JavaScript in XenForo has a hard dependency on jQuery. If scripts are loaded with async/defer then jQuery might not be available when execution starts. This is the primary reason I'm still using the old method of deferring scripts in my add-on (placing them before the </body> tag). Additionally, async will make execution start immediately, before the page is finished loading, which is probably incompatible with most XenForo JavaScript.
I haven't played around with it too much, but I'm not sure there's a reliable way to use <script defer> without breaking dependencies. Callbacks might be an option, I suppose.