It seems the XenForo.ajaxBaseHref setting isn't being used when the AJAX call to "attachments/do-upload.json" is made when pasting an image into editor (which makes it fail when on a different sub-domain).
The URL comes from the uploader's data-action attribute, which is actually a "full" URL by default. Pasting actually does have canonicalization with the ajaxBaseHref, but the URL is likely canonical already.
Does that mean that a "full" URL in your setup generates a URL that points to a different sub-domain?
Ah... okay, not a bug then. Yeah, data-action is set to the main sub-domain, but strange normal uploads work set up like that (I guess maybe Flash doesn't have to adhere to cross-domain same-origin policies or something?).
I specified the right sub-domain in data-action, but pretty strange it worked in all ways except the copy/paste method of uploading images.