You can't import attachments and avatars without also importing users and user groups.
Unfortunately, though you'll be able to retain ids in the first import, when you try do a second import with the hope of matching ids, you won't be able to since Xenforo's importer will not allow the retain content id setting to be on after the first import.
Merging multiple imports so their users, attachments, avatars and threads/posts match up properly requires custom work. If you need that kind of service, contact me via conversation message.
@Slavik can probably handle something like this as well.