I seem to recall the various things that you can import (with the importers) as users, user groups, etc... and you might be able to import users only doing it through the web interface.
My memory isn't too clear on this, but I do believe you can uncheck posts for it to not import those, etc.
Might want to run a dev instance with of the main forum, upload the other forum's DB, install the importers, and then run it from ACP to see if you can just import the users.