When you set up xenforo, all users should be in registered as primary. Then you keep them like that and just use secondary groups for different permission sets. So you'd never need to search for primary group as that is just the entire user list.
You just search secondary group (so my post above is wrong, I'll edit it)