How many of those are unconfirmed (waiting for email confirmation) or banned/invalid? By default, XF doesn't count any invalid, banned or members who've not confirmed their email address.
You can search in your AdminCP for different group combinations and status to see where those members are located. I would at least start there to figure out how many accounts have statuses that wouldn't be counted in the actual (active/valid) member count.