When viewing "Banned Users" in the ACP entries are showing with no user name. Clicking on the entry gives shows the xf_user_ban data. Clicking on "User Info" shows "The requested user could not be found" and looking in xf_user confirms that the user entry is not there.
So the question really is.... what's the best way of clearing up xf_user_ban to get rid of entries for users that don't exist?
You could likely delete them with a query that left joined to the xf_user table and removed the ones without a matching user. I wouldn't totally recommend that just from an ease-of-mistakes perspective.
You may just need to remove them manually via the DB.