Set everything back to Not Set. Create a test user. Ban the test user. Have a play around and then see if the banned user has access to anything they shouldn't Pretty much it is the case that they can do nothing.
If that's the case, then you don't need to change any permissions (Leave them as default, e.g. Not Set).
And with that configuration, the ban reason you mentioned in the original post should be visible.