Thanks, I think I figured this out now.
I've set all the users to be Registered as their primary, with all the permissions set to 'Not Set'.
In the forum where I only wanted the Admins and Moderators to be able to create new threads, I revoked the 'Post New Thread' for Registered and...
I am having real trouble with this concept. If everyone is in the Registered as their primary, then as an Admin how do I add a new thread? The Registered group seems to take priority and therefore there is no Post New Thread button, even though I am admin.