This thread was extremely helpful. I created several forums which are only visible to the users who self-joined. I used to do this on a previous forum software, but it was far too resource intensive and buggy.
In this way, it creates a basic group or social group type system within Xenforo without having to heavily modify it.
Here are a few notes of things I found along the way that may help:
1. This current thread about using custom user fields with user group promotions. I used a multiple choice drop down and called it Group Memberships. I put the location in preferences. Added a link in the description of the field that says click here to go to the group area after joining. It seems to work fine as long as everything is done through the selection of the custom user field. I'm not clear if the dropdown options stay in sync if an admin changes the groups directly for a user. So, if I manually change anything for a specific user for some reason on the admin side I just do the custom user group and promotion at the same time.
2. Used this add on to trigger the promotions right away.
Adds an per-custom field criteria option to match an empty value. Preforms a user promotion when: A user updates their own profile (Routes under /account). When admin's edit a user's profile. When a user upgrade is being applied/removed...
xenforo.com
3. Created a parent level node with a description and link for how to join a group (the link goes to their preferences or wherever the custom user profile is). Then made child special forums as private then explicitly gave them permission based on the group they joined.
4. Used Notices, based on being in that forum, to create a header explaining they are in a group, a link to create a new thread, view existing threads, and show members who are in the group (see below). A sort of submenu if you will. Otherwise, it may look a little confusing as to where you are.
5. Create and lock an intro thread in each new forum explaining the group, rules, etc.
6. To show members who are in the group. Go into member statistics, create a separate member statistic based on each group. Then you can link to that specific member statistic as a list of who is in the group. I put this in the Notice and first thread for each group forum. Might want to cache the results for at least 10 minutes. Can also use a member stat widget in a similar way.
7. In the custom user group setup, give each group a linked banner. This helps promote the group since it will show throughout the forum in their posts.
8. Create a widget for use in advertising the groups throughout the forum. Link to the top Group node, list each group (link to the top node), and the join link optionally (preferences page or wherever).
9. Refer to the groups throughout top navigation menus and other places as needed.