Check all User Groups, not just Primary

I'm making a second member list that is for admin people tracking purposes (and only they can see it), and it will have the group name followed by everyone who is in that group. The thing is, people are in multiple user groups, but they are only showing up in their Primary group when I'd like them to show up in ALL of the groups they're a part of.

This is the code for what one of the groups looks like:
    <ol class="section memberList">
        <xen:foreach loop="$users" value="$user">
            <xen:if is="{$user.user_group_id} == 4">
            <xen:include template="member_list_item">
I suspect it has something to do with the variable $user.user_group_id that it only checks the primary group instead of all of the groups they're in. Is there some bit of code I can add in to have it poll all of their groups?