I wonder if it's because you're using one of the XenFocus range of styles? "Basik" in the case I've just seen on your Infinity Program forum.
XenFocus ones tend to use smaller avatars in the sidebar_visitor_panel (57 x 57 as opposed to the 96 x 96 default).
I use "Uniform" as an extra style, so that would happen to me too.
Also, unrelated but btw: the link in your XF sig for The Admin Zone is broken up when I tried it earlier. An extra [/URL is present in the URL (at the end).