As disjunto mentioned, this is mostly do to your browser waiting while the signature images are generated. The script for the images is fetching data from XenForo, processing that data, and then dynamically drawing an image. This takes some time, and once it is completed, your browser still has to download the image from the server. So yes, dynamic signature images like these (and any user-added images, for that matter), can artificially slow down the page-loads and make it appear like the site is slow, when in reality the apparent slowdown is being caused by an external server.
If you disable signatures, the site will probably be noticeably faster.