XF 1.2 Page Load Time Issues for Certain Users

I have gotten a couple of complaints about extremely slow load times on my forum, which is located HERE. I ran a poll on the site on load times and it appears that the vast majority of users are seeing good load times- it's just a couple of users that are having issues. However, they indicate that they don't have similar problems with other sites or even other forums. The issues that are being reported are not browser-specific (happening on both Chrome and MSIE) and are not device-specific (happening on both desktop and mobile. From what I can tell the folks who have indicated the worst troubles (pages simply stalling out and not loading) have Verizon as an ISP.

I've done my best to try and optimize performance and have not had any issues with load time from my end and, again, most users report no problems. Has anyone experienced similar issues and are there any suggestions? Many thanks in advance for any support.
@Jeremy - I have not discouraged any IP's. I banned a number of spammer IP's initially, and still am struggling with whether that is the proper way to deal with them, but other than that, no other actions taken with regard to IP's. @Russ - I will double-check with the users but my understanding is that it's happening on the front end (I'm runinng XenPorta), the forum category page and in individual threads as well.