Here's what I get from Firebug:
View attachment 65813
The longest wait time appears to be the thumbnails for the gallery sidebar (further up the waterfall) - but they're not drastic:
View attachment 65814
If I visit one of the forums and then go back to the home page (cached) - the longest wait time to load anything is 164ms for the skimlinks.js
Even on a forum page, all your local stuff seems to be loading quickly and the only lags are the usual third-party resources such as Skimlinks and Adsense/DoubleClick.
Caching seems to be on and working because when I first request a thread the local images are taking up to 430ms, but on refresh they're down to 20+ms.
Knowing you you'll have optimised it as best you can already, but I don't think you've got anything to worry about here. The numbers seem okay and it visually loads very quickly for me, here in the UK (your target audience at the moment I presume?).
Gets a good score with WebPageTest too (Firefox):
http://www.webpagetest.org/result/140127_QB_MFZ/ - (Chrome):
http://www.webpagetest.org/result/140127_21_MTJ/ [NB: All of the static content giving you a D is off-server anyway, so nothing you can do about those!
]