XF 1.0 Real-World Page Loading Speed

This isn't so much a 'have you seen this feature' post as one that goes some way to explaining why interacting with XenForo is such an immediate, pleasurable experience.

The following is screen-grabbed from Google Webmaster Tools.

 

[hescominsoon]

http://forums.Spacebattles.com is a forum with approaching five million posts, and maintains a high level of concurrency. It is highly reliant upon Sphinx as its search engine to prevent it from murdering this little server. Therefore, Spacebattles won't switch to XenForo until a viable Sphinx implementation is available.

why wouldn't you move to xf? IS sphinx required due to the MySQL's fulltext engine limitations?

 

[RobParker]

what kind of hdd's are in your box? XF will handle it..it's a matter of your hardware. p-4 xeons are a bit long in the tooth. That and your disk configuration are probably your biggest limitations. Give us a top when you are at max load so we can see cpu/ram/io usage.

Cheers, I'll do that when we get busy.

SCSI Devices
-DELL VIRTUAL DISK IM (Direct-Access)
-IBM IC35L073UCDY10-0 (Direct-Access)
-SDR GEM318P (Processor)
-SEAGATE ST373207LC (Direct-Access)

 

[Kier]

why wouldn't you move to xf? IS sphinx required due to the MySQL's fulltext engine limitations?

This server was brought to its knees by the SpaceBattles forums running with vBulletin's MySQL fulltext search, deploying sphinx search has made the load manageable. MySQL Fulltext search with multi-millions of posts is going to cause problems without lots and lots of very fast hardware, regardless of the application using it.

 

[hescominsoon]

exactly.. So why not get sphinx into XF for larger sites?

 

[Kier]

exactly.. So why not get sphinx into XF for larger sites?

To get ourselves to a position where we will be able to support larger sites, we first have to make enough money to pay for development of such features that won't necessarily benefit all customers - a small site with less than a million posts (ie: most of them) would likely see very little if any benefit from a Sphinx solution.

 

[Mikey]

to develop sphinx integration would cost you money?

 

[TerminalAddict]

wow spacebattles must be full of chatty people
I've got 20k user and only 300k posts .. much smaller user to post ratio.

Sphinx helped a lot in my IPB forum; but mainly with search speed .. MySQL searching was taking ages (20 seconds for a response) now it's fast as!

also of interest the cluster of db servers I run on can handle these queries but sphinx is still faster.

if I ever get to this size again (which I'm hoping to) I will absolutely be looking for a sphinx solution

 

[R1Lover]

If you were waiting 20 seconds for a search it was more likely due to a configuration or server issue imo. If setup properly there is no reason it would be that bad on a proper server.

 

[Kier]

If you were waiting 20 seconds for a search it was more likely due to a configuration or server issue imo. If setup properly there is no reason it would be that bad on a proper server.

That's really not true, the nature of MySQL fulltext is such that speed suffers very badly when a lot of results are returned and need to be sorted. A test search for a common term on my test machine here with 5.5 million posts just took 74 seconds to return, and there is nothing wrong with its configuration, it's just a MySQL limitation.

Conversely, searching the same database for a very rare term returned almost instantly.

 

[kingston]

That's really not true, the nature of MySQL fulltext is such that speed suffers very badly when a lot of results are returned and need to be sorted. A test search for a common term on my test machine here with 5.5 million posts just took 74 seconds to return, and there is nothing wrong with its configuration, it's just a MySQL limitation.

Conversely, searching the same database for a very rare term returned almost instantly.

Does mod_pagespeed help Xenforo much?

 

[Kier]

Does mod_pagespeed help Xenforo much?

I've never tried it, but it might give a very modest speed improvement. However, it would have absolutely no effect whatsoever on the speed of searching.

 

[kingston]

I've never tried it, but it might give a very modest speed improvement. However, it would have absolutely no effect whatsoever on the speed of searching.

Searching is for wimps. I just remember where all the posts are supposed to be always.

 

[R1Lover]

That's really not true, the nature of MySQL fulltext is such that speed suffers very badly when a lot of results are returned and need to be sorted. A test search for a common term on my test machine here with 5.5 million posts just took 74 seconds to return, and there is nothing wrong with its configuration, it's just a MySQL limitation.

Conversely, searching the same database for a very rare term returned almost instantly.

He was referring to a forum with only 20k user and 300k post.... I have several in this range and mysql search works just fine. Now when your talking 5.5 million post as you mentioned, that's a different story as you have shown.

 

[hescominsoon]

I've killed mod_deflate and php gzipping of content.. Obviously it's actually made things faster for me..in my numbers less than 1% are dialup so i don't feel bad about making things "faster" for the majority of my visitors..

 

[hescominsoon]

I also understand that most folks have a transfer cap and therefore compression is a transfer saver...i'm not disputing that aspect at all..

 

[Wildcat Media]

That's really not true, the nature of MySQL fulltext is such that speed suffers very badly when a lot of results are returned and need to be sorted. A test search for a common term on my test machine here with 5.5 million posts just took 74 seconds to return, and there is nothing wrong with its configuration, it's just a MySQL limitation.

On the "big board" forum we run on a dedicated server, we only have limited funds to keep it running. So, I have to maximize what resources we have, as we don't have money to put toward better hardware. Last time I worked on it, we were running vB 3.7 series. Searches were taking 30-45 seconds. I figured out how to use the Sphinx modifications, and searches now take place around one second on average. The server load was our big concern, and Sphinx has really helped there as well. Members have made many comments on how much faster searches are.

We'd probably have to wait for a Sphinx implementation before we'd consider using XF on that board. Then again, with that many posts, it would be a huge task to convert it all over.

I will say this, though: I have converted a small, newer forum over to XF from SMF, and the overwhelming compliment we've received thus far: it's fast. Members really like it! And they are actually a lot more active now. This is their first exposure to XenForo, after years of vBulletin (where many of us met), or phpBB2 or SMF from my other smaller forums. Most seem to be having no problems at all finding anything.

I must say that with XenForo, you've "nailed it." We're all very pleased with the outcome after our conversion! I just think from a usability standpoint, the speed, and the improved "social" aspects of the system, it is a step or two ahead of all the others out there. (And I've been using forums since I was a Compuserve Sysop, using the text-based interface inside Procomm Plus!)

 

[hellreturn]

Kier, the screenshot posted in first topic is from July or so. Would it be possible for you to share the latest screenshot ?
Thank you.

 

[kingston]

I've never tried it, but it might give a very modest speed improvement. However, it would have absolutely no effect whatsoever on the speed of searching.

Using Googles webmaster tools when I turned on mod_pagespeed I saw a .2 second drop in average load time. While that doesn't sound like much, considering its already around a 2 second average there wasnt much more time for it to drop to (I load a lot of graphics).

 

[AzzidReign]

Can we see an updated image of how this site is loading now since it's getting much more traffic than when you first posted?

 

[groundup]

I misspoke. I have unmetered transfer but metered bandwidth. What folks call bandwidth isn't. bandwidth is how fast something can go irrespective of how much you transfer. Transfer is how much you move irregardless of speed. I have unmetered transfer and a good port speed(umetered 10 megabit connection so unlimited transfer at 10megabits); So I would rather use those cpu cycles for other things considering my target audience has more than a 90% b-band penetration.

The faster you free up that connection, the faster your CPU can get on to doing other things. You're thinking about it all wrong. Gzip is nowhere near the CPU eating process as you think it is.