XF 1.4 General laggyness in very active threads

[Denny Crane]

I run a sports site and the posters like to congregate in a single thread during the basketball games. Tonight they made about 570 posts in about 2 hours, 4.75 posts per minute.

Many of them complained that the pages were laggy. There were two specific kinds of complaints.

1) Users said their keystrokes didn't appear on screen in the editor quickly
2) When the "there are new posts" indicator appears

while the game was on, I checked out the servers and they were at 99% idle. I think it's the JavaScript or maybe related to work being done in response to frequent Ajax calls or timer callbacks.

Anyone else experience this and know a fix?

 

[Jake Bunce]

1) Users said their keystrokes didn't appear on screen in the editor quickly

Input lag would be at the client, though it can be as a result of the client's resources being consumed by something intensive on the page such as an embedded flash object, runaway javascript, or excessive CSS animation.

Basically you can try disabling addons and activating a default style. Also try a different browser and/or computer because it might be a problem with a specific client.

2) When the "there are new posts" indicator appears

Which indicator exactly? If it's an alert then know that alerts are not updated instantly. New alerts are shown on page load and also at regular intervals, but it still amounts to polling.

Forum threads aren't ideal for high rates of real time discussion. Your users are probably constantly refreshing the thread. You might consider hosting a chat room for events like this:

https://xenforo.com/community/resources/8wayrun-com-xenhabla-irc.100/

 

[Denny Crane]

Right, I realize input laggyness is a function of the user's computer. But they never complained about vb4 with similar "add ons" like twitter postings in the posts. We've been hosting these big game time threads since 2003 on various message board software, including ipBoard and vBulletin 3 and 4.

I don't think the chat room idea is going to make my users think going to xenforo would be a great upgrade if it can't perform. We are the official message board of the Portland Trail Blazers NBA team and they drive a lot of traffic to us specifically for game threads (they have 1M twitter followers, etc.).

I want to point out that we are extremely happy with the feature set and functionality except for this.

I've found you and others on this site and involved to be extremely helpful and good folks. I'm pretty good at the technical stuff, including the hardcore PHP and JavaScript coding and am fine with the system administration end of things, too. I'm basically asking for your help in resolving this and offering my help in diagnosing it.

They tested and do not report the laggyness in a separate thread at the time. They tested and do not report the legginess in not busy 30 page threads.

The "there are new posts indicator" is one that appears in the middle of a thread while you are posting. It's a blue bar (in the default theme) that says "there are unread messages in this thread, do you wish to load them?"

I'm guessing that the issue is related to frequency of ajax polling and the amount of frequent DOM manipulation that is triggered by high activity.

 

[Denny Crane]

You can read about it here:
http://www.sportstwo.com/threads/testing.277500/

 

[Denny Crane]

To clarify, this site is not running on a shared host or even a single server. It's running on several vps instances: 2x load balancers, 3x WWW servers, 2x MySQL in master-master with one as hot spare, 2x glusterfs for the data/ and internal_data/ directories. The three WWW servers also run memcached and memcached caching is turned on.

All this is secured by iptables rules and the only ssh access is from a vps instance that I only spin up when I need command line access to the other instances.

I ran ab against the whole setup and got 75 requests per second out of it.

 

[Jake Bunce]

I read over your thread. It certainly sounds like a general problem of client lag. As you said in that thread, if the problem is reproducible then it can be debugged and fixed. Otherwise we are left to make educated guesses without the ability to diagnose and confirm.

I quickly scanned your thread pages for anything obvious and nothing occurred to me. The JS profiler looks good when viewing a thread, but I didn't try posting.

Just a thought...

Admin CP -> Appearance -> Style Properties -> General -> Enable Animations, Using Multiplier

Animations can cause bursts on the CPU, and they are employed when AJAX-loading new posts.

I wonder if the users in question are constantly quick replying while never refreshing the entire page. In the context of a highly active thread that could result in a huge DOM as new posts are repeatedly inserted into the page.

These are just guesses.

 

[Denny Crane]

I'm certain they are using quick reply constantly. In fact, a few people complained that the default setting for time required between posts was too high (15 or 30 seconds or whatever).

If that is what the issue is, what can be done about it?

 

[Jake Bunce]

The quick reply form could be made to do a standard submission thereby reloading the entire page.

Remove the "AutoValidator" class from this template:

Admin CP -> Appearance -> Templates -> quick_reply

Of course this makes replying and viewing recent posts less seamless.

 

[Denny Crane]

It would be awesome if this was a setting the users could choose in their personal settings, no?

 

[Jake Bunce]

It would be awesome if this was a setting the users could choose in their personal settings, no?

That can be coded up as an addon.

We really should try to lock down the problem before putting too much effort into a solution. I will create this situation on my test forum to see if it does indeed cause performance issues for the client. At the very least it could be a use case for the developers to consider, but it still amounts to guesswork with respect to your problem.

 

[Jim Boy]

The quick reply doesn't sound right - We have game day threads as well, and at a similar higher throughput than mentioned in the OP and we dont get these sorts of complaints, at least not related to game day.

There could well be some bottleneck in the network, or db. I'd be doing more troubleshooting first. For example using the timings function within developer tools on firefox to see if there is a big delay in network.

It could well be some third party javascript loaded by a an add-on that is causing the issue.

 

[Jake Bunce]

This guy in your thread (regarding input lag):

I just did a re-fresh and it helped a little. But it still lags behind.

Try to set him up with a default style and no addons for the purpose of troubleshooting.

If you can get his IP address then you can add this code to the library/config.php file to disable addons for him:

if ($_SERVER['REMOTE_ADDR'] == '11.22.33.44')
{
	$config['enableListeners'] = 0;
}

 

[Denny Crane]

When I trace timers triggered in chrome developer tools, there are a lot fired per second.

 

[Denny Crane]

This guy in your thread (regarding input lag):



Try to set him up with a default style and no addons for the purpose of troubleshooting.

If you can get his IP address then you can add this code to the library/config.php file to disable addons for him:

if ($_SERVER['REMOTE_ADDR'] == '11.22.33.44')
{
    $config['enableListeners'] = 0;
}

Done. Now we wait for him to come online and try it.

 

[Jim Boy]

Info on browser and version might be helpful. Does task manager suggest that cpu or memory is excessively used? You ca also use about:memory to see what memory is being used.

Overall though I suspect it is related to the number of posts per page being displayed, which I take it you allow a fair bit of freedom on. Why not try disabling that add-on and see if the problem goes?

FWIW, if you were to set the posts per page to something like 25 then I think you should be right. You'll get the usual people complaining about it, but I doubt they would really miss it. When we shifted from VB to XF the users lost that option, and other than a little hand wringing at the beginning, ts never been requested or talked about again.

 

[Denny Crane]

I appreciate the suggestion. That would be a last resort type of fix. If forcing this user to use no add ons doesn't fix the issue, then it's not posts per page, right?

 

[Jim Boy]

Yes, posts per page is an addon.

Be a little wary about disabling listeners if any of your add-ons perform critical tasks.

 

[Denny Crane]

I know what add ons I installed

What I was saying is that if I force the user to not use add ons and the basic skin and the problem persists, then it's not likely any add ons or the skin causing the problem.

FWIW, I only modified CSS rules for the skin so far.

 

[Jim Boy]

Yeah, that is true. Note that some browsers, such as Opera, have had lag problems with the text editor.

 

[Denny Crane]

He ran virus checker and all that "stuff" you need with Windows. That sped things up considerably, but still a bit of lag he says. Then he turned off the rick text editor and it's zippy fast for him.