XF 1.4 Long wait times on thread creation, resulting in dup threads

kontrabass

kontrabass

Well-known member
Hi all,

So lately, in some of our more popular forums, we've been getting a lot of duplicate thread submissions. This is happening because it takes 10+ seconds for the thread to "go through". The progress bars appear, user waits, clicks "create thread" again, and the thread is posted twice (or 3 or 4 times heh). During peak hours, sometimes the user gets no confirmation the thread was posted (no drop-down, no redirect to the thread). The web server and db server loads are very low.

I can test this again and again, always the same wait. Oddly, this is only a problem in one or two subforums (our classifieds forums, which run a couple addons). I've tried disabling the addons, but there's no change. Here's the query list when the wait is occuring:

Code: 
| 76052008 | xfdbuser  | 192.168.0.1:48566 | xenforodb | Execute        |       2 | update                                                                | INSERT INTO xf_data_registry
                (data_key, data_value)
            VALUES
                ('deferredRun', 'i:1417669409;')
            ON DUPLICATE KEY UPDATE
                data_value = VALUES(data_value) |    0.000 |
| 76052036 | xfdbuser  | 192.168.0.1:48602 | xenforodb | Execute        |       2 | update                                                                | INSERT INTO xf_data_registry
                (data_key, data_value)
            VALUES
                ('deferredRun', 'i:1417669409;')
            ON DUPLICATE KEY UPDATE
                data_value = VALUES(data_value) |    0.000 |
| 76052043 | xfdbuser  | 192.168.0.1:48611 | xenforodb | Execute        |       2 | update                                                                | INSERT INTO xf_data_registry
                (data_key, data_value)
            VALUES
                ('deferredRun', 'i:1417669409;')
            ON DUPLICATE KEY UPDATE
                data_value = VALUES(data_value) |    0.000 |
| 76052050 | xfdbuser  | 192.168.0.1:48619 | xenforodb | Execute        |       1 | update                                                                | INSERT INTO xf_data_registry
                (data_key, data_value)
            VALUES
                ('deferredRun', 'i:1417669410;')
            ON DUPLICATE KEY UPDATE
                data_value = VALUES(data_value) |    0.000 |
| 76052068 | xfdbuser  | 192.168.0.1:48647 | xenforodb | Execute        |       1 | update                                                                | INSERT INTO xf_data_registry
                (data_key, data_value)
            VALUES
                ('deferredRun', 'i:1417669410;')
            ON DUPLICATE KEY UPDATE
                data_value = VALUES(data_value) |    0.000 |
| 76052073 | xfdbuser  | 192.168.0.1:48654 | xenforodb | Execute        |       0 | update                                                                | INSERT INTO xf_data_registry
                (data_key, data_value)
            VALUES
                ('deferredRun', 'i:1417669411;')
            ON DUPLICATE KEY UPDATE
                data_value = VALUES(data_value) |    0.000 |


Any ideas?

Thanks!
 
Sort by date Sort by votes
This sounds like a server performance issue. It may be anything from addons, the webserver config to database tuning.

Updating xf_data_registry should not be that expensive.

An idea of activity level and server specs would be needed.

It may still be addon related even if you disable it, as sometimes the addons do silly things and stay effective even if you disable them. Especially if they have inflicted expensive changes to the database.
 
Last edited:
Upvote 0 Downvote
If it only happens in a few forums, that could indicate it's down to people watching the forums and thus XF having to do a large amount of mail processing. You can prevent people from watching particular forums and sending emails, though changing the value won't apply to previous settings. As a test, you could change XenForo_Model_ForumWatch::sendNotificationToWatchUsersOnMessage() to immediately return an empty array.

Those queries should indeed be inexpensive. Is that the full processlist output?
 
Upvote 0 Downvote
Mike said:
If it only happens in a few forums, that could indicate it's down to people watching the forums and thus XF having to do a large amount of mail processing. You can prevent people from watching particular forums and sending emails, though changing the value won't apply to previous settings. As a test, you could change XenForo_Model_ForumWatch::sendNotificationToWatchUsersOnMessage() to immediately return an empty array.

Those queries should indeed be inexpensive. Is that the full processlist output?
Click to expand...

Yes that's pretty much the whole output, besides the normal processes (a couple sleeping, binlog, etc).

Here's another proc list output from phpmyadmin this time:

Code: 
Processes    ID    User    Host    Database    Command    Time    Status    SQL queryShow Full Queries
Kill    36798    slave    192.168.0.2:42668    None    Binlog Dump    1669537    Master has sent all binlog to slave; waiting for binlog to be updated    ---
Kill    77462767    DELAYED    localhost    xenforodb    Delayed insert    1    Waiting for INSERT    ---
Kill    77478562    DELAYED    localhost    xenforodb    Delayed insert    100    Waiting for INSERT    ---
Kill    77521862    xfdbuser    192.168.0.1:56474    xenforodb    Sleep    1    ---    ---
Kill    77522026    xfdbuser    192.168.0.1:56654    xenforodb    Execute    18    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522072    xfdbuser    192.168.0.1:56704    xenforodb    Execute    18    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522082    xfdbuser    192.168.0.1:56716    xenforodb    Execute    17    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522096    xfdbuser    192.168.0.1:56732    xenforodb    Execute    17    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522098    xfdbuser    192.168.0.1:56735    xenforodb    Execute    17    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522116    xfdbuser    192.168.0.1:56754    xenforodb    Execute    16    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522152    xfdbuser    192.168.0.1:56792    xenforodb    Execute    16    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522165    xfdbuser    192.168.0.1:56806    xenforodb    Execute    15    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522190    xfdbuser    192.168.0.1:56834    xenforodb    Execute    15    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522198    xfdbuser    192.168.0.1:56843    xenforodb    Execute    15    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522202    xfdbuser    192.168.0.1:56847    xenforodb    Execute    15    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522215    xfdbuser    192.168.0.1:56861    xenforodb    Execute    15    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522220    xfdbuser    192.168.0.1:56866    xenforodb    Execute    14    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522225    xfdbuser    192.168.0.1:56872    xenforodb    Execute    14    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522237    xfdbuser    192.168.0.1:56884    xenforodb    Execute    14    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522243    xfdbuser    192.168.0.1:56891    xenforodb    Execute    14    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522280    xfdbuser    192.168.0.1:56931    xenforodb    Execute    13    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522310    xfdbuser    192.168.0.1:56962    xenforodb    Execute    13    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522330    xfdbuser    192.168.0.1:56983    xenforodb    Execute    13    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522423    xfdbuser    192.168.0.1:57091    xenforodb    Execute    11    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522428    xfdbuser    192.168.0.1:57096    xenforodb    Execute    11    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522448    xfdbuser    192.168.0.1:57117    xenforodb    Execute    11    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522455    xfdbuser    192.168.0.1:57125    xenforodb    Execute    10    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522463    xfdbuser    192.168.0.1:57140    xenforodb    Execute    10    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522468    xfdbuser    192.168.0.1:57145    xenforodb    Execute    10    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522496    xfdbuser    192.168.0.1:57177    xenforodb    Execute    10    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522567    xfdbuser    192.168.0.1:57267    xenforodb    Execute    9    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522570    xfdbuser    192.168.0.1:57270    xenforodb    Execute    8    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522573    xfdbuser    192.168.0.1:57273    xenforodb    Execute    8    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522598    xfdbuser    192.168.0.1:57309    xenforodb    Execute    8    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522609    xfdbuser    192.168.0.1:57328    xenforodb    Execute    8    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522616    xfdbuser    192.168.0.1:57343    xenforodb    Execute    8    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522621    xfdbuser    192.168.0.1:57350    xenforodb    Execute    8    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522645    xfdbuser    192.168.0.1:57381    xenforodb    Execute    7    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522647    xfdbuser    192.168.0.1:57383    xenforodb    Execute    7    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522667    xfdbuser    192.168.0.1:57406    xenforodb    Execute    7    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522690    xfdbuser    192.168.0.1:57431    xenforodb    Execute    7    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522719    xfdbuser    192.168.0.1:57463    xenforodb    Execute    6    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522720    xfdbuser    192.168.0.1:57464    xenforodb    Execute    6    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522724    xfdbuser    192.168.0.1:57468    xenforodb    Execute    6    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522730    xfdbuser    192.168.0.1:57475    xenforodb    Execute    6    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522735    xfdbuser    192.168.0.1:57480    xenforodb    Execute    6    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522742    xfdbuser    192.168.0.1:57488    xenforodb    Execute    6    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522749    xfdbuser    192.168.0.1:57495    xenforodb    Execute    6    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522761    xfdbuser    192.168.0.1:57508    xenforodb    Execute    6    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522785    xfdbuser    192.168.0.1:57532    xenforodb    Execute    5    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522792    xfdbuser    192.168.0.1:57541    xenforodb    Execute    5    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522793    xfdbuser    192.168.0.1:57542    xenforodb    Execute    5    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522802    xfdbuser    192.168.0.1:57552    xenforodb    Execute    5    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522871    xfdbuser    192.168.0.1:57626    xenforodb    Execute    4    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522923    xfdbuser    192.168.0.1:57682    xenforodb    Execute    3    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522928    xfdbuser    192.168.0.1:57687    xenforodb    Execute    3    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522936    xfdbuser    192.168.0.1:57695    xenforodb    Execute    3    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522948    xfdbuser    192.168.0.1:57709    xenforodb    Execute    3    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77522963    xfdbuser    192.168.0.1:57726    xenforodb    Execute    2    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77523025    xfdbuser    192.168.0.1:57796    xenforodb    Execute    1    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77523042    xfdbuser    192.168.0.1:57814    xenforodb    Execute    1    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77523049    xfdbuser    192.168.0.1:57822    xenforodb    Execute    1    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77523055    xfdbuser    192.168.0.1:57828    xenforodb    Sleep    0    ---    ---
Kill    77523056    xfdbuser    192.168.0.1:57829    xenforodb    Execute    1    update   
INSERT INTO xf_data_registry (data_key, data_value) VALUES ('deferredRun', 'i:14177082
Kill    77523058    phpmyoliver    192.168.0.1:57831    mysql    Query    0    ---   
SHOW PROCESSLIST
Kill    77523059    xfdbuser    192.168.0.1:57832    xenforodb    Sleep    0    ---
I'd love to test this like you mentioned, but I have no idea how to do what you suggest :oops:

I'm currently using smtp to send notification emails through mandrill - should I use the default email transport method instead? NVM that didn't help

I think you are correct in your guess - our bass guitar classifieds forum, where this occurs, is very heavily watched.

Thanks for your help! :)
 
Last edited:
Upvote 0 Downvote
Mike said:
As a test, you could change XenForo_Model_ForumWatch::sendNotificationToWatchUsersOnMessage() to immediately return an empty array.
Click to expand...

I'm wondering if someone can show me how to make this change, as Mike has suggested? Still trying to get to the bottom of this :( Thanks!!
 
Upvote 0 Downvote
Mike said:
As a test, you could change XenForo_Model_ForumWatch::sendNotificationToWatchUsersOnMessage() to immediately return an empty array.
Click to expand...
Open /library/XenForo/Model/ForumWatch.php find "function sendNotificationToWatchUsersOnMessage(" after the { put
Code: 
 return array();
and save it. Make sure there's a space between "{" and "return".
 
Upvote 0 Downvote
Mike said:
If it only happens in a few forums, that could indicate it's down to people watching the forums and thus XF having to do a large amount of mail processing. You can prevent people from watching particular forums and sending emails, though changing the value won't apply to previous settings. As a test, you could change XenForo_Model_ForumWatch::sendNotificationToWatchUsersOnMessage() to immediately return an empty array.

Those queries should indeed be inexpensive. Is that the full processlist output?
Click to expand...

@Mike You hit the nail on the head. I modified XenForo_Model_ForumWatch::sendNotificationToWatchUsersOnMessage() to return an empty array, and instantly thread creation in the "laggy" forum was zippy as can be (< 1 second).

So the question now is what to do about it :(. According to my Mandrill outbound logs, there are only 10-15 people signed up for email notification of new threads in this subforum. That shouldn't cause a 15 second delay when submitting a new thread, should it? So then it must be the folks watching the thread with no email notification? There must be something I can do on the database server to help get the delay down to like 5 seconds. The server itself is very hefty, and hardly registers a load at all.

Edit: Additional stats. Using phpmyadmin to search the xf_forum_watch table, there are 232 users watching this subforum (compared to 82 in our next-most-popular forum). 157 users have send_alert on. Only 11 have send_email enabled.

Should 157 watchers with send_alert enabled really grind the thread creation process to a halt?
 
Last edited:
Upvote 0 Downvote
I wouldn't have expected that much delay from that, but it does require more work (to do the inserting and alerting) so there will definitely be an overhead.

I will look to see if there are tweaks that can be made, but there wouldn't be scope for any major changes. The only thing that could be done from your side is to limit the forum watching options (though that wouldn't necessarily change the existing watch settings; I believe that would need to be changed in the DB directly).

On a side note, our announcement and HYS forums have more people watching those forums than the listed one on your site; I can't say I've noticed any particularly significant delay when posting there but I haven't really looked for it.
 
Upvote 0 Downvote
Mike said:
I wouldn't have expected that much delay from that, but it does require more work (to do the inserting and alerting) so there will definitely be an overhead.

I will look to see if there are tweaks that can be made, but there wouldn't be scope for any major changes. The only thing that could be done from your side is to limit the forum watching options (though that wouldn't necessarily change the existing watch settings; I believe that would need to be changed in the DB directly).

On a side note, our announcement and HYS forums have more people watching those forums than the listed one on your site; I can't say I've noticed any particularly significant delay when posting there but I haven't really looked for it.
Click to expand...

Thanks for looking into it Mike. Honestly I don't know much about this but, as one of my admins asked me, "shouldn't those kinds of notifications should be fired off asynch by some kind of service and should have no bearing on thread posting. and only ~300 subs causing it? ...." That does kind of make sense to me... it seems like the user experience of creating a thread, could "go through" and then the task of sending notifications could be done in the background?

The thing is, the thread creation seems actually pretty fast (every click of the button does make a new thread). It's giving the user confirmation of the new thread creation (ie, the "your thread has been posted" pulldown, and the redirection to the new thread), that is delayed 20 seconds (hence the button mashing).

Anyway, I hope a tweak is possible ;) If not, we'll look at hardware I guess. Our master DB server is a couple years old but has an Intel E3 1230v2 and dual SSD's in Raid 1...

Thanks again for your time!
 
Upvote 0 Downvote
Emails are deferred, but alerts are not.

https://xenforo.com/community/threads/assorted-small-things.48937/#post-524533

Mike said:
...

New general-purpose system for long running processes
1.2 adds a new "deferred" system for running any code that can be broken up into pieces and may take a long time (multiple seconds or more) to run. Deferred tasks can be triggered automatically by the code as necessary and they will either be run automatically in the background (by page views) or by displaying the current running state to the administrator (in the case of some cache rebuilds). This effectively unifies the cache rebuild systems with cron and other potentially long running processes (such as the bulk user change system discussed in the past).

Mail queue system
Using the "deferred" system above, a mail queue system has been implemented for email that does not have to be sent immediately, such as thread watch emails. Other emails (such as registration confirmation or lost password) emails can still bypass the system.

This creates performance improvements, particularly for heavily watched threads or when using a slow, external mail server.
Click to expand...
 
Upvote 0 Downvote
The problem with deferring the alerts is of course that some people will not get theirs right away.
I've ran a test with 180 users and depending on how many alerts you process per batch, it can take up to a few seconds for those at the bottom of the list to get their alerts.
While this might be preferable in your case where the thread posting process is slow, it would however mean that some users might notice that a thread was created, or has new responses, before they are actually alerted about it.
 
Last edited by a moderator:
Upvote 0 Downvote
Syndol said:
The problem with deferring the alerts is of course that some people will not get theirs right away.
I've ran a test with 180 users and depending on how many alerts you process per batch, it can take up to a few seconds for those at the bottom of the list to get their alerts.
While this might be preferable in your case where the thread posting process is slow, it would however mean that some users might notice that a thread was created, or has new responses, before they are actually alerted about it.
Click to expand...

I wonder if an admincp option to "Defer Alert Processing?" would be doable... The only other thing I can do I guess is to build a more robust server and hope that helps. Thanks guys for your input.
 
Upvote 0 Downvote
The following SQL can give you an idea on the number of forum watchers you have:

Code: 
SELECT xf_forum_watch.node_id, xf_node.title, count(*)
FROM xf_forum_watch
join xf_node on xf_node.node_id = xf_forum_watch.node_id
where xf_forum_watch.send_alert = 1
group by xf_forum_watch.node_id
order by count(*) desc;

And the following for thread watchers for receive alerts in new posts:
Code: 
SELECT xf_thread_watch.thread_id, xf_thread.title, count(*)
FROM xf_thread_watch
join xf_thread on xf_thread.thread_id = xf_thread_watch.thread_id
where not exists (select * from xf_user_alert_optout where alert in ('post_insert','post' ) and xf_user_alert_optout.user_id = xf_thread_watch.user_id)
group by xf_thread_watch.thread_id
order by count(*) desc
limit 20;


A 2gb Linode VM appears capable of easily handling alerting threads with ~828 watches who require alerting on a thread reply, and new thread watching only peaks at ~70 or so.

Obviously getting hard timings on 'sendNotificationToWatchUsersOnReply' and 'sendNotificationToWatchUsersOnMessage' would be best before requiring an addon to change this functionality to use the deferred system.
 
Last edited:
Upvote 0 Downvote
Xon said:
The following SQL can give you an idea on the number of forum watchers you have:

Code: 
SELECT xf_forum_watch.node_id, xf_node.title, count(*)
FROM xf_forum_watch
join xf_node on xf_node.node_id = xf_forum_watch.node_id
where xf_forum_watch.send_alert = 1
group by xf_forum_watch.node_id
order by count(*) desc;

And the following for thread watchers for receive alerts in new posts:
Code: 
SELECT xf_thread_watch.thread_id, xf_thread.title, count(*)
FROM xf_thread_watch
join xf_thread on xf_thread.thread_id = xf_thread_watch.thread_id
where not exists (select * from xf_user_alert_optout where alert in ('post_insert','post' ) and xf_user_alert_optout.user_id = xf_thread_watch.user_id)
group by xf_thread_watch.thread_id
order by count(*) desc
limit 20;


A 2gb Linode VM appears capable of easily handling alerting threads with ~828 watches who require alerting on a thread reply, and new thread watching only peaks at ~70 or so.

Obviously getting hard timings on 'sendNotificationToWatchUsersOnReply' and 'sendNotificationToWatchUsersOnMessage' would be best before requiring an addon to change this functionality to use the deferred system.
Click to expand...

That's interesting. Thank you for your insight. When you say "only peaks at 70 or so", do you mean it starts lagging/slowing?

Either an addon or hardware, one way or the other we need to fix this asap. Today we had the same threads posted 14 times, while users waited for visual confirmation. What if I took our RAID 1 SSD's and put the SSD's with two more in RAID 10? Writes are faster with RAID 10, right?
 
Upvote 0 Downvote
kontrabass said:
That's interesting. Thank you for your insight. When you say "only peaks at 70 or so", do you mean it starts lagging/slowing?

Either an addon or hardware, one way or the other we need to fix this asap. Today we had the same threads posted 14 times, while users waited for visual confirmation. What if I took our RAID 1 SSD's and put the SSD's with two more in RAID 10? Writes are faster with RAID 10, right?
Click to expand...

If disk performance is in question then you should run a benchmark first. Slow disk performance could certainly be an issue here.

Actually, it may be worth getting debug information from the "add thread" request. That would reveal times for individual queries including alerts.

I can debug this if you give me FTP and admin access.
 
Upvote 0 Downvote
@Mike , @Jake Bunce figured this out. It's TAPATALK. @#$#% Tapatalk. I would have never guessed, given that this only happens in select subforums with lots of watchers. But as Jake told me, Tapatalk has a bug where alerts are duplicated/triplicated etc. After I disable Tapatalk, thread creation in the suspect forum is fast as lightning.

Whew. I'm sorry for blaming your software Mike :whistle:. I'm truly grateful for all the tremendous support and help that comes from the XF community and staff. Thank you all.
 
Last edited:
Upvote 0 Downvote
FYI, this is the original report that I remembered about Tapatalk and alerts:

https://xenforo.com/community/threads/upgraded-to-1-1-3-from-1-1-2-double-alerts.35861/

You will see there are some links to Tapatalk's site regarding fixes.

I never actually established the nature of problem with Tapatalk. But I remembered at least one version of Tapatalk had problems with alerts, and kontra's problem was likely related to alerts. So it was just a guess based on that.
 
Upvote 0 Downvote
Daniel Hood said:
As someone that has never used Tapatalk and only read about it, it seems like some incredibly bad software. :/
Click to expand...

There was a time when the Tapatalk BYO (ie branded for talkbass) app was so buggy it was laughable. Recently (in the last 6 months) it's finally gotten to the point where it's reasonably stable and we've actually had a few good 5 star reviews in the app stores. And we've pulled in new users through the app stores.

However, their plugin is certainly a different story. I'm no expert, but IMHO, there are ongoing security vulnerabilities and this latest bug (which apparently hasn't been fixed in over a year) is the last straw. Heck, the version they have here in the xenforo resource area right now has a known XSS vulnerability. It's been fixed at tapatalk.com but who knows when they'll update it here. Over the past year security issues have been swept under the carpet, NO mention to us admins that we're in danger, and sometimes from what I've read, there's not even a version update (just a quiet file fix).

Boo, hiss.

Yay responsive XenForo!

(Now I get the fun job of telling our 2-3,000 daily app users that we're going to wipe the app) :coffee:o_O
 
Upvote 0 Downvote
You must log in or register to reply here.

Similar threads

X
Fixed Lock contention on thread-read tracking under very high load
Replies
1
Views
648
XF Bug Bot
XF Bug Bot
A
Not a bug Data too long for column 'data_value'
Replies
2
Views
420
au lait
A
Eran
  • Question Question
XF 2.2 Deadlock while import with more than 2 processes (Vbulletin4.x => xenforo 2.2)
Replies
3
Views
802
Eran
Eran
DJA
  • Question Question
XF 2.2 The table 'xf_session_activity' is full
Replies
2
Views
1K
DJA
DJA
PaulB
Fixed xf_session_activity: MySQL query error [1205]: Lock wait timeout exceeded; try restarting transaction
Replies
13
Views
2K
PaulB
PaulB
Top Bottom