XF 2.3 High CPU usage when jobs get added to XF_Job

S

Sicilian

New member
We recently moved from Vbulletin 4.2 to XF2.3.

We're having an issue that when a large amount of jobs get added to xf_job table we get high CPU usage.

If we try to send out a mailshot to all members the forum more or less grinds to a halt.

Today I decided to batch delete approx 66,500 inactive zero posters who haven't visited in 5 years or more. This has added 1000's of jobs to xf_job table.

The forum has slowed a lot and we're getting high CPU used by

/usr/sbin/mariadbd

Using around 50% cpu.

Any ideas why we are experiencing this issue? Surely these jobs should work their way through without using so much of the recources?

Forum is run on a dediciated server:

Brand DELL
Model Dell R210-II (2xLFF)
Processor 1x Intel Xeon E3-1230v2
RAM 16 GB

AlmaLinux v9.6.0 STANDARD standard
cPanel Version
126.0.16
 
Sort by date Sort by votes
Sicilian said:
Any ideas why we are experiencing this issue?
Click to expand...
M

Confirmed Thread 'Inefficient reaction cache rebuild on username change'

On a large forum with some 22 million posts and very high reaction usage, coupled with high volume of username changes (mostly from a high volume of account deletions), the process to update the reaction user cache on content is painfully inefficient and leads to frequent table locks that then cause performance and stability issues on the site.

We have an addon that allows users to delete their own accounts, which are queued and fed in in small batches (3 at a time one an hour) so as not to overwhelm the database. However even with queuing 3 at a time, we were seeing table locks every...
  • Like
K

Confirmed Thread 'Service XF:User\ChangeContent runs twice when renaming a user upon deletion'

This is a rather complicated bug, but I hope it's understandable.

Method XF\Service\User\Delete::delete sets entity options to avoid running jobs XF:UserRenameCleanup and XF:UserDeleteCleanup when renaming a user before deletion:
PHP: 
if ($this->renameTo)
{
    $user->reset();
    $user->setOption('admin_edit', true);
    $user->setOption('enqueue_rename_cleanup', false);
    $user->setOption('enqueue_delete_cleanup', false);

    $user->set('username', $this->renameTo);

After deletion,. those jobs are enqueued in method runPostDeleteJobs
PHP: 
if...
  • Like

Sicilian said:
Surely these jobs should work their way through without using so much of the recources?
Click to expand...
No :)

Btw: Your hardware is kinda stone age (the CPU is from 2012!) , that certainly doesn't help as well.
 
Upvote 0 Downvote
Thats a bit over my head.

What do we do, be paitent and let the jobs run though? Or do we have to start editing files? Which I'd prefer not to do.

We have worse issues trying to send out a Digest with Dragonbyte mail. 1000's of jobs end up in xf_job and the forum becomes unusable. We have to literally flush xf_job to stop all the emails so we can use the Forum again. Surely this can't be right?
 
Upvote 0 Downvote
Sicilian said:
What do we do, be paitent and let the jobs run though?
Click to expand...
Yep.

Sicilian said:
Or do we have to start editing files? Which I'd prefer not to do.
Click to expand...
The issue is complex and there is no patch available yet so:
Yeah, in theory you could patch the issue if you know exactly what you are doing.
But as you are asking I'd assume you don't have that knowledge, so: No.

Sicilian said:
We have worse issues trying to send out a Digest with Dragonbyte mail. 1000's of jobs end up in xf_job and the forum becomes unusable. We have to literally flush xf_job to stop all the emails so we can use the Forum again. Surely this can't be right?
Click to expand...
It's normal for XF 2.3 to create one job per email, if your system can't handle that I am sorry but you just might need more powerful hardware (for this usecase at least).

Do you use server based trigger?
If not I'd try that, this might help a bit.
 
Upvote 0 Downvote
You are mixing up things, you mail issue has nothing to do with the bugs I linked above.
It is normal that XF 2.3 generated one job per email, so if you are sending 1K emails you will have 1K jobs.
But this should not cause significant issues (at least not when using server based trigger).

Those bugs I linked are releated to
Today I decided to batch delete approx 66,500 inactive zero posters who haven't visited in 5 years or more.
Click to expand...
 
Upvote 0 Downvote
Then you just might need more powerful / up-to-date hardware, unrelated to the bugs I linked (which even kill up-to-date hardware if there is enough load).

You could still try to use a server based trigger though, this might help for the emails.

And check / optimize you MariaDB config.
 
Upvote 0 Downvote
Today I decided to batch delete approx 66,500 inactive zero posters who haven't visited in 5 years or more.
Click to expand...
That is a lot to run in one shot, break that stuff up by dates a little better and shoot for batches of 1000 to 1500 at a time.

You didn't mention if this was shared hosting so I am assuming it is since you are running cPanel. Do you have access to change any MySQL settings?
 
Upvote 0 Downvote
GrnEyedDvl said:
That is a lot to run in one shot, break that stuff up by dates a little better and shoot for batches of 1000 to 1500 at a time.

You didn't mention if this was shared hosting so I am assuming it is since you are running cPanel. Do you have access to change any MySQL settings?
Click to expand...
As per my initial post, see below. Yes have full access to MySQL settings. Its our own dedicated server.

Brand DELL
Model Dell R210-II (2xLFF)
Processor 1x Intel Xeon E3-1230v2
RAM 16 GB

AlmaLinux v9.6.0 STANDARD standard
cPanel Version
126.0.16
 
Upvote 0 Downvote
Sicilian said:
Brand DELL
Model Dell R210-II (2xLFF)
Processor 1x Intel Xeon E3-1230v2
RAM 16 GB
Click to expand...
That CPU was released in 2012. 16 RAM isn't bad, but how big of a site are we talking about?
Then there's the tuning involved for the database, etc. etc.
I'd start thinking about a path forward to get on newer hardware and probably better DB tuning.

Edit:
Kirby said:
Btw: Your hardware is kinda stone age (the CPU is from 2012!) , that certainly doesn't help as well.
Click to expand...
Oh, whoops!
 
Upvote 0 Downvote
ENF said:
That CPU was released in 2012. 16 RAM isn't bad, but how big of a site are we talking about?
Then there's the tuning involved for the database, etc. etc.
I'd start thinking about a path forward to get on newer hardware and probably better DB tuning.

Edit:

Oh, whoops!
Click to expand...
Its not a huge busy forum anymore. These days we're lucky to get 10 posts a day.
 
Upvote 0 Downvote
You must log in or register to reply here.

Similar threads

C
  • Question Question
XF 1.5 Extremely high CPU usage - don't want to get suspended again!
2
Replies
29
Views
3K
Slavik
Slavik
Back
Top Bottom