XF 2.1 High CPU Usage - What is my host suggesting? And is it safe?

Sal Collaziano

Sal Collaziano

Well-known member
My server's CPU usage has been quite high lately and I asked my dedicated server host what could be done to help. Here's what they suggested:

In an attempt to reduce CPU usage, I would suggest tuning Apache with respect to the MPM used (i.e. Apache's configuration should be configured via an includes file when using "event" or "worker" MPMs) and switching from the "worker" MPM to "event". Additionally, I would highly suggest enabling "KeepAlive" for Apache as that can help reduce CPU usage.
Click to expand...

Is there any down side to this? Anything that could affect XenForo negatively?
 
Sort by date Sort by votes
I know this doesn't answer your question specifically but as a suggestion you may want to look at your logs and bot activity and consider blocking some overly aggressive or unfamiliar bots in your robots.txt and .htaccess. If a bunch of indexing bots are constantly hitting your server, it could be part of what is driving up your usage.
 
Upvote 0 Downvote
You don't have to ditch Apache to move to php-fpm and cPanel makes that dead simple. Make sure you're running CSF and blocking countries that have no business being there. The other thing that helped on one forum I manage was installing and enabling a bare bones Redis cache using the options from the XF2 manual:

Cache support | Manual | XenForo

End-user documentation for XenForo
xenforo.com xenforo.com
Redis
Code: 
$config['cache']['enabled'] = true;
$config['cache']['provider'] = 'Redis';
$config['cache']['config'] = [
    'host' => '127.0.0.1',
    'password' => 'password'
];

Redis has a number of additional configuration options. The following list will demonstrate the default values of all of the supported configuration items available:
Code: 
'host' => '',
'port' => 6379,
'timeout' => 0.0,
'password' => '',
'database' => 0,
'persistent' => false,
'persistent_id' => ''
Click to expand...

One other thing I did was disallow several more aggressive bots in robots.txt, although a lot of bad bots ignore it:

Code: 
User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: AspiegelBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: proximic
Disallow: /

User-agent: trendictionbot0.5.0
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: AspiegelBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: YandexBot/3.0
Disallow: /

User-agent: proximic
Disallow: /

User-agent: trendictionbot0.5.0
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: TTD-Content
Disallow: /

User-agent: grapeshot
Disallow: /

User-agent: ADmantX
Disallow:/

User-agent: Mediapartners-Google
Allow: /

User-agent: *
Disallow: /whats-new/
Disallow: /account/
Disallow: /attachments/
Disallow: /goto/
Disallow: /posts/
Disallow: /login/
Disallow: /admin.php
Allow: /

You can also block bots in .htaccess but for me that ended up creating more problems than it solved.
 
Upvote 0 Downvote
StarArmy said:
I know this doesn't answer your question specifically but as a suggestion you may want to look at your logs and bot activity and consider blocking some overly aggressive or unfamiliar bots in your robots.txt and .htaccess. If a bunch of indexing bots are constantly hitting your server, it could be part of what is driving up your usage.
Click to expand...
Thank you. I may be using an add-on by AndyB for bots...
andybond said:
Something must have changed

If you havent changed anything in terms of config/upgrades or addons then what @StarArmy said is most likely.

Investigate a bot throttle or block position
Click to expand...
Thank you. I'm looking into this...
djbaxter said:
You don't have to ditch Apache to move to php-fpm and cPanel makes that dead simple. Make sure you're running CSF and blocking countries that have no business being there. The other thing that helped on one forum I manage was installing and enabling a bare bones Redis cache using the options from the XF2 manual:

Cache support | Manual | XenForo

End-user documentation for XenForo
xenforo.com xenforo.com


One other thing I did was disallow several more aggressive bots in robots.txt, although a lot of bad bots ignore it:

Code: 
User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: AspiegelBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: proximic
Disallow: /

User-agent: trendictionbot0.5.0
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: AspiegelBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: YandexBot/3.0
Disallow: /

User-agent: proximic
Disallow: /

User-agent: trendictionbot0.5.0
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: TTD-Content
Disallow: /

User-agent: grapeshot
Disallow: /

User-agent: ADmantX
Disallow:/

User-agent: Mediapartners-Google
Allow: /

User-agent: *
Disallow: /whats-new/
Disallow: /account/
Disallow: /attachments/
Disallow: /goto/
Disallow: /posts/
Disallow: /login/
Disallow: /admin.php
Allow: /

You can also block bots in .htaccess but for me that ended up creating more problems than it solved.
Click to expand...
Thank you. I'll be doing some reading on caching now... :)
 
Upvote 0 Downvote
You must log in or register to reply here.
Top Bottom