XF 2.1 High CPU Usage - What is my host suggesting? And is it safe?

Sal Collaziano

Well-known member
My server's CPU usage has been quite high lately and I asked my dedicated server host what could be done to help. Here's what they suggested:

In an attempt to reduce CPU usage, I would suggest tuning Apache with respect to the MPM used (i.e. Apache's configuration should be configured via an includes file when using "event" or "worker" MPMs) and switching from the "worker" MPM to "event". Additionally, I would highly suggest enabling "KeepAlive" for Apache as that can help reduce CPU usage.

Is there any down side to this? Anything that could affect XenForo negatively?
 
I know this doesn't answer your question specifically but as a suggestion you may want to look at your logs and bot activity and consider blocking some overly aggressive or unfamiliar bots in your robots.txt and .htaccess. If a bunch of indexing bots are constantly hitting your server, it could be part of what is driving up your usage.
 
You don't have to ditch Apache to move to php-fpm and cPanel makes that dead simple. Make sure you're running CSF and blocking countries that have no business being there. The other thing that helped on one forum I manage was installing and enabling a bare bones Redis cache using the options from the XF2 manual:
Redis
Code:
$config['cache']['enabled'] = true;
$config['cache']['provider'] = 'Redis';
$config['cache']['config'] = [
    'host' => '127.0.0.1',
    'password' => 'password'
];

Redis has a number of additional configuration options. The following list will demonstrate the default values of all of the supported configuration items available:
Code:
'host' => '',
'port' => 6379,
'timeout' => 0.0,
'password' => '',
'database' => 0,
'persistent' => false,
'persistent_id' => ''

One other thing I did was disallow several more aggressive bots in robots.txt, although a lot of bad bots ignore it:

Code:
User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: AspiegelBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: proximic
Disallow: /

User-agent: trendictionbot0.5.0
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: AspiegelBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: YandexBot/3.0
Disallow: /

User-agent: proximic
Disallow: /

User-agent: trendictionbot0.5.0
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: TTD-Content
Disallow: /

User-agent: grapeshot
Disallow: /

User-agent: ADmantX
Disallow:/

User-agent: Mediapartners-Google
Allow: /

User-agent: *
Disallow: /whats-new/
Disallow: /account/
Disallow: /attachments/
Disallow: /goto/
Disallow: /posts/
Disallow: /login/
Disallow: /admin.php
Allow: /

You can also block bots in .htaccess but for me that ended up creating more problems than it solved.
 
I know this doesn't answer your question specifically but as a suggestion you may want to look at your logs and bot activity and consider blocking some overly aggressive or unfamiliar bots in your robots.txt and .htaccess. If a bunch of indexing bots are constantly hitting your server, it could be part of what is driving up your usage.
Thank you. I may be using an add-on by AndyB for bots...
Something must have changed

If you havent changed anything in terms of config/upgrades or addons then what @StarArmy said is most likely.

Investigate a bot throttle or block position
Thank you. I'm looking into this...
You don't have to ditch Apache to move to php-fpm and cPanel makes that dead simple. Make sure you're running CSF and blocking countries that have no business being there. The other thing that helped on one forum I manage was installing and enabling a bare bones Redis cache using the options from the XF2 manual:


One other thing I did was disallow several more aggressive bots in robots.txt, although a lot of bad bots ignore it:

Code:
User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: AspiegelBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: proximic
Disallow: /

User-agent: trendictionbot0.5.0
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: AspiegelBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: YandexBot/3.0
Disallow: /

User-agent: proximic
Disallow: /

User-agent: trendictionbot0.5.0
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: TTD-Content
Disallow: /

User-agent: grapeshot
Disallow: /

User-agent: ADmantX
Disallow:/

User-agent: Mediapartners-Google
Allow: /

User-agent: *
Disallow: /whats-new/
Disallow: /account/
Disallow: /attachments/
Disallow: /goto/
Disallow: /posts/
Disallow: /login/
Disallow: /admin.php
Allow: /

You can also block bots in .htaccess but for me that ended up creating more problems than it solved.
Thank you. I'll be doing some reading on caching now... :)
 
Back
Top Bottom