PHP-FPM Problem

Sheratan

Sheratan

Well-known member
From the beginning, I used default php5-fpm configuration in www.conf
Code: 
pm = dynamic
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3

2 days ago, some warning appeared in php5-fpm.log
Code: 
[16-Feb-2014 09:02:11] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it

I trying to tweak my www.conf
Code: 
pm = dynamic
pm.max_children = 10
pm.start_servers = 3
pm.min_spare_servers = 2
pm.max_spare_servers = 5
pm.max_requests = 500

But the the warning is still there:

Code: 
[17-Feb-2014 09:24:29] WARNING: [pool www] server reached pm.max_children setting (10), consider raising it

Then I try to change pm.max_children to
Code: 
pm.max_children = 13

Still warning...
Code: 
[18-Feb-2014 15:47:27] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 10 total children
[18-Feb-2014 15:47:28] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 12 total children
[18-Feb-2014 15:47:29] WARNING: [pool www] server reached pm.max_children setting (13), consider raising it
[18-Feb-2014 15:49:58] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 10 total children
[18-Feb-2014 15:49:59] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 12 total children
[18-Feb-2014 15:50:00] WARNING: [pool www] server reached pm.max_children setting (13), consider raising it

Any suggestion?

It's a 1GB RAM VPS, running PHP 5.5.9 with xcache 3.1.0. My xenforo forum is the only website in there. 25-15 user active each day, with average 40-50 post each day.
 
It would really require more information on your VPS.

Based on 1GB RAM, try...
Code: 
pm.max_children = 32

There's several place where you can optimise settings (PHP, NGINX, etc). Most people stick the default values and don't read up on how to calculate values for the amount of resources available to you.
 
Sheratan said:
Do you have any formula?
Click to expand...

I'm sure, we had one. But I don't remember. ;-)

Our needs are much larger than your's. We run 4 load-balanced dedicated web servers for our forum and each of them uses this php-fpm setting.
 
You should not need 256 max FPM workers for 15-25 users per day. You have a problem that needs addressing. Inflating your server workers to far beyond reasonable limits for your traffic isn't going to solve it.

Have you actually looked at your FPM process usage? Are you restarting PHP-FPM (not Nginx) after changing your config? Have you read the output of PHPinfo to make sure the settings reported being used by the PHP handler is what's in your config? Are you running analytics software so you can see your actual traffic instead of just XenForo statistics?

Take these steps:

Install htop. It's in the Debian repos. Run htop -u followed by your web server user. I think on Debian it's www-data. I'm not sure. I compile my server stack from source so I don't know the package defaults. This will show you all the running processes. Count the PHP-FPM workers and note the "time" field values. That will tell you how long each process has been running.

Create a phpinfo.php page and post the results from loading it in a browser. It will list every config option recognized by your PHP installation:
Code: 
<?php
phpinfo();
?>

Run Google Analytics or even better, if you're comfortable setting it up, Piwik Analytics. This will shed light on any traffic hitting your site that isn't from your users. It's possible your site is being targeted by spam bots or something else less sinister that is causing PHP-FPM to believe it needs to ramp up in processes.
 
I think I've been under some kind of attack. A spam attack, perhaps. Lot's of guest from static.softlayer and a lot of IP from OVH trying to register to my forum.
 
Sheratan said:
I think I've been under some kind of attack. A spam attack, perhaps. Lot's of guest from static.softlayer and a lot of IP from OVH trying to register to my forum.
Click to expand...

Nginx is easy as store-bought pie to block addresses on. Try use the deny config option to filter out the shady traffic you're getting. you can filter SoftLayer wholesale and if any other spam is coming from a predictable range of IPs you can block them with a simple filter as well. This will prevent the connections from ever reaching the PHP handler. If that solves your problem you can implement the same changes at the firewall level which will free up even more resources by preventing them from ever hitting your web server in the first place.
 
Ended up blocking all 5.10.83.xyz range

Code: 
location / }
deny 5.10.83.1/32;
                deny 5.10.83.2/31;
                deny 5.10.83.4/30;
                deny 5.10.83.8/29;
                deny 5.10.83.16/28;
                deny 5.10.83.32/27;
                deny 5.10.83.64/26;
                deny 5.10.83.128/25;
                allow all;
.... }
 
There are no reached pm.max_children setting anymore, replaced by this:

Code: 
[24-Feb-2014 00:51:16] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 23 total children
[24-Feb-2014 00:51:17] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 6 idle, and 31 total children
[24-Feb-2014 19:17:52] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 7 idle, and 23 total children
[24-Feb-2014 21:08:55] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 7 idle, and 23 total children
[26-Feb-2014 22:02:17] NOTICE: [pool www] child 45632 exited with code 0 after 98661.098606 seconds from start
[26-Feb-2014 22:02:17] NOTICE: [pool www] child 47169 started
[26-Feb-2014 22:02:25] NOTICE: [pool www] child 45630 exited with code 0 after 98670.043448 seconds from start
[26-Feb-2014 22:02:25] NOTICE: [pool www] child 47170 started
[26-Feb-2014 22:04:44] NOTICE: [pool www] child 45631 exited with code 0 after 98809.710953 seconds from start
[26-Feb-2014 22:04:44] NOTICE: [pool www] child 47171 started
[27-Feb-2014 10:41:28] NOTICE: [pool www] child 46441 exited with code 0 after 99673.498877 seconds from start
[27-Feb-2014 10:41:28] NOTICE: [pool www] child 47916 started
[27-Feb-2014 10:41:35] NOTICE: [pool www] child 46440 exited with code 0 after 99682.877664 seconds from start
[27-Feb-2014 10:41:35] NOTICE: [pool www] child 47917 started
[27-Feb-2014 10:41:45] NOTICE: [pool www] child 46442 exited with code 0 after 99680.437769 seconds from start
[27-Feb-2014 10:41:45] NOTICE: [pool www] child 47918 started
[27-Feb-2014 10:42:09] NOTICE: [pool www] child 46438 exited with code 0 after 99719.989047 seconds from start
[27-Feb-2014 10:42:09] NOTICE: [pool www] child 47919 started
[27-Feb-2014 10:56:40] NOTICE: [pool www] child 46443 exited with code 0 after 100573.755350 seconds from start
[27-Feb-2014 10:56:40] NOTICE: [pool www] child 47922 started
[27-Feb-2014 14:50:34] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 23 total children
[27-Feb-2014 14:50:35] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 4 idle, and 31 total children
[28-Feb-2014 00:55:14] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 23 total children

www.conf
Code: 
pm.max_children = 64
pm.start_servers = 12
pm.min_spare_servers = 8
pm.max_spare_servers = 16
pm.max_requests = 768
 
HittingSmoke said:
Take these steps:

Install htop. It's in the Debian repos. Run htop -u followed by your web server user. I think on Debian it's www-data. I'm not sure. I compile my server stack from source so I don't know the package defaults. This will show you all the running processes. Count the PHP-FPM workers and note the "time" field values. That will tell you how long each process has been running.
Click to expand...
16 PHP-FPM process, with average "time" 15-25 minutes

HittingSmoke said:
Create a phpinfo.php page and post the results from loading it in a browser. It will list every config option recognized by your PHP installation:
Code: 
<?php
phpinfo();
?>
Click to expand...
The page loaded smooth. What configuration should I notice?

HittingSmoke said:
Run Google Analytics or even better, if you're comfortable setting it up, Piwik Analytics. This will shed light on any traffic hitting your site that isn't from your users. It's possible your site is being targeted by spam bots or something else less sinister that is causing PHP-FPM to believe it needs to ramp up in processes.
Click to expand...
This is done. It's softlayer bot. :D
 
Sheratan said:
There are no reached pm.max_children setting anymore, replaced by this:

Code: 
[24-Feb-2014 00:51:16] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 23 total children
[24-Feb-2014 00:51:17] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 6 idle, and 31 total children
[24-Feb-2014 19:17:52] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 7 idle, and 23 total children
[24-Feb-2014 21:08:55] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 7 idle, and 23 total children
[26-Feb-2014 22:02:17] NOTICE: [pool www] child 45632 exited with code 0 after 98661.098606 seconds from start
[26-Feb-2014 22:02:17] NOTICE: [pool www] child 47169 started
[26-Feb-2014 22:02:25] NOTICE: [pool www] child 45630 exited with code 0 after 98670.043448 seconds from start
[26-Feb-2014 22:02:25] NOTICE: [pool www] child 47170 started
[26-Feb-2014 22:04:44] NOTICE: [pool www] child 45631 exited with code 0 after 98809.710953 seconds from start
[26-Feb-2014 22:04:44] NOTICE: [pool www] child 47171 started
[27-Feb-2014 10:41:28] NOTICE: [pool www] child 46441 exited with code 0 after 99673.498877 seconds from start
[27-Feb-2014 10:41:28] NOTICE: [pool www] child 47916 started
[27-Feb-2014 10:41:35] NOTICE: [pool www] child 46440 exited with code 0 after 99682.877664 seconds from start
[27-Feb-2014 10:41:35] NOTICE: [pool www] child 47917 started
[27-Feb-2014 10:41:45] NOTICE: [pool www] child 46442 exited with code 0 after 99680.437769 seconds from start
[27-Feb-2014 10:41:45] NOTICE: [pool www] child 47918 started
[27-Feb-2014 10:42:09] NOTICE: [pool www] child 46438 exited with code 0 after 99719.989047 seconds from start
[27-Feb-2014 10:42:09] NOTICE: [pool www] child 47919 started
[27-Feb-2014 10:56:40] NOTICE: [pool www] child 46443 exited with code 0 after 100573.755350 seconds from start
[27-Feb-2014 10:56:40] NOTICE: [pool www] child 47922 started
[27-Feb-2014 14:50:34] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 23 total children
[27-Feb-2014 14:50:35] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 4 idle, and 31 total children
[28-Feb-2014 00:55:14] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 23 total children

www.conf
Code: 
pm.max_children = 64
pm.start_servers = 12
pm.min_spare_servers = 8
pm.max_spare_servers = 16
pm.max_requests = 768
Click to expand...

Are you sure you're not still getting hammered with spam bots? Also, is that 16 PHP-FPM process count from your peak time? You should have a hell of a lot more than 16 processes running if your max is 64 and your www pool is I/O locked.

The notices are normal. Those are processes being recycled from pm.max_requests.
 
You must log in or register to reply here.

Similar threads

K
  • Question Question
Xenforo throws MySQL timeout errors while searching even with Elasticsearch installed
Replies
1
Views
1K
KevinP
K
Sunka
  • Question Question
MG 2.1 Upgrading to XFMG 2.1.7 stuck through ACP/browser
Replies
5
Views
1K
eL_
eL_
Marcus
php-fpm: Avoiding "Cannot allocate memory" Error with better configuration
Replies
1
Views
8K
Floren
Floren
Nentify
  • Question Question
XF 1.2 WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
Replies
6
Views
3K
Nentify
Nentify
R
Php-fpm configuration: encountered and error
6 7 8
Replies
140
Views
31K
Jim Boy
Jim Boy
Top Bottom