My dedicated server is slow

cmeinck

Well-known member
I run a number of sites on my own dedicated server. The specs are:
  • Dual Processor, Quad Core Xeon Clovertown 5335
  • 4GB RAM
  • Seagate Cheetah 15K
I recently had them do an OS reload on this server and moved both the OS, mySQL and files to the Cheetah, utilizing the second drive for backups only.
I run a number of sites on this, none of which are what I would say heavily trafficed.
I have an IT group that manages my servers and they handled the migration, etc. My concern is that since the move, things have been dog slow. Both the front end of the sites and backend. These guys are great for support issues, but aren't able to deal with slowness -- since there are so many factors which could contribute.
There are times when the site's performance is ok, but I'm confident that something is off.
What's the best way to troubleshoot a speed issue? Are there any reputable companies whose primary service is to optimize sites?
Here are my Android forums experiencing issues. Would love any and all feedback.
 
First you have to find out what causes the slowness. Most of the time it's either a) CPU load (very unlikely, currently you only have 20 users online, this server should fly!) or b) IO problems (very common).

What does top and vmstat say on the command line?
 
Here are the results.

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 0 88 337664 86324 3020240 0 0 76 115 5 5 13 2 85 0 0
root@redmond [~]#
---
top - 11:00:50 up 10 days, 6:47, 1 user, load average: 1.10, 1.40, 1.66
Tasks: 213 total, 3 running, 210 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.7%us, 2.0%sy, 0.2%ni, 84.5%id, 0.5%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 4148812k total, 3842688k used, 306124k free, 86140k buffers
Swap: 1052248k total, 88k used, 1052160k free, 3019696k cached
 

Attachments

High load for such a server.
How much memory have you dedicated to mysql min/max? Currently 3GB is used for caching, that's not bad, but maybe you could give mysql a bit more ram instead of using it for the OS cache.

BTW - an "IT group" who isn't able to handle a slow server isn't worth much.
 
What are those 3 php processes doing? You aren't running PHP via CGI are you? (Am not too familiar with Apache so forgive me if mod_php uses separate processes like that :p)

Also post your my.cnf
 
Very frustrated with the support I'm receiving. They seem reluctant to acknowledge how slow the sites are now. I've got about 8 sites on the server. Some see hardly any traffic, so they shouldn't have much impact at all. The site I referenced is actually the busiest site of them all. I'm seeing slow loading when editing styles properties -- stuff I never see on a normal site.

The previous configuration had one hard drive for the OS and I believe they used a second drive for mySQL. All of it is on one drive now -- which should actually be a faster drive at 15k.

The whole situation is maddening.

Can anyone recommend someone who has experience in optimizing servers, sites? The server is quite expensive and the fact that it runs like a $10 shared hosting plan is just not going to cut it.

I have Litespeed licenses that I'm going to renew and replace Apache. I don't want to put a bandaid on a bigger issue. My thought is, correct the current problem and then aim for massive improvements.
 
Below is my.cnf.

These figures seem much different then what I'm seeing in this Apache tuning thread.

Code:
# Example MySQL config file for large systems.
#
# This is for a large system with memory = 512M where the system runs mainly
# MySQL.
#
# MySQL programs look for option files in a set of
# locations which depend on the deployment platform.
# You can copy this option file to one of those
# locations. For information about these locations, see:
# http://dev.mysql.com/doc/mysql/en/option-files.html
#
# In this file, you can use all long options that a program supports.
# If you want to know which options a program supports, run the program
# with the "--help" option.

# The following options will be passed to all MySQL clients
[client]
#password    = your_password
port    = 3306
socket    = /var/lib/mysql/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
local-infile=0
max_connections=1000
port    = 3306
socket    = /var/lib/mysql/mysql.sock
skip-locking
key_buffer_size = 256M
max_allowed_packet = 16M
table_open_cache = 256
sort_buffer_size = 2M
read_buffer_size = 2M

join_buffer_size=2M
max_heap_table_size=256M
tmp_table_size=256M
 
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
#
#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
log-bin=mysql-bin

# binary logging format - mixed recommended
binlog_format=mixed

# required unique id between 1 and 232 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id    = 1

# Replication Slave (comment out master section to use this)
#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
# the syntax is:
#
# CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
# MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
# where you replace <host>, <user>, <password> by quoted strings and
# <port> by the master's port number (3306 by default).
#
# Example:
#
# CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
# MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
# start replication for the first time (even unsuccessfully, for example
# if you mistyped the password in master-password and the slave fails to
# connect), the slave will create a master.info file, and any later
# change in this file to the variables' values below will be ignored and
# overridden by the content of the master.info file, unless you shutdown
# the slave server, delete master.info and restart the slaver server.
# For that reason, you may want to leave the lines below untouched
# (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 232 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id = 2
#
# The replication master for this slave - required
#master-host = <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user = <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password = <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port = <port>
#
# binary logging - not required for slaves, but recommended
#log-bin=mysql-bin

# Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /var/lib/mysql
#innodb_data_file_path = ibdata1:10M:autoextend
#innodb_log_group_home_dir = /var/lib/mysql
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 256M
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 64M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[myisamchk]
key_buffer_size = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout
 
That my.cnf is not optimised in the slightest.

Firstly, what is the size of all of your *.MYI files?
Secondly, I take it that you use InnoDB for XF?
Finally, are you using replication?
 
That my.cnf is not optimised in the slightest.

Firstly, what is the size of all of your *.MYI files?
Secondly, I take it that you use InnoDB for XF?
Finally, are you using replication?

That's what I thought. This is all above my head and it's tough because the IT group I employ is more reactive than proactive. I appreciate the help and have asked them to provide answers to 1 and 3. I'm almost certain I use InnoDB and noticed it was commented out. :(
 
That is hard. I have running a XF Forum with peaks of 90User online on a shared hoster for 1 Euro/Month.
I would quit this and go to another provider
 
I've initiated the changes in the Apache tuning thread and the innodb settings from Deebs outlines here. Fingers crossed. Can't be worse right?

Does anyone know how these changes will be affected when I switch to Litespeed from Apache?
 
Cmeinck,

Can you do a show status in MySQL for me as well please.

As for changes when you switch to Litespeed, it depends on the memory usage of all the applications. Your server only has 4gig of RAM so I hope that you didn't include "innodb_buffer_pool_size=6G" in your my.cnf. If you did change it to 2G for now.
 
Cmeinck,

Can you do a show status in MySQL for me as well please.

As for changes when you switch to Litespeed, it depends on the memory usage of all the applications. Your server only has 4gig of RAM so I hope that you didn't include "innodb_buffer_pool_size=6G" in your my.cnf. If you did change it to 2G for now.

Thanks, I just pinged them to make that change and to see if they were aware of the 500 Internal Server Error that's lighting up all my sites.

Thanks for your help. I'll get the show status in mySQL as soon as they finish all the changes and get the sites back online.
 
Wanted to circle back and thank everyone for their help. As it turned out, there was something misconfigured at the host. When SoftLayer made those corrections, I made changes noted and started using CloudFlare. The results have been excellent. There's probably some more performance that could be squeezed out, but I'm happy for now. I also noticed that Google Webmaster Tools showed the issue. The crawl speed was super low.

Now it's time to work on my other server!:D
 
I'm no mysql guru, but have you tried something like tuning-primer.sh to see if your my.cnf is optimized? (https://launchpad.net/mysql-tuning-primer). That, and a good mysql hacker should get you more optimized.. another idea is you can mount your filesystem with noatime (or relatime) to stop access time writes to the filesystem every time a file is accessed.
 
Top Bottom