1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Server has high load

Discussion in 'Server Configuration and Hosting' started by Marks534, Feb 23, 2016.

  1. Marks534

    Marks534 Member

    Hi,

    I have quite a beefy machine, did some simple MySQL optimizations, tried caching, etc yet my machine has a high CPU usage and load. I have 700+- members at peaks. 25k registered and 880k posts. I do have elasticsearch enabled.


    So yeah, quite a machine yet I'm getting slow loading times and high CPU usage. I use MaxCDN which does a great job so far. See blow all my configuration files involved.

    Code:
    [mysqld]
    skip-external-locking
    query_cache_type = 0
    query_cache_size = 0
    query_cache_limit = 1M
    query_cache_strip_comments = 0
    thread_cache_size = 256
    max_heap_table_size = 128M
    tmp_table_size = 128M
    connect_timeout = 10
    max_connections = 500
    interactive_timeout = 300
    wait_timeout = 300
    thread_stack = 256k
    table_open_cache = 24000
    open_files_limit = 24000
    key_buffer_size = 768M
    max_allowed_packet = 4M
    sort_buffer_size = 1M
    join_buffer_size = 512k
    #net_buffer_length = 64K
    read_buffer_size = 512K
    #read_rnd_buffer_size = 512K
    myisam_sort_buffer_size = 128M
    innodb_buffer_pool_size = 8G
    innodb_thread_concurrency = 7
    innodb_buffer_pool_instances = 4
    innodb_flush_method=O_DIRECT
    innodb_file_per_table=1
    innodb_log_buffer_size=8M

    This is a configuration I've assembled from several threads on here and tweaked it to my server specifications.

    Code:
    [site]
    listen = 127.0.0.1:9001
    listen.allowed_clients = 127.0.0.1
    
    user = admin
    group = admin
    
    pm = dynamic
    pm.max_children = 2000
    pm.start_servers = 50
    pm.min_spare_servers = 50
    pm.max_spare_servers = 90
    
    env[HOSTNAME] = $HOSTNAME
    env[PATH] = /usr/local/bin:/usr/bin:/bin
    env[TMP] = /tmp
    env[TMPDIR] = /tmp
    env[TEMP] = /tmp
    


    Let me know if you guys need any other configuration files.

    The setup is a VestaCP installation. As stated above, i've tried caching systems and none really improved load significantly so I removed them temporarily until someone can help me to get it configured and setup properly.

    htop

    [​IMG]
     
  2. RoldanLT

    RoldanLT Well-Known Member

    Try this for PHP-FPM:
    Code:
    pm = dynamic
    pm.max_children = 32
    pm.start_servers = 8
    pm.min_spare_servers = 4
    pm.max_spare_servers = 16
    pm.max_requests = 1000
    
    With similar PHP-FPM config, on one large forum I have.
    Online stats last 15 minutes: Total: 4,477 (members: 1,162, guests: 3,242)
    Average system load: 2% / 0.17 0.28 0.27

    For Mysql:
    Code:
    innodb_io_capacity = 3000
    innodb_read_io_threads = 64
    innodb_write_io_threads = 64
    innodb_flush_neighbors = 0
    For server, I highly recommend PHP 7.0 w/ Zend Opcache
    Memcached or Redis for Data Caching that should help a lot!

    For Nginx and Guest Caching: https://xenforo.com/community/threads/110806/


    This is really small board and can be hosted on Linode 4GB plan.
    I have mine large a bit than this just hosted on Linode 4GB/$40 plan only and works very fine.
    Current Average system load: 10% 0.40 0.38 0.41 (Just a bit high than normal usage as I just upgraded XF and restarted PHP-FPM and all Cache)
     
    Last edited: Feb 23, 2016
  3. Marks534

    Marks534 Member


    Thanks for your help. I've applied the php-fpm config options and I can notice a huge improvement on performance. As for the MySQL config, can you check what I currently have is okay? Mainly the innodb settings. For some reason, when I applied your settings mariadb refused to restart.
     
  4. RoldanLT

    RoldanLT Well-Known Member

    MariaDB version your using?
    My current config is use on 10.1.11.
     
  5. Marks534

    Marks534 Member

    5.5.44-MariaDB
     
  6. RoldanLT

    RoldanLT Well-Known Member

    Use this instead:
    Code:
    innodb_io_capacity = 3000
    innodb_read_io_threads = 64
    innodb_write_io_threads = 64
     
  7. The Forum Heroes

    The Forum Heroes Well-Known Member

    Ditch a lot of that RAM, upgrade to a better processor (hex core) and add 2 more SSD's to move to RAID10 and you will see a big improvement. 64GB RAM is overkill for a server with 700 online. You should only need 8GB RAM at very most. Upgrading processor to a hex core, will give you more CPU power then a E5 by a long shot.
     
  8. Marks534

    Marks534 Member

    Applied the fixes but don't see any direct improvements via htop.

    I don't think my server is the cause of bad performance, it's probably just configuration.
     
  9. Marks534

    Marks534 Member

  10. The Forum Heroes

    The Forum Heroes Well-Known Member

    What database is causing the load? How large is it? Did you run mysqlchk on it to assure no crashed tables?
     
  11. RoldanLT

    RoldanLT Well-Known Member

    Your PHP version? Opcode caching use?
     
  12. Marks534

    Marks534 Member

    PHP 5.6.17 with OpCache 7.0.6-dev
     
    RoldanLT likes this.
  13. Marks534

    Marks534 Member

    @RoldanLT It really seems like a MySQL issue. I sometimes get all my MySQL instances hitting +200% CPU
     
  14. RoldanLT

    RoldanLT Well-Known Member

    Try to upgrade your MariaDB to 10.1.11 first.
     
  15. Joeychgo

    Joeychgo Well-Known Member

    That much better than 5.5.48-MariaDB?
     
  16. Marks534

    Marks534 Member

    I honestly think it would improve performance a little however I don't think that's causing my CPU usage to hit +200%.
     
  17. Joeychgo

    Joeychgo Well-Known Member

    I was asking because I'm running 5.5.48-MariaDB and things seem a little slow. Don't want to hijack your thread though.
     
    Michael McLevie likes this.
  18. Tracy Perry

    Tracy Perry Well-Known Member

    DDOS issue?
    Code:
    netstat -ntu | grep :80 | grep -v LISTEN | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | grep -v 127.0.0.1
    and see how many active connections from IP's you have... I've seen similar happen on a system that was having a layer 7 DDOS attack.
    If running SSL, change :80 to :443.
     
    WSWD likes this.
  19. RoldanLT

    RoldanLT Well-Known Member

    Based on experience, Yes.
     
  20. Marks534

    Marks534 Member

    That was also my first thought however I think it's very unlikey to get hit with Layer 7 attacks 24/7. I have a normal amount of connections.

    I'm seriously out of ideas.
     

Share This Page