1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Big optimization problems!

Discussion in 'Server Configuration and Hosting' started by Skyuser, Mar 31, 2014.

  1. Skyuser

    Skyuser Member

    Recently i started to monitor my server activity because pages with 20 posts ( large threads) opens very slowly.
    Page Time: 1.3822s
    Memory: 4.1461 MB (Peak: 5.3929 MB)
    Queries (10, time: 0.0136s, 1.0%)

    I have DigitalOcean VPS with 1 Gb Ram, nginx 1.4.7 with ngx_pagespeed + Apache, MySQL 5.5.6, PHP 5.4 and APC.
    My.cnf settings:
    query_cache_size = 128M
    query_cache_limit = 2M
    thread_cache_size = 4
    innodb_buffer_pool_size = 300M
    innodb_additional_mem_pool_size = 15M
    innodb_flush_log_at_trx_commit = 2
    log-slow-queries = /var/log/mysqlslowlog/slow.log
    tuning-primer output:
    The slow query log is enabled.
    Current long_query_time = 10.000000 sec.
    You have 0 out of 4925 that take longer than 10.000000 sec. to complete
    Your long_query_time seems to be fine
    The binary update log is NOT enabled.
    You will not be able to do point in time recovery
    See http://dev.mysql.com/doc/refman/5.5/en/point-in-time-recovery.html
    Current thread_cache_size = 4
    Current threads_cached = 3
    Current threads_per_sec = 0
    Historic threads_per_sec = 0
    Your thread_cache_size is fine
    Current max_connections = 151
    Current threads_connected = 1
    Historic max_used_connections = 5
    The number of used connections is 3% of the configured maximum.
    You are using less than 10% of your configured max_connections.
    Lowering max_connections could help to avoid an over-allocation of memory
    See "MEMORY USAGE" section to make sure you are not over-allocating
    Current InnoDB index space = 19 M
    Current InnoDB data space = 60 M
    Current InnoDB buffer pool free = 89 %
    Current innodb_buffer_pool_size = 300 M
    Depending on how much space your innodb indexes take up it may be safe
    to increase this value to up to 2 / 3 of total system memory
    Max Memory Ever Allocated : 472 M
    Configured Max Per-thread Buffers : 405 M
    Configured Max Global Buffers : 459 M
    Configured Max Memory Limit : 864 M
    Physical Memory : 1006 M
    Max memory limit seem to be within acceptable norms
    Current MyISAM index space = 4 M
    Current key_buffer_size = 8 M
    Key cache miss rate is 1 : 55
    Key buffer free ratio = 87 %
    Your key_buffer_size seems to be fine
    Query cache is enabled
    Current query_cache_size = 128 M
    Current query_cache_used = 1 M
    Current query_cache_limit = 2 M
    Current Query cache Memory fill ratio = 1.47 %
    Current query_cache_min_res_unit = 4 K
    Query Cache is 23 % fragmented
    Run "FLUSH QUERY CACHE" periodically to defragment the query cache memory
    If you have many small queries lower 'query_cache_min_res_unit' to reduce fragmentation.
    Your query_cache_size seems to be too high.
    Perhaps you can use these resources elsewhere
    MySQL won't cache query results that are larger than query_cache_limit in size
    Current sort_buffer_size = 2 M
    Current read_rnd_buffer_size = 256 K
    Sort buffer seems to be fine
    Current join_buffer_size = 132.00 K
    You have had 79 queries where a join could not use an index properly
    You should enable "log-queries-not-using-indexes"
    Then look for non indexed joins in the slow query log.
    If you are unable to optimize your queries you may want to increase your
    join_buffer_size to accommodate larger joins in one pass.
    Note! This script will still suggest raising the join_buffer_size when
    ANY joins not using indexes are found.
    Current open_files_limit = 1024 files
    The open_files_limit should typically be set to at least 2x-3x
    that of table_cache if you have heavy MyISAM usage.
    Your open_files_limit value seems to be fine
    Current table_open_cache = 400 tables
    Current table_definition_cache = 400 tables
    You have a total of 229 tables
    You have 250 open tables.
    The table_cache value seems to be fine
    Current max_heap_table_size = 16 M
    Current tmp_table_size = 16 M
    Of 428 temp tables, 15% were created on disk
    Created disk tmp tables ratio seems fine
    Current read_buffer_size = 128 K
    Current table scan ratio = 26 : 1
    read_buffer_size seems to be fine
    Current Lock Wait ratio = 0 : 5220
    Your table locking seems to be fine
    mysqltuner.pl output
    -------- Storage Engine Statistics -------------------------------------------
    [--] Data in MyISAM tables: 5M (Tables: 8)
    [--] Data in InnoDB tables: 60M (Tables: 176)
    [--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
    [--] Data in MEMORY tables: 504K (Tables: 4)
    [!!] Total fragmented tables: 178
    -------- Security Recommendations  -------------------------------------------
    [OK] All database users have passwords assigned
    -------- Performance Metrics -------------------------------------------------
    [--] Up for: 25m 3s (5K q [3.918 qps], 644 conn, TX: 33M, RX: 2M)
    [--] Reads / Writes: 58% / 42%
    [--] Total buffers: 475.0M global + 2.7M per thread (151 max threads)
    [!!] Maximum possible memory usage: 880.8M (87% of installed RAM)
    [OK] Slow queries: 0% (0/5K)
    [OK] Highest usage of available connections: 3% (5/151)
    [OK] Key buffer size / total MyISAM indexes: 8.0M/5.0M
    [OK] Key buffer hit rate: 98.3% (6K cached / 104 reads)
    [OK] Query cache efficiency: 40.3% (1K cached / 2K selects)
    [OK] Query cache prunes per day: 0
    [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 187 sorts)
    [!!] Joins performed without indexes: 95
    [OK] Temporary tables created on disk: 15% (87 on disk / 567 total)
    [OK] Thread cache hit rate: 99% (6 created / 644 connections)
    [OK] Table cache hit rate: 51% (252 open / 494 opened)
    [OK] Open file limit used: 6% (64/1K)
    [OK] Table locks acquired immediately: 100% (8K immediate / 8K locks)
    [OK] InnoDB buffer pool / data size: 300.0M/60.3M
    [OK] InnoDB log waits: 0
    -------- Recommendations -----------------------------------------------------
    General recommendations:
        Run OPTIMIZE TABLE to defragment tables for better performance
        MySQL started within last 24 hours - recommendations may be inaccurate
        Reduce your overall MySQL memory footprint for system stability
        Adjust your join queries to always utilize indexes
    Variables to adjust:
        join_buffer_size (> 128.0K, or always use indexes with joins)
    Here i have 1 question - Should i run OPTIMIZE TABLE every time i got this message? Does this command increase the size of my DB?

    httpd.conf settings:
    # prefork MPM
    <IfModule prefork.c>
    StartServers 4
    MinSpareServers 5
    MaxSpareServers 10
    ServerLimit 50
    MaxClients 25
    MaxRequestsPerChild 1000
    # worker MPM
    <IfModule worker.c>
    StartServers 4
    MaxClients 25
    MinSpareThreads 25
    MaxSpareThreads 75
    ThreadsPerChild 25
    MaxRequestsPerChild 1000
    apc.ini settings:
    apc.optimization = 2

    I would really appreciate if you can help me with my problem :)
  2. MattW

    MattW Well-Known Member

    Are you using nginx as a reverse proxy to apache?
  3. Skyuser

    Skyuser Member

  4. Skyuser

    Skyuser Member

    Additional info:
    First Byte Time (back-end processing): 30/100

    952 ms First Byte Time
    261 ms Target First Byte Time

    Nothing good :(
  5. craigiri

    craigiri Well-Known Member

    If you have goog analytics, check your total page speed in there and what the various parts of the delays are.

    I don't think optimize works on innodb, but others know more than I do. I think they self-repair to some degree.
  6. craigiri

    craigiri Well-Known Member

    One quick and dirty check is to put a plain html page on your server and try loading it with pagespeed and other tools - and from other sites that check (goog pagespeed).....to determine if the problem is somewhere on the network or general server as opposed to in the php and mysql.

    I think sometimes you have to figure out what it is NOT before you figure what it is....
  7. Skyuser

    Skyuser Member

    Well other threads which contains few posts 5-6 ( not 20) are loading superfast.
    Look at this:
    Sometimes Pagetime goes up to 1.5 sec.
  8. craigiri

    craigiri Well-Known Member

    Google pagespeed gives that page an 86 out of 100 (that's good) for desktop.

    Based on my stats, if the average page load time (over many pages and over time) is about 6 seconds total, you are doing well IMHO. On my servers, the average in google is about 6.8 seconds in busy times - even with a dedicated server that rarely hits a load of 1.

    Unless you have google-type of replicated content (everywhere), I don't think you are going to get total page serve/render times under 4 seconds (when averaged out). On my very slow months I may average as low as that.
  9. Skyuser

    Skyuser Member

    Do you think switching to MariaDB can burst some speed? Tryed Percona - much more ram usage + even slower work.
  10. Adrian Schneider

    Adrian Schneider Active Member

    If MySQL is only taking one hundredth of a second for the entire page, your problems are likely elsewhere. Any reason for proxying nginx to Apache? Doing it directly seems like it would be much faster with php-fpm.

    Have you tried disabling all of your add-ons to see if the problem goes away?

    FYI I'm also on the cheapest DigitalOcean droplet and it runs quite respectably. No Apache though.
  11. Skyuser

    Skyuser Member

    I think ill try to disable all addons first. Never worked with php-fpm. I heard you need smth to do with .htaccess files with fpm onboard :>
    If disabling addons won't help, ill try to use fpm thing.
    Any suggestions about switching MySQL to Maria for speed?
  12. Adrian Schneider

    Adrian Schneider Active Member

    Even if Maria is 5x faster, you would only be reducing 0.01s to 0.002s. That won't solve your problem here, though speed is always nice. I was under the impression the speed differences between the two are mostly negligible anyway. There should be much lower hanging fruit.
  13. craigiri

    craigiri Well-Known Member

    The real question is whether it IS slow or just whether it seems slow to you at some times.

    If you don't already have it installed, open a google analytics accounts and see what the averages are over time. I can't imagine a page with 20 posts taking much longer than one with 5 - especially looking at the nature of your posts.

    Don't create problems where there may be none. Your site seemed fine to me (from the east coast of the USA).
    Skyuser likes this.
  14. Skyuser

    Skyuser Member

    @Adrian Schneider @craigiri
    Thanks for help!

Share This Page