Big optimization problems!

[Skyuser]

Recently i started to monitor my server activity because pages with 20 posts ( large threads) opens very slowly.
Page Time: 1.3822s
Memory: 4.1461 MB (Peak: 5.3929 MB)
Queries (10, time: 0.0136s, 1.0%)

I have DigitalOcean VPS with 1 Gb Ram, nginx 1.4.7 with ngx_pagespeed + Apache, MySQL 5.5.6, PHP 5.4 and APC.
My.cnf settings:

character_set_server=utf8
collation_server=utf8_unicode_ci
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
query_cache_size = 128M
query_cache_limit = 2M
thread_cache_size = 4
innodb_buffer_pool_size = 300M
innodb_additional_mem_pool_size = 15M
innodb_flush_log_at_trx_commit = 2
log-slow-queries = /var/log/mysqlslowlog/slow.log

tuning-primer output:

SLOW QUERIES
The slow query log is enabled.
Current long_query_time = 10.000000 sec.
You have 0 out of 4925 that take longer than 10.000000 sec. to complete
Your long_query_time seems to be fine

BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See http://dev.mysql.com/doc/refman/5.5/en/point-in-time-recovery.html

WORKER THREADS
Current thread_cache_size = 4
Current threads_cached = 3
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine

MAX CONNECTIONS
Current max_connections = 151
Current threads_connected = 1
Historic max_used_connections = 5
The number of used connections is 3% of the configured maximum.
You are using less than 10% of your configured max_connections.
Lowering max_connections could help to avoid an over-allocation of memory
See "MEMORY USAGE" section to make sure you are not over-allocating

INNODB STATUS
Current InnoDB index space = 19 M
Current InnoDB data space = 60 M
Current InnoDB buffer pool free = 89 %
Current innodb_buffer_pool_size = 300 M
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory

MEMORY USAGE
Max Memory Ever Allocated : 472 M
Configured Max Per-thread Buffers : 405 M
Configured Max Global Buffers : 459 M
Configured Max Memory Limit : 864 M
Physical Memory : 1006 M
Max memory limit seem to be within acceptable norms

KEY BUFFER
Current MyISAM index space = 4 M
Current key_buffer_size = 8 M
Key cache miss rate is 1 : 55
Key buffer free ratio = 87 %
Your key_buffer_size seems to be fine

QUERY CACHE
Query cache is enabled
Current query_cache_size = 128 M
Current query_cache_used = 1 M
Current query_cache_limit = 2 M
Current Query cache Memory fill ratio = 1.47 %
Current query_cache_min_res_unit = 4 K
Query Cache is 23 % fragmented
Run "FLUSH QUERY CACHE" periodically to defragment the query cache memory
If you have many small queries lower 'query_cache_min_res_unit' to reduce fragmentation.
Your query_cache_size seems to be too high.
Perhaps you can use these resources elsewhere
MySQL won't cache query results that are larger than query_cache_limit in size

SORT OPERATIONS
Current sort_buffer_size = 2 M
Current read_rnd_buffer_size = 256 K
Sort buffer seems to be fine

JOINS
Current join_buffer_size = 132.00 K
You have had 79 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.

Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.

OPEN FILES LIMIT
Current open_files_limit = 1024 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine

TABLE CACHE
Current table_open_cache = 400 tables
Current table_definition_cache = 400 tables
You have a total of 229 tables
You have 250 open tables.
The table_cache value seems to be fine

TEMP TABLES
Current max_heap_table_size = 16 M
Current tmp_table_size = 16 M
Of 428 temp tables, 15% were created on disk
Created disk tmp tables ratio seems fine

TABLE SCANS
Current read_buffer_size = 128 K
Current table scan ratio = 26 : 1
read_buffer_size seems to be fine

TABLE LOCKING
Current Lock Wait ratio = 0 : 5220
Your table locking seems to be fine

mysqltuner.pl output

-------- Storage Engine Statistics -------------------------------------------
[--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MRG_MYISAM
[--] Data in MyISAM tables: 5M (Tables: 8)
[--] Data in InnoDB tables: 60M (Tables: 176)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
[--] Data in MEMORY tables: 504K (Tables: 4)
[!!] Total fragmented tables: 178

-------- Security Recommendations  -------------------------------------------
[OK] All database users have passwords assigned

-------- Performance Metrics -------------------------------------------------
[--] Up for: 25m 3s (5K q [3.918 qps], 644 conn, TX: 33M, RX: 2M)
[--] Reads / Writes: 58% / 42%
[--] Total buffers: 475.0M global + 2.7M per thread (151 max threads)
[!!] Maximum possible memory usage: 880.8M (87% of installed RAM)
[OK] Slow queries: 0% (0/5K)
[OK] Highest usage of available connections: 3% (5/151)
[OK] Key buffer size / total MyISAM indexes: 8.0M/5.0M
[OK] Key buffer hit rate: 98.3% (6K cached / 104 reads)
[OK] Query cache efficiency: 40.3% (1K cached / 2K selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 187 sorts)
[!!] Joins performed without indexes: 95
[OK] Temporary tables created on disk: 15% (87 on disk / 567 total)
[OK] Thread cache hit rate: 99% (6 created / 644 connections)
[OK] Table cache hit rate: 51% (252 open / 494 opened)
[OK] Open file limit used: 6% (64/1K)
[OK] Table locks acquired immediately: 100% (8K immediate / 8K locks)
[OK] InnoDB buffer pool / data size: 300.0M/60.3M
[OK] InnoDB log waits: 0
-------- Recommendations -----------------------------------------------------
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours - recommendations may be inaccurate
    Reduce your overall MySQL memory footprint for system stability
    Adjust your join queries to always utilize indexes
Variables to adjust:
    join_buffer_size (> 128.0K, or always use indexes with joins)

Here i have 1 question - Should i run OPTIMIZE TABLE every time i got this message? Does this command increase the size of my DB?

httpd.conf settings:

# prefork MPM
<IfModule prefork.c>
StartServers 4
MinSpareServers 5
MaxSpareServers 10
ServerLimit 50
MaxClients 25
MaxRequestsPerChild 1000
</IfModule>

# worker MPM
<IfModule worker.c>
StartServers 4
MaxClients 25
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 1000
</IfModule>

apc.ini settings:

extension=apc.so
apc.enabled=1
apc.shm_segments=1
apc.shm_size=128M
apc.num_files_hint=7000
apc.user_entries_hint=4096
;apc.ttl=7200
;apc.user_ttl=7200
apc.gc_ttl=3600
apc.optimization = 2
apc.slam_defense=0
apc.stat=1
apc.include_once_override=0
apc.enable_cli=0
apc.file_update_protection=2
apc.max_file_size=2M

​

I would really appreciate if you can help me with my problem

 

[MattW]

Are you using nginx as a reverse proxy to apache?

 

[Skyuser]

Are you using nginx as a reverse proxy to apache?

Yes

 

[Skyuser]

Additional info:
First Byte Time (back-end processing): 30/100
952 ms First Byte Time
261 ms Target First Byte Time

Nothing good

 

[craigiri]

If you have goog analytics, check your total page speed in there and what the various parts of the delays are.

I don't think optimize works on innodb, but others know more than I do. I think they self-repair to some degree.

 

[craigiri]

One quick and dirty check is to put a plain html page on your server and try loading it with pagespeed and other tools - and from other sites that check (goog pagespeed).....to determine if the problem is somewhere on the network or general server as opposed to in the php and mysql.

I think sometimes you have to figure out what it is NOT before you figure what it is....

 

[Skyuser]

One quick and dirty check is to put a plain html page on your server and try loading it with pagespeed and other tools - and from other sites that check (goog pagespeed).....to determine if the problem is somewhere on the network or general server as opposed to in the php and mysql.

I think sometimes you have to figure out what it is NOT before you figure what it is....

Well other threads which contains few posts 5-6 ( not 20) are loading superfast.
Look at this:
http://tools.pingdom.com/fpt/#!/dm4....php?threads/voprosy-pervokursnikov.41/page-2
Sometimes Pagetime goes up to 1.5 sec.

 

[craigiri]

Google pagespeed gives that page an 86 out of 100 (that's good) for desktop.

Based on my stats, if the average page load time (over many pages and over time) is about 6 seconds total, you are doing well IMHO. On my servers, the average in google is about 6.8 seconds in busy times - even with a dedicated server that rarely hits a load of 1.

Unless you have google-type of replicated content (everywhere), I don't think you are going to get total page serve/render times under 4 seconds (when averaged out). On my very slow months I may average as low as that.

 

[Skyuser]

Google pagespeed gives that page an 86 out of 100 (that's good) for desktop.

Based on my stats, if the average page load time (over many pages and over time) is about 6 seconds total, you are doing well IMHO. On my servers, the average in google is about 6.8 seconds in busy times - even with a dedicated server that rarely hits a load of 1.

Unless you have google-type of replicated content (everywhere), I don't think you are going to get total page serve/render times under 4 seconds (when averaged out). On my very slow months I may average as low as that.

Do you think switching to MariaDB can burst some speed? Tryed Percona - much more ram usage + even slower work.

 

[Adrian Schneider]

If MySQL is only taking one hundredth of a second for the entire page, your problems are likely elsewhere. Any reason for proxying nginx to Apache? Doing it directly seems like it would be much faster with php-fpm.

Have you tried disabling all of your add-ons to see if the problem goes away?

FYI I'm also on the cheapest DigitalOcean droplet and it runs quite respectably. No Apache though.

 

[Skyuser]

If MySQL is only taking one hundredth of a second for the entire page, your problems are likely elsewhere. Any reason for proxying nginx to Apache? Doing it directly seems like it would be much faster with php-fpm.

Have you tried disabling all of your add-ons to see if the problem goes away?

FYI I'm also on the cheapest DigitalOcean droplet and it runs quite respectably. No Apache though.

I think ill try to disable all addons first. Never worked with php-fpm. I heard you need smth to do with .htaccess files with fpm onboard :>
If disabling addons won't help, ill try to use fpm thing.
Any suggestions about switching MySQL to Maria for speed?

 

[Adrian Schneider]

Even if Maria is 5x faster, you would only be reducing 0.01s to 0.002s. That won't solve your problem here, though speed is always nice. I was under the impression the speed differences between the two are mostly negligible anyway. There should be much lower hanging fruit.

 

[craigiri]

The real question is whether it IS slow or just whether it seems slow to you at some times.

If you don't already have it installed, open a google analytics accounts and see what the averages are over time. I can't imagine a page with 20 posts taking much longer than one with 5 - especially looking at the nature of your posts.

Don't create problems where there may be none. Your site seemed fine to me (from the east coast of the USA).

 

[Skyuser]

The real question is whether it IS slow or just whether it seems slow to you at some times.

If you don't already have it installed, open a google analytics accounts and see what the averages are over time. I can't imagine a page with 20 posts taking much longer than one with 5 - especially looking at the nature of your posts.

Don't create problems where there may be none. Your site seemed fine to me (from the east coast of the USA).

@Adrian Schneider @craigiri
Thanks for help!