xf_session' is marked as crashed and should be repaired keeps happening

[XxUnkn0wnxX]

ok i believe this all started after the 2.2.27 Apache upgrade with php 5.4.29 + MySQL upgrade from 5.5-5.6

i started to have these issues b4 the mysql upgrade.

now i usually use fcgi but it kept giving me server errors internal errors and it could not be solved

so i went back to suphp which worked fine

but now i keep getting this xf_seasons corruption with 24 hours and i am not sure what is causing it

i have to run mysqlcheck hidden_database -r
mysqlcheck hidden_database -o just to get it running again

i have moved the mysql temp dir to tmpdir=/mysqltmp

i have set perms to mysql and properly permissions and i do have tmpwatch running on that directory

i believe every 12-16 hours i get this corruption mysql shts its self i get emails from server like this:

now i could try deleting contents of the temp directory and dumping the xf_seasons table within xenforo db but i am unsure if that is safe to do that with xenforo db..

my next guess would be one of these add ones causing the issues as memebrs said they got time issues when leaving there pages open on any thread:
http://xenforo.com/community/resources/*******-thread-live-update.3053/
http://xenforo.com/community/resources/*******-conversation-live-update.2978/
http://xenforo.com/community/resources/live-update.274/

now i could try to disable one of these for a a day or so and see if site crashes but i doubt these would couase such a thing to happen

i have hidden the sensitive marital and names in this log

140628 10:20:37 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
140628 10:20:41 [Note] Found 535 of 532 rows when repairing './hidden_XenForo/xf_session'
140628 10:23:41 [Note] /usr/sbin/mysqld: Normal shutdown
140628 10:23:41 [Note] Event Scheduler: Purging the queue. 0 events
140628 10:23:43 [Warning] /usr/sbin/mysqld: Forcing close of thread 1  user: 'eximstats'
140628 10:23:44  InnoDB: Starting shutdown...
140628 10:23:45  InnoDB: Shutdown completed; log sequence number 110852851784
140628 10:23:45 [Note] /usr/sbin/mysqld: Shutdown complete
140628 10:23:45 mysqld_safe mysqld from pid file /var/lib/mysql/hidden.hidden.pid ended
140628 10:24:28 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
140628 10:24:28 [Note] Plugin 'FEDERATED' is disabled.
140628 10:24:28 InnoDB: The InnoDB memory heap is disabled
140628 10:24:28 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140628 10:24:28 InnoDB: Compressed tables use zlib 1.2.3
140628 10:24:28 InnoDB: Using Linux native AIO
140628 10:24:28 InnoDB: Initializing buffer pool, size = 1.0G
140628 10:24:28 InnoDB: Completed initialization of buffer pool
140628 10:24:28 InnoDB: highest supported file format is Barracuda.
140628 10:24:30  InnoDB: Waiting for the background threads to start
140628 10:24:31 InnoDB: 5.5.37 started; log sequence number 110852851784
140628 10:24:31 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
140628 10:24:31 [Note]  - '0.0.0.0' resolves to '0.0.0.0';
140628 10:24:31 [Note] Server socket created on IP: '0.0.0.0'.
140628 10:24:33 [Note] Event Scheduler: Loaded 0 events
140628 10:24:33 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.37-cll'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
140628 10:54:05  InnoDB: ERROR: the age of the last checkpoint is 9449311,
InnoDB: which exceeds the log group capacity 9433498.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.
140628 11:28:13 [Note] /usr/sbin/mysqld: Normal shutdown
140628 11:28:13 [Note] Event Scheduler: Purging the queue. 0 events
140628 11:28:15  InnoDB: Starting shutdown...
140628 11:28:16  InnoDB: Shutdown completed; log sequence number 111622679818
140628 11:28:16 [Note] /usr/sbin/mysqld: Shutdown complete
140628 11:28:16 mysqld_safe mysqld from pid file /var/lib/mysql/hidden.hidden.pid ended
140628 11:28:17 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
140628 11:28:17 [Note] Plugin 'FEDERATED' is disabled.
140628 11:28:17 InnoDB: The InnoDB memory heap is disabled
140628 11:28:17 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140628 11:28:17 InnoDB: Compressed tables use zlib 1.2.3
140628 11:28:17 InnoDB: Using Linux native AIO
140628 11:28:17 InnoDB: Initializing buffer pool, size = 1.0G
140628 11:28:17 InnoDB: Completed initialization of buffer pool
140628 11:28:17 InnoDB: highest supported file format is Barracuda.
140628 11:28:18  InnoDB: Waiting for the background threads to start
140628 11:28:19 InnoDB: 5.5.37 started; log sequence number 111622679818
140628 11:28:19 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
140628 11:28:19 [Note]  - '0.0.0.0' resolves to '0.0.0.0';
140628 11:28:19 [Note] Server socket created on IP: '0.0.0.0'.
140628 11:28:19 [Note] Event Scheduler: Loaded 0 events
140628 11:28:19 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.37-cll'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
140701  2:40:27 [Note] /usr/sbin/mysqld: Normal shutdown
140701  2:40:32 [Note] Event Scheduler: Purging the queue. 0 events
140701  2:40:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 582957  user: 'hidden_hidden'
140701  2:40:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 582956  user: 'hidden_hidden'
140701  2:40:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 582955  user: 'hidden_hidden'
140701  2:40:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 582954  user: 'hidden_hidden'

140701 15:39:06 mysqld_safe Number of processes running now: 0
140701 15:39:12 mysqld_safe mysqld restarted
2014-07-01 15:40:24 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2014-07-01 15:40:38 1657 [Note] Plugin 'FEDERATED' is disabled.
2014-07-01 15:40:46 7f0687e1e720 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
2014-07-01 15:40:46 1657 [Note] InnoDB: Using atomics to ref count buffer pool pages
2014-07-01 15:40:46 1657 [Note] InnoDB: The InnoDB memory heap is disabled
2014-07-01 15:40:46 1657 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2014-07-01 15:40:46 1657 [Note] InnoDB: Compressed tables use zlib 1.2.3
2014-07-01 15:40:46 1657 [Note] InnoDB: Using Linux native AIO
2014-07-01 15:40:46 1657 [Note] InnoDB: Using CPU crc32 instructions
2014-07-01 15:40:50 1657 [Note] InnoDB: Initializing buffer pool, size = 1.0G
2014-07-01 15:40:51 1657 [Note] InnoDB: Completed initialization of buffer pool
2014-07-01 15:40:51 1657 [Note] InnoDB: Highest supported file format is Barracuda.
2014-07-01 15:40:54 1657 [Note] InnoDB: The log sequence numbers 114320554920 and 114320554920 in ibdata files do not match the log sequence number 114769974298 in the ib_logfiles!
2014-07-01 15:40:54 1657 [Note] InnoDB: Database was not shutdown normally!
2014-07-01 15:40:54 1657 [Note] InnoDB: Starting crash recovery.
2014-07-01 15:40:54 1657 [Note] InnoDB: Reading tablespace information from the .ibd files...
2014-07-01 15:43:29 1657 [Note] InnoDB: Restoring possible half-written data pages
2014-07-01 15:43:31 1657 [Note] InnoDB: from the doublewrite buffer...
2014-07-01 15:46:29 1657 [Note] InnoDB: 128 rollback segment(s) are active.
2014-07-01 15:46:39 1657 [Note] InnoDB: Waiting for purge to start
2014-07-01 15:46:40 1657 [Note] InnoDB: 5.6.17 started; log sequence number 114769974298
2014-07-01 15:47:12 1657 [Note] Server hostname (bind-address): '*'; port: 3306
2014-07-01 15:47:12 1657 [Note] IPv6 is available.
2014-07-01 15:47:13 1657 [Note]  - '::' resolves to '::';
2014-07-01 15:47:14 1657 [Note] Server socket created on IP: '::'.
2014-07-01 15:48:55 1657 [Note] Event Scheduler: Loaded 0 events
2014-07-01 15:48:55 1657 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.17-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
2014-07-01 15:54:45 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
2014-07-01 15:54:47 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
2014-07-01 15:54:47 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
2014-07-01 15:54:47 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
2014-07-01 15:54:47 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
2014-07-01 15:54:47 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
2014-07-01 15:54:48 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
2014-07-01 15:54:48 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired
2014-07-01 15:54:48 1657 [ERROR] /usr/sbin/mysqld: Table './hidden_XenForo/xf_session' is marked as crashed and should be repaired

 

[XxUnkn0wnxX]

and this what i have in my.cnf configuration:

open_files_limit=20000
innodb_file_per_table=1
character_set_server=utf8
innodb_additional_mem_pool_size=30M
wait_timeout=60
log-output=TABLE
innodb_buffer_pool_size=1024M
key_buffer_size=32M
table_open_cache=8K
table_definition_cache=3072
max_user_connections=600
innodb_log_buffer_size=29M
socket="/var/lib/mysql/mysql.sock"
max_heap_table_size=119M
myisam_sort_buffer_size=16M
innodb_thread_concurrency=4
query_cache_size=200M
table_open_cache=8192
connect_timeout=30
collation_server=utf8_unicode_ci
long_query_time=5
max_connections=8000
max_allowed_packet=268435456
tmp_table_size=120M
default-storage-engine=MyISAM
innodb_open_files=1000
tmpdir=/mysqltmp

i need some support i have no idea why this is happening and i analyzed the logs best i could

 

[XxUnkn0wnxX]

strange thing this is only happening to my main forums my testing boar d which under the same domain/account just in another folder does not get this issues but hey no one else eve uses it

same with my word press front page no issues only my main forums

 

[Dillon Lawrence]

There is no reason why the xf_session table cannot be cleared/truncated in phpMyAdmin. The worst that could happen is that users are logged out.

Are you sure you have enough storage space?

 

[XxUnkn0wnxX]

well my hosting had this to say about the matter and i have also been viewing my access logs seems like a another Dos but it should not cause such corruption:

+ i have plenty of disk space + CPU usage is Fine but only 4gb of ram installed.

I have reviewed the server and the load is causing mysql to crash. Since the server is trying to handle all requests and mysql is reading and writing to the tables when the servers crashed it caused the open tables to become corrupt. That is why there is an option and command to check and repair databases in mysql. I have repaired all tables and reviewed the mysql settings. Everything is properly configured.

If we increase the memory limit or connection limit of mysql it will just allow for more traffice from the attacks to consume server resources quicker. It wouldn't be a wise solution to this issue. You will want to block the bad user agents and secure any posting or email scripts with a captcha to prevent bots from spamming the form and crashing your server.

 

[Dillon Lawrence]

There is no reason why the xf_session table cannot be cleared/truncated in phpMyAdmin. The worst that could happen is that users are logged out.

Are you sure you have enough storage space?

Have you tried this?

 

[XxUnkn0wnxX]

Have you tried this?

trying to but my server is under heavy dos attack i cannot even ssh...

i have blocked out all the bad ip's with CSF but still its being attacked

 

[TPerry]

trying to but my server is under heavy dos attack i cannot even ssh...

i have blocked out all the bad ip's with CSF but still its being attacked

WP Pingback? If so, that's a layer 7 attack and if on OVH I don't think they protect against it. If you have CSF installed you can try to limit the number of concurrent connections.

 

[XxUnkn0wnxX]

WP Pingback? If so, that's a layer 7 attack and if on OVH I don't think they protect against it. If you have CSF installed you can try to limit the number of concurrent connections.

well if they cannot protect against it i have other services that may be able to protect against it as a reverse proxy.

they said they can i have been chatting with them for a while but i will wait and see

if there protection doest work out i might go to one of these services:
https://ddosdeflect.com
http://www.hyperfilter.com/web-protection/

yes initially i was moving to OVH because i have had enough of hostgator i have been attacked many times and they haven't done anything to protect the server

but the second reason is the better hardware for a cheaper price

i when with hostgator in the first place because i wasn't very good at managing a server but i am fairly confident now, and linux isn't that hard as i thought to be.

but i am absolutely disappointed with hostgators dedos protection.

and there is some much the CSF fire wall can do.

+ my site is hitting the point where in general it is overwhelming the current servers resources in such a way an upgrade to OVH is well worth it.

 

[Vladislav Rastrusny]

default-storage-engine=MyISAM

This should be changed to:

default-storage-engine=InnoDB

You can also do the following

REPAIR TABLE xf_session EXTENDED
ALTER TABLE xf_session ENGINE = InnoDB

to permanently change table format. But xf_session is MySQL intentionally for speed. So this may slow down your forum a little.