1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Bit of tech help? - Database tables keep crashing

Discussion in 'Off Topic' started by AdamD, Mar 18, 2013.

  1. AdamD

    AdamD Well-Known Member

    No idea why this is suddenly an issue, but earlier today a few database tables came up as marked as crashed

    So I shut down Mysql, ran a repair/optimise on all and everything seemed ok

    An hour later, some more tables came up with marked as crashed

    I checked the logs and saw lots of errors like this:

    130318 16:33:56 [ERROR] /usr/sbin/mysqld: Incorrect key file for table './x/jos_session.MYI'; try to repair it
    and
    130318 16:34:06 [ERROR] /usr/sbin/mysqld: Table './x/jos_session' is marked as crashed and should be repaired

    I checked the tmp folder, it had some 899mb free, so it wasn't that, but repairing, the tables just didn't seem to "hold"

    I've now changed the mysql tmp folder to another location, to see if it helps, but any tips?
     
  2. Slavik

    Slavik XenForo Moderator Staff Member

    As youve already worked out, the usual cause is /tmp/ filling up..

    maybe install tmpwatch?
     
  3. AdamD

    AdamD Well-Known Member

    Yea it's odd, the tmp directory wasn't full, not anywhere near.

    It has the correct permissions and there are tmp files in there, so I'm a bit at a loss

    I see mysql writing to the new tmp folder ok, at least.
     
  4. Biker

    Biker Well-Known Member

    Had a similar issue a couple of years ago where tables were showing up as corrupted when everything had been running just fine for over 2 years. I'd repair the tables only to have more show up as corrupted shortly afterwards. After a couple of hours of fighting the issue, the entire server took a dump due to the motherboard going Tango Uniform.
     
  5. Slavik

    Slavik XenForo Moderator Staff Member

    Maybe time for Adam to make a backup... just in case :)
     
  6. Biker

    Biker Well-Known Member

    Might be wise. Fortunately, the server does a full backup of all databases and user accounts every night and then shoots them over to the FTP server. Lost about 12 hours worth of data and posts, but I can live with that.
     
  7. AdamD

    AdamD Well-Known Member

    Hmm well this is a VPS with Raid 10, but, I suppose that wouldn't matter much, if the motherboard was dying lol
     
  8. The Forum Heroes

    The Forum Heroes Well-Known Member

    Has been a lot of issues with filling /tmp lately For us to the point of moving our sql temp directory to a folder in /home to have the space. Odd thing though for everyone with this issue is, there hasn't been a xF update in a long time to be a reflection on a recent change.
     
  9. AdamD

    AdamD Well-Known Member

    Hmm, still getting problems with /tmp

    Noticed another site's admin panel kept giving me the nginx 500 internal server error, but other pages were working ok, so checking the log, I see:

    Code:
    [Wed Mar 20 13:06:10 2013] [warn] [client 66.249.73.91] mod_fcgid: stderr: PHP Warning:  Unknown: open(/tmp/sess_aef044dfe3defd98e6931b4df9c19e8d, O_RDWR) failed: No space left on device (28) in Unknown on line 0
    [Wed Mar 20 13:06:10 2013] [warn] [client 66.249.73.91] mod_fcgid: stderr: PHP Warning:  Unknown: Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/tmp) in Unknown on line 0
    So again, I check the tmp folder, it's practically empty
    I really can't understand this heh, why is the server having issues with /tmp?
    Tmp files ARE being written to it, but it seems to report that the folder is full, when it's not.
    The session.save_path is set correctly, too.
     
  10. SilverCircle

    SilverCircle Well-Known Member

    I would first do a file system check and see what it says. This could very well be a corrupted file system - rare thing, but it still can happen due to software or even hardware issues. No RAID or other hardware can protect against software issues or other problems causing corrupted data to be physically written to the file system.

    Where is you /tmp? Maybe the partition is really full? Remember that most Unix systems reserve a certain amount of disk space to the root user, so any application running as non-root can report a full file system when there is still some space left on the device (typically 5-10% are reserved).

    Or maybe you've run out of inodes (df -i will show you inode usage on the filesystem), but that's unlikely unless you have some stuff that needs an insanely high number of files or directories.
     
  11. AdamD

    AdamD Well-Known Member

    Yea I found out what's causing it

    I completely forgot about the Inode limit that VPS' have, seems nginx wasn't clearing out it's temporary files in the tmp folder, so even though it was empty, it had reached it's inode limit.

    At least, I hope that's it! :p
     

Share This Page