[SolidMean] ForumBackup 1.3.01

[SneakyDave]

@theraptor, I can't seem to reproduce the problem you're experiencing, that the board doesn't turn back on after a backup, especially after version 1.0.1.

Can you leave your add-on debug mode enabled, and then copy me the last 50 lines of it when it happens again?

Also, the next version (1.1.0) will have the following new options:

• Forum backup and code backup are separate cron entries.
• Option to email the administrator when backups are complete
• Admins can separately disable the board for database backups, or code backups.

 

[theraptor]

Im gonna more tests and if that happens again i will report back.

 

[Robru]

Second try with automatic backup:

2014-09-18 01:21:31 | INFO | Backing up code with: tar czvf /media/usb0/forumbackup_18-09-2014-011723.code.tar.gz /var/www/html 2>> /media/usb0//forumbackup.log
tar: Removing leading `/' from member names
2014-09-18 04:08:20 | INFO | Deleting old code backups...
2014-09-18 04:08:20 | INFO | No code backups to delete...
2014-09-18 04:08:20 | INFO | Verified that database backup exists...
2014-09-18 04:08:20 | INFO | Verified that database backup has data...
2014-09-18 04:08:20 | INFO | Verified that code backup exists...
2014-09-18 04:08:20 | ERROR | Code backup exists but has 0 length.

 

[Mr. Goodie2Shoes]

@SneakyDave I was looking at the code and found this in the line 185:

$backup['filename'] = str_replace('.str','',$backup['filename']);

in the file /library/SolidMean/ForumBackup/CronEntry/Backup.php
shouldn't it be .sql instead of .str?

 

[Robru]

does /media/usb0 has global write permission?

Yes
If I manually backup it's going to be fine.

 

[Robru]

@SneakyDave I was looking at the code and found this in the line 185:

$backup['filename'] = str_replace('.str','',$backup['filename']);

in the file /library/SolidMean/ForumBackup/CronEntry/Backup.php
shouldn't it be .sql instead of .str?

We wait for the answer from @SneakyDave

 

[Mr. Goodie2Shoes]

okay, I think I got why this is happening:

tar: Removing leading `/' from member names

Adding the option

--absolute-names

to the tar command might fix it...

 

[SneakyDave]

We wait for the answer from @SneakyDave

Yes, that should be .sql, not .str, not sure how that got in there like that, thanks! It would only make a difference if your backup file name included a ".sql" extention, but because that filename now only accepts alphanumeric characters ('.' not allowed), it should never have a .sql extension when it gets to that point. That code has been removed in the soon to be released 1.1.0. Good catch.

okay, I think I got why this is happening:

tar: Removing leading `/' from member names

Adding the option

--absolute-names

to the tar command might fix it...

Yes, that is just notice output to STDERR that shows up while running the tar, but I've been meaning to suppress or correct it. Reading about that option now: http://www.gnu.org/software/tar/manual/html_node/absolute.html

Thanks again.

 

[Robru]

I hope this is the solution

 

[SneakyDave]

Unfortunly, I think everybody gets that warning message on their code backup, but the next version will have that error message resolved.

Second try with automatic backup:

Were you able to run that tar command manually, that I gave you earlier, and did you receive any errors? Unfortunately, without access to your system, I'm not much help .

Can you try this set of commands, creating an empty file in a directory and seeing if you can tar it up, just to see if your tar command is working?

mkdir /var/www/backuptest
touch /var/www/backuptest/junk.txt
tar czvf /media/usb0/testbackup.tar.gz /var/www/backuptest

 

[Robru]

Were you able to run that tar command manually, that I gave you earlier, and did you receive any errors?

Yes, with your manually command it works perfect without errors!
I wait for v 1.1.0 en try it again

Can you try this set of commands

Works good!

 

[SneakyDave]

I don't suspect that version 1.1.0 will help resolve your issue, but you're welcome to try.

I don't suspect that it's a permission issue, because your database backup works fine.
I don't suspect that it's a problem with gzip because your database backup works fine.
I don't suspect that its a problem with the web server user not knowing where tar is, because it runs for 3 hours.
I wouldn't think it would be a file system problem because your database backup works on that file system.

Backing up a bit....
The code backup that was created by the add-on, that took 3 hours, you received an error that it was 0 length, but you mentioned that it was actually about 36 gig? Is that how big the backup was, or how big your /var/www/html/ forum is?

If the code backup is indeed 36 gig, then there is a problem with the filesize() function in php. I'll look into that too see if using that function is problematic. Maybe your backups are fine, and the filesize function is borked.

Maybe related to this limitation:

Note: Because PHP's integer type is signed and many platforms use 32bit integers, some filesystem functions may return unexpected results for files which are larger than 2GB.

 

[SneakyDave]

@Robru, I suspect you may be a victim of PHP's filesize() limit on 32bit or 64bit PHP versions.

To confirm that, can you upload this as a .php file (such as file.php) and then browse to it in a browser, and see what it says? It should echo out the size of the code backup you created:

<?php 
echo "Filesize of the code backupk is: " . filesize('/media/usb0/forumbackup_18-09-2014-011723.code.tar.gz');
?>

If it says the filesize is 0 or null, then that's the problem. I'll have to come up with a better way to determine large file sizes.

 

[Robru]

@Robru, I suspect you may be a victim of PHP's filesize() limit on 32bit or 64bit PHP versions.

To confirm that, can you upload this as a .php file (such as file.php) and then browse to it in a browser, and see what it says? It should echo out the size of the code backup you created:

<?php
echo "Filesize of the code backupk is: " . filesize('/media/usb0/forumbackup_18-09-2014-011723.code.tar.gz');
?>

If it says the filesize is 0 or null, then that's the problem. I'll have to come up with a better way to determine large file sizes.

Filesize of the code backupk is: -617112293

 

[SneakyDave]

lol, ok, so that's the problem, I'm checking to see if the file size is greater than 0 to determine if it has data, which yours isn't. I think as a quick solution, I'll just check to see if the file size IS '0' to tell me if it's empty.

Thanks for your help. New 1.1.0 version should be available with this fix, and other stuff, in the next couple of hours.

 

[Robru]

Many thanks for your help!

 

[SneakyDave]

New email option, separate cron entries for database and code, separate checkbox to turn off board for database and code backup, and bug fixes. Visit your admin options and make any corrections you need. Then visit the 2 cron entries ("ForumBackup / Database" and "ForumBackup / Code") to adjust when they should run. The default schedule for the database is everyday at midnight, the default schedule for backing up the code is every day at 00:30.

SneakyDave updated [SolidMean] ForumBackup with a new update entry:

Update 1.1.0 New Features and Bug Fixes

This version splits out the cron entry into 2 separate ones. Make sure to visit your options and update accordingly!

09/15/2014: Version 1.1.0
- Separated the cron processes for database and code backups. There are now 2 cron entires (database and code) that can be scheduled at different times.
- Added email option
- Improved install routine.
- Added separate options to turn off forum for database and code.
- Made sure everything is fully phrased.
- Added a P option to the tar archive of the...

Read the rest of this update entry...

 

[rainmotorsports]

Awesome update. I was half tempted to request separate crons as well as being able to only turn the board off for SQL. Very useful additions!

 

[rainmotorsports]

@SneakyDave What about passing --single-transaction to mysqldump? I believe its commonly used when people are trying to do clean backups that have innodb tables.

This may solve the "lock up" people are having. I tested my usual dump parameters and the board was usable during the dump. I took the single-transaction switch out and the site became unresponsive during the sql backup, same as it does when using the plugin. The board being closed doesn't even show due to the severity of that, though after the dump it does show as closed during the file backup since the server is rather responsive at that point.

My backups only take so many seconds, especially if done server side. But lets just say we wanted to try hourly backups (useful for those of us with customer data), a 30 second lockup isn't going to work for that. But if that can be solved it would be awesome.

In addition to --single-transaction I see some talk about adding the --quick switch to that for very large tables.

 

[rainmotorsports]

In fact I finally figured out the switch has to come after the extra config.

$command = $backup['mysqldumppath'] . ' --defaults-extra-file=' . $backup['dbconfigfile'] . ' ' . '--single-transaction ' . $backup['dbname'] . $backup['exportoptions'] . ' 2>> ' .  $log->getPath();;

So I was able to test it. During the database backup I was able to load the site unlike before. So pending any consult I would request that switch to at least be an option (if not appropriate as a default).