Quickest Way to Copy Files?

[taylor_smith]

I like to test plugins on a mirror of our live community. Our files are hefty--around 2.5 gb zipped. I need a faster/successful way to copy these to a new location.

Originally, I was using cPanel to create an archive. Now it times out.

Next, I tried: "tar -zcvf". After 12 hours, it finished . . . but the backup was incomplete. Was missing /library, among other things.

Is there a faster/better way? We manage the box (and my DBA's a volunteer, so I try to do as much as I can myself).

 

[0ptima]

Dump your database and then rsync everything over.

 

[taylor_smith]

The database is fine. I can dump it and restore it to a new database.

It's the files I can't get to backup.

 

[0ptima]

no need to backup the files. rsync will copy everything over with no problems.

 

[0ptima]

Here is an example
http://www.thegeekstuff.com/2010/09/rsync-command-examples/

 

[rdn]

/usr/bin/rsync -aruvP --itemize-changes --human-readable --stats /current/ /target/

 

[rdn]

scp -r user1@from-server:/dir1 user2@to-server:/dir2

scp -r local-directory user2@to-server:/dir

scp -r user1@from-server:/dir local-directory

cp -Rf

 

[AndyB]

I like to test plugins on a mirror of our live community.

Where is the mirror located in relation to your server? Same box? Another server? At home?

 

[JonathanW]

Where is the mirror located in relation to your server? Same box? Another server? At home?

The answer to this question will help us help you the most, as it will let us know what you're being bound by...network, disk, etc.

What's the limiting factor? Sounds like CPU to compress the files is limiting you and that perhaps ample bandwidth is available to just straight rsync/scp them?

 

[taylor_smith]

Where is the mirror located in relation to your server? Same box? Another server? At home?

All on the same box.

public_html/live
public_html/subdomains/test

Server specs:

Dual Intel L5420 2.5 Ghz (Quad Core)
Port Speed: 100 Mbps (10 TB Bandwidth)
Operating System: Centos 6
Hard Drive #1: 500GB Sata Western Digital Black
Hard Drive #2: 500GB Sata Western Digital Black
Hard Drive #3: 500GB Sata Western Digital Black
Hard Drive #4: 500GB Sata Western Digital Black

 

[AndyB]

All on the same box.

I suggest you use Rsync. This wonderful program allows one to quickly copy files from one foler to another. It will automatically sync the files so the mirrored folder will be identical to the main folder. Best of all only files that are new will be copied. It's super fast.

 

[BoostN]

Rsync is the way to go!

 

[AndyB]

Here's an example on how to use Rsync

rsync -a --delete /home/name/public_html/live /home/name/public_html/subdomains/test

This would copy everything from your live folder to your test folder.

 

[WSWD]

As everyone else said, rsync is the way to go. However, I'm quite curious...is the entire backup 2.5GB, or do you have a bunch of 2.5GB files, or what is the total size? We routinely backup 40-50GB accounts or larger on some of our cPanel boxes, and have absolutely no issues at all. There is absolutely no reason a 2.5GB backup would fail like that.

 

[taylor_smith]

Our DBA used rsync. Says a mistype can be damaging, but will teach me in a sandbox.

I'm not sure why the server isn't allowing me to archive with cPanel. I submitted tickets, the host shrugged. We manage the box, so maybe it's something we're doing. But the guy who set it up has been doing this for nearly 30 years. I dunno. :<

 

[AndyB]

Our DBA used rsync. Says a mistype can be damaging,

This is very true. What I do is put command lines into a thread that I have in a private forum, I then copy the exact command line I need to do something. That way I don't have to type anything.