1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Quickest Way to Copy Files?

Discussion in 'Server Configuration and Hosting' started by taylor_smith, Apr 6, 2014.

  1. taylor_smith

    taylor_smith Well-Known Member

    I like to test plugins on a mirror of our live community. Our files are hefty--around 2.5 gb zipped. I need a faster/successful way to copy these to a new location.

    Originally, I was using cPanel to create an archive. Now it times out.

    Next, I tried: "tar -zcvf". After 12 hours, it finished . . . but the backup was incomplete. Was missing /library, among other things.

    Is there a faster/better way? We manage the box (and my DBA's a volunteer, so I try to do as much as I can myself).
     
  2. 0ptima

    0ptima Well-Known Member

    Dump your database and then rsync everything over.
     
  3. taylor_smith

    taylor_smith Well-Known Member

    The database is fine. I can dump it and restore it to a new database.

    It's the files I can't get to backup.
     
  4. 0ptima

    0ptima Well-Known Member

    no need to backup the files. rsync will copy everything over with no problems.
     
    taylor_smith likes this.
  5. 0ptima

    0ptima Well-Known Member

  6. RoldanLT

    RoldanLT Well-Known Member

    Code:
    /usr/bin/rsync -aruvP --itemize-changes --human-readable --stats /current/ /target/
     
    taylor_smith likes this.
  7. RoldanLT

    RoldanLT Well-Known Member

    scp -r user1@from-server:/dir1 user2@to-server:/dir2

    scp -r local-directory user2@to-server:/dir

    scp -r user1@from-server:/dir local-directory

    cp -Rf
     
    taylor_smith likes this.
  8. AndyB

    AndyB Well-Known Member

    Where is the mirror located in relation to your server? Same box? Another server? At home?
     
  9. jwright

    jwright Active Member

    The answer to this question will help us help you the most, as it will let us know what you're being bound by...network, disk, etc.

    What's the limiting factor? Sounds like CPU to compress the files is limiting you and that perhaps ample bandwidth is available to just straight rsync/scp them?
     
  10. taylor_smith

    taylor_smith Well-Known Member

    All on the same box.

    public_html/live
    public_html/subdomains/test

    Server specs:

    Dual Intel L5420 2.5 Ghz (Quad Core)
    Port Speed: 100 Mbps (10 TB Bandwidth)
    Operating System: Centos 6
    Hard Drive #1: 500GB Sata Western Digital Black
    Hard Drive #2: 500GB Sata Western Digital Black
    Hard Drive #3: 500GB Sata Western Digital Black
    Hard Drive #4: 500GB Sata Western Digital Black
     
  11. AndyB

    AndyB Well-Known Member

    I suggest you use Rsync. This wonderful program allows one to quickly copy files from one foler to another. It will automatically sync the files so the mirrored folder will be identical to the main folder. Best of all only files that are new will be copied. It's super fast.
     
    0ptima likes this.
  12. BoostN

    BoostN Active Member

    Rsync is the way to go!
     
    0ptima likes this.
  13. AndyB

    AndyB Well-Known Member

    Here's an example on how to use Rsync

    Code:
    rsync -a --delete /home/name/public_html/live /home/name/public_html/subdomains/test
    
    This would copy everything from your live folder to your test folder.
     
  14. WSWD

    WSWD Well-Known Member

    As everyone else said, rsync is the way to go. However, I'm quite curious...is the entire backup 2.5GB, or do you have a bunch of 2.5GB files, or what is the total size? We routinely backup 40-50GB accounts or larger on some of our cPanel boxes, and have absolutely no issues at all. There is absolutely no reason a 2.5GB backup would fail like that.
     
  15. taylor_smith

    taylor_smith Well-Known Member

    Our DBA used rsync. Says a mistype can be damaging, but will teach me in a sandbox.

    I'm not sure why the server isn't allowing me to archive with cPanel. I submitted tickets, the host shrugged. We manage the box, so maybe it's something we're doing. But the guy who set it up has been doing this for nearly 30 years. I dunno. :<
     
  16. AndyB

    AndyB Well-Known Member

    This is very true. What I do is put command lines into a thread that I have in a private forum, I then copy the exact command line I need to do something. That way I don't have to type anything.
     
    taylor_smith likes this.

Share This Page