• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Compress/decompress Large directories?

Dean

Well-known member
#1
Did some testing a while ago, and it seems the linux zip command will not unzip if the resulting compressed file size is 2GB or greater.

Any suggestion how to compress/decompress large directories on a linux server?
 

Fred Sherman

Well-known member
#4
It has nothing to do with gzip, zip, tar, cpio or any other command. Its a kernel parameter that isn't allowing files larger than 2GB. Try and FTP a file larger than 2GB. It won't work either.

# ulimit -a

Look for "file size"

It should look something like this:

[fasherman@localhost ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7833
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

unlimited is what you want to see.

append the line:

* hard fsize 0

to /etc/security/limits.conf and reboot.
 

Dean

Well-known member
#5
It has nothing to do with gzip, zip, tar, cpio or any other command. Its a kernel parameter that isn't allowing files larger than 2GB. Try and FTP a file larger than 2GB. It won't work either.

# ulimit -a

Look for "file size"

It should look something like this:

[fasherman@localhost ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7833
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

unlimited is what you want to see.

append the line:

* hard fsize 0

to /etc/security/limits.conf and reboot.
Bold line: yes that is what it says.

However while I have root access on the server that I am using, it is not my server. So I am reluctant to make any changes to the limits.conf file. So I will use a work around that I discovered shortly after I started this thread.

But.... now I have the answer, thanks Fred! (y)