XF 2.2 internal data is full

[javakhir]

the first 500Gb NVMe is used for forum

I have mounted separate 14TB HDD as /internal_data
to keep files & resources in one place

but now facing a problem with space, 14TB HDD as /internal_data is full

questions is:

is there any way we could leave that disc as it is, connect the second new HDD as another folder like internal_data2 to extend space?

point is to continue uploading files without touching /internal_data

 

[Chris D]

That’s a lot of storage.

Unfortunately, your only option would be to either migrate to remote storage, such as AWS S3 or a compatible service, migrate over to a larger internal disk or extend your current one using some sort of RAID array setup.

 

[javakhir]

AWS S3 never will be option for me, $0.023 per GB = 1TB = $23, 14TB would be $322

I pay only $30 per month for 14TB HDD, another $30 for the second HDD for backup.

 

[Chris D]

Sure, but there are cheaper and perfectly reasonable compatible alternatives including Backblaze B2 which would be $0.005 / $71.68 for 14TB (or thereabouts).

But if that cost is prohibitive you either need a larger single disk or to add another disk with RAID.

 

[javakhir]

I knew this before starting import VB to XF, that would face this anyway. adding another 18TB now will stretch time, after 6 months when the second disc is full, will face this again.

Willing to go through hard way but correct one, decided to buy separate server with 4 HDDs each,
download all files to my pc & bulk upload to new server via FTP
next step link each file as external link in resources

because need about 200TB in total

just wanted to be sure, before starting this

 

[Sim]

What do you have in internal_data that is taking up so much space?

I'm guessing videos?

I have 500,000 images on one of my sites and it takes less than 200GB of space.

14TB is huge.

 

[javakhir]

we have mobile phones firmware, attachments - Showing 20 of 213,886 items

we had another 2 slots for HDD, so connected 2 x 14TB & mounted as separate folders, files1, files2

now I can bulk upload files via ftp, after just creating resources with external URL, example forum.com/files1/SM-G998B_SER_G998BXXU3AUIE.zip
which is already in my server.

 

[duderuud]

What I tend to do is adding a bigger disk and create a symlink for the data and internal_data directories.

(For me it's much easier, I use a VPS with attached storage. I can just add more storage to that virtual disk).

 

[javakhir]

using raid to mount disk into one directory & symlink are dangerous, no one guarantees that one day one of HDD will go off, during 19 years we lost our data 7 times with German hetzner, all of them were Seagate, finally, we moved to another hosting, now using only western digitals,
so far, users still upload files in resources, from my side, I download one by one, sort them nicely according to file names & folders alphabetically, now it is getting easier this way, I did put 500GB to upload via FileZilla & went to sleep, till the morning all uploaded

I remember myself uploading files one by one, 700 GB which is 500 resources took me 1-2 month,
now this is a correct way to build forum with 100k resources in short period of time

 

[duderuud]

during 19 years we lost our data 7 times

That's why I create daily/weekly backups.