RM 2.2 Storing files in S3 with their original name and extension

zeus3005

New member
Hello!

My community is currently storing our resources in S3 via:

However, we need to change how the files themselves are stored in the bucket.

We basically want our S3 bucket to double as a CDN, and as such we want to be able to store certain file extensions (.bsp and .bsp.bz2) to a specific location and to not have the file renamed as it is stored.

Example:
Instead of storing a file as
Code:
internal_data/attachments/0/13-6198a98605f0c2ed5a18685e17835dd0.data
We would like it to be
Code:
maps/mapname.bsp


These files are generally on the order of 10's of megabytes and we have hundreds of thousands of these. As it is today, we have to move these files around a lot and redownload the files with their original names to service our community. Getting S3 to double as a CDN for these files will reduce our bandwidth cost and storage costs by over 50%.


I'm comfortable writing code for this, I've been a developer for over 10 years, but haven't done anything with PHP or Xenforo before. If someone could point me in the right direction for doing this that would be very helpful. I assume the storage format is as it is today to avoid file name conflicts, we are fine with rejecting any files with a name collision.

Thanks!
 

Jim Boy

Well-known member
The 'renaming' is done by Xenforo as it stores the data - not by S3. Really you have four choices as I see it:
1) Create an addon that creates an additional copy of the data in the naming format you want and stores on s3 (tip - use the php sdk here, you'll know exactly what calls you are making and therefore what perms you'll need)
2) Reconstruct the file name from data stored in the database
3) See if there are any existing addons developed by someone else that suits your needs
4) Cool your heals for a while, absorb your current high cost and then switch to CloudFlare R2 when it is available

You have to brush up on S3 though, it is not file system storage, its an object store and that means some basic things just aren't doable, like renaming files.
 
Top