HUGE Space Size Savings On Images

ProCom

Well-known member
As I posted here, I'm in the process of converting a huge site to XF.

I also posted here asking for info about resizing uploaded pics.

Well, I have about 4 TB of images that I'm moving from my old host / site. A TON of the images are around 9mb in size.

We ran a bunch of tests on some big images and we’re going to run a script to resize and recompress all the huge files. This should save us a TON of space and bandwidth!

We’re using mogrify (script with ImageMagick) to do two things:

1. Resize images to a max of 2000 x 2000
2. Compress images to 85% quality

Check out the attached screenshot of 3 version of each file we tested:

a = full size original image
b = resized and default 92% compression
c = resized and 85% compression

upload_2017-4-22_14-17-18.png

So, we'll set the script to find all images over 3mb and resize them all down.

Incredible savings on sizes! Most went from 9mb down to around 500 kb!!!
 

Mouth

Well-known member
Incredible savings on sizes! Most went from 9mb down to around 500 kb!!!
Do your images still work properly? I always understood there was a problem modifying/resizing images outside of XF since XF stores a checksum of the image file in the DB and after resizing the checksum changes and XF then has a mismatch and your have image display issues.
 

ProCom

Well-known member
Sounds great. Please share the script?
I sure will once we finish and fully test it.

Do your images still work properly?
Hmm... that's a fantastic question, and definitely something I should have thought about. We're applying this to the images from the previous platform that are not stored within a filesystem like XF, and we're making the changes before importing them into XF.
 

Mouth

Well-known member
If the images are modified before importing into XF, that should be fine.
Is there a similiar script available that will also action the DB appropriately when the images are already listed within XF?
 

eva2000

Well-known member
Well, I have about 4 TB of images that I'm moving from my old host / site. A TON of the images are around 9mb in size.

We ran a bunch of tests on some big images and we’re going to run a script to resize and recompress all the huge files. This should save us a TON of space and bandwidth!
nice, you inspired me to whip up optimise-images.sh for testing https://github.com/centminmod/optimise-images
 
Last edited:

MattW

Well-known member
Do your images still work properly? I always understood there was a problem modifying/resizing images outside of XF since XF stores a checksum of the image file in the DB and after resizing the checksum changes and XF then has a mismatch and your have image display issues.
The images aren't going to be in the XF attachment system, they are going to be served directly via nginx using the previous cdn URL.
 

Anthony Parsons

Well-known member
I personally like leaving the image optimisation process at cloudflare's server, mirage and polish handle it all (including webp), across all devices. I only downsize the image at my server on upload.
 

eva2000

Well-known member
I personally like leaving the image optimisation process at cloudflare's server, mirage and polish handle it all (including webp), across all devices. I only downsize the image at my server on upload.
i similarly do this via my Centmin Mod Nginx's ngx_pagespeed integration which does on the fly image optimisations including jpg/png to webp conversions for detected web browsers that support it https://community.centminmod.com/threads/benefits-of-ngx_pagespeed.1032/ :)

Example of my Xenforo's attachments being on the fly converted to webp for web browser that supports it
auto optimised conversion of user attachments from PNG to WebP format where browser supports WebP. Almost 88KB size reduction for those 3 images combined by converting on the fly to WebP format.
Code:
hment-files/2014/05/198_upload_*.png' (24863 bytes) to `*attachmentlink*/2014/05/x198_upload_*.png.pagespeed.ic.3Exc-iTnt7.webp' (6268 bytes)
*threadlink*/:1241: Shrinking image `*attachmentlink*/2014/05/199_upload_2014-*.png' (42496 bytes) to `*attachmentlink*/2014/05/x199_upload_2014-*.png.pagespeed.ic.MtXR6WHd7d.webp' (12718 bytes)
*threadlink*/:1248: Shrinking image `*attachmentlink*/2014/05/200_upload_2014-*.png' (53259 bytes) to `*attachmentlink*/2014/05/x200_upload_2014-*.png.pagespeed.ic.QRsR_7kaYd.webp' (14484 bytes)
 

Anthony Parsons

Well-known member
Yer, used the page speed version. Does the same thing, absolutely. I simply like mine done at cloud flare, as it removes another burden from the server, and images resized on upload are reduced enough from a storage view for me. The cached version at cloud flare is then optimal and using their system to do it.
 

MattW

Well-known member
We need to shrink these down at source initially, as the old system stored full fat versions, along with multiple smaller ones as well. Will make managing them easier in the long run.
 

eva2000

Well-known member
We need to shrink these down at source initially, as the old system stored full fat versions, along with multiple smaller ones as well. Will make managing them easier in the long run.
Does that mean the old forum system actively referenced different sizes of the same image i.e. image srcset usage ? If you're keeping the old url format, does that mean you need to also make multiple sizes of same image but just optimise each set of images as well ? That would be alot of images !
 

MattW

Well-known member
I'm hoping @Slavik knows the answer to that, as he's doing the conversion from the previous software. Inspecting the source shows most are pulling the full CDN URL when they are embedded, and allowing them to pop out in the lightbox

Not sure how the thumbnails are going to work thorugh, as they look to be a mapped URL

Code:
content/type/61/id/7803445/width/350/height/700/flags/LL
 
Last edited:

CvP

Member
Does that mean the old forum system actively referenced different sizes of the same image i.e. image srcset usage ? If you're keeping the old url format, does that mean you need to also make multiple sizes of same image but just optimise each set of images as well ? That would be alot of images !
Yes, to all.
 
Top