HUGE Space Size Savings On Images

ProCom

ProCom

Well-known member
As I posted here, I'm in the process of converting a huge site to XF.

I also posted here asking for info about resizing uploaded pics.

Well, I have about 4 TB of images that I'm moving from my old host / site. A TON of the images are around 9mb in size.

We ran a bunch of tests on some big images and we’re going to run a script to resize and recompress all the huge files. This should save us a TON of space and bandwidth!

We’re using mogrify (script with ImageMagick) to do two things:

1. Resize images to a max of 2000 x 2000
2. Compress images to 85% quality

Check out the attached screenshot of 3 version of each file we tested:

a = full size original image
b = resized and default 92% compression
c = resized and 85% compression

upload_2017-4-22_14-17-18.webp

So, we'll set the script to find all images over 3mb and resize them all down.

Incredible savings on sizes! Most went from 9mb down to around 500 kb!!!
 
ProCom said:
Incredible savings on sizes! Most went from 9mb down to around 500 kb!!!
Click to expand...
Do your images still work properly? I always understood there was a problem modifying/resizing images outside of XF since XF stores a checksum of the image file in the DB and after resizing the checksum changes and XF then has a mismatch and your have image display issues.
 
Mouth said:
Sounds great. Please share the script?
Click to expand...
I sure will once we finish and fully test it.

Mouth said:
Do your images still work properly?
Click to expand...
Hmm... that's a fantastic question, and definitely something I should have thought about. We're applying this to the images from the previous platform that are not stored within a filesystem like XF, and we're making the changes before importing them into XF.
 
ProCom said:
Well, I have about 4 TB of images that I'm moving from my old host / site. A TON of the images are around 9mb in size.

We ran a bunch of tests on some big images and we’re going to run a script to resize and recompress all the huge files. This should save us a TON of space and bandwidth!
Click to expand...
nice, you inspired me to whip up optimise-images.sh for testing https://github.com/centminmod/optimise-images
 
Last edited:
Mouth said:
Do your images still work properly? I always understood there was a problem modifying/resizing images outside of XF since XF stores a checksum of the image file in the DB and after resizing the checksum changes and XF then has a mismatch and your have image display issues.
Click to expand...
The images aren't going to be in the XF attachment system, they are going to be served directly via nginx using the previous cdn URL.
 
I personally like leaving the image optimisation process at cloudflare's server, mirage and polish handle it all (including webp), across all devices. I only downsize the image at my server on upload.
 
Anthony Parsons said:
I personally like leaving the image optimisation process at cloudflare's server, mirage and polish handle it all (including webp), across all devices. I only downsize the image at my server on upload.
Click to expand...
i similarly do this via my Centmin Mod Nginx's ngx_pagespeed integration which does on the fly image optimisations including jpg/png to webp conversions for detected web browsers that support it https://community.centminmod.com/threads/benefits-of-ngx_pagespeed.1032/ :)

Example of my Xenforo's attachments being on the fly converted to webp for web browser that supports it
auto optimised conversion of user attachments from PNG to WebP format where browser supports WebP. Almost 88KB size reduction for those 3 images combined by converting on the fly to WebP format.
Click to expand...
Code: 
hment-files/2014/05/198_upload_*.png' (24863 bytes) to `*attachmentlink*/2014/05/x198_upload_*.png.pagespeed.ic.3Exc-iTnt7.webp' (6268 bytes)
*threadlink*/:1241: Shrinking image `*attachmentlink*/2014/05/199_upload_2014-*.png' (42496 bytes) to `*attachmentlink*/2014/05/x199_upload_2014-*.png.pagespeed.ic.MtXR6WHd7d.webp' (12718 bytes)
*threadlink*/:1248: Shrinking image `*attachmentlink*/2014/05/200_upload_2014-*.png' (53259 bytes) to `*attachmentlink*/2014/05/x200_upload_2014-*.png.pagespeed.ic.QRsR_7kaYd.webp' (14484 bytes)
 
Yer, used the page speed version. Does the same thing, absolutely. I simply like mine done at cloud flare, as it removes another burden from the server, and images resized on upload are reduced enough from a storage view for me. The cached version at cloud flare is then optimal and using their system to do it.
 
We need to shrink these down at source initially, as the old system stored full fat versions, along with multiple smaller ones as well. Will make managing them easier in the long run.
 
MattW said:
We need to shrink these down at source initially, as the old system stored full fat versions, along with multiple smaller ones as well. Will make managing them easier in the long run.
Click to expand...
Does that mean the old forum system actively referenced different sizes of the same image i.e. image srcset usage ? If you're keeping the old url format, does that mean you need to also make multiple sizes of same image but just optimise each set of images as well ? That would be alot of images !
 
I'm hoping @Slavik knows the answer to that, as he's doing the conversion from the previous software. Inspecting the source shows most are pulling the full CDN URL when they are embedded, and allowing them to pop out in the lightbox

Not sure how the thumbnails are going to work thorugh, as they look to be a mapped URL

Code: 
content/type/61/id/7803445/width/350/height/700/flags/LL
 
Last edited:
eva2000 said:
Does that mean the old forum system actively referenced different sizes of the same image i.e. image srcset usage ? If you're keeping the old url format, does that mean you need to also make multiple sizes of same image but just optimise each set of images as well ? That would be alot of images !
Click to expand...
Yes, to all.
 
You must log in or register to reply here.
Back
Top Bottom