XF 2.3 Image optimization, enhanced image resizing, and more!

Image optimization with WebP support​

Arguably, we could have talked about this in our bumper Boosting performance entry but it wasn't quite ready to be shown at that point. (I started working on it this week and I haven't yet mastered the art of time travel).

Initially the scope of our WebP support in XenForo 2.3 was to merely allow users to upload WebP files and have them correctly displayed inline. This would have been a positive change on its own as the format becomes more prevalent across the web, but it doesn't really do much on its own to solve the issue of disk usage which, of course, then also has a positive effect on performance.

But, this wasn't enough.

hys_4_option.png


If you wish to optimize, automatically, all future image uploads you simply need to enable this option.

On upload, all currently supported image types (except GIFs) will be saved to WebP format.
Side note: As is the case now, thumbnails, profile banners and avatars (and anything else that has a programmatically set file name) will be served with a .jpg extension, regardless of the underlying file format.

If you have custom content types which already use the attachment system, such as we do with Media Gallery and Resource Manager, images uploaded to those will be automatically optimized too.

In fact, if your add-on handles uploads via the default approach, namely the XF\Http\Upload class, all new image uploads will be optimized automatically. This extends out into pretty much every system we have, including the admin asset upload system.

As a developer, if you wish to opt-out of this behaviour for any reason, you can do so with the following one-liner:

PHP:
$upload->setImageOptimize(false);

That deals with future uploads, but, many of you will be wondering how to optimize existing files. So you'll be pleased to hear that you are able to rebuild all of your existing attachments, avatars and profile banners automatically.

hys_4_rebuilds.png


These are somewhat typical rebuilds that you can kick off from within your admin control panel on the "Rebuild caches" page.

With this being a rather intensive and lengthy process, if you'd prefer to run this without the risk of browser timeouts and supervision, you can also use one of the built in commands:

Code:
xf-rebuild:attachment-optimization
xf-rebuild:avatar-optimization
xf-rebuild:profile-banner-optimization

As a developer, adding support for your own content types is trivial by extending the AbstractImageOptimizationJob class.

We have run this already on a development copy of the XenForo Community forum. In doing so, the file size reported via the xf_attachment_data table before the conversion was around 40GB. The file size reported after the conversion is around 19GB.

These are significant savings and will also have a positive impact on performance.

The "not so good" news​

WebP is now supported by every single major browser. Anyone using an up to date browser will not experience any issues with viewing images. However, certain Apple devices and browsers prior to September 2020 do not support WebP.

Specifically, if you are using iOS 14 or above, there is no issue at all. Safari 14 and above is great, too but you must be running at least macOS 11 Big Sur for WebP images to display.

In earlier browsers, these users will simply not see WebP images at all. They will appear as broken images.

As time progresses, this is naturally an ever decreasing issue as people update their software and hardware. But it is something you should be aware of and think about before blanket converting all images to WebP.
 
Does the rebuild process change xf_attachment_data.file_hash?
Sorry for going off-topic.
If I manually add a watermark to all the images (data files of the script), the file's hash will change, and it won't match the hash in the database. Is this critical?
 
I modified just a few webp files on the forum, and I received a message from a user. They mentioned they have a 2013 MacBook with Safari, which doesn't support webp. They also mentioned that Apple no longer releases updates. What should I do? Will there be support for older formats in version 2.3, perhaps through picture/srcset?
 
Tbh up-to-date safari support more image formats like jxl compared to up-to-date Chrome. But yes. It's a problem in cases where people are either using safari on old macos or any browser on old iOS. 🫣
 
Tbh up-to-date safari support more image formats like jxl compared to up-to-date Chrome.
Yeah, it's a shame that JXL support was removed from Chromium and Firefox despite community protest :(

IMHO the format has so much potential, hopefully Apple market share can generate enough pressure to revert the decisions.
 

Enhanced image resizing​

[...]

Starting with XenForo 2.3, the "attachment manager" which powers the file upload system for things like resources, media gallery and other attachments, is now able to resize images before they reach your server.
The attachment manager uses flow.js, Froala insert image dialog doesn't (in 2.2).
Will client side image resizing also work with images inserted via Froala insert image dialog?

Which format will be send to the server, always the original image format?
If this is the case it could mean that the browser converts HEIC to JEPG, XenForo JS resizes that, sends it to the server - and XenForo PHP on the server might convert that to WebP?
 
Last edited:
Will client side image resizing also work with images inserted via Froala insert image dialog?
It's on our radar but a little tricky.

Which format will be send to the server, always the original image format?
If this is the case it could mean that the browser converts HEIC to JEPG, XenForo JS resizes that, sends it to the server - and XenForo PHP on the server might convert that to WebP?
We attempt client-side conversion and fall back to server-side conversion, except for GIFs where client-side conversion is not supported.
 
It's on our radar but a little tricky.
I am sure you will find a way.

To support our chunked uploads Add-on I ended up just bypassing Froala uploads (via Insert Video dialog) and routing everything through flow.js, a similar approach could work for Images as well.

While you are working on uploads the following kinda annoying bugs could be fixed as well:

We attempt client-side conversion and fall back to server-side conversion, except for GIFs where client-side conversion is not supported.
Hmm, yeah - that doesn't really answer my question ;)

Assuming a user with Safari 17.0 attempts to upload a 4000x4000 HEIC image while XenForo is configured for a maximum attachment size of 1920 x 1920 and to automatically covert to WebP.

What will be send to the server in this case:
  • A 1920x1920 JPEG
  • A 1920x1920 WebP
  • A 1920x1920 HEIC
  • A 4000x4000 HEIC
  • Smth. else?
?
 
To support our chunked uploads Add-on I ended up just bypassing Froala uploads (via Insert Video dialog) and routing everything through flow.js, a similar approach could work for Images as well.
Yeah we're looking at just routing everything through the attachment manager.

Hmm, yeah - that doesn't really answer my question ;)
It does, but to be clearer we attempt WebP conversion client-side prior to upload. So, a 1920x1920 WebP.
 
The first step is to use the AssetVariantTrait in your Entity which has one required method to implement:

PHP:
public function getAssetVariantSizeMap(): array
{
return [
'image_url' => [
's' => 128,
'm' => [512, 456]
]
];
}

[...]

HTML:
<img src="{{ base_url(asset_display($entity, 'image_url', 's')) }}" />
Am I right to assume that there are also changes to <xf:assetuploadrow /> ?
 
Only a subset, as the full set bloats the JS from 18kb to 70kb and we only use 9 values in XFMG. We can consider unbundling the library to make it easier for add-ons to swap in a build with a bigger set if it's an important use case, but we're unlikely to ship more ourselves.
Just circling back to this one.... any subsequent thoughts about letting the full EXIF be captured for add-ons that use it?

If not, will the client side processing be something we can disable so we can carry on using the same as currently?

I'm looking forward to playing with 2.3, just have some hesitation about losing add-on functionality of capturing EXIF.
 
Not yet, but we'd likely just take the approach I outlined where add-ons are free to swap in a build of the library with a bigger set. We don't have a built-in option to disable client-side processing, but it is a simple toggle on the attachment manager (data-resize-images="false").
 
We don't have a built-in option to disable client-side processing, but it is a simple toggle on the attachment manager (data-resize-images="false").
IMHO an option to entirely turn this off (and probably have this off by default for upgrades) is a must have.

Otherwise customers who have not read this info might just upgrade and notice after some time that previously available EXIF data is no longer recorded for new uploads and there is no way to recover this (without applying the mentioned template-modification and re-uploading the affacted images).

So I would consider this a serious data-loss regression.

Ideally there should be fine-grained control where to apply client-side processing, eg. some admins might be okay with that for attachments in posts, but would like to have full EXIF data for gallery uploads.

 
Back
Top Bottom