Fixed Unexpectedly high CPU usage during add-on install - CSS repeatedly invalidated

[Xon]

Affected version

2.1.4

On my own sites and another large site, I've observed that XF will use massively more CPU during the add-on installs & adding new templates/styles operations above what is expected (that is, every other session not just the admin session doing the changes). With the is_processing flag, these add-ons should not be actually running and points to some cache being repeatedly invalidated.

On investigating; I believe it is the TemplateRebuild/TemplatePartialCompile (and others) that use the \XF\Service\Template\Compile which updates all styles last modified date, which then invalidates all cached css.

This style invalidation appears to occur when templates are imported, template modifications are imported, and phrases are imported.

This happens per web-request rather than at the end of the template/phrase rebuild/compiling jobs. With batch installing, and complex add-ons, this can result in a large number of requests during the installing phase.

The xf_css_cache table is also evacuated per web-request, but before the style last modified date is updated, resulting in this table receiving a lot more write traffic than expected. This can result in xf_css_cache causing unexpected deadlocks as well.

On a few client sites; I've got some install scripts which simply prevent updateAllStylesLastModifiedDate doing any work installing/updating add-ons massively decreased the performance hit of installing add-ons.

 

[Kirby]

I've seen this increased load as well though didn't bother to track it down.

Ideally, updateAllStylesLastModifiedDate should be called only once after everything is finished.

 

[GameNet]

couple days ago I have noticed this as well

 

[ShikiSuen]

Looks like this can be a joint issue with mine:

XF 2.1 - ErrorException: Fatal Error: Out of memory (allocated 35651584) (tried to allocate 4096 bytes)

I got these errors happening everytime I do mass modification to templates or something similar (i.e. replacing the Terms and Rules article content.) My server is Vultr VPS which uses the cheapest plan. MySQL Ngram smallest length is 2 (for searching Chinese texts). Database Table types are...

xenforo.com

 

[XF Bug Bot]

Thank you for reporting this issue, it has now been resolved. We are aiming to include any changes that have been made in a future XF release (2.2.0 RC 1).

Change log:

Attempt to reduce number of CSS cache invalidations, particularly when batch installing add-ons.

There may be a delay before changes are rolled out to the XenForo Community.

 

[Mike]

So to follow up with a bit more detail, we're using the system that allows us to dequeue run later jobs in 2.2 so that we remove the last modified time update when we know that we can trigger it later. This means for things like add-on data imports and some template rebuild jobs.

In general, this means that the date will only be updated once per add-on data import and that should significantly improve this, particularly on slower servers/with bigger add-ons. (The amount of rebuilds triggered also related to the number of "pages" there were in the rebuild process so a fast server might've only triggered it a couple times across all the jobs anyway, though that was still more than needed.)