XF 2.1 Add-on list loads extremely slowly

[Jake B.]

As of 2.1.1 (I think this is when it started) our add-on list is extremely slow. We've got around 80 add-ons or so saved in our repository and it takes around 7 seconds to fully load the page. If you have a lot of add-ons installed as well the admin index page becomes extremely slow (15 add-ons takes around 5 seconds)

 

[Chris D]

I’m not certain what would be causing a bottleneck exactly. We’ve not had any other reports of this and I’ve come across one or two boards with many add-ons that aren’t exhibiting the behaviour.

Is this reproducible? Is the query count on the add-on list unexpectedly high? Is the debug page time also long, or is it short therefore indicating an issue with the browser rendering the page?

Is this reproducible on multiple, different installations?

Are the add-on icons all FA icons or images? If you remove those icons (maybe a find/replace on all the addon.json files is simplest) does the issue persist?

If not, then what size and format are the icon files?

 

[DragonByte Tech]

I didn't want to bog down the bug report with "works for me" but... well, it works for me

I have 30 addons installed; 5 FontAwesome icons, 2x xenForo logo and 23x DBTech icons. My dev machine is a bit of a potato (2945 in Geekbench), and I'm running it in Docker, the Add-ons page loads in 2 seconds on average.

That's maybe a second or so slower than the average XF page on this machine, for reference.

 

[Jake B.]

Is this reproducible? Is the query count on the add-on list unexpectedly high? Is the debug page time also long, or is it short therefore indicating an issue with the browser rendering the page?

Yeah, we're able to reproduce it on all of our installs using our XF2 add-on repository. Debug says page generation is around 7 seconds

Is this reproducible on multiple, different installations?

Yeah, happens for both myself and @Lukas W.

Are the add-on icons all FA icons or images? If you remove those icons (maybe a find/replace on all the addon.json files is simplest) does the issue persist?

They're all images, around 150x150px. If I remove all of the icons it drops down to around 400ms page generation time, so it's definitely related to reading and then rendering the icons as base64 inline

 

[Lukas W.]

Is this reproducible? Is the query count on the add-on list unexpectedly high? Is the debug page time also long, or is it short therefore indicating an issue with the browser rendering the page?

Absolut constant issue, happens every single time. Regardless of whether debug and/or development mode is enabled, whether add-ons are disabled or not, etc. Only thing that reduces the time is reducing the number of add-ons. ACP index runs 38 queries on 5.1~5.8s average load time, 18MB memory, add-on list runs on 12 queries, 6.5~7.7s 18MB memory average.

If not, then what size and format are the icon files?

PNG, 256x256px or 150x150px.

 

[Jake B.]

Did some testing, and in \XF\AddOn\AddOn::getIconUri I've done the following:

$start = microtime(true);
$imageManager = \XF::app()->imageManager();
$image = $imageManager->imageFromFile($tempFile);

$return = $image ? $image->getDataUri($tempFile) : null;
$end = microtime(true);

and just from this block of code it ends up taking around 4.5 seconds to handle converting the image contents to the inline data value

 

[Chris D]

There's got to be more to this. I've just created this script:

<?php

$dir = __DIR__;
require ($dir . '/src/XF.php');

XF::start($dir);
$app = XF::setupApp('XF\Pub\App');

for ($i = 1 ; $i <= 500; $i++)
{
   $addOnId = 'TestAddOn' . $i;

   $tempFile = \XF\Util\File::getTempFile();
   $path = \XF::getAddOnDirectory() . '/' . $addOnId . '/icon.png';
   \XF\Util\File::copyFile($path, $tempFile, false);
   clearstatcache(); // older PHP versions appear to cache the 0 byte size when the file was created

   $imageManager = \XF::app()->imageManager();
   $image = $imageManager->imageFromFile($tempFile);

   $uri = $image ? $image->getDataUri($tempFile) : null;

   if ($uri)
   {
      echo '<img src="' . $uri . '" style="width: 64px" /><br /><br />';
   }
}

\XF::dump(number_format(microtime(true) - $app['time.granular'], 4));

(Yes, I have 500 unique add-ons I had to write a script to create that many for this purpose)

And the diff dumped at the end is 2.1815 seconds.

Of course loading the add-on list could be different, but it isn't:

Queries 10 Time 2.4894s Memory 11.32MB

To be honest, I'd be happy with 2.5 seconds even at 100 add-ons. This is 5 times that.

Not sure if I'm missing something significant?

 

[DragonByte Tech]

If the issue is getting the image from file, then the issue may be in the GD or ImageMagick library and/or a problem with the image causing a performance-draining issue in said library, whichever you're using.

I don't know if TH uses a standardised Docker or similar VM image that might explain why multiple members of your team would experience the same issue, but if so, there might be something worth investigating there.

Other than that, the only other thing I can think of that might sort this out is if XF were to modify the xf_addon table to perhaps store the base64 image data there, updating that data the same way you currently update the rest of the data if the json_hash has changed?

 

[Jake B.]

Out of curiosity, what is the filesize of the "icon.png" you're using? The largest one we have being loaded that I could track down is 256x256px and 7.82kb (i've attached it here)

 

[Jake B.]

I don't know if TH uses a standardised Docker or similar VM image that might explain why multiple members of your team would experience the same issue, but if so, there might be something worth investigating there.

We have a docker image, but I'm not using it currently. Just have Nginx, php 7.3, mariadb, etc installed locally on my MBP through brew

 

[DragonByte Tech]

Out of curiosity, what is the filesize of the "icon.png" you're using? The largest one we have being loaded that I could track down is 256x256px and 7.82kb (i've attached it here)

View attachment 200032

Attached mine, it's a transparent 96x96 @ 12 KB.

We have a docker image, but I'm not using it currently. Just have Nginx, php 7.3, mariadb, etc installed locally on my MBP through brew

The plot, like me when I look at a piece of bread, thickens If you're both using different image libraries then that would rule out my theory of it being a bug.

Have you tried installing the other library than the one you're using and switching XF over to using it, in case that makes a difference?

 

[Chris D]

It does take longer now for 100 but that's still 2.1291 seconds which isn't too bad IMO.

 

[Lukas W.]

The systems are ultimately about as different as it gets. Nginx php7.3 on Mac via Brew vs. Apache php7.1 on Windows via Xampp. They've been installed and raised individually as well. The only thing they have in common are the files.

Just to verify, I've spun up a 2.1 install on my community server and uploaded our repository files there as well. Nginx php-fpm (7.2.) on CentOS, fresh XF install, no add-ons installed. The result is entirely the same there as well, 7s on the add-on list.

If it helps, I can send the metadata package with the addon.json files and the icons over in a bundle tomorrow. Maybe a single image isn't enough to reproduce the issue, or the particular image we provided is not what is causing it.