Lack of interest Batch add-ons installation/upgrade with a single cache rebuild

[ManOnDaMoon]

Hi,

I thought about this suggestion while working on my local development copy, and while deploying on my production site: add-on installation can be a very time consuming process, and therefore multiple add-on installation process can take quite a while:
1- Launch add-on1 installation
2- Go take a coffee
3- Check installation process: still buiding templates
4- Go take a second coffee
5- Check installation process: installation completed, go to step 1 with add-on2

Would it be feasible to feed XenForo with several add-ons XML files at the same time, thus sensibly shortening the installation/upgrade time? User interaction would be limited to the first step, all caches would be rebuild only once, maintenance time would probably be reduced. Also, would reduce the amount of money spent at the coffee machine.

 

[Chris D]

Interesting.

I think it's a great idea, but are you exaggerating, just a little bit?

On my local development copy, the rebuild phase takes less time than it takes to boil the kettle.

There may be an underlying issue with your development copy. Is it running on something like WAMP? Or is it in a virtual machine?

Personally, I use WAMP and it's pretty quick on a not too special laptop.

Alternatively, do you have loads of styles installed? The most common reason for the rebuild phase taking a while is having lots of styles installed. Removing unnecessary ones might be beneficial.

 

[ManOnDaMoon]

Hi Chris,

My working copy is locally installed on a not-so-old Vista-designed laptop running WinXP and XAMPP where I do my developments. I also use it as a preprod env. I can assure you the rebuild phase is just long enough to drink a coffee (the coffee-machine coffee, not the boiler one).

My live website, though on a shared host, is quicker in the rebuild phase. I only have the default style with a couple of customizations (you can check in my sig).
Anyway, even on the live environment, the rebuild phase takes up to 30-60s (not precisely timed). I believe websites with a long list of add-ons to maintain would gain in simplifying the install/upgrade process.

 

[Chris D]

I agree it's definitely a good suggestion and very beneficial regardless of how long the rebuild phase takes. Whether it takes 1 minute or 10, this suggestion is definitely a good one.

 

[Deebs]

Was just talking with my developer and he mentioned this exact same thing. Either this or somehow maximise more than 1 PHP process when rebuilding the templates to speed it up.

 

[Chris D]

My add on installer does this.

You can upload multiple zips and it only does one cache build.

On a test install I installed 10 add ons and it took just over one minute including cache rebuild. The bulk of the time was the copy and extract process.

Coming soon...

Upload multiple XML files ... So effectively good old traditional add on install (upload files yourself via FTP the install with the XML) but with multiple xml and only one cache rebuild.

 

[Deebs]

My add on installer does this.

You can upload multiple zips and it only does one cache build.

On a test install I installed 10 add ons and it took just over one minute including cache rebuild. The bulk of the time was the copy and extract process.

Coming soon...

Upload multiple XML files ... So effectively good old traditional add on install (upload files yourself via FTP the install with the XML) but with multiple xml and only one cache rebuild.

Chris,

Sounds great but I refer you to my private conversation I started with you.....

 

[Biker]

While this all sounds wonderful, I think it could create issues for someone who uploads 10 addons at once, and finds the site is now broken in some way. At least with single additions, you know what the cause is when something goes wrong. Trying to troubleshoot which addon is the cause will become more difficult with batch processing.

 

[Chris D]

That's fine.

Many add ons one rebuild already works for automatic zip upload.

I'm talking about if you were to do it the old fashioned way then you can upload the files manually and then upload multiple XML files with one rebuild.

 

[ManOnDaMoon]

Thanks Chris. I never had a look at your add-on because I dont find manually installing addons such a difficult task, but I'm interested in such features that deeply enhance add-ons install processes. I'll have a look.

One other use case during which I encounter multiple template rebuild processes is when I install a language pack.
1) Install add-on or add-on update, wait for cache rebuild.
2) Install language pack, wait for second cache rebuild.

 

[Deebs]

My preference is to spawn multiple processes to rebuild the template cache between them.

 

[Chris D]

I nearly put this in small tweaks, but actually it's bigger than you might give it credit for.

There is a HTML5 attribute on file inputs that allows you to specify the allowed file types, and set it to allow multiple files:

<input type="file" name="upload_file[]" value="" id="ctrl_upload_file" multiple="multiple" accept="application/zip">

That is from one of my add-ons and it allows you to upload multiple ZIP files in one go.

If you're installing many add-ons, it makes sense for your time and sanity to only do the cache rebuild once.

This is possible right now with a work around. You can upload the files for all of your add-ons, then click back immediately when you see the Rebuilding Caches screen come up. You can repeat this for each XML you need to import.

Ideally, if you could allow multiple XML files to be imported at once then it makes the task much easier.

Where this becomes a bigger feature, more than a small tweak is the code that is required to loop through the files. But also, how you handle errors. If you import 10 XML files, ideally you want to catch an error, continue with the other XML files, log further errors and only report the errors at the end on a single screen. The code currently isn't geared up to handle that.

But overall, would be a nice thing to have.

 

[tyteen4a03]

For now you have the config option $config['rebuildMaxExecution']. Setting it to 300 or so sped up rebuilding speed for me.