Lack of interest Single file upgrading

[Gladius]

I'm not really sure if this is feasible or not but I'm throwing it in here anyway. The XF upgrade package alone is around 3000 files, which takes a ridiculous amount of time to upload even on a very fast connection simply due to the huge number of individual files.

Sure, those of us with the technical know-how can upload the zip and handle this on the server end, but I expect that we're in the minority. And it's a chore to have to do it this way even for us, especially because the same method/commands don't necessarily work in all of the various server environs that we use if we manage more than one instance of XF. Also, any mistakes in the process can be very painful.

So what I would propose would be a solution where we would only need to upload a single upgrade file, which would be properly extracted by XF itself in the upgrade process. Obviously, the option would still have to be there to upload individual files when/if needed, so two packages would have to be provided. But in the majority of cases, I expect that a single file could be used.

 

[Chris D]

The only real barrier to this is ensuring there is something server side to actually unpack the single file.

As an example, it's surprising how frequently I've seen sites that don't have basic ZipArchive support in PHP.

So we would have to make that a requirement and that may not always be feasible.

 

[Gladius]

Couldn't you include something to unzip with XF itself?

But honestly, with as many requirements that XF already has, adding another that would make otherwise painful upgrades for most a breeze instead would really not be viewed as a bad thing I expect. I'm sure that many admins now skip a lot of releases that aren't critical simply because upgrading isn't a quick & easy process -- but it really could and should be.

 

[Chris D]

Couldn't you include something to unzip with XF itself?

No. If it was a zip file (there may be alternatives) it would effectively need to be the ZipArchive class which is standard within PHP if PHP has been compiled with it. It's the latter bit that's problematic.

But honestly, with as many requirements that XF already has, adding another that would make otherwise painful upgrades for most a breeze

Well, therein lies the paradox. It's only a painless upgrade and a breeze if you happen to meet that requirement. If you'd just spent $140+ on software and then couldn't install it or upgrade it because your host hasn't enabled the proper support, then it's far from painless. XF doesn't actually have that many requirements and those that it does have are almost guaranteed to be automatically available on any host. Unfortunately if we added ZipArchive (or some other), we would be commonly sending people away to have components installed that they didn't already have.

Suggestion noted, and all, but realistically we can't put barriers in people's way to ultimately save what is no more than a 10 minute job every 4-6 weeks.

 

[Gladius]

PHP extensions: MySQLi, GD (with JPEG support), PCRE, SPL, SimpleXML, DOM, JSON, iconv, ctype

I seriously doubt that there are many hosts out there that aren't specializing in XF hosting that will have all of those extensions installed by default. Just sayin'...

Also, since you'd still need to provide the zip with individual files, you wouldn't really be preventing anyone from taking hours to do it the hard way like they're doing it now, right? This isn't an everything or nothing situation. Nothing would change for those without proper unzip support.

Also, uploading all the files via FTP took me over an hour over a FTTH connection so I'm not sure where your 10 minutes figure is coming from. Doing it via a slower connection would likely take several hours.

 

[Slavik]

Also, uploading all the files via FTP took me over an hour over a FTTH connection so I'm not sure where your 10 minutes figure is coming from. Doing it via a slower connection would likely take several hours.

Sounds like you need to change your ftp client settings or ftp server?

I can upload 3000 files in just shy of 30 seconds if I set my simultaneous transfers to 20.

 

[Gladius]

What should I look for? I'm getting pretty much the same speeds when uploading large numbers of files across several different servers.

 

[Slavik]

Just some things to look for off the top of my head would be max simultaneous connections, any transfer speed limit throttling, ensure transfers are set manually to binary rather than auto-negotiation or ascii.

First and last file of a full package upload, so, 1 minute 8 seconds to upload the full XenForo package. (done with 8 concurrent connections)



FWIW however, I would like to see a Magento SUPEE style upgrade offered for more technically able users, upload 1 file, run the script and it does the rest. I've also seen it done in a way where you only ever upload 1 upgrade script once, and just keep going back to it and running it each time you want to check or run an upgrade, its somewhat "secure" as it only downloaded the diffs, not the full package so even if someone unlicensed got hold of it, it wouldn't do them much good unless they already had the software installed.

 

[Gladius]

Hmm, I'm using Total Commander and it doesn't have much in terms of FTP options. Binary is my default, the only thing that looks like it might be slowing things down is "Compress during transfer (MODE Z)" so I turned that off.

 

[Slavik]

Hmm, I'm using Total Commander and it doesn't have much in terms of FTP options. Binary is my default, the only thing that looks like it might be slowing things down is "Compress during transfer (MODE Z)" so I turned that off.

I use FileZilla. Maybe try that.

 

[Gladius]

I have used FZ in the past, but I don't recall there being much of a difference with large batches of files, though it might be setting-dependent so I'll check again.

 

[Gladius]

Ah, this explains it: http://www.ghisler.ch/board/viewtopic.php?t=38562

 

[Jeremy P]

I seriously doubt that there are many hosts out there that aren't specializing in XF hosting that will have all of those extensions installed by default. Just sayin'...

What makes you say that?

PCRE is a core extension, and can't be disabled in PHP 5.3+
SPL is enabled by default, and can't be disabled in PHP 5.3+
SimpleXML is enabled by default in PHP 5.1.2+
DOM is enabled by default
JSON is enabled by default in PHP 5.2+
iconv is enabled by default
Ctype is enabled by default in PHP 4.2+

The only two outliers are MySQLi, which I guarantee any standard LAMP host has enabled, and GD, which is the defacto PHP image manipulation extension and required by just about every widely used PHP application.

ZipArchive (and similar) is not enabled by default, or quite as popular as MySQLi and GD.

With all of that said, I still like the idea. There's nothing to say that ZipArchive support couldn't be detected and used when available, falling back on manual upgrades when it's not. In theory, it could even use the exact same zip file as the manual upgrade. It just seems unclear what percentage of XenForo users on shared hosting would be able to make use of it, while those with their own servers can probably just upload the zip and unzip it over SSH already.

 

[Paul B]

It just seems unclear what percentage of XenForo users on shared hosting would be able to make use of it

In a lot of cases, those on shared hosting typically have cPanel (or similar) available, so zip archives can be extracted directly on the server using the File Manager (or other) app.

 

[Jeremy P]

In a lot of cases, those on shared hosting typically have cPanel (or similar) available, so zip archives can be extracted directly on the server using the File Manager (or other) app.

I hadn't even thought of that. I've never used cPanel and the lot, but they must be at least as common as ZipArchive support in PHP, if not more.

I don't know if they support the creation of symbolic links, but what I do is create a regular directory beneath the web root, let's say 'xenforo', and create a symbolic link in that directory named 'upload' that links to my installation. Then updating the installation is simply a matter of uploading and unzipping the package into the 'xenforo' directory (and running the upgrade process).

 

[Gladius]

What makes you say that?

PCRE is a core extension, and can't be disabled in PHP 5.3+
SPL is enabled by default, and can't be disabled in PHP 5.3+
SimpleXML is enabled by default in PHP 5.1.2+
DOM is enabled by default
JSON is enabled by default in PHP 5.2+
iconv is enabled by default
Ctype is enabled by default in PHP 4.2+

The only two outliers are MySQLi, which I guarantee any standard LAMP host has enabled, and GD, which is the defacto PHP image manipulation extension and required by just about every widely used PHP application.

ZipArchive (and similar) is not enabled by default, or quite as popular as MySQLi and GD.

With all of that said, I still like the idea. There's nothing to say that ZipArchive support couldn't be detected and used when available, falling back on manual upgrades when it's not. In theory, it could even use the exact same zip file as the manual upgrade. It just seems unclear what percentage of XenForo users on shared hosting would be able to make use of it, while those with their own servers can probably just upload the zip and unzip it over SSH already.

The last few dedicated cPanel/CentOS servers I used for XF didn't have even half of those extensions enabled by default and I had to recompile PHP every time, so your info on what's on by default is somewhat off or dependent on other factors that you're not taking into account. It'd certainly be nice if it was all enabled by default, though!

 

[Jeremy P]

All of those are correct as of PHP 5.3+ per the official PHP documentation and would require compile-time flags to disable in the first place. I've edited the post to link the relevant documentation. Some distros/cPanel may have packaged PHP with a non-standard configuration, however. I've never used CentOS, but in Debian, for example, you'd need:

• php5
• php5-gd
• php5-json
• php5-mysql

Still, no (re)compilation necessary. The cPanel documentation says all of the modules are included too, even GD and MySQLi. I was talking more about shared hosts though. I'd be surprised if there was any half-decent PHP/MySQL host without all of those extensions available.

 

[Gladius]

Well, my own experience was apparently with a bunch of non-standard configs then... *shrugs* If the XF install script says it's not there, it just isn't, not much point arguing that it should be because that won't magically make it so without a recompile. I think that your cPanel link just lists what's included with a certain version of cP/PHP, not what's enabled by default, btw. Various cheapo shared hosts have everything and the kitchen sink enabled usually but you can't run big boards on dime hosts.