Why /upload?

[feldon30]

I've been with XenForo since vBulletin 3.5 days, although I don't have as many licenses as I'd wish. Since the point of XenForo is to reevaluate everything and not just carry the same habits forward because that's how we've "always done it" can we have a conversation about packaging?

Can I ask -- why is the xenforo_2.2.12_xxxxxxxxxxxxx_#############_upgrade package arranged like this:

/        -- literally nothing except a 500 byte text file that tells you go to the website
/upload/ -- the software

It seems like this makes upgrading unnecessarily complicated?

To work around this, we have
Option A:

1. Unzip xenforo_2.2.12.....zip on your PC and spend 30 minutes replacing files with FileZilla cause individual FTP socket connections are slower than a glacier (I'm on Gigabit Fiber, it's not the speed, it's the handshaking, overhead, and individual file checking).

Option B:

1. SSH
2. Very carefully do a recursive replace cp
3. Do an rm -rf * on the upload directory and don't f--- up.

Option C (my preferred):

1. Rename public_html to upload, unzip -o xenforo_2.2.12.....zip
2. Rename upload back to public_html
3. All told this takes 30 seconds

Could we just rid of this /upload path? Clearly it was problematic otherwise the XF devs wouldn't have spent considerable time and effort developing one-click upgrades.

 

[Mouth]

Option C (my preferred):

1. Rename public_html to upload, unzip -o xenforo_2.2.12.....zip
2. Rename upload back to public_html
3. All told this takes 30 seconds

Urrgghhh, that's very scary! If using a upgrade package, you're assuming (unwisely!) that all files are being replaced, and this may not be the case and thus leaving you with missing files.

Option B:

1. SSH
2. Very carefully do a recursive replace cp
3. Do an rm -rf * on the upload directory and don't f--- up.

Step 3 isn't need. Don't copy the upload dir into your XF installation, just copy the upload files.

 

[Paul B]

If using a upgrade package, you're assuming (unwisely!) that all files are being replaced, and this may not be the case and thus leaving you with missing files.

Upgrade packages contain all of the files a new installation package does, except for data , internal_data , and .htaccess .

Even so, that's still not a way I would ever upgrade.

 

[Mendalla]

I always used Option A before one-click upgrades was added (I've been using XF since 1.3) and didn't find it too terrible over my 5/30 cable connection. Never had problems save when I ****ed things up one time and that was entirely on me, not the method. Would need to reinstall Filezilla now if I ever had to do it again since I've been using One-Click ever since it came out so never bothered to install FZ on my current box.

 

[feldon30]

Urrgghhh, that's very scary! If using a upgrade package, you're assuming (unwisely!) that all files are being replaced, and this may not be the case and thus leaving you with missing files.

Upgrade packages contain all of the files a new installation package does, except for data , internal_data , and .htaccess .

Even so, that's still not a way I would ever upgrade.

Unzipping the upgrade package ZIP over top of your live install (public_html et al.) is mechanically identical to uploading the unzipped package on top of your live install with FileZilla which is the officially supported and recommended method, except it takes 3 seconds instead of 20-30 minutes. The XenForo 2.2 upgrade package is 4,886 Files in 769 Folders. There's a ceiling to how fast FileZilla can do that that is not lowered by having faster internet.

Step 3 isn't need. Don't copy the upload dir into your XF installation, just copy the upload files.

You can't delete a full directory. If you're doing your forum migration late at night, you may be a little bleary-eyed. rm -rf * in those circumstances is always worthy of a triple check. I'd rather simply not have to touch that command when I'm half asleep.

 

[digitalpoint]

I think there's never going to be a one-size fits all approach. If you have an upload folder, it's not going to suit some people's way of uploading. If there wasn't an upload folder, it would be problematic for some users for other reasons. As an example, I don't use one-click upgrade (I don't allow my web server to have write permission to normal PHP files for security reasons, and I also have multiple web servers... so letting the web server upload the files is not only a security issue, but you would also be left with only 1 web server with the updated files). I upload the archive to one of the servers, unzip xxx;cp -R zzz;cluster-sync as needed, and having the archive not have a root folder and just spraying files and directories all over the place in whatever directory you are in (I'm not going to put the .zip file in the root of a publicly accessible folder that the web server serves) is just different problems.

I don't upload the individual files from a computer, just the archive, but for those that don't want to do that (or it's not an option), you might want to look for a FTP/SFTP client that supports concurrent uploads. For the times when I do upload a ton of files, I have the client I use set to 50 concurrent uploads.

 

[Kirby]

Thee are probably infinity +1 ways to extract the ZIP that don't require rm -rf - I for example usually extract to a tmpfs, adjust owner/perms, copy to files to target, umount the tmpfs.

A temporary symlink (assuming there is none in the current directory) would also work

ln -s /target upload && unzip xenforo.zip 'upload/*' && php /target/cmd.php xf:upgrade && rm upload

You could even create an alias or a little shell script to further simplify this.

Really, upload is a non-issue

 

[feldon30]

Really, upload is a non-issue

"There are a million ways to work around this eccentricity."

Once I get this forum upgraded from 1.5 to 2.2 this problem goes away. It's just been odd to see the adherence to a directory structure that makes no sense to me. There's nothing above the /upload directory except a text file telling you to go to the website you literally just downloaded this from.

 

[feldon30]

I think there's never going to be a one-size fits all approach. If you have an upload folder, it's not going to suit some people's way of uploading. If there wasn't an upload folder, it would be problematic for some users for other reasons. As an example, I don't use one-click upgrade (I don't allow my web server to have write permission to normal PHP files for security reasons, and I also have multiple web servers... so letting the web server upload the files is not only a security issue, but you would also be left with only 1 web server with the updated files). I upload the archive to one of the servers, unzip xxx;cp -R zzz;cluster-sync as needed, and having the archive not have a root folder and just spraying files and directories all over the place in whatever directory you are in (I'm not going to put the .zip file in the root of a publicly accessible folder that the web server serves) is just different problems.

I don't upload the individual files from a computer, just the archive, but for those that don't want to do that (or it's not an option), you might want to look for a FTP/SFTP client that supports concurrent uploads. For the times when I do upload a ton of files, I have the client I use set to 50 concurrent uploads.

Interesting. I was trying and failing to think of benefit to the /upload structure.

As for # of downloads/uploads per client, the default setting for proFTPd server is 4 per IP address. I increased it to 6. Maybe I should increase it further? 50 seems extreme though.

 

[digitalpoint]

As for # of downloads/uploads per client, the default setting for proFTPd server is 4 per IP address. I increased it to 6. Maybe I should increase it further? 50 seems extreme though.

Ya, you would need to also check if the server has any limitations on concurrent connections. The servers are use are my own physical hardware with no user accounts other than me, so I didn't set them up to protect against users doing annoying things (like too many concurrent connections). Most servers shared with other users will have some sort of concurrency limits.