My awful, awful, upgrade process

Rigel Kentaurus

Well-known member
Half of this is my fault. But to prevent "accidents", probably the process can be improved.

So, I went to update to beta 3. And.. my method was to ..
1) Rename old folder
2) Move the "upload" folder in place
3) Start moving data, internal_data, config.php and my hacks to the new folder

So.. you might think, why don't I overwrite? One is security, Second is coherence. After countless vb installations where for some random reason only 1 file would not upload, I decided to go the way of extracting the zip in the server and promoting the upload directory to base installation. Also, that way I wipe anything that could be there that is alien.

So .. what happened?

First of all, I didn't shutdown apache (silly me), that meant that the users could still try to enter the site. No bid deal, I thought. I was wrong.

1) As soon as I moved internal_data, somehow install-lock.php dissapeared
2) Since the users were redirected to the upgrade page, after I uploaded the config.php they were given the option to reinstall the forum including wiping the database (because I included config.php)
3) I could not fix it fast ... because I didn't know how to resume the upgrade process. The request index.php?upgrade redirected me to index. Figuring out that the lock disappeared was not fun

I am pretty sure that some people here would like to make this my fault for not uploading/merging contents instead. But I don't like doing that for the reasons explained. And I do have some suggestions:


1) Upgrade should not rely only on internal-lock, if my database is a valid xf one, I have the directories chmoded and in place .. I should be able to resume the upgrade somehow.
2) install-lock is magically removed, why? So, that means that if for some reason I delete config.php (say I rename it, and then rename it back, and that can happen), any user can go to /install and wipe my whole forum
3) Add a "master password" (vb used the customer id) to the install process. At least, if this accident happens the user cannot continue because he does not have the number


Myself .. I have added .htaccess in place, and next time I will shutdown apache, move everything in place and then start it up again.
 
This is how I upgrade,

I close the board and backup the files and the database.

I put the .zip in say beta3/
go to the beta3/ folder and unzip xen*
then I type cp -rf upload/* ../public_html/

And I run upgrade from the browser.

The users get "the board is closed while it's being upgraded"

And 2 seconds later it's all done.

Then I double check everything, including re-doing the style and plugins.
Then I open the board again.

And delete the beta3/ folder. And I keep the backup of beta2/ for a few days.
 
It is your fault when you have an upgrade package to download ... the upgrade package doesn't contain the Data and Data_internal folders or the config file ;)
 
Myself .. I have added .htaccess in place, and next time I will shutdown apache, move everything in place and then start it up again.

.. and merge files and folders you do not need to replace them ... if for some reason merge is not wokring for you contact your host because it's most likely your ftp server is not configured properly -check the permission mask-
 
On an active website, whenever I do an upgrade or complete installation, I setup a portal page first and then lock it with a .htaccess file until I'm ready to go. I also setup the config file locally first thing (I've had a problem that you've had previously and thus is why I do that now).
Most of the stuff that happened is User Error, which really can't be XenForo's fault. However, I do think that they need to have a build-in "master password" like most other boards have.
The main reason why I setup the config file first locally, is if that gets upload during the process (even after they've setup a default one), normally the installation has an error and you'd have to restart. With XenForo, if the database is an active one (meaning that there are accounts on there), it won't let just any user upgrade it. It has to be a SuperAdmin and once the upgrade or installation starts the forum is shutdown.
I wouldn't say that going as far as shutting down apache would be the best thing, but rather redirect all visitors to a portal page which could be on a subdirectory until you get the forum setup and running.
 
The install-lock.php is created during the initial installation of XenForo, and not during an upgrade. When it is not present, an installation will take place when it detects a pending upgrade. For testing, I moved my internal_data folder to various directories on my server, and placed it back in my XF directory. No files went missing, and my site works as it should. I then removed it completely, and the site still works normally (I placed it back for the next upgrade).
 
Although I appreciate the comments (who wouldn't?), the point of the thread is not for asking suggestions about the proper process to do an upgrade, but to help into making the platform more robust as a whole. The points I am trying to make are 2:

1) If a forum is already installed, an upgrade process should be suggested. With my scenario, it was not, only installation was suggested. The forum did detect an existing database but didn't offer an upgrade path, it only offered to completely wipe the database and restart. That is probably not acceptable. The file lock mechanism seems too weak as a detection, and the fact that the forum even auto-deletes that makes it even worse. A second check on the database should be in place.

2) The forum should not be compromised should the admin use a different upgrade path. It doesn't matter how stupid the admin is, the forum software shouldn't facilitate a single user to wipe the complete installation. In my scenario, just because of something unpredictable, the users were able to reach the install screen, and thankfully, I did this really early and had a backup, I do not want to imagine what would have happened if a user just went through.

a) The redirect to /install from home is cute, but too much a security risk
b) The lack of password in the install directory is a no-go, this should be a default, not an extra thing with .htaccess
 
I understand, and agree with, the points you are making Rigel. Things should be set up as robust as possible. I can test some things, and perhaps make other suggestions as well.
 
When I do an upgrade I usually duplicate the forum directory, merge the new files with the duplicate, then swap directory names and run the upgrade. It sounds like you were doing the opposite... merging the old files with the new. It is better to merge new with old. That ensures that all new files exist while still preserving any extra files from hacks. If something goes wrong with the merge (e.g. you mentioned the problem of some files not uploading) then you have the opportunity to sort that out before swapping the directory names.

With that said, some xenForo fixes for your problems might include:

1) Add authentication to the install script.

2) Change the install-lock.php system. I believe I saw Mike say that they are trying to come up with a different system for that.
 
I understand you have suggestions, but could this have been avoided if you read the upgrade instructions?

I read the upgrade instructions, and because I am a power user I chose to go other path. Other people may do that, too. In fact, if you read some of the suggestions given in this thread, they are not following the exact upgrade instructions either.

My impression is that, if you avoid the upgrade instructions, you are expected to do some workarounds and/or encounter some barriers on the way. That is ok.

However, never, for any reason, completely wiping the board should be the consecuence for not following exact instructions. That is an unfair and ridiculous punishment. I am not asking that the developers account for every single scenario, if the user deletes his database himself or replaces instead of merging the install_data folder they are on their own.

This is a tragic scenario.. rename your "config.php" once, and anyone, any user, can wipe all your board.
 
When I do an upgrade I usually duplicate the forum directory, merge the new files with the duplicate, then swap directory names and run the upgrade. It sounds like you were doing the opposite... merging the old files with the new. It is better to merge new with old. That ensures that all new files exist while still preserving any extra files from hacks. If something goes wrong with the merge (e.g. you mentioned the problem of some files not uploading) then you have the opportunity to sort that out before swapping the directory names.

With that said, some xenForo fixes for your problems might include:

1) Add authentication to the install script.

2) Change the install-lock.php system. I believe I saw Mike say that they are trying to come up with a different system for that.

Although I explained that in a simplified way, it is a bit more complicated. I have a folder with all the new and changed files somewhere on my server. Whenever I upgrade the forum I just symlink all the files to the proper places.

That has proven to be really convenient for me because I keep all the personalized files in a centralized location, where I can quickly inspect everything that has been modified. A clean separation between "core" package and "my" files has helped me a lot into customizing and extending the software.
 
Really? If your config.php is not in place, how does anyone know what database to use, or the connection credentials?

Kier, the scenario is like this

1) Rename config.php to something else
2) People are browsing the board, actively, the lock is deleted
3) Rename config.php BACK
4) The board continues working, you don't notice anything wrong
5) Someone goes to install/, since the config.php is there, the database information is available. The user can wipe the forum.


Two scenarios for this.

1) Social engineering. A person convinces somebody to "rename" config.php for a while, and later wipes the forum.
2) Someone, as a workaround for disabling the forum, just renames config.php (thinking that way every user would get an error)
 
I read the upgrade instructions, and because I am a power user I chose to go other path. Other people may do that, too. In fact, if you read some of the suggestions given in this thread, they are not following the exact upgrade instructions either.

If you have downloaded the upgrade package instead of the full install none of this would have happened.
 
Blandt, I downloaded the upgrade package. And this happened with the upgrade package. Please read my full posts for the detail.
I read your post :) I know you are a little angry ... good thing you have backup :) but the upgrade package doesn't contain the Data and the internal_data folders ;) So I'm not sure how you managed to erase the lock file with just the upgrade package :)
 
I read your post :) I know you are a little angry ... good thing you have backup :) but the upgrade package doesn't contain the Data and the internal_data folders ;) So I'm not sure how you managed to erase the lock file with just the upgrade package :)

If you rename or move the config.php, the forum auto-deletes the lock file. The fact that the upgrde package doesn't contain the data and internal_data folders is not relevant at all, I am not even doing the upload-replace thing for the upgrade.

I am in no way angry and I am sorry if my post sound like that. I am trying to give constructive criticism and suggestions to make this product even better.
 
Maybe, if there is an existing XF board in the database, the installer should require the Super Administrator Email and Password be successfully entered before the XF data can be deleted, similar to the upgrade process (although it must be a Super Admin account). That should solve the security issues.(?)
 
Top Bottom