XF 1.4 Upgrading Load Balanced Servers

[Bob Hennessey]

We run on a system with 2 load balanced apache servers. Yesterday we upgraded to 1.4.3. I have a few questions about how to deal with this.

1. Run upgrade on each server?
Prior to the upgrade (weeks ago) I had pointed my hosts file to a single server, not the LB. So when I ran the upgrade it apparently only ran on that 1 server. So I had to then point my hosts file to the 2nd server and re-run the upgrade. Once I did that everything was good.

Should this be the normal upgrade process for load balanced servers, to run the upgrades on each server?

2. Notices
A general question about notices - we put a notice up prior to the upgrade to notify users, when we took the notice down it took about 1/2 hour or so before it stopped appearing on all pages. Is this just a cache clear issue and should I rebuild certain caches in this case? I did rebuild forum cache but thread cache could take several hours due to site.

 

[Jake Bunce]

1) You only need to run /install one time to upgrade the database. I assume you have just one database server? Of course both web servers need the new XF files uploaded.

2) The notice should update instantly unless something is out of sync with your two servers. Make sure both library/config.php files are pointing to the same database. If you have a memory cache defined then that should also be the same in the config file. If, for example, both config files are pointing to their own local file system cache then that would be a point of divergence.

 

[Mike]

If you're running multiple web servers, you need to make sure that you maintain consistency in two particular components (outside the database):

1. Caching. If you're using a cache in config.php, it needs to be shared amongst all web nodes. This would mean only memcache and all pulling from a central server (or multiple with sharding). (If you're not using a cache in config.php then the DB is being used.)
2. Files, including the ones XF writes out. This means not only the XF files, but all the data written to data/ or internal_data/ (likely using something like NFS).

The behaviors you've mentioned sound a bit like #1 not being handled.

 

[Bob Hennessey]

I wasn't sure with the version upgrades if it was 100% database upgrades or if anything was done on the file system as part of any upgrades, which would then account for why it needed to run on each server.

The code we deploy to each server is identical, all pointing to the same database, and config files are identical. We did just enable memcache as part of this latest release in shared mode, but I do question how well it gets shared so maybe that would account for the notices not coming down immediately.

We are also using apache which does tend to "flake out" sometimes when we deploy code, I sometimes get sporadic errors or strange behavior from a single site but not all (we're currently setting up our 3rd site on Xenforo). But an apache restart tends to fix these things and could possibly be the cause of the upgrade not seeming like it was run completely. Next time I see something strange like this I'll restart apache and see if it clears up the issue.