Just to add, I do read the release notes, I do follow proper procedures, and I've never had an issue. With add-ons or xf release. Other than a theme or 5 during the responsive upgrade...everything had went as it should or as I expected.
I also upgrade almost immediately after the new version is out. I just have a quick look to see if any addons are replaced by core functionality so I can kill them, otherwise I just upgrade. Never had an issue so far.
I rarely upgrade - it's simply a matter of my community being more important than the latest gadgets...and any upgrade takes some time and there is some risk involved. In reality, that means I usually upgrade once a year or so....
I suspect they jump to the latest release because these developers are top-notch coders from proven past history. Read up on these developers. They are a family, and somehow I feel as an extension to their family too.
Interesting because that was not my experience at all.. 1.2 was a nightmare that took weeks to work through all the various broken plugins where the latest update from 1.2 to 1.3 nothing broke on my forum..
I'm about to finally upgrade from 1.3 to 1.4.3 and have just gone through all of my add-ons one by one to check compatibility. Looks like there isn't a huge change from 1.3 to 1.4 that will break my add-ons. I'm also updating my theme, which will create more work than anything. Anyone have good threads describing tips and a check list to follow when running through the XF upgrade process?
For larger upgrades I usually create a guide.
The 1.4 upgrade doesn't require any specific instructions though; essentially just ensure any third party add-ons and styles are compatible and updated as necessary.
Then set up any new features and functionality as required.
It all depends on what the cost of failure is: risk v. reward, and how you make decisions. If you are one man/woman who doesn't stand to lose much if your site is screwed up for a few days and you can make decisions on-the-fly then you will be less conservative than an organization with a set process for managing the forums or where having the forums screwed up for a few days will cause a major disruption.
Arbitrary and manual "testing" doesn't change the risk, only the appearance of risk in the same way buying 2 lottery tickets doesn't double you odds of winning. You could go for a week or more before discovering one of the hundreds of features is critically broken, far too long to simply rollback to upgrade. The only way to actually mitigate risk is comprehensive unit and functional tests. If we had those we could validate the complete application stack before deploying to production rather than simply waiting until someone (probably a customer) discovers that something isn't working right.
XF2 must have a complete TDD test suite. It's the single biggest problem with using XF as a business tool. Right now I need to manually code review all the diffs but that doesn't mean it works. If XF pushes out a new version I need to know, through proof, that subscription renewals still process correctly and I'm not all of a sudden giving away free subscription renewals because someone accidentally wrote a>b instead of a<b or if ("foo" == 0) instead of if ("foo" === 0).
I try to wait for a few days before upgrading just in case. If I'm in a hurry or if the update is critical, the only thing I check is if the theme I use is compatible with the latest version.