XF 2.0 New XF Resource Standards & your Software Quality Assurance / Test methods?

[Alpha1]

There is a vast difference between developers in regards to the issues in releases. Some developers are really good in releasing solid relatively bug free software, while others need more releases or in some cases even a lot of bug fixes.

Could you please share how you test your software before you release it? What works for you and what does not?
What methods do you use? Are there any Software Quality Assurance protocols that you follow or any standards?
Do you use any software tools? For example performance / stress testing tools to see how a busy big board would handle your software?
How do you account for different browsers? Or for different stacks?

Does your IDE help you to increase quality? What IDE do you use?

Now that the new resource standards require a certain quality and ban certain methods, have you used the opportunity to revisit your Software Quality Assurance methods?

 

[S Thomas]

That sounds like a pretty basic question, yet it's a pretty big one. I will try to cover at least just one aspect.
I can't speak for others, but it might be helpful to go through this:
https://en.wikipedia.org/wiki/Systems_development_life_cycle#Testing

Could you please share how you test your software before you release it? What works for you and what does not?
What methods do you use? Are there any Software Quality Assurance protocols that you follow or any standards?
Do you use any software tools? For example performance / stress testing tools to see how a busy big board would handle your software?
How do you account for different browsers? Or for different stacks?

For XF2, you would mostly just assure that everything's escaped, filtered and using as many resource optimizations as possible. At the end of the day, XF2 is a big framework and most of the work is done by that framework. Try to avoid overriding methods where possible, follow basic rules like MVC, KISS and DRY, extend correctly and you're on the safe side.
All of this works towards one goal: minimize the amount of time spent with testing and minimize the impact on everything else. Because XF2 is already a heavily tested framework, extending that framework saves a lot of time. Extending correctly saves everyone's time.

That being said, the most benefits apart from properly coding come from a proper plan. Preparation is key to success. We used to work with the prototyping workflow where you create a lot of prototypes and integrate them step by step, evolving into an ultimate release candidate. We still use that workflow sometimes, mostly when it's some small work, but working directly with customers can be tedious, so rapid deployment can backfire pretty hard pretty fast.
Our experience is that waterfall typish workflows are way better - you integrate once when you are ready for it. This ensures that everything's working in theory, thus already raising code quality by default. Obviously, the downside is that integration is pushed towards the end, which could be a huge drawback. But as I said, XF2 is a really great framework, so integration should be easy if everyone follows the rules. And yea, obviously, customer feedback can't be considered that often either. So you need to communicate a lot and prepare a lot.

Performance and stress tests should be relevant only when doing actual heavy work, like database optimization, transfering a lot of files, importers and all that stuff. Your casual addon *should have* pretty much no impact on anything. Reason for that is again, we are using XF2. If XF2 does scale with your numbers, then most addons will aswell.
Yes, there is something called micro-optimization where you would hunt for every useless query, misplaced if-clauses etc. Like Xon usually denotes in his addons what the query impact is. But that's really another story.

Considering different systems - we have a strict policy where we officially only support the current generation of browsers + their ESR pendants. We use Blisk and BrowserStack for different environments, and for most systems we also have an actual physical device.

 

[Lukas W.]

Does your IDE help you to increase quality? What IDE do you use?

The right IDE is always an important choice. While Notepad++ will do a solid job at syntax highlighting for you, it's just not enough to ensure quality. PHP errors can be very roughly grouped in two major groups of issues: Runtime issues and syntax issues. A good IDE will smash syntax issues into your face the very moment you type them, and resolve them for you with the click of a button.

IDEs like phpStorm are powerful enough - when used right - to give you hints on methods available in classes you call, tell you the return type of functions and the expected input parameters, etc. You'll rarely find yourself in a situation where your code runs into an unexpected state if you thoroughly use the capabilities of your IDE. Yet this all eats time, and while lots of it can be automated, not all can be. At the end of the day it's a balancing act between not wasting time on stuff that it not necessary yet possible and stuff that actually helps your development process. So if you're prone to making type errors in functions, you want to make sure to use your IDEs capabilities to mitigate that.

At the end of the day we're not writing software for space rockets, and while bugs are always nasty for people that run into it, and we will want to resolve them when they do, and it naturally would've been better not to make them in the first place, no one will die because of them. Software testing is a balancing act between the goals of releasing timely (as we know, time is money) and releasing software that's as free of bugs as it gets. You'll never get all of them, and the larger your user base grows, the more likely they'll pop up.

 

[Alpha1]

@DragonByte Tech @NixFifty @ThemeHouse @truonglv @AndyB What are your thoughts about the above questions and quality control methods?

 

[Sim]

A good IDE really helps with identifying potential bugs - especially typos and such. PhpStorm works really well for XF development.

XF2 is a great framework to develop with - there are quite a few development patterns utilised in the core which we can take advantage of to make our code more reliable and save us a lot of development time. The main examples most devs will be familiar with are the Entity/Finder/Repository framework which are fairly well documented.

But when you also understand the power of Containers (singleton pattern!), SubContainers, Factories, the Registry data store (with rebuilders and decorators!), the SimpleCache, Events, Jobs, Cli tools etc - then you can build functionality quickly which leverages the well tested framework to improve our own code.

Another thing I generally do for my addons is build a small test-harness which I can use to validate the functionality is working as expected - useful for regression testing too. I include it in the addon (in the Admin>Tools UI) for end users to test the functionality as well - especially where I'm integrating external systems via APIs and such.

When I get a chance, I will write up a tutorial on how to build such a test harness in XF2.

Logging is also useful for when you have back-end processing such as Jobs, or external integration - which is why I built the Monolog addon to allow me to provide a central location and API for integrating that functionality across my addons. You can add debug code throughout your addon to check that things work as expected, and then in production logging can be set to ignore those debug messages.