Thursday, July 31, 2014

Quality of software vs. quality of trucks

When company is to choose between two trucks (cheaper and more expensive) to buy, the schema is pretty straightforward. One is cheaper, of lower quality - the other offers quality and reliability. Some companies will choose the cheaper, most will probably go with reliability.

This is easy decision, basic competence of any company owner or manager - ask them about such a choice and not even a second and they start talking.

This is not the case with software

Most people ordering software cannot even understand what a quality and reliability of software is. With trucks you have publicly available reliability reports.
Trucks quality evaluation is a common knowledge. Evaluating reliability of software is area clouded with misconceptions and poor measure ideas.

Site availability in % per month is one of greatest tools to evaluate quality. But it does not say whether a user could even log in into system for the whole time.

You then go to measuring process availability in % per month which this is difficult and needs special agreements between companies.

The deeper you go into trying to understand quality and reliability the bigger obstacles you face.

This is why we stick with known errors or problems

Simple accounting tells us some fixes are too expensive to be made. Too expensive means: it's cheaper to constantly fix bugs due to that problem than fix the problem.

Simple accounting cannot account for software reliability, final user experience and so on.

Possible measures

By investing into Real User Monitoring we could really reach some level of confidence in software quality. Ammount of error statuses per hour gives you pretty good picture about whether processes work or not.

Other ideas:

  1. RUM: ammount of errorpages per hour for logged-in users
  2. window.onerror -> log problem via ajax for reporting and analysis
  3. ammount of errors in system logs (assuming java here, you need to count stacktraces)

Processes for quality

No logging or reporting will improve software if you do nothing with gained knowledge. Logs need to be inspected and bugs need to be reported.
Then those bugs need to be fixed and your client will have to agree to pay for this.

Information about found and fixed bugs need to be avaliable and transparent for both developing company and it's clients.

Third-party testing raids and code inspections should be as common and obvious as asking other doctor for opinion or visiting independent mechanic workshop.
Are we ready for this?

Paying more for better, more reliable software is a difficult choice. Let's not blame managers for their inability to understand our point of view.

Yes, quality is expensive as commonly said. Let's forge this slogan into real procedures and measures.

Maybe one day simple accounting will tell manager: you should pay more, you'll earn more with better software.