The now-retired Space Shuttle had over 400,000 lines of code in 1996. Even with an extraordinary development process and very high stakes, bugs were still part of the Shuttle’s software development. Testing and bug tracking were always part as well, given that those 400,000 lines of code had a 20 year old database to document and keep track of bugs.
Given that software has bugs, who should find them? All developers do this as they write code, as well as when the project is finished. This review is part of checking that the work is finished (with a bit of pride in their work mixed in.) As another review step, project managers and clients will also check the product to ensure it meets their expectations. This type of testing fulfills a particular need—developers have the deepest understanding of their code, while PMs and clients can have a greater understanding of client expectations—but, just as authors have editors and lawyers often don’t represent themselves in court, there is a need for an independent QA review.
The ultimate testers are the end users, who wield tremendous resources (due to their numbers) and amazing coverage. At this point, however, the cost of fixing a bug is a huge multiplier of the cost of fixing the bug during a QA phase. A NIST study quoted that fixing bugs in production can be anywhere from 5.3 to 32 times the cost of repairing the bug during QA (Table 1-5, plotted below.) While these multipliers may be lessened due to the rapid turnaround times of website releases, they still have a potential to eat up an unnecessary chunk of time. The implication, then, is that spending time on QA prior to release will save significant time over the life of the project.
The points above should make it clear that spending proper time on testing is a worthy investment. A dedicated QA process will not only reduce the development time spent on repairing bugs (making for happier clients,) it will increase the quality of the project prior to launch (making everyone happier.)
If you’d like some more details on some ways to start your own QA process, I highly recommend Andrew Riley’s Five Easy Ways to QA Your Site. I’ll also be making future posts, detailing some topics such as wrangling test machines for browser compatibility testing and the benefits of good bug writing—so stay tuned!