Of all the software testing techniques out there, there is one the boss always likes best: how to cut costs without sacrificing quality.
My goal in this tip is to build on an earlier piece, which offered concrete steps any test team can take to move faster without having to buy new tools or make other costly investments.
The advice I offer here is based on research for a book on keeping testing costs down and software quality up.
Here are four new testing techniques to help make that happen. Let’s get down to business.
Tip No. 1: Consider common failures
Mistakes happen. They are part of what makes us human. Trying to prevent every mistake turns out to be horribly expensive and inefficient.
Repeating mistakes over and over again is optional.
The Japanese term for avoiding repeated mistakes is "poka-yoke"; loosely translated, it means "mistake proofing." Cars manufactured by Toyota, for example, use RFID chips combined with a seat-pressure check to prevent drivers from locking their keys in the car.
Mistake proofing applies to software, too. Finding the common defects in software can be as simple as running a query in your bug-tracking system. Once you have a list of the last two hundred bugs, invest an hour or two in looking at each defect to determine the root cause. When a root cause occurs more than once, that presents an opportunity for improvement. (If your team fixes bugs as they occur and doesn’t do bug tracking, you might keep a list for a month, then throw it away.)
One company I worked with that applied this particular software testing technique found a great number of problems with transparent images in older browser versions. To fix the problem, we simply established a policy that all images be in Portable Network Graphics format, instead of the Graphics Interchange Format. With the new policy in place, that defect went from a weekly occurrence to a memory.
And of course, the less time the team spends documenting, fixing and retesting, the faster they deliver the product to market.
Tip No. 2: Automate test below the GUI
This means that the programmers ended up writing high-level functions such as "save profile," "send a signal," "add to cart," "check out" and so on.
When it came to test automation, we focused on writing code to call those high-level functions on every build. The programmers wrote those checks in the same computer language as the source code, which meant they did not have to learn a tool or new techniques.
This changed the tester role from verifying every major input/output combination to verifying the front-end graphics and confirming that the "poke" — the system call into the back end — worked correctly.
Our test process for that feature was reduced from hours to minutes. Creating that application programming interface also allowed the team to reuse the back-end code without having to retest it, making the launch of our client/server application proceed more smoothly than the initial, Web-based project.
Tip No. 3: Reduce reliance on paperwork
Testers spend a lot of time doing things that are not hands on the keyboard, flexing the application. One of those things is documenting: Planning the work, defining it and updating the documents to keep them current.
There are two problems with this. First, every hour spent documenting is an hour spent not testing. That means the documentation slows down the process. Second, documentation that gets out of date must be maintained, and maintenance takes more time.
The combination of these two factors means that comprehensive software testing techniques, where every click is documented in detail, actually slow down the team.
At Agile Testing Days in Germany this October, a consultant named Dawn Haynes suggested something lighter: Just list the features that are going to be tested today. This is very similar advice to what a friend of mine, Rob Sabourin, suggested at about the same time. Sabourin explained that a "test idea" can be expressed in less than two hundred characters — that is, in a sentence or two.
This kind of test document is quick to write, easy to recreate and much less likely to go out of date. The result: more time testing, less time documenting.
Tip No. 4: Don’t script — train your testers
The first question Haynes was asked at Agile Testing Days was how to deal with new testers who aren’t familiar with the system. Instead of solving the problem with documents that tell testers exactly what to do, Haynes suggested training them to turn them into subject matter experts. One way to do this is to pair the tester up with a more experienced tester, who can demonstrate some ways to run "test undo/redo" into specific, actionable steps.
Putting it all together
For years, we’ve been told, "Fast, cheap, good; pick any two," and that certainly makes sense — unless you have some way of building the skills of your team; skills make the key difference.
A skilled team will look back at what goes wrong often and work to prevent it in the future. These teams will look at common repeated activities that can be automated and automate those steps. They will spend less time documenting and more time testing, which means the team will get the software to market faster.