Belgium Testing Days focused on this theme: “QA versus Testing: Antagonism or Symbiosis?” Nathalie Rooseboom de Vries and Daniel Maslyn, wearing eye-catching outfits that symbolized both sides of the debate, charged into the opening session resulting in a noisy “argument.” Throughout the conference, Daniel and Nathalie circulated around the conference asking delegates to write their opinions on iPads, and these were beamed directly to a giant monitor in the large common area, generating more informal discussions.
The “QA vs. Testing” question wove through many different sessions. In this two-part series, we look at highlights from the sessions I attended. Whether your role is labeled as QA or software test, manager or engineer, thought-provoking questions and discussions help in continual learning with a focus on providing quality.
Serendipity in testing
Ever found a serious bug by accident? According to Elalami Lafkih, we can make our own luck. Serendipity in testing is a combination of curiosity, sagacity and perseverance. Elalami advised us to learn all we can about our product and customers, so we can take advantage of “lucky” discoveries. Leverage the work of others and exploit knowledge gained from one problem to find more. For example, mine system logs to see how exceptions are correlated to user actions, and use this knowledge to design repeatable scenario tests.
Perseverance, combined with diverse skills and viewpoints, helps us uncover issues. Like a team of archeologists learning about an ancient culture, we may find something totally different than what we thought we were hunting. Serendipitous testing is about feedback and continuous learning. Subsequent sessions built on this theme, contrasting this type of testing with QA.
Regulatory requirements, QA and testing
Jean-Paul Varwijk’s topic, “Regulations: Where Quality Assurance Meets Testing” fit right in with the conference theme. Jean-Paul explained that QA designs standards, procedures and controls to measure adherence to standards via regular checks. In Jean-Paul’s view, testing aligns with QA, but is more about providing information to someone who matters about the product or service in a certain context.
The purpose of testing is not to fulfill a process, it’s to supply info by critically thinking and investigating in order to defend value for its clients. This represents antagonism between QA and testing. But there is a symbiosis as well, if testing goes beyond just checking off the deliverables to provide valuable information.
Jean-Paul pointed out that we don’t need fixed processes to do good testing, we need the right information so we can tell our testing story. When an auditor requires metrics to see if some standard was met, dig into what the auditor really wants to know. It’s an opportunity for deeper learning.
Feedback is what matters
Johanna Rothman noted that defects are a lagging indicator. Test and dev should partner up, and use multi-dimensional product data to gauge quality. We need to ask whether our work provides feedback to the business for product and release decisions, and whether our development teams get enough feedback about our work to be able to improve it.
QA departments endeavor to put processes in place and police them, but it works better if process improvement is done by the people doing the work. What we are called sets expectations up and down the organization. Johanna pointed out that product quality is a business decision, with lots of factors, such as market position and product complexity. Johanna presented this challenge: Given the fact that owning quality starts with the business executives, what should we call ourselves to reflect what we can do to improve product quality?
Losing the “Ifs”
In a lightning talk, Lee Copeland used the old pot roast story to open our eyes about some testing “truths.” Mom cuts the ends off the pot roast before cooking it, because that’s what her mom always did. Come to find out, her mom had to cut the ends off the roast only because she didn’t own a big enough roasting pan.
The testing (and/or QA) profession has plenty of pot roast stories. We’ve lost the “Ifs” that went with the “Thens.” “IF the code is difficult to change, THEN we must have extensive documentation.” “IF we have inexperienced testers, THEN we must have detailed test cases.” If we focus on the practice and not the purpose behind it, we end up with a lot of meaningless work.
In part two of this conference summary, we’ll hear more about the QA versus test debate and what the industry notables had to say in sessions at Belgium Testing Days 2012.