Daily Archives: 02/05/2012

Belgium Testing Days conference: The purpose of software testing

Belgium Testing Days focused on this theme: “QA versus Testing: Antagonism or Symbiosis?” Nathalie Rooseboom de Vries and Daniel Maslyn, wearing eye-catching outfits that symbolized both sides of the debate, charged into the opening session resulting in a noisy “argument.” Throughout the conference, Daniel and Nathalie circulated around the conference asking delegates to write their opinions on iPads, and these were beamed directly to a giant monitor in the large common area, generating more informal discussions.

The “QA vs. Testing” question wove through many different sessions. In this two-part series, we look at highlights from the sessions I attended. Whether your role is labeled as QA or software test, manager or engineer, thought-provoking questions and discussions help in continual learning with a focus on providing quality.

Serendipity in testing

Ever found a serious bug by accident? According to Elalami Lafkih, we can make our own luck. Serendipity in testing is a combination of curiosity, sagacity and perseverance. Elalami advised us to learn all we can about our product and customers, so we can take advantage of “lucky” discoveries. Leverage the work of others and exploit knowledge gained from one problem to find more. For example, mine system logs to see how exceptions are correlated to user actions, and use this knowledge to design repeatable scenario tests.

Perseverance, combined with diverse skills and viewpoints, helps us uncover issues. Like a team of archeologists learning about an ancient culture, we may find something totally different than what we thought we were hunting. Serendipitous testing is about feedback and continuous learning. Subsequent sessions built on this theme, contrasting this type of testing with QA.

Regulatory requirements, QA and testing

Jean-Paul Varwijk’s topic, “Regulations: Where Quality Assurance Meets Testing” fit right in with the conference theme. Jean-Paul explained that QA designs standards, procedures and controls to measure adherence to standards via regular checks. In Jean-Paul’s view, testing aligns with QA, but is more about providing information to someone who matters about the product or service in a certain context.

The purpose of testing is not to fulfill a process, it’s to supply info by critically thinking and investigating in order to defend value for its clients. This represents antagonism between QA and testing. But there is a symbiosis as well, if testing goes beyond just checking off the deliverables to provide valuable information.

Jean-Paul pointed out that we don’t need fixed processes to do good testing, we need the right information so we can tell our testing story. When an auditor requires metrics to see if some standard was met, dig into what the auditor really wants to know. It’s an opportunity for deeper learning.

Feedback is what matters

Johanna Rothman noted that defects are a lagging indicator. Test and dev should partner up, and use multi-dimensional product data to gauge quality. We need to ask whether our work provides feedback to the business for product and release decisions, and whether our development teams get enough feedback about our work to be able to improve it.

QA departments endeavor to put processes in place and police them, but it works better if process improvement is done by the people doing the work. What we are called sets expectations up and down the organization. Johanna pointed out that product quality is a business decision, with lots of factors, such as market position and product complexity. Johanna presented this challenge: Given the fact that owning quality starts with the business executives, what should we call ourselves to reflect what we can do to improve product quality?

Losing the “Ifs”

In a lightning talk, Lee Copeland used the old pot roast story to open our eyes about some testing “truths.” Mom cuts the ends off the pot roast before cooking it, because that’s what her mom always did. Come to find out, her mom had to cut the ends off the roast only because she didn’t own a big enough roasting pan.

The testing (and/or QA) profession has plenty of pot roast stories. We’ve lost the “Ifs” that went with the “Thens.” “IF the code is difficult to change, THEN we must have extensive documentation.” “IF we have inexperienced testers, THEN we must have detailed test cases.” If we focus on the practice and not the purpose behind it, we end up with a lot of meaningless work.

In part two of this conference summary, we’ll hear more about the QA versus test debate and what the industry notables had to say in sessions at Belgium Testing Days 2012.

Source:

http://searchsoftwarequality.techtarget.com/news/2240149290/Belgium-Testing-Days-conference-The-purpose-of-software-testing

Did you like this? Share it:

Quality assurance vs. testing the focus of Belgium Testing Days

“QA versus Testing: Antagonism or Symbiosis?” was the theme for the 2012 Belgium Testing Days Conference.  QA and software test have often been used synonymously, but there are some subtle and not-so-subtle differences, depending on who you talk to.  In this second part of a two-part series, I continue with a summary of many of the conference sessions, starting with keynote from industry leader and SSQ contributor Karen Johnson.

Is it important what I’m called?

The “QA vs. Test” debate was a force in Karen Johnson’s keynote. Karen asked us to keep an open mind and look at different viewpoints of one’s job title. A job title can help define your abilities to others, it can affect your salary, or give you access to influential people.

On the other side of the coin, influence is earned, not assigned via job title. Titles may contribute to pigeonholing us in one position, and they don’t measure career accomplishments. Karen quoted many software professionals talking about why they care or don’t care what they are called. Do you want to be a “Senior QA Engineer” or a “Tester”? In the end, the professional contributions we leave behind may be more important than our job titles.

Solving large-scale testing and QA problems

One of the most valuable sessions I attended was a workshop on large scale Agile testing and QA, facilitated by Kirsi Korhonen and Eveliina Vuolli. Kirsi and Eveliina asked each table group to come up with our biggest challenges in large-scale agile projects. We wrote sticky notes to post on the wall and then voted on which topic to tackle first.

I asked for help with a colleague’s problem: If you’re given a huge, traditional requirements document, how do you engage stakeholders in slicing those up into small user stories, and prioritizing which to do first? Johanna Rothman, who was in my table group, suggested asking the stakeholders, “What do you want to see in the first demo?” I left the workshop with several experiments for my own team to try.

What’s in a name?

Stefaan Luckermaan explored the “what’s in a name” quandary generated by the “Test vs. QA” debate. He used the metaphor of a bowler hat, which was originally designed to protect gamekeepers’ heads, as it was shorter than a top hat and let them slide under low-hanging branches on horseback. Just as we have different names for software quality and testing, bowler hats are called by other names, such as “derbies.” Different people and characters have used bowlers for different reasons: a trademark, a status symbol, part of a uniform, comedy, an artistic statement.

Like a particular style of hat, quality doesn’t fit one stereotype. Quality is the result of a carefully constructed cultural environment. It has to be the fabric of the organization, not just part of the fabric. Stefaan suggests creating a Quality Mission Statement that is similar to a balanced scorecard – quality according to us, according to our customer, the benefits, the results of our work, a learning process. We must keep experimenting, and get out of our comfort zone, to grow our skills and improve quality.

One project, two perspectives

Adrian Rapan and Tony Bruce shared lessons learned from an ill-fated project where cultures clashed. Permanent employees of the organization didn’t want to change, so contractors’ attempts to improve quality met resistance. Different teams working on different parts of the same system didn’t talk, and they also didn’t talk to customers, citing that they couldn’t afford the time. Long feedback cycles left testers “in the dark” and resulted in last-minute requirement changes.

Tony and Adrian had to find new ways to learn business needs. One lesson learned was the importance of making requirements visible to the whole team in order to get feedback on potential gaps. One way to accomplish this is to find a format that’s accessible to everyone. Automated regression testing provided “living documentation” as well as coverage metrics.

Their experience showed that the purpose of testing is to provide information, and the purpose of QA is to use that information as part of the effort to optimize processes to improve quality. The problems Tony and Adrian described are common, and one important step they took was to understand the conflicting cultures and find ways to work around resistance to change.     

Both testing and QA are pro-quality

In Sigge Birgisson’s view, testing is product-centric; QA is process-centric, but both are pro-quality. For Sigge, QA in a product context is checking, a verification activity. Testing moves the project forward.

Sigge recounted his experiences working in a Waterfall project. Their manager pressured them to update documentation “for the future,” and provide metrics such as the number of tests completed. Sigge’s team felt that adding business value “now” was more important, so they intentionally incurred “document debt” by delaying documentation updates. They turned to the overwhelming number of defect reports, analyzed bugs from a business perspective, and told the business what was and was not working. Business experts could then identify what really mattered. In the end, they went beyond both testing and QA, to focus on delivering the value needed.

Focus on outcomes

Are we doing QA or testing? After the excellent discussions at Belgium Testing Days, my own conclusion is that we should focus on whether the features we deliver are providing the expected value to the business. A continual feedback and learning loop will help us stay on track to ever-improving quality.

Source:

http://searchsoftwarequality.techtarget.com/news/2240149291/Quality-assurance-vs-testing-the-focus-of-Belgium-Testing-Days

Did you like this? Share it:

5 Myths of Software Testing

As I scan the software testing stories of the day, I’m amazed at the frequency of certain misconceptions. While there are too many to list, I wanted to share five of the most common testing myths (in my brief experience). The first three I find to be prevalent in mainstream news articles, while the other two are more common within the tech industry in general.

Take a look and see if you agree with me.

Myth 1. Testing is boring: It’s been said that “Testing is like sex. If it’s not fun, then you’re doing it wrong.” The myth of testing as a monotonous, boring activity is seen frequently in mainstream media articles, which regard testers as the assembly line workers of the software business. In reality, testing presents new and exciting challenges every day. Here’s a nice quote from Michael Bolton that pretty much sums it up:

“Testing is something that we do with the motivation of finding new information.  Testing is a process of exploration, discovery, investigation, and learning.  When we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated, we’re testing.  We’re testing when we’re trying to find out about the extents and limitations of the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before.”

Myth 2. Testing is easy: It’s often assumed testing cannot be that difficult, since everyday users find bugs all the time. In truth, testing is a very complex craft that’s not suited for your average Joe. Here’s Google’s Patrick Copeland on the qualities of a great tester:

“It’s a mindset and a passion. From the 100s of interviews I’ve done,  “great” boils down to: 1) a special predisposition to finding problems  and 2) a passion for testing to go along with that predisposition. In  other words, they love testing and they are good at it. They also appreciate that the challenges of testing are, more often than not,  equal or greater than the challenges of programming. A great “career”  tester with the testing gene and the right attitude will always be able  to find a job. They are gold.”

Myth 3. Testers only find bugs: Yes, testers do find bugs, but that’s not their sole purpose. Here’s a good summary on this myth from Ankur of freesoftwaretesting.info:

This view of the tester’s role is very limited and adds no value for the customer. Testers are experts with the system, application, or product under test. Unlike the developers, who are responsible for a specific function or component, the tester understands how the system works as a whole to accomplish customer goals. Testers understand the value added by the product, the impact of the environment on the product’s efficiency, and the best ways to get the most out of the product.

Myth 4. Machines will make human testers obsolete: With advances in automated technology, it’s often assumed  that computers will someday render human testers obsolete. But since the ultimate users of an application are not robots or machines, but rather live human beings, it stands to reason that human testing will always have an important role to play. Here’s testing author James Whittaker on the importance of manual testing:

“Test automation is often built to solve too big a problem. This broad scope makes automation brittle and flaky because it’s trying to do too much. There are certain things that automation is good at and certain things humans are good at and it seems to me a hybrid approach is better. What I want is automation that makes my job as a human easier. Automation is good at analyzing data and noticing patterns. It is not good at determining relevance and making judgment calls. Fortunately humans excel at judgment.”

Myth 5. Testers don’t get along with developers: It’s not hard to see why this myth persists. As testing guru James Bach once wrote: “Anyone who creates a piece of work and submits it for judgment is going to feel judged. That’s not a pleasant feeling. And the problem is compounded by testers who glibly declare that this or that little nit or nat is a “defect,” as if anything they personally don’t like is a quality problem for everybody.”

What’s not widely known is that many testers are actually former developers (and vice-versa), so there’s a mutual understanding and appreciation for the challenges each camp faces. This is not the case inside all companies, but to say that the majority of testers and developers don’t get along is false, in our experience.

What are some other myths of testing? Let us know in the comment section.

Source:

http://bostinno.com/channels/5-myths-of-software-testing/

Did you like this? Share it: