Tag Archives: quality

Software testing techniques: Four ways to cut costs

Of all the software testing techniques out there, there is one the boss always likes best: how to cut costs without sacrificing quality.

My goal in this tip is to build on an earlier piece, which offered concrete steps any test team can take to move faster without having to buy new tools or make other costly investments.

The advice I offer here is based on research for a book on keeping testing costs down and software quality up.

Here are four new testing techniques to help make that happen. Let’s get down to business.

Tip No. 1: Consider common failures
Mistakes happen. They are part of what makes us human. Trying to prevent every mistake turns out to be horribly expensive and inefficient.

Repeating mistakes over and over again is optional.

The Japanese term for avoiding repeated mistakes is "poka-yoke"; loosely translated, it means "mistake proofing." Cars manufactured by Toyota, for example, use RFID chips combined with a seat-pressure check to prevent drivers from locking their keys in the car.

Mistake proofing applies to software, too. Finding the common defects in software can be as simple as running a query in your bug-tracking system. Once you have a list of the last two hundred bugs, invest an hour or two in looking at each defect to determine the root cause. When a root cause occurs more than once, that presents an opportunity for improvement. (If your team fixes bugs as they occur and doesn’t do bug tracking, you might keep a list for a month, then throw it away.)

One company I worked with that applied this particular software testing technique found a great number of problems with transparent images in older browser versions. To fix the problem, we simply established a policy that all images be in Portable Network Graphics format, instead of the Graphics Interchange Format. With the new policy in place, that defect went from a weekly occurrence to a memory.

And of course, the less time the team spends documenting, fixing and retesting, the faster they deliver the product to market.

Tip No. 2: Automate test below the GUI
A few years ago, I worked with a team that used good design principles, separating the graphics from the business logic. This particular piece of code covered what happens after the user clicked the Submit button. (In HTML and JavaScript, you can only click Submit if all values are filled in.)

This means that the programmers ended up writing high-level functions such as "save profile," "send a signal," "add to cart," "check out" and so on.

When it came to test automation, we focused on writing code to call those high-level functions on every build. The programmers wrote those checks in the same computer language as the source code, which meant they did not have to learn a tool or new techniques.

This changed the tester role from verifying every major input/output combination to verifying the front-end graphics and confirming that the "poke" — the system call into the back end — worked correctly.

Our test process for that feature was reduced from hours to minutes. Creating that application programming interface also allowed the team to reuse the back-end code without having to retest it, making the launch of our client/server application proceed more smoothly than the initial, Web-based project.

Tip No. 3: Reduce reliance on paperwork
Testers spend a lot of time doing things that are not hands on the keyboard, flexing the application. One of those things is documenting: Planning the work, defining it and updating the documents to keep them current.

There are two problems with this. First, every hour spent documenting is an hour spent not testing. That means the documentation slows down the process. Second, documentation that gets out of date must be maintained, and maintenance takes more time.

The combination of these two factors means that comprehensive software testing techniques, where every click is documented in detail, actually slow down the team.

At Agile Testing Days in Germany this October, a consultant named Dawn Haynes suggested something lighter: Just list the features that are going to be tested today. This is very similar advice to what a friend of mine, Rob Sabourin, suggested at about the same time. Sabourin explained that a "test idea" can be expressed in less than two hundred characters — that is, in a sentence or two.

This kind of test document is quick to write, easy to recreate and much less likely to go out of date. The result: more time testing, less time documenting.

Tip No. 4: Don’t script — train your testers
The first question Haynes was asked at Agile Testing Days was how to deal with new testers who aren’t familiar with the system. Instead of solving the problem with documents that tell testers exactly what to do, Haynes suggested training them to turn them into subject matter experts. One way to do this is to pair the tester up with a more experienced tester, who can demonstrate some ways to run "test undo/redo" into specific, actionable steps.

Putting it all together
For years, we’ve been told, "Fast, cheap, good; pick any two," and that certainly makes sense — unless you have some way of building the skills of your team; skills make the key difference.

A skilled team will look back at what goes wrong often and work to prevent it in the future. These teams will look at common repeated activities that can be automated and automate those steps. They will spend less time documenting and more time testing, which means the team will get the software to market faster.

Source: http://searchsoftwarequality.techtarget.com/news/2240181160/Software-testing-techniques-Four-ways-to-cut-costs

Did you like this? Share it:

Whatever happened to the art of software testing?

Over the last year I’ve had the opportunity to attend a number of extremely interesting and mind-expanding conferences focusing on emerging and somewhat disruptive technologies and companies: APIs, mobile, cloud, big data – the works. Coming from a quality background, it has stricken me how little focus these companies give to testing. They talk plenty about continuous integration, agile methodologies, user groups and continuous deployment – but testing? Nope.

Why is that?

First, let me elaborate a little on what I mean (and don’t mean) by "testing." I don’t mean unit tests. I don’t mean BDD or TDD based on some cool-named framework using a custom DSL to describe scenarios. I don’t mean recorded and automated scripts of your website or application. Much of this is being done by many of these companies – which is great, and I’m positive it increases the quality of their products.

What I do mean with “testing” is testers who try to break stuff; who do those things with your software that users weren’t intended to do; who provoke the hardware hosting your application to behave like it usually doesn’t; and who continuously challenge the design and architecture of your products by thinking outside the box and applying this in a methodological and structured way. When it comes to quality, these testers will be your greatest pain and your biggest gain. They take the quality of your products to the next level by doing all that crazy stuff your users do (and don’t do), giving your team the opportunity to fix them first.

So, back to the question: why is it that these oh-so-crucial testers and the art of testing are so absent from these companies and conferences?

I have three possible explanations in my mind:

Strike One: Developers aren’t testers

Developers – I love you – but you aren’t testers. A tester’s mentality and talent when it comes to quality is to find defects and ultimately break stuff. Developers, on the other hand, want to make sure things work. This might sound like a small difference but the implications are huge. The “developer” of a steering wheel will make sure it turns the car left or right when you turn the wheel left or right (at the required rate/degree/whatever). The tester, on the other hand, will jerk the wheel back and forth to bring it out of balance, will submit the wheel to extreme (but plausible) conditions at the north pole and in the Sahara desert, and he might even compare the wheel to the competitors and tell you why theirs is better. Developers confirm, testers overhaul. That’s just how it is. Unfortunately, though, developers are usually at center stage in a development team, and often lack the insight into both the craft of testing and the time it takes to do it right. You need both on your team for your quality to be top-notch – neither can stand in for the other.

Strike Two: Agile Testing = Automated Testing

Don’t get me wrong – agile development can be fantastic when executed correctly, and has surely improved the lives of many a developer/tester/product-owner out there. Unfortunately, though, agile teams (at least in their infancy) often put testing efforts into the hands of developers (see point 1), who often believe that you either need to be able to automate all your tests or that a BDD/TDD specification is a valid substitute for testing. Neither is correct. Using a BDD/TDD specification as a test is just another way of checking that your software performs as required/designed/specified. And, as already argued above, exploratory testing is key to finding those out-of-bounds conditions and usage-scenarios that need to be fixed before users encounter them.

Strike Three: Cheap-skating quality

OK – you’ve convinced your agile team they need to do exploratory testing during their sprints, and your developers have reluctantly agreed that they aren’t testers at heart. So what happens when you approach the management team with a request to hire an expert tester?! Hands in the air if you think they might answer something like:

+ "We have a deadline – we need to release – we’ll invest in development now and testers later."
+ "Don’t our developers have 90% code coverage? Do we really need testers?"
+ "Our users will help us iron out those out-of-bounds issues and quirks. That will be ample feedback for future improvements."
+ <any other “explanation” that is based on the reluctance to spend money on quality>

No one raised their hands? Phew, that’s a relief! Otherwise, given the already stated arguments, this is an obvious and probably the most common mistake. Your story-telling talents will be put to test. Hopefully you can convince management to make the investment.

What to learn from this mini-rant? To put it simply:

+ Understand that testing, just like development, is a craft of its own
+ Cherish your testers and their expertise
+ Invest in quality – your users will love you for it.

Source: http://www.networkworld.com/community/node/82273

Did you like this? Share it:

The next generation software testing

In today’s IT World finding smarter ways to test is a continuous goal for many software testers, whether it is implementation of agile testing, improved test design techniques, exploratory testing. This event explores the most effective ways of scaling “smarter testing” to benefit the whole team and the organisation.

Benefits of Attending
- Hear best practices and case studies
- Learn from experts who are passionate about software testing
- Network with your peers
- Find out about new software testing tools and services
- Take part in interactive panels and get information fit for your purposes
- Leave with fresh ideas on how to approach your software testing challenges.

Who Should Attend

Business Managers
CEOs
CIOs
CTOs
Heads of Software Engineering
IT Directors
IT Managers
Programme Managers
Project Managers
Quality Directors and Managers
Software Customers and Users
Software Engineering Managers
Software Engineers and Developers
Software Managers
Software Process Improvement Managers
Software Test Managers
Software Testers

Agenda of the Seminar

Few of the themes that would be covered, includes:

- Agile Project Scaling
- Assuring quality in a virtualised operating environment
- Automating Requirements and Risk Based Testing
- Behaviour Driven Development (BDD)
- Building a QA Team for the Agile Age
- Cloud Testing
- Coping with complex systems
- Expressing Software Testing in terms of Business Outcomes
- Improved Software Test Design
- Moving to Agile from traditional projects
- New approaches and ideas for managing software testing both traditional and Agile projects
- Quality Management in an Agile Context
- Radically disruptive approaches to Software Testing
- Software Test Automation Strategies

Source: http://apnaindia.com/classified/the-next-generation-software-testing-50090.html

Did you like this? Share it:

Unit Testing is a Means to an End

Most professional software developers these days understand the importance and value of writing and using unit tests. A nice summary of some of the oft-touted and oft-realized benefits of unit testing can be found in the StackOverflow.com thread Is Unit Testing worth the effort? [my only very minor criticism is the mixing of more specialized Test-Driven Development (TDD) with the more general unit test concept]. As with most good things, however, even unit testing enthusiasm can go too far. The benefits of unit testing can lead to overly enthusiastic unit testing developers forgetting that unit tests are not the end themselves, but rather are a means to the real end.

The "end" that most software developers are striving for is delivery of software solutions that make their users’ lives easier and more productive. Unit tests can be extremely valuable in obtaining this end and certainly add to software quality, but the overly zealous unit tester must beware of allowing the unit tests themselves to displace this end goal. It’s all too easy to allow oneself to get so bound up in writing exhaustive and "perfect" unit tests that one puts the true end goal at risk. In the remainder of this post, I look at ways in which developers can allow unit testing to move from helping achieve the desired end to unintentionally displacing the real end and putting it at risk.

Overreaching Unit Testing

Steven Sanderson has written "the benefit of unit testing is correlated with the non-obviousness of the code under test." I largely agree with this sentiment as a general guideline. I see little value in unit testing trivial "get" and "set" methods. Some methods are more readily evaluated via code review than via unit test.

The concept of code coverage can be a useful one as long as it’s not taken too far. Code coverage appears to provide high return for the effort for a while, but there comes a point of diminishing returns when gaining additional code coverage comes at much greater cost and may not be worth that cost. It’s also important to recognize that even the often highly expensive 100% code coverage typically means only all lines of code were executed and does not check all possible paths through the code.

All Code’s Unit Testability is Not Equal

The post Selective Unit Testing – Costs and Benefits clearly articulates well the differences in difficulty (cost) and advantages (benefits) of unit testing of different types of code. In cases where the advantages/benefits of a unit test are high and the cost/effort is low, the value of unit testing is obvious. On the opposite extreme, there are types of code that receive little benefit from unit testing.

Source: http://www.javaworld.com/community/?q=node/8354

Did you like this? Share it:

Software Testing Tools to Test Cloud Computing Applications

With its dynamic scalablity, flexibility and virtualized resources are provided as a service, Cloud Computing is seen as the dawn of a new era for application services and has received its fair share of plaudits.  With Google Documents, Flickr, Buzzword and Zoho as examples of general purpose applications that use Cloud Computing Technology it is only a matter of time just before Cloud Computing is noticed as the most viable alternative for application development and deployment.

With IT Giants such as Microsoft, Google and Amazon all vying for a position inside the Cloud Computing space you’d anticipate there to be a plethora of Cloud based Software program Applications  at present available.  Even although this is certain to be the case in the near future, at present Software program Testing appears to be the present favoured use of Cloud environments.

A recent survey by Evans Information, an independent investigation firm that conducts periodic surveys of developers , discovered that of those using cloud facilities to run applications, 49.8% said they were performing so experimentally or for prototyping 28.6% for non-essential business applications and 21.6% for organization crucial applications.  They see Cloud environments as being “good for testing simply because they can be set and torn down quickly, occasionally at less expense than on-premise facilities”. 

The question to answer then is what Software testing tools are available to aid developers and Quality Assurance people in their application development and testing procedures.  Software program Testing tools that are used for testing of conventional applications are of small use when applied to Cloud Testing as there is a need for tools to allow Software program developers and Tester to analyse the network, desktop and implications of adjustments within the Cloud.

A growing selection of Cloud based Open Source Software Testing Tools are being published.

Cloud Tools for example is a set of tools for deploying, managing and testing Java EE applications on Amazon’s Elastic Computing Cloud (EC2). Containing 3 primary parts, which consists of machine images that can be configured to run on Tomcat and Maven &amp Grails plug-in this is an remarkable tool to use for Open source cloud software program testing.

PushToTest TestMaker is a distributed test environment that can run tests on test equipment, or in a Cloud Computing environment.  It introducing particular commands to support automatic Cloud Testing services.

Cloud Tools and PushToTest Test Maker represent examples of products that will support shape the future of robust Cloud based Software program Testing Applications.  Although the technologies is in its infancy, a number of testing tools are emerging that can supply help in cloud based software testing.

Source: http://www.technethuancavelica.com/software-testing-tools-to-test-cloud-computing-applications.htm

Did you like this? Share it:

Heavy Load Testing

Visual Studio Team Agents 2010 consists of Test Load Agent, Test Load Controller and Lab Agent. Team Test Load Agent monitors a request for a new test from the controller by running a service locally. Team Test Load Controller is run by a service, and the service controls the test agents and reports the status and errors of the test. It can also transfer the sources into running test and data collection in logic, or has effect on the system running the testing environment again.

The main function of Test Load Agent is to cascade various machines to produce heavier load and higher stress when doing the load test. It is rarely seen that an average level PC can produce a load of 50 people, which still cannot compare with some great websites or systems with thousands or more people online. Thus, enough load and stress can be produced when we connect several server level machines, so that the results will be meaningful to us.

More than 3 computers are required when using Load Agent, and VS 2010 For Tester, Controller, Agent1, Agent2, etc. are supposed to be installed. If the number of computers is limited, you can install the Controller and Agent in the same computer, but the test will be limited, as the picture below shows.

clip_image002

As the picture above shows that, Team Agents include the Agent and Controller software, which are highly extendable and customizable, so that the testers enjoy great flexibility. Testers can do load testing directing at Web apps and organizations to improve service quality by measuring the performance of Web apps and servers under load more accurately.

Application & Practice: to modify and test x64 bits CLR program set with VS 2010

Visual Studio Team Test 2010 Load Agent testing platform provides host process for the test. You can enjoy the new feature that network emulation has been introduced for performance testing, and what’s more, Visual Studio 2010 has developed another feature, which is that it can test the program set in 64bits CLR. With Visual Studio 2008, the host process can only be tested in the 32bits mode, and the internal storage has been limited to 2GB. But with Visual Studio Team Test 2010, there is no such limit.

Here are the steps to modify and test x64 bits CLR program set with VS 2010.

1) Set your program set as opening your program set on “Any CPU” platform, and open the “Build” tab, then set your platform target as “Any CPU”.

clip_image004

2) Open test settings, and set the host process platform as “MSIL”. Open the “Local.testsettings” to set and then choose “Hosts” and at last choose the MSIL under the Host Process Platform options.

clip_image006

Tips:

If you have not installed Test Load Agent, you will find that only one CPU will be occupied by the local requests when doing performance test with VS 2010. It is because the process to generate stress is VSTESTHost.exe. Accordingly, if you want every CPU of a multinuclear CPU to produce stress, you have to use Visual Studio Team System Test Load Agent.

Did you like this? Share it:

Seven Tips about Software Testing

The software is to control people’s thoughts, but people’s thoughts cannot be controlled, which determines that software with no bug is impossible. “The testing will end anyway” is the principle law in the testing circle, which means the testing should be moderate, without doing too much testing regardless of the cost and dreaming of the perfect software with no bug. I was inspired a lot by several experts in this industry by interviewing them, and I found seven tips about software testing.

1. The goal of software testing should be higher, not just for testing, but for improving the quality of software. The job of testers is not just to find out bugs, but to create higher-quality software cooperating with software developers and servicers.

2. For every executor of any kind, software testing requires weighing the input and the output, and the cost and the benefit. Thus, the company should decide how to build the test team, how to build the test environment, how to choose test tools, how to manage the test process and whether to do out sourcing according to its own demands.

3. The software development and testing is a unity of opposites, and the two are not supposed to separate entirely in terms of duties and examinations. The company can further the study and practice of test management with the guidance of the management science and the psychology.

4. The division of labor on the testing work will be further specified. There will be a tendency of division of work of testers and testing agencies.

5. The capabilities of testers in our country have to be improved. We should keep an eye on testers who can handle test requirements, design, process and results, on the overall level of development and testing, besides finding someone who is skilled in finding bugs. Testers are supposed to know new testing concepts, possess more testing skills and accumulate more experience.

6. Test management is something engaged in all software life cycles. Defect management, test requirements management, test environment management, test cases management and test execution management are all boards of the same wooden barrel, each board cannot be shorter than others.

7. How to evaluate the test work is still an unsolved question, and it needs research and practice.

Did you like this? Share it:

The Differences between Performance Testing and Stress Testing

Performance testing is used to test the performance of software in a system. The performance testing can be done in every period of testing. Even in the element layer, the performance of a single module can also be evaluated by doing white-box testing. However, when the whole system is integrated, the actual performance of the system can be really tested.

Performance testing is often done with stress testing, and needs hardware and software testing equipments, which means, it is necessary to measure the use of resources in a demanding environment (for example, the processor cycle). The outward testing equipments can monitor the testing, when something happens (such as interruptions), it can record it. By testing the system, testers can find what reduces the efficiency and what causes the system bugs.

Stress testing is a kind of test which adds stress continuously to the system. By determining a system’s choke point or a failed performance point, the testing will enable you to know the maximum service level of the system. For example, you can measure when the system’s performance will decline or when it breaks down.

Performance test is a common used term when doing load test and force test alternately. Performance testing puts emphasis on the overall system. It has close relationship with the intensity, stress/load testing. Accordingly, stress and intensity testing should be done with performance testing.

The difference between stress testing and performance testing is their aim of the testing.

The aim of stress testing is to figure out the maximum load the system can support. The precondition is that the performance of the system can be tolerated. For example, it is usually agreed that the page should respond within 3 seconds. In a word, it tests the maximum load the system can bear on the condition that the performance can be tolerated.

Performance testing is for testing the response and the speed of the system and other performance indexes. Its precondition is that there is a certain load. For example, it will test the performance indexes when 100 users are online, and see if every user can operate normally. To sum up, it tests the performance of the system according to some indexes (such as the time of response) with different load (or with a certain load). If we say, some website performs badly, but to be exact, we should say, when how many people are online, the website performs badly.

In general, it is like a formula: overall performance=the load quantity*performance index. The overall performance is fixed, and stress testing is to test the biggest load when the performance indexes are the lowest, and performance test is to test the performance indexes when the load quantity is determined.

Did you like this? Share it:

BACK END TEST GUIDE for SQL

1. INTRODUCTION

This document is to discuss general test specification issues for SQL server back end testing and to provide testers a test design guide that includes test methodology. Most systems, i.e. Forecast LRS, Delta, KENAI, KBATS and so on, that are developed by ITG have client-server architectures. However, only a few projects have their back end completely tested.

1.1 Why back end testing is so important

A back end is the engine of any client/server system. If the back end malfunctions, it may cause system deadlock, data corruption, data loss and bad performance. Many front ends log on to a single SQL server. A bug in a back end may put serious impact on the whole system. Too many bugs in a back end will cost tremendous resources to find and fix bugs and delay the system developments. It is very likely that many tests in a front end only hit a small portion of a back end. Many bugs in a back end cannot be easily discovered without direct testing. Back end testing has several advantages: The back end is no longer a "black box" to testers. We have full control of test coverage and depth. Many bugs can be effectively found and fixed in the early development stage. Take Forecast LRS as an example; the number of bugs in a back end was more than 30% of total number of bugs in the project. When back end bugs are fixed, the system quality is dramatically increased.

1.2 Differences between back end testing and front end testing

It is not easier to understand and verify a back end than a front end because a front end usually has friendly and intuitive user interfaces. A back end has its own objects, such as, tables, stored procedures and triggers. Data integrity and protection is critical. Performance and multi-user support are big issues. Slowness in operation can be vital to the project’s future. There are no sufficient tools for back end testing. SQL language is mainly a testing tool. MS Access and MS Excel can be used to verify data but they are not perfect for testing. However, there are a large number of test tools available for front end testing. To be able to do back end testing, a tester must have strong background in SQL server and SQL language. It is relatively difficult to find testers who understand both SQL server and SQL testing. This causes a shortage of back end testers.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=17246&document_srl=590218&listStyle=&cpage=

Did you like this? Share it:

Time Estimation for Software Testing

In the process of creation of a successful software product, there is an inevitable problem of finding a balance between the quality and the release date of the software product. The testing allows of obtaining a product that satisfies all requirements. But the covering of each product risk with various test cases and compiling them take too long. The correctly prepared testing process should provide a required quality level without exceeding project time and budget . If the time for testing was estimated wrongly, it can lead you either to the late product delivery, or to the decrease of its quality and competitiveness . The estimation of time resources in software testing is a rather complicated and volumetric process but its significance for the creation of the successful project shouldn’t be underestimated.

This article contains recommendations that can help you to obtain more realistic and functional time estimates for the testing of a new project.

Written by:
Olga Volzhanina,
QA Specialist of Driver Testing Team

1. Decomposition of Testing Tasks
1.1. Testing Process Planning
1.2. Test Plan and Test Case Development
1.3. Test Environment Configuration
1.4. Execution of Test Cases
2. Debugging of Test Cases After the First Run or Product Change
2.1. Regression Testing
Conclusion

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=469996&document_srl=526798&listStyle=&cpage=

Did you like this? Share it: