Tag Archives: automation

Automation Testing Framework Approach

A framework approach to create an Automated Regression Test Bed has many benefits over the traditional simple capture/playback or custom scripts methodology. However evolving a keyword driven framework approach for automating the regression test bed of scripts, may take time and is generally not recommended if at all any major changes are expected to be met up in near or even near far future.
How to do Automation Testing :

• Reuse of Tests : When a certain set of test scenarios require frequent execution it makes more sense to automate them and reap the benefits of unattended automated execution of test scripts.

• Another example will be in cases where a single script needs to be executed with multiple data sets. In such cases the effort to automate a single script and running it with multiple data sets is far less than the manual execution effort for all those data sets.

• Time Save : Running unattended automated test scripts saves human time as well as machine time than executing scripts manually.

• Better use of people : While automated scripts are running unattended on machines, testers can do more useful tasks.

• Cost Saving : On test engagements requiring a lot of regression testing, usage of automated testing reduces the people count and time requirement to complete the engagement and helps reduce the costs.

• Machines are more reliable than humans : Great confidence will be gained when a system is released, when we use automated testing . With some approach instead fo just record and is playback , it becomes very easy easy to maintain and update the scripts with newer releases or changes done to the application. Also the degree of scripting gets minimized as the resources working on the automation scripting mature to successfully understand and develop some set of library for their reusability.

• Selective testing : There should be an easy way to only test the breadth of the application without doing a lot of depth testing. This is to be used for stability testing of an application • There should be an easy way to select test scenarios at a more granular level in order set up more targeted regression.

• Recovery : If test script execution is stopped at any moment for any reason, the test suite should be able to resume testing from that point at any time later. We should not need to re-test functionality that had passed earlier.

• Scalability : It should be possible to execute tests from different machines at the same time in order to gain time efficiencies. 

• Inter-environment portability : The scripts should run on any given test environment without any code change (technical intervention).

• Usability : With minimal amount of training, anyone on the team must be able to execute the automated suite . The script must run without any manual intervention.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=12632&document_srl=539680&listStyle=&cpage=

Did you like this? Share it:

Apply QA Testing in an Agile Development Methodology

The growing popularity of utilizing the Agile Development methodology to develop systems has changed the way software is tested. Certainly, high quality is still a requirement but the approaches, techniques and artifacts have changed. In Agile development, there is a common “principle” that for a specific period called a Sprint (usually 30 days), all software will be fully developed, tested and documented and “releasable ready” for production use. The speaker will educate on the appropriate approaches, artifacts, and metrics to successfully deliver high quality products on time in an Agile methodology. This will include customer experiences to show real life examples and techniques for successful “Agile” Testing.

  1. Customer Agile Experiences
  2. QA Processes and Artifacts
  3. Automation Testing Approaches
  4. Performance Testing Approaches

Source: http://www.vietnamesetestingboard.org/zbxe/?document_srl=425387

Did you like this? Share it:

Top 20 practical software testing tips

Here are some of the best testing practices I learned by experience:

1) Learn to analyze your test results thoroughly. Do not ignore the test result. The final test result may be ‘pass’ or ‘fail’ but troubleshooting the root cause of ‘fail’ will lead you to the solution of the problem. Testers will be respected if they not only log the bugs but also provide solutions.

2) Learn to maximize the test coverage every time you test any application. Though 100 percent test coverage might not be possible still you can always try to reach near it.

3) To ensure maximum test coverage break your application under test (AUT) into smaller functional modules. Write test cases on such individual unit modules. Also if possible break these modules into smaller parts.

4) While writing test coverage, write test cases for intended functionality first i.e. for valid conditions according to requirements. Then write test cases for invalid conditions. This will cover expected as well unexpected behavior of application under test.

5) Think positive. Start testing the application by intend of finding bugs/errors. Don’t think beforehand that there will not be any bugs in the application. If you test the application by intention of finding bugs you will definitely succeed to find those subtitle bugs also.

6) Write your test cases in requirement analysis and design phase itself. This way you can ensure all the requirements are testable.

7) Make your test cases available to developers prior to coding. Don’t keep your test cases with you waiting to get final application release for testing, thinking that you can log more bugs. Let developers analyze your test cases thoroughly to develop quality application. This will also save the re-work time.

8 ) If possible identify and group your test cases for regression testing. This will ensure quick and effective manual regression testing.

9) Applications requiring critical response time should be thoroughly tested for performance. Performance testing is the critical part of many applications.

10) Programmers should not test their own code.

11) Go beyond requirement testing. Test application for what it is not supposed to do.

12) While doing regression testing use previous bug graph (Bug graph – number of bugs found against time for different modules).

13) Note down the new terms, concepts you learn while testing.

14) Many times testers or developers make changes in code base for application under test.

15) Keep developers away from test environment.

16) It’s a good practice to involve testers right from software requirement and design phase.

17) Testing teams should share best testing practices, experience with other teams in their organization.

18) Increase your conversation with developers to know more about the product.

19) Don’t run out of time to do high priority testing tasks.

20) Write clear, descriptive, unambiguous bug report.

Source: http://www.vietnamesetestingboard.org/zbxe/?document_srl=565689

Did you like this? Share it:

The Distributed Agile Model

Since proponents of Agile processes prefer face-to-face conversation as the means for communication within a development team, in the “distributed Agile model” – in which members of an Agile team are situated in more than one workplace — this may seem unlikely.

But Agile processes are not rule-bound; rather, they optimize for methods and dynamics by which organizations are able to build, maintain, and apply systems of knowledge within changeful, context-dependent, collaborative work situations.

So when we evaluate the viability of distributed Agile (DA), our concern is not whether it complies with a particular definition of Agile, but whether it can support iterative, incremental, and sustainable development; promote teamwork; promote self-management; adapt readily to emergent changed requirements; and increase customer participation in the development process. Based on my experience I would say DA is, in this sense, viable. But special measures must be taken, especially in the area of communication. I will briefly describe those measures in this paper.

First, though, it may be helpful to state some background. We worked with organizations who, following their companies’ recommendation or mandate, had adopted Agile project management, but also wanted to outsource their testing.

Some varieties of testing are a very good fit for DA, such as running full regression suites on the most recent known-good build, performance testing, automation development, and end-to-end testing.

One of the best fits for DA has been a sort of cluster of remote Scrums, where the home group drives development of the core engine, while a remote group is spawned whenever an adapter or significant new feature is required — each remote group communicating closely with one member of the home group who attends daily standups etc.

Source: http://www.vietnamesetestingboard.org/zbxe/?document_srl=459513

Did you like this? Share it:

From manual testing to test automation

Costs associated with software testing

The process of software development is complicated and can last years, there can be and there will be some bugs, mistakes, errors, functions that stop working under certain conditions etc. Sure, there are better testers and users that help to find bugs in software, but the cost of having version of software with bugs is too high.

Software testing might be a minor problem application with a army of free testers and beta-tester, but there also expensive corporate software applications that cannot be released without full testing.

There are different method for software testing – manual testing or automated testing, but the answer about if company should test its software is always "Yes", because costs of having software with bugs available in the market are too high.

Advantages and disadvantages of the black box

There are different concepts about how to test software. Some approaches involves unit tests or functional tests. Software developers usually do these tests in the beginning of the development process. Once the first version of application was released, it is a good idea to start with black box testing.

When tester do a Black Box Testing he don’t know about any functions or units inside the application. He only knows how the application should work, e.g. what should user do and what results he will have.

The main advantage of Black Box Testing is that:

Tester will act exactly as user will do. Tester will click exactly the same buttons and work with exactly the same objects. Thus, the black box testing is a testing from user viewpoint.

The main disadvantage of Black Box Testing is that:

Black box testing is hard to automated, while functional or unit tests can be completed in just few seconds, black box testing requires tester to click buttons, enter texts and work with software graphical interface (GUI). RoutineBot software was designed to make software black box testing faster.

So, how to do black box testing in a case of software? We can talk about three methods: manual testing, "record and playback" testing, automation test.

Source: http://www.comparesuite.com/solutions/tests-automation/from-manual-testing-to-test-automation.htm

Did you like this? Share it:

Practical side of Agile Testing: Insights for software testing

03 Conferences are an opportune time to learn, network and just have fun. As inspired as I was by the many talks at Agile Testing Days about how we learn and what makes our teams jell, I also took away good ideas from the practical, technical sessions. In this second part of a two-part series, you’ll find out more of the many insights that were shared at the conference.

Automation

Mike Scott explained how his team evolved the Selenium Page Objects concept to provide automated tests that were easier to maintain and understand. In addition, the tests are flexible; they are able to continue even when an unexpected result occurs.

A balanced test strategy

Another practical session I attended was Anko Tijman’s presentation on a balanced test strategy. Anko pointed out that we need not only potentially shippable software, but maintainable software. He has found that risk is best mitigated by covering five different areas: test cases with user stories, unit and integration tests, non-functional tests, exploratory testing and customer acceptance.

Visualizing quality

Gojko Adzic’s talk on “Five Key Challenges for Agile Testers Tomorrow” took us back to the idea of visualization that earlier speakers discussed. His Agile Testing Donut provided an excellent example of visualization – if you had two seconds to eat, where would YOU take a bite? Gojko started visualizingquality.org to help our community find good ways to represent quality and risk and guide our testing and coding approach.

Self-organizing

Like most conferences, the most valuable discussions occurred during the excellent long lunch breaks and evening social events. Additionally, there was an entire “collaboration day” with Open Space, facilitated by Brett Schuchert, and the Test Lab, produced by James Lyndsay and Baart Knaak.

Making tacit knowledge explicit

Michael Bolton started off his keynote with some memorable one-liners, such as, “When a manager asks you to show him your test case, ask to see his management cases”, and “When a manager says you can lower the cost of testing with test automation, ask him if he can lower the cost of management with management automation.”

Source: http://searchsoftwarequality.techtarget.com/news/2240111928/The-practical-side-of-Agile-Testing-Days-Insights-for-software-testing

Did you like this? Share it:

Testing Your Test Code

In the world of software testing and quality assurance, we all know the value test automation brings by improving test coverage, overall product quality and the tester’s productivity. But all of this value flows in and the return on investment occurs only when the automation code is robust and reliable enough to produce consistent results to catch product bugs. The term “product bugs” is very important here. If the automation code does not catch bugs (if this is truly because the product has reached a steady state and is largely bug free, then it is acceptable) or shows more false negatives (due to test code issues rather than product issues), there will be a lot of wasted effort including:

Test Automation Effort – code design, implementation and maintenance resulting in wasted time, cost and human resources

Triage time – involving the product team to look into the invalid bugs reported resulting in expended time, cost and human resources and more importantly the reputation of the test team

Resource Usage – machine, other infrastructure and software usage for automation execution

Source: http://www.ctoedge.com/content/testing-your-test-code

Did you like this? Share it:

Testing Your Test Code

In the world of software testing and quality assurance, we all know the value test automation brings by improving test coverage, overall product quality and the tester’s productivity. But all of this value flows in and the return on investment occurs only when the automation code is robust and reliable enough to produce consistent results to catch product bugs. The term “product bugs” is very important here. If the automation code does not catch bugs (if this is truly because the product has reached a steady state and is largely bug free, then it is acceptable) or shows more false negatives (due to test code issues rather than product issues), there will be a lot of wasted effort including:

Test Automation Effort – code design, implementation and maintenance resulting in wasted time, cost and human resources

Triage time – involving the product team to look into the invalid bugs reported resulting in expended time, cost and human resources and more importantly the reputation of the test team

Resource Usage – machine, other infrastructure and software usage for automation execution

One of the traditional solutions adopted to solve this problem and make the test code reliable and robust is to “test” the test code itself, as if it were product code. While this sounds great in theory, are all teams really practicing it? Also, it is all the more important to carefully choose a smart strategy and decide “How much to test” your test code because your goal is very different in this test effort compared to product testing goals. Any extra test effort here is again going to lead to a lot of wasted resources and is not going to add any value. Choosing what areas to test, how to test and agreeing upon your goals of testing the test code are vital. Here are some best practices on "testing your test code," which will help you make these important decisions.

Source: http://www.ctoedge.com/content/testing-your-test-code

Did you like this? Share it:

Apple is testing Siri on the iPhone 4

It is reproted that Apple is testing Siri specifically on the iPhone 4. Currently Siri only runs on Apple’s latest device, the iPhone 4S. However, it doesn’t seem to need anything extra to run on the older model. It would be a big surprise if Siri was to become available the update via a software upgrade. Moreover, Appadvice reports that test was successful, with the exception of the iPad Touch.

At this moment, Apple testing something does not automatically mean the feature will appear on the iPhone any time soon. Apple is also suggested to restore its employee’s phones to original stock software.

Source: http://recombu.com/news/apple-testing-siri-on-the-iphone-4_M15749.html

Did you like this? Share it:

Quality Assurance and Quality Control

We define the processes, policies, strategies, establish standards, developing checklists and so on in ‘QA’, while in ‘QC’ we follow all those defined processes, standards and policies to make sure that the project has been developed with high quality and at least meets customer’s expectations. QA does not assure quality, rather it creates and ensures the processes are being followed to assure quality. Meanwhile, QC does not control quality, rather it measures quality. QA and QC are both powerful techniques which can be used to ensure that the deliverables meet high quality expectations of customers.

Source: http://www.softwaretestinghelp.com/does-quality-assurance-remove-need-for-qualiqu-control/

Did you like this? Share it: