Category Archives: Test Automation

What’s so scary about test automation?

In every testing environment and QA lab today, management and engineers perform hundreds of thousands of repetitive actions and procedures when testing and validating components and infrastructures. Ongoing proper execution of these actions is essential to ensure the quality of the network and products under test.

Unfortunately, when attempting to automate, most of these procedures are script-based and are difficult for others to reuse. So, every small change in an element or network layout may require editing of the script in multiple locations. When the question of automating these processes arises, more often than not test teams shy away from the subject. “Once bitten, twice shy” equals reluctance regarding the possibility of creating user-friendly, cost-saving automated frameworks for even the most seasoned testing personnel. Past problems could have been:

• self-developed automated systems that were so complicated that only the developing team could use them
• “automated” systems that relied heavily on “capture and replay” scripting that required tremendous effort for recording thousands of component-specific scripts.
• long automation projects that weren’t worth the development time expended instead of performing the actual testing work; and
• the overhead for adding and maintaining automation of new features grew over time until it was no longer worthwhile to continue to do so on top of the same framework

Despite these obstacles, there are ways to prevail over testing monstrosities and automate all or part of test procedures.

Conquer the fear
Through the development of basic building blocks that are repeatedly re-used to create test automation with little or no need for coding knowledge, test engineers can easily triumph over test automation horrors. A simple “drag and drop” scenario enables test designers to design a logical flow chart and then manipulate actions without having to perform actual scripting. Although the frameworks can be implemented differently, the below guidelines are commonly needed:

+ Test engineers must be able to build the automation on their own without involving code developers.

+ The time to automate testing of a new feature cannot be much greater than the manual testing time.

+ All the automated commands, parsing, reporting, etc. must be easily contained within re-usable and replaceable building blocks from which the automation process can be designed and built top-down.

This means that:
+ Changing a component or a small part of an existing layout will only require changing specific building blocks, not recreating the entire automation from scratch.

+ We can leverage the knowledge of our domain experts by letting others use their complex procedures packaged into simple building blocks.

+ The user interface should not vary regardless of vendor or component. All topologies, devices, interfaces, procedures, tests, regressions, results, reports and dashboards need to be supported within a single integrated platform.

+ Automation generates numerous data that must be easily aggregated from multiple tests, testers and stations and then filtered, grouped and tabulated according to test-specific requirements into a concise report.

Your ‘out of the box’ approach needs
For building successful test automation, we discussed the guidelines and now let’s define what we are looking for:

1. Usability. This includes number of users, ease of training and how quickly it can be implemented

2. Maximum functionality. Ability to support lab management, asset reservation and scheduling, device integration, topology definition and setup, automation development and execution, data collection aggregation and reporting and dashboards.

3. Flexibility and scalability. Ease to upgrade and integrate new capabilities

4. Strength of the interface library. The interfaces support all the device and application controls needed and can deal with steps that are not “out of the box.”

5. Vendor. What is the vendor’s core business? Does the product have a roadmap, and how aligned is it with your requirements? What support can you expect? And perhaps the most important factor: What references of successful automation deployments does the vendor have?

In summary, test automation doesn’t have to be scary. Adopting the right approach can lead to great results.

Source: http://www.tmworld.com/electronics-blogs/test-voices/4410272/What-s-so-scary-about-test-automation–

Did you like this? Share it:

Apple testing new iPhone, iOS 7: report

Apple Inc has started testing a new iPhone and the next version of its iOS software, news website The Next Web reported.

The company’s shares rose as much as 4.3 percent but eased a little to trade up 3 percent at $546.11 by mid-day on the Nasdaq.

Application developers have found in their app usage logs references to a new iPhone identifier, iPhone 6.1, running iOS 7 operating system, the website reported.

Apple’s iPhone 5 bears the identifiers "iPhone 5.1" and "iPhone 5.2" and is powered by the iOS 6 operating system.

Developer logs show that the app requests originate from an internet address on Apple’s Cupertino campus, suggesting that Apple engineers are testing compatibility for some of the popular apps, the website said.

"Although OS and device data can be faked, the unique IP footprint leading back to Apple’s Cupertino campus leads us to believe this is not one of those attempts," the website said.

Raymond James analyst Tavis McCourt, however, expects the next version of the iconic smartphone to be called iPhone 5S and not iPhone 6.

Apple typically tags the interim version of its phones with an "S" before moving on to a new version. iPhone 3GS followed iPhone 3G and the iPhone 4S followed iPhone 4.

McCourt also said he wouldn’t be surprised if Apple looked at an earlier launch because of the stress on its supply chain caused by late-year launches.

Apple launched iPhone 5 in September and it has been reported that the new iPhone will be released in the middle of 2013.

Techradar.com reported last month that Apple could unveil the next version of its iPhone as early as the spring of 2013.

Soucre: http://news.yahoo.com/apple-testing-iphone-ios-7-report-125657766–finance.html

Did you like this? Share it:

Automation Anywhere Launches Testing Anywhere for Cloud Applications and HTML5

"Automation Anywhere and our Testing Anywhere product line have grown rapidly because we make it easy for people in the application lifecycle including QA testers, software developers and people in non-technical roles to automate tests intelligently," said Mihir Shukla, founder and CEO of Automation Anywhere. "80% of our customers are already building cloud or web applications and asked us to expand into the cloud. So in this release we set the goal to make it easier to test cloud, web and HTML5 applications, which are increasingly important in mobile and online. Our mission is to build products that provide our customers intelligent, powerful and easy to use automation."

– Automatic Language Identification for Objects: Testing Anywhere’s
intelligent technology was developed for the cloud. It detects an
object’s programming language and then automatically chooses the best test automation technology for that language. This increases the
efficiency of the testing process, especially when there are multiple
types of objects and languages. Testing Anywhere works with languages including .Net, Java, WPF, HTML,Silverlight, Flash, Flex and more.

– Application Testing in the Cloud: Automation Anywhere customer
feedback says more than 80% of recorded test cases initially fail in
the cloud using other testing tools. Testing Anywhere aims to
significantly improve that percentage and also extends support for
nested inline frames and framesets.

– HTML5 support: With the growth of HTML5 in both web and mobile
environments, Testing Anywhere now supports automated HTML5 testing.

– Power User Functionality: Although Testing Anywhere was initially
designed to be used by a wide range of users, many advanced users have requested more control over their testing environments. As a result, Testing Anywhere has added some power user functionality such as a free-flow editor for testers to quickly type and create test cases, a multi-tab editor, advanced debugging support, and support for regular expressions in many actions.

– PDF testing support: Testing Anywhere now automatically tests PDF
capabilities for applications that export or work with PDF documents.

– Email testing support: More advanced email integration allows testers to easily connect to an email server from within Testing Anywhere and test email functionality of any software or web application.

– Enhanced Testing for Mainframes: Testing Anywhere has always used terminal emulator technologies to test new and legacy applications. Testing Anywhere 7.5 offers more advanced mainframe testing and now supports SSH1 and SSH2 protocols

– Testing Anywhere first launched in December 2009 and is being adopted by large system integrators and other software development teams looking for significantly better efficiencies and quality around
automated software testing. Testing Anywhere tests any application on any Windows platform; tests web applications on Explorer, Firefox,
Opera, Safari and Chrome; and tests custom applications written in
more than 20 languages including Python, Perl, C++ and C#. People
interested in Testing Anywhere can download the free trial version

Read More:http://www.marketwatch.com/story/automation-anywhere-launches-testing-anywhere-for-cloud-applications-and-html5-2012-05-23

Did you like this? Share it:

Automating embedded software testing with Electric Cloud

The 2012 UBM Survey showed that, for the first time, QA engineers are becoming a significant portion of embedded software teams, and there is less concern about the quality of debugging tools for those teams, However, the size of those teams is, in general, dropping and concern for tool quality is still number one, all of which makes hitting schedules on time the greatest challenge for those teams.

According to Dax Harfang at Electric Cloud, those pressures are even greater in hardware-centric companies who would rather not make a large investment in software QA, especially smaller companies that may be using resources around the world. Farhang stated that “homegrown” approaches are hard to manage, can be very slow, and often lack documentation that a distributed team can access. “Development teams need to address “back end” software production processes to save time, improve product quality and deliver software to market faster.” New Tech Press talked with Harfang about meeting automating embedded test at the 2012 Design West Conference in San Jose.

http://www.newtechpress.net/2012/04/10/automating-embedded-software-testing-with-electric-cloud/

Did you like this? Share it:

My Opinions on Automated Software Testing

Automated Software Testing is to test automatically with automated tools according to the schedule, with an aim to reduce the amount of labor of manual test. The goal of automated testing is to find out old defects of the system, while that of manual testing is to discover new defects.

Automated testing mainly focuses on automation management and automation of dynamic testing in the software testing process (Unit Test, Functional Test, Performance Test).

Advantages of Automated Testing

1) Doing regression testing to the new versions: every time a new version is released, most functions and interfaces are similar to the last version.

2) Solving problems left by manual testing in non-functional aspects: stress testing, concurrency testing, large data volume testing, breakdown testing

3) Consistency and repeatability: for the automation is achieved by scripts, data can remain the same every time the test is conducted, and the scripts can be used repeatedly.

4) Able to do repeated and large numbers of tests: after the development work is finished, regression and integration testing should be done, and every functional point in the system has to be tested frequently.

5) Making the most of time of the weekends and the nights

Suitable for the Following Situations

1) Product type projects

2) Incremental development and continuous integration projects

3) Automatic compiling and releasing projects

4) Regression testing: it can find out whether you have introduced new defects and whether the old ones have been modified. To some extent, the automated testing tools can be called regression testing tools.

5) Repeated and mechanical tests: operations like repeated data input or key typing leads to unnecessary time and labor wasting.

What Requires Attention about Automation Testing

1) Selecting fewer automatic products and more platforms as possible to reduce costs

2) During testing process management automation, we should always remember that the testing team’s needs for process management support should be met.

3) When the investment is limited, functional testing automation should be put after performance testing automation

4) Besides cost performance, support service and service integrity should also be taken into full account.

5) Choosing the main-stream product, in order that it will be convenient to communicate with other companies to acquire more experience and business opportunities.

Misunderstandings about Automated Testing

1) Automated testing can replace manual testing, and the subject of testing is still the people.

2) Automated testing can discover old defects effectively, but can’t find out new ones. The more the new defects, the higher the failure rate of automated testing is.

3) Automated testing can’t test the user experience and interface appearance and something like that.

4) Automated testing requires large amount of test script maintenance work.

Not Suitable for the Following Situations

1) Customization projects

2) Projects with a short cycle

3) Business rules are complex

4) Light work load of testing

5) Unstable programs: there are many serious bugs or it is likely to break down

Did you like this? Share it:

What kind of automated testing does Facebook do?

We do several kinds of testing. Some specifics:

For our PHP code, we have a suite of a few thousand test classes using the PHPUnit framework. They range in complexity from simple true unit tests to large-scale integration tests that hit our production backend services. The PHPUnit tests are run both by developers as part of their workflow and continuously by an automated test runner on dedicated hardware. Our developer tools automatically use code coverage data to run tests that cover the outstanding edits in a developer sandbox, and a report of test results is automatically included in our code review tool when a patch is submitted for review.
For browser-based testing of our Web code, we use the Watir framework. We have Watir tests covering a range of the site’s functionality, particularly focused on privacy—there are tons of "user X posts item Y and it should/shouldn’t be visible to user Z" tests at the browser level. (Those privacy rules are, of course, also tested at a lower level, but the privacy implementation being rock-solid is a critical priority and warrants redundant test coverage.)
In addition to the fully automated Watir tests, we have semi-automated tests that use Watir so humans can avoid the drudgery of filling out form fields and pressing buttons to get through UI flows, but can still examine what’s going on and validate that things look reasonable.
We’re starting to use JSSpec for unit-testing JavaScript code, though that’s still in its early stages at this point.
For backend services, we use a variety of test frameworks depending on the specifics of the services. Projects that we release as open source use open-source frameworks like Boost’s test classes or JUnit. Projects that will never be released to the outside world can use those, or can use an internally-developed C++ test framework that integrates tightly with our build system. A few projects use project-specific test harnesses. Most of the backend services are tied into a continuous integration / build system that constantly runs the test suites against the latest source code and reports the results into the results database and the notification system.
HipHop has a similar continuous-integration system with the added twist that it not only runs its own unit tests, but also runs all the PHPUnit tests. These results are compared with the results from the same PHP code base run under the plain PHP interpreter to detect any differences in behavior.

Our test infrastructure records results in a database and sends out email notifications on failure with developer-tunable sensitivity (e.g., you can choose to not get a notification unless a test fails continuously for some amount of time, or to be notified the instant a single failure happens.) The user interface for our test result browser is integrated with our bug/task tracking system, making it really easy to associate test failures with open tasks.

Source: http://www.quora.com/What-kind-of-automated-testing-does-Facebook-do

Did you like this? Share it:

Test automation with user interface

It is the assumption that GUI testing is much harder to pull off correctly than data-driven testing that many teams have given up on the approach, but the Geoff Bache, writer of the “Making GUI testing productive and Agile” advised teams to rethink the user interface testing and make the approach productive.

Combining domain-language descriptions of use cases with recorded user interface action will do help to get confidence and make easy maintenance. In order to make recorded test cases easier to understand, Bache advise teams to record actions and assign domain names to such recorded actions. And he also advises verification by logging and inspecting textual log outputs of the system to validate a function was performed correctly.

By allowing teams to record domain activities, combing them into readable scripts and saving detect behavior changes, teams can inspect changes and decide whether the changes were expected or not, furthermore, it can allow teams to create new tests required.

Source:http://www.onestoptesting.com/testing-articles/automated-testing/details/rethinking-user-interface-test-automation-366.asp

Did you like this? Share it:

Mix of automatic and manual testing

Now days, enterprises turn to automation due to the benefits of reducing costs, saving time, increasing repeatability of test execution, and effectively utilizing resources under extreme pressure. Automation also helps QA teams to effectively plan test cycles by measuring the risk of test requirements.

Even though automation offers several benefits, it doesn’t mean suiting for all the cases. If the application changes are minor and can be quickly tested manually, then there is no need to use automating testing, or an early application which may undergo many modification to the test scripts, so it is not available to adopt automation. Apart from this use case, the other factors such as automation support technologies, money and resources etc.

Therefore the trick in the success of enterprises’ testing efforts lies on how well they set in place a right mix of automation and manual testing.

Source: http://technology.ezinemark.com/automation-and-manual-testing-mix-them-right-7d2de3cad027.html

Did you like this? Share it:

Six Major Components of a Test Automation Framework

A test automation infrastructure, or framework, consists of test tools, equipment, test scripts, procedures, and people needed to make test automation efficient and effective. The creation and maintenance of a test automation framework are key to the success of any test automation project within an organization.
The implementation of an automation framework generally requires an automation test group. The responsibility of this group is to develop test automation infrastructure, test libraries, and tests tools.

The idea behind an automation infrastructure is to ensure the following:

a) Different test tools and equipment are coordinated to work together.

b) The library of the existing test case scripts can be reused for different test projects, thus minimizing the duplication of development effort.

c) Nobody creates test scripts in their own ways.

d) Consistency is maintained across test scripts.

e) The test suite automation process is coordinated such that it is available just in time for regression testing.

f) People understand their responsibilities in automated testing.

Components of a typical test automation framework are as described in the following figure.

1) System to Be Tested:

This is the first component of an automation infrastructure. The subsystems of the system to be tested must be stable; otherwise test automation will not be cost effective. All the subsystems must be stable and work together as a whole before the start of an automation test project.

2) Testing Platform:

The testing platform and facilities, that is, the network setup on which the system will be tested, must be in place to carry out the test automation project. For example, a procedure to download the image of the SUT, configuration management utilities, servers, clients, routers, switches, and hubs are necessary to set up the automation environment to execute the test scripts.

3) Test Case Library:

It is useful to compile libraries of reusable test steps of basic utilities to be used as the building blocks of automated test scripts. Each utility typically performs a distinct task to assist the automation of test cases. Examples of such utilities are ssh (secure shell) from client to server, exit from client to server, response capture, information extraction, rules for verdicts, verdict logging, error logging, cleanup, and setup.

4) Automated Testing Practices:

The procedures describing how to automate test cases using test tools and test case libraries must be documented. A template of an automated test case is useful in order to have consistency across all the automated test cases developed by different engineers. A list of all the utilities and guidelines for using them will enable us to have better efficiency in test automation. In addition, the maintenance procedure for the library must be documented.

5) Testing Tools:

Different types of tools are required for the development of test scripts. Examples of such tools are test automation tool, traffic generation tool, traffic monitoring tool, and support tool. The support tools include test factory, requirement analysis, defect tracking, and configuration management tools. Integration of test automation and support tools, such as defect tracking, is crucial for the automatic reporting of defects for failed test cases. Similarly, the test factory tool can generate automated test execution trends and result patterns.

6) Test Administrator:

The automation framework administrator does the following
a) Manages test case libraries, test platforms, and test tools;
b) Maintains the inventory of templates;
c) Provides tutorials; and
d) Helps test engineers in writing test scripts using the test case libraries.
e) Provides tutorial assistance to the users of test tools and maintains a liaison with the tool vendors and the users.

Source:http://www.softwaretestinggenius.com/articalDetails.php?qry=957

Did you like this? Share it:

M	T	W	T	F	S	S
« May
					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30