Tag Archives: automation

Software Testing is not a commodity!

Stick in software testing long enough, and you will see enough ideas come and go to be able to sort out the ones that look promising to work, and the ones that you just hope will go away soon enough so that no manager will pay any of her attention to it. There have been quite a few in the history of software testing, and from my experience the worst things started to happen every time when someone tried to replace a skilled tester with some piece of automation – whether that particular automation was a tool-based approach or some sort of scripted testing approach.

Why do we test software?

If we were able to write software right the first time, there would be clearly no need to test it. Unfortunately we humans are way from perfect. Take for example the book I wrote mostly through 2011. 200 pages, lots of reviewing, production planning, and stuff happening in the end. And still, while reviewing the German translation, I spotted a problem in the book – clearly visible at face value. I had spend at least 2 weeks after work to go through the book once more, and get everything right. Yet, I failed to see this obvious problem.

The problem lies in our second-order ignorance: the things we don’t know that we don’t know them. These are the things of good hope, and prayers that it will work. Murphy’s Law also has a role to play here.

The very act of software testing then becomes to find out as much information about our unawareness as possible. This includes not only exercising the product, but also finding out new things about the product. Skilled testers learn more about the product and the product domain and the development team over the course of the whole product lifecycle.

Why do we repeat tests?

But how come we focus on regressions to often in our industry? It has to do with first-order ignorance. A regression problem is a bug that gets introduced a second time, although it already had been fixed in the meantime. Since we were already fully aware of the problem, the bug is no longer something that we don’t that we don’t know it. It has become something that we know now, but we don’t know whether we will know it still tomorrow. That’s why we introduce a regression check for tomorrow, so that it will remind us about the problem that we tried to avoid at this time.

Read that sentence again. Yes, it’s speculation. We speculate that we might break the software tomorrow again. With this speculation comes a whole lot of costs. We have opportunity costs for doing the test, for automating it, and with every run, we have the opportunity cost of analyzing the result (if we have to).

We wouldn’t need this if we were able to realize that a regression bug introduced in our software is an opportunity to learn what is not working in our current process that caused that bug to re-occur. Every regression bug discovered should be an invitation to start a root cause analysis and fix the underlying problem rather than deal with the symptoms.

Source: http://www.shino.de/2013/02/04/software-testing-is-not-a-commodity/

Did you like this? Share it:

iPhone Vulnerability: Return of the Lock Screen Bypass

iphone lock screen

Reports yesterday of a lock screen bypass in the iPhone 5 noted that a "similar" bug was found in iOS 4.1 and fixed in 4.2. In both cases, the lock screen, which is only supposed to let you make emergency calls or enter the lock code, allows the user to perform other functions, like make other phone calls. How do these errors resurface after being fixed? In Apple’s case, the problem could be a weakness in their test plans or procedures.

The iPhone lock screen
When an error that was fixed shows up again later it is called a regression error. Regression errors generally are when some change to the program, a new version or software patch, breaks some feature of the program. Security fixes are one type of feature that could be broken.

Controlling regression errors is a matter of proper documentation and testing. Good code documentation should at least give future developers the chance to recognize that changes will affect the feature. But it’s testing that is the key to preventing regressions.

Any well-designed software project has a formal test plan as part of it. As new features and bug fixes are added, test should also be added to the test plan to make sure that new fixes don’t break old features or fixes. In the case of security patches, a test needs to be added to the plan to check for each vulnerability that is fixed.

The real key to making regression testing practical is to automate it. Back around 2007 and 2008, Mozilla had a very bad problem with security patches causing regressions of other security patches. They finally got it under control and attributed their success, in part, to increased automated testing.

Almost any test can be automated, even by simulating user interface actions by hardware through the USB connection to the device. But the lock screen on iOS is a problem for test automation. The lock screen is designed not to allow external hardware to break out of it, lest someone else take your phone and gain control of it. There’s no automated way to test it, so you have to test it manually.

In all likelihood, Apple has some manual tests to perform as well, but it’s easy to see how they would get shrugged off in a hurry or given to some intern who didn’t execute them properly. Expect an angry memo to go around at Apple about this, but deadlines are deadlines and one day the manual testing will again seem like a corner worth cutting.

Source: http://www.informationweek.com/byte/personal-tech/iphone-vulnerability-return-of-the-lock/240148663

Did you like this? Share it:

HP, Wavefront Partner on Mobile Testing and Monitoring Services

Wavefront is pleased to announce it has been selected by Hewlett-Packard (HP) to act as a premier reseller of HP’s Unified Functional Testing (UFT) and Quick Test Professional (QTP) software.

As Canada’s Centre of Excellence for Wireless Commercialization and Research, this new partnership will enable Wavefront to connect large enterprise companies with best-in-class HP software for accelerated mobile application testing, automation and monitoring.

According to James Maynard, President and CEO, Wavefront, “By tightly linking HP’s UFT and QTP software with our Perfecto Mobile cloud-based mobile testing and monitoring services, Wavefront facilitates a complete end-to-end solution that improves efficiency and quality assurance for enterprise-grade mobile application testing, while accelerating time to market.”

Wavefront’s cloud-based testing and monitoring service, integrated with HP’s UFT and QTP software and Perfecto Mobile’s best in class mobile application testing solutions, can be accessed either remotely or on-site, with a direct connection to one of the country’s most comprehensive mobile device libraries.

With more than 1,000 handsets, smartphones and tablets available to test on live global networks, businesses can identify exactly how their applications will perform on actual mobile browsers, networks and devices.

“HP Canada is extremely happy and excited to announce our strategic partnership with Canada’s Centre of Excellence for Wireless Commercialization and Research – Wavefront. Wavefront’s thought leadership position around wireless communication in conjunction with HP’s recent partnership with Perfecto Mobile position HP uniquely in the Mobile app delivery space. The strategic partnership with both Wavefront and Perfecto are what will launch HP’s enterprise class testing, monitoring and security portfolio into unparalleled dominance of the mobile application delivery market,” said Frances Newbigin, Vice President and General Manager – HP Enterprise Software and Solutions at Hewlett-Packard Canada.

Source: http://www.mediacastermagazine.com/pressroom/productDetail.aspx?id=10772

Did you like this? Share it:

How to Do Automation Testing of iPhone Applications

Why automation? It saves precious time spent in running manually one and the same tests over and over again with each new build of the application. And secondly, it improves the stability of the application and decreases regression bugs by making it easy to run tests after each non-trivial code change or even at the end of each development day.

How about automation testing of mobile applications? Well, this area has always been foggy.

There are lot of tools for web application testing and some of them could fit automation testing of mobile applications either independently, or with an extension. For example, you could use M-eux Test as an extension on top of HP’s Quick Test Professional (QTP). M-eux Test recognizes and activates the GUI objects on the screen of the mobile device while utilizing best automation testing practices already incorporated into QTP. The restriction of M-eux Test comes from the lack of supported OSs – in fact, it supports only Windows Mobile apps (6.x and 5.0 Mobile PC).

What happens if you want to test an iPhone app? Recently Gorilla Logic, an enterprise IT consulting services company has launched free functional testing tool for iPhone apps named FoneMonkey. It provides the ability to capture, edit, and replay user interface tests on iPhone, thus covering the functionality of an iPhone app with automation tests.

Further, FoneMonkey provides options for integrated script editing while playing a recorded scenario, and modifiable assertion checks. Sounds familiar?

I think FoneMonkey will quickly become a standard for iPhone test automation. Besides, the people from Gorilla Logic are handling issues and change requests in no time, there is an extensive and easy-to-follow documentation, and a dedicated forum hosted by the company. If you are into the mobile application development business, I could highly recommend you the tool!

Source:http://www.bianor.com/blog/how-to-do-automation-testing-of-iphone-applications/

Did you like this? Share it:

Automation Anywhere Launches Testing Anywhere for Cloud Applications and HTML5

"Automation Anywhere and our Testing Anywhere product line have grown rapidly because we make it easy for people in the application lifecycle including QA testers, software developers and people in non-technical roles to automate tests intelligently," said Mihir Shukla, founder and CEO of Automation Anywhere. "80% of our customers are already building cloud or web applications and asked us to expand into the cloud. So in this release we set the goal to make it easier to test cloud, web and HTML5 applications, which are increasingly important in mobile and online. Our mission is to build products that provide our customers intelligent, powerful and easy to use automation."

–  Automatic Language Identification for Objects: Testing Anywhere’s
intelligent technology was developed for the cloud. It detects an
object’s programming language and then automatically chooses the best test automation technology for that language. This increases the
efficiency of the testing process, especially when there are multiple
types of objects and languages. Testing Anywhere works with languages including .Net, Java, WPF, HTML,Silverlight, Flash, Flex and more.

–  Application Testing in the Cloud: Automation Anywhere customer
feedback says more than 80% of recorded test cases initially fail in
the cloud using other testing tools. Testing Anywhere aims to
significantly improve that percentage and also extends support for
nested inline frames and framesets.

–  HTML5 support: With the growth of HTML5 in both web and mobile
environments, Testing Anywhere now supports automated HTML5 testing.

–  Power User Functionality: Although Testing Anywhere was initially
designed to be used by a wide range of users, many advanced users have requested more control over their testing environments. As a result, Testing Anywhere has added some power user functionality such as a free-flow editor for testers to quickly type and create test cases, a multi-tab editor, advanced debugging support, and support for regular expressions in many actions.

–  PDF testing support: Testing Anywhere now automatically tests PDF
capabilities for applications that export or work with PDF documents.

–  Email testing support: More advanced email integration allows testers to easily connect to an email server from within Testing Anywhere and test email functionality of any software or web application.

–  Enhanced Testing for Mainframes: Testing Anywhere has always used terminal emulator technologies to test new and legacy applications. Testing Anywhere 7.5 offers more advanced mainframe testing and now supports SSH1 and SSH2 protocols

–  Testing Anywhere first launched in December 2009 and is being adopted by large system integrators and other software development teams looking for significantly better efficiencies and quality around
automated software testing. Testing Anywhere tests any application on any Windows platform; tests web applications on Explorer, Firefox,
Opera, Safari and Chrome; and tests custom applications written in
more than 20 languages including Python, Perl, C++ and C#. People
interested in Testing Anywhere can download the free trial version

Read More:http://www.marketwatch.com/story/automation-anywhere-launches-testing-anywhere-for-cloud-applications-and-html5-2012-05-23

Did you like this? Share it:

Automating embedded software testing with Electric Cloud

The 2012 UBM Survey showed that, for the first time, QA engineers are becoming a significant portion of embedded software teams, and there is less concern about the quality of debugging tools for those teams,  However, the size of those teams is, in general, dropping and concern for tool quality is still number one, all of which makes hitting schedules on time the greatest challenge for those teams.

According to Dax Harfang at Electric Cloud, those pressures are even greater in hardware-centric companies who would rather not make a large investment in software QA, especially smaller companies that may be using resources around the world.  Farhang stated that “homegrown” approaches are hard to manage, can be very slow, and often lack documentation that a distributed team can access.  “Development teams need to address “back end” software production processes to save time, improve product quality and deliver software to market faster.”  New Tech Press talked with Harfang about meeting automating embedded test at the 2012 Design West Conference in San Jose.

Read More:

http://www.newtechpress.net/2012/04/10/automating-embedded-software-testing-with-electric-cloud/

Did you like this? Share it:

My Opinions on Automated Software Testing

 

Automated Software Testing is to test automatically with automated tools according to the schedule, with an aim to reduce the amount of labor of manual test. The goal of automated testing is to find out old defects of the system, while that of manual testing is to discover new defects.

Automated testing mainly focuses on automation management and automation of dynamic testing in the software testing process (Unit Test, Functional Test, Performance Test).

Advantages of Automated Testing

1) Doing regression testing to the new versions: every time a new version is released, most functions and interfaces are similar to the last version.

2) Solving problems left by manual testing in non-functional aspects: stress testing, concurrency testing, large data volume testing, breakdown testing

3) Consistency and repeatability: for the automation is achieved by scripts, data can remain the same every time the test is conducted, and the scripts can be used repeatedly.

4) Able to do repeated and large numbers of tests: after the development work is finished, regression and integration testing should be done, and every functional point in the system has to be tested frequently.

5) Making the most of time of the weekends and the nights

Suitable for the Following Situations

1) Product type projects

2) Incremental development and continuous integration projects

3) Automatic compiling and releasing projects

4) Regression testing: it can find out whether you have introduced new defects and whether the old ones have been modified. To some extent, the automated testing tools can be called regression testing tools.

5) Repeated and mechanical tests: operations like repeated data input or key typing leads to unnecessary time and labor wasting.

What Requires Attention about Automation Testing

1) Selecting fewer automatic products and more platforms as possible to reduce costs

2) During testing process management automation, we should always remember that the testing team’s needs for process management support should be met.

3) When the investment is limited, functional testing automation should be put after performance testing automation

4) Besides cost performance, support service and service integrity should also be taken into full account.

5) Choosing the main-stream product, in order that it will be convenient to communicate with other companies to acquire more experience and business opportunities.

Misunderstandings about Automated Testing

1) Automated testing can replace manual testing, and the subject of testing is still the people.

2) Automated testing can discover old defects effectively, but can’t find out new ones. The more the new defects, the higher the failure rate of automated testing is.

3) Automated testing can’t test the user experience and interface appearance and something like that.

4) Automated testing requires large amount of test script maintenance work.

Not Suitable for the Following Situations

1) Customization projects

2) Projects with a short cycle

3) Business rules are complex

4) Light work load of testing

5) Unstable programs: there are many serious bugs or it is likely to break down

Did you like this? Share it:

Software Testing And Maintenance Makes The Software Reliable

Introduction:- There are so many software companies running in the market, which provide the software services like:- software development, web development, application development, software testing, software maintenance, search engine optimization and many more. Software testing plays a major role in the success of any software because compatibility and accuracy of software matters on its testing portion. It stated as the validation of a software program, guides the software design and development according to the requirement. In simple words, software testing is the process of execution of a program finding errors in its functionality, security and productivity of the product.

Software testing can be implemented at any time during development process, but it is mostly evaluated after the completion of designing and coding portion of the software.

People think like software testing makes the software risk free but it is not true. Basically, it reduces the risk of application that occurs in the software but not completely. The two major areas of testing are:- correctness testing and reliable testing.

Now-a-days, most of the software engineers ready to move in testing field for its creativity, its challenge of automation, its elation to system thinking. Most of the software companies in India also work as outsourcer, they do the project of another company on the contract basis, that type of companies are called Offshore Software companies. These software companies provide the services: – software application testing, outsourced software testing, security testing, unit testing, insurance testing and mobile domain testing.

There are two techniques used in testing White Box Testing and Black Box Testing. Black box testing is used to check the application of software externally whereas; white box testing is used to check internally. Thorough analysis and testing of application program needs the broad knowledge of testing techniques and requires the testing tools.

Testing engineers have the capability to handle:-

  1. Both manual as well as automation testing.
  2. Open source technology as well as commercial

Source: http://www.flightcasteranalytics.com/2012/03/software-testing-and-maintenance-makes-the-software-reliable/

Did you like this? Share it:

IBM Said New Attacks Appeared

According to foreign reports, IBM found that the current Internet security has been significantly improved. Application security vulnerabilities, malicious code and spam have been greatly reduced. But such achievements are paid a certain price, that is, it forces the attacker to rethink their attack strategy.

IBM security team, X-Force, released the 2011 trend and risk report, which surveyed about 4,000 consumers. Compared to 2010, the amount of spam dropped 50%. There is only 36% of software vulnerabilities not repaired in 2011 while 43% in 2010. Although some loopholes in the software have never been restored, the percentage declines. Thanks to the improvement of the quality of the web application code, the cross-site scripting attacks decline half than four years ago. But the cross-site scripting vulnerabilities still occurs 40% of the program scanned by IBM. In addition, the amount of attack code relying on the security vulnerability reduces 30% than four years ago.

The new security attacks include Shell command injection vulnerability, automatic password guessing, increased phishing attacks, mobile malware and so on.

Did you like this? Share it:

Study: Cloud computing becoming pervasive, and IT needs to take control now

Cloud computing may be taking the business world by storm, but its success could mean a “perfect storm” that endangers the role of IT.

As a result, IT needs to step up now and change its approach to cloud services. This includes building trust with the lines of business, beginning to manage public cloud services, and pursuing increased automation for service provisioning and operations.

These are the key findings of a survey commissioned by BMC Software and conducted by Forrester Research. The study, “Delivering on High Cloud Expectations,” shows that business units’ demand for speed and agility is leading them to circumvent IT and acquire cloud services, more than half of them from what were termed “unmanaged” clouds.

Brian Singer, Lead Solutions Marketing Manager for BMC, said his company commissioned the survey in an effort to confirm what the company was hearing anecdotally from customers. “Cloud and software as a service (SaaS) are in enterprises in a big way,” Singer said, “and we wanted to see how IT was dealing with them.”

For the study, researchers polled 327 enterprise infrastructure executives and architects in the United States, Europe, and Asia-Pacific. Among the key findings:

  • Today, 58 percent run mission critical workloads in unmanaged public clouds, regardless of policy. The researchers use “unmanaged” to describe clouds that are managed by the cloud operators, but not by the company buying the service.
  • In the next two years, 79 percent plan to run mission-critical workloads on unmanaged cloud services.
  • Nearly three out of four responders, 71 percent, thought that IT should be responsible for public cloud services.
  • Seventy two percent of CIOs believe that the business sees cloud computing as a way to circumvent IT.

Source: http://www.zdnet.com/blog/gardner/study-cloud-computing-becoming-pervasive-and-it-needs-to-take-control-now/4561?tag=content;search-results-river

Did you like this? Share it: