At the STAR EAST conference this year, mobile solutions saturated the vendor exhibition area. It’s no wonder that mobile application testing is the hot in the testing community right now. Testers are now faced with the task of testing mobile versions of their applications and figuring out how to adapt to the new paradigm (and challenges) mobile applications bring. It’s not as if they didn’t have enough to test already, right?

After listening to some of the conversations around mobile and seeing some interesting questions being raised, I’m wondering if this has become more difficult than it should be. A common question I hear is “How do I figure out how to test all of these different devices with various screen sizes by different manufacturers with a different OS on each?”  A quick look at the market reveals at least four major operating systems (Apple, Android, Windows, and Blackberry), with multiple versions for an untold number of devices for each.  I think every tester would agree, automation is a must.

So, should the device matter? For years we have been testing applications on various laptops, desktops, and servers without regard to the model, size, or parts inside. Aren’t we just replacing the PC with a mobile device? The reason I could see the specific device playing a key role today has to do with the way automation is being accomplished – namely through bitmap recognition and/or OCR technologies. These technologies are useful, and serve their purpose for building automated test scripts. However, when screen sizes change (i.e. iPhone to iPad), this breaks the scripts. Worse case, multiple versions of the script need to be in place to accommodate different devices.

As an HP reseller, we’re focused on the automation capabilities of HP’s Unified Functional Test, which includes QuickTest Pro and Service Test. There are a few add-ins that can be purchased to add mobile automation. They include ZAP-fiX, DeviceAnywhere, and Perfecto Mobile. All of these handle automation using bitmap recognition and/or OCR. They also currently require the device to be jail broken (iPhone) or rooted (Android) to start testing. This causes a major dilemma for the QA Manager committed to quality. How can you start with a fundamental change to the OS code? It’s a hack, which means it is an undocumented change to the release being tested that does not reflect the production end user build (unless they have the exact jailbreak too) with no control over that changed code. Nevertheless, testing must get done.

What if there was one solution that approached this problem a little differently? JAMO Solutions provides code that is integrated into the application as it is delivered to the device. This code allows for object recognition so that an automation product like QuickTest Pro can “hook” into the application and see all of the objects, their name, and many other properties. Why is that important? Now we have access to all the objects of the application various known properties that bitmap recognition methods could never be aware of (like hidden objects that might still need some form of verification). Scripts won’t break due to screen size because the object’s property is still the same. This means the OS matters, but the device doesn’t. If the device supports the OS, then the script will still work even when the device changes. This is much like how testers approach web based application test automation. They are not concerned whether they are testing against a Dell desktop, a Lenovo laptop, or an HP Proliant server. They obviously do care with regards to if the OS is Windows 2008, Windows XP, or Red Hat Linux.  However, if the OS supports the test case, that is what matters.

The leadership team at Northway Solutions Group spent a great deal of time and research in determining what type of partner we wanted for mobile testing solutions. Jamo Solutions was the obvious choice.  Offering JAMO as our mobile automation solution exemplifies our commitment to meeting the ever increasing needs in the Software Quality Management marketplace. It allows us to talk to our existing customers who already use HP’s QuickTest Professional and help them understand how they can extend its use into mobile applications in a pretty seamless fashion. It doesn’t require they rethink how they test applications today, so they can use the same approach, and the code for the OS is exactly the same in production.

What do you think? Should the device matter? Why or why not? We look forward to your comments.

Source:

http://northwaysolutions.com/our-work/blog/2012/05/mobile-application-testing-should-the-device-matter/