Tag Archives: service

HP, Wavefront Partner on Mobile Testing and Monitoring Services

Wavefront is pleased to announce it has been selected by Hewlett-Packard (HP) to act as a premier reseller of HP’s Unified Functional Testing (UFT) and Quick Test Professional (QTP) software.

As Canada’s Centre of Excellence for Wireless Commercialization and Research, this new partnership will enable Wavefront to connect large enterprise companies with best-in-class HP software for accelerated mobile application testing, automation and monitoring.

According to James Maynard, President and CEO, Wavefront, “By tightly linking HP’s UFT and QTP software with our Perfecto Mobile cloud-based mobile testing and monitoring services, Wavefront facilitates a complete end-to-end solution that improves efficiency and quality assurance for enterprise-grade mobile application testing, while accelerating time to market.”

Wavefront’s cloud-based testing and monitoring service, integrated with HP’s UFT and QTP software and Perfecto Mobile’s best in class mobile application testing solutions, can be accessed either remotely or on-site, with a direct connection to one of the country’s most comprehensive mobile device libraries.

With more than 1,000 handsets, smartphones and tablets available to test on live global networks, businesses can identify exactly how their applications will perform on actual mobile browsers, networks and devices.

“HP Canada is extremely happy and excited to announce our strategic partnership with Canada’s Centre of Excellence for Wireless Commercialization and Research – Wavefront. Wavefront’s thought leadership position around wireless communication in conjunction with HP’s recent partnership with Perfecto Mobile position HP uniquely in the Mobile app delivery space. The strategic partnership with both Wavefront and Perfecto are what will launch HP’s enterprise class testing, monitoring and security portfolio into unparalleled dominance of the mobile application delivery market,” said Frances Newbigin, Vice President and General Manager – HP Enterprise Software and Solutions at Hewlett-Packard Canada.

Source: http://www.mediacastermagazine.com/pressroom/productDetail.aspx?id=10772

Did you like this? Share it:

Hackers expose 453,000 credentials allegedly taken from Yahoo service (Updated)

Hackers posted what appear to be login credentials for more than 453,000 user accounts that they said they retrieved in plaintext from an unidentified service on Yahoo.

The dump, posted on a public website by a hacking collective known as D33Ds Company, said it penetrated the Yahoo subdomain using what’s known as a union-based SQL injection. The hacking technique preys on poorly secured Web applications that don’t properly scrutinize text entered into search boxes and other user input fields. By injecting powerful database commands into them, attackers can trick back-end servers into dumping huge amounts of sensitive information.

To support their claim, the hackers posted what they said were the plaintext credentials for 453,492 Yahoo accounts, more than 2,700 database table or column names, and 298 MySQL variables, all of which they claim to have obtained in the exploit.

"We hope that the parties responsible for managing the security of this subdomain will take this as a wake-up call, and not as a threat," a brief note at the end of the dump stated. "There have been many security holes exploited in webservers belonging to Yahoo! Inc. that have caused far greater damage than our disclosure. Please do not take them lightly. The subdomain and vulnerable parameters have not been posted to avoid further damage."

In a statement published by TechCrunch, Yahoo representatives confirmed a breach that hit the site’s Contributor Network (previously Associated Content) on Wednesday. The stolen data was contained in an "older file," and only about 5 percent of the exposed credentials were still valid on Yahoo.

"We are fixing the vulnerability that led to the disclosure of this data, changing the passwords of the affected Yahoo! users and notifying the companies whose users accounts may have been compromised," the statement continued. "We apologize to affected users. We encourage users to change their passwords on a regular basis and also familiarize themselves with our online safety tips at security.yahoo.com."

Because many people use the same credentials for multiple accounts, Ars isn’t identifying the address of the website that published the disclosure. But at time of writing, the URL wasn’t hard to find.

The TrustedSec blog is reporting that the hacked service may be Yahoo Voices, aka Associated Content. That speculation is based on the string "dbb1.ac.bf1.yahoo.com" included in the dump. The subdomain is associated with the voice service, the post said.

Article updated to reflect TrustedSec now says the compromised property is Yahoo Voices. Later updated to add official comment from Yahoo.

Editor’s Pick: Promoted Reader Comment
IntergalacticWalrus | Ars Praetorian jump to post At this point I guess we should always assume that every password we give to an online service is stored in plain text, and therefore avoid password reuse at all costs. Companies can’t be trusted to give a shit about your personal security, and lawmen and/or politicians are too fucking clueless about technology to understand that storing unencrypted passwords should be considered criminal negligence and dealt as such.

SevenFactors wrote:
Given all the resent hacks, not to mention the massive PlayStation Network incident [hopefully they learned & now are encrypting, hashing & salting] One would think that companies who know are stashing users credentials in plain text would be proactive and not wait till they get hacked to then take action.

Well what to expect, Yahoo got stuck in 1998

PSN passwords were encrypted and salted. There’s this common misconception that they were not because the initial disclosure of the attack stupidly used ambiguous terms, which they clarified later.
Last edited by IntergalacticWalrus on Thu Jul 12, 2012 8:42 am

                                                      563 posts | registered Nov 17, 2009

mohaine | Ars Centurion jump to post El Chupageek wrote:
The problem here was SQL Injection (which, btw Dan, is not caused by failure to scrutinize input but rather by NOT using prepared statements and properly binding the user input. There is a difference).

This statement couldn’t be more wrong.

Scrutinizing input (white list and/or blacklists) MIGHT stop SQL injection, but it only works if you happen to get it completely right. This damn hard with UTF and more advanced SQL engines. Proving you are doing this correctly is impossible to do. The best you can do is "Mostly Correct". Don’t trust your data to "Mostly Correct".

Property binding completely removes user input from the SQL parser, which fixes the issue with no worries.
                                                      

                                                       301 posts | registered Jun 15, 2004

source:http://packetstormsecurity.org/news/view/21233/Hackers-Expose-453-000-Credentials-Allegedly-Taken-From-Yahoo-Service.html

Did you like this? Share it:

Demand for software testers a boon to economy

The rising demand for software testing engineers can help position Malaysia as one of the leading niche players in the growing software testing business running into billions of dollars annually, says the President of the Malaysia Software Testing Board (MSTB), Mastura Abu Samah.

A software testing engineer is normally responsible for testing new computer software or programs before the product is delivered to ensure that they work properly, perform the desired functions, and are free from defects.

Mastura said the current trend in "our highly automated world is for software testing engineers to become the ‘third eye’ to identify ‘bugs’ or problems in computer systems before the procurer or purchaser takes possession of the system from the vendor for use or deployment."

"Against such a scenario, the potential of software testing engineering is tremendous with the business able to soar to a RM20 billion industry by 2020," she told Bernama in an interview.

Mastura said as European and other Western countries increasingly outsource their software testing business to the Asia Pacific, Malaysia could take advantage of the opportunities being offered to specialise in this sector and in the process enhance the national income.

Starting with only 20 software testing engineers in Malaysia more than 15 years ago, MSTB has helped to increase the number to 1,600 engineers and the board is targeting 10,000 software testing engineers in the country by 2015, she said.

MSTB is the national body representing industry interests in promoting Software Quality Assurance (SQA) and software testing as core competencies in the development of IT-dependent quality products and services.

As a member of the International Software Testing Qualifications Board (ISTQB), MSTB regulates the accreditation and certification processes for Malaysia.

"The board has set up a platform to promote the industry and we need to ensure that the applications being tested are tested well and at the same time ensure that there are multi-skilled people undertaking the tasks," said Mastura, who is regarded as a pioneer in this specialised field.

She said companies usually clamour for multi-tasking engineers who can design, test and deploy software rather than being confined to "doing just one thing."

Mastura said MSTB initiated the Malaysia Software Testing Hub (MSTH) programme, a strategic public-private partnership with the government to "pilot" an eco-system that would also identify new opportunities in the global economy against the backdrop of an innovative economic model for high-income growth.

"In other words, MSTB is undertaking one of the niche areas which our former Prime Minister Tun Dr Mahathir Mohamad had envisioned in the Multimedia Super Corridor (MSC) initiative in 1996 towards creating a multimedia hub of knowledge workers," said Mastura.

MTSB, she said, is now working with several local universities to ensure that software engineers performing such tasks would be able to "skill-up" and adapt to the different domains in the market such as banking, telecommunications or human resource management.

Mastura said MSTB hopes to help produce software testing engineers who can prove their worth anywhere in the world with their skills, adding studies have shown that these highly-skilled specialists can easily earn between RM15,000 and RM18,000 a month.

To help Malaysian software testing engineers keep abreast of the latest developments, MTSB has invited 14 renowned international experts in the field to speak at Softec, a three-day conference and workshop beginning here on Tuesday.

This year’s event, themed "The Art of Testing", focuses on the finer points of software testing, particularly on testing techniques and their effective application.

Source:
http://www.thesundaily.my/news/427696

Did you like this? Share it:

10 Free Website Speed Test Tools

Website Speed Test Tools

Page Speed Insights

PageSpeed Insights analyzes the content of a web page, then generates suggestions to make that page faster. Reducing page load times can reduce bounce rates and increase conversion rates.

website-speed-testing-1

Website Speed Check

The website speedtester shows the duration of a given website. This value can be used for showing how long a website take to load and if it is better to optimize the website.

website-speed-testing-2

Neustar Web Performance

Test your website performance with this free cutting edge tool. This tool accesses your website from different location and generates a website performance analysis report.

website-speed-testing-3

PingDom

The Full Page Test tool to help you analyze the load speed of your websites and learn how to make them faster. It examines all parts of a web page, shows performance overview and you can also share the results with your friends.

website-speed-testing-4

Load Impact

Load test your website online. We offer load testing and reporting as an online service to e-commerce & B2B sites all over the world.

website-speed-testing-5

Which Loads Faster

Pages compete head-to-head in your browser to see who’s fastest!

website-speed-testing-6

Octagate Site Timer

Web Monitor allows you to monitor how long it takes for a user to download one or more of your web site pages.

website-speed-testing-7

Show Slow

Show Slow is an open source tool that helps monitor various website performance metrics over time. It captures the results of YSlow, Page Speed, WebPageTest and dynaTrace AJAX Edition rankings and graphs them, to help you understand how various changes to your site affect its performance.

website-speed-testing-8

Web Pagetest

Run a free website speed test from multiple locations around the globe using real browsers (IE and Chrome) and at real consumer connection speeds.

website-speed-testing-9

GTmetrix

GTmetrix uses Google Page Speed and Yahoo! YSlow to grade your site’s performance and provides actionable recommendations to fix these issues.

website-speed-testing-10

Source:

http://www.flashuser.net/resources/website-speed-test-tools.html

Did you like this? Share it:

Simulation software marks its might in safety, testing

When you are transporting nuclear reactors, you have only one chance for a safe, zero-error shipment. Perkins, a specialised highway transportation contracting company realised this only too well when it had to transport four massive steam generators for a nuclear power plant from southern California to Utah.
It used a completely digital approach to create a massive transportation vehicle — a 400-foot-long truck with over 192 wheels. It began with 2D sketches of the transporter in AutoCAD software then brought alive in 3D models which were then put through a simulation software. Thus, Perkins was able to simulate every possible accident or "situation" that might have happened and optimise design accordingly to avert the worst.

Similarly, simulation software allowed a new elevator system for US Navy aircraft carriers to be tested for shocks — like those produced by underwater explosions. And the US National Institute of Science and Technology (NIST) could have safer and more effective respirator masks for firefighters.
It could digitally capture reality, using the same software, to analyse the fitting process between the respirator mask and a human head and assess airflow for the mask, in two-and-a-half days.

Augmented reality, the blending of virtual and real worlds and simulation are mainly linked with gaming, animation movies, entertainment and other consumer applications.
But digital recreation of real-world environments and situations have almost infinite applications in safety and research as well because they allow you to see and prepare for the worst, without necessarily having to go through it in reality, which is incredibly expensive. The US Navy for instance, saved hundreds of millions of dollars on those tests alone.

Moreover, simulation means that multiple situations can be dealt with simultaneously, which may not always be possible in regular testing. This is better preparation since in real life, situations rarely occur one at a time. It also aids in research because you can see things that you might not have seen otherwise.

CIM Data, a consultancy firm focused on product lifecycle management, forecasts that the simulation and analysis software market will exceed $3.1 billion by 2014.
"In as little as 10 years, all kinds of testing will be done digitally with a big focus on simulation," says Buzz Kross, vice-president, design, lifecycle & simulation at Autodesk. The design software giant has been among the strongest supporters having invested over half a billion dollars in simulation technology.

Source:http://economictimes.indiatimes.com/tech/hardware/simulation-software-marks-its-might-in-safety-testing/articleshow/13192785.cms

Did you like this? Share it:

Database Testing – Practical Tips and Insight on How to Test Database

Database is one of the inevitable parts of a software application these days. It does not matter at all whether it is web or desktop, client server or peer to peer, enterprise or individual business, database is working at backend. Similarly, whether it is healthcare of finance, leasing or retail, mailing application or controlling spaceship, behind the scene a database is always in action.

Moreover, as the complexity of application increases the need of stronger and secure database emerges. In the same way, for the applications with high frequency of transactions (e.g. banking or finance application), necessity of fully featured DB Tool is coupled.

Currently, several database tools are available in the market e.g. MS-Access2010, MS SQL Server 2008 r2, Oracle 10g, Oracle Financial, MySQL, PostgreSQL, DB2 etc.  All of these vary in cost, robustness, features and security. Each of these DBs possesses its own benefits and drawbacks. One thing is certain; a business application must be built using one of these or other DB Tools.

Before I start digging into the topic, let me comprehend the foreword. When the application is under execution, the end user mainly utilizes the ‘CRUD’ operations facilitated by the DB Tool.

C: Create – When user ‘Save’ any new transaction, ‘Create’ operation is performed.
R: Retrieve – When user ‘Search’ or ‘View’ any saved transaction, ‘Retrieve’ operation is performed.
U: Update – when user ‘Edit’ or ‘Modify’ an existing record, the ‘Update’ operation of DB is performed.
D: Delete – when user ‘Remove’ any record from the system, ‘Delete’ operation of DB is performed.

It does not matter at all, which DB is used and how the operation is preformed. End user has no concern if any join or sub-query, trigger or stored-procedure, query or function was used to do what he wanted. But, the interesting thing is that all DB operations performed by user, from UI of any application, is one of the above four, acronym as CRUD.

Database Testing

As a database tester one should be focusing on following DB testing activities:

What to test in database testing:
1) Ensure data mapping:

Make sure that the mapping between different forms or screens of AUT and the Relations of its DB is not only accurate but is also according to design documents. For all CRUD operations, verify that respective tables and records are updated when user clicks ‘Save’, ‘Update’, ‘Search’ or ‘Delete’ from GUI of the application.

2) Ensure ACID Properties of Transactions:

ACID properties of DB Transactions refer to the ‘Atomicity’, ‘Consistency’, ‘Isolation’ and ‘Durability’. Proper testing of these four properties must be done during the DB testing activity. This area demands more rigorous, thorough and keen testing when the database is distributed.

3) Ensure Data Integrity:

Consider that different modules (i.e. screens or forms) of application use the same data in different ways and perform all the CRUD operations on the data. In that case, make it sure that the latest state of data is reflected everywhere. System must show the updated and most recent values or the status of such shared data on all the forms and screens. This is called the Data Integrity.

4) Ensure Accuracy of implemented Business Rules:

Today, databases are not meant only to store the records. In fact, DBs have been evolved into extremely powerful tools that provide ample support to the developers in order to implement the business logic at DB level. Some simple examples of powerful features of DBs are ‘Referential Integrity’, relational constrains, triggers and stored procedures. So, using these and many other features offered by DBs, developers implement the business logic on DB level. Tester must ensure that the implemented business logic is correct and works accurately.

Above points describe the four most important ‘What Tos’ of database testing. Now, I will put some light on ‘How Tos’ of DB Testing. But, first of all I feel it better to explicitly mention an important point. DB Testing is a business critical task, and it should never be assigned to a fresh or inexperienced resource without proper training.

How To Test Database:
1. Create your own Queries

In order to test the DB properly and accurately, first of all a tester should have very good knowledge of SQL and specially DML (Data Manipulation Language) statements. Secondly, the tester should acquire good understanding of internal DB structure of AUT. If these two pre-requisites are fulfilled, then the tester is ready to test DB with complete confidence. (S)He will perform any CRUD operation from the UI of application, and will verify the result using SQL query.

This is the best and robust way of DB testing especially for applications with small to medium level of complexity. Yet, the two pre-requisites described are necessary. Otherwise, this way of DB testing cannot be adopted by the tester.

Moreover, if the application is very complex then it may be hard or impossible for the tester to write all of the needed SQL queries himself or herself. However, for some complex queries, tester may get help from the developer too. I always recommend this method for the testers because it does not only give them the confidence on the testing they have performed but, also enhance their SQL skill.

2. Observe data table by table

If the tester is not good in SQL, then he or she may verify the result of CRUD operation, performed using GUI of the application, by viewing the tables (relations) of DB. Yet, this way may be a bit tedious and cumbersome especially when the DB and tables have large amount of data.

Similarly, this way of DB testing may be extremely difficult for tester if the data to be verified belongs to multiple tables. This way of DB testing also requires at least good knowledge of Table structure of AUT.

3. Get query from developer

This is the simplest way for the tester to test the DB. Perform any CRUD operation from GUI and verify its impacts by executing the respective SQL query obtained from the developer. It requires neither good knowledge of SQL nor good knowledge of application’s DB structure.

So, this method seems easy and good choice for testing DB. But, its drawback is havoc. What if the query given by the developer is semantically wrong or does not fulfill the user’s requirement correctly? In this situation, the client will report the issue and will demand its fix as the best case. While, the worst case is that client may refuse to accept the application.

Conclusion:

Database is the core and critical part of almost every software application. So DB testing of an application demands keen attention, good SQL skills, proper knowledge of DB structure of AUT and proper training.

In order to have the confident test report of this activity, this task should be assigned to a resource with all the four qualities stated above. Otherwise, shipment time surprises, bugs identification by the client, improper or unintended application’s behavior or even wrong outputs of business critical tasks are more likely to be observed. Get this task done by most suitable resources and pay it the well-deserved attention.

Source:http://www.softwaretestinghelp.com/database-testing-%E2%80%93-practical-tips-and-insight-on-how-to-test-database/

Did you like this? Share it:

Security Testing of Mobile Applications

Security Testing of Mobile Applications  resized 600

 

Development of mobile applications has accelerated at a tremendous rate recently. Mobile applications are used to store personal data, perform banking transactions, make travel arrangement, enhance social media and much more. Mobile applications should also be tested from security perspective.  Compared to web applications mobile applications are more complicated to test due to the factors mentioned previously in this series. In general, mobile applications are not as secure, compared with web applications.

Mobile applications should be tested for security in the following ways:

1. Installable mobile applications

Check for modifications in registry entries, configuration settings and the creation of new files/folders in the file system, after installing the application. This should be compared it with the structure before installing the application.  Application reversing and analyzing the components helps in penetration testing of the application.

2. Browser based mobile applications

Setting up web proxies for intercepting application traffic can help viewing application behavior and to modify the data for validation and authorization.  By intercepting client server data flow, one has complete control over application data flow and can thus perform a thorough test.

3. Certificates of Trusted authorities:

Most of the mobile applications use HTTPS. They rely on the device’s certificate store to determine trusted certificate authorities. This connection is not established if the certificate is not a trusted one.

4. Application Permissions/User Permissions: For all the points mentioned below, user permissions are required.:

PIM: Personal Information Management: This includes address book, calendar events, to do, file system, key injections.

Protected APIs:  It includes Camera functions, Location data (GPS), Bluetooth functions, Phone functions, SMS/MMS functions, Network/data connections.

Cost-Sensitive APIs:  A cost sensitive API is any function that might generate a cost for the user or the network. The user will have to grant explicit permission to third-party applications requesting use of cost sensitive APIs. These APIs include Telephony, SMS/MMS, Network/Data, In-App Billing, and NFC Access.

In addition to this, if an application wants to know the user’s location, the application requires a permission to access the user’s location. Upon installation, the installer will prompt the user asking if the application can access the user’s location. At any time, if the user does not want any application to access their location, then the user can disable location based services for all applications on the user’s device.

For testing the Security of Mobile Applications, one needs to employ following methods:

Static analysis includes analysis of Source code, Source code scanning, Manual source code review, Binary Reverse engineering etc.

Dynamic Analysis includes Debugger execution, Traffic capture via proxy, Forensic Analysis, File permission analysis, File content analysis etc.

Testing the security of mobile applications is not only testing the application but also the third party web services, enterprise services as the weaknesses and vulnerabilities we find in mobile applications come mostly from interactions with supporting services.

In addition to above mentioned points, security testing involves:

Authentication and Authorization: Both these techniques should be used for restricted user access to the application.  Moreover, roles and rights management should be implemented within the different areas of the application.

Input Validation: Wherever user input comes into picture, validations must be done. All the user inputs should be verified and filtered based on the expected type. Restricted inputs should be used wherever user input is expected. The rewards of good input validation are resilience to dangerous attacks and a high level of information assurance.

Session Management: Maintaining sessions for users for specified time period helps in security of a particular user data. Requirement for timeouts of user logons must be in place for sessions. Identifying the maximum age of any given session ID as well as a timeout for sessions is essential. There is often the requirement to re-authenticate users during a session.  For example a net banking application would re-authenticate the user prior to transferring funds. This second authentication should also prompt the creation of a second session ID and the destruction of the original ID.

Encryption: All the sensitive data must be encrypted to make it secure. Encryption should be strong especially for sensitive data like passwords of user accounts, credit card numbers or other business critical information. Proper security measures must be adopted when flow of sensitive data or business critical data occurs. Whether this data floats between different modules of same application, or is transmitted to different applications it must be encrypted to make it safe.

SQL injection checks

SQL injection is an approach in which malicious script is used by the hackers in order to manipulate the application.  To check this, there should be some restriction to the input field. Also in such fields any html tags or script tag input must be prohibited. Also the application should not support anonymous access methods.

The last part of this series will deal with Reliability Testing of Mobile Applications. 

Source:http://info.nttdata.ca/bid/136831/Security-Testing-of-Mobile-Applications

Did you like this? Share it:

iPay? iPaid? Apple Testing Mobile Payments

We’re not sure what it’ll be called, but we do know that Apple has signed on with Seattle-based Pirq to offer food and drink daily deals to Bay-area employees. As far as beta testing projects go, it doesn’t get much more exciting than this.

Here are the details via TechCrunch:

The deal with Apple being announced today will see discounts of between 20 percent and 50 percent at nearly 50 venues in Cupertino, Santa Clara, Sunnyvale and Mountain View, and it is Pirq’s jumping-off point for offering a wider service in San Francisco further down the line — within the next 12 months, according to Pirq. To date, the company’s service has only been available in Seattle, where it launched in September 2011.

Pirq’s deal with Apple is partly the result of an existing relationship that both companies have with Passport Unlimited, which has been working with Apple for the past six years offering eating discounts to its employees. Pirq’s chairman, Roger Blier, is the founder and CEO of Passport Unlimited.

Although Pirq was getting ready to announce this news itself today, we actually heard about it first from a tipster at Apple, who got in touch, enthusiastically, to say how great the service was.

As the tipster pointed out, the difference between what Pirq does and what, say, Groupon offers is that Pirq pre-sets the discounts with restaurants and doesn’t require users to pay for the service upfront before redeeming it.

“I just think it’s really cool that our family gets deals like 50% without ever having to buy them like Groupon,” the anonymous tipster noted to me. (A screen shot of how the Apple offering looks is the illustration for this post.)

Did you like this? Share it:

Pen Game: A Mean to Fathom Software Testing

A handful of testers find the Pen Test an easier way to explain software testing. In a Pen Test, the presenter will hold up a pen in his hand and repeat some questions. With only two possible answers, he would like to know what makes the answer Yes and what makes the answer No.

Michael Larsen, in his blog brought out the relationship between the Pen Game and Software Testing. While testing, the testers are being presented with the program (the pen). The behavior of the testing is displayed (words and actions to display the pen) and based on the behavior, the testers can determine the state of the program.

The next step for a tester is to create a hypothesis and test it. If the result accepts the hypothesis, then we can still continue testing, however, if the results disregard the hypothesis, then we have to abandon the model or the assumptions.

The answer can be determined based on the behavior of the application. We can bring out different clues for the answer. The clues are being tried and if it doesn’t work we can replace it with another clue. Thus, coming closer to the answer.

Sometimes when we have no valid answers at our disposal, we tend to make guesses. However, if we make too many right guesses, there can be a possibility that we might create an inaccurate mental model. For example, if we are able to make four correct answers, we might think that the next answer is a yes because we have created a mental model that all answers are Yes.

However, if a game becomes too popular, it loses its potency as we tend to focus more on the answer. Therefore, a tester would need to look out for other games that can be applied for software testing.

Source:

http://qa.siliconindia.com/news/Pen-Game-A-Mean-to-Fathom-Software-Testing-nid-115076.html

Did you like this? Share it:

Load Testing: What Tool to Choose?

Classifying and evaluating load testing tools is not easy as they include different sets of functionality often crossing borders of whatever criteria are used. In most cases, any classification is either an oversimplification (which in some cases still may be useful) or a marketing trick to highlight advantages of specific tools. There are many criteria allowing to differentiate load testing tools and it is probably better to evaluate tools on each criterion separately.

First, there are three main approaches to workload generation and every tool may be evaluated on which of them it supports and how exactly.

Protocol-level recording and the list of supported protocols. Does the tool support protocol-level recording and, if it does, what protocols it supports. With quick Internet growth and popularity of browser-based clients, most products support HTTP only or a few Web-related protocols. According to my knowledge, only HP LoadRunner and Microfocus SilkPerformer try to keep up with support of all popular protocols. So if you need recording of a special protocol, you probably end up into looking at these two tools (unless you find a special niche tool supporting your specific protocol). That somewhat explains the popularity of LoadRunner at large corporations where you probably have almost all possible protocols used. The level of support of specific protocols differs significantly too. Some HTTP-based protocols are extremely difficult to correlate if there is no built-in support, so you may look for that kind of specific support. For example, Oracle Application Testing Suite may have better support of Oracle technologies.

UI-level recording. The option was available for a long time, but it is much more viable now. For example, there was a possibility to use Mercury/HP WinRunner or QuickTest Professional (QTP) scripts in load tests, but you needed a separate machine for each virtual user (or at least a separate terminal session). That limited the level of load you may achieve drastically. Other known options were, for example, Citrix and RDP (Remote Desktop Protocol) protocols in LoadRunner – which always were the last resort when nothing else was working, but were notoriously tricky to playback. New UI-level tools for browsers, such as Selenium, extended possibilities of the UI-level approach allowing to run multiple browser per machine (so scalability is limited by resources available to run browsers). Moreover, we got UI-less browsers, such as HtmlUnit, which require significantly less resources than real browsers. There are multiple tools supporting this approach now – such as PushToTest directly harnessing Selenium and HtmlUnit for load testing or LoadRunner TruClient protocol and SOASTA CloudTest using more proprietary solutions to achieve low-overhead playback. Still questions of supported technologies, scalability, and timing accuracy remain largely undocumented, so the approach requires evaluation in every specific non-trivial case.

Programming. There are cases when you can’t (or can, but it is more difficult) use recording at all. In such cases using API calls from the script may be an option. Other variations of this approach are web services scripting and using of unit testing scripts for load testing. And, of course, you may need to add some logic to your recorded script. You program the script using whatever way you have and use the tool to execute scripts, coordinate their executions, report and analyze results. To do this, the tool should have ability to add code to (or invoke code from) your script. And, of course, if tool’s language is different from the language of your API, you would need to figure out a way to plumb them. Tools, using standard languages such as C (e.g. LoadRunner) or Java (e.g. Oracle Application Testing Suite) may have an advantage here. However you should know all details of the communication between client and server that is often very challenging.

Other important criteria are related to the environment:

Deployment Model. There were a lot of discussions about different deployment models: lab vs. cloud vs. service. There are some advantages and disadvantage of each model. Depending on your goals and systems to test you may prefer one deployment model over another. But I still believe that for comprehensive performance testing you really need both lab testing (with reproducible results for performance optimization) and realistic outside testing from around the globe (to check real-life issues that you can’t simulate in the lab). Doing both would be expensive and makes sense when you really care about performance and have a global system – but it not rare and if you are not there yet, you can get there eventually. If there are such chances, it would be better to have a tool which supports different deployment models.

If it is lab or cloud, an important sub-question would be what kind of software / hardware / cloud the tool requires. Many tools use low-level system functionality, so is may be unpleasant surprises when the platform of your choice or your corporate browser standard is not supported.

Scaling. When you have a few users to simulate, it usually is not a problem. The more users you need to simulate, the more important it becomes. The tools differ drastically on how many resources they need per simulated user and how well they may handle large volumes of information. It may differ significantly even for specific tool depending on protocol used and specifics of your script. As soon as you get to thousands of users, it may become a major problem. For a very large number of users some automation, like automatic creation of a specified number of load generators across several clouds in SOASTA CloudTest, may be very handy.

Two other important sets of functionality are monitoring of the environment and result analysis. While theoretically it is possible to do it using other tools, it significantly degrades productivity and may require building some plumbing infrastructure. So while these two areas may look optional, integrated and powerful monitoring and result analysis are very important. And the more complex system and tests, the more important they are.

Of course, non-technical criteria are important too:

Cost. There are commercial tools (and license costs differ drastically) and free tools. And there are some choices in between: for example SOASTA has the CouldTest Light edition free up to 100 users. There are many free tools (some, as JMeter, are mature enough and well-known) and many inexpensive tools, but most of them are very limited in functionality.

Skills. Considering a large number of tools and a relatively small number of people working in the area, there is a kind of labor market only for the most popular tools. Even for the second-tier tools there are few people around and few positions available. So if you don’t choose the market leaders, you can’t count that you find people with this tool experience. Of course, an experienced performance engineer will learn any tool – but it may take some time until productivity will get to the expected level.

Support. Recording and load generation has a lot of sophistication in the background and issues may happen in every area. Availability of good support may significantly improve productivity.

This is, of course, not a comprehensive list of criteria – rather a few starting points. Unfortunately, in most cases you can’t just rank tools on the better – worse scale. It may be that a simple tool will work quite well in your case. If your business is built around a single web site, it doesn’t use sophisticated technologies, and load is not extremely high – almost every tool will work for you. The further you are from this state, the more challenging it would be to pick up the right tool. And it even may be that you need several tools.

And while you may evaluate tools with above mentioned criteria, it is not guaranteed that a specific tool will work with your specific product (unless it uses a well-known and straightforward technology). That actually means that if you have a few system to test, you need to evaluate the tools you consider using your systems and see if the tools can handle them. If you have many, choosing a tool supporting multiple load generation options is probably a good idea (and, of course, check it with at least the most important systems).

Source:

http://alexanderpodelko.com/blog/2012/05/10/load-testing-what-tool-to-chose/

Did you like this? Share it: