Tag Archives: functional

Maximizing the value of cloud-based development and testing environment

Historically, development and testing environments have been built and managed at the project level, and often remain underfunded, under-resourced and underutilized for significant periods of time. The development and testing demand and the IT infrastructure management processes differ in their DNA. Development and testing is unpredictable and has variable demand cycles while the IT managers look at smoother predictable operations, gradual capacity building and higher utilization. Despite being a crucial IT function, the inability to quickly provide the capacity needed by development and testing teams delays the application development life cycle and hampers the delivery of an application quickly and efficiently.

As the pace of change and the level of competition is growing, businesses today need agile IT environment to match the highly dynamic and resource intensive needs of the application development and testing – a business critical function.

According to Gartner, cloud and mobility will drive the worldwide application development market to exceed USD 10 billion in 2013. By leveraging cloud, developers, test engineers, and QA teams can develop and perform extensive scenario testing in shorter cycles. Here’s how:

Cloud provides developers and test engineers with a self-service model for requesting and almost instantly receiving resources from within a pool of secured, shared and scalable infrastructure resources. This capability can shave days or even weeks off of application development project times, speeding time to market. Cloud also enables these teams to build configuration templates and machine snapshots in seconds, run them in parallel, and customize them to meet the needs.

Test engineers can quickly deploy configurations and scale performance on-demand heavy load testing, saving time and operational costs over traditional on-premise development and testing environments.

Benefits of moving development and testing to the cloud include:

• Achieve faster time-to-market and greater flexibility for new products and services.
• Automate approval workflows and reduce the cost of IT infrastructure management.
• Enhance ID & Access control and safeguard data with a private or a hybrid solution.
• Utilize infrastructure capacity efficiently with granular monitoring and management of infrastructure resources.

Considerations for cloud-based development and testing

While development and testing in a cloud-based model addresses the traditional roadblocks of cost, scalability, and lack of process and methodology, it has its own challenges:

Security & Control – Businesses may have applications that need to comply with regulatory and corporate restrictions around security and data privacy for e.g. access control for offshore and sub-contractors or an applicable local law that mandates compliance to data residency and hence restricts usage of a public cloud for data/devices. These may not even move to an off-premise cloud instance for development and testing because of proprietary/legacy systems, as well as intellectual property security considerations. In addition, control & governance mechanism needs to be set for integrating workflows, identity management, usage metering, chargeback, etc. to ensure efficiency and quality.

Interoperability – Businesses may be confronted with issues surrounding legacy systems development & testing on the cloud as connecting to legacy systems from the external cloud may pose interoperability issues. The ability to integrate with existing systems and share data between different platforms may need multi-tier technology architecture.

Performance – As development and testing environment on cloud maybe shared by numerous users, there may be cases where businesses may have to wait for the required bandwidth. Uptime is an important consideration when developing and testing on the cloud to assess the performance characteristics of an application. The IT admins have to ensure that underlying hardware provides adequate performance levels across storage, network and compute in a private cloud, while such tweaking may not be available in a public cloud.

Monitoring – Monitoring of application, which is in a distributed format, one spanning multiple servers, on multi-cloud environment or accessing multiple applications through web services, becomes difficult from a performance, security and availability perspective. A full-featured logging and tracing mechanism for troubleshooting becomes imperative. Measuring the cloud utilization by various teams and business units enables better capacity planning for future.

Management – Servicing and managing development and testing environment has been challenging because of the bursty workloads and the dynamic service requests. The current processes are designed around current IT service delivery models. The processes such as provisioning, procurement, configuration and de-provisioning of the resources are manpower intensive at the transactional levels and automation is limited by the technology. While cloud provides a wide variety of build/integration systems, test harnesses, and development and testing tools, there is still a need to bring all of this together in a turnkey and managed model to reduce the burden of managing development and testing infrastructure on cloud.

Developing for the cloud

– Developing applications to run on cloud is different from developing applications for a traditional or virtualized IT environment. Developers must build applications that consider resource unavailability and is able to recover from such incidences. For e.g. a multi-tier application should be loosely coupled and ready for any other tier failure. The application should be built in a way that allows multiple instances of a component to run concurrently so that in case an instance fails, the components could easily switch to another instance.

Critical Success Factors

A thorough planning and selecting the right technology and cloud service provider must be done in order to maximize the value of the cloud-based development and testing environment.

To understand cloud-ready environments, some key architectural requirements of a cloud-based development and testing environment must be known:

+ What hardware/compute resources will be used and will it be capable of achieving development and testing objective?
+ What resources (wiring and cabling, SAN and storage, rack space etc.) will be needed before any servers or workloads are installed?
+ What networking and data storage capabilities in terms of capacity will be required?
+ What workloads/applications will be placed in the cloud?

Businesses must make sure that the cloud-based development and testing environment is aptly architected for hybrid environment and does not lead to application performance degradation.

Once the development and testing environment in the cloud is established, businesses must take into consideration the following to ensure an agile development and testing life cycle:

+ Template library of ready-to-use VMs, defining server, capacity & storage requirements along with application components, must be created. Such templates allow team members to quickly duplicate environments and streamline provisioning.

+ Services in the cloud should be integrated with the right chargeback/metering processes and tools. This will enable enterprises with the financial thresholds and control of costs in the development and testing cloud.

A comprehensive cloud solution for development and testing provides increased control over projects, speed of deployment, ease of collaboration, and the ability to access environments on demand, enabling efficient and quality application development and testing.

Source: http://www.informationweek.in/cloud_computing/13-02-08/maximizing_the_value_of_cloud-based_development_and_testing_environment.aspx

Did you like this? Share it:

Non-Functional Software Testing

If functional software testing is the practice of verifying and validating a software package — in other words making sure the software works and does was it was designed to do — then non-functional software testing is everything else beyond that. One of the first of these “everything-else” categories is performance testing.

Performance Testing

Testing the performance of a new or existing program is the process of determining how well an application performs under actual working conditions. Performance testing was not a great concern for software developers until the rise of client/server technology. In the earlier days of computer development, most programs were standalone, independent creations that were restricted to local environments. Having thousands and thousands of users attempting to access the same information simultaneously was not a consideration.

All of that emphasis on standalone software went away with the advent of the Internet. Suddenly, a program was expected to concurrently service similar requests of any number of users. The need for performance testing became evident in a hurry.

Performance Testing Requirements

The first requirement for testing performance is that the program in question be completely functional. That means that all functional testing should have been completed successfully, and all bugs should have been corrected. Performance testing also requires that the testing be done automatically, with the aid of a service that is designed to simulate multiple users performing identical actions in a short (or, at least, defined) period of time.

Performance testing attempts to verify that an application can perform satisfactorily while serving a given number of users. Typically, the number of virtual users will be determined by the testing program but it can often be more than a thousand simulated users. Hence the requirement for automation. Endurance testing, an element of performance testing sometimes referred to as “stability testing,” verifies that the application can continue to perform for an extended period of time under conditions that are as near to real-life parameters as possible. Load testing can easily fit into this category, as it attempts to verify either the number of simultaneous users that can be serviced or the level of parallel requests for data manipulation (this can include transfer quantity and/or data modification activity).

Stress testing, another form of performance testing, attempts to find the point at which the application begins to break down or fall apart under duress. One of the important elements of stress testing is to determine how the program reacts to overload situations: does it just quit, or does it systematically shut down, sending warning messages to users and management as appropriate?

Usability Testing

Usability testing is the method of checking to see if the program is easy to use and performs things the user wants in an intuitive manner. A relatively new practice employed by software developers is to have the users perform these tests themselves. The results are solicited through the use of surveys, which contain questions such as “Did you find the program easy to use?” and “Do you have any suggestions for improvements?” This form of testing is extremely useful to the developer because it often gets to the direct root of any existing usability problems.

The term “user-friendly” comes to mind under the umbrella of usability testing. “User-friendly” often refers to the user being able to progress through the program without having to make any abstract or unexpected decisions, which could cause the program to inadvertently abort. This may take the form of yes/no questions, radio-button pushing, or having the user make a selection from a dropdown list of choices. The more user-friendly a program is, the less likely it is to fail the usability test. Of course, writing user-friendly code for such a program puts a much greater demand on the programmer.

Security Testing

The latest addition to the whole software testing scheme involves the testing of security. Again, the need for this type of testing became evident with the creation of the Internet. Hackers originally took pleasure in breaking into a system and modifying the performance of an application. Their motivation was primarily self-identification to show the world just how smart they were. As the Internet became more commercial, though, it quickly became evident that there was money to be made in the practice of hacking.

There are numerous programs and services available to scan soon-to-be-released software for security breaches. They present a risk-based analysis as to the vulnerability of the program indicating whether the defects found are in need of immediate attention, or if they can possibly wait for the next release. At any level, they merely point out the security problems – it’s still up to the developers to come up with the solutions.

The difficulty with testing security is that it involves the human element. Passwords, personal questions, or other means of authenticating the user must be safeguarded by the user themself. If the user is not dutifully inclined to protect said security information, then all the security testing in the world is for naught. Writing passwords down, giving passwords to others, or just leaving a computer open and unattended are invitations for a security disaster.

Wrapping It Up

I hope you have enjoyed and gotten something out of this exploration on non-functional software testing. Obviously, my approach was merely an overview of the whole concept. Entire books (emphasis on the plural) have been written on the subject. My intent was to present you with the major facets and, hopefully, spark some interest in the subject.

Source: http://buildmobile.com/non-functional-software-testing/

Did you like this? Share it:

Watir Trial in Software Testing

Watir, shoft for “Web Application Testing in Ruby”, is an automated functional testing tool based on web mode. It is simple and flexible, and allows you to write tests that are easy to read and maintain. Watir keep good maintenance for its website user manual, sample code and FQA.

Now let us have a simple look at its trial:

1. Script Example

It is very convenient to use Watir to write script. The following example is the script of Watir comes with a period of testing Google’s search function:

require ‘watir’ # the watir controller
# open the IE browser
ie = Watir::IE.new
# Step 1: go to the test site: http://www.google.com
ie.goto (http://www.google.com)
# Step 2: enter ‘pickaxe’ in the search text field
ie.text_field(:name, "q").set("pickaxe") # q is the name of the search field
# Step 3: click the ‘Google Search’ button
ie.button(:name, "btnG").click # "btnG" is the name of the Search button
# Actual Result: Check that the ‘Programming Ruby’ link appears on the results page
if ie.contains_text("Programming Ruby")
puts "Test Passed. Found the test string: ‘Programming Ruby’. Actual Results match Expected Results."
else
puts "Test Failed! Could not find: ‘Programming Ruby’"
end
# End of test: Google search

This script is to open Google home page, then enter “pickaxe” in the sole text box of Google. Next, Press the “Google Search” button and after that verify pages of search results whether contains the “Programming Ruby”. Finally, according to results, use puts function to print different information on the screen. In the script, the part behind “#” is the content of comment.

2. Needed Environment

Ruby: It is implemented by Ruby and the script is Ruby script as well, so you need to install Ruby in the machine.

Watir

WINDOWS 2000 or XP + IE 5.5 later

3. Necessary Knowledge Background

Development experience, for any language, is required to write code. If you wrote VBScript or other code, you would find Watir is very simple. However, you need to know Web development, such as the basic syntax of HTML. As we need to determine the means of identification of Web objects according to Web page’s source code when developing script. Of course, you can make use of other tools to achieve, such as DOM Viewer provided by Mozilla Firefox.

In addition, basic knowledge of testing and automated testing is required as well.

4. Script Generation

Write Watir script is much more convenient and quicker than tool recording of QTP or Rational.

5. Identification, Operation and Automatic Verification of Object

Watir provides support for the identification and operation of a variety of common Web objects, such as Hyperlinks click, Checkboxes selected and remove, Radio Buttons selected and remove, drop-down box and list box selection, text box input, a variety of button clicks and Frame access, pop-up window control, etc. Of course, we can access these objects to make automate the validation of test results.

6. Other Features

Combined with other tools, many features can be extended on the bisis of Watir framework, such as external file read, modular development, reusable libraries, data driven, keyword driven, script version control, test results management, and so on.

Did you like this? Share it:

White Box and Black Box Test Cases

To write black box test cases software tester needs the requirement document and design plan. These documents are easily accessible at the very beginning of the project.

At the same time white box test cases cannot be written at the initial start of the project. It is so because white box test cases need more architecture clearness which is not accessible at the beginning of the project.

White Box and Black Box Test Cases

So it is a common situation when white box test cases are written after black box ones.

Black box test cases don’t need system comprehension but white box testing requires more structural comprehension.

Structural comprehension is more intelligible in the later stages of project.

For black box testing QA engineer needs to only analyze from the functional perspective which is easily accessible from a simple requirement document.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=12675&document_srl=588605&listStyle=&cpage=

Did you like this? Share it:

Black Box Testing. Part I

Black Box Testing is also known as functional testing. Well it is software testing technique whereby the tester doesn’t know the internal workings of the item being tested. It means that in a black box test on a software design the tester only knows the inputs and what the expected results should be. Software tester doesn’t ever inspect the programming code and doesn’t require any further knowledge of the program other than its specifications.

There are some benefits of this kind of software testing:

  1. The test is done from the viewpoint of the user, not the designer
  2. The tester doesn’t require knowledge of any special programming languages
  3. The test is impartial because the designer and the tester are independent of each other
  4. Test cases can be designed as soon as the specifications are complete

There are some disadvantages of this kind of testing:

  1. The Test Cases are complex to design
  2. The test can be redundant if the software designer has already run a Test Case

It is unreal to test every possible input stream; the reason is it would take a lot of time; consequently, many program paths go unchecked.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=12674&document_srl=565454&listStyle=&cpage=

Did you like this? Share it:

When is testing in Agile complete?

In Agile development, testing isn’t a separate phase, it’s tightly integrated with coding. In fact, we start development on each user story by writing business-facing tests that will tell us what to code and help us know when we’re done. Testers, business analysts and developers work with the business stakeholders to elicit examples of desired and undesired behavior for each user story and feature, and turn these into executable tests. This is called Acceptance Test-Driven Development (ATDD) or Specification by Example. The development team collaborates with the customers to decide which tests will prove a particular user story delivers what the customer expects. This will include automated functional tests, manual exploratory tests and non-functional testing such as performance or security testing. When those tests are all passing, you’re done with that user story.

User story estimates must include time for all testing activities, including test automation and manual exploratory testing. When we plan our iteration, we only plan the user stories which can be completed, including all testing activities. New Scrum teams often over-commit, planning more work than they can possibly finish. They end up in a mini-waterfall, with testing pushed to the end, and their features are not done just because the last day of the sprint arrived. This leads to a death spiral of stories dragging from one iteration to the next, with the testers unable to ever catch up.

Agile teams have an advantage – they include all the roles necessary to understand the customer needs and deliver high quality software. Diversity of skills and experiences allows Agile teams to find ways to help business stakeholders define their needs with concrete examples, and translate those examples into the tests which define “done” for each user story and each feature.

Source: http://searchsoftwarequality.techtarget.com/answer/The-end-of-an-iteration-When-is-testing-in-Agile-complete

Did you like this? Share it:

Brief introduction of Quick Test Professional

Quick Test Professional (QTP) is an automated functional Graphical User Interface (GUI) testing tool that allows the automation of user actions on a web or client based computer application. The features and benefits of QTP are:

1. Key word driven testing

2. Suitable for both client server and web based application

3. VB script as the script language

4. Better error handling mechanism

5. Excellent data driven testing features

Ways to handle the exceptions using recovery scenario manager in QTP:

1. Triggered Events

2. Recovery steps

3. Post Recovery Test – Run

Source: http://www.softwaretestinghelp.com/preparation-of-qtp-interview-questions/

Did you like this? Share it:

LaodUI – a free and open source laod testing tool for load testing

LoadUI is a free and open source load testing tool that allows you to do complex load tests and test the performance by simply dragging the different components around. LoadUI consist of building blocks called Components that you drag from the Component Toolbar to the Project you’re working on. Moreover, these components are connected by wires and what’s more there’s no limit for how many components that you can create and connect. Following are pro tips of loadUI:

1. Let a manager test

2. It’s interactive, use it

LoadUI is highly interactive and configurable in real-time, so take advantage of that.

3. Record your tests

Reproducing the tests is difficult as loadUI is too interactive. Hence, using a screen recorder to be able to replay your tests and see what caused the service to crash.

4. Work sturctured

LoadUI has support for test cases.

5. Don’t double-work

If you have created functional tests in soapUI, you can simply run these in loadUI with the soapUI component.

6. Write your own component

LoadUI is open-source, which means you can write your own components, in Groovy, if you wish.

Source: http://www.softwaretestinghelp.com/load-testing-using-loadui/

Did you like this? Share it:

Ways to test software requirements specification (SRS)

Most of the bugs in software are due to incomplete or inaccurate functional requirements. Hence, it’s better to catch the requirement ambiguities and fix them in early development life cycle. Moreover, to have requirement analysis and catch these incorrect requirements before design specifications and project implementation phases of SDLC is important, as the cost of fixing the bug after completion of development or product release is too high.

While measuring functional software requirement specification documents, we need to define some standard tests. Once each requirement is passed through these tests you can evaluate and freeze the functional requirements. Moreover, requirements should be clear and consistent. Then next criteria for testing the requirements specification is discover missing requirements. If the requirements are related to the project goal, we need check them.

Source: http://www.softwaretestinghelp.com/how-to-test-software-requirements-specification-srs/

Did you like this? Share it:

Software Testing and Quality Assurance Certifications

We have discussed software testing certifications before, namely CSTE, CSTP, CTM, and HP QTP Certification. Today we will talk about ISEB & ISTQB, CMST, CASQ, CSQA, and CMSQ here.

1. ISEB & ISTQB

ISEB, short for The Information systems Examinations Board, certification is the step taken forward by a chartered institute for IT called BCS, to support the career of IT professionals in IT field. It is available in foundation, practitioner and advanced level to cater the needs of each IT testing aspirant. ISEB and ISTQB are similar certifications. The only diffience is that ISEB is UK specific whereas ISTQB is international certification.

2. CMST

CMST, standing for Certified Manager of Software Testing, is meant to establish the competencies of testing professional in software industry at the interantional level.

3. CASQ

CASQ, short for Certified Associate in Software Quality, is focused on quality assurance skills and principles at the fundamental level of understanding. It provides the rapid carrer progression at a potentially affordable cost.

4. CSQA

CSQA (Certified Software Quality Analyst) certification is an intermediate level exam, considered as the atandard in best practices and principles of software quality assurance in IT industry.

5. CMSQ

CMSQ (Certified Manager of Software Quality) certification is an international level assessment for the skills in software quality assurance leaders and managers. It is the advanced level certification in software quality field.

Source: http://www.softwaretestinghelp.com/software-testing-and-quality-assurance-certifications-part-2/

Did you like this? Share it: