Tag Archives: document

What Should Testers Do in the Early Design Stage?

Are software testers needed in the early design stage of the products? If they are, what should they do? I often hear such questions asked by my friends. As far as I am concerned, many medium and small-sized companies in our country do not attach importance to the testing job in the early stage or even the whole process of developing products. Example one: some companies hold the view that the investment in testing during the early stage is high and meaningless, so they will start design test cases after the first period of development ends. Example two: some hold that testers do not have to participate in the requirements mapping work, they just need to accept. As a result, the marketing department and the development department will discuss about the demands directly, and the test manager will just refer to the demand design documents. Example three: the test manager does participate in the requirements design at the beginning, and work out the test plan. But the quality of the products is what the engineer deploying on the scene determines. At the user’s place, something unsatisfactory and failed will be found. To save time, the testers have to adjust the software at the user’s place, with no regard for whether the product needs an overall testing.

Not to say whether these methods are even more costing, it is easier for us to get the answer of “what should testers do during the early stage”: it will be nice if the test plan is made during the early stage, and it is okay if it is not made. But the testing should be done. It is also uncertain that what kind of product will be handed to the user. But there is only one aim: the product should be able to be used by the users, no matter for a mature or a newly started company.

For Microsoft, there are three product life cycles: Product Definition, Product Development and Product Serving. To make the best use of the resources, the testers mainly participate in the Product Development and Product Serving work. Their major duties are: to determine the priority levels of all the products’ functions; to review the definition of the product using conditions; to review the user experience documents.

Of course, every company should have its own development mode that fits itself, but there is no difference between their goals when deciding whether to let the testers do these jobs: firstly, the testers should know what the customers demand; secondly, the tester should tell what demands are reasonable and what are not according to his own experience and from the angle of testing and preserving, then gives feedbacks to the project manager or the marketing department. Lastly, the tester should make test plans according to the demands and the software framework documents.

Each company and each team of the company does not have to take the same pattern. We should act according to actual circumstances, but the focus should not vary, neither should the goal, just like some companies in our country apply the CMM blindly, which is just imitation, but with no soul.

Did you like this? Share it:

How to Achieve Level 5 Maturity for QA and Testing Process

For any process whether it is a QA process, development process or any non-technical process, there are levels of its maturity. By levels of maturity we mean that the level of formality and processes improvement, like ad-hoc processes – to formally defined steps – to managed result metrics – to optimization of the processes.

CMM (Capability Maturity Model) is process based model which is used to assess the maturity of an organization for different domains. Although this model is normally termed as the software development model but eventually it was used for other processes as well like QA and testing.

It has 5 different levels of maturity from 1 to 5. As we go towards level 5 from 1, variability and inconsistency reduces. Below are the details of 5 levels. Here we will go through the 5 CMM levels with respect to QA process and what all output/result is expected for each level to mature a QA/testing process and reach up to level 5.

CMM Levels

Level 1 – Ad-Hoc: Unplanned, unsystematic, and inconsistent

As the word ‘Ad-Hoc’ states: unplanned, unprepared, at this level significance is not given to planning, following processes, guidelines and standards. There is no standardized & consistent way of doing any task. The only thing which is important at this level is meeting the timelines, irrespective of the quality of the end product and deliverables.

As there are no pre-defined standards and processes, same task is done in different ways by different people.

And this becomes even more unsystematic and inconsistent if same task is done differently next time.

Level 2 – Control: initiate defining processes at high level:

Solution to the problem which we saw at Level 1 of unavailability of QA processes, methodology & standards would be to have all these in place. The standards and processes are not only finalized but also are well documented, so that those can be re-used by any one for similar task.

Level 3 – Core Competency: Come up with a generalized process for wider audience and domains:

At this level 3, people are motivated to follow the standards and processes defined at level 2. For this first of all the processes need to be conveyed to all people and need to identify what all skills are needed to use those effectively and efficiently and also if any training is required for that and then motivated and supported to follow those standards and processes. Here people having more experience share their knowledge with others.

Level 4 – Predictable: Measure the processes

At this level processes defined at level 3 are measured quantitatively. This is done to control the effort required on any task. Based on this quantitative analysis, processes can be adjusted if needed, and that to without degrading the quality of the end product. Analysis is done by dividing complete process into smaller sub-processes and then quantitative techniques are applied on these sub-processes and as per the result, sub-processes are adjusted if needed. This level is called predictable as based on prior experience; we can predict the process quantitatively and make use of that for the upcoming processes.

Level 5 – Innovative: Continuous Improvement

At this level, innovative ways are identified to further improve the pre-defined processes and standards. This is a continuous process. For this our own processes are watched and re-engineered continuously by adding new tools technologies, by continuous studies and by keeping ourselves updated with new information in the market. This can also be achieved by benchmarking other organizations and learn from them and try to improve our process by adding new innovations to it.

Source: http://www.softwaretestinghelp.com/achieve-level-5-maturity-for-qa-testing-process/

Did you like this? Share it:

BACK END TEST GUIDE for SQL

1. INTRODUCTION

This document is to discuss general test specification issues for SQL server back end testing and to provide testers a test design guide that includes test methodology. Most systems, i.e. Forecast LRS, Delta, KENAI, KBATS and so on, that are developed by ITG have client-server architectures. However, only a few projects have their back end completely tested.

1.1 Why back end testing is so important

A back end is the engine of any client/server system. If the back end malfunctions, it may cause system deadlock, data corruption, data loss and bad performance. Many front ends log on to a single SQL server. A bug in a back end may put serious impact on the whole system. Too many bugs in a back end will cost tremendous resources to find and fix bugs and delay the system developments. It is very likely that many tests in a front end only hit a small portion of a back end. Many bugs in a back end cannot be easily discovered without direct testing. Back end testing has several advantages: The back end is no longer a "black box" to testers. We have full control of test coverage and depth. Many bugs can be effectively found and fixed in the early development stage. Take Forecast LRS as an example; the number of bugs in a back end was more than 30% of total number of bugs in the project. When back end bugs are fixed, the system quality is dramatically increased.

1.2 Differences between back end testing and front end testing

It is not easier to understand and verify a back end than a front end because a front end usually has friendly and intuitive user interfaces. A back end has its own objects, such as, tables, stored procedures and triggers. Data integrity and protection is critical. Performance and multi-user support are big issues. Slowness in operation can be vital to the project’s future. There are no sufficient tools for back end testing. SQL language is mainly a testing tool. MS Access and MS Excel can be used to verify data but they are not perfect for testing. However, there are a large number of test tools available for front end testing. To be able to do back end testing, a tester must have strong background in SQL server and SQL language. It is relatively difficult to find testers who understand both SQL server and SQL testing. This causes a shortage of back end testers.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=17246&document_srl=590218&listStyle=&cpage=

Did you like this? Share it:

White Box and Black Box Test Cases

To write black box test cases software tester needs the requirement document and design plan. These documents are easily accessible at the very beginning of the project.

At the same time white box test cases cannot be written at the initial start of the project. It is so because white box test cases need more architecture clearness which is not accessible at the beginning of the project.

White Box and Black Box Test Cases

So it is a common situation when white box test cases are written after black box ones.

Black box test cases don’t need system comprehension but white box testing requires more structural comprehension.

Structural comprehension is more intelligible in the later stages of project.

For black box testing QA engineer needs to only analyze from the functional perspective which is easily accessible from a simple requirement document.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=12675&document_srl=588605&listStyle=&cpage=

Did you like this? Share it:

5 common solutions to software development problems

  1. solid requirements – clear, complete, detailed, cohesive, attainable, testable requirements that are agreed to by all players. Use prototypes to help nail down requirements.
  2. realistic schedules – allow adequate time for planning, design, testing, bug fixing, re-testing, changes, and documentation; personnel should be able to complete the project without burning out.
  3. adequate testing – start testing early on, re-test after fixes or changes, plan for adequate time for testing and bug-fixing.
  4. stick to initial requirements as much as possible – be prepared to defend against changes and additions once development has begun, and be prepared to explain consequences. If changes are necessary, they should be adequately reflected in related schedule changes.If possible, use rapid prototyping during the design phase so that customers can see what to expect.This will provide them a higher comfort level with their requirements decisions and minimize changes later on.
  5. communication – require walkthroughs and inspections when appropriate; make extensive use of group communication tools – e-mail, groupware, networked bug-tracking tools and change management tools,     intranet capabilities, etc.; insure that documentation is available and up-to-date – preferably electronic, not paper; promote teamwork and cooperation; use protoypes early on so that customers’expectations are clarified.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=12646&document_srl=432592&listStyle=&cpage=

Did you like this? Share it:

Testing Web application security using Google’s ratproxy

To help developers audit Web application security, Google has released an open source tool called ratproxy. It is a non-disruptive tool designed for Web 2.0 and AJAX applications that produces an easy-to-read report of potential exploits.
Ratproxy is a local program designed to sit between your Web browser and the application you want to test. It logs outgoing requests and responses from the application, and can generate its own modified transactions to determine how an application responds to common attacks. The list of low-level tests it runs is extensive, and includes:
    * potentially unsafe JSON-like responses
    * bad caching headers on sensitive content
    * suspicious cross-domain trust relationships
    * queries with insufficient XSRF defenses
    * suspected or confirmed XSS and data injection vectors
Many more tests are performed, all enumerated in the messages.list file included in the package.
The ratproxy package consists of source code, documentation, and an optional component for testing Adobe Flash content called flare-dist. A standard Make file is included, but no configure script. To build the program, simply run make:

$ make cc ratproxy.c -o ratproxy -Wall -O3 -Wno-pointer-sign -D_GNU_SOURCE http.c mime.c ssl.c -lcrypto -lssl
*** WARNING: flare-dist/flare bianry is not operational.
*** Please see flare-dist/README and update it for your OS.

Source: http://www.vietnamesetestingboard.org/zbxe/?document_srl=105685

Did you like this? Share it:

When the requirements fail

When we got a new task to test an application, or a new feature,  we start by getting to know the purpose of it. Then we read all documentation and detailed requisites. It is at this point we find possible flaws.

How does this happen? This is an interesting point – how and when we realize that something is wrong?

The first thing a tester does is to try to understand what is the purpose of it and the motivations behind it, only then starts to know about how it was done, or will be done. Immediately he starts imagine scenarios, putting questions and firing a lot of ‘What if’s.

Normally,  a Tester only goes forward on Planning and Test Execution, if he doesn’t find any problem during this first step. Sometimes it is not possible to get an answer and it is necessary to move on with some tests to get it.

I believe this skill is stronger in a Tester than in a Programmer. The Programmer is focuses in analysis and programming.

In the same way the Tester’s brain triggers thousand of test cases, trying to figure how the application will react to it, the brain of a Programmer triggers functions, procedures and how to link them, trying to find an opportunity to create a new class.

Both brains react immediately to the requisites reading. But it works in a different way to each of them, Testers and Programmers. Each one focuses on its own area. The availability of each one is different.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=51883&document_srl=497443&listStyle=&cpage=

Did you like this? Share it:

ETL & Data Test Guidelines for Large Applications

QA Guidelines for Data Warehouse Quality Verification

This document describes testing guidelines and steps for verifying data, ETL processes, and SQL during the construction, unit testing, system and integration testing of an application’s data warehouse operational tables and data mart.

1.) Verify and Maintain the Data Low Level Design (LLD)

A first level of testing and validation begins with the formal acceptance of the logical data model and “low level design” (LLD). All further testing and validation will be based on the understanding of each of the data elements in the model.

Data elements that are created through a transformation or summary process must be clearly identified and calculations for each of these data elements must be clear and easily interpreted.

During the LLD reviews and updates, special consideration should be given to typical modeling scenarios that exist in the project. Examples follow:

1. Verify that many-to-many attribute relationships are clarified and resolved.

2. Verify the types of keys that are used: surrogate keys versus natural keys.

3. Verify that the business analyst / DBA reviewed with ETL architect and developers (application) the lineage and business rules for extracting, transforming, and loading the data warehouse?

4. Verify that all transformation rules, summarization rules, and matching and consolidation rules have clear specifications.

2.) Analyze Source Data Before & After Extraction to Staging

Testers should extract representative data from each source file (before or after extract to staging tables) and confirm that the data is consistent with its definition; QA can discover any anomalies in how the data is represented and write defect reports where necessary. The objective is to discover data that does not meet “data quality factors” as described in specification

3.) Verify Corrected, Cleaned, Source Data in Staging

This step works to improve the quality of existing data in source files or “defects” that meet source specs but must be corrected before load.

4.) Verifying Matched and Consolidated Data

There are often ETL processes where data has been consolidated from various files into a single occurrence of records. The cleaned and consolidated data can be assessed to very matched and consolidated data.

Much of the ETL heavy lifting occurs in the transform step where combined data, data with quality issues, updated data, surrogate keys, build aggregates, are processed.

5.) Verify Transformed / Enhanced / Calculated Data to Target Tables

At this stage, base data is being prepared for loading into the Application operational tables and the data mart. This includes converting and formatting cleansed, consolidated data into the new data architecture and possibly enhancing internal operational data with external data licensed from service providers.

6.) Front-end UI and Report Testing Using Operational Tables and Data Mart

7.) Operational Table and Data Mart: Build Sanity Test

8.) Sanity Test: Exit and Suspension Criteria

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=17246&document_srl=595418&listStyle=&cpage=

Did you like this? Share it:

Apply QA Testing in an Agile Development Methodology

The growing popularity of utilizing the Agile Development methodology to develop systems has changed the way software is tested. Certainly, high quality is still a requirement but the approaches, techniques and artifacts have changed. In Agile development, there is a common “principle” that for a specific period called a Sprint (usually 30 days), all software will be fully developed, tested and documented and “releasable ready” for production use. The speaker will educate on the appropriate approaches, artifacts, and metrics to successfully deliver high quality products on time in an Agile methodology. This will include customer experiences to show real life examples and techniques for successful “Agile” Testing.

  1. Customer Agile Experiences
  2. QA Processes and Artifacts
  3. Automation Testing Approaches
  4. Performance Testing Approaches

Source: http://www.vietnamesetestingboard.org/zbxe/?document_srl=425387

Did you like this? Share it:

Software testing guide: software Test Plan

The following are some of the items that might be included in a test plan, again that will be depending on the project to Project: 
- Title 
- Identification of software including version/release numbers 
- Revision history of document including authors, dates, approvals 
- Table of Contents 
- Purpose of document, intended audience 
- Objective of testing effort 
- Software product overview 
- Relevant related document list, such as requirements, design documents, other test plans, etc. 
- Relevant standards or legal requirements 
- Relevant naming conventions and identifier conventions 
- Overall software project organization and personnel/contact-info/responsibilties 
- Test organization and personnel/contact-info/responsibilities 
- Assumptions and dependencies 
- Project risk analysis 
- Testing priorities and focus 
- Scope and limitations of testing 
- Test outline – a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable 
- Outline of data input equivalence classes, boundary value analysis, error classes 
- Test environment – hardware, operating systems, other required software, data configurations, interfaces to other systems 
- Test environment validity analysis – differences between the test and production systems and their impact on test validity. 
- Test environment setup and configuration issues 
- Software migration processes 
- Software CM processes 
- Test data setup requirements 
- Database setup requirements 
- Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs 
- Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs 
- Test automation – justification and overview 
- Test tools to be used, including versions, patches, etc. 
- Test script/test code maintenance processes and version control 
- Problem tracking and resolution – tools and processes 
- Project test metrics to be used 
- Reporting requirements and testing deliverables 
- Software entrance and exit criteria 
- Initial sanity testing period and criteria 
- Test suspension and restart criteria 
- Personnel allocation 
- Personnel pre-training needs 
- Test site/location 
- Outside test organizations to be utilized and their purpose, responsibilities, deliverables, contact persons, and coordination issues 
- Relevant proprietary, classified, security, and licensing issues. 
- Open issues 
- Appendix – glossary, acronyms, etc.

Source: http://www.softwaretestingfullguide.com/2011/12/software-test-plan.html

Did you like this? Share it: