Tag Archives: function

Attacking the Recovery Capability of Applications during Software Testing

Recovery testing is an important & generally overlooked technique. Instead of ignoring the inevitably of bugs, it faces them head-on by investigating how software will react in the face of a trouble. It is applicable across all phases of software testing, and is especially productive at exposing bugs on systems under heavy load and stress. It is essential that the software testing engineers give due consideration to recovery implications while developing their test plan.

Software’s ability to recover from a failure is an important contributor to its robustness. Recovery can also be one of the most interesting test focus areas. How much recovery testing is needed largely depends upon the nature of the target program, as well as the operating system environment it will operate within.

At the other end of the spectrum are environments in which an application that fails is simply expected to crash and the entire operating system will need to be rebooted before the application can be restarted cleanly. Most software lies somewhere in between.

Various forms of recovery testing covers Function Verification Test (FVT), System Verification Test (SVT) and integration test disciplines.
Here in this post I am discussing the Function Verification Test (FVT) & System Verification Test (SVT).

A) Methods of attacking programs during Function Verification Test

According to the situation, there are many different ways in which we can attack a program’s recovery capabilities during FVT. Few of them I am describing below. However before we can check how well a program recovers from an error, we need a way to generate that error in the first place.
Some of the options I am describing here are given below.

Option –1: By using Special Tools and Techniques
In some cases, an error can be easily created through external means, such as filling up a log file or killing a process. But many times such techniques aren’t enough during FVT. As a software testing engineer we need to simulate a bad parameter being passed from one module to another, or force an error interrupt to occur just as the module reaches a critical point in its processing. It may not be obvious to us how to go about injecting such errors, but several techniques are available to us.

a) Stub Routines:
If we need to force another module or component to pass bad input into our target software, we need to replace that module with a small stub routine. The stub routine will do little more than accept incoming requests, then turn around and reply to them in a reasonable way. However, it will purposely corrupt the one parameter we are interested in. Alternatively, rather than replacing a module with a stub we can tamper with the module itself, altering it to pass back bad data when called by our target software.

These approaches will only work if the module we intend to "stub out" is called infrequently under conditions, which we can externally generate. Ideally, it would only be called by the module under test. We don’t want to insert a bogus stub routine that will be invoked millions of times per second for routine tasks by many other modules in the component. If we do, its identity as an impostor will quickly be revealed and the software will surely stumble. This stubbing approach obviously creates an artificial environment, so it’s probably the least desirable method listed here. But under the right circumstances, it can be useful.

b) Zapping Tools:
Some systems have tools that allow the software testing engineer to find exactly where a particular module is loaded in memory on a running system, display its memory, and change bytes of that memory on the fly. This dynamic alteration of memory is called a zap. If we can’t find such a tool for the system we are testing on, we can consider writing our own. We will probably find that creating a crude zapping tool is not a major undertaking.

A zapping tool gives us an easy means to selectively corrupt data. We can also use it to overlay an instruction within a module with carefully constructed garbage, so when that instruction is executed it will fail. As with the stub routine case, care must be used not to meddle in an area that is frequently executed on the running system, or the volume of errors we will generate will be overwhelming. However, zapping is not nearly as artificial a technique as stub routines. In the right situations it can be very effective.

c) Error Injection Programs:
Another approach is to create a small seek-and-destroy program to inject the desired errors into the system. To create such a program we must first determine exactly what error we wish to inject by studying the target software. Let us say the module in question maintains a queue of pending requests, and a counter which indicates the current length of the queue. When the module scans the queue, it relies on this counter to determine if it has reached the end. We decide to corrupt that counter so that the queue scanning code will fall off the end of the queue and throw an error.

To implement this plan, software testing engineers write a small program that operates with full system privileges. It follows a chain of system control structures until it locates our target module in memory. Our program establishes addressability to this module’s dynamic area (i.e., access to its variables), examines the current contents of the counter variable, doubles it, and then exits. The next time the target module tries to traverse the full queue, it’s in for a surprise.

This is a simple example, but try to imagine other cases where our error injection program corrupts the contents of a control structure shared by multiple modules within a component, or performs other nasty deeds. In essence, this is nothing more than automating the function of a manual zapping tool. But because the seek-and-destroy program is operating at computer speeds, it can be much more nimble and precise in its attacks.

d) Emulators and Hypervisors:
Through things called emulators and hypervisors, it’s possible to create what is known as virtualized environments. For this discussion all we need to realize is that they create another layer of software between an operating system and the hardware it runs on. In some implementations, this extra layer has special debugging capabilities that can be used to set breakpoints. These breakpoints can freeze the entire system when triggered. This gives the software testing engineer an opportunity to stop the system at a specific point, corrupt memory or register contents, then restart it and watch the recovery support take action.

This is quite different from the sort of breakpoint function available in interactive debuggers, which can create a very artificial environment. In virtualized environments, the operating system and all of the middleware and applications running on top of it are unaware of the existence of this extra layer. When a breakpoint is hit, the entire system stops not just one module. At that point, the virtualization layer hands control over to the software testing engineer.

Such technology is not universally available. But if we have access to a virtualized environment that supports break-pointing capabilities, it probably offers the most powerful mechanism for injecting errors during FVT.

Option –2: Enabling the Restartability of Program
The most basic recovery option is enabling a program to restart cleanly after a crash. In FVT, the focus is placed on failures within individual components of the overall product. We will generally need to trick a component into crashing. We can do this in a virtualized environment by setting a breakpoint at some specific location in its code. When the breakpoint hits we can insert carefully corrupted data, set the system’s next instruction pointer to the address of an invalid instruction, or zap the component’s code itself to overlay a valid instruction with some sort of garbage that’s not executable. We then resume the program after the breakpoint, watch it fail, and ensure it generates the appropriate failure messages, log entries, dump codes, etc. If it has robust recovery support, it may be able to resume processing as if nothing had happened. If not, it may force the entire product to terminate.

If the program terminates, software testing engineer can then restart it and determine if it restarts successfully and is able to process new work (or resume old work, depending on its nature). If we resorted to zapping the component’s code with garbage to force it to crash, and that code remains resident in memory, then we will need to repair the overlay prior to restarting the program (or it will just fail again).

Option –3: Using Component level Recovery out of Anticipated Errors
Most commercial software has some sort of component-level (or object-level) recovery, whether it is operating system-managed, or more basic signal try-and-catch mechanisms employed by some programming languages. At a high level, the idea is to establish a recovery environment around a chunk of code, such that if an error interrupt (e.g., program check, I/O error) occurs, the recovery routine will be given control to take some sort of action. That action could be as simple as issuing an error message. Or, it could be as complex as generating a memory dump, logging or tracing the error, releasing program-owned resources and serialization, and freeing held memory. It might even restore overlaid data in key control structures and retry the failed operation.

There may be a long list of anticipated error types for which the recovery routines take unique actions. At a minimum, our FVT plan should include scenarios for forcing each of those errors. After each error, we need to ensure the recovery code processes them correctly. It should issue the correct error messages, trace entries, log records, generate a valid memory dump, or perform whatever action the code is designed for. When choosing locations within a component to inject errors, prime consideration should be given to points where memory is obtained, shared resources are in use, or serialization mechanisms (e.g., locks, mutexes) are held. These areas are complicated to handle properly during recovery processing, and so are good grounds for test exploration.

Sufficient Diagnostic Data:
Our test plan should also include an attempt to verify that any error information generated is sufficient for its intended purpose. If a message is presented to the end user, is there enough information so the user can make an intelligent decision about what to do next? Or, if there’s no reasonable action the user can take, is the message necessary at all or will it just lead to needless confusion? If diagnostic data is generated, will it be sufficient to determine the root cause of the problem? This is where we go beyond simply testing to the specifications, and instead determine in a broader sense if the function is "fit for purpose." As a software testing engineer, we bring a different perspective to the table than does the developer. We need to be sure to leverage that perspective to ensure the program’s actions are useful and helpful.

Option –4: Using Component-level Recovery out of Unanticipated Errors
A thorough test plan will go beyond errors that the program’s recovery support was coded to handle. It will also investigate how the program responds to unanticipated errors. At a minimum, the code should have some sort of catchall processing for handling unknown errors (if it doesn’t, we may have found our first bug). We need to be a little devious here. We need to use the instruction zapping approach if necessary, but find a way to force the code to react to errors it hasn’t attempted to address, and then ensure it reacts reasonably. Again, software testing engineers use their own end-user view to determine what "reasonably" means for this program.

Also included in this category are errors that occur at the system level but also impact the individual component. These errors can include memory shortages, hardware element failures, network problems, and system restarts. Force or simulate as many of these types of errors as seem relevant, and discover if the component handles them gracefully – or if it takes a downward nosedive.

B) Methods of attacking programs during System Verification Test (SVT)
The objective of SVT is also similar to FVT, namely to wreak controlled havoc and see how the software responds. But in SVT, the focus shifts from a narrow, component-level view to an entire product view. It also folds load / stress into the picture. This is critical, because it’s common for recovery processing to work perfectly on an unloaded system, only to collapse when the system is under heavy stress.

Restartability: In System Verification Test, there are two aspects to restartability.

1) Program crash
2) System crash

1) Program crash: Here, because we are operating at an end-user level in which techniques such as setting breakpoints are not applicable, there must be an external way to cause the program to fail. Such external means could include bad input, memory shortages, or a system operator command designed to force the program to terminate fast and hard. Alternatively, input from software testing engineers during the software’s design might have led to the inclusion of special stability features that can aid with error injection.

An advantage to using external means to crash the program is that we are able to send normal work to the program so it is busy doing something at the time we force the crash. Programs that die with many active, in-flight tasks tend to have more problems cleanly restarting than idle ones do, so we are more likely to find a bug this way.

2) System crash: This case is similar to program crash, except that any recovery code intended to clean up files or other resources before the program terminates will not have a chance to execute. The approach here should be to get the program busy in processing some work, and then kill the entire system. The simplest way to kill the system is simply to power it off. Some operating systems, like z/OS, provide a debugging aid that allows a user to request that a particular action be taken when some event occurs on a live system. That event could be the crash of a given program, the invocation of a particular module, or even the execution of a specific line of code. The action could be to force a memory dump, write a record to a log, or even freeze the entire system. In z/OS, this is called setting a trap for the software. If such support is available, then another way to kill the system is to set a trap for the invocation of a common operating system function (like the dispatcher), which when sprung will take the action of stopping the system immediately so we can reboot it from there.

After the system reboot, restart the application and check for anomalies that may indicate a recovery problem by watching for messages it issues, log entries it creates, or any other information it generates as it comes back up. Then send some work to the program and ensure it executes it properly and any data it manipulates is still intact. Restartability is the most basic of recovery tests but, if carefully done, will often unearth a surprising number of defects.

Clustered System Failures:
Some software are designed to operate in a clustered environment to improve its scalability or reliability characteristics. Devise scenarios to probe these capabilities.

For example, consider a group of Web application servers clustered together, all capable of running the same banking application. An additional system sits in front of this cluster and sprays incoming user requests across the various systems. If one server in the cluster fails, the sprayer should detect the loss and send new work elsewhere. We can try crashing a server and restarting it, all the while watching how the remaining systems react. Another scenario might be to crash several members of the cluster serially before restarting any of them, or crashing multiple members in parallel

# What if the sprayer system crashes?

# Does it have a hot standby that will take over to keep work flowing? Should it?

# Does any data appear corrupted after completion of the recovery process?

# All such possibilities are fair game for the wily tester.

Environmental Failures:
Depending on the nature of the software under test, it may need to cope with failures in the underlying environment. In the case of operating systems, this usually means failure of hardware components (e.g., disk drives, network adapters, peripherals). For middleware and applications, it usually means the failure of services that the operating system provides based on those hardware components.

# What happens if a file system the application is using fills up, or the disk fails?

# What if a path to a required Storage Area Network (SAN) device fails or is unplugged by a careless maintenance person?

# What if a single CPU in a multiprocessing system fails?

# Are there any cases in which the operating system will alert the application of environmental failures?

# How does the application respond to such information?

Even if the application has no specific support for such events, it may still be worthwhile to see how badly it is compromised when the unexpected happens. In the mainframe world, tools are used to inject error information into specific control structures in memory on a running system, and then force a branch to the operating system’s interrupt handler to simulate the occurrence of various hardware failures. Similar tools can be created for Linux or other operating systems. This sort of testing is very disruptive, so unless we have our own isolated system, we will need to schedule a window to execute these scenarios to avoid impacting everyone else’s work.

Natural Failures:
During the course of normal load / stress or longevity runs, the software being tested will almost surely fail on its own, with no help from the software testing engineer.

# Rather than cursing these spontaneous, natural errors, take advantage of them.

# Do not look only at the failure itself; also examine how the program dealt with it.

# Monitor recovery processing to see how the software responds to unplanned failures.

source:http://www.softwaretestinggenius.com/attacking-the-recovery-capability-of-applications-during-software-testing

Did you like this? Share it:

Software Testing Using Function/Inverse Pairs

One of the hardest problems in software testing is finding convenient, strong oracles—programs that can tell us whether the software under test worked correctly. Weak oracles, such as checking to make sure a program doesn’t crash, are always available, but not always of much use. An example of a strong oracle would be a reference implementation.

Sometimes software comes in function/inverse pairs, meaning that we get a program implementing some function f and another that implements f-1. When this happens we can (ideally) make up an input x and then check to make sure that f-1(f(x))=x. Often this works only in one direction. For example, let’s consider an assembler/disassembler pair. If I take some arbitrary assembly language, assemble it, and then disassemble the resulting object code, I’m unlikely to get exactly what I started with. On the other hand, if I take some arbitrary object code, disassemble it, and then assemble the resulting assembly code, I should get back the same object code. The question is whether or not f or f-1 is prepared to accept inputs in non-canonical forms. This problem can usually be solved by running a second cycle through the system. For example, Jesse Ruderman of jsfunfuzz fame reports taking JavaScript code in non-canonical form, compiling and decompiling it, and then doing the same thing again. If the second and third versions of the program differ, a bug has been found. I don’t know how many of the impressive 1500+ bugs found by jsfunfuzz come from this method.

Here’s a list of common function/inverse pairs that I came up with:

- pickle/unpickle, or save/load, of in-memory data
- checkpoint/restore of process or virtual machine state
- assemble/disassemble and compile/decompile
- encode/decode of data for transmission
- encrypt/decrypt
- compress/decompress, either lossless or lossy

The thing I would like to learn from you readers is what other function/inverse pairs you can think of. I feel like I must be missing many.

This topic was suggested by Alastair Reid who additionally suggested that an assembler/disassembler pair for an architecture such as ARM could be exhaustively tested in this fashion since there are only 232 possible instruction words. Very cool!

Source: http://blog.regehr.org/archives/708

Did you like this? Share it:

Heavy Load Testing

Visual Studio Team Agents 2010 consists of Test Load Agent, Test Load Controller and Lab Agent. Team Test Load Agent monitors a request for a new test from the controller by running a service locally. Team Test Load Controller is run by a service, and the service controls the test agents and reports the status and errors of the test. It can also transfer the sources into running test and data collection in logic, or has effect on the system running the testing environment again.

The main function of Test Load Agent is to cascade various machines to produce heavier load and higher stress when doing the load test. It is rarely seen that an average level PC can produce a load of 50 people, which still cannot compare with some great websites or systems with thousands or more people online. Thus, enough load and stress can be produced when we connect several server level machines, so that the results will be meaningful to us.

More than 3 computers are required when using Load Agent, and VS 2010 For Tester, Controller, Agent1, Agent2, etc. are supposed to be installed. If the number of computers is limited, you can install the Controller and Agent in the same computer, but the test will be limited, as the picture below shows.

clip_image002

As the picture above shows that, Team Agents include the Agent and Controller software, which are highly extendable and customizable, so that the testers enjoy great flexibility. Testers can do load testing directing at Web apps and organizations to improve service quality by measuring the performance of Web apps and servers under load more accurately.

Application & Practice: to modify and test x64 bits CLR program set with VS 2010

Visual Studio Team Test 2010 Load Agent testing platform provides host process for the test. You can enjoy the new feature that network emulation has been introduced for performance testing, and what’s more, Visual Studio 2010 has developed another feature, which is that it can test the program set in 64bits CLR. With Visual Studio 2008, the host process can only be tested in the 32bits mode, and the internal storage has been limited to 2GB. But with Visual Studio Team Test 2010, there is no such limit.

Here are the steps to modify and test x64 bits CLR program set with VS 2010.

1) Set your program set as opening your program set on “Any CPU” platform, and open the “Build” tab, then set your platform target as “Any CPU”.

clip_image004

2) Open test settings, and set the host process platform as “MSIL”. Open the “Local.testsettings” to set and then choose “Hosts” and at last choose the MSIL under the Host Process Platform options.

clip_image006

Tips:

If you have not installed Test Load Agent, you will find that only one CPU will be occupied by the local requests when doing performance test with VS 2010. It is because the process to generate stress is VSTESTHost.exe. Accordingly, if you want every CPU of a multinuclear CPU to produce stress, you have to use Visual Studio Team System Test Load Agent.

Did you like this? Share it:

Test Strategy versus Test Plans

Test Strategy is part of the Test Plans. Test Plans describes who will do what? Describes roles and responsibilities of test team, resources required any risks associated, major functions to be tested. Test Strategy is a very important part of the test plan. When requirements analysis phase is finished and requirements are somewhat mature then testing team starts preparing test plans, how to carry on testing and what strategy to be followed.

In very simple precise terms there are two basic testing strategies:

 To test the software in its entirety, once the completed package is available; otherwise know as “big bang testing”

 To test the software piecemeal, in modules, as they are completed (unit test); then to test groups of tested modules integrated with newly completed modules (integration tests). This process continues until all the package modules have been tested. Once this phase is completed, the entire package is tested as a whole (system test). This testing strategy is usually termed “Incremental Testing”

Incremental Testing is further divided into two classes Bottom-up Testing Strategy and Top-Down Testing Strategy.

Did you like this? Share it:

Glass Box: The Next Phase of Web Application Security Testing?

For years, security researchers have used the terms "black box" and "white box" to describe dynamic and static web application security analysis, respectively. IBM is now seeking to combine the best of both approaches by introducing a new approach called "Glass Box."

"We use the terms ‘black box’ and ‘dynamic analysis’ interchangeably, and basically that’s looking at a functioning application in a web browser and evaluating its state to identify potential vulnerabilities," Patrick Vandenberg, program director for IBM Security, told InternetNews.com. "’Static analysis’ we use interchangeably with white box testing and that’s looking at source code before it is compiled to root out potential vulnerabilities."

IBM historically has provided black box testing by way of its AppScan portfolio. AppScan was expanded in 2010 with a source code edition that can do static, white box analysis.

With its latest release of AppScan standard edition 8.5, IBM is now taking that capability one step further by introducing the new Glass Box approach. With Glass Box, AppScan installs agents on a server to instrument the code, while also applying dynamic analysis techniques.

"In so doing we’re getting the real-world validation that you get from black box testing as well as getting inside the box, and that delivers phenomenal improvements in accuracy," Vandenberg said.

When it comes to root cause analysis using Glass Box, Vandenberg noted that users are limited in what they can see from an instrumentation perspective. That said, Vandenberg added that the system is able to provide coverage for all the vulnerabilities that a user would be able to find from a static analysis perspective within the context of a web application.

Full static analysis is still required for non-web applications as well as from a process perspective.

"Really the root cause is sitting in development where all these vulnerabilities are first introduced to the code," Vandenberg said. "You want to find those flaws as early as you can."

IBM also has production software capabilities with its Tivoli software division that could benefit from the enhanced security analysis that Glass Box can provide.

"We can push the vulnerability data there so that policies can be tuned and pushed out to all the devices that are being managed," Vandenberg said.

Source: http://www.esecurityplanet.com/network-security/glass-box-the-next-phase-of-web-application-security-testing.html

Did you like this? Share it:

Test Script Language Part-1

TSL stands for “Test Scripting Language”.TSL is procedural language-similar to C programming language.TSL is having different catagory to perform different tasks.It has constructs,like statements,comments,variables,mathematical operators,control statements,and functions.

Feature list and description:

Comments
Allows users to enter human readable information in the test script that will be ignored by the
interpreter.
Naming Rules
Rules for identifiers within TSL.
Data Types
The different types of data values that are supported by the language.
Data Storage
Constructs that can be used to store information.
Operations
Different type of computational operations.
Branching
Avoids executing certain portions of the code unless a condition is met.
Loops
Repeats execution of a certain section of code.
Functions
Group of statements used to perform some useful functionality.

Basically this language is created by Mercury Interactive to support their own products.All the recorded scripts that WinRunner/Loadrunner/X-runner creates when you perform an operation is in TSL code syntax. Keep in mind that while mastery of TSL is not required for creating automated tests, knowledge of the language helps to enhance the recorded tests and to create highly sophisticated tests. Skillful usage of TSL can limit the need for manual intervention when running your test. It can also make your test less errorprone.The TSL language is very compact containing only a small number of operators and keywords.A very interesting thing about TSL is that it is really easy to understand if you have little amount of knowledge in programming language.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=12652&document_srl=593121&listStyle=&cpage=

Did you like this? Share it:

Security of Mobile Communications

Wireless networking has witnessed a tremendous growth in recent years. Wireless networks offer attractive flexibility and coverage to network operators and users. Ubiquitous network coverage, for local and wide areas, can be provided without the cost of deploying and maintaining wired-based infrastructures. The current wireless networks are based on the concept of (radio) cells, which divide a telephone service zone into small areas for efficient use of low-power transmitters with minimum interference. The terminal devices are generally unintelligent terminals, meaning that the call management, routing, and other services are handled in the network. The terminals can move from one area to another, requiring their calls to remain uninterrupted during mobility.

A simplified architecture of the wireless network is depicted in Figure 1.1. When a mobile terminal is turned on, it locates a nearby cell, which tells it what radio channels and what transmission power to use. As the terminal moves, it will repeat this process as it enters new cells. The mobile terminals identify themselves to the network using a unique identifier (ID), having two components that we denote, temporarily, by the electronic serial number (ESN) and the mobile identification number (MIN). While the first component physically identifies the terminal, the second identifies the terminal holder. The second component is actually the telephone number. The ESN and MIN are continuously validated by the network.

A common attack on the communication systems, to obtain fraudulent services, is called cloning attack, in which the ESN and MIN are duplicated in another terminal and submitted to the system by the other terminal. To perform its tasks, a mobile communication system implements several fundamental functions, including call management, call billing, call routing, data communication, and data protection. For billing needs, the mobile communication system handling the communication connection is responsible for sending to the communication’s service provider a billing record, translating the raw connection details, according to rates and plans, into particular charges (for the MIN holder). The signal carriers, involved in the connection establishment, set up accounts with each other for the services provided on the other networks. End users typically pay a combination of monthly service charges and per-minute charges, although prepaid plans with only per-minute charges are becoming more popular.

Source: http://www.vietnamesetestingboard.org/zbxe/?document_srl=516943

Did you like this? Share it:

The False Sense of Database Security

In many web sites, passwords are usually stored in a database. I think this is well known to anybody who ever created any web page with user accounts.

Storing passwords in plain text is considered harmful. The only reason that I find is that when someone gets the access to the database, or has the database backup, the passwords are just plain text, so everyone can read them.

Quite nice solution to this problem is hashing the passwords. The database stores no the plain text passwords, but the result of some hash function.

1
hashed_value = HASH_FUNCTION(password);

The hash function is a one way function, so you can use it for encrypting the password. Decryption is impossible, what’s more, you could probably find many different passwords that have the same hash value. The mostly used hash function is, unfortunately still, md5(). It returns 128 bit hash value. It means that there are only 2^128 possible values… what is a quite huge number: 340,282,366,920,938,463,463,374,607,431,768,211,456. The md5 algorithm contains some flaws and currently is not so secure.

Hash functions can be “decrypted” using the rainbow tables. This is not normal decryption, so we get the original plain text password. This kind of decryption allows to find a password that gives the same hash value that we have. This can be some other text, not the one that user entered, but it will be enough for logging into a system.

This problem was partly solved using a ‘salt’, this is some random text that is added to the password before calculating the hash value. So the function now looks like:

1
hashed_value = HASH_FUNCTION(password + salt);

There is no decryption of the password in databases. Instead the password that user provides for logging in, is hashed using the same algorithm and the same salt, and the hash value is compared to the values stored in the database. There are some collissions, so two different strings generate the same hash value, so there is a possibility that you can login to a system using other password than user provided.

Source: http://simononsoftware.com/the-false-sense-of-database-security/

Did you like this? Share it:

From manual testing to test automation

Costs associated with software testing

The process of software development is complicated and can last years, there can be and there will be some bugs, mistakes, errors, functions that stop working under certain conditions etc. Sure, there are better testers and users that help to find bugs in software, but the cost of having version of software with bugs is too high.

Software testing might be a minor problem application with a army of free testers and beta-tester, but there also expensive corporate software applications that cannot be released without full testing.

There are different method for software testing – manual testing or automated testing, but the answer about if company should test its software is always "Yes", because costs of having software with bugs available in the market are too high.

Advantages and disadvantages of the black box

There are different concepts about how to test software. Some approaches involves unit tests or functional tests. Software developers usually do these tests in the beginning of the development process. Once the first version of application was released, it is a good idea to start with black box testing.

When tester do a Black Box Testing he don’t know about any functions or units inside the application. He only knows how the application should work, e.g. what should user do and what results he will have.

The main advantage of Black Box Testing is that:

Tester will act exactly as user will do. Tester will click exactly the same buttons and work with exactly the same objects. Thus, the black box testing is a testing from user viewpoint.

The main disadvantage of Black Box Testing is that:

Black box testing is hard to automated, while functional or unit tests can be completed in just few seconds, black box testing requires tester to click buttons, enter texts and work with software graphical interface (GUI). RoutineBot software was designed to make software black box testing faster.

So, how to do black box testing in a case of software? We can talk about three methods: manual testing, "record and playback" testing, automation test.

Source: http://www.comparesuite.com/solutions/tests-automation/from-manual-testing-to-test-automation.htm

Did you like this? Share it:

What Is The Real PostgreSQL Triggers Ordering

During the last database training that I led, I was talking about selecting rows from database and sorting them (including some problems with collations). Later I was talking about triggers. For this entry assume that there are only row level triggers.

When you perform e.g. INSERT in PostgreSQL, there are fired triggers (if they exists, of course). At the very beginning there are fired all BEFORE INSERT triggers. Then INSERT is made, and later all triggers AFTER INSERT are fired.

If more than one trigger is defined for the same event on the same relation, the triggers will be fired in alphabetical order by trigger name.
trigger-definition

And then suddenly I was asked:

- OK, that’s clear, but which collation is used for ordering the triggers?

Good point, I really had no idea. So let’s check that.
First of all create the function which then will be used as a trigger:

CREATE FUNCTION trigtest() RETURNS TRIGGER AS $$
BEGIN
  RAISE NOTICE '%', TG_ARGV[0];
  RETURN NEW;
END;
$$ LANGUAGE PLPGSQL;

This function has to be used with a parameter. It just prints the parameter that was passed.

Now let’s create a bunch of triggers, with different names. In PostgreSQL I can use any chars I want, even crazy ones, what usually is stupid, but I can. So let’s check them:

CREATE TABLE tab (i int);

CREATE TRIGGER "ąą" BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('ąą');
CREATE TRIGGER "$$" BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('$$');
CREATE TRIGGER "[]" BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('[]');
CREATE TRIGGER "AA" BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('AA');
CREATE TRIGGER "aa" BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('aa');
CREATE TRIGGER "aA" BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('aA');
CREATE TRIGGER "Aa" BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('Aa');
CREATE TRIGGER "b"  BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('b');
CREATE TRIGGER "B"  BEFORE INSERT ON tab FOR EACH ROW EXECUTE PROCEDURE trigtest('B');

The best way to check the ordering of the triggers is to make a simple insert:

# INSERT INTO tab(i) VALUES(0);
NOTICE:  $$
NOTICE:  AA
NOTICE:  Aa
NOTICE:  B
NOTICE:  []
NOTICE:  aA
NOTICE:  aa
NOTICE:  b
NOTICE:  ąą
INSERT 0 1

The collation used for ordering the triggers is the normal POSIX locale, as describedhere.

So beware when the definition says “triggers will be fired in alphabetical order by trigger name”. The order is the same all the time, the collation you use doesn’t really matter.

Source: http://simononsoftware.com/what-is-the-real-postgresql-triggers-ordering/

Did you like this? Share it: