SOA has promised to have higher flexibility, longer app life cycle, better system integration and lower cost. To realize these is not an easy thing to do. Creating a complicated integrated system is not easy at all, but it requires the comprehensive application of various complicated tools according to the reality and in a creative way, to implement, test and deliver the SOA system.

SOA testing includes Service Testing, Process Testing, TDM and UI automation. It also includes some other assisting technologies, such as continuous integration testing and Service Virtualization. The testing team has to do testing to both the service provider and the client side, to make sure that the system will not go wrong. The tests have to be merged in a proper way, forming a regression testing suite. To let regression testing suite collaborate with the workflow data system is the key of SOA testing.

Challenges We Shall Meet in the Reality

One of the challenges SOA brings to the testing team is that, they have to learn new technologies, process and tools. Many testing teams always consider whether they can go on using their old tools and process in the new SOA environment. However, they finally find out that these do not work when starting the SOA testing. Here are some reasons and problems:

Need for Modern Tools

1) SOA has a unique architecture. Can those old tools test suites without user interfaces? Can those tools deal with subscriptions to agent components? Can they explain messages transmitted cross-ESB?

2) SOA has a unique protocol suite. Can those automated tools support protocols like SOAP, WS-Security and something like that?

3) Can old tools help you isolate and position failure in a series of component calling process?

Changes of Testing Methodology

1) A new architecture needs a new testing method. The current black-box life cycle process and tools is not fast enough to meet the demands of deeply grey-box testing. For instance, we need to test user interface, as well as the suite behind.

2) SOA’s life cycle is agile and iterative. So automatic regression test becomes more important than before. How do we conduct those continuous integration testing? Manual testing is too slow and the old tools are not competent.

End-to-end Business Process Testing

1) Can the current tools support cross-technology multi-tool collaboration? Such as: user interface automation, middleware testing, service testing and database query.

2) End-to-end process testing may generate some complex TDM requirements, which will not be seen in the non-SOA system.

Testing of Access Restricted System

1) SaaS system charging according the times of use will increase the cost of regression test

2) Some systems, like the mainframe system, may not have resources or be not easy to be authorized to be used for testing.

SOA Testing Route Map

Every SOA project can be divided into 4 stages: Requirement Management, Analysis and Design, Development (and testing) and Deployment and Production Management. Most SOA manufacturers can provide data warehouse and BPM/BAM system supporting deployment and production management. The two stages are managed by SOA teams, not the concern of testers. CIT and SIT should be done while designing, developing and testing SOA life cycle, which should be attached importance to by the SOA testers.

The picture below shows the stages of the SOA method: