Tag Archives: unit

Continuous Integration and Testing :Unit and Integration Tests With Maven and JUnit Categories

Introduction

This example shows how to split unit and integration tests using Maven and JUnit categories.
It is especially useful for existing test suites and can be implemented in minutes.

Why use this?

My previous post showed how we to use a maven profile to split unit and integration tests.
http://johndobie.blogspot.co.uk/2011/06/seperating-maven-unit-integration-tests.html
This has been a very well read post and I like how it uses seperate directories. However this example show a much simpler technique that can easily be applied to legacy test suites.
It offers most of the benefits of the original, and sits more comfortably in the Maven world.

Code

The code for the example is here. 

svn co https://designbycontract.googlecode.com/svn/trunk/examples/maven/categor...
mvn clean install

JUnit Categories

As of JUnit 4.8 you can define your own categories for tests. This enables you to label and group tests.

This example shows how easy it is to separate unit and integration test using the @Catgegory annotation.

http://kentbeck.github.com/junit/javadoc/latest/org/junit/experimental/categories/Categories.html

Define the Marker Interface

The first step in grouping a test using categories is to create a marker interface.

This interface will be used to mark all of the tests that you want to be run as integration tests. 

public interface IntegrationTest {}

Mark your test classes

Add the category annotation to the top of your test class. It takes the name of your new interface.

import org.junit.experimental.categories.Category;
@Category(IntegrationTest.class)
public class ExampleIntegrationTest{

 @Test
 public void longRunningServiceTest() throws Exception {
     
 }
}

Categories can be used to mark classes or methods. Really in my opinion you should only mark a class.

If you have both unit and integration tests in a single class then split it.

Configure Maven Unit Tests

The beauty of this solution is that nothing really changes for the unit test side of things.

We simply add some configuration to the maven surefire plugin to make it to ignore any integration tests.


org.apache.maven.plugins
maven-surefire-plugin
2.11

 
  org.apache.maven.surefire
  surefire-junit47
  2.12
 


 
  **/*.class
 
 com.test.annotation.type.IntegrationTest


There are 2 very important parts. The first is to configure surefire to exclude all of the integrations tests.

com.test.annotation.type.IntegrationTest

Surefire will run all of your tests, except those marked as an integration test.

The other important part is to make sure the surefire plugin uses the correct JUnit provider. The JUnit47 provider is needed to correctly detect the categories.


 
  org.apache.maven.surefire
  surefire-junit47
  2.12

Running the unit tests

To make sure this works correctly we can run the unit tests

mvn clean test

You can see from the output below that the unit test is run, but not the integration test.

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.test.EmptyUnitTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Configure Maven Integration Tests

Again the configuration for this is very simple.

We use the standard failsafe plugin and configure it to only run the integration tests.


 maven-failsafe-plugin
 2.12
 
  
   org.apache.maven.surefire
   surefire-junit47
   2.12
  
 
 
  com.test.annotation.type.IntegrationTest
 
 
  
   
    integration-test
   
   
    
     **/*.class

The configuration uses a standard execution goal to run the failsafe plugin during the integration-test phase of the build.

The following configuration ensures only the integration tests are run.

com.test.annotation.type.IntegrationTest

And again the JUnit provider must be correctly configured.


 
  org.apache.maven.surefire
  surefire-junit47
  2.12

That’s it!

Running the integration tests

We can now run the whole build.

mvn clean install

This time as well as the unit test running, the integration tests are run during the integration-test phase.

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.test.AnotherEmptyIntegrationTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec

Running com.test.EmptyIntegrationTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec

Results :
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
source:
http://www.javaworld.com/community/?q=node/8374
Did you like this? Share it:

More Than 90% of Developers Believe Unit Testing Is Most Effective For Reducing Bugs

[…]

80 percent of respondents said that bugs are the responsibility of developers and only 8 percent said it was QA’s. More than 54 percent of respondents noted that they were responsible for bugs in their company’s software. An additional 26 percent of respondents felt that another developer was most responsible for software bugs.

Finding and fixing bugs takes up a significant portion of developers’ time, according to the survey. 48 percent of respondents said that they spend up to 5 hours each week finding and fixing bugs and 38 percent said they spend up to 10 hours a week on this alone, which is 25 percent of the average work week. An additional 12 percent spend over 10 hours of their week finding and fixing bugs. According to data received from customers using Typemock Team Mate, tracking developers’ actual usage, the amount of time developers spend using the debugger is even higher than they think – around 50-55 percent of their time.

Today, a company’s value is greatly affected when customers experience bugs in their software. Over 50 percent of respondents noted that, as developers, it is most damaging to their reputation when end users report a bug in the program.

As the survey shows, unit testing is more crucial than ever and with proper unit testing developers will be more efficient, catch more bugs, and create higher-quality code.

“As professional programmers, we know that unit testing is the best way to get rid of software bugs,” said Eli Lopian, CEO of Typemock. “Over 90 percent of developers see unit testing as the most effective way to find and fix bugs. We know that unit testing is difficult to start and the rewards are great. This is exactly the problem that Typemock is out to solve – helping beginners, novices, and experts to reach their unit testing/TDD goals. Automated unit testing with Typemock’s Isolator V7 ensures software bugs are caught instantly so they can be corrected immediately.”

[…]

Read More:

http://www.darkreading.com/vulnerability-management/167901026/security/news/232800324/more-than-90-of-developers-believe-unit-testing-is-most-effective-for-reducing-bugs.html

Did you like this? Share it:

Unit Testing is a Means to an End

Most professional software developers these days understand the importance and value of writing and using unit tests. A nice summary of some of the oft-touted and oft-realized benefits of unit testing can be found in the StackOverflow.com thread Is Unit Testing worth the effort? [my only very minor criticism is the mixing of more specialized Test-Driven Development (TDD) with the more general unit test concept]. As with most good things, however, even unit testing enthusiasm can go too far. The benefits of unit testing can lead to overly enthusiastic unit testing developers forgetting that unit tests are not the end themselves, but rather are a means to the real end.

The "end" that most software developers are striving for is delivery of software solutions that make their users’ lives easier and more productive. Unit tests can be extremely valuable in obtaining this end and certainly add to software quality, but the overly zealous unit tester must beware of allowing the unit tests themselves to displace this end goal. It’s all too easy to allow oneself to get so bound up in writing exhaustive and "perfect" unit tests that one puts the true end goal at risk. In the remainder of this post, I look at ways in which developers can allow unit testing to move from helping achieve the desired end to unintentionally displacing the real end and putting it at risk.

Overreaching Unit Testing

Steven Sanderson has written "the benefit of unit testing is correlated with the non-obviousness of the code under test." I largely agree with this sentiment as a general guideline. I see little value in unit testing trivial "get" and "set" methods. Some methods are more readily evaluated via code review than via unit test.

The concept of code coverage can be a useful one as long as it’s not taken too far. Code coverage appears to provide high return for the effort for a while, but there comes a point of diminishing returns when gaining additional code coverage comes at much greater cost and may not be worth that cost. It’s also important to recognize that even the often highly expensive 100% code coverage typically means only all lines of code were executed and does not check all possible paths through the code.

All Code’s Unit Testability is Not Equal

The post Selective Unit Testing – Costs and Benefits clearly articulates well the differences in difficulty (cost) and advantages (benefits) of unit testing of different types of code. In cases where the advantages/benefits of a unit test are high and the cost/effort is low, the value of unit testing is obvious. On the opposite extreme, there are types of code that receive little benefit from unit testing.

Source: http://www.javaworld.com/community/?q=node/8354

Did you like this? Share it:

Unit Test For ASP.NET Page With WatiN

Unit Test For ASP.NET Page With WatiN

WatiN is a tool from Watir, used to test Web pages. WaitN stands for Web Application Testing in .NET.

Here we will test a simple ASP.NET page, with which to demonstrate the scene of identity and acceptance (agreement aclearcase / “target =”_blank ”> acceptance). Users enter name in the text box, click “I agree” check box, then press submit button. This is obviously a very simple page. You can use this idea for large pages testing when you are familiar with the working mechanism of WatiN framework.

Test Agreement Page:

Add a class library project to solution, as well as test tool (here used MbUnit, but you can use Nunit or VSTS) and WatiN library reference.

The following test code to ensure that users have already recognised.

[TestFixture(ApartmentState =

ApartmentState.STA)]publicclassTestAgreementPage{[Test]

publicvoidTestCanAcceptUserAgreement(){IE ie =

newIE(ConfigurationManager.AppSettings["DefaultPageUrl"]);

ie.TextField(“txtName”).TypeText

(“Mohammad Azam”);ie.CheckBox(“chkAgree”).Checked =

true;ie.Button(“btnAgree”).Click(); Assert.AreEqual(“Valid”,

ie.Span(“lblMessage”).Text);}} .csharpcode, .csharpcode pre

{ font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op

{ color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }

This class has TestFixture Attribute and STA value. Ensure the test runs in the state of STA (Single Threaded Apartment). That is because the test code needs to be loaded IE.

IE class in WatiN completed the main work. IE class opens IE, refers html control by name or id. ie.TextField(“txtName”).TypeText(“Mohammad Azam”) refers id “txtName” text box. After browser is loaded, WatiN will enter “Mohammad Azam” to “txtName” text box. You could see this process during the test. After that, id “chkAgree” check box will be selected. Finally, WatiN will press the submit button to submit the form.

Did you like this? Share it:

GDC 2012: How Vector Unit made the leap from console to mobile games

Matt Small, creative director at independent developer Vector Unit, thought his small studio would focus on consoles forever when it was working on Hydro Thunder Hurricane for Xbox Live Arcade.

But then the smartphone and tablet market picked up, and mobile devices started to offer increasing amounts of processing power. That increase in power opened up new opportunities for the people at Vector Unit, which had years of developing for consoles.

So the studio worked with Tegra chipmaker Nvidia and created the Android and iOS game Riptide GT, a jet ski racing game that took five months to develop. After that came another multiplatform water racing game, Shine Runner, which took about four months to develop.

At GDC 2012, Small laid out 10 tips for console game developers who are considering making the leap to mobile.

1. "Embrace the brutal truth of the mobile market."
Small warned that the vast majority of sales in the mobile game market come from a small percentage of developers — it’s an extremely crowded and competitive market. "The top 1 percent is very hard to get into unless you’re Angry Birds," said Small.

2. "Rethink roles"
At small studios in particular, people need to be able to do more than one thing well. "When you’re building a team especially when you’re hiring a startup, you need people who are flexible," said Small.

3. "Build Efficiently"
Vector Unit’s development process was built around efficiency. That’s reflected in the short development cycles of Riptide GT and Shine Runners, which took five months and four months to develop, respectively.

4. "Identify your audience"
Small also said it’s important to know who you’re making your games for. "Hardcore" players and "casual" players use their mobile devices for games, so there are lots of opportunities for mobile game makers.
"These audiences are totally merging. There’s ton of hardcore gamers on mobiles, and there and also lots of casual players," he said.

"I think there’s an audience for just about any kind of game you want to make on mobile. …Use the [audience] as a benchmark…don’t try to be all things to all people."

5. "Be ruthless with your feature set"
Vector unit was careful from the beginning to identify the core features of the game in development. Everything must relate back to this feature set. In Hydro Thunder, for example, "We wanted a really tight moment to moment gameplay," said Small, and that remained a focus throughout development.

6. "Get ‘organizized’"
"I think actual task-tracking and organization, that is really important," said Small. It takes extra effort to track tasks and development in spreadsheets, but when a studio records all of the details of making a game, it then has a concrete point of reference for the time and effort it takes to perform specific tasks (e.g. adding a new race track). This practice is also something that developers with console game backgrounds may already have experience with.

7. "Don’t be precious"
"Don’t be precious with your ideas," said Small. Developer don’t need to make a game top secret during its development. Small teams have the tendency to become very insular. It’s best to look for input outside the walls of the studio.
"It’s really important to get it in front of friends and family, you want to pull people off the street [to try your game and give feedback]," said Small.

8. "Ship your Beta?"
Console game developers may not be used to shipping a game that isn’t 100 percent "done." Mobile platforms offer developers opportunities to constantly update games with new features. But that doesn’t mean developers should ship broken games. "It’s really important to ship something that actually works… something that is truly a minimum viable product." He added, "Make sure that you schedule time afterwards to respond to things."

9. "Control! Style! UI!"
Small said control, style and UI are three elements that mobile developers should focus on. Regarding a distinct style, Small added, "With a small team you can’t necessarily out-Chair [Infinity Blade developer] Chair. … But come up with an art style that is compelling and works to the strengths of your team."

10. "Go Cross Platform"
Making multiplatform mobile games takes effort, but it paid off for Vector Unit. "In our experience on Riptide, [sales of the] iPhone version basically made up for our development costs," he said. But the Android version helped Vector Unit actually turn a meaningful profit.

"Our Android sales have pretty much topped iPhone sales," said Small. He added that games hold sales levels over a longer period of time. "There is a thicker tail for Android sales," he said.

While he highly encouraged cross-platform mobile development, Small added, "Your mileage may vary with this."

Source: http://www.gamasutra.com/view/news/165387

/GDC_2012_How_Vector_Unit_made_the_leap_from

_console_to_mobile_games.php

Did you like this? Share it:

Test Strategy versus Test Plans

Test Strategy is part of the Test Plans. Test Plans describes who will do what? Describes roles and responsibilities of test team, resources required any risks associated, major functions to be tested. Test Strategy is a very important part of the test plan. When requirements analysis phase is finished and requirements are somewhat mature then testing team starts preparing test plans, how to carry on testing and what strategy to be followed.

In very simple precise terms there are two basic testing strategies:

 To test the software in its entirety, once the completed package is available; otherwise know as “big bang testing”

 To test the software piecemeal, in modules, as they are completed (unit test); then to test groups of tested modules integrated with newly completed modules (integration tests). This process continues until all the package modules have been tested. Once this phase is completed, the entire package is tested as a whole (system test). This testing strategy is usually termed “Incremental Testing”

Incremental Testing is further divided into two classes Bottom-up Testing Strategy and Top-Down Testing Strategy.

Did you like this? Share it:

Typemock CEO Predicts Unit Testing to Grow in Importance Due to Software Explosion in 2012

Tel Aviv, January 18, 2012 – Typemock, (http://www.typemock.com/) the leading provider and pioneer of easy unit testing solutions, announced today their prediction that a rise in unit testing will occur in 2012 as software development continues to climb significantly. Eli Lopian, founder and CEO of Typemock , speaking at a developer seminar said, “In 2012, we will see a major shift in the efforts of software developers to focus on better code. Unit testing is significantly more important today as we see an eruption in software development in the coming years.” In addition, Lopian added “Unit testing is a key aspect of all Agile methodologies, and, in order to remain agile, unit testing must be properly executed.”

Software development has grown exponentially in recent years as more companies lean toward becoming software providers as well as product vendors. This rise in software increases the risk of faulty programming code which can lead to mission critical failures that cost organizations both in financial damages and consumer satisfaction. Software quality and on time releases are critical to any organization and unit testing, as part of Agile development, enables developers to produce improved software within deadlines.

In an article on the Testing Tools Landscape, a leading analyst firm noted, “Now it is time for quality management and testing to respond to this faster-moving environment. Functional testing tools are not enough. Quality must move beyond the purview of just the testing organization and must become an integrated part of the entire software development life cycle (SDLC) to reduce schedule-killing rework, improve user satisfaction, and reduce the risks of untested non-functional requirements such as security and performance.” In addition, a recent Forrester report recommended that developers engage in automated testing, including unit testing, in order to build higher quality software.

Read More:

http://www.prurgent.com/2012-01-18/pressrelease220212.htm

Did you like this? Share it:

A Set of Unit Testing Rules

Teams that adopt agile practices often adopt Test Driven Development (TDD), which means, of course, that they end up writing a lot of tests. In general, thats great but there is a failure case for teams that attempt to get test infected; you can end up writing very slow tests that take so long to run that they essentially start to feel like baggage even though they help you catch errors.

This issue with unit tests isnt a new issue, its been around for a while, and there are a couple of ways of handling it. In Extreme Programming, the typical way of handling it is to periodically go back and optimize your tests as they get too slow. In many cases this works well, but the amount of optimization that you have to do can be rather large if you havent been conscious of how long your tests run during development. In one case that stands out in my memory, I visited a team on the east coast about four years ago that wrote oodles of tests against their EJB environment. The tests hit a server and went through session beans, entity beans, down to the bowels of the database and then up again. Their refrain? We dont like writing unit tests any more; they take too long to run. I didnt blame them for feeling that way, but I also didnt agree that they had written any unit tests.

The problem is rather common. Ive spoken to other XPers about it over the years and I sort of figured that the way that I handled it was common, but I was surprised to discover (on the XP yahoo group this week) that it was also a bit contentious. Heres what I typically say when I run into teams that have this problem.

A test is not a unit test if:
# It talks to the database
# It communicates across the network
# It touches the file system
# It can’t run at the same time as any of your other unit tests
# You have to do special things to your environment (such as editing config files) to run it.

Tests that do these things aren’t bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.

Source: http://www.vietnamesetestingboard.org/zbxe/?mid=download&category=12646&document_srl=459161&listStyle=&cpage=

Did you like this? Share it:

Unit Testing – Best Practices & Techniques

Unit testing is a software development process in which the smallest testable parts of an application, called units, are individually and independently scrutinized for proper operation. The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as expected. Each unit is tested separately before integrating them into modules to test the interfaces between modules.By means of effective Unit testing large percentage of defects are identified. Unit testing is performed by developers.
Each module that is developed by designers need to be tested individually to verify proper operation so that any faulty module can be fixed immediately rather than let it exist and then cause some major issue in the integration phase. Once all of the units in a program have been found to be working efficiently and without any bugs, larger components of the program can be evaluated by means of integration testing. Though Unit testing may be time consuming, tedious and requires thoroughness on the part of the development team, but in the long run it can avoid major pitfalls in the software.
Benefits of unit testing:

· The modular approach during Unit testing eliminates the dependency on other modules during testing.

· We can test parts of a project with out waiting for the other parts to be available.

· Designers can identify and fix problem immediately, as the modules are best known to them. This helps in fixing multiple problems simultaneously.

· Cost of fixing a defect identified during the early stages is less compared to that during later stage.

· Debugging is simplified.

· Structural coverage of code is higher.

· Unit testing is more cost effective compared to the other stages of testing

Source: http://www.vietnamesetestingboard.org/zbxe/?document_srl=443792

Did you like this? Share it:

Unit Test Your Database

I observe the endless war about TDD. On one side there are those who claim that TDD is the best thing since sliced bread. On the other side there are those who claim that TDD is just a waste of time and is too slow.

TDD Your Indexes

Imagine you have got a database filled by some ETL process, which fills an empty database with huge amount of data. This means you can have many different databases with different versions of the data and structure. For one database you could notice that the database is slow. It is too slow. Simple index could help. You check that, and it really helps. OK, good.

You just fix the ETL to create this index after loading the data. You are happy – you’ve already fixed the problem. Do you fixed the real problem?

What with the rest of databases? Well… they need that index too. So you just create the index on all the databases/schemas you remember about.

  • Have you created the index on all?
  • What is the probability of forgetting about some of schemas and databases?
  • What if suddenly someone used the old ETL version instead of the new one?
  • What if the old version is still used because you will release the new version in a week?

The answer is simple: TDD.

TDD you database.

If you are using PostgreSQL, just write a simple test, use pgTap something like:

SELECT has_index(
  'public',
  'tab',
  'my_index',
  ARRAY['a', 'b']
);

Next time when the database is slow, just run this simple test. It just tells you if the index exists or maybe you should create it.

Source: http://simononsoftware.com/unit-test-your-database/

Did you like this? Share it: