Filed under: Java, Maven, TDD, Test automation, — Tags: Acceptance test, Integration test, Separation of concern, System tests, Test suit — Thomas Sundberg — 2012-08-21
The way Maven file structure is defined is a great way to separate unit tests and production code. Unit tests are fast. So fast that developers developing using Test Driven Development, TDD, doesn't have any performance problems when applying the Red-Green-Refactor cycle in their work. A large unit test suit is expected to be executed in seconds.
There is, however, no standard on how the files should be structured for slower tests (integration/acceptance/system et.al.). A common property for these tests is that they are slow. They often require a complete system setup in a known state. They use the filesystem, network, database and similar slow resources. Slow tests are normally not executed often. Developers seldom have the patience to wait for a build a long time before writing the next thing. Using them in a Red-Green-Refactor cycle is not practical. It takes too long time.
So what options do we have to separate the fast units tests and the slow tests? There are two main tracks that I have explored.
When separating tests based on their names, you either name the test classes after a specific pattern or place them in a specific package. When you want to execute them, you filter the tests based on the pattern you used for the separation.
This option will allow you to write all tests intermingled and execute them at different times. Suppose that we
place all our unit test in a package that contains the part unit and all integration tests in a package
called integration. Then all that is needed is to set up some filtering in the Maven build. One way to
do this is to use include and exclude in Surefire. By excluding a package in the test phase and include
it in the integration-test phase, you are able to separate the execution.
An example could look like this:
<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>se.waymark.educational</groupId>
<artifactId>name-separation</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.2</version>
<executions>
<execution>
<id>default-test</id>
<configuration>
<includes>
<include>**/unit/**</include>
</includes>
<excludes>
<exclude>**/integration/**</exclude>
</excludes>
</configuration>
</execution>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>
<exclude>**/integration/**</exclude>
</includes>
<excludes>
<include>**/unit/**</include>
</excludes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
I have hacked the maven-surefire-plugin and defined that it should exclude everything from the
integration package in its default execution.
Then I have added an execution to the integration-test phase where the goal test will be executed. The tests in the package unit are now excluded and the tests in the package integration included.
This separation enables me to prepare the system under test before the integration-test phase. If I need I can setup an application server and seed a database before the integration tests are executed.
Executing the build with
mvn integration-test
will execute all tests. Executing it with
mvn test
will only execute the unit tests.
An alternative to this setup is to separate the tests in different modules. Lets explore a setup for module separation.
A Maven multi module project is needed to be able to separate the test to a specific test module. The parent pom may look like this:
<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>se.waymark.educational</groupId>
<artifactId>module-separation</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<modules>
<module>main</module>
<module>test</module>
</modules>
</project>
The tests are implemented in the test module. They may be located in any package. The tests in the test module will only be executed in the integration-test phase.
The test module pom may look like this:
test/pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>se.waymark.educational</groupId>
<artifactId>test</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.2</version>
<configuration>
<skip>true</skip>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>se.waymark.educational</groupId>
<artifactId>main</artifactId>
<version>1.0-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
I do two things here. First I skip all tests in the default execution of Surefire. Then I define that the goal test should be executed in the integration-test phase and that no tests should be skipped.
This separation requires less configuration of Surefire plugin compared to separating the tests using a naming convention.
Execute the build with:
mvn integration-test
This may be done in the project root or in the test module.
I have heard rumours that separating things in different modules is a problem if you are using Eclipse. This may be a valid reason for some people not to separate a project like this. In my opinion, it is not a valid reason. I use better tools than Eclipse and don't have any issues separating a project like this. A better tool is IntelliJ IDEA.
Separating the slow tests in a specific module enables a separation of concern. It also applies the single responsibility principle. The test module is responsible for the slow tests and nothing else.
These are my main argument for preferring separation in different modules instead of separating them using naming conventions.
RSS