Thomas Sundberg

February 18, 2012

Test coverage in a multi module Maven project

Filed under: Automation, Java, Maven, Test Automation — Tags: , , , , , — Thomas Sundberg @ 09:35

This post has been migrated to

Test driving a project should result in a test coverage of 100% of the code. But how can we know what test coverage we have? Is it even important?

Know what you measure

Numbers is a complicated subject. They are a little like alcohol. To much and dependant of them is a problem. None may be boring. The Swedish word ‘lagom’ is very applicable here. It means not to much and not to little, just enough.

To measure is to know

As an engineer I know that if you can’t measure a value, you don’t know anything about it. But measuring the wrong thing is just as bad. You must know what the values you get represents. If you don’t, then you should at least not talk about them.

Is it important

Is test coverage even important? What is important is that you are certain that the functionality you want to support actually works as expected and required. Numbers are easy to misunderstand and test coverage test coverage can be useless if the tests are bad. Suppose that all code is executed but nothing is asserted? Then we might think that we have a great test coverage, but if nothing is asserted then it is actually worthless. So test coverage may be a good value to track, but it may also be totally useless information. The only way to know is to have confidence in the test code that actually generated the execution. A bad test will result in an execution that may or may not be correct.

But enough with ranting about numbers. How do you measure the test coverage? As usual, use the right tools for the job. One tool is Cobertura. It works great, but it has some problems when it comes to Maven projects with multiple modules.

Instrumented classes

Cobertura works like this: It instruments classes and writes to a log file each time a line is executed. The log is then combined with the source code to calculate which rows that actually were executed.

Let’s start with the simple case that works out of the box.

The simple case – one Maven module

A simple example is when you have a simple project in one module. It is easy in this case to measure the test coverage using a Maven plugin.

The project I use in ths example looks like this:

|-- pom.xml
`-- src
    |-- main
    |   `-- java
    |       `-- se
    |           `-- sigma
    |               `-- calculator
    |                   `--
    `-- test
        `-- java
            `-- se
                `-- sigma
                    `-- calculator

We have a really simple calculator that calculates fibonacci numbers. The source looks like this:


package se.sigma.calculator; public class Calculator { public int nextFibonacci(int a, int b) { return a + b; } }

The test code is also simple and looks like this:


package se.sigma.calculator; import org.junit.Test; import static; import static org.junit.Assert.assertThat; public class CalculatorTest { @Test public void shouldCalculateFibonacci() { Calculator calculator = new Calculator(); int expected = 13; int actual = calculator.nextFibonacci(5, 8); assertThat(actual, is(expected)); } }

The test coverage is 100% and it can be measured using Cobertura as shown in the Maven Pom below:


<?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <groupId>se.thinkcode</groupId> <artifactId>one-module-example</artifactId> <version>1.0</version> <properties> <>UTF-8</> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> </properties> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>cobertura-maven-plugin</artifactId> <version>2.5.1</version> <executions> <execution> <phase>process-classes</phase> <goals> <goal>cobertura</goal> </goals> </execution> </executions> </plugin> </plugins> </build> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.10</version> </dependency> </dependencies> </project>

The key part here is the cobertura-maven-plugin that is executed in the process-classes phase. It will instrument the compiled classes so Cobertura can record the execution of each line.

The report from a build looks like this:

It tells us that we have 100% coverage of the system and that two out of two lines of code has been executed. We have the complexity 1. There is only one path through the code. There are no conditions that will give us a different paths of execution.

Drilling down into the package se.sigma.calculator will give you the view below:

Further drilling into the class Calculator will give you the view below:

We note that each executed line is counted and the number of times it has been executed is reported. This doesn’t tell us anything about the quality of a test, it just tells us that the line has been executed. The quality of the tests has to be validated by examining the test cases themselves. I don’t know of any good tool that will help you here. You must use your experience as developer and your domain knowledge about the domain problem to know if this is a good test that tests the correct things or not.

As a side effect, we are publishing a nice way to browse the source code.

This case was easy since it is just one Maven module. A multi module Maven project is trickier if the production code and test code test is defined in different modules. The reason lays in the design of Maven. Maven will execute all build steps for the entire module before moving on to the next module. This is the only reasonable way to design a tool like Maven, but it clashes somewhat with a measuring tool like Cobertura.

A more complicated case – multiple Maven modules

If we have tests defined in one module and the production code in another module then we will have a problem with the behaviour of Maven. It performs all it’s build phases for each module before it continues with the next module. This is by design and normally a good thing.

Given a setup as below:

|-- pom.xml
|-- product
|   |-- pom.xml
|   `-- src
|       `-- main
|           `-- java
|               `-- se
|                   `-- sigma
|                       `-- calculator
|                           `--
`-- test
    |-- pom.xml
    `-- src
        `-- test
            `-- java
                `-- se
                    `-- sigma
                        `-- calculator

The production code and the test code is the same as before. A dependency has been added from the test module to the product so the Calculator can be found from the test. The only real difference is that I choose to separate production and test code in different Maven modules. The reasons for doing that may differ, in my cases I tend to have slow integration or acceptance test defined like this. It normally works very well, but test coverage is difficult to calculate using Maven.

I include the Maven poms for reference. They are the only thing that differs in this example compared to the first example.

The root pom is defined as:


<?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <groupId>se.thinkcode</groupId> <artifactId>multi-module-failing-example</artifactId> <version>1.0</version> <packaging>pom</packaging> <modules> <module>product</module> <module>test</module> </modules> <properties> <>UTF-8</> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> </properties> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>cobertura-maven-plugin</artifactId> <version>2.5.1</version> <executions> <execution> <phase>post-integration-test</phase> <goals> <goal>cobertura</goal> </goals> </execution> </executions> </plugin> </plugins> </build> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.10</version> </dependency> </dependencies> </project>

It has the same cobertura-maven-plugin defined and a report is generated. It also refers to the two modules that contains the product and the test in it’s modules section.

The product pom is defined as:


<?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <parent> <groupId>se.thinkcode</groupId> <artifactId>multi-module-failing-example</artifactId> <version>1.0</version> </parent> <groupId>se.thinkcode</groupId> <artifactId>calculator</artifactId> <version>1.0</version> </project>

The test pom is defined as below:


<?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <parent> <groupId>se.thinkcode</groupId> <artifactId>multi-module-failing-example</artifactId> <version>1.0</version> </parent> <groupId>se.thinkcode</groupId> <artifactId>calculator-test</artifactId> <version>1.0</version> <dependencies> <dependency> <groupId>se.thinkcode</groupId> <artifactId>calculator</artifactId> <version>1.0</version> </dependency> </dependencies> </project>

It has a dependency to the production code so the test can get hold of the Calculator and get something to execute.

The result is a below:

We see that we have 0% code coverage. Even though we know that we should have 100% in this case.

Drilling down into the package tells us the same thing, no execution of the code has been recorded.

Further drilling revels the Calculator class. We see that we had the possibility to run two lines. We also notice that Cobertura didn’t record any execution of the code.

This is by design. The classes the test executes has not been instrumented yet.

A solution is to use a combination of tools. Maven and Ant in combination can solve this problem. Maven is great for compiling the classes and executing the tests. All issues with class path and similar are handled nicely by Maven. Ant is extremely flexible and can be used to instrument the compiled classes and then merge the result after executing the tests to a coverage report.

A working multiple Maven module example

The file layout in this example is the same as to the example above, with the exception that I have added an Ant build.xml into the mix.

|-- build.xml
|-- pom.xml
|-- product
|   |-- pom.xml
|   `-- src
|       `-- main
|           `-- java
|               `-- se
|                   `-- sigma
|                       `-- calculator
|                           `--
`-- test
    |-- pom.xml
    `-- src
        `-- test
            `-- java
                `-- se
                    `-- sigma
                        `-- calculator

The execution has to be done in four steps.

  1. Compile all code
  2. Instrument the code
  3. Execute all tests
  4. Consolidate and build the report

It means that you have to execute the commands below in this order:

mvn clean compile
ant instrument
mvn test
ant report

The root pom is defined as below:


<?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <groupId>se.thinkcode</groupId> <artifactId>multi-module-example</artifactId> <version>1.1</version> <packaging>pom</packaging> <modules> <module>product</module> <module>test</module> </modules> <properties> <>UTF-8</> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> </properties> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-clean-plugin</artifactId> <version>2.4.1</version> <configuration> <filesets> <fileset> <directory>.</directory> <includes> <include>**/*.ser</include> </includes> </fileset> </filesets> </configuration> </plugin> </plugins> </build> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.10</version> </dependency> </dependencies> </project>

It doesn’t use any Cobertura plugin. The cobertura instrumentation and reporting will be done by the Ant script.

It uses the Maven Clean plugin to remove files Cobertura uses to store which lines has been executed. They are called cobertura.ser as default. I add sum.ser in the Ant script and therefore I want to remove all *.ser files from the project during the clean phase. This really doesn’t matter for this example, it is just house keeping.

The Ant build script is defined as:


<project> <target name="instrument"> <!-- Add all modules that should be included below --> <!-- <antcall target="instrumentAModule"> <param name="module" value="MODULE_NAME_TO_REPLACE"/> </antcall> --> <antcall target="instrumentAModule"> <param name="module" value="product"/> </antcall> </target> <target name="report" depends="merge"> <property name="src.dir" value="src/main/java/"/> <cobertura-report datafile="sum.ser" format="html" destdir="./target/report"> <!-- Add all modules that should be included below --> <!-- fileset dir="./MODULE_NAME_TO_REPLACE/${src.dir}"/ --> <fileset dir="./product/${src.dir}"/> </cobertura-report> </target> <target name="merge"> <cobertura-merge datafile="sum.ser"> <fileset dir="."> <include name="**/cobertura.ser"/> </fileset> </cobertura-merge> </target> <target name="instrumentAModule"> <property name="classes.dir" value="target/classes"/> <cobertura-instrument todir="./${module}/${classes.dir}"> <fileset dir="./${module}/target/classes"> <include name="**/*.class"/> </fileset> </cobertura-instrument> </target> <property environment="env"/> <property name="COBERTURA_HOME" value="/Users/tsu/java/cobertura-"/> <property name="cobertura.dir" value="${COBERTURA_HOME}"/> <path id="cobertura.classpath"> <fileset dir="${cobertura.dir}"> <include name="cobertura.jar"/> <include name="lib/**/*.jar"/> </fileset> </path> <taskdef classpathref="cobertura.classpath" resource=""/> </project>

The two main targets to understand are

  • instrument
  • report

The instrument target will prepare the compiled classes with Cobertura instrumentation so they are able to record their usage.

The report target is used to combine the result from the execution and build to coverage report.

To get this example up and running, you need to download Cobertura. I used the link and downloaded the binary version of Cobertura I unzipped the downloaded file into the directory /Users/tsu/java/cobertura- on my computer. You will ned to update the reference in the build.xml to match your specific location.

The production pom is identical to the production pom in the previous example, it doesn’t know anything about code coverage or Cobertura.


<?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <parent> <groupId>se.thinkcode</groupId> <artifactId>multi-module-example</artifactId> <version>1.1</version> </parent> <groupId>se.thinkcode</groupId> <artifactId>calculator</artifactId> <version>1.1</version> </project>

The test project is almost identical to the test module pom above.


<?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <parent> <groupId>se.thinkcode</groupId> <artifactId>multi-module-example</artifactId> <version>1.1</version> </parent> <groupId>se.thinkcode</groupId> <artifactId>calculator-test</artifactId> <version>1.1</version> <dependencies> <dependency> <groupId>se.thinkcode</groupId> <artifactId>calculator</artifactId> <version>1.1</version> </dependency> <dependency> <groupId>net.sourceforge.cobertura</groupId> <artifactId>cobertura</artifactId> <version></version> </dependency> </dependencies> </project>

The test project need a dependency to the Cobertura plugin to be able to record the execution during the test phase. The production code has been instrumented to use Cobertura so it needs to be available in the test classpath.

Executing the commands

mvn clean compile
ant instrument
mvn test
ant report

should now create a report like this:

It tells us that we have been able to count the times all lines has been executed.

Drilling down into the package se.sigma.calculator will give you the view below:

Further drilling into the class Calculator will show each executed line of code.

That’s it folks!
If the classes are properly instrumented before the test execution, then Cobertura will be able to record the execution of each line and a coverage report is possible to compile afterwards.


This post has been reviewed by Malin Ekholm.

Thank you very much for your feedback!



  1. Great post. I’ve used it and it works excellently. I adapted it using the maven antrun plugin so you can have everything in the parent pom file, optionally with external ant script files. Using the plugin you don’t need the cobertura installation and only need to run mvn twice and you have your report. If you’re interested in more details, let me know.

    Comment by Wouter — August 8, 2012 @ 14:19

    • hi Wouter,
      can you please provide your pom.xml for multi module project.

      Comment by gourab — November 5, 2012 @ 22:14

      • Hi Wouter,
        Yes, that would be great if you could post your pom file here. Thanks a lot.

        Comment by moudug — November 15, 2012 @ 13:29

    • Hi Wouter,
      I’m having the problem of multi-module and test coverage. I read this post, but it works with cobertura plugin and in my project we don’t use this plugin. Could you please send me your pom without cobertura? is that has to do with the command run? Is it possible to run ant command with a project maven?


      Sarang (

      Comment by Ignacio — November 12, 2012 @ 16:49

      • We use coberturašŸ˜›.

        Comment by Ignacio — November 13, 2012 @ 13:37

    • can uu post u r pom please ..i am facing issues/////

      Comment by anandkatti108 — February 7, 2013 @ 13:35

  2. Hi
    we are getting stuck in maven pom.xml to get coverage report if the test case are executing fine but data tables are not available now becasue they are old for some modules and we wont use further but it is affecting the overall coverage report

    can you please help and my email id is

    Comment by shanker — August 20, 2012 @ 08:51

  3. Hi Thomas,

    I received the error: [taskdef] Could not load definitions from resource It could not be found. could you help me please? Thanks



    Comment by Ignacio — November 14, 2012 @ 14:51

    • Hi!

      Does the cobertura.classpath contain the proper jars? If it doesn’t then I would expect this error to occur.
      Double check your paths, there might be an error there.


      Comment by Thomas Sundberg — November 14, 2012 @ 20:29

      • I could resolve all problems. The path of my cobertura.dir was wrong. Now, I have another problem. When I run report and want to see the result of the coverage, it appear all in red, 0% coverage over all class tested. What could be?

        Comment by Ignacio — November 14, 2012 @ 23:36

      • One reason may be that you have somehow failed to annotate the classes before executing the tests. Another may be that you haven’t combined the .ser files properly. I would double check the order the steps are executed in.


        Comment by Thomas Sundberg — November 15, 2012 @ 06:14

      • I tried to post the ant.xml with the structure of my project and I did not have success. Please, send me your e-mail, and I will send you if you do not matter.

        Comment by Ignacio — November 15, 2012 @ 14:38

      • The files cobertura.ser and sum.ser are located in a folder call ant, which contains the ant.xml file. Is correct
        to have these files in the root project?

        Structure project:

        -Project root
        — ant
        — ant.xml
        — cobertura.ser
        — sum.ser
        — target
        — reports

        * The commands are ran in the next steps:

        — mvn clean compile
        — ant instrument
        — mvn test
        — ant report

        Comment by Ignacio — November 15, 2012 @ 14:43

      • Hi Thomas,
        I am facing the same problem, like [taskdef] Could not load definitions from resource It could not be found, and my build.xml is similar to the above and i just updated the path COBERTURA_HOME to be D:\cobertura-1.9.41
        Can you please help me on this.

        Majeed H.

        Comment by MohamedKMajeed Haja moideen — December 18, 2015 @ 11:50

  4. Thomas!! I got it. One path was wrong. Now, I have to configure jenkins to run these commands in order to get the correct coverage. Any suggestion for that?


    Comment by Ignacio — November 16, 2012 @ 15:01

    • Hi!

      Glad you found the problem.

      My suggestion if you want to run this using Jenkins is to create freestyle project. Check out the source and add four consecutive steps. First the Maven compile step, then the Ant instrument step, then the Maven test step and finally the Ant report step. The last thing you would like to do is to have Jenkins to display the result. You can add support for Cobertura reports at the end of the job configuration. I don’t remember the details and I don’t have a Jenkins that I could try with to get the commands correct. But Jenkins has a Cobertura plugin that you can use. I think it s a plugin at least, it could be built in as well.


      Comment by Thomas Sundberg — November 17, 2012 @ 08:55

      • [cobertura-merge] Cobertura – GNU GPL License (NO WARRANTY) – See COPYRIGHT file
        [cobertura-merge] Exception in thread “main” java.lang.NoClassDefFoundError: org.apache.log4j.Logger

        Comment by anandkatti108 — February 7, 2013 @ 13:34

  5. Hi Thomas,

    Thanks for this interesting post. I’m wondering how surefire plugin knows about the generated .ser file in your example.
    I tried to pass it via or -D option but it seems it doesn’t work.
    Tests are executed against instrumented libraries but .ser file is not updated.
    If I run unit tests with ant and by providing the .ser it works fine.


    Comment by jnc — April 30, 2013 @ 15:22

    • Sorry for the wrong information, but writing it down finally helped me to find the trick.

      Surefire was running tests against non instrumented libraries (the one declared in project dependency section), as I instrument previously generated artifacts.

      By excluding these project dependencies in surefire configuration and by adding additional classpath element pointing to the instrumented dependencies it works like a charm.

      Thanks again for your post.

      Comment by jnc — April 30, 2013 @ 15:41

      • Hi!

        I am glad that you found my post valuable and that you found the problem. I obviously acted as a rubber duck without even knowing it:)


        Comment by Thomas Sundberg — May 1, 2013 @ 10:01

  6. Thomas,

    THANK YOU for posting this solution. I spent a day looking for a solution, many of which are either out of date or simple don’t work. My frustration has ended with your post. It was easy to read and easy to understand and I don’t even know how to use Ant, but picked up on it quickly.

    Thanks again!! =)

    Comment by Billy — April 30, 2013 @ 23:00

  7. Thanks A lot Thomas, it helped me a lot .. I implement your solution with maven-antrun-plugin it may help

    Comment by worldpython — May 27, 2013 @ 16:49

  8. I do have an antrun example, but not one that describes how to solve the problem with multi module test coverage.


    Comment by Thomas Sundberg — October 8, 2013 @ 22:07

  9. Thanks for the nice post. I found this article very helpful as I am using a separate module for testing. The report is working fine except for a small issue – unable to locate the source code. I am using the ANT script as-is and run from the root. Would you be able to help with this?

    Comment by patb23 — March 31, 2015 @ 20:30

    • Hi!

      It is very hard to know what is causing your problem. I would assume that you need to change something in the Ant script but i am not certain what you should change. I would double check the paths in the Ant script.

      So, sorry. No I can’t say what is wrong without running on your system. And I can’t do that.


      Comment by Thomas Sundberg — March 31, 2015 @ 20:43

    • Please ignore the request for help. Added a task that copies all src to the destination. it worked. Thanks a lot

      Comment by patb23 — March 31, 2015 @ 21:12

  10. Hi Thomas, this article is very useful to me. but i found a problem:

    when i run ‘mvn test’, i get error ‘java.lang.NoClassDefFoundError: net/sourceforge/cobertura/coveragedata/TouchCollector’; however, when i run ‘mvn cobertura:cobertura’, it’s all right.

    thanks again!

    Comment by onlyjdoc — May 18, 2015 @ 09:14

  11. To run this in Jenkins, do you need to download the Cobertura jar file and put it somewhere in your project?

    Comment by David — October 26, 2015 @ 20:12

  12. Excellent information and I’ve implemented it myself. Since the original post in 2012 I wonder if changes to Maven have made a pure Maven solution possible.

    Comment by Mike West — April 9, 2016 @ 00:16

    • Hi!

      I haven’t checked if any change in Maven has made it possible to do this with a pure Maven solution.

      But I kind of doubt it. The problem is that Maven uses depth first when it builds a multi module project. Each module is built completely before it starts with next module. Instrumenting the classes is done before the tests are executed. But the classes in an earlier module will not be touched again once the module is built and especially will they not be instrumented. Therefore, a pure Maven solution is not possible.

      The better solution? Keep your unit tests within the same module and make sure the test coverage is relevant using unit tests.


      Comment by Thomas Sundberg — April 9, 2016 @ 16:58

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Silver is the New Black Theme. Create a free website or blog at


Get every new post delivered to your Inbox.

Join 81 other followers

%d bloggers like this: