Separating tests in Maven

The way Maven file structure is defined is a great way to separate unit tests and production code. Unit tests are fast. So fast that developers developing using Test Driven Development, TDD, doesn’t have any performance problems when applying the Red-Green-Refactor cycle in their work. A large unit test suit is expected to be executed in seconds.

There is, however, no standard on how the files should be structured for slower tests (integration/acceptance/system et.al.). A common property for these tests is that they are slow. They often require a complete system setup in a known state. They use the filesystem, network, database and similar slow resources. Slow tests are normally not executed often. Developers seldom have the patience to wait for a build a long time before writing the next thing. Using them in a Red-Green-Refactor cycle is not practical. It takes too long time.

So what options do we have to separate the fast units tests and the slow tests? There are two main tracks that I have explored.

• Separating tests with name or package conventions
• Separating slow tests in a test module

Performing an action when a test fails

JUnit supports annotations so a method can be executed first in a test class or before each test method is executed. It also has annotations that supports methods to be executed after each test method or after the test class. This is very good if you need a common setup and a common tear down.

The after methods don’t give you access to the test result. You cannot know if a test method failed or not. This may pose a problem if you have a need to perform some specific actions after a failed test. A solution could be to implement an onError() method. But JUnit doesn’t support it.

A solution is to employ a @Rule annotation.

Pair Programing

How do you solve a problem best? In your solitude alone or in a group? I know that I solve problems better in a group. The group has to be small so you don’t get disturbed. But it has to be large enough so a discussion can occur.

Programing is not about writing text fast. Typing is never the bottleneck when programming. The bottleneck is problem solving and solving the right problem.

Pair programming is about two persons that solve a problem together. Problem solving in a group tend to result in better solutions. Different people see a problem from different angles. The result is usually better than a solution that one person came up with.

Execute tests in random order

It works on my machine!

Ever heard that from a developer? I have, and it happens that the reason is that it actually works on their machine. It may also be the case that the order in which tests are executed matters. Test classes depends on each other and the order they are executed in is important. The solution is to execute the tests in random order so that any dependencies between tests are found and can be removed.

What is a good test?

A colleague asked that question the other day. What is a good test? It is a really difficult question to answer. There are a number of properties that hold true for a good test. I will list some of them and discuss why I think they are important. Others may think that the order of importance is different or that other properties are more important then those that I list.

I will show some examples using Java and JUnit of what I think are bad tests and how they can be refactored to something better.

Tests are automated in my world. I always try to avoid manually testing. The main reason for this is that manual testing is slow and hard to repeat arbitrary many times. Therefore, test should be read as automated test in for the rest of this text.

The simplest possible solution

The simplest possible solution that could work. Ever heard that expression? What does it mean? Really?

The answer is obviously something that in a really simple way satisfies the test you currently are working on. Nothing more, nothing less.