Thomas Sundberg

January 31, 2013

Take control of your time

Testing is all about verifying that something works as expected. What are the challenges then? There are many, one very obvious is to actually know how it should work. Another challenge is to isolate the system under test so that it is possible to know what is tested and determine if that specific part works as expected.

Isolating the system under test means that you must have control over the environment that you perform the testing in. This is true for almost anything that is tested. I work with software systems so I am mostly interested in software testing. But the general idea here is applicable on any testing. If you don’t have control over the surrounding systems then how can you actually test something in a deterministic way?

Time dependence

How would you do to test a behaviour that is dependent on the current time? Suppose that you should test that the shipping cost in an online store is calculated correctly. There is a business rule that says that shipping should be free a month before Christmas and that it should cost 17 the rest of the year. This means that the cost will depend on the date an order is placed.

An implementation of this rule could be something like this:

public int calculateShipping() { int aMonth = 31; Calendar today = GregorianCalendar.getInstance(); int year = today.get(Calendar.YEAR); Calendar christmas = GregorianCalendar.getInstance(); christmas.set(year, Calendar.DECEMBER, 24); int christmasDay = christmas.get(Calendar.DAY_OF_YEAR); int currentDay = today.get(Calendar.DAY_OF_YEAR); int daysBeforeChristmas = christmasDay - currentDay; if (daysBeforeChristmas < aMonth && daysBeforeChristmas > -1) { return 0; } else { return 17; } }

This method has at least two problems. A minor issue is that the day Christmas is celebrated is different in different countries. Some celebrate it 24 of December, others 25 of December.

The largest problem is not which date Christmas is celebrated though, but the fact that it is only possible to verify the behaviour during a few days of the year.

Suppose that this method is implemented in mid December, tested with a few unit tests and executed in a continuous integration server. Everything would be nice and green. The build will, however, start breaking after Christmas and break every time it is executed. The test will probably be disabled and probably forgotten. The result may be that we build up a set of disabled tests and all of a sudden it may be ok to have ignored tests.

Take control

How can this method be refactored so it gets testable? It will become testable if you take control over the context where the test will be executed. Supply the method with a date you have control over. This means that you can create tests for all interesting dates surrounding the edges for free shipping. One day before the shipping should be free, the first day that should be free, the last free day, the first day after and so on. You can very easy create all cases that will box in the shipping rule and always be sure that you will know when somebody breaks the functionality.

An implementation may look like this:

public int calculateShipping(Calendar today) { int aMonth = 31; int year = today.get(Calendar.YEAR); Calendar christmas = GregorianCalendar.getInstance(); christmas.set(year, Calendar.DECEMBER, 24); int christmasDay = christmas.get(Calendar.DAY_OF_YEAR); int currentDay = today.get(Calendar.DAY_OF_YEAR); int daysBeforeChristmas = christmasDay - currentDay; if (daysBeforeChristmas < aMonth && daysBeforeChristmas > -1) { return 0; } else { return 17; } }

The only difference is that you now take control of the time. You can set the parameter today to whatever you want in a test. You don’t depend on the system date where this code is executed.

This pattern of taking control can be extended to almost anything that surrounds a piece of software. If you don’t have control, then you really can’t execute your tests and expect a deterministic behaviour. This goes for any surrounding system, a main frame, a database with a production copy and so on.

Conclusion

If you are not in control, you will always be depending on other systems to behave deterministic. Surrounding systems in a development environment seldom behave deterministic all times. This may be one of the largest problems that you must solve when you implement test automation.

Acknowledgements

This post has been reviewed by some people who I wish to thank for their help

  • Malin Ekholm
  • Johan Helmfrid

Thank you very much for your feedback!

Resources

About these ads

3 Comments »

  1. Hi, good points about testing time dependent functionality.
    There are more possible problems, e.g. the definition of a month. The silent assumption in your code is that a month counts 31 days. For Christmas on Dec. 25th, a month earlier should start on Nov. 25th. With your current code it will start on Nov. 24th. Thus, I would rephrase the condition in the method and check if the current month is the month of Christmas (December) or one earlier (November) + checking days.

    Another issue with time-related tests are timezones. Should we count time in the timezone of the application calling the method or the place where the method is called? e.g. someone in Canada is running this code in Germany, where the code is stored and run – what timezone should be used to get consistent results?

    Comment by Slawek — February 1, 2013 @ 15:21

    • Hi!

      You are absolutely right, there are many things that could be issues with the code in my example. The length of a month can be argued. Should it be 28, 29 (it happens) or 30 days? How should timezones be handled? These are things you have to understand and decide. Then there is the control part, avoid to rely on the system date for testing. It moves on and your test has to cope with a moving target. Moving targets are always more difficult to handle than stationary targets.

      Testing is difficult without understanding the problem and controlling the testing environment.

      /Thomas

      Comment by Thomas Sundberg — February 1, 2013 @ 16:46

      • There are two different categories here.
        * The business part, the length of a month, when is Christmas celebrated and how should timezones be handled.
        * The technical part, how can you isolate the system under test so you don’t get dependencies that you don’t have control over.

        The business part is something you might have to accept and just relate to. The technical part is where you, as a developer and tester, have to take control and implement what you need in order to solve the problem.

        Comment by Thomas Sundberg — February 2, 2013 @ 08:32


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Customized Silver is the New Black Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 49 other followers

%d bloggers like this: