Clearer Unit Tests: Assert, Guard, and Throw
I hate the book “xUnit Test Patterns” for being over 800 pages long. I am convinced a good editor can turn any 800 page book into a 700 pager with the exact same content… that includes you Leo Tolstoy! The problem is that xUnit Patterns is still the definitive reference to test concepts and implementation. There is no better book on testing available, but the sheer weight and size the book means few have actually read it. Well I have, and I highlighted the damn thing just to prove it. In this post you’ll find 3 techniques to improve test clarity that, in my opinion, are underutilized in the development world today.
Custom Assertion Methods
I commonly see assertions repeated throughout a test case in sets. For instance, my current system deals in User objects, and you can see the same 3 lines of code sprinkled throughout a test case:
User user = ...
assertEquals("userId", user.getUserId());
assertEquals("fname", user.getFirstName());
assertEquals("lname", user.getLastName());
This is a form of test code duplication. It plain English, you want to say “make sure that the user we got back is the same one we expect”. This points to two concerns: an assertion (“make sure”) and equality (“the same”). It is temping to refactor using Extract Method, but this is a poor solution:
User user = ...
assertEquals("userId", "fname", "lname", user);
static assertEquals(String userid, String fname, String lname, User user) {
assertEquals("userId", user.getUserId());
assertEquals("fname", user.getFirstName());
assertEquals("lname", user.getLastName());
}
This isn’t bad, it just isn’t good. The test reads better, but the parameter list of the custom assertEquals is quite busy. It sort of half encapsulates equality. Equality is defined in the assertion method, but the attributes that contribute to equality are defined in at the place where assertEquals is invoked. A better solution is proper, canonical Assertion Method: two parameters and static.
User actualUser = ...
User expectedUser = ...
assertEquals(expectedUser, user);
static assertEquals(User expected, User actual) {
assertEquals(expected.getUserId(), user.getUserId());
assertEquals(expected.getFirstName(), user.getFirstName());
assertEquals(expected.getLastName(), user.getLastName());
}
I follow this pattern because it keeps my options open. The assertEquals method is static. When I need to move it to my AssertionUtils or CommonAssertions class, then it is just a keystroke away (Ctrl+Alt+V or F6). Also it is common to construct a factory method for a canonical, or expected user. Having assertEquals written with a (User, User) parameter list means that you can hide the expected data behind a factory method and not have it clutter up your test. For me, a custom assertion method is the cleanest solution. The test code is readable because it captures the equality concern without showing the equality implementation. Hey, it’s an abstraction!
Guard Assertion
In tests, decisions are the enemy of clarity. If you read a test method, any decision you come across is a detail that can clutter and obfuscate the meaning of the test. Mock object setup is classic clutter: “when x is called then return y” is a setup step that seldomly is a meaningful part of the test case. But assertions can contain decisions as well. Consider this test:
List users = ...
if (users != null) {
if (users.size() == 1) {
User actualUser = users.get(0);
assertEquals(expectedUser, actualUser);
} else {
fail("user list wrong size");
}
} else {
fail("null users");
}
On first glance it looks complicated. This is how we sometimes write production code… be defensive and always guard against null pointers and illegal state. But remember, tests are different. What do you want your test to do when there is a null pointer? Blow up with a null pointer exception, of course! Forget these fail() methods and just write it so it blows up:
List users = ...
assertEquals(1, users.size());
assertEquals(expectedUser, users.get(0));
It is shorter, and less to take in. But there are less decisions in the test as well. No deciding on sizes and nullability. If the user list of user is null then it blows up with a stack trace. Which is what you want in a test! The conversion from conditional statements to assertions is called a Guard Assertion. Use them to clarify your tests by eliminating unneeded conditionals and decisions.
Throw Everything
I blogged a while back about testing exceptions. Now it is time to say don’t test exceptions. Consider this approaches to handling exceptions in test methods:
public void testSomething() throws IOException,
LoggedException,
EnterpriseException {
...
}
This happens with fancy IDEs. A quick shortcut adds the exception to the method signature. But what’s the point? There is absolutely no reason a test shouldn’t just throw Exception. It is simpler, you can add it to your test method template, you never have to look at it again, and you never have to think about it. Throw Exception. It’s the simplest thing you can do. Another common occurrence is to see this type of exception code in a test method:
public void testSomething() {
try {
...
} catch (IOException ex) {
fail("Could not execute test");
}
}
Yes, it is important to fail the test when an exception occurs. But do you know what happens when fail executes? It throws an unchecked exception. This code suppresses the cause of the exception only to throw a different one. The try catch block is a form of an unneeded decision in your test case. It can be completely eliminated by just throwing Exception in the test method. This way your stack trace points to where the problem occurred, not where it was handled. Both these examples are simplified into the same result:
public void testSomething() throws Exception {
...
}
That’s it: 3 tips for clearer test code. And in far less than 800+ pages!
Igor Popov said,
September 28, 2010 @ 3:24 pm
For the first example why don’t you override Object’s equals method? That way you don’t need to define a custom assertion method… you’ll just use the generic method from the jUnit API:
static void assertEquals(java.lang.Object expected, java.lang.Object actual);
So, what I’m saying can be resumed to:
public class User {
private int id;
private String firstName;
private String lastName;
// other methods, ctors, setters, getters
public boolean equals(Object obj) {
if(!(obj instanceof User)) return false;
User user = (User) obj;
if(id != user.getId()) return false;
if(!firstName.equals(user.getFirstName())) return false;
if(!lastName.equals(user.getLastName())) return false;
return true;
}
}
and in your test method you’ll have just:
@Test
public void testUsers() {
User user1 = new User(1, “Bob”, “Morice”);
User user2 = new User(2, “Joe”, “Elvis”);
assertEquals(user1, user2); // should fail; it will use user1.equals(user2);
// so you don’t need custom asserts (at least for this example)
}
Is this a better abstraction? At least the User is better encapsulated… you don’t need to know it’s fields or other things… you just ask if 2 users are the same… you don’t ask them for each particular attribute if it’s equal if the other…
What do you think?
PS. I didn’t compile this code so it might contain errors, but you should get the idea from this example.
Igor Popov said,
September 28, 2010 @ 3:25 pm
For the first example why don\’t you override Object\’s equals method? That way you don\’t need to define a custom assertion method… you\’ll just use the generic method from the jUnit API:
static void assertEquals(java.lang.Object expected, java.lang.Object actual);
So, what I\’m saying can be resumed to:
public class User {
private int id;
private String firstName;
private String lastName;
// other methods, ctors, setters, getters
public boolean equals(Object obj) {
if(!(obj instanceof User)) return false;
User user = (User) obj;
if(id != user.getId()) return false;
if(!firstName.equals(user.getFirstName())) return false;
if(!lastName.equals(user.getLastName())) return false;
return true;
}
}
and in your test method you\’ll have just:
@Test
public void testUsers() {
User user1 = new User(1, \
Hamlet said,
September 28, 2010 @ 8:01 pm
Equality for testing purpose’s is often different than Object equality. So I avoid this.
Tweets that mention Rich Internet Applications (RIA) » Blog Archive » Clearer Unit Tests: Assert, Guard, and Throw -- Topsy.com said,
September 29, 2010 @ 1:10 am
[...] This post was mentioned on Twitter by HamletDRC, Aleš Najmann, Richard Laksana, Rene Gröschke, Zak Jacobson and others. Zak Jacobson said: Great, concise. RT @DZone "Clearer Unit Tests: Assert, Guard, and Throw" http://dzone.com/rHpb [...]
Jon Davis said,
September 29, 2010 @ 8:19 am
I agree with this article, however I have a reservation about one caveat. In your example regarding “just throw null exceptions”, my concern is that it is helpful to break down the scope of the null exception to scope of the variables being called upon.
In C# there is an ArgumentException which tells the developer which parameter of a method was invalid, i.e. null. When a parameter is needed for something and its value is null, let’s say it is of class type Foo and the code invokes the property Bar, the param name might be foo and the code inside the method would invoke the property foo.Bar. If foo was null, the test will fail with a null reference exception. Were this a unit test, this is exception is considered sloppy and unhelpful because we don’t know what’s null, we just know that something died.
We could implement decision logic inside the catch statement, going against what you had described, to see if the variable that was null was foo and to re-throw based on that information. Alternatively, an approach I’d recommend, is that the unit test should start off just validating the parameters; if foo==null throw new ArgumentException(“foo is null”, “foo”). Unfortunately, in real-world unit testing this gets to be a burden.
It seems it is a limitation of the languages, unfortunately, that a null reference exception will not describe the name of the variable that was called upon which was null. Due to this, there is sometimes no getting around a little bit of “excess” logic that must be written in defensive business logic as well as in helpful unit tests in order to fully communicate the essence of a programmatic error.
Alan Franzoni said,
September 29, 2010 @ 10:07 am
In my opinion, you\’re still using too many assertions.
http://www.artima.com/weblogs/viewpost.jsp?thread=35578
http://blog.jayfields.com/2007/06/testing-one-assertion-per-test.html
Take this code:
List users = …
assertEquals(1, users.size());
assertEquals(expectedUser, users.get(0));
I would have split it into two separate, tests, one called something like testRepoReturnsJustOneUserWhenInvokedWith…, the other one called testReturnedUserHasCertainProperties…
If users.size() is 0 or users is null, the second test would give an error by whatever means. If users.size() > 1 it is probably none of the second test concerns; the logic populating users is probably separate from the logic that creates a User object, and should be tested independently.
If it isn\’t your case, it\’s probably an issue with your code more than with your tests.
Hamlet said,
September 29, 2010 @ 6:42 pm
@Jon – Be careful. I did not say Throw a null pointer exception from the production code. THat is not OK and you need a better message. But from the test code, if a certain assertion line throws a NPE then you know exactly where the problem is without anything but the line number.
@Alan – Don’t confuse “each test method should test one thing” with “each test method should have one assertion”. The number of assertions a test method has depends on context not a rule.
Alan Franzoni said,
October 1, 2010 @ 9:34 am
It seems that my next comment got stuck somewhere… I’m reposting it.
@Hamlet:
Would you produce a better example, then? In your code you’re using two assertions to test two different things in the very same test. Will you give me an example that would justify multiple assertions in one test?
The only one I usually think of is the case where my testing framework has bad support for collections and I don’t want to get mad at looking for difference in unordered data structures, e.g. when using Python’s standard unittest module I usually do something like that:
self.assertEquals(1, actual_d["k"])
self.assertEquals(2, actual_dd["j"])
but the proper way to do it – and that’s the way is done in unittest2 – would be
expected = { “k”:1, “j”:2 }
self.assertEquals(expected, actual_d)