What makes a good unit test?

The other day, we had the task of excluding some projects from a report our system creates. The projects belonged to a category that we could easily identify. We set out to build a test, so we could start coding.

How do we test a report? We can look at it, and see if it did what we wanted, but we were looking for a unit test that we could write to do this. This is where we remembered that we could mock the report. We set about looking at what information we could get from it.

All we came up with was counting the number of lines, and making sure it was the same as the number of projects not in Category A. I’m pretty sure that was lame. The way I described it to my pair was this:

If someone comes and does something to exclude a different category (B), and that breaks our code somehow, our test could still pass, because there were the same number of excluded lines from Category B as from Category A. Or a page break enforcement could change the number of lines.

One thing I’m thinking of is that we can produce red then green, but still not have learned much. And I’m aware that from an empirical perspective, passing tests really don’t tell us anything. It’s the failing tests that tell us stuff. Passing tests are just indicators of an absence of failing tests.

What I want to know is this: how do we write strong tests? How do we focus in on the real essence, and write a test that’s tight enough to be really meaningful?

GeePawHill sez…

Your insight is correct. The right number of passing tests is always a judgment call, and in the beginning, that judgment is still pretty weak. It’s also easy to “overcode”, that is, write more code than your tests call for. It’s why TDD is such a tricky sport to master.

Anyway, with a report, I wonder if what’s needed is a more micro approach. (I  call them microtests , actually.)

If I’m creating a report, there are several things I might want to demonstrate. Since I restrict a single object to having a single responsibility, I suspect my Report class will be what I call a host class. All it does is hook up a few other classes and let them run. For me, that’s liable to be untested. After all, declaring a couple of instances and cross-connecting them is pretty hard to break. Remember that TDD is about testing things you think can break.

My feeling is that I just don’t know enough about the rules you wanted in the report. Does it have to be paginated? Does it need to deal with 0 line-items? many? different types? Or is the important thing that it loads its incoming fields and calculates derived fields? Does it sort? and so on and so forth. Give me some more detail and we can sketch how the TDD might really go.

I constantly start one test, e.g. ReportTest, only to immediately decide that its too much, and I need something much much smaller to get started. Thank God for stacks: I push the starting test onto the stack and go after the next one, and so on, until I find something I’m sure I can do.

4 thoughts on “What makes a good unit test?

  1. Sean McMillan

    First Level: There is always a test you can write that will be better than no test at all.

    Next Level: The point of the tests is to give you confidence that you haven’t broken functionality when you make changes. They protect you from errors, not from intentional sabotage. You need exactly the right tests to allow you to make changes without fear.

    Now, what is a good strategy to drive out fear? If you start with Testivus you’ll be in a pretty good place. Some of the advice in there comes from the great mocking wars of 2007, when people argued back and forth over if a test was truly a “unit” test. I suggest that you do not worry about that definition, but rather ask if your tests are useful and flexible, and don’t worry about the level they’re testing at.

  2. Tracy Harms

    The situation with software tests is not the same as with the tests of science. When it comes to scientific theories we’re interested in universal claims. In software we can be working with an existential claim. In this case: There exists a report that excludes this category of items.

    But, you might counter, a test might be flawed. And indeed it might. But what we’re working on here is a problem of making, not explaining. That difference affects what we need think about in writing tests. Unlike scientific theories, which are more valuable when they’re very general, software features gain value from being very specific and narrow.

    The test of the moment keeps us in contact with the specific enhancement we’re coding. If future enhancements end up breaking features while allowing original tests to pass, I venture that’s simply not something we can prepare for as the original tests are written.

    1. Angela Post author

      Thanks, Sean & Tracy!

      @Sean You said “The point of the tests is to give you confidence that you haven’t broken functionality when you make changes.” I’m puzzled by that. I guess I’ve thought of it as a benefit of tests, but I’d imagined that the point was to create a clear statement of the problem (or need) you’re coding to solve (fulfill). If that’s not the point, I’m definitely missing something in my understanding here.

      Lost my copy of TDD by Example. Better remedy that. :)


      So tests can be useful, without being as rigorous as scientific tests. That makes sense to me. Still, doesn’t it make sense to ask how they can be stronger? It’s definitely possible to write tests that don’t even ask the question we mean to ask. As an extreme example, I can write a test that tells me whether a method returns an integer. If I tell myself that I’m testing for a *particular* integer, my test will turn green, and I’ll go on my way “fat, dumb and happy” as my Dad used to say, and I won’t have accomplished anything. Right?

      You said this: “If future enhancements end up breaking features while allowing original tests to pass, I venture that’s simply not something we can prepare for as the original tests are written.” I have to question that. We can’t do anything to make our tests tightly focused, so they’re less likely to break? Is that really what you meant?

      I know, more practice. I need lots. But I’m also still *curious*. Dang that constant nagging curiosity. :)

  3. Sean McMillan

    Looking at The Extreme Programming Core Practices, I see that things may be changing, so take this as my personal understanding.

    Tests come in two flavors: Unit and Acceptance. Acceptance tests define your problem, and tell you when you’ve solved it. When all the acceptance tests pass, you’re done with that story. (Acceptance tests are defined, not necessarily coded.) Acceptance tests communicate from the customer to the developers.

    Unit tests are coded, and they are a tool written by the developers for themselves. They help the developer to build the best design possible, by forcing them to think about the design from a client’s perspective first.

    For me, it all comes back to good design. The best design for a piece of software is the one that perfectly fills the needs of the system today, with no hacks, and no unused extension points. The way you get there is by refactoring to improve the design of the code at each step. In order to refactor, you need tests to assure you that you haven’t broken the code as you change the design. Hence, unit tests.


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>