The term ‘unit test’ was being thrown about today at work but so many things about its context were just plain wrong. Just because our testing framework is called xUnit and each method happens to be called testX doesn�t make it a unit test. People have spent (far too much) time attempting to classify all types of tests coming up with categories like functional, acceptance, integration, or unit-integration tests. In my humble opinion, I tend to think of all tests falling along a spectrum, with unit tests (those that test just enough) on one end and the functional/acceptance tests (those that test too much) on the other.
Each test along this spectrum is important but the speed and quality of these tests will be one of the factors that determine how fast you can change or add to your code base. By all means, slower, bigger tests further along in the spectrum can be substituted for faster, lighter unit tests, but its consequences usually lead to a project that is less agile and more waterfall-driven.
Here are a few signs that your tests may be masquerading as something more than just (better, and faster) unit tests:
- Your application must be �deployed� to run some of your tests;
- A user must interpret console output or some other type of result for the test to fail;
- Each test takes too much time to run (it should be possible to many of these within a second);
- Excessive amounts of set up are required for each test (a smell that this might be integration test and/or testing too much); and
- Tests cannot be run without some external resource (such as a database, specific computer or some file);
Smells like mutiny to me 😉
I’m hoping it’s not a side-effect of the two weeks I spent on board a sailing ship earlier this year then… 🙂
Pat, Amen to that. Not that I totally agree with the implication that other types of tests are not useful, but they certainly aren’t unit tests. I have heard the term “in-container unit test” thrown around and that always makes my skin crawl. Hence my aversion to things like cactus and those frameworks that try to unit test struts actions…icky poo!
James, my post was written in a little bit of a haste, so I did not intend at all to imply that other types of tests are not useful. In fact, I am a big believer in having a good suite of integration and unit tests to ensure that the code works in the “real world”. I suppose my biggest gripe about people that discount the value of unit tests (which I know you are not one of) is that they fail to realise that by having these, the feedback/build/test/develop loop increases significantly, making me less confident when making code changes and effectively slows me down.
I also use an integration/acceptance test when initially working on a bug (because this is the form that they can most easily be reproduced after being reported). As I track down the source of the bug, I tend to write unit tests along the way, so that when I have isolated the exact part that causes the bug, I have a much faster mechanism for reproducing the bug, allowing me to fix it quickly and easily.