Tell Me What You’re Testing

It still seems that no matter how mainstream particular technologies get, many developers do not clearly understand how those technologies should be applied. One developer I’ve worked with in the past before always stated that they wrote ‘JUnits’, which is admittedly, better than people not writing tests, but they still didn’t make it clear what sort of testing they were doing. I’ve also seen this same mistake applied in a build for a suite of tests marked as ‘FIT’ (a different framework for testing).

The thing that bothers me about statements like this is that JUnit and FIT libraries are a means to an end. JUnit is great because it is so mainstream and simple that people can quickly understand how it works. Stating that someone is about to write a JUnit is like saying to me that they are about to go and write java (well duh!).

One of the most useful things I find on a project is building up a common understanding of the groups of tests. I have my own starting point, and adapt labels depending on how strongly people feel about their own understanding levels. I like to begin with at least three levels of tests including:

  1. Unit tests
  2. Unit integration tests
  3. Acceptance tests

The first level of tests are sometimes better referred to as class level tests, and are intended for testing code at their most granular level (such as a method, etc). They should be able to run without any external dependencies.

Unit integration tests – something that I may refer to as functional tests should be applied when classes have a higher level of interaction – sometimes with each other, or more importantly something that will slow down the tests suite – such as a file system, a networked system or a DB (yes, even an in memory DB). These tests typically are IO bound, and may even require external configuration or setup before execution.

Acceptance tests are intended to test the configuration and the ‘wiring’ up of your application. Tests will be the longest running part of your suite since they are testing parts of the code over and over again through various paths, but these ones validate that the application is meeting the business value that drives it. These tests should simulate real life scenarios and interactions, but a failure in this suite should typically have a matching failure in the unit test build.

The choice of technology should always be secondary to the effort of communicating your intent, and should be reflected by the words that you choose. Saying what sort of test you writing is always much better than how it is going to be done.

One Reply to “Tell Me What You’re Testing”

  1. Agreed. In picture format, I penned this a while back:
    http://photos1.blogger.com/blogger/7546/664/1600/testing-pyramid.jpg
    … which is set in context here:
    http://agiletesting.blogspot.com/2006/02/thoughts-on-giving-successful-talk.html

    You’ll notice my drawing is Python-centric, but the concepts mostly match… Also, I too think that folks should say “UI test/smoke test/acceptance test” instead of saying “Selenium test”… And that follows that program directories should have a folder called “tests/acceptance” or “tests/ui”, not “tests/selenium”. 🙂

Comments are closed.