I’ve been working with a large scale legacy system (in the Michael Feathers’ sense). Strangely enough there are a small handful of tests, unfortunately most of them falling by the way side having not been added into an Continuous Integration process. Many of them no longer look relevant, others, unable to run without a specific person’s machine (due to configuration files or special databases). Others look like a mechanism for people to simply diagnose issues with plenty of tracing statements. All of this was an inevitable system without someone running all the tests all the time.
Any serious team serious about continuous integration will think hard about their development testing strategy, considering approaches that allow you to run tests all the time. The strategy needs to consider a good way of preventing side effects from other tests (global state, global configuration, or left over state in external resources) balancing out speed with good quality feedback. Staggering build processes into smaller chunks, such as using build pipelines are a great way of achieving this.
Please don’t let leave your tests to die a sad and lonely death. It will take discipline and effort to keep them running yet the results will often pay back for themselves very quickly on any system with a reasonable lifespan.
Couldn’t agree more. It’s a shame to waste tests like this, and it seems like the only reason that these tests end up falling by the wayside is that there wasn’t a proper (read: simple) test framework that was put in place to make it easy for anyone to add tests as they went.
The test framework is the one that gets integrated into the CI build system, and anything added to it is always automatically run. That way the programmers can easily add tests and don’t have to worry about special setup.
Keep tests simple, easy to maintain, and you’ll rapidly grow a lot of them. Incorporate them into a simple testing framework and they’ll have a place to live and continue to give useful results for the life of your project.
Thanks for the post!
Patrick,
Good post. I was once in a position that I was the only one writing tests in the team. I was checking them in, but the concept of a full build that runs the tests also was so far away, no one cared about looking at them. I was just trying to get people writing tests, and unfortunately, that stopped there.
You’re absolutely right, and I’ve learned from it. I hope other people don’t throw away usable tests. Integrate them into the build.
Gil Zilberfeld
Typemock