Code Coverage Is Not The Be All And End All

Patrick Work Related

My pair and I were talking about getting some code coverage results for our project (it has been a while since we have the last report because of various changes in environments and breakages caused by the move to Java 5).

We managed to finally get a report and we were talking about parts of the system that came out tested. We picked a few parts at random and it was interesting to see the different styles of tests that had been written over time. I like to think that my style of tests (at least my current style) tends to be very user-centric, aimed at representing whatever use case(s) I am working on at the time. I like to make sure that the tests I write leave behind the business reason why the code that I wrote actually exists.

In contrast to my own personal style, some of the tests we inspected seemed to actually test the structure of the code that was written instead of just the business rules. Although I was glad to see the code there was actually tested, I find that these sorts of tests (probably closer to white-box testing) are not as useful as those based on business rules (think black box testing) because I find they tend to require more changes and therefore are much more resistant to refactoring. The reason for this is that if I change to code to support a similar but different business rule, can I be confident that all the previous business rules hold as well? Probably not.

I find that some of my tests might actually end up testing a little bit of code a few more times, but I have more confidence in the system that if someone decides to change the implementation, the business will still be getting the same behaviour out of the system. Oh, and I don’t think that code coverage metrics do not add value – they certainly highlight parts of the system that need testing, but like everything else in software development, actually needs just that little bit more judgment and thought to interpret the results.

Can I Have That, and While You’re At It?

Patrick Agile, Work Related

Agile tells us to the do the simplest thing that could possibly work for the things that we need right now. Why do we do the simplest thing? The answer: Because it is cost effective, especially because the future is unsure. For example, when the business does have something similar, are we sure they’re not going to want it to do something more that will need more work done? Or perhaps the cost of making something “general” could be better spent on more immediate priorities, because if it never gets used, the cost of building (not to mention maintaining) the general infrastructure is wasted.

So what if your business tells you that they want something for now, but make it general, so if they want to do something like it (but not exactly like it) they can reuse the infrastructure to do it again? Even if the business or the analyst is driving this requirement, it reeks of having a bad smell, and is probably driven by a Fear of Waiting. To me this could mean one of several things:

  1. Release Cycles are not often enough – Waterfall projects typical schedule a release for anything between 3 months up to a year or more. If requirements do not get in, it is a very long time until they are revisited. Agile addresses this with the Release Often principle that emphasises more requirements prioritisation over requirements identification and freezing.
  2. Too many competing priorities – With smaller release cycles providing more restrictive timelines, the next issue is working out what can be fit into that release cycle. Businesses typically have many competing priorities, with waterfall models trying to fit all of those priorities in (leading to bigger release cycles). Agile attempts to get people to prioritise what is most important to them right now.
  3. Not Enough Bandwidth – The amount of business value that can be delivered is a function of the fixed time that can be available and the amount of work that can be produced by available resources. I am specific amount the amount of work produced instead of simple the number of people because it is not simply about the number of developers available (read the Mythical Man Month if you want to learn more). Typical waterfall methods rely on judgement (estimates that become promises) to give a guide as to how much can be provided. Agile provides better metrics in the form of a historical moving average of points (e.g. storycard points) per iteration that indicate a better guide as to what can be delivered.

So what can we do to alleviate this fear? Agile has many values and two key ones that spring to mind is around visibility and communication. Visibility is about making people aware of issues (in this case, a fear) and to try to work out what the real motivation is (especially if it is not a real business need). Communication is about making the process transparent and how it is (or can be helping if it is not) to alleviate those concerns.

The Parrot’s Theorem

Patrick Books, Work Related

I realised that most (ok, all) of my reading of late has been entirely non-fiction (and probably excessively IT related) so I thought I should get back to the fictional genre. Since I was in the city on Monday just gone, I dropped into the library and, for no other reason except that the blurb read well, picked up a copy of The Parrot’s Theorem written by Denis Guedj (and translated from French by Frank Wynne).

Anyone who knows me I’m not too fond of studying mathematics, but for some reason, this book, classified as Math Fiction, seemed to catch my eye. This French best seller’s is set in a Parisian bookstore, detailing the story of a mysterious cargo shipment of a Mathematical library, a talking Amazon blue parrot and somehow managing to form a link between the rest of these as well as the mafia and the death of the bookstore owner’s long lost army friend. After finishing the book last night, I would not be inclined to say that this book’s bizarre and intertwining plotline was definitely not its strongest point. As funny as this may sound, the plotline simply set the stage for the author to describe (what I hope to actually be factual) the history and evolution of over 2000 years of mathematical theorem’s and proofs.

Unlike the maths classes that I remember, that simply detailed formulas and repetitive rehearsals, this book gave life to a number of world famous mathematicians and the circumstances in which they developed their theorems. I had no idea, for example, that mathematics (a simple definition of a link between a theory and a proof involving numbers) was really the result of a number of philosophical arguments and there continues to be a strong relationship between mathematics and philosophy today. Most of the famous ancient mathematicians were fore mostly philosophers, and then followed by their role as mathematicians. The author continues to demonstrate that modern day fundamentals are in fact, only fairly modern. He discusses the concept of zero and how it was only invented in 733AD (that’s 1272 years ago) even though mathematics has been around almost double that time.

The biggest thing that this book has to offer is not really the education of the number of theorems it covers, but the really interesting way that they came into being.

TheKua Rating: Fictional value (6 out of 10), Educational value (9 out of 10).

The Forces That Be And What To Do About Them

Patrick Work Related

Every project is influenced by a number of forces that make it less than ideal. Sometimes these forces can be business decisions that create a rate of change faster than a development team can meet with perfection, other times it is technical limitations that prevent ideal solutions from being implemented, and sometimes it is simply the misappropriate application of tools and libraries that can’t fit all of the needs of the business.

Regardless of the source of these forces, anything that has a significant impact on a team delivering perfect software is worthy of documenting. Identifying and clarifying such influential forces provides newcomers with a better context to the decisions that a team might have made along the way, and better yet, provide a way of highlighting them and their undesirable effects, so they could be removed if they no longer apply.

When it comes to the technical forces affecting the level of code being produced (e.g. a library not behaving accordingly, or an unforgiving production environment), the best way that these forces should be documented is in the form of tests. Developers would not be expected to run these, and should be left for execution by a build machine running in a production-like environment. Continuous integration serves as a great mechanism to inform us when the undesirable conditions that forced the creation of less-than-perfect-code, goes away, and highlights what can be changed to bring it that much closer to perfection.

Meaningful Names

Patrick Work Related

Bar the Foo, no one really cares about you.

I like good names for all things, even for test variables, even though you may not care what value they have. However, even if you don’t care what value it has, please think about people who will maintain it and tell them (by giving it a good name) what it supposed to represent (how about something like anyName, or anyEmail?).

Pair Pong Perfection

Patrick Agile, Work Related

Our last iteration introduced full time pair programming as an accepted practice (in contrast with our previous practice of “pairing”). The last couple of days I’ve been pair programming with one of client’s developers and because we seem to work well together, it hasn’t been hard to all building the synergy. As part of pair programming, I suggested we give pair pong programming a go and despite it being a new practice, we’ve found it excels at providing us with the rhythm that makes us amazingly productive.

I think there are a couple of reasons why pair pong programming works. Firstly, it’s got all the aspects of pair programming (such as constantly discussing design, questioning and trying to improve on the code that is written). As I mentioned earlier, it’s got the rhythm of proper TDD (writing a failing test, and then getting that test to pass) and finally it’s really good for actually forcing pairs to drive and navigate for an evenly distributed time. I think we’ve almost got to the point where when we write the test for each other, the other pair already understands what it would take to implement it and then the code just flows.

With the right partner, pair pong programming can be such bliss…

Pair Programming Patterns

Patrick Work Related

On our current project full pair programming has been more of the exception than the rule. In our current iteration we are trying full pair programming and we’ve seen some really good things come of it. I was thinking of writing up some entries here for Pair Programming Patterns/Anti-Patterns because it would be good for future reference but maybe there is some resource out there already. Does anyone know of such a resource and would it be worth writing some up?

Why Rely On A Compiler When You Have Tests?

Patrick Work Related

I’ve been playing around with Ruby and reading up on Ruby on Rails lately (yes, I have joined the bandwagon) and it interests me how Ruby’s popularity (at least in some of the circles I partake in) seems to have resparked the debate of dynamic languages being better than static languages. Static language advocates will suggest that they have frequently less bugs in the class of wrongly assigned data types because the compiler provides them with one level of feedback. Of course this does not mean that static languages (such as Java) exclude this class of bug completely (which is why we have our most favourite ClassCastException). This level of feedback comes at the cost of its programmers having to endure a much higher level of verbosity.

Although I code everyday in a static language for my current project, I’ve written some development tools and worked on a project all based in Perl (gasp!) and enjoyed the freedom its dynamic typing provided. My suggestion for people looking at using dynamic languages for a production system is that the higher level of power that the language provides also requires a corresponding higher level of discipline from developers who use it.

My response to people who strongly advocate for static languages solely based on its type safety is that this issue goes away if one actually writes tests for it (and even better, test drives the entire system). It amuses me to no end to see how many projects (both production and even many open source projects) fail to even compile when you take a copy from their source control system. Although seeing a project fail to compile gives me one level of feedback (yes, you won’t even be able to run it), I think it’s bad that when you do see a project compile successfully, it gives people a false sense that the system must be working correctly. Yes this requires high discipline from developers to write useful unit tests but is this any different from what you are currently doing or what you expect your team to be doing? I really hope not.