I just came off my last holiday where I got through reading two books that I wanted to. One of these was the heavy but insightful tome, that I dragged in my suitcase, Continuous Delivery. Before I begin, I should warn you that I work for ThoughtWorks, and that both Jez and Dave (the authors) are and were my colleagues so take this into account with my review.
What I enjoyed
Continuous Delivery is a very practical book that covers a huge range of topics from cover to cover. I feel that they found a good balance between giving just enough advice (and more importantly why) with plenty more references for those who want to find a topic to delve deeply. I felt it really lays down the ground work for many projects (but not all) that need to see the light of day.
I think it’s also great that they not only outlined the practices and tips, but also outlined the principles. It gives the book much more merit and will make sure it stays relevant as tooling and new techniques are invented. On the flip side, they do give enough advice on tooling, and, at least, current up to date advice on how one might achieve things.
I like that the authors expressed their opinons strongly on some of their tooling such as, if you’re going to use a commercial source control system, how it should be used and why, or what the problems with the build process in maven or the limitations of various toolings are.
The book is peppered with war stories. I think I know some of the examples they cite, or at least, heard about it from other people and I think some of them are really strong stories about some of the ways people approach deployment.
What would make it perfect (for me)
I’m not sure if they framed the context of Continuous Delivery enough in terms of the types of applications it’s appropriate for. I know that there’s a current (constant) movement in the startup community of “just putting out there.” I think that is an alternative approach that works for some amount of time (or at least changes some of the testing/feedback process). I think for throw away apps (this one is always hard to judge), it may not be worth investing in the same degree of how much effort you put into the process. I think probably the closest advice but should have been expanded upon was, “be pragmatic about how much you automate.”
One particular headline (out of the very many) grabbed my attention, “Test targets should not fail the build.” After reading that section I think I understood what they meant, but it should have been relabelled, “Failing test targets should not immediately *halt* the build” -> it should still fail the build at the end.
Finally almost all of the chapters, for me, had lots of great advice and in working reaffirmed many of the conversations and thoughts I’ve had over my career. I did find the final chapter about “Risk Management” a bit thin, feeling like it was added at the end and either deserved much more attention or should have been dropped. It felt like it had a different tone, or was written at a different time and couldn’t quite see how it would fit in.
Having said that, these last bits are only minor points in a huge volume of rich information and is essential reading for anyone who cares about their profession.
Putting this into practice
For most of the clients I work on, the clearest and most difficult part of the organisation will be the (im)maturity of the operations side of the organisations. Most are already not set up for success with completely different parallel hierarchies from development and competing goals that go against constant change. I still see this, “last mile” problem being the hardest (at least for me to solve). I am hopeful about the DevOps movement and hope that the IT parts of an organisation can only take it much more seriously. I’m hoping this book shows how more companies can achieve this.
Thanks for the review Pat.
I think you’re right about the last chapter. It’s the one that has least stood the test of time for me too. I’m actually planning to write a new (and much shorter) book on the management side of things next year to address this deficiency.
In terms of “throwaway apps”, I think you’re referring to the developer anarchy model. I do have real reservations about this model, for three reasons. First, I don’t think that it’s appropriate for all (or even most) startups. Many of the successful startups I have seen actually use high degrees of test automation. IMVU for example has an enormous set of automated end-to-end acceptance tests that are highly parallelized to run in 30m and that form a gate upon deployment. My observation is that usually the minimum viable product should involve very little software – and probably no tests that aren’t A/B tests – but once you start building software for real, you need to be pretty rigorous about all the things we discuss. Advanced class is that your monitoring and your automated functional tests become the same thing. We didn’t cover monitoring and A/B testing in much detail simply because we hadn’t seen much of it – that for me is the most fascinating growth area right now.
The second reason I have reservations about developer anarchy is that it is being done by a bunch of really great developers working in very small teams in a very particular niche (even within startup world). If you need to read a book to learn how to do this stuff, it’s almost guaranteed you shouldn’t be doing developer anarchy, and thus those people aren’t a target audience for the book. It’s something that people who are at ri level are doing. Nobody who is at shu or ha should be contemplating it. Thus I think that even teaching developer anarchy to people who aren’t at ri carries its own dangers.
Finally I just don’t like the term developer anarchy. My observation is that in real life, in political terms, anarchy is usually created by somebody as a way to gain power without being seen to do so. Developer entrepreneurialism I can get behind.
Thanks again for the review!
I look forward to reading the next book as well. I’m not so sure I would classify it as the “developer anarchy” model as well. I think the person espousing this exaggerates some of the practices and I know that behind the scenes, what is said isn’t exactly what’s going on behind as well. I also agree with you that I don’t like that term.
I guess I found interesting that git hub doesn’t seem to have a lot of test automation (but I could have misinterpreted that).
Perhaps I was thinking more of some stories I’ve heard from the lean startup movement as well as those apps that have a very limited lifespan and unlikely to evolve (e.g. website portal for a single marketing event – like a launch or something). Perhaps there’s not enough of them to warrant and understand there’s a risk these things always become more than just a “throwaway app”.
Pat, thanks for the review. You are right about our meaning failing tests fail the build, but they should wait until the run is finished to fail it so that you can get a full picture of the nature of the failure.
At my company we display results from our build grid as we run so that you can get fast feedback on failures, but won’t get a complete picture until the build fails. This means that we have often fixed the failure before the whole test run is finished (for us about 30 – 40 minutes).
I think that CD is appropriate for most projects. Like TDD it can be easy to say “it isn’t worth it for my project” and if testing alone is your focus then you may be correct. But like TDD, there is more to CD than testing. It changes the way that a team interacts with the software it creates in a variety of ways. Deployment flexibility is an immediate side-benefit.
My own application runs on 95 servers in production at two different data-centres or I can run it all on my laptop. The software that deploys it in each of these scenarios is the same. This has a big effect on how the software is used during development, and sometimes later.
There is an in-built assumption that CD is costly, but I am not sure that it is costly compared to a true evaluation of costs for the life of a project. If the deployment of your software is simple, and not likely to cause an error, then it is simple, and low cost, to automate too. If your business logic is simple enough that you don’t think you will create a bug, then building and maintaining tests for it will be simple too. The level of assurance that the automation of these facets of a project brings significantly enhances the willingness of the team to be bold and make valuable changes.
In reality there is certainly a cut-off, a point at which CD is too heavy-weight, but I am convinced that this is at such a low-level that it is largely irrelevant. If I was working, alone, on a project where the results would be discarded in less than 3 months time, or that was only ever going to be used by me, then perhaps I would skip the creation of a deployment pipeline. For anything else I think that you are likely to see some pay-back.
The problem is that the the pay-back is amortized over a long period of time, whereas the cost of creating the tests and deployment tools is more clearly visible up front. The perennial problem of the difficulty of measuring software projects means that these sorts of costs can be hard to justify up front. However, I think that you can make the same argument against unit-testing and TDD, and I know that you think that that is a good idea 😉
Hey Pat
I think it’s important to distinguish between the lean startup movement and throwaway apps.
As Dave says there’s probably cases where CD is not worth it, but they’re really a tiny minority.
As for the lean startup, Eric (from what I’ve read, and when I interviewed him) is pretty adamant about good automated testing. One of the quotes I like from his book is “Defects … interfere with our ability to learn and so are dangerous to tolerate in any production process.”
@Dave – Thanks for your comments. Good to hear your point of view
@Jez – I agree it’s useful to distinguish between throwaway apps. Fair enough about the lean startup stuff. I haven’t done that much reading around it – just more talking to people and I didn’t get the impression that automated testing had such a heavy emphasis. Thanks for making me more aware of it.