Project management is a funny thing. It seems like once something is converted into a number, the number is often all that matters to traditional schools of project management. Plans get created, promises get made, yet little diligence goes into ensuring the number is based on sound thinking, and that the basis for that number remains correct. Maybe that also explains the state of the current banking industry, though that’s another post entirely.
Most people fail to understand that estimates have a half life, and often a very short one. Even results produced with agile estimation techniques, have an expiry date that is often ignored. Magic numbers (aka, the estimates) have an expiry date because in order for them to be even slightly reasonable, they need assumptions to be made, and it is often these assumptions that expire. Gantt charts are awful at highlighting what those assumptions are, let alone tracking when they fail to be met.
A recommended practice for agile estimation is to get the people who are going to do the work, to estimate. Put these estimates on a shelf to ripen for a year and, presto, you have a new team to which, the original estimates no longer apply. Even given the same team, set to work on something else only to return to the system, will have lost some flow, forgotten some things and need more time than if they just continued to work on the same system. Environments often change, and the underlying needs for the business change even more frequently.
What can we do about it? I refer to Lean Software Development’s principle of the Last Responsible Moment, to not only make decisions, but when collecting (and keeping) estimates. Collect estimates now if it helps you make a better decision, discarding the estimates if you choose not to immediately implement the project. You set yourself up for failure if you choose to estimate now, knowing that the project is to be shelved for “only three” months and keeping those numbers as if they are in any way relevant. If you don’t need to make a decision now, work out when you need to make the decision and defer collecting estimates until just before then.
Hi,
You make an interesting point, I would argue though that the half life is even shorter. Consider the Cone of Uncertainty (see The Mysterious Cone of Uncertainty), I interpret that as suggesting that irrespective of whether you leave the estimates until the last responsible moment (a principle that I wholeheartedly agree with and support) your estimate still carries the potential to be wrong and as such, the estimate degrades pretty much instantaneously.
I think that rather than working out how long you can put off estimating people should focus instead on getting their units of work in to similar sizes so that they can measure how long a card takes to get through their pipeline and base their velocity / delivery predictions on that instead.
Hey Pat,
Thanks for echoing what I wrote a while ago on my blog.
http://journal.bitshaker.com/articles/2008/05/01/what-is-the-shelf-life-of-a-requirement/
I tried to break down when estimates and even the stories themselves expire and need to be reevaluated. Good to know someone else out there shares my pain.
Joe
>> “Even given the same team, set to work on something else only to return to the system, will have lost some flow, forgotten some things and need more time than if they just continued to work on the same system.”
If your estimates are in relative terms (i.e. points), then “need more time” will come out in the new velocity. There would be absolutely no need to revisit estimates. None.
Also – I disagree that estimates with a new team, “no longer apply”. A new team may disagree with relative estimates (“This 5-pointer should be an 8 pointer because we really understand these other 5-pointers but don’t really understand this one). Fundamentally, however, without specific gaps in knowledge about specific domain areas or technical areas, the original estimates should be relatively close.
I agree with the conclusion – delaying to the last responsible moment – but the arguments that lead to the conclusion are, I think, misleading.
At one point or the other, you have to collect estimates. As a Project Manager, you have to get the data from the team, and then add logic and common sense to it to provide your estimates. I agree that waiting till the last moment probably is the best time to make your estimates. But the question is: When is the best moment? Is it the day you start the project? a week before? a month before? or even a year before?
Now about estimating in relative terms, does this kind of estimate actually fly with stakeholders?
I invite you to read a series of articles on lessons learned in estimating, it does provide some good thoughts on the subject.
Thanks all for your comments.
@Dan – I agree that this is one approach that can work, particularly when someone is encouraging the use of Kanban, Flow and Cadence. I’m not sure if all organisations would accept this.
@Joe – Thanks!
@Adrian – I think relative estimates get you to a certain point. Knowing how this is interpreted by project manager, in too many organisations they still need to project this onto budget and time and this is what I suggest is what degrades. Changes in velocity is definitely one way of determining this impact, but it does have downstream effects. I guess this is what I was thinking of more so.
@PM Hut – When is the best moment? There will be no specific answer that will suit all projects. As a guide, I would ask what would be the consequences of delaying estimation? Does this mean other plans cannot be made or committed to? As I said before, I would have some time set aside for estimation, and work out what’s the latest I could run these estimations so that I can still make a decision, but no earlier. Relative estimates are one approach encouraged by agile estimation. It is suggested as a more advanced technique to help people understand sizing (versus timing). I’d recommend reading the Agile Estimation and Planning book linked above for more information.
Interpretation by the project manager is an issue no matter how recent the estimates are, no? I guess I don’t understand – are you suggesting that the interpretation degrades? Or is it that different project managers will interpret differently (which could happen the day after the estimates if the PM gets hit by a bus).
Changes in velocity have downstream effects. I don’t understand. Changes in velocity impart changes to the release schedule, it’s true, but it doesn’t invalidate the estimates. If you estimate and your lead developer gets run over by a truck, you have an impact to velocity, but it doesn’t invalidate the estimates. This is the beauty of relative estimates. Their shelf-life is like Spam…
I’m sorry if I appear stubborn, but I’m missing your premise.
Hi Adrian,
You’re not being stubborn. I’m just not communicating well enough. I’d love to chat (because we all know how the written word works perfectly…)
Yes, interpretation is an issue regardless. However, I did mean the first question you posed (Interpretation degrades). For some organisations, PMs need to map estimates into some sort of plan (either budgeting, resourcing, or just to decide go vs no-go decisions). Their mapping process often involves different scenarios (likely, optimistic, pessimistic, etc) and often assumes the same team will be doing this. Obviously all these assumptions (explicit and implicit) start to break down over time.
I agree that velocity doesn’t invalidate estimates if you do relative.
I am trying say though that even relative estimates have a shelf life because assumptions about what that story involves start to degrade, or change over time (and don’t necessarily get reflected). The relative work changes as assumptions change therefore I start to have less confidence even in relative estimates.
Hope this makes sense now!