Why Your Estimates are Optimistic

I had an “Ah-hah!” moment at the recent Agile Edmonton meeting. I don’t get a lot of those so I wanted to write about it.

Brian Donaldson from Quadrus was in town to present on the Quadrus Estimation Methodology. You can view Brian’s presentation on the Agile Edmonton site.

Most people expect software estimates to follow a Bell Curve, or Normal Distribution. That is, they expect the most common estimate (the mode) to match the average estimate (the mean). A Normal distribution implies that your actual progress is just as likely to be over your estimate as it is to be under.

Normal Distribution

However, Brian persuaded us that software estimates typically follow a LogNormal distribution. In a LogNormal distribution, the mode does not match the mean, which results in a skewed graph. For software development estimates, the mean is usually higher than the mode, which means that your actual progress is more likely to be over. This made sense to me when Brian pointed out that unexpected things in a project don’t often speed up an iteration (and even if they do the gains aren’t that significant) but they can often slow down a project (and the slowdown can be massive).

Lognormal Distribution

I’m getting to the “Ah-hah!” moment. But let me pull out the soapbox for a sec.

I’ve often been asked for a quick estimate to help get an idea of cost. The typical thing people say to make me comfortable with giving a high-level estimate is “just make it plus or minus 50%”. I’m sure you’ve been in that situation as well. This request always makes me laugh. For, you see, this type of hallway estimate is never based on anything close to a good understanding of the problem. That means that it’s pretty much a useless estimate that will probably come back to haunt me in the future. I have to laugh at this request because people look at you funny if you start to cry.

OK, enough soapbox and back to the point of this story.

Here’s the “Ah-hah!” – plus or minus 50% doesn’t exist. Assuming that your estimates do indeed follow a LogNormal distribution (and I’ve certainly seen that occur over and over again), minus 50% is nowhere near as likely as plus 50%. If anything, minus 10% might be as likely as plus 50%. Ah-hah!

So, next time somebody asks you for a “plus or minus 50%” estimate, you can feel free to explain the inaccuracy of their statement. Ask them if they’re willing to take a “plus 50% or minus 10%” estimate. Or, since they’ve already got a number in their head that is significantly lower than the number you’ve got in your head, just do what you’ve always done and pull a number out of your ass. Just try not to cry while you’re doing it.

Image credit: jespis