The “magic” of just enough estimation

Estimates are worthless for project planning. They're always done wrongly. Here’s a better approach.

Photo by Jon Handley on Unsplash

Are your estimates accurate?

If your estimates are accurate, you can skip this story. The other 99.9% of you should read on. Seriously.

I’ve never met anyone — myself included — who can estimate with reasonable accuracy. Occasionally we get something right, but those occasional accurate guesses are limited to things we’ve already done, and even then our estimates tend to be wrong. There’s science behind this, by the way.¹

It turns out that we humans are horrible at estimating things. Back in 1979, two researchers, Kahneman and Tversky described “The Planning Fallacy.” It’s now a well established foundation for subsequent research. The gist of the fallacy is simply this:

• Humans systematically underestimate how long it will take to do a task.

• Humans are over confident in their own estimates.

But there’s an interesting corollary to this research — and it’s one we can use in estimating our tasks. Allan Kelly summarized the corollary:

Curiously the literature does show that although human’s can’t estimate how long a task will take, the estimates they produce do correlate with actual time spent on a task. In other words: any time estimate is likely to be too small but relative to other estimates the estimate is good.

So, we’re absolutely lousy at predicting how long it will take to get something done. But, our lousy estimates are, relatively speaking, consistent when compared to our other lousy estimates.

Steve Jobs really understood this. It’s why for decades Apple never preannounced a product. Do you remember watching their annual product launch events? Where Steve would get up on stage and say, “hey, we built this new phone, here it is — you can buy it after the show. Enjoy.” It was a surprise and everyone was stunned, even though Apple had clearly been working on the concept for years.

If you’re new, welcome to Customer Obsessed Engineering! Every week I publish a new article, direct to your mailbox if you’re a subscriber. As a free subscriber you can read about half of every article, plus all of my free articles.

Anytime you'd like to read more, you can upgrade to a paid subscription.

Good estimation

When it comes to estimation, I don’t believe we can accurately predict the future. I’ve seen this play out over 40 years — my empirical results across many projects have convinced me estimates are basically worthless for project planning.

So why estimate?

There is value in estimating, but we need to be strategic when it comes to applying our estimates. We need to be crystal clear about why we’re doing it and equally clear about how estimates will not help us. The why is simple:

• We want to deliver value consistently, at a steady pace, without big surprises.

• We want everyone to have advance knowledge about what we are doing.

In regard to the will not, it’s important to recognize that software development is not manufacturing chain assembly. It’s not construction. It’s research and development — and by its very nature, this means we are exploring the unknown. Throughout much of our day-to-day as developers we’re in new and often startling territory.

We can’t predict what’s going to happen reliably and especially not far in advance — not at the task level. But we might be able to establish a trajectory that we can use for planning.

Knowing when you’re “done”

“But,” I can hear you saying, “my boss needs to know when we’ll be done.” Your employer or client wants a predictable release date. You have to get something out the door before you run out of funding. Your customer needs to beat their competitor to market.

I’ve heard all the reasons before — and yes, they are very real and very tangible reasons for wanting a hard-and-fast launch date.

They are also exactly why we cannot give a hard-and-fast date.

Look at it this way. If you put forward a delivery date based on estimates (for some unique, new thing that nobody has ever built before), and that date is wrong — what goes wrong? If the business has been planning on that date being firm from day one, a lot.

Instead, you have to plan around the fuzziness and uncertainty of the unknown. What happens if you don’t get all the features done in time? What if you can’t support as many customers as you thought you could handle on day-one? What if the system isn’t as fast as we want it to be? If you honestly anticipate problems ahead of time, they won’t be fatal — but if you pretend it won’t happen, any misstep becomes fatal.

This is why estimates are worthless for project planning. They are always wrong.

What generally won’t be wrong is your project trajectory. Most Agilists call it “velocity.” Once we have established a track record, a solid velocity, once we are approaching a reasonably finished state, we can begin to predict a launch target. Not until then.

What we can estimate

Ok, so we’ve established a few ground rules — a few basic facts that we can work from:

1. Humans are horrible at estimation.

2. We’re even worse at fooling ourselves with our bad estimates.

3. Our bad estimates, relative to each other, have consistency.

Basically, we can’t know exactly when. The further out we try to estimate with exactitude, the worse our answer becomes.

But, if we can’t know exactly when maybe we can settle for how fast we’re moving. And, maybe we can strive for a reasonable, though imperfect, degree of consistency in measuring “how fast.”

You can probably see where this is going. We want to estimate our work in a way that establishes our velocity, and lets us predict that velocity.

In other words, if we say, “we’re going to take on these 5 features in the coming sprint,” we want to be pretty darned sure we can handle it. This is our promise to our customer. We need to have a strong track record, we need to deliver what we promise. The worst thing we can do when it comes to customer interaction is constantly break our promises. That will destroy trust, and hurt our customer relationship.

What about planning poker?

Just about every developer has used — or has heard of — planning poker. I’ve absolutely used it, several times over the years. The problem is, every single time the results have been… bad. Not just at an individual level, but organizationally. For instance, at Accenture we almost unanimously agreed, planning poker isn’t worth the time that goes into it.

That’s not to say there aren’t some good things that come from planning poker.

One of those benefits is incentivizing discussion. By now, you probably know how much I value good communication — and this is no exception. More discussion about a given feature leads to better understanding, often discovery of hidden unknowns, and more generally a clarity across the team about what a feature does.

I also like how planning poker strives to protect the estimation exercise itself from outside influence. This probably predates planning poker, finding its roots in “wideband delphi.” Basically, it leans on anonymous estimation because scientific evidence demonstrates that we tend to gravitate towards data we’ve already been exposed to. This happens even if the data is totally random or incongruous. This effect is a cognitive bias and is called “anchoring.” Remember, humans are horrible at estimating.²³

But there’s a lot about planning poker that I don’t like.

It’s not accurate, and it’s not relative. We already know, from the research cited above and likely our own experience, the results are wildly inconsistent. At the same time, it doesn’t focus on truly relative estimation. It’s not about “this” being more or less than “that.” Instead, it often confuses time, effort, value, complexity, and impact. That simply means that everyone involved is measuring something different — so no wonder accuracy is poor.

There’s another problem planning poker runs right into — it turns out, if a deadline is mentioned before an estimate is given people tend to estimate within the deadline. So again we see how we humans are just not suited to create accurate estimates.⁴

Planning poker set out to be a quick, easy-to-use solution to a complex problem. In fact, it’s neither. Across numerous projects, I’ve seen planning sessions turn into long, drawn out affairs that go down deep rabbit holes. At the same time, the game of planning poker itself isn’t really that easy. I’ve seen many teams get confused and lost in the details unless they have careful training and periodic coaching. There just aren’t enough guardrails to keep things on track without a lot of oversight.

So, are there really better alternatives to planning poker? There are. Several techniques have emerged, each of which focuses on:⁵

• Visual estimation (I’m a huge believer in spatial thinking).

• Measuring work relative to other work, never using “points” or “time.”

• Estimation flexibility, usually by moving cards around a white board.

What does good estimation look like?

Before we dive into a better way of estimating, let’s talk about what good estimation should achieve. Exactly what benefits do we hope to garner from estimation? How can we balance it so we don’t waste our time on fruitless activity?

First, let’s set our intention. We want to plan the delivery of our next iteration — be that one sprint, or several — with reasonable certainty. That, basically, is the sum total scope of what we can estimate.

Our intention is to plan the delivery of our next iteration with reasonable certainty. Period.

To maximize the benefits of our estimation, we want to get everyone on the same page about our intended outcome. With that in mind, we want everyone to come away from the exercise with:

1. Clear expectations about what the iteration will deliver.

2. Sufficient understanding and agreement on those delivery goals.

3. An understanding that progress may fluctuate but, by and large, will set a consistent, repeatable cadence. These are estimates not hard facts, after all.

Magic estimation

I’m going to describe an approach I call “magic estimation.” It’s closely based on the technique developed by Boris Gloger, though I recommend a few subtle changes.⁶

If you use the referral button below, you’ll earn free premium access to Customer Obsessed Engineering. Just three referrals will earn a free month!

The planning exercise is visual, ideally done with a white board or wall space so everyone can participate. Estimates are established by placing post-it notes describing your stories on the board. This can work well with digital tools too, like Miro.

Also, the activity is strictly time boxed. My rule of thumb is 30 seconds per story, so if you plan to estimate 60 stories you’ll have a 30 minute time limit. Most of the time, we’re looking at half that number, more or less. As you’ll see below, I strictly time box estimation of each individual story — but, with this technique, time boxing is rarely an issue. Estimation moves very fast, which is the whole point. It really does work like magic.

If you’ve been following along in the Delivery Playbook, this activity should be happening right around chapter 3.3 Identify product increment. Otherwise, plan your activity just before sprint planning, and focus on just one or two sprints at a time.

Lightning fast setup and execution

The rules are simple. Basically, you just have to draw your estimation board — once everyone sees it, the exercise is almost intuitive in nature.