The project ran over budget. Was it the estimate?

The project is 40 percent over budget, the board wants to know why, and the first question in the room is who did the estimate. It is a reasonable question. It is also, most of the time, the wrong place to start.

An overrun usually has several causes, and the estimate is only one of them. The useful work is to separate them, because each one is fixed by a different change, and blaming the wrong one guarantees the next project overruns the same way.

What the research says, and what it does not

The numbers are sobering, and each comes with caveats worth knowing before you quote it.

McKinsey and the University of Oxford studied more than 5400 IT projects and published the results in 2012. Large projects, those with an initial budget above $15 million, ran 45 percent over budget on average, 7 percent over time, and delivered 56 percent less value than predicted. Their explanation of the causes, though, comes from surveys of IT executives rather than from the project data itself.

Bent Flyvbjerg and Alexander Budzier, in Harvard Business Review in 2011, looked at 1471 projects. The average cost overrun was 27 percent, but one project in six was a black swan, with an average cost overrun of 200 percent. Their sample was mostly public sector, with an average project cost of $167 million.

Closer to the question of causes, the Project Management Institute’s 2018 Pulse of the Profession asked practitioners for the primary causes of the projects that failed, allowing several answers. Inaccurate cost estimates were cited by 28 percent of respondents. Changing organizational priorities were cited by 39 percent, changing project objectives by 37, and inaccurate requirements gathering by 35. The same survey found that 52 percent of completed projects experienced scope creep or uncontrolled changes to their scope, up from 43 percent five years earlier.

None of these studies is about a twelve-person company commissioning its first business application. What they agree on is that estimation is a real cause, and rarely the only one.

The estimate is guilty more often than vendors admit

It would be convenient to conclude that estimates are fine and clients keep changing their minds. The research does not allow it.

A 2003 review of software estimation surveys by Kjetil Moløkken and Magne Jørgensen, at the Simula research laboratory, found that 60 to 80 percent of projects overrun on effort or schedule. In one of the surveys it covers, over-optimistic estimates were cited for cost overruns almost exactly as often as changes in design or implementation: 51 percent against 50.

So the estimate does fail, and often. It fails alongside other causes, and that the share of each is different on every project. You only find out which one dominated by looking.

Why the post-mortem meeting will not tell you

The meeting where everyone explains the overrun produces the least reliable answer available.

In a 2004 study of 68 projects in one software company, Jørgensen and Moløkken-Østvold found that the reasons people gave for estimation errors depended on their role, and tended to point to causes outside their own control. Experience reports blamed incomplete specifications, while the project data showed indications of the opposite.

Nobody in that meeting is lying. The vendor genuinely remembers the requests that arrived in month four. The client genuinely remembers a quote that looked firm. Both memories are accurate and both are partial, which is why the question needs a method rather than a discussion.

The four-bucket test

Take the final cost and split the gap between it and the original estimate into four buckets. The test needs two documents: the scope as it was written when the price was agreed, and a record of what was actually done. If the second one does not exist, that absence is your first finding.

Estimate error

The only bucket that belongs to the estimator. If a feature described in the scope took three times what was priced, the estimate was wrong, and on a fixed fee it is the vendor’s margin that absorbs it. That is the arrangement we work under, and it is the most honest test of an estimate there is.

Unwritten scope

The bucket most easily misfiled as estimate error, because the overrun shows up on the same invoice. The data migration that was assumed to be an export and an import. The integration with the accounting system that “had an API”. The three months of corrections after go-live that nobody priced. None of it was in the scope, all of it was necessary, and the quote was never wrong about the work it described. These are the same three omissions that make quotes impossible to compare in the first place.

Change

Not a failure in itself. Some changes are the project discovering something true about the business. The failure is a change absorbed without a price or a decision, because it turns into overrun without anyone choosing it.

Waiting

The bucket nobody measures. McKinsey describes a bank whose finance department joined the project only a few months before go-live, which cost a delay of more than three months and more than $8 million. On a smaller project, the same mechanism looks like a key user who is never available, test data that arrives six weeks late, or a decision that waits for a monthly committee.

Why the unwritten part is the one to fix

Of the four buckets, unwritten scope is the one you can shrink before the next project starts, and the cheapest to shrink.

Steve McConnell’s description of the cone of uncertainty explains why. At the initial concept stage, a software estimate can be off by a factor of four in either direction. The cone narrows only as decisions are made that remove variability, including decisions about what you are not going to build. If those decisions are never taken, McConnell writes, the cone does not narrow at all, and it becomes a cloud.

In practice, writing the scope is making those decisions early, on paper, when they cost an afternoon instead of a sprint. A short, paid scoping that produces the written scope, the estimate and the acceptance criteria is how we start every fixed-fee project, and the technical due diligence page describes it. The deliverable is yours even if you take it to another vendor.

What changes on the next project

Four habits, each aimed at one bucket.

Ask how similar projects actually turned out. Flyvbjerg calls this reference class forecasting: instead of building the estimate up from the tasks, start from what a class of comparable projects really cost. Dan Lovallo and Daniel Kahneman called the opposite habit the planning fallacy. A vendor who can show you how its last five comparable projects ended is giving you better information than a detailed task list.

Get the assumptions written into the quote. Every estimate rests on assumptions about data, integrations and availability. Written down, an assumption that turns out false is a change with a clear cause. Left implicit, it becomes an argument.

Keep a change log with a price on each line. One line per change: what, why, how much, who decided. It takes minutes per week and it is the document that makes the four-bucket test possible afterwards.

Put dates on your own decisions. If a key user signs off screens within five working days, write it in the plan. Waiting is the only bucket that is entirely on the client side, and the only one that a schedule can actually prevent.

None of this makes an estimate exact. It makes the overrun explainable, which is what lets the next one be smaller. Whether a fixed fee or time and materials suits your project depends on how much of this can be written down, and our custom software development page sets out what we commit to once it is.