Demand and Benefit Forecasts by the Numbers
Chapter 1 of 10 · 13 min read
The Hiding Hand and the evidence on optimistic forecasts
A canning plant forecast at 25.32m cans and an NPV of A$6.08m, Hirschman's Hiding Hand, what the published tests found, and a board paper that cited one good year.
Every business case for new capacity rests on a forecast of how much the capacity will be used and what that use will earn. The cost of the plant can be quoted, the lease can be read and the crew can be priced, but the volume is a judgement about customers who have not yet bought anything. That judgement is usually made by the person who most wants the project to go ahead, and it is usually too high. There is also a respectable argument that this does not matter, because the people who underestimate the difficulties of a venture also underestimate their own ability to overcome them.
This chapter sets out the running case: a contract-canning business, its proposal for a fixed canning plant, and the forecast in its business case. It states the argument that optimism pays for itself, summarises what the published tests of that argument found and what their critics said in reply, and asks who carries the loss when a small business’s forecast falls short. It ends with a board paper that cited the business’s one good year as proof that shortfalls get made up, and what happened when all six years were tabulated.
The business case on the table
This is an illustration. A contract-canning business near Geelong, Victoria, has run mobile canning rigs since 2018, visiting craft breweries to fill and seam their beer on site. Rig 1 started in 2018 and rig 2 was added in mid-2021. In February 2024 the managing director put a business case to the board for a fixed canning plant in a leased industrial unit: one line rated at 200 cans a minute, with a capital cost of A$5.4m, excluding GST. All money in this book is in Australian dollars excluding GST.
The plant would can for four segments of customer, labelled A to D throughout the book:
- A, craft brewers without their own line: 30 customers, 3.240m cans a year.
- B, ready-to-drink and seltzer brands: 5 customers, 12.000m cans a year.
- C, non-alcoholic drinks such as kombucha, cold brew and functional drinks: 12 customers, 7.200m cans a year.
- D, overflow work from larger brewers with their own lines: 3 customers, 2.880m cans a year.
That totals 25.32m cans a year from the third year of operation, 2027. Each can earns a contribution, the service fee less the gases, power, water, chemicals, cartons and consumables used to fill it, of between A$0.10 and A$0.20 depending on the segment, so the plant would earn A$3,201.6k a year of contribution, an average of A$0.126 a can. Fixed costs, covering the lease, salaried and line crew, maintenance, quality assurance and insurance, are A$1.30m a year, leaving steady-state EBITDA of A$1,901.6k and a simple payback of 2.84 years.
The case allowed for a ramp-up: 70% of the steady volume in the first year, 2025, and 95% in the second. It also counted the sale of mobile rig 2 for A$320,000 in June 2025. Discounted at 9% a year over ten operating years, with the capital spent in 2024 and no residual value, the cash flows were −A$5,400.0k, then A$1,261.1k, A$1,741.5k and eight years of A$1,901.6k, giving a net present value of A$6,081.5k and an internal rate of return of 29.3%.
The numbers look comfortable. The plant covers its fixed costs at 10.281m cans a year at the forecast mix, only 40.6% of the forecast, so on the face of it the forecast could be badly wrong and the plant would still pay its way. The rest of the book tests that impression. For now, note what the comfort depends on: a business that had never filled more than 4.19m cans in a year was forecasting six times that, from a different kind of customer, on a new line.
Hirschman’s argument
In 1967 the economist Albert Hirschman published a study of development projects financed by the World Bank, and in its first chapter set out what Hirschman called the principle of the Hiding Hand. Hirschman’s observation was that many projects ran into difficulties their planners had not foreseen, and that, had the planners foreseen them, many worthwhile projects would never have been started. Yet the same planners also failed to foresee their own capacity to solve those difficulties once they met them. The two errors, Hirschman argued, tend to offset each other: ignorance of the problems is hidden by ignorance of the solutions, and the hidden hand of that double ignorance leads people into ventures that turn out to be worth doing.
The argument has an obvious appeal to anyone who has built a business. The managing director had started with one rig, added a second, and solved problems at every step that no plan had anticipated. When the board asked about the risk in the volume forecast, the answer was the Hiding Hand in plain words: “whatever we have underestimated, we will find a way, as we always have”. It is not a foolish answer. Ingenuity is real, and a business run only on what can be proved in advance would never start anything.
The difficulty is that the argument is a claim about outcomes, and outcomes can be counted. If the Hiding Hand works as stated, a large sample of projects should show creativity making up for underestimated difficulty more often than not, so that the benefits actually delivered come out at least as large as forecast on average. That is a testable prediction.
What the tests found
The most direct test is Flyvbjerg (2016), published in World Development. It set the Hiding Hand against a large sample of projects and reported, in its abstract, that statistical tests reject the principle at p < 0.0001. The average project in the sample was not rescued by ingenuity. Instead it suffered substantial cost overruns compounded by substantial benefit shortfalls, a combination the abstract calls a “double whammy”. In other words, the planners’ two errors did not offset each other; they pointed the same way.
Demand forecasts have also been studied directly. Flyvbjerg, Skamris Holm and Buhl (2005), in the Journal of the American Planning Association, examined forecasts for 210 transport projects in 14 nations worth US$59 billion. Their abstract reports three results that matter here:
- for nine out of ten rail projects, passenger forecasts were overestimated, and the average overestimation was 106 per cent;
- for half of all road projects, actual traffic differed from forecast traffic by more than ±20 per cent;
- forecasts had not become more accurate over the 30-year period studied.
To read the rail figure, it helps to do the arithmetic for a single project. An overestimation of 106% means the forecast was 2.06 times the traffic that turned up, so the actual was about 0.485 of the forecast. That is a reading aid, not the study’s average ratio, but it shows the scale: a rail forecast in that sample typically promised about twice what arrived. For roads, the result is about spread rather than direction. A band of ±20% means actual traffic between 0.8 and 1.2 of the forecast, and half the projects fell outside it.
These are large public transport projects, not a canning plant, and their forecasts were made in a different setting, often with public funding at stake and long horizons. They do not tell a small business what its own error will be. What they establish is that demand forecasts made to justify an investment have, in a large and well-documented sample, run high far more often than low, and that experience over three decades did not cure it. Measuring forecast error and reading base rates in Chapter 5 returns to these base rates and to the question of how to compare a forecast with what happened.
The case against the tests
The test did not settle the matter. Ika (2018), also in World Development, argued that the 2016 test rests on a straw man. Hirschman’s principle, on this reading, was never a claim that cost and benefit forecasts would balance on average. It was a claim about how projects get started and how people solve problems along the way, and a fair test would look at full life-cycle costs and benefits, at unintended effects, good and bad, and at the difficulties met and the problem-solving they prompted, not only at a comparison of forecast and outturn at one point in time.
Flyvbjerg’s reply in the same issue (2018) accepted that the Hiding Hand exists. The reply’s abstract reports it in about one fifth of projects in the dataset, while arguing that it is not the typical case: for most projects, the evidence fits better with optimism and underestimation that are not later made good.
- Ignorance of difficulties leads people into worthwhile projects
- Creativity, once needed, solves problems nobody foresaw
- Fair tests need full life-cycle benefits and unintended effects
- A single comparison of forecast and outturn is too narrow
- Hiding Hand rejected at p < 0.0001 (2016)
- Cost overruns compounded by benefit shortfalls on average
- Hiding Hand found in about one fifth of projects (2018)
- Rail demand overestimated in nine of ten projects, by 106% on average
A board does not need to decide who won the academic argument. It needs a working rule. The evidence supports this one: a Hiding Hand happens, perhaps in one project in five, and when it happens it is welcome, but a business case that relies on it is betting on the minority outcome. Upside surprises and honest accounting, Chapter 9, looks at the plant’s own upside surprise in these terms.
The inside view
Why do forecasts run high so consistently? Kahneman and Lovallo (1993) gave the explanation that most later work builds on. A forecaster who builds a forecast from the plan in hand, its customers, its steps and its likely obstacles, is taking the inside view. The inside view feels thorough, and it is how forecasts have to be built, but it tends to imagine the plan going roughly as intended. It neglects the outside view: the statistics of how similar ventures actually turned out. Kahneman and Lovallo paired this with a second tendency, timidity about risk in individual choices, and argued that the two together produce bold forecasts behind cautious-looking decisions.
The plant’s forecast is an inside-view forecast in this sense. Every one of its 50 customers is a business the managing director could name, and every segment has a plausible story. What it lacks, so far, is any comparison with the business’s own record of turning named prospects into volume. The inside view and the plan on the table in Reference Class Forecasting by the Numbers applies the same idea to a project’s cost and schedule; this book applies it to volume and benefits.
There is an earlier question, too: what the plant is for. The board’s stated aim was to diversify the business beyond beer, and the forecast puts 75.8% of volume in segments B and C. A forecast is easier to test when the outcome it serves is written down first, as From a request for new plant to the outcome behind it in Outcome-First Projects by the Numbers sets out.
Who absorbs a shortfall in a small business
In the published studies, a shortfall in a road or rail project is absorbed by a government budget, spread across taxpayers and decades. A small private business has no such cushion. The plant’s fixed costs of A$1.30m a year are committed by a lease and a crew whether the cans arrive or not, while contribution falls in proportion to volume. At the forecast mix, each million cans not filled costs about A$126.4k of contribution.
That gearing works against the business. A 20% shortfall in volume, 5.064m cans, removes A$640.3k of contribution, which is 33.7% of steady-state EBITDA: EBITDA falls from A$1,901.6k to A$1,261.3k. Every 1% of volume moves EBITDA by about 1.68%. And the people whose ingenuity the Hiding Hand counts on are the same few people who must also run two mobile rigs, sell to new customers and manage the build. In a large organisation a shortfall is a variance; in a business of this size it is the owners’ equity, the bank’s patience and the crew’s jobs.
The error: one good year as proof
This is an illustration. The board paper defended the forecast with the business’s own history. It noted that in 2020 the mobile business had beaten its budget, filling 2.81m cans against 2.50m, a ratio of actual to forecast of 1.124, and presented that year as proof that “shortfalls get made up”.
The independent reviewer, a board adviser asked to test the forecast, requested the budgets and actuals for every year since 2018 and tabulated them. Five of the six years came in below budget. The ratios ran from 0.728 in 2018 to 1.124 in 2020, with a mean of 0.879, a median of 0.857 and an aggregate ratio of 0.877 (17.81m cans delivered against 20.30m budgeted over six years). Without 2020, the mean was 0.830. The 2020 result had a specific cause: an unplanned surge in packaged orders that the business had not forecast and had not created by any action of its own. It was a windfall, not a shortfall made up by ingenuity.
The error was one of selection. Citing a single year, chosen because it supported the argument, turned a record of persistent over-forecasting into evidence for the opposite. It was detected because the reviewer asked for the full series rather than the example offered. The correction was simple and became a standing rule for the board papers that followed: show every year of a record, not a chosen one, and state the cause of any year that departs from the pattern.
The board’s decision
The reviewer’s tabulation was before the board when it decided. The board approved the plant anyway. It was persuaded partly by the managing director’s Hiding Hand argument and partly by the strategic aim of diversifying beyond beer, which it valued for its own sake. That is a legitimate decision: a board may accept a risk it understands. What it did not do was approve the forecast as though it were reliable. It attached two conditions:
- Quarterly benefit tracking against the business case, with leading indicators that would show a shortfall before it reached the accounts.
- A post-completion review at about 21 months of operation, re-appraising the plant on its actual results and recording what the forecast got wrong.
Both conditions are the subject of later chapters. They turned a disagreement about the Hiding Hand into something that could be checked against the plant’s own numbers.
What happened, in brief
The plant opened in January 2025, which this book calls month 1, and the book follows it to September 2026, month 21, and the review held in October 2026. The final cost of the plant was A$6.10m, 13.0% over the A$5.4m budget. At month 21, after adjusting for the season, the plant was running at 16.95m cans a year, 66.9% of the forecast. That is a cost overrun and a benefit shortfall at once, the combination the 2016 study found in the average project, although one segment did very much better than planned, and Chapter 9 examines whether that counts as the Hiding Hand at work. Chapter 5 sets out how to measure the shortfall properly.
Exercise: a top-down correction
Before building anything more elaborate, the reviewer tried the simplest outside-view correction: scale the forecast by the business’s own record. Apply the median annual ratio, 0.857, and the mean excluding 2020, 0.830, to the inside-view volume of 25.32m cans, keeping the inside-view mix, ramp, fixed costs, capital and rig sale, and ignoring any other correction. What steady volume and NPV does each give, and at what steady volume does the NPV reach zero?
Worked answer. At 0.857, the steady volume is 25.32 × 0.857 ≈ 21.7m cans (21.688m from the unrounded ratio), and the NPV at 9% is A$3,279.7k. At 0.830 it is about 21.0m cans (21.003m), and the NPV is A$2,751.8k. Solving for the volume at which the NPV is zero gives 17.436m cans, 68.9% of the forecast. On this evidence the plant survives a typical year’s optimism with room to spare. The caution is that the six ratios measure forecasts of volume from customers the business already had. The plant depends mostly on customers it did not yet have, and Chapter 6 shows why a forecast built on new customers needs a larger correction than the annual budgets suggest.