Study: Cost-Benefit Analyses for Projects Are 'Worse than Worthless'
A large new global study of project performance over 86 years confirms that ingrained optimism bias results in forecast-based cost-benefit analyses "so misleading as to be worse than worthless," say UK researchers. While construction technology and practice have developed over the decades, failures of cost-benefit analysis "seem universal across space and time," they add.
Developed long before the emergence of behavioral science, "cost-benefit theory and practice ... have yet to adapt to its findings about the nature and causes of bias," say study authors from the University of Oxford’s Saïd Business School. As a result, they find that benefit-cost ratios for various project types continue to be overestimated by between 50% and 200%.
To improve project assessment, the researchers call for de-biasing by systematic reference to actual performance of previous projects, incentivizing forecasters for accuracy with rewards and punishments, as well as completing independent audits. They also advocate integrating analysis results into real-life democratic decision-making processes "which consider other concerns than the results of cost-benefit analyses."
The study, "The Cost-Benefit Fallacy: Why Cost-Benefit Analysis Is Broken and How to Fix It" is based on validated data on costs and benefits of 2,062 public projects built between 1927 and 2013 in 104 developed and developing nations on six continents, mostly in the US and Europe. Projects included bridges, buildings, bus rapid transit, dams, power plants, rail, roads and tunnels.
Researchers say "strong and consistent" biases led to cost overruns not being compensated by benefit overruns, "but quite the opposite, on average." In addition, projects with large average cost overruns "tend to have large average benefit shortfalls," the study says.
"A rationalist would expect humans to improve over time through learning. [But] for a behavioral scientist, not so much. People had the same brains in 1927 as in 1960 and 2013, with the same biases, so they produce the same inaccuracies, as the data show," says research leader Bent Flyvbjerg, chair of major program management in the business school.
Cost forecasts were found to be highly inaccurate and biased with average overruns ranging from 24% for highways to 85% for dams. Benefit forecasts for bridges, buildings, power plants and roads were "fairly accurate,". but average benefit shortfalls for bus rapid transit and rail projects were 58% and 34%, respectively.
Looking for quick answers on construction and engineering topics?
Try Ask ENR, our new smart AI search tool.
Ask ENR →
However, in about 20% of cases, underforecasting of benefits more than compensated for actual cost overruns. Denmark's Great Belt toll-road bridge, for example, had a 90% benefit overrun, double that of the cost forecast. The researchers caution that the number of benefit samples for most investment types is small.
Overoptimism bias lies behind what the researchers call "the cost-benefit fallacy." They cite behavioral science evidence that planners and managers repeatedly underestimate the importance of schedules, geology, prices, scope changes and complexity and overestimate mitigation measures.
"The problem with cost-benefit forecasts is not error but bias," they say. "As long as we allow ourselves to be blinded by optimism bias, overconfidence bias, probability neglect and other behavioral biases, we will keep reproducing the cost-benefit fallacy."
