Cohesion Policy evaluation –
Cost and time estimates of major projects
Warsaw 30 November
Hugh Kelly
Importance of reliable estimates
Estimates are critical:
• Public policy decisions
• Investment decisions
• Accountability
Cost overruns and benefit shortfalls:
• Lead to an inefficient allocation of resources
• Lead to delays and further cost overruns
• Destabilize policy, planning, implementation and project
operation1
1. See Megaproject Policy and Planning: Problems, Causes and Cures, Flyvberg, Department of
Development and Planning, Denmark
Estimates and evaluation
Ex post evaluation
• What estimates were used in the ex ante CBA?
• How did the outcome compare to estimates?
• What were reasons for differences?
• How did they impact on the effectiveness of the project?
But major projects can go spectacularly over
budget
A$102m
$5.0bn
A$7.2m
Sydney Opera House
$1.5bn
£9.5bn
$14.6bn
£4.7bn
Channel Tunnel
Denver's International
Airport
$3.7bn
Boston’s Central Artery “The Big Dig”
http://miller-mccune.com/business_economics/bill-of-goods-the-world-s-biggest-boondoggles-601
And the evidence suggests that the problem is
not restricted to flagship projects
ERDF 2000-2006 Evaluation (51 projects):
• Average cost overrun
• Rail +27%
• Road +9%
• Urban transport +45%
• Water +11%
• Energy +21%
Flyvberg: 44 Urban rail projects:
Average overrun 45%
Traditional explanations
Source: European Commission DG XVI (1998), “Understanding and Monitoring the Cost-Determining
Factors of Infrastructure Projects”, A User’s Guide, Brussels
Numerous studies confirm the
reasons for over runs
Source: ECORYS et al. (2005), “Ex post evaluation of a sample of projects co-financed by the
Cohesion Fund (1993-2002)”, Synthesis Report, Rotterdam
Le D e
s
En gis ign
vi lat c
ro io ha
nm n ng
c
A en han es
W uth tal ge
or o i m s
k rit p
su y ac
sp d e t
en lay
Pe
si s
Si rm
on
P
i
t
La e c ts/C oli s
te ha o tic
n
C co r ac se al
on m te n
s m r i ts
Fu tr uc enc stic
St n ti e s
ak di on me
eh n g p n
ol p er t
de ro io
r bl d
In T nu em
m s
n
C ov ech be
on at n r s
i o
D tr ac on log
es t d y
C ig or eg
on n s re
Pr tr a co pe e
oj ct m cif
ec c pl ic
t m o m ex
an ple ity
ag xi
em t y
In e
Pu Sit fla nt
e
bl a tio
ic c n
Fo r e ces
la s
Po rce tio
S
B u o m ns
us pp r p aj
in li la ur
es er n e
s di nin
in sp g
ad u
eq tes
ua
c
Ex E O y
t
ch co he
an n o r
ge mi
ra c
te
s
Score
Recent review of ERDF projects
1.6
Cost overrun score
Time delay score
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
Traditional approach to dealing with the risk
• Risk management
– PPP
• Contingency funding
• More and more detailed budgets
Traditional solution – better planning
and contingency
Source: Flyvbjerg,
Bent in
association with
COWI (2004),
“Procedures for
dealing with
Optimism Bias in
Transport
Planning:
Guidance
Document”, The
British
Department for
Transport,
London, June.
But if we understand the reasons for overrun
why do we keep getting it wrong ?
£9.3bn
£2.3bn
Original
bid cost
Current
estimate
Alternative approach: Flyvberg: Delusion and
Deception in large Capital Projects1
• Delusions
 Planning fallacy
 Human optimism
 Anchoring and adjustment
 Tendency to ‘stick’ to initial estimate
 Deception
 Principal-Agent Problem
 Sources of strategic deception
Delusions and Deception in Large Infrastructure Projects, Bent Flyvberg, Massimo
Garbuio, Dan Lovallo California Management Review Vol 51 No 2 Winter 2009
Principal agent issues
Sources of Strategic Deception
• Actors self interests
• Asymmetric information
• Different risk preferences
– Different time horizons
– Diffuse accountability
Different types of risks for different
projects
Solutions – (a) risk sharing
• Pass risk of over runs to:
– Member states
– Local Governments
– Contractors
Solutions – (b) reference class
projects
• Select reference class
• Assess distribution of outcomes
• Make an intuitive prediction of project’s position in the
distribution
• Assess the reliability of prediction
• Correct intuitive estimates