Early Warning or Just Wise
After the Event?
The Problem of Using Cyclically Adjusted
Budget Deficits for Fiscal Surveillance in
Real Time
Estonian Economic Association Annual Conference
Pärnu, 12 January 2007
Andrew Hughes Hallett, George Mason University
John Lewis, De Nederlandsche Bank
Rasmus Kattai, Bank of Estonia
Disclaimer: The views expressed are those of the authors and not necessarily those of the institutions to which they are affiliated
Introduction
Revenues
Expenditures
Economic c ycle
Output gap = 0
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
The Dutch Experience
Cyclically Adjusted Budget Deficit: Economic Forecasts (European Commission)
1996
1997
1998
1999
2000
2001
2002
2003
2004
Spring 2004
-0.9
-0.9
-1.5
-1
0.9
-2
-2.9
-2.4
-1.9
Autumn 2002
-1.0
-0.9
-1.5
-0.9
-0.6
-1.2
-0.8
-0.5
-0.4
Autumn 2001
-1
-0.8
-1.3
-0.6
0.1
0.8
0.8
1.4
June 2004:
Excessive Deficit
Procedure initiated
against NL for 2003
budget deficit figure
Was the fiscal slippage
apparent earlier on from
CAB data?
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Overview of the Paper
• A simple model of fiscal surveillance
• Describing the real time dataset
• How Robust are CAB estimates over time?
• Effectiveness of CABs as an early warning
indicator
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
The Basic Problem
B  B  (Yt  Y * )(    )
Budget balance CAB Output gap
• We don´t know what the output gap is
• We don’t know the final budget balance is until several years after
the event
• Estimating the CAB s years after the event we make the error:
Sensitivity
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Revision to Revision to
Output gap deficit ratio
The Real Time Dataset
• Output gap, actual balance, CAB from successive issues
of OECD Economic Outlook: December 1995 (58) December 2006 (78)
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Data Revisions: Output Gap
• Difference between
data at time t+s and
ex post data
• n=7
• 1995-2001
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Data Revisions: Deficits
• Difference between
data at time t+s and
ex post data
• n=7
• 1995-2001
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Data Revisions: CAB
Summary of OECD
cyclically adjustment
methodology
• Estimate elasticities of various
budget components with
respect to output gap
• Multiply these by output gaps
– Some lags included for tax
revenues
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
How well does the OECD Real Time
Estimate Perform?
• Simple benchmark: construct our own simple output
gap and CAD measure
• HP filter output
• Plug in our estimate of output gap to OECDs elasticity
and deficit figures
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Comparison: Output Gap
• RMSE:
– Thick line: OECD’s real time
output gap series vs OECD’s
final output gap series
– Thin Line: Our real time
output gap series vs OECD’s
final output gap series
• OECD figures perform significantly better in real time
• Over longer time horizons (s = 2,...,4) the difference is almost
negligible
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Comparison: CAB
• RMSE:
– Thick line: OECD’s real time
CAB vs OECD’s final CAB
– Thin Line: Our real time CAB
series vs our final CAB
– Dotted line: Our real time CAB
series vs OECD’s final CAB
• OECD figures perform marginally better
• Even if you take OECD final numbers as “correct”. OECDs own real
time data is only marginally better at estimating that completely
different (and far simpler) methodology
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Early Warning?
• Measure of accuracy of real time figures
• Fiscal Slippage: ex post CAB worsens by 1.5pp of GDP
• If real time CAB slips by certain amount- trigger value- then an
alarm is sounded
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Early Warning?
• Two definitions of fiscal slippage
– Fall of 1.5pp in CAB over 1yr
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
1 year slippage definition
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Early Warning? (2)
• Two definitions of fiscal slippage
– Fall of 2.0pp in CAB over 2yrs (t-2 to t, time t data)
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
2 year slippage definition
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Deficit Revisions or Cyclical
Adjustment?
• Apply RT cyclical adjustment to ex post deficit data
• Eliminates most of the missed alarm problem
• …but doesn’t help the false alarm problem
Real time Data
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Deficit Revisions or Cyclical
Adjustment? (2)
• 2 pp definition: similar story
• Eliminates most of the missed alarm problem
• …but doesn’t help the false alarm problem
– False alarm problem even gets worse!
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Real time Data
Data Revisions and Slippages
• CAB
calculations are
at their most
unreliable
when they are
the most
needed!!!
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007
Conclusions
• Using CABs for fiscal surveillance requires good
quality figures in real time
• Real time CAB figures are unreliable
– Output gap revisions
– Deficit revisions also play a big role
– Simple approach can almost match OECDs
methodology in real time
– Can’t reliably pick out fiscal slippages
– Real time CAB figures are most unreliable when they
are most needed
John Lewis
Economics and Research Division
De Nederlandsche Bank
Early Warning or Wise After the Event?
Estonian Economic Association
12 January 2007