OECD STESWP 2008
Using results from revision
analysis to improve
compilation/estimation methods
An application to the Italian IIP
Anna Ciammola – ISTAT
Meeting of the OECD Short-term Economic
Statistics Working Party (STESWP)
Paris, 23-24 June 2008
OECD STESWP 2008
Outline
 Introduction
 A case study: the Italian IIP

Description of the approach
 Presentation
Paris, 23-24 June 2008
of the results
2/19
OECD STESWP 2008
Introduction
For users
Objective  Availability of all the relevant information for
using appropriately the estimates of ST
indicators at different stages of the revision
process
provision of
information about
past revisions
real-time databases
gathering all the vintages
Paris, 23-24 June 2008
schedule
future revisions
(statistical and definitional)
analysis of size,
bias and efficiency
of revisions
3/19
OECD STESWP 2008
Introduction
For producers
Underlying issues
 Bias in the revision process
 Inefficiency in compilation of preliminary estimates
Targets
 Reduction of (the size of) “avoidable” revisions
 Detection of the source for bias / inefficiency
Paris, 23-24 June 2008
4/19
OECD STESWP 2008
A case study
Italian Index of Industrial Production (IIP)
1. Source and timing of revisions
2. Revision analysis
3. Top-down approach
4. Results
Paris, 23-24 June 2008
5/19
1. Source and time of revisions
OECD STESWP 2008
Y(t-3)
M
A
M
J
J
A
S
O
N
D
J
F
Y(t-2)
Y(t-1)
J
Current Year Y(t) – Reference month
F M A M J J A S O N
D
LR CE
LR
CE
PC
LR CE
LR CE
LR CE
LR CE
LR CE
LR
CE
LR CE
LR CE
LR CE
LR CE
LR CE
First estimate
LR
Late respondents
Paris, 23-24 June 2008
Second estimate
CE
Six-month revision
Correction of errors
PC
Annual revision
Productivity coefficients
6/19
OECD STESWP 2008
2. Revision analysis
IIP - Revisions on raw year-on-year growth rates
h=1
h=12
60
48
MAR
0.142
0.246
RMAR
0.053
0.087
MR
0.075
0.083
SD of MR(HAC)
0.021
0.056
T-value
3.564
1.489
Significance of MR
Yes *
No *
Period: Jan-03 / Dec-07
# of revisions
Legend
Paris, 23-24 June 2008
h=1 – after one month
h=12 – after 12 months
MAR – Mean Absolute Revision
RMAR – Relative MAR
MR – Mean Revision
SD – Standard Deviation
* a = 5%
7/19
2. Revision analysis
OECD STESWP 2008
IIP - Revisions after one month on raw year-on-year growth rates
0.5
0.4
0.3
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
-0.5
Jun
Paris, 23-24 June 2008
Dec-03
Jun
Dec-04
Jun
Dec-05
Jun
Dec-06
Jun
Dec-07
8/19
OECD STESWP 2008
3. Top-down approach
Tools
 Revision measures
► Mean Revision
► Mean Absolute Revision
► Mean Squared Revisions (together with its
decomposition)
►…
 Weighted response rates
 Average contribution of components to the MR of IIP index
Paris, 23-24 June 2008
9/19
3. Top-down approach
OECD STESWP 2008
Diagram describing the top-down approach
IIP
Migs
Divisions
DINT,1
…
CND
CDU
DINT,j
…
CAP
GINT,1
Paris, 23-24 June 2008
CINT,1
…
CINT,m
…
ENE
DINT,7
Groups
Classes
INT
…
...
…
GINT,k
…
GINT,20
...
CINT,n
...
…
10/19
OECD STESWP 2008
3. Top-down approach
Computation of the contribution to the MR
 Revision of July 2004 and January months also
affected by the revision of the productivity coefficients
 Simulation exercise aimed at:
1. highlighting the effect of the imputation of late
respondents
2. fulfilling the condition necessary to compute the
average contribution of each components
Paris, 23-24 June 2008
11/19
4. Results
OECD STESWP 2008
MIGS - Revisions after one month on raw Y-o-Y growth rates
Paris, 23-24 June 2008
Period: Jan-03 / Dec-07
CND
CDU
CAP
INT
ENE
Weights %
22.9
6.1
23.8
35.5
11.7
MAR
0.272 0.415 0.378 0.223 0.149
RMAR
0.084 0.081 0.088 0.073 0.040
MR
0.092 0.072 0.042 0.143 -.003
Contribution to MR °
0.019 0.006 0.010 0.047 -.003
SD of MR(HAC)
0.047 0.103 0.071 0.030 0.040
T-value
1.962 0.694 0.589 4.724 -.079
Significance of MR
No *
No *
No *
Yes *
No *
Legend
CND – Consumer non durables
CDU – Consumer durables
CAP – Capital goods
INT – Intermediate Goods
ENE – Energy
° Period Jan-04 / Dec-07
* a = 5%
12/19
4. Results
OECD STESWP 2008
Revisions after one month on raw Y-o-Y growth rates
Capital goods
Capital goods by month
1.5
1.5
1
1
0.5
0.5
0
0
-0.5
-0.5
-1
-1
-1.5
Dec-03
Dec-04
Dec-05
Intermediate goods
Dec-06
Dec-07
-1.5
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
-0.2
-0.2
-0.4
Dec-03
Dec-04
Paris, 23-24 June 2008
Dec-05
Dec-06
Dec-07
-0.4
Jan Feb
Mar
Apr May Jun Jul Aug Sep
Intermediate goods by month
Oct
Nov Dec
Jan Feb
Mar
Apr
Oct
Nov Dec
May
Jun
Jul
Aug
Sep
13/19
OECD STESWP 2008
4. Results
Average weighted response rates
Year Estimate IIP
CND
CDU
CAP
INT
ENE
First
91.5
93.9
94.3
90.3
88.3
97.4
Second 95.0
95.7
96.1
93.4
93.8
99.6
90.2
90.6
93.5
88.1
87.9
98.7
Second 93.3
93.1
95.4
91.3
92.4
100.0
88.7
89.0
90.4
87.4
86.4
97.3
Second 91.7
91.4
92.5
90.1
90.1
99.9
83.7
84.7
82.4
80.8
80.6
97.6
Second 87.6
88.4
86.0
85.6
84.9
99.0
2004
2005
2006
2007
Paris, 23-24 June 2008
First
First
First
14/19
OECD STESWP 2008
4. Results
Revisions after one month on raw Y-o-Y growth rates
S
NS
Weights %
32.3
67.7
MAR
0.362
0.263
RMAR
0.100
0.082
MR
0.263
0.071
Contribution to MR of INT
0.088
0.047
SD of MR(HAC)
0.066
0.050
T-value
3.985
1.407
Significance of MR
Yes *
No *
Period: Jan-04 / Dec-07
Legend
S – Selected subset of INT (19 NACE classes)
NS – Complement of S in INT (S U NS = INT)
Paris, 23-24 June 2008
* a = 5%
15/19
OECD STESWP 2008
4. Results
Revisions after one month on raw Y-o-Y growth rates
S
SC
11.5
88.5
MAR
0.362
0.159
RMAR
0.100
0.055
MR
0.263
0.056
Contribution to MR of IIP
0.030
0.049
SD of MR(HAC)
0.066
0.032
T-value
3.985
1.766
Significance of MR
Yes *
No *
Period: Jan-04 / Dec-07
Weights %
Legend
S – Selected subset of INT (19 NACE classes)
SC – Complement of S in IIP (S U SC = IIP)
Paris, 23-24 June 2008
* a = 5%
16/19
OECD STESWP 2008
4. Results
Some evidences
 Sectors in the subset S different in terms of either
business concentration or production process (on order
or not)
 Reasons for revisions traced back to:
► partial
information previously provided by
respondents (especially small firms) and revised the
month after
► estimation
of the production levels of non
respondents at the first release
Paris, 23-24 June 2008
17/19
OECD STESWP 2008
4. Results
Possible countermeasures
 Intensive follow up of specific groups of units (especially
for large firms that work on orders)
 Different methods for the imputation of non responses
► some
methodological proposals already implemented
in the production process of IIP
 taking into account firm size
 several estimators
Paris, 23-24 June 2008
18/19
OECD STESWP 2008
Acknowledgements
 Teresa Gambuti – ISTAT IIP survey
 Anna Rita Mancini – ISTAT IIP survey
Thank you!
Paris, 23-24 June 2008
19/19