Comparing Forecast Accuracy of Di¤erent Models for
Prices of Metal Commodities
João Victor Issler (FGV) and Claudia F. Rodrigues (VALE)
August, 2012
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
1 / 19
Stylized Facts on Forecasts and Forecast Combinations
Interest in forecasting yt , stationary and ergodic, using information up
to h periods prior to t – where h is treated as …xed. Risk function is
MSE. Then, optimal forecast (Min. MSE) is:
Et
h
step
ahead forecast error = yt
(yt ) ,
Et h (yt ) .
h
Hendry and Clements (2002): Let fih,t be the i-th h-step-ahead forecast
of yt , i = 1, 2, . . . , N. Then, N1
N
∑ fih,t
performs very well compared to
i =1
individual forecasts fih,t .
This suggests that fih,t cannot approximate Et h (yt ) very well, since
Et h (yt ) is optimal.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
2 / 19
Stylized Facts on Forecasts and Forecast Combinations
Forecast combination works from a risk diversi…cation point-of-view
(Bates and Granger, 1969, and Timmermann, 2006): if the number of
forecasts in the combination is large (N ! ∞), the idiosyncratic
component of forecast errors is wiped out due to the law of large
numbers.
However, there is the Forecast Combination Puzzle: consider
N
∑ ωi fi h,t , where jωi j < ∞ and
i =1
ωi =
N
∑ ωi = 1.
In practice, equal weights
i =1
1/N outperform “optimal weights” designed to outperform it in
a MSE sense; Stock and Watson (2006).
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
3 / 19
Issler and Lima (JoE, 2009)
Work within a panel-data framework, where N, T ! ∞, with
sequential asymptotics: …rst T ! ∞ with N …xed. Then, N ! ∞,
written as (T , N ! ∞)seq .
Propose the use of equal weights combination (1/N ) coupled with an
N
average bias correction term (BCAF):
1
N
∑ fi h,t
i =1
b
B.
Propose a new test for the need to do bias correction: H0 : B = 0.
Show that there is no Forecast Combination Puzzle in large samples:
for N, T large “optimal weights” have the same limiting MSE of the
BCAF.
The bad performance of estimated “optimal weights” is then linked to
the curse of dimensionality.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
4 / 19
Issler and Lima (JoE, 2009)
They decompose fi h,t , as follows – where h is treated as …xed:
fi h,t = Et
h
(yt ) + ki + εi ,t ,
We can always write:
fi h,t
= Et h (yt ) + ζ t , with Et
Then,
= yt ζ t + ki + εi ,t , or,
fi h,t
= yt + ki + η t + εi ,t , where, η t =
fi h,t
= yt + µi ,t,
yt
h
(ζ t ) = 0.
ζ t , or
i = 1, 2, . . . , N, with µi ,t = ki + η t + εi ,t
The error µi ,t has a two-way decomposition (Wallace and Hussain
(1969), Amemiya (1971), Fuller and Battese (1974)) with a long
tradition in the econometrics literature. It depends on h, but this is
omitted for notational convenience.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
5 / 19
Issler and Lima (JoE, 2009) Assumptions
Time framework:
E
R
P
1________T1 ________T2 _______T
= T1 = κ 1 T ,
R = T2 T1 = κ 2 T ,
P = T T2 = κ 3 T .
E
Assumption 1 ki , εi ,t and η t are independent of each other for all i and t.
Assumption 2 ki is an identically distributed random variable in the
cross-sectional dimension, but not necessarily independent,
ki
J.V. Issler and C.F. Rodrigues
i.d.(B, σ2k ).
()
Forecast Models for Metal Prices
(1)
August, 2012
6 / 19
Issler and Lima (JoE, 2009) Assumptions
Assumption 3 The aggregate shock η t is a stationary and ergodic MA
process of order at most h 1, with zero mean and variance
σ2η < ∞.
Assumption 4: Let εt = (ε1,t , ε2,t , ... εN ,t )0 be a N 1 vector stacking the
errors εi ,t associated with all possible forecasts, where
E (εi ,t ) = 0 for all i and t. Then, the vector process fεt g is
assumed to be covariance-stationary and ergodic for the …rst
and second moments, uniformly on N. Further, de…ning as
ξ i ,t = εi ,t Et 1 (εi ,t ), the innovation of εi ,t , we assume
that
1 N N
(2)
lim 2 ∑ ∑ E ξ i ,t ξ j ,t = 0.
N !∞ N
i =1 j =1
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
7 / 19
Issler and Lima (JoE, 2009) Assumption for Nested Models
Continuous of N models (i = 1, .., N) split into M classes (or blocks),
each with m nested models:
= mM. Let,
M = N 1 d , and m = N d , where 0
N
1
d = 0, all models are non-nested;
2
d = 1, all models are nested and;
3
0 < d < 1 gives rise to N 1
Nd .
d
d
1.
blocks of nested models, all with size
Notice that d is a choice parameter from the point-of-view of the
researcher combining forecasts.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
8 / 19
Issler and Lima (JoE, 2009) Assumption for Nested Models
Partition matrix E ξ i ,t ξ j ,t into blocks: M main-diagonal blocks, each
with m2 = N 2d elements. M 2 M o¤-diagonal blocks. Index the class by
r = 1, .., M, and models within class by s = 1, .., m.
Within each block r , we assume that:
0
1
lim 2d
N !∞ N
Nd Nd
∑∑
E ξ r ,k ,t ξ r ,s ,t
1
m !∞ m2
lim
m
m
∑∑
E ξ r ,k ,t ξ r ,s ,t
=
k =1 s =1
< ∞,
k =1 s =1
being zero when the smallest nested model is correctly speci…ed.
Across any two blocks r and l, r 6= l, we assume that:
1
lim 2
m !∞ m
m
m
∑∑
k =1 s =1
E ξ r ,k ,t ξ l ,s ,t
1
= lim 2d
N !∞ N
Nd Nd
∑∑
E ξ r ,k ,t ξ l ,s ,t
= 0.
k =1 s =1
Here, the assumption in the previous page will still hold in the
presence of nested models when 0 < d < 1.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
9 / 19
Issler and Lima (JoE, 2009) Main Results
If Assumptions 1-4 hold, the following are consistent estimators of ki , B,
η t , and εi ,t , respectively:
kbi
b
B
b
ηt
bεi ,t
=
=
=
1 T2
fh
∑
R t =T 1 +1 i ,t
1 N b
∑ ki ,
N i =1
1
N
= fi h,t
J.V. Issler and C.F. Rodrigues
N
∑ fi h,t
i =1
yt
b
B
kbi
1 T2
yt ,
∑
R t =T 1 +1
plim
(T ,N !∞)seq
yt ,
b
ηt ,
b
B
plim kbi
T !∞
ki
= 0,
B = 0,
plim
(T ,N !∞)seq
plim
(T ,N !∞)seq
()
Forecast Models for Metal Prices
ηt
(b
(bεi ,t
η t ) = 0,
εi ,t ) = 0.
August, 2012
10 / 19
Issler and Lima (JoE, 2009) Main Results
If Assumptions 1-4 hold, the feasible bias-corrected average forecast
N
1
N
∑ fi h,t
i =1
b obeys:
B
plim
(T ,N !∞)seq
1
N
N
∑
fi h,t
i =1
b
B
!
= yt + η t = Et
h
(yt ) ,
and has a mean-squared error as follows:
E
"
plim
(T ,N !∞)seq
1
N
N
∑
i =1
fi h,t
b
B
!
yt
#2
= σ2η .
Therefore it is an optimal forecasting device.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
11 / 19
Issler and Lima (JoE, 2009) Main Results
Consider the sequence of deterministic weights fω i gN
i =1 , such that
jω i j 6= 0, ω i = O N
1
N
N
uniformly, with ∑ ω i = 1 and lim ∑ ω i = 1.
N ! ∞ i =1
i =1
Then, under Assumptions 1-4:
E
"
N
plim
(T ,N !∞)seq
∑
i =1
ω i fi h,t
N
∑ ωi kbi
i =1
!
yt
#2
= σ2η .
Therefore it is an optimal forecasting device as well.
For optimal population weights there is no Forecast Combination
Puzzle.
Thus, the Forecast Combination Puzzle must be a consequence of the
inability to estimate consistently the optimal population weights. This
happens when R is small relative to N.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
12 / 19
Issler and Lima (JoE, 2009) Main Results
The optimality results above are based on
fi h,t = Et
h
(yt ) + ki + εi ,t ,
where the bias ki is additive. If the bias is multiplicative as well as
additive, i.e.,
fi h,t = βi Et h (yt ) + ki + εi ,t ,
where βi 6= 1 and βi
β, σ2β , the BCAF is no longer optimal if β 6= 1.
Optimality can be restored if the BCAF is slightly modi…ed to be
!
kbi
fi ,t
1 N
,
∑
b
N i =1 b
β
β
where kbi and b
β are consistent estimators of ki and β, respectively.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
13 / 19
Issler and Lima (JoE, 2009) Main Results
Under the null hypothesis H0 : B = 0, the test statistic:
b
B
bt = p
b
V
d
!
(T ,N !∞)seq
N (0, 1) ,
b is a consistent estimator of the asymptotic variance of
where V
N
B=
1
N
∑ ki .
i =1
b is estimated using a cross-section analog of the Newey-West
V
estimator due to Conley (1999), where a natural order in the
cross-sectional dimension requires matching spatial dependence to a
metric of economic distance.
N
If B = 0, the average forecast
1
N
∑ fi h,t
is an optimal forecasting
i =1
device.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
14 / 19
Issler and Lima (JoE, 2009) Monte-Carlo
The DGP is a stationary AR (1) process:
yt = α0 + α1 yt
1
+ ξ t , with ξ t
i.i.d.N (0, 1), α0 = 0, and α1 = 0.5,
One-step-ahead forecasts are generated as:
fi ,t = b
α0 + b
α1 yt
1
+ ki + εi ,t ,
and the bias is generated as:
ki = βki
1
+ ui ,
where ui i.i.d.U (a, b ), with β = 0.5.
The error εi ,t is drawn from a multivariate Normal with zero mean and
variance-covariance matrix Σ = (σij ), which has zero covariance imposed
as long as ji j j > 2.
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
August, 2012
15 / 19
Issler and Lima (JoE, 2009) Monte-Carlo Results
R = 50,
Bias
BCAF Average
mean
0.000
0.391
mean
0.000
0.440
mean
0.000
0.465
J.V. Issler and C.F. Rodrigues
B = 0.5,
Weighted
N
-0.001
N
-0.002
N
0.000
σ2ξ = σ2η = 1
MSE
BCAF Average
= 10
1.561 1.697
= 20
1.286 1.466
= 40
1.147 1.351
()
Forecast Models for Metal Prices
Weighted
1.916
2.128
6.094
August, 2012
16 / 19
Issler and Lima (JoE, 2009) Monte-Carlo Results
R = 50,
Bias
BCAF Average
mean
0.000
0.000
mean
0.000
0.000
mean
-0.002
0.000
J.V. Issler and C.F. Rodrigues
B = 0,
Weighted
N
-0.001
N
-0.002
N
0.000
σ2ξ = σ2η = 1
MSE
BCAF Average
= 10
1.561 1.547
= 20
1.286 1.272
= 40
1.147 1.133
()
Forecast Models for Metal Prices
Weighted
1.916
2.128
6.094
August, 2012
17 / 19
Issler and Lima (JoE, 2009) Monte-Carlo Results
N = 10, R = 500, 1, 000,
B
Bias
BCAF Average Weighted
N = 10,
mean 0.000 0.391
0.000
N = 10,
mean 0.000 0.392
0.000
J.V. Issler and C.F. Rodrigues
= 0.5,
σ2ξ = σ2η = 1
MSE
Average Weighted
BCAF
R = 500
1.532 1.697
R = 1, 000
1.535 1.695
()
Forecast Models for Metal Prices
1.559
1.541
August, 2012
18 / 19
Issler and Lima (JoE, 2009) Monte-Carlo Results
N = 40, R = 2, 000, 4, 000,
B = 0.5,
σ2ξ =
Bias
MSE
BCAF Average Weighted BCAF Average
N = 40, R = 2, 000
mean 0.000 0.466
0.000
1.127 1.355
N = 40, R = 4, 000
mean 0.000 0.465
0.000
1.127 1.355
J.V. Issler and C.F. Rodrigues
()
Forecast Models for Metal Prices
σ2η = 1
Weighted
1.149
1.137
August, 2012
19 / 19