Asymptotic Expansions of MM and ML
Estimators for the First Order Moving
Average with Mean Models
Antonis Demos and Dimitra Kyriakopoulou1
Athens University of Economics and Business
and
University of Piraeus
November 2006
Current Version: January 2008
are grateful to Stelios Arvanitis, Enrique Sentana, the participants at the 18th
EC2 Conference, Faro Portugal and the seminar participants at the University of Piraeus. Of course the usual caveat applies. Address for correspondence: Dimitra Kyriakopoulou, University of Piraeus, Department of Banking and Financial Management,
80 Karaoli & Dimitriou St., Piraeus 185 34, Greece. Tel:+30-210-8203125, e-mail: [email protected].
1 We
Abstract
The second order asymptotic expansions, of the Edgeworth type, of the MM and
ML estimators for the MA(1) with mean model are given. We also derive the
second order expansions of the sample autocorrelation and the autocorrelation
based on the ML estimator of the MA parameter. By employing Nagar type
expansions, the properties of the estimators in terms of bias and mean squared
error are discussed and compared with the zero-mean case. The results presented
here are important for deciding on the estimation method we choose, as well as
for bias reduction and increasing the efficiency of the estimators.
Keywords: Edgeworth expansion, moving average process, method of moments, maximum likelihood, autocorrelation, asymptotic properties.
JEL: C10, C22
1
Introduction
Techniques for approximating probability distributions like the Edgeworth expansion have a long history in econometrics.1 In time series models, starting with
Phillips (1977a), there is a fair amount of papers dealing with Edgeworth expansions in autoregressive or mixed models (e.g. Tanaka 1983 and 1984, Bao
and Ullah 2007, Kakizawa 1999a etc.). However, there are relatively few papers
concerning the limiting distribution of estimators of the Moving Average (MA)
parameters and their properties. Durbin (1959) proposes an estimator for the
parameter of the MA(1) model that can reach the asymptotic efficiency of the
Maximum Likelihood Estimator (MLE), Davis and Resnick (1986) discuss the
limit distribution of the sample covariance and correlation for the MA(∞) model
under the assumption of infinite variance of the errors. Bao and Ullah (2007)
present the second order bias and Mean Squared Error of the MLE of the MA(1)
under non-normality but without mean. Tanaka (1984) develops a technique for
1
the up to T − 2 order, T being the sample size, Edgeworth expansion of the MLEs
for autoregressive moving-average (ARMA) models and presented the first order
expansion of the MLE of the MA(1) model with and without mean.2
Here we develop and compare the second order asymptotic expansions of two
estimators of the following MA(1) model with mean, MA(1|µ) say,
yt = µ + ut + θut−1 ,
t = ..., −1, 0, 1, ...,
|θ| < 1,
iid
ut v N(0, σ 2 ).
(1)
As the asymptotic distribution of the estimators of θ depends on whether the
mean µ is estimated or it is known and not estimated, we set in this case µ = 0,
without loss of generality. We denote the true value of the parameter θ for the
MA(1|µ) model, and θ0 for the zero mean model, say MA(1) (see Tanaka 1983
for this notation).
Traditionally, the parameters of the model have been estimated by the ML
1
principle, mainly due to efficiency considerations (see e.g. Durbin 1959). In this
respect, we extend the results in Tanaka (1984) to include also terms of order T −1 .
We denote the MLEs with a tilde, i.e. e
θ and µ
e for the MA(1|µ) model and θe0 when
we consider the MA(1) one. On the other hand, the 1st order autocorrelation of
the model, say ρ for the MA(1|µ) case and ρ0 for the zero mean case, is estimated
by its sample equivalent. However, one could, in principal, employe e
θ and θe0 to
estimate ρ, i.e. e
ρ=
h
θ
2
1+h
θ
and ρe0 =
θh0
3
2.
1+θh0
We call these estimators the MLEs of
ρ and ρ0 . We derive their second order expansion, as well. The expansions of e
ρ
and ρe0 are based on an extension of Sargan (1976).
On the other hand, one could equate the sample 1st order autocorrelation, say
b
ρ or ρb0 when there is no mean, with the theoretical one,
θ
,
1+θ2
and solve for the
unknown parameter. We call these the MM estimators of θ and θ0 , and denote
them by b
θ and θb0 , respectively (see also Davis and Resnick 1986 p.556). We derive
ρ, and ρb0 , as well as
the second order expansion of all four estimators, i.e. b
θ, θb0 , b
the expansion of µ
b up to the same order.4 The analysis follows Phillps (1977),
Sargan (1976) and Tanaka (1983).5
The second order expansions reveal that none of the estimators is uniformly
superior in terms of bias. Furthermore, the asymptotic superiority of the MLEs,
in terms of variance, does not hold for small number of observations.
In Section 2, we present the expansions of the MM estimators, i.e. the expansions of b
ρ, ρb0 , b
θ, θb0 , and µ
b. In Section 3 the expansions of the MLEs, e
θ, θe0 , e
ρ, ρe0 ,
and µ
e, are presented. In Section 4 the estimators are compared in terms of bias
and asymptotic efficiency, and Section 5 concludes. All proofs, rather lengthy and
tedious, are collected in Appendices at the end.
2
2
The Expansions of the MM Estimators
The following analysis is based on Tanaka (1983) and Phillips (1977). Given
the observations y = (y0 , ..., yT )0 , the estimators of the first order autocorrelation
coefficient and the mean are given by:
b
ρ =
where:
y 0 C1 y −
y 0 C2 y
µ
b = q4 + µ4 ,
⎛
⎜
⎜
⎜
1⎜
⎜
C1 = ⎜
2⎜
⎜
⎜
⎝
0
1
T
(d03 y) (d04 y)
−
1
T
(d03 y)2
1
.
1 ..
.
0 ..
.. . .
.
.
0 0
... ...
... ...
...
0
0 ···
0
1
=
q1 + µ1 − (q3 + µ3 ) (q4 + µ4 )
,
q2 + µ2 − (q3 + µ3 )2
⎛
⎞
0
⎟
.. ⎟
. ⎟
⎟
⎟
0 ⎟,
⎟
⎟
1 ⎟
⎠
0
⎜
⎜
⎜
⎜
⎜
C2 = ⎜
⎜
⎜
⎜
⎝
are (T + 1) × (T + 1) symmetric matrices and
1
0
..
.
..
.
0
(2)
⎞
··· ··· 0
⎟
. . . . . . .. ⎟
1
. ⎟
⎟
. . . . . . . . . .. ⎟ ,
. ⎟
⎟
⎟
... ...
1 0 ⎟
⎠
··· ··· 0 0
0
d4 = (0, 1, ..., 1)0 (T + 1) × 1 vectors
d3 = (1, ..., 1, 0)0 ,
y 0 Ci y − E (y 0 Ci y)
d0i y − E (d0i y)
, (i = 1, 2) ,
qi =
, (i = 3, 4) ,
qi =
T
T
1
1
E (y 0 C1 y) = θσ 2 + µ2 ,
µ2 = E (y 0 C2 y) = (1 + θ2 )σ 2 + µ2 ,
µ1 =
T
T
1
1
0
0
E (d3 y) = µ,
µ4 = E (d4 y) = µ.
µ3 =
T
T
Let e (q) = b
ρ − ρ, where ρ is the true parameter value and q = (q1 , q2 , q3 , q4 )0 ,
so that:
q1 + µ1 − (q3 + µ3 ) (q4 + µ4 )
− ρ,
q2 + µ2 − (q3 + µ3 )2
√
− ρ = 0. Thus, we can develop T e (q) in a Taylor series
e (q) =
and e (0) = 0 as
µ1 −µ3 µ4
µ2 −µ23
expansion as follows:
³ 3´
√
1
1
T e (q) = ei q i + √ eij q i q j +
eijk q i q j q k + O T − 2 ,
6T
2 T
3
where q i =
√
T qi .
In order to derive the Edgeworth expansion of b
ρ, we should define the char-
acteristic function of q i 0s and the cumulant generating function, say ψ (t). The
partial derivatives of e (q) and ψ (t) up to the third and fourth orders respectively
determine the so-called Edgeworth coefficients.
Let Σ denote the covariance matrix of y with (i, j)th element given by
⎧
³
´|i−j|
θ
⎨ (1 + θ2 )σ 2
,
f or |i − j| = 0, 1
2
1+θ
Σij =
⎩
0,
for |i − j| ≥ 2, ...
and m = (µ, ..., µ)0 the mean vector of y. Then it follows that
y v N(m, Σ).
/
The characteristic function of the y 0 Ci y (i = 1, 2) and di y (i = 3, 4) and the cumulant generating function of q i are given in Phillips (1980) and Tanaka (1983).
The derivatives of the cumulant generating function evaluated at zero are the cumulants of qi ; all first derivatives are zero, since q is standardized about its mean.
The second, third and fourth order cumulants of q i and the derivatives of e (q)
evaluated at 0 are presented in Appendix A1 [In Appendix A1 for Referees there
are detailed calculations].
2.1
The Expansion of the MM 1st Order Autocorrelation
³√
´
The second order asymptotic expansion of P
T (b
ρ − ρ) < x is given by:
³x´µ
³ x ´2
³ x ´3
³ x ´5 ¶
x
P (x) = Φ
,
−φ
c0 + c1 + c2
+ c3
+ c5
ω
ω
ω
ω
ω
ω
³x´
(3)
where the coefficients ci , i = 0, ..., 5 are given in Appendix A1 (see also Sargan
³ 3´
1976 and 1977, and Phillips 1977), and the error is O T − 2 .
4
Hence, we get [see Appendix A2 for Referees for calculations and the Edgeworth coefficients]
θ2 + 4θ4 + θ6 + θ8 + 1
ω =
¢4
¡ 2
θ +1
2
and
c0
c1
c2
¡
¢
¡
¢
¡
¢
(1 + θ)2 1 + θ10 + 3θ3 1 + θ6 + 5θ4 1 + θ4
¡
¢
+7θ5 1 + θ2 + 2θ6
1
√
=
¢2 ¡
¢
¡ 2
ω T
θ + 1 θ2 + 4θ4 + θ6 + θ8 + 1
⎛
¡
¢
¡
¢
¡
¢
1 + θ24 + 8θ 1 + θ22 + 11θ2 1 + θ20 + 20θ3 1 + θ18
⎜
¡
¢
¡
¢
¡
¢
⎜
⎜
+49θ4 1 + θ16 + 72θ5 1 + θ14 + 36θ6 1 + θ12
⎜
¡
¢
¡
¢
¡
¢
⎜
⎜ +132θ7 1 + θ10 + 213θ8 1 + θ8 + 228θ9 1 + θ6
⎝
¡
¢
¡
¢
+97θ10 1 + θ4 + 308θ11 1 + θ2 + 338θ12
1
= −
¢3
¡ 2
4T
θ + 4θ4 + θ6 + θ8 + 1
¡
¢¡
¢
θ θ4 + 1 6θ4 + θ8 + 1
1
√ ¡
=
¢ ¡
¢,
ω T θ2 + 1 3 θ2 + 4θ4 + θ6 + θ8 + 1
(4)
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎠
⎞
¡
¢
¡
¢
24
3
22
1 + θ + 206θ − θ 1 + θ − 8θ 1 + θ
⎜
¡
¢
¡
¢
¡
¢ ⎟
⎟
⎜
5
18
6
16
7
14
⎟
⎜ −52θ 1 + θ + 13θ 1 + θ − 84θ 1 + θ
⎟
⎜
¡
¢
¡
¢
¡
¢
⎟
⎜
⎜ −62θ8 1 + θ12 − 224θ9 1 + θ10 + 141θ10 1 + θ8 ⎟
⎜
¡
¢
¡
¢
¡
¢ ⎟
⎜
11
6
13
2
12
4 ⎟
⎜ −276θ 1 + θ − 376θ 1 + θ − 199θ 1 + θ ⎟
⎠
⎝
¡
¢¡
¢
3
23
−4θ
1
−
θ
1
−
θ
1
c3 =
,
¢2 ¡
¢3
¡ 2
4T
θ + 1 θ2 + 4θ4 + θ6 + θ8 + 1
¢2 ¡
¢2
¡ 4
6θ + θ8 + 1 θ4 + 1 θ2
1
c5 = −
.
¢14
¡ 2
2T ω 6
θ +1
√
ρ − ρ) (see Fuller 1976).
Notice that ω2 is the asymptotic variance of T (b
⎛
28
14
,
2
(5)
To evaluate the bias, Mean Squared Error (MSE) and the cumulants, needed
in the sequel, we employ an alternative second order representation, deduced by
a Taylor series expansion of the Edgeworth formula (see Sargan 1976 and Phillips
5
1977), i.e.
³x´
³ x ´2
³ x ´3 ¶
x
+ O(T −3/2 ),
+ d0 + d1
+ d2
+ d3
P (x) = Φ
ω
ω
ω
ω
µ
(6)
where
d0 = c0 ,
d1
d3
d2 = c2 ,
c2
= c1 + 0
⎛2
¡
¢
¡
¢
¡
¢
¡
¢ ⎞
20
4
16
5
14
6
12
1 + θ + θ 1 + θ − 3θ 1 + θ + 8θ 1 + θ + 66θ 1 + θ
⎠
⎝
¡
¢
¡
¢
¡
¢
¡
¢
8
8
9
6
10
4
11
2
12
1
+
θ
+
16θ
1
+
θ
+
173θ
1
+
θ
−
24θ
1
+
θ
+
26θ
+5θ
1
,
=
¢3
¡ 2
4T
θ + 4θ4 + θ6 + θ8 + 1
= c3 + c2 c0
⎛
⎞
¡
¢
¡
¢
¢
¡
1 + θ2 + 17θ4 1 + θ16 + 6θ6 1 + θ12
⎝
⎠ 6θ2 + θ4 + 1
¡
¢
¡
¢
¡
¢
+81θ8 1 + θ8 + 25θ10 1 + θ4 + 122θ12 + θ22 1 + θ2
1
=
.
¢2 ¡
¢3
¡ 2
4T
θ + 1 θ2 + 4θ4 + θ6 + θ8 + 1
24
2
Setting
z = d0 + (1 + d1 )
³x´
ω
+ d2
³ x ´2
ω
+ d3
³ x ´3
ω
+ O(T −3/2 )
we find that
¡
¡
¢
¢
x
= −d0 + (1 − d1 + 2d0 d2 ) z − d2 z 2 + 2d22 − d3 z 3 + O T −3/2 ,
ω
(7)
where z can be considered as a standard normal variable. Consequently,
E
³x´
ω
and it follows that
¡
¡
¢
¢
= (−d0 − d2 ) + O T −3/2 = −c0 − c2 + O T −3/2
¢
¡
a4
= − √ + O T −3/2 ,
2ω T
¡
¢¡
¢
³√
´
¡
¢
1 θ + θ2 + 1 2θ − 2θ2 + 2θ3 + θ4 + 1
k1 = E
T (b
ρ − ρ) = − √
+ O T −3/2
¢3
¡ 2
T
θ +1
6
and
¡
¢
k2 = ω 2 + k2& + O T −3/2
¡
¢
¡
¢
¡
¢
13θ4 1 + θ4 − 7θ2 1 + θ8 − 2θ 1 + θ10
¡
¢
−6θ5 1 + θ5 − 22θ6 + θ12 + 1
¡
¢
2
= ω2 −
+ O T −3/2 .
¢6
¡ 2
T
θ +1
Employing equation (7), taking expectations, using the connection between
moments and cumulants (see Kendal and Stuart 1969), and keeping terms up to
order O (T −1 ) we get
¡
¢¡
¢
1 θ θ4 + 1 6θ4 + θ8 + 1
k3 = −6 √
¢7
¡ 2
T
θ +1
and
¡
¢
¡
¢
¡
¢ ⎞
16
4
12
6
8
1 − 10θ 1 + θ + 30θ 1 + θ − 106θ 1 + θ
⎠
⎝
¡
¢
8
4
10
20
+129θ 1 + θ − 216θ + θ
−6
k4 =
.
¢10
¡ 2
T
θ +1
⎛
2
In Figure 1 we present the difference between the exact cumulative distribution
√
of T (b
ρ − ρ), based on 20000 simulations for T = 30, θ = −0.3 and µ = 5,
and the normal approximation (thick line) and between the exact and the T −1
approximation in equation (6).6 It is apparent that the approximation is much
better than the asymptotic normal, although the normal is not far away from the
exact. We should emphasize here that although the approximations are close to
the exact one, this is not always the case with Edgeworth type expansions (see
e.g. Ali 1984).
In terms of MSE, squaring (7) and keeping terms up to order O (T −1 ), we
7
have that
E
2.1.1
´2
³√
θ2 + 4θ4 + θ6 + θ8 + 1
T (b
ρ − ρ)
=
¢4
¡ 2
θ +1
⎞
⎛
3
2
4
5
6
6θ − 2θ − 12θ + θ − 36θ + θ
⎠
(θ + 1)2 ⎝
7
8
9
10
+6θ − 2θ − 12θ + θ + 1
1
−
.
¢6
¡ 2
T
θ +1
The Zero-Mean Expansion
For the zero mean case, i.e. when µ = 0 or µ is known and one analyses the
demeaned data, we have
e (q) =
q1 + µ1
− ρ0 ,
q2 + µ2
µ1 = θ0 σ 2 ,
The derivatives ei =
∂e(0)
,
∂qi
eij =
∂ 2 e(0)
,
∂qi ∂qj
µ1
− ρ0 = 0.
µ2
¡
¢
µ2 = 1 + θ20 σ 2 .
as
eijk =
∂ 3 e(0)
∂qi ∂qj ∂qk
and the cumulants of qi can
be found from equations (24) and (25) and (21), (22) and (23), respectively, by
setting µ = 0.
Consequently, we could evaluate the Edgeworth coefficients and substituting
into equation (26) (Appendix A1) we could get the expansion coefficients cj , j =
1, ..5. Notice that the summation now runs up to 2. However, we get that c2 and
c5 are the same as in equations (4) and (5), respectively, the only difference is
that now we denote θ0 the true value of the parameter. Furthermore, as expected,
√
the asymptotic variance of T (ρb0 − ρ0 ), ω 2 , is the same as in the non-zero mean
8
case, i.e. ω 2 =
θ20 +4θ40 +θ60 +θ80 +1
. The coefficients that differ are given:
4
(θ20 +1)
¡ 4
¢2
θ0 + 1 θ0
1
¢¡
¢,
√ ¡
=
ω T θ20 + 4θ40 + θ60 + θ80 + 1 θ20 + 1
¢¡
¢
¡
¢
¡
¢
¡
¢
¡ 4
+ 48θ60 1 + θ12
+ 21θ80 1 + θ80 + 97θ10
1 + θ40 + 50θ12
θ0 − θ20 + 1 1 + θ20
0
0
0
0
=
,
¡ 2
¢4 ¡ 2
¢2
4
6
8
4ω 2 T θ0 + 1 θ0 + 4θ0 + θ0 + θ0 + 1
¡
¢
¡
¢
¡
¢
¡
¢
1 + 3θ20 1 + θ24
+ 16θ40 1 + θ20
+ 69θ60 1 + θ16
+ 62θ80 1 + θ12
0
0
0
0
¡
¢
¡
¢
10
8
12
4
14
28
+393θ
1
+
θ
+
177θ
1
+
θ
0
0
0
0 + 606θ 0 + θ 0
1
=
,
¢6 ¡
¢2
¡ 2
4ω2 T
θ0 + 1 θ20 + 4θ40 + θ60 + θ80 + 1
c0
c1
c3
Again, employing the representation in (7), it is worth noticing that the 1st
cumulant, i.e. the bias, is this time
¡
¢
³√
´
³ 3´
2 θ40 + 1 θ0
k1 = E
T (ρb0 − ρ0 ) = − √ ¡ 2
+
O
T −2 ,
¢3
T θ0 + 1
(8)
which is different from the case of non-zero mean, i.e. when µ 6= 0. In fact, we can
√
plot the absolute values of the two biases (multiplied by T ) against the common
values of θ0 and θ (see Figure 2). For values of θ between −0.2 and 1 it is obvious
that ρb0 is less biased than b
ρ. In fact for θ = 0 the bias of b
ρ is − T1 . However, for
the interval (−1, −0.204) the opposite is true.
Furthermore, the higher order cumulants are given by:
12
¡
¢
2 17θ40 − 5θ20 − 18θ60 + 17θ80 − 5θ10
0 + θ0 + 1
+ O T −3/2
¢6
¡ 2
T
θ0 + 1
¡ −3/2 ¢
2
&
= ω + k2 + O T
¡
¢¡
¢
¡
¢
1 6θ40 + θ80 + 1 θ40 + 1 θ0
k3 = −6 √
+ O T −3/2 .
¢7
¡ 2
T
θ0 + 1
k2 = ω 2 −
and
¡
¢
¡
¢
¡
¢
1 + 30θ40 1 + θ12
− 10θ20 1 + θ16
− 106θ60 1 + θ80
0
0
¡
¢
20
+129θ80 1 + θ40 − 216θ10
¡
¢
0 + θ0
6
+ O T −3/2 .
k4 = −
¢10
¡ 2
T
θ0 + 1
9
Notice that the third cumulant in this case, is equal to the one for the non-zero
µ one. Hence, the non-normality of the approximations is affected in the same
way for both cases. However, the 4th order cumulant for the non-zero µ is much
bigger than the one for the zero case, at least for positive values of θ.
The MSE is given by
³√
´2
θ20 + 4θ40 + θ60 + θ80 + 1
E
T (ρb0 − ρ0 )
=
¢4
¡ 2
θ0 + 1
¢¡
¢
¢2 ¡ 4
¡
¡
¢
θ0 − 4θ20 + 1 θ40 − θ20 + 1
2 1 − θ20
−
+ O T −3/2 .
¢6
¡ 2
T
θ0 + 1
This is different from the non-zero mean case. In fact, if we plot the two MSEs,
for T = 10 and common values of θ and θ0 (Figure 3), we observe that there is an
interval, when θ < −0.2, where the MSE of b
ρ, the non-zero mean estimator of
ρ, is less than the MSE of ρb0 . Of course, for bigger values of T the two MSEs
collapse to the common asymptotic variance.
2.2
Expansion of Mean MM Estimator
The asymptotic expansion for µ
b can be done almost in the same way as that for
the first order autocorrelation. We denote the error in the estimate µ
b defined in
(2) as e (q) = q4 + µ4 − µ. Now, all partial derivatives of e (q), ei (q), are zero
apart from e4 (q) which is 1. Note that the partial derivatives of ψ (t) remain
unchanged. Consequently,
ω 2 = (1 + θ)2 σ 2 .
√
µ − µ).
Notice that, again, ω 2 is the variance of the asymptotic distribution of T (b
³√
´
The asymptotic expansion of P
T (b
µ − µ) < x is obtained as in (3). Notice
that all Edgeworth coefficients, aj j = 1, ..., 10, are zero. Consequently, we have
that
¡
¢
E (b
µ) = µ + O T −2 ,
10
and the second order Edgeworth expansion of P
P (x) = Φ
³x´
ω
³√
´
T (b
µ − µ) < x is given:
³ 3´
+ O T −2 ,
as the coefficients, cj for j = 1, ..., 5, in the expansion, are zero. Hence, the second
order approximate distribution of µ is normal.
2.3
The Expansion of the MM MA Coefficient Estimator
In this subsection we follow the method developed in Sargan (1976, 1977). Notice that in our case k1 is non zero. Furthermore, the second order cumulant of
√
T (b
ρ − ρ), apart from the O (1) term, includes a term of O (T −1 ). This term can
not be ignored in the expansion (as it is ignored in Sargan 1976) (see Appendix
A2 [Appendix A3 for Referees]). In these respects we can say that the method
developed is an extension of Sargan (1976).
The second order Edgeworth approximation of P
h√ ³
´
i
T b
θ − θ ≤ x is:
³x´ ∙
³x´
³ x ´2
³ x ´3
³ x ´5 ¸
,
G(x) = Φ
+ c3
+ c5
−φ
c0 + c1
+ c2
δ
δ
δ
δ
δ
δ
³x´
(9)
where
c0
1
= √
6 T
c1
1
=
24T
c2
c5
µ
Ã
3b4 + 6b5 b1 + 3b3
−
δ
δ3
¶
,
12b9 + 12b25 + 9b24 + 12b7 + 36b4 b5 + 12b8
(b1 + 3b3 )2
+ς +5
δ2
δ6
Ã
!
(b1 + 3b3 )2
1 b1 + 3b3
1
10
= √
, c3 = −
+ς ,
72T
δ3
δ6
6 T
1 (b1 + 3b3 )2
=
,
72T
δ6
ς = −3
b2 + 4b1 b5 + 4b6 + 12b3 b5 + 14b4 b1 + 18b3 b4
,
δ4
11
(10)
!
,
δ
2
b2
b3
b5
¢¡
¢
¡
θ θ4 + 1 6θ4 + θ8 + 1
θ2 + 4θ4 + θ6 + θ8 + 1
=
, b1 = −6 ¡ 2
,
¢¡
¡
¢2
¢3
1 − θ2
θ + 1 1 − θ2
¡
¢
¡
¢
¡
¢
¡
¢
1 + 30θ4 1 + θ12 − 10θ2 1 + θ16 − 106θ6 1 + θ8 + 129θ8 1 + θ4
(11)
−216θ10 + θ20
= −6
,
¢2 ¡
¡ 2
¢4
θ + 1 1 − θ2
¢2
¢2
¢¡
¢ ¡
¢2 ¡
¡
¡
2θ 3 − θ2 θ2 + 4θ4 + θ6 + θ8 + 1
2θ 3 − θ2 θ2 1 + θ2 + θ4 + 1
¢
¡ 2
, b4 = ¡
,
=
¢
¡
¢5 ¡ 2
¢3
θ +1
1 − θ2
θ +1
1 − θ2
¢h ¡
¢ ¡
¢i
¡
2
2
2 2
θ + θ + 1 2θ 1 + θ + 1 − θ
¡ 2
¢
¢¡
,
= −
θ + 1 1 − θ2
b6
b7
b8
¢ ¡
¢3
¡ 2
11θ − 5θ4 + θ6 + 1 θ4 θ2 + 4θ4 + θ6 + θ8 + 1
= 2
,
¢6
¡
¢8
¡ 2
1 − θ2
θ +1
¢ ¡
¢2
¡ 2
11θ − 5θ4 + θ6 + 1 θ4 θ2 + 4θ4 + θ6 + θ8 + 1
= 2
,
¢6
¡
¢6
¡ 2
1 − θ2
θ +1
¢¡
¢
¢ ¡
¡
12 3 − θ2 θ2 θ4 + 1 6θ4 + θ8 + 1
= − ¡
, and
¢2
¢4
¡ 2
1 − θ2
θ +1
13θ4 − 7θ2 − 2θ − 6θ5 − 22θ6 − 6θ7 + 13θ8 − 7θ10 − 2θ11 + θ12 + 1
b9 = −2
.
¢2 ¡
¡ 2
¢2
θ + 1 1 − θ2
Note that, as in Fuller (1976), δ 2 is the variance of the limiting distribution of
´
√ ³
T b
θ − θ (see Appendix A2 [Appendix A3 for Referees]), which is also the
´
√ ³
b
asymptotic variance of T θ0 − θ0 .
In Figure 4 we present the difference between the exact cumulative distribution
´
√ ³
b
of T θ − θ , based on 20000 simulations for T = 30 true θ = −0.3 and µ =
5, and the normal approximation (thick line) and between exact and the T −1
approximation in equation (9). It is apparent that the approximation is much
better than the asymptotic normal, apart from values bigger than 1.77.
Furthermore, keeping terms up to order O (T −1 ), we can find the bias of
12
´
√ ³
T b
θ − θ as
E
³√ ³
´´
³ 3´
1
T b
θ−θ
= − √ (b4 + 2b5 ) + O T − 2
2 T
1 2θ4 − 6θ3 − 2θ2 − 3θ5 − 2θ6 + θ8 + θ9 + 1
.
= √
¡
¢3
T
1 − θ2
In terms of MSE we have that, keeping relevant terms
⎞
⎛
2
2
³√ ³
´´2
1 ⎝ 12b9 + 12b5 + 9b4 + 12b7 + 36b4 b5 + 12b8 ⎠
E
T b
θ−θ
= δ2 −
2
3)
12T
−10 (b1 +3b
4
δ
2
=
2.3.1
4
6
8
θ + 4θ + θ + θ + 1
¡
¢2
1 − θ2
⎛
6θ + 88θ2 + 36θ3 + 59θ4 + 84θ5 + 533θ6 + 84θ7 + 63θ8
⎜
⎜
⎜
+132θ9 + 471θ10 + 60θ11 − 91θ12 − 12θ13 + 59θ14
⎜
⎜
+12θ15 − 32θ16 − 18θ17 + θ18 + 1
1⎜
⎜
4
2
+ ⎜
(1−θ2 ) (θ2 +1)
T⎜
2
⎜
(θ2 +4θ4 +θ6 +θ8 +1) (92θ2 +115θ4 +2θ6 −25θ8 −6θ10 +3θ12 +27)θ2
⎜
−
6
6
⎜
(θ2 +1) (1−θ2 )
⎜
2
⎝
(2θ12 −12θ4 −23θ6 −13θ8 −11θ10 −5θ2 −θ14 +θ16 −2) θ2
+30
2
6
(θ2 +4θ4 +θ6 +θ8 +1) (1−θ2 )
The Zero-Mean Expansion
For the µ = 0 case, we have that δ and bi for i = 1, 2, 3, 4, 6, 7, 8 are the same as
in equation (11) (the only difference is that now θ0 denotes the true value of the
parameter instead of θ). The other two coefficients are:
¢
¡ 4
12
θ0 + 1 θ0
17θ40 − 5θ20 − 18θ60 + 17θ80 − 5θ10
0 + θ0 + 1
¢¡
¢
b5 = −2 ¡ 2
,
and
b
=
−2
.
¢2 ¡
¡ 2
¢2
9
θ0 + 1 1 − θ20
θ0 + 1 1 − θ20
Consequently, the 1st cumulant is given by
¡ 8
¢
4
6
2
³√ ³
´´
³ 3´
θ
−
2θ
−
θ
−
5θ
−
1
θ0
1
0
0
0
0
b
E
T θ0 − θ0 = √
+ O T −2 .
¡
¢3
2
T
1 − θ0
(12)
√
Plotting, again, the absolute values of the two biases (multiplied by T ), i.e.
¯ ³ ³
¯ ³ ³
´´¯
´´¯
¯
¯
¯
¯
b
b
¯E T θ − θ ¯ and ¯E T θ0 − θ0 ¯, against the common values of θ and θ0 ,
13
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟.
⎟
⎟
⎟
⎟
⎟
⎠
we can observe that for values of θ bigger than about 0.3 the bias of b
θ is smaller
than the one of θb0 (see Figure 5).
Furthermore, for the zero mean case we have that the MSE of the estimator is
E
³√ ³
´´2
θ2 + 4θ40 + θ60 + θ80 + 1
T θb0 − θ0
= 0
¡
¢2
1 − θ20
¢2 2
¡
4
6
8
10
2
14
16
θ0
30 2θ12
0 − 12θ 0 − 23θ 0 − 13θ 0 − 11θ 0 − 5θ 0 − θ 0 + θ 0 − 2
+
¢
¡ 2
¡
¢
2
6
T
θ0 + 4θ40 + θ60 + θ80 + 1 1 − θ20
12
158θ60 − 36θ40 − 31θ20 + 952θ80 + 2672θ10
0 + 3928θ 0
16
18
20
22
+4001θ14
0 + 2216θ 0 + 463θ 0 − 546θ 0 − 346θ 0
26
28
30
−134θ24
0 − 8θ 0 + 22θ 0 + 3θ 0 − 2
¢6 ¡
¡ 2
¢6
θ0 + 1 1 − θ20
1
−
T
We can plot the MSE for the non-zero and zero mean cases, for T = 10
(Figure 6). It is apparent that even for such a small number of observations,
there is little difference between the two. Still, for values of θ < 0.1 the MSE
of b
θ, the estimator of θ in the non-zero mean case, is less than the one of the
estimator in zero mean one.
3
The Expansion of the ML Estimators
In this section we extend the analysis in Tanaka (1984) to include terms of order
T −1 in the approximation of the MLE of the MA (1|µ) parameters, θ and µ, say
e
θ and µ
e. These are the solutions to the following equations:
³ ´
∂ e
θ
∂ (e
µ)
= 0 and
= 0,
∂µ
∂θ
where
2
(θ) = −
T
X
u2t
T log(2πσ )
− t=1 2
2
2σ
and
14
ut = yt − µ − θut−1 .
The first order conditions are
³ ´
µ
¶¯¯
T
∂ e
θ
∂ut−1
1 X
¯
ut θ
= 0⇒ 2
+ ut−1 ¯
¯
∂θ
σ t=1
∂θ
and
¯
T
∂ (e
µ)
1 X ∂ut ¯¯
ut
= 0⇒ 2
¯
∂µ
σ t=1 ∂µ ¯
=0
θ=h
θ
=0
µ=h
µ
In Appendix B1 we present the up to 4th order derivatives of the log-likelihood
function and their expectations [in Appendix B1 for Referees there is analytic
derivation of the results].
³√ ³
´ √
´/
¡
¢/
Now let ϕ = θ1 , θ2 =
T e
θ − θ , T (e
µ − µ) . Consider the Taylor
³
³ ´/
´/
ϕ)
1 ∂ (h
e
e
√
e = θ1 , θ2
= e
θ, µ
e around the true value
expansion for T ∂ϕ , where ϕ
ϕ = (θ1 , θ2 )/ = (θ, µ)/ as:
2
X ∂ 3 (ϕ)
1 X ∂ 2 (ϕ)
1
θk +
θk θl
0 = gj +
T k=1 ∂θj ∂θk
2T 3/2 k,l=1,2 ∂θj ∂θl ∂θk
+
1
6T 2
where j = 1, 2 and gj =
∂4
∂θj ∂θn ∂θl ∂θk
X
∂ 4 (ϕ∗ )
θn θk θl
∂θj ∂θn ∂θl ∂θk
n,k,l=1,2
2
3
√1 ∂ , ∂
, ∂
T ∂θj ∂θj ∂θl ∂θj ∂θl ∂θk
are evaluated at the true values and
are evaluated at an intermediate point, say ϕ∗ , between the estimates
and the true values. The above expression can be written as
¶
¶
2 µ
2 µ
X
wji
1 X
qjik
Aji + √
Kjik + √
θi + √
θi θk
0 = gj +
T
2
T
T
i=1
k,i=1
(13)
2
³ 3´
1 X
+
Mjikl θl θi θk + Op T − 2
6T l,k,i=1
³ 3´
≡ fj (ϕ, h) + Op T − 2
j = 1, 2
³ 2 ´
³ 3
³
´
´
∂ (ϕ)
∂ (ϕ)
∂ 4 (ϕ)
1
1
1
where Aij = T E ∂θj ∂θi , Kjik = T E ∂θj ∂θi ∂θk , Mjikn = T E ∂θj ∂θi ∂θk ∂θn ,
µ 2
µ 3
¶
¶
∂ (ϕ)
∂ (ϕ)
1
1
wij = √
− T Aij , qijk = √
− T Kjik , (14)
T ∂θj ∂θi
T ∂θj ∂θi ∂θk
15
for i, j, k = 1, 2. Let us define a vector h containing the non-zero elements of gi ,
wij , qijk , q112 for i, j, k = 1, 2. As however w22 = q122 = q222 = 0 we define h as
h = (h1 , h2 , h3 , h4 , h5 , h6 )/ = (g1 , g2 , w11 , w12 , q111 , q112 )/ . Solving equation (13) for
θj , and j = 1, 2, as continuously differentiable functions of h, gives:
θj (h) =
6
X
∂θj (0)
a=1
≡
6
X
∂ha
ea(j) ha
a=1
(j)
where ea =
∂θj (0)
,
∂ha
6
6
³ 3´
1 X ∂ 2 θj (0)
1 X ∂ 3 θj (0)
ha +
ha hb +
ha hb hc + Op T − 2
2 a,b=1 ∂ha ∂hb
6 a,b,c=1 ∂ha ∂hb ∂hc
6
6
³ 3´
1 X (j)
1 X (j)
+ √
eab ha hb +
eabc ha hb hc + Op T − 2
6T a,b,c=1
2 T a,b=1
(j)
eab =
√ ∂ 2 θj (0)
T ∂ha ∂hb
∂ 3 θ (0)
(j)
and eabc = T ∂ha ∂hj b ∂hc .
Now the derivatives can be found by solving the following system of equations,
for j, k = 1, 2 and a, b, c = 1, ..., 6 (see Appendix B2) [Appendix B2 for Referees]:
0 =
0 =
2
X
Ajk ea(k) +
k=1
Ã
2
X
k=1
∂fj (0, 0)
,
∂ha
(15)
2
1 X
∂ 2 fj (0, 0)
(l)
√
Kjkl eb +
T l=1
∂hb ∂e
θk
!
ea(k)
2
1 X
(k)
Ajk eab ,
+√
T k=1
+
2
X
∂ 2 fj (0, 0)
k=1
∂ha ∂e
θk
(k)
eb
(16)
and
⎛
2 ⎜
X
⎜
⎜
0 =
⎜
k=1 ⎝
1
T
2
X
(l) (p)
Mjlkp eb ec
p,l=1
2
X
∂ 3 fj (0,0) (l)
+
e
∂hc ∂h
θl ∂h
θk b
l=1
1
T
(l)
Kjkl ebc
l=1
+
2
X
p=1
∂ 3 fj (0,0) (p)
ec
∂h
θp ∂hb ∂h
θk
⎞
⎟
⎟ (k)
⎟ ea
⎟
⎠
!
3
∂
f
(0,
0)
j
(k)
+
eb
Kjkl e(l)
e(p)
ac +
c
e e
p=1 ∂ θ p ∂ θ k ∂ha
k=1
l=1
Ã
!
2
2
2
X
∂
f
(0,
0)
1X
1
j
(k)
eab
+
Kjkp e(p)
c + √
e
T
T ∂hc ∂ θk
p=1
k=1
2
X
Ã
+
2
X
1
T
2
X
2
X
2
2
2
1 X ∂ 2 fj (0, 0) (k)
1 X ∂ 2 fj (0, 0) (k) 1 X
(k)
+√
ebc + √
eac +
Ajk eabc .
T k=1
T k=1 ∂e
T k=1 ∂hb ∂e
θk ∂ha
θk
16
(17)
The derivatives for j = 1, 2 and the actual values of Ajk , Kljk and Mnljk for
n, l, i, k = 1, 2 are presented in Appendix B2.
To find the cumulants of h let us rewrite the Likelihood function of the model
as
¢ 1
T ¡
1
ln 2πσ 2 − ln |Σ| − 2 (y − µd)/ Σ−1 (y − µd)
2
2
2σ
³
³
´/
´/
/
E [(y−µd)(y−µd) ]
where Σ =
,
y
=
and
d
=
y1 y2 · · · yT
1 1 ··· 1 .
σ2
(θ) = −
Consequently, ha (a = 1, 3, 5) is of the form
Ã
!
/
(y − µd) Ca (y − µd)
1
ha = √
− tr (Ca Σ)
σ2
T
and hb (b = 2, 4, 6) is of the form
1
hb = √
T
Ã
/
cb (y − µd)
σ2
!
where
C1
C5
/
c2
µ 2
¶
1 ∂ Σ ∂Σ −1 ∂Σ
1 −1 ∂Σ −1
−1
=
−
Σ
Σ , C3 = Σ
Σ
Σ−1 ,
2
∂θ
2 ∂θ2
∂θ
∂θ
µ
¶
∂Σ −1 ∂Σ −1 ∂Σ 1 ∂ 2 Σ −1 ∂Σ 1 ∂Σ −1 ∂ 2 Σ
−1
Σ−1 , and
Σ
Σ
−
−
Σ
= 3Σ
Σ
∂θ
∂θ
∂θ
2 ∂θ2
∂θ
2 ∂θ
∂θ2
µ
¶
2
/
/
/ −1
/ −1 ∂Σ −1
/
−1 ∂Σ −1 ∂Σ −1
−1 ∂ Σ −1
.
Σ , c6 = d 2Σ
Σ
Σ −Σ
= d Σ , c4 = −d Σ
Σ
∂θ
∂θ
∂θ
∂θ2
Now under our assumptions we have that
(y − µd)
∼ N (0, Σ)
σ
and the characteristic function of
(y−µd)/ Ca (y−µd)
σ2
/
(a = 1, 3, 5) and
cb (y−µd)
σ2
(b = 2, 4, 6)
of the same form as in Phillips (1980). In Appendix B3 [Appendix B3 for Referees]
the cumulants of h are presented.
Now from Sargan (1976) and (1977), and Phillips (1977) we have for l = 1, 2
³ x ´³
´
³ 3´
³ x ´
(l)
(l)
(l) 2
(l) 3
(l) 5
P (x) = Φ (l) − φ (l) c0 + c1 x + c2 x + c3 x + c5 x + O T − 2 ,
ω
ω
17
(l)
(l)
where c0 , ..., c5 are given by equation (26) and the Edgeworth coefficients are
defined in the same way as in equations (27) and (28) (see Appendix A1), with
the difference that the summations now run up to 6.
3.1
Expansion of the ML MA Coefficient Estimator
´
√ ³
e
For l = 1, i.e. for T θ − θ , we get
¡ (1) ¢2
¢
¡
ω
= 1 − θ2 ,
(1)
= −1,
γ1
(1)
γ3
i
4i 5θ2 + 2
(1)
(1)
¢,
β 1 = − √ 6θ, β 3 = √ ¡
T
T 1 − θ2
−4θ
3θ2 + 1
(1)
(1)
(1)
(1)
=
,
γ
=
−6
¡
¢2 , γ 2 = γ 4 = γ 6 = 0,
5
2
2
1−θ
1−θ
6θ
i
4i 5θ2 + 2
(1)
= −√
, δ 13 = √ ¡
¢ ,
T 1 − θ2
T 1 − θ2 2
¡
¢
i 4 1 − θ2
= √ 2
,
T σ (θ + 1)4
(1)
δ 11
(1)
δ 24
2i (1 − θ)
(1)
δ 22 = − √
,
T σ 2 (1 + θ)2
and
(1)
a1
(1)
a4
(1)
a7
(1)
a9
¡
¢
i
= − √ 6θ 1 − θ2 ,
T
¢¡
¡
¢
¢
6¡ 2
(1)
7θ + 3 1 − θ2 , a3 = 2θ 1 − θ2 ,
T
¢
¡
¢
¢¡
¡
i
(1)
(1)
= 2 (1 − 2θ) , a5 = 4 √ 20θ2 − 2θ + 5 , a6 = −6 1 − θ2 θ2 + 1 ,
T
¡ 2
¢
¡
¢
¢¡
(1)
= 4 θ − 2θ + 6 , a8 = −2 1 − θ2 θ2 + 3 ,
¡
¢
¢
¢¡
8i ¡
(1)
= 4 θ2 − 2θ + 4 , a10 = − √ 1 − θ2 4θ2 + 1 .
T
(1)
a2 =
Hence we have that
¢
1
1 ¡
2 θ
(1)
√ , c(1)
8θ + 79θ2 + 17 , c2 = − √
1 = −
2
4T ω
ω ¡T
Tω
¢
2
θ2
1 8θ + 113θ − 3
2
(1)
(1)
¡
¢.
c3 =
,
c
=
−
5
4T
ω2
T 1 − θ2
´
√ ³
th
e
Furthermore, we shall need in the sequel the j order cumulants of T θ − θ ,
(1)
c0
=
say kj for j = 1, ..., 4. Consequently, employing the representation in equation (7)
18
we get:
(1)
³√ ³
´´
¢
¡
a4
1 − 2θ
e
√
T θ−θ =−
k1 = E
=− √
+ O T −3/2 ,
2 T
T
as in Tanaka (1984),
¢
2¡
2θ + 37θ2 − 6 ,
k2 = ω 2 −
T
¡
¡
¢
¢
12
(1)
k3 = −6c2 ω 3 + O T −3/2 = √ θω2 + O T −3/2
T
and
¢
¡
¢
ω2 ¡ 2
49θ − 3 + O T −3/2 .
k4 = −6
T
Notice that the first order approximation is given by (see Tanaka 1984 as well)
³ x ´2 ¶
³x´µ
³x´
¡
¢
+ O T −1 .
−φ
c0 + c2
P (x) = Φ
ω
ω
ω
Consequently, the closer to the boundary values θ is and the further away from
zero x is, the greater the discrepancy between the two approximations.
In Figure 7 we present the difference between the exact cumulative distribution
´
√ ³
θ − θ , based on 20000 simulations for T = 30 true θ = −0.3 and µ = 5,
of T e
and the normal approximation (thick line), the T −1 approximation in equation
√
(6) (thin line), and the T approximation (dotted line). It is apparent that the
T −1 approximation is much better than the asymptotic normal, apart from the
√
interval (1.5, 2). However, the T approximation is better than both for the
interval (−6.2, −1.8) and (0, 0.8), although for values smaller than −7 it behaves
very badly.
In terms of MSE we get
³√ ³
´´2 ¡
¢
¢ 1¡
E
8θ + 70θ2 − 13
T e
θ−θ
= 1 − θ2 −
T
3.1.1
The Zero-Mean Case
Now for the case that µ = 0, we have that equation (13) is a function of only w11 ,
q111 , A11 , K111 and M1111 , which are as in (14) and (32). The only difference is
19
that the true θ is now denoted by θ0 , and all other coefficients of both equations
are 0. Hence, the vector h is now h = (h1 , h2 , h3 )/ = (g1 , w11 , q111 )/ . Following the
procedure of Section 3 and setting µ to 0 we find that the expansion coefficients
are given by:
c0
c3
¢
¡
1 1
1 97θ20 + 7
θ0 , c1 = −
= √
,
4T
ω2
Tω
1 121θ20 − 3
2 θ20
=
,
c
=
−
.
5
4T
ω2
T ω2
2 θ0
,
c2 = − √
T ω
It follows that
´´
³√ ³
¢
¢
¡
¡
θ0
a4
k1 = E
T θe0 − θ0 = − √ + O T −3/2 = √ + O T −3/2 ,
2 T
T
(18)
the same result as in Tanaka (1984), and Bao and Ullah (2007),
and
¡
¢
¡
¢
k2 = ω 2 1 + 14c22 − 6c3 − 2c0 c2 − c20 − 2c1 + O T −3/2
¢
¡
¢
1¡ 2
37θ0 − 4 + O T −3/2 ,
= ω2 − 2
T
¡
¡
¢
¢
12
k3 = −6c2 ω 3 + O T −3/2 = √ θ0 ω2 + O T −3/2 ,
T
¢
¡
¡
¢
¡ −3/2 ¢
¢
ω2 ¡ 2
4
2
49θ0 − 3 + O T −3/2 .
= −6
k4 = ω −24c3 − 24c2 c0 + 96c2 + O T
T
Notice that the 3rd and 4th order cumulants are the same as the cumulants of
the non-zero mean case, for common values of θ and θ0 . Furthermore, it is worth
noticing that the absolute bias of e
θ is greater than the one of θe0 for the interval
¡
¢
¡ ¢
−1, 13 and smaller for the interval 13 , 1 .
In terms of MSE we have
E
³√ ³
´´2 ¡
¢
¡
¢ 1¡ 2
¢
73θ0 − 8 + O T −3/2
T θe0 − θ0
= 1 − θ20 −
T
Plotting, in Figure 8, the MSEs for the non-zero and zero mean cases, for
T = 10, we can see that for the interval that the MSEs are positive the non-zero
20
case estimator has a higher MSE. Furthermore, for θ > 0.48 both MSEs become
negative! This of course is a peculiarity of the order of the approximation and
of the small number of observations, i.e. for higher order approximations and/or
bigger T this effect disappears.
3.2
Expansion of the Mean Coefficient MLE
For l = 2 we get
6
X
¡ (2) ¢2
(2) (2)
= −
ψij ei ej = σ 2 (1 + θ)2 ,
ω
(2)
(2)
(2)
β2 = β4 = β6 = 0
i,j=1
(2)
= −1,
(2)
= ψ112 e2 = 0,
γ2
δ 11
(2)
γ4 =
(2)
−6
(2)
(2)
(2)
γ 1 = γ 3 = γ 5 = 0,
2,
(θ + 1)
1
4
2i
i
(2)
, δ 14 = √
= −√
.
T (1 + θ)
T (θ + 1)2
2
,
(θ + 1)
(2)
δ 12
(2)
γ6 =
(2)
It follows that all the Edgeworth coefficients, aj
j = 1, ..., 10, are 0 and conse-
quently,
³√
³ 3´
´
³x´
P
+ O T −2 ,
T (e
µ − µ) < x = Φ
ω
i.e. the second order approximation is normal (see Tanaka 1984 for the first order
√
µ − µ).
approximation), as it is for the MM estimator T (b
3.3
Expansion of ML 1st order Autocorrelation
Let us define the MLE of ρ, e
ρ say, as
and
e
ρ=
³ ´
m e
θ =
e
θ
2
1+e
θ
e
θ
,
2 − ρ,
1+e
θ
where ρ is the true value of the parameter. Then we have that
¢
¢2
¡
¡
θ 3 − θ2
1 − θ2 − 4θ2
∂m (θ)
1 − θ2
∂ 2 m (θ)
∂ 3 m (θ)
=¡
= −2 ¡
= −6 ¡ 2
¢4
¢2 ,
¢3 ,
2
3
∂e
θ
1 + θ2
1 + θ2
θ +1
∂e
θ
∂e
θ
21
³ ´
Again, employing equation (30), in Appendix A2, with the derivatives of m e
θ
instead of the derivatives of m (b
ρ), we get
¢
¢
¢
¡
¡
¡
2 3
2 4
2 5
¡
¢
1
−
θ
1
−
θ
1
−
θ
2
δ2 = ¡
(19)
¢ , b1 = 12θ ¡
¢ , b2 = −6 49θ − 3 ¡
¢8 ,
2 4
2 6
1+θ
1+θ
1 + θ2
¢
¢
¡
¡
¢
¢
θ 3 − θ2 ¡
θ 3 − θ2 ¡
1 − θ2
2 4
2
b3 = −2 ¡
,
b
=
−2
=
−
(1
−
2θ)
1
−
θ
1
−
θ
,
b
¢7
¡
¢3
¡
¢2 ,
4
5
1 + θ2
1 + θ2
1 + θ2
¢2
¢2
¡
¡
¢
¢3
1 − θ2 − 4θ2 ¡
1 − θ2 − 4θ2 ¡
2 6
b6 = −6 ¡ 2
1 − θ , b7 = −6 ¡ 2
1 − θ2 ,
¢10
¢6
θ +1
θ +1
¡
¢
¢2
¡
2
¡
¡
¢ 1 − θ2
¢ 2 1 − θ2
2
2
b8 = −24 3 − θ θ ¡
¢5 , b9 = −2 2θ + 37θ − 6 ¡
¢4
1 + θ2
1 + θ2
√
where δ2 is the Asymptotic Variance of T (e
ρ − ρ). The second order Edgeworth
√
approximation of the distribution function of T (e
ρ − ρ) follows from equation
2 3
1−θ
)
(
(10) where this time ω 2 = δ 2 =
4 and
(1+θ2 )
¡
¢¡
¢
¡
¢2
1 − θ2
1
1 8θ + 79θ2 + 17 1 − θ2
,
c0 = − √ ¡
¢4
¢ , c1 =
¡ 2
4T δ 2
δ T 1 + θ2 2
θ +1
¡
¢
¢
1 θ 1 − θ2 ¡ 2
−
1
,
3θ
c2 = √ ¡
¢
3
δ T 1 + θ2
¢2
¡ 2
3θ − 1 θ2
1 103θ2 − 4θ + 8θ3 + 171θ4 + 12θ5 + 133θ6 + 1
1
c3 = −
, c5 =
¢2
¢ .
¡
¢¡
¡
¢¡
4T
2T 1 − θ2 θ2 + 1 2
1 − θ2 θ2 + 1
In Figure 9 we present the difference between the exact cumulative distribution
√
of T (e
ρ − ρ), based on 20000 simulations for T = 30 true θ = −0.3 and µ =
5, and the normal approximation (thick line) and between exact and the T −1
approximation in equation (6). It is apparent that the approximation is much
better than the asymptotic normal, although the normal is a better approximation
for the interval (0.75, 2.7).
√
The bias of T (e
ρ − ρ) is
³√
´
¢
¡ −3/2 ¢
1 (1 − θ)2 ¡ 2
T (e
ρ − ρ) = √ ¡
+
2θ
+
1
+
O
T
k1 = E
3θ
¢
T 1 + θ2 3
22
In terms of MSE, and keeping terms up to O (T −1 ), we get
¢3
¡
³√
´2
1 − θ2
E
T (e
ρ − ρ)
= ¡
¢4
1 + θ2
¢2 113θ2 − 10θ + 4θ3 + 79θ4 + 14θ5 + 295θ6 − 7
1¡
1 − θ2
+
.
¢6
¡ 2
T
θ +1
3.3.1
The Zero-Mean Case
For the case µ = 0 we have that δ 2 , and the bi coefficients are the same as in (19)
apart from b5 and b9 which are now given by:
¡
¢
¢2
¡
¡ 2
¢ 1 − θ20
θ0 1 − θ20
b5 = ¡
¢2 , and b9 = −2 37θ0 − 4 ¡
¢4 .
1 + θ20
1 + θ20
Consequently,
¡
¢2
³√
´
¡ −3/2 ¢
2 1 − θ20 θ0
k1 = E
T (ρe0 − ρ0 ) = √ ¡
+
O
T
(20)
¢
T 1 + θ20 3
³√
´
and recalling that in the case of non-zero mean, i.e. µ 6= 0, we have E
T (e
ρ − ρ) =
√
(1−θ)2 (2θ+3θ2 +1)
√1
,
we
can
plot
absolute
value
of
the
two
biases
(multiplied
by
T)
3
2
T
(θ +1)
against the common values of θ0 and θ (see Figure 10). It is obvious that ρe0 is
less biased than e
ρ for all θ ∈ (−1, 1). In fact for θ = 0 the bias of e
ρ is − T1 .
Furthermore, the 3rd and 4th order cumulants in this case, is equal to the one for
the non-zero µ case. Hence the non-normality of the approximations is affected
in the same way for both cases.
In terms of MSE, and keeping terms up to O (T −1 ), we get
¢
¡ 2
4
6
´2 ¡1 − θ2 ¢3
³√
¡
¢
+
39θ
+
152θ
−
1
54θ
2
2
0
0
0
0
1 − θ20
T (ρe0 − ρ0 ) = ¡
.
E
¢6
¢ +
¡ 2
2 4
T
1 + θ0
θ0 + 1
√
√
ρ − ρ) and T (ρe0 − ρ0 ) for T = 10, in Figure 11,
Plotting the MSE of T (e
we can observe that the second order approximate MSE of the estimator when
23
the mean is non zero is not higher from the zero mean MSE one over the whole
interval (−1, 1). Of course, for higher values of T both MSEs tend to the common
asymptotic variance.
4
Comparing the Estimators
Let us start our comparisons by considering first the biases of the two estimators
of θ and ρ. To facilitate the comparisons we consider the absolute values of the
√
biases of the estimators multiplied by T . It is apparent that when µ is estimated
there are areas of the admissible region of θ that the MM estimator of either θ or
ρ is less biased than the MLE ones (see Figure 12 and Figure 13). For example,
for −.3 ≤ θ ≤ 0, b
θ and b
ρ are less biased than e
θ and e
ρ, respectively. However, the
opposite is true for θ ≥ 0. When µ is known the bias of the MLEs is less from
the bias of the MM ones uniformly over the whole area of the admissible values
of θ (compare equation 12 with 18 for θ0 and 8 with 20 for ρ0 ).
In terms of second order approximate MSEs, we plot the ones of the two
estimators of θ in Figure 14 and the corresponding for the estimators of ρ in Figure
15. Notice that in both graphs we set T = 10 and in both cases µ is estimated. It is
apparent that there is not uniform superiority of neither the MLEs nor the MM
ones, over the whole range of the admissible values of θ. The same applies when
µ is not estimated (the graphs are not presented to conserve space). Of course for
a large data set i.e., high values of T , the MSE of either b
θ or e
θ approaches their
asymptotic variance and the MLE, e
θ, is, as expected, uniformly superior to the
MM one over the admissible interval (−1, 1) (see Kakizawa 1999b). The same
applies for the estimators of θ0 , e
ρ and ρe0 .
Hence, to conclude this section, we can say that asymptotically the MLEs of
either the MA parameter or the 1st order autocorrelation are more efficient than
24
the MM ones. However, for small samples and under the maintained assumption
that µ is known, the MSE of MM is smaller for the estimation of θ0 and ρ0 ,
provided that the true value of θ0 is close to zero. Furthermore, when µ 6= 0, if
the objective is less biased estimators, then for moderate negative values of θ, i.e.
θ ∈ (−0.4, 0), the MM estimation should be employed for the estimation of either
θ or ρ. For negative values of θ, i.e. θ ∈ (−1, −0.39), the ML method should be
utilized for the estimation of θ and the MM one for the estimation of ρ, whereas
for positive values of θ the ML method is preferred for both parameters. Finally,
in terms of MSE and for as small T as 10, the ML method is more efficient for
the estimation of both parameters only for the interval (−1.0, −0.6) ∪ (0.0, 1.0).
5
Conclusions
This paper contributes to asymptotic expansions of the MM and ML estimators
of the 1st order autocorrelation, the mean parameter and the MA parameter for
the MA(1) model. First, the second order expansions of the MM estimators are
derived and second, the first order expansions in Tanaka (1984) are extended to
include terms of order T −1 for the ML ones.
The results can be utilized to provide better approximation of the distributions
of the estimators, as compared with the asymptotic ones. In fact the results on
the bias of the estimators can be very useful for bias reductions along the lines of
Linton (1997) and Iglesias and Linton (2007), as well as to provide more efficient
estimators along the lines of Bao and Ullah (2007). As an example consider e
θ, the
h
MLE of θ. Then it is easy to prove that θe∗ = e
θ+ 1−2θ is second order unbiased and
T
has the same asymptotic variance as e
θ. Finally, as e
ρ and b
θ are Indirect Inference
estimators our results can be considered as applications of the results in Arvanitis
and Demos (2006) on the MA(1) model.
25
The analysis presented here can be extended to any ARMA(p, q) or ARMA(p, q|µ)
model. However, the algebra involved is becoming extremely tedious even for
small values of p and q. Another interesting issue could be the expansion of the
estimators as the parameter θ reaches the boundary of the admissible region, i.e.
when θ → ±1. In this respect the work of Andrews (1999) as applied in Iglesias
and Linton (2007) can be very useful. Furthermore, along the lines of Durbin
(1959) and Gourieroux et al. (1993), the properties of the MM estimators can be
improved by considering the expansions not only of the first order autocorrelation
but higher order ones. Finally, one could consider asymptotic expansions of the
estimators under the assumption of nonstandard, i.e. second moment infinite, or
non-normal error distributions. We leave these issues for future research.
Notes
1
Nagar (1959), Sargan (1974), Phillips (1977b) and Sargan and Satchell (1986), to quote
only a few papers. Rothenberg (1986) gives a review on the asymptotic techniques employed
in econometrics. For a book treatment of Edgeworth expansions, one may consult Hall (1992),
Barndorff-Nielsen and Cox (1989) and Taniguchi and Kakizawa (2000).
2
1
From now on we will refer to the up to T − 2 order expansion as first order one and for the
up to T −1 order as second order expansion.
3
In a general set-up for ARM A models, Ali (1984) presents the Edgeworth expansion of this
autocorrelation but in the zero-mean case and does not provide explicit formulae.
4
Notice that e
ρ is the Indirect estimators of ρ, when the true model is an AR(1) and the
auxiliary is an M A(1) where the parameter θ is estimated by M M , or by M L in the Constraint
Indirect estimation setup (see Canzelori, Fiorentini and Sentana 2004). On the other hand, b
θ is
an Indirect estimator of θ when the true model is an M A(1) and the auxiliary is an AR(1) one
(see Gourieroux, Monfort and Renault 1993).
26
5
The validity of the expansions of either the M LEs or the M M Es is justified by our main-
tained assumptions in equation (1) (see Magdalinos 1992, Phillips 1977, Sargan 1974 and 1976,
and Sargan and Satchell 1986).
6
Throughout all simulations we set σ2 = 1 and we do not estimate this parameter. This does
not affect the expansions of the parameters (see Tanaka 1984).
27
References
ALI, M.M. (1984) Distributions of the sample autocorrelations when observations are from a stationary autoregressive-moving-average process. Journal of
Business and Economic Statistics 2, 271-278.
ANDREWS, D.W.K. (1999) Estimation when a parameter is on a boundary.
Econometrica 67, 1341-1383.
ARVANITIS, S. and DEMOS, A. (2006) Bias Properties of Three Indirect
Inference Estimators, mimeo Athens University of Economics and Business.
BARNDORFF-NIELSEN, O.E. and COX, D.R. (1989) Asymptotic Techniques
for Use in Statistics. Chapman and Hall.
BAO, Y. and ULLAH, A. (2007) The second-order bias and mean squared
error of estimators in time-series models. Journal of Econometrics 140, 650-669.
CANZELORI, G., FIORENTINI, G. and SENTANA, E. (2004) Constrained
Indirect Estimation. Review of Economic Studies 71, 945-973.
DAVIS, R. and RESNICK, S. (1986) Limit theory for the sample covariance
and correlation functions of moving averages. Annals of Statistics 14, 533-558.
DURBIN, J. (1959) Efficient estimation of parameters in Moving-Average
models. Biometrika 46, 306-316.
DURBIN, J. (1980) Approximations for densities of sufficient estimators. Biometrika
67, 311-333.
FULLER, W. (1976) Introduction to Statistical Time Series. Wiley.
GOURIEROUX, C., MONFORT, A. and RENAULT, E. (1993) Indirect Inference. Journal of Applied Econometrics 8, S85-S118.
HALL, P (1992) The bootstrap and edgeworth expansion. Springer.
IGLESIAS, E.M. and LINTON, O.B. (2007) Higher order asymptotic theory
when a parameter is on a boundary with an application to GARCH models.
Econometric Theory forthcoming
28
KAKIZAWA, Y. (1999a) Valid Edgeworth expansions of some estimators and
bootstrap confidence intervals in first-order autoregression. Journal of Time Series Analysis 20, 343-359.
KAKIZAWA, Y. (1999b) Note on the asymptotic efficiency of sample covariances in Gaussian vector stationary processes. Journal of Time Series Analysis
20, 551-558.
KENDAL, M. G. and STUART, A. (1969) The Advanced Theory of Statistics.
C. Griffin & Comapny LTD London.
LINTON, O. (1997) An asymptotic expansion in the Garch(1,1) model. Econometric Theory 13, 558-581.
MAGDALINOS, M. A. (1992) Stochastic expansions and asymptotic approximations, Econometric Theory 8, 343-367.
PHILLIPS, P.C.B. (1977a) Approximations to some finite sample distributions
associated with a first-order stochastic difference equation. Econometrica 45, 463485.
PHILLIPS, P.C.B. (1977b) A general theorem in the theory of asymptotic
expansions as approximations to the finite sample distributions of econometric
estimators. Econometrica 45, 1517-1534.
PHILLIPS, P.C.B. (1980) Finite sample theory and the distribution of alternative estimators of the marginal propensity to consume. Review of Economic
Studies 47, 183-224.
ROTHENBERG, T.J. (1986) Approximating the distributions of econometric
estimators and test statistics, In The Handbook of Econometrics. vol. II, Amsterdam: North-Holland.
SARGAN, J.D. (1974) Validity of Nagar’s expansion. Econometrica 42, 169176
SARGAN, J.D. (1976) Econometric estimators and the Edgeworth approxi-
29
mation. Econometrica 44, 421-448.
SARGAN, J.D. (1977) Erratum ”Econometric estimators and the Edgeworth
approximation”. Econometrica 45, 272.
SARGAN, J.D. and SATCHELL, S.E. (1986) A theorem of validity for Edgeworth expansions. Econometrica 54, 189-213.
TANAKA, K. (1983) Asymptotic expansions associated with the AR(1) model
with unknown mean. Econometrica 51, 1221-1232.
TANAKA, K. (1984) An asymptotic expansion associated with the maximum
likelihood estimators in ARMA models. Journal of Royal Statistical Society B 46,
58-67.
TANIGUCHI, M. and KAKIZAWA, Y. (2000) Asymptotic theory of statistical
inference fo time series. Springer Series in Statistics
30
Appendix A1
The derivatives of the cumulant generating function evaluated at zero are the
cumulants of q i ; all first derivatives are zero, since q is standardized about its
mean. For a, b, c = 1, 2 and p, q, r = 3, 4, the second order are given by (see
Phillips 1980, p.209-210)
2
4
ψab = − tr (Ca ΣCb Σ) − m0 Ca ΣCb m,
T
T
1 0
ψab = − db Σda ,
T
2
ψ ap = − d0p ΣCa m,
T
the third ones are
ψabc = −
ψabp = −
8i
3
2
T
8i
T
3
2
tr (Cc ΣCb ΣCa Σ) −
m0 Ca ΣCb Σdp ,
24i
T
3
2
m0 Ca ΣCb ΣCc m,
ψ apq = −
2i
T
3
2
d0p ΣCa Σdq ,
ψ pqr = 0,
whereas the fourth order cumulants are
48
192
tr (Cd ΣCc ΣCb ΣCa Σ) + 2 m0 Cd ΣCc ΣCb ΣCa m,
2
T
T
48 0
8
=
m Cc ΣCb ΣCa Σdp , ψ abpq = 2 d0p ΣCb ΣCa Σdq ,
2
T
T
= 0, and ψ abcd = 0.
ψabcd =
ψabcp
ψapqr
Hence we need to evaluate the following traces: tr (C1 Σ), tr (C1 Σ)2 , tr (C2 Σ),
tr (C2 Σ)2 , tr (C1 ΣC2 Σ) etc., as well as the quadratics d03 Σd3 , m0 C1 ΣC1 m, etc.
Here we present only tr (C1 Σ)2 and m0 C1 ΣC1 m, as the rest follow by the same
method.
2
tr (C1 Σ)
σ4T
∼
2π
Z
π
−π
π
¯
¯4
(cos λ)2 ¯1 + θeiλ ¯ dλ
µ
¶2
¢2
eiλ + e−iλ ¡
1 + θeiλ + θe−iλ + θ2 dλ
2
−π
µ
¶2
¶2 µ
Z
1
1
1
2
4 T
= σ
z+
1 + θz + θ + θ
dz
8π |z|<1
z
z
iz
2
³
¡ 2
¢2 ´
d4 (z 2 + 1) (1 + θz)2 (θ + z)2
2
4T 1
4T
lim
3θ + θ + 1
=σ
= σ
4 24 z→0
dz 4
2
4
σ T
=
2π
Z
31
Furthermore,
T 2
µ
tr (m C1 ΣC1 m) = tr (mm C1 ΣC1 ) ∼ σ
2π
0
0
where g (λ) = 1 + 2
∞
X
2
Z
π
−π
¯
¯2
(cos λ)2 ¯1 + θeiλ ¯ g (λ) dλ
cos kλ. Hence
k=1
Z
2
(z 2 + 1) (1 + θz) (θ + z)
dz
z4
|z|<1
¢¡
¢
2¡
∞ Z
X
(z 2 + 1) z + θz 2 + θ + θ2 z z 2k + 1
2 T
2
+σ
µ
dz
8πi k=1 |z|<1
z k+4
#
"
2
2
T
+
1)
(1
+
θz)
(θ
+
z)
(z
= σ 2 µ2 Re s
4
z4
"
¢ ¡ 2k
¢#
2¡
∞
2
2
2
X
+
1)
+
θ
+
θ
z
z
+
1
z
+
θz
(z
T
Re s
+σ 2 µ2
4 k=1
z k+4
T 2
µ
tr (m C1 ΣC1 m) = σ
8πi
0
2
2
T 2
d3 (z 2 + 1) (1 + θz) (θ + z)
= σ
µ lim
24 z→0
dz 3
¢¡
¢
2¡
∞
X
dk+3 (z 2 + 1) z + θz 2 + θ + θ2 z z 2k + 1
1
2T 2
lim
+σ µ
4 k=1 (k + 3)! z→0
dz k+3
2
= µ2 σ 2 T (θ + 1)2
as limz→0
1
(k+3)!
d3
dz 3
limz→0
¡
¢
2
(z 2 + 1) (1 + θz) (θ + z) = 12 1 + θ2 , and
2
¡
¢
dk+3 (z 2 +1) (z+θz 2 +θ+θ2 z )(z 2k +1)
2
=
2
4θ
+
θ
+
1
.
k+3
dz
Consequently, we get
h
¡
¢2 i
− 4µ2 (1 + θ)2 σ 2 ,
ψ11 = −σ 4 3θ2 + θ2 + 1
¡
¢
ψ12 = −4θ 1 + θ2 σ 4 − 4µ2 (1 + θ)2 σ 2 ,
¡
¢
ψ22 = −2σ 4 θ4 + 1 + 4θ2 − 4µ2 (1 + θ)2 σ 2 ,
ψ13 = ψ14 = ψ23 = ψ24 = −2µ (1 + θ)2 σ 2 ,
ψ33 = ψ34 = ψ44 = − (1 + θ)2 σ 2 ,
32
(21)
the third ones are:
and
¡
£
¢
¤
2i
ψ111 = − √ σ 4 σ 2 θ 9 + 28θ2 + 9θ4 + 12µ2 (θ + 1)4
T
¢
¤
4i 4 £ 2 ¡ 2
ψ121 = − √ σ σ 12θ + 12θ4 + θ6 + 1 + 6µ2 (θ + 1)4 ,
T
¢
¤
24i 4 £ 2 ¡
ψ122 = − √ σ σ θ 1 + 3θ2 + θ4 + µ2 (θ + 1)4 ,
T
¢
£ ¡
¤
8i
ψ222 = − √ σ 4 σ 2 9θ2 + 9θ4 + θ6 + 1 + 3µ2 (θ + 1)4 .
T
(22)
8i
8i
ψ113 = ψ114 = − √ σ 4 µ (θ + 1)4 ψ123 = ψ124 = − √ σ 4 µ (θ + 1)4 ,
T
T
¢
8i 4 ¡
ψ223 = ψ224 = − √ σ µ 4θ + 6θ2 + 4θ3 + θ4 + 1 ,
T
2i
ψ314 = ψ413 = − √ σ 4 (θ + 1)4 ,
T
¢
¡
2i
ψ234 = ψ233 = ψ244 = − √ σ 4 4θ + 6θ2 + 4θ3 + θ4 + 1 .
T
Finally, the fourth order cumulants are:
ψ1111 =
ψ1112 =
ψ1122 =
ψ1222 =
ψ2222 =
¢ 192 2 6
6 8¡ 2
σ 72θ + 173θ4 + 72θ6 + 3θ8 + 3 +
µ σ (θ + 1)6
T
T
¢ 192 2 6
48 8 ¡
σ 3θ + 19θ3 + 19θ5 + 3θ7 +
µ σ (θ + 1)6
T
T
¢ 192 2 6
24 8 ¡ 2
σ 22θ + 52θ4 + 22θ6 + θ8 + 1 +
µ σ (θ + 1)6
T
T
¢¡
¢ 192 2 6
192 8 ¡ 2
σ θ θ + 1 5θ2 + θ4 + 1 +
µ σ (θ + 1)6
T
T
¢ 192 2 6
48 8 ¡ 2
σ 16θ + 36θ4 + 16θ6 + θ8 + 1 +
µ σ (θ + 1)6
T
T
ψ1113 = ψ1114 =
48 6
µσ (θ + 1)6 ,
T
ψ1123 = ψ1124 = ψ1223 = ψ1224 = ψ2223 = ψ2224 =
8 6
σ (θ + 1)6 f or a, b = 1, 2 and p, q = 3, 4
T
= 0, and ψ abcd = 0 for a, b, c = 1, 2 and p, q, r = 3, 4.
ψabpq =
ψapqr
48 6
µσ (θ + 1)6
T
33
(23)
Furthermore, the derivatives of e (q) evaluated at 0 are:
e1 =
e4 =
e22 =
e33 =
e11 =
1
−θ
−µ (1 − θ)2
,
e
=
,
e
=
(24)
¡
¢2
¡
¢2 ,
2
3
(1 + θ2 )σ 2
1 + θ2 σ2
1 + θ2 σ 2
−µ
1
2µ
,
e
=
−
,
e
=
,
12
13
2
2
(1 + θ )σ 2
(1 + θ )2 σ 4
(1 + θ2 )2 σ 4
¢¢
¡
¡
µ −4θ + 1 + θ2
2θ
µ
, e23 =
, e24 =
,
¡
¢
¡
¢
2 3 4
2 3 4
(1 + θ2 )2 σ 4
1+θ σ
1+θ σ
¡
¢
¢
¡
θσ 2 1 + θ2 − 2µ2 (1 − θ)2
− 1 + θ2 σ 2 − 2µ2
2
, e34 =
,
¡
¢3
¡
¢2
1 + θ2 σ 4
1 + θ2 σ 4
e14 = e44 = 0,
and
(25)
e111 = e112 = e113 = e114 = e124 = e134 = e144 = e244 = 0,
¢
¡ 2
2 2
2
σ + 4µ + θ σ
2
µ
,
e122 = ¡ 2
¢3 , e123 = −4 ¡ 2
¢3 , e133 = 2 ¡ 2
¢3
θ + 1 σ6
θ + 1 σ6
θ + 1 σ6
¢
¡
−2 θ2 − 6θ + 1 µ
θ
−2µ
, e224 = ¡ 2
e222 = −6 ¡ 2
¢4 , e223 =
¢4
¢3 ,
¡ 2
θ + 1 σ6
θ + 1 σ6
θ + 1 σ6
¢
¢
¡
¡ 2
4 2µ2 − θσ 2 − 6θµ2 − θ3 σ 2 + 2θ2 µ2
σ + 4µ2 + θ2 σ 2
e332 =
, e432 =
¢4
¢3
¡ 2
¡ 2
θ + 1 σ6
θ + 1 σ6
where ei =
∂e(0)
,
∂qi
eij =
∂ 2 e(0)
,
∂qi ∂qj
eijk =
∂ 3 e(0)
∂qi ∂qj ∂qk
for i, j, k = 1, ..., 4.
The expansion coefficients are:
c0 =
c1 =
c2 =
c3 =
c5 =
a4
a3
ia
√ + 13 +
√ ,
(26)
6ω
2ω T
2ω3 T
µ
µ
¶
¶
a7
a23
ia5
1
a24
a9
15 a21
ia1 a3
3
√ +
− 2
+
+
+ 6
− √ −
+ 4 ζ,
ω
2T
8T
4T
ω
72 12 T
8T
ω
2 T
¶
µ
1 ia1
a3
,
− 3
+ √
ω
6
2 T
µ
¶
a23
1
10 a21
ia1 a3
− 4ζ − 6
− √ −
,
ω
ω
72 12 T
8T
µ
¶
ia1 a3
a2
ia10
a3 a4
a8
1 a21
a23
ia4 a1
a6
− √ −
, ζ=
− √ − √ −
−
−
6
ω
72 12 T
8T
24 2 T
6T
4T
2T
12 T
34
and the adapting the tensor summation convention, the so called Edgeworth coefficients are defined by:
a1 = ψijk ei ej ek ,
a4 = ψab eab ,
a2 = ψijkl ei ej ek el ,
a5 = δ ab eab ,
a8 = γ a eab ψbc ecd γ d ,
a3 = γ a eab γ b ,
a6 = eabc γ a γ b γ c ,
a9 = ψad eab ψbc ecd ,
(27)
a7 = eabc ψab γ c ,
a10 = γ a eab β b ,
where also we have:
ω2 = −ψij ei ej ,
β a = ψaij ei ej ,
γ a = ψai ei ,
δab = ψabi ei .
(28)
Appendix A2
To write the explicit expression of b
θ we solve this equation for θ and we choose
the solution with the constraint |θ| ≤ 1 (see Fuller, 1976):
p
1 − 4b
ρ2
1
−
b
= f (b
ρ) , if |b
ρ| ≤ 0.5
θ =
2b
ρ
or p
p
2
1 − 4ρ2
1
−
1
−
4b
ρ
1
−
b
−
= f (b
ρ) − f (ρ) = m (ρ) ,
θ−θ =
2b
ρ
2ρ
where θ is the true parameter value.
Notice that for − 12 ≤ ρ ≤
1
2
the function is monotonic and one-to-one. Notice
also that
q
(1 − 4ρ2 )3 − 1 + 6ρ2
∂ f (ρ)
q
=
,
3
∂b
ρ2
3
2
ρ (1 − 4ρ )
p
∂f (ρ)
1 1 − 1 − 4ρ2
p
≥ 0,
=
∂b
ρ
2 ρ2 1 − 4ρ2
2
5
(1 − 4ρ2 ) (1 − 2ρ2 ) − 4ρ2 (1 − 6ρ2 ) − (1 − 4ρ2 ) 2
∂ 3 f (ρ)
=
3
.
5
∂b
ρ3
(1 − 4ρ2 ) 2
We employ the following notation:
´ √
√ ³
√
θ= T b
θ − θ = T m (b
ρ) = T
Ã
35
1−
!
p
p
1 − 4b
ρ2 1 − 1 − 4ρ2
−
2b
ρ
2ρ
cfθ (s) =
Z
¡ ¢
exp isθ dF (ρ) ,
which stands for the characteristic function of θ.
Taking a Taylor series expansion about the correlation coefficient to get:
³ 3´
1 ∂ 2 m (ρ) 2
∂m (ρ)
1 ∂ 3 m (ρ) 3
y+ √
θ=
y +
y + O T −2
2
3
∂b
ρ
6T ∂b
ρ
ρ
2 T ∂b
√
where y = T (b
ρ − ρ0 ).
In the characteristic function of θ, taking the Taylor expansion of exp
³
´
is ∂ 3 m(ρ0 ) 3
and exp 6T
and setting s ∂m(ρ)
y
= z we get
∂e
ρ
∂e
ρ3
³
is ∂ 2 m(ρ0 ) 2
√
y
∂e
ρ2
2 T
is ∂ 2 m (ρ) d2 cfy (z)
s ∂ 3 m (ρ) d3 cfy (z)
cfθ (s) = cfy (z) − √
−
dz 2
6T ∂b
dz 3
ρ2
ρ3
2 T ∂b
¶2
µ
³ 3´
s2 ∂ 2 m (ρ) d4 cfy (z)
−
+ O T −2 .
8T
dz 4
∂b
ρ2
´
(29)
The last equation is the characteristic function of θ as a series involving the
√
characteristic function of y = T (b
ρ − ρ) and its subsequent derivatives, with
coefficients depending on the derivatives of m (b
ρ) evaluated at the the true ρ.
Denoting by kr the rth degree cumulant of y, expanding, aroung z = 0,
³
³
´
´
³ 4´
2
3
z
exp −k2& z2 , exp (k1 iz), exp −k3 iz6 and exp k4 24
we get
¶µ
¶
µ
³ 3´
2
2
6
iz 3
z 4 k12 z 2
z4
2z
&z
2z
cfy (z) = exp −ω
1 − k2
− k3
− k3 + k1 iz + k1 k3 −
+ k4
+O T − 2 .
2
2
6
72
6
2
24
Taking equation (29), substituting out the derivatives (evaluated by the above
expression) and z = s ∂m(ρ)
and collecting terms we get:
∂e
ρ
⎛
is
√
2 T
3
is
√
6 T
1+
[b4 + 2b5 ] −
[b1 + 3b3 ]
⎜
£
¤
2
µ 2 2¶⎜
− s2 T1 b9 + b25 + 34 b24 + b7 + 3b4 b5 + b8
sδ ⎜
⎜
cfθ (s) = exp −
⎜
s4
2
⎜ + 24T
[b2 + 4b1 b5 + 4b6 + 12b3 b5 + 14b4 b1 + 18b3 b4 ]
⎝
s6
− 72T
[6b3 b1 + b21 + 9b23 ]
36
⎞
⎟
⎟
³
´
⎟
⎟+O T − 32
⎟
⎟
⎠
where
δ
2
=
b3 =
b6 =
b8 =
µ
µ
¶2
¶3
¶4
√
∂m (ρ)
∂m (ρ)
∂m (ρ)
ω
, b1 = T k3
, b2 = T k4
, (30)
∂b
ρ
∂b
ρ
∂b
ρ
µ
¶2
√
∂m (ρ)
∂ 2 m (ρ) 4 ∂m (ρ)
∂ 2 m (ρ) 2
ω
,
b
=
ω
,
b
=
T k1
,
4
5
2
2
∂b
ρ
∂b
ρ
∂b
ρ
∂b
ρ
µ
¶3
∂ 3 m (ρ) 6 ∂m (ρ)
∂ 3 m (ρ) 4 ∂m (ρ)
,
4
ω
,
b
=
ω
7
∂b
ρ
∂b
ρ
∂b
ρ3
∂b
ρ3
µ
¶2
∂m (ρ)
∂m (ρ)
∂ 2 m (ρ) √
&
T k3
.
, b9 = T k2
∂b
ρ
∂b
ρ
∂b
ρ2
2
µ
Inverting the characteristic function of θ term by term, we deduce the corresponding asymptotic expansion of the density g(m) and the probability function
h√ ³
´
i
G(m) = Pr T b
θ − θ ≤ m as T → ∞. To do so, we use the next relations:
Z ∞
1
2
(it)n e−t /2 e−itz dt ⇒
(−1) φ (z) =
2π −∞
Z ∞
1
2
(it)n e−t /2 e−itz dt
Hn (z)φ(z) =
2π −∞
n (n)
where φ(z) denotes the standard normal density function, and Hn (z) are the
Hermite polynomials, for which we have:
H0 (z) = 1,
H1 (z) = z,
H2 (z) = z 2 − 1,
H3 (z) = z 3 − 3z,
etc.
and for z = δs we have:
Z
³m´ ³m´
1 ³x´
b4 + 2b5 1 ³ m ´ b1 + 3b3 1
G(m) =
φ
dx − √
φ
− √
φ
H2
δ
δ
δ
δ
2 T δ
6 T δ3
−∞ δ
3 2
³
´
³
´
2
b9 + b5 + 4 b4 + b7 + 3b4 b5 + b8 1
m
m
−
φ
2 H1
2T
δ
δ
δ
³m´ ³m´
b2 + 4b1 b5 + 4b6 + 12b3 b5 + 14b4 b1 + 18b3 b4 1
−
H3
φ
24T
δ
δ
δ4
³m´ ³m´
¡ −3/2 ¢
6b1 b3 + b21 + 9b23 1
φ
+O T
H5
.
−
72T
δ
δ
δ6
m
37
and emplying the formulae of Hermite polynomials we get:
⎡
b4 +2b
√ 5 1 − b1 +3b
√ 3 1
2 T δ
6 T δ3
⎢
⎢
b +b2 + 3 b2 +b +3b b +b
+ 9 5 4 4 2T7 4 5 8 δ12 mδ
⎢
⎢
ς m
³m´
³m´ ⎢
⎢
+ 24T
δ
⎢
G(m) = Φ
−φ
¡ ¢2
2 +9b2
⎢
6b
b
+b
1
3
b1 +3b
m
1
1
3
δ
δ ⎢ +15
√ 3 13 m
+
6
δ
δ δ
δ
6 T ´
⎢
³ 72T
¡ m ¢3
⎢
6b1 b3 +b21 +9b23 1
1 ς
⎢ − 3 24T + 10
72T
δ
δ6
⎣
¡
2
2
6b1 b3 +b1 +9b3 1 m ¢5
+
72T
δ6 δ
¡ −3/2 ¢
+O T
,
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
where
b2 + 4b1 b5 + 4b6 + 12b3 b5 + 14b4 b1 + 18b3 b4
.
δ4
The last equation stands for the Edgeworth approximation of the distribution
´
√ ³
b
function of T θ − θ , written conpactly in subsection 2.3.
ς = −3
Appendix B1
The derivatives of the log-likelihood function, w.r.t. θ, are:
Ã
µ
¶2 !
T
T
∂ (θ)
∂ 2 ut
1 X
∂ut
1 X ∂ut
∂ 2 (θ)
ut 2 +
ut
=− 2
= − 2
,
∂θ
σ t=1 ∂θ
σ t=1
∂θ
∂θ2
∂θ
¶
µ
¶
T µ
∂ 3 ut
∂ 3 (θ)
1 X
∂ut ∂ 2 ut
ut 3 + 3
= − 2
σ t=1
∂θ ∂θ2
∂θ3
∂θ
Ã
!
µ 2 ¶2
T
4
4
3
X
∂ ut
∂ (θ)
1
∂ ut
∂ut ∂ ut
ut 4 + 3
.
=− 2
+4
4
2
σ t=1
∂θ ∂θ3
∂θ
∂θ
∂θ
Noting that
X
∂ut
=−
(−θ)i ut−1−i ,
∂θ
i=0
∞
and
X
∂ 2 ut
(i + 1) (−θ)i ut−2−i ,
2 = 2
∂θ
i=0
∞
X
∂ 3 ut
=
−3
(i + 1) (i + 2) (−θ)i ut−3−i
3
∂θ
i=0
∞
38
X
∂ 4 ut
=
4
(i + 1) (i + 2) (i + 3) (−θ)i ut−4−i ,
4
∂θ
i=0
∞
it follows that
¶2
µ 2 ¶2
µ
2
σ2
∂ ut
∂ut
2 1+θ
=
,
E
=
4σ
E
¡
¢3 ,
∂θ
1 − θ2
∂θ2
1 − θ2
¶
¶
µ 3
µ 2
2θσ 2
6θ2 σ 2
∂ ut ∂ut
∂ ut ∂ut
= ¡
=¡
E
¢2 and E
¢3 .
∂θ2 ∂θ
∂θ3 ∂θ
1 − θ2
1 − θ2
Hence the expectation of the derivatives of the log-likelihood function evalu-
ated at the true θ
µ
¶
¶
µ 2
∂ (θ)
T
∂ L (θ)
E
=−
= 0, E
2
∂θ
∂θ
1 − θ2
¶
¶
µ 3
µ 4
Tθ
1 + 3θ2
∂ (θ)
∂ (θ)
= −6 ¡
= −12T ¡
E
¢2 , E
¢3
∂θ3
∂θ4
1 − θ2
1 − θ2
The derivatives of the log-likelihood function with respect to the parameter µ:
"µ
#
¶2
T
T
2
2
X
X
1
∂
1
∂ut
∂ ut
∂ut
∂
= − 2
,
ut
=− 2
+ ut 2 ,
2
∂µ
σ t=1 ∂µ
∂µ
σ t=1
∂µ
∂µ
" µ
#
∙
¸
¶2
T
T
∂3
∂ 3 ut
∂ 4 ut
∂ 2 ut
∂4
1 X ∂ut ∂ 2 ut
1 X
∂ut ∂ 3 ut
3
3
= − 2
+ ut 3 ,
=− 2
+4
+ ut 4
∂µ3
σ t=1
∂µ ∂µ2
∂µ
∂µ4
σ t=1
∂µ2
∂µ ∂µ3
∂µ
where
∂ut
1
=−
,
∂µ
1+θ
∂ 2 ut
∂ 3 ut
∂ 4 ut
=
=
= 0.
∂µ2
∂µ3
∂µ4
Hence their expectations at the true θ are:
¶
¶
µ 4
¶
¶
µ 3
µ
µ 2
T
∂ (θ)
∂ (θ)
∂ (θ)
∂ (θ)
=−
=E
= 0.
=E
, E
E
∂µ2
∂µ
∂µ3
∂µ4
(1 + θ)2 σ 2
Furthermore,
à T
!
T
1 X ∂ut ∂ut X ∂ 2 ut
∂2
ut
= − 2
+
,
∂µ∂θ
σ
∂µ ∂θ
∂µ∂θ
t=1
t=1
¶
µ 2 ¶
µ
1
∂
∂ut
= −
, E
= 0.
E
∂µ
1+θ
∂µ∂θ
and
Furthermore,
∂ 2 ut
1
,
=
∂µ∂θ
(1 + θ)2
∂ 3 ut
2
,
2 = −
∂µ∂θ
(1 + θ)3
39
∂ 3 ut
= 0.
∂θ∂µ2
We also have
¶
T µ
∂3
1 X ∂ 2 ut ∂ut
∂ut ∂ 2 ut
∂ 3 ut
= − 2
+2
+ ut 2
,
∂µ2 ∂θ
σ t=1 ∂µ2 ∂θ
∂µ ∂µ∂θ
∂µ ∂θ
2T
1
∂3
=
2
2
∂µ ∂θ
σ (1 + θ)3
where
So, its expected value is
E
µ
∂3
∂µ2 ∂θ
¶
=
2T
,
(1 + θ)3 σ 2
as we have
E
µ
∂ 2 ut ∂ut
∂µ2 ∂θ
¶
= 0 and E
µ
∂ut ∂ 2 ut
∂µ ∂µ∂θ
¶
=−
1
(1 + θ)3
Next
¶
T µ
∂ 3 ut
∂3
∂ 2 ut ∂ut ∂ut ∂ 2 ut
1 X
2
,
+
= − 2
+ ut
σ t=1
∂µ∂θ ∂θ
∂µ ∂θ2
∂µ∂θ2
∂µ∂θ2
µ 3 ¶
∂
= 0
E
∂µ∂θ2
Moreover,
¶
T µ
∂ 4 ut
∂4
∂ 3 ut ∂ut
1 X
∂ 2 ut ∂ 2 ut ∂ut ∂ 3 ut
3
,
= − 2
+
+ ut
+3
σ t=1
∂µ∂θ ∂θ2
∂µ ∂θ3
∂µ∂θ3
∂µ∂θ2 ∂θ
∂µ∂θ3
µ 4 ¶
∂ 4 ut
1
∂
=
0
as
,
E
3
3 = 6
∂µ∂θ
∂µ∂θ
(1 + θ)4
Ã
!
µ 2 ¶2
T
∂ 3 ut ∂ut ∂ 2 ut ∂ 2 ut
1 X
∂ ut
∂ut ∂ 3 ut
∂4
∂ 4 ut
2 2
= − 2
+2
+2
+ ut 2 2 ,
+
σ t=1
∂µ ∂θ ∂θ
∂µ2 ∂θ2
∂µ∂θ
∂µ ∂µ∂θ2
∂µ2 ∂θ2
∂µ ∂θ
µ 4 ¶
T
∂ 4 ut
∂
=
−6
as
=0
E
∂µ2 ∂θ2
∂µ2 ∂θ2
(1 + θ)4 σ 2
and
¶
T µ
∂4
∂ 4 ut
1 X ∂ 3 ut ∂ut
∂ 2 ut ∂ 2 ut
∂ut ∂ 3 ut
= − 2
+3 2
+3
+ ut 3
,
∂µ3 ∂θ
σ t=1 ∂µ3 ∂θ
∂µ ∂µ∂θ
∂µ ∂µ2 ∂θ
∂µ ∂θ
µ 4 ¶
∂ 4 ut
∂
=
0
as
= 0.
E
∂µ3 ∂θ
∂µ3 ∂θ
40
Appendix B2
The derivatives can be found by solving the system of equations (15), (16) and
(17), which is coming from implicit differentiation of fj (ϕ (h) , h) in equation (13),
i.e. for j = 1, 2 we have:
u (ϕ (h) , h) =
2
X
∂fj (ϕ (h) , h) ∂θk (h)
∂ha
∂θk
k=1
+
∂fj (ϕ (h) , h)
= 0.
∂ha
(k)
= ea and
Evaluating the above expression at 0, and noting that ∂θ∂hk (0)
a
¯
w ¯
Ajk + √jkT ¯
= Ajk we get the result in equation (15).
h=wjk =0
∂fj (0,0)
∂h
θk
=
By implicit differentiation of u (ϕ (h) , h) gives, again for j = 1, 2
v (ϕ (h) , h) =
2
X
∂u (ϕ (h) , h) ∂θl (h)
l=1
Substituting out
∂u(ϕ(h),h)
∂h
θl
and
∂hb
∂θl
∂u(ϕ(h),h)
∂hb
+
∂u (ϕ (h) , h)
= 0.
∂hb
(31)
by
∂u (ϕ (h) , h) X ∂ 2 fj (ϕ (h) , h) ∂θk (h) ∂ 2 fj (ϕ (h) , h)
=
+
∂ha
∂θl ∂θk
∂e
θl
∂e
θl ∂ha
2
k=1
and
¶ 2
2 µ
∂u (ϕ (h) , h) X ∂ 2 fj (ϕ (h) , h) ∂θk (h) ∂fj (ϕ (h) , h) ∂ 2 θk (h)
∂ fj (ϕ (h) , h)
+
=
+
∂hb
∂ha
∂hb ∂ha
∂hb ∂ha
∂hb ∂θk
∂θk
k=1
collecting terms and evaluating at 0 we get
à 2
!
2
2
2
2
X
X
X
∂ fj (0, 0) (k)
1
∂ fj (0, 0) (k)
√ Kjkl e(l)
ea
eb +
0 =
b +
e
T
∂hb ∂e
θk
k=1 ∂ θ k ∂ha
k=1
l=1
2
1 X
∂ 2 fj (0, 0)
(k)
+√
Ajk eab +
.
∂hb ∂ha
T k=1
As now fj (ϕ, h) is linear in h we have that
fj2 (ϕ,h)
∂hb ∂ha
a, b = 1, ..., 4. and we get the result in equation (16).
41
= 0 for all j = 1, 2 and
Finally, for the third derivatives, implicitly differentiating, with respect to hc ,
for c = 1, 2, ..., 6, of v (ϕ (h) , h) gives
2
X
∂v (ϕ (h) , h) ∂θp (h)
p=1
∂θp
∂hc
+
∂v (ϕ (h) , h)
= 0.
∂hc
aand consequently,
2
2
X
∂ 3 fj (ϕ (h) , h) ∂θk (h) X ∂fj2 (ϕ (h) , h) ∂ 2 θk (h)
∂v (ϕ (h) , h)
=
+
∂hb
∂hb ∂ha
∂θp
∂θp ∂θk ∂ha
∂θp ∂θk
k=1
k=1
Ã
!
2
2
X
X
∂ 3 fj (ϕ (h) , h) ∂θl (h) ∂ 3 fj (ϕ (h) , h) ∂θk (h)
+
+
∂hb
∂ha
∂θp θl ∂θk
∂θp ∂hb ∂θk
k=1
l=1
and
¶
2 µ 3
X
∂v (ϕ (h) , h)
∂ fj (ϕ (h) , h) ∂θk (h) ∂ 2 fj (ϕ (h) , h) ∂ 2 θk (h)
=
+
∂hc
∂h
∂hc ∂hb
∂h
∂θ
∂h
∂θk ∂ha
b
c
k
a
k=1
¶
2 µ 2
X
∂ fj (ϕ (h) , h) ∂ 2 θk (h) ∂fj (ϕ (h) , h) ∂ 3 θk (h)
+
+
∂h
∂hc ∂hb ∂ha
∂h
∂θ
∂θk
b ∂ha
c
k
k=1
Ã
!
⎡
2 ³
´
X
3
2
3
2
∂ fj (ϕ(h),h) ∂θl (h)
∂ fj (ϕ(h),h) ∂ θl (h)
∂ f (ϕ(h),h)
∂θk (h)
+ ∂hj ∂h ∂θ
+ ∂θ
2 ⎢
∂h
∂h
∂h
∂ha
∂h
∂θ
∂θ
∂θ
c
b
b
c
c
l
k
l
k
b
k
X
⎢ l=1
⎢
!
à 2
+
⎢
X ∂ 2 f (ϕ(h),h) ∂θ (h) ∂ 2 f (ϕ(h),h) ∂ 2 θ (h)
j
j
l
k
k=1 ⎣
+
+ ∂h
∂hb
∂hc ∂ha
∂θ ∂θ
∂θ
l
k
b
k
l=1
Substituting in equation (31) the above expressions of
∂v(ϕ(h),h)
∂θp
and
∂v(ϕ(h),h)
,
∂hc
collecting terms, evaluate at 0 and taking into account the linearity of fj (ϕ, h)
we get equation (17).
1
1
A12 = 0, A22 = −
,
2,
1−θ
σ 2 (1 + θ)2
6θ
2
= −¡
,
K
=
, K222 = K112 = 0
¢
122
2
(1 + θ)3 σ 2
1 − θ2
A11 = −
K111
1 + 3θ2
M1111 = −12 ¡
¢3 ,
1 − θ2
M1122 = −
6
,
(1 + θ)4 σ 2
42
(32)
M1112 = M2222 = M1222 = 0.
⎤
⎥
⎥
⎥
⎥
⎦
To evaluate the derivatives, first consider j = 1 and observe that
and
∂f1 (0,0)
∂ha
∂f1 (0,0)
∂h1
= 1,
= 0 for a = 2, ..., 6. Hence,
(1)
e1 = 1 − θ2 ,
For j = 2 observe that
∂f2 (0,0)
∂h2
(2)
(1)
(1)
(1)
(1)
(1)
and e2 = e3 = e4 = e5 = e6 = 0
∂f2 (0,0)
∂h1
= 1, and
e2 = σ 2 (1 + θ)2 ,
(2)
(2)
= ... =
(2)
∂f2 (0,0)
∂h6
(2)
= 0. It follows that
(2)
e1 = e3 = e4 = e5 = e6 = 0
Applying the same logic we find that the non-zero second derivatives for j = 1
are
(1)
e11
(1)
e22
¡
¡
¢
¢2
(1)
= −6θ 1 − θ2 , e13 = 1 − θ2 ,
¡
¢
¡
¢
(1)
= 2σ 2 (1 + θ) 1 − θ2 , and e24 = σ 2 (1 + θ)2 1 − θ2 ,
whereas for j = 2 we get
¡
¢
(2)
e12 = 2σ 2 (1 + θ) 1 − θ2 ,
¡
¢
(2)
e14 = σ 2 (1 + θ)2 1 − θ2 .
Finally, we have
(1)
¡
¡
¡
¢¡
¢
¢2
¢3
(1)
(1)
−12 + 72θ2 1 − θ2 , e113 = −18θ 1 − θ2 , e115 = 1 − θ2 ,
¡
¢
¡
¢
¡
¢
(1)
= 2σ 2 (1 − 7θ) (1 + θ) 1 − θ2 , e124 = 2σ 2 2 − 3θ − 5θ2 (1 + θ) 1 − θ2 ,
¡
¡
¢2
¢3
¡
¢2
(1)
(1)
= σ 2 (1 + θ)2 1 − θ2 , e133 = 2 1 − θ2 , e144 = 2σ 2 (1 + θ)2 1 − θ2 ,
¡
¡
¢2
¢2
(1)
= σ 2 (1 + θ)2 1 − θ2 , e223 = 2σ 2 (1 + θ) 1 − θ2 ,
e111 =
(1)
e122
(1)
e126
(1)
e234
and
¢
¡
(2)
(2)
e112 = −2 1 − θ2 (7θ − 1) (θ + 1) σ 2 , e114 = 2 (5θ − 2) (θ − 1) (θ + 1)3 σ 2 ,
¡
¢2
¡
¢2
¡
¢2
(2)
(2)
(2)
e116 = σ 2 (1 + θ)2 1 − θ2 , e123 = 2σ 2 1 − θ2 (1 + θ) , e134 = σ 2 (1 + θ)2 1 − θ2 ,
¡
¢
¡
¢
¡
¢
(2)
(2)
(2)
e222 = 12σ 4 1 − θ2 (1 + θ)2 , e224 = 6σ 4 1 − θ2 (1 + θ)3 e244 = 2σ 4 (1 + θ)4 1 − θ2 .
Whereas all the other derivatives are 0.
43
Appendix B3
To get the expression for h notice that
T
T ¡ ¢ 1
1
ln (2π) − ln σ 2 − ln |Σ| − 2 (y − µd)/ Σ−1 (y − µd)
2
2
2σ
µ
¶ 2
∂ (θ)
∂Σ
∂Σ
1
= tr Σ−1
+ 2 (y − µd)/ Σ−1
Σ−1 (y − µd)
∂θ
∂θ
2σ
∂θ
¶
µ
2
∂ 2 (θ)
−1 ∂Σ −1 ∂Σ
−1 ∂ Σ
Σ
+Σ
= tr −Σ
∂θ
∂θ
∂θ2
∂θ2
µ 2
¶
∂ Σ
∂Σ −1 ∂Σ
1
/ −1
Σ
Σ−1 (y − µd)
+ 2 (y − µd) Σ
2 −2
2σ
∂θ
∂θ
∂θ
¶
µ
¶
µ 2
2
1 −1 ∂ 2 Σ
∂ (θ)
−1 ∂Σ −1 ∂Σ
−1 ∂ Σ
−1 ∂Σ −1 ∂Σ
= tr −Σ
Σ
+Σ
Σ
+ Σ
−Σ
E
∂θ
∂θ
2
∂θ
∂θ
∂θ2
∂θ2
∂θ2
(θ) = −
and
∂
⎛
−1
Σ
(θ)
= tr ⎝
3
∂θ
3
³
∂Σ −1 ∂Σ
Σ ∂θ
∂θ
⎛
−
∂2Σ
∂θ2
⎡
´
Σ−1 ∂Σ
∂θ
−
Σ−1 ∂Σ
∂θ
³
´ ⎞
−1 ∂Σ −1 ∂Σ
−1 ∂ 2 Σ
−Σ ∂θ Σ ∂θ + Σ ∂θ2
⎠
2
Σ−1 ∂∂θΣ2
−Σ−1 ∂Σ
∂θ
Σ−1 ∂Σ
Σ−1 ∂Σ
−3 ∂Σ
∂θ
∂θ
∂θ
+
∂ 2 Σ −1 ∂Σ
Σ ∂θ
∂θ2
⎤
⎞
⎦ Σ−1 ⎟
⎜ Σ−1 ⎣
1
2Σ
⎟
/⎜
∂
∂Σ
−1
− 2 (y − µd) ⎜
⎟ (y − µd)
+ ∂θ Σ ∂θ2
σ
⎝
⎠
³
´
1 −1 ∂Σ −1 ∂ 2 Σ
∂ 2 Σ −1 ∂Σ
−1
+2Σ
Σ ∂θ2 + ∂θ2 Σ ∂θ Σ
∂θ
´
⎛ ³
⎞
−1 ∂Σ −1 ∂Σ
−1 ∂ 2 Σ
−1 ∂Σ
Σ ∂θ Σ ∂θ − Σ ∂θ2 Σ ∂θ
µ 3
¶
⎜
³
´ ⎟
∂ (θ)
⎜
⎟
−1 ∂Σ
−1 ∂Σ −1 ∂Σ
−1 ∂ 2 Σ
E
=
tr
⎜
⎟
−Σ
−Σ
Σ
+
Σ
2
3
∂θ
∂θ
∂θ
∂θ
⎝
⎠
∂θ
−1 ∂Σ −1 ∂ 2 Σ
−Σ ∂θ Σ ∂θ2
⎛
i ⎞
h
−1 ∂Σ −1 ∂Σ −1 ∂Σ
−1 ∂ 2 Σ −1 ∂Σ
−1 ∂Σ −1 ∂ 2 Σ
Σ
Σ
+
Σ
Σ
+
Σ
Σ
−2
−3Σ
1
∂θ
∂θ
∂θ
∂θ
∂θ
∂θ2
∂θ2
⎠
+ tr ⎝
2
2
2
−1 ∂Σ −1 ∂ Σ
−1 ∂ Σ −1 ∂Σ
Σ
−Σ
Σ
−Σ
∂θ
∂θ2
∂θ2
On the other hand
µ
¶
2
1 /
∂ 3 (θ)
−1 ∂Σ −1 ∂Σ −1
−1 ∂ Σ −1
(y − µd)
Σ
Σ −Σ
= 2 d 2Σ
Σ
σ
∂θ
∂θ
∂µ∂θ2
∂θ2
Now under our assumptions we have that
(y − µd)
∼ N (0, Σ)
σ
44
∂θ
and the characteristic function of
(y−µd)/ Ca (y−µd)
σ2
/
(a = 1, 3, 5) and
cb (y−µd)
σ2
(b = 2, 4, 6)
of the same form as in Phillips (1980). Consequently, taking into account that
C1 , C3 , C5 , and Σ are symmetric matrices we have for a, b, d, f = 1, 3, 5 and
p, q, r, s = 2, 4, 6:
¡
¢
2
1
ψab = − tr (Ca ΣCb Σ) , ψ ap = 0, ψ pq = − 2 tr c/p Σcq
T
Tσ
8i
2i
ψabd = − 3 tr (Cd ΣCb ΣCa Σ) , ψ abp = 0, ψ apq = − 3 c/p ΣCa Σcq
T2
T2
48
tr (Cf ΣCd ΣCb ΣCa Σ) ,
ψabdf =
T2
8
ψabdp = 0, ψ abpq = 2 c/p (ΣCb ΣCa Σ) cq , ψ apqr = 0, ψ pqrs = 0.
T
and ψ pqr = 0
We present the derivation of only ψ44 . All the other cumulants follow with the
∞
X
cos kλ, we have
same logic. As g (λ) = 1 + 2
k=1
ψ44
µ
¶
´
³
1
1
/
/ −1 ∂Σ −1 ∂Σ −1
Σ
Σ d
= − 2 tr c4 Σc4 = − 2 tr d Σ
Tσ
Tσ
∂θ
∂θ
¢2
¡ iλ
Zπ
Zπ
e + e−iλ + 2θ
4 (cos λ + θ)2 g (λ)
1
1
dλ = −
∼ −
¡
¢3 dλ
2πσ 2
2πσ 2
|1 + θeiλ |6
1 + θ2 + θeiλ + θe−iλ
−π
−
1
2πσ 2
Zπ
¢2
¡ iλ
e + e−iλ + 2θ
−π
= −
⎛
1 1 1⎜
⎝
σ 2 2π i
Z
|z|<1
k=1
−π
¡ ikλ
¢
e + e−ikλ
¡
¢3
1 + θ2 + θeiλ + θe−iλ
2
∞
X
(z 2 + 1 + 2θz)
dz +
(zθ + 1)3 (z + θ)3
k=1
Z
dλ
|z|<1
¢
¡
(z + 1 + 2θz) z 2k + 1
2
2
zk
3
3
(zθ + 1) (z + θ)
⎞
⎟
dz ⎠
¢!
2¡
∞
2
X
d2 (z 2 + 1 + 2θz)
1
dk−1 (z 2 + 1 + 2θz) z 2k + 1
1
+
lim
lim
z→0 dz k−1
2 z→−θ dz 2
(k
−
1)!
(zθ + 1)3
(zθ + 1)3 (z + θ)3
k=1
¢
2¡
∞
1 X1
d2 (z 2 + 1 + 2θz) z 2k + 1
− 2
lim
σ k=1 2 z→−θ dz 2
z k (zθ + 1)3
1
= − 2
σ
Ã
∞
X
2
and notice that
1
2
2
d2 (z +1+2θz )
limz→−θ dz
2
(zθ+1)3
(2θ2 +1)
= −2 (θ+1)3 (θ−1)3 and
45
2
2
2
4
2
2
2
2
2k
2
1 k(1−8θ +7θ )+k (1−θ ) +4θ (1+2θ )
d2 (z +1+2θz ) (z +1)
limz→−θ dz
=
−
3
2
3
3
k
2
z k (zθ+1)
(θ+1) (θ−1) (−θ) θ2
2 +4θ2 +8θ 4 +8kθ 2 −7kθ4 −2k2 θ 2 +k2 θ 4 (−θ)k
−k+k
(
)
− 12
(θ+1)3 (θ−1)3 θ2
1
2
2
Furthermore,
1
(k−1)!
limz→0
2
2k
dk−1 (z +1+2θz ) (z +1)
3
dz k−1
(zθ+1) (z+θ)3
=
1
(k−1)!
limz→0
2
2
dk−1 (z +1+2θz )
,
dz k−1 (zθ+1)3 (z+θ)3
2
(z2 +1+2θz)
2
1
1
1
2θ+4θ3
1
4θ2 +2
= 3θ2 −3θ
+ 1−4θ
+ 1
− 3θ2 −3θ
−
4
4
+θ6 −1 (zθ+1) θ−2θ3 +θ5 (zθ+1)2 θ3 −θ (zθ+1)3
+θ6 −1 (z+θ)
(zθ+1)3 (z+θ)3
1
1
3θ
− θ21−1 (z+θ)
3.
θ4 −2θ2 +1 (z+θ)2
³
´
k−1
dk−1
dk−1
1
1
1
1
Notice also that (k−1)!
=
limz→0 dz
=
(−θ)
,
lim
z→0 dz k−1 (zθ+1)2
k−1 zθ+1
(k−1)!
k−1
and
,
k (−θ)
1
1
1
dk−1
dk−1
= k(k+1)
(−θ)k−1 , (k−1)!
limz→0 dz
k−1 (z+θ)
2
dz k−1 (zθ+1)3
1
1
dk−1
limz→0 dz
= k θ12 (−θ)1k−1 ,
k−1
(k−1)!
(z+θ)2
dk−1
1
1
1
1
and (k−1)!
limz→0 dz
= k(k+1)
we get
k−1
2
θ3 (−θ)k−1
(z+θ)3
1
(k−1)!
limz→0
=
1
1
,
θ (−θ)k−1
2
2
4
2
4
2
2
2 2
2 4
1
1 (k −k+4θ +8θ +8kθ −7kθ −2k θ +k θ )
dk−1 (z +1+2θz )
limz→0 dz
3
3 = 2
3
3
k−1
(k−1)!
(zθ+1) (z+θ)
(θ+1) (θ−1) θ
2
4
2
4
2
2 2
2 4
1 (k+k +4θ +8θ −8kθ +7kθ −2k θ +k θ )
1
2
(θ+1)3 (θ−1)3 θ3
(−θ)k−1
(−θ)k−1 −
Hence
ψ44
⎛
¡
¢
−2 2θ2 + 1 +
4θ2 (1+2θ2 )
θ
∞
X
(−θ)k−1
⎜
⎜
1
k=1
⎜
∼ −
∞
∞
2
4 X
2
2
−1+8θ
−7θ
1−2θ
+θ4 ) X 2
σ (θ + 1)3 (θ − 1)3 ⎜
)
(
(
k−1
⎝ +
k (−θ)
+
k (−θ)k−1
θ
θ
k=1
= −
k=1
4
1
2
σ (θ + 1)4
and the result follows by collecting terms and as for |x| < 1 we have that:
∞
X
x
kx =
(1 − x)2
k=1
k
and
∞
X
k=1
46
k2 xk =
x2 + x
.
(1 − x)3
⎞
⎟
⎟
⎟
⎟
⎠
Hence the cumulants of h are:
ψ11 = −
1
,
1 − θ2
4θ
ψ 13 = − ¡
¢2 ,
1 − θ2
3θ2 + 1
ψ 15 = −6 ¡
¢3 ,
1 − θ2
1
1
3 + 7θ2
1
2
,
ψ
=
−
,
ψ
=
,
ψ33 = −2 ¡
¢
22
24
3
2
σ 2 (1 + θ)
σ 2 (θ + 1)3
1 − θ2
6
4
ψ26 = −
, ψ 44 = −
4
σ 2 (θ + 1)
σ 2 (θ + 1)4
1
6θ
i
4i 5θ2 + 2
2i
√
ψ111 = − √ ¡
,
ψ
=
¢
¡
¢3 , ψ 122 = − √ 2
113
2
2
2
T 1−θ
T 1−θ
T σ (1 + θ)3
¢
¡
1
4i
6 7θ2 + 3
ψ124 = √
, ψ 1111 = ¡
¢ .
T 1 − θ2 3
T σ 2 (θ + 1)4
47
0.04
0.03
0.02
0.01
0
-3.5
-2.5
-1.5
-0.5
0.5
1.5
2.5
3.5
-0.01
-0.02
True-Normal
True-T Approximation
Figure 1: Differences from Exact Distribution of
√
T (b
ρ − ρ).
1.5
1.25
1
0.75
0.5
0.25
0
-1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
Theta
Figure 2: |E [T (b
ρ − ρ)]| (thick line) and |E [T (ρb0 − ρ0 )]|.
48
4.5
1
0.875
0.75
0.625
-1
-0.75
-0.5
0.5
-0.25
0
0.25
0.5
0.75
1
Theta
Figure 3: MSE of
√
√
T (b
ρ − ρ) (thick line) and T (ρb0 − ρ0 ), T = 10.
0.1
0.08
0.06
0.04
0.02
0
-5
-4
-3
-2
-1
0
1
2
3
4
-0.02
-0.04
True-T Approximation
True-Normal
Figure 4: Differences from Exact Distribution of
49
´
√ ³
T b
θ−θ .
5
10
7.5
5
2.5
0
-0.5
-0.25
0
0.25
0.5
Theta
¯ h ³
¯ h ³
´i¯
´i¯
¯
¯
¯
¯
b
b
Figure 5: ¯E T θ − θ ¯ (thick line) and ¯E T θ0 − θ0 ¯
4.5
4
3.5
3
2.5
2
1.5
-0.3 -0.25 -0.2 -0.15 -0.1 -0.05 0
0.05 0.1
0.15 0.2
0.25 0.3
Theta
Figure 6: MSE of
´
´
√ ³
√ ³
T b
θ − θ (thick line) and T θb0 − θ0 , T = 10.
50
0.2
0.1
-9
-7
-5
-3
0
-1
-0.1
1
3
5
7
9
-0.2
-0.3
-0.4
-0.5
-0.6
-0.7
True-T Approx.
True-root T Approx.
True-Normal
Figure 7: Differences from Exact Distribution of
´
√ ³
T e
θ−θ .
2
1.5
1
0.5
0
-0.5
-0.375
-0.25
-0.125
0
0.125
0.25
0.375
0.5
Theta
Figure 8: MSE of
´
´
√ ³
√ ³
T e
θ − θ (thick line) and T θe0 − θ0 , for T = 10.
51
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
-0.5
-0.02
-1.5
0.5
1.5
2.5
3.5
4.5
-0.04
-0.06
True-T Approx.
True-Normal
Figure 9: Differences from Exact Distribution of
√
T (e
ρ − ρ).
1
0.75
0.5
0.25
0
-1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
Theta
Figure 10: |E [T (e
ρ − ρ)]| (thick line) and |E [T (ρe0 − ρ0 )]|
52
0.8
0.6
0.4
0.2
0
-1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
Theta
Figure 11: MSE of
√
√
T (e
ρ − ρ) (thick line) and T (ρe0 − ρ0 ) for T = 10.
3
2.5
2
1.5
1
0.5
0
-0.5
-0.375
-0.25
-0.125
0
0.125
0.25
0.375
0.5
Theta
¯ h ³
¯ h ³
´i¯
´i¯
¯
¯
¯
¯
Figure 12: ¯E T b
θ − θ ¯ (thick line) and ¯E T e
θ − θ ¯.
53
1.5
1.25
1
0.75
0.5
0.25
0
-1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
Theta
Figure 13: |E [T (b
ρ − ρ)]| (thick line) and |E [T (e
ρ − ρ)]|.
4.5
4
3.5
3
2.5
2
1.5
-0.3
-0.25 -0.2
-0.15 -0.1
-0.05 0
0.05 0.1
0.15 0.2
0.25 0.3
Theta
´
´
√ ³
√ ³
b
e
Figure 14: MSE of T θ − θ (thick line) and T θ − θ for T = 10.
54
1
0.75
0.5
0.25
0
-1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
Theta
Figure 15: MSE of
√
√
T (b
ρ − ρ) (thick line) and T (e
ρ − ρ) for T=10.
55