BIOMATHEMATICS TRAINING PROGRAM
DISTRIBUTION OF LIKELIHOOD RATIO
TE3T STATISTICS FOR NONSTATIONARY TTh1E SERIES
by
David A. Dickey
Wayne A. Fuller
Mimeograph Series No. 1171
Raleigh, N.C. April 1978
DISTRIBUTION OF LIKELIHOOD RATIO
TEST STATISTICS FOR NONSTATIONARY TIME SERIES
David A. Dickey
North Carolina State University, Raleigh, North Carolina 27650
and
Wayne A. Fuller
Iowa State Universi ty, Ames, Iowa 500ll
ABSTRACT
Let the time series
Y = 0
l
and
ret }~=l
random variables.
series be given.
(a,p) = (0,1)
Yt
satisfy
Yt = a + PYt - + e t ' where
l
is a sequence of normal independent
Let
n
observations
Y , Y2 , ... , Y
l
n
(0,02)
from the time
The likelihood ratio test of the hypothesis that
is investigated and a limit representation for the test
statistic is presented.
Percentage points for the limiting distri-
bution and for finite sample distributions are estimated.
tribution of the least sCluares estimator of
a
The dis-
is also discussed.
A similar investigation is conducted for the model containing a time
trend.
David A. Dickey, North Carolina State University,
Raleigh, North Carolina 27650
- 2 -
Let
Y
t
satisfy the model
t =
wheree
mean
°
2,3, ... ,n,
(1.1)
is a sequence of normal independent random variables with
t
and variance
estimators of
0
cr2 [e
,... NID(0,cr 2 )].
t
and a, conditional on
The ma.ximwn likelihood
Y , are the least squares
l
estimators
1\
1\
(1.2)
where
( n-l) -1
y( -1)
=
Y(O)
= (n-l) -1
n
!:
t=2
Yt - l
n
!:
t=2
Y
t
1\
In this article we investigate the limiting distribution of
given that
(0.,0)
= (0,1).
a
~
We also study the limiting distribution
of the likelihood ratio test of the hyPOthesis that
(0.,0)
= (0,1).
- 3 -
An al.ternative model for
Y1 =
Y
t
is
°
t
were, as- before, e
of a
and
~
t
,.., N1D (0,0"2) .
= 2,3, ... ,n,
We study the least squares estimators
of (1.3) under the assumption that (a,~,p)
We also study the likelihood ratio test of the hypotheses
=
(1.3)
=
(0,0,1) •
(a,~,p)
(0,0,1).
Empirical. distributions of the statistics are generated by Monte
Carlo methods for finite sample sizes.
Representations based on the
results of Dickey (J.976) are presented for the limiting distributions
and the li.miting distributions are simulated using these representations .
- 4. -
2.
Distribution
of " normalized
regression
coefficients for finite
n .
• _ ... _ - •
....
"O4"
...........
....
.~
The statistic constructed by analogy to the regression "t-statistic"
. for the estimated ex of model (1.1) is
-1
or
. CXIJ.
A
= SCXIJ. exIJ.'
-1 [A
e(O) - (p
1 n-1
= SCXIJ.
IJ.
- l)(n - 1)- . ~ (n ~ t)e t ] ,
t=l
(2.1)
where
S
2
CtIJ.
2 [(
= SelJ. n-1
) -1
+
::2
-1 n
S2
elJ.
= (n-3)
= (n-1)
(n
-
Y(-l) ~ (Yt - 1 - y(-l))
t=2
A
~
2}-l
II
n
~
t=2
e
(2.2)
A
(Y - ex - P y _ )2
t=2 t
IJ.
~ t 1
-1
J
t
Using the model
t
= 2,3, ... ,n,
(2.3)
- 5 -
the sampling distribution of
T
(Y , Y ,
2
l
a 1;'atio of quadratic· forms in
distribution of
T
QI.1
was simulated.
QI.1
is symmetric.
Because
O+L
for 50,000 samples with
the distribution is bimodal.
n
=
- 1
is
it follows that the
••• , Y )
n
Therefore cells equidistant from
zero were pooled to create a symmetric histogram.
T.
"p~
25
The histog.ram of
is shown in Figure 1.
This is because
"
n(p
~
- 1)
Note that
has a non-
n-l
zero mode for positive values of
l: (n-t)e
t=l
t
as well as for negative
n-l
values
l: (n-t)e
t=l
Let
and let
e
,...
t
.
· whose J..th row J.S
( n- 1) x 3 mat ru
X d enot e th e
'"
I'
= (Y , Y , ..• , Y )·
2
n
3
= (a,~,p)'
is
" " "p
",...e = (aT'
~T'
T
Let
C..
J.J
)
I
=
(~,~-l !'I .
denote the ijth element of
J. "
(C
S2) -2 a
T
crr = 11 eT
T
T~,.
Then the least squares estimator of
=
(C 22 S:T)
(2.4)
(~'~ -1 and define
,
-i"~T ,
(2.6)
- 6 ...
where
S2
eT
= (n-4) -1 Y' [I _ X(X'X' -1 X' Jy
/"'oJ
,..,
~
,..,
-:::J
.-..J.
Figures 2 and 3 contain histograms for
1"131"
constructed from 50,000 samples of size
I"'W
1"
.
ar
and
n = 25.
All of
the distributions are symmetric and the histograms were constructed
to be
symmetric~
The distributions of the 1"-statistics are distinctive in two
respects; the distribution is bimodal and the "spread" of the distribution is much larger than that of Student's t-distribution.
The esti-
mation of percentiles for the distributions is discussed in Section
5.
- 7 -
3.
Likelihood ratio tests
We construct the likelihood ratio tests for testing the null
hypothesis that the true model is a random walk with zero drift.
consider first the test under the alternative (1. 3).
We
The logarithm
of the likelihood function for a sample of n observations from model
( 1. 3), conditiona! on
Y , is
l
log L = - 'i(n-l) log(2TT) - (n-l)log
(J
n
- (2a2)-1 L [Yt - a - ~(t-l-in) - pY
t=2
t-l
J2 .
Under the null hypothesis, HO: (a,~,p) = (0,0,1), the likelihood is
maximized with respect to a2 to obtain
Under the alternative hypothesis the maximum of the like"lihood occurs
A
at
A
(ci2 ' ~,) , where
l
~
A
A
e = (a ,
/'OW,.
Thus the likelihood ratio is
A
A
~ ,. , p,. )'
was defined in (2.2), and
(3.1)
- 8 -
Using
we obtain
= l+3(n-4) -1
~2'
where
and
S2
eT
was defined in (2.7).
H for large values of
O
of the hypothesis
~2
Thus the likelihood ratio test rejects
' where
H : (a,~,p)
O
=
~2
is the usual regression "F-test"
(0,0,1).
In a similar manner, it can be shown that the likelihood ratio statistic
for testing
H : (a,p)
O
= (0,1)
for the model (1.1) is
where
and
1\
~1 =
(28 2 ) -1 [(n-1)~0 - (n-3)S2 ]
S2
was defined in (2.3J.
e~
e~
.e~
- 9 -
The likelihood ratio test of'the hypothesis
(/3,p) = (0,1)
against
the alternative specified in model (1.3) is a monotone function of
As with the other tests the statistic .i
·would construct for the hypothesis.
3
is the common !IF-test" one
In this situation the null
hyJ;lothesis is a random walk in which drift
a
is permitted.
It is
easily demonstrated that the distribution of the test statistic does
not depend upon a.
- 10 -
4.
~ting dis.tributio~~.
The several statistics that we have discussed can all be expressed
as functions of few sample statistics.
~t
= (n-l) -i ~
e(O)
T
n
t=2
3
W
n
=
(n_l)-2
= (n-lri
et
n-l
~ (n-t)e
t=l
_~
t
• (n-I)
_
Y(-I)'
n-l
= (n-l)-5 / 2. ~ (n-t)(t-l)e
n
t=l
t
(4.1)
V
Then, for example,
1.
A
(n-l)2 a:
A
~
= T
n
- (p
~
-l)W
n
and
A
Given that
0-
2
~ 1, Dickey (1976) ha.s shown that
[r,
n
T , W , V , n(p - 1) J
n
n
n
~
- 11. -
converges in distribution to
(r,
T, W, V, 0) , where
CD
r
T
i:
=
· 1
J.=
= · ~1
Y.2 .Z2. ,
J.
J.
,)Y.Z.
W
~ 2t Y~J. z.J. ,
=
· 1
J.=
CD
V
and
,
J. J.
J.=
=
:3
i: (2 2
· 1
J.=
.do.
2
- 2 2 Y. )Z.
Y~
,J.
. J.
2
(_l)i+l
(2i-l)TT
Yi
=
0
= (r _ W2) -1 [.~(Tid-
[Z. J~ 1
,
,
-1)- TW]
(4.2)
is a sequence of normal. independent
J. J.=
variables.
J.
It follows that
•
nZ
-1
(j
A
0:
!
;>
T - oW ,
1..1.
under the assumption that
(0:,0) =
(O,~).
In a similar rilamler
because
S2
el..l.
converges in probability to
r;r2.
(O,J.) random
-12-
For model (1. 3) with the assumption that
and that
=1
(]'2
,.... ""
=
= (0,
0, 1)
, we have
1
X'X
(0;, t3, p)
(n-l)
12
°
°
-1
(n-l)~
n(n-2)
i V
1
-(n-l)
2
n
(n_l)i W
n
W
n
1
2
-(n-l) Z V
2
n
(n-l)r
n
Letting
~
Rn = diag[(n-l)2
1
A
=
°
W
, (n-l)
°
W
1
12
k2,
~
2
r
2
Z ,
n-l] ,
,
we obtain
(4.4)
A
Under our assumptions,
e
""
e=
is estimating
,....
(0,0,1)'
and we have
,
where
! =
(T, iT - W, ~ ('! - cfl))'.
The mtrix is invertible with
probability 1 and it is readily verified that
(4·5)
- 13 -
Q. + W2
1 =Q.-l
A,..,
6vw
6VW
12Q. +
- w
where
Q. =
r -
W2 - jV2.
36v2
- 6v
-w
-6v
,
(4.6)
1
Thus
and
A
1,
6~ - - ; >
..-.0.",,;';
D (6_
The third element of
A
'n(P -1)
T
-1
A
,..,
A-1 f
f
,..,
..
is the limit random variable for
as given in Dickey (1976).
(4.8)
- 14 Using (4.6) and the fact that
S;'T'
converges in probability
a2,
we obtain
and
(4·9)
Likewise
(4.10)
J:
-1
:> 3
-1
!' ~ !
= 3
-1
[T
2
+ l2(~T - W)
2
2
+ 'T''T'J,
(4.1J.)
and
J: "- 2- 1 (f' A- 1 f _ T2)
12
2
i 3 -.-;....
,...;,...,...
• 2[12(~T - W) .+ 'T''T']
where 'T' 'T' is the limit random variable for the
p - 1 in model (1.3).
r~gression
(4.12)
lit-statistic" for
- 15 -
SimuJ.ation
........
_ ..... w
5·
The distributions of the statistics for finite samples were
simuJ.ated for time series generated by the model with
YO
=0
and
Yt = Yt - l + et,t' = 1,2, .•. ,n for n = 25, 50, 106, 250, and 500.
For each time series 50,000 samples of size n were generated and the
statistics computed for those samples.
generated for
n
= 500.
n
= 25,
two for
n
= 50,
Three replicates of
100, and
50,000 were
250, and one for
The simuJ.ation of the limit case was conducted using Dickey's
(1976) procedure.
Three replicates of
50,000 were generated for the
limit case.
For each of the nine estimators and for each sample size, the 0.01,
0.025, 0.05, 0.10, 0.90, 0.95, 0.975, and 0.99 percentage points of the
distributions were calculated.
plotted against n.
form
P
=a
These empirical percentiles were then
Based on the plots, regression functions of the
+ /3n Y were fitted to the percentiles of the empirical
distributions.
The regression smoothed percentiles are given in Tables
1 through 9.
David (1970 section
2.5) gives a method for
constructingdistri~
bution free confidence intervals for the percentiles of a distribution
based on empirical percentiles.
We used the half length of a 68.~
confidence interval as an estimated standard error (based on the fact
that 0.6826 is the probability that a normal random variable will differ
f'romits mean by no more than one standard deviation).
through 8 the number in the raw labeled
errors constructed for
n
=25
Tl
In Tables 1
s . e. Tl is the largest of the standard
and for the limit case.
These standard errors
- 16 provide. an upper bound for the standard errors of the regression
smoothed percentiles.
Since several observations on each percentile were available for
n = 25, 50, 100, 250, and for the limit case, regression F-tests for
lack of fit for the smoothing regressions were computed.
Of the 48
lack of fit statistics computed, 13 were significant at the 0.25 level,
3 at the 0.05 level, and none at the O. Ol level.
- 17 -
In this section we demonstrate that the test statistic s
investigated in the previous sections can be applied in higher order
autoregressive processes.
Consider data generated. py the model
(6.1)
where
is a stationary autoregressive process and the
e
are
t
NID(O,cr2 ).
The model can also be written
Yt
where
p
= PYt - l
=1
and
+
Zt
p
~
i=l
= Yt
e·(yt · - Yt 1" .) + e t
~
-~
- -~
- Y- .
t l
assume, without loss of generality,
'
To simplify the presentation we
a2 = 1
•
Consider the regression equation
p
Yt
+p
t
= 1,2, ... ,n-p
=a
.
+ ~ [t - i(n-p+l)] + p Yt + -
P l
Let
H
'-n
denote the
+
~
i=l
(P+3) x (P+3)
e.z
.
~ t +p-~
+ et '
sums of squares
- 18 and products matrix needed to compute the regression, let
!!n'
the square roots of the diagonal elements of
let
~
M
.-.n
=
denote
(el, t3, 0,
A
8 , 8 , ••. , 8p )
2
1
~
and let
denote- the least squares estimator of
Then
y' .
"-n
where
~'
xn
n-p
=
~
t=l
(1, t, Yt+p-1' Zt+p-1'
Fuller (l976,p. 374 ) has demonstrated-that
~
to n(1
-
P
~e .)
i=1
t
-l
J.
~
e.
j=l
as
t
increases.
-l
n
~
t=2
n-
2
Zt
=0
~
(n
-'2
)
P
n
~ [t - !(n+p-1)]Zt
. =
t=l
+P-J
n
~ Yt - l Zt_l
t=2
Therefore
= Open)
.
~
0 (n- )
P
Yt
is converging
By the results of Fuller,
J
we have,
n
n-~
- 19 -
where
!!u=
1
0
r""' w
0
1
r-i
r-i w
r~ 3~
~2
is the
and
V were defined in (4.2).
p xp
V
3'
V
1
correlation matrix of the process
Zt' and F, W,
It follows that the limiting distribution
A
of the vector composed of the first three elements of
is the same as the limiting distribution of
discussed in section
4.
M (y - y )
",n ",n ""n
- 20 -
7·
~:c~le
Friedman and Schwartz (1963) give yearly observations of the
velocity of money from 1869 through 1960 (n = 92).
Gould and Nelson
(1974) ·concJ.ude that the logarithms of the observations are consistent
.
WJ.th the model
.
X = X _l + e ' where
t
t
t
e
2
,.,. NID(O,O" ).
t
To illustrate
the use of Tables ·1 through 9 in hypothesis testing, we fit two models
to the data.
Below we list the models with the fitted coefficients,
the standard errors of the coefficients, the regression error mean
square and the regression
~
statistic.
Note that using
X - X _
t
t l
on the left side of the regression equation yields an estimator of
p - 1 for the regression coefficient of
X _ .
t l
The regression statistics
are:
Xt - Xt _ =
l
0.016
-
(0.017)
0.034X _
t l
(0.058)
S2 = 0.0050, ~l = 2·99
e\-L
X - X _ =
t
t l
0.086
(0.04 7)
S2 =
eT
-
(7·1)
0.0013(t-4 7) (0.0008)
0 ..l2OXt~1
(0.058)
0.004 9, i = 2·77
2
(7. 2 ).
For regression equation (7.1) the test statistic is
~l
=.2.99
which is smaller than the tabular 0.90 value of 3.88 (obtained by
interpolating between
n = 50
and
n = 100
in Table 7).
Therefore
the hypothesis of a random waJ.k :wi. th zero dr1:ft is accepted at the
0.10 level when tested against the alternative (1.1).
- 2l -
For regression equation (7.2) the statistic for testing
(0:,. (3, p) = (0, 0, l) is
~2 =
2.71.
HQ:
Comparing this to the value
4.l9 obtained by interpolation from Table 8 the hypothesis of a random
~
is accepted at the O.lO level when tested against the alternative
(1.3).
- 22 REFERENCES
Anderson, T. w., 1959, On asymptotic distributions of estimates of parameters
of stochastic difference equations, Ao.D.als of Mathematical. Statistics
30, 676-687·
Box, G. E. P. and G. M. Jenkins, 1970, Time series analysis forecasting and _
control (Holden-Day, San Francisco).
David, H. A., 1970, Order statistics (Wiley, New York).
Dickey, D.. A., 1976, Estimation and hypothesis testing in nonstationary time
series, Iowa State University Ph.D. thesis.
Friedman, M., and A. J. Schwartz, 1563, A monetary historJ of the United
States 1867-1560 (Princeton University Press, Princeton, New Jersey).
Fuller, W. A., 1976, Introduction to statistical time series (Wiley:, New York).
GOUld, J. P., and C. R. Nelson, 1974, The stochastic structure of the velocity of money, The American Economic Review 64, 405-417.
Hasza, D., 1977, Estimation in nonstationary time series, Iowa State UniveISity
Ph. D. thesis.
Mann, H. B., and A. Wa1d, 1943, On stochastic limit and order relationships,
Anna.l.s of Mathematical Statistics 14, 217-226.
Rao, M. M., 1961, Consistency and limit distributions of estimators of parameters in explosive stochastic difference equations, Anna.l.s of Mathematical Statistics 32, 195-218.
Rubin, H., 1950, Consistency of maximum-likelihood estimates in the explosive
cas"e, in: T. C. Koopmans, ed., Statistical inference in" dynamic economic
models.
- 23 -
Table
1. . Empirical Distribution of nt
0"
-1 ~
IJ.
.(Symmetric Distribution)
Sample
size
n
25
50
100
250
500
CD
s.e.
Probability of a smaller value
0·90
0·95
0·975
0·99
4.32
4.40
4.42
4.43
4.43
4.43
5.77
5.88
5·93
5.96
5·97
5·99
7·17
7·31
7.39
7. 45
7.47
7.50
8.92
9.15
9.27
9.34
9.37
9.40
0.01
0.02
0.02
0.03
- 24 -
.Tab1e 2.
Sample
size
n
25
50
100
250
500
CD
s.e.
Empirical Distribution of T
OIJ.
(Symmetric Distribution)
Probabili ty of a smaller value
0.90
0.95
0·975
0·99
2.20
2.18
2.61
3.41
3.28
3.22
3.19
3.18
3.18
0.008
2.17
2.16
2.16
2.16
2·53
2·52
2.52
2.97
2.89
2.86
2.84
2.83
2.83
0.003
0.004
0.006
2.56
2.54
- 25 -
Table 3.
Sample
size
n
25
50
100
250
500
CD
s.e.
Empirical Distribution of ni
(Symmetric Distribution)
0--
1
~T
Probability of a smaller value
0·90
0·95
7.63
7·95
8.11
8.20
10.49
11.07
11. 34
11.50
8.23
8.26
0.02
0.975
0·99
11·55
11·59
13.41
14.22
14.62
14.87
14.95
15.04
18·37
19·00
19. 4 3
19;60
19.82
0.03
0.05
0.07
17.26
- 26 -
Table 4..
Sample
size
Empirical Distribution of TaT
(Symmetric Distribution)
Probability of a smaller value
n
0·90
0·95
0·975
0·99
25
50
100
250
500
2·77
2.75
2·73
2·73
2·72
2.72
3.20
3.14
3·11
3.09
3.08
3.08
3·59
3.47
3.42
3.39
3.38
3.38
4.05
3.87
3.78
3.74
3·72
3.71
0.004
0.005
0.007
0.008
CD
s.e.
- 27 -
Table
5. Empirical Distribution of ni
0--
1
~'T"
. (Symmetric Distribution)
Sample
size
Probability of a smaller value
n
0.90
0.95
25
19·73
14.28
14·57
14.76
14.83
14.90
19.06
19.98
20.48
20.81
20.93
21.06
24.34
25·81
26.60
27.11
27·29
27·50
31.47
33.74
34.98
35.80
36.11
36.48
0.04
0.06
0.10
0.14
50
100
250
500
ex>
s.e.
0·975
0.99
- 28 -
Table
6.
Empirical Distribution of
T
t3T
(Symmetric Distribution)
Sample
size
n
25
50
100
250
500
CD
s.e.
Probabili ty of a smaller value
0·90
2.39
2.38
2.38
2.38
2.38
.2.38
0.004
0·95
2.85
2.81
2.79
2·79
2.78
2.78
0·975
0·99
3.25
3.18
3.14
3·12
3.11
3·11
3.74
3·60
3·53
3.49
3.48
3.46
0.005
0.006
0.009
- 29 -
Table .7.
Sample
size
n
0.01
25
50
100
250
500
co
s.e.
0.025
Empirical Distribution or 11
Probability o£ a smaller value
0.10
0.05
0·90
0·95
0.29
0.29
.0.29
0·30
0.30
0·30
0.38
0·39
0·39
0.39
0.39
0.40
0.49
0·50
0·50
0·51
0·51
0·51
0.65
0.66
0.67
0.67
0.67
0.67
4.12
3.94
3.86
3.81
3.79
3·78
0.002
0.002
0.002
0.002
0.01
0.975
0.99
4·71
4.63
4.61
4.59
6.30
5.80
5·57
5. 45
5. 41
5.38
7.88
7.06
6.70
6.52
6.47
6.43
0.02
0.03
0.05
5.18
4.86
- 30 -
Table 8.
Sample
size
0.01
n
25
50
100
250
500
CJ)
s.e.
0.025
0.61
0.62
0.63
0.63
0.63
0.63
0·75
0.77
0·77
0·77
0·77
0·77
0.003
0.003
Empirical Distribution of '2
Probability of a smaller value
0.10
0.05
0·90
0·95
0.975
0·99
4.67
4.31
4.16
4.07
4.05
4.03
5.68
6.75
8.21
0·91
0.92
0·92
0.92
0.92
1.10
1.12
1.12
1.13
1.13
1.13
5·13
4.88
4.75
4.71
4.68
5·94
5·59
5. 40
5·35
5·31
7·02
6.50
6.22
6.15
6.09
0.003
0.003
0.01
0.02
0.03
0.05
0.89
- 31 -
Table 9.
Sample
size
n
Empirical Distribution of
~3
Probability of a smaller value
0.01
0.025
0.05
0.10
0.90
0.95
0.975
0.99
25
0.74
0.90
1.08
1.33
5.91
7.24
8.65
10.61
50
0.76
0.93
1.11
1.37
5.61
6.73
7.81
9.31
100
0.76
0.94
1.12
1.38
5.47
6.49
7.44
8.73
250
0.76
0.94
1.13
1.39
5.39
6.34
7.25
8.43
500
0.76
0.94
1.13
1.39
5.36
6.30
7.20
8.34
00
0.77
0.94
1.13
.1.39
5.34
6.25
7.16
8.27
s.e.
.004
.004
.003
.004
.015
.020
.032
.058
- 32 -
·-3
Figure 1.
o
Histogram for 50,000 values of
3
T
CXIJ.
constructed with n
6
=25
- 33 -
-0
'"
Figure 2.
o
Histogram for 50,000 values of
3
T etr
6
constructed 'With n = 25
- 34 -
"
-0
Figure 3.
-3
o
Histogram for 50,000 values of
3
'T"
t3'T"
constructed 'With n
6
= 25