The I'Bsea:Poh in this I'Bpon 7J1Q8 pat'tiaZ'ty supponed by the U. S. Ail'
Fo1'ae Offiae of Scientifio RsssaZ'Ch unde:r Contl'am; No. AFOSR-GS-1416.
THE INVERSE AUTOCORRELATIONS OF A TIME SERIES
by
William S. Cleveland
Depazrtment of Statistics
Univel'Bity of Nonh Caro'tina at Chapel BiZZ
Institute of Statistics M1meo Series No. 689
June, 1970
THE UiVERSE AUTOCORRELAXIONS OF A TIME SERIES
by
William S. Cleveland
Depanment of Statistics
Univemi ty of North Carolina
Chapel Hill" N.C. 27514
1.
INTR>DOC11ON.
In this paper, the inverse autocorrelations of a time series are introduced and their use described.
stationary time series with
(1)
Let
s
EXt == 0
X
t
for integer
t
and spectral density
is continuous and nonzero on
v ' for k· 0,1, ... ,
k
lations of X so that
t
and
Let
be a covariance
s
such that
[0,1].
be the autocovariances and
r k = vk!vo• The inverse autocovariances of Xt
rk
the autocorre-
are defined by
and the inverse autocorrelations by
•
The researeh in this report !Vas partia'tl,y supported by the U. S. Air'
FOr'CB Office of Seientific Resear'()h under' Contm(ft No. AFOSR-68-1416.
2
Thus
s
r;
are the autocorrelat1ons of a time series with spectral density
-1
Interest in
r;
arises from the following three properties, which
will be illustrated in the remainder of the paper.
(2)
Some characteristics of
in
r ,
k
are better perceived in
r;
than
particularly if fitting a moving-average, autoregressive
model to
(3)
X
t
X
t
is the goal.
The knowledge and intuition that one has of the relationship
between time series and their autocorrelations will serve to
interpret the inverse autocorrelations.
That is, no new body
of knowledge regarding the relationship between a time series
and its inverse autocorrelations need be developed.
(4)
r;
2.
is easily estimated.
UsING ESTIMATES OF THE INVERSE AUrOCORRELATIONS TO
AID IN THE IlENTIFICATIOO OF A PARSIKWIOlS,
KMNG-AVERAGE, AUTOREGRESSIVE K>JEL.
The most common method of parametrically estimating, from a sample
Xl' ••• '
x..r,
the mechanism generating X
t
is to assume it satisfies a moving-
average autoregression (which will be abbreviated to
where
e"
B is the backward shift operator defined by
MA),
BX
t
== X - ;
t l
Rt ,
the
residual process, is a sequence of independent (normal) random variables
with
ERt = 0
nomials in
and
ER;. (12;
and
a(D)
and
pCB)
are products of poly-
B each of whose roots lie outside the unit circle.
a(B)
is
3
e
the autoregressive part of the model and
)J(B),
the moving-average part.
For example. the model might be
Having chosen the particular type of MA to be fit, 'the unknown parameters,
that is
0
2
and the coefficients of the specified polynomials, are estimated.
Accounts of fitting these models may be found in the papeTS by Box and
Jenkins and the paper by Bannan cited in the bibliography.
The initial step of identifying the particular type of MA to be fit is
an important one.
It is important to keep the number of parameters as small
as possible while leaving enough to be able to get a good approximation of
the truth. for each added parameter makes understanding the likelihood or
leut squares function more difficult.
Calculating and plotting estimates
!
T
of
r
k
is a good way to begin the identification procedure, for much can be
said about the relationship between
nomials
T
I
x2
tol t
a(B)
and
)J (B) •
r
k
and the coefficients of the poly:-
Stralkowski and Wu (1968) have developed charts
to aid in understanding this relationship, particularly for low order models.
calculating and plotting estimates
A
P
k
of
P I
k
the partial autocorre-
lation between Xt
end Xt _k given Xt_l' ••• ,Xt-k+l' bas also been advoA
cated (Box and Jenkins I 1967). One use that can be made of Pk is to choose
•
the value of
(6)
a
that would be needed if the autoregression
•
Rt
4
e
is fit.
(Any series whose spectral density satisfies (1) may be approx-
imated by such an autoregression with
should fit well if
A
Pk
is nearly
a
0
euffic1eatly large.)
for
employed in Section 3 to form ext1JDates of
eating this value of
is given by
A
r •
k
does not hold for
a,
k > a.
r;.
This use of
Such a model
A
P
k
will be
However, other than indi-
A
Pt seems to add little information beyond what
Furthermore,
A
Ptt
suUers from the disadvantage that (3)
Pt. This po1D.t will be illustrated in the examples of
Section 4.
That (3) holds for
lk-
can be seen by denoting the autocorrelations
and inverse autocorrelations of the MA in (S) by
respectively.
__
and
r;(a, 1.1)
Now the spectral density of this MA is
Thus
,
J;
r;(Ol, \1)
Thus in practice, estimates
•
r (Cl, 1.1)
k
J;
•
A~
e2wikf .-l(f)df
of
-
rk
.-l(f)df
from the sample
Xl' • • •
,X-r
be-
having like the autofJ017Blations of an MA with autoregressive polynomial
pCB)
and IIOving-average polynomial
Ol(D),
suggest the MA in (5) be fit to
5
Xt"
Furthermore, the autocorrelation charts of Stralkowski and Wu may be
used to aid in interpreting
A-
r
k
simply by interchanging the words "auto-
regressive', and "moving-average".
For example, if
Ar
k
::::: r k
where
Il' I
< 1,
a first order moving-average
is suggested since
gression.
r
k
are the autocorrelations of a first order autore-
A-
for k > 1 then a first order autore-
and
If r 1 • r
gression
.,
.i
t
is suggested since the above values are the autocorrelation, of a first
order moving-average.
Of course, in these simple examples
A
r
k
would serve
equally well to indicate the model.
Calculating and plotting estimates
since
A-
lk
A-
r
k
supplements the use of
can reveal effects not apparent in
opinions formed from
A
r
k
A
r
k
If
X
t
about the choice of the MA model.
EsTIMAnm OF n£ INVERSE AUroCORRELATICWS.
is the
a-th
order autoregression in (6), then
a
•
l
jaO
a2
j
r
k
or can help to solidify
illustrated in the examples of Section 4.
3.
A
This will be
6
e
for
k" 1 •••••a,
Section 2, a
Qo" 1.
where
and
r; .. 0
for
k > a.
may be chosen in practice by examining
~k.
As stated in
The choice of
a
-~
can also be guided by examining the variance of the fitted residuals from
the model in (6);
a
would be chosen to be the point where the variances
cease to be significantly reduced.
X •••• ,Xwr
1
sample
A
Qj'
by
-
rk
Then if
I
j-G
A
Q
o
= 1.
If
will be biased.
a
are estimated from the
j
can be estimated by
a
where
Q
~2
j
is chosen too small, the resulting estimates
If
is chosen too large (wi th respect to
a
T.
ber of observations) the estimates will have large variances.
practice, it will sometimes be worthwhile to form
values of a.
A-
If the
A-
the values of
r
r
k
A-
r
k
r
k
the num-
Thus in
for a few different
resulting from a particular
should not change drastically when
k
A-
a
a
are reliable.
is increased by a
moderate amount.
Estimates
A
Q
j
recursive formulas.
assuming X
t
is an
j .. 0 ..... n.
where
in n
A
Pk may be got by using the LevinsOD (1949. p .147)
and
This is more easily explained if the estimates of
n-th order autoregression. are denoted by
A
A
Q
Q
o(n) = 1. Then
(n)
A
IX (n)
n
.. -
I
k=O
n-l
i
k=O
A
for
may be calculated recursively
using the formulas
n-l
e"
j
A
IXj(n)
Qj'
A
a.. (n-1)v -k
K
n
A
A
ak(n-l)vk
7
and
=
Pk
~j(n-l) + ~n (n)~n-j(n-l)
for
j • l, ••• ,n-l.
is estimated by
1\
1\
Pk
1\
a
j
•
- 'it(k) •
are appro ximate 1y, if t h e roots
0
f
\,a a Bj
L.j=O
j
are not too close
to the unit circle, the least squares estimates got by finding the minimum
The least squares estimates are approximately the maximum likelihood estimates if Rt
is normal.
squares estimates of
estimates if· R
t
al"" ,aa"
If
1\-
Thus
are approximately the least
r;: , •• " ,r; and approximately the maximum likelihood
is normal, since
Rt
1\-
r l' • " • ,ra
r;:, ••• ,r~ is a one-to-one function of
is dangerously nonnormal, then
X
t
should be transformed
or some modification of the least squares procedure used (Anscombe, 1967).
The remainder of this section will be devoted to the asymptotic joint
distribution of
R
(7)
t
1\-
1\-
rlt ••• ,r •
a
It is assumed that X
t
is i.Ld. with
An easy computation shows that
of
a
j
and
r;
ER: <
satisfies (6) where
co.
and therefore
1\-
r
k
described by Walker (1964, p.366, (2».
are the estimates
(6) and (7) insure
that assumptions (1) and (5) on p.366-1 and the conditions of Theorem 2,
e
.
p.31l of (Walker, 1964) hold.
plained by Walker on p.3S3.)
implies the following results:
(Assumption (3), p.367, is not needed, as exThus Walker's Theorem 2 is applicable and
8
are each asymptotically normal with mean
0
and
c~variance
r1
matrices
and
r 2 respectively; let y~k and y~k denote the k,j-th elements of
r 1-1
and
r 2-1
respectively and let
a(B) == 1
+ alB + ... + aaB
a
then
and
alogla(e 21rif )!2
ar;
An easy calculation gives
This result, together with an application of the chain rule yields
where the k,j-th element of
Ii· is
Thus
e
r1
can be expressed in terms of
adopt the convention
a
j
• a
-
a , r ,
j
k
-
and
for all integer
va.
j
To simplify notation,
with
j
< a
or
j
> a.
9
Then
(Xj+k + a j _ + 2a j r
k
v
r2
and the k,j-th element of
k
o
is (cf. (Parzen, 1961, p.968»
a
f
n:l
(a
a
- a
a
I).
k-n j-n
n+a-k n+a-j
4. E"xPwLES.
Table 1 shows the first 18 autocorrelations, partial autocorre1ations,
and inverse autocorrelations of a time series.
From the autocorrelations ,
it is clear that the model has an autoregressive part, presumably with a
first order term that is causing the alternation of +
and
-
signs.
The
partial autocorrelations reveal more; there is no moving-average part and
6
is the highest order autoregressive parameter needed.
•
would adequately describe the series.
That is, the model
R
t
The inverse autocorrelations lead
also to this last conclusion but in addition suggest strongly that
a2
=a3
0=
a4
= as = 0,
since
r
k
are the autocorrelations of a moving-
. average
.
xt •
The values of the table are, in fact. 'those of the autoregression
10
Table 1
e
Lags
1-9
10-18
1-9
10-18
Autocorrelations
.60
.18
- .84
- .46
- .84
0
- .03
- .41
- .30
.13
- .38
0
.36
.61
.79
.67
- .65
- .70
Partial Autocorrelations
.32
.33
- .31
- .4
0
0
0
0
.68
.33
- .80
- .54
0
0
0
0
0
0
0
0
0
0
0
0
Inverse Autocorrelations
1-9
10-18
.42
0
0
0
0
0
0
0
.17
0
The inverse autocorrelation of order
5, .17,
.24
0
results from the 'interaction t
between the first and sixth order terms of the model.
e~
metry between
r
last example,
Pk
k
and
r~
does not exist between
rk
Note that the symand
Pk'
In this
is not the autocorrelation function of the above moving-
average.
An example of the use of the inverse autocorrelations in identifying a
time series model is provided by the monthly inward station movement data
in (Tiao and Thompson, 1970).
The reciprocals of the original
215
ob-
servations were taken to remove the trend in the size of the seasonal oscillations.
These transformed variables are the monthly average times between
installations.
Then a second order polynomial was fit to remove the trend
in the level of the reciprocals.
Let
X
t
be the residuals after this re-
gression.
Table 2 gives the estimates
The seasonal component of order
A
r
k
and the high values
A
r
l2
and
12
= .74,
A-
r (with
k
a = 36)
is evident from the oscillations in
A
r
24
= .57,
A
r 36 • .49,
and
These last four values suggest that the seasonal component can be explained
by an autoregress i ve term
1 + a Bl2 '
lZ
The decrease i n
r~
l2k
i s a li tt1e
slower than geometric, but these autocorrelations will be affected by terms
of
other orders.
Furthermore, the inverse autocorre1ations sugges t
AA+ (112B12 since Aare close to O.
r
= -.21 while r 24 and r
12
36
1
The values of
18.
A-
rk
greater than
.1
1, 3, 9-12,
occur at lags
and
Some of these, however, might result from the interaction of low order
components (month to month) with the very strong seasonal component.
To
get a cleaner look at the nonseasonal components, the model
•
Rt
was fit and the fitted residuals
were analyzed.
Table 3 gives
•
A
r
36
• .03
and
A
r
48
A
A
r , Pk
k
and
A-
r
k
(with
a=24)
(1)
for Rn
•
In addition
= .03, which indicates the seasonl4 component has been
12
RU )
Table 3:
e
A
Lags
1-12
12-24
r
.30
.01
.23
.17
.36
.02
.12
-.10
t
A
(v =26.5 x 10
k
O
.23
-.16
.21
-.27
.00
-.17
-12
)
.16
-.1
.19
-.09
.05
-.09
.29
-.01
.09
-.02
-.12 .08
.05 -.01
.11
.19
-.02
.00
.22
.15
-.11
.07
-.12
.16
.04
-.24
-.09
.07
-.05
A
Pk
1-12 .30
12-24 -.05
.15
.01
.29
-.04
-.08
-.19
.15
-.26
.03
-.17
A-
rk (9=24)
1-12 -.09
12-24 -.04
.12
-.11
-.17
-.08
-.03
.06
A-
adequately accounted for.
rk
-.16
.12
for
-.05
.21
-.05
.04
k == 1, ••• ,6
.05
.10
-.11
suggests that the low order
components can be accounted for by an autoregressi ve term
(8)
A-
r
1
has been reduced to
-.09,
whereas it was
-.29
for
X •
t
There are
two possibilities.
One is that a first order autoregressive term
not present in
and
X
t
A-
r
1
== -.29
•
is being obscured by the interaction of higher order terms.
r
1
• .30
(1)
Rt
on
A-
r1
But the value
suggests the latter so a first order term was added to (8).
high order components in
is
arises from the interaction of other
terms; the second is that the effect of a first order term in
A
alB
The
R~l), particularly those of orders 11 and 18,
were ignored for the moment and the model
=
Rt
13
was fit.
A
The parameter estimates are
A
aa = -.291,
and
A
as
A
R(2)
n
A
r , Pk
k
11.-
r
k
1
Table 4:
A
r
Lags
.11
.16
l
"
-.188,
k
r
(with
k
a-24) for the
The low order components seem
In all three functions
11
X
A
A
r , Pk
k
18. Therefore, the
18 was fit to R~2)
B
and
+ aliB ll + a 1s
with the result that the final model for
1-12 -.02
13-24 .05
a
11.-
and
the two largest values occur at lags
, autoregressive model with polynomial
e'
A
-.791,
from this last model.
to have been adequately accounted for.
and
"
l2
= -.18.
Table 4 gives the estimates
fitted residuals
a
is
t
R(2)
t
(v .19.4 x 10
O
-12 )
.03 -.06 -.04
.00 -.06 -.06
.00 -.08 -.11 -.24 -.18 -.08
.13 -.06
.00
.02
.24 -.07
.10
.07
.14 -.04
.08 -.01
.21 -.08
.09
.07
A
Pk
1-12 -.02
13-24 .02
.11
.18
.03 -.07 -.05
.01 -.04 -.07
.01 -.11 -.13 -.23 -.14 -.15
A-
rk (a=24)
1-12 .09
13-24 -.02
.01 -.03 -.04 -.04 -.1
-.14 -.02
.05
.10
.19
Table 5 gives
A
A
r • P
k
k
and
11.-
r
k
(with
-.03
.13
.02 -.14
.11 -.04
.03
.00
.11
-.19
-.08 -.05
a=24) for the fitted residuals
R~3). The estimates seem to indicate that there is at least no major disorder in the residuals.
14
e
R(3)
t
Table 5:
1\
Lags
r
1-12 .06
13-24 -.03
.16
.12
.03
-.04
.02
-.09
1\
= 11.9
(v
k
O
-.04
.02
-.02
-.12
x
10
-12
)
-.03
-.18
.03
-.07
-.02
-.12
-.03
.14
.01
-.13
-.06
-.01
.18
.03
.00
.15
-.16
-.04
.14
.07
.04
.02
-.12
-.02
-.03
1\
Pk
1-12 .06
13-24 -.02
.15
.13
-.01
-.14
.01
-.06
-.05
.04
-.01
-.12
1\-
r
1-12 -.06
13-24 .00
e"
-.17
-.13
.02
.06
-.08
.10
5.
.05
-.08
k
1\
s(f),
.04 -.0'
(a=24)
.05
.06
-.02
.11
-.07
.02
-.11
.13
.00
.05
TopICS FOR FURrHER STUDV.
Another possible method of estimating
by
.04
r; would be to estimate
s(f)
one of the standard nonparametric estimates got by smoothing the
periodogram, and then es timate
r;
by approximating
J~ ~-1(f)e2.ikfdf.
I do not know the properties of such estimates or how they compare with the
estimates of Section 3.
The definition of inverse autocorre1ations can be generalized to mu1tivariate time series.
But more discussion is needed about the relationship
between the correlation matrix function and multivariate, moving-average
autoregressions.
15
BIBUOGRAPHV•
Anscombe, F.J. (1967) Topics in the ilvestigation of linear relations
fitted by the method of least squares. Joumal of the Royal.
Statistical Society (B) 29, 1-52.
Box, G.E.P., Jenkins, G.M. and Bacon, D.W. (1967) Models for forecasting
seasonal and non-seasonal time series. Spectml Analysis of
Time Senes, ed. B. Harris. Wiley, New York.
Box, G.E.P. and Jenkins, G.M. (to be published)
Control. Holden Day, New York.
Models for Prediction and
Hannan, E.J. (1969) The estimation of mixed moving nerage autoregressive
systems. Biometnka 56, 579-593.
LeVinson, N. (1949) The Weiner RMS Error Criterion in Design. Appendix B
in E:.ctrapolation.l Interpolation.l and Smoothing of Stationazry
Time Series by N. Weiner. Wiley, New York.
Parzen, E. (1961)
An approach
~
time series analysis.
matical Statisti08 :iL., 951-989.
e'
AnnaZs of Mathe-
Stralkowski, C.M. and Wu, S.M. (1968) Charts for the interpretation of low
order autoregressive-moving average models. Technical Report
No. 164.1 Depazrtment of Statistics.l University of Wisconsin.
Tiao, G.C. and Thompson, H.E. (1970) Analysis of telephone data. TechnicaZ
Report No. 822" Department of Statistics" University of Wisconsin.
Walker, A.M. (1964) Asymptotic properties of least-squares estimates of
parameters of the spectrum of a stationary non-deterministic
time series. Journal of the Australian MathematicaZ Society
IV, 363-384.