'f
, ~
DISTRIBUTIONS OF SOME SERIAL
CORRELATION COEFFICIENTS
by
Mohammed Moinuddin Siddiqui
University of North Carolina
Special report to the Office of Naval
Research of work at Chapel Hill under
the contract for research in probability
and statistics. Reproduction in whole
or in ~art is permitted for any Durpose
of the United states government.
~;"
0'
,'.
... ..
,-,.
'{
f'f,.--'
of Statistics
Series No. 164
January" 1957
Institut~
Mimeogra~h
ii
ACKN01AJLEDGEMENT
I wish to ex?ress my feelings of gratitude to
Professor Harold Hotelling for his thought provoking
suggestions and valuable advice at various stages of
this research.
I also wish to express my thanks to the Institute
•
of Statistics and to the Office of Naval Research for
providing me With financial and other facilities, to
Mrs. Hilda
~attsoff
for careful typing of the manuscript
and to Miss Elisabeth Deutsch for helping me in reading
and correcting proofs.
M. Moinuddin Siddiqui
iii
TABLE OF OONTENTS
..
CHAPT'I:'R
.
. . ..
ACKNO"\tJLEDGEMENT
TABLE OF
NOTATION
I.
•
.
II.
CONTE~ITS
..
TEST CRITERIA
41
•
•
•
•
•
•
•
•
ii
•
•
•
•
•
•
..
•
iii
.. . ... . . ..
"
......
1
. ..
1.
Introduction and summary
••••••
2.
The distribution of a sample in an autoregressive
scheme
• • • • • .
3.
Least-squares estimates
4.
Likelihood-ratio criteria
1
4
13
. ... . ..
16
DISTRIBUTION OF A FIRST SERIAL CORRELATION COEFFICIENT NE.AR THE ENDS OF THE RANGE
Introduction
2.
Geometrical representation
4.
5.
6.
·.. .
. ....... .. .
1.
3.
III.
PAGE
21
21
...
·...
Approximation to the value of r * • • • . .
Integral expression for p(r*~ r O) •• . . . . .
Discussion about g(\)
. . . . ..... ...
The value of r * near the curve
7. Bounds on the integral IN
•••.••
8. Bounds on Per * ~ r o)
..•..
?4
29
34
37
39
42
...
55
JACOBI POLYNOMIALS AND DISTRIBUTIONS OF ST£RIAL CORREtATION COEFFICIENTS
•
•••••••
f
•
•
•
•
•
•
58
iv
CHPPTER
PAGE
1.
Introduction. • • • • •
2.
Jacobi polynomials
II
...
• •
•
•
•
• • •
... . . .
58
59
3. Distributions of certain statistics as series of
•••••.•
4.
Independence of r l
and p
5.
Cumulants of ql
6.
~symptotic
•
•
•
• • • • •
•
•
•
63
•
•
66
. .. .
.., .
• • • •
0
•
11. Correction for the sample mean
12.
IV.
,
.. . . . . .
Distribution of the circular serial correlation
coefficient • • • • • • • •
• • • • • • ••
••••••
....
• •
67
73
75
78
81
87
90
92
97
DISTRIBUTIONS OF PARTIAL fiND MULTIPLE SERIAL CORRELATIOW COEFFICIENTS
....... .
. , . . . . .. · .
... . . ..
1. Summary of the chapter
.....
.....
2.
Cumulants of qs
3.
The exact distribution of r 2s when N is even
The distribution of r
2
Bivariate moments of r l and r 2 ••••
4.
.
•
• • • • • •
normality of q1
13. Concluding remarks
.'t
Ii
7. Moments of r l
••••
• •
8. Approximate distribution of r l •••• • •••
9. Asymptotic validity of the distribution of r l •
10 Some other properties of the series (8.6) • • •
•
.
J Beobi polynomials
5.
.,
~
... .. . ...
99
99
100
103
1°5
108
6. The distribution of partial serial correlation
coeffici ant • • • • • • • • • • ••
••••.•
111
v
CHAPTER
•
PAGE
1. The distributi.on of multiple serial correlation
coefficient • • • • • • • • • • • •
...
9.
v.
Bivariate distribution of r 1
Concluding remarks • • •
2
119
•• •
120
.. . ..
123
LEAST-SQUARES ESTIMATION lNHF.N RESIDUALS ARE CORRELATED
124
......
1.
Summary
•
124
2.
Least-squares estimates and their covarianees ••
124
3.
Bounds on a quadratic and a biU.near form •
130
•
•
t
•
•
.. , .
•
,
•
•
•
•
•
•
.. ·..
Linear trend .' • • • . . . .
·..
6. Trigonometric functions
...
..
Concluding remarks . . . . . .
..
BIBLIOGRAPHY • • • • • • . . . . . . . . . . . . · . .
Bounds on covariances ,
..
and r
"
.0 • •
10.
..
... ...
8. Extension to multivariate case
115
135
137
140
143
145
vi
NOTATION
•
The following notation and convention will be used, any departture being indicated at the proper place •
•
Numbers in
L- 7 refer
-
to the bibliography.
"Variate" stands for "random variable"
IIN(IJ.,o-) variate" means "a variate distributed normally with mean
and standard deviation
(J.ll
"p(S)tr stands for "the probability of the statement S."
Cx"
It
•
"a
X
\I
II
rlX
stands for "the expectation of the variate x."
stands for "the standard deviation of the variate x."
=
x r , i.e. the rth moment of the variate x.
G
x.
rt'x denotes the rth cumulant of the variate
f(x) denotes the probability density function of x.
I':
denotes the cumulative distribution function of x.
F(x)
¢x (t)
denotes the characteristic function of x.
Xx (t) denotes the cumulant generating function of x.
.OV
stands for "is approximately equal to. I'
~
stands for
o(r.. ) stands for
lIis asymptotically equivalent to."
"a quantity of the order of magnitude of L. II
If A denotes a matrix then AI
If m is a positive number
...
'.
than or equal to m•
L-m_7
denotes its transpose.
denotes the largest integer less
IJ.
CHAPTER I
TEST CRITERIAI
..
1.
Introduction and summary.
The distribution theory of serial correlation coefficients has
received considerable attention lately. R. L. Anderson /-2 72 obtain-
--
ed exact distributions of serial correlation coefficients defined oircularly at Hotellingts suggestion.
The distributions thus obtained are
complicated and are given by different analytical functions in different parts of the range.
- -7 of smoothing
Following Koopmans l method /-29
the characteristic function of the variate, a simple approximation to
the exact distribution of the circular serial correlation coefficient
- -7 and Rubin -~4S-7 for uncor-
with known mean was found by Dixon /-9
related normal variates.
This aoproxirnation is accurate for large
sample Size, N, in the sense that
fi~st
N moments of the serial cor-
relation coefficient agree with the moments of the fitted distribution.
--7 extended
Leipnik r31
-
Dixon's distribution, following a method due
-
to Madow /-32 7, to the case of a circular autoregressive scheme of
-
order 1.
Quenouille L-43 7, T'Tatson ~S2 7 and Jenkins /-21, 22_7, all
-
- -
-
adopting the circular definition and the method of smoothing the characteristic function of the variates, have made considerable progress
...
•
lSponsored by the Office of Naval Research under the contract
for research in probability and statistics at Chapel Hill. Reproduction in whole or in part is permitted for any purpose of the United
States Government.
2Numbers in the square brackets refer to the Bibliography.
towards obtaining the joint distribution of several serial correlation
coefficients under null and non-null hypotheses.
.
Daniels
L-8_7,
More recently
by the application of the method of steepest descents,
obtained the already known results and some new ones for circular and
non-circular statistics.
It is difficult to give a comprehGnsive history of the work
done in this field as the subject has been under extensive research
by scores of mathematicians and statisticians.
Since the present study
is confined to the distribution of non-circular serial correlation coefficients under the null hypothesis, no attempt has been made to include all the names of the persons who have contributed to the problems of estimation and testing; nor is any history of the classical
approach to time series analysis included. However, the bibliography
includes papers and books dealing with the general problem of time
series analysis. The names of B3 rtlett
Von Neumann £49, 50,
Kendall
L-'?4, 2,_7,
,1_7,
1IIlhittle
Mann and T.Tald
L-" 6_7,
L-53_7:;
['33_7
Quenouille L-4l, 4~h
T. 1,11. .~nderson
L-3, 4_7,
may be specially no·ted in
this respect.
The major part of this paper deals with the distributions of
serial correlation coefficients and their functions, in uncorrelated
normal variates with known means.
Since these coefficients are inde-
pendent of scale parameter there is no loss of generality in assuming
the variance of the normal variates to be unity.
In this chapter the distribution
~f
the sample from autore-
3
gressive schemes of orders land 2 will be obtained.
A method of in-
verting the covariance matrix of the sample from a general order autoregressive scheme will be suggested.
Estimates, which are approxima-
tions to least-squares estimates of the parameters in the autoregressive schemes of orders 1 and 2 will be ?iven; and statistics' for testing the null hypothesis,which are approximations to the likelihoodratio oriteria, will be constructed.
These are functions of first and
second serial correlation coefficients.
Two alternative definitions will be proposed for the first serial correlation coefficient.
r* is the ordinary correlation coeffi-
cient between the sets (x ,x 2 , ••• ,x _ ) and (x2 ,x , •.• ,xN) except that
N l
l
3
the deviations will be taken from the population mean, which is assumed to be
zer~
rather than the sample mean.
rl
is the ratio of two
quadratic forms obtained by replacing the denominator of r* by
N 2
i • Bounds for P(r* > r ) . will be obtained in Chapter II, using
Ex
i=l
a geometrical method.
- o
A geometrical. approach was introduced into
statistical theory by Fisher
~12,
13_7 to obtain the distributions
of ordinary, multiple and partial correlation coefficients.
geometrical methods have been utilized by Rotelling
taining some important distributions.
~17,
Other
18_7 in ob-
He also shows how to determine
the order of contact of frequency curves of some statistics, with the
variate axis near the ends of the range, even though the actual distribution is unknown.
In the third chapter a method will be developed to obtain dis-
4
tributions of serial correlation coefficients including those of partial and multiple serial correlation coefficients.
Each of these dis-
tributions is represented asa product of a beta distribution and a
series of Jacobi polynomials.
The Hermite polynomials which are
associated with the normal distribution have long been in use in distribution theory.
Hotelling suggested the use of Laguerre polynomials
for approximating the distribution of positive definite quadratic
forms.
Similarly, series of other functions such as Bessel functions
have been utilized for such
pt~poses
L-15_7.
The theory of Jacobi polynomials is well developed and many results will be quoted from Szego ~47_7 without pro9f.
The same tech-
nique will be extended to obtain a bivariate distribution in Chapter
IV, and could be extended to multivariate distributions as well.
Since
the knowledge of moments is necessary for this purpose, a considerable
part of the third and fourth chapters will be devoted to calculating
the cumulants and moments of the variates.
A comparison with the al-
ready known distributions will be made to demonstrate the power of
the proposed methodo
In Chapter V, which is the last chapter, least-squares estimation of regression coefficients will be considered when the hypothesis
of the independence of residuals is not satisfied.
2.
The distribution of a sample in an
au~~regressive
scheme.
Let xl' • '," xN be a realization of a time series at time
t
= 1,
••• , N.
It is assumed that the underlying model is an auto-
5
regressive scheme of order k, i.e.,
where St
is independent of x t _l ' x t _'2'
... ,
and each e is a
N(O, a) variate, independent of all other ets.
In the case of a j = 0 for j
instBi."'l.ce, 1:2°_7) for all t
>
1, it is known that (see, for
,
2
e 2
where Cl'x:::
C; xt
and is independent of t.
the distribution of
(2.2 )
xl' •.• , xN
is
It is also known that
~iven
by
'2 -N/'2 ( l-a '2 ) 1/2
dF ( xl' ... ,xN) ,., ( 2na)
l
_
ex·pL -
1
(N 2
2 N...l 2
N-l
7.
~ xi -2a l ~ xixi+lS_7dxl ... ebeN •
2r:i l ff~ctl
We will study a second order scheme, i.e., a.=O for j > 2, in more
J
Changing the notation we put a
1
an autoregressive scheme of order 2 is
detail.
=a
and a 2 =
-~
so that
Kendall L-26, Vol. II, p. 406 ff._7 has shown that the most general
solution of (2.3) in our notation is
6
(2.4)
Xt=~
Here
t/2
'/2
2~J sin jQ
(A cos t ~+B sin t 9) + j:O (4~ _ a 2)r/"/. 6 t _j +l
00
A and B are arbitrary constants and 2 cos Q = a~-1/2, ~1/2
is taken with positive si(ln and it is assumed that
assume that 0 <
~
< 1,
4~
2
> a • We also
the first ineauality being necessary for a
real non-vacuous prooess, and the second ineauality insuring that
X
t
will not increase without limit.
proc~ss
Further, if we suppose the
to have been started a long time
a~o,
then the first term on
the right hand side of (2.4) may be considered to be damped out of
existenoe.
Thus
The justification of representing xt as a sum of infinit~ly many
random variables is provided by the following theorem taken from Doob1s
Stochastic Pr-ocesses'/-lO 7,
-----------"THEOREM.
page 108:
Let Y1' Y2' ••• be mutually independent random variables,
2
'2
with variances 0'1' 0'2' ••• • Then, if
00
~
\ n)
converges,
00
and if
~.x j = ~E t y
n=l
xn
=
2
(J
< co
Z y =x is convergent with
1
n
probability 1 and also in the mean.
E
2
~n
00
00
and if Z E ~ y
1
i
Moreover
2
2
t
;
E t x J-E tx} =
n
= 1~ [Yi-E fYil 7
.J-
P~L.U.B.lx(~J)1
> e}~0'2/e2.
\..
n
n
f_
2
ct
7
Conversely, if (.i.m
n-> 00
in Y = x exists,
j
i a2
00
the series
j
00
and
i E 1Yj ~ converge."
We need only to point out that in the notation of the theorem
above
,
_.
co
[xt =
(2.6)
E [Yj
j=O
=0
,
Furthermore
(2.8)
-t
Ex
x
t t+k
=2
a~ k / 2 2
a;-
(4t:l_a2)
p
-
cos kr.e...
1 ...~
cos kr.e'" tQ cos (k - 2) e 7
1 - 2~ cos ?Q + f3 ~ - •
If we define the autocorrelation coefficient of xt
(2.9)
we have
p
It-st
-r
=
-7
26
(2.10)
Pk satisfies the equation
(2.11)
and X
s
\
as
8
with
Q.
'u
=
1,
(J
- "
- t - '"' t •
From (2.11) taking k=t
~
and
in turn and solving for
8
a
and
we get
Pt
a ==
s-2
- P s P t-2
t-1 p s-2 -
p
~
(J
p
t-2 PS - 1
t P 8-1 - p t-1 P s
p t-1 p s-2
t-2 p8-1
(J
:--p
...
In particular
(2.12)
a ::
2
P1 ....
=
~
.,
.... -
:HE0R~
O
2
P2
1
I
2.1. Let A be the covariance matrix of x =(x1,x 2' •.• ,xN)
where the model is (2.3), so that
(2.14)
A = (J2
x
r...
i
j
1
P
I
1
I °2
Cl
O2
.•
p
1
PI
•
.
P
PI
1
lp I/-~
1/-1 P
then
N-2
PN-3
.
• PN-3
N··l
1
I
I
9
Ii
-1
1
=2
1
-a
-a
l+a
~
-a-a~
o
~
-a-a13
o
o
o
o
o
o
o
C1
o
~A!
(2.16) (ii)
=
2
(1-~ )
~
0
-a-a~
~
2 2
l+a +13
o
o
o
o
o
o
·..
-a-ap
2 2
1+a +13
·..
·..
o
-a
-a
1
2 {<_ 1+~.,~)
t
-a J
and
(iii)
(2.17)
PROOF.
if x
I
denotes the transpose of X , its distribution is
-r
dF ( xl' ~ ••• xl\T )_(
- 2n )-N/2 II A I -1/2 exp_/- - 1
Pi
=a
'::\_1 - ~ P i-2' 1:=2,3, ••. ,N-l
PI = a - ~ -P1
and
A-Ix_ 7dx
The proof of part (i) is completed by actual multip1ic~ion,
taking into consideration that
(2.18)
f
,
10
The nrocess by which we arrive at
A-I is the following.
The distribution of (S3, ••• ,eN) is
2· -(N-? )/2
dF ( &3'" .,eN) =( 2no )
Sunnosing xl
./-
exp_ -
I
';)(l
N 2 7
~ 6 i _ de) ••• deN •
and x 2 given, make the transformation
The Jacobian of the transformation is 1, hence) given Xl and x 2 ' the
distribution of (x ' •.. , xN) is
3
Now Xl
where
and x 2 are distributed as bivariate normal given by
PI
=
~, and ax
eXPL-
is given by (2.7), hence
2~2 {(xi+X~)(1-~2)-?a(1+~)xIX2}_7rlx1 dx2
Multipl~Ting (2.19) and (2.20) vie get (2.17) and also the value of
•
IA I .
11
Thus parts (i1) and (iii) are proved.
Another method.
l~Ti
th the help of (2.19) we write the matrix A-I
,
with the first two rows and the first two columns blank and to be
filled in, as
r---
*
*
*
-1
A
1
=~
c
il-
*
*
*
*2
* ·...... '*
* -·...... '*
*
2
1+0: +~ -a-a~
*
2 2
Ha -+p
-a-aa
*
.
J,fo
e
tl$o
... 0
... 0
'*
*
*
i~
*
*
0
0
0
0
0
0
•
*
*
*
0
0 • • • ,,".
0
0
0
0
"a"'a~
·...... a
· ... ... 0
"
1+(12+~2
...a...o;~
~
-a-np
1+0;2
-a
~
-0;
1
Now, we observe that the matrix
A is persymmetric, i.e.,
symmetric about both diagonals, therefore A- l has the same property.
l~Te
fill up the first two rows and the first two columns with the help
of the last two rows and the last two columns, only writing them in
reverse order, and obtain (2.15).
The proof is then completed by
actual multiplication.
If the underlying scheme is autoregression of order k given
by (2.1), A- l can be obtained easily by this method if N > 2k.
Thus,
(2.21)
dF(xk+l,·",xNJ x1' ••• ,xk )
=
,
12
If k is small we can find the distribution of (X , ..• ,x k ),
1
and by multiplying it with (2.21) obtain not only the exponent but
I AI.
also the value of
In any case, however, we can 1I\Trite A-I with the help of (2.21),
filling in the first k rows and the first
k columns wi'bh the help
of the last k rows and the last k columns.
for
-1
k
A =
Thus, for instance,
= 3,
1
-0:1
-0: 2
-c;.
1+0:12
-0:1-0:10::?
-0:2-C: l c£3
I+CI12+0: 22
-0:1-0:ICI2-0:20:3
-0: 2
-CI1-0:10: 2
-0:
-0:2- CI10:3
3
-0:
-0:1-CI I0:2-0:2CI 3
-0:
3
-0: 2-0:10:
-0:1-0:10:2-0: 20:3
-0:1-0:10::?-0:20:3
2
1+0: 2
+0:2+0:
1 2 3
-0:
0
0
0
0
0
0
0
0
0
0
3
3
2
1+0: 2
+0:2+0:
1 2 3
0
3
-0: 2-0:10:
0
3
0
0
0
0
0
0
0
0
0
0
1
"2
Cf
• 1+0: 2
1
.
-0:1
-0:1
1
.
13
J . lATise
L-54_7 has
given another method of inverting
A
in
the general case.
)_
Least-squares estimates.
In the analogy with the elementarv theory we find formally the
least-s"'uares estimates of the parameters i.n autoregressive schemes
of orders 1 and 2.
This will serve as a guide in constructing simpler
statistics based on serial correlation coefficients.
For a Markov scheme
least-squares estimates (which are also the maximum-likelihood estimates
when xl is supposed to be fixed) of a and
(J
2
are
N-l
a
1
=
i~lxixi+l
N-l 2
2: x.
i=l ~
and
(J .2)
respectively.
The estimate of the variance. of
L. Hurwicz
L-20_7 has
shown that
a
l
is
a is biased in small samples.
l
Mann and Wald ~J3_7 showed that this bias tends to zero as N tends
14
to infinity.
For an autoregressive scheme of order 2, that :l.s,
the least-squares estimates of a,
a2 =
~,
and
~
2
are
..
_1Xi )(~i 2 )-( ~1-~i )(Zxi _2x i _l )
2
2
(
(ZXi_l)(~Xi_2)- ZX _2x _ )=D
i
i l
{ZXi
(~xi_2Xi_l)(zxi ..lxi)-(ZXi_l)(ZXi_2xi)
D
and
respectively, where all the summations extend from 3 to N.
mate of the covariance matrix of
s
The esti-
a2 and b2 is
XXi.. l~i"2
2
B = 1)
ZX 2_
i l
Obviously these estimates are of
litt~.e
practical use.
To
simplify we propose the following estimates instead, which are suggested by and approximated to the least-squares estimates.
1ATriting
N-?
Z x.x +
1=1 ~ i 2
p ==
N 2
Z xi
i=l
let us define a first and a second serial correlation coefficient as
and
if "P -I
o.
We propose r
gressive
l
as an estimate of a
in a first order autore-
scheme~
as an estimate of
(]
2
. Similarly an estimate of variance of
r
l
In the analogy of elementary theory we expect
t*
=
to be distributed approximately as Student t, under the hypothesis
a :: O.
Similarly for a scheme of order 2, given by
a =
b
=
(2.3~
is
16
and
(3.6)
s
2
=
p(1-2ri+2rir2-r~)
- (N-2 )(l-ri)
2
pel - R )
N-2will be our estimates of a, f3
and (J2.
Here we shall define R as
a multiple correlation coefficient between x
ffild
t
(X _ , x _ ).
t l
t 2
It may be noted that
1
rl
r2
r
1
r1
r1
1
1
.".
e
(3.1)
2
1 - R
"'2
=:
:1 I
1
r1
Again, it is expected that when a = f3
=:
0,
2
2
l_R
F*(N-2,2) = ~
-7\N-.:::
R':::
will be distributed, at least approximately, as
F with
N-2
and
2
degrees of freedom.
4.
Likelihood-ratio criteria.
Consider the maximum of likelihood under the following
for an autoretressive scheme of order
2 ~iven by (2.3),
HOO :
a
=:
0,
~
=0
H :
IO
a
r 0,
f3
=:
0
hj~otheses,
17
Notation:
HOI:
a
= 0,
Hll :
a
I
0, ~
~
10
I
0
gij(x) denotes the likelihood under Hij ; and &ij'
B..
.
~J
the maximum likelihood estimates and
/'\
g..
~J
'~he
aij ,
and
maximized
likeli:."1ood.
Since we propose to study only null distributions, i.e.,
distributions of statist:i.cs under H ' we will not oonsider
OO
any non-null hypothesis against another non-null hypothesis.
Furthermore, for mathematical simplifications we shall
assume that either xl' or xl and x 2 ) are givan. This re,
suIts in a slight loss of information but saves many complications.
2)-(N-l)/2
goo ( x2 '·· • ,xNtxl ) = ( 21£0'
~~O = (
2
°10
x~)/(N-I)
6
2
~
2)-(N-l)/2
1- 1 N(
2 7
exp_ - 20'2 ~ Xi -ax i _l ) _
N
= E(x.-ax.
1)2/(N-l)
2 ~
~=
N-l
alo = ( E
1
I
20'
N
glO ( X2 '··· ,xNJ Xl ) = ( 2110'
/-
exp_ - ?
xixi~l)/(
N
Exi)
2
18
Hence the likelihood-ratio criteria Ala for testing HOC against
H is
IO
~
1
---::T"
rv
for large N •.
l-alO
We note that replacing ala by r l results in some loss of
power but since a -"" r l this loss is negligible for large N. HowlO
ever, this or some other approximation seems necessary for further
progress in the theory.
For testing HOO against Hal or Hil we will compare
" ( x3'.' .,xNfxl' x'2 ) • ,,2
gij
0'00 could be considered modified accordingly.
19
Using BPproximat:l.ons similar to those used for
"
~11 :::
•
Thus ~ for testing HOO
against
2
nl
H
OO
against
H
11
2
(4.4)
(l
11
)N--2 ,....
=
1
----2
1 - R
where
HOI' we ha've
1
(l )"N=2
and for
~Ol
and
~Ol' we find
20
Though AOl ' Ala and All are not exact likelihood-ratio oriteria,
and their use ~dll involve some loss of power, it seems that to develop any theory based on serial correlation coeffioients some suoh
approximations are to be used for maximum-likelihood estimates.
Other writers
L-2,
9, 21, 43, 52_7 have used circular definitions of
serial oorrelation coefficients to oiroumvent some of the theoretioal
difficulties. We have preferred to avoid the oircularity definitions
and have dealt with non-circular statistios.
In the following chapters the distributions of these statistics
P(r* ~ r O) has been obis a serial correlation coefficient defined on the
have been obtained.
tai.ned iAhere r *
.Also an upper bound for
pattern of an ordinary correlation coefficient between two series.
CHAPTER II
DISTRIBUTION OF A FIRST SERIAL CORRELATION
COEFFICIENT NEAR THE ENDS OF THE RANGE
1.
Introduction
If xl' x2 ' ••• ,
x~
are observations on the variates Xl' X2 ,
••• , XNt and the population means are known which may, without loss
of generality, be taken to be zero, the most natural definition of
the serial correlation coefficient with lag unity would be
N-l
r
*
1
.L1x.x.
1 1+
=
1=
:=::::;::=:;;::;;=== if the denominator
i
0,
N..l 2
N-l 2
(LX.)( Lx.+ l )
i=l 1 i=l 1;
which is an analogue to the definition of the ordinary correlation
coefficient between two different sets of observations.
However,
instead of taking deviations from the sample mean, we have taken
deViations from the known popUlation means which are assumed to be
zero.
Due to the seemingly insurmountable mathematical difficulties
"
involved in obtaining" the distribution
of r * , several alternative
definitions have been proposed as approximations to r * • The most
suitable among the alternatives seems
to be
22
Even this presents difficulties as far as exact distribution is concerned.
Further, r
never attains the values -1 and 1, its minimum
l
and maximum values being - cos N:l and cos : l respectively.
N
But it
has the advantage of being (i) a good approximation to r * for large
N, (ii) a ratio of two quadratic forms, (iii) 'independently distriN :2
buted of L xi on null hypothesis, and (iv) of not bringing any
i=l
unnatural assumptions about the universe, particularly in non-null
cases, as the circular definition or some other proposed definitions
do.
*
If we consider the cummulative probability curves of rand
r l , for large N, the two curves will be very close in the middle of
the range but will deviate apart toward the upper end of the range.
This becomes clear if we observe that
~ ~
2: x.
i=l
i. e.
two
XIS
~
.1
2
> ~ Xl
- c. •
+
~
Xc>
c;..
2
1 ~
+ •••+ x.. -1 + '7) xN
N
'-
-1
n
~
N-1 2
( r x.)( I x. 1)
>
'1'1 . 11+
J.=
~=
Ir11 ~lr*1 , the inequality being true whenever at least
are different from zero and different from each other.
Hence
for a given number r O between 0 and 1 we have p( Irll ~ ro)<P( Ir*l~ro)'
23
On the other hand we have
2
r1
2
~
(--.)= (1 - ---2)(1
r
*
Lxi
2
Writing "N = (1 -
~2)(1
we observe that if xi are independent
Ex.1
N(O,l) variates
2
and the variances of these quantities are 0(N- ).
to 1 in probability as N
->00.
Hence from section 20.6, page 254
of Cramer's Mathematical Methods of Statistics
the distribution function of r
1
Therefore"N tends
L:7_7,
we conclude that
is asymptotically equivalent to that
of r* •
It is therefore desirable to consider the mathematical behavior
of the distribution function of r * near the ends of the range, and
function of r l for the values of r (or r * )
1
in the middle of the range. However, it may be pointed out that r
l
to use the
distrib~tion
is 'easier to calculate and can be used in the whole of its range.
In this chapter the distribution of r * near the extremities
of the range is considered.
The geometrical approach seemed to be
particularly suitable in obtaining the order of contact of the distribution curve at r *
=+
1.
It will be shown that if for a given
number r O < land close to 1, p(r* ~ r O) is expanded in a series of
powers of (1 - r O)' the first non-zero coefficient in the expansion
is that of the term (1 - r )(N-2)/2.
Upper and lower bounds for the
o
coefficient of this power in the expansion will be calculated.
The
lower bound is positive and the upper bound gives an approximation to
the upper bound of P(r *
2.
~
r O)'
Geometrical representation.
Let Xl' ""X be N independent N(O,l)
N
r
variates.
Define
* = ;:::;;::;:=::;;::;::::
/ N-1 2 N-1 2
/ ( fX.)( fX.
V
1
1
1
1)
1+
if the denominator is not equal to zero, then r * is a variate with
For every set of observations Yl"."Y on these variates, we
N
take a point S in N-dimensional Euciidean space, which may be regarded
as a representation of sample space.
ordinates are taken to be (Y , ,o.,Y ).
1
N
S is determined i f its coDenoting the origin by 0, we
25
see that the points S are distributed with spherical symmetry about
O.
Furthermore, a
uni~ue
value of r * corresponds to all the points
on a straight line OS, excepting the origin for which r * is not defined.
Let the straight line OS meet the (N-l)-dimensional unit
I
sphere in Q and Q , where Q is on the same s ide of the origin a.s S
is.
(2.2)
Denoting by (xl'"
.,XN)
N
\~~ x.2
. 1
:1.=
:I.
the Coordinates of
Q,
we have
= 1,
which may also be taken as the
e~uat1on
of the unit sphere.
points Q and QI may be considered to determine a
uni~ue
The
value of r * •
Considering only the point Q, it is easily seen that the distribution
of Q is uniform over the unit spherej that is, denoting the total
(N-l)-dimensional surface area of (2.2) by SN_l' the probability of
ISN-1'
Q fall ing in an area A on the sphere is A
For a given r 0 in
L:-l,l_7 there exists a set of points on the unit sphere such that
for each point in this set the correspoinding value of r * lies in
the interval ~ro,1_7, and for no other point.
If this set of point
covers an area A on the surface of the sphere (2.2), it follows that
P(r *
2:
r 0)
= A/sN_l
•
26
We observe that r * = 1 if and only if
Xi
= Ax i _l ,
i
= 2,3, ••• ,N
and ~
> 0, Xl~O,
that is,
xi
= Ai-lXl'
i
= 2,3, .•• ,N
and
~
> 0, XliJ O•
Since the point (xl""'~) lies on (2.2), we obtain for the value
whence
I
c = t (1 - ~ 2 ) /( 1 - ~2N).
Denote the variable point (c, hC, ••• ) A. N-l c ) by P and
( - c, - AC, ••• , - AN-l c) by Pl.
of P and
pI
As A varies from zero to 00, each
describes a curve for every point of which -- excepting
the two points of each curve obtained by A =
the value of r * = 1.
° and
00 --
corresponds
27
Since both these curves are exactly alike, except for their
position in space, we confine our attention to the curve
1-1
;:: A xl' 1 = 2, ••• , N, 0
(2.4)
< A. < co
•
Further, from now on we reserve (x 1 ' ••• 'x ) to denote the
N
point on curve (2.4) which corresponds to the parameter value A,
and we use (£l' ••• '£N) to denote any other point on the unit sphere.
To find -the probability of r*
exceeding a given" value
ra
which is close to 1, we consider the points within a "tube" of goe..
deaic radius
9
on the surface of the sphere (2.2) with its axial
curve (2.4) •.
Let the length of the curve (2.4) measured from
Pa(l,a, ••• ,
) be denoted by 8, or more explicitly seA),
l
N
and an element of curve by ~s. Denoting by primes the differential
0)
to P(x , ••• , x
coefficient with respect to s, the direction cosines of the tangent
to the curve at
Pare
... , x
,
,
N
where
xit
= L-(J."-1),i-2
1\
e
de _,1
1 2, ••• , N •
+ ,1-]
I\.
•• QA _/1\ , i= ,
We note that
(2.6 )
N
'2
Zx
1=1 i
=1
,
28
and since
N
2
Z xi:&< 1
i=l
we have
N
I
Z xix i :: 0
(27)
i=l
Let the coordinate axes be rotated, so that the new coordinates
B, ~ and ~denote the column vectors (~l' .'., ~) and (~, ••• ,
rw)
respectively.
The "1
at P and the
axis:i.s how narallel to the tangent of the curve
~2
axis coincides With the line OPe
The (N-J)-dimensional sphere given by the set of equations
"i
with
=
Yi sin 8, i = 3,4, ••• ,N,
29
lies entirely on the (N-l)-dimensional sphere
N
2
N
IT) =1= 1:~2
i=l i
1=1 i
(2.10)
The sphere (2.10) i8 the same as (2.?) with a change of notation.
Each point on (2.9) is at a geodesic distance
measured on the sphere (2.10).
plane
~
Q
from P
Further, since (2.9) lies in the
= 0, this hypersphere is perpendicular to the tangent of
curve (? .)4) at
P.
Changing back to the original coordinates we have
~ = B
,
T) or
,
t i =X1T)1+X iT)2+b31T)3 + ••• + bNiTlN' i
=
1, ••• ,N
•
Eauations (2.9) become
(2.11)
~j
= x j cos
with
Q
+ sin
e
N
1: b ijYi , j=1,2, ••• ,N
i=3
N 2
Z Yi ,.. 1
i=3
3.
The value of
r * near the curve.
tet us calculate the value of r * corresponding to a point
(~"'.'~N) on the hypersphere (2.11),
We have
since
N-1 2
Z
tj
j=l .
=
1 -
~
and
N-1
l:
~ 1
j=l J+
=
2
1 - ';1.
30
Now
N-1
(3.1)
N-l
Z ~j~" i= Z Lx.cOg G+sin
j=l
J~
j=l
J
_
r Xj+lcOS
2
= cos e
N
e Z b i ·Yj. 7
i=3 J N
g+sin e Z b
. Y
i=3 i,J+l i -
7
N-l
z x,x +
j=l J. j l
It is convenient to introduce here a double suffix summation
convention. We will reserve the letter
j
appearing at two places
as the suffix in a product term to represent summation over
from 1 to
N-l.
The letters
i
and
k
appear ing twice as
suffices with stand for summaticn from 3 to
ample
means
and
Now
j
N.
N-l
Z x,x.+
j=l J J 1
Thus, for ex-
31
Therefore,
Similarly
Rememhering that the point
P(x1 , •.• ,XN) is on the curve (2.4) for
each point of which r* ~ 1, we have
(3.4)
32
Since B is an orthogonal matrix, we obtain
b ij xj +1 ~ A bijX j
= - A b~
N
= - A biNc ,
b i ,j+1Xj
1
= ~ bi,j+lx j +l = -
: -f
1~Te
note that
N
b il c •
is not a dUlillTI:y suffix and does not stand for
sUl"f1mation.
Finally from (3.1) -
(3.6) r*:: l-c
r1 bilxl
(3.5) we obtain
33
Now,
,
The coefficient of sin Q cos Q
=0
•
After substituting the values from (3.1) in terms of >..
replacing cos 2e by 1 - sin? 0 and simplifying, we get
and
(
(3.8) 1-;*
sin
e
It is understood that terms of order sin?Q in (3.6) have been omitted
in (3.8) and in the later discussion.
34
4.
Approximation to the value of r * •
As an approximation, replace the terms in the square bracket
in (3.8) by their expectations.
Since Y3,o"'YN
are Cartesian
coordinates of a point on (N-3)-dimensional unit sphere
N ')
2: 'Y'''' = 1
i=3 i
,
we have from considerations of symmetry
;l
t
2
1
Yi = N~' i=),4"";iN ,
Rearranging the terms of (3.8), we get
Since
or
B
is orthogonal, we have
35
Similarly
and
and
where the double suffix summation convention is still being followed.
Hence,
rx
- J
I
I
l+xjx'·
J+
. J+ l -
.X.
Now.
and
(4.4)
Therefore
dx i
i-2
i-I
ax= (i-l)X xl+A
t
dx l
dX' 1=2, ••• , N
7.
36
Furthermore"
XjXjJ>_l
-
Y
and, since
2 N-I 2j-?
= Ax l EI A
•
J=
,
I
A(A 2N- 2_1)
= _.~,
"\ <::'1\;
~
0,
-1
,
x. I = Ax j + x.A ,
J+
J
we have
12
= A(l-x N )
~
from (?6) and (2.7).
f
- A
t
VN
Substituting from (4.4)·(4.7) in (4.2) we get
f
1
Inserting the values of xl' xl' xN' xN' rearranging terms and
simplifying, we finally obtain
37
or
5.
Integral expression for
Per* ~ r )'
o
To find the probability that r* ~ r O' where r O < 1 and
close to 1, we proceed in the following manner. For a given A, r O
determines a unique positive value of sin 8; hence a unique value of
e in,the interval ~O, ~ _7, say eO(A). For
a ~iven
A, the
probability that a random point (~l' ••. , ~N) falls in the elemental
area of the surface of the sphere
Q
ds by
,
f o SN_3
~~i = 1
given by the product of
1
dG
is the ratio of this area to the area of the unit
o
sphere; here
denotes the surface area of
N
~
3
This ratio equals
2
Y'i = sin
2
Q.
,
~(N-2)
sinN-3 0 dQ
21C
N-?
..'1_
=-u.n
2n
sinN-3 e dO
~emembering that for r* ~ I there correspond two curves on the unit
sphere, one traced by P and the other by
I
P,
and noting that
38
changing the signs of x's in (3.6) does not change the expression,
we get
00
P(r* ~ r o)
J jf
N;2
..,
9
•
It may be pointed out that for
0
P.)
N
sin - 3ed9dsb.. ).
°X °° and
=
00,
r * is not de-
fined, but the inclusion of these values in evaluating (,.1) does
not affect the probability as the integrand remains finite near these
values and is zero for X
=0
and
as will be seen later.
00,
N
sin - 30 cos QdQ
(,.2)
cos 13
• N~?
=
s:m
2
eO
:.'
sin 13
/-1- N cos °eo
....
3 sec 60
2~'
(N-2)cos eO -
r
N sinN-' QO
if N is large and eO small.
Therefore,
Per .:: rO
From
.
J
o
00
,'l-
)
--
1
n
(4.,) and (4.9)
*
(l-r )(N-2)/2
(,.3) Per ~ 1'0' ::: __0;....-_ _
n
N-2 BOds ()
A.
s~n
00
Now,
39
Let us denote the integral on -the right hand side by IN
and note
that a change of variable A =- 1/).. leaves the integral l..l.i'1chcn ged;
l
to 1 is the s arne as from 1 to 00. Thus
hence the integr al from
°
(,.4)
IN
l
=2
°
where
~
l"g(\LT(N,.2)!2h(>.)d>.
and
We note here that E.
s.
~eeping ~23_7
has studied a similar
but much less complicated integral.
6.
Discussion about
g(A).
Let us consider the behavior of the function
rm ge
["0, 1_7.
Writing
(6.1)
and
(6.2)
g2 ( y ) -
i
( l-yN-? ( l-yN)
N-- ~ ..
(l-y
)
,
g(A) in the
40
we have
(6.3)
Dividing out the factor (l-y) from the numerator and the
denominator of gl (y), we have
gl
=
l+yN
'2
N-l > 0 for y > 0
y+y ..... ·+Y
1. t
:: Denom~na
or
r y2N-?+?y?N-3 + ••• + ( N-l)yN- ( N-l )YN-2......-2Y-!r.
,7
L
Hence, from Descartes' rule of sign,
dgl/dy
= 0 can have
at most one positive root, as after supplying the missing term yN-l
we see that there is only one variation of the sign whether the coefficient of yN-l be considered positive or negative.
is given by y
= 1. It follows that for 0
<
This root
y < 1, dgl/dy has
a constant sign which is easily seen to be negative.
Hence
gl
a monotcnically decreasing function of its argument in the range
(6.4)
Differentiating
g?(y) with respect to
y and arranging
terms,
dg 2
---d. = y
yN-3(1_y)2
N-1
NI' 3 ;-rN-2)y
(l-y -)
-
N-2
-2y
N-3
-2y
- •.• -2y+(N-2)
-
7.
is
41
<.
Let the expression in the square bracket be denoted by
g2*
Descartes' rule of sign,
=0
*
g2°
From
can have at most two positive roots o
Now'
and
=0
Hence
y
= 1 is a repeated root of
g;(y)
=0
Theref<0r e ,
dg
N-3
4
N-3()
-2
= _ y (l- Yj ¢(y)=. -y
1-y
dy
,)
where 0(Y)
f.
(N-l)
l-y
0
for
( l+y+ •.•+YN-?)3
.
¢(y)
,
0 < y < 1, and since
- -
*
g2(Y)
¢(y) has the same sign as
= (1-y) 2¢(y) ,
g;(y) which is easily seen to be posi-
tive. Hence dg 2!{jy < 0 for 0 < y < 1, and for 0:: y ~ 1, g2(Y)
is a monotonically decreasing function of y~ and
(6.5)
g (n)
2
= 1,
g
?
(1) = ~2)N
(N_l)2
-7,
Since \2 is a monotonically increasing function of A in /-0,1
-
,
we conclude from (6.3) that g(\) is a monotonically decreasing
function of \
(6.6)
in
L-O, 1_7
g(o)
and
= 00,
42
( )
1 2
g 1 = 1+ 2'. N-1
N
= ('N-!)
2
+
1
(N-2)N
N-2 • (N-l)~
,
Write
(6.7)
then 1'(A) is a monotonically increasing function of A inL-0,1_7
with
(6.8)
7.
,+'(0) = 0
Bounds on the integral
From
(5.4)
and ''/1(1)
=
(1 _
~)2 •
IN'
and (6.1)
1
r J:
IN = 2
'ijt'(A)_7(N-2)f2h (A)dA •
J
o
Now
(7.1)
yeA) can be written as
2 ] A2N- 2 ~
"'Y'(A)=2A. (M .~.
);-1+
l"'A2N -
m 2 (1_,2N-4)
w\o
'"
;::2) (l-A 2N)-
Hence
/
-
Put
_
1
(1_A 2 /
N2,2N-2
'"
- (I-A2N/
_7
1/~
C
dA
•
43
then
1 2
sinh ( lW')x N-2 _ N sinh (1- i)x
(N-2)/2
sinh i -_7 L 1+
)
_T
(N-2 sinh x
r
L
o
00
~
r
-
t
00 (
4
1
sinh 2 ~
N
h N.
it w . eo th x
COSL
.•
1
Ol.:D:n
X)N-2/x If"
_2 cosech2~71/2~.
N. _ cosh 2 NU.I\,
)
o
r
N
2x
.
2x) 7(N_?)/2
_ 1+ N-2( cosh N - coth x s:mh T _
Now
for every x, while for
where Bn
lxl
n ,
<
is the nth Bernoulli number.
th
x
cos h N - co
'00 x
X S1
1- 1(1
N= - N
x
+ j
Therefore, for
246
:x:
2x
- 45 + 945
Ix J
2
~ -coth x
sinh
i)= - (1+ ~ - ~+
1t,
x
O( -3)
+ ••• + 2N 2+ N
)
and
(N-2)1og(cosh
<
••• )+
~(~ + ~
44
and finally
x
xN2
(1.4) (cosh N -ooth x sinh 'N) - "'exPL-(1+
-r -'[;
2
~
+ ••• )
Further
1+
.1!~(oosh
~
N-~
N
Ii
2
- coth x sinh
~)=2r11(::_
- ;..,..
~
N'3
4~
6
-I-
92~
1 (X 2
2x4
.
3
... -2 - + -,-p + ••• )+ O(N- )
N 3
45
+
... )
_7t
and
N-2
N (2x
.
2x) _7
... ""2
log .t..r 1+ N-2
cosh 1r ... ooth x slnh
N
=-
1 x 2 x4 2x 6
1 ( 2 x 4 2x6
....,...log 2 + '2(j - 4; + 945 + ••• )- TiN 2x - '9 + n~
N-2
2
+ ... ) + 0(N- );
and so
(7.5) L-1+
}'J~2(cOSh ~
-(N-2)/2 exp
"" 2
- ooth x sir.h
~L7 -(N-2)/2
6
2 4
i .,1(x'.3-2 - 45
x4 + ~
2x +
1(x
x
... ) - N
-r . . W
... )+0 (-2).~
N _I"
_
Finally
246
-x/N) 7(N-2)/2 -1
/- x
x
x
(
=e exp_ - '6 ... 90 - '91.3 +
r "f e _
( 7•6) _
3
... mJ(l +
2,,.2
x4
"'::9 - 30
0 ••
-·2
+ ••• ).... O( N
>-7
45
Ix I <
the expression being valid for
(7 .7 ) _r cosech
2x
N -:N 2
.
2
1/2
cosech x_7
no
Also
2
::
N /- x
I/'J'"'
_ 1-!O
IN into the ranges
We split the range of the integral
£0,1.] and
L-1, 00_7.
4
137
... l.~OO x + •••
Denoting the in~jjegra1 from
° to 1
by
I N1 , we have f:::-om (7.4) - (7.7) to order N-1 ,
-1
(7.8) I =.L-
lJ
Nl
-
_/ 1-
f1 (1- '0
x 2 x4
+ 40 +
a -:1
=0
x2
!O
-
137
+ 12600
x
4+
7
.""_ dx
137x4
" .. )( 1- yO + 12606 + ".. ) dx
x2
130
a-I
=-
ff
f1
x 2 x 2 x4 x 4
137 4
0 (1-?", - 10 + 40 + bo + 12600 x + •.•
_ a-I (
- ./3-
1
1-
1
1
1
131
T8 - 3U ... "2"00 ... 300 ... b3000 - .".)
-1
= ~(.9216) :: 0.196
13
Hence, to 0(N-1 ),
I =0.196+2 N/ 2
N
o
•
)
dx
Denote the second term on the right hand side by J
and substitute
A = yl/2
.
dX .,
'21 y -1/2 dy ]
so thlt
2 N-l. 1 )2
t -y
r l- N y
The result
(1.8)
(l_r~)(N-2)/2
(l_yN)
~
71 / 2
-
d
y
•
is sufficient to prove that the coefficient of
in the expansion of
now proceed to set bounds on
J.
p(r~.:: rO)
Now, for
0
is non zero. 1nTe
~
y < 1 ,
N 3N+
2 Y2N+(3)Y.r1+(N-2)yN+(N-I)
...
_7
=1_(N_2)yN-lCI _y )+ {(N;2)y2N-2_ CN _2)2 y2N-l+(N;1)y2N}
N-?) 3N-3
- {( 3
y-
-
(N-2)2(N-3) 3N-? (N-2)2(N-1)3N_l
-"'21
T
+
2
r
- (~h':3NJ ~ ...
47
and
J:iilrthermor e,
and hence
Therefore
_ Ny(1_~r-2)
e-1/ 2
-(N-2)/2
1
-(l..-y) ('1'2
< / 1..- tr
7
<
. in
N-2)
(N-2 )(l-y) (hy) t N-2
Using (7.10) - (7.12) in (7.9), we obtain
where
~2
~(N-2) r
e -2/N
f
- 0
(N,-3)/.?d
y.
(1_Y)(1~y)(N-2)1?
y
·
48
-(N-2)
+~
-2/N
(N;?)y(5N-7)/2_(N_2)2y(5N-5)/2+(N;1)y(5N-3)/2dy
o
(1_y)(1+y)(N-2)/2
e... 2/ N
f
(l-y)
{y(5N-7)/~_i5N-5)/21dY
(1_~J)2(1+y)\N-~)/2
o
We observe that we have to evaluate integrals of type
e -2/N
f
o
and
e-?/N
J
(7.16)
o
where
q = (N-2)/2, P
= sq+b,
s
>
0
and b = 0(1) • Substituting
y = e- 2/ Nz and expanding (1+e-2/ Nz)-q in the powers of (l-z) and
integrating term by term, we obtain
49
M(p;q,e
-2/N
~ r~r»
e... 2(p+1)/N
)- (
-2/N q
l+e
)
=
e- 2 (p+1)!N
(p+1)(1+e-
21M
)
r(p+1) ( 1 )r
f,(p+r+2j
2/N
. 1+e
q
r=O
q F(1,q,p+2,
1
2/q);
l+e r
and
-2/N _ e- 2(p+1)/N 00 e- 2k / N (
,
1
t(p,q,e
)-2/N
X +k+1 F\1,q,p+K+2,
2/N)·
(l+e
)q k=O P
l+e
Now if s > 1 and c, t > 0,
t
t( t+1)
2 +
F(l,t,st+c,x )=1+ ts+c x + (ts+c)(ts+c+1) x
x
x
s
s
2
x3
> 1 + - + -2 +'-3 + •.•
a
...
1
= l-xTS
and
If
t
is large, then the right hand side of the last inequality
2
1+ -x +;:;, x
'" 0 (1
i t). NoW, for
s 6 (l-x)
we have
(1.17)
26
(
2s-1 < F 1, t, ts+e,
1 ) 1+8+2s
2TN <. 2
l+e N
2s
2
=
Therefore, taking
q
=! - 1
2
and p
= sq v b, to O(N-1 ), we find that
and
(7.18)
Let
L- q_7 denote
the largest integer in q; also, let
largest integer in
q
~,
q2 in
2q
IO
and
so on.
q1 be the
Then
00 -2k/N
1
~ e+k+1 F(1,q,p+k+2, - 2/N)
k=O P
l+e
Lee7 -1
e -2k/N
1
+•••+ ~
v . +1 F(1,q,P+Q9+2 ,
2/N)
k=q p q9
l+e
9
1
l+e
2/N)
•••
+ e-?q/N(1_e-?q/N) ( ~+(S+1)+2(S+1)~ )
2(s+1)2
(S+1)Q(1_e- 2/ N)
+
-.2 I
2\
e, ., ll. 2+s+ ( e+ 02) j + •••
(S+ .2)
+ !:.e-1
e
•
J
{S+2+2(8+1)2 + e-2 3lf 8+3+2(8+2)2
(8+1)
.
(8+2)
L- e-1j
-r {
2(
+ ••• + e. 3 . s+r+1+2(s+r) J + •••
(s+r)
J
Similarly
(7.20)
e-· 9(28+1.8)
_7
+ •••+ (s+1)(28+1.8-1)
e
For
8
= 1,
p
~
q
=~ -
1, we have
,
+
l-,a-'" { -1 11
-2 22
-3 37
-4
2
e . 1f +e '"'-1 + e • or; + e •
hence,
L (p,q,e
.629
-2/N)
.<
2
(N-1r;~
I
·
56
W
+
A similar
shows that for the above values
calculation
ot
S,p and q,
Denoting the integrals on the right-hand side of (7,14) by Ql' Q2
etc. as they occur in order, and neglecting th,e sign, we see that
Q
l
= 2(N-2)/2 L(~:~J
¥,
e- 2/N)
•
Hence from (7 .:?1) and (7.22)
0.54:?
We may note here that
IN
J <
= .196
<
~
0.629
<
01 and hence
+ J <
.196
+
.629 = .8:?5 •
The upper bound can be reduoced further by evaluating
and we proceed to do this.
02' Q3' •••
Now,
N-. 5 N-2
= 2(N-2)/2(N°- 2)M(3-2·~'
-2/N)
-2 ' e
o
°
(7.18),
2
Hence, from
putting s
=3
and
qN!,
2
we get
or
0.065
< Q2 <
0.066 •
Similar evaluations give the following results:
•
54
0.101 < Q < 0.103
3
...5
04
,
=<
O(!.-)
N
0.028 <
Q~ <
0.0?9
0.0055 < Q6 < 0.0056 •
Now
Q = Ql - Q2 - Q + Q + Q - Q6 - •.•
5
3
4
<.629-.065-.101+.029-.005= .487
and
Q > .542-.066-.103+.028-.006
~
.395.
The latter terms diminish very rapidly and do not affect to the
second decimal place.
Now,
e -1/2 Q < J < Q '
that is,
.?39 < J < .487 •
Since
IN
=<
.196 + J, we have
These calculations are valid to O(N.'~) and to two decimal places.
Hence the first term, PO' in the expans i on 0 f
r
* =1
is given by
p
o
=~
(l-r )(N-2)/2
1t
where IN is bounded as in (7.24).
0
p( r ~l- ~
r
)
O
ab out
"
8.
Bounds on
Per* ~ r o).
From (4.9), we have
sin Q = (1_r*)1/2
Hence fOr a given value of
L- "f' (".L71 / 2
' the maximum value of
O
'r
QO( ".),
say
I
9 , is given by
0
(8.1)
~
( l-r )1/2
O
or
(8.2)
FUrthermore, from
s i nN-200
N-2
(,.2), we have to O(N- 2),
" N-2 8
s:m
0
<----(N-2)cos eO
<
Therefore,
co
1
n
fo
or
N-2
sin
SOds(".
From (1.24) it follows that
0.43'(1-rO )(N-2)/2< P( r *> r ) < 0.68~(1 -r )(N-2)/2
n
- O
nrO
0
_v
"
p
IllTe remember that the results EPPly to such values of r O
)
56
for which sin3 QO is negligible in comparison to 1.
If, for in-
stance, we consider 0.01 as negligible, we may take the maximum
value of sin 90 = .21, i.e., r O ~ .978. In fact the bounds may be
valid even for much l~er values of r O' as (8.~) suggests that the
neglected terms
in (3.6) may be of 0(sin4e) rather than 0(sin3e).
r~f-
A similar argument shows that
= ..
1 if and only if
i=2,3, ••• ,N and X < 0, xl f 0 •
If -1 <
1"
o
is close to 1, it follows from (4.9)
< 0 and
that near r*= - 1,
,
and if the integral corresponding to
IN be denoted by IN '
~ ~ / £ 1f' (l.)jN-2 )/~().)dA •
-00
_4 chen ge of variable from
X
to -X shows that
,
~
= IN. Therefore,
the first term, p~, in the expansion of p(r~ ~ r~) about r
(8.4)
p
,=
o
IN (
-n
f
)(N-2)/2
1 + rO
~f-
= -1
is
'
and
It is instructive to compare (1.?5) with the probability of an
ordinary correlation coefficient r, based on a sample of size (N-I),
57
exceeding a given value r O close to 1.
If the population means are known the frequency function of
r
for a sample of size N-l is given by
r(N-l)
2
=_
rn f(N;2)
(1_r)(N-4)!2
t2- (l-r) ~ (N-4)!2
)
N-4
T r(B)
_2
::z
_
~?_
N 2
rn r( 2-)
(1
-r
)(N-4)!2 {1
Therefore the first term in the expansion of
Hence
(8~6)
This ratio tends to zero as
N -> co •
N-4(tl)+ - ..'
- 2
•• • J
2
per
~ r
) is
O
•
CHAPTER III
JACOBI POLYNOMIALS AND DISTRIBUTIONS
OF SERIAL CORRELATION COEFFICIENTS
1.
Introduction.
It was pointed out in section 1 of Chapter 2 that a statistic
closely related to r* is given by
(1.1)
r
1
=
N-l
Z xix'+l
i=l 1.
N
2
Zx
i=l i
which has the advantaqe of being a ratio of two quadratic forms.
this chapter the approximate distribution of
•
rl
In
will be obtained in
terms of a beta distribution and a series of Jacobi polynomials.
Denoting by ql
the numerator of the right hand side of (1.1),
the characteristic roots of the matrix of ql will be utilized to
calculate the cumulants and the moments of ql'
The asymptotic normal-
ity of ql
and of r l will be proved separately. Following a discussion about the properties of Jacobi polynomials a method will be
developed to obtain an approximate frequency function of the ratio of
two quadratic forms.
Of course
this method has limited applicability
but suggests the possibility of using orthogonal polynomials in approx-
•
imating the distributions of certain statistics whose exact distribution cannot be obtained expressly in terms of known functions and integrals.
It can also be used for calculating percentile points, even
when the exact distribution is available, but is tedious to handle.
2.
Jacobi polynomials.
the distribution of r l it is necessary to
state some properties of Jacobi Do1ynomials. These properties are
~efore obtainin~
given below without proof.
For the proofs a reference can be made
to Orthogonal Polynomials, by G.Szego
L-47_7.
Write
f(x;a,~)= (l_x)a(l+x)~
.
2a+13+1B(a+l,~+1)
The Jacobi polynomial
nomial of degree r
;a,~
> -l,-l;x
~
1
p~a,~)(x) is defined as the orthogonal poly-
with we:i.ght function f(x;a,~). The normaliza-
tion is achieved by the restriction
Explicit1v
r
m
_ (r+a)(r+~)(x-l)r-m(_x+l)
~
~, r:O,l, 2 , •••.
m r-m
t:.
c
m= O
Yo
It can be shown that
if r
r/
k
if r = k = 0
if r
if r
A very useful property is
=:
=k
k
>
=1
1
60
'( )k ( .
)
B(a+1,13+1)
= -1 f x,a,13 B(a+k+l,~+k+l'
kt (ll B)( )
2k Pk "
x
p(a,~)(x)
satisfies the differential eauation.
r
2
(2.4) (I-x)
2
d
--4
dx~
+ /-~-a-(a+13+2)x
-
d
_7 ~
aX
+r(r+a+~+l)y
=0
and is related to a hyper?eometric function by the equation
Changing the variable from x to z by
x = 2z - 1
dx= 2dz
let
f(x;a,~)dx
Q(O:,P)(z).
r
be transformed into
(Note. o(a,p)(z)
-
r
g(z;a,13)dz
and
p~a,p)(x)
has no oonneotion with the second solu-
tion of (?I.d, a no'liation used by Szego).
Then
,
and
(2.6)
Equation (2.3) go~s into
to
61
Now, consider the characteristic function of a variate x
quency function is
whose fre-
g(x;a,~). We have
0') t)= te tx==
1
J
etX g (x;a,{3 )dx
°
The partial sums of the series under the integral sign are dominated
by the function
range.
(2.8)
e
~xl
in /-0,1
-
-7
which is integrable over this
Therefore, we can integrate term by term and thus obtain
=
co
2:
r=O
r(p+1+r)f(a+@+?)
-
r(~+1)r(a~~+2+r)
tr
r.
y
== F(~+l, a+~+?, t)
where F({3+1, a+~+?, t) is the confluent hypergeometric function
whose Taylor series expansion about t=O is given by (2.8).
The series
(2.8) is absolutely convergent and since a, ~ > -1,
+ •.•
<1+Jfr+J~l2
Hence,
+
•••
_
Q
-
v
ItI
•
F({3+1,a+{3+2,t) -is ~i entire function and so
From the inversion theorem of characteristic
fun~tions,
if
,0
~
x < 1,
/
( 2.1 0)
1
~ni
Jje-txF{~+1~a+~+2~t) dt- B(I1+1,~+1)
(l_X)I1X~
io
(f .... dt
where
stands for
lim
)
c
J ...
->00
d'ii, and
i
is imaginary
-ie
unit.
Replacing a
entiating both sides
(2.11)
1
2ni
and
~
by a+k
and
p+k in (2.10 )' and differ-
k times with respect to x we obtain
j-+ t k
e -txF ( ~+k+l,~+o:+2k+?, t ) dt
= g(X;I1,/3)
B(a+l,p+l)
(a C!) )
k' Q ' to' (x
B(a+k+l,/3+ k+l ) • k
--:---~'""'--"("
Similarly, the characteristic function of a variate y with frequency
function
f(y;a,/3) is
e- t F(p+1,a+/3+?,2t)
and for -1 < y < 1 ,
and
•
63
3.
Distributions of certain
statistic~
as series of Jacobi poly-
nomia1s.
Let R be a statistic whose range is contained in ~o,1_7.
Let either its characteristic function or its exact moments up to a
certain order and a few dominating terms in all its moments be known.
Further, let it be possible to write its characteristic function 1
¢R(t), in the form
(3.1)
¢R(t)=aoF(~+I,a+S+2,t)+alt F(~+2,n~~+4,t)+ •••
+B tkF(~+k+l,a';'~+:'k+2, t)+ •.•
k
00
= Z
k=O
where
a,~
> -1.
k
~ t F(~+k+l,a+~+2k+2jt)
It is possible that
which case the series will terminate.
itl
~ e' , for all
,
ak = 0 for all k ~ r, in
Since IF(~+k+l, a+~+?k+2, !t/)I
k ~ 0, the series on the right hand side of
(3.1)
converges absolutely if
'. a,..
t I
.J..iil
~t.
1 ....
k ->00 iak_i!
I
< 1;
and it converges absolutely for all values of t
8
lim
k ->
•
00
k
ak _1
if
=0
Suppose either the series ().1) terminates or converges abso:}.ute1y f or all values of t.
From the inversion theorem of char-
acteristic functions the freouency function of R is given by
64
Let us formally carry out the term by term integration.
From
(2.11) we will then write, fQrmally, the frequency function of R as
in the range of R, and f(R)=O
othe~~Jise.
It will be seen that the
terms of the expansions (3.1) and (3.3) correspond by means of the
relation (2.11).
This process is justified analytically in the case
when partial sums in the integral (3.2) are dominated by an integrable
~unction
of t. However, in practical applications in most cases we
are seldom interested in the convergence properties of our expansions.
As Cramer
states~
"What we really want to know is whether a small
number of terms - usually not more than two or three - suffice to
give good approximation to
seen later that
by
-7.
f(x) and F(x)" /-7, p. 224
-
It will be
this method we ob'tain expansions for frequency
functions and distribution functions of serial correlation coefficients
in series which are asymptotic, and have some other desirable proper-
ties; for example, the moments of the fitted series agree to a high
order of accuracy with the moments of the sta'Gistic under consideration.
Now the maximum value of
Q~O:'~) (R) in the rmge fO,lJ is
max
Q(a'~)(R) = (k1:Q) where q =
O<R<l k
~
max(a,~) •
6,
Hence, by the ratio test, the series (3.3) converges for all values of
14kak
lim
k ->
I k _1
00
B
<1
A similar procedure shows that if.
0.4)
¢R(t) = e- t
00
E akt~(f3+k+l,o;+f3+2k+2,2t)
k=O
then we can formally develop the frequency function of R as
This series converges if
I 2k8 l
k -> 00 I "ak-~ I
lim
1{
< 1
•
The above results are summarized in the following theorems.
If R is a variate with range L-c ,c _7 where 0 ~ 01
l 2
~ 02 ~. 1, and if the characteristic function of R, ¢R(t), can be
THEOREM 3.1.
written as (3.1) and the partial sums are dominated by a function
~ (t)
such th at,
e
'tR~( t)
is integrable over (-00, 00) then the fre-
quency function of R is given by (3.3).
THEOREM 3.2.
In 'theorem 3.1 replace
(i) 0 ~ c1
~ c2 ~ 1
by -1 ~ c1 ~ 02 ~ 1,
In the following sections we will obtain distributions of
66
serial correlation coefficients by the application
or
the method de-
ve10ped in this section.
4.
Independence of r
and 1'.
l
Let xl' x 2' •.. , xN be independent N(O, 1) variates.
N-1
ql.
=
p
=:
l
xjx'+ l
N
2
j=l
j
j=l
'
J
l x
,
and
THEOR~M
4.1.
rl
is distributed independently of p.
The distribution of xl' •.• , x
PROOF.
dF(xl ,···,xN)=(2n)
-N/2
e
2
-lxi /2
is given by
N
dx1 •.• dX N
=(2n)-N/2 e-P/ 2 dX
1
e"
dX
N
Let
x.=z.p1 /'2
J
J
,
j
=1,2,
e."
N, with
The Jacobian of the transformation is ~1, p.
N
l
2
z. = 1
j=l J
385_7
P
T~erefore
the distribution of p, zl' "', zN_l
(N-l)/2
is given by
Define
67
1
Hence the conmined variable (zl' ••• , zW)
of P, as
is distributed independently
zN can be expressed in terms of zl' ••• , zN.1 only. Now
and is, therefore, distributed independently of p.
'/'., qk
(. 1
COROLLARY.
;t
k
CP
5.
Cumulants of ql .
Before proceeding to investigate the distribution of r , we
l
will investigate the properties of the distribution of ql'
As a preliminary, we will state a theorem about, the cumulants
of a general quadratic form in normally distributed variates.
THEOREM 5.1.
I
Let
x =(~, ""
xN) be distributed normally with
mean vector zero and covariance matrix Z, where Z is a positive
defin5.te.
Let
I
u = x Ax, where x is the transpose of x
is a real symmetric matrix.
,
and
If the characteristic roots of AZ
A
are
08
PROOF.
The characteristic function of u, ¢. (t), is given by
u
¢u( t)=(2n)-N/21
:r(2n)-N/2
j expL:' ~X Z-lX tx
z \-1/2
I Z 1-1 / 2
I AX
_7dX
J expL--~x '(Z-1-2tA)x_7dx
:: t Z 1-1 /? I Z-1_2tA i
"" 7T
+
1
-1/2
(l_2tc )-1/?
j
j::l
Therefore~
X (t)
u
1 N
=- -
E log(1-2tc.)
2 j=l
J
If by cl we denote the characteristic root of
in magnitude, then for ~t 1 < --2:.--,
2
(~E)
which is largest
l0ll
• Q.E.D.
Note:
If Z is the identit. y mat rix,
cl ' •.• , c will be the
N
characteristic roots of A.
THEOREM
,~.
N-l
ql "" i:1XiXi+l
(5.2)
•
PROOF.
The characteristic roots of the matrix Q
l
are
Aj
1I\1e have
j'l
= cos 'N+I
'
j=l, 2, •• " N
~
of the form
0
0
0
0
10
1
0
0
0
1--- .
...
...
0
1
1
0
Write
DN=
i -j.L
1
0
0
1
"j.L
1
0
1
-j.L
1
0
0
0
II
i
"
I
II 0
I
I •
I
I,
e
0
Then it can be easily shown that
Put
~ =-
x
N
,
so that
x
or
2
+j.Lx+l=-O
2)1/2
-IJ. + i(4 -lJ.
x =
-
2
Let
lJ. = 2 cos
~
so that
x = -e t¢ ,-e-i¢
·..
·..
·..
i
- ...i
0
0
0
-j.L
70
Now
Introducing DO' which satisfies the difference enuation for N=2,i.e.,
we obtain
The most general solution of
where
and
(,.4)
is
A and B are arbitrary constants.
From the values of
DO
Dl' we obtain
A+ B =1
~r>!
-Ae .J..'P. Be
-i~
= -IJ.
which give
e
B = -
A
.i¢
2i sin ¢
=>
21 sin
¢
and
DN =
2i sin ¢
=
..
-r
e
i(N+l)¢
-e
~i(N+l)¢ '1
-'
(_l)N sin (N + 1) ¢
sin ¢
The characteristic roots of 2Q are the roots of the equation
l
in IJ.~ which is equivalent to the equation in ¢
DN=
0
71
sin(N+l)¢ - 0
sin t3 This pives
¢
We note that
¢:
gree equation in
0
~,
~j =
~
i:r '
j :
Since D = 0 is an Nth deN
the N roots are given by
is not a solution.
jn .
2 cos N+l
= 1,
, j
and hence the characteristic roots of
,
Aj -- cos 'N+!
jn
COROLLARY 1-
=
N
1t
j=l
,
2, ••. , N
°1
j : 1, 2,
are
... ,
N•
The characteristic function of ql
r/J (t)
ql
GOROLL4RY 2.
1,2, ••• ,N
jn
(1-2t cos IIl+1 )-
is
1/2
The limits of the variation of r
1
•
are
+ cos
Cumulants of ql
We observe that if N: 2m, th0n
AN_j +1
and
= - Aj ,
j
= 1,
2, "0' m;
if
N = 2m + 1 ,
= 0, AN-j+ 1 =
Am+ 1
=
1, 2, ... , m
In any case
N
..
~
~
j:l
e.
From theorem 5.1,
'\ 2.k+1 = 0
J
~
, k -- 0 , 1 , 2, ...
n
N+l
72
N
r - 1 ( -1)' ~ ,r.
_
2
r:ql r . . . . . flo
j=l J
K'
I
,
hence
(5.6)
K'
Now
1
t1
0, r = 0, 1, 2, ...
;=:
rr+ :ql
-inj
Z ).,2r = Z~(e N~l + e N+l)
j=l j
j=l 2 r
N
ir.j
N
.-._"
2
1
r 2r
= --2r Z ( )
2
If k-r = 0,
k=O k
- - 2r
2nij(k-r)
N
~
j=l
-+ (N+1), -+ 2(N+l), ... ,
N
2j.t-( k-r )j
Ze
N+l
=
j=l
N+l
e
then
N
Z1
=N
j=l
Otherwise,
N
Ze
2in(k-r)j
N+I
2iR(k-r)
e
N+I
2in(k-r)
- e
?in(k-r)
1 _ e N+l
j=-l
= -1
Hence, if
(i) r ~ N,
=
1~r _
/-(2r)
r N
2
2r) _
- 2 2r + (r
7
73
(ii) r
= s(N+l)+k
k~
where
N and
s
is a natural number
(5.8)
+ 2(
Finally, from theorem
r
r-s?N-s )(N+l) - 2
2r
_7
5.1,
if s(N+l) ~ r < (s+l)(N+l) where s = 1,2, •••
6.
Asymptotic normality of ql'
THTi:OREM
6.1.
N(O,l) variates
If xl' x 2 ' ..• is a seouence of independent
N-l
and ql = .E xix i +l then ql' when standardized, is
1.=1
asymptotically N(O,l).
PROOF.
First we note that the Central Limit Theorem is not applicable
in this case, as the successive terms in the SUIllil1ation are not inde-
pendent of each other.
Secondly, the theorem 6.1 is valid for the
case when the variance of each x is the same finite quantity.
74
From (5.6) and (5.9) we have
,
/' q1 ::
K
"2 :ql::
K.
-..'
:: 0
l :ql
= N-1 •
?:q1
Let
q1
IN=I
(6.1)
From (5.9) and (6.1) we have
(6.?)
,
t2
z
2
X ( t)= -
't.
N
E
t 2r
(? 1) t
_..:.!.:.2(N_l):r."
r=2 (2r) t
00
+
E
(s+l)(N+l)-l
E
00,
for every r
(2r-1)i
(2r) 1 . 2(N-ll
s=l r=s(N+l)
As N ->
t?r
we have
,
lim
K,
= lim (?r-l)l
N ->00 2r i Z N->oo 2( N_l)r
~ (2r)( N+l)_2 2r
\
r
\
l
)
=
0
•
Furthermore, the double summation term in (6.2) recedes to infinity
as N ->
00;
hence it follows that
lim
Q.E.D.
N ->00
The rapidity of converpence to normality could be measured by the
kurtosis, as the skevIness is zero.
75
The freouency function of ql' r(ql)' in type A Edgworth series up
to 0(N- 2 ) is given by
1
=:-
ffn
where
H (x-x
)_ r - r(r-l) x r-2
r
7.
+
r(r-l)(r-2)(r-3)
-2
2-lt
x
r-4
- •••
2 . 21
Moments of r •
l
From theorem 4.1 the moments of r
l
are obtained by
utilizing the relations between moments and cumu1ants we obtain
1
2k+ :ql
W
=:
0, k
where
\I
For N.::
4,
tl
-- '.
q
- //-1
l ·Q
"1 '. -
and
=:
0, 1, 2, •.•
76
(7.1)
=N-
Vt')o
c· q
1 ,
l
Y4"
= 3N2+12 N-27 ,
Y6'
·ql
= 15N3 +225N?'"5?5N-2205 ,
·ql
v8"
·ql
~ l05(W-l)4 + 1260(3N-5)(N-l)2+1260(3N_5)2
+672 0(5N-ll)(N-l)+l008(35N-53) •
In general the first two terms in the highest powers of
N in
are
p
is distributed as
/}
" r
c· p
)(,2
with
N degrees of freedom; hence
2r(N
r' -2 +r )
= ~-
=N(N+2) ••• (N+2r-2)
r(-)
2
, Therefore,
(7.3)
2k+J.- :r
= 0,' k = 1,2,3,
Y
i
y.
2:r,.I.
:0
l
N-l
N(N+2)
N(N+2)(N+4)(N+6 )
2
15N3+225N +525N-2205
N(N+2)(N+4)(N+6)(N+8)(N+IO)
In general
v
2k:ql
77
1.3.5 .... (2k-l) {N k+k(3k_4)N k- l ) +0(Nk- 2)
(7.4)
N(N+?) ... (W+4k-2)
=- O(N-k) •
Asymptotic normality of r l .
Let
v 2k+l :x
= 0, k
=
1,2, ... ,
lim
"2k 1x
N ->00 .
If ¢.x (t;N)
~.1.3.S... (2k-l)Nk/-l 0(1) 7
2k
- + -N-
=
lim
N ->00
=
1.).5 ... (2k-l)
N
denotes the characteristic function of x, then
for every t
It follows that the distribution function of x
tribution function of a w(O, 1) variate.
!!'EOREM 7 ~l.
:i:'l '
when
standardi~ed,
converges to the dis-
Hence
is 9sy:nptotic ally N( 0, 1).
The rapidity of approach to normality can be measured by kurtosis
as the skewness is zero.
NoW
~ = v~ = 3N2+1?~~~~
2
v~
N(N+2)(N+4)(N+6)
78
= 3( 1- -2
N
~2 -
8.
= - -6 •
3
Nt'
N3
1
0(_)
N
flnproximate distribution of
10
1
• - • 0(_) )
N2
r •
l
From (7.3) and (7.4) the characteristic function of ri
=z
can be written, if N > 6,
¢ (t)
(8.1)
=
1.
~-l
~
+
N(N+2) If
z
•
2
3N .l2N-27
N(N+2)(N+4)(N+6)
l5{N3"1:;?rl+35N-47)
~
N(N+2)(N+4)(N+6)(N+8)(N+lO) 3t
•
Let us write formally,
¢z ( t)=F(~2'
(8.2 )
N+2 t)+
?'
Comparing (8.1) and (8.2) we determine
¢ (t) = 1 +..l
z
!... +
N+2 11
4
00
~ a tkF( 2k+l
k='l k
-r
I.)
'
N+ k+2
t)
- 2 ' .
SkIS.
2
~ +•
(N+2)(N+4) 2l
Writing (8.2) in detail,
1.).5 ..• (2k-l) t k
+ •••
(N+2) •.• (N+2k) k!
•+ -
3.,
2
~+
(N+6)(N+B) 2!
..
-+ •••
79
+ a t r rl + ?r+l.!. +
N+4r~2 1!
r
k r
t + ~(-.;2r;,.-+.:;;1..::-;)(~2~r...;+)~)...;.•.;...
•.;.,.;;(..,;.2k..,;.-. .;:l;.:. ) ........;..-(N+4r+~) •.• (N+2r+2k)
(k-r)!
+ •••
-
7
+
,•
or
~ t k 5a + k:l a (2r+l)(2r+) •.. (?k-l)
kl Jt
k
k=l k! l
r=O r (N+4r+?) ••• (N+?r+~k) (k-r)t
( 8 .:3) ¢z ( t ) = 1 +
where
aO
Lo
-,
Lo
= 1. Therefore, we obtain
a
Sol vinp for
8
Z
k!
+ k l a (?r+l)(2r+) ••• (2k-l)
= \I
2k:r1
k r=O r (N+4r+2) ••• (N+~r+2k) (k-r) 1
1,
a~:"
a), we get
a
1
1
N-l
N(N+2)
+-=-~-
N+~
or
1
hence}
a
•
2
... -
)
2N(N+2)(N+4)(N+6)
Si..milarly,
B
J
... -
45
)! N(N+2)(N+4)(N+6)(N+8)(N+IO)
80
Now, all the cumulants of q1
~ossib1e
and
are
~nown;
to calculate all the moments of
hence, it is theoretically
and therefore of r 1
Then we are able to determine a , a , .,0' to any desired
1 2
No systematic way of determining all the coefficients could
z.
number.
q1
emerge because of the irregular behavior of the cumulants of
From corollary 2 of theorem ,.2 we see that
order greater than N.
the range of r 1 is
-/2
- 0, cos
q1 for
£"-
cos
N:r '
cos N~I
_7
and that of
z
is
11
i+!
_7. Hence from (3.1) and (3.3) an approximate distri-
button of z
to the first four terms in the expansion is
_ (l-z) ( N-1) / 2z -1 / 2 _
r(z)N1 r
I 1B(T)' 2)
-
N+l
B(~ ,
1
2)
N+3 3
Ql(z)
N(N+2)B(T'-)
2
3 B(.! , ~)
-
N(N+?)(N+4~(N+6~B(2,
~~
2
2
Q2(z)
45 B(l , R:1:)
2
-
N(N+?)(N+4)(N+6)(N+8)(N+IO)B(~ ,~)
where the superscript~ for each polynomial
(1_z)(N-l)/2z-1/2dZ
f ()
v..dz --_.
B(N+l
}-, " -
2 ' '2)
N+8
: 2
--:::--- ~ Z
l.2!N i\.
3(N+12)
23 .3 rN
6z(1-z)
- -
(N+1)
7
2
Q3(z)-
Q are (N;l , -~) • Or
L--.l.
N+4
- W
'I;
'; z
3(1-z)2
+ --"----(N+1)(N+3) .
(J.-Z)
(N+l)
81
2 'It
for 0 < z < oos -1
-
= o
-
N+
otheI'tvise.
Henee, an aporoxtmate distribution of r l
is
( 8.6)
+ _3(1-ri)2_
~
(N+l)(N+3) )
t: 2(_l.-r 2)2
4;>:r
l
l
(N+l)(N+3)
for - cos
=
9.
l5(1-ri)3
\
-----\
(N+l)(N+3)(N+5)
1t
N+:r .:: r l ::: oos
J-
7
1t
N+J:
otherwise •
0
Asymptotic validity of the distribution of
r •
l
In the previous section we have given an expansion of ¢z(t),
first in a power series of
t
t
and then in a mixed series of powers of
and hyper geometric functions.
tained the distribution of
polynomials.
arise.
From this let ter series 1I1e have ob-
r
in terms of a beta function and Jacobi
l
Naturally questions about the behavior of series (8.5)
The question of convergence of (8.5) and of (8.2) depends on
the knowledge of
ak for large k" which in t~Tn depends on the specification of all the moments of r.,. Such information is not available
.L
in exact specifications.
However, first few dominant terms in all the
82
moments can be specified.
From this it is possible to prove the asymp-
totic validity of the distribution of
z
=
2
r"t'
.J.
This proof is carried
out in several steps.
§tep (1). 'oITe observe that cos ~ '" 1 + 0(N- 2 ).
L~+.1.
Since we are going
to consider the asymptotic behavior of the series we replace
of z
= ri by i-o,
I
_7.
th~
range
This change will have negligibly small effect
on the value of probaryilities or of moments.
ForexaIDple, considering
its effect on the kth moment from the first term of series (8.,),
t
I
cos
2
zk-l/2(1_z)(N+l)/2 dz
n
w.;:r
n )(n-l)!2
(1-cos 2 'i\T+I
...
<
'B(~ 1)
2 ' 2
=
Sim e
t zk
=
00
k
it will be negligible for all k.
Let us write
where vO(z), v1(z),
instance,
N~i/')
N
O(N- k ), this change is negligible for any
less than N and as N -->
S~ep (2).
O(
.0'
are the successive terms in series (8.,); for
83
Furthermore, let us write
and so on; and
So (t), sl(t), ••.
for the characteristic functions and
FO(Z), Fl(Z), ••• for the cumulative distribution functions associated
with fO(Z)' fl(z), ""
respectively; then
1
N+2
so(t) = F(2' ~, t) ,
N+6
,,
2
t)
at cetera.
Step (31,
•••
boy'
cient of
Denote the sth moment of the distributions
v(O)
s:z'
~
vel)
s:z'
•.• , resp'ectively.
in the expansion of
so(t),
v(O)
s:z
Then
v~;~
fO(Z)' fl(Z),
is the coeffi-
is the coefficient
~f
s
~
s. in the expansion of sl(t) and so on.
Step (4).
From the relations among moments and cumulants of a distri-
bution and remembering that odd cumulants of ql
are zero and that
every even cumulant is of order N, we find
>,
54
(9.1)
... .
Therefore,
v2 • =1.3 •••• (2s-1)(N-l)s~3.,••• (2s-1)S(S-1)(3N-')(N-l)s-2~l 2(N)
s.ql
swhere
v
Rs (N)
stands for
an
unspecified polynomial of degree
in N.
s
can be arranged in the following form:
•
2s·ql
v
2s :ql=1.3., ••• (2s-1)L-N(N+2s+2) (N+2s+4) ••. (N+4s-2)
-s(N+4)(N+2B+4)(N+2s+6) •.• (N+4s- 2 )+f _2 (N)
s
_7.
Therefore, we can write
v
_
s:z
-
1.3.' ... (2s-1)
(N+2)(N+4) •.• (N+2s)
""
(2s-1)(2s-3) •• '.3.1.s
------.-~~-N(N+2)(N+6)(N+8) ••. (N+2s+2)
-
ls.. 2(N)
------..;.,-.;;..---N(N+2)(N+4) ••• (N+4s-2)
We note that the first term is of O(N-s), the second term of O(N- s - l )
-s~2
and the third term of 0 ( N
).
Also that the first term is
and the first two terms are equal to
(9.2 )
v
stz
(1)
v • ,
s·z
i.e., for s
= v (0) = - -fs_I(N)
s:z
_
N(N+2) ••• (N+4s-2)
>
1,
~
(0)
s~z
85
Furthermore, it can be easily verified that
" l:z =
"2:z
(1)
"l:Z
_ (2)
- "l:z
=:
= ,,(2)
,,(3)
=:
=
/3)
...
==
"3~z
l:z
2~z
?:z
(J)
"32Z
=:
...
...
,
That is, by taking more of the terms of the series (8.,) more of the
first moments of z become equal to the moments of the constructed
distribution.
Step. (5) .
Now,
...vs
00
i:
s"2
+
N(N+2) ••• (N+4s-2)
Si
Therefore,
lim
N
~N t¢z(t)-~o(t)) j
- > 00
lim
=:
it I
-
I,'
I
F( 3
-2'
.N+6
2
N
->00
+
00
(s_2(N)
i:.
--",-=.~-s==? (!J+4) ••. (N';'4s-2)
N+2
'
{. )
v
st
R.'S(N)
N(N+~) ••• (N+4s-2)
Thus
0z (t)
is asymptotically equivalent to
tS
s!
~o(t), ~l(t), •.•
== 0 •
86
with successively better apnroximations and hence F(z)
ally equivalent to
mations.
is asymptotic-
FO(Z), Fl(Z), •.• with successively better approxi-
lATe can improve our approximations by taking more terms of
the series (8.2), stopping before the term for which ak/~_l is no
longer of O(N-l ). Thus series (8.2) is to be regarded the sum of
a finite number of terms, stopping either at the term but one for
which the coefficient
have already
achie~d
a is smallest or at some earlier one when we
k
as much accuracy as we want. In our case, taking
one, two or three terms from series (8.2), which is the same as taking
the same number of terms from series (8.5), means that the moments of
constructed distributions agree to the moments of z within relative
orders N-l , N- 2 and N-3 respectively. This is clear from step (4)
equat ions (9.2).
~tep
(6). It can be demonstrated that if we integrate the series (8.5)
from 0
to zo'
we actually obtain an asymptotic expansion.
f(z)dz
we have
,ZO
,Zo
F(zo) =
j
o
Now,
V6(Z)dZ
+
/
'0
vl(z)dZ+
...
~Triting
87
in the notation of
,
~arl
Pearson.
Furthermore,
.
!~(zo)l
,•
and
similarly,
•
Therefore
~
We may, therefore, write
P(r l2 <
)
- Zo
10.
.-J
I
t
N+4
3 N+l
-2
.(-~ I· (_,--)-1
N N+2)
~o
2
2
1 N+3) \
\
J
Zo (2- ,2-
Some other' properties of the ser1.es (8.6).
(i)
If the series for
(8.6) is written
where
1 N+l
Zo ("~,-)2 2
f(r ) given on the right hand side of
l
88
(N+4)(1_r 2/ N- 1 )/2
v (r )... _
1
1
.
2N B(l
2'
~~
!.:!)
_
1
2)
Jr 2 _ ~ ~ ,
(1
N+o£. j
I
2
and so on, heuristic considerations show that over the effective range
v 2(r1 )
].
OCr J == 0(;2)
l
v
To show this we observe that the variance of r
is
O(N-I),
so that values of f(r ) outside some rcnge on either side of zero,
l
I
which is 0(N- !2), are negligibly small. Thus r, may be regarded
...
as
0(N-1/ 2 ) over the effective range of rIo
i.e., for
!rll
<
Hence, over this range,
A/NI/2 , where A is a constant independent of N,
we have
...,
-
and
and so on.
This statement is further strengthened by comparing the
maxima of vO' vI' v 2 ••• over the range of r l • It is found that
89
and so
max \ vII
---- =
1
0(-)
N
Similarly,
(ii)
1111'e
make the transformation
=
t
in
(8.6). The frequency function of t
=a
This shows that
is given by
otherwise .
t
(N+l) degrees of freedom.
is cp?roximately a Student statistic with
90
11.
Correction for the sample mean.
In almost all practical cases we do not know the population
mean of the
relation
r1,
XIS.
It is, therefore, necessary to define serial cor-
corrected for the sample mean.
Let
(11.1)
where
and let
(11.3 )
This correction complicates the mathematics considerably, as the characteristic roots of the matrix Ql of the form q1
are not tractable.
However, by straightforward expectations,
-N-1
N
and
•
The independence of the distribution of r 1 from that of p
-/
can be proved easily. Now, [p = N-l, Gp2= (N-1)(N+1), we obtain
/
91
t! -
(ll.h)
(3.4)
From
and
(3.5),
if N >
... 0
where
of
N-l
1
1
v r l ... - N ' 'N=!'" - N
cN_
1
Ql'
3,
we obtain
otherwise,
is the least and
One root of
c
is the highest characteristic root
1
Ql being zero, we always take c N = 0 ~ For
N= 2,
rl
; - 1/2 with probability
For N'" 3, the characteristic roots of
Ql
2.
0, 0, -2/3.
are
Hence
2
.. 2Yl/3
2 2
Y1+ Y2
where
Y1
and
Y2
are independent
N( 0, 1).
Therefore,
f (-r 1 ) dr
1
3 (- 2
3 -r )-1/2 (1+ 2
3 -r )-1/2dr- ,
=;;
1
l
1
For
4,
N=
-2 / 3 ~ -r 1 ~
the characteristic roots of Q'l are
o.
92
c1
= ( jFJ -1)/4, c 2
=- -
1/4, c
3
= -(
J5
+1)/4 •
For N=5,
c1 = 1/2, c 2
It
12.
'8
1.
=<
(-fti
-4)/10'03= -1/2'04= -(
2n ,and
1 = cos N+l
c1 (-Q)
a guess th a t
J2J.
+4)/10
2n
0N_1 < - cos N+f
•
Distr:i.bution of the circular serial correlation coefficient.
Consider
'...,.
=
(12.1 )
p
where xl\T+ 1
=~
and xl' ••• , ~
R1
The exact distribution of
are independent
N( 0, 1) variates.
is givan by R. L. Anderson
The characteristic function, ¢ (t), of c
c
L-2 _7.
is
N-l
(12.2)
(t) =-
rj;
11" (1-2t oos 21!J.)-1/2
j=1
C
N
•
Let
OJ
=
21!-i
cos -N,
•
J
=<
1" 2
••• ,
N- 1
'~ N-l
~ i·/N -en~~)
~ 4~/N' k
~ ( en J
",c='l{",e
e
j=l j
2 j=l
N-1
k
~.
=_~ ~ (~) Nil
..
2
If k
= 2m+1
<
N
r=1
j=l
e 2nij (?r-k)/N
•
93
N~l e2nij(2r-2m-l)/N
== _
1 •
jel
Hence
N-l 2m+l
Z c.
j=-l J
1
k
k
2
r=l
r
=-:I{
Z-()
=-1
If k
2m
==
<
N
m-l
1 _r(?m)N
_ 22m
6 c 2m =- :Tm
m
j=l j
2
(12.4)
For k
~
_
7.
N, we oan proceed on lines similar to those used for calcu-
lating the cumulants of ql.
For N > 2m
Hence for N >
4~
K
l:c = -1
t\"
2:0 = N-2
1<:
3:c = -8
K
4:c
=- 6(3N-8)
.
;
and
"
v1 ·0
= -1
v
= N-l
?:o
v3:0.
= -3( N+l)
/'
94
Furtherml:"re,
t~ pr = (N-l)(N+l)(N+3) ... (N+?r-3)
-
Therefore the moments of HI' up to order
4,
•
are
1
(12.4)
\1
I:R1
-
v2
•
N-l
-
-
N-l
1
CN-l)(N+l) = N+l
,R1 =
= - IN:..13....(N-.+-)-)
v3 :R1
Henoe
1 t
1 t2
t3
3
rbR (t) = 1- N..l TI + N+l -:IT - (N-l)(N+3) 3T + •••
1
•
which can be written as
t
(12.,) ¢R1(t).... e- (1+
t
t
2
IT + -2;
3
+
t3
51
(N-l)(N+j)
=e
-t(
N-2 t
2
1 t
+ ..• )(1- 1\T~l It + N{.l2r
1
t3
Ji
+
t
3
t4
1N+1HN+j) 4T +
2
N -N-4
t
••• )
2
1+ '1\T:I n + (N-!HN+lj 21
+
N3+5N 2-14N-48""1.=+
N3+3N2-10N-24 -+-......,.....,
t3
t4
(N2~1)(N+3)
3L
(N 2-1)(N+3)
4\.
••• ) .
Let also
(12.6) ¢nR (t)=e-t;-F(~,N_l,2t)+a1tF(~,N?1,2t)+a2t2F(N+2,N+3,2t)+ ••• 7.
1
2
2
2
-
95
Comparing (12.5) and (12.6) we determine the values of
a , a2~ a
l
3
and
a4 as
t;. = 0
From (3.4) and (3.5), taldng
~.
(N-4)/2,
~
= (N-2)/2, we obtain
Or,
=0
otherwise,
where b , b
... can be determined in the way outlined above, and
. ~~perscripts of the polynomials are (N-2
where tne
.---, N~i')
t . The series
5
6
(12.7) is exact up to
4
2
2
terms, that is, the first four moments of the
fitted series are equal to the moments of RIo
The remaining moments
96
are approximated at least to 0{N- 2 ).
terms, close to
5 per
Neglecting b P5(Rl ) and later
5
cent values of the positive tail of the
bution are oalculated for three values of N.
comparison with the exact
N
It serves as a rough
5 per cent values given by R.L. Anderson
Calculations
Exact
5 per oent
value
10
peRl .:: .36)= .0496
.360
30.
peRl .:: .26)= .0483
.257
IP~l ~
.18)= .0610
.191
~
.20 ),* .0439
60
peRl
distri~
The closeness of the results is very satisfaotory.
The calcu-
lations were carried out with the help of Karl Pearson's IIT ables of
the Incomplete Beta-Functions", which give values of the argument only
up to two decimal places.
Since our object is to illustrate the theory
rather than providing a table of percentage
point~,more
refined calcu-
lations were not attempted.
A proof of the asymptotic behaviottr of the distribution function
of Rl obtained from f(R l ) can be developed on the lines of a proof
which is given in section 9 for the distribution of r •
i
A similar method applied to the circular serial correlation coefficient without mean correction, R , where
I
97
gives an approximate distribution of R , if N is even, as
l
(12.8)
::: 0
otherwise
e
The first N+4 moments of (12.8) are exactly equal to the moments of Rle
13.
.90ncluding remark.!.
We conclude this chapter with the remark that the method of de-
veloping the distribution of a serial correlation coefficient in a
product of a beta distribution and a series of Jacobi polynomials
produces better approximations to the true distributions as compared
to the method of smoothing the characteristic function.
method has been widely used by several authors.
This latter
For example,
Dixo~
-7 and Jenkins -.;-21-7 have obtained smoothed distributions of cir-
~9
cular serial correlation coefficients and their functions.
If we com-
pare Dixon's distribution of the first circular serial correlation coefficient uncorrected for the mean, i.e., in our notation R , with
l
the distribution (12.8), we observe that the first term of the series
(12.8) is the same as Dixon's frequency function.
But whereas in the
smoothing method there is no way of improving the accuracy for a fixed
sample size, our teohn1que provides a method of improving the accuracy
of the fitted distribution to any desired extent.
Even for small sample
98
sizes, by taking sufficient nuraber of terms in the series; we can improve the accuracy to any desired level, as the successive terms decrease very rapidly.
This was demonstrated in section 12 by computing
the probabilities of the circular serial correlation coefficie.t corrected for the mean, RI , from (12.7) and comparing them with the exact
values given by R.L. Anderson
Even for a sample of size 10,
L-2_7.
and utilizing only first two terms of the series, we obtained a result
which is correct within 0(10-4).
Finally, it may be observed th&t approximate distributions of
partial, multiple and other ordinary serial correlation coefficients
can elso be obtained by the proposed methodo
In the Case of first
two serial correlation coefficients and their functions, it will be
carried out in Chapter
4.
J
e.
CHAPTER IV
DISTRIBUTIONS OF PARTIAL AND MULTIPLE
SERIAL CORRELATION COEFFICIENTS
1c
Strrllll1ary •
It is indicated by section 3 of Chapter 1 that we require the
distribution of a partial serial correlation coefficient, b, between
and x _ eliminating x _ ; and of a multiple serial correlation
t
t 2
t l
coefficient, R, between x
and the combined variable (xt _ ' x t _2 ),
l
t
to test different hypotheses concerning an autoregressive soheme of
X
order 2
Since band R are functions of 1'1
and r 2, it is necessary
to investigate the joint distribution of 1'1 and r • A method which
2
is due to Koopmans L-28_7 yields the exact distribution of 1'2' when
the sample number, N, is even.
It is also applicable to the deriva-
tion of exact distributions of serial correlations with even lags when
N is even.
sections
These distributions will be developed in the following two
0
The distribution of rr.
~
for N even is defined by
par ts of the range; furthermore;> its exact distribution for
not available.
diff~rent
N odd is
r , N even
2
or odd, will be obtained in terms of a beta distribution and Jacobi
polynomials.
Therefore, an q>proximate distribution of
In later sections approximate distributions of band
R will be obJuained
0
lno
?•
Cumulants
of_.....:5.
q
_
____
Let
•
¢s(tjN) and xs(t;N) denote the characteristic and cumulant
generating functions, respectively, of the form
qs
where s
=
N-s
~ x(i)x(i+s),
i=l
is a natural number < N/2 and x( i), i=1,2, ••. ,N are inde-
pendent 1\T(O,l) variates. We will use the notation xCi) and xi
terchangeably.
If C denotes the (N x N) matrix
C
then the matrix Qs
where
,
C
0
1
0
0
0
1
0
0
0
0
0
0
·..
·..
qs
0
0
0
0
1
·..
0
0
of the quadratic form qs
Q
s
is
= (C s + C's)/?
is the transpose of
60
0
::0
c.
Let ms be the greatest multiple of
remainder,
in-
6
in Nand
j
the
that N = ms + j.
can be written as the sum of the following
s
quadratic
forms ~
rn-l
q ~ ~ x(r+ks)x(r+k+l s); r
sr k=O
=
ly2, ••• yj
m-2
qsr "., k:OX (l:'+ks)x(r+k+l s); r""j+l, •.• , s •
,
101
If s = 1 then j
= 0,
s > 1, then qsl' qs2'
and we have found the cumu1ants of q1.
~.t,
qss
If
are all forms like ql' and no x
which appears in one of them appears in any other of them; hence they
are mutually independent.
of the x' s
Each of the first
j
forms involves (m+l)
and each of the las t (s-j) forms involves m of the x IS.
Therefore,
•
Or
m+l
=L- )('
k=l
.
(1-2t cos
m
j
k\.) -1/2 /- )( (1-2t
m+2
- k=l
7
cos ~) -1/2 7S-j
m+J.-
which, incidentally, proves
THEOREM 2.1.
The characteristic roots of the matrix Qs
cos~, k=1,2, ••• ,m~1, each repeated
are
j times;
kn
cos m;I'
k=1,2,o.t,m, each repeated ( s-j ) times.
We observe that if N is a multiple of
m distinct roots of Qs
B,
given by
cos ~:1 ,k=1,2, ••• ,m, each repeated
Since
i-G,
n
_7,
cos
i.e., j = 0, there are only
s
e is a decreasing function of e
10
in the range
we have
kn
kn(k+l)n
k 1
> cos ~~
J "= ,
cos -:-'2
m.... > cos ~l
JI'-~
In+£:
If j
times.
and the distinct roots of Qs
0.',
,
nl.v_
are denoted by 01' 02' __ .,
102
02m+l
in decreasing order of magnitude, La.,
°1, >
(2.6)
02 >
0'.
> c 2m+ 1
'
then
kn
c 2k _1 : cos m+2'
C
= cos ~n:'
2k
k=l, 2, •.• ,
~l;
k=l, 2, •••
m.
Taking the logarithm on both sides of (2.4)
If the rth cumulant of
qs
is denoted by
K
r
(q ~N), then it follows
s
from (5.6) of Chcpter 3 that
(2.8)
K
2r+l
(q :N)
s
= 0,'
r= 0,1, •••
Furthermore, if r ~ m, then from (5.9) of Chepter 3,
For r > m, we can similarly write down expressions for
It
2r( qs :N)
with the help of equation (5.9) of Chapter 3. Hence, all the cumulants
of q
s
are known.
THEOREM 2.2.
qs' when standardized, is an asymptotically N(O, 1)
variate.
The proof is similar to that given for the asymptotic normality of ql.
103
3.
The exact distribution of r 28 when N is even.
Let
P --
N
'-'~x·2i
i=l
and
JHEOREM
3~1.
If
N:::: 2sm + 2j, where 2sm is the greatest multiple
of
28 in N, so that
r 2s
=.
y, when j
:J
j <
s, then the freauenoy funotion
f(y)
of
0, is given by
(3.1)
(N-2)/2 r-l _ cs - j - 1
+-_.1:/
.1
( s-j-l ) i=l-.~w8-J-
if c?L-I' < y -< c 2r- l' r=l, 2, ..• } m ,
=0
otherwise ,
f
where 7t" indicates that from the product that factor is omitted
which involves the characteristic root with respect to which
differ~
entiation is being performed.
If j
PROOF.
it.
~ O~
the first summation is to be disregarded.
We only consider the case
j:f
0, j :::: 0 being a special case of
104
The joint characteristic fUnction of p
¢(t,Q), where
t
corresponds to p
and
and Q2s
Q to
denoted by
Q2s' is given by
From here we can proceed on the lines of the argument given by
Koopmans ~28_7 in his discussion of the distribution of a ratio of
tWQ quadratic forms.
Without reproducing his arguments, we arrive at
the following integral expression for the distribution of y:z r 2"s '
f( y )=. (N-2)/2
2ni
where Yy
is any curve proceeding from z
plane, crossing the real axis at a point
z
~
=y
into the lower half-
z > 01' and returning to
y through the upper half-plane.
Now, if g(z) is a function analytic within and on a closed
curve Y, we have
I
•
'2nI
(
g(z)dz
1
I
r = (r-l}t
!
(z-a)
Y
2l r - 1
() r(a
1: g(a)
,
,.. 0, if a :f.s outside of
Hence the tb-aorem follows.
•
if a is within or on y,
Y •
105
4.
The distribution of r •
2
If N = 2m, the distribution of r
3.1 by taking
s = 1 and j
=0
2
is obtained
fr~m
theorem
I
, if c 2 2
u+ < r 2 .::: c 2u '
u
:: 0
=:
1, 2, ••• ,m..l
otherwise,
where
un
c 2u "" cos jji;I ,
u". 1, 2, •.• , m.
The frequency function (4.1) is exact but of little practical
use if N is even moderately large, say 10 or greater, due to the
involved calculations for percentile points.
For
N odd, the exact distribution can be put only in the in-
tegral form.
We therefore use the theorem 3.1 of Chapter 3 and obtain the
distribution of r
2
in terms of a beta distribution and Jacobi poly-
nomials.
Follo'tl1ing the method whi.ch gives cumulants and moments of
we obtain the moments of q2' if N.:: 6, up to order 6 as
v
2k+ 1tq 2
== 0
"2:Q2 :: N-2
k ""' 1, 2, ••.
ql'
106
2
V4:Q? ~ 3N +6N-48
v6:Q?
==
15N3+180N 2-60N-3600.
In general, the term of highest power in N of
k
( 2k··1 ) N'"
v
•
2k ·Q2
is 1.3.5 •••
Hence, th e moment s of r 2 are
k=1,2, ... ,
N-2
)\T(N+2)
==
N(N+2) (N+4)(N+6)
==
3+180N2-60N-3600
15N
•...;;.,;;.;---.......;"-'-------N(N+? )(N+4) (N+6 )(N+8) (N+10)
_
and in general
On the lines of the proof given for the ;proach to normality of r ,
l
it can be proved that r 2, when standardized, tends to a N(o,l) variate
(4.2)
as
N -> co.
Writing r 22 == z,
¢ (t)=-l+ _N-2
Z
N(N+::»
Let us also write
0z (t)
J.
c an be written
~+
It
t2
3N2+6N-48
-+
N(N+2)N+4)(N+6) ?1
... .
107
ri (t)=
'fl Z
(4.3)
Comparing
~o
- r ' N+4k+~
2 -,
J.-)
F(2k+l':
~O 8 k .
u
(4.2) and (4.3), we dete:CTilL,a up to
8
3
,
Hence from section 3 of Ohanter 3, the expansion of the frequency
function of z, up to third degree polynomial, is
if 0 <
=0
(N-l,_
Now,
Q~
1)
2
(r~)
otherwise
Z
<
006 2
n
NT1 7
I-2
- +1 '
•
(!-2.~,N2-1)
can be expressed in terms of
P2
(r 2 ), and
thus we obtain
=
if -
n
n. ..
< r < cos
rN~. 7+)1
L-~ 7+1 - 2 -
006----
2-
:' 0
otherwise •
-
2-
103
Now, if Zo
= r~,
then from (4.4) ,
2
)
p ( r 2~ ~ r O
=I
~
= F(z
zO) = F( Zo ) ' say.
1 N+l --r,;",;\
N+4
/- I (-,--)-I
3 N+1
1 N+l 7 •
(-,-)(~,--)
N,N+l) ~ Zo 2 2
Zo 2 2 -
Zo 2 2
Hence, apart from the fact that the difference
I
Zo
(1 }l:!)_
I
2' 2
Zo
(o! !:];)
2'
will be considerably smaller than I
2
Zo
(1~) we observe that the
2' 2
'
second term is O(N-l ) as compared to the first term; and therefore
the series is an asymptotic series. We can write
(4 •6) P\r 22 .:5 TO2) =3:.
I
(1
N+l
(N+4)/-
2'"2,-r)- NnT+!J-
ZOo
(3 N+l)
1
I 2 ~2'T -I 2("2
TO
rO
N·:·l)
s2
-
7·
Since
a2 : : .- 8 : : .- 0, the error in (4.6) will be O(N- 3 ) or less.
3
5.
Bivariate moments of r
l
and r •
2
To find the bivariate moments of r 1 and r 2, it is not possible
to use the joint characteristic function of ql and Q2' as the
matrices of these forms are not commatative and cannot be reduced to
their diagonal forms simultaneously. Hence we proceed with the straightforward method of calCUlating the expectations of products.
We first note that
109
2r 1
"B. X i +
= 0,
r=O,1,2, ... ,
and, therefore, in taking the expectations we disregard all the terms
which contain odd powers of
J
~
2 2
t,qlq~ ~ ~(
c
XiS
as a factor.
N-l
2
For example:
N-2
Z x:-x + ) ( ~ x.xo+ 2 )
i=l ~ i l
j:l J J
powers of
XIS
as a factor_7,
the double summation extends over values of i
subscript of x
2
and
is less than 1 or greater than N.
j
such that no
Thus
or
e
2 2
C~qlq2
2
: N +9N-30
t
. We have used the fact that· x 2=1,
f x~3
The results of the calculations are the
•
fol1owing~
lIn
t- 23
2
6 e
4
G
qlq2= 6 N +48N-21 , (., qlq?= 0 ,
(:
q~q2=
0,
t qtq~= 3N3+108N2+159l\J-l~.J4, t:)qiq~= 0,
tqiq~= 3N3
'''''riting
'lJ
ks
+
=
2
66N + 417N - 25?6,
tqlq~ = 0
,c' k s
urlr2' we can summarize the moments, including
the univariate moments, up to order six as follows:
Order 1:
Order 2:
Order 3:
Order 4~
Order
5:
III
Order 6:
_ 3N3+66N2+417N-2526
))24- - N(N+2) ... (N+101 '''15 =
(5.1)
0
,
_ 15N3+180N 2-60N-3600
"06- -N(N+2) •.. (N+IO)
6.
The distribution of partial
seri~~
correlation coefficient.
In an autoregressive scheme of order 2 ,
we have seen that
(6.1)
where
PI
lag 1 and
and
P2
are population autocorrelation coeffioients wi th
2 respectively.
It is easy to see that
partial autocorrelation between x
ed.
Substituting the estimates of
estimate of
~
(6.2)
t
and
PI
is the
-~
x _ when x t _l is eliminat~
t 2
and P we get a consistent
2
as
b ::
Further, define
as
the multiple autocorrelation between x
t
and
112
A sample multiple serial correlation R can be defined by
1 - R2 =( l-b 2) (1-r12 )=
"2
Since r 1 < 1, the condition R > 0 gives
We already know that
if m -- /- -N+l 7,
?-
from which we conclude that the limits of the variation of Rare
['0, 1
_7
and thoBe of bare
.r- cos m.~ , 1 _7.
We determine approximate moments of band R by the following method.
Fr<ri' (6.2)
(6.5)
+ (
k+2
(5
3 )r1
+ •••
..
"
113
Taking the expectations on both sides of (6.5) with the help of (,.1),
and rearranging the terms, we obtain
(6.6)
,: (1 . )k_ 1
~.
+0
-
+
2
-5k+6)
U{N+?)
k(k+l) + k(f'2k
?(N+?)
k(k4~16k3+,lk2_76k+16)
... ---sN(N+2)(N+4)
This approximation for i(l+b)k is valid for
compari.son to N, i. e ., for
need more terms.
k <
IN.
1
I
+ O'N(N+2)(N+4J{l\J+6)'
k small in
For higher values of k we will
OmittL1g the terms of o(N- 4) we have
;:l
( (l+b)
and
Unfortunately it is not possible to determine exactly the moments of
(l~b). However, working with the approximations to the order N- 4 for
these moments, let us
~~ite
terminating the series at
the characteristic function of (l+b) as
a2 " Proceeding in the manner of section 3
of Chapter 3 we obtain
N+3
1ii+'2 ...
or
_ N+3
81
3
- "N+"2 - N(N+2}(N+4)
,
114
a =
1
N+5 N+5
2a 2 + ?a1 N+G + ~
=
3
•
- l\T( N+:? J( N+4 ) '
N+5
1
11
N+2 - N(N+2J - N(N+2)(N+4)
~
or
Hence,
0:
0
otherwise.
Now,
and
+
N+5 ( I-b) 2 _.
7
N+i
Furthermore,
and
Hence, for large N, the effective
ran~e
of the variation of b is
115
1/2
Therefore, in thi.s ranl'.e, the second and third terms :ot·. the
series in the square bracket are O(N-51?) and O(N- l ) respectively.
Ordinarily, we would expect the second term to be O(N-1/2 ) ; but in
O(N-
).
a is O(N- J ) instead of O(N-l). HowQyer
l
it can be proved, as in the case of r , that the distribution funcl
tion of b obtained from (6.7) will be asymptotically valid reprethis particular case
sentation of the true distribution.
n
If we consider c o s - - - - - -
-
r(N+l)I?+l
.-..J
7
1
and denote by
-
[r(l+b)k the expectation of (1 + b)k with respeot to the function
(6.7), we find after rome reduction
f.
v
(l+b)k= 1 + k(k+l) + k(k J -2k?-5k+6 + O( 1 )
f
2{N+2)
6N(Nv?)
3
'N
1'1
comparison with (6.6) shows that Cf(l+b)
O(N- 3 ) •
A
7.
k
k
The distribution of multiple serial correlation coefficient.
In section 6 a definition of a multiple serial
between x t
Hence
e
agrees with ~(l+b) to
and
correl~ion,
(x t _l ' x _2 ) was given by the equation
t
J
Expanding the factors, we obtain after rearranging
R,
116
: 1 _ 2k ~ k(4k-l)
N+2 !,frN+2)
+
0(1 )
;J'
the approximat ion being good for k small as compared to N.
Writing T = 1 - R2 and the characteristic function of T,
¢T(t),
rl
in the form
'{IT
(t)=FCM
2'
~)
(N+2
N+6
2' t + ~ t F""2 ' 2' t) + •••
,
we can only determine al effectively as a2 , 8 , •.. require more
3
exact knowledge of the moments of T. We therefore consider the
series for ¢T(t)
terms
to be terminated after two terms.
omitting the
O(N-3 ) and less, we have
3
N(N+2)
Thus the distribution of T may be written up to the first degree
Jacobi polynomial as
otherwise.
Or, the distribution of R2 is, to the s&~e degree of approximation,
~
0
(7.3) f(R 2)dR 2= E(I_R 2 )(N-2)/2dR 2rl+
.
2
-
3(N+4)~(1_R2_~) 7
4N
1'1+2 -
for 0 < R2 < I
-
=
0
-
,
otherwise
First we observe that the moments of the distribution (7.3)
2 Secondly,
agree to the order N-2 with the actual moments of R.
2
since [. R = O(N- I ) and the variance of R2 is O(N-2 }, :in the effective
117
l
range orR the second term in (7.3) is O(N- )
first term.
as compared to the
Thirdly, the cumulative distribution function of R2
is
or
If R2 is O(N-l ), the second term is
o
O(N- l ) rel~ive to the first
term.
Another method
The kth moment of the distribution
is
xk =
N
T+'2T{" •
If 2k < N, this can be expanded as
2k
(1 - ~~ +
4k2
---2
+ ••• ).
N
If
k is small compared to N, these moments agree with the moments of T
to order N-l • Let us, then, write an approximate distribution of T
as
where (N;2T -
¥) is first degree orthogonal polynomial with weight
.
i'uneticn
T(w-2)!2
. 'J
and range
r
7•
- 0,1-
From this, we have
118
j 1Tf*( T)dT= mN
+
o
N+2
'2N 81(lmi
N
- N+2)
Equating this to the first moment of T, we obtain
a
1
= 3(N+~)
2N
Therefore, negleoting terms
+ O( 1 ) •
;'-
0( N-2 ) in
8
1
,
which agrees with (7.2). This series may be extended by taking additional terms and equating corresponding moments.
In (7.3), make the transformation
Then, the distribution of F is given by
N+2
)
F(N-2 /2
=(.N)2
2
(1+
-
3(N+4)
~)(N+2)72 L 1+ -fiN"-
7
¥) ,- ,
F-l
(1+
... -
O<F<oo
,
which shows that F is approximately distributed as Fisher I s e 2z ,
with
Nand
2 degrees of freedom.
119
8.
Extension to multivariate case.
We shall state a theorem suggested by theorems 3.1 and 3.2 of
Chapter 3.
The proof is similar
L~
nature.
THEOREM 8.1.
Let ¢(t l , ••• , t u ' 91 , "', 8v ) be the joint characteristtc function of variates zl'
Y],
•.• , y,
where 0 ~ b i
..
v
.:: Z i
.:: c i .:: 1, i
2 ••.. ,
Vo
= 1,
,-.1.:: d j .:: Yj .:5 e j :; 1,
and
2, .•. , u;
j= 1,
Let it be possible to write
00
( 8 .1)
¢( t , ••. ,t ,9 , •.. ,0 ) =
l
u 1
v .'
'. L;
t' •.
~l,·····~,:"'u,Jl"'~'
v
7t
j' 0
a
va
. j
.
i l' .• J. 1'" J
v
u
j
9 r F(6 +j +1, Y +6+2j +2,2Q), all a,~,y,6 > -1 ,
r=l r
r
r
r
r
r
the series (8.1) being absolutely convergent for all values of t ,
l
" ' J t ' el,.·.,Qv.
If the partial sums of the series are dominated
u
by a function
~
of these variables such that
~ exp
ltlzl+ •••+tuzu +
QlYl+ •• 0+8vYv l is integrable, then the joint frequency function of the
variates is
v
(8.2)
IT
r=l
L
......
00
B(a +l;P.!_
+1) i s 1__
u ~_
"IT
il'
••
j'j
•..
j
i l' ••• ~ j 1""=O
v s=l B(n +1 +1,~ +i +1)
s s
s s
L;
a
1:'0
B(r +1,8 +1)
r
r
r:l B(y +j +1,8 +j +1)
v
IT
'r r
r
r
within the range of the variates and zero otherwise.
9.
Bivariate distribution of r 1
For a given r
as ["2r 2 - 1, 1
1
_7.
1
and
the condition R ~ 0 gives the range of r 2
But
since r
2
in absolute value cannot exceed
1t
7 we hav'"e
m = ; - ~N.I,'
....,;..:::,
cos -1
m+ where
2-
Similarly, for a given r 2,
For a given r , the range of the variation of b is
1
n
1- cos -~"'i'
m~·.1.
( - 1 +-';--0
h'
•
1 - 1'1
As
N ->
00
so does
Furthermore, (I'J.b -
cos
1t
m+ l
)
m and this range approaches
f.b . . . O.
L--1, 1 _7.
and
l
b, though not independent, are asymptotically independently distributedo
b
[rl
to
This indicates that
r
Hence, we shall consider the joint distribution of r
l
and
as an intermediate step to finding the joint distribution of r
l
and
and
let
¢( t, Q)
be 'bha joint characteristio
121
function of
of z
where
z
and b
aOO=l.
and
b.
From the knowledge of the first few moments
let us write
But
Therefore, we obtain the equations
e _
v
z - alO
1
+
N+2
tZ(l+b)
and so on.
m-3 we
above.
Since the moments of (l+b) have been calculated to order
do not attempt to determine coefficients beyor.d those listed
1ftTe
find to order
a
_
a01 -- - N[fti. 3
2 ) ( H+4)"' 10 - -
1
t"N+2) ,
flT
,
1?2
Now~
t' z(l+b) = ~ - N(N+2~(N+4j + O(~)
The value of all turns out to be
O(N-4),which we shall neglect.
Hence to the second order polynomials the joint distribution of z
b
and
is
in the range of z
and
b
and zero otherwise.
Making the transforma-
tion
we finally obtain the joint distribution of r
of'
,
.~
(9.,) ... (rl'r 2 Jdr1 dj.2
_
r i+
(1- 2r
-1\.
r 2) (1-!'2)
2
l-r1
l
1(N-I}/2
- J'
and r
2
(1-r 2 )
as
r""
-T~ dr1 "'.l.2
(1-r )
1
123
(
.
...
i ? a)(-im
1 N+8
I- 2r +r2
l-r l
within the range of variation of T1 and r , and zero otherwise.
2
The asymptotic behavior of the series (9.1) fer the characteristic function of r l and b can be established in the same manner
as was given in section 9 of Chapter 3 fer the series representation
of the characteristic function of r • Also, in the effective range
l
of r
and r , it can be shown that the terms in (9.5) decrease
2
l
rapidly and the first term may serve a good approximation for even
moderately large N.
10.
Concluding remarks.
Starting from the joint distribution of r
and r
given by
l
2
(9.2), we can obtain an approximate distribution of any function of
these variates.
It may be noted that the distributions obtain::d in third and
fourth chapters are quite good even for small samples as the theory
..
permits development of the series to any desired order of approximation.
CHAPTER V
• LEAST-SQUARES ESTIMATION lNHEN RESIDUALS ARE CORRELATED
1•
St'1.:mmary.
In this chapter we will study the effect on the least-squares
estimates of regression coefficients and of the variance of residuals
when the condition of independence of residuals is violated.
The
maxima and minima of a bilinear and a quadratic form will be obtained.
These will be utilized to set up bounds on the variances and covariances of the least-SQuares estimates
of regression coefficients.
The
cases of a linear trend and of trigonometric functions will be considered in more detail.
The material in section 2 up to equation (2.11)
is not new except for its oresentation, and is adapted from Hotelling's
lectures on the theory of least squares and regression analysis.
The
rest of the chapter is original.
2.
feast-squares estimates and their
Let
covar~£~.
Y1' .", YN be observations on a variate and let
p
y~;
~ ~iXiy + 6 v ,
'i=l
y = 1, 2, "', N
0
I
The observations on Xl' "', x p are considered fix8d from sample to
sample. Let us write 6'::t (6 ,
, 6N) and 6 for itif3 trBnspose. In
1
regression theory it is usually assumed that N > P and tha~
...
(i) ['!J
= 0
y
nn)
( l.l
;.,,,2
2 f
LI = 0'
Y
L
~r
or 1 ,
2,
••. ,
1N
,
125
(iii)
C6y "0
(iv)
If
NC~,~)
for y ,,6
,
is distributed normally.
6
stands for a normal distribution with mean vector
covariance matrix
(1)
°
=
~,
the above conditions may be
b is distributed as
identity matrix and
°
NCO, IN)' where
stands for the N x 1
Write
,
,
=
Y
(2.2)
~
x(p x N)
=:
(Yl '
. ., YN)
(~l'
... , ~p)
summ~ized
~
and
as
IN is the N x N
zero vector.
=:
,
l._.Xpl
(X
==
iy ); i
xp2
==
x pN
1,2, •.• ,p,
y=1,2, .•• , N ,
a(p x p):=- xx'
-1
c
=
a
g
=:
xy
and
We note that
a
,
=:
a
and c
, =e
We will use a prime on a letter to denote the transpose of a matrix
or a vector represented by that letter.
126
We suppose that N > P and that the rank of x
the least-squares estimates, b of ~ and
.
8
~.
Than
of a 2, are given by
2
= cg
(2.3)
b
(2.4)
82
t
= if
,
y - b g
N-p
= y'y
_ bIg
:I
n
where n.: N - p.
is
Furthermore, if we write 6q
=q
-
lq,
where q
is a variate, then
,:>
B(p x p) =
C,6bl5b
1
= (]
2
C
Now,
I
1
j; '
= -':'(xx
)-
t~b = Ccg
•
= (xx t )-1
xy
xx I ~
=~
as
£ y:::: r(x'~
+ 6)
~ x' f3.
It is also known that under the condi-
tion (1)
~ 2
cs
=
(j
2
We propose to consider these estimates when condition (iii) is
replaced by
(iii)*
r /'), y If6 = a2 P f y-s i '
I...-
~~ich means that condition (1) is replaced by
(2)
6.
is distributed N( 0, (]2 p), where
127
...-
peN x N) =
(2.8)
1
PI
P2
• ••
PM-1
PI
I
PI
·..
P
I
·..
P l'J_ 3
p
P2
P N- 2
N-l
I
P N-3
-'
N-I
i
'i
Z
Pi(C + C )
+
...,.
.......·.
P
= IN
I
,
i=-l
and where
C
r,
::-
;
I
0
I
0
·..
0
0
0
0
1
·..
0
0
•
.
•
e
I
.
0
0
0
0
0
0
' ..-
.... .
·..
We still find
Let
,
v=y-xb
Then
I
"
,
,
- b x)(y - x b)
,
=- yy-bg
,
and
s
Also
2
,
=-
- .
vv
n
,
v=x~+6-xb
=
•
0
I
0
0
-
...
v v =- (y
N-2
x' (~ - b) + I::.
I
128
I
= (IN - x ex) /).
as
b=~+cx/)..
lrV'rite
>.. is
a
m
1 2
:0
= m and m = m.
x ox; then ml
It follows that if
root of m then >..:: >..2, so that, >.. = 0 or 1. 't1Triting tr
for trace of a matrix, we have
tr m = tr
I
:it
ex
tr xx
=
I
0 =
tr I p
=p •
It follows that
p
roots of mare 1
and N-P
=n
of them are
zero. This can also be seen by observing that
,
x(IN-m) = x - xx ox
.
,
=0
i.e., there are P relations among the columns of
e
(/ V
I
I
I
- t tr
6 (IN-m) (IN-m) 6-
= f. tr
II I (IN-m) II
::: t tr
M
=>
If p:: IN then tr
vI v
v:: ttl'
=0.
2
I
tr P -
N cr
2
Now,
~-m.
-
C1
(IN-m)
,l2
tr P
In
tr Pm.
)
2
I
Pm=- tr m =- P, ~:. v v = n
(J
.
,
2
and (. s =
C1
2
as
stated in (2.7).
If L::
(fij ),
i, j
= 1,2, ••• ,N, 1s a matrix,
N..k
tr L(e k + elk) :: 2 Z
1=1
and
f k,k+i
we observe that
129
Therefore,
N-1
tr P m ~ tr m P = tr /-mI +
-
:: p +?
and
(2.10)
p
(6
2
TIT-I
N-k
E f) k
k=l
Z m k+
y=l y, Y
,
-
'k
)_7
N-1 N-k
~
v v
2/-1 2
=L-=a
-(!
n
k
E f) ktr m( C +C
N k=l
'I:'
£J
n
£J
k""l y:1
Now,
"
,
..
2
:: (1
'I
t l5y5y x
t 6g15g = x
X
P
X
t
and
(2.11)
B
=
<>
1..ob6b
I
=
(1
2
C
I
x p x c
Write
d(p x N)=-(d
jy
)= ex, j = 1, 2, ... , p,
and let Bij = Bji be the element of B
L~
y:::: 1, 2,
ith row and jth.eo1umn.
We obtain from
'4
B =
2
(1
dPd
2
,
-
= cr d/ T7~ +
-
and
that
L_d(C k+C 'k )d ' _7ij
=
.'1
... , N ,
N-l
k 'k
'
E P k( C +0 L7d
1<=1
N-k
Z (d. d
+ d d
)
y=l
JY i,k+y
iy j,k+y
1.30
(2.12)
Bij=cov(bi'b j )
2
=CJ
Since eaoh b i
N-l N-k
N
r i: dj; d. + i: i: k( dJ'ydi,k+y+di.y+dJ' ,k+Y) _7.
- "(=1 Y JY k=l y = l '
P
is a linear function of 6's, the veotor b is
N(~,B).
If by a proper ohoioe of XIS
f
XX
so that
a
= c = I p,
2 _
(2.1.3) Bij =
CJ
L 0i'+
J
= I
we make
p
,
we have
N-l N-k
i:
i:
k=l y.l
P k(Xj; x. k+ +x j Xi k+ )
Y J, Y Y , Y
where
0ij = 0 if i " j and 0ii = 1. .
.3.
Bounds on a quadratic and a bilinear form.
We notice that i~ s~
and the
_7,
Bij'S contain bilinear and quad-
ratio forms in the x t s. We wish to set some uniform bounds on these
quantities for different sets of non-stochastic variables.
THEOR~M
.3.1. Let A = (aij ) be an N X N real symmetric matrix. Let
a l , a2' ••• , aN be the characteristic roots of A where
a l :: a 2 ::; ••• .:: a W_ l .:: aN •
,
f
•..• x ), y ~ (Yl' ••• , YN) are row vectors in real
N
quantities with x and Y their transposed column vectors, then,
If x
= (xl'
under the oonditions
('
,
(i) x x=l, ii) y y=l, and (iii)x y=O, ~ xrAy has a maximum
,
(a~ral )/2
and a minimum -( '1J-al )/2
131
PROOF.
Since the problem is invariant under any. simultaneous orthoga
anal transformation of x
y, we assume that
and
A
is a diagonal
matrix. Hence
Usi ng Lapl ace's method of multipliers, we write
The ma:.'Clmul11 and minirm:un values of )"
and T are the same.
values
;;T
- - - - == a y.
~xi
,i ~
- my. - kx i = 0 , i = 1, 2, .•• , NJ
~
Therefore,
Zy. -
Zx
aT
1
aXi
j
.J.!.
2fij
= Zu; i y i2
- m == 0
== Ea .X~
-
J J
Zy. ~~- == )" J
01,-;
v
which give
f =k
== 'X,
m == 0
f =0
,
For these
132
The equations (3.4) become
-Ax i
(ai-v)Yi
+
(ai-v)xi - AYi
t
Let z =(x
I
i
y).
= 0,
i
= 1,2, ••• ,N
0,
i
= 1,2, •.• ,N •
a
,
Then for a non-null solution in z of the above
equations it is necessary that
-AI
A-vI
A-vI
-AI
I
I
a
0
I
!
or
N
Jr ~a.-(v+~)
7/-ai-(v-l) 7 = 0
~1
~
--
•
This means that either
v+l
= a i for i =1 or 2 or ••• or N
v-A
= a i for i = 1 or 2 or •.• or N.
Qr
Hence the stationary values of A are obtained from the above given
values of v + A and
ditions (1) - (iii).
\I
-
A provided they are consistent with con-
Now,
= ..
= ..
1.6
ij
>"61j
,
133
L-
i,j=1,2, ••• ,N and 0ij ~
°1 ifif i::
i f
j
j
The Messian, 1. e., the matrix of the second partial derivatives, H,
is given by
(3.6)
H(2N x 2N)
L
>:
-AI
A-vI
A-vI
-AI
The characteristic roots of H are given by the equation in k
=
Loe. ,
k = (v-A)-a.,
a.-(v+A),
i=1,2, ••. ,N.
~
~
°,
~11
the ?N roots will be
non-negative if and only if
V-A = a N
Le., if
A
= -(aN
-( 1 ) /2
V
= (a1
aN)/2
+
Hence
the mruhmum value of A is -( aN-a ) /2.
1
A attains its mil1tmum is
z
i.e.,
,
.
=(1/ /2,0, uo,O,l/
n,l/./~,O, •.. ,O~-l/ .(2)
x' = (1/ I~,O,
y'= (1/
one vector for whi ch
0
..
,0,1/
/?)
\/2,0, ... ,O,.el/ ~).
Similarly, all the 2N roots of H are non-positive if
V
+
\I -
A :: a
!IT
A = a1
,
134
and the maximum value of A is (aN - a l ) /2.
Thus
-(aN-a )/2 ~ Amni < A -max
< A
= (aN-a l )/2 .
1
3.2.
-THEOREM
tiona
In the notation of theorem 3.1 and under the same oondi-
where
,
,
v=xAx=yAy
PROOF •
.
If aN_1 = aN' the veotors
x* = (0" 0,
* = (0,
y
make
v
0,
... , 0,
1/ /2, 1/ /2)
... , 0,-1/ /2, 1/ .(2)
= aN and the maximum of v, in this case, is aN.
a~_l I aN' v r aN' as this implies x~ = y~ = 1, xl = x 2 =
•.• = x N_1 ~ Yl = Y2 = ••• = YN-l ~ O~ which is contrary to the condiIf
tion x'y = 0.
and for
If
.
2
xT\T
v
max
< ...L /:"~,
For the vectors x* and
y*
the value of v is
135
2
Simi.larly, YN ~ 1/2, i.e.,
(3.8 )
But
(X YN)2 ... (xlYl+ ••• + xN_1YN_l)2
N
N-l 2 N-l 2
Z x.)( z Y )
1
~ (
1
1.
(l-X~)(l-Y~) ~ 1/4 for xN'
...
.
1
It follows from (3.8) and (3.9 )'thct for
B
For these values of JCN
a'l
v ...
where
vmax '
(tN/2
+
2
(alx1 +
N~l CttXi
under the condi-
1
~_1/2
1
2
which is achieved for x N_l ... 1/2,
xl = ...... x _2 ... O. Similarly,
N
Thus the maximum value of
Y~-l'" 1/2, Yl = ...... YN-2 = O.
v,
vmax .. (aN + a _l )/2
N
which is attained for
2
(tN_lxN_l)
••• +
1
~12
x~ ... 1/2, Y~ ... 1/2.
2
YN
d
N-l 2
Z xi ... 1/2. The maximum value of
tion Z xi ... 1/2 is
YN ~ 1/2 •
z ... (x*y*).
,
A similar proof shows that
Q.E.D.
4.
Bounds on covariances.
In .this section we revert to the notction of section 2 and
1.::;6
..
define x, y, a, ~ etc., as in (2.?) •
If xi
of the
XIS
denotes the ith row of x
and if by proper choice
we make
N
Z x. x.
y=l 1.Y JY
= 0 •• ,
1.J
then a '" c '" I p and we get (2.13), i.e.,
B . ::: 0'
iJ
2
-
N-l
/ 0 .. +
1.J
-
Z P kXi (C
k=l
From theorem 2.1 of Chapter
..
lk
k
of C + C
is
4 the
11
2 cos
k
kl'
)x j
+ C
7
_ •
maximum characteristic root
which is less than 2.
There-
fore from theorem 3.1,
i B1.'j.
(4.1)
2
I
- 0' 6.IJ.
.J..
i
2
< 20'
•
i
N-l
! k=l
Z
;
\
Pk I
.
For first order autoregressive scheme
where
6
t
satisfies the conditions given in section 2 of Chapter 1,
we have
::: a k
Hence, if a >
(4.2)
and
0:;
c
137
< 11
•
ij
< 20'
2(
N
a.-a. )
i-a
•
Similarly with the help of (2.10) we can set bounds on Bij for a
second order scheme. We note that (4.1) gives a uniform bound for
all schemes and for all x's such that
,
xx = I.
In special cases
p
we can improve these bounds to a considerable extent.
5.
Linear trend.
Let us suppose that N = 2s+1 and consider the linear trend
in the form
where
(5.2)
u2
= s(s+1)(26+1)/3
In the notation of section 2
(5.3)
x ly
= (26+1)-1/2
x2y = (y-s-l)/u,
b = (2s+1)1/2
1
b
2
=
'!,yy'
y y
-
y=l, 2, ... , N ,
y = Nl / 2 Y
(s+1) Ey
y y
u
where the summation over y
is from 1 to N.
Also
aac=JP
b
=g
m =(x'x)1J'= -N1 + ~(2i-N-l)(2j-N-l)
ij
N(N 2_1)
138
p
"
=2
n == N-2
s2 =
f
.!...!
~ 2
b2
~Yy - 1
== .:..y
n
•2
- °2
_
n
and
2
BiJ· ==
CJ
N-l N-k
r- y==lt:,N x.Joyx.JY + k=l
t:,
Z
y==l
f)k(x i X. k+ +x. xi k+ ) 7,i,j==1,2 •
Y J, Y JY , y -
After substituting the values of x
(5---5)
B11 ==
CJ
1y
and x 2y l<Te obtain
2 _ . N-l (N-k)
h;l
N
Pk
L
N-l
~~
k=l
_7
k3 P
k
-
7
and
fa
2
91
2
["1-
4 N-1
if
2
Z Pk +
k==l
2
iiN(4+~)
N -1
N-1
4
N-1 3
E kP k - ' - ?
Z k p k _7.
k==l
nN(N -1) k=l
For the first order autoregressive scheme,
•
number
(5.6)
z =/1, we have
q
t:, z
k=l
k
==
z_zQ+1
1-z
P k ==
Ct
k
•
Now, for any
139
q
k
Z kz ::
•
.
k::l
q+l
q+2
z- ( q+l)z
+gz
(1_z)2
Hence for the first order scheme
r?
s
2
1
.c,-:J=
4 a-aN
- ii I-a
'" I
-
+
8
liN
4
2
N 1\1+1
a-Na +a
+
(1_a)2
nN(N-l)
4a
- n{l-a) ,
(5.7)
•
B22 ,..
-
If we put
(j
2 /- l+a
6a
'l:'=7i - 2
-a
N(l-a)
_7 •
2 . -, 42
a_NaN+aN+1
(I-a)
N-.1 3k
Zk a
nN(N -1) 1
140
If2
=
b2
N(N 2..{)
we have
and under the first order scheme for
by'
(5.8)
.
•
Since b
1
and
b
2
are linear functions of
their covariance is zero, we conclude that
distributed as
= I,?
N(Si' Bii ), i
Ars and since
b
are independently
i
A consistent estimate of a
\. may be taken
r1
6.
=
N-l
N 2
Z v v +1 /
1
Y Y
Zv
1
y
Q
Trigonometric functions.
Consider
(6.1)
+/\ rr
-N i=1
z ~2. 1
~+
p
sin 'A.y
1.
+ bY.,y=I,2, ••• ,N,
141
satisfies condition (2) of section 2.
Now
In our notation
(6.2)
x2 •
1.,1
x
is a
= cos
'X.. y
1.
I
'P+I x N matrix such that xx = I 2P+1 " Therefore
a = c = I 2p+1
and we
'
find
"
Furthermore,
1 2 P
+ ~ cos (y-6)A
N N i=l
i
=-
142
n
=N -
2p - 1
\-\~
and
i~ 2
Cs =
CJ
2
2 N-l
2 N-l
r l - N L; Pk + -nN E k P k
k=l
k=l
4
TIT-I
P
E (N-k)p k cos kAi
K=l i=1
-!iN . E
For the covariances of
2
Ell
b
N-l N-k
= CJ .rl+2~1 N
and
Bl ,2i+l
for
Bl ,2i
= CJ £1+ N
and
and
_7
Pk
Z
k=l
B2i+1 ,2i+l
B2i ,2i
2
B2i 2j+l
,
we obtain from (2.13)
j
4 N-l N-k
2
B2i 2i
~
b
md
i
7.
~
r=l
P k coS(k+V)A. cos VA i
~
are obtainable from the expressions
by replacing cosine by sine.
N··l N-k
= CJ ~ ~
_7
L;_
k=l "(:1
Finally
2 .
)eos VA i sin (k+y)X i
Nl
-
l
If we consider P2' P 3' •.• to be negligible, we obtain to order N-
We can get more precise expressions if the underlying scheme is
known.
For eXaIn1jle, if the underlying scheme is first order auto-
regressive scheme, we have
2
a~
[~ ~ 1
~~
observe that the vector b
7.
Concluding remarks.
1~Te
,
2
4 P
cos A.~a
a
a
Z (
)..)
- n{i-a) -if i=l 1-2a cos A.+a~
•
).
is distributed as
N(~,B).
conclude this chapter with the remarks that in most practical
cases the underlying scheme for
6's will not be known.
Even if it
is known, we will be required to estimate the parameters involved
144
which will further reduce the degrees of freedom of s 2 • Similarly,
'Bij'S will have to be estimated from the sample. It may not be very
difficult to estimate 0 2, but the question of estimating PI' P 2,
•••
remains open.
ore suggestion obviously is to estimate these
quantities by sample serial correlations; or, in case when higher
order autocorrelations are functions of first
to estimate the first
serial
correl~ions.
k
k
autocorrelations,
autocorrelations by corresponding sample
In any case, the distribution of resulting esti-
mates will be extremely difficult to obtain. However, these estimates
will be a definite improvement over the estimates generally used whenever there is evidence of significant serial correlations among the
residuals.
145
BIBLIOGRAPHY
A.itken} A.C., liOn Least Squares and Linear Combinations of
Observations", Proceedings of the Royal ~ociety of ~din­
burgh, Vol. tV(1935),pp.42-48.
Anderson, R. L., "Distribution of the Serial Correlation Coefficient", Annals of Mathematical Statistics, Vol.XIII
(1942), pp. 1-13.
--Anderson, R. L., and Anderson!} T ).r., "Distribution of the Circular Serial Correlation Coefficient for Residuals from
Fitted Fourier Series"~ Annals of Mathematical Statistics,
Vol.XXI(1950), pp. 59-81:.
,
£4_7
Anderson, T.W., "On the Theory of Testing Serial Correlation fl J
Skandir:~:..visk .Aktuarietidskrift, Vol. XXXI( 1948) ,pp .88-116.
.[5 7
Bartlett, H.S., "Some Aspects of the Time-Correlation Problem
in regard to Tests of Significance fl , The Journal of the
Royal Statistical Society, Vol. XCVIII, part III(1935),
pp. 536..543.
/-6 7
Bartlett, M.S., 1l4pproximate Confidence Intervals tl , Biometrika,
Vol. XL(1953), pp. 306-317.
£7 _7
Cramer, Harald, Mathematical Methods of Statistics, Princeton
University Press, Princeton, 19510
-
e
-
i- 8 _7
Daniels, H.E., It ,o\Pproximate Distribution of Serial Correlation Coefficientsll., Biometrika, Vol. XLIII(1956), pp.169185 ~
Dixon, W. J., "Further Contributions to the Problem of Serial
Correlation t' , Annals of Mathematical Statistics, Vol.XV
(1944), pp. 119-144.
Doob, J. L., Stochastic Processes,
porated,-New York, 1953 ~_._-
John Wiley and Sons, Incor-
Durbin, J., and 1A!atson, G. S., "Testing of Serial Correlation
in Least Squares Regression 11, Biometrika, Vol.XXXVII
(1950), pp. 409-428; Vol. XXXVIII{1951), pp. 159-178.
-r127
-
Fisher, R. A., "Frequency-Distribution of the Values of the
Correlation Coefficient in Samples from an Indefinitely
Large Population It, ~iometrika, Vol.X(1915), pp. 507-521.
146
Fisher, R.A., ltThe General Sampling Distribution of the
Multiple Correlation Coefficient/If, Proceedings of Royal
Socl.ety Se~ies A, Vol. CXXI(19 28), pp. 654-673.
Ghurye, S. Go. "A Method of Estimat ing the Parameters of an
Autoregressive Time Series ", Biometrika, XXXVII(1950),
pp. 173-178.
Gurland. John, "Distribution of Definite and of Indefinite
Quadratic Forms ", .Annals of Mathematical Statistics,
Vol. XXVI (1955), pp. 122-127.
- -7
/-16
•
Hart, B. I., "Significance Levels for the Ratio of the Mean
Square Successive Difference to the Variance", Annals
of Mathematical Statistics, Vol. XIII(1942b),pp. 445"
41(1:"-Hotelling, Harold, "An Application of Analysis Situs to
Statistics ", Bulletin of the Amerioan Mathematical
Society, Vol.!XXIII(1927), pp. 467-476.
Hotelling, Harold, "Tubes and Spheres in n-Spaces and a
Class of Statistical Problems II, 1\merican Journal of
Mathematics, Vol. LXI(1939), pp. 440-460.
P.L., "On the Asymptotio Distributions 6f Oertain Statistics used in Testing the Independence between Successive
Observati.ons from a Normal Population", Annals of Mathematical Statistics, Vol. XVII(1946), pp. 350"354.
Hurwicz, Leonid, "Least Squares Bias in Time Series ll , Chapter
15 in Koopmans', T., Statistical Inference in Dynamic
Economic Models, John-wIley and Sons, Incorporated,
New york, 1950.
•
Jenkins, G.M., "Tests of Hypothesis in the Linear Autoregressive Model. I. Null Hypothesis Distributions in the
Yule .Scheme fl , ~~, Vol. XLI(1954), pp. 405-419.
Jenkins, G.M., t~ests of Hypothesis in the Linear Autoregressive Model. II. Null Distributions for Higher Order
Sohemes: Non-m!ll Distributions", Biometrika, Vol. XLIII
(1956), pp. 186-199<
Keeping, E.S., "A. Significance Test for Exponential Regr.ess:ton tt,
Annals of Mathematical Statistics, Vol. XXII(1951), pp.
180-198.
147
Kendall, M.G., "On Autoregressive Time Series", Biometrika,
Vol. XXXIrI(1944), pp. 105-12'.
Kendall, M.G., Contributions to the Study of Oscillator
Time-Series, Cambri ge University Press, 19 o.
Kendall, MG., The Advanced Theory of Statistics, Vols.I and
II, Charles Griffin and Company Liniitea, London, 1948.
_._--
Kendall, M.G., "Note on Bias in the Estimation of Autocorrelationa", Biometrika, Vol. XI,I(1954), pp. 40.3-404.
Koopmans, Tjalling, "Serial Correlation and (...l uadratic Forms in
Normal Variables", Annals of Mathematical Statistics,
Vol. XIII(1942), pp. 14-2.3.
£29_7
Koopmans, TjaLling, Statistical Inference in Dynamic Economic'
Models, John 'Wiley and Sons, Incoropor ated, New York;, 1950.
£3o_7
Lehmann, E.L., "On Optimum Tests of Composite Hypothesis with
. One Constraint", A1mals of Mathematical Statistics, Vol.
XVIII(1947), pop. 4'73-494 •
£31_7
Leipnik, R.13., "Distribution of the Serial Correlation Coefficient in a Circularly Correlated Universe", Annals
of Mathematical Statistics, Vol. XVIII(1947), pp. 80-87.
".
•
e
Madow, W.G., r~ote on the Distribution of the Serial Correlation Coefficient li , Annals of Mathematical Statistics,
Vol. XVI(1945), pp. 308-310.
Mann, H.'!3. and lrlald, Abraham, "On the Statistical Treatment
of l:.1.near Stochastic Difference 'Equations", Econometrica,
Vol. XI(1943), pp. 173-220.
Marriott, F.H.C., and Pope, J.A., "Bias in the Estimation of
Autocorrelations", Biomet!':t~, Vol. XLI(1954), pp. 390-402.
-r35·7
-
Moran, P.A.P., "Some Theorems on Time Series", Biometrika, Vol.
XXXIV(1947), pp.281-29l} Vol.XXXV(1948),pp.25;-260.
Moran, P.A.P., III!. Test for the Serial Independence of Residuals",
Biometrika, Vol. XXXVII(1950), pp. 178-181.
Orcutt, G.H., "A Study of the Autoregressive Nature of the Time
Series used for Tinbergen's Model of the Economic System
of the United States 1919-32 11 , ,Journal of the Royal Sta~is­
tical Societl S~pplement, Vol. X(1948), pp. 1-53-
148
-r38-7
Orcutt, a.H., en d James, S.F., "Testing the Significance of
Correlation between Time Series", Biometrika, Vol.XX:XV
(1949), Pp. 397-413.
Orcutt, a.H., and Cochrane, D., "~ Sampli.n(7 Study of the Merits
of Autoregressive and Reduced Form Transformations in
Regression Analysis", Journal of the American Statistical
~ssociation, Vol.XLIV(1949), pp. 356-372.
-r407
£41
-
7
Pearson, Karl, Tables of the Incomplete Beta-Function, Cambridge University Press, 1934.
Quenouille, M. H.• "Some Results in the Testing of the Serial
Correlation Coefficient", Biometrika, Vol. XXXV(1948), pp.
261-267.
Quenouille, M.H., "Approximate Tests of Correlation in TimeSeries If, Journal of the Royal Statistic a1 Society Supple~~ Vol. XI(1949a), pp. 68-84.
Quenouil1e, M.H., "The J oin"1i Distributions of Serial Correlation Coefficients TI, Annals of Mathematical Statistics, Vol.
XX(1949b), pp. ,61-571.
['44...7
Rao, S.R., and Som, R.K., II Autoregressive Series", second
article in Indian Statistical Institute's '~he Applicability of Large Sample Tests for Moving Average and Autoregressive Schemes to Series of Short Length rr, Sankhya,
Vol. XI(19,1), pp. 239-2,6.
RUbin, Herman, liOn the Distribution of the Serial Correlat ion
Coefficient", Annals of Ma"~hematical Statistics, Vol.XVI
(1945), pp. 211-215.
.
Sastry, A.S .R.. "Bias in Estimation of Serial Correlation Coefficients", Sankhya, Vol.XI(1951a),pp. 281-296.
Szego, aabor, OrthogoEal Polyoomials, American Mathematical
Society Colloquium PUblications, Vol.XXIII, New York,1939.
Tintner, Gerhard, Econometrics, John Wiley and Sons, Incorporated, New YorK: 19;2.
1:49_7 von Neumann, John, Kent,R.H., Bellinson,H.R., and Hart,B.I.,
"The Iv1ean Square Successive Difference ", Annals of I'1athematical Statistics, Vol.XII(1941a), pp. 153-162.
149
von Neumm n, JOhn, "Distribution of the Ratio of the Mean
Square Successive Difference to the Variance ll ; Annals
of Mathematical Statistics, Vol. XTI(1941b), pp: 357=395.
151_7 von Neumann, John, !fA Further Remark on the Distribution of
the Ratio of the Mean Square Successive Difference to the
Variance tl , Annals of }Jfathematical Statistics, Vol.XIII
(1942), pp. ~-B6.
-r527
-
Watson, G.S., rQn the Joint Distribution of the Circular Serial
Coefficients", ~iometrika, Vol-XLIII( 1956), pp. 161-l68~
-r537
-
lNhittle, Peter, Hypothesis Testing in Time ~eries, Almquist
and l,vikse1ls 'Roktryckari P,.B., Uppsa1a, 1951.
£54-.7 lnTise, J., liThe Autocorrelation Function and the Spectral Density
Function", 12iome~~, Vol. XLII(1955), pp. 151-159.
-r557
-
'"
e
Wold, Herman, A Study in the Analysis of Stationary Time Series,
Almquist andWlksells, Uppsala, 1938 •
.(56_7 Yule; G.U., tlOn the Time-Correlation Pr oblemll 3 .:!ournal of the
Royal Statistical Society, Vol. LXXXIV(1921), pp.496-537.
£57_7 Yule,
a.u., "Why do we Sometimes get Nonsense Correlations
between Time Series?", Journal of the Royal Statistical
Society, Vol. LXX~IX(1920Y, pp. 1-64.
Yule, G.U., nOn a Method of Investigating Periodicities in
Disturbed Series with Special Reference to Wo1fer's
Sunspot Numbers", Philosophical Transa~ions A, Vol.CCXXVI
(1927), pp. 267-29~.
.-