I
Ie
I
I
I
I
I
I
I
••
I
I
I
I
I
I
I
•e
I
CONTRIBUTIONS TO CENTRAL LIMIT THEORY
FOR DEPENDENT VARIABLES
by
R. J. Serfling
University of North Carolina
Institute of Statistics Mimeo Series No. 514
March 1967
This research was supported by the National
Institute of Health, Institute of General
Medical Sciences, under Grant GM-00038-13,
and by the Air Force Office of Scientific
Research, Contract AF 49(638)-1544.
DEPARTMENT OF STATISTICS
UNIVERSITY OF NORTH CAROLINA
Chapel Hill, N. C.
I
..I
I
I
I,
ii
TABLE OF CONTENTS
Chapter
Page
ACKNOWLEDGEMENTS
INTRODUCTION
1
REVIEW AND SUMMARY
4
NOTATION AND MATHEMATICAL ASSUMPTIONS
I
iii
10
GENERAL THEOREMS
I
1.
Description of Results
11
2.
Central Limit Theorems Under Condition (A)
12
I
I
3.
On Moments of Sums
26
4.
Relationships Among the Various Assumptions
33
5.
Comparisons with some Previous Theorems
40
.e
II
I
I
I
I
I
I
I
.e
I
PARTICULAR THEOREMS
6.
Description of Results
44
7.
Weakly Stationary Sequences
45
8.
m-correlated Sequences
49
9.
Sequences with Markov-bounded Second-order Dependence 54
10.
III
Bounded and Nearly-bounded Sequences
56
APPLICATIONS
11.
The Wilcoxon 2-sample Statistic
61
12.
The Empirical Distribution Function
71
13.
Covariance Function Tests
72
14.
Tests for Renewal Processes
73
APPENDIX
74
BIBLIOGRAPHY
76
I
~
I
I
I
I
I
I
I
f
iii
ACKNOWLEDGEMENTS
The contributions of others are acknowledged with my gratitude.
I am especially indebted to Professor M. R. Leadbetter for encouragement at the outset of the investigation and for assistance in my endeavor to become a researcher.
Professor Wassily Hoeffding has provided elegant simplications of
some proofs.
Professor W. L. Smith has offered stimulating ideas for
advancing the investigation.
I accord my most heartfelt thanks to my wife, Jackie.
By her
efforts, our house has been a home, our children happy, and the student
years grand.
I
I
I
I
I
I
~
I
I
My graduate study was supported by the National Institutes of
Health.
This research was initiated during summer employment with the
Research Triangle Institute, Statistics Research Division, which also
supported the completion of the work.
Finally, it is my pleasure to thank Mrs. Ruby Monk for her cheerful
and competent typing of the manuscript.
I
..I
iv
I
I
I
I
I
I
,I
I
I
I
I
I
II
I
"
the experimentalist would argue that in most physically realizable
situations where a stationary process has been observed during a time
interval long compared to time lags for which correlation is appreciable,
the average of the sample would be asymptotically normally distributed.
Unfortunately none of the extensions of the central limit theorem
to dependent variables seems to answer this problem in terms well
adapted for practical interpretation."
Grenander and Rosenblatt, Statistical Analysis of Stationary Time Series .
I
~
I
I
I
I
I
I
I
~
I
I
I
I
I
I
~
I
I
INTRODUCTION
Considerations on stochastic models frequently involve sums of dependent random variables.
In many such cases, it is worthwhile to know
if asymptotic normality holds.
If so, inference might be put on a non-
parametric basis, or the asymptotic power of a test might become more
easily evaluated for certain alternatives.
Of particular interest, for example, is the question of when a
weakly stationarY'sequence
l
of random variables possesses the central
n
limit property, by which is meant that the sum
~ X. , suitably normed,
1 1
Our results make headway in this direction
is asymptotically normal.
and also have wider applicability.
The general class of sequences (X.}oo
considered here will be that
1 _00
n
Var(~ X.) ~
which satisfies
1
nA
2
.
Included in this class are the weakly
1
stationary sequences whose covariances (r.} have finite sum ~ r . .
J
J
An
example of the latter type of sequence is any sequence of mutually independent random variables having common mean and common variance.
For definiteness, we define the class of sequences under investigation
to be those sequences (X.}oo
which satisfy the variance condition
1 _00
~n
(A)
Var(
~
X.)
~l1
~
ldefined in Section 7.
nA
2
uniformly in a, as n ->
00
•
2
As a mathematical convenience, we may and shall assume, without
00
loss of generality, that the sequences (X.} _00 satisfy E X. = 0 (all i).
~
~
We shall say that the sequence (X.}oo_00 has the central limit property.
~
or that n -~ (Xl + ... + X ) is asymptotically normal, if
n
Xl
(1)
+ ..._+_X.;:.n
< z} -:;:. (2rr) -~
P(~_--:-
n~A
l
2
e -~t dt , as n ->
00
•
_00
The plan of our theoretical investigation is to obtain relatively
mild restrictions upon the dependence in a sequence satisfying condition
(A), under which the central limit property holds.
Other investigators
have weakened considerably the requirement (A) by assuming that certain
strong restrictions on the dependence (e.g. m-dependence, defined in
"Review and Summary") are satisfied.
Thus it is important to
have a body of central limit theorems which take advantage of condition
(A) when it holds, in order to diminish other requirements necessary for
the utilization of the central limit property.
Within this plan, effort has been made to arrive at conditions which
are amenable to verification, such as conditions on moments of the Xi's
or, for applications in weakly stationary processes, conditions in terms
of the covariance function.
It would be possible to extend our investigation, without substantially
changing its character, to a wider class of sequences through replacing
nA
2
by nh(n) in condition (a), where h(n) is a slowly varying function .
However, the class of sequences we have adopted covers the applications
of interest and, moreover, is intuitively satisfying.
2
i.e., lim
n->OO
[h(nk)jh(n)]
=1
, for any k.
I
I
I
I
.,
I
,
I
practical interest, condition (A) is satisfied but other typical re-
2
..I
I
However, in many situations of
strictions on the dependence cannot be assumed.
I
I
I
I
I
.'I
I
I
3
A more important outlet for further effort is found in the need for
investigating the behavior of various useful statistics, usually defined
on sequences of mutually independent random variables, when dependence
I
I
exists in the sequence.
In this way, the scope of established statis-
tical procedures can be extended to more general stochastic processes,
and when applicable the powerful central limit theory can be utilized.
In particular, the most interesting contribution of our practical inves-
I
I
I
I
f
I
I
I
I
I
I
~
I
I
tigation is an examination of the Wilcoxon 2-sample statistic for sequences of dependent random variables.
4
I
~
I
REVIEW AND SUMMARY
A general treatment of central limit theory for sums of dependent
random variables was given by Bernstein [1]1 in 1927.
His Theorem A
deals with sequences in which variables sufficiently far apart are independent of each other.
Theorem B allows dependence to persist, in di-
minishing degree, as the variables become far apart.
The work advanced
the study of finite Markov chains, but some of the conditions assumed,
while quite natural for that investigation, limit the applicability of
the theorems to general sequences satisfying condition (A).
Certain con-
ditional absolute moments higher than the second are required to be
bounded and.to satisfy strong regularity conditions.
Even in the case of
a bounded sequence, the regularity conditions would be too severe for our
purposes.
We do not wish to exclude, for example, sequences such as a sequence
of jointly normal random varibles.
Such a sequence has, of course, the
central limit property and exemplifies the apparent departure from mutual
independence that may exist without losing the central limit property.
The work of Lo~ve [13] in 1945 also requires certain conditional
moments to be bounded.
Later results in his book [14] entail other re-
strictions unsuitable for our purposes.
As with Bernstein, other goals
motivated the work.
All of the difficulties mentioned are avoided, for the case of an
m-dependent sequence, in the work of Hoeffding and Robbins [10] in 1948.
l[ ] denotes reference to Bibliography.
I
I
I
I
I
I
,
I
I
I
I
I
I
.-I
I
I
Ie
5
Their theorem is quite suitable for practical application in the case of
I
a weakly stationary m-dependent sequence and, in order to fix ideas,
I
I
I
I
I
m-dependence of a sequence (X.}oo
I
••
I
I
I
I
I
I
I.
I
I
shall be stated here for that special case.
~
_00
First, we clarify that by
is meant that (X
a-r
... ,
,
Xa- l' Xa }
and (Xa+ s ' Xa+ s +1' ... , Xa+ s + t } are independent if s > m .
Theorem (Hoeffding-Robbins).
Let (X.} be a weakly stationary m-dependent
~
sequence of random variables such that
(M
<
00
5
;
>
Then as n ->
0) .
00
E X. = 0
~
,
Elx.
~
2 5
+ < M
1
the random variable n -~ (X. + ... + X )
~
n
has a limiting normal distribution with mean 0 and variance
2
A
=
E
xi
m
+ 2 L:
j=l
E XlXl + .
J
A weakly stationary sequence has, of course, finite and equal second
2
moments E Xl
=c
.
Thus we see that, for a weakly stationary m-dependent
sequence (X.}, the assumption
~
Elx. 12+ 5
~
< M is not a severe additional
restriction.
For strictly stationary m-dependent sequences, the (2+5)-moment
condition becomes simply
sequence
t he
2
Elx l
2 5
/ + <
00.
But since a strictly stationary
does not by definition guarantee the existence of any moments,
.
requ~rement
I
E Xl /2+5 <
00
.
~s
.
a major
a dd·
it~ona 1
.
restr~ction.
Thus, for some purposes it would be natural to seek a generalization
of the Hoeffding-Robbins theorem by weakening the moment requirements.
A succession of results by Diananda ([4], [5], [6]) and Orey [15] pursued
this endeavor, retaining the assumption of m-dependence.
The result of
Orey actually permits the X. 's to have no moments whatsoever.
~
Our interest, of course, is in retaining a condition such as
2defined in Section 7.
6
2
since at least Elx. 1 < M' is guaranteed by condition (A),
1
-
and pursuing ways of lessening the dependence restrictions.
One might conjecture that any weakly stationary and m-correlated
2 5
sequence with Elx. 1 + < M possesses the central limit property.
1
3
Evidence
to the contrary may be seen in Grenander and Rosenblatt [9; l80-l8lJ.
They exhibit a strictly stationary sequence of uncorrelated random vari2 5
ables with Elx / + <
l
00
which does not have the central limit property.
Also presented is a bounded sequence of uncorre1ated variables satisfying
condition (A) which does not possess the central limit property.
Efforts to weaken the independence requirements have relied upon
certain regularity conditions which are but a first step away from
m-dependence and furthermore which are not in general amenable to verification.
b
In order to state these conditions, let M denote the a-algebra
a
generated by events of the form (X. , ... , X. ) E E} , where
1
1
1
k
a
~
i
1
< ... < i
k
~
band E is a k-diminensiona1 Borel set.
Then the
conditions (see [llJ) are
(I)
oo
/P(B/M _00 )
-
p(B)1 ~ ~(n) + 0 , as n ->
00
,
with probability 1,
00
for all B E Ma +n '
and the strong mixing condition
(II)
Ip(AB) - P(A) P(B)/ < a(n)
+
0 , as n ->
00
,
for all
a
00
A E M_00 and B E Ma+n .
It is seen in Section 4(d) that (I) implies (II).
An excellent study of the central limit question for strictly stationary sequences satisfying (I) or merely (II) has been done by
3defined in Section 8.
I
~
I
I
I
I
I
I
I
,
I
I
I
I
I
I
JI
I
I
Ie
I
I
I
I
I
I
I
,.
7
Ibragimov [11] in 1962.
For example, he gives necessary and sufficient
conditions for a strictly stationary sequence satisfying (II) to have the
central limit property.
Rosenblatt [16] in 1956 has stated a central limit theorem under
condition (I), condition (A) and a severe (2+o)-moment condition.
though strict stationarity is not required, the (2+o)-moment condition
is more severe than would hold for a strictly stationary sequence satisfying condition (I), condition (A) and
Elx l
2 O
/ + <
00
Thus, the con-
ditions of the theorem thwart practical application of it.
However,
the fact that it circumvents the strict stationarity assumption makes
the theorem of interest.
It is also of interest to circumvent such strong regularity requirements as (I) and (II).
We wish to make use of those conditions that are
known to hold for many typical stochastic processes of interest and
which furthermore contribute toward the central limit property.
I
I
I
I
I
I
I.
I
I
Al-
condition is condition (A).
Also, information in terms of covariances
often is easy to obtain and seemingly easy to interpret.
dition in terms of covariances would be useful.
(C)
~
j=l
IE
Such a
Hence a con-
We will make use of
X X +. / converges uniformly in a .
a a J
Let us now consider a result of Ibragimov's [12] which does not require either (I) or (II).
Theorem (Ibragimov).
of martingale differences
limi t property.
4defined in Section 8.
(X.}
1
Let
4
00
_00
be a strictly stationary ergodic sequence
with E
X71 = A2 <
00.
Then
(X.}
has the central
1
I
8
It can be seen without difficulty that conditions (A) and (C) are
implicit in the conditions of the theorem.
Thus although (I) and (II)
have been circumvented, we see that conditions like (A) and (C) are somewhat basic to the central limit property.
Our investigation adopts as basic assumptions conditions such as (A)
or (C), and reasonable (2+5)-moment conditions, and then obtains further
conditions, under which the central limit property holds, but which are
amenable to practical interpretation.
See "Remarks about Theorem lA" for
further discussion.
Chapter I gives general central limit theorems under condition (A).
A variety of conditions are explored, of which the theorems developed embrace the more useful or interesting combinations.
General results on
moments of sums and a discussion of the relationships among conditions of
interest augment and support the general central limit theory.
Finally,
some interesting comparisons are made between our results and those of
previous authors.
Chapter II gives particular central limit theorems for some special
classes of sequences.
In each case, the general theory of Chapter I is
specialized to provide theorems directly suited to the class under consideration.
In one case, a new result is also provided.
Chapter III gives material relevant to applications of central limit
theory.
The Wilcoxon 2-sample statistic is shown to be asymptotically
normal for certain cases of sequences of dependent variables, including
the case of two independent sequences of m-dependent variables.
The be-
havior of the empirical distribution function is examined for stochastic
processes.
Some tests of covariance functions are given for weakly
stationary sequences.
..I
I
I
I
I
I
I
-.
I
I
I
I
I
I
--I
I
I
Ie
I
I
I
I
I
I
I
.e
I
I
I
I
I
I
I
.e
I
9
An appendix lists for easy reference the conditions assumed in the
various theorems stated.
10
NOTATION AND MATHEMATICAL ASSUMPTIONS
a
We shall use Pa to denote the cr-algebra M_00 defined in the previous
remarks, that is, the cr-algebra generated by the variables (X ,X l' ... J.
a
aHence, conditional expectations given the "past" from time t
=a
backwards
will be represented by E( . Ip ).
a
E will denote mathematical expectations, and D will denote variance:
D(Y) = E y 2 _ (E y)2
Conditional variance D(ylp) denotes the quantity
E(y2 Ip) _ [E(ylp)]2
We will assume that expectations and conditional expectations, whereever expressed, exist and are finite.
I
..I
I
I
I
I
I
I
-.
I
I
I
I
I
I
--I
I
I
Ie
I
I
I
I
I
I
I.
I
I
I
I
I
I
I
I.
I
I
CHAPTER I
GENERAL THEOREMS
1.
Description of Results.
Section 2 gives some general central limit theorems under condition
(A).
The theorems merit interest in differing ways.
ditions of Theorems 1A and 1B carry intuitive appeal.
Certain of the conCertain of the
conditions of Theorems 1C, 1D and 1E are particularly amenable to verification in practical applications.
pressed in terms of covariances.
A condition of Theorem IE is ex-
The proofs of the theorems make use of
a single basic method, but in each case either these basic steps are
proved differently or other results are called upon to yield the particular theorem.
Thus, while collectively the theorems constitute a
single result, their differing merits and differing details of proof make
individual consideration advantageous and appropriate.
However, in
summary of Section 2, the omnibus theorem is stated as Theorem 1.
Auxiliary theorems and lemmas utilized in proving the theorems of
Section 2 largely fall into two areas of interest, to which Sections 3
and 4 are devoted.
dom variables.
Section 3 gives theorems on moments of sums of ran-
Section 4 discusses the relationships among the alterna-
tive conditions of Theorem 1 and augments the introductory remarks concerning conditions assumed in central limit theorems.
Finally, Section 5 explores the relationships between our central
limit theorems and certain previous theorems.
It is seen, for example,
that the Hoeffding-Robbins Theorem is generalized by Theorem 1.
I
12
2.
Central limit theorems under condition (A)
00
(i)
EI
=0
Let (X}
satisfy E X.1
i _00
Theorem lAo
a+n
2 5
l y
L X. 1 + < Mn +
a+l 1
-
(M <
, condition (A),
0 < 5/2 ~ Y < 5 < 1) ,
00
and assume that, uniformly in a
and
a+n
(iii) E ID( L X. Ip
)
a+ 1 1 a -m
a+n
- D( a+L 1
X.)
1
l+~E
where ~ ~ 0 , E ~ 0 , and ()
* 2(1-~)+E
I<
n on mel) ,
,
<-2..
2y
Then (X.} has the central limit property.
1
Proof:
The proof includes adaptations of techniques used by Hoeffding
and Robbins [10], Ibragimov [12] and
Since the restrictions
(2(~~$)+E
2~)
Rosenb~ltt
E, 5, Y insure that the open interval
on~,
~
·
Letting [ a ] d enote t h e greatest 1nteger
=
=
[n/v] and j
Then n
=k
(~,
is a non-degenerate subinterval of
a number of a from this interval such that
k
[16].
[n~] , where ~
v + r (0
~
r < v) and k
~
<a < 1 .
~
a, set v -_ [nl-a] ,
~ .
=a -
a as n ->
n
00
•
Let
Ui
= X(i-l)k+l
St = U
l
+ ... + Xik _j
+ ... + Uv '
1), we may choose
(i=l, ... ,v) ,
,}
I
I
I
I
I
I
.-I
I
I
I
I
I
I
.-I
I
I
13
Ie
I
I
I
I
I
I
v-I
T = I: (X, k- '+1 + ". + Xik ) + (Xvk-J'+l + ,'. + Xvk+ r ) ,
i=l 1 J
Thus, S
= S' + T ,
To prove (1) it suffices by a theorem of Cramer [3; 254) to show
that
n -~ T --? 0 in probability as n
(2)
--~
00
,
and
(nA2)-~S'
(3)
n
--~
has a standard normal limiting distribution as
00
Let us first consider (2),
2
showing that E T
= o(n)
Since E T
=0
, (2) may be proved by
and applying Chebyshev's inequality,
Now, by (i) along with the Holder and Minkowski inequalities, we
I.
I
I
I
I
I
I
I
I.
I
I
have
2 1:
v-I
2 ~
X )2)~
(E T ) 2 < I: [E(X'k '+1 + ' .. + Xik ) ) + [E(Xvk_J'+l + .. , +
- i=l
1 -J
n
<
v-I
I:
( Mj
l+y)l!(2+5) + [M(j+r)l+Y) 1/(2+5)
i=l
~ M1 /(2+5) [(v_l)(j)(1+y)/(2+5) + (j+r)(1+y)/(2+5]
< 2 M1/(2+5)v(j) (1+y)/(2+5) , since j+r < j+v < jv
__ o(n1-a+1-L..) , as n ->
= o(n~)
2
Therefore E T
= o(n)
, as n ->
as n ->
since
00
00
00
,
l-a+~ = ~ ,
so (2) is proved,
Proceeding to (3), let f (t) denote the characteristic function of
n
(nA 2)-~
S' •
That is, f (t)
(3) is equivalent to
n
= E e it(nA2)-~
S'
It is well known that
14
f (t) -'
/ e-~t
(4)
~
2
n
as n ->
00
,
for all t ,
I
I
I
Define
(m
= 1"."v) , Zo = 0 ,
2
~
(t)
m
*
m(t)
=
v-m t
exp(- -;- 2
= ~m(t)
)
(m
= O,l"",v)
,
m
(m
= 0,1",. ,v)
,
itZ
E e
I
Then
I
I
2
f (t) n
e-~t = * (t) - * (t)
v
0
v
=~
(5)
r*rn+1(t) - *m(t)]
~O
JI
A term of this sum is
=
itZ 1
itZ
~rn+1(t) E e m+ - ~m(t) E e m
_t
2
2v
(6)
Now, for real y ,
e
(7)
-y
=
1 - y + Q(y)y
2
e iy
=1
+ iy _ ~y2 + R(y) ,
where it is seen easily that !Q(y)/ ~ ~
IR(y)1
~
lyl3 , IR(y)
I ~ y2
Using (7), we obtain
, whence IR(y)
if
I~
I
Y ~ 0 and that
ly12+5 since 0 < 5 < 1
I
I
I
I
I
I
JI
I
I
..I
15
E(e
2 -~
it(nA) Um+l
II
I
I
2
2 -1
2
E(Um+l Pmk - j ) - t (2nA)
E(Um+l Pmk - j )
224
t i t
t
t
2
n2A
m -J
v
v 4v
+ E[R(-r Um+l) P k .] + -2 - Q(-2)
Hence, by (6),
I
I
I
I-
I
I
I
I
I
I
- e
I
.
2 -~
= ~t(nA)
I
I
I
Ipmk -J.)
_t 2 /(2v)
From (ii), since (k-j)
~
= n
k , we have uniformly in m,
2af3 [O(n -cx€ ) + O(n -~€ )]
= n 2a-l
(9)
u~€
since 2(1-~)+€ < a
0(1),
L
==> 2~ < a€ - ~€
+ 2a - 1 = ~€ + 2a - 1 .
Hence, by the Schwarz inequality,
(10)
I
I
2 -~ t E E(Um+l P - )
(nA)
mk j
I<
na-I 0(1), uniformly in m.
By condition (i),
2 5
t
E IR(T
Um+l) I
E ITt Um+l 1 +
nAn A
:s.
:s..
(11)
2+5 -~(2+5) l+~
Mk I
) n
( t/ A
= na-I
0(1), uniformly in m .
I
16
Now, noting that E Um+l =
°,
2 -1
2 I
-I!
E I(2nA)
E(Um+l Pmk - j ) - (2v) I
~ 2~(j+l)
+
2~2
+
~
2nA
EID(Um+l/Pmk_j) - D(Um+l)I
E E2(Um+lIPmk_j) +
2~2/D(Um+l)
2
- (k_j)A /
Hence, by (9) and conditions (A) and (iii), we have, uniformly in m ,
2
2
E/(2nA )-1 E(U~lIPmk_j) - (2v)-11 ~ n~-l 0(1) + kn- l 0(1) + n (a-l) 0(1)
+(k_j)n- l 0(1)
a-I
= n
(12)
0(1)
Combining (10), (11) and (12) in (8), we obtain
I~m+l(t)
(13)
~m(t)
-
/<
a-I
n
0(1), uniformly in m
= O,l, ... ,v-l
.
Therefore, by (5),
.
(14)
2
/fn(t) - e-\t /
~
0(1) , as n ->
00
Hence (4) is true, completing the proof.
Remarks about Theorem lA.
An important feature of conditions (ii) and (iii) is their intuitive
a+n
appeal.
Let T
a
denote the normed sum n-\
~ X.
a+l
1
(T
l
' then, is the
normed sum which is stated to be asymptotically normal in the result of
the theorem.)
Let e
a
denote the difference between the variance of T
and its conditional variance given P
.
a-m
a
Then condition (iii)
~
I
I
I
I
I
I
I
-.
I
I
I
I
I
I
--I
I
I
..I
I
I
I
I
I
I
17
large.
:I
I be
a
small when nand m both are
Condition (ii) is a restriction on the variance of the condi-
tional expectation of T , given P
a
a-m
We now examine the stringency of the conditions.
First let it be noted that condition (i) is considerably milder
than the (2+5)-moment condition in Rosenblatt [16] which would require
that, uniformly in a ,
EI
a+n
~ X. 12+ 5 = o(nl+~5) , as n ->
00
•
~
a+l
To achieve perspective with respect to condition (iii), we observe
that a restriction of the same type is implicit in condition (A).
For,
by (A) and Schwarz' inequality,
.-
I
I
I
I
I
I
of Ie
requires that the expectation
2 a+n
E E
(~
a+1
x.lp
~
a-m
)<
a+n
E E(
I ~ x./ 2 Ip
~
a+l
a-m
)
a+n
=
EI ~ x.1 2
a+1
:s n
(15)
~
0(1) .
Now (15) has the form of (ii) with
not suffice for (ii), since
l+~€
2(1-~)+€
2~
=1
<..§.. < 1 , which implies
2r-
+
2~
€
2~
=1
+
€
(However, (15) does
is excluded by the restriction
< 1 +
~€
.)
In terms of T
a
, (15)
states that the variance of the conditional expectation of T , given
a
p
a-m
is uniformly bounded.
a mild restriction.
Thus, condition (A) , via (15) , imposes
By comparing (ii) with (15) , it is easily seen
that (ii) does not require a great deal more than what is already provided by condition (A).
Hence, given (A) , (ii) is not a very stringent
I
18
additional restriction.
Similarly, we observe that a restriction very much like condition
(iii) is contained in (A).
~n
E/D( L:
a+l
x.lp
1.
a-m
For, by (A) and (15) ,
) - D(
~n
E
a+l
~n
X.)/
<
1.
<
(16)
x.1 2 +
2 EI L:
a+l
n 0(1)
n 0(1)
1.
.
Of course, (16) does not suffice for (iii), since the latter requires
n 0(1) on the right hand side.
But (16) shows that, given (A) , (iii)
is not a very stringent additional restriction.
Finally, on the matter of stringency, it should be noted that conditions (ii) and (iii) allow considerable freedom in particular realizations of the sequence (X.}.
In a particular realization, the deviations
1.
Ie
I and
a
a+n
E( L:
a+l
x.lp
) may
1.
a-m
take large values, as in the case of a se-
quence of jointly normal random variables.
It is merely the expected
value and variance, respectively, of these errors that is being restricted by (iii) and (ii).
Further, it is merely the asymptotic behavior of
these moments, when both of m and n are large, that is restricted.
We
might say that the asymptotic second-order dependence of the sequence is
being curtailed.
An interesting class of sequences satisfying conditions
(ii) and (iii) is examined in Chapter II, Section 9.
Let us now consider the fact that the conditions are stated as propa+n
erties of the sums
L:
a+l
X
i
f
S
X. , rather than in terms of properties of the
1.
or in terms of other aspects of the basic probability model.
We
have seen that this permits easy comparison of condition (A) with (ii)
and (iii).
Moreover, in some situations, properties of the sums can be
easily obtained.
For example, in the case of a weakly stationary sequence
..I
I
I
I
I
I
I
-.
I
I
I
I
I
I
--I
I
I
..I
I
I
I
I
I
I
••
I
I
I
I
I
I
I.
I
I
19
with convergent sum of covariances, condition (A) holds (Lemma 11).
In seeking to utilize central limit theory, which states a property of
n
the sum
~
1
x. ,
consideration of other properties of this sum may some-
1
times be of use.
We now show that conditions (ii) , (iii)} in Theorem lA may be replaced by the following alternative pair of conditions:
a+n
(ii') E IE(
x. IP
~
a+l
a-m
1
)
I s:
~
n [O(n
-~E
) + O(m
-1:- E
2
)]
,
uniformly in a ,
and
a+n
(iii') EIE(I
~
a+l
a+n
2
x.llp a -m 11
~
EI
x.1 2 1<n
1
a+l
on m(1) '
'
uniformly in a,
where as previously
~ ~
0 and E
~
0 .
Returning to the proof of Theorem lA, we see that (10) follows
directly from (ii') , for, uniformly in m ,
E/E(Um+lIPmk_j)I
"""2""(1;;.1~;..o~i.;=):"'+-E < a:
since
->
~
-
~~E
< a: -
s: k~[O(k-~E)
+
O(j-~E)]
=
n~[O(n-~E) + O(n-~~E)]
=
n~-~~E 0(1)
=
n
a:-~
>2a13 < a:E -
0(1),
~E
+ 2a: - 1
= ~E
~ .
Further, by (iii') and (A), uniformly in m
+ 2a: - 1
I
20
2 1 E(Um+l
2 IP - ·) - (2v) -I,
E I(2nA)mk
J
1
2 IP - )
S ----2
E IE(Um+1
mk j
2nA
2
- E(Um+1)/
-(k-j)1 + --1--IE(U 2 )_(k_j)A 2 ,
2nA2
m+1
2n v
Jl/£
+
< na - 1 0(1) + n~-l 0(1) + na - l 0(1)
a - l 0(1) .
= n
Hence (12) follows from (iii') and (A).
Since (8) and (11) resulted without recourse to (ii) or (iii), we
have (13).
The remaining steps of the proof did not require (ii) or
(iii), so that we have proved:
00
Let [X.}
satisfy E X.1
1 _00
Theorem lB.
and (iii'), with M <
( *)
00
,
0 < 5/2
=0
Sr <
5
and conditions (A), (i), (ii')
S
1 ,
~ ~
0 ,
€
~
0 and
<...£
l+jE
2(1-~)+€
2r
Then (X.} has the central limit property.
1
Remarks about Theorem lB.
Clearly Theorem lB has the same features as Theorem lA so that the
"Remarks about Theorem lA" substantially apply also to Theorem lB.
Of
course, the particular intuitive interpretation corresponding to conditions (ii') and (iii') is different from that corresponding to (ii) and
(iii), but is equally simple.
We note that condition (ii) implies condition (ii').
..I
I
I
I
I
I
I
-.
I
I
I
I
I
I
--I·
I
I
Ie
I
I
I
I
I
I
I
••
I
I
I
I
I
I
21
It is of interest to replace the unintuitive condition (i) by
the simpler and weaker condition
where M <
and
00
a<
0 < 1
Theorem 2, proved in Section 3, shows that
condition (i), with y = 0/2 , follows from the conditions (i') and
n l+~o , a 11 m> m
(B)
-
0
Hence, combining Theorem 2 with Theorems lA and lB, we obtain:
Theorem lC.
Let [X.}oo_00 satisfy E X. =
~
~
a ,
conditions (A), (B) and
(i'), and also either [(ii), (iii)} or [(ii'), (iii')} , with
(*)
2~
< 1 +
~E
•
Then [X.} has the central limit property.
~
Although (it) is in fact both simpler and weaker than (i), the
combination [(it), (B)} implies a condition stronger than (i), since
0/2 < Y < 0 is excluded.
of Theorems IA and lB.
Thus, Theorem IC is less general than either
It is stated in order to take advantage of the
greater intuitive appeal of [(it), (B)} over (i).
(A) implies a condition like (B), but with 0
=a
(A), (B) is not a severe additional restriction.
Note that condition
Thus, given condition
Therefore, in some
practical situations, in which condition (A) is assumed, it might be
easier to assume [(i'), (B)} on the basis of intuitive justification
than to adopt the weaker condition (i) without benefit of a justifying
criterion or argument.
I.
I
I
In the interest of further simplicity, we can eliminate condition
(B) from the preceding theorem by slightly strengthening the restrictions
I
22
((ii), (iii)} or ((ii'), (iii')} .
As an example of the type of result
that can be achieved, let us strengthen the primed conditions.
Consider the condition
(iii 'B)
°> 0
uniformly in a , with
.
It is shown in Section 4, Lemma 4, that (iii') follows from (iii'B) and
that (B) follows from ((A), (iii'B)} .
Therefore we have
Theorem ID.
Let (X.}oo
satisfy E X.1.
1. _00
and (iii'B), with (*)
2~
< I +
=0
and conditions (A), (i'), (ii')
~€
Then (X.} has the central limit property.
1.
Theorem ID offers the advantage of rather simple conditions, but is
slightly weaker than Theorems IC and lB.
Another approach toward weakening condition (i) introduces a condition (C) in terms of covariances.
The theorem we obtain will replace
(i) by a condition intermediate between (i) and (ii'), and for certain
combinations of €,
° will allow (i)
to be replaced by (i').
These as-
peets, we shall see, will make the theorem very effective with respect
00
to certain classes of sequences (X.}
1. _00
required on
~,
€,
Moreover, the restriction (*)
0, r in the previous theorems will be replaced by a
lesser restriction (**).
Theorem IE.
00
Let (X.}
satisfy E X.1. = 0 , conditions (A),
1. _00
,}
I
I
I
I
I
I
I
-.
I
I
I
I
I
I
I
-.
I
I
23
Ie
I
I
I
I
I
I
I
I·
I
I
I
I
I
I
I.
I
I
a+n
2 0
X" 1 +
ElL:
(i" )
~
(C)
i=l
a+l
1.
IE
X X +"
a a 1.
< Mn 1+y
-
I converges
(M
<
00
o<
~o
< y < 1 + 0 < 2) ,
uniformly in a
and also either {(ii), (iii)} or {(ii'), (iii')} , with
1
(**) 2(1-~)+E <
0
2y
Then {X.} has the central limit property.
1.
Proof:
We can follow the plan of the proof of Theorem lAo
The restrictions on
E, 0, Y insure that the open interval
~,
o
2y) is non-degenerate.
.
Hence, for Q > 0 chosen sufficient-
~y) is also non-degenerate.
1+QE
ly small, the 1.nterval (2(1-~)+E
chosen small enough, we may choose a number
that 0 < Q <
For Q
a from this interval such
a < 1 , since JL
< 1
2y -
We define v, k and j as in the previous proof, except now we put
~
=a -
Q , so 0
<
<a .
~
Statement (2) will be deduced from conditions (A) and (C), instead
of from (i) as earlier.
Let
(i
T = X _ + +
v
vk j l
Then
v
T = L: T.
l.
i=l
T
2
v
=
L:
i=l
2
T
1.
,
...
+ X
vk+r
and
+ 2
v-I
v
L:
L:
i=l e=i+l
T.T
1. e
= 1, ... ,v-l)
,
I
24
v
=
v-I
T~ +
L:
2
~
i=l
v
T. ( L: Te )
i=l ~ e=i+1
L:
Now, by condition (C), there exists a function g(w) , where
g(w) ->
a
(17)
as w ->
.
IE
~
00
X X
a a
~=w
such that
,
:s. g(w)
+.1
~
, for all a .
For 1 < i < v ,
v
IE [T .
L:
T ]
~ e=i+1 e
ik
I<
L:
- ik-j+1
IEX
ik
<
L:
a
~
. k -J'+1 w--J
k'
v
L:
(
IE
X X
~
:s.
T)
e=i+1 e
a a+w
I
I
j g(k-j) , by (17) ,
2 = ,
l
~
O(j) i f 1 < i < v
O(j+v) if i = v .
Therefore,
2
ET
:s.
(v-I) O(j) + O(j+v) + 2(v-l)jg(k-j)
= O(n l -ffi- Il )
= o(n) , since Il < a .
By Chebyshev's inequality, (2) follows.
Now, it was seen in the proof of Theorem 1A that (A), (iii), (9)}
(12), if Il < a , and in the proof of Theorem 1B that
implies
(A), (iii')} implies (12), if Il < a
Also, (ii) ==> (9) ==> (10) and (ii') am> (10), since by definition
of a we have
also
~
-
~
a[2(1-~)
Il€ < a -
+ €] > 1 + Q€ , whence
~
.
I
I
I
I
I
I
I
-.
By condition (A),
E T.
~
~
- Il€ < 2a - 1 and thus
I
I
I
I
I
I
--I"
I
I
~
I
I
I
I
I
I
I
f
I
I
I
I
I
I
~
I
I
25
Thus, (12) and .:10) follow from each of the sets (A), (ii), (iii)}
and (A), (ii'), (iii')}, since ~ < a .
And we have (11) from (i"), since a <
~r
Therefore, it is clear that (4) holds, completing the proof.
As discussed in Section 4, (i") and (i') are equivalent in the case
r =
that
1 + 5.
Therefore, if (iri() can be satisfied with
+
1
5)
On
<
€
•
the other hand, if (i'), (B)} replace (i"), then r
(**) becomes
1 + 5 ,
In such a case, (**) becomes
then (i') may replace (i") in Theorem 1E.
2(~
r =
2~
< 1+
= 5/2
so
€
Hence we have
Corollary 1E. 1.
00
Let (X.}
satisfy E X.1
1 _00
=0
, conditions (A), (C) and
(i'), and either (ii), (iii)} or (ii'), (iii')} , with
Then (X.} has the central limit property.
1
Corollary 1E. 2.
Let (X.}oo
satisfy E X.1
1 _00
=0
, conditions (A), (B), (C)
and (i'), and either (ii), (iii)} or (ii'), (iii')} , with
(**)
2~
< 1 +
€ •
Corollary 1E. 3.
Then ( X.J has the central limit property.
1
oo
Let (X.J
satisfy E X.1
1 _00
(i'), (ii') and (iii'B), with (**)
2~
=0
< 1+
€
conditions (A), (C),
Then (X.} has the cen1
tral limit property.
Comparing the last corollary with Theorem 1D, which shares the
advantage of simple conditions, we can see the effect of adding condition
I
26
(C) by considering the desirability of (**) over (*).
For
condition (A), via (15), implies a condition of type (iiI) with
2~
= 1 + € , so that (iiI) with (**) requires but the merest assumption
beyond what is provided by (A).
All three corollaries have relevance in Chapter II.
The advantages of Theorem IE are gained at the expense of adding a
further condition, (C).
However, in many situations of interest, such a
condition is readily available and should be utilized.
Thus, the five theorems we have chosen to present cover a variety of
interesting and useful situations.
In summary, we give the omnibus version:
Theorem 1.
Let (X.}oo
satisfy E X.1
1 _00
=0
, condition (A) and one of the
following sets of additional conditions:
,
(a)
((i) , (ii) , (iii) } with (*)
(b)
(i) , (ii I), (Hi')} with (*)
l
,
(d)
(i'), (B), ('11. ') , (iii')} with (*) ,
(e)
(i'), ('11. ') , (iii'B)} with (*)
(f)
(i"), (C) , (H) , (iii)} with (**) ,
(g)
((i"), (C) , ('11. ') , (iii')} with (**) ,
),
(B) , (ii) , (iii)} with
(~'<')
(i
l+~€
where (*) 2(1-~)+€
<...2....
2y
and
()
**
,
1
2(l-~)+€
< ...2....
2y
Then (X.} satisfies the central limit property.
1
3.
On Moments of Sums.
Lemma 1.
a+n
CX
a
If Elxi,a:s M, then EI L: x.l < Mn
a+l 1
-
, CX>
I
I
I
I
I
I
I
-.
I
I
I
I
I
I
I
,
(c)
,}
1 .
-.
I
I
Ie
I
I
I
I
I
I
I
I·
I
I
I
I
I
I
27
Proof:
Minkowski's inequality is used.
1
a+n
(EI L: X. la} a< L: (E
~
a+l
a+l
a+n
a+n
<
~
1
(M}a
a+l
1
= -Jin
The preceding lemma shows what a uniformly-bounded moment condition
implies with respect to moments of sums.
In particular, it shows that
E/x.1 2+O < M
(i')
~
is equivalent to
a+n
ElL:
(18)
X. 12+ 0
<
Mn 2+0
~
a+l
It is of interest to know under what circumstances conditions such
as (i) and (i") may be supplanted by the weaker condition (i'), which
has more intuitive appeal and is easier to verify.
In Section 4 it is
seen that (i) and (i") are indeed stronger than (i'), so a further condition is necessary.
The following theorem provides a sufficient addi-
tional condition.
Theorem 2.
Let (X.}oo_00 satisfy, for some 0 < 0 < 1 , and some m0
~
(i')
E
Ix.~ 12+0 < M
'
a+n
(B)
E IE(
I
a+n
(19)
EI L:
a+l
X.
L:
~
a+l
Then there exists M' <
I.
I
I
L:
Ix. ra}
1
a
00
12 1p
such that
x./ 2+ O <M'
~
-
a-m
)
nl+~o.
n
l+~o
m>m
-
0
I
28
Proof:
Let j
= [log nJ and k =
a+n
S
=
[~(n-j)J
.
a+k
~
X. , Tl
a+l 1
= 6 Xi ' T2 =
a+l
,}
Define, for j > m
o
a+n
6
X.,
a+n-k+l 1
and put S' = T + T2
l
=
Since S - S'
a+n-k
6
X. is a sum of at most (j+2) terms, we obtain by
a+k+l 1
(i') and Minkowski's inequality,
By Loeve's c -inequalities [13J ,
r
whence
(21)
l O
l
Z o
Z o
Els' /2+0:5 EIT / + + E/TZI + + ZE/Tl//TZl + + 2E/T / +0/'T 2 /
l
1
+ EIT1IZ/TZ/o + E/TlloITZ/2
c r
inequalities, for 0 < s :5 Z and q
2+5
= --s-s
E[E(IT2Is/Pa+k)Jq :5 E[(kA
Z
)2 +
,
s
2
/61 Jq
< zq-l E[(kA2)1+~5 + /6ll+~OJ
=
But,
o<
2q - l [kA2)1+~0 + E/6ll+~5] .
EI6/l+~0 :5 BZ kl+~o , by condition (B) . Hence, with C
2+0
s < 2 and q = --s-- ,
=
A2 + B2 ,
I
I
I
I
I
I
I
-.
I
I
I
I
I
I
••I
I
I
29
Ie
I
I
I
I
I
I
I
I·
I
I
I
I
I
I
(22)
Now, by Holder's inequality, if r+s = 2+0
Hence, making use of (22), if r+s = 2+0 and 0 < s < 2
(23)
a+n
Let A = n-(l+~o) sup EI ~
n
a
a+l
,2+0
By (i') and Lemma 1, A is
n
1.
finite.
Then EITlI2+0 :s; Rkkl+~O and
Let R_
-K be any numb er >
_ Ak·
Thus, we have from (21) and (23), with
D = 4 max (1, C) ,
1
(24)
E/S,,2+0 ~ kl+~o R [2 + r(~)] ,
k
where rex) = D[x-(1+0)/(2+o) + x- 1 /(2+o) + x- o /(2+0) + x- 2 /(2+5)]
Since rex) -> 0 as x ->
00
there exist
,
E
> 0 and x
o
> 1 such that
1
[2 + r (x) ] 2+5 <
-
v'2 -
if x > x
E
-
0
For this same e, we may choose, since j =
which (j+2) <
Define a
k~
E
k
(24), i f n > N
I.
I
I
x.
=
o(k~)
, a number N for
if n > N .
max (A
k
'
X
o
' M).
Then, by (20) and
I
30
1
1
( E I S 12+ 6 }2+6
E) k~ (a )2+6 +
:5 CV2 -
k
1
(a ) 2+6
k
Therefore, since 2k < n
EI
(25)
a+n
6 X. /2+6
a+1 ~
But then a k
~
:5 a n1+~6 , if n > N
k
For, by definition, a
an if n > N
n
Now, if n > N , define (k.} 1 by k
~
Then a
0
0
=n
Hence, a
n
k
> x
-
0
,
, and
< a
n -
m
, for some m < N.
Therefore,
Defining M'
= max
fa1 , ... , aN} , (19) is proved.
We now investigate what information about a moment of a sum is conveyed by a higher-moment condition in conjunction with a lower-moment
condition.
Theorem 3.
(a)
(b)
Suppose, for r
:5 a <
~
, and M <
00
,
..I
I
I
I
I
I
I
-.
I
I
I
I
I
I
.-I
I
I
Ie
I
I
I
I
I
I
I
.e
I
I
I
I
I
I
I
.e
I
31
Then, if 0 < a < Q <
~
,
(27)
Proof:
1
1
- + p
q
Let r, s, p, q be positive numbers satisfying r+s
=
1.
=Q
,
Then, assuming the stated moments exist, we have by
Holder's inequality,
(28)
In order to use assumptions (a) and (b) simultaneously in (28),
we wish to choose rand p so that rp
sa and
sq
S~ .
_1
•
S~nce
p
+
_1 -_ 1 ,
q
we must require in particular that ~ + ~ < 1 , so that
u.
ar +
Q-r
~
~~
= r(a~
)
Q
+ ~
S
~-
1 , whence a necessary requirement is
(29)
Assuming for the moment that appropriate r, s, p and q have been
determined, (28) gives
EI
(30)
a+n
L:
a+l
x.I
~
Q
!.2.
!
§.g,
< ([Mnl'] a }p ([Mnt3 ] t3
!
}q
'Y
-r + -s Q + (....
- l)r
="tf ~n
a
To obtain the best inequality possible by this method, let us choose
r so as to minimize the exponent Q + (~ - l)r in (30).
restriction (29), we wish to have r =
In view of the
a(~) if p can be chosen suitably.
~~
I
32
It is easily seen that with p
sq
Q
= ~.
Q
a
=2
a
Corollary 3. 1.
+ (y-a)(~-Q)
(~-a)
= a(~) ,
we have rp
=a
2
Q
+ a-y
a
Q
(a-y)
=Z
a
<
-
Q + ~(a-y)(Q-a)
a(~-a)
M ,
x.r 2 <Mn
1.
-
Proof:
:s Mn
In Theorem 3, put
~
l+~
h
=
2+h,
a = 2,
=1
y
and Q
Further, since random variables bounded Ix.
1.
E/XiI2+h
I<
= 2+0
.
1 satisfy
~ 1 for all h > 0 , we have
Corollary 3. 2.
If the random variables (X.} are bounded by 1 and
1.
a+n
satisfy EI L X. /2 < Mn (M > 1) , then for any 0
a+l 1.
a+n
ElL X. /2+0 < Mn 1+0
a+l 1.
-
..I
-.
Then, for 0 < 0 < h ,
a+n
2 0
ElL Xi 1 +
a+l
and
I
I
I
I
I
I
(~-Q)
(~-a )
a
Suppose, for h > 0 ,
1.
EI L
a+l
=
Q + (a-y)[~ _~]
a
~-a
2 h
E IX. 1 +
a+n
(b)
and r
Hence (30) holds and reduces to (27) since
+ (2 _ l)r
(a)
= ~~
> 0 ,
I
I
I
I
I
I
I
-.
I
I
Ie
I
I
I
I
I
I
I
.e
33
4.
Relationships Among the various Assumptions,
(a)
Alternative Hoeffding-Robbins Condition.
The Hoeffding-Robbins Theorem, stated and discussed in Section 5,
involves a condition (H) which lacks the simplicity and intuitive appeal
of condition (A).
theorem, (A) and (H) are equivalent.
.e
I
Thus their theorem has an interest-
ing and useful formulation in terms of condition (A).
Let [X.}
Lemma 2.
1
condition (it).
Proof:
1
1
(H)
lim
n
->
00
00
_00
be an m-dependent sequence satisfying E X. = 0 and
1
Then conditions (A) and (H) are equivalent.
Letting A.
=E
n
~ A +.
n j =1 a J
Now, for n
I
I
I
I
I
I
I
We show that, under the other conditions of their
2
=A
exists, uniformly in a .
> m
a+n
Er
m
X.+ + 2 ~ E X.+ ,X,
,condition (H) is:
1 m
j=l
1 m-J 1+m
a+n-l
~
X. /2 = E
a+l
1
I ~
X. /2
a+l
1
a+m
EI~ x.1 2
a+l
1
a+m
(31)
= E
Ia+l
~ x.1
1
By (it), and Lemma 1,
Hence
EI
a+m
~
x.1 2
a+l
1
E
I~
a+l
a+m
~
+
A
X,/2
1
=
= •••
a n-m
n
+
~ A+,
j=m+l a J-m
n-m
+ ~
j=l
2
X. 1
a+l 1
A
+'
a J
is bounded uniformly in a .
o(n) uniformly in a .
a+n
(32)
EI
2
+
Hence (31) may be written
n-m
~
j=l
A +' + o(n) , with the
a J
"0"
being uniform in a ,
I
34
Assuming (H) , we have from (32), uniformly in a ,
a+n
lim
EI2:
a+l
n-:>oo
x.1 2
1
n-m
= lim
n ->
which is condition (A).
(b)
(!!.:!!!)
n
00
1
2: A . = A
n-m . 1 a+J
J=
,
Likewise, (H) follows from (A) through (32).
Conditions (i). (i') and (i").
These three conditions may be stated in the form
a+n
2 5
X. 1 + :s. Mn 1+r
E I 2:
(33)
a+l
,
5
>
0 ,
1
where r is restricted, respectively:
(i)
~5 :s.
(i')
r
(i")
~5:s.
r < 5
= 1+5
r
:s. 1+5
This representation is the definition in the cases of conditions (i)
and (i"), and in the case of (i') follows from Lemma 1, Section 3.
Thus we have
(34)
(i)
On
==> (i") -> (i')
the other hand, we do
~
have (i') ==> (i) . For, in the case of
extreme dependence, i.e. all X. 's identical, we have
1
satisfying (i') but contradicting (i) .
But, as we have seen by Theorem 2, (i'), (B)} implies (i) with
r =
~5.
Since, as Lemma 3 will show, (B) holds in the case of m-
dependence and condition (A), it follows by Theorem 2 that
(35)
(i'), (A), m-dependence} -> (i) with r =
~5
.
Therefore, condition (i) clearly restricts the dependence possible
..I
I
I
I
I
I
I
-.
I
I
I
I
I
I
I
-.
I
I
35
Ie
I
I
I
I
I
I
I
.e
I
I
I
I
I
I
I
.e
I
in the sequence (X.}, whereas (i') does not.
~
investigate in what way the parameter
It would be interesting to
r in condition
(33) measures the
dependence in a sequence satisfying at least (i').
Lemma 3.
Let (X.}oo_00 be an m-dependent sequence satisfying (i') and
~
condition (A).
Proof:
Then condition (B) holds.
Let k > m.
By Lo~ve's c -inequalities,
r
(36)
By m-dependence, the first right-hand term is zero.
the second right-hand term
.
~s
[o(n)]
1+~5.
,
~.e.
By condition (A),
o(n
1+~5
).
Hence (B)
is easily satisfied.
(c)
Conditions (ii), (iii)}, (ii'), (iii')}, (B) and (iii'B).
Note that (B) cannot follow from (ii), (iii)}, or (ii'), (iii')} ,
a+n
2
2
since (B) requires that the quantity
~ x.
k) - nA
have a
a+l ~
amoment higher than the first, while the latter sets of conditions require
IE(I
only the first moment of this quantity.
I Ip
I
However, the implication of (B)
for the first moment of this quantity is also an implication of (A).
The
implication is:
m>m
-
0
This illustrates that, once condition (A) is assumed, none of conditions
(iii), (iii'), (B) and (iii'B) drastically increase the restrictions on
I
36
We now show that, under condition (A), each of (iii') and (B) is
generalized by (iii'B).
(A), (iii'B)}
Lennna 4.
(B) and (iii'B) ==> (iii').
By ffd1der's inequality and Loeve's c -inequalities, (iii'B)
Proof:
=>
~>
r
(iii ').
Now, referring to the inequality (36) used in proving Lennna 3, it
is easily seen that (A), (iii'B)} -=> (B).
(d)
Implications of Ibragimov's condition (I).
b
Let M denote the a-algebra generated by events of the form
a
(x.1. , ... ,X.1.
1
)€ E} , where a
k
< i
-
1
< ... < i
< band E is a k-dimensiona1
k -
Borel set.
Ibragimov's conditions (I) and (II) [llJ are
a
Ip(B IM _00 ) - P(B) I <
cI>(n) 1- 0 , as n ->
-
(I)
00
all B
€
Ma +n
00
,
with probability 1 for
'
and the "strong mixing condition"
fp(AB) - P(A)P(B)I ~ a(n) 1- 0 , as n ->
(II)
and B
00
,
for all A
€
M~oo
00
€
Ma+n .
The following result of Ibragimov shows that (I) ==> (II) .
Lennna 5.
condition:
The regularity condition (I) is equivalent to the following
For any events A
€
a
M_00 and B
00
€
Ma+n '
..I
I
I
I
I
I
I
-.
I
I
I
I
I
I
I
-.
I
I
Ie
I
I
I
I
I
I
I
.e
I
I
I
I
I
I
I
.e
I
37
/P(AB) - P(A)P(B) I ~ ~(n) peA) .
(37)
Proof:
See [11; 351] .
We now prove a result by which condition (I) can be related to our
conditions (i'), (ii') and (iii') .
Lemma 6.
Suppose the sequence [X.J satisfies condition (I).
1
random variable
EI~IP <
00
,
~
If the
00
is measurable with respect to M
and if
a+m
p> 1
then
(I' )
Proof:
Let Q denote the space corresponding to the random variables
Let P(·) denote the probability measure induced
[Xa+ n ' Xa+n+l , ... J
on M;+n by the basic probability model. Let Pc· IM~oo) denote a ~egular
oo
a
conditional probability measure on Ma+n ,given M-00
Let ~ denote the signed measure Pc· I~_00 )
Q+UQ-
=Q
P(.) and let
denote a Hahn decomposition of Q . Then Ann+
measurable, and
of U.
-
~(AnQ+) > 0 , ~(AnQ-)
and AnQ-
are
< 0 , for any measurable set A
Also, since Q is measurable, Q+ and U- are measurable.
Hence we have
IE( ~ IM~oo)
(38)
- E
d
=
If
Q
HP(dwIM~oo) - P(dw)]
I
~ J+I~I[p(dml~oo) - pedro)] + J_I~I[p(dw) - P(dmIM~oo)]·
Q
Q
I
38
,}
Now
f+lsl[p(drul~oo) - P(dru)]
U
1
1
S (f+lsIP[P(druIM~oo) - P(dru)]}P(f+[P(drul~oo) - P(dru)]}q
U
U
-1
-1
= [p(u+l~oo) - P(U+)]q(flsrp[p(drul~oo) + P(dru)]}P
U
-1
-1
< [~(n)]q(E(IsIPI~oo) + ElslP}p .
Performing similar steps with the right-most term in (38), we have
-1
(39)
-1
IE(sf~_00 ) - E sl <
2[~(n)]q(E(!sIPI~_00 ) + ElslP}p ,
-
whence, by Holder's inequality,
a
-1
-1
EIE(s!M _00 ) - E sl <
2[~(n)]q(2ElsIP}p
-
(40)
Hence we obtain the following relationship between (1) and
(i'), (ii'), (iii')} .
I
I
I
I
I
I
I
-.
I
I
I
I
I
I
I
-.
I
I
..I
I
I
I
I
I
I
.e
I
I
I
I
I
I
I
.e
I
39
00
Lemma 7.
y
Let eX.}
satisfy condition (I) and also condition (i) with
1. _00
= ~o.
Then (iiI) and (iii l
Uniformly in a,
a+n
EIE(L:
(41)
a+l
x.lpa-m)1
EIE(
Proof:
< n
2
I L: x.1 Ip
a+l
1.
a-m
)-
By condition (i) with y
a+n
ElL:
(43)
Putting ;
=
I2»
+U
X.
a+l
~
o(m
-E
) ,
E
> 0
1.
a+n
(42)
hold and have the following formulations:
)
1.
<
Mn
a+n
2
x.1 1< n
EI L:
a+l
A
o(m- ) , A> 0
1.
= ~o ,
0 > 0 , M<
00
,
1 1: 0
+2
-
a+n
L:
X. and p = 2+0 in Lemma 6, (II) and (43) yield (41).
a+l
1.
a+n
=
Again, putting ;
I L:
a+l
2
X. 1 and p
=
l+~o
in Lemma 6, (It) and (43) yield
1.
(42).
It is important to see, however, that (iiI) without (iii l
from (I) and (A).
=
L:
a+l
Lemma 8.
follows
The stringent condition (i) is not required.
a+n
putting ;
)
X. and p
=2
For,
in Lemma 6, (I') and (A) yield (41).
Hence
1.
00
Let eX.}
satisfy conditions (I) and (A).
1. _00
Then (iiI) holds,
with the formulation (41).
(e)
Implications of m-dependence.
In addition to Lemmas 2 and 3, we trivially have, for E Xi
=0
,
I
40
Lemma 9.
m-dependence implies (ii), (iii), (ii'), (iii') and (iii'B).
If also (i'), then (C) holds.
5.
Comparisons with Some Previous Theorems.
(a)
Hoeffding-Robbins Theorem generalized.
The Hoeffding-Robbins Theorem (1948) is
Theorem (Hoeffding-Robbins).
00
Let (X.}
be an m-dependent sequence of
1 _00
random variables satisfying E X.
=0 ,
Elx. /2+5
< M (M <
1 1 -
(H)
lim
n ->
n
-1 n
~
h=l
00
2
where Ai
= E X.+
+
1 m
Then as n ->
00
; 5 > 0) , and
A + = A exists, uniformly in a ,
a h
m
2
00
~
j=l
E X.+ . x.+ .
1 m-J
1 m
the random variable n -~ (Xl +...+ X ) has a limiting normal
n
distribution with mean 0 and variance A.
By Lemma 2, the conditions of this theorem imply condition (A).
Then also condition (B) holds, by Lemma 2.
Further, by Lemma 9, we see
that (ii), (iii), (ii'), (iii'), (iii'B) and (C) are implied.
by Theorem 2, (i) holds with r
= ~5.
Finally,
Therefore, the Hoeffding-Robbins
Theorem is generalized by each of our Theorems lA, lB, lC, lD and IE.
(b)
Comparison with Rosenblatt's Theorem.
We shall show that a corollary of our Theorem lB provides an interesting parallel to a theorem of Rosenblatt.
This Rosenblatt theorem
(1956) is, in slightly specialized form for the class of sequences satisfying condition (A),
J
I
I
I
I
I
I
I
~
I
I
I
I
I
I
I
~
I
I
Ie
I
I
I
I
I
I
I
.-
I
I
I
I
I
I
I
.I
41
Theorem (Rosenblatt).
Let rX.}oo
satisfy E X.1
1 _00
0, condition (A), the
strong mixing condition (II), and for some 5 > 0
EI
(44)
a+n
~ X. 12+ 5 = 0(n1+~5) , uniformly in a, as n ->
a+1
00
•
00
•
1
Then
(1)
P
X + ... +X
l
n}
r
n~A
<
Z
"
-->
(2~)
-72
JZ
e
-Z2
t
2
dt , as n ->
_00
A closely parallel theorem may be obtained as a corollary of Theorem
lB:
Let (X.}oo
satisfy E Xi
1 _00
Corollary lB. 1.
regularity condition (I) with
EI
(45)
~(m)
=0
= o(m -Q )
, condition (A), the
, Q > 0 , and for some 5 > 0 ,
a+n
~ X. 12+ 5 = O(nl+~5) , uniformly in a, as n ->
a+l
00
•
1
Then (X.} has the central limit property.
1
Proof:
Condition (45) is a representation of condition (i) with r
Hence, by Lemma 7, (ii') and (iii') hold, with
(*)
2(i~~)+€ < ~r
is satisfied.
e=~ ,
E
= ~5
> 0, so
Therefore the conditions of Theorem 1B
are satisfied.
We note that (45) is a weaker condition than (44), while (I) is
slightly stronger than (II).
Thus, the two theorems are quite close.
However, Corollary lB. 1 may prove more advantageous in practice, since
condition (45) is implied by conditions (if), (B)}, while (44) does not
.
I
42
seem so amenable to verification or intuitive justification.
(c)
Comparison with Results of Ibragimov.
We shall show that Corollary lB. 1 just discussed generalizes usefully an important theorem of Ibragimov, whereas the close parallel
,
theorem of Rosenblatt does not afford such a generalization.
It will be
seen that Rosenblatt's (2+6)-moment condition (44) is unreasonably strong
for situations of common interest.
Ibragimov's theorem (his Theorem 1.4 in [11], 1962) is, in slightly
specialized form for the class of sequences satisfying condition (A),
Theorem (Ibragimov).
satisfies E X.
1
00
Suppose the strictly stationary process (X.}
1 _00
= 0 , condition (I), condition
(M
Then the random variables
<
n-~(Xl
+ ...+ X ) are asymptotically normal.
n
To assist comparison, we shall require the following interesting
lemma, which is Lemma 1.9 in Ibragimov's text and may be found also in
Doob [7; 225].
Specialized to sequences satisfying (A), the result is
Lemma 10.
00
Let (X.}
be a scrictly stationary process satisfying
1 _00
E X. = 0 , condition (I), condition (A) and
1
(M
<
00
;
6
>
0 •
Then (45) holds, i.e.
(45)
2
EI a+n
~ x. I +6
a+l
1
=
O(n
1+~6
.
.
) , un1formly 1n a, as n ->
I
I
I
I
I
I
I
-,
(A) and
6 ::..- 0) .
00
~
00
•
I
I
I
I
I
I
I
-.
I
I
~
I
I
I
I
I
I
I
f
I
I
I
I
I
I
I
f
I
43
Note that Lemma 10 offers alternative conditions to those of our
Theorem 2.
In order, under condition (A), to have (i') ==> (45), our
Theorem 2 states that condition (B) suffices, while Lemma 10 states that
strict stationarity combined with regularity condition (I) suffices.
By Lemma 10 we see that the conditions of Ibragimov's theorem imply
those of Corollary lB. 1, except that we further require that
be o(m-Q) , Q >
o.
(Of course, it is implicit in (I) that
~(m)
in (I)
~(m) is
0(1).)
The advantage of Corollary lB. lover Ibragimov's result applies to
situations when strict stationarity is not assumed, for our result requires only (45) instead of stationarity.
Rosenblatt's theorem does not generalize Ibragimov's theorem because
(44) is too stringent.
Thus, Corollary lB. 1 seems to have greater util-
ity than Rosenblatt's theorem.
I
J
I
I
CHAPTER II
PARTICULAR THEOREMS
6.
I
Description of Results.
Attention is given to cases of special interest.
The theory pre-
sented in Chapter I is utilized, through supplementary considerations,
to provide theorems apropos to various classes of sequences within the
realm of sequences [X.} satisfying condition (A).
l.
For convenience, and
without loss of generality, we shall assume as usual that E X. - 0 .
l.
Weakly stationary sequences are examined in Section 7.
This class
includes, of course, the strictly stationary sequences satisfying (A).
Results for weakly stationary sequences which are m-correlated are deferred to Section 8.
Section 8 considers m-correlated sequences.
Included here are se-
quences of m-th order martingale differences, which in turn include
m-dependent sequences with Elx.
l.
I<
00.
Interesting results are obtained
for sequences of martingale differences, which also have been investigated by Ibragimov [12J.
In Section 9 we look at a class of sequences having a property
which we choose to call Markov-bounded second-order dependence.
Finally, Section 10 specializes our theory for certain sequences of
bounded random variables and sequences satisfying bounded moment conditions of relatively high order.
I
I
I
I
e.
'I
I
I
I
I
I
I
e.
I
I
Ie
45
7.
I
I
Weakly Stationary Sequences.
00
We call eX.}
1 _00 weakly stationary if E X.1 = C , Var X.1 = r 0 <
and Cov [ X., x.+.J = r. (a function depending only on j).
11]
J
00
However, we
shall assume also E X. = 0 .
1
I
I
I
I
I
II
I
I
I
I
I
I
,I
We say that exi}:oo is strictly stationary if, for any choice of
n> 1 and kl, ... ,kn ' the joint distribution of (Xa+kl""'Xa+kn) does
If, further, E
not depend upon a .
X~1 <
00
,
then it is easily seen that
eX.}OO
is stationary in the weak sense also.
1 _00
In the case of a weakly stationary sequence, condition (C) of Theorem
IE reduces to
zlr j I <
(46)
00
•
1
We shall see that (46) is sufficient for a weakly stationary sequence to
satisfy condition (A).
In fact, more generally,
00
If eX.}
is weakly stationary and if
1 _00
Lennna 11.
00
L: r. converges,
(R)
1
]
00
then (X.}
satisfies condition (A).
1 _00
a+n
Proof:
EI L:
a+l
x.1
1
2
a+n
=
L:
a+l
= nr
= nr
0
0
n-l
2
E X. + 2 L:
1
n
L:
i=l j=i+l
n
n-l
2
L: r.
L:
+
i=l j=i+l ]-i
n-l n-i
L: r.
+ 2 L:
i=l j=l J
E X .X .
a+1 a+]
I
46
i
Thus, with R. =
L: r.
~
j=l J
a+n
(47)
2
ElL:
X.
~
a+l
I
= n[r + 2!
o
n i=l
n
a+n
! ElL:
lim
(48)
n
->
n
00
a+ 1
2
X. 1 = r
1
1
n-l
L:
R. = lim
R (=9, say) ,
I
+ 29 , uniformly in a .
I
2
Thus condition (A) holds with A = r
00
o
n i=l
~
n ->
00
+ 29 .
Therefore, the reduction of our Theorem IE for weakly stationary
sequences in general is
Let (X.}OO
be a weakly stationary sequence satisfying
1 _00
Theorem 4A.
E X.1
=0
,
(iff)
~ Ir . I <
1
EI
00
,
and
J
a+n
2
X. 1 +5 < Mn l +r
a+l ~
L:
I
n
->
o
R.] .
!
lim
Since by (R) there exists
I
n-l
L:
J
(M
<
00
o<
~5
:s:
'Y
and either (ii), (iii)} or ( i i f ), (iii f )} , with (**)
:s:
1+5
:s:
2) ,
2(1~~)+€ < ~r
Then (X.} has the central limit property.
1
Typically, when weakly stationary sequences arise in the analysis of
stochastic processes, knowledge of the covariance function is available,
so that condition (46) is a natural way of investigating applicability of
central limit theory.
However, if condition (46) cannot be investigated, or cannot be met,
but condition (A) can be assumed on some grounds, then a reduction of
I
I
I
-.
I
I
I
I
I
I
I
-.
I
I
Ie
I
I
I
I
I
I
I.
I
I
I
I
I
I
I
I
••
I
47
Theorems lA and lB provides
Theorem 4B.
00
If (X.} 00 is weakly stationary and satisfies E X.
1-
1
=a ,
condition (A) ,
(i)
EI
a+n
2 5
l
L X. 1 + < Mn +r
a+l 1
(M
<
00
a<
~5
S
and either (ii), (iii)}or (ii'), (iii')}, with (*)
y
< 5 < 1) ,
2(i~~)+E <
:y
then (X.} has the central limit property.
1
Note that (i) and (*) of Theorem 4B have much less flexibility than
(i") and (**) of Theorem 4A.
y
For example, recall that (i") with
= 1+5 is equivalent to the simple restriction (i ' ).
For applications under regularity condition (I), various further re-
ductions are possible.
We shall confine ourselves to two interesting
cases, that of a very flexible (2+5)-moment condition and that of a rigid
(2+5)-moment condition.
We now show that if
A>
(Y-~5)o~;:~~
For, putting p
and
~E =
=
~(m)
in (I) satisfies
~(m)
O(m
, then (ii') and (**) are satisfied with
2+5 and
1
A(l--) =
=
) , where
~ =~
a+n
L X. in Lemma 6, we have (ii') with
a+l 1
~
=~
Then (**) reduces to the requirement that
P
,
(1+5)
A > (y-~5)O(2+5)'
~
-)..
Hence Corollary 4A.
Using Theorem 4B and Lemma 7, or else Corollary lB. 1 of Section 5(b), we
obtain Corollary lB. 2 .
I
48
00
Let [X.}
be a weakly stationary sequence satisfying
1 _00
Corollary 4A.
E X.
1
o , I: Ir . I <
1
00
I
and
,
J
a+n
(i ' ') E I 2: X. /2+0 < Mn 1+y
a+l 1
-
(I) with ~(m)
-A
= Oem ) , A >
(M <
00
I
0 < ~O ~ y ~ 1+0 ~ 2) ,
(1+0)
(Y-~0)0(2+0)
I
I
I
I
,
and, uniformly in a ,
a+n
2
a+n
2
E/E(I a+l
2: x.1 /p
)
EI
2: x.1 1< n
a-m
a+l
(iii')
1
1
0
n,m
(1).
Then [Xi} has the central limit property.
Let [X.}oo
be a weakly stationary sequence satisfying
1 _00
Corollary lB. 2.
00
E X.
=0
~(m)
= Oem-A)
1
(45)
, 2: r. convergent [or (A)] , condition (I) with
1 J
EI
, A>
a+n
2:
a+l
a ,
2
X. 1 + 0
I
, 0 > 0 .
1
Then [X.} has the central limit property.
1
As discussed in Section 5, condition (45) is implied by (I) and (i')
in the case of a strictly stationary sequence.
To achieve the same
simplification for a weakly stationary sequence, we state
Corollary lB. 3.
Let (X.} be weakly stationary and satisfy E X.
1
1
~ r. convergent, [orCA)] , Elxil2+0 ~ M <
1
00
(0 > 0) , regularity
J
condition (I) with ~(m)
= Oem-A)
.'I
I
and
= O(nl+~o)
,}
(A> 0) , and
=0
,
I
I
I
I
I
••
I
I
..t
49
a+n
2
(BI) EIE(Il: x.1
a+l 1
Ip
a+n
a-m
) - Ell:
2 I 1,
x.1 1+ 25
a+l
1
< B2 nl+~5 , m>mo
Then eX.} has the central limit property.
1
I
I
I
I
I
I
I·
Proof:
By (36) and condition (A), (B') ==> (B).
(45 holds.
Hence, by Theorem 2,
Thus Corollary lB. 2 applies.
We see that the Ibragimov Theorem, quoted in Section 5(c), for
strictly stationary sequences, is generalized, except for a mild restriction on
~,
by Corollary lB. I to non-stationary sequences.
Coroll-
ary lB. 2 specializes this to weakly stationary sequences, and Corollary
lB. 3 offers useful alternative assumptions.
Our results for weakly stationary sequences also apply to strictly
2
stationary sequences satisfying E Xl <
In this respect, Corollary
00
lB. I nearly coincides with the Ibragimov theorem quoted, but theorems
I
I
I
I
I
I
I.
I
I
4A and 4B provide interesting complements to the theorems already available for strictly stationary sequences.
Note that under stri.ct station-
arity the uniformity requirements in conditions (ii), (iiI), (iii) and
(iii') hold automatically.
Results for certain weakly stationary and m-correlated sequences
appear in the following section.
8.
m-correlated Seguences.
00
We say eX.}
is m-correlated if Cov [Xa ,xa+].J = 0 whenever j > m.
1 _00
We shall assume, as ususal, that E X. = 0 .
1
A sequence of m-th order martingale differences is one satisfying
I
50
Elx,
1
I<
00
and E(X
a
Ipa-J,)
=
0 if j > m,
If m = 0, it is called simply a
sequence of martingale differences.
Since, if j > 0, E X X +'
a a J
=
E(X
a
E(X
,Ip)} , a sequence of m-th
a+J
a
order martingale differences is m-correlated,
m-dependent
<
i f E lx, r
1
1
Also, it is clear that an
sequence is a sequence of m-th order martingale differences
00
We will make use of Theorem IE via
Lemma 12.
If (X,}oo
is m-correlated, condition (C) holds.
1 _00
2
Elx. 1 < M <
1
00
,
-
Proof:
If also
then conditions (A) and (H) are equivalent.
The first statement holds trivially.
The second follows by re-
ferring to Lemma 2 and noting that its proof suffices for this slight extension.
Hence, for m-correlated sequences in general, we have
Theorem SA.
Let (X.}oo
be m-correlated and satisfy E X.1 = 0 , condition
1 _00
(A) [or (H)] ,
a+n
(i ") ElL:
a+l
X. 12+ 5 < Mn 1+1'
1
(M
<
00
o<
~(5 ~ l' ~ 1+5 ~ 2)
-
and either (ii), (iii) } or (ii'), (iii')} , with (**)
2(1-~)+€ <
:1'
,}
I
I
I
I
I
I
.J
I
I
I
I
I
I
Then (Xi} has the central limit property.
I
As an example, let us consider a sequence of uncorrelated and
I
Ide fined in the Introduction.
••
I
I
Ie
I
I
I
I
I
I
I.
I
I
I
I
I
I
I
I.
I
I
51
identically distributed random variables.
correlated and weakly stationary, if E
X~ <
Such a sequence is both un00
,
and satisfies (A).
Hence, by Theorem 4A or SA,
Let [Xi}~oo be a sequence of uncorrelated and identically
Example SA.
distributed random variables, with E Xl = 0, E
X~ <
00
either [(ii), (iii)} or[(ii'), (iii')} hold, with (**)
If (i") and
2(1:~€ < 2~
,
then (1) holds.
Let us now narrow attention to sequences of m-th order martingale
differences.
It is easily seen that our conditions (ii) and (ii') are
satisfied automatically, with any
superfluous.
~
and E.
Hence (*) and (**) become
Therefore a rigid (2+5)-moment condition offers no ad-
vantage, so that the simple condition (i') will be satisfactory for our
purposes.
Moreover, conditions (iii) and (iii') become equivalent.
shall also use:
Lemma 13.
If [X.}oo
is a sequence of martingale differences, then
1 _00
a+n
E(I I:
a+l
(48)
a+n
Proof:
II: x.1
a+l
But, for j > i
1
x./ 2 Ip)
1
2
a
a+n
=
a+n
=
2
E(X. Ip ) .
a+l
1 a
I:
n-l
I: X: + I:
1
n
I: X .X
.
. '+1 a+1 a+J
i=l J=1
a+l
I
= 0
E[Xa+.1 Xa +.J Pa ] = E[Xa+.1 E(Xa +.Ip
J a+.)
1 Ipa }
By Lemma 13, a simple sufficient condition for a sequence of
martingale differences to satisfy (iii') is
We
I
52
2
2
) - E x 1= 0(1) uniformly in a , as m ->
(iiim) E/E(x Ip
a a-m
a
00
•
I
I
For, by Lemma 13,
a+n
a+n
2
a+n
2
a+n
2
EIE(/ L x.1 1p
) - EI L x. I 1- EI L E(XiIP
) - L E xi!
a+1 ~
a-m
a+1 ~
a+1
a-m
a+1
a+n
< L EIE(X~lp
a+1
~
a-m
) -
Ex~1
~
If (iiim) holds, the right-hand side is n 0(1), uniformly in a, as m ->
00.
Hence
Condition (iiim) ==> condition (iii'), in the case of a
Lemma 14.
sequence of martingale differences.
From this discussion, we may conclude the following theorems for
sequences of m-th order martingale differences and of martingale differences.
Theorem 5B.
Let (X.}oo_00 be a sequence of m-th order martingale differ~
I /2+0
ences satisfying condition (A) [or (H)] , E Xi
o>
0 , and condition (iii t ) .
Theorem 5C.
~
00
for some
~
00
Let (X.} _00 be a sequence of martingale differences satisfy~
form1y in a , as m ->
(X.~
M<
Then (X.} has the central limit property.
ing condition (A) [or (H)] , Efxil2+0
Then
~
00
~
M<
00
,}
for some 0 > 0 , and, uni-
,
has the central limit property.
Theorem 5C is, of course, contained in Theorem 5B, but is stated for its
I
I
I
I
.'I
I
I
I
I
I
I
.-I
I
I
..I
I
I
I
I
I
I
••
I
I
I
I
I
I
53
intrinsic interest.
Corollary 5C. 1.
I
I
(X.}oo
be a weakly stationary sequence of martin1 _00
Let
2
gale differences, with E X.
1
formly in a, as m ->
Elx.1 12+0 <
M (M
<
00
o>
0) and, uni-
00
Then (X.} has the central limit property.
1
This result provides an interesting complement to the following
theorem of Ibragimov [12]:
Theorem (Ibragimov).
Let
(X.}oo
be an ergodic strictly stationary se1 _00
quence of martingale differences, with E
X~1 = A2
Then (X.} has the
1
central limit property.
For completeness, we shall mention the implications of our theory in
the case of m-dependent sequences; but it should be noted that, except for
an equivalent version of the Hoeffding-Robbins theorem, our theorems do
not give new results for m-dependent sequences.
Thus, for m-dependent sequences in general, our results in conjunction with the Hoeffding-Robbins theorem, yield
Theorem 5D.
Elxil2+0
I.
In the weakly stationary case, Theorem 5C reduces to
~
00
Let (X.}
be an m-dependent sequence with E X.1
1 _00
M (M <
00
;
0> 0) and satisfying condition (A) [or (H)].
Then (X.} has the central limit property.
1
o,
I
54
In the weakly stationary case, we have
Let (X.}oo
be m-dependent and weakly stationary, with
1. _00
Corollary 5D. 1.
2+5
E X. = 0 , E IX. 1
1.
1.
<
M
(M
<
5
00
>
0) .
Then (X.} has the central
1.
limit property.
9.
Sequences with Markov-bounded Second-order Dependence.
In Section 2, "Remarks about Theorem lA", we discussed certain
a+n
(n)-~IE(
"errors" Ie I and
a
L:
a+l
X.
1.
IFa-m )1
between conditional expectations,
given P
, and their corresponding unconditional expectations.
a-m
Such
quantities measure, in a sense, the second-order dependence in a sequence.
If these quantities are bounded by a quantity which depends upon the past
P
a-m
only through the last observation, X
, of P
, and which dia-m
a-m
minishes as m and n
~
-?
00
,
we shall say that the second-order
dependence is Markov-bounded.
With these intuitive concepts in mind, and speaking as usual of
sequences satisfying E Xi = 0 , we give the following definition:
00
=0
A sequence (X.}
satisfying E X.1.
1. _00
is said to have Markov-
bounded second-order dependence (MSD) if, uniformly in a,
(49)
a+n
IE(I L:
a+l
(50)
with
€ ~
0 and 0
2
x.1
1pa-m ) 1.
~ ~ ~ ~
a+n
2
EI L: X.
< n
a+l 1.
-
0
> 0 or
~
, and either
11
€
n,m
2
J,
a-m
(1) [1 + X
< \ .
We shall refer to (49) , (50)} as condition (MSD)
J
I
I
I
I
I
I
I
-.
I
I
I
I
I
I
--I
I
I
Ie
I
I
I
I
I
I
I
••
I
I
I
I
I
I
I.
I
I
55
Lennna 15.
00
If (X.}
satisfies E X.1
1 _00
=0
and conditions (A), (it) and
(MSD) , then also conditions (ii), (ii t ), (iii), (iii'), (i) with
y
= ~5
Proof:
, (*), (**), (iii'B) and (B) hold.
By (i') and Holder's inequalit,y, there exists M' for which
Hence, taking expectations in (49) and (50), we immediately have
(ii') and (iii').
Moreover, squaring both sides of (49) and then taking expectations,
we obtain (ii) in the form
(52)
By the restrictions on
~
and
E,
the right-hand side of (52) is n
0
n,m
(1).
Therefore, (52) with (iii') implies (iii).
Moreover, by (50) and (i'), it is clear that (iii'B) is satisfied,
and so byLennna 4, Section 4, (B) holds.
Therefore, from Theorem 2, (i) holds with y
(*) and (**) hold, since
2~
< 1 +
~E
= ~5.
This shows that
•
Using Lennna 15 with Theorem lA under (*), we have
Theorem 6.
Let (X.}oo
1
_00
satisfy
conditions (A) and (MSD).
E X.1 = 0
,
Elx.1 12+5
<M <
00
(5 > 0), and
Then (X.} has the central limit property.
1
It should be noted that the class of sequences which satisfy condition (MSD) is fairly wide, and hence Theorem 6 is fairly general.
example, m-dependent sequences satisfy condition (MSD).
For
I
56
10.
Bounded and Nearly-bounded Sequences.
00
In this section we assume that the sequences eX,}
satisfy
1. _00
E X.
1.
=0
and condition (A).
If the X. 's are bounded by 1, then by Corollary 3. 2 a condition of
1.
form (i") is satisfied with y
=
5
That is,
(53)
In this case (i") can be dropped as a hypothesis of certain of our
theorems.
Moreover, by Corollary 3. 1, such a simplification may be made
even when the X. 's are not bounded by 1, but rather satisfy a bounded
1.
moment condition of high order.
We shall see that any possible such sim-
plification will not be of advantage in the case of a low-order moment
condition.
Note that if the X. 's satisfy a high-order moment condition,
1.
E
(54)
Ix. 12+h
1.
<M ,
then (under condition (A», according to Corollary 3. 1, (i") holds with
y
1
= 5(1~)
and (**) becomes (**)
2~
bounded by 1, then (**) becomes (**)
2
+ h<
€
<
€
2~
If, further, the X. 's are
1.
Under either of these conditions, (i") may be deleted from the hypotheses of the theorem, if (**) is changed as indicated.
Thus (i") need
not be checked in order to apply the theorem if the new (**) is satisfied.
The flexibility of (i") is sacrificed to gain this simplification.
If h
is small in (54), it is clear that (**) becomes a severe restriction, in
which case it would be more advantageous to retain (i"), to allow for the
case of y being small enough to make (**) more favorable.
If h is large,
or if the X. 's are bounded, (**) will be satisfied in some situations, so
1.
that this simplification deserves a place in our repertoire of theorems.
tJ
I
I
I
I
I
I
I
-.
I
I
I
I
I
I
I
-.
I
I
57
Ie
I
I
I
I
I
I
I
••
I
I
I
I
I
I
I.
I
I
Hence we have, extending Corollary lEo 1,
Theorem 7.
00
Let (X.}
be bounded
1 _00
2
and satisfy E X.1
=0
, conditions (A)
and (C), and either (ii), (iii)} or (ii'), (iii')}, with (**) 2~ <
2 h
Or, if merely Elx / +
i
S M , (**) 2~ + ~ <
€
€
•
•
Then (X.} has the central limit property.
1
The generality of this theorem can be seen in that, for example, it
nearly contains the following result.
Theorem 8.
Let (X.}oo
be bounded and satisfy E X.1
1 _00
(A), (C) and (I) with ~(m)
= O(m- A)
, A> 1.
=0
, and conditions
Then (X,} has the central
1
limit property .
Proof:
by 1.
Without loss of generality we may assume that (X.}oo
is bounded
1 _00
For, if Ix.1
I<M,
and (X./M}oo
has the central limit property, so
1
-00
Applying condition (A) and Corollary 3. 2, there exists M <
00
such
that
(55)
a+n
2+5
1+5
ElL: X. I
< Mn
,for all 5 > 0 .
-a+l 1
Hence, for p > 1 , (55) with 5
(56)
= 2(p-l)
gives
1
1 2-1
a+n
2p
ElL: X. 1 )p < MP n p , for all p > 1 .
a+l 1
Hence, by Lemma 6,
(57)
2
Clearly the extension for bounds> 1 is immediate.
See proof of Theorem 8.
I
58
Letting p ->
in a , as m ->
00
00
~(m) =
in (57) and putting
Oem -:A ) , we have uniformly
I
I
,
(58)
Also, by Lemma 6 with P
EIE(
(59)
a+n
L
a+l
=
x.lp
) I <1
a-m
uniformly in a , as m ->
00
n
2 , we obtain from (55)
-\
-\
m-:A 0(1) ,
•
We now show that (59) and (58) suffice in place of (ii') and (iii'),
respectively, in the proof of Theorem IE.
(59)
> (10) and (58) --> (12).
We need to show that
We will use without change the notation
of the proofs of Theorems IE and lAo
(60)
Then (10) will follow if it can be shown that
(59) is of form (ii') with
~
(l~~~ , ~).
= -\ ,
:A
=E
Since:A
===>
)[y,
~
~ ~:A
<a -
and (55) permits
,
~.
Since
r = 0 , we
===> 1 < a + :A(a-Q) ->
~
a . Then l+Q:A
1+:A < a ....>
-
-\~:A
<a -
(10) .
By (58), uniformly in m
(61)
-
> 1 and Q can be chosen arbi-
trarily small> 0 , we can choose such an
1 + Q:A < a +
I
I
I
I
I
-.
By (59), uniformly in m
want a to lie in
,}
$. n
)a-I
0(1),
~.
Hence (60)
I
I
I
I
I
I
I
-.
I
I
..I
I
I
I
I
I
I
I·
I
I
I
I
I
I
I.
I
I
59
since it was just shown that
a -
~A
<
~
- 1 .
But then, by condition (A), we see that (12) follows.
It is easily seen that the'remainder of the proof of Theorem IE
remains intact under these alterations.
This theorem has several corollaries of interest.
Corollary 8. 1.
satisfying E X.
1
~(m)
Let (X.}oo
be a bounded and weakly stationary sequence
1 _00
= 0,
~Ir. I <
1
00
and regularity condition (I) with
,
J
= O(m -A ) , A> 1. Then (X.} has the central limit property.
1
Note that Corollary 8. 1 offers considerable improvement over
Corollaries lB. 2 and lB. 3 in the case of a bounded weakly stationary
sequence.
Corollary 8. 2.
fying E X.
1
~(m)
=0
= O(m -A)
Remark.
Let (X.}oo
be a bounded and m-correlated sequence satis1 _00
, condition (A), and regularity condition (I) with
, A> 1 .
Then (X.} has the central limit property.
1
There is an interesting comparison between Theorem 8 and
Corollary lB. 1, Section 5, which closely parallels Rosenblatt's theorem
and generalizes a result of Ibragimov.
We see that under condition (C)
and boundedness it is possible to eliminate the condition
(45)
required in Corollary lB. 1.
I
60
We conclude this section, and the chapter, with an example that
arises in Chapter III.
Sum of Distribution Functions.
Example.
Let (X.)oo
be a sequence of
1 _00
identically distributed random variables with common distribution
function F.
F* (X.)
1
00
Then the sequence (F* (X.»)
is bounded, where
1
_00
= F(X i )
- E F(X.) .
1
Moreover, E F* (X.) 2
1
·
= A2 (constant).
If we assume weak stationarity of (F*(X.») , with
1
~ r. convergent,
1
J
condition (A) holds.
Then, under the further assumption that (I) applies to (F(X.») ,
1
with
-l
= O(m ) ,
~(m)
l
> 1 , we may apply Corollary 8. 1 to conclude
n
that
n-~ ~ F(X.) is asymptotically normal.
1
1
Or, if (I) is too strong [in this case, it seems virtually to con-
stitute asymptotic independence], we may assume ~Ir. I < 00 so that (A) and
1 J
(C) hold. Then we need only convince ourselves that either (ii), (iii»)
or (ii'), (iii'») hold, in order to apply Theorem 7.
~
I
I
I
I
I
I
I
~
I
I
I
I
I
I
JI
I
I
..I
I
I
I
I
I
I
CHAPTER III
APPLICATIONS
11.
(a)
Preliminary Remarks.
Let Xl' X '··· and Y , Y2 ... be independent sequences of possibly
2
1
dependent variables, with respective continuous marginal distribution
functions F and G.
••
I
I
I
I
I
I
The Wilcoxon 2-Sample Statistic.
The hypothesis to be tested is H:
Define
s(u)
= pry
and put 9
If F
=G
>
xl , r = 29
=~
, then 0
tively by testing 9
H : F
o
=G
u
> 0
o
u
=0
-1
u
< 0
Hence H:
F
o
= 29
- 1
=G
may be tested conserva-
0
=r ,
we thus may test the hypothesis
by the U-statistic
U
flO(t) = 1
E fOley)
=
1
- 1 .
= ~ , i.e. r =
Noting that E s(Y-X)
=
1
=r
m
~
n
~
mn 1.=
. 1 J=
. 1
Define flO(t)
I.
I
I
F = G .
o
=E
2G(t) , f
s(Y.-X.)
J
1.
s(Y-t) , fOl(t)
01
=E
set-X)
Then
(t) = 2F(t) - 1 and hence E flO(X) = r ,
I
62
We first investigate the question of when Z = m\(U-y) has the same
limiting distribution as
W= m
-\
n
-1
n
~ [fOl(Y') - yJ ,
J
j=l
as m, n ->
00
such that min has a limit c .
Secondly we investigate what conditions suffice for the limiting
distribution of W to be normal.
Let us note that E Z = E W = 0 , so that Z and W will have the same
limiting distribution (if any) if E(Z-W)
2
-> O.
For in this case
(Z-W) -> 0 in probability by Chebyshev's inequality and hence Z and W
have the same limiting behavior.
Our work parallels in past some work
quoted in Fraser [8J.
(b)
Conditions under which E(Z-W)2 -> 0 .
Let g(x,y) = [s(y-x)-rJ - [flO(x)-yJ - [fOl(y)-rJ
Then
Z - W = m-\ n- l
m
n
~
~
i=l j=l
g(X., Y.)
1.
J
Hence
(62)
E(Z-W)
2
-1-2
= m
n
6,
where
(63)
6 =
m
m
n
n
~
~
~
~
6(a,b,i,j) ,
i=l j=l a=l b=l
and
(64)
6(a,b,i,j) = E g(X.,Y )g(x.,Y )
1.
a
J b
We now evaluate 6(a,b,i,j) .
..I
I
I
I
I
I
I
••
I
I
I
I
I
I
••I
I
I
..I
I
I
I
I
I
I
I·
I
I
I
I
I
I
63
Since E g(X,y) = 0 = E g(x,Y) ,
s(Y -X,)s(Y -X,) b
a
J
y2 - 4
1
Cov[G(X,),G(X,)]
J
1
Now
E s(Yb-X,)s(Y -X,) = E s[(Yb-X,)(Y -X,)]
a
J
a
J
1
(66)
1
=
2P[(Y -X.)(Y -X.) > 0] - 1
b J
a 1
=
2P[Yb < X"Y < X,] + 2P[Yb > X"Y > X,] - 1.
J a
1
J a
1
Let F" denote the joint distribution function of (X, ,X.) , G b the
1J
1 J
a
joint distribution function of (Ya,Y ) , Then
b
P [Y
b
<
X., Y
J a
<
X,] 1
(1-9)
2
(67)
=
=
E G b(X, ,X,) - (1-9)
a
2
J
1
E[G b(X"X,) - G(X.)G(X.)]
a
J
1
1
J
+ Cov[G(X,),G(X,)] ,
1
J
and similarly
(68)
P[Y > X"Y > X,] - 9
b
J a
1
2
= E[F,1J,(Ya ,Yb )
- F(Y )F(Y )]
b
a
+ Cov[F(Ya ), F(Yb )] •
Thus, since
y2
= 2(1_9)2 +
29
2
- 1 , it follows from (66), (67) and (68)
that
(69)
I.
I
I
=E
(65)
l:>(a,b,i,j)
= 2E[F,1J,(Ya ,Yb )
- F(Y )F(Y )] - 2 Cov[F(Y ).,F(Yb )]
b
a
a
+ 2E[G b(X"X,) - G(X,)G(X.)]- 2 Cov[G(X,), G(X,)],
a
1
J
1
J
1
J
I
64
The conditions under which E(Z-W)
2
-> 0 will be given in terms of
the quantities on the right-hand side of (69).
The following result will
also be of use.
Lemma 16.
co
Suppose we have a function r(k), with L r(k) convergent, such
that 16(a,b,i,j)1
S r(max[la-~,li-j
o
I)
2
Then 6 = o(mn ) , as m, n ->
co
such that min has a limit.
Proof:
Let us partition 6 as follows:
n
n
m n
m
m
n
L
L
L 6(a,b,i,i) = L
L
L 6(a,a,i,j) - L
L 6(a,a,i,i)
i=l a=l b=l
i=l a=l j=l
i=l a=l
m
6 =
m
n
+ L
L
m n
L
L 6(a,b,i,j) .
i=l a=l j=l b=l
j;'i b;'a
Then
n
n
m m
L r( /a-b I) + n L
L r(/i-j
161 <mL
a=l b=l
i=l j=l
+ 4
I)
+ mn reo)
m
n
m-l n-l
L
L
L
L r(maxr!a-b I, li-j I J)
i=l a=l j=i+l b=a+l
By the same method used in proving Lemma 11, it is easily seen that the
first two terms of the right-hand side are O(mn) , since
Hence, since
(70)
161
co
~
m-l m-i
m-l i
L
L f(j) = L
L f(j)
i=l j=l
i=l j=l
s
O(mn) + 4
Now, letting T =
m-1 i n-l a
L
L
L
L r(max[j,bJ)
i=l j=l a=l b=l
.
m-l i n-l a
L
L
L
L r(max [j , b J), we have
i=l j=l a=l b=l
r(k) converges.
..I
I
I
I
I
I
I
••
I
I
I
I
I
I
••I
I
I
..I
I
I
I
I
I
I
I·
I
I
I
I
I
I
I.
I
I
65
T =
m-l
i
L:
L:
i=l j=l
=
m-l
i
L:
L:
i=l j=l
=
m-l
i
L:
L:
i=l j=l
m-l
=
L:
I
j
a
L:
L:
n-l a
r(max[j ,b]) + L:
L: r(max[j ,bJ) - j r(j) !
a=l b=l
a=j b=l
j
{
n-l
a r(j) +
L:
a=l
1
a
L: [j r(j)
a=j
m-l
L:
[i 0+1) + j(n-j) - n] r(j) +
2
i
+
L: (n-j 12)(j -1) r(j)
i
+
L: n j r(j)
L:
a
L:
L:
a=j b=j
r(b)!
n-l
L: (n-a) r(a)
m-l
i
L:
L:
n-l
L: (n-a) r(a)
m
+ mn
L: i r(i)
L: R(i) ,
i=l
where R(i) =
i
L:
n-l
i=l j=l a=j
m
< mn
m-l
r(b) - r(j)] - j r(j)!
i=l j=l a=j
i=l j=l
(71)
L:
b=j
i=l j=l
<
+
i=l
00
L: r(j)
j=l
m
It is easily seen that L: R(i) = o(m) since R(i) -> 0 as i ->
i=l
00
•
Moreover, taking an integer-valued function g(m) which satisfies g(m) ->
and g(m) = o(m) , as m ->
00
,
m
g
L: i r(i) =
L: i r(i)
i=l
i=l
we may write
+
m
L:
i r(i)
i=g+l
g
~
g
L: r(i)
+ m R(g)
i=l
=g
0(1)
= o(m)
+
m 0(1) , as m ->
, as m ->
00
00
•
2
2
Therefore the right-hand side of (70) is 0(m n) , which is 0(mn ) if
m, n ->
00
such that min
has a limit.
00
66
Theorem 9.
Suppose, for some function h(k) with
00
~
o
h(k) convergent,
/E[F .. (Y ,Y ) - F(Y )F(Y )]I < h(li-j I) ,
b
1J a b
a
-
I Cov[G(Xi )
, G(Xj)]I~ h(li-j
I) ,
IE[G b(X.'X.) - G(Xi)G(X.)]I < h(/a-bl) ,
a
1
J
J-
and
Then Z and W have the same limiting behavior as m, n ->
00
such that min
has a limit.
Proof:
This is seen easily by considering (69) and Lemma 16.
Theorem 9 shows how to determine conditions in terms more conventiona1 than those of Lemma 16.
In some situations, the conditions of
Theorem 9 are suited to direct verification.
Otherwise, the following
result may be of use.
Corollary 9. 1.
(I), with
~(n)
If both (Xi} and (Y } satisfy the regularity condition
j
satisfying
ing behavior as m, n ->
00
00
< 00 , then Z and Whave the same 1imit1
such that min has a limit.
~ ~(n)
This corollary follows from Theorem 9 because of
Lemma 17.
(a)
and
If (X.} satisfies regularity condition (I), then
1
/E[F .. (Y ,Y ) - F(Y )F(Y )] < ~(Ii-j
b
1J a b
a
-
I) ,
I
~
I
I
I
I
I
I
I
~
I
I
I
I
I
I
JI
I
I
..I
I
I
I
I
I
I
•e
67
Proof:
Let A and B denote the events (Xi < t } and (X < t } , respectj
Z
l
ively.
Then by Lemma 5, Section 4,
Ip(AB) - P(A)P(B)I
S
~(Ii-j
I) ,
which is the same as
IF ij (tl,t Z) - F(tl)F(t Z)
(72)
Is
~(/i-j
I) ,
which implies (a) .
Now, for j > i ,
Cov[G(X.) , G(X.)]
J
1
= E[G(X.)G(X.)
- E G(X.) E G(X.)
1
J
1
J
= E(G(X.)[G(X.)
- E G(X.)]}
1
J
J
= E(G(X.)[E(G(x.)lp.)
- E G(X.)]}
1
J
1
J
so that, since
I Cov[G(X.)
(73)
1
, G(x.)]1 < EIE(G(X.) Ip.) - E G(x.)1 .
J
-
J
J
1
But, IG(x.)1 being bounded by 1, we can take p arbitrarily large in
1
I
I
I
I
I
I
I
•e
I
Lemma 6, Section 4, to yield
whereby (73) implies (b) .
(c)
Conditions under which Z and Ware asymptotically normal.
m
Now W is asymptotically normal if each of
~
flO (X.) and
i=l
1
m
is, or equivalently if each of
~
i=l
For the two sums
~
G(X ) and
i
~
n
G(X.) and
1
~
j=l
F(Y.) is
J
F(Y ) are independent, so the characterisj
tic function of W is the product of the characteristic functions of the
normed sums .
I
68
Hence it suffices to investigate conditions under which a sequence
such as (G(X.)} has the central limit property.
1
Let G* (Xi) denote
G(X.) - E G(X.) and A2 denote E[G* (X.)] 2 .
1
1
1
If we assume (G(X.)} is weakly stationary, and that
1
~ C~7[G(X.) , G(X.)] converges, then [G* (Xi)} satisfies conditions (A)
1
11
and (C) of the previous chapters (as well as condition (b) of Theorem 9).
If we further assume that [X.} satisfies regularity condition (I),
1
then so does (G(X.)} and hence also [G* (X.)}, since events about X 's may
1
i
1
be represented as events about G(Xi)'s and vice versa.
Under these assumptions, with
~(k) = O(k- A) , A > 1 , Theorem 8 of
00
Section 10 applies to [G* (X.)}
and also the conditions of Corollary 9. 1
1
_00
are satisfied.
The condition that [G(X.)} be weakly stationary is satisfied if [X.}
1
1
is strictly stationary of order 2, i.e. if F.. depends only upon /i-j
1J
I.
Hence Z and Ware asymptotically normal if the sequences [X.}, [Y.}
J
1
are strictly stationary of order 2 and satisfy regularity condition (I)
with
~(k) = O(k- A) , A > 1
In some circumstances, any strict stationarity assumptions may not
be feasible. But Lemma 17 shows condition (C) holds if [X.} satisfies (I)
1
with
~(k) =
O(k) ,A
A> 1. ·
Thus Theorem 8 applies if [G* (X.) } satisfies
1
condition (A).
These considerations constitute the following theorem.
Theorem 10.
00
00
Let (X.}
and [Y.}
be two independent sequences of
1 _00
J _00
possibly dependent random variables identically distributed with continuous marginal distributions F and G respectively.
satisfies regularity condition (I) with ~(k)
Assume each sequence
= O(k- A)
, A > O.
If
~
I
I
I
I
I
I
I
~
I
I
I
I
I
I
I
~
I
I
Ie
I
I
I
I
I
I
69
(x.), (Y.) are each strictly stationary of order 2 (or, more generally,
1.
J
if condition (A) is satisfied by each of the sequences (G(X.»), (F(Y.»)),
1.
J
then as m, n ->
00
such that min has a limit c, the statistic
has a limiting normal distribution with mean zero and variance
1
4 lim
m
(d)
->
00
m
- Var( ~ G(X.»
m
i=l
1.
+ 4c lim
n _>
1
00
n
- Var( ~F(Y.»
n
j=l
J
Remarks relevant to Application of Theorem 10.
While the limiting behavior of
m-~(U-r)
is distribution-free under
appropriate conditions, the limiting variance is parametric.
Therefore,
in some instances, application of the theorem will require an estimate of
I.
I
I
I
I
I
I
I
I.
I
I
2
.
A .
t h e var1.ance"
This is under study.
However, for some purposes the theorem is immediately applicable.
For instance, an evaluation of the power can be accomplished by specify-
2
ing alternatives in terms of rand A .
2
Let us examine some representations of A , assuming the existence of
conditions sufficient for Theorem 10 to apply and for (G(X.»), (F(Y.») to
1.
J
be weakly stationary sequences.
Under such conditions we may write
2
A = 4(Var G(X ) +
l
2~
Cov[G(X ), G(X »))
l
i
(74)
Under the null hypothesis, this quantity is
(75)
A~
= 4(1+C)(f2+
2~
CoV[F(X l ) , F(Xi »))
2 A2 is
Now, by Lemma 17, an upper bound for each of A,
o
I
70
8(l+c)(~(o) + 2~ ~(i)}
(76)
1
Hence conservative tests and estimates of power may be based upon (76).
2
Another approach is possible if A can be known, or approximated, by
o
some criterion.
For example, suppose we are dealing with a class (F} of
2
distributions for which A does not greatly vary.
o
[In some situations,
it is feasible to assume that the covariance structure of the sequence
under consideration does not depend a great deal upon the particular distribution F].
2
2
A will not differ appreciably from A
o
Thus, once a
2
single parameter A is determined, our theory will be non-parametric for
o
a class of distributions (F}U(G}.
(e)
Particular Application:
Comparison of Rates of Occurrence.
Two series of events can be compared by forming for each series the
sequence of intervals between successive events and then using methods
for the comparison of the means of samples, the mean interval being the
reciprocal of the rate of occurrence.
See Section 9.4 of Cox and Lewis
[2].
Specialized techniques apply if the series under comparison are
Poisson processes.
When this assumption is not feasible, but the series
can be assumed to be renewal processes [that is, the sequence of intervals is a sequence of independent and identically distributed random
variables], various general methods may be used.
For example, if a
convenient 2-parameter distribution, such as Gamma, lognormal, or
Weibull, can be assumed, the standard maximum likelihood method is
effective.
Or, a convenient non-parametric approach is the Wilcoxon
2-sample statistic.
However, a renewal process cannot always be assumed.
In
~
I
I
I
I
I
I
I
~
I
I
I
I
I
I
JI
I
I
Ie
I
I
I
I
I
I
I
I-
71
applications such as machine stoppages, it is common to find appreciable
correlation present in the sequence of intervals.
parametric procedures, which require the assumption of independence, cannot be invoked.
II
I
Our Theorem 10 shows, however, that the Wilcoxon pro-
cedure is of use in these more general situations.
12.
The Empirical Distribution Function.
Let (X.)
1.
00
be strictly stationary of order 2.
_00
Let F denote the con-
tinuous distribution function of X. and F. the joint distribution function
J
1.
of (X. ,X.+.) .
1.
J
1.
A non-parametric estimator of F(t) is the "empirical distribution
function"
=
F (t)
n
number of x 's < t
--=i'---_ _
1
0
u>
0
<
0
u
n
=1
L: l(t-x.)
n i=l
n
-I
where leu)
I
I
I
I
I
I
Thus, the standard non-
1.
The sequence (l(t-X.)} is then weakly stationary, with
1.
= F(t)
E l(t-X.)
1.
Cov[l(t-X. )
1.
, Var l(t-X.)
1.
l(t-X.+.)]
1.
J
= F(t)
= F.(t,t)
J
- F(t)
- F(t)
00
2
Suppose (X.}
satisfies (1) with ~(m)
1. _00
does (l(t-X )} , so IFj(t,t) - F(t)2,
i
2
.
= Oem -A )
~ ~(j),
is finite.
Thus, from Corollary 8. 1 we obtain
, and
, A> 1.
Then so
according to (72).
Then
72
Theorem 11.
Let (X.)oo
be strictly stationary of order 2 and satisfy
1 _00
~(m)
regularity condition (I) with
= O(m -:A )
, :A > 0 .
Let X. have
1
I
..I
continuous distribution F and (X, ,X,+,) have joint distribution F.
1
1
J
J
Let F (t) denote the empirical distribution function,
n
n~[F n (t)
- F(t)] has a limiting normal distribution with mean zero and
variance A
13,
Then
2200
2
= [F(t) - F(t) J + 2 ~ [F,(t,t) - F(t) ] ,
j=l J
Covariance Function Tests,
2
Let (X,) be a weakly stationary sequence with E X, = 0 , E X. = cr
1
1
2+5
and for some 5 > 0 , E IX, 1
<
M < 00 ,
1
r.
J
=E
2
1
Suppose that the covariances
X X +' have absolutely convergent sum R
aaJ
00
=~
1
r.
J
(Hence con-
ditions (A) and (C) are satisfied),
If, also, conditions (i") and either (ii), (iii») or (ii'),
.lL
(iii'») are satisfied, with (**) ~2~(~1--~~)-+-E < 2r ' then by Theorem 4A
1
n
n
-~ ~
, 1
X, is asymptotically normal with mean zero and variance
1=
1
2
2
A = cr + 2R
n
2
, asymptot1ca
, 11 y ch'1-square W1t
, h
In this case, (nA )-1 ("
w X,)2 1S
1
1
1 degree of freedom.
Test 1.
H :
o
Let (Xi) satisfy the above conditions.
R = R versus HI:
o
Then a test of
R = R may be based upon the asymptotically
1
n
chi-square (1) statistic
Test 2.
n-1(~
1
x.)2
1
Let (X,), (Y.) satisfy the above conditions and be independent
of each other.
1
1
Then a test of H:
o
R
x
= Ry
may be based upon the
I
I
I
I
I
I
-.
I
I
I
I
I
I
_.
I
I
I
Ie
I
I
I
I
I
I
I
••
I
I
I
I
I
I
73
F(l,l) statistic
n
n
l
2
2
n2(~ X.) /nl(~ y.)
1
Remark.
1
1
The asymptotic power of these tests can be evaluated without
difficulty.
14.
1
2
A shortcoming, however, is their lack of consistency.
Tests for Renewal Processes.
As stated by Cox and Lewis [2; Section 6.4], the problem of testing
for independence of the intervals (between successive events of a series)
is complicated by the broadness of the null hypothesis and the lack of
clearly specified alternatives.
lations, there is also the difficulty of combining the tests for the
various lags.
Test 1 of the preceding section offers a quick test which circumvents these difficulties.
Under the null hypothesis, R
= O.
Alterna-
tives may be specified by values of R, within the class of weakly stationary sequences possessing the central limit property, and the power easily
calculated.
And the test takes into account all of the serial corre-
lations.
Some more sophisticated techniques are presented in Cox and Lewis
[2].
One of these, the Sherman statistic,is asymptotically normal under
the null hypothesis.
If the same is true for certain weakly stationary
alternatives, the power can be determined against such alternatives.
Our "particular theorems" of Chapter II may be of use in defining suitable classes of alternatives.
I.
I
I
With respect to tests for serial corre-
I
74
APPENDIX
We list for easy reference the various conditions assumed.
(1)
Ip(BI~~- P(B)/ ~ ~(n)
to,
as n ->
00
with probability 1,
,
00
for all B E Ma+n .
(II)
Ip(AB) - P(A)P(B)I ~ a(n)
as n ~
to,
00
,
for all A E M~oo
00
B
(A)
E
a+n
M
a+n
Var( 6 X.)
a+l 1
~ nA 2 uniformly in a , as n ->
00
•
m>m
(B)
-
0
(B' )
(C)
~ Icovrx ,X +,]1 converges uniformly in a .
j=l
- a
a J
(R)
a+n
(i)
EI 6
a+l
2
X 1 + 6 < Mn1+1'
i
a+n
(i')
ElL:
a+l
(i' I)
(ii)
X.
(M <
00
-
12+6 ~ Mn1+~6
(M
<
0 < ~6 ~ I' < 6 ~ 1)
6
00
>
0)
1
(M
<
00
0
<
~6 ~ I' ~ 1+6 ~ 2)
2 a+n
2~
E E (L: X. Ip
) ~ n [O(n -E) + Oem -E)] , uniformly in a .
a+l 1 a-m
tJ
I
I
I
I
I
I
I
-.
I
I
I
I
I
I
--I
I
I
Ie
I
I
I
I
I
I
I
.I
I
I
I
I
I
I
.I
75
(ii' )
(iii)
(iii')
a+n
EID( ~
a+l
EIE(/
x.lp
1.
~n
~ x.
a+l
a-m
/21p
1.
(iii'B)
uniformly in a
(*)
(**)
)-
a-m
a+n
D( ~ X.)
a+l 1.
) - EI
I<
~n
~ x.
a+l
n
0
-
1.
n,m
(1), uniformly in a .
2
I 1< n
-
0
n,m
(1), uniformly in a .
I
76
BIBLIOGRAPHY
[1]
Bernstein, S. (1927). Sur l'extension du theor~me du calcul des
probabilites aux sommes de
Mathematische Annalen.
[2]
Cram~r,
dependantes.
2l, 1-59.
Cox, D. R. and Lewis, P. A. W. (1966).
Series of Events.
[3]
quantit~s
The Statistical Analysis of
Methuen, London.
Harald (1946). Mathematical Methods of Statistics.
Princeton University Press.
[4]
'Diananda, P. H. (1953).
Some probability limit theorems with
statistical applications.
[5]
Diananda, P. H. (1954).
Proc. Camb. Philos. Soc. 49, 239-246.
The central limit theorem for m-dependent
variables asymptotically stationary to second order.
Proc.
Camb. Phil. Soc. 50, 287-292.
[6]
Diananda, P. H. (1955).
variables.
The central limit theorem for m-dependent
Froc. Camb. Philos. Soc. 51, 92-95.
[7]
Doob, J. L. (1953).
Stochastic Processes.
[8]
Fraser, D. A. S. (1957).
Wiley, New York.
Non-parametric Methods in Statistics.
Wiley, New York.
[9]
Grenander, U. and Rosenblatt, M. (1957).
Stationary Time Series.
[10]
Statistical Analysis of
Wiley, New York.
Hoeffding, Wassily and Robbins, Herbert (1948).
The central limit
Duke Math. J.
theorem for dependent random variables.
11,
773-780.
[11]
Ibragimov, I. A. (1962).
processes.
Some limit theorems for stationary
Theory Frob. Applications.
I,
349-382.
J
I
I
I
I
I
I
I
~
I
I
I
I
I
I
I
~
I
I
~
I
I
I
I
I
I
I
f
I
I
I
I
I
I
I
f
I
77
[12J
Ibragimov, I. A. (1963).
A central limit theorem for a class of
dependent random variables.
[13J
Loeve, Michel (1945).
a1~atoires 1i~es.
[14J
Loeve, Michel (1960).
Theory Prob. Applications.
~,
83-89.
Etude asymptotique des sommes de variables
J. de Math. 24.
Probability Theory, (2nd ed.). Van Nostrand,
Princeton, New Jersey.
[15J
Orey, Stephen (1958).
random variables.
[16J
Rosenblatt, M. (1956).
condition.
A central limit theorem for m-dependent
Duke Math. J. 25, 543-546.
A central limit theorem and a strong mixing
Proc. Nat. Acad. Sci. 42, 43-47.