.
,
UNIVERSITY OF NORTH CAROLINA
Department of Statistics
Chapel Hill, N. C.
Mathematical Sciences Directorate
Air Force Office of Scientific Research
Washington 25, D. c.
AFOSR Report
N~.
6,::·
-1;
CENTRAL LD4IT THEOREM FOR SUMS
OVER SETS OF IWIDOM VARIABLES
by
Friedhelm Eicker
•
University of North Carolina
March, 1961
Contract AF 49(638)-929
The convergence of weighted sums
of random variables taken in arbitrary order out of a set F is
defined. Necessary and sufficient conditions on F and the
weight coefficients for convergence to the normal law are given.
Qualified requestors may obtain copies of this report from the ASTIA Document Service Center, Arlington Hall Station, Arlington 12, Virginia.
Department of Defense contractors must be established for ASTIA services,
or have their I1 need-to-lmowl1 certified by the cognizant military agency
of their project or contract.
Institute of Statistics
Mimeograph Series No. 279
CENTRAL LIMIT THEOREM FOR SUMS
eVER SETS OF RANDOM VARIABLES
*
By Friedhelm Eicker
University of North Carolina
Introduction and summary.
The classical problem of the probabilistic
limit laws is to determine conditions for random variables in order that
their sums for increasing number of terms tend to a limit law.
In statis-
tical e.stimation the situation occurs frequently where the distributions
of theee random variables are not exactly known ~!7.
They may assumed
to be members of a certain family of distribution functions.
This case has
been considered for instance by A. Wald and by E. Farzen ~~7, and conditions for the uniform convergence of these families have been stated.
In a more general approach presented in this paper the family is replaced
by a set F of random variables.
Then conditions are given for this set and
the linear coefficients occuring in the sums in order that these sums converge for any selection of random variables out of F to a normal law.
It
seems to be much more difficult if not impossible to find similar simple
conditions for the convergence to other limit laws.
1.
Convergence to the normal law.
Let ibn 1be a sequence b ,b 2 , ••• of random variables
l
(r. v.), and let each bn be a function b (E i ,e'2' ... ) of at most countably
n
Definition:
* This research was supported b7 the United States Air Force through
the Air Force Office of Scientific Research of the Air Research and Development Command, under Contract No. AF 49(638)-929. Reproduction in
whole or in part is permitted for any purpose of the United States Government.
2
many other r. v. -s
sequence
C€ i}'
For all n the
€ ••
1
€
's are taken out of the same
Let furthermore all e 's be elements in a set F of r. v. -s.
Then {bnJ is said to converge on F (in a sense to be specified) if this
convergence holds for each possible choice of the sequence {e i \ .
Let A=
{ank~
be an infinite real matrix of generalized triangular
shape (type (T) say), i.e., each of its rows contains only a finite but
positive number of non-zero elements.
~,k2"" ~
integers
Further, there is a sequence of
co such that ani
any k let there be infinitely many a
nr
=0
for all i > kn • Finally, for
# 0 (this will be seen to be no
essential restriction).
Henceforth, each element e in any considered set F shall have zero
mean and positive, finite variance.
~olumn vector~.
written as an (infinite)
~1'~2' ..•
r.v.-s
Let any sequence
€l'~""
'Et g F, be
Consider now the sequence of
given by the vector
(1)
z
= B-1A e
where B is a diagonal matrix the square of whose n-th diagonal element is
k
2
B =
(2)
n
~
12,
27 of
p. 10
k=l
= 1 for all n. We now make use of the Lindeberg-Feiler version
Then var
n
n
.E
the Central limit theorem:
In order that the distribution functions of the sums
of independent random variables gNl,E N2' •.. ,e NN with
3
(4 )
E SNk= 0
(5)
E
for k = 1,2, .•• ,N, and
N
var
S =1
Nk
k=l
converge to the normal distribution N(O,l):
J
x
!(x)
= (27t)-1/2
2
e- Z
/2
dz
-00
and that the r.v.-s gNk be infinitesimal it is necessary aad sufficient that
Lindeberg's condition
(6)
be satisfied for every 8 > O.
If one puts Snk=
FNk(Z) is the distribution function of
B~l ank ek' ~n as in (1), conditions (3), (4), and
(5) are satisfied, and the following theorem can be concluded:
Theorem:
-
Let F be the space of all random variables with zero mean
and positive finite variance.
Let F be a non-etppty subset of
F.
Then
it is necessary and sufficient for the convergence of ~ *}to the normal
n
law N(O,l) on the set F for
n~
CO that the following conditions regarding
F and a matrix A of type (T) are simultaneously satisfied:
a
(I)
(II)
max
k=l, ... ,kn
k
n
2
nk
for n";>
~O
E a
k=l nk
There exists a bounded function g(c) for c > 0
-
*) Footnote:
00
2
~th
lim gee)
e-;>oo
=0
If the fourth moments of all €gF are uniformly bounded then their
E,a statistic similar to ~n yet not containing the ~VIS and tending to
N(O,l); see ~!7
4
such that for each r.v. €
g
F with distribution function G(€), say,
holds.
(III)
There exists a positive constant m such that for each E
var
€
eF
>m
holds.
As the theorem shows, uniform boundedness of all variances from
above is not sufficient.
Proof:
(for a special matrix A a proof was already given in ~17).
1) Sufficiency: Let a' be the n-th row vector of A. Then for any
-n
k < k by (I) and (III)
-
n
a
2
~k •
2
2
a
nk < 1
nk, 0
iii ~x a 2 --n
-n
7 -
Let ! be any sequence of "errors" €.g F.
1
condition
f
'
or n__
00
•
We see that the sufficient
(6) in the central limit theorem is satisfied because for any
5>0
Z2 dF
()
nk Z
5
2
-n
~
a
B
n
max
k=l, ... ,It
n
-2
2
-1
We have namely Bn ~n < m ,and by (II), with
G the distribution function of €k' for any k ~ k
k
n
which tends to zero.
5
because
max
~~ O.
k=l, •.• ,kn
2) The necessity of both of the conditions (II) and (III) is proved
Condition (I) is seen to be identical with the inti-
in several steps.
nitesimality of the ; terms' .
required instead of
En k in
~
n.
.
The latter could have been
(I) from the beginning. By choosing all r.v.-s €k
to be identically distributed like €o' say, we have from
max p( 10 k€
k
10 0
I > 8) = max
k
-
2
for any 8>0 that max ~k= (var €)
k
3)
-1
0
I >~ ) ~
p(le:
0
2
a nk
-
~k
0
max ~ ~ O.
k a
-n
We assume that the variances of the r.v.-s in F are not bounded
where F is any subset of
F.
We construct a sequence {€v} with O"~";=" 00
such that (8) is bounded away from zero for all n for which a
nk
n
~
O.
To this end, we write (8) in the form
k -1
n
2
E
+ Clnk
k=l
n
Jr
2
z dGk (z)
Izl>e/Clnk
n
n
2
and, assuming 8 < 1/2, we determine the last integration interval so that
k - 1
n
( E
k=l
This is always possible for a k
n n
~
2
a nk 2 2
2 2
2 O"k+ O"k ) < 'Y O"k •
a nk
n
n
n
0 and a constant 'Y < 1 and already fixed
O"k'S, k=l, ••. ,kn-l, by making O"k sufficiently large.
n
the k -th term, (6) is larger than
n
Then, taking only
6
J
2
Z
Izl>rO"k
dG
2
k
(z) > (1 - r )
n
n
Hence all the r.v.-s in F have uniformly bounded variances •
4)
..,;'
Let
€
Its distribution function may
be any element in F f F.
be denoted by Ge(z), and a uniform bound for the variances of all the E1slet
be .. M > var E.
Let f
(c) =
e
J
z2 dG (z) and consider for c > 0 the
E
Izl>c
non-increasing function
M(c) = lim sup fe(C).
E Q F
F has property (II) if and only if
lim M(c) = O.
We assume now
c~
that F does not have property (II).
We will then construct a sequence
of errors such that (6) does not tend to zero, thus coming to a contradiction.
sequence c
Obviously, F must have infinitely many elements, and for any
we can find a sequence of r.v.-s e Q F and functions
v
f (c) such that for v=l,2, •.•
v
~ 00
v
fv(C) =
J
z2 dG)Z) > C > 0
Izl>c v
where
lim M(c) > C > 0, C a suitable constant;
c~
C < 0"2
v
= var
e < M.
v
By selection of a suitable sub-sequence {e
1 from E and identifying
vk l
v
them with the components of E we show that at least for a sub-sequence
7
n
p
j
p=1,2, ••• in the set of positive integers, (6) is not true.
Because
of (9) we have
(10)
which tends necessarily to zero because of 2).
constant <
a- l .
Let for each n
K
n
Let
~
be a positive
be the largest integer such that
K
1
2
a -n
a
n
E
k=l
fK n } is non-decreasing. Evidently Kn ~
yet such that the sequence
00
because of (10).
Now determine an infinite sequence nl,n2' ••• ~
such that for
00
p=1,2, •••
K
<
P-
K
n p +l
Here to simplify the notation we have written K for k .
p
n'p
Let
now
€
Y
be the same r. v. for
k
k = Kn +l)
" .•. ,Kp , say EK '
p
p
with
Here j runs from K
to K except those j for which an' j= O.
n+l
p
p
of c -->
Y
(x),
a c
Y
<
(X)
can always be found.
k
We now get for (6) a contradiction because for each n
o<
K
P
E
k=1
p
and any 6,
8 < 1,
2
fJnk
,p
Because
p
J
K
z2 dGk(Z) > (
Izl>8/~ k
p
p 2
E ~ k)
k=K + 1 p
np
J
Izl>C K
p
2
Z dG
K
p
(z) >
8
r.
n
2
p
I:
a:n
k
~ ) > ~ (1 - C~) > const > O.
a.
k=l -n
p
Thus the necessity of (II) has been proved.
5)
It remains to prove the necessity of (III).
arbitrary subset F
CF
We assume that an
does not have property (III) and construct a
contradiction by showing that (6) remains above a positive constant for
a
Be~uence
of positive numbers n
p
~ 00.
Any such set F contains neces-
sarily infinitely many r.v.-s and especially
lim inf (J'
v-~ 00
v
se~uences
{€
v
1 with
=0
We first give the proof for a special set F of this kind, say S, with
r.v.-s whose distributions are 2-step functions S(J' (z) each having steps
at + and - (J' with weight 1/2 each.
The k-th term in (6) for a se~uence
1
{Sk C S and variance (J'k
gives the contribution
(11)
if only we have for the integration interval (J'k > 5/qkN (which is the
same inequality as (11)).
with
€
v
If we can construct an error sequence
S S such that (11) holds for some 5 > 0 and for infinitely many
nand for at leosta:ek at each n
p
{€vl
0
p
then we have a contradiction.
Clearly,
9
if (11) holds for some 8 > 0 it holds also for any positive 8 < 8 .
o
0
2
We write (11) in the form, putting 8 = ~:
o
k
(12)
(1 -
2 2
~) ~kank - ~
n 2
Ea
j=l j
j#l
Let
2
max
a
nk
k=l, ..• ,k
n
and consider for fixed k and n~ 00 the sequences skiS {(ank/A )2 } •
n
Subcase ,5a):
Let there exist a number k =
limit value greater than c < 1, say.
It
for which sk has a positive
select from sk a sUbsequence all
whose terms are greater than c' < c for n=n ,n , ••• ~ co.
l 2
n and It, (12) is greater than
Then for these
p
2
00
2
«l-~)~K c'- ~ E ~k) An
k=l
p
co
2
< 00 and ~ > 0 to baa sufficiently
j=l
small constant independent of n. Thus (11) is a lower bound for (6) and
which is positive if we choose
E
~j
p
under our assumption about sk' (III) is proved to be neaessary .
Subcase 5b):
We now assume that there is no
each sequence sk tends to zero.
It
as defined above, i. e.
It follows that there can be found
nondecreasing sequences of integers m
n
~
co (for n <
0-
n~)
such that
m 2
n a
E
nk<J..L<l.
2
k=l An
Determine now for a suitably selected sequence m
n
integers n ~ co from m
p
n p +l
= kn
~ 00
:: K (K as defined in
p
p
p
a sequence of
4.».
Let A2
n
10
2
be taken on by ank*' Clearly mn < k*n . If n'ow 0"1,0"2"" 'O"m for n=n p
n
n
are any constants we then determine inductively O"m+l""'O"k such that
n
n
for (12) the lower bound holds, putting k=k*,
n=n p ,
n
} >
This can be done
simultan~ou91y
k
O.
:for anY'D eql;1al to . an' n P by;choosing
(i) O"k*= const > 0, (ii)
En <T~ < const < 00, (iii) T} ~ flo a sufficiently
n
k=m+l
n
small constant (all constants independent of n). Hence also in this
p
subcase condition (III) is shown to be necessary for the set
6)
(III).
s.
We now turn back to an arbitrary subset F not having property
There are sequences of r.v.-s with distributions G (z) in F such
v
that for their variances lim inf
v~
<T
v
ConsidHr for each G (z) a
= 0 holds.
v
step function S (z) as defined in 5) which has the sane variance ct.
v
v
then have for any v, any 6 > 0 and any number
J
Izl~/~k
z2 dG (z) > 1/2
with 6'=
v
-
~k
J
l
Izl>6'/a
-
We
dB (z)
11k
y25 . It is sufficient to prove this for
v
r'
_?... = 0" - €.
~lk
v
The
2
v
left side is not smaller than 0" (1 - 1/2) which is also the value of the
right side.
It now follows from 5) that for any matr:lx A of type (T) we
can find a sequence of errors with d.f.-s G such
k
tha1~
for n=n p '
p~ 00
11
K
P
E
K
P
E
k=l
2
~k
k=l
p
2
C!nk
P
2
vi:
Izl>
2
z dS k (Z»28 >0.
2 8/~ k
p
This completes the proof of theorem 2.
2.
The law of large numbers for sums over spaces F.
While it turned out to be possible in the above case 1. of conver-
gence to the standard normal distribution to find simple separate conditions for the matrix A and the space F, this does not seem to be the case
for convergence to other limit distributions.
So for instance for the
convergence to the law of large numbers over a space F there are here given
only sufficient conditions under rather restrictive a p!iori conditions
placed upon F.
If one wants necessary and sufficient conditions it seems
unlikely that they can be separated into conditions for A and F only.
In
fact, the limit theorems to be used suggest this for the case of general
limit laws.
The central limit theorem thus represents
situation.
The following well known theorem
L2,
an exceptional
p.13~ is used to obtain
conditions for sums of random variables in order that they obey the law
of large numbers.
Let
n=1,2, •••
be a doublesequence of
in each row independent r.v.-s, and let Fnk(X),
2
mnk , IJ.nk , (ink be resp. the cumulative d.f., the median, ,the mean and the
variance of Snk'
Theorem: In order that the sums
.~
n
= Sn 1+
Sn 2+'" + Sn k obey the law
n
of large numbers (LLN), it is necessary and sufficient that as n-->OO
12
k
n
1)
E
k=l
Ixl
k
n
2)
k
n
3)
{IXr
E
k=l
for every
dF
nk
(x + m
nk
)
~ 0
>
IXr
E
k=l
(13)
J1
x
2
dFnk(X + mnk ) -...> 0
1
T
x dFnk (x + m ) + mnk }
nk
-> 0
>0 •
T
We now put as before Snk= anksk, ~k g F, and with Fk(X) being the
d.f. of
~k
it comes
One observes the following
Lemma:
If
the LLN holds over F and F contains at least one non-degenerate
ku
2
r.v. (i.e. its d.f. is not a unit step function) then E a k~ 0 as
k=l n
n~
Proof:
is necessary.
Because of
(z -~)
2
dFk(Z)
m
where m. = ~
K ank
= median of Sk' cond1t10n
2) in the above theorem has for
identical, non-degenerate r.v.-s Sk the lower bound
ml +
-1
an
J
-1
m1 - exn
(z
a=maxa
n
k
nk
13
As the first factor remains positive the second one must tend to zero
Lemma:
If the LLN holds over F then the medians of all elements in F
are necessarly uniformly bounded.
Proof:
If the assertion is not true then F contains infinitely many
elements, and there are sequences of r.v.-s such that the moduli of their
medians
elements
I~I---'> 00.
an*k*~
We now select any sequence of non-zero matrix
0 for k*--;> oo-from among the elements
alkl,a2k2,···,ankn'·"
and determine the Sk* inductively such that
k*-l
C
(14)
+ T +1
E
Imk*l>
I
1
I an*k* I
k*-l
which is always possible.
Here C is a positive constant and
IE
Imeans
1
the modulus of the sum (13) up to the (k*-l)-st term.
(13) putting n=n*
k*
Ik=l
E
an*k{~+
We then have for
-1
mk+Ta *k
n
J
-1
mk-Tan*k
according to (14) for any choice of
Sl""'Sk*_l'
Thus there is a con-
tradiction because (13) should tend to zero.
It seems to be difficult if not impossible to find necessary
condi~
tions to be imposed on the means and variances of the elements of F independent of the given matrix A.
This problem is considerably simplified
if not only the sequences of r.v.-s Sl'S2'S3""
are left undetermined
"
14
but also the matrix A.
The problem then would be to find properties of
a class of matrices A and of sets F which are necessary or sufficient
(or both) in order that the LLN holds uniformly over these classes.
this case is of less practical importance.
way' are similar to that of lemma 2)
Yet
Without proofs (which by the
we observe that in this situation
the following conditions are necessary:
(15.)
•
uniformly for all r.v.-s
g~F
for
a~
O.
If the matrix A is fixed then
these conditions are required only for a-values out of certain sets of
1
f
1
numbers {ank
resp. Tank
If certain weak smoothness conditions
of the F(g)IS are satisfied then the distinction between these discrete
sets and the continuum of a-values does not make any difference.
Thus
it is practically reasonable to consider (15) as necessary (but these
conditions are still far from being sufficient).
In order tofind sufficient conditions for A and F one may start
with the strong a
prior~
assumption that for all g
var g < const <
00
~'F
holds
•
Besides the above shown necessary unifor.m boundedness of the medians m
we then also have uniform boundedness for the means
~,
namely