UNIVERSITY OF NORTH CAROLINA
Department of Statistics
Chapel Hill} N. C.
ON THE ELEMENTARY RENEVlAL THEOREH FOR
NON-IDENTICALLY DISTRIBUTED Vl\.HIABLES
by
vlalter L. Smith
April 1962
This research vTas supported by the Office of Naval
Research under contract No. Nonr-855(09) for research
in probability and statistics at the University of
North Carolina, Chapel Hill, N. C. Reproduction in
vThole or in part is permitted for any purpose of the
United States Government.
Institute of Statistics
Mimeo Series No. 322
ON THE EIEi:<IENTARY RENEHAL TIillOREM
FOR NON-IDENTICALLY DISTRIBUTED VARIABLES
I
by
Halter L. Smith
1.
Introduction
Let
(X )
n
be a sequence of independent, identically dj.stributed, rand.om
a<E
X < co; vrri.te Sn == Xl + X + ••• + Xn ; let Nx be
n
2
S? N·x· The
the number of partial sums which satisfy S < x; vr.l:'i te H(x) == \.J
n-
variables vrith
elementary rene'Wal theorem states that under certain conditions
( E X
n
as
} -1
x
->
H(x)jx-->
co
Ka'Wata (1956) has proved a result which is equivalent to a generalization
of the elementary renewal theorem to the case in which
~ally
distributed.
Unfor~~lately,
restrictions upon the distribution
(x }
n
are non-identi-
he found it necessary to inwose quite
functions involved.
heav~'
In this note we prove
a generalization of Kawata f s result, but under rather ioleaker restrictions.
He write
F
n
(x)
~ p(X
< x} and
G (x) S P (8 <
n
n-
n --
x
we make use of the Heaviside unit function U(x) == prO ~
the sequence of independent random variables
H
(X )
a
n
J
x
Where necessary,
J. 11e shall call
vi-sequence vT:i.th averag<:
if the follo1ung two conditions hold:
.\
( lJ
(; X
n
exists for all
1
-
n
2:
n r=l
e
<..::>
X
r
->
nand
J.l
,
as
n ->
co
~his research v~s supported by the Office of Naval Research under contract
No. Nonr-855(09) for research in probability and stati.stics at the University
of North Carolina, Chapel Hill, N. C. Reproduction in whole or in part is
permitted for any purpose of the United states Government.
2
(ii)
S /n
n
tends to
IJ.
It is assumed, of course, that
that
[X}
n
be a
n ->
in probability as
IJ.
co •
is some finite constant.
The requirement
i'l-sequence merely amounts to requiring that it satisfy a cer-
tain weak law of large numbers.
conditions upon
[F (x)}
The determination of necessary and sufficient
for the satisfaction of such a law is a classical
n
problem in probability theory and its solution is well-knovm (see, for example,
Gneo.enlw and Kolmogorov (1954), p. 135).
of s~ow grov~~; the function
He shall also make use of functions
L(x), defined for all sufficiently large
said to be a function of slow groirth if, for every
L«X~ _>
1
Lx
Let
k
(i)
a
00
(ii)
(iii)
00
We shall then say that
a ) - sequence if
(L, k, "
of constants is an
> 0, all n;
n-
as
x
->
1 -
°,
1
a L( ---)
n
a x
(1 - x)' L:
n
n==l
He shall call a
->
L a certain function of slow growth.
(an)
(1.1)
x
> 0,
a and y be certain non-negative
be a non-negative integer,
constants, and
a sequence
as
c
x, is
l-x
the limit-value of the sequence.
For example, the Taylor
expansion of
1
log
1
(---)
1 - x
generates a
(log, 1,
3
2'
1)
~sequence
culty that the relevant coefficients
a
n
(it can be shown without too much diffiare
l 2
0(n / log n) in fact).
The main result of this paper is as follows.
3
Theorem.
(Tl)
Suppose the
fL,
(a ) is an
n
L(x), the sequence of
-sequence.
h:, y, a)
The sequence of mutually independent random variables
'-lith positj.ve average
J..l.
n
1
-
L:
11
j;:;l
such that
.f
fU(x)
J?j (".)
.,.
) dx
for all
n
Then it follows that,
as
t
>
0
-00
K(x)
of a negative-valued random
variable with a finite moment of order
-
is a
A
There is a distribution function
K(x) > F (x)
ex11 )
(> 0).
There is a large positive constant A
lim inf
n -> 00
(T4)
0, some limit value
-------_.----------_._--
lv~sequence
(1.2)
y?-
0, and some function of slow growth
constants
(T3 )
hold.
For some integer k:::- 0, some constant
a~
(T2)
follow~n~ condi~ions
->
n and
(k+2) such that
x
,
00
00
(1.3 )
an Gn(t)
L:
--> a.
n=l
--------------_..
Notice tha,t, in view of
_-
the easily proved fact that
H(x);:; L:~ G (x), we
n
easily have the following.
Corollary.
If conditions
(T2)
and
(T3)
,
as
of the Theorem hold, and
i.!...co~dition.
(T41 holds with k;:; 0, then
(1. 4)
->
1
J<:
->
00
J..l.
The Corollary is, of course, a version of the elementary renewal theorem for
non-identically distributed variables.
fixed
h >0
From
(1.6)
we can infer that for any
4
t+h
1
t
J
H(x) CL'C
,
->
as
t ->
,
00
t
whence
i
t
J
[H(x+h) - H(x)}
dx
h
Il
->
,
as
t
->
00
o
or, in other words,
t
(1.6)
1
lim
t ._>
00
t
J'
00
~
x < S <x + h }dx
n::::l
0
This last limit
p [
(1,6)
is Kawata's result
-hIl
::::
n-
(1956).
We shall explain later in
what way his conditions are more restrictive than ours.
It is interesting to see,
however, that one can quickly recover the simpler result
(1. 4)
(1.6)
is equivalent to
seems an unduly complicated statement.
ancl from (1.. 5), since
H(x)
lim sup
t -> 00
For
(1.6)
from
(1.6);
thus
(1.5) )
is non-decreasing,
H(t)
-t-
<
1
Il
and
lim inf
t -> 00
H(t+h)
t--
>
1
Il
A word about the conditions of the theorem seems appropriate.
(Tl) is merely a regularity condition
on the coefficients
[a}
n
Condition
,and there··
fore is irrelevant from the point of view of the elementary J.'enewal theorem given
in the Corollary.
Condition (T2)
seems a natural generalization of the con-
ditions holding for the simpler case of identically distributed variables, and
indeed the
elementa~J
renewal theorem can be proved by means of the weak law of
large numbers in this case.
rene-\'lal function
Condition (T4) is concerned with ensuring that the
H(x)) or the more general function
L;
a G (x), shall be finite;
n n
5
we co~nent at more length on this type of condition elsewhere (Smith) 19620),
but some condition like it seems to be necessary.
are non-negative the need for
(T4) disappears.
[x }
n
Of course) when the
Condition
(T3») on the other
hand, does seem extraneous, and we suspect that a different attack on the present
problem may be able to dispose of it.
there are sequences
(T3).
Nevertheless it should be noticed that
[X } satisfying
n
(T2)
ano.
(T4) llhich eto not satisfy
Such a sequence is obtained if indepenCl.ent random variables
(x } are
n
chosen such that
-
1
---
/n'
It can be shown that this particular sequence (X}
n
1) and it is trivial to verify
vi-sequence.
(T4).;
hOvlever, (T3)
Some Lemwas.
(T3)'
In what follows we denote the familiar Stieltjes convolution
of t'i'fO distribution functions, say A(x)
and
A * A(X) by A*2(x), ano., generally,
A~(-n(X)
n ::: 1, 2,
A *
B(X), by A * B(x).
3,
00
<
Truncate the
(X
"n }
vI
.I\.
n
00
at A, thus:-,r
< A
:::
.A
"n
if
:::
A
othervlise
,,~
n
\'le
by A*(n+l)(x), for
Lemrra 1.
Proof.
---
ODes not hold for this
He do not knoll llhether in this special case the elementary renews.l
theorem actually holds in spite of the failure of
2.
is a W-sequence with average
)
o.enote
6
~In =
Let
GXln
.
lim
,
> 0
such that
> 0 > 0
j=l
n > n (0) •
o
> Fn(X)
are bounded; let
x
n
2:
j=l
00
r.
Since K(X)
all
-n1
n
-n1
= Xln
inf
6 > 0 and n (6)
o
so we can find
Zn
(1.2) states that
->
n
for all
Then
- ~'
n
c
= P [Zn-<
n
U(x - A)
This proves that
n
and all
it is clear that the
x
[~I ).
be an upper bound for these
L (x)
and
for all
[Zn)
x).
n
[~I
n
) .
Let us put
Then it follovTS that for all
nand
< Ln(X) ::: K(x + c)
is a stochastically st~a~e ~qu_e~, as defined by Smith
(1962a), whose Theorem 7 allows us to draw the following conclusion.
For every integer
p [Z
for all
n
p
there is a distribution function K (x)
p
+ Z 1 + ... + Z
1.
n+
n+p-.
n and all
x,
where
K (x)
ax
<
P x)
as
p ->
such that
< Kp (x)
o
(2.4)
Ip -
J
is finite for all
p
p, and
I p -> 0
€ < 0, therefore, we can find
Given
po(€)
a random variable Vii. th distribution function
that
e
K
Po
y
>
-€.
Moreover, it is clear that
co
such that
I
Po
<
€.
then it follows from
If
Y is
(2.4)
7
for the supremum of ? (Zl + Z2 + .•. + Z < P x) for
r 0
is a random variable vTith distribution
1, 2, ... , Po - 1. Then if Y
Vlrite J.vI(x)
r
=:
o
function J.vI(x)
i t is also clear that
Now choose and fix
r
GClmin (O,Y
0, or 1, or 2
=:
o
)1 1>:+2
<00.
)
, .. " or p -1,
'0
It follows from
,'That we have established so far that
nPo+r
p(
z.J -<
Z
j=l
Thus, in view of
P
0
x }
(2.2), if we suppose
Therefore, on putting
x == -n
€
n
>
n (0) ,
o
and observing that
X. >
J -
X~
J
, we deduce
that
np +r
o
p(
Z
j==l
Let
X
j
Yl , Y2 , ... , be a sequence of independent random variables, identi-
cally distributed, '\-lith distribution function. K (x); let Y , Y , ...
l
2
(J
Po
independent of Yo' Then G (Y j + €) > 0, for j =: 1, 2, 3,
and
~ (lmin(O, Y. + €)/ k+2 <
J
Smith (1962b) that
00
for
j
=:
0, 1,2,
Thus it follows from
be
8
00
n
P [y + L: (Y. +
o j=l J
k
n
L:
n::::l
€)
:5
<
O}
00
;
that is,
(2.6)
From
(2.5)
(2.6)
and
we may conclude that
00
<
L:
00
n::::l
lemw~
The
Ler.~a
r:::: 0,1,2, ... , P
o
- 1
in
~:::: ~c
(0 - €) so o
that n p (0 o
- €) > (n p + r)~
turn, and by putting
for all
(2.7) by letting
follows from
n and all
r.
Under the conditions of the Theorem
2.
ro .
J
(2.8)
(1 - Gn(nx)} dx
->
0,
as
x ->
00
I-"
Proof.
----
Let us iITite
nl' n
,Q
::::
t'
Iv ,:)
•
nJ
then
+00
I'n
::::
J
(U(x) - G (nx)} d.x
n
-00
Thus, by property (i) of a W-sequence, we have that
us next vITite
I' n ::::
j
(1 - Gn(nx)} d.x
00
I-"
as n -->
Let
00
+
o
n + Bn - Cn
A
The sequence
->
-->
00
::::
n
n
('
I-"
as
I'
(X)
for all
n
x
-00
say
G (nx) :::: peS < nx} -->
n
nas
Thus, by bounded convergence, B -> I-"
is a W-sequence, so that
<
1-".
n
°
9
n
->
00
But
•
f
-->
n
proved that An -> 0
~
if we can show C -> 0
n
Let us employ the notation of Lemma 1.
and all
(2.9) shows that we will have
also, and therefore
(2.3)
Then
shows tl~t for all
x
p
P
P(
1
L: X .
j=l J
<
P x
P(
p
i:
P
L: lJ., )
j==l J
+
<
K (x)
p
i-Thence
-
j==l
Therefore
X.
J
< K (x)
< px )
-
-
p
P ->
C < In' and, as i'Te know that
p.l'
vThen
Under the conditions of the theorem, as
s -> 0+ ,
00,
the leJ11.ma
is proved.
Lemma 3 .
a
L:
Proof
n
00
a
n=l
rv
ex
e -~sn
n
N
L (~)
s
~fSf
e -~s -> 1 - 0
Plainly,
L:
e -lJ.sn
as
s -> 0+
ex
---(l_e-~s )f
IJ
Therefore, as
(
L
1
1 - e -~s
1
- e
Karamata (1930, p. 45)
uniformly for
as
r
has shown that
1 -
)
) .
L(rx)/L(x) ~> 1 as
in any interval not containing
s -> 0+, since
-~s
for small
O.
s.
s -> 0+ ,
x =>
00
,
Thus it follows that
Thus the lenma is proved.
10
Le!l1J11.a 4.
s --> 0+ ,
Under the conditions of the theorem, as
00
n a
L:
n=l
Proof.
0
<
Il n e -Il
ns
Choose
~,
~
"I
n
a
< 1. Then
e- p.sn~ _ e-1J·sn
<
-0 -
11)S
00
Thus
00
Il .
n a
<
n
(1 - il)S
and so
"1+1
00
-IL(- )
L:
IlS
n a
n=l
'Sl
e -~lsn
n
~
sy
<
1
00
L:
(1-'1) )L(8) (n=l
a
e
-ll sn Tj
n
It follows therefore, from Lemma 3, that
(2.10)
sup
lim
s -> 0+
~
If \Ve let
IJ·S
"1+ 1
00
n a
~(~)-
n
S
-> l O i n (2.10) we obtain
00
(2.11)
lim
s
sup
-~>
n a
0+
Similarly, by taking
Il n e
-Ilns
11 > 1
>
<
n
and using the fact that
e
-Ilsn
- e
Tj -
1
-Ilsnil
00
- L:
n=l
a
n
e-~sn
1
)
11
'we can show
00
(2.12)
lim inf
s -> 0+
n a
The lffinma follows from
Proof of the Theorem.
:Ln accordance
,.,i
>
n
(2.11) and (2.12)
We shall
vn~ite
~
for an upper bound to the numbers
a / nIt , and we shall suppose that
n
th Le:unm. 1; we may suppose T) < I..l. •
~
>
0
is chosen
Consider, to begin with,
G (x)
n
e
ax
-nsx G (nx) dx
n
Evidently,
o -_.< Kn-<
But
G
n
(nx) .~>
n e -n~s
as
0
n
->
00,
for all
x < IJ.,
so vre can appeal to bounded
convergence and write
1
on
1
,.,here
0
n
->
0
n ->
as
00
,uniformly in
Next consider
e
-sx
[1 - G (x)
n
} ax
s > 0
12
e
-nsx [1 - G (nx)} <L'C
n
In view of Lemwa :2 and the assmnption that
we way thus conclude
~< ~
that
(3. 4)
i{here
0"
n -> 0
n ->
as
'rhus) if we "n'ite
::::
01
n (Ln - Kn )
::::
0
n
n
00
(3.5)
Given an arbitrary
n >n
- 0"n
€
n a
L;
n::::l
0
n
e -nYls
n
> 0) we can find n(d
0
then trivial.
s -> 0+
as
(0)
since the alternative case is when
'0 I <
In
€
for
~
a
}
is convergent and our theorem is
n
Thus
00
I~
n a
n::::l
e -llrls
0
n n
n
<
0
~
n::::l
n an
0
n
00
e -nYls
+
E
0
n a
~
n=l
Therefore) by Lenuna
lim sup
s -> 0+
f
n a
~
n::::l
n
(3.7)
such that
IvIoreover "le can assume that
0
->
(3.6)
)
00
a
L;
n::::l
all
s >0
uniformly in
(0)
n
e -n71
n
e - n11s
s
4)
(~s)l'+l
L(l:)
s
00
~
n::::l
n a
n
0
n
e -n~s
I
<
€
r ex
13
But
that, as
is arbitrary, and we can therefore deduce from
€
(3.7) and (3.5)
,
s -> 0+
a n (Ln - Kn ) ->
(3.8)
o
Now consider the function
00
H(X)
~
Z
n=1
Evid.ent1y
decreasing.
(3·10)
N
H(x)
a.n
G
n
(x)
U
n~)
(x -
.
is non-decreasing, since each term in the sUll1lll8.tion is non-
We also note that
00
'\I
H(x)
an U(x - n~) U (x - n~)
= Z
n=l
00
-
Z
n=l
a
n
1. u(x
- n~) - Gn(X)} U(x - nTI)
Let us denote the Laplace transform of a function A(X), say, thus: --
J
00
0
A (
s)
=
e- sx
JI~( x)
d.x
o
Then, from (3.10), ife have
00
(3.11)
+ Z a n (Ln - Kn ) ;
n=l
the term-by-tena integration being justified by monotone convergence,
From (3.11), (3.8), and Lemma 3, i t appears that
(3.12)
f'lHo(s)
->
ex
~
,as
s
-> 0+
An appeal to Doetsch (1950, p. 511) then allows the inference
14
J;1Y r (l+y)
(3.13)
But, by
-> ex,
H(t)
t ->
as
00
tYL(t)
(3.9) ,
00
(3.14)
Ai
a
L:
n:::l
G (x)
n n
H(x)
;;;:
+
ljr(x) ,
1 -
U(x - n11)
say
where
~
'!rex)
n:::l
a
n
G
n
~.
(x)
1
1
(3.16)
He have already explained that '!,.:re are assumlng the d.ivergence of L: a
to avoid triviality.
out bouno. as
(3.17)
t ->
Thus, by (1.1), '!,.,e may suppose
00
:::
lim
t->
increases with-
0
00
The theorem follows from
(3.13), (3.14), and (3.17) .
Some comments on the theorem of Kawata.
end of fl
,
(3.16) and LenmB 1,
Hence, by
•
tYL(t)
n
He have already explained at the
that the conclusion of Kawata's theorem (Kawata,
to the simpler conclusion of the corollary to our theorem.
1956) is equivalent
It is of interest to
see that Kawata's conditions are actually rather more restrictive than those
needed by this corollary.
(Kl)
There is an
s
o
In our notation, Kawata:s conditions are as follows:
> 0 such that, for all o < s -< s 0
o
,J
-00
e- SX 0. F (x)
n
<
00
,
15
(K2)
Uniformly in
n,
00
cf
lim
A-:>OO
x d Ii'n(X)
0
A
Uniformly in both
(K3)
: .:
nand
e"SX d F (x)
lim
: .:
n
A->oo
o<
s,
s < s
o
0
-00
(G Xn }
Kavle,ta also requires, like us, that all the expectations
shall
exist ano. that
( 4.1)
n
I:
1
lim
n
n-->OO
1'::.::1
He first remarl\: that
that
r...(-A) -> 0
as
Thus our condition
ex
==
€ >0
0
(K3) implies the existence of a function
A ->
00
and
,
F (-A) < r...(-A) exp (-s A}
n
-
0
(T4) is satisfied by a distribution function
decreases exponentially rapidly as
foY.' any small
>
IJ.
l'
x
<
€
for all
K(x)
such
n.
which
-> -co. A consequence of this j.s that
we can find a J~rge
( 4.2)
r...( -A)
M (€)
o
such that
,
-00
uniformly in
n.
From (4.1)
1
n
Also, from
vIe infer the existence of
n
I:
1'==1
&x
l'
>
(K2) , we can find
1-1.
-
n (€)
o
€
N (€)
o
such that
such that for all
n > n (€)
o
16
(4.4)
x d F (x)
n
uniforrruy in
n
>
n
o
n.
<
)
Thus we can infer from
(4.3)
and
(4.4)
that for all
(€)
1
-
n
L:
n
r=l
>
x d F (x)
n
IJ. - 2 €
-00
It follows easily from (4.5) that condition
(T3) is satisfied.
It remai.ns to be proved that Kawata1s sequence
In view of
(1.~.1)) therefore) all we have to prove is that
probability as
n
-->
00.
[I GXn I)
is a bounded. sequence; we shall 'Yrrite
:=
n
L:
m.
m=l
and it follows that
(4.6)
(4.2)
and (4.4)
1
0(-)
n
Thus
d F
111
n
(x + C~ Xm)
In
ll-seguence.
->
~t
in
(4.2) and (4.4))
+ max (M ) N )
o
0
2 €
He also remarll: that the uniformity of
uniformly in
S
To do this we first observe that) by
<
'l'hus
is a
(X)
n
=
1
n oC n _ C )
C
for any upper bound.
iw.ply that
17
n > C ,
]urthermore, for
1
n
n m~l
J
lxl d Fill(x
+
eX)
m
Ixl>n
<
n
L:
1
n
.J
m=l
C
n
/xl>n-C
J
Ix/>n-C
d F
ill
(x)
Thus, by (4.2), (4. 4), and (~.. 6), I'le can deduce that for all sufficiently large
n
n
L:
1
-n
J
m==l
d Fill(x + ~ Xm)
Ixj
<
Ixl>n
therefore we may conclude that
J
n
1
(4·7)
L:
n
m=l
eX)
x d Fm(X +
m
-> 0, as
Ixl<n
lastly we note that, for
2
X
n
> 2C ,
d Fm(x + ~ Xm)
/x/<n
<
J
I
+
n
r
m=l
x l<2C
<
-n
+
1
n
L:
m=l
n
L:
Ix ~>2C
(x/d Fm(x +
GXm)
n
._>
00
•
18
<
-n
+
<
-n
+
n
I:
2
n
m;;l
2€,
(4.2)
by
and
(4.4)
Thus
J
Ixl<n
From
(4.6), (4.7),
and
X
2
(4.8)
d F.
(x +
m
.0 X) \.:J
m
>
0J
as
n"- >
it follows that Kav~ta's sequence
co
( X )
n
satisfies the conditions of the classical weak law of large numbers, and
our demonstration is complete.
19
REFERENCES
G. Doetsch (1950), Handbuch der laplace-Transformation, Vol. I.
Birknauser, Basel •
Verlag
B. V. Gnedenko and A. N. Kolmogorov (1954) , 1illit distributions for sums
of independent random variables (Translated from the Russian by K. 1. Chung),
Cambridge ,Mas~s~- AddIs
ley .
on-lles
T. Kaiffita (1956), A renewal theorem, J. Math. Soc. Japan, ~, 118-126 •
w.
1. Smith (1962a), On some general renev~l theorems for nonidentically
distributed variables, Proceedings of the Fourth Berkeley Symposium on
Mathematical Statistics and Probability, Vol II, 467-514, Ber1::eley, University
of California Press.
(1962b), Some remarks on the renewal function, ~o appear.