UNIVERSITY OF NORTH CAROLINA
Department of Statistics
Chapel Hill, N. C.
ON THE ELEMENTARY RENEWAL THEOREM FOR
NON-IDENTICALLY DISTRIBUTED VARIABLES.
(Revised version)
by
Walter L. Smith
February 1963
This research was supported by the Office of Naval
Research under contract No. Nonr-855(09) for research
in probability and statistics at the University of
North Carolina, Chapel Hill, N. C. Reproduction in
whole or in part is permitted for any purpose of the
United states Government.
Institute of Statistics
Mimeo Series No. 352
ON THE EIEMENTARY RENEWAL THEOREM FOR
NON-IDENTICALLY DISTRIBUTED VARIABLES. (Revised version).l
by
Walter L. Smith
= ~ = = = = = = = = = = = = = = = = = = = = = = = =; = = = = = = = = ~ = = = = ~
1.
Introduction
Let
'I,
X
n
5
be a sequence of independent, identically distributed random
variables with 0 < EX <
n
00;
write
Sn
= Xl
+ X + ••• + X ;
2
n
let
Nx
be the
S < X; write H(X) = ENx ' The Elementary Renewal
nTheorem states that under certain conditions H(x)/x --> \ EX ~ -1 as X _>
n
number of partial sums
00
Kawata (1956) has proved a result which, as we shall see below, is equivalent to a generalization of the Elementary Renewal Theorem to the case in which
the
i.
X } are non-identically distributed.
n
Unfortunately, he found it necessary
to impose quite heavy restrictions upon the distribution
functions involved.
In this note we shall also be concerned with the proof of the Elementary Renewal
r.l i ~ r r ; b ij i: (",J
Theorem for non-identically, random variables, but under substantially weaker
v\
conditions than Kawatafs.
totic estimate to the sum
This renewal theorem, essentially, provides an asymp00
~n=l
P
S
L
Sn
~
)
x5 ; actually, we shall discuss in
this paper the asymptotic behavior of more general sums
~:=l an P ~ Sn ~ x}
,
for certain general classes of positive coefficient - sequences
,
I
1an')
•
It is well if we point out that there is another line of inquiry which could
be pursued in the present context, one with which the present investigation must
not be confused.
Instead of considering
N, one could define a random variable
x
IThiS research was supported by the Office of Naval Research under contract No.
Ncnr-855(09) for research in probability and statistics at the University of
North Carolina, Chapel Hill, N. C. Reproduction in whole or in part is permitted
for any purpose of the United states Government.
2
8 > x, and then study the asymptotic behavior
m
M as the least m for which
x
of EMx/x.
The latter problem (also for non-identically distributed" X \ ) has
n
been tackled in recent work announced by Robbins. c.nd ChOvl (1962).
However, as
might be expected, the problem we consider and the problem considered by Robbins
end Chavdiffer in important respects, in general.
Indeed, a reference to Theorem
A, which we quote below, v1ill show that one can construct a sequence of indepen-
dent and identically distributed random variables with a finite first moment, for
which
EM
x
is finite but EN
Evidently conditions which are ade-
is infinite.
x
quate for a study of M may prove inadequate for a similar study of
x
ever, when all the
then M
x
= Nx
1Xn }
are non-negative we have an exceptional case;
N.
x
How-
for
+ 1, and the distinction between the two lines of inquiry disappears.
Our main result, Theorem 1 announced below, is more general (albeit much less
simple) than the one announced by Hobbins and ChO\v, for the case of non-negative
random variables.
Let us write
Fn(x) :;;: P
x
1and
L(x), defined for all sufficiently large
L(cx)
-> 1
,
x, is said to be
c >0
a function of slow growth if, for every
(1.1)
Gn(x) :;;: P ) Sn:5 x \ ; we shall
u(x):;;: P { 0 ~ x ~ •
also need the unit function
The function
i Xn ~
as
x
_>
00
L(X)
It follows from the work of Karamata (1930), that a non-negative function of
slow growth can always be represented thus:
x
(1.2)
L(X)
=
a(x)
x
e
r a(u)
Jl
u
du
,
3
where
a(x)
is a function which tends to unity as
x
tends to infinity.
An
easy consequence of this re$presentation (1.2) is that the convergence (1.1)
takes place uniformly with respect to
c
in any interval not containing the
origin.
..
As a final preliminary we must say a word about the non-negative coefficientsequences ~"an
1 which \ve consider.
a>
exist numbers
For such a sequence we shall suppose there
0, y ~ 0, and some non-negative function of slow grow"th
L(X),
such that
(1.3 )
l:
n
x ....
a
n=l
as
n
x->
By an appeal to a Tauberian theorem due to Karamata (Hardy,
(1.3)
possible to deduce from
N
(1.4)
a
L: a
n:::l n
°.
1949,
p.
NY L(N)
r(l+y )
,
as
N
->
ro
(1.4)
follows.
Thus
~(. a n )<-
We also note, as an easy
(1.5)
a
n
and
:::
0
it is
,
Conversely, if one starts from
then an appeal to an appropriate Abelian theorem will show that
(1.3)
166)
that
although we shall omit details of' this deduction.
(1.4)
1 -
(1.3)
are equivalent assumptions on the non-negative
(ny L( n
deduc~ion
»,
as
(1.4),
from
n _>
that
ro
In connection with these sequences ~ an ~ we need to define an index:Definition.
that
The index k
of the sequence
~ an
1 is the least integer
k
such
4
From (1.5) it is clear that k.5 i,p. ly_7 + 1, where
mean the integer part of y.
k
?
y - 1, and, when
L(x)
If
Lex)
is
is to
On the other hand, we can infer from (1.4) that
is an unbounded function, we can even infer the
stronger inequality k > Y - 1.
Lemma 1.
Lp. fY_7
We have, therefore,
bounded and y
is an integer then the index k
Y-l' Y, Y + 1.
have one of three possible values:
In all other cases k
may
may
have one of only two possible values :
The main result of this paper can now be stated.
Theorem 1.
Suppose the fo11ovnng conditions hold.
1
(Tl) {Xn is a sequence of independent random variables with distribution
functions
Fn(X) ~ and finite expectations Iln ::: EX , such that
n
I
(1.6)
->
n
where
(T2)
11
For every
,
n ->
co
as
->
>0
€
r=l
.,
~ 1 - F (x) ~
For some ex > O}
grov~h
as
,
is finite and strictly positive.
n
L:
(T3)
11
dx
r
->
0
,
n
co
Y? 0, and some non-negative function of slow
L(x), the sequence of non-negative constants
fies either of the equivalent asymptotic relations (1.3) or (1.4);
00
assume further that L: 1 a
n=
Then, if we write
n
diverges.
Gn(X)::: p ~ Xl + X2 + ••• + Xn ~ x.~,
in order that
5
(1.8)
a L(x)
r(l+Y)
E a G (x)
n=l n n
,
as
x ->
co ,
it is sufficient that one of the following two sets of conditions, (T5) or (T6),
hold.
•
(T5)
The ~I. XnJ~ are non-negative, and for some unbounded non-decreasing
function
r(n), no matter how slowly it may increase,
1
lim inf
n
->
n
E
n
co
>
r=l
r
(T6)
(a)
If k
is the index of
function K(x)
i
0 •
,
an S ' then there is a distribution
of a negative-valued random variable with a finite
moment of order (k+2), such that K(x) > F (x)
-
x;
(b)
If
and every
€
(1.10 )
lim inf
n -> co
-K
€
>
for all
n and all
is the first moment of K(x), then for some v >
K
0 ,
nl10g n n
~ E u(x) - Fr(X)} dx
_co
r=l
J
n
1
> 2j(k+l) v
co
In condition (T3) above we have required that En=l an
€
•
shall diverge; it
will be appreciated that this assumption is made only to avoid triViality.
co
sequence of the divergence of En=l an
is that the index k
~
A con-
-1; therefore the
distribution function K(x) which appears in condition (T6)(a) will always have
a finite
mean; this justifies the introduction of
The special case of Theorem 1 in which the ~ Xn
given the following form.
-K
in condition
(T6)(b).
1 are non-negative can be
Theorem 2.
such that
1
Suppose that (i) ~ X is a sequence of' non-negative random variables
n
(Tl), (T2), and (T5) of Theorem 1 hold; (ii) A(n) is a non-decreasing
a > 0, Y ~ 0, and a non-negative function of
function of
n for which constants
slow growth
L(n), can be found such that
(loll)
~
A(n)
a nY L(n),
as
n
->
00
Then it follows that
~
(1.12)
xY
a (-)
=n
We note that by letting A(n)
as
L(X),
IJ.
x
_>
00
•
in Theorem 2 we obtain a version of the
Elementary Renewal Theorem for independent, non-identically distributed, nonnegative random variables.
Alternatively, by taking a
=1
n
for all n
in
Theorem 1, we obtain the following version for the case when the random variables
may assume negative values.
Theorem 3.
If conditions
(T6) hold for k
= 0,
(Tl) and (T2) of Theorem 1 hold; and if both parts of
then
->
(1.13 )
1
IJ.
,
as
_>
x
00
,
is the expected number of partial sums
S
< x •
n-
From (1.13) we can infer that, for any fixed h > 0,
t+h
(1.14)
1
t
J
1t
H(X)
ax ->
t
Therefore
-IJ.h
,
as
t ->
0:>
•
t
1
t
a
H(x+h)
H(X)
~ ax
->
-hIJ. ,
as
t _>
00
,.
7
or, in other words,
(1.15 )
i
lim
t->oo
t
1;
pix < 8 n
o n=l
:5 x
+ h
1
h
=
dx
I-l
This last limit (1.15) is the form taken by Kawatafs result (1956); we see
that it is implied by the simpler statement (1.13).
On the other hand it is
not difficult to deduce (1.13) from (1.15), so that (1.15) seems an unduly complicated form for the result.
For (1.15) is equivalent to (1.14); and from
(1.14) and the monotone character of H(X) we can infer that lim sup
R(t + h)/t ~ I-l-
and lim inf
l
R(t)/t
:5
I-l
-1
•
We close this introduction with some remarks about the conditions of Theorem
1.
The easiest proofs of the Elementary
~ally
Renev~l
Theorem for the case of identi-
distributed random variables make use of the weak law of large numbers to
show toots
n
In
->
I-l
in probability, as
n ->
CD.
The present investigation will
also depend on establishing such weak convergence of
8
n
In ,
and conditions (Tl)
and (T2), aided by (T6)(a) when the random variables can take negative values, are
concerned with this task.
To understand the raison dt~tre of the alternative conditions (T5) or (T6)(b)
it is
necessari~y
to inquire a little into our mode of proof.
noted, begin by establishing that
8
n
In
->
not deviate too much from its "average value"
I-l
in probability.
nil
easy consequence of this weak law of large numbers.
deviation of
S
n
from nl-l
is presented by sequences
We shall, as just
If only
S would
n
our theorems would then be an
Unfortunately, considerable
is possible; the main obstacle we have to overcome
,
l
~
8
n
~ which tend to adhere, for long stretches of
n,
to the same value (or even gradually decrease over such stretches), and then indulge in a rare, but very large, increase in value.
This kind of pathological
8
behavior is exemnlified by a sequence St Xn)t for which P
'J;'
arbitrarily near unity as
n
increases.
Conditions
i Xn
becomes
(T5) or (T6)(b) are con-
cerned with controlling this kind of aWkwardness.
Condition (T6)(a), which is unnecessary when the random variables are nonnegative, is introduced to ensure the finiteness of the quantities with which we
deal; it will be understood better in relation to the following theorem, which we
shall use later in this paper but prove elsewhere (Smith (1963)).
If ) X 1. is a. sequence of independent and identically distributed ran-
Theorem A.
- n \
dom variables vnth
° < E Xh
~
00,
and if k
is a non-negative integer, then a
necessary and sufficient condition for the convergence of the series
00
(1.16)
L.
n=l
is that
E
k
'
n P Xl + X2 + ••• + Xn -< x )I,
t
~ !min(O,x )!k+2 \ <
00.
,
Furthermore, when this condition is met,
n
--------_-.:.:--_------------------::----_._---j
if X is any other random variable, independent of the ~ Xn ~, such that
o
(
k+2 '
E i Imin(O,X ) I
~ < 00 , then
o
(1.17)
L
,
n=l
is also convergent.
Thus we see that (T6)(a) must be satisfied when the ~ X ') are identically
n
distributed, and it therefore seems reasonable to require the satisfaction of
some condition like (T6)(a) even in the general case, if we are to ensure the
finiteness of the left-hand side of (1.8).
9
2.
Some preliminary lemmas.
We begin by sho1nng that the conditions of Theorem
ensure that
S
n
In
->
n ->
in probability as
I-l
(Xl.
1
are sufficient to
This could be done by
appeal to classical results; however, it is not difficult to proceed from first
principles, and our proof conveniently introduces an argument of a sort which we
shall use several times in the course of this paper.
Lemma 2.
If conditions (Tl) and (T2) hold, then a sufficient condition for
ensuring that
S
n
In --->
in probability as
I-l
S are
--
(
random variables J~ X
n
n --->
(Xl
is either: (a) the
non-negative; or (b) condition (T6)(a) holds with
k :::: -1.
Proof.
Consider first the case of a sequence of independent, non-negative, ran-
dom variables
i Xn'~ whose
distribution functions satisfy (T2) and whose mean
values satisfy the condition that
III + 1-l2 + ••• + I-l
n :::: O(n), a condition less
restrictive than (1.6). Write mn :::: n-l(l-ll + 112 + ••• + I-l n } and let ~ be a
small strictly positive number. Then, in virtue of a familiar inequality, for
every fixed
t > 0 we have
< e
n(m -~}
n
\ -ts )
E
(e
n~
Furthermore, if we make use of the Laplace-Stieltjes transforms
1(Xl
Qi j
(t) ::::
e-tx dF j (x)
,
we may rewrite this last inequality thus:
P
~l
S
n
< n(ron _ ~) 1S <
~
e
n(m ..
n
~)
n
rr
j=l
r
But, as may easily be verified,
<P.(t) - 1
Qi.(t} < e J
J
-
,.
Qi.(t) •
J
10
and so we have that
W (t)
,
< e n
p \ s < n(m - 'n) 1
1 n n
)
(2.1)
where, after some integrations by parts, Ife see that
co
W (t) :::: n(m
n
n
Choose a small
€
- 'n)t - t
~
j==l
1 t
e-
tx
1 - F.(X)
J
0
0n(€) ---> 0
If we observe that
(2.4)
w(
1
as
n __> co.
<
)
From (2.2) we then deduce that
and put
t:::: (n .j€fl
e -j€
mn - 11
I
rm
-
n
m is positive and bounded; thus
n
n, by choosing
€
n, by choice of
p )"
l
s <
n-
€.
0 (€) n
n
- 'n)
l
)
< exp
-
1(_
J
n
Hence the expression in
< - 21 'n , for all sufficiently large
Therefore, from (2.1), we see that
n(m
'n (
m (l_e-J€) can be made arbitrarily
sufficiently small.
braces in the last inequality can be made
in (2.3), we find that
-o(€)7
n
-
+ e -j€
small for all
•
;
0 (0) == m
n
n
n nj€
Recall that
dx
> 0 and set
-n1
by (T2),
1
11
2j€"
,
11
for all sufficiently large
n.
Since
can be chosen arbitrarily small we
€
are led to the conclusion that
p ~S
< n(m
l n -
1\)
~ ->
as
0
)
n
->
co
Tl > 0 •
for every
Let
n
> 0 be arbitrarily small.
p
Since
ES
n
=n
m
n
we have that
)
,
~ S > n(m + p) (
L nn
.J
> n(mn + p) p
n-
n m
+ n(m
n
M Tl)p~n(m M 1\) < S < n(m + p)
(n
n
n
I),>
,
and from this inequality it follows that
(2.6)
If we choose
Tl
arbitrarily small, and observe once again that
we can infer from
(2.5)
(2.7)
~ Sn
p
for every
p
P
is bounded,
that
::: n (mn + p)
~ ->
as
0
n ->
co
Tl > 0
S
iI ~
f
(2.6)
n
> O. The coupling of (2.5) and (2.7) produce the desired conclusion
that for every
(2.8)
and
m
)
M
mn
I:::
1\
~ ->
0
as
n ->
co
We remark that (2.8) proves the lerruna for the case when the
nonMnegative (see the result due to
Gnedenko and KoL.-nogorov (1954».
~ Xn ~ are
A. A. Bobrov quoted on page 141 of
We turn now to the general case, and begin by
12
defining
=
=
0
,
if
xn-> 0 ,
,
otherwise
,
-
-
and X = X+ - X • Thus both x+ and X are non-negative random variables
n
n
n
n
n
+
+
+
+
and we shall write 11 =EX, Il
m+ ::::: n-l( III
+ Il~ ),
+ 112 +
n
n n = E Xn
n
m := n-1 ( III- + IJ. +
+ Il~) , and, of course, m := n-1 ( 111+ 11 2+
+ 11n )
n
2
n
-,
- ...
Since
this mean
(T6)(a) holds for k = -1, the mean of K(x) is finite; if we call
-
hence m- <
n -
...
...
K
K.
< ~ for all n, and
nMoreover, the fact that K(x) > F (x), for all n and all x,
then it also follows from (T6)(a) that
-
ensures that (1.7) will hold when the
11-
n
j Fn(x) l in that condition are replaced by
i X~ ~.
the corresponding distribution functions of the variables
We can now
infer from the result established for non-negative random variables at the start
X~ + •••
of this proof that, if we write S-:= X- +
n
l
>
for every
n>
T]
lS
->
~
0
n
as
+ X- ,
n
_>
,
00
0 .
Let us turn now to a consideration of S~
= X~
+ X+ •
n
+ X~ +
We note
+
But 'Vle have just
Il+
n - Iln : : : Iln ' we have mn = mn + mn
is bounded by K ; and we may now suppose m -> 11 as n ->
first that, since
shown that
n
Thus m+ is a bounded function of the integer
n
to our preliminary result to deduce that
by (1.6).
+
(2.10)
for every
- mn
n>
O.
>
-> 0 as
n
_>
n, and we can appeal
co ,
The lemma follows from (2.9) and (2.10) •
co
,
13
Lemma 3.
Under the conditions of Lemma 2 ,
co
(2.11)
Proof.
Gn(nx.)
(
S dx.
-> 0 as x ->
00
It is easy to verify that
+<:Xl
mn
11
J~
=
u(x) - Gn(nx)
~
1
,
dx
00
=
1 - Gn(nx)
\-l
~
dx.
a
-K:
-J
Gn(nx) dx.
-It
(2.12)
G (nx) ---> 0
By Lemma 2,
and
C
n
->
0
n
as
Thus we can see frcm
An
A + B - C - D
n
n
n
n
=
-->
n
->
00,
as
1
-
,
dx.
Gn(nx)
_00
,
n ->
say
00,
x < \-l.
for all
by bounded convergence.
But
m
n
Thus
->
\-l'
B ->\-l
n
by (1.6).
(2.12) that in order to establish the required result,
0, we need only prove that
C
n
->
)
O.
In the case ·when thei X
n
~
are
non-negative there is, of course, no need for further argument.
Write
itself.
Kn (x) for the familiar
Then plainly, since
K (x) > G (x)
n
-
n
for all
n
n-fold Stieltjes convolution of K(X) vlith
K (x) .> Fn (x)
and all
x.
for all
n
and all
Thus it will be enough if we can prove
that
(2.13 )
x,
as
n
_>
co
14
However,
,
80
for all.
n,
that (2.1.3) vlOuld follOvl if we proved that
(2.14)
as
n _>
<Xl
To prove (2.14) we need only remark that, by the weak law of large numbers for
K (nx) ---> 1 as n
> 00 for
n
follows from the theorem on bounded convergence.
identically distributed random variables,
all
x>
~
K
.
)
thus (2.14)
This proves the lemma.
Lemma 4.
If the non-negative constants
ro
E
a
n=l
Proof.
Plainly,
e -Il
s
n
e -Ilsn
l""
1an ~
satisfy (1..3) then, as
L(
s -> 0+.
l..l.
s
1 .. e -Il
s
But, as i'1e have remarked in our introduction,
unifonnly for
r
Therefore, as
s -> 0+ ,
1
)
-liS
1 - e l""
L ( __1__ )
y
0+ ,
~
-> 1 .. 0 as
y
s ->
in any interval not containing O.
•
L(rx)/L(x) _> 1 as x _>
Thus it transpires that
co
15
1
L(---)
-lls
1 - e
as
S -
.
1 - e -lls "" lls
> 0-1-, sJ.nce
for small
Thus the lemma is proved.
s.
5. Under the same conditions as Lemma 4, as s ---> 0+ ,
Lemma
CXl
Proof.
Y ex
Choose Tj , 0 < Tj < 1.
Then
<
(1 - Tj)s
co
Thus
E a
co
n a
II •
n=l
n
n
(1 - l1)s
and so
y+1
co
CXl
II S
L
(l-Tj )L(-)
s
It follows therefore, from Lemma
(2.15)
lim sup
s -> 0+
y+1
II s
L(2:)
s
If we let 11 -> 1- 0
(2.16)
lim sup
s -> 0+
E a
1
(2:)
s
4,
n==l
that
CXl
Ena
n
n",l
e -llsn
<
ex
llY
in (2.15) we obtain
y+l
II s
L(1:")
s
n
CXl
-lJ.sn
E na n e
n:::l
<
ay
lJ.y
"
~
Tl,-Y - 1
1 - 11
(
16
Similarly, by taking
~
> 1 and using the fact that
~ - 1
we can show
j..t s
lim inf
s -> 0+
(2.17)
y+l
co
L: n a
L(l:)
s
n=l
n
e
-j..tsn
>
ay
j..ty
The lemma follows from (2.16) and (2.17) •
3. Proof of Theorem 1.
We shall write f3
for an upper bound to the numbers
is the index of the non-negative coefficient sequence
n> 0
write
t an/n
k ~
,
1 an \
;
), 'where k
we shall also
for an arbitrary small number; it is supposed that
~
<
j..t
•
Consider, to begin inth,
(3.1)
Kn
=
l
e -sx G (x) dx ,
n
nT)
=
n
J
e
-nsx
Gn (nx) dx •
11
EVidently,
But
G (nx) -> 0 as
n
n ->
<Xl
,
for all
x < j..t, by Lemma 2.
to bounded convergence and write
where
0t
n
-> 0 as
n ->
co ,
uniformly in
s > 0 •
Hence we can appeal
17
Next consider
CfJ
L
n
=
J
e
-sx ,-
i1
- Gn (x)
~
dx
nJl
co
n
=
{
e
)
-
-nsx
L1
'n
In view of T..Jemma 3 and the assumption that
(3.4 )
,,;{here
Ln
0 II -> 0
=n
-nT)s
n ->
as
n
e
Thus, if we write
CfJ
on = 5'n
~ II
un
,
=
a (L - K )
n=l n n
n
n>
since
'Vle
n
vie
may thus conclude that
,
- 0"n
e- nlls
0
n
n
> 0 , we can find n (€) such that 10n l <__ ~_ for
o
1
>
as 8 -> 0+ ,
Moreover we can assume that s-Y L(8- )
Given an arbitrary
all
Jl
s > O.
uniformly in
n a
<
,
co
~
dx
- Gn(nx) )
€
CfJ
o
suppose
~
a
to be divergent.
n
Thus
(3.6)
n
<
o
~
n=l
n a
n
+
€
~
n=l
e
n a
-nT)s
n
Therefore, by Lemma 5 ,
lim sup
s -> 0+
But
as
€
<
€
Y a:
is arbitrary, and we can therefore deduce from (3.7) and (3.5) that,
s -> 0+ ,
18
(f.l S )Y+l
1
L(--s )
Now consider the function
co
(3.9)
~
:::
Evidently
non-decreasing.
Hn(x)
Gn(x) u(x - n~)
an
n:::l
is non-decreasing, since each term in the summation is
We also note that
co
(3.10)
Hn(X)
U(X - nll)
Let us denote the Laplace transform of a function A(x), say, thus:-
1
co
=
ax
e-sx A(X)
o
Then, from
(3.11)
(3.10), we have
~(s)
;:::
1
-
s
co
~
n:::l
a
n
-nf.ls
e
co
+
the term-by-term integration being justified by monotone convergence.
From
(3.11), (3.8), and Lemma 4, it now appears that
(3.12)
An appeal to Doetsch
~(s)
- > a: f.l ,
as
s - > 0+
(1950, p. 511) then allows the inference
~(t)
- > a: as
t ->
co
19
But, by (3. 9) ,
co
(3.14 )
L:
n:=l
a
G (x)
n
n
:=
IL. (x)
+ !n(x)
--II
,say,
v1here
co
~l1(x)
:=
L:
n::::l
a
n
i
G (x)
n
1 - U(x - nil)}
If we 1'iere to prove that
lim
Ti ->
°
:=
lim sup
x
->
°
co
then the theorem would follow from (3.14) and (3.13).
The proof of (3.16) under
fairly weak hypotheses is qUite involved, however, and we therefore present it
in the following two separate sections.
4
Completion of proof under
If {X ~
n
when
Xn < 0,
(T6).
X-n
:=
° when
distribution function K(X)
Xn> 0.
1'1e
have
°<
Let us also write
\!
which, as has already been explained in
n-< '"
Therefore, if we write
-K.
for all
n.
X+:::: X + X- , S+ := X+ + X+ + .•• + x+,
n
n
n
n
n
l
2
S~ :::: X~ +
x; +
Lemma 6.
When (T6) holds we can find 11 > 0, 0 > 0, such that
n
where
k
and
••• + Xn
1
° ( "-'k:-+-::o-+"""l-)
(4.1)
n
n
~fuen (T6) holds we can introduce the
be assumed to have a finite first moment
\in := E X~ ,
X- : : -X
is the renewal sequence under study let us write
is the index of the coefficient sequence
~. an
1.
,
§ 1,
may
20
Proof.
T1 > 0, it is plain that
P
S s+ < n("\)
2 nn
+
11) 1 < e
)
(4.2)
= e
n(~ + "\)t
-t s+
nEe
n
n(l1 + vn )t
n
IT
j=l
vThere
e -tx dF ()
x
==
j
0+
If we now use the familiar inequality already employed in
deduce from
(4.2)
P
§2
we can
that
S S+ <
I
n
n
-
(v + 11)
n
W (t)
15
< e n
"There, after some integrations by parts, we now have
1
00
~
(4.4)
W (t) == n('ll +"\) )t - t
n
n"l
J==
e-tx
0
Sl.. 1
- F. (x)
J
1
S
dx.
In this section we are assuming (1.10) to hold, and so, given any fixed
€ > 0, we have for all sufficiently large
n
L;
j=l
where
v >
n
that
€rylOg n
JJ U(x)
- F. (x)
-co
II:
J
j
>
dx
is independent of
2nJ(k+l) v
€
€ •
We can rewrite this last inequality as follows.
(4.5)
~
j==l
h
lOg n
ll-F.(x)1 dx
0
J
> n
'J
n
+ 2n J(k+l)
'J
€
21
(4.4), it is plain that
But, from
If, in the latter inequality, we make the substitutions
t
t
=
=
n
€
and if we note incidentally that
for all
i-. log n
~
n
>
-v
n
n, and 1 - e -x < x
for all
x > 0, then we find that
W (t )
n
(4.6)
n
<
2:..l.
log n
By taking
+ (k + 1) - )", e-i-.(k+l)
€
€
sufficiently small we can make
fi.
e-i-.
arbitrarily near unity
and thus make
J""v-i
k+l) - .~ e -i-. (k+l)
<
(
for some small
All <
€
0
(4.7)
0 >
° (recall that
and it follows from
v > K).
Next choose Il
so small that
(4.6) that
Wn (tn ) < - (k + 0 + 1) log
Lemma
< - (k + 20 + 1)
n.
6 follows from (4.7) and (4.3) .
In what follows we denote the familiar Stieltjes convobltion of two distribution functions, say A(x)
A*2( x), and, generally,
and
B(x), by A * B(x) ..
A * A*n( x)
by A*(n+1)( x), for
We denote
n
= 1,
A*A(x) by
2, 3, ...
22
Lemma 7.
When (T6) holds
is convergent for every
Proof.
Z
n
v <
n-
Define
recall that
This proves that
0 > 0 .
0 + v
=
}
K
for all
2
n
and write L (x) = P ) Z < x
n
n -
- Xn
n, then we easily see that
is a stochastically stable sequence as defined by Smith
Z
n
(1962 ), whose Theorem 7 allows us to draw the following conclusion.
For every integer
there is a distribution function
~
,"
(4.8)
P :~- Z n + Zn+ 1+'" + Z n+p- 1 < px .),
for all
n and all
<
K (x)
such that
p
K ( x)
p
x, where
o
Ip =
-
1
K (x) dx
_CXl
is finite for all
P
p, and I
Thus we can find
p
-> 0 as
po(~ 0) such tha t
variable with distribution function
EY> - ~ O.
Po
Moreover, it is clear that
can certainly suppose
Ylrite
for
r
= 1,
K
M(x)
->
en
•
< } [,
If Y is a random
Po
2
then it follows from (4.9) that
I
E
~ Imin(O,Y) Ik+2i
<
since
en,
"le
K (x) < K*P(PX) •
p
-
for the supremum of
2) ..• , P - 1.
o
function M(X)
P
Then if
Y
0
it is also apparent that
is a random variable with distribution
E: !min(o'Y )lk+2 ~
o
<
en
•
2)
Now choose and fix
r
~
0, or
1, or
2, .•• , or
P
o
It follovlS from
- 1.
what we have established so far that
(4.10)
M
*
*n
K (x)
Po
Let
Y , Y , ..• be a sequence of independent random variables, identi2
l
cally distributed, with distribution function K (x); let Y , Y ,
be
2
l
Po
1
and
independent of Y. Then E(Y. + -2 0) > 0, for j ~ 1, 2, ),
o
J
( Imin(o'Y +21 0 )1 k+2!j < co for j = 0,1,2,
Thus it follows from
E
i
j
Theorem A, quoted in ~ 1, that
n
1
r.
j~l
(Y j +
2
<
0)
co
that is,
co
r. nk M * K*n
j~l
Po
(4.11)
1
(
<
- 2 no)
From (4.10) and (4.11) we conclude that
r.
n:::l
k
1
n p)( (n Po + r)( -2 0 + \} np +r ) --< S·np +r a
~2
0
l
npoo j
<
co
,
whence,
(4.12)
Sn·p +r
o
> (np 0 + r)(o + \) np +r )
0
-
The lemma follows from (4.12) by letting
Lemma 8.
~fuen
~
(T6) holds we can find
<
co
•
r
~
<
0, 1, 2, ... , Po • 1 in turn.
> 0 such that
24
Proof.
We observe that
:::
P
~l
S+ <
n-
nll
+
S~ ~
S- < n( 0 +
<
0 > O.
for every
n
vn ) .~)
Hence
The lemma now follows from Lemmas 6 and
~
7 if we make
+ 0
sufficiently
small.
The proof of (3.16) is now straightforward.
<
We see from (3.15) that
L.
n:::l
l
'(
(
for all
x.
Therefore, since k
follows from Lemma 8 that
x --->
as
5
0:>,
~ll(x)
is the index
of the
is bounded above.
Since
a ') sequence, it
n
XYL(x) _>
0:>
the truth of (3.16) is established.
Completion of proof under
(T5).
We begin by showing that we can assume, with no loss of generality, certain
convenient properties for the function
!(n)
which appears in the condition (T5).
All that actually matters are the values taken by
but we may clearly assume
x > 1.
~(x)
~(n)
for integer values of
n;
to be a continuous function defined for all
More to the point, we observe that if (1.9) holds for the function
then it also holds for any function
~l(x) ~
~(x).
~(x)
In this connection we prove
25
the following:
~emma
9.
If, for
function of
x
~(x)
x? 1,
is an unbounded, continuous, and non-decreasing
on the same domain, with the additional property that
for all sufficiently large
Proof.
For
x > 1
=
x, and tends to zero
~1 (x)
To begin With, since
+
inf
:5 x
15 y
1~(y) - log
~(y) - log
at which this lower bound is attained.
=
Evidently
00
log
y(x)
x
+
fact that log
x
is non-increasing
00
•
(y)\ .
Y is a continuous function in
y(x):5 x
if
y(x)
log
y(x) .
x.
is unbounded shows, in
tends to a finite limit as
is unbounded shows, also in
Incidentally, it is an easy deduction from
Next choose an arbitrary value of
y-value
Then
f(y(x»
~(x)
~l, ~7
for the largest
is a non-decreasing function of
then the fact that
is also unbounded;
~(x), defined
has all the requisite properties.
it attains its lower bound; we shall write
(l(X)
~l(x)/x
x --->
as
5
define
log x
He she,ll show that
x ->
~l(x)
then we can find another such function
(5.2), that
(5.1)
x, Xl
that
say.
If
y(x) --->
(5.2), that
x --->
~l(x)
~l(x):5
00
,
00
as
~l(x)
then the
is unbounded.
f(x) .
Our argument will be given
in two cases.
y(x ) < Xl' The continuity of ~(y) - log y ensures the existence
l
of seme open interval G, containing Xl' within which y(x) = y(x l ). Hence,
Case:
in
G,
26
from this equation it is clear that
~l(x)/x
differentiation,
Case:
Since
i.e,
xl +
~l(x)
Y(xl )
h'::
= xl'
~l(x)
is increasing in G and, by simple
is decreasing in G,
In this case, for any
h > 0,
xl'
But, from
(5.1)
xl; thus
~l(Xl + h) .:: ~l(xl)'
y(x l + h), it follows from (5.3) that
is increasing at
y(x l + h) ~
J
from which we can infer that
X~
.L
+ h
The right hand side of
~l(x)
< -
h
+
X~(X~ + h)
xl(x1 + h)
X~
.L
(5.4)
h ~l(xl)
.L-
.L
is negative for all sufficiently large
is unbounded and non-decreasing,
~l(x)/x
Thus
xl' because
is non-decreasing at
xl'
Finally J we remark that
~l(x)
~
log X + ~(l)
~l(x)/x
from which it is obvious that
-> 0 as X _>
We turn now to the proof of (3.16) under
(T5); for this argmnent we may, by
the irr@ediately preceding discussion, assume that
decreasing,
r * I~(r* )
5
r*
We can then define
x.
We also vrrite
greatest integer such that
s*
s*
= r * (x)
x/f(x)
is unbounded and non-
as the greatest integer such that
= s *(x) = r *«1
If(8 * ) ~
<Xl.
(1 + e) x.
+ e)x); thus
s *(x)
is the
27
Choose a large positive
all of which
(i)
2
Cx
'Vle
x
C and consider the following three cases, in
is assumed to be large.
< n < r * (x).
By considerations similar to those in the proof of Lemma
have
W (t)
< e n
G (x)
n
,
where
t:::
1
~
W (t) ::: tx - t
n
If we substitute
co
j:::l
-tu 5
()
e l l - F. u)i duo
J
0
1/(2x)
t~uncate
and
,)
x
the integrals at
in
(5.6)
then we find
.~
2x
Since
(5.8)
1 (j)
E
j=l
1f
o
1 - F. ( u)
J
for all sufficiently large
that
j/f(j)
x/f(x)
~
x
n.
for
~
large) .
and so, from
1 - F .(u)
J
~
du
n~
But
j:::
n 0
r * so that (since
I, 2, ••• , n.
(Actu~lly
is non-decreasing for all sufficiently large
Hence we can infer from
x
n
(5.9)
0
>
du
quate for our purpose if we note that
n
1f
(T5) is assumed to hold we can, by (1.9), find & > 0 such that
n
decreasing)
J:::l
x
j~J
~l-F.(U)~.
J
)
(5.7),
Ttl
x/f(x) ->
(5.8)
that
> n 6
du
that
(1/(2x)) <
n
1
2
-
e -1/2 n 6
2x
co
as
x/f(x)
is non-
we have only shown
x;
but this is ade-
x ->
co
and assume
28
(5.5) we deduce that
If we use this last inequality in
(5.10)
r *(x) < n ~ s*(x).
(ii)
been using on G (nx).
n
e- l / 2 n 0
.~
2x
)
,
\
n
<
r
*
For this case we modify the kind of inequality we have
Plainly
p.~'-
<
< x for all
X.
n
\\
:::
j
J -
~)
F .(x)
J
j=l
(
n
1 -j~l ["1 - Fj(X) _7
< exp
By forming the geometric mean of this last inequality and
(
$
(5.5) we obtain
a new inequality:
(5.11)
G (x)
n
<
R (t)
,
e n
,,,here
l'
00
Rn (t) : : '21 tx - -1 t
2
n
L:
j:::l
du
0
1
- 2
If ue truncate the integrals in
(5.12) at x and substitute t::: l/X
then we can infer
(5.13 )
1
Rn (l/x) < "2
-
e -1
2x
1
x
j~ll i
"
.~ du
1 - F .(u)
J
n
i
I:
".' 1 - F.(x)
"2 J=r
. *+1
2
J
)
-.
~
29
At this point it is convenient to write
j!!(j)
;: J
~
.~
1 - F j ( u)
du
;
o
then
(5.9) can be rewritten
> no,
(Al + A2 + ..•• + An)
n sufficiently large.
We shall also write
x
=
ex}x)
Therefore, for
j
~
~
1 - F j ( u)
~
du
> r * (and, consequently,
;:
j/~(j)
> x)
'Vle have
A. - ex.(x)
J
J
A consequence of the last equation is that
(
j
-
X)(l - F.(x))
J
f(j)
s *(x), then
However, if
> A. - ex.(x)
-
j / ~( j )
J
J
< (1 + e)x , and from this inequality
it follOi'lS that
( _J_'_
_
x)
<
ex
f(j)
Thus we have, from
(5.15), that
1 - F. (x)
J
,
>
ex
r* < j <
/'"
\)
Using the last inequality we can infer from
s*
4~\
'.J }
(,..~~~
that
r
30
*
;.:::
:=
1
1
- 2ex
J
*
r *+1
(A j - a.(x))
J
n
Aj
L.
1
n 6
--2ex
-
'2
a. (x)
r +1
~.
2ex
1
<
- - 2ex
n
1
--
'2
1
~'j
1
n
1:
by (5.8) .
Therefore, from (5.11), we discover that
(
G (x) < exp
(5.16)
n
~
(
1
-2
-
n 6
2ex
1
,
I
~
r * < n < s*
Fran (5.10) and (5.16) we can conclude that
s*(x)
a
1:
n:=Cx
n
G (x)
n
< e1/2
ex1')\
l:'
i-
a
n
n=Cx
1:
n 6
2ex
~J
We quote here a theorem which can be immediately deduced from some results
of Karamata (Hardy (1949), pp. 166-169, especially Theorems 110, Ill).
Theorem B.
r(y) :=
J~,j)
(5.18)
where y ~
Suppose that
e- yt d aCt)
a(t)
is a non-decreasing function of
° and
as
L(X)
y
=> 0+
is a function of slow grovnh.
tinuous function of bounded variation in
and that
y > 0, and that
is convergent for
r(y)
t
(0,1), as
Then, if
y - > 0+
g(x)
1'1e
have
is a con-
31
(a)
in case y > 0
co
1
1
...
r(y)
o
(b)
in case y
=
0:
1
00
1
e-
yt
g(e-
yt
~
) d o:(t)
gel)
o
Let us put, in this theorem,
0:( t)
=
a
L.
<
n
t
n
Then Lellill'a 4 shows that a relation like
(5.18) holds
Define
g(x)
=
=
2
=
Clearly this
vie
g(x)
1
o<
- -ax
a
0
2a<x< 1.
<
a
<x <
~a
satisfies the conditions of Theorem B.
can deduce that
lim
y->
x
sup
t
1
0+
a
1
n
n > - log- y
a
co
<
~y r(y)
J'
1
2a.
log-
e
-ny
,
Thus" when y > 0
In this last result, substitute
being given any prescribed
Y = o/(2ex)
1
and
log -a = Co/(2e).
On
> 0 we can choose C sufficiently large for us
€
to deduce, via (5.16), that
lim sup
x -> co
=0
When y
s *(x)
1
Z
n = Cx
<
a n Gn (x)
€
a similar result to (5.19) can also be proved by appeal to
Theorem B.
s * (x) < n
n/r(n) >
x •
For this range of values for
R
If we define
and
n
A
n
we always have
as for case (ii) then by arguments
n
similar to the ones employed in that case we find
R (x-I) < 1
n
"2
-
s*
1
2ex
Z
1
-
Aj
n
1
2ex
a.(x)
Z
J
s *+1
')
-
n
1
Z
"2
*
\
~
( roJ
j
s +1
However, when
j > s*
(5.20)
(
~
Aj - aj(x)
j/r(j) >
j
x)
10)
-
x
(1 + e)x and one can infer that
> e x
and deduce therefore that
(5.21)
R (x-I) < 1
n
'2
-
s*
1
2ex
Z
1
/\,j
Write
Tn
=
s*~J
Aj
j
f(J) - x
"\
~\
)
1
'2
n
(
Z
<.
7\j
)
I
s *+1 (
j
rrn
- x
~
33
and
=
I\j
A.
+ A.
l
Then
T
n
2
I\n
n
=
+ ... +
rcnJ -
n-l
~
+
~~
1\*
s
s *+1
ns *+1)
x
1
1\.J
~d)
s +1
s*
We may assume
n
f(ti) -
!\*
s
s *+1
x
-f{s *+1)
*+1
~(s *+1)
S
x
-
x
(
j
/\ . > j 0 for all
J
j
1
- x
(5 s *
1
1
j+l
nj+l)
- x
+ 0
::::
x
Thus
> __n~o__
n
-
-
to be large, so that, by (5.14),
in the range of consideration.
T
\1
1\ s *)
j+l
f(j+l)
-
x
+
x
If we use this last inequality in (5.21), and also make use of (5.20), then
we find
2Rn (x-I) <
1
s* 0
- s *+1
s*
<
f( s
<
1
(
n
2::
I)
1--".,...---s *+1
--- *
- x
\
S
;
1
/
s *+1
- x
f( S *+1)
\
n
E
I)
34
,
(
C.
j
rrn
- x
)
f(j)
*+1
j
+1)
s* 6
- --;;---s +1
- x
- 6
*
f( s *
+ 1) log (n+1
-r )
s +1
f(S\l)
We may therefore conclude, from (5.11), that
G (x)
n
< (s
*
1
*
"2
0 f(s +1)
1
s*
e t\f(x)
+ 1)
n + 1
n > s*
,
'Vlhere
Hx)
=
"2 - --
21
*
I)
s +1
'f(s*+i)
x~
"
f
At this point in our argument we need information about the order of magnitude of
for
p
>Y
a
co
uN =
+ 2.
is some constant
2::
N
To this end, define
c
such that
Evidently
'
J
,
n
aCt) =
aCt) <
c t Y L(t)
co
=
N-o
En< t an'
d a(u)
as before.
for all large
Then there
t, by
(1.4).
35
r -- a(u)- 7
- uP
1
:::
co
co
p
+
N-o
J
a(u)
U
N
du
p+l
co
<
J
c p
L(u)
N
if we ignore a negative form,
du
u p-y+l
Hence, by an obvious substitution in the integral,
vie have
co
J
cpL(N)
p-y
N
But, from (1.2)
L(Nv)
dv
v p-y+l
L(Nv)
L(N)
1
J
:::
a(NV)
-v1
a(N)
L(N)
where a(x) ---> 1, as
x --->
co,
exp
(
!
Nv
:,~u)
du
i
exp
} (1+<)
L(N)
Nv
i
d~
(
)
:::
Hence we can appeal to dominated convergence to infer that
J
1
as
N --->
co,
'f
co
co
L(Nv)
L(N)
}
Therefore, given an arbitrarily small
we can choose N so large that for all v > 1
L(Nv)
l
dv
p-y+l
v
........>
'-
1
It then transpires, from (5.23), that
dv
. p-y+l
v
€
> 0,
36
x we shall have ~ 6 f(S*+l) > Y + 2) and
For all sufficiently large
2
hence may infer from (5.22) and (5.24)
that
and hence that
(5. 25)
a
L:
n=s *+1
G (x) ::: 0
n n
Y L(s *+1)
Lex)
*
0:>
1
XYL(X)
e~(x) )
« s +1
)
x
Since
s *+1
f(s *+1)
:::
*
( s :1 ) ( f(: * ) )
s
f(s +1)
(S *:1
<
s*
(
)
f(s*)
) (1 + e)x
)
s
it is clear that for all large
s* + 1
x
<
f(s * + 1)
(1 + 2e) x
•
()~
Therefore) from the definition of
~(x)
which follows
1. ,~:)
(2~g,)
.:f
*( x)
<
1
-2
-
0
s
-r,-'+
ex
In addition) we can deduce from (1.2) that
L(s * + 1)
L(x)
L(S * + 1)
L(X)
for any
€
we infer that
>0 •
a(s * + 1)
a(x)
:::
:::
o«
s
*
..:- .\- \
I
(
x
s* + 1
€
+ 1) )
x
)
exp
\
)
)
L
I
\
\
)
;1:
a(u)
U
du
l\
\
)
37
(5.26) and (5.27) with (5.25) we discover that
If we combine
00
(5.28) __
1_
xY L(X)
a
E
*
n:::;s +1
G (x)
n
n
But (s *+1) > (1 + e) x f(s *+1), so that
We can therefore deduce from
1
E
*
n:::;s +1
G (x) - > 0 ,
n
a
n
as
x --->
1
sup
as
x _>
00
so that
E a G (x)
n:::;Cx n n
en
en
(5.19) we find that given any
C> 0
we can choose u sufficiently large
x->
00
(5.28) that
On combining this last result with
lim
(s *+l)/x --->
<
€
This result establishes (3.16) and completes our proof, since
00
=
E
a G (x)
x n n
n>1'i
€
> 0
•
38
REFERENCES
G. Doetsch (1950), Handbuch der Laplace-Transformation, VoL I, Verlag
Birkhguser, Basel.
B. V. Gnedenko and A. N. Kolmogorov (1954), Limit distributions for sums of
independent random variables (Translated from the Russian by K. L. Chung),
Cambridge, Mass., Addison-Wesley.
G. H. Hardy (1949), Divergent Series, Oxford University Press.
J. Karamata (1930), Sur un mode de croissance reguliere des fonetions,
Mathematics (Cluj), ~, 38-53.
T. Kawata (1956), A renewal theorem, J. Math. Soc. Japan, ~, 118-126.
H. Robbins and Y. S. Chow (1962), Abstracts of Short Communications, International
Congress of Mathematicians, Stockholm.
W. L. Smith (1962), On some general renewal theorems for nonidentically distributed
variables, Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics
and Probability, Vol. II, 467-514, Berkeley, University of California Press.
(1963), On functions of characteristic functions and their application
to some renevTal-theoretic random walk problems, to appear.
k~ifu:~ttdurn
to the report v80N THE EIDmNi'ARY RENF..'WAL THEORJrM FOR NON-IDENTICALLY
D!STRIBUTED VARIABLES". (Revised version)
It 'turns out t.hat, 'When the randall variables are non-negative, conditions
(Tl) and (T2) autanatically imply that (109) Will hold for sane function
Thus
(T5)
l~
can be g:t"eatly simplified to read merely
X
n
f(n).
are nm...negative U
The proof runs as follows: ..
(1'2), being true for every
llI:I
(Al)
J
> 0,
:!Jrq>lies the existence of an unbounded
A.{n), say, such that
increasing f'unction
-n1
~
~
f
1 .. Fr(X)
n/A(n) :r=l
~
dx
-->
°
as n -=> llI:I
•
Therefore, fran (1.6) ,
e
ret (n) be another unbounded increasing :f'un.ction, but one which increases
V9r.! much more slowJ;y than A(n).
•
all r > p(n). Then the result (A2)
.(A3)
sultbs.i;
Let p(n) be
r/f(r) > nfl,.(n)
implies that
p(n) n / . A ( n ) · ·
n
r!!(r)
.! I:
1 '" F (x) ~ dx + 1:
1:
~ 1 .. F ex) ~ dx
n r=l
0
r
n p(n)+l 0
r
'"
Jt
J
~
1-11+
>0
0
•• 0
J
+ J.1p{n)
n
and so we my fran (loG) conclude t.b9.t 'rl(n) .-.~.-.> 0
if we can shaw p(n)/n
It would then follow that T (n) > (J.l ... 2e) tor all large n and tram this,
2
in view of the arbi"lirariness of ti,
would mve
we
=>
00
0
2
n
.t
1
11m 1n:f
n ._> Q)
n
r-l
f'ram. which" of course"
1:
n
n r1<r>
- f 1 ... F (x)}
1'=1
r
E
ax --> t.t"
0
which is far more than we need.
Now for all large n
A.(n)
~
A. (n
l~nj(n)
)
and so" i f we put s(n):: (log A.(n) )/A.(n)
ns(n)
W my"
by Lemma
>
-
~refore" i f we let
0
o
i(n)
9" assume n/>..(n)
this it follows easi:l¥ tIlat
all r ~ ns{n)
n
,
is increasing far all large
n/(ing >..(n»
is
inciSing
n;
fran
for all lArge n.
f(n) ... log '),.(n)"" we lave that r /fer)
~
n/A.{n) for
Hence p{n) (ns(n) and it my be seen tlBt sen) -=> 0 ;
tlms our claims are proved.