;.
~tf
r'f~
t
~
ik,;q
INFINITESnvIAL RENEWAL PROCESSES
by
Walter L. 8m1th
University of North Carolina
This research was supported by the Office of
Naval Research under Contract No. Nonr-855(06)
for research in probability and statistics at
Chapel Hill. Reproduction in whole or in part
for any purpose of the United States Govermnent
is permitted.
Institute of Statistics
Mimeograph series No. 237
August 1959
nWnUTESIMAL RENEWAL PROCESSES
by Walter L. Smith
University of North Carolina
•
1.
Introduction.
Renewal theory (8mith Ll_7) is concerned pri-
marily with the renewal function H(x), which gives the expected number of
partial sums Sn
= Xl
+
~
+ ••• + Xn of the independent and identically
distributed random variables
tXi ~ which satisfy the inequality Sn S x.
Generally the Xi are non-negative, and we then call the sequence ~ Xi'} a
renewal process; when we wish to drop the non-negativity restriction, i.e.,
consider unrestricted Xi' we can draw attention to this extra generality
by speaking of an ext.ended renewal proces.!.
The renewal function H(x) can be related to the following physical
model concerned with the motion of a particle P on the real axis.
At zero
time P is assumed to be at the origin and P remains there until time t
= 1,
at which instant P moves instantaneously to the point Sl
= Xl
to remain for a further time interval of un!t duration.
The motion of P
where it is
then continues in an obvious way so that, for instance, P is at Sn during
the time-interval
in,
n+l).
In this model the renewal function H(x)
measures the expected amount of time which P spends in the interval
(-co J
xJ.
It is assumed here, as is usual in renewal theory, that
gXi > 0 and it is known that in this case H(x) is necessarily finite
for all finite x.
One can now imagine the following development of our model at' a renewal
process.
.
e
Suppose we change our time and distance scales so that the indi-
Vidual shifts in the position of P become) in general, very small but at
the same time occur extremely frequently.
We are led to consider a process
- 2 -
x(t) with stationary independent increments for which x(o)
~l
=
= ° and
x(l) > 0, and to consider the e~cted amount of time that this
process satisfies the condition x(t)
~
x.
We shall call this expected
duration H(x) and show that H(x) has all the properties of a conventional
renewal function.
In particular we shall show that, under suitable con-
ditions, the following three theorems hold.
Blackwell's Theorem.
For any a > 0,
H(x + a) • H(x) ---> a ~il
as x -> + CD; the limit being zero if
second Renewal Theorem.
~l
= + co.
For a certain constant
J.l.
2
(Which is actually
the second moment, assumed finite, of the renewal 'lifetimes' in a
certain ordinary renewal process Which is associated With the process
x(t); the details of this ordinary renewal process are given below)
1
H(x)
+
0(1)
as x -> + co.
Renewal Density Theorem.
hex) -> ~il
->
°
If
hex) = Ht (x) exists then
as x -> + co,
as x ---> -
00.
The name for the second of these theorems is coined here, for the
first time, as a convenience.
It has only previously been proved for
renewal processes satisfying the non-negativity assumption (smith
["2_7);
we give a proof of the second renewal theorem which is valid for extended
renewal processes in section 3 below.
:Because of the physical model which we have described, and which
gives rise to the analogy between our present study and standard renewal
·e
- :5 -
theory, we shall speak of infinitesimal renewal processes.
In Theorems 1, :2 and 3 we , essentially, complete the program outlined above.
we
In particular, when x(t) has a frequency function f(Xj t),
shall see that
co
hex)
Jo
=
f(x;t )dt •
Integrals of this kind are of particular interest in the theory of queues
and dams, as we hope to show elsewhere, and it was in considering their
asymptotic behavior that we were led to the present investigation.
Un-
fortunately the most general sufficient conditions available for the renewl density theorem (those of Snith [)J) are, in the present application, bound up with the behavior of the density function
co
f(x) =
e -t f(x;t)dt •
J
o
One obstacle is that we need f(x) to belong to the class Ll + for some
5
5 > O. Theorems 4 and 5 are concerned with providing sufficient restraints under which this condition will be automatically satisfied.
We
should point out incidentally that the renewal density theorem for the
infinitesimal renewal process can apply to cases where x(t) is not absolutely continuous.
Finally we consider the particular process associated with the density
-x t-l
f(x, t) = e r(~f '
t >0 •
We shall see that none of our conditions covers this
though the renewal density theorem does hold.
s~le
case even
In fact we shall deduce
from this process a conventional renewal process for which the renewal
- 4density theorem holds, but for wbichf(X) does not belong to any class
Ll +8 for
e > O.
This raises the intriguing question as to how the class
of functions, considered in
L3_7,
for which the renewal density theorem
holds, can be Widened.
A heuristic discussion of the present ideas has already been given
in Smith
Ll-J.
It is proposed to consider elsewhere the theory of in-
finitesimal renewal processes in relation to the particular homogeneous
process which arises in the theory of dams.
It will there be shown that
the infinitesimal renewal density function hex) gives the expected number
of up-crosses of the real number x by the process.
2.
The infinitesimal renewal function.
Let
1Xi}
be a sequence
of independent, identically distributed random variables such that
o < 8 Xi
~ +
ClO.
Let Sn
= Xl
+ X2 + ... + Xn and write Fn(X) for the
distribution function of Sn and F(x) for Fl(x).
tion corresponding to
~Xi
(1)
J is
H(x)
Then the renewal func-
determined by
=
00
Fk{X) ,
t
k=l
(see, e.g., ~l-J).
We write x( t) for a process with stationary independent increments
such that x(O)
=0
and 0 <
&x(l) ~
(X)
and suppose, wherever necessa.ry,
that x(t) is both separa.ble and measurable.
processes we refer to Doob
[4J,
(For a discussion of such
Gnedenko and Kolmogorov ~5J,
Levy
["6J). We shall also write F(x;t) = prob { x(t) S x}, defined for all
t
> O.
The function U(x) is defined by
- 5~e
U(x)
=1
for
=0
for
x<O
x>O
•
Sinoe U{·) is Borel-measurable, U(x-x(t» Will be a measurable prooess
for any fixed x (Balmes
J:7J,
p. 81).
Thus, by Fubini's theorem, we
oan introduoe the random variable
Jo
CD
(2)
T{x) =
U{x-x(t»dt •
Evidently T(x) is the total amount of time for whioh x(t)
sinoe
g U(x-x(t»
:&:
S x.
Further,
F{x;t), we oan deduoe from (2) by a seoond appearl
to Fubini's theorem that
Jo
00
g T(x) =
(3)
although the finiteness of
Theorem 1.
g T(x)
F(x;t)dt ,
is not yet established.
If we define the distribution funotion
Jo
00
(4)
F(x)
=
e- t F(x;t)dt ,
and the renewal funotion H(x) by (1), in whioh of oourse Fk(x) will be
k-fold oonvolution of F(X) with itself, then
Hex) = ~ T(xl •
Consequently (]T(X) is a finite and monotone non-deoreasing funotion of x.
Proof.
Let ~Yn1 be a sequenoe of independent, non..negative J and
identioally distributed random variables with the distribution funotion:
prob {Yn S y} = U(Y)["l ... e- YJ, i.e., the Y are exponential variables.
n
It is assumed that the variables {Yn are independent of x( t). Write
1
....
Zn = Yl + Y2 + ••• + Yn ' for the n-th partial sum of the Y' s.
Then
- 6 prob {x(Yl )
S
x
J
QO
1=
y
e- prob {x{y)
S
x} dy
o
= F{x)
by (4).
I
Consider the equation
r=n
x{Z ) = E ~x(Z) - x{Zr_l)-7 •
r=l
r
n
Since x(t) is a process With stationary indel'endent increments the n
brackets in the summation in (5) enclose n independent and identically
distributed random variables.
For ~x(Zr) - x(Zr_l)-71 saYI is the
change in the value of x{t) during a period of time of length Y ; thus
r
all terms in the summation will have the same distribution function as
x(Yl)1 namely F{x).
Hence x{Zn) will have Fn(X) as its distribution
function.
By a well-known property of the exponential distribution the random
variable Zn has the probability density function
e -x xn-l
(n-l)!
for x
I
~
0 •
Consequently we can infer that
CD
=jo
e
-y n-l
.
y
()
(n-l)r F x;y dy •
Thus by equating the distribution functions we have obtained for the
left- and right-hand sides of (5) we conclude that
CD
Fn{x)
Jo
=
-y n-l
e(n_lJ!
F(xjy)dy.
Hence l by a standard theorem from integration theory (Balmes ~7_71
....
p. 112 1 Theorem B)I we find that
- 7
=
H(x)
M
co
r: Fn(X}
n=l
CD
=Jo
n-l
00
-y
n=l
(n-l)!
r: e
y
F(Xjy)dy
00
=
This, by
(3),
Jo
F(xjy)dy
proves H(x} = gT(X).
•
The fact that H(x) is finite and
nonMdecreasing is well-known, since H(x) is a conventional renewal function (see, e.g., Blackwell
["8J).
We can call F(x) the associated distribution function, and the random
variables } Xn~ defined by Xn = X(Zn} - x(Zn_l} the associated renewal
process.
The argument in which we have just indulged to prove Theorem 1 has
glossed over a non-trivial point of rigor, namely the question of Whether
x(Y ), say, is a random variable. It can be shown fairly easily that
l
this question has an affirmative answer, but as the necessary measuretheoretic elaborations would seem a trifle out of place in the present
setting we shall not discuss this point further here.; moreover Theorem 1
can equally well be proved without mentioning random varia.bles, either
by an argument using characteristic functions or by a somewhat tiresome
manipulation of multiple integrals and Jacobians.
the present demonstration as being more vivid.
Write
+00
I!r =
J
MOO
and
xrdF(x) ,
However,
"We
prefer
- 8+co
mr(t)
=
J
r
x dx F(x;t) ,
-CD
Also write mr = mr(l). Since" as
is well-known" the cumulant generating function of x(t) (given x{O) =0)
whenever these moments are definable.
is a direct multiple of t, the relations between the moments {mr(t)}
and the moments
t mr1 are easy to determine.
In particular, one finds
that when the app1"opriate moments are finite
ml(t) = mlt
(6)
~(t)
=
~t
,
+ mit(l-t) •
Thus, by (4), we can find that
e
(7)
-t
t dt
= ml
CD
1J2
Jo e-tL~t
=
+ mit (l-t)_7dt
= m2
+ mi •
Renewal processes may be continuous or £iscrete (see" e.g., ~l_7).
For every theorem which is true for a continuous renewal process there
is a closely analogous result for a discrete renewal process.
For the
remainder of this section and for § 3 we shall assume that the associated
renewal Rrocess is continuous.
Corresponding results for the case in
which the associated renewal process is discrete may be proved on lines
similar to those adopted here (by appealing to appropriate standard
theorems for discrete, instead of continuous, renewal processes and by
dealing with sums instead of integrals" etc.).
We shall briefly discuss
in § 4 the characterization of those stationary processes with independent
increments which lead to discrete associated renewal processes.
Theorem 2.
(i)
For any
a > 0, as
x
-> co
where the limit is to be zero i f ml = + co •
(ii)
If'
1'lI2 <
then as x -:--> + co
00,
8 T(X) =-!
+ ~
m
2m
l
r?
where
+
0(1)
,
l
= m2 - mi.
-Proof.
Part (i) is simply a re§tatement of Blackwell's Theorem for
the extended renewal process (Blackwell
~
6T(x)
H(x)
L8_7).
We have merely substi-
(7),
m for ~1.
l
Part (ii) follows immediately from the Second Renewal Theorem quoted
tuted
for
in section 1 above.
and, by
We have used (7) to express the result in terms of
the moments ~ m } instead of the moments
r
{lJ.r }. However, the second
Renewal Theorem has nowhere been proved for the extended renewal process,
so we must now prove this theorem.
3.
Proof of the Second Renewal Theorem.
Consider the function
Q(x) defined by
x
(8)
Q(x) = 1J.1U(x) ..
J
LU(y) .. F(y)JdY •
-co
Since
co
Jo Ll and
F(x)_7dx =
o
J
-co
J
CD
xdF(x)
0
0
F(x)dx = -
J
-co
xdF(x)
.. 10 ..
we can also write (8) as
J ["1 00
Q{x)
=
~
0,
for x <
o.
for x
F{y)JdY ,
x
x
J
= ..
F{y)dy
-co
Notice that, since we are supposing 1-1 to be finite, the first ab2
solute moment of F must also be finite.
Let us write F+(X)
and 1-1~ for the first moment of F+(x).
= U{x)F(x),
Then for x ~ 0 we have, by (9), that
x
Q{x) =
1-1~
..
Jo
Ll - F+(y)_7dy •
Then by the results of smith ["9_7 concerning I derived distributions I , or
by direct computation, it follows that
co
Jo
Q(y)dy =
~
co
J
y2dF +(y) •
0
A simila.r argument, coupled with this result, thus proves
+co
Lemma 1.
J
Q(y)dy =
~
1-1 , the integral converging absolutely.
2
-co
Given any
€
> 0 we can, by
Lemma 1, find a. large 6 so that
r
lyr~6-l
IQ(Y) Idy
<
€
•
Consider
I+
=
J
Q{y-z)dR(z)
y-z~6
=
00
E
n=l
J
6+n.l~ y-z<
Q(y-z)dR(z) •
6+n
• 11 •
Because Q(.) is non-increasing for positive values of its argument we
therefore have
00
I+ ~ E
Q(6 + n - l)["H(n + 6) - H(n + 6 - l)J •
n=l
But there is a finite ~ such that H(x + 1) - H(x) < ~ for all x, for
obviously H(x + 1) - H(x) < 1 + H(l) • H(-l).
I+
S f3
Thus
00
E
Q(6 + n - 1)
n=l
~~
ex>
J
Q(y)dy
I
6-1
by the monotonicity of
Q
again.
Hence I+ can be made arbitrarily small
for all y by choosing !::l. large enough.
A similar argument can be applied to
I.
=
J
/Q(y-z) /dH(z)
I
y-z~6
which, coupied with the result just obtained,l shows that if
(10)
I =
J
Q(y-z)dH(z)
Iy-zi ~6
then I can be made arbitrarily small, uniformly in y I by choosing l:i large
enough.
Let ~, be the class of bounded step-functions each of which vanish
Outside some bounded interval (which may depend on the function) and have
finitely many discontinuities in that interval.
Then it is an easy de-
duction from Blackwell's Theorem for the extended renewal process that ,
as x -> + co
I
+cD
(11)
J
-co
G{x-z)dH{z) ->
~
+00
J
-co
G(z)dz
I
- 12 whenever G(·)
E
%.
Since Q(x) is monotone in (-00 1 0) and in
it is not difficult to see that for any
G+(.)
E
'~SUCh
E
>
°we can find G-(.)
LO,
E
+00)
~ and
that for all Ixl S A we have
(12)
G-(x) S Q(x) S G+(x) ,
and
J
°S
LG+(X) - G-(x)JdX S
E '.
Ixl~A
Plainly
J
J
G- (x-z)dH(z) S
Ix-zl ~A
Q(x-z)dH(z)
Ix-zlsA
J
S
G+(x-z)dH(z),
Ix-zlsA
and since (11) holds for all functions in't we discover that
J
G-(y)dy S
J
J
;:~
Iyl ~A
Q(x-z)dH(z)
Ix-zISA
S IIiii
Q(x-z)dH( z)
x=oo Ix-zISA
S
J
G+(y)dy.
Iyls A
This chain of inequalities in conjunction with (12), (13), and the arbitrariness of e: prove that as x ->
f
00
J
Q(x-z)dH(z) ->
Q(y)dy.
-
-
Ix-zl <6
Iyl< A
Because of Lemma. 1 and of what we have already proved concerning the
integral I of (10) we have thus established
Lemma 2.
As x
->
+ co,
w.
- 13 +00
J
Q(X-Z)dH(z) =
~
2 + 0(1) •
J.L
-(I)
We shall presently make use of the elementary renewal theorem, which
states that as x
->
+
00,
H(x)/x ->
in many places, in particular in
1:1J.
1•
J.Li
It will be found discussed
The proof' given in
L1J can be
adapted easily to cover the extended renewal process.
Since
< 00 it is an easy deduction that z2F(z) -> 0 as z ->
J.L
2
Thus we can write
x-z
J
F(y)dy = p(x-z)
x-z
-00
where p(x-z)
->
0 as z
-> + 00, x fixed. Hence
~z
J
z::C H(z)
~z
LU(y) - F(y)JdY
= z::C -
B(z)
-00
J
F(y)dy
-00
=
lim H(z)p(x-z)
x-z
Z=+oo
•
In view of' the elementary renewal theorem we can thus conclude
z::' H(z)
(14)
x-z
J
LU(y) - F(y)JdY = 0 •
-CD
B(z) must decrease to 0 as z ->
numbers would be contradicted.
-00,
Thus
x-z
lim
Z=-(I)
H(z)
J
or the strong law of' large
LU(y) - F(y)Jdy
= z:~
J.L1H(Z)
-00
=0
,
where we have used the fact that
+00
J.L1
=
J
-co
I:U(Y) - F(y)JdY
,
-00.
••
- 14 in the intermediate step,
If we now write
+co
K=
J
+co
J
dB(z)
-<II)
LU{y) .. F{y)JdY
-(X)
and appeal to (14) and (15) then integration by parts r:Lelds
J
~j j
+(J)
K=
LU(x-z) - F(x-z)JH(z)dZ
-(I)
dz
-QI)
U(z-y)["U(x-z) - F(x-z)JdB(Y) •
-<II)
By Fubin! t S theorem, therefore,
+00
K=
J
-fG)
dH(y)
-CD
J
U(Z-Y)L(x-Z) - F(x-z)JdZ •
.. (X)
We change the variable of integration in the inner integral from Z to u
by. putting Z
=x
+ y -
Uj
+co
+co
K=
J
this yields
dH(y)
-CD
J
U(x-u)["U(u-y) - F(u-y)Jdu •
-(X)
A further appeal to Fubin1 t s Theorem then shows
+co
+00
K=
J Jco
du
..(X)
LU(u-y) - F(u-y)JdH(Y) •
..
However, the integral equation of renewal theory (see, e.g.,
that the inner integral is simply F(u).
x
(16)
...
K=
J
L1J)
shows
Thus
.
F(u)du •
-(X)
If we hark back to equation (8), which defines Q('), we realize that (16)
implies
- 15 Lemma 3.
+cD
+00
J
-co
Q(x-z)dH(z) = 1-11H(X) -
J
F(u)du •
-co
We are now in a position to finish our proof I for
x
J
F(u)du
= xU(x)
... 1-11 + 0(1)
-00
as x ->
00 •
Thus Lemma 2 and Lemma 3 Jointly imply
H(x)
as x ->
4.
00.
The renewal density theorem for infinitesimal processes.
We
prove first
Theorem ~.
(i)
non-negative, p + q
If' z (Dl): F!x)
= 1,
= pU!x)
"
+ q F(x),
where p and q are
A
and F(x) is an absolutely continuous distribu-
tion functionj
"
"
COO): fex)
is a frequency function corresponding to F(x)
and belongs
to Ll+8 for some 8 > 0;
then hCx~
= HI (x)
(a) i f t(x)
"
(b) if f(x)
(ii)
eXists for all x
t 0 and
--->
0 as x
->
+
00 I
hex) ->
->
0 as x
-> ...
CD I
hex)
->
l-1il
as x
0 as x
--->
-> -
+
CD j
00 •
A sufficient condition for F(x) to satisfy CD1) is that
,.
F!x,t)
p(t)U(x) + q(t)F(x,t) I
=
•
(17)
..
where pC·) and q(.) are non-negative measurable functions such that
pet) + q{t)
function.
= 1,
"
and F!xzt) is an absolutely continuous distribution
In this case pet)
= e- rt
tor some
r > O.
- 16 -
-e
(iii)
If F(x,t) is of the for.m (17) then a sufficient condition for
F(x) to satisfy (D2) is that if
define
we
,
.
J
TOO
I l +8 (t) =
:1+8
Lt(x,t)J
dx
-00
Proof.
(i)
If F(x)
,..
= pu(x)
+ q rex) then, in an obvious notation,
(1) shows that
kk
00
H= t
k
,..
00
t (r) p -r qr Fr + t
k=l r=l
L_
= pq--u
so that for all x
~
P
k
U
k=l
(I)
+ t
r=l
Lqr'"Fr
00
t
s=O
r+s
s
(r) p J
0
,.
The renewal function H(x) is one to which the available renewal density
theorem will apply (Snith
L3J)
and so 3(i) is proved.
Notice that
p < 1, since F(x) is to have a strictly positive first moment.
(ii)
If (17) holds then, by (4),
J
00
(18)
F(x) = U(x)
J
00
e- t p(t)dt +
o
e-
t
q(t) F(x,t)dt •
0
,..
But there will be a frequency function f(x,t) such that
,..
F(x,t) =
x
Jr
-00
,..
f(z,t)dz,
- 17 and so (18) 1eads l via Fubini's Theorem to
ex>
x
co
t
F(X) = U(x)
e- p(t)dt +
dZJ e- t q(t) f(Zlt)dt •
o
-00
0
This shows F(x) must satisfy (D1).
J
J
If (17) ho1ds l moreover, then prob
o},
i x(t1 + t 2 ) =0) = prob 1x(t1 ) =0 }
x prob ~ x(t1 + t 2 ) .. x{t1 ) =
for any positive tl' t • Thus
2
p(t1 + t ) = p(t1 ) p(t )1 by the homogeneity of the x(t) process. Since
2
2
pet) must be monotone non-increasing, we must have pet) = el't, as announced.
The constant I' may not vanish for this would make F{x)
= U(x)
and then F(x) would have a zero mean value.
(iii)
When (17) holds,
,. =J
co
rex)
e- t (1 - e-I't)"
f(x,t)dt •
o
t
BB1der s inequality then gives
s.!
00
Lf(X)J
l +8
e-t(l - e-I't)["f(x,t)Jl +8 dt
o
t
so that, by Fubin1 s Theorem,
J
s.J
00
+(I)
t:f(x)Jl+8 dx
-co
e-tel - e -I't) I + (t)dt •
l 8
0
This proves 3(1ii).
Corollary 3A.
When F(x;t) has the form (17) then for all x
J
i0
ex>
hex)
=
(1 - e-I't) f(x1t)dt •
o
notice that if F(x,t) is absolutely continuous for all t > 0, With
a density function f(x,t) then (17) automatically holds (with pet) = 0).
- 18 -
,..
We can then take f
.
= f.
The more "general" form (17) with p( t) ~ 0 can
arise, however, when we have
F(x,t)
where
A(.)
=
Q)
I:
n=O
e
-rt ('Vt)n
;'
An(x)
n.
Au(·)
is some distribution function and
is its n-fold convol-
lution.
It is desirable to obtain, if possible, conditions under which (Dl)
and (D2) are satisfied which relate to more intimate properties of the
x(t) process.
~(Q)
We shall now introduce the cumulant generating function
of x(l), i.e., we suppose
(19)
eiQX(t)
(Recall that x(O)
= 0).
= etV(Q)
•
Then since (19) is to be a characteristic tunc-
tion for all t we must bave
(20)
for otherwise the modulus of this characteristic function would exceed
unity for large values of t.
If ¢(g) is the characteristic function of F(x) then it follows from
(4) and (19) that
J
ex>
¢(Q) =
e-t+tV(Q) dQ
o
(21)
Theorem 4.
(i) A necessary condition for F to satisfy (Dl) is that
there exist
lim
IQI=a:>
v(Q) = A ~ •
00
•
- 19 (ii)
.
It A is finite then p
= (1
- A)-l > 0, and in this case a
sufficient condition for F to satisfy (Dl) and (m) is that LvtQ) .. AJ
be the Fourier Transform of a function in Ll and Ll +8, for some 8 > O.
(111)
If A is 1nfinite then p
= 0,
and in this case a sufficient
condition for F to satisfl (Dl) and (O2) is that for some
€
>0
l1m~>O.
(22)
~E
/9/=CD
---~~---
Proof.
(i)
If (D1) holds then ~(9)
Fourier Transform of the density function
~besgue ~mma" {I(Q)
-> P as 191 ->
'"
= P + q '"~(9)" where ~(9)
is the
'"
rex).
Thus, by the Riemann-
(I).
Reference to (21) then proves
4(i).
(ii)
then p
From the previous remarks and (21) it is clear that i f A > -
= (1
_A)-l > O.
00
From (21) it transpires that
Suppose g(x) is the function in ~ and Ll +6 whose Fourier Transform is
Lvt(9) - AJ. Then (23) implies that for almost all x
J
+clO
q r(x) = p
g(x-z) dF(z)
I
-00
where we have appealed to the fact that SELlin writing down the righthand side.
A familiar application of B6lder's inequality then gives
J Lf{X)J
q1+6
1
+8 dx
~
pl+8
-co
(iii)
It (22) holds then for some
J
+co
+(I)
l:S(X)Jl +8dX < CD •
-00
1')
> 0 and all sufficiently large
- 20 -
Thus" by (21)" ¢(Q)
= o( IQI- l / 2- E).
Hence
¢(Q)E~
and so" by Titchmarsh
(L10J" p. 6) there mus·t exist a density f(x) which belongs to L2 •
It follows from the discussion of rAVY J:6J that for the most
general possible pg:ooceos with stationary independent increments one can
in the form
iQQ -
~
J
+1+0
(j2 g2 + "'1
iQu
e
- u~ -
iQu
dn(u)
-1-0
(24)
J
lui
+ A2
(e
iQu
-l)dN(u) •
>1
In this expression (j2" Ap
A are arbitrary non-negative real numbers"
2
a is an arbitrary real number, and N(·) is an arbitrary distribution
function which is continuous at O.
The form (24) is not the most com-
pact one given in the literature (see, e.g." Gnedenko and KolInofP'O'lL5J)
but is more convenient for our present purposes.
a drift a if a
# 0,
We shall say there is
and a Brownian component if (j2 > O.
It it turns out that the distribution function N in (24) is such
that
+1
J
furdN(U)
< (J)
-1
then "'(9) can be given the simpler and more convenient form
(25)
iQa -
~ (j2Q2
+ex>
+ A
J~
dM(u) ,
-(X)
where .. here" (j2 and A are arbitrary non-negative numbers, a is an arbitrary real number" and M is an arbitrary distribution function which is
continuous at O.
We might call the third term in (25) the pure
~
term.
- 21 Notice that if N is such that (24) can be thrown into the form (25)
then the constant a will be modified in the conversion.
is relevant to Theorem 5(iii) below.
This observation
Before we proceed to Theorem 5,
however, the introduction of the canonical form for '!r (g) provides us with
an opportunity to give the brief discussion, promised in § 1, of conditiona under which the associated renewal process will be discrete.
such a discrete process to arise there must be a real
...
(I)
For
> 0 such that, With
w.
probability one.. x(Y ) is divisible by
Plainly this requires that
l
¢(21C/W) = 1 and hence, by (21) that '!r(21C/W) = 0, The real part of (24)
is, in general, strictly negative and can only vanish at
0'2
=0
and if the sole points of increase of N(u) are
tive or negative (but not zero)
multiple
of
...
21C/W
(~) i f
where u is a posi-
Thus '!r(g) may be given
(I).
the simpler form (25) with 0'2 = 0 and M(u) the distribution function of
a lattice variable.
a
= 0,
But Theorem 5(iii) below shows that we must have
or else '!r(g) would be the cumulant-generating function of an ab-
solutely continuous distribution.
We may therefore conclude that dis-
crete renewal processes only arise if, for some
...
(I)
> 0 and some sequence
p = 1, we can write
such that Po = 0 and t+CO
-00 n
...
n=+co eIW.>iQ • 1
'!r (Q)
I'"
t
=")..
n=-oo
Theorem 5.
nw'
Pn
•
SUfficient conditions for F(e) to satisfy
are that, in the exx>ression (24) for '!r(g), either
(i)
or
0'2 > 0, i. e., x( t) has a Brownian component
(ii) 0'2 = 0, but for some
lim
-
h
l
E
< 1/2.
+h
J
l+E
~
n=v+ h
-h
dN(u) > 0
(Dl) and (D2)
- 22 +1
or
(iii)
N(.) is such that
A
in the
fo~
(25) then a
J ~I>
dN(u) <
CD
and when o/(Q) is cast
-1
r 0,
i.e., there is a drift.
NOtice that sufficient conditions 5(i1) and 5(iii) might apply when
x(t) is not absolutely continuous, in fact 5(11i) might apply when x(t)
has a discrete distribution.
Thus it is not necessary, apparently, for
x(t) to be absolutely continuous before the infinitesimal renewal density
theorem holds.
Proof.
(i) Let G(x;t) be the distribution function of the process
that would have the cumulant generating function (24) .. but with 0'2
= O.
Then, if F(x;t) corresponds to (24) as it stands, it is clear that F(x;t)
is obtained by convoluting G{x;t) and a normal distribution function
with zero mean and variance 0'2t •
Thus F(x;t) must be absolutely con-
tinuous, with a frequency function
+00
f(x,t)
=
J
e
dG(z,t) •
-00
Thus, again by H&lder's inequality,
dG(z,t) •
A use of Fubin1's Theorem yields, in the notation of Theorem 3(iii),
I (t)
2
< 1
•
-2O'.firt,
Thus e- t I {t) is in Ll and 5(1) follows from Theorem 3(1ii).
2
(ii) If w{Q) has the fo~ (25), but w1th a2 = 0, then
- 23 +1+0
~ '!r(9) = A.l
CoS(~)
J'"'
- 1 dN(u)
-1-0
J
+ A.2
(cos(Qu)-l) dN(u)
1
lul>l
= J l (9)
Trivially",e have
IJ2 (9) I So
+ J 2(9), say.
But
2A. •
2
J
+1+0
IJ1 (9) I = 2A.1
2 1
sin (2'19 )
'2
dN(u) •
-1-0
u
Thus , for fixed 'I} > 0, and IQI sufficiently large
IJl (91 > 2A.1
r
+'I}/I91
.
sin2(1:uQ)
22
dN(u)
-'I}j'91
u
+'I}/IQI
If
7)
-~~gl
is small enough sin'2x/~ > ~ for all Ixl < 'I}.
IJ1(Q)1 >
+7)/191
T .r
A. Q'2
Thus
dN(u)
1
-'I}719/
and so
> 0, by hypothesis.
Since
E
< ~ and J 2o (9) is bounded
lim
1;1
=Cl)
"Ie
can conclude that for some v > 0,
..lli&J.
>°
1:+\1
'
191 2
- 24 -
1'!r(Q) I ~ 1.1t'!r(Q) f.
for
An appeal to Theorem 4(iii) finishes the proof
of 5(ii).
It' '!reg) has the form (25) with a ~ 0 then
(i11)
~ :IQl =" + A j
(26)
•
clM(u) •
-CD
It we can show that the integral on the right of (26) tends to zero
as
19 I -> co
then it will appear that
Since 1'!r(9) f
values of 191.
r~ '!r(9) I .. fal 191
for large
~ 1~'!r(9) I we will then have
hH9) I > 0
lim
191 =CO Ii'r
and 5(iii) follows from Theorem 4(i1i).
For any
E
> 0, by the Riemann-Lebesgue lemma.,
9 Jl
as
191 -> CD.
Thus J
l
=
J
Tui
lul>e
9u
) dM(u)
->
0
approaches zero even more rapidly, and we need
only consider
ISin
xl S lxi,
so
IJ 2 1
But
M(u)
is continuous at
be made arbitrarily small.
(26) tends to zero as
e
0,
s
f
dM(U).
lul~E:
so by choosing e sufficiently small IJ21 can
This shows that the integral on the right of
191 -> CD,
and thereby completes the proof of 5(iii).
In connection with Theorem 5(ii) it is discouraging to note the fol-
lowing.
.. 25 Theorem 6.
It we.) is any increasing function then there is a dis-
= 0,
tribution function N(.) such that N(O+)
1
J*dN(u) =+00 ,
o
h
Y!
h=O+
-but
Proof.
we!)
h
J
o
=0
dN(u)
•
We have to show there is a strictly positive random variable
XI say" such that
gXl = +
!!!...
h=O+
00" but such that
w(-hl) prob
{X <- h} = 0•
Our task is made easier if rephrased in terms of Y
= X· l •
We want to
find a positive random variable Y with infinite ex,pectation such that
1::0 wet) prob \
Y
<
t} = 0
•
With no loss of generality we may suppose w(x) > 1 for x > 1.
tEn}
Let
be any strictly decreasing sequence of strictly positive real num-
bers such that:
.,." and~n
= gl
El < I" and En -> 0 as n
->
00.
We define
~l"
g2"
+ g2 + ••• + gn' as follows:
~l
= 1,
gn+l = wHn)/e n "
for n > 1.
Thus
",
and for n > 1"
w(~n_l)
gn+l > E
n-l
Therefore
fgn1
= gn
•
is a strictly increasing sequence and
~n ->
CD
as n
->
00.
- 26 Define the function G{x) as follows.
In the interval Ltn , ~n+l)'
n = 1,2, ... ,
€
1 • G{x} = w(t:, •
Then G(x) increases to unity as x tends to infinity, and hence, if we define G(x) to be zero for all x
However,
< 1, it will be a distribution function.
~n+l
J
~
Ll - G(x)Jdx
= w(~n,
~n+l = 1
n
n
so that
co
J ["1 -
G(x)Jdx
= co
o
and G(x) therefore has an infinite mean value.
But
= €n
Since €n -> 0 as n ->
CD
•
this implies that
!!!
r-l x=oo w(x) L·
G(X)- 7
=0
,
and the theorem is complete.
We close this section by drawing attention to a serious omission.
We have been able to obtain some reasonably general conditions, in
Theorems 4 and 5, which ensure that F(x) satisfies CD1) and (D2).
But
A
to apply the density theorem (Theorem 3) we must also have that rex) is
vanishingly small at infinity.
Indeed, this requirement is a necessary
one for the validity of the density theorem.
worthwhile results in this direction.
So far we have obtained no
- 27 -
5. A special infinitesimal renewal process.
Consider the process given by the density function
f{x;t)
=e
-x t-l
r(~r--'
t
> O.
The associated cumulant genera.ting function is
~(Q)
so that, by
=-
log
(l-iQ)
(21),
Plainly this characteristic function belongs to no class
~,
so there is
no hope that the various conditions derived in the last section will be
of any use in applying the density theorem to this special case.
However, by Corollary ;A,
(27)
h(X)
=
e: j ri:,
X
dt •
o
'!be integral on the right of (27) may be evaluated by a method similar to the one of steepest descents (see, e.g., Jeffreys and Jeffreys
["llJ, p. 501 et seq.).
However, the present case is not quite routine"
so we shall give some details of the calculation.
Let A be a large positive constant, and write
e -x xt-l
r(t)
= Il
We first show that I
e
x, if' A is large enough.
+ 12 + I"
l
dt
say •
and I; are asymptotically negligible for large
t- 1 -t
If we use the inequality r( t) > t
~ e If21C)
(see, e.g... Whittaker and Watson
f:12J, p. 253) then we easily obtain
- 28 -
<
I
1
xr-
1
Jr2iiXf
x
e-(x-t) (t)
t
dt.
o
If we put u = x - t and use the further inequality
(!)
t
t
<e
u2
u --2
x
which is valid for t < x, then we find
x
u2
I < 1
e 2x du •
l
j(21tX) Aft.
1 --
Evidently I
can be made arbitrarily small by choosing A large enough.
l
Next we find, in a similar way, that
1
00
t-< 1
e-(x-t) ( x 1)
2 dt •
J(21tx) x+A/X
t - 2
J
13
On putting t - ~ =
T
and B = A - 1 we have, more tidily..
1
I}
< e
J
2
00
T
e-(x- )
(~)
T
dT.
J(21tX) x+Bji
We now observe that for
T
>
X
one has
•
Thus
1
2
I
}
< ~e~:;:
J(21tx)
J
00
dT •
e
x+B[i
But in the range of integration
(T2~)2 > f B2 + .1L
."
Thus
afX
(T -
X -
B,/i) •
- 29 1 1 B2
--li'
2
I3 < e
J,
and so
X,I
B .- .
A)
I
also I will be arbitrarily small if A is sufficiently large.
Finally we consider I , For large XI in view of stirling's asymp2
totic approximation to the gamma function l we can evidently write
x+A[i
J
x-Ali
I 2 ...
e
-(x-t) Xt-l
dt
1
t--
2%
t
x+'}fi
J_ eg(xlt) dt
1
.If2n1 x-Ali
where g(xlt)
=t
- x + (t-l)(log x) - (t- ~)(lOg t),
One finds that
,
It is then simple to show that g has a
uniq~e
maximum
integra.tion for I J it occurs at
2
t
= x + -21 + 0(1)
,
x
o
Routine analysis will then show that
g(xl to)
4
dt
=- ~
= - 1x
+
log x + 0(10~ x)
O(~)
x-
to
and for all t in the range under consideration
in the range of
- 30 -
~Q:
=
~
at'
1
0(-) •
Y?
Thus, for a.1l t in this range we have, by Taylor I s Theorem,
) ~ g(
x,t
1
2
log x -
(t_x)2
2x
+
1
0(--) ;
jX
in this expansion the final correction term is uniform in t.
It follows
from this last expansion that as x tends to infinity we have
_ (t-x):
x+P{i e
I 2 ...
2x
J_ ~;::;:;::::;-- dt
x-Afi j{2nx)
•
This final integral can be made arbitrarily near unity by choosing A
large enough.
Thus, as x
-> co, hex) -> 1 and the renewal density
holds.
The density function
CD
f(x)
Jr
e -t.. ~. x t-1
=
o
(t)
dt
is therefore one which does not satisfy the conditions required by
smith
L3J,
but it is yet one for which the density theorem 1s valid.
- 31 •
REFERENCES
L1J
Walter L. smith, "Renewal theory and its ramifications/'
:!:..!!:.
Statist. Soc., B 20 (1958), pp. 243-302•
.t:2J
Walter 1. Snith" "Asymptotic renewal theorems,," Froc. Roy. Soc.
Edinb."A 64 (1954), pp. 9-48•
.t:3J
Walter L. 8m1th, "Extensions of a renewal theorem," Froc. Camb.
Phil. Soc., 51 (1955), pp. 629-638•
.t:4J
J. L. Doob, Stochastic processes.. John Wiley and Sons, New York,
L5J
B. V. Gnedenko and A. N. Kolmogorov, Limit distributions for sums
of independent random variables.. translated from the Russian
by K. L. Chung.. Addison-Wesley, Cambridge.. Mass., (1954).
~
I
/
Paul 'Ii3VY.. Theorie de l'addition des variables aleatoires"
Gauthier-Villars, Paris.. (1937).
L7J
Paul R. Balmos, Measure theory.. D. Van Nostrand, New York (1950).
L8_7
D. Blackwell" "Extension of a renewal theorem.... Pacific J. Math."
3 (1953) .. pp. 315-320.
L9_7
Walter L. smith, "On the cumulants of renewal processes," ~
metrika.. 46 (1959), pp. 1-29•
.t:10J E.
C. Titchmarsh.. Introduction to the theory of Fourier integrals,
Oxford Univ. P., Oxford.. second edition (1948).
LllJ H.
Jeffreys and B. S. Jeffreys .. Methods of mathematical physics,
cambridge Un1v. P... Cambridge, second edition (1950).
"0\
e
L12J E.
T. Whittaker and G. N. Watson, A course of modern analysis,
Cambridge Univ. P... Cambridge.. fourth edition (1927).