*The research was supported by NIH Grant No. S.T32.HL0700S·03.
t
EXPLICIT CHARACTERIZATION Of OPTJ:MAL
STOPPING TIMES*
by
Anthony Mucci
Department of Biostatistics
University of North Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 1115
April 19.77
.ABSTRACT
.
A large class of continuous time optimal stopping problems is shown
to have solutions explicitly determined by roots of equations
where
H involves Laplace transforms.
xH(x)
These results motivate the
=I
speci~
fication of discrete time optimal stopping problems whose solutions are
approximated by solutions to corresponding continuous time problems,
ing rigorous a procedure sometimes employed in the literature.
A fairly
self-contained treatment of continuous time optimal stopping is also
included, albeit for highly structured situations.
mak~
I NTRODUCTI ON
(I)
This paper is centered around Section (2) where we determine explicit
solutions for the problem of optimally stopping
Hunt process.
for a representative class of Hunt processes,
where
X
o
where
X is a
F and Too where
More precisely, we determine
of Section (2) that
e"'At Xt
It is shown in the theorems
T00 satisfies
is the largest
x
for which
x H(x) s 1
where, if T
x
is the hitting time f0r
= lim
H(x)
In particular, when
x,
then
-AT
dd
y+xy
f
e
xdPy
y <x .
X has independent homogeneous increments, continuous paths:
where
e
f
-AT
xdPy
= e-K(A)(x-y) ,
The technique determining these results can be partially adapted to more
general situations, for instance, it is easily established that the
...
.. 3-
optimal hitting time
Too for the return
nction
\ where
a+t
x
is
standard Brownian motion has form
Too
although the constant
(II)
= inf
t
such that
X ~ K la+t
t
K must be determi ed by other methods.
Section (1) represents an attempt t
provide a simple treatment
of continuous time optimal stopping by im
strong conditions on
the processes and payoffs under considera
In many applications
these restrictive conditions are shown to be more apparent than real
this is made precise in Remarks (1,8).
(III)
Section (3) provides two represent
situations in which the
solution to a continuous time optimal sto
problem is a good appro-
ximation to the solution of a discrete ti e optimal stopping problem
when the underlying processes are close i
1be first two sections were motivate
[13].
the sense of weak convergence.
by a close reading of Taylor
The last section developed from re arks made in Chernoff [3]
and Shepp [12].
Section (1).
Continuous Time 0 timal
ing
processes
We restrict attention to a class of
X = { \ , t ~ O}
which are real valued and governed by an
ntial distribution
transition
t.
p
in the sense that for all
5,
p {X ~ y} =
x s
x, y:
JY p(s,x,dz)
00
u
and a
-4 ..
and
The subclass of such processess to be considered is a restricted class of
Hunt processes which satisfies the following conditions
II
(1.1)
X is strong Markov and
px{X o = xl = 1
(1.2)
X has right continuous paths with left limits
(1.3)
Feller property:
For all bounded continuous
f, (ptf)(x)
= !f(Xt)dPx
is bounded continuous.
(1.4)
X is extended
quasi~left
continuous:
If T
n
sequence of stopping times and if Tn .,.. Too ~
Px
as a finite value
is an increasing
00
then
X exists
T00
a.e. and
X
to the set where T00 < 00
Our more
T00
restrictive condition, implying at least that X00 exist a.e. is conveni-
Taylor [13]
restricts
~n
-+
ent for the development of the theory of this section and presents no real
obstacle for applications of the usual sort, as will soon become evident.
We remark also that the conclusion
X -+ X
or Too < 00 obtains whenever
Tn
Too
one has the following (see Dynkin [7], Volume 1, p. 104):
c
lim sup p(t,X,[X-E, X+E] )
t"O XEK
all compact
K.
In all that follows our state space
ture
=0
S will be the time ..space struc-
[0,00] x R with its product topology and Borel sets.
this space will usually be denoted
be thought of as a process
z
The points in
= (t,x), and our process X will
Z = {(t ,X ), t
t
~
ol
still governed by our
e
~s-
"
transition
p
in the sence that
p(s,x,dyJ
We now define our optjmal stopping problem,
(1,5)
f,
a
(1.6)
For all
non~negative
(t,x) ES.
and continuous function on
f~Ll(t.x)
Js~p f(t + s,
Let's call
f
The clements are:
where this means
Xt+s)dP (t,x) <
the return function.
S,
00
The optimal stopping problem is the
determination of the optimaL payoff f
and the optimal stopping time
Too
where
(1.7)
where
T runs through all stopping times.
(1. 8)
Remarks
The determination of
sup fC(t + T)X+ TdP(t
T~t
,x
) ,
cet)
~0
where is the exclusive concern of the applications in later sections is the
problem determined by the process
+
c(t)x .
It is seldom the case that
X = {Xt' t
~
O}
and the return
X00 exists. i.e"
f(t,x)
=
that (1,4) holds.
Our development of the theory of continuous time optimal stopping uses
(1.4) essentially.
This discrepancy between theory and application is only
apparent, not substantial.
The accomodation of such applications under our
theory merely involves a redefinition of processes, that is, we define
y = { \ , t ~ O}
where
Y = c (t) Xt
t
and consider
-6-
sup
T~t I\
where
c(t)
+
dP
(t,y)
decreases rapidly enough so that
c(t)X
t
~
0,
i\e,~
so that
Y00 = O.
We now consider classes of processes
decreasing
c(t)
X and well-behaved
n0n~negative
for which (1.1) through (1.3) hold and for which (1.4)
and (1.6) hold in the form
p(O,O){~~: c(t)Xt
(1.9)
s~p
f
=
o} = 1
c(t) IxtldP (0,0) <00.
These are the versions of (1.4) and (1.6) proper to
= c(t)Xt , We consider only the starting point
t
computational convenience,
Y
f(t,y)
(t,x)
= y+
= (0,0)
where
for
Class (1)
(1.10)
X
t
= Xo
+
JtO(X )dW
o s s
+
ftm(X )ds
0
s
where
o and m are bounded,
square integrable,
X
o is
W is standard Brownian motion and
X
o in independent
of W.
Class (2)
X is non-explosive pure jump on the integers wi th bounded intensities,
t
i.e., there exists {Ax' x an integer} and {Qx' x an integer} where if
Tx
is the jump time from
x,
then
-7-
=e
P {T > t}
x
x
-A
t
x
(1,11)
sup Ax
A00 <
~
x
00
J y2QxCdY)
sup
<
00
•
x
Both classes of processes are easily seen to satisfy the decomposition
(1.12)
X
is square integrable
V
is a martingale and
o
t
Clearly, for decreasing
f(Yt+s - Yt ) 2 ~ Ks
f s~p
c(t),
f s~p
small
c(t)lxtldPCO,O) <
c(t)IYtldP(O,O) <
00
s
provided
00
and
(1.13)
sup c(t)t <
00
•
t
Now, by right continuity and with obvious notation
f sup c (t) IY IdP(O
t
t
,
0) =
=
S
lim
~UO
f sup c (k6t) IYk~t IdP
k
(0 0)
'
lim (OOp (0 0) {sup c (k6t) IYk6 I > a}da
~UO Jo ' k
t
lim
~t.j.O
2
JOO{~
L c (k~t) J(y (k+1)~t - Yk~t) 2dP (0 0) }da
0 a k
'
-8-
by the Hajek-Renyi-Chow inequality, (see Chow, Robins, Siegmund
[5]) .
Using (1.12):
from which
(1.14)
It is easily seen that
2
00
10
c (t)dt <
00
=> sup c(t)t <
00
\
t~O
Further, it is shown in Chow [4] that the hypothesis in (1.14) also
~ o.
Thus, in order that (1,9) hold, we need c(t)Zt + 0,
t
which certainly holds if tc(t) + O. We collect these criteria for ready
implies
c(t)Y
reference.
Proposition (1.1)
Let
X
t
= Xo + Yt
+ Zt
and
(A)
satisfy (l,12)!
J;c 2 (t l dt
<
f sups c(t + s) IXt+s!dP(t,X)
(1.15)
00
,
<
Then
then
00,
all
and
all
. (8)
The
~onclusions
(t,x)
(1.15) obtain if the hypotheses in (A) are
replaced by
(1.16)
(t,x)
tc(t)
+
0 .
-9 ..
Remarks
The stationary Ornstein-Uhlenback process has representation
Xt = e
..at
t as
(Xo + hab 0 e dWS)
and appears in the applications.
J
One can model the arguments above, via
the Hajek-Renyi-Chow inequality to determine that
f
sup
s
p(
)
e-A(t+s)Ix
t+s
\dP
(t ,x)
<
m
and
{lim e-A(t+s)~t+s =o}
t,xs~
= 1 .
All these results overlap those of Walker [14].
We now turn to the determination of
F and
Tm
from (1.7).
Our
development is a modification and generalization of that found in Taylor
[13] and Dynkin [8].
(1.17)
(A)
We always assume 0.1) through 0.6).
Definition
G:
S -.. R
+
is called excessive if
G is universally measurable
and if
and
(8)
An excessive
G ~ f
(C)
G is called an excessive majorant for
f
if
•
An excessive majorant
G for
f
is
c~Jled
the least excessive
majorant if H ~ G for all excessive m;.,; ·..lrants
H of
f
-10~
Grigelionis-Shiryaev [9]
f~,
show that
f
has a least excessive
majorant~
determined by the recursion
f
Further, the continuity of
n
= sup
t
f
t
P f
n-
f
1 '
;: t f
~
n
and the Feller property make
semi-continuous - see Taylor [13].
We define (allowing
t
lower
f~
=~
as a
value):
rw
(1.18)
=
{(t,x)j f(t,x)
Again, the lower semi-continuity of
fw
= f~(t,x)}
makes
r~
a closed set,
The
hitting time
T~ = inf (t,Xt )
(1.19)
t
E
r
00
will be achieved with probability one from any starting point
if the following holds
(1. 20)
We prove this by adapting Neveu [11] to our context:
For fixed
Let
r
~
00
t < to < r:
we have
so that, by lower-continuity of
f 00''
(t,x)
e
-11-
lim f 00 (r,X r )
r-+oo
Letting
to
~
00
S
sup f(s,X )
and using the continuity of
and the other direction is obvious,
p(
s
s~to
f
t,x )" a.e.
we have
We turn now to our principal result,
Theorem (1.1)
Proof:
Dynkin [8] shows the following.
f(t,x)
+
d ,
then with T
E
If
the hitting time for
It is clear that
rE
by our
continuity assumptions
q~asi-left
Letting
E ~ 0
f
S
is closed, that
r
rE
E
f,
foo and since
and with
X
TE
~
X .
T00
f
TE .,. T
00
00
bounded:
so that
Further,
X,
foo
and continuity
is excessive and bounded, we've established
We now consider the unbounded case, sub-
Set, for each
a > 0:
S
00
we have
our result in the bounded case.
ject as usual to (1.6).
= {(t,x): f (t,x)
r E {- r-and that
and using the quasi-left continuity of
and boundedness of
Since
and
E>O
-12 ..
f
a
= min(f,a)
fa
= least
fa
= {fa = fa}
f
= {f
a
excessive majorant of
f
a
""'
~
f }
a
T
a
= hitting
time for
f
Na
= hitting
time for
ra
a
""'
""'
f cf
a' that Na ~ Ta and that b ~ a implies f b ~ f a by
a
,...,
properties of least excessive majorants, Let f 00 = t ""'f
as a~oo,
a
Clearly ""'f 00 is excessive and ""'f 00 ~ f. therefore ""'f oo ~ f 00
On the other
Note that
,...,
from which
for if we set
tN,
by lower semi-continuity of
which implies
(Noo'X N )
E
while
then
a
""'
fb ,
But then
thus
foo
00
Next.
since
is excessive and
T
a
~
N •
a
N00
~
T00
f oo
=
f oo '
so that
Na
7
Too
-13-
Since
~
f
a
is bounded,
h'C
have hy nynkin's
rC~\I1t.
so that
Now, (1.6) allows us to use Lebesgue dominated convergence I
= at
lim
co
J feN a 'XNa )dP(t.,x ) ~
lim
f
atco a
(t ,x)
~
= fcoCt ,x) = fco(t ~x)
Since
f co
~
f
and
•
f co is excessive, the result follows,
Theorem (1. 2)
F(t,x)
=
J f(Tco'~co)dP (t,x) ,
all (t,x)
C
S•
Proof:
F(t,x)
= s~p
='
~ J f(Tco,XTco)dP(t,X)
f f(T,XT)dP(t,x)
Jfco(Tco'~co)dP(t,X)
=
~ s~p
Jfco(T,XT)dPCt,x)
~ s~p
f
= F(t,x)
fco(t,x)
f(T,XT)dP(t,x)
Q.E.D.
-14-
(1.21)
Remarks on Discrete Time Optimal
Let
X = {X kr , k
=0, I, 2
Stoppin~
where
t ' • ,}
moves on the discrete time lattice
r >0
{kr}
be a Markov process which
and which is governed by the
transition
= Px{Xk r
P(kr, x, dy)
In the case
r
E
dy} .
= 1, we write
p(r,x,dy)
=
p(x,dy) •
Suppose
(A)
If
f
is bounded and continuous, then
(Pf)(x)
=
I
f(t+r,y)p(x,dy)
is bounded and continuous.
(B)
=X
p(
t,x ){lim
k-t<O Xkr
(C)
f
00
exists and is finite}
is positive continuous and
f
= 1 , all
(t ,x)
sup f(t+kr, Xt+kr)dP(t,X) <
k
.
00
Under (A), (B), and (C) the previous development for continuous time
will hold
for discrete time - it will in fact be simpler.
stopping times,
where. Too
T,
to live on the lattice
is the hitting time for
f oo
{kr},
Restricting
we have
= {f =fool where
tf
n
A self-contained and much more general development of the discrete-time
theory appears in Neveu [11] and in
Chow, Robbins, Siegmund [5].
e
-15-
Section (2),
Let
Determination of Optimal Hitting Sets and Payoffs
X = {\, t ~ o}
For all
(2.1)
be a Hunt process such that
and all
A> O.
(a)
I
(b)
P
::;
P
x
and
00
{lime-A(t+S)x
=O}=l
(t,x) st oo
t+s
T . for
f(t,x)
00
applies since (1.1) through (1. 6) obtain.
f(t,x) = e -At x
= e-Atx+.
It is easily seen that we can
T where
XT is negative. given
that we can do better by continuing on to time infinity,
investigate the structure of
f
F(t,x) = sup
T~t
We assume throughout that
e
Our theory
since (2,lb) implies that it is
unreasonable to examine stopping times
4It
(t~x)
(t,x) e S
sup e-A(t+s) Ix
IdP
<
t+s
(t ,x)
s
We want to determine
just as well consider
p
Thus. we'll
F, roo' Too where
-AT
~dP(t,X) =
I
e
-AT00
XT_dP(t,x)
~
P(t ,x) = Px •
Theorem (2.1)
(A)
Let
X = {X , t ~ O}
t
0 ~ a < b < 00,
that for all
T
a,b
be a Hunt process satisfying (2.1),
there exists
is the exist time from
P {T
x
(2.2)
{
•
Then there exists
X ~ 0
o
X~
a,b
fe
b
a,
such that
then
=1
< oo}
-AT
[a,b],
x e (a,b)
~
dP
a,b
x
'
roo = {(t, x), x ~ x
o}.
Suppose
such that if
-16-
X = {X , t ~ O} be a Hunt process satisfying (2.1) \ Then
t
..At
the optimal hitting set for f(t,x) = e
x again has form foo
(8)
Let
x~xO}
{(t,x):
if
=
X has homogenous independent increments, Le.,
if
= Ft (a.-x)
P {X sa} = PO{X sa-x}
(2.3)
x
where
Ft
t
t
is a distribution for each
t
~
O.
Proof:
We write
(t,x) ;. foo'
= Px
p(t,x)
'
T
then there exists
e -Atx <
If
(t ,x)
E:
e "At x
x
,
T
= XT",x.
Note then that if
T with
J
e-A(t+T) (x+X )dP .
. T x
e
then for all
foo'
= T-t
~
J
e-A(t+T) (x+X )dP
. . T X
Consequently, by straightforward algebra
(t ,x)
(2.4)
E:
f co
e-ATx dP
T x
<=> x ~ sup .L---r--'A:-'-- - H(x)
pO 1- e- TdP
x
where we can restrict
T to run through hitting times of closed sets in
If (2.3) holds,
is a constant and we're finished.
(2.2) holds.
H(x)
Now
foo
and since
rco
X €
~
foo
= {(t,x): t
~H(x)}
is closed and contains no
{(t,x): x ~ x
(a, b)
So let's assume
while
o},
a
~
(t,x)
then there must exist
H(a),
we'd have simultaneously
b ~ H(b),
For
0 Sa
with
<
x negative, if
b < 00 with
x < H(x),
x as determined by (2.2),
S.
.. 17~
and
Q.E,D,
a contradiction.
An
Explicit Characterization of
X
o
for Diffusions
X = {X 1 t
t
our theorem and having continuous paths.
We restict attention to
(2.5)
~ o}
satisfying the hypotheses of
We impose
x <x .
be the time to reach
Let
T
all
A> 0,
o
X
the derivative
a further reqirement
o
Then for
exists as a con..
d:
tinuous function with limit
Theorem (2.2)
Under the hypotheses in theorem (2.1) and the condition (2.5) we can
characterize
X
o
by
(2.6)
X
o
Proof:
If
·e
then
which we re-write as
= sup
x
such that
xHA(x)
~
1 .
-18-
L
-AT
1, e
I
XOdPX
..AT-Xo
dP x
By the Mean Value Theorem of Calculus
where
x
<
y
<
x '
o
Letting
On the other hand, if
o
o ~ x < Xl'
X
e -At x
we have our conclusion
X
x
~
Xl
that
then
I
-A(t+T )
e
Xl dP
x
and repeating our reasoning we have
Q,E.D,
Applications
(I)
X = {X , t
~O} has homogeneous independent increments, continuous
t
paths, then there exists non-negative G(A) with
If
Consequently
so that
Thus, if
m,
X is Brownian Motion with variance
0
2 and drift
then
from which
and
e -Atx
F{t,x)
1' f
X >
- x
=
o
if
x < xo
'
This agrees with Taylor [13],
(II) The stationary Ornstein-Uhlenbeck process has representation
where
W is standard Brownian motion,
Ware independent.
Xo is
and
X
o and
Conditions (2.l) and (2,2) are easily seen to
hold for this process; for (2,2) one uses
We consider the optimization
sup
T~t
·e
n{O,a),
f
e
~T
~
dP rt
~
,x
)
-20-
-A:r
solved by Taylor [13].
The transform
Je
X
o dP
x
which is solved in Darling-Siegert [6] and which 1 for the case
takes the relatively simple form
Straightforward calculations lead to
1 2
- -x
=
d
1 irn dx
xtx o
e 2 0
satisfies
A = 1,
-21-
Thus, from (2.6), we need
1 2
co
f e- It dt
=1
-x O
o-
which has solution
payoff
An
X
.839,
agreeing with Taylor [
].
The
F has form
Explicit Characterization of
X
o
for Compound Poisson Processes
X = {X , t ~o} live on the integers as a pure jump process which
t
has independent homogeneous increments. Thus, if T is the jump time from
Let
x
x to some other state, then
P {T >t}
x x
P {X
x
T
x
·e
e(Z)
We assume further that
(1.8), it is clear that
.
some finite positive
= x + z} = e(Z) ,
Let's assume that
ing.
= e- at
=0
if
LZ2SCZ)
a
e a fixed probability,
Z
~
<: co
0
so that the process is increasso that, using Section (1), Remarks
X qualifies for the hypotheses of Theorem (2.1), in
-22~
fact, the existence of second moments is much more that is needed,
Inter~
preting (2,4) in the present context, it is seen that
where
X
T
is the size of the jump taken at jump time
x
T '
x
Now the right
side in the inequalities above is constant since the process has homogeneous independent increments.
This constant is
-AT
Je
0
I
~o
dP o
=
-AT
1- e
Thus
.
0 dP
I L ze(z)
.
O
is the smallest integer for which
The optimal payoff for the case
F(t,x)
eel)
=1
is
=
These results agree with Taylor [13],
e·
~23~
Remarks
Let
..
c(t)
be smooth and decreasing to zero as
t
-+
00
I
let
X be
a Hunt process and consider
Arguing as before,
(t ,x)
x
€
~
r00i f f
X dP
C(t+T) T (t,x)
J ~J!L
sup
l-f
T>O
If the process
c(t) dP
c (t+T)
(t ,x)
X has homogeneous independent increments, then
Po and the right side above is a function of t
thoroughly treated in the literature has
~
c(t)
alone,
1
and
= l+t
P (t ,x)
=
The case most
X
standard
Brownian motion.
A generalization treated by Walker [15], slightly modified here, has
X standard Brownian motion and
c(t)
The characterization of
r
00
=
1
1
r>2
CA+Bt) r
for this case is
~
sup
T l-CA+Bt)rJ
Using the fact that
X
t
~
a Xt
for
2
a
making this substitution above with
·e
00
iff
xT
(A+B(t+T))r
-=--_"----'e..:-_
CA+Bt)rJ
x
(t,x)er
1
CA+B(t+T))r
X standard Brownian motion, and
a =~A+Bt
B
'
we calculate that
(t,x)
€
r(X)
iff
x ~ K
r
•
~+Bt:
J-B
where
K
r
Thus,
T(X)
= sup
'[
is the hitting time for the square root boundary
some appropriate constant,
The determination of
Kr
KrjA+BBt •. Kr.
is achieved by Walker
[15] and in special cases by Shepp [13], Taylor [12], and others; we won't
pursue this problem here.
We remark only that on the basis of this example
and those considered previously, it is not too much to expect that the time
T(X)
is the hitting time for some smooth curve
G(t),
This assumption is
exploited in the next section,
Section (3).
Weak Convergence in Optimal Stopping
We consider a class of discrete time optimal stopping problems whose
solutions are approximated by solutions of related continuous time problems.
To begin with, suppose
where
X (t)
n
<j>(n)
O.
-+
Let
has
f
t
{X}
n
to be a sequence of processes in discrete time
restricted to the lattice
{m<j>(n), m = 0,1,2, , ,}
be our return function having its usual domain
where
S = [O,(X)] xR
and define
(3.1)
where
F (t,x)
n
T
= sup
f
T
f(T,X (T))dP(t
n
,x )
runs through stopping times on the lattice
{m<j>(n)},
Suppose we
e·
interpolate
CIO~oo)~
so that it is a process in
X
n
x (t)
(3.2)
i.e l
~
we set
= ((m+1)¢(n)-t) X (m¢(n))
¢(n)
n
n
+
(t-m¢(n)) X ((m+l)¢(n))
<I>(n)
n
for
t e [m¢(n), (m"'1) <I>(n)]
Each discrete time process
X
n
t
is Markovian and
satisfies~
for all
t,s
in the lattice {m¢(n)}
(3.3)
for appropriate fixed transition probabilities
4It-
p.
n
For each fixed
(t,x),
the continuous time process
X
n
under initial
(t,x)
Xn conditioned on Xn(t) =x will be called
and the resulting probability on C[O,oo) will
or
the latter denotation being adequate
p ,
x
be denoted either
P
in view of (3.3).
We assume in what follows that there exists a Markov
process
X00 on
(t ,x)
C[O,oo)
governed by a transition probability poo(t,x,dy)
such that
X
n
under all initial
(t,x)
1l.
X
00
where the indicated convergence is
gence - see Billingsley [2] and Whitt [16]
Suppose there exists a closed set
.
·e
satisfies
weak'conver~
t
roo in
S whose hitting time
Too
-26-
We define
Tn
as the approximate hitting time of
T
n
= minimal
hits
X,
n
the~discrete
such that the continuous time process
m~(n)
roo
((m-l)~(n) 1
in
It is reasonable to expect that
time
for
xn'.
time process
(3,4)
roo
i . e,
T
n
X
n
mHn)] ,
is approximately optimal for discrete
1
Fn (t,X) ....,
J F(Tn Xn (Tn ))dP(t , x) ·.
1
We now proceed to lay down assumptions which given this approximation
a precise meaning.
It is not assumed in this development that
lim X (t)
t-+oo n
exists.
Assumptions
(3.5)
For all
n
~
00
and under initial
Thus, we interpret
(3.6)
f(t,x)
is increasing in
tinuous in
(3.7)
f (T 'X (T))
n
There exists
x1
(t,x):
=0
on
{T
=oo}
decreasing in
t
non-decreasing in
t
.
1
uniformly con-
t .
G€C[O,oo),
with
roo = {(t,x); x~G(t)} ,
We interpret
Too =
00
on paths where
Xco(t) < Get),
all
t,
e·
~27~
(3.8)
For all
(t,x) e S
U
{Xoo(Too+S)
P
{(
l
(t ,x)
£>0
0<5<£
{T00 < oo} •
conditional on
(3.9)
For all large
=1
G(T+ S)}}
is excessive for discrete time
n, F00
Foo(m(/>(n) ,x)
f Foo((m+l)<p(n) tY)Pn
~
X,
n
i ~ e.,
(<p(n) ,x,dy) •
Theorem (3.1)
Let
{m<p(n)}
X
whose continuous time interpolation converges weakly to
all initial
tit
for any
be the discrete time Markov process on the time lattice
n
(t.x).
£>0,
Under assumptions (3,5) through
there exists
N(£)
0 S F (t,x)
n
S Foo(t ,x)
~
(3~8)
n~N(£)
such that
X00 under
we have it that
implies
f f(Tn ,Xn·(Tn))dP(t .x)
- f
f(T n ,Xn (Tn ))dP(t ,x )
S e:
Proof:
By Skorokhod's theorem - see Billingsley [2] - there exists a probability triple
(Q,B,P)
supporting random elements
Y: Q-+-C[O,oo). nS oo
n
such that
(3.10)
y
..
y
Since the metric on
C[O,oo)
n
n
~ Xn
-+- Y
00'
all
W E:
Q
is uniform convergence on all closed intervals
it is easily seen from assumptions (3.7) and (3,8) that
, if
is the hitting time of
Sn
G(t)
for the continuous time process
vn ~
all
S
n
n
and if
Yn 1 then
is the corresponding hitting time for
(3.11)
X
Vn
n
S
00
and
Sn
(3.12)
+
{S00 < oo} .
S00 on
Clearly (3.8) implies (3.12) by making hitting times
But then, by continuity of
ous where they occur.
(3.13)
f(S n ,Yn (S n )) + f(S 00 ,Y00 (S 00)) ,
Here we use (3.5) on
{Soo
= co}.
f
all
and
C[O, oo)
Y ,
n
continu-
we have
WE;n.
Now (3.11) demands that
(3.14)
1"
and then Fatou's lemma applied to the right and interpreted for the left
Yields (note V00 = T00):
(3.15)
J f(T
00
,X00 (T00))dP(t ,x )
~
J f(Vn ' Xn (Vn ))dP (t , x)
Now we want to replace
so that we can relate behaviour of dis-
Xn
the uniform continuity of
Xco '
crete time
(3.16)
f
(3.17)
f
in
f(Vn ,Xn (Vn ))
Further, since
and
V by T
n
n
to continuous time
lim
n
ITn -Vn I ~ lj>(n),
t demands that for large
~
and
lj>(n) +0,
n
f(T n ,Xnn
(V)) + E •
G is non-decreasing and
is increasing in
Since
V
n
is the hitting time for
x:
f(T n ,Xn (Tn ))
I
G,
-29-
Consequently;
f f(T
X (T ))dP
00'0000
(t.x)
S
lim
n
J f(Tn'nX (Tn·))dp·
.
(t.x)·
Finally. assumption (3.9) and the properties of least excessive
ants gives
(3.19)
F00
~
F00 (t.x) =
Fn
major~
so that. using (3,18)
f f(T.X
00
00
(T00 ))dP(t .x )
lim
S
n
f f(Tn .Xn (Tn ))dP(t ,x)
Q.E.D.
A Related Result for
0[0. 00)
{X } be a discrete time sequence of integerpvalued processes
n
where X has time lattice {m~(n)}. and where Xn has transitions
n
governed by p where. for all integer x.y;
Let
n
P{Xn ((m+l)~(n)) = ylXn (m~(n)) = xl = pn (~(n) ,x,y) .
Let the continuous time version of
X (t)
n
Each
X
n
m~(n).
X
n
= Xn (m~(n)) if t
Let
X00 be a pure-jump continuous time process which lives on the
non-explosive, i.e.,
"
f
(3.20)
€ [m~(n). (m+l)~(n))
is a pure-jump process with jumps occurring only at time values
integers, hence is a random element in
That is,
be defined by
0[0,(0).
and assume that
X00
is
Xoo has at most finitely many jumps in finite time.
-30 ..
where
{sm}
is the sequence of jump times for
Xoo
and this scqu('nce
satisfies
(3.21)
= oo} = 1,
P {lim s
x n~ n
xn ~ X
Now suppose
on
00
all integer
x ,
this type of weak convergence is
D[O,oo) ;
treated in Whitt [17] and Lindvall [10].
We want then to relate
F
n
to
Foo where, as usual
Fn (t,x)
= sup
T~t
f f(T,Xn-,T))dP(t,x)
,
Assumptions
(3.22)
f(t,x)
is increasing in
x,
decreasing in
all initial
(3.24)
There exists non ...decreasing
G € C [0 ,00)
t,
continuous in
t.
(t,x)
with
roo = {(t,x): x~G(t)}
and we again interpret
all
(3.25)
Too
=
00
on paths where
X00 (t) < G(t) ,
t
For all large
n, F00
is excessive for discrete time
Xn.
Theorem (3.2)
above satisfy
(3.25) hold, then if T
n
Foo(t ,x)
Xn
= minimal
RX
00
m~(n)
= lim
n~
I
and if
on
where
X
n
(m~(n))
~
(3~22)
through
G(t),
we have:
f(T n ,xn (Tn ))dP(t
. ,x ) •
Proof;
and
Yn , n s 00 where Xn R y
n
metric for each path. Since all paths are inte ..
Just as before , we replace
X
n
be
~ Y
in the 0[°1 00 )
00
n
ger valued, and since Yoo executes at most finitely many jumps in finite
Y
time, we see that for a particular path
Y
n
w,
Yn (w)
~
eventually makes exactly the same jumps as
where
n
s
m
n
the time axis, we have
S
n
as in the last theorem.
{S00 < oo},
and since
f(Sn,Yn(S
n ))
-+
looks like
Y
f
-+
S00 on
vious theorem, noting that
00
00
= y (s )
n
s
m
00
00
~
s ,
m"
Since
m
G is
Y00 except for a slight distortion of
{S00 < oo}
where
S , S00 are defined
n
Y00 (S00) for large n on
(S ) =
n n
is continuous in t, we have, given (3.23), that
Also, since
f(S00 1 Y0 0(S0 0)).
J f(T
if and only if
i • e, ,
is the time of the m..th jump and further
continuous and since
Y00 (ui)
T
n
y
Continuing the logic and notation of the pre=
V,
n
we see that
lim
,X00 (T00))dP(t ,x ) S --n
J f(Tn ,Xn (Tn ))dP(t ,x)
and that
so that all conclusions follow •
·e
Q.E.D.
REFERENCES
[1]
Billingsley, P. (1968),
and Sons, Inc,
[2]
Billingsley, P. (1971).
Frobability.
Convergence of FrQbability
Measures~
John Wiley
Weak convergence of measures, Applications in
SIAM Publications, No.5, Philadelphia,
[3]
Chernoff, H. (1968).
221--252.
[4]
Chow, Y.S. (1960).
Optimal stochastic control.
Sankhya, Ser. A
~,
A martingale inequality and the law of large numbers.
!!, 107-111.
Froc. Amer. Math. Soc,
[5]
Chow, Y.S., Robbins, H., Siegmund, D. (1971), Great Expectations,
Theory of Optimal Stopping. Houghton Mifflin Co., Boston.
The
[6]
Darling, D. and Siegert, A. (1953). The first passage problem for a
continuous Markov process, Ann. Math, Stat, ~, 624~639,
[7]
Dynkin, E.B. (1965).
[8]
Dynkin, E.B. (1963). Optimal selection of stopping time for a Markov
process. Soviet Math, !' 627-629.
[9]
Grigelionis, B.I. and Shirgaev, A.N. (1966). On Stefan's problem and
optimal stopping rules for Markov processes. Theor. Frob. Appl .
.!..!., . 541-558.
Markov Process, Vol. 1, Academic Press, New York.
[10]
Lindvall, T. (1973). Weak convergence of probability measures and random
functions in the function space D[O,oo). J. Appl. Frob. l£, 109-121.
[11]
Neveu, J. (1972).
[12]
Shepp, L.A. (1969). Explicit solutions to some problems of optimal stopping. Ann. Math. Stat. 40, 993-1010.
[13]
Taylor, H.M. (1968). Optimal stopping in a Markov process,
Stat. 39, 1333-1344.
[14]
[15]
Martingales
a temps
discret.
Masson et cie,
Ann. Math.
Walker, L. (1974). Optimal stopping variables for stochastic processes
with independent increments. Annals Frob, ~, 309~3l6,
Walker, L, (1974).
Annals Frob,
Otpimal stopping variables for Brownian motion,
~,
3l7~320.
[16]
Whitt, W. (1970), Weak convergence of probability measures on the function
space C[O,oo). Ann. Math. Stat. ~, 939-944,
[17]
Whitt, W. (1970). Weak convergence of probability measurcs on the fun.:ti\~·~
space D[O,oo). Technical Report. Yalc University,
I
!
,,
t
f