BIOMATUf
MAnes TRAINING PR-oGRAM
.
..
EXISTENCE AND EXPLICIT DETERMINATION
OF OPTIMAL STOPPING TIMES
~,
.'.
by
Anthony Mucci
Department of Biostatistics
University of North Carolina at Chapel Hill
•
Institute of Statistics Mimeo Series No. 1170
April 1978
EXISTENCE AND EXPLICIT DETE~MINATION
OF OPTIMAL STOPPING TIMES
Abstract
We consider large classes of continuous time optimal stopping
problems for which we establish the existence and form of the optimal
stopping times.
These optimal times are then used to find approximate
optimal solutions for a class of discrete time problems .
•
Introduction
This paper focuses on three problems in the theory of optimal
stopping:
(1)
The determination of a broad class of continuous time optimal
stopping problems for which optimal stopping times exist.
This
problem is treated in Section (1).
(2)
The explicit determination of existing optimal stopping times.
This problem is treated in
(3)
Se~tion
(II).
The connection between weak convergence and optimal stopping,
aspects of which are treated in Section (III).
A more detailed survey of the contents is given directly below.
We remark here that we made no use of the "free-boundary" approach to
optimal stopping which one finds, for instance, Van Moerbeke [
..
Chernoff [
Let
].
X= {\, t
~
O}
be a standard Markov process.
f: [0, 00) x R ~ R be continuous.
probabili ty
P{·/ \ = xl.
for
f
where
X and
T
] and
Let
P(t ,x)
Let
represent the conditional
We define the optimal stopping problem
as the problem of determining
runs through stopping times and
Too,
F, T00
where
if it exists, is a
stopping time that realizes the supremum. We prove in Section (I)
..
•
that for a broad class of processes
the optimal time
Too
X and continuous functions
exists and is the hitting time of the closed
f
set
r00
= {F =
n.
Our principal result is Theorem (1.3).
This result
covers at least the following cases:
(A)
X is standard Markov,
f(t,x)
= c(t)x
where
c
is continuous
non-increasing and
and
Jsup
s~O
(B)
c(t+s) IX t .ldP(t ) <
+s
,x
X is standard Markov,
continuous
00,
f(t,x) = x - at
for
all
(t,x)
a> 0,
X has
paths, and
f°
SUP(Xt +s -a(t+s)tdPCt ,x ) <
00,
all
(t,x)
s~
(e)
b > 0,
X is a pure jump process with positive bounded jumps;
g: [0,00)
-+-
R is continuous, and
P (t,x)
and
{~: g(X.) - aX. - bs
f(t,x)
= g(x)
=-
= I,
00 }
a > 0,
- ax - bt where
all
(t,x)
In Section (II) we treat the subclass of (A) where
A> O.
c(t)
=-At,
We show by elementary arguments that for a class of processes
X which includes, in particular, martingales and processes with
stationary independent increments, there exists a positive
that
Too
is the hitting time of
r 00
If
= {F=f} =
[x o'
{(t,x):
00)
or
X is a diffusion the value
Xo such
for the process
t
X, i. e.
=oo}
can often be characterized as a
solution to
xH(A,x)
=I
H is a smooth function related to Laplace transforms of first
where
passage times.
(2.3).
There results are treated in Theorem (2.1),
(2.2),
Applications are given covering Brownian Motion, Ornstein-
Uhlenbeck processes, and Poisson processes.
Our solutions in these
three cases agree with those obtained previously by Taylor [
];
our methods are different.
In Section (III) we consider the relationships between weak
convergence and optimal stopping.
converging weakly a process
Fn (t,x)
= sup
Suppose we can find
Too'
n
=
00
above.
T~t
Can we use
X00 .
We have a sequence of proceSSes,
Xn-,
We define
If(T,Xn(t))dP(t,X),
the optimal stopping time for the case
Too as an approximation to
stopping time in those cases above where n
is large?
T ,
n
An
the optimal
affirmative
answer is given in Theorems (3.2) and (3.3) under restrictions
motivated by Theorem (3.1).
paraphrased as follows:
The latter theorem may be loosely
The optimal hitting
T00
defined by
where
X is brownian motion and
where
G is continuous, postive, non-decreasing.
c
is decreasing, is given by
Similar results,
subject to slight modifications, would hold for most processes
stationary independent increments.
that when
motion,
c(t)
we have
=
1
(A+Bt)r
G(t)
=
, r >
K~+Bt·
X with
OUr methods show, in particular,
1
and when
,
~
,
k
X is brownian
constant.
B
OUr methods yield the square root form almost immediately; the
determination of
Shepp [
].
k requires other tools - see Walker [
] or
(I)
Some Optimal Stopping Theory
Let
X = {X , t
t
~ O}
be a real valued Markov
process with the
following properties
X is strong Markov,
(1.1)
px{X =x} = 1,
all
o
(1.2)
Ttf
right continuous with left limits and
x € R.
X has the Feller property, i.e., for all bounded continuous
f,
is continuous where
(1.3)
X is extended quasi-left continuous, i.e., for all increasing
T ,
sequences of stopping times,
n
if
Tn
r'"
then
T00 ,
X
exists
T00
P
x
a.e. and
P a. e., all
x
x.
We remark that (1,1), (1.2) and the version of (1.3) restricted to
~ + ~
n
00
on
{Too < oo}
defines a standard process; see Dynkin [
].
Z = {(t,X ), t ~O} be the space-time version of X and let
t
be non-negative and continuous on [0,(0) xR. We will call f a
Let
f
return funation.
(1.4)
For all
We assume
f
satisfies
(t,x):
f
sup f(t+s,X t )dP(t ) <
s~O
+s,x
00
where, of course, the integral is taken as the canonical version of
the conditional expectation, i.e.,
..
Remarks
We assume throughout this section that
a.e., i.e., f is continuous at infinity.
is well-defined
P
(t ,x)
Statement of the Problem
To determine the optimal payoff F and the optimaZ Btopping time
where
T00
F(t,x) = sup Jf(T X )dP
T~t
'T
(t,x)
(1. 5)
(1.6)
•
Remarks
(I)
The most interesting processes
hypothesis central to our development.
and
= ff('T'oo,XToo)dP(t,X)
X will not 'satisfy (1.3), a
However, a re-definition of
X
will often suffice to transform the problem stated in (1. 5)
f
into one where
X satisfies (1.3).
For example, if
X is standard
-At
f(t,x) = e
x+,
Brownian motion, and if
then the problem (1.5) is
....
identical to that problem involving the process X and the function
....
f
t
where
~
00
....
-At
....
+
....
X = e
x
and f(t,x) = x, for now Xt ~ 0 a.e. as
t
t
and so (1.3) obtains. This construction will be elaborated on in
the next remark.
(II)
Let
X be a Markov process which satisfies (1.1), (1.2) and
where
(1. 7)
(b)
Xo is square integrable
(c)
y
= {Yt ,
for all
t
~
o}
t
~
0, s
E{ (Yt +s - Yt ) 2 1 \
(d)
We
list
two
Iztl
$
Kt,
classes
all
is a martingale where,
}
~
$
t
0
and for some fixed
K > 0:
Ks
~
0
of processes which have these properties:
Class (1)
X
t
where
~ Xo + ftO(X
)dW
o ,s s
+ Jtm(x )ds
0
o,m are bounded, W is Brownian Motion,
grable, independent of
s
X
o
is square
inte~
W.
Class (2)
x = {Xt ' t ~ o} is pure jump, range the integers, having
intensities {A} and transition probabilities {Q} where, if T
x
x
x
is the time for a jump from state x:
P {T
x x
px{~
(1. 9)
-A t
x
= x+Y} = Qx{y}
x
sup Ax <
x
e
=e
> t}
CIO
sup flQx{dY} <
x
00
•
Our concern in later applications will be with determining
F, Too
for
processes of Class (1) or Class (2) subject to a damping factor, i.e.,
we consider
(1.10)
where
C decreases in
t.
In order to use our soon to be developed
theory on such problems, where, as explained in the last remark, we'll
rename our process as
X = {C(t)Xt ,
t ~ o}
we must determine that (1.3), (1.4) hold.
and our return
occur in Walker [
].
f(t,x)
=
+
x ,
We state a result - see Mucci,
[ p. 7] for a proof - which gives us what we need.
.
,.,.
Similar results
Proposition (1.1)
Let
X satisfy (1,7) and let
be non-increasing,
C: [0,00) -+ [0,00)
Then
(A)
If
t
all
Zt - 0,
P(t , x)
(1.11)
°
Xs =°} = 1
JooC2 (t)dt
and
0
{~.: c (s )
then
< 00
all
(t, x)
and
J
(8)
~
For general
sup c(t+s) IXt IdP(t ) < <xl,
.0
+s,x
ull
(t,x)
S2
Z,
(1.11) continues to hold provided
tc(t)
(1.12)
° as
-+
t
-+ 00 •
Development of the Theory
Let (1.1) through (1.4) hold.
of
F, T00 defined in (1.5).
(1.13)
(A)
We will determine the general form
Definitions
G: [0,00) x R -+ [0,00)
and if, for all
all
(t,x) ,
G(t,x)
is called exaessive if
~
s
~
G is measurable
0:
= fG(t+s,Xt+s)dP(t,X)
(TsG)(t,x)
and
G(t,x)
= lim
(TSG) (t,x)
54-0
excessive
.
G is called an exaessive majorant of
(8)
An
(C)
An excessive majorant
G of
f
f
if
G
is called the Zeast exaessive
majorant if, for all excessive majorants
H, of
f,
H ~ G.
~
f.
Grigelionis-Shirgaey [
majorant,
(1.14)
t
= t~O.
sup T f l'
n-
n
] shows that the continuity of
(1.2) made
f
has a least excessive
defined by the recursion
f O = f, f
Taylor [
] determine that
f OC)
f OC)
f
= t fn
and the Feller property
lower semi-continuous.
Definitions
roo
= {(t,x):
Too = inf t
~
0
The lower semi-continuity of
hitting time,
TOC),
= foo(t,x)}
f(t,x)
fOC) makes
is well-defined, since
rOC) a closed set, and the
foo(oo,X oo)
= f(oo,Xoo),
for we
have
fooCoo,Xoo)
(1.15)
= lim
s-t<lCl
exists
fooCs,X s )
P
Ct,x)
a.e.
and
Pet ,x ){f00 (oo,X00)
= f(oo,X
00
)}
=1
A proof of (1.15) can be adapted from Neveu [
].
We begin with the
observation that
(1.16)
foo(t,x)
S
J sup
s~O
This we can show inductively.
fO(t,x)
~ I
f(t+s,X t
Clearly
sup fCt+s,X t )dP Ct ) .
s~O
+s
,x
Assume this inequality obtains with
•
)dP(t )
+s,x
f
n
on the left.
Then
= sup
f
s~o
f n+ l(t , X)
f (t+s,X t )dP(t )
n
+s
,x
:>; sup
If
= SUp
f SUp f(t+r,X t +r )dP( t,x )
s~o
s~O
sup f(t+s+r,X t
)dP(t
x )dP( )
+s+r
+5, t+s
t,x
r~O
r~s
SUp f(t+s,Xt+s)dP(t,x)
s~O
f n + f 00 .
Ineqality (1.16) now follows from
Next, fix
t < to < r,
let
Br = a{x ' s:>; r} ,
and let
s
be a conditional expectation relative to the measure P
E(t ,x) {-I Br }
(t, x)
field
B.
Then, using (1.16):
r
foo(r,X r ) s
(1.16a)
~
Letting
r
(1.16b)
~
we have
00,
lim foo(r,X )
r
~
Letting
to
~
~
E(t,X){:~~
s~
lim E(t
~
,x
~
~
foo(r,X r )
~
f(r,X ),
r
0
a.e.:
){sup f(s,X )\B }
s~to
s
r
and using continuity of
00
f(s,X s ) IBr }
E(t ,x ){su>tp f(s,X s ) IB r }
p(t,x)
lim f 00 (r,X r )
Since
and the
f
f(oo,X 00)
= sup f(s,X )
s
>t 0
s_
and (1.3):
p(t,x)
a.e.
the other direction is obvious.
Theorem (1, 1)
foo(t,x) ::;
f foo(:roo,XToe»dP (~,x)'
Proof:
Dynkin [ ] shows the following.
foo(t ,x)
S
with
f
f(t ,x) + d,
bounded:
then with
T£
all
If
(t,x)
£>0
E
[O,oe»
and
x
R •
r £ ={(t,x):
the hitting time for
r £ and
f E + foo and that
f E is closed, that
It is clear that
that by our quasi-left continuity assumptions
Letting
E ~
0
~
f::; f oo and since
f,
foo
f
a
fa
a
so
Further,
and con-
we have
We now consider the unbounded case, sub-
Set, for each
a> 0:
= min(f,a)
= least
f a = {fa
f
X,
~Too
is excessive and bounded, we've established
our result in the bounded case,
ject as usual to (1,4),
E
X .
T00
and using the quasi-left continuity of
tinuity and boundedness of
Since
X
TE
T
excessive majorant of
f
a
= f a}
= {f ~ f }
a
Ta = hitting time for
f
N = hitting time for
a
ra
a
.....
c f
that N ST
and the b~a implies f ~ fa by
a'
b
a
a
a
.....
.....
properties of least excessive majorants, Let f oo = t f
as a ~ 00 •
a
.....
.....
.....
Clearly f 00 is excessive and f oo ~ f, therefore f 00 ~ f 00 . On the
Note that
other hand,
..
f
f 00 ~f a'
hence
.....
f 00 ~fa'
from which
f 00 =f00'
so that
for if we set
N00
by lower semi-continuity of
which implies
(Noo'XN )
E
=t
Na'
....
f .
b
roo'
then
N00 'S T00 whi Ie
But then
thus
~
Noo
00
Too
Next,
....
since
f
is excessive and
a
....
Since
f
a
T
a
<!: N •
a
is bounded, we have by Dynkin's result,
J fa(Ta'~a)dP(t,X) = f a (t,x)
so that
I f(Na,XNa )dP(t,x) ~ fa(t,x)
.
Now, (1.4) allows us to use Lebesgue dominated convergence
J f(Too,~oo)dP(t.X) = f ;;:
= ~~:
f f(Na,X Na )dP(t,x)
....
=f
Since
f co
~
f
and
f co
00
~ lim f (t,x)
atoo a
(t,x)
is excessive, the result follows.
Q.E.D.
Theorem Cl.2)
= fooCt,x)
FCt,x)
=
I fCToo,~}dP
Ct,X)'
all
Ct,x)
EO
{O,oo) x R
•
Proof:
FCt,x)
= ~~i f f(T,XT)dPCt,x)
=
~
~
I
foo(Too,XToo)dPCt,X)
~f
= fooCt,x)
sup
f fooCT,XT)dPCt,x)
sup
J
T~t
T~t
fCToo,XToo)dPCt,x)
fCT,~)dPCt,x)
= F(t,x) .
A
Q.E.D.
Generalization
Consider the following situations:
(I)
where
g
To determine
F, Too where
~ O}
is pure jump,
is continuous, non-negative, and where, for
F(t,x)
= sup
T~t
CII)
N = {Nt' t
To determine
paths, where
g
f (g(T,NT)
F, Too where
x=
A >0
ANT)dP(t,x)
{X , t ~ O}
t
has continuous
is continuous, non-negative, and where for
FCt,x)
= sup f
T~t
A> 0:
CgCT,XT) - AT)dP
) .
Ct ,x
In both cases we have an added cost factor which removes the integrand
from the positive and, in a sense, bounded, type to which our previous
theory applied.
Since situations like (I), CII) are very natural, we
will extend our results to include a broad class of such problems.
(1.17)
Definition
We will call a process
extended standard.
Dynkin [Harkov Processes I, 104] defines as stan-
dard those processes
left continuous.
(1.18)
Let
G.
Result
X which satisfies (1.1), (1.2), (1.3)
X which satisfy (1.1), (1.2) and are
quasi~
We will need a result from Dynkin:
[Dynkin, Markov Processes I, Theorem 10.3]
X be a standard,
G be open,
T be the exit
G
time from
x = {Xti\T
' t~O} is standard.
G
The replacement of standard by extended standard leaves this result
Then
valid.
We remark also that the statement of Theorem (10.3) in Dynkin is
somewhat finer, involving measure theoretic conditions; however, Dynkin
shows that there can always be met if
(1.19)
Let
X is standard.
Definitions
f: [0,00] x [_00,00)
decreases moderately with
~
[_00,(0)
be continuous.
We say that
f
X if there exist two increasing sequences
!
{a }, {bnl where a
a <b < a
00
n+I ' and
n
n
' n
n
(a) If A = {f> -an }, then [0,(0) x (~oo,oo) c U An
n
n
is the exit time from A , then f(a,X ) ~ -b
(b) If a
n a
n
n
n
n
_ 00
The idea behind this definition is that although f(t,X ) +
t
as t + 00 is possible, this convergence doesn't take.place too rapidly.
of positive constants
.
Theorem (1.3),
Let
moderately with
(A)
J1f(t+S'
X.
X be extended standard and let
Suppose for all finite
xt +s ) IdP(t,x)
<
00,
all
5
~
(t,x):
0
f
decrease
(8)
I
(C)
For all sequences
sup
f+ (t+s,X
s~O
t
+~
)dl'(t
T
,x
) <
00
,
of increasing stopping times,
n
we have
Let
F(t,x)
Too
Then
foo
is closed,
sup
T~t
f f(T,Xr)dP(t ,x)
hitting time of
=
Too
=
f
is a well-defined, possibly extended hitting
time, and
Proof:
Let
A
n
then the closure
satisfies
= {f> -a}
n
is the exit time from A,
n
n
of A union its exit points (a ,X )
n
n an
x(n)
=
By Dynkin's theorem
{\"a} is
B ,
n
Bn c An+ I'
where, if a
n
B.
extended standard when confined to
is bounded below by
Let
F (t,x)
n
•
n+l'
let
n
= T€[t,a
sup
n
fn
= {(t,x)
T
= hitting
n
f
restricted to
we can use Theorem (1,2) as follows:
-a
(t,x) € B;
n
Since
]
ff(T,X-)dP(t
-I
)
= Pn (t,x)}
€B,
f(t,x)
n
time of f
,x
n
for paths beginning in
B
n
Then
F
n
of
r n is closed in B,T
is
n n
a well ",defined hitting time with
T s; (J ,
n n
is lower semi",continuous on Bn , and is th(" least excessive majorant
f for the process X(n) restrictod to B, and
n
(t ,x)
Fn
Note that
increases on
A
'''nO
,
all
n
any finite
(t,x)
and any stopping time
B
•
n
F00 = t Fn
so that
~nO'
well",defined and lower semi",continuous on
E:
[0,(0) x (_00,00).
T beginning at
is
Now for
(t,x),
(C)
demands that
(1. 20)
But then we have
tion of
F00 = F,
F
for (1, 20) shows
demands FOl) :S F.
n
We want now to be able to define
s; F
00
while the defini-
F
Too
as the hitting time of {F = f},
and since we want this definition to make sense at infinity, we'll estabUsh that
(1.21)
p(t,X){~': F(s,X s ) =f(CO,Xoo)}
= l,
all finite
(t,x)
We refer the reader back to the proof of (1.15), noting here only that
(1.16a) holds by reducing to
Fn
(l.16b) holds from the fact that
and then
passing to limits, while
sup f(s,X)
s
is bounded above by
s~to
sup f+(s,X ) and below by
S
s->t 0
Too is well",defined since
f(to'X t );
Let's next consider the times
o
then we use (A),(B).
Thus,
is well-defined.
Tn'
Clearly, for
n
~nO
and on paths
A , T is increasing in n, thv.s Tn! T*, and our proof
nO
n
will be completed if we can show that T. = Too, for, by (C):
starting in
e
Fix finite
(t,x)
p(t,x) {f(T*,~}
at
(t, x) ,
E:
A
nO
Now
= _ co} = 0,
there exists
(T. ,X ) EA.
T•
n
F(t,x) >
_
thus
00,
so that for almost all paths beginning
n,
possibly dependent on the path, with
Then by extended quasi-left continuity, there exists
m such that the entire path,
~
{(s,X ): s
s
lim F (T ,X_ )
- - m n -,.
n
E
~
n
[t,T.]}
cA.
m
But then
F (T.,X_ )
m
-I *
where the last inequality depends on the lower semi.continuity of
F
m
and where the next to last inequality depends on the monotonicity of
Fn
in
n
on
A.
m
Thus,
T* t:: Too'
On the other hand,
T sT
n
001\011'
Q.E.D.
(1.22)
Remarks on Discrete Time Optimal Stopping
The theory developed above holds for Markov
a discrete time lattice
processes that move on
{kr, k = 0,1,2, ... }, r > 0 provided the discrete
analogues of hypotheses involving transitions
p(t,x,dy)
are used.
point of fact, the theory falls out much more simply and with fewer
hypotheses and we will not develop it here,
•
In
(II)
Applications
X = {X ,
t
Let
in time, Le.,
t~O}
be a standard process which is homogeneous
p(t,x) = Px '
specify othePiJ)ise.
This assumption wiU hoZd unless we
Assume further that for all
(a)
fsup e -A(t+s) IXt +s IdP(t ,x )
(b)
P
s~O
A > 0:
<
00
(2.1)
{lim .-A(t+S)X
(t,x) s-+<>o
t+s
.o}
= 1
Then, we have from Theorem (1.3) in conjunction with Remarks (1.6)
that there exists
roo'
a closed set, and its hitting time,
Too
such that
(2.2)
F(t,x) = sup
T~t
Je
-AT
~dP (t,x)
It will be convenient to use a different notation for stopping times.
Since
Too
is a hitting time, we can restrict our stopping time
considerations exclusively to hitting times, and given any closed
r,
we will let
(t ,x) ,
and
X
T
T be the time it takes to enter
r,
starting from
the increment in distance required.
We then re-write
(2.2) in the form
F(t,x)
where
Too
= sup
e-
f
A(t
)
+T (x+X )dP
=fe -A(t+T
00
)
(x+x
T X T
is the time it takes to travel from
)dP
00
(t,x) into
X
f oo
Now
r
(t,x)
(2.4)
is specified by
00
£
r
00
<=> e -At x ~ sup
T>O
Je -A(t+T) (x+x
)dP
T
X
which we write in the e4uivalent from
if
(2.5)
(t,x)
£
r
00
<=>
x~H(x)
x 10
=
if
In most of the analysis which follows we will consider
x =0
x 1 0, the case
x = 0 being either simi lar or irrelevant.
r
The description (2.5) demands that
(2.6)
r = {(t,x):
If we use
for the hitting time of
00
x~H(x)
we have
I
e
H(x) =
(2.7)
-
or
t
r
00
= oo}
starting from
(t,x),
h
0 X dP
TO
X
---~--
1 (2.8)
have form
00
I
e
-ATo
x#o
dP x
Remarks
We introduce some definitions and notation which will facilitate
the statement and proof of the next theorem.
·
.
"
...
a<x<b
Let
(1)
exit time from
(~.b),
travelled from
x
have
x+ x
T
(II)
Two
a <b
T be the
and let
x.
we will say that
states
Let
x be the distance
T
r f, along all paths where T<OO we
starting from
at exit time.
{a. b},
£
X.
be in the state space of
X doesn't skip states.
in the state space of
non-contiguous if there exists
c
X will be called
X and
in the state space of
c £ (a,b).
Theorem (2.1)
Let
X
= {\,
t
~
O} be a standard process which satisfies (2.1)
and which doesn't skip states.
OS; a < b
s;
x £ (a,b)
00
where
a
where, if
and
b
Suppose further that for all
are non-contiguous, there exists
T is the exit time from
(a,b), starting at
ph <oo}=l
x
(2.9)
X ~
Then there exists a finite
roo
=
x
{(t,x):
e
J
-AT
(x+x) dP
T
X
o ~ 0 such that
x~xO
or
t=oo}
We have
e -At x
if
e -At X
Je"-AT °dP
F(t ,x) =
where
o
is the hitting time for
x~x
x '
O
o
x
if
x<X
o
starting from
x.
x:
BIOMATffEMATICS TRAINING PROGRAM
Proof:
Note first that
x < 0 implies
t
(t ,x)
can always do better than the negative value
Xoo '
on to
Thus, if
x<H(x),all
time,
TO'
to exit
xE(a,b).
to travel from
(a,b),
(2.7), for
e
-At
x simply by continuing
is not of the proposed form, then (2.6) demands
f oo
the existence of non-contiguous
but
for from (2.1), we
f oo '
O~a<b~oo
a~H(a),
b~H(b)
We note next that for all such
r
into
(t,x)
starting at
with
x,
00
x,
is the same as the time
X does not skip states.
for
the
From
x E (a,b)
x < H(x)
On the other hand, for some
equivalence
=
and using (2.9) and the
x E (a,b),
T = TO:
X
~ e
J
-AT
0 (x+x
TO
) dP
x
so that we reach a contradiction.
Thus
f oo
= {(t, x):
x
o= '
F(t,x)
= 0,
If
is finite.
we would have
to the contradiction
(2.10)
x~
00
X
or
o
t
=oo}
f 00 = {(t,oo): t ~O}
then
all finite
F(t,x) <e
At
and it remains to show that
x
for
t
(t,x)
with
(t,x)
t
and using (2.1)
t
f 00 ,
leading
f 00 , x>O.
Remarks
The class of processes
quite large.
X for which (2.9) holds is probably
It includes, for instance, super martingales for which
T,
the first half of (2.9) holds, for we have, with
0
~
a < x < b,
and
T
the exit time from (a,b):
X ~
>
J(x+x) dP
fe
-AT (x+x) dP
TXT
X
An Application to Diffusions
Let
X = {X , t ~ O}
For
0
(I)
P h < oo}
x y
be a difusion.
t
x
~
~
y < 00
we ass wne
= 1,
T
the hitting time for
y
y
-AT
(II)
f
e
dP
(1 II)
(IV)
x,
is continuous in
x
is continuous in
H(A,y)
=
lim: fe
A>O
all
x,
all
x < y,
A>O
-AT
xty
x
y dP
exists and is continuous in
x
y
Theorem (2.3)
Let
X be a diffusion satisfying (2.1), (2.9) and (I) through
(IV) above.
Assume
call it
Then
f 00
(2.11)
Further,
xH(A,x) = 1 has a unique non-negative solution,
X
= {t, x):
x <::
X
o
or
t
=oo}
o > O.
Proof:
First
X
o > 0,
for if
(0, t) e: f oo
then for all
T
we would have
but using (I) with
1"=1"
we would then have
y'
o 2: ye
.
-AT
f
YdP a > 0 ,
a contradiction.
If
x<xo '
and if
is the time to hit
La
starting from
x, then, from (2.5):
(2.12)
Using the Mean Value Theorem on the denominator, there exists
Z E
(x, xo)
with
d
= (x o - x) dz
1 -
•
f -AT adP
e
z
so that, (2.11) becomes
-AT
X <
J
d
-
dz
Now let
On
xl'
x~x
a
°dP
e
x
Je -ALadP
z
so that
the other hand, if
starting from
x,
X
Our hypothesis (III, IV) leads to
o
$X $X
1
,
and if
L
l
is the time to reach
then since the best policy at
x is to stop at
x,
we have
-At
ex>
f ->"(t+T 1)
c
xl
i . c. ,
dP){
-AT
x ?
J
ex 1 - x) e
1 -
f
e
-AT
0 dP x
ldP
x
Repeating our reasoning above, we have
Q.E.D.
(2.13)
Example
The stationary Ornstein-Uhlenbeck process has the representation
•
where
and
W is standard brownian motion,
X
is
o
n(o,b),
and
X
o are independent. The transform
~(x,xO) = f e->"T OdPx
x<x
o
satisfies
ab~"
-axl/!'
= AI/!
This equation is solved in Darling-Siegert [
a = b = >.. = 1,
].
When
the solution takes the relati voly simple form
W
xt - t%
oo
(X'X
o) = f
c
dt
o
The process
X can be shown to satisfy the hypotheses of Theorem (2.2),
consequently the optimal
is the unique
X
o
00
I
o
xt -
t
value for which
2
'12
e
dt
=1
Straightforward calculations reduce this to the equation
X
2
o+
_1/2 X2
xOe
0
=1
00
I
-x
for which the solution is
_112t 2
e
dt
0
o-
X
.839,
agreeing with Taylor [ ].
Theorem (2.3)
Let
X = {Xt ,
t~O}
be a standard process satisfying (2.1) and
having independent stationary increments.
p {-r
X
or
y
<oo} > o.
t =oo}.
Then there exists finite
Assume for
X
o>0
with
x<y
that
r ={(t, x) :
00
Proof:
P iT <oo} > 0,
Since
(t,x)
€
,x
y
(t,x)
we see that
€
r
00
~~>
x> O.
if and only if
roo
x
(2.14)
The assumption that
-AT
x dP
f
T>O - - - - -
sup e
T
X
X has stationary independent increments implies
that the right side above is a constant not dependent on
constant be
(2.15)
2
As in the last theorem,
x.
Let this
is necessarily finite.
Applications
(1)
a.
Thus
Then
so that if
Let
X ={\,
t
~ O}
be Brownian Motion with drift
m variance
X has continuous paths and stationary independent increments,
TO
is the hitting time of
have
where
G( )
from which it follows immediately that
x '
O
starting from
x < xO'
we
-M
and
e
=
F(t,x)
This agrees with Taylor [
(2)
Let
X
-At x
= {X t ,
].
t~O}
be a stationary independent increment
pure jump process with state space the non-negative integers and with
all jumps positive.
times, and if
exists fixed
.
More specifically, if T is generic for jump
x, z are non-negative integers, with
a> 0
and some fixed probability
z > 0,
then there
8 on the posi ti ve
integers with
P {T>t} = e- at
x
We will assume
2
E z 8(z) <co,
from which it follows fairly easily
that conditions (1.7) are met and Theorems (1.2), (2.3) are available.
Letting
TO
be the time to exit
stopping value, we see that
X
o- 1 <
,
I
e
-AT
0 z dP _1
TO Xo
where
is the optimal
However,
TO
= T~
the jump time from starting point
X
o - 1,
so that
we have
x - 1 <
o
(2.16)
Likewise, since
X
o
is optimal:
(2.17)
where again we are using
given starting point.
is
~ E ze(z),
A
so that
T generically as the jump time from any
One calculates that the right side of (2.17)
X
o
is the smallest integer with
(2.18)
For the case
e (1) = l,
we have as optimal payoff:
e- At x
(2.19)
F(t,x)
if
x s
X
o
=
e
and this agrees with Taylor [
].
-At
if
Weak Convergence Related to Optimal Stopping
(III)
Let
assume
tinuous.
X = {X , t~O} be brownian motion with non-negative drift. We
t
P{Xo=O} = 1. Let c:[O,oo) ~ (0,00) be strictly decreasing, conWe want to determine
roo' Too where
(3.1)
We will be able to use the characterization of Theorem (1.3) for
roo pro-
vided, for instance
p(O,O){~~ c(t)X t
(3.2)
sup
J t~O
The restriction to
exists and is finite} = 1
c(t)lxtldP(O 0) < 00 •
'
in (3.2) is allowable since X has stationary
p(O,O)
independent increments.
Conditions (3.2) will hold if, for instance, in
analogy with Proposition (1.1), we have
(3.3)
lim tc(t)
= M < 00
t~
We restrict
c
(3.4)
c',
so that not only (3.3) holds but also
the derivative of c,
is continuous with
strictly decreasing absolute value, and
Theorem (3.1).
Let
X be brownian motion with non-negative drift; let
satisfy (3.3) and (3.4).
c
Then roo' the optimal hitting set for the problem
defined in (3.1) is given by
r : ((t,x): x ~G(t) or
t :oo}
00
where
G is non-decreasing
(I)
For all
(II)
s>O,
all
O~r~t~s:
G(t) - G(r)
~
where M is a universal constant and
Proof:
(t,x) € f 00
(3.5)
PX{T<OO} = 1,
and we can
B:[O,oo)
~
~
f C(t+T) (x+xT)dP(t,x) = F(t,x)
sup
T>O
with
T the time to trave 1 from
therefore
re~write
F(t,x) > 0.
hence
and that the fact that
T,
the hitting time of
T',
y,
we have
implies
~
sup
T>O 1 ...
x dP
c (t)
I
T
(t,x)
C(t+T) dP
c(t)
(t,X)
f,
X
has stationary independent increments gives
starting from
the hitting time of
r~
(t,x),
(t,x),
the same stochastic
starting from (0,0).
This allows us to re-write (3.6) as
(3.7)
x>O,
can be restricted to hitting times of closed sets,
T
f,
structure is
to
(3.5) as
(t,x)€foo <=> x
Now we know that
x
(t,x) € foo
C(t+T)
(3.6)
(0,00).
if and only if
c(t)x
Given any x < y,
M
B(5) (t-r)
(t , x)
€
Further, (3.4) implies
f 00 <->
X ~
G(t) =
sup
T>O
f
C(t+T)
c(t) XTdP(O,Or
1-
f c (t) dP (0 , 0)
C(t+T)
(3.8)
s ~ t,~
so that
G(t)
Next, if
c(s+r)
c (s)
is non-decreasing in
O<x<G(t),
I
c(t+'t'o)
X dP
c(t)
TO (0,0)
C(t+TO)
1 - J crt)
dP(O,O)
G,
1
to define
starting from
(t,x).
(3.11)
G(O)
x=-Z-,
f
c(t+'t')
c(t) dP(O,O)
G(O)
is the time to travel from zero to -2-'
That is, we are free
G through stopping times which are strictly bounded away
zero so that the denominator above doesn't get too small.
(II) .
Fixing
G with
G(t) = sup
T2:T
T
G(O) > 0.
-'7-:-'or--'
1 1 -
where
It is also clear that
--..--,:--~---
is the time to rea.ch
(3.10)
c( t,)
G:
G(t) =
we can re-define
c(t+r)
the reasoning which led from (3.5) to (3.7) allows the
following definition of
(3.9)
t.
~
We set
G(t,T)
J
C(t+T)
crt) X dP(O,O)
=-------1 -
I
C(t+T)
crt) dP(O,O)
We first get bounds on G(S,T) - G(t,T),
5 ~
t.
from
Let's now prove
By straightforward calculations
~ Jsu /c(s+r) - c(t+r)1 Ix IdP
r~~
S
c(s)
c(t)
r
2Icc'(~oll- cls) -It-sl
I ~~g
(0,0)
c(r)lxrldP(O,O)
for some universal
=M- ~
c (s)
Likewise
M>
°.
f C(S+T)
I M I I
c(t) dP(O,O) c(s) dP(O,O) ~ C(S) t-s
If C(t+T)
where we again use
for a generic universal constant, not necessarily the
M
same from case to case.
But then,for all
T~ T
l
B(T,S)
T is as in (3.10): G(S,T) - G(t,T) ~
l
M
C(S+T)
and the latter quanwhere
= (1 2
C(s) dP(0,0))2,
c (s) • B T,s
tity, with T ~ Tl , is bounded away from zero. Finally, given E > 0,
~
t
~
s
~
So
choose
T~ T1
where
f
so that
G(S)
G(S,T)
S
+ C .
We then have
G(s)
$
G(t,T)
M - Is-tl
+ E + ~-------­
2
c (s)B(T ,s)
~
where
G(t)
+ E
s-t
+~
B So
Q,E.D.
e
Remarks
(3.12)
Let
(I)
B> O.
let
X be standard brownian motion;
c (t) =
1
(A+Bt)r
1
, r > 2' A > 0,
Then
•
X
dP
(A+B(t+T))r
(0,0)
f
(A+Bt) r ---~~..::-...::..:-.::------l_(A+Bt)r f
1
r dP(O 0)
,
(A+B(t+T))
T
G(t) = sup
T>O
Using the fact that
X ~
ax
r
ria
2 for
X standard brownian motion, setting
'"
. t egra 1 above, we have
a =jA+Bt
-,r- and sub
stltut1ng
1n th e 1n
G(t)
= Kr
• JA+:t
where
X
sup
r = T>O
K
T dP
(0,0)
f (l+T)r
1 -
I
1
(l+T)r
dP
(0,0)
We see then that
The constant
[
Kr is determine by Walker [
] and in special cases by Shepp
] and Taylor [
]; we will not pursue this problem here.
(II)
The last theorem, although restricted to Brownian motion with non-
negative drift, can be extended, with minor changes in hypotheses and conelusions, to a large class of processes with stationary independent incre•
ments.
Again, we will not develop these generalizations here .
Weak Convergence
For each
n,
X = {X (t), t = m<p(n), m=0,1,2, ... } be a discrete
let
n
n
time Markov process whose time parameter lines on the lattice
divides <p(n),
n
<p(n) = 2- . We assume
<p(n+l)
<Pen)
Xn
f: [0,(0) x R
[0,(0)
~
(3.14 )
0
~
E
be continuous.
where it is to be understood that
for instance, we might have
00,
Pn'
i.e.,
We define
(n)
J
,x
)
T runs through stopping times on the
pen)
(t,x) is the probability on the process Xn
In order that extended times T be available
and where
under initial state
~
f(T,XT)dP(t
T~t
n
{m<p(n)}
n
where
dylX n (t) = x} = pn (x,dy) .
F (t,x) = sup
lattice
as
is governed by a transition
p{Xn (t+<p(n))
(3.13)
Let
and
{m<P(n)}
(t,x).
we'll assume always that
pen)
(3.15)
(t, x)
For each n,
{f(ooX(oo))=O}=l
, n
extend the process
'
Xn
all
(t,x),
to
c[O,oo)
all
n.
by the standard inter-
polation:
(3.16)
for
t
X (t) = (Cm+l)<pCn)-t) X Cm<PCn)) + Ct-m<PCn)) X ((m+l)<p(n))
n
<pen)
n .
.Cn)
n
E
[m<p(n),(m+l)<p(n)].
We will continue to use
interpolated process and we'll also use
Xn(t) = x.
transition PooCt,x,dy)
X,
00
as notation for the
p~~~x) for Xn conditioned on
Where no ambiguity arises we'll write
Now assume there exists
Xn
p(t,x)
for
(n)
p(t,x)'
a standard continuous path Markov process with
such that
X
n
!l
Xoo as n
~ 00
where the intended convergence is weak convergence on
initial
•
(t,x)
(t,x)
where
in the state space of X
is on the
~(n)
n,
i. e . I under all
lattice, all large
n.
For treatments
of weak convergence see Billingsley [
p
We'll assume that
(t,x
) {f
(00
under all
for large
n
t
C[O,oo)
'
] and Whitt
X (co))
00
=O}
::: I
'
].
all
and that
(t ,x)
Too
exists where
(3.17)
and Too
the least excessive majorant under
Fro
r ={f=F}
is the hitting time of the closed set
to
Fn through Too
with some assumptions.
poo of
00
f.
00
,
and
Foo
is
Our objective is to relate
in a manner to be made precise below.
We begin
Assumptions
(3.18)
For all
n S
Thus, we interpret
(3.19)
f(t,x)
tinuous in
(3.20)
00
(3.21)
= 0,
f(oo,Xn(co))
increases in
x,
in the space-time range of Xn :
(t ,x)
as already indicated above.
decreases in
t,
and is uniformly con-
t.
There exists
G E C[O,co)
roo
•
and for all
For all
conditional on
= {( t, x):
(t,x):
{Too < oo}.
non-decreasing in
x ~ G(t)
or
t
= oo}
t
with
.
, (3.22)
For all large
n, F00
is excessive for discrete time
X ,
n
i.e. ,
•
(3.23)
(I)
Comments on the assumptions
The determination of
f oo
though non-decreasing, continuous
G
is
motivated by Theorem (3.1).
Assumption (3.21) prevents
(II)
f oo '
from moving away from
X00
r
once it hits
00
Xoo is brownian
This is a rather weak assumption, true, for instance, when
G has less than infinite slope everywhere - a proof is easily
motion and
established by the local law of the iterated logarithm.
Let's now define
Tn
n'
minimal
=
{
00
roo for the
X.
discrete time process
T
n
as the approximate hitting time of
m<jl (n)
3
if no such
Our main result is that
X (t)
n
E
r
for
t
€:
((m-~)<jl(n)
,m<jl(n)
00
m exists .
is approximately optimal for the problem defined
T
n
in (3.14).
Theorem (3.2).
xn ,
range of
Let
large
Xn
gX
n.
00
under all initial
in the space-time
(t ,x)
Assume (3.18) through (3.22).
Then, for all such
(t,x) :
F00 (t,x)
Proof:
= lim
F (t,x)
n-+<Xl n
= lim
~
n~
ff(T n ,Xn (Tn ))dP(t ,x ) .
By Shorokhod's theorem - see Billingsley [
bability triple
such that
(n,B,p)
supporting random elements
] - there exists a pro-
yn : n +
C[O,oo), n
~
00
•
Y ~ X
n
n'
(3.25)
Yn (w)
Since the matric on
[O,t
C.[O,oo)
all
w
E
Q .
is uniform convergence on all closed intervals
o] , it is easily seen from assumptions (3,20) and (3.21) that if
is the hitting time of
S
n
Y00 (w)
+
for the continuous time process
G(t)
is the corresponding hitting time for
vn ~
(3.26)
all
S
n
n
n
and if
then
Y
n'
n s
X
V
00
and
Sn
(3.27)
S00 on
+
{S 00
>
oo} .
Clearly, (3.21) implies (3.27) by making hitting times
ous where they occur.
(3.28)
But then, by continuity of
f(S n ,Yn (S n ))
Here we use (3.18) on
{Soo
+
f(S 00 ,Y00 (S00)) ,
= co}.
ff(Vn ,Xn (Vn))dP(t ,x )
(3.29)
and
f
all
C[O,oo) - continu-
Y ,
n
we have
wd1.
Now (3.26) demands that
=
ff(S n ,Y n (S n ))dP,
all
n soo
and then Fatou's lemma applied to the right and interpreted for the left yields
(note
Voo =
T0 ).
0'
(3.30)
•
Now we want to replace
so that we can relate behaviour of dis-
crete time
X.
co
Xn
Vn by Tn
to continuous time
the uniform continuity of
(3.31)
f
in
f(Vn ,Xn (V))
n
t
:s;
Since
IT n -Vn I
:s; <p(n) ,
demands that for large
f(Tn ,Xn (V))
n
+ €
•
n
and
<p(n) +0,
Further, since
and
f
G is non-decreasing and
is increasing in
V
n
is the hitting time for
G,
x:
(3.32)
Consequently:
~
Jf(Too,Xoo(Too))dP(t,X)
(3.33)
-lim
n
Jf (T n ,Xn (Tn )) dP ( t,x ) .
Finally, assumption (3.22) and the properties of least excessive majorants
gives
Foo
~
F
n
so that, using (3.33)
~
lim F (t,x)
-- n
n
A Related Result for
Let
where
by
{X}
n
X
n
P
Foo(t,x) .
Q.E.D.
0[0,00)
be a discrete time sequence of integer-valued processes
has time lattice
{m¢(n)},
where, for all integer
n
~
and where
X
n
has transitions governed
x,y:
P{X n ((m+l)¢(n)) = ylXn (m¢(n)) =x} = Pn(x,y)
Let the continuous time version of
Each
m¢(n).
X
n
X
n
be defined by
is a pure-jump process with jumps occurring only at time values
Let
Xoo be a pure-jump continuous time process which lives on the
integers, hence is a random element in
non-explosive, i.e.,
0[0,00),
and assume that
Xoo
is
Xoo has at most finitely many jumps in finite time.
1t
•
. (3.35)
Note:
•
We'll assume also that the probability under all intial
(t ,x)
that
xn g X
Now suppose
00
treated in Whitt [
Xoo
jumps at some fixed
on
0[0,(0);
= sup
F (t ,x)
n
T~t
I
>t
is zero.
this type of weak convergence is
] and Lindvall [
F00 where, as usual
5
].
We want then to relate
f(T,Xn(T))dP(t,x) , n s
00
F
n
to
•
Assumptions
x,
(3.36)
f(t,x)
is increasing in
(3.37)
Pet ,x ){lim
f(s,x n (s))
s-+oo
(3.38)
There exists non-decreasing
decreasing in
=o}. =I
t,
all initial
G C[O, ) •
roo = {(t,x):
x~G(t)
continuous in
t.
(t,x)
with
t = oo}
or
.
and we again interpret
(3.39)
Too =00 on paths where
X00 (t) < G(t),
X.
n, F00 is excessive for discrete time
For all large
all
n
Theorem (3.3)
above satisfy
(3.39) hold, then if
Tn
=
Xn ~ X00 on
minimal
m<j>(n)
D[0, ),
where
and if (3.35) through
Xn (m<j>(n))
G(t),
~
have
.
= lim
n-+oo
F (t,x)
n
= n-+oo
lim
I fCTn ,xn (Tn ))dP Ct ,x )
.
we
t.
Proof:
xn fl
We remark, before entering into particulars, that
intended as weak convergence under all allowable intial
(t,x).
X
00
is
Also, note
(3.35) constitutes an assumption which substitutes for (3.21) of the last
theorem in that (3.35) implies that for small
{T
co
s,
Xco(Tco+s)
~
G(Too+s)
on
< co}.
and
X be Y , n s; co where X Ry
n
n
n
n
metric for each path. Since all paths are integer
Just as before, we replace
Y + Y00 in the 0[0,00)
n
valued, and since Yco executes at most finitely many jumps in finite time,
we see that for a particular path
w, Yn(w)
eventually makes exactly the same jumps as
where
s
n
m
Yoo(w)
+
if and only if
Le.,
Yoo '
is the time of the m-th jump and further
continuous and since
Y (w)
looks like
n
the time axis, we have
Sn
{S00 < oo}
Soo on
+
as in the last theorem.
Also, since
{S00 < co},
is continuous in
and since
f(S n ,Yn (Sn )))
+
f
f(S co ,Y00 (5 co)).
vious theorem, noting that
Y00 (w)
Y (S )
n n
t,
s
n
m
+
00
s .
m
Since
G is
except for a slight distortion of
where
~
Y
n
n
co
Yn (s m) = Y00 (s m)
y (S )
00
co
"
S,
Sco are defined
n
for large
n
on
we have, given (3.37), that
Continuing the logic and notation of the preT
n
= V
n'
~
we see that
lim ff(T X (T ))dP
n
n' n n
(t,x)
and that
so that all conclusions
fol~ow
•
Q.E.D.
•
REFERENCES,
•
•
[11
Bi 11 ings 1ey, P. (1968). COrlve:r'genae of P'1 obabi U ty Measures, John
Wiley and Sons, Inc,
[2]
Billingsley, P. (1971). Weak convergence of measures. AppZiaations in
ProbabiZity, SIAM PubZiaations, No.5, Philadelphia.
[3]
Chernoff, H, (1968).
221-252.
[4]
Darling, D. and Siegert, A. (1953), The first passage problem for a
continuous Markov process. Ann. Math. Stat. 24, 624-639.
[5]
Dynkin, E.B. (1965).
[6]
Dynkin, E.B. ,(1963). Optimal selection of stopping time for a Markov
process. Soviet Math. !, 627~629,
[7]
Grigelionis, B.I. and Shirgaev, A.N. (1966), ~l Stefan's problem and
optimal stopping rules for markov processes. Theor. Probe AppZi.
!!., 541-558 .
[8]
Lindvall, T. (1973). Weak convergence of probability measures and random
functions in the function in the function sapce D(O,). J. AppZ.
Prob. 10, 109-121 .
[9]
Mucci, A.G. Explicit Characterization of Optimal Stopping Times.
Technical Report, Institute of Statistics Mimeo Series #1115,
University of North Carolina, Chapel Hill.
•
l
Optimal stochastic control.
Sankhya~
Ser. A 30,
Markov Process, Vol, 1, Academic Press, New York.
Martingales l temps discret.
[10]
Neveu, J. (1972).
Masson et cie.
[11]
Shepp, L.A. (1969). Explicit solutions to some problems of optimal stopping. Ann. Math. stat. 40, 993-1010.
'
[12]
Taylor, H.M. (1968). Optimal stopping in a Markov process.
Stat. 39, 1333-1344.
[13]
Van Moerbeke, P.L.J. (1974). Optimal Stopping and free boundary problems. Pro. of the Conf. on Stochastic Differential Equations,
(July 1972) Edmonton (Alberta). Rooky Mountain Math. J. 4, 3,
539-578.
[14]
Walker, L. (1974). Optimal stopping variables for stochastic processes
with independent increments. AnnaZs Prob. ~, 309-316.
[15]
Walker, L. (1974), Optimal stopping variables for Brownian motion.
AnnaZs Prob. ~, 317-320.
Ann. Math.
[16]
Whitt, W, (1970), Weak convergence of probab~lity measures on
the function space C[O,oo) \ Ann, Math, Stat, ~, 939-944,
[17]
Whitt. W. (1970), Weak convergenc of probability measures on
the function sapce D[O,oo), Tehcnica1 Report, Yale University.
•
•