Dept. of Math. University of Oslo
Pure Mathematics
No. 23
ISSN 0806–2439
June 2004
Optimal stopping with delayed information
Bernt Øksendal1,2
Revised in May 2005
Center of Mathematics for Applications (CMA)
Department of Mathematics, University of Oslo
P.O. Box 1053 Blindern , N–0316, Oslo, Norway
and
2 Norwegian School of Economics and Business Administration,
Helleveien 30, N–5045, Bergen, Norway
1
Abstract
We study a general optimal stopping problem for a strong Markov process in the
case when there is a time lag δ > 0 from the time τ when the decision to stop is taken
(a stopping time) to the time τ + δ when the system actually stops. Equivalently, we
impose the constraint that the admissible times for stopping are stopping (Markov)
times with respect to the delayed flow of information. It is shown that such a problem
can be reduced to a classical optimal stopping problem by a simple transformation.
The results are applied
(i) to find the optimal time to sell an asset
(ii) to solve an optimal resource extraction problem,
in both cases under delayed information flow.
MSC (200): 93C41, 93E20, 60J25, 91B28
Key words: Optimal stopping, delayed information flow, strong Markov processes, optimal
time to sell, optimal time to stop resource extraction.
1
Introduction
Let Y (t) be a strong Markov process in Rk on a filtered probability space (Ω, F, {Ft }t≥0 ,
{P y }y∈Rn ), where P y is the probability measure giving the law of {Y (t)}t≥0 when Y (0) =
y ∈ Rk . Let δ ≥ 0 be a fixed constant. In this paper we consider optimal stopping problems
of the form
hZ α
i
y
f (Y (t))dt + g(Y (α))
(1.1)
Φδ (y) := sup E
α∈Tδ
0
1
where E y denotes expectation with respect to P y and f : Rk → R, g : Rk → R are given
continuous functions such that
hZ α
i
y
(1.2)
E
|f (Y (t))|dt + |g(Y (α))| < ∞
for all α ∈ Tδ
0
where we interpret g(Y (α)) as 0 if α = ∞. Here Tδ is the set of δ-delayed stopping times,
defined as follows
Definition 1.1 A function α : Ω → [δ, ∞] is called a δ-delayed stopping time if
{ω; α(ω) ≤ t} ∈ Ft−δ
(1.3)
for all t ≥ δ
or, equivalently,
{ω; α(ω) ≤ s + δ} ∈ Fs
(1.4)
for all s ≥ 0
The set of all δ-delayed stopping times is denoted by Tδ .
In other words, if we interpret α(ω) as the time to stop, then α ∈ Tδ if the decision whether or
not to stop at or before time t is based on the information represented by Ft−δ . In particular,
if δ = 0 then Tδ = T0 is the family of classical stopping times and (1.1) becomes the classical
optimal stopping problem, discussed in many texts (see e.g. [Ø, Ch. 10]).
In the delayed case problem (1.1) models the situation when there is a delay δ > 0 in
the flow of information available to the agent searching for the optimal time to stop. An
alternative way of stating this, is that there is a delay δ > 0 from the decided stopping
time τ ∈ T0 (based on the complete current information available from the system) to the
time α = τ + δ ∈ Tδ when the system actually stops. This new formulation is based on the
following simple observation:
Lemma 1.2 (i)
τ ∈ T0 ⇐⇒ α := τ + δ ∈ Tδ
(ii) α ∈ Tδ ⇐⇒ τ := α − δ ∈ T0
Proof. It suffices to prove (i).
First, assume τ ∈ T0 . Then, for t ≥ δ,
{ω; τ (ω) + δ ≤ t} = {ω; τ (ω) ≤ t − δ} ∈ Ft−δ ,
and hence α := τ + δ ∈ Tδ .
Conversely, if α := τ + δ ∈ Tδ then
{ω; τ (ω) ≤ t} = {ω; τ (ω) + δ ≤ t + δ} = {ω; α(ω) ≤ t + δ} ∈ F(t+δ)−δ = Ft ,
and hence τ ∈ T0 .
2
Remark 1.3 In view of this result we see that it is possible to give another interpretation
of problem (1.1), namely
h Z τ +δ
i
y
(1.5)
Φδ (y) = sup E
f (Y (t))dt + g(Y (τ + δ))
τ ∈T0
0
In this formulation the problem appears as an optimal stopping problem over classical stopping times τ ∈ T0 , but with delayed effect of the stopping: If the stopping time τ ∈ T0 is
chosen, then the system itself is stopped at time τ + δ, i.e. after a delay δ > 0.
Note that Tδ ⊂ T0 for δ > 0 and hence
Φδ (y) ≤ Φ0 (y)
and we can interpret Φ0 (y) − Φδ (y) as the loss of value due to the delay of information.
In this paper we show that the delayed optimal stopping problem (1.1) can be reduced
to a classical optimal stopping problem by a simple transformation (Theorem 2.1).
We call α∗ ∈ Tδ an optimal stopping time for the problem (1.1) if
h Z α∗
i
y
(1.6)
Φδ (y) = E
f (Y (t))dt + g(Y (α∗ )) .
0
This paper may be regarded as a partial extension of [AK2], where the geometric Brownian motion case is studied and solved (see Example 3.1), with a more general (Markovian)
delay δ(X) ≤ 0. See also [AK1]. For a related type of problem involving impulse control
with delivery lags see [BS].
2
Optimal stopping with δ-delayed information
We are now ready to state and prove the main result of this paper:
Theorem 2.1 a) Consider the two optimal stopping problems:
hZ α
i
y
(2.1)
Φδ (y) := sup E
f (Y (t))dt + g(Y (α))
α∈Tδ
0
hZ τ
i
y
(2.2)
Φ̃(y) := sup E
f (Y (t))dt + g̃δ (Y (τ ))
τ ∈T0
0
where
(2.3)
g̃δ (y) = E
y
hZ
δ
i
f (Y (t))dt + g(Y (δ)) .
0
Then we have
Φδ (y) = Φ̃(y)
for all y ∈ Rk , δ ≥ 0.
3
b) Moreover, α∗ ∈ Tδ is an optimal stopping time for the delayed problem (2.1) if and only
if
α∗ := τ ∗ + δ
(2.4)
where τ ∗ ∈ T0 is an optimal stopping time for the non-delayed problem (2.2).
Proof. a) Define
(2.5)
J
(α)
(y) = E
y
hZ
α
i
f (Y (t))dt + g(Y (α)) ;
α ∈ Tδ ,
i
f (Y (t))dt + g̃δ (Y (τ )) ;
τ ∈ T0 .
0
and
(2.6)
hZ
J˜(τ ) (y) = E y
τ
0
Choose α ∈ Tδ and put
τ = α − δ ∈ T0
Then α = τ + δ and hence
hZ α
i
(α)
y
J (y) = E
f (Y (t))dt + g(Y (α))
0
i
h Z τ +δ
y
f (Y (t))dt + g(Y (τ + δ))
=E
0
Z τ +δ
hZ τ
i
y
=E
f (Y (t))dt +
f (Y (t))dt + g(Y (τ + δ))
0
τ
oii
h nZ δ
hZ τ
y
y
(2.7)
f (Y (t))dt + g(Y (δ))
,
f (Y (t))dt + E θτ
=E
0
0
where θτ is the shift operator, defined by
θτ {h(Y (s))} = h(Y (τ + s)) for s ≥ 0, for all measurable h : Rk −→ R,
and we have used that
θτ
Z
δ
Z
f (Y (t))dt =
0
τ +δ
f (Y (t))dt.
τ
We refer to [BG] for more information about Markov processes. By the strong Markov
property we now get from (2.7) that
hZ τ
h nZ τ
o ii
(α)
y
y
f (Y (t))dt + E θτ
f (Y (t))dt + g(Y (δ)) Fτ
J (y) = E
0
0
hZ τ
hZ δ
ii
f (Y (t))dt + E Y (τ )
f (Y (t))dt + g(Y (δ))
= Ey
0
0
hZ τ
i
(2.8)
= Ey
f (Y (t))dt + g̃δ (Y (τ )) = J˜(τ ) (y).
0
4
Hence, by Lemma 1.2 (ii),
Φδ (y) = sup J (α) (y) = sup J (α) (y)
α∈Tδ
α−δ∈T0
˜(α−δ)
= sup J
(y) = sup J˜(τ ) (y) = Φ̃(y),
τ ∈T0
α−δ∈T0
as claimed.
b) Suppose τ ∗ ∈ T0 is optimal for (2.2). Define
α∗ := τ ∗ + δ.
Then α∗ ∈ Tδ by Lemma 1.2 and by (2.8) combined with a) we have
∗
∗
J (α ) (y) = J˜(τ ) (y) = Φ̃(y) = Φδ (y).
Hence α∗ is optimal for (2.1).
Conversely, if α∗ ∈ Tδ is optimal for (2.1) a similar argument gives that τ ∗ := α∗ − δ is
optimal for (2.2).
3
Application 1: The optimal time to sell an asset
In this section we illustrate Theorem 2.1 by solving the following problem:
Example 3.1 (The optimal time to sell an asset)
This case (without the jump part) was first solved by [AK1], with a more general (Markovian)
delay δ(X) ≥ 0.
Suppose the value X(t) of an asset at time t is modelled by a geometric Lévy process of
the form
Z
i
h
−
X(0) = x > 0,
(3.1)
dX(t) = X(t ) µ dt + σ dB(t) + z Ñ (dt, dz) ,
R
R tR
where µ, σ and x are constants. Here B(t) and η(t) := 0 R z Ñ (ds, dz) is a Brownian motion
and an independent pure jump Lévy process, respectively, where
Ñ (dt, dz) = N (dt, dz) − ν(dz)dt
is the compensated Poisson random measure of η(·), N (dt, dz) is the Poisson random measure
of η(·) and ν(dz) is the Lévy measure of η(·). We assume that
(3.2)
0 ≥ z ≥ −1 a.s. ν.
5
This guarantees that X(t) never jumps down to a negative value. For convenience, we also
assume that
E[η 2 (t)] < ∞
(3.3)
for all t ≥ 0.
Then by the Itô formula for Lévy processes (see e.g. [ØS]) the solution of equation (3.1) is
h
X(t) = x exp (µ − 12 σ 2 )t + σB(t)
Z tZ
Z tZ
i
(3.4)
+
{ln(1 + z) − z}ν(dz)ds +
ln(1 + z)Ñ (ds, dz) ;
t ≥ 0.
0
0
R
R
We now study the following problem
(3.5)
Φδ (s, x) = sup E s,x [e−ρ(s+α) (X(α) − q)],
α∈Tδ
where E s,x denotes expectation with respect to the probability law P s,x of the time-space
process
dt
s
dY (t) =
; Y (0) =
dX(t)
x
and ρ > 0, q > 0 are constants. We assume that
(3.6)
ρ > µ.
One possible interpretation of this problem is that Φδ (s, x) represents the maximal expected
discounted net payment obtained by selling the asset at a δ-delayed stopping time (ρ is the
discounting exponent and q is the transaction cost).
It is well-known that in the no delay case (δ = 0) the solution of the problem (3.5) is the
following (under some additional assumptions on the Lévy measure ν):
Φ0 (s, x) = e−ρs Ψ0 (x)
(3.7)
where
(
x−q
Ψ0 (x) =
C0 xλ
(3.8)
;
;
x ≥ x∗0
0 < x < x∗0
Here λ > 1 is uniquely determined by the equation
Z
1 2
(3.9)
−ρ + µλ + 2 σ λ(λ − 1) + {(1 + z)λ − 1 − λ z}ν(dz) = 0,
R
and
x∗0
(3.10)
(3.11)
and C0 are given by
λq
λ−1
1
C0 = (x∗0 )1−λ .
λ
x∗0 =
6
The corresponding optimal stopping time τ ∗ ∈ T0 is
(3.12)
τ ∗ = inf{t > 0; X(t) ≥ x∗0 }
Thus it is optimal to sell at the first time the price X(t) equals or exceeds the value x∗0 . We
refer to [ØS, Example 2.5] for details.
To find the solution in the delay case (δ > 0) we note that we have f = 0 and
g(y) = g(s, x) = e−ρs (x − q)
Hence, by (2.2),
g̃δ (y) = E y [g(Y (δ))] = E s,x [e−ρ(s+δ) (X(δ) − q)]
= e−ρ(s+δ) (E x [X(δ)] − q) = e−ρ(s+δ) (x eµδ − q)
(3.13)
= e−ρs+δ(µ−ρ) (x − q e−µδ ) = K e−ρs (x − q̃),
where
(3.14)
K = eδ(µ−ρ)
and
q̃ = q e−µδ .
Thus g̃δ has the same form as g, so we can apply the results (3.7)–(3.12) to find Φ̃(y) and
the corresponding optimal τ ∗ :
(3.15)
Φ̃(y) = Φ̃(s, x) = e−ρs Ψ̃(x)
where
(3.16)
(
K(x − q̃) ;
Ψ̃(x) =
C̃ xλ
;
x ≥ x̃∗
0 < x < x̃∗ ,
with λ as in (3.9). Here x̃∗ and C̃ are given by
(3.17)
(3.18)
λ q̃
λ−1
1 ∗ 1−λ
C̃ = (x̃ ) .
λ
x̃∗ =
The corresponding optimal stopping time for problem (2.2) and (2.1), respectively, is
(3.19)
τ̃ ∗ = inf{t > 0; X(t) ≥ x̃∗ }
(3.20)
α∗ = τ̃ ∗ + δ.
Using Theorem 2.1 we conclude the following:
7
Theorem 3.2 The value function Φδ (y) for the delayed optimal stopping problem (3.5) is
given by
Φδ (y) = Φ̃(y),
where Φ̃ is as in (3.15)–(3.18). The corresponding optimal stopping time α∗ ∈ Tδ is
α∗ = inf{t > 0; X(t) ≥ x̃∗ } + δ.
Remark 3.3 Assume for example that
µ > 0.
Then comparing (3.17) with the non-delayed case (3.10) we see that q̃ > q and hence
x̃∗ < x∗0
Thus, in terms of the delayed effect of the stopping time formulation (see (1.5)), it is optimal
to stop at the first time t = τ̃ ∗ when X(t) ≥ x̃∗ . This is sooner than in the non-delayed case,
because of the anticipation that during the delay time interval [τ ∗ , τ ∗ + δ] X(t) is likely to
increase (since µ > 0). See Figure 1.
X(t)
x∗0
x̃∗
(δ = 0 case)
(delay case)
z
τ̃ ∗
δ
}|
{
τ0∗
α∗ = τ̃ ∗ + δ
Figure 1. The optimal stopping times for Example 3.1 (µ > 0)
8
x∗ =
λq
λ−1
x̃∗ =
λqe−µδ
λ−1
t
4
Application 2: An optimal resource extraction problem
In the no delay case the following example was discussed in [Ø] (continuous case) and [ØS]
(jump diffusion case). Our example models the situation when there is a time lag δ > 0
between the decided stopping time τ ∈ T0 and the time α = τ + δ ∈ Tδ when the result of
the stopping decision comes into effect.
Example 4.1 (Optimal time to stop resource extraction) Suppose the price P (t) at
time t per unit of a resource (oil, gas, . . . ) is given by
Z
i
h
−
P (0) = p > 0
(4.1)
dP (t) = P (t ) µdt + σdB(t) + z Ñ (dt, dz) ;
R
where, as in Example 3.1, µ and σ are given constants and we assume that z ≥ 0 a.s. with
respect to ν.
Let Q(t) denote the amount of remaining resources at time t. As long as the extraction
field is open, we assume that the extraction rate is proportional to the remaining amount,
i.e.
dQ(t) = −λQ(t)dt;
(4.2)
Q(0) = q > 0
where λ > 0 is a known constant.
If we decide to stop the extraction and close the field at a (delayed) stopping time α ∈ Tδ ,
then the expected total discounted net profit J α (s, p, q) is assumed to have the form
(4.3)
α
J (s, p, q) = E
(s,p,q)
h Zα
i
e−ρ(s+t) (λP (t)Q(t) − K)dt + θe−ρ(s+α) P (α)Q(α)
0
where K > 0 is the (constant) running cost rate and ρ > 0, θ > 0 are other constants. The
expectation E (s,p,q) is taken with respect to the probability law P (s,p,q) of the strong Markov
process


 
s+t
s



(4.4)
Y (t) := P (t) , which starts at y = p at time t = 0.
Q(t)
q
The explanation of the quantity J α (s, p, q) in (4.3) is the following:
As long as the field is open (i.e. as long as t < α) the gross income rate from the
production is price times production rate, i.e. P (t)λQ(t). Subtracting the running cost rate
K we get the net profit rate
λP (t)Q(t) − K
for 0 ≤ t < α.
9
If the field is closed at time α the net value of the remaining resources is estimated to be
θP (α)Q(α). Discounting and integrating/adding these quantities and taking expectation we
get (4.3).
We want to find the value function Φδ (s, p, q) and the corresponding optimal delayed
stopping time α∗ ∈ Tδ such that
∗
Φδ (y) = Φδ (s, p, q) = sup J α (s, p, q) = J α (s, p, q)
(4.5)
α∈Tδ
In the case of no delay (δ = 0) it is shown in [ØS, p. 158–162] that if the following relations
between the parameters hold:
0 < θ(λ + ρ − µ) < λ
(4.6)
then the optimal stopping time τ0∗ ∈ T0 is
τ0∗ = inf{t > 0; P (t)Q(t) ≤ w0∗ },
(4.7)
where
w0∗ =
(4.8)
(−r2 )K(λ + ρ − µ)
,
(1 − r2 )ρ(λ − θ(λ + ρ − µ))
r2 < 0 being the negative solution of the equation
Z
1 2
(4.9)
h(r) := −ρ + (µ − λ)r + 2 σ r(r − 1) + {(1 + z)r − 1 − rz}ν(dz) = 0
R
In this case we have
f (y) = f (s, p, q) = e−ρs (λpq − K)
and
g(y) = g(s, p, q) = θe−ρs pq
Thus
g̃δ (y) = E
y
h Zδ
i
e−ρ(s+t) (λP (t)Q(t) − K)dt + E y [θe−ρ(s+δ) P (δ)Q(δ)]
0
Zδ
=
e−ρ(s+t) (λE[P (t)Q(t)] − K)dt + θe−ρ(s+δ) E y [P (δ)Q(δ)]
0
Zδ
=
e−ρ(s+t) (λpqe(µ−λ)t − K)dt + θe−ρ(s+δ) pqe(µ−λ)δ
0
10
(4.10)
(4.11)
(4.12)
h
i
K
= e−ρs {(λ + ρ − µ)−1 λ(1 − e−(λ+ρ−µ)δ ) + θe−(λ+ρ−µ)δ }pq − (1 − e−ρδ )
ρ
−ρs
= e [F1 pq − F2 ], where
F1 = (λ + ρ − µ)−1 λ(1 − e−(λ+ρ−µ)δ ) + θe−(λ+ρ−µ)δ
and
K
F2 = (1 − e−ρδ )
ρ
Therefore, according to Theorem 2.1 we have
(4.13)
Φδ (y) = sup E
y
h Zτ
τ ∈T0
−ρ(s+t)
e
i
(λP (t)Q(t) − K)dt + E y [e−ρ(s+τ ) (F1 P (τ )Q(τ ) + F2 )]
0
The method used in [ØS] to provide the solution (4.7)–4.9) in the no delay case can easily
be modified to find the optimal stopping time τ ∗ for the problem (4.13). The result is
(4.14)
wδ∗ =
(−r2 )K(λ + ρ − µ)e(λ−µ)δ
= w0∗ e(λ−µ)δ .
(1 − r)ρ[λ − θ(λ + ρ − µ)]
We have proved:
Theorem 4.2 The optimal stopping time α∗ ∈ Tδ for the delayed optimal stopping problem
is
(4.15)
α∗ = τδ∗ + δ,
where
(4.16)
τδ∗ = inf{t > 0; P (t)Q(t) ≤ wδ∗ },
with wδ∗ given by (4.14).
Remark 4.3 Note that the threshold wδ∗ for the decision to close down in the case of a time
lag in the action only differs from the corresponding threshold w0∗ in the no delay case by
the factor e(λ−µ)δ .
Assume, for example, that λ > µ. Then we should decide to stop sooner in the delay case
than in the no delay case, because of the anticipation that P (t)Q(t) will probably decrease
during the extra time δ it takes before the closing down actually takes place.
References
[AK1] L. H. R. Alvarez and J. Keppo: The impact of delivery lags on irreversible investment
demand under uncertainty. Manuscript March 1998.
11
[AK2] L. H. R. Alvarez and J. Keppo: The impact of delivery lags on irreversible investment
under uncertainty. European J. Operational Research 136 (2002), 173–180.
[BG]
R. M. Blumenthal and R. K. Getoor: Markov Processes and Potential Theory. Academic Press 1968.
[BS]
A. Bar-Ilan and A. Sulem: Explicit solution of inventory problems with delivery lags.
Math. Operations Research 20 (1995), 709–720.
[Ø]
B. Øksendal: Stochastic Differential Equations. 6th edition. Springer-Verlag 2003.
[ØS]
B. Øksendal and A. Sulem: Applied Stochastic Control of Jump Diffusions. SpringerVerlag 2004.
[S]
A: Shiryaev: Optimal Stopping Rules. Springer-Verlag 1978.
12