Utility Maximization with Constant Costs
Christoph Belak
Department IV – Mathematics
University of Trier
Germany
Joint work with Sören Christensen (University of Hamburg) and Frank Seifried (University of Trier).
11th Bachelier Colloquium, Métabief, France
January 16, 2017
The General Impulse Control Problem
Consider an Rn -valued system X = X Λ controlled by an impulse control Λ =
{(τk , ∆k )}k∈N as follows:
dX(t) = µ(X(t))dt + σ(X(t)) dW (t),
t ∈ [τk , τk+1 ),
X(τk )= Γ(X(τk −), ∆k ),
where
• the stopping times τk do not accumulate, i.e. P[limk→∞ τk > T ] = 1,
• the impulses ∆k are chosen from a set Z(X(τk −)) ⊂ Rm .
The General Impulse Control Problem
Consider an Rn -valued system X = X Λ controlled by an impulse control Λ =
{(τk , ∆k )}k∈N as follows:
dX(t) = µ(X(t))dt + σ(X(t)) dW (t),
t ∈ [τk , τk+1 ),
X(τk )= Γ(X(τk −), ∆k ),
where
• the stopping times τk do not accumulate, i.e. P[limk→∞ τk > T ] = 1,
• the impulses ∆k are chosen from a set Z(X(τk −)) ⊂ Rm .
The objective is to maximize
hX
i
Λ
Λ
V(t, x) = sup E
K(Xt,x
(τk −), ∆k )1{τk ≤T } + g(Xt,x
(T )) .
Λ∈A(t,x)
k∈N
The Quasi-Variational Inequalities
The martingale optimality principle of stochastic control suggests that the value
function V can be linked to the quasi-variational inequalities (QVIs)
min −∂t V − LV, V − MV = 0
on [0, T ) × Rn ,
V(T, ·) = g
on Rn ,
The Quasi-Variational Inequalities
The martingale optimality principle of stochastic control suggests that the value
function V can be linked to the quasi-variational inequalities (QVIs)
min −∂t V − LV, V − MV = 0
on [0, T ) × Rn ,
V(T, ·) = g
on Rn ,
where
• the operator L is the infinitesimal generator given by
LV(t, x) , µ(x)> Dx V(t, x) + 12 tr σ(x)σ(x)> D2x V(t, x) ,
The Quasi-Variational Inequalities
The martingale optimality principle of stochastic control suggests that the value
function V can be linked to the quasi-variational inequalities (QVIs)
min −∂t V − LV, V − MV = 0
on [0, T ) × Rn ,
V(T, ·) = g
on Rn ,
where
• the operator L is the infinitesimal generator given by
LV(t, x) , µ(x)> Dx V(t, x) + 12 tr σ(x)σ(x)> D2x V(t, x) ,
• and the operator M is the maximum operator given by
MV(t, x) , sup V t, Γ(x, ∆) + K(x, ∆) .
∆∈Z(x)
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
A Candidate Optimal Control
Observe that
• if V(t, x) > MV(t, x), an impulse in state (t, x) cannot be optimal, and
• if V(t, x) = MV(t, x), an impulse in state (t, x) is expected to be optimal.
I = {V = MV}
C = {V > MV}
Problem: Verification requires a C 1,2 -solution of the QVIs.
A Formal Optimal Stopping Problem
By the dynamic programming principle, we expect that
V(t, x) = sup E MV τ, Xt,x (τ )
τ ∈Tt
A Formal Optimal Stopping Problem
By the dynamic programming principle, we expect that
V(t, x) = sup E MV τ, Xt,x (τ ) = sup E G τ, X(τ ) ,
τ ∈Tt
τ ∈Tt
which is nothing but an optimal stopping problem with reward G , MV.
A Formal Optimal Stopping Problem
By the dynamic programming principle, we expect that
V(t, x) = sup E MV τ, Xt,x (τ ) = sup E G τ, X(τ ) ,
τ ∈Tt
τ ∈Tt
which is nothing but an optimal stopping problem with reward G , MV.
The general theory of optimal stopping lets us expect that
• V equals the smallest superharmonic function V dominating G, and
A Formal Optimal Stopping Problem
By the dynamic programming principle, we expect that
V(t, x) = sup E MV τ, Xt,x (τ ) = sup E G τ, X(τ ) ,
τ ∈Tt
τ ∈Tt
which is nothing but an optimal stopping problem with reward G , MV.
The general theory of optimal stopping lets us expect that
• V equals the smallest superharmonic function V dominating G, and
• if V is lower semi-continuous and G is upper semi-continuous, then V = V
and the first hitting time of the set {V = G} = {V = MV} is optimal.
A Formal Optimal Stopping Problem
By the dynamic programming principle, we expect that
V(t, x) = sup E MV τ, Xt,x (τ ) = sup E G τ, X(τ ) ,
τ ∈Tt
τ ∈Tt
which is nothing but an optimal stopping problem with reward G , MV.
The general theory of optimal stopping lets us expect that
• V equals the smallest superharmonic function V dominating G, and
• if V is lower semi-continuous and G is upper semi-continuous, then V = V
and the first hitting time of the set {V = G} = {V = MV} is optimal.
Remark: Under standard assumptions, G = MV is upper semi-continuous if V is
upper semi-continuous. That is, to verify optimality, continuity of V should suffice!
The Verification Theorem
Let H be the set of upper semi-continuous functions h : [0, T ] × Rn → R with
The Verification Theorem
Let H be the set of upper semi-continuous functions h : [0, T ] × Rn → R with
(H1) h is superharmonic with respect to the uncontrolled state process,
The Verification Theorem
Let H be the set of upper semi-continuous functions h : [0, T ] × Rn → R with
(H1) h is superharmonic with respect to the uncontrolled state process,
(H2) h dominates the reward, i.e. h ≥ Mh,
The Verification Theorem
Let H be the set of upper semi-continuous functions h : [0, T ] × Rn → R with
(H1) h is superharmonic with respect to the uncontrolled state process,
(H2) h dominates the reward, i.e. h ≥ Mh,
(H3) h satisfies the terminal condition h(T, ·) ≥ g on Rn .
The Verification Theorem
Let H be the set of upper semi-continuous functions h : [0, T ] × Rn → R with
(H1) h is superharmonic with respect to the uncontrolled state process,
(H2) h dominates the reward, i.e. h ≥ Mh,
(H3) h satisfies the terminal condition h(T, ·) ≥ g on Rn .
Define V : [0, T ] × Rn → R to be the pointwise infimum of the members of H.
The Verification Theorem
Let H be the set of upper semi-continuous functions h : [0, T ] × Rn → R with
(H1) h is superharmonic with respect to the uncontrolled state process,
(H2) h dominates the reward, i.e. h ≥ Mh,
(H3) h satisfies the terminal condition h(T, ·) ≥ g on Rn .
Define V : [0, T ] × Rn → R to be the pointwise infimum of the members of H.
Verification Theorem
If H 6= ∅, then V exists and V ≥ V. If V is continuous and the candidate
optimal impulse control defined in terms of {V = MV} and {V > MV}
is admissible, then (up to integrability) we have
V=V
and the candidate optimal impulse control is optimal.
The Verification Theorem
Let H be the set of upper semi-continuous functions h : [0, T ] × Rn → R with
(H1) h is superharmonic with respect to the uncontrolled state process,
(H2) h dominates the reward, i.e. h ≥ Mh,
(H3) h satisfies the terminal condition h(T, ·) ≥ g on Rn .
Define V : [0, T ] × Rn → R to be the pointwise infimum of the members of H.
Verification Theorem
If H 6= ∅, then V exists and V ≥ V. If V is continuous and the candidate
optimal impulse control defined in terms of {V = MV} and {V > MV}
is admissible, then (up to integrability) we have
V=V
and the candidate optimal impulse control is optimal.
Note: If V is known to be continuous, then V ∈ H and things become easy.
Continuity of V
How can we prove the continuity of V?
Continuity of V
How can we prove the continuity of V?
Stochastic Perron Method: Squeeze V between a subsolution bigger than V and
a supersolution smaller than V and apply a comparison principle:
Continuity of V
How can we prove the continuity of V?
Stochastic Perron Method: Squeeze V between a subsolution bigger than V and
a supersolution smaller than V and apply a comparison principle:
(1) Show that V is an upper semi-continuous viscosity subsolution of the QVIs.
Continuity of V
How can we prove the continuity of V?
Stochastic Perron Method: Squeeze V between a subsolution bigger than V and
a supersolution smaller than V and apply a comparison principle:
(1) Show that V is an upper semi-continuous viscosity subsolution of the QVIs.
(2) Approximate V from below by a sequence {Vk }k∈N of iterated optimal stopping problems (by restricting to the problem to at most k impulses).
Continuity of V
How can we prove the continuity of V?
Stochastic Perron Method: Squeeze V between a subsolution bigger than V and
a supersolution smaller than V and apply a comparison principle:
(1) Show that V is an upper semi-continuous viscosity subsolution of the QVIs.
(2) Approximate V from below by a sequence {Vk }k∈N of iterated optimal stopping problems (by restricting to the problem to at most k impulses).
(3) Show that the limit V , limk→∞ Vk is a lower semi-continuous viscosity
supersolution of the QVIs.
Continuity of V
How can we prove the continuity of V?
Stochastic Perron Method: Squeeze V between a subsolution bigger than V and
a supersolution smaller than V and apply a comparison principle:
(1) Show that V is an upper semi-continuous viscosity subsolution of the QVIs.
(2) Approximate V from below by a sequence {Vk }k∈N of iterated optimal stopping problems (by restricting to the problem to at most k impulses).
(3) Show that the limit V , limk→∞ Vk is a lower semi-continuous viscosity
supersolution of the QVIs.
(4) Then V ≤ V ≤ V. Now apply viscosity comparison so that V ≥ V and hence
V=V=V
is continuous.
Application: Portfolio Optimization with Constant Costs
Successful application: Portfolio optimization with constant costs.
Application: Portfolio Optimization with Constant Costs
Successful application: Portfolio optimization with constant costs.
• The market is multidimensional; prices are driven by factor processes.
• Transaction costs: γi |∆i | + Ki .
• Maximize utility of terminal wealth for general lower bounded, possibly nonconcave utility functions.
Application: Portfolio Optimization with Constant Costs
Successful application: Portfolio optimization with constant costs.
• The market is multidimensional; prices are driven by factor processes.
• Transaction costs: γi |∆i | + Ki .
• Maximize utility of terminal wealth for general lower bounded, possibly nonconcave utility functions.
Additional Difficulties:
• Constrained state space!
• Set of impulses possibly empty!
• The maximum operator does not preserve continuity!
Application: Portfolio Optimization with Constant Costs
Successful application: Portfolio optimization with constant costs.
• The market is multidimensional; prices are driven by factor processes.
• Transaction costs: γi |∆i | + Ki .
• Maximize utility of terminal wealth for general lower bounded, possibly nonconcave utility functions.
Additional Difficulties:
• Constrained state space!
• Set of impulses possibly empty!
• The maximum operator does not preserve continuity!
But still: The method adapts very well. We even can show admissibility of the
candidate optimal controls.
Thanks for your attention!
Belak, Christensen, Seifried (2017):
A General Verification Result for Stochastic Impulse Control Problems
To appear in SIAM Journal on Control and Optimization
Belak, Christensen (2017):
Utility Maximization in a Factor Model with Constant and Proportional Costs
Available at: www.belak.ch/publications/