The Uniqueness of Equilibrium for Time-Inconsistent
Stochastic Linear–Quadratic Control
Ying Hu, Hanqing Jin, Xun Yu Zhou
To cite this version:
Ying Hu, Hanqing Jin, Xun Yu Zhou. The Uniqueness of Equilibrium for Time-Inconsistent
Stochastic Linear–Quadratic Control. 2015.
HAL Id: hal-01139343
https://hal.archives-ouvertes.fr/hal-01139343v1
Submitted on 4 Apr 2015 (v1), last revised 28 Jan 2015 (v2)
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
The Uniqueness of Equilibrium for Time-Inconsistent
Stochastic Linear–Quadratic Control
Ying Hu∗
Hanqing Jin†
Xun Yu Zhou‡
April 2, 2015
Abstract
We prove the uniqueness of an equilibrium solution to a general time-inconsistent
LQ control problem under mild conditions which ensure the existence of a solution.
This is the first positive result on the uniqueness of the solution to a time inconsistent
dynamic decision problem in continuous-time setting.
Key words. time-inconsistency, stochastic linear-quadratic control, uniqueness of equilibrium control, forward–backward stochastic differential equation, mean–variance portfolio
selection.
AMS subject claaification 91B51, 93E99, 60H10
1
Introduction
Time inconsistency in dynamic decision making is often observed in social systems and
daily life. The study on time inconsistency by economists can be dated back to Strotz
[10] in1950’s, who initiated the formulation of time inconsistent decision making as a game
between incarnations of the player herself.
The game formulation is quite easy to understand when time setting is (finitely) discrete.
When time setting is continuous, the formulation can be generalized in different ways. It
is still not clear which is the best one among different definitions of a solution to the time
inconsistent decision problem. Mathematically, both the existence and the uniqueness of a
solution make a definition more acceptable. Although it is common that a game problem
∗
IRMAR, Université Rennes 1, 35042 Rennes Cedex, France. The research of this author was partially
supported by the Marie Curie ITN grant, “Controlled Systems,” GA 213841/2008.
†
Mathematical Institute and Nomura Centre for Mathematical Finance, and Oxford–Man Institute
of Quantitative Finance, The University of Oxford, Oxford OX2 6GG, UK. The research of this author
was partially supported by research grants from the Nomura Centre for Mathematical Finance and the
Oxford–Man Institute of Quantitative Finance.
‡
Mathematical Institute and Nomura Centre for Mathematical Finance, and Oxford–Man Institute
of Quantitative Finance, The University of Oxford, Oxford OX2 6GG, UK. The research of this author
was supported by a start-up fund of the University of Oxford, research grants from the Nomura Centre for Mathematical Finance and the Oxford–Man Institute of Quantitative Finance, and GRF grant
CUHK419511.
1
admits multiple solutions, the time inconsistent decision problem is a decision problem for
one player, and hence, even if the control in the solutions is not unique, an identical value
process for all solutions sounds more reasonable.
Unfortunately to our best knowledge, neither existence nor uniqueness of a solution for
general time inconsistent decision problem is available yet in continuous-time setting. Yong
[12] and Ekeland and Pirvu [4] studied the existence of equilibrium solutions, with their
own definitions for equilibrium solutions, for the time inconsistence arising from hyperbolic
discounting. Grenadier and Wang [5] also studied the hyperbolic discounting problem in
an optimal stopping model. In a Markovian system, Björk and Murgoci [2] proposed a definition of a general stochastic control problem with time inconsistent terms, and proposed
some sufficient condition for a control to be a solution by a system of partial differential
equations. They constructed some solutions for some examples including an LQ one, but it
looks very hard to find not-too-harsh condition on parameters to ensure the existence of a
solution. Björk, Murgoci and Zhou [3] also constructed an equilibrium for a mean-variance
portfolio selection with state-dependent risk aversion. Basak and Chabakauri [1] studied
the mean-variance portfolio selection problem and got more details on the constructed solution. In our previous paper [7], we generalized the discrete-time game formulation for
an LQ control problem with time inconsistent terms in a non-Markovian system slightly
different from the one in Björk and Murgoci, and constructed an equilibrium for quite
general LQ control problem, including a non-Markowvian system.
The study of the uniqueness is even less available in literature. Vieille and Weibull [11]
studied the non-uniqueness for a finitely discrete system because of the non-uniqueness of
optimal solution in backward steps. Otherwise we do not know any other results on the
uniqueness for the solution in a continuous-time other than trivial cases (e.g., the time
consistent control problem), neither positive results nor negative ones.
In this paper, we prove that the solution we obtained in [7] is unique. This result not
only justifies our definition of a solution to the time inconsistent LQ control problem, but
also justifies the game formulation of a time inconsistent dynamic decision problem. It also
sheds a light on the search of conditions on the uniqueness of the problem.
The rest of this paper is organized as follows. In Section 2, we recall the formulation of
the LQ control problem without time consistency studied in our previous work [7]. Then
we develop an equivalent characterization of a solution by a system of stochastic differential
equations in Section 3. Finally in Section 4, we prove that the solution obtained in [7] is
the unique solution to our LQ control problem in cases when a solution is constructed in
[7].
2
Problem Formulation
The Linear-Quadratic control problem without time consistency is the same as the one in
our previous work [7]. Here we recall it for the convenience of reading, where we inherit
notations for different spaces.
We consider the same LQ control problem with the controlled system in a finite time
horizon t ∈ [0, T ],
(2.1)
dXs = [As Xs +
Bs′ us
+ bs ]ds +
d
X
[Csj Xs + Dsj us + σsj ]dWsj ;
j=1
2
X0 = x0 ,
where W = (W 1 , · · · , W d )′ is a standard d-dimensional Brownian motion on a filtered
probability space (Ω, F , (Ft )t∈[0,T ] , P) with filtration generated by W , A is a bounded deterministic function on [0, T ] with value in Rn×n . The other parameters B, C j , D j are all
essentially bounded adapted processes on [0, T ] with values in Rl×n , Rn×n , Rn×l , respectively; b and σ j are stochastic processes in L2F (0, T ; Rn ). The process u ∈ L2F (0, T ; Rl ) is
the control, and X is the state process valued in Rn with initial value x0 ∈ Rn .
At any time t with the system state Xt = xt , our aim is to minimize
Z T
1
1
△
Et
[hQs Xs , Xs i + hRs us , us i]ds + Et [hGXT , XT i]
J(t, xt ; u) =
2
2
t
1
(2.2)
− hhEt [XT ] , Et [XT ]i − hµ1 xt + µ2 , Et [XT ]i
2
over u ∈ L2F (t, T ; Rl ), where X = X t,xt ,u , and Et [·] = E [·|Ft ], Q and R are both given
non-negative definite and essentially bounded adapted processes on [0, T ] with values in
Sn and Sl , respectively, G, h, µ1 , µ2 are all constants in Sn , Sn , Rn×n , and Rn , and G is
non-negative positive definite.
We also define an equilibrium by local perturbation. Given a control u∗ . For any
t ∈ [0, T ), ε > 0 and v ∈ L2Ft (Ω; Rl ), define
(2.3)
ut,ε,v
= u∗s + v1s∈[t,t+ε) , s ∈ [t, T ].
s
Definition 2.1 Let u∗ ∈ L2F (0, T ; Rl ) be a given control and X ∗ be the state process
corresponding to u∗ . The control u∗ is called an equilibrium if
J(t, Xt∗ ; ut,ε,v ) − J(t, Xt∗ ; u∗ )
lim inf
≥ 0,
ε↓0
ε
where ut,ε,v is defined by (2.3), for any t ∈ [0, T ) and v ∈ L2Ft (Ω; Rl ).
Notice that here we changed lim to lim inf in this definition, which makes no critical
difference other than making the definition more robust.
3
An Equivalent Condition of Equilibrium Controls
In this section we present a general characterization for equilibria. This has been done
partially in our previous paper [7] for the sufficiency by the second-order expansion in the
local spike variation, in the same spirit of proving the stochastic Pontryagin’s maximum
principle [8, 9, 13], here we will study an equivalent condition for equilibria.
We also start with the obtained result from our previous paper [7]. Let u∗ be a fixed
control and X ∗ be the corresponding state process. For any t ∈ [0, T ), define in the time
interval [t, T ] the processes (p(·; t), (k j (·; t))j=1,··· ,d ) ∈ L2F (t, T ; Rn ) × (L2F (t, T ; Rn ))d as the
solution to

P

dp(s; t) = −[A′s p(s; t) + dj=1 (Csj )′ k j (s; t) + Qs Xs∗ ]ds



P
(3.1)
+ dj=1 k j (s; t)dWsj , s ∈ [t, T ],




p(T ; t) = GXT∗ − hEt [XT∗ ] − µ1 Xt∗ − µ2 ;
3
Define (P (·; t), (K j (·; t))j=1,··· ,d ) ∈ L2F (t, T ; Sn ) × (L2F (t, T ; Sn ))d as the solution to

n


dP (s; t) = − A′s P (s; t) + P (s; t)As



o

Pd

j ′
j
j ′ j
j
j

+ j=1 [(Cs ) P (s; t)Cs + (Cs ) K (s; t) + K (s; t)Cs ] + Qs ds
(3.2)
Pd

j
j

+

j=1 K (s; t)dWs , s ∈ [t, T ],




 P (T ; t) = G.
The following estimate under local spike variation is from [7, Proposition 3.1].
Proposition 3.1 For any t ∈ [0, T ), ε > 0 and v ∈ L2Ft (Ω; Rl ), define ut,ε,v by (2.3).
Then
Z t+ε
1
∗
t,ε,v
∗
∗
(3.3)
J(t, Xt ; u ) − J(t, Xt ; u ) = Et
(hΛ(s; t), vi + hH(s; t)v, vi)ds + o(ε)
2
t
△
where Λ(s; t) = Bs p(s; t)+
Pd
j ′ j
∗
j=1 (Ds ) k (s; t)+Rs us
△
and H(s; t) = Rs +
Pd
j ′
j
j=1 (Ds ) P (s; t)Ds .
In the rest of this section, we will keep the notations Λ and H.
By the estimate in Proposition 3.1, it is straightforward to get the following characterization of an equilibrium.
Corollary 3.2 A control u∗ ∈ L2 (0, T, Rl ) is an equilibrium if and only if
Z
1 t+ε
lim
Et [Λ(s; t)] ds = 0, a.s., ∀t ∈ [0, T ).
ε↓0 ε t
Before going to the main result of this section, let us prove a key property for the
solution to the system of BSDEs for (p(·; t), k(·; t)) ∈ L2F (t, T ; Rn ) × (L2F (t, T ; Rn ))d and
P
△
the special form of the process Λ(s; t) = Bs p(s; t) + dj=1(Dsj )′ k j (s; t) + Rs u∗s .
Theorem 3.3 For any pair of state and control processes (X ∗ , u∗ ), the solution to (3.1)
in L2F (t, T ; Rn ) × (L2F (t, T ; Rn ))d satisfies k(s; t1 ) = k(s; t2 ) for a.e. s ≥ max (t1 , t2 ).
l×n
Furthermore, there exist λ1 ∈ L2F (0, T ; Rl ), λ2 ∈ L∞
) and ξ ∈ L2 (Ω; C(0, T ; Rn )),
F (0, T ; R
such that
Λ(s; t) = λ1 (s) + λ2 (s)ξt .
Proof: Define the function ψ(·) as the unique solution for the matrix-valued ordinary
differential equation
dψ(t) = ψ(t)A(t)′ dt, ψ(T ) = In ,
where In means the n × n identity matrix. It is well known that ψ(·) is invertible, and
both ψ(·) and ψ(·)−1 are bounded.
Define p̂(s; t) = ψ(s)p(s; t) + hEt [XT∗ ] + µ1 Xt∗ + µ2 and k̂ j (s; t) = ψ(s)k j (s; t) for
j = 1, · · · , d, then in the interval s ∈ [t, T ], (p̂(·; t), k̂(·; t)) satisfies

 dp̂(s; t) = −[Pd ψ(s)(C j )′ ψ(s)−1 k̂ j (s; t) + ψ(s)Qs X ∗ ]ds + Pd k̂ j (s; t)dW j ,
s
s
s
j=1
j=1
(3.4)
 p̂(T ; t) = GX ∗ .
T
4
From the uniqueness of solutions to Lipschitz BSDE, the solution (p̂(s; t), k̂(s; t)) does not
depend on t, hence we denote its solution as (p̂(s), k̂(s)).
Now we have k(s; t) = ψ(s)−1 k̂(s) := k(s) and
p(s; t) = ψ(s)−1 p̂(s) − ψ(s)−1 (hEt [XT∗ ] + µ1 Xt∗ + µ2 ) := p(s) + ψ(s)−1 ξt ,
where k(·) is in (L2F (0, T ; Rn ))d , ξt := −hEt [XT∗ ] − µ1 Xt∗ − µ2 defines the process ξ ∈
L2F (Ω; C(0, T ; Rn)) and p(s) := ψ(s)−1 p̂(s) defines the process p ∈ L2F (Ω; C(0, T ; Rn )).
Hence
Λ(s; t) = Bs p(s; t) +
d
X
(Dsj )′ k j (s; t) + Rs u∗s
j=1
= Bs p(s) +
d
X
(Dsj )′ k j (s) + Rs u∗s + Bs ψ(s)−1 ξt
j=1
:= λ1 (s) + λ2 (s)ξt ,
P
where λ1 (s) := Bs p(s) + dj=1 (Dsj )′ k j (s) + Rs u∗s and λ2 (s) := Bs ψ(s)−1 .
Q.E.D.
Theorem 3.4 For any control u ∈ L2F (0, T ; Rl ), the following two statements are equivalent:
R t+ε
(i) limε↓0 1ε t Et [Λ(s; t)] ds = 0, a.s., ∀t ∈ [0, T ).
(ii) Λ(t; t) = 0, a.s., a.e.t ∈ [0, T ].
Proof: We know Λ(s; t) = λ1 (s) + λ2 (s)ξt . Since λ2 is bounded and ξ is continuous, we
have
Z t+ε
Z
1
1 t+ε
lim Et
|λ2 (s)(ξs − ξt )|ds ≤ c lim
Et [|ξs − ξt |] ds
ε↓0
ε↓0 ε t
ε t
= 0,
where the last equality is because Et [|ξs − ξt |] is a continuous function of s and vanishes
at s = t.
By this fact, we know
Z
Z
1 t+ε
1 t+ε
Et [Λ(s; t)] ds = lim
Et [Λ(s; s)] ds.
lim
ε↓0 ε t
ε↓0 ε t
If (ii) holds, then
Z
1 t+ε
Et [Λ(s; t)] ds = lim
Et [Λ(s; s)] ds = 0.
ε↓0 ε t
t
R t+ε
If (i) holds, then limε↓0 1ε t Et [Λ(s; s)] ds = 0. According to the following lemma 3.5,
we know Λ(s; s) = 0, a.s., a.e.t ∈ [0, T ].
Q.E.D.
1
lim
ε↓0 ε
Z
t+ε
5
Lemma 3.5 Y ∈ L2F (0, T ; Rl) is a given process. If limε↓0
[0, T ), a.s., then Yt = 0, a.e.t ∈ [0, T ), a.s..
1
ε
R t+ε
t
Et [Ys ] ds = 0, a.e.t ∈
Proof: Since L2FT (Ω; Rl ) is a separable space, we can take a countable dense subset D ⊂
l
L2FT (Ω; Rl ) ∩ L∞
FT (Ω; R ), such that for almost all t, we have
1
lim
ε↓0 ε
(3.5)
Z
t+ε
E [hYs , ηi] ds = E [hYt , ηi] ,
∀η ∈ D,
t
R t+ε
and limε↓0 1ε t E [Ys2 ] ds = E [Yt2 ] .
For any η ∈ D, define ηs = Es [η], then E [hYs , ηi] = E [hYs , ηs i]. Since
sZ
Z t+ε
Z t+ε
t+ε
1
2
lim 1
E [Ys ] ds
E [(ηs − ηt )2 ] ds
E [hYs , ηs − ηt i] ds ≤ lim
ε↓0 ε
ε↓0 ε
t
t
t
s Z
s Z
t+ε
1
1 t+ε
2
= lim
E [Ys ] ds
E [(ηs − ηt )2 ] ds
ε↓0
ε t
ε t
s Z
r
1 t+ε
E [Ys2 ] ds
sup E [(ηs − ηt )2 ]
≤ lim
ε↓0
ε t
s∈[t,t+ε]
s
Z
p
1 t+ε
E [Ys2 ] ds E [(ηt+ε − ηt )2 ] = 0.
≤ 2 lim
ε↓0
ε t
Hence for any η ∈ D,
E [hYt , ηt i] = E [hYt , ηi]
Z
1 t+ε
E [hYs , ηi] ds
= lim
ε↓0 ε t
Z
1 t+ε
= lim
E [hYs , ηs i] ds
ε↓0 ε t
Z
1 t+ε
E [hYs , ηt i] ds
= lim
ε↓0 ε t
Z
1 t+ε
E [hEt [Ys ] , ηt i] ds
= lim
ε↓0 ε t
Z t+ε
1
= lim E h
Et [Ys ] ds, ηt i .
ε↓0
ε t
Since
" Z
2 #
Z t+ε
Z t+ε
1
1 t+ε
2
Et [Ys ] ds
≤ E
ds
Et [Ys ] ds
E
ε t
ε2
t
t
Z t+ε
1
2
E
Et [Ys ] ds
=
ε
t
Z
1 t+ε 2 E Ys ds,
≤
ε t
6
1
ε
R t+ε
E [Ys2 ] ds = E [Yt2 ], there exists a constant δt > 0, such that
" Z
2 #
1 t+ε
E
Et [Ys ] ds
< 2E Yt2 , ∀ ε ∈ (0, δt ).
ε t
R t+ε
This implies that 1ε t Et [Ys ] ds is uniformly integrable when ε ∈ (0, δt ). Hence
Z t+ε
Z t+ε
1
1
Et [Ys ] ds = E lim Et [Ys ] ds = 0.
lim E ε↓0 ε t
ε↓0
ε t
and limε↓0
t
Then there exists a constant c > 0, such that
Z t+ε
Z t+ε
1
E h 1
E
[Y
]
ds,
η
i
E
[Y
]
ds
≤
cE
t
s
t
t
s
ε
ε t
t
→ 0,
which means
Z
1 t+ε
Et [Ys ] ds, ηt i = 0,
lim E h
ε↓0
ε t
and hence E [hYt , ηi] = 0, a.e.t ∈ [0, T ] for any η ∈ D, which implies
Yt = 0, a.e.t ∈ [0, T ], a.s..
Q.E.D.
We summarize the main theorem into the following characterization for equilibria.
Theorem 3.6 Given a control u∗ ∈ L2F (0, T ; Rl ). Denote X ∗ as the state process of u∗ , and
(p(·; t), k(·; t)) ∈ L2F (t, T ; Rn ) × (L2F (0, T ; Rn ))d as the unique solution for the BSDE (3.1),
P
△
with k(s) := k(s; t) according to Theorem 3.3. Define Λ(·; t) = B· p(·; t) + dj=1(D·j )′ k(·)j +
R· u∗· .
Then u∗ is an equilibrium if and only if
Λ(t; t) = 0, a.s., a.e. t ∈ [0, T ].
4
Uniqueness
Now we prove the system of equations (3.1) and the dynamics, together with the condition
Λ(t; t) = 0, has a unique solution in cases studied in our previous paper [7], where we focus
on the scalar system, i.e., n = 1 and the system is governed by the SDE
(4.1)
dXs = [As Xs + Bs′ us + bs ]ds + [Cs Xs + Ds us + σs ]′ dWs ;
X0 = x0 .
In this case, we can rewrite the system of backward SDE (p(·; t), k(·; t)) into

 dp(s; t) = −[As p(s; t) + C ′ k(s; t) + Qs X ∗ ]ds + k(s; t)′ dWs , s ∈ [t, T ],
s
s
(4.2)
 p(T ; t) = GX ∗ − hE [X ∗ ] − µ X ∗ − µ ,
T
t
T
1
t
2
and the corresponding Λ(s; t) is now in the form
△
Λ(s; t) = Bs p(s; t) + Ds′ k(s; t) + Rs u∗s .
7
4.1
Cases with Deterministic Parameters
In this section, we assume all parameters A, B, b, C, D, σ, Q and R are deterministic. We
have constructed a solution to the joint SDEs and Λ(t; t) = 0 in [7] by solving the following
system of ordinary differential equations:

2


 0 = Ṁ + (2A + |C| )M + Q

(4.3)
−M(B ′ + C ′ D)(R + MD ′ D)−1 [(M − N − Γ(1) )B + MD ′ C], s ∈ [0, T ],




MT = G;


0 = Ṅ + 2AN



(4.4)
−NB ′ (R + MD ′ D)−1 [(M − N − Γ(1) )B + MD ′ C], s ∈ [0, T ],




NT = h;

 Γ̇(1) = −AΓ(1) , s ∈ [0, T ],
(4.5)
 Γ(1) = µ ;
T
(4.6)
1


0 = Φ̇ + {A − [(M − N)B ′ + MC ′ D](R + MD ′ D)−1 B}Φ + (M − N)b



+C ′ Mσ − [(M − N)B ′ + MC ′ D](R + MD ′ D)−1 MD ′ σ, s ∈ [0, T ],




ΦT = −µ2 .
If this system of equations admits a solution (M, N, Γ(1) , Φ), then the feedback control
law
u∗s = αs Xs∗ + βs
defines an equilibrium, where
△
′
αs = −(Rs + Ms Ds′ Ds )−1 [(Ms − Ns − Γ(1)
s )Bs + Ms Ds Cs ],
△
βs = −(Rs + Ms Ds′ Ds )−1 (Φs Bs + Ms Ds′ σs ).
We assume the existence of (M, N, Γ(1) , Φ) to the system of equations (4.3, 4.4, 4.5, 4.6),
then we have an equilibrium constructed as above. In [7], there are several mild sufficient
conditions for this assumption to hold. Now we claim that the equilibrium constructed
above is the unique equilibrium.
Theorem 4.1 In the case with deterministic parameters, when (M, N, Γ(1) , Φ) exists, the
equilibrium is unique.
Proof: Suppose there is another equilibrium (X, u), then the equation system (3.1), with X ∗
replaced by X, admits a solution (p(s; t), k(s), Xs , us ), which satisfies Bs p(s; s) + Ds′ k(s) +
Rs us = 0 for a.e. s ∈ [0, T ].
Define
p̄(s; t) = p(s; t) − [Ms Xs − Ns Et [Xs ] − Γ(1)
s Xt + Φs ],
8
k̄(s) = k(s) − Ms (Cs Xs + Ds us + σs ).
By the equilibrium condition, we have
′
Bs (p̄(s; s) + (Ms − Ns − Γ(1)
s )Xs + Φs ) + Ds [k̄(s) + Ms (Cs Xs + Ds us + σs )] + Rs us = 0.
Since Rs + Ds′ Ms Ds is invertible, we have
′
us = −[Rs + Ds′ Ms Ds ]−1 Bs p̄(s; s) + Ds′ k̄(s) + (Bs (Ms − Ns − Γ(1)
s ) + Ds Ms Cs )Xs
+Bs Φs + Ds′ Ms σs ] .
Hence
dp̄(s; t)
= dp(s; t) − d[Ms Xs − Ns Et [Xs ] − Γ(1)
s Xt + Φs ]
′
′
= −[As p(s; t) + Cs ks + Qs Xs ]ds + ks dWs − d[Ms Xs − Ns Et [Xs ] − Γ(1)
s Xt + Φs ]
′
′
′
′
−1
= − Ap̄(s; t) + C k̄(s) − [C MD + MB ][R + D MD] [B p̄(s; s) + D ′ k̄(s)]
+NB ′ [R + D ′ MD]−1 Et B p̄(s; s) + D ′ k̄(s) ds + k̄(s)′ dWs ,
where we suppress the subscript s for A, B, C, D, M, N, R, and we
hRhave used the
i equations
T
(1)
2
for M, N, Γ , Φ in the last equality. It is easy to prove that E 0 |k̄(s)| ds < +∞ and
supt∈[0,T ] E sups≥t |p̄(s; t)|2 < +∞.
We will prove in the next theorem that the equation for (p̄(s; t), k̄(s)) admits at most
one solution in the space L1 × L2 , where
)
(
X(·; ·) : X(·; t) ∈ L2F (t, T ; R), sup E sup |X(s; t)|2 < +∞ ,
L1 :=
t∈[0,T ]
and
L2 :=
(
Y (·; ·) : Y (·; t) ∈ L2F (t, T ; Rd), sup E
t∈[0,T ]
s≥t
Z
t
T
)
|Y (s; t)|2 ds < +∞ .
Hence p̄(s; t) ≡ 0 and k̄(s) ≡ 0.
Finally, plugging p̄ ≡ k̄ ≡ 0 into u, we get the u being in the same form of feedback
control law as the one specified by M, N, Γ(1) , Φ, and hence (X, u) is the same as we got
before.
Q.E.D.
For the uniqueness of (p̄(s; t), k̄(s)), we study a more general equation
(4.7)

 dp̄(s; t) = −f (s, p̄(s; t), p̄(s; s), Et [l1 (s)p̄(s; s)] , k̄(s; t), Et l2 (s)k̄(s; t))ds + k̄(s; t)′ dWs ,
 p̄(T ; t) = 0,
where li (s) are uniformly bounded, adapted vector process with finite dimension, f (s, · · · )
is uniformly Lipschitz for all variables except for s.
Theorem 4.2 Equation (4.7) admits at most one solution (p̄, k̄) in the space L1 × L2 .
9
Proof: For any t ∈ [0, T ], s ∈ [t, T ], by Itô’s formula, we have
Z T
2
|p̄(s; t)| +
|k̄(u; t)|2du
s
Z T
= 2
p̄(u; t)f (u, p̄(u; t), p̄(u; u), Et [l1 (u)p̄(u; u)] , k̄(u; t), Et l2 (u)k̄(u; t) )du
s
Z T
−2
p̄(u; t)k̄(u; t)′dWu .
s
By this equality, there exists a constant c1 > 0, such that
Z T
2
2
E |p̄(s; t)| + E
|k̄(u; t)| du
s
Z T
≤ c1 E
|p̄(u; t)| |p̄(u; t)| + |k̄(u; t)| + |p̄(u; u)| + Et [|p̄(u; u)|] + Et |k̄(u; t)| du
s
Z T
Z T
1
2
2
2
|k̄(u; t)| du ,
≤ c2 E
|p̄(u; t)| + |p̄(u; u)| du + E
2
s
s
where we have used the inequality cxy ≤ c2 x2 + 41 y 2 for any nonnegative c, x, y. Hence
there exists a c3 > 0 such that
Z T
Z T
2
2
2
2
(4.8)
E |p̄(s; t)| + E
|k̄(u; t)| du ≤ c3 E
|p̄(u; t)| + |p̄(u; u)| du .
s
s
Furthermore, we have for any s ∈ [t, T ] that
"
#
Z T
E |p̄(s; t)|2 +
|k̄(u; t)|2du ≤ c3 (T − t) sup E |p̄(u; t)|2 + sup E |p̄(u; u)|2
u∈[t,T ]
s
≤ 2c3 (T − t)
u∈[t,T ]
sup
E |p̄(s; u)| .
t≤u≤s≤T
Hence
(4.9)
sup
t≤u≤s≤T
E |p̄(s; u)|2 ≤ 2c3 (T − t)
2
sup
t≤u≤s≤T
E |p̄(s; u)|2 .
Now take δ ∈ (0, 1/(4c)), then for any t ∈ [T − δ, T ], we have
1
sup E |p̄(s; u)|2 ,
sup E |p̄(s; u)|2 ≤
2 t≤u≤s≤T
t≤u≤s≤T
which means supt≤u≤s≤T E [|p̄(s; u)|2] = 0, hence p̄(s; u) = 0, a.s. almost everywhere in
{(s, u) : t ≤ u ≤ s ≤ T }.
For t ∈ [T − 2δ, T − δ] and s ∈ [T − δ, T ], since p̄(u, u) = 0 for any u ∈ [s, T ], by (4.8)
we have
Z T
Z T
2
2
2
(4.10)
E |p̄(s; t)| + E
|k̄(u; t)| du ≤ c3 E
|p̄(u; t)| du .
s
s
By Grownwall’s inequality, we have p̄(s; t) = 0, k̄(s; t) = 0.
For t ∈ [T − 2δ, T − δ] and s ∈ [t, T − δ], since we have p̄(T − δ; t) = 0, we can apply
previous trick for the region t ∈ [T − δ, T ] and s ∈ [t, T ] to confirm that p̄ ≡ 0, k̄ ≡ 0.
We can repeat the same analysis to t ∈ [T − 3δ, T − 2δ], and again and again until time
t = 0.
Q.E.D.
10
4.2
Mean-Variance Equilibrium strategies in A Complete Market with Random Parameters
In this subsection, we consider the mean-variance investment problem in a complete financial market. In terms of the LQ control problem, we study the system governed by the
SDE

 dXs = rs Xs ds + θ′ us ds + u′ dWs ,
s ∈ [t, T ],
s
s
(4.11)
 X =x,
t
t
where r is the deterministic interest rate function, θ is the random risk premium process.
The objective at time t with state Xt = xt is to minimize
(4.12)
1
Vart (XT ) − (µ1 xt + µ2 )Et [XT ]
2
1
Et [XT2 ] − (Et [XT ])2 − (µ1 xt + µ2 )Et [XT ]
=
2
△
J(t, xt ; u) =
with µ1 ≥ 0.
In [7], we constructed an equilibrium by the solutions (M, U), (Γ(1) , γ (1) ), (Γ(2) , γ (2) ),
and (Γ(3) , γ (3) ) for BSDEs:



dMs = −[2rM + (θM + U)′ α]ds + Us′ dWs , MT = 1,





(1)
(1)
 dΓ(1)
= −rΓ(1) ds + (γs )′ dWs , ΓT = µ1 ,
s
(4.13)
(2)
(2) ′
(2)

(2)
′

 dΓs = −[rΓ + (θM + U) β]ds + (γs ) dWs , ΓT = −µ2 ,




(3)
(3)
 dΓ(3)
= −[rΓ(3) + (θM + U)′ β]ds + (γs )′ dWs , Γ = 0,
s
T
with
△
(1)
αs = −Ms−1 −θs Γ(1)
+
U
−
γ
,
s
s
s
△
(3)
(2)
βs = −Ms−1 θs (Γ(2)
.
s − Γs ) + γ s
In this case, the BSDE for p(s; t) for a control u with state process X is
dp(s; t) = −rs p(s; t)ds + k(s; t)′ dWs ,
p(T ; t) = XT∗ − Et [XT∗ ] − µ1 Xt∗ − µ2 ,
and the corresponding Λ(s; t) is
Λ(s; t) = p(s; t)θs + k(s; t).
It is proved in [7, Proposition 5.1] that the system of BSDEs admits a unique solution,
both M and M −1 are bounded, and U · W is a BMO martingale. Furthermore, the feedback
control policy
i
h
RT
u∗s = −Ms−1 (Us − θs µ1 e s rv dv )Xs∗ + Γs θs + γs(2)
defines a control in the space L2F (0, T ; Rd), which is an equilibrium for the mean-variance
investment problem.
11
For any q > 1, define
L3 (q) := {X(·; ·) : For ∀ t ∈ [0, T ], X(·; t) ∈ LqF (Ω; C(t, T ; R))} ,
and
L4 (q) :=
(
Y (·) : Y is adapted, E
"Z
T
2
|Y (s)| ds
t
q/2 #
)
< +∞ .
In this subsection, we claim that the equilibrium above is unique.
Theorem 4.3 For the mean-variance problem with random θ, the equilibrium obtained is
unique for the control problem.
Proof: Suppose that there is another equilibrium, then there exists another solution for the
equation system (5.4) in our previous paper, denoted as p(s; t), k(s), Xs , us which satisfies
θs p(s; s)+k(s) = 0 for a.e. s ∈ [0, T ]. We will also use the notation M, U, Γ(1) , Γ(2) , γ (2) , Γ(3)
and γ (3) .
As we know from [7], M, M −1 , Γ(1) , Γ(2) and Γ(3) are all bounded, γ (2) · W and U · W
are both BMO martingales.
By the fact that U · W is a BMO martingale, in view of John-Nirenberg inequality
(see
that there exists a ε > 0 such that
i [6, Theorem 2.2,
hRp.29]), weqknow
h Kazamaki
i
RT
2
T
E eε 0 |Us | ds < +∞, hence E
|Us |2 ds
< +∞ for any q > 0.
0
h
i
(2)
(3)
(1)
Define p̄(s; t) = p(s; t) − [Ms Xs + Γs − Et Ms Xs + Γs − Γs Xt ] and k̄(s) = k(s) −
(2)
[Ms us + Us Xs + γs ].
It is easy to check that p̄(·; t) ∈ L3 (2). Since k ∈ L2F (0, T ; Rd), Mu+γ (2) ∈ L2F (0, T ; Rd),
and for any q ∈ (1, 2),
"Z
"
Z T
q/2 #
q/2 #
T
E
|Us Xs |2 ds
≤ E sup |Xs |q
|Us |2 ds
s∈[0,T ]
0
≤
E
"
0
sup |Xs |2
s∈[0,T ]
< +∞,
#!q/2
E
"Z
T
|Us |2 ds
0
q/(2−q) #!1−q/2
together with L2F (0, T ; Rd) ⊂ L4 (q) for any q ∈ (1, 2) we can conclude that k̄ ∈ L4 (q) for
any q ∈ (1, 2).
Furthermore,
(3)
(1)
(2)
0 = θp̄(s; s) + k̄(s) + θ[Γ(2)
s − Γs − Γs Xs ] + [Ms us + Us Xs + γs ],
(2)
(3)
(2)
us = −Ms−1 [(Us − θΓ(1)
s )Xs + θ p̄(s; s) + k̄(s) + θ(Γs − Γs ) + γs ]
= αs Xs + βs − Ms−1 [θp̄(s; s) + k̄(s)],
where αs , βs are the same as we defined in (5.14) in our previous paper.
12
Now we plug us into the



dp̄(s; t) =






(4.14)







 p̄(T ; t) =
calculation of dp̄(s; t),
− rs p̄(s; t) − (θs + Us Ms−1 )′ [θs p̄(s; s) + k̄(s)]
+Et (θs + Us Ms−1 )′ [θs p̄(s; s) + k̄(s)] ds
+k̄(s)′ dWs ,
0.
We will prove in the next theorem that this equation admits at most one solution
(p̄(s; t), k̄(s)) in the space L3 (q) × L4 (q)
h for some q i∈ (1, 2), which means p̄ ≡ 0 and k̄ ≡ 0,
(1)
(2)
(2)
(3)
and hence p(s; t) = Ms Xs + Γs − Et Ms Xs + Γs + Γs Xt , k(s) = Ms us + Us Xs + γs .
So we have us = αs Xs + βs . Plugging this control to dXs , we have the wealth process and
the control (X, u) are the same.
Q.E.D.
Theorem 4.4 For any q ∈ (1, 2), the equation (4.14) admits at most one solution (p̄, k̄) ∈
L3 (q) × L4 (q).
Proof: Firstly, take Et [·] of the equation for p̄, and notice that k̄ · W is a martingale, we
get
Z T
Et [p̄(s; t)] =
rν Et [p̄(ν; t)] dν,
s
which implies Et [p̄(s; t)] = 0 for any s ≥ t. Especially at s = t, we have p̄(t; t) = 0, hence
the equation (4.14) turns into

h
io
n
 dp̄(s; t) = − rs p̄(s; t) − (θs + Us )′ k̄(s) + Et (θs + Us )′ k̄(s) ds + k̄(s)′ dWs ,
Ms
Ms
(4.15)
 p̄(T ; t) = 0.
RT
As r is deterministic and bounded, we can discount p̄(s; t) by e− s rv dv to remove the
linear term −rs p̄(s; t), and hence we can assume r ≡ 0 without loss of generality. Define
RT p̃(s; t) = p̄(s; t) − s Et (θv + Uv Mv−1 )′ k̄(v) dv, then p̃(T ; t) = 0 and
dp̃(s; t) = (θs + Us Ms−1 )′ k̄(s)ds + k̄(s)′ dWs .
In p̃(·; t), it is given that p̄(·; t) ∈ L3 (q). Furthermore, for any q̄ ∈ (1, q), denote q̂ = q/q̄,
13
and 1/p̂ + 1/q̂ = 1, then
#
"Z
i q̄
T h
′
sup E Et θν + Uν Mν−1 k̄ (ν) dν s∈[t,T ]
s
"Z
#
q̄
T ′
≤ E
θν + Uν Mν−1 k̄(ν) dν
t
≤ c0 E
≤ c1 E
"Z
"Z
≤ c3 + c2
T
|θν′ k̄
t
(ν) |dν
T
|k̄ (ν) |2 dν
t
E
< +∞.
"Z
q̄ #
+ c0 E
|Uν |2 dν
t
T
Mν−1 |Uν′ k̄
t
q̄/2 #
T
"Z
+ c2 E
"Z
q̄p̂/2 #!1/p̂
q̄/2 Z
T
|Uν |2 dν
t
RT
T
|k̄(ν)|2 dν
t
E
"Z
T
|k̄(ν)|2 dν
t
By this inequality, we have E sups∈[t,T ] |p̃(s; t)|q̄ < +∞.
1
(ν) |dν
q̄ #
RT
Us 2
q̄/2 #
q/2 #!1/q̂
Us ′
Define ξ = E(−(θs + Us Ms−1 ) · W )T = e− 2 0 |θs + Ms | ds− 0 (θs + Ms ) dWs . Since U/M is a
= ξ, under which
BMO, E [ξ] = 1, and it can be used to define a new measure Q by dQ
dP
Rs
−1
Ŵs = Ws + 0 (θv + Uv Mv )dv is a standard Brownian motion. Furthermore,
dp̃(s; t) = k̄(s)dŴs , p̃(T ; t) = 0.
By Itô’s formula,
dMs−1 = −Ms−2 dMs + Ms−3 Us2 ds
("
#
)
(1)
(1)
2
Γ
U
U
Γ
|θ
|
s
s
s
s
s
θ(
= Ms−1
ds −
− 1)
+
dWs ,
Ms
Ms
M
Ms
hence
MT−1
=
−
M0−1 e
RT
0
[
R U′
Us′ θs
(1) |θ |2
(1) Us′ θs
|U |2
−Γs Ms + 21 s2 −Γs
]ds− 0T Ms dWs
Ms
s
s
Ms
Ms2
.
Comparing ξ and MT−1 , we can see
ξMT = M0 e−
RT
RT
0
(1)
(1)
Γs |θs |2 M1 ds −
s
e
RT
0
′
(1) θs
Us
ds
Ms Ms
Γs
2 1
1
1
e− 2
RT
RT
0
2
|θs |2 ds−
RT
RT
0
θs dWs
.
′
It is obvious that M0 e− 0 Γs |θs | Ms ds is bounded, e− 2 0 |θs | ds− 0 θs dWs ∈ Lq̄ for any q̄ > 1.
Furthermore, we know for any q̄ > 1, any ε > 0, there exists a constant C > 0, such that
R
q̄ h RT 2 i
(1) θs Us
ds
− 0T Γs M
s Ms
≤ CE eε 0 |Us| ds .
E e
i
h RT
2
As we claimed before, there exists a ε > 0 such that E eε 0 |Us | ds < +∞, hence
RT
(1) θs Us
e− 0 Γs Ms Ms ds ∈ Lq̄ . This proves ξMT ∈ Lq̄ , and therefore, by the fact that M −1 is
bounded, we know ξ ∈ Lq̄ for any q̄ > 1.
14
Now for any q̄ ∈ (1, q) and q̂ ∈ (1, q̄), we have
"
EQ [ sup |p̃(s; t)|q̂ ] = E
s∈[t,T ]
sup |p̃(s; t)|q̂ ξ
s∈[t,T ]
≤
E
"
#
sup |p̃(s; t)|q̄
s∈[t,T ]
#!q̂/q̄
< +∞,
(q̄−q̂)/q̄
E ξ q̄/(q̄−q̂)
which means p̃(·; t) is a Q-martingale, and hence p̃ ≡ 0.
Q.E.D.
5
Concluding Remarks
Surprisingly, the LQ control problem we studied in this paper admits a unique solution. The
uniqueness of the solution implies the uniqueness of the value process, and the latter relieves
the worry like “why an equilibrium is defined like that”, or “what should the decision maker
do if there are multiple solutions”. Since equilibria are defined via perturbation for the
game formulation of the control problem, we believe that the definition of equilibria, unlike
the optimal solution, is not conceptually linked to the uniqueness of the value process. We
believe that the uniqueness (of the value process) is from the LQ structure of the system
and the objective. In our proof, the linear feedback structure of our equilibrium plays a
key role, which sounds hard to be generalized for other time inconsistent control problem.
References
[1] S. Basak and G. Chabakauri, Dynamic mean–variance asset allocation, Rev.
Financial Stud., 23 (2010), 2970–3016.
[2] Björk T. and A. Murgoci, A General Theory of Markovian Time Inconsistent
Stochastic Control Problems, 1694759, Social Science Research Network (SSRN), 2010.
Available online at
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1694759.
[3] T. Björk, A. Murgoci, and X.Y. Zhou, Mean-variance portfolio optimization
with state dependent risk aversion, Math. Finance, to appear. Published online Feb.
3, 2012; DOI: 10.1111/j.1467-9965.2011.00515.x.
[4] I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,
Math. Financial Economics, 2 (2008), 57–86.
[5] S.R. Grendadier and N. Wang, Investment under uncertainty and timeinconsistent preferences, J. Financial Economics, 84 (2007), 2-39.
[6] Kazamaki N., Continuous exponential martingales and BMO. Berlin: SpringerVerlag, 1994.
15
[7] Y. Hu, H. Jin and X. Zhou, Time-Inconsistent Stochastic Linear–Quadratic Control, SIAM Control, 50 (2012) 1548–1572.
[8] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze, and E.F.
Mishchenko, The Mathematical Theory of Optimal Processes, John Wiley, New
York, 1962.
[9] S. Peng, A general stochastic maximum principle for optimal control problems, SIAM
J. Control Optim., 28 (1990), 966–979.
[10] R.H. Strotz, Myopia and inconsistency in dynamic utility maximization. The Review of Economic Studies (1955), 165-180.
[11] N. Vieille and J.W. Weibull, Multiple solutions under quasi-exponential discounting.
Economic Theory, 39 (2009), 513-526.
[12] J. Yong, Time-inconsistent optimal control problems and the equilibrium HJB equation. Math. Control Relat. Fields, 2 (2012), 271329.
[13] J. Yong and X.Y. Zhou, Stochastic Controls: Hamiltonian Systems and
HJB Equations, Springer–Verlag, New York, 1999.
16