Optimization problems in finance under
full and partial information
Wolfgang Runggaldier
University of Padova, Italy
www.math.unipd.it/runggaldier
Tutorial for the Special Semester on Stochastics with Emphasis on
Finance, Linz, September 2008 - December 2008.
OUTLINE
1. Financial market models
• Discrete time
• Continuous time
? Continuous and discontinuous trajectories
2. Investment strategies self financing strategies
• Discrete time
• Continuous time
3. Optimization problems
• Standard portfolio optimization
• Optimization in Insurance
• Hedging and benchmark tracking
5. Optimization, arbitrage and martingale measures
6. Dynamic Programming (for standard portfolio optimization)
• Discrete time
• Continuous time (HJB equations)
• Approximations
6. Martingale method
• Preliminaries from hedging
• Discrete time
• Continuous time
• Discussion of the martingale method vs DP
7. Elements of portfolio optimization (and hedging) under
incomplete information
• min-max approach
• Adaptive approaches
? A. A first discrete time case
? B. A second discrete time case
? C. A continuous time case
• robust approaches
8. Portfolio optimization for a purely discontinuous market
model
Market models (discrete time ∆ = 1)
• A locally riskless asset (money market account)
Bn+1 − Bn
= rn
Bn
↔
Bn+1 = Bn(1 + rn)
(rn known at n)
• Risky assets (for the moment just one)
Sn+1 = Sn(1 + an+1)
Sn+1 − Sn
= an+1
↔
Sn
l (an+1 unknown at n)
Sn+1 = Snξn+1
• Example:
ξn =
u
d
with probability p
with probability 1 − p
Market models (transition to continuous time)
• Locally riskless asset
Bt+∆ − Bt = Btrt∆ −→ dBt = Btrtdt
(cont. compounding)
• Risky asset (at+∆ = at∆ + σtξt+∆ with ξt+∆ ∼ N (0, ∆))
St+∆ = St (1 + at∆ + σtξt+∆)
Let wt be a process s.t.
∆wt := wt+∆ − wt ∼ N (0, ∆) (Wiener process)
St+∆ = St (1 + at∆ + σtξt+∆)
↓
dSt = St [atdt + σtdwt]
Price processes with continuous trajectories
dSt = St [atdt + σtdwt]
(∆wt := wt+∆ − wt ∼ N (0, ∆) −→ dwt ∼
√
dt

1
2

d log St = at − 2 σt dt + σtdwt






hR
i
R
t+∆
t+∆
→
St+∆
= St exp t
as − 21 σs2 ds + t
σsdws







= Stξt+∆
→ All trajectories of St are continuous functions of t
Price processes with discontinuous trajectories
• Let τn (0 = τ0 < τ1 < · · · ) be a sequence of random times
where a certain event happens
• Let Nt = n if t ∈ [τn, τn+1) ⇔ Nt =
P
n≥1
→ Nt is a counting process and dNt ∈ {0, 1}.
1{τn≤t}
• Let St change only at times τn with return γn > −1 at τn
Sτn − Sτn−
Sτn−
= γn ↔ Sτn = Sτn− (1 + γn) ↔ dSt = St− γtdNt
⇒ St = S0
QNt
n=1(1
= S0 exp
hR
+ γτn ) = S0 exp
t
log(1
0
+ γt)dNt
i
i
Nt
n=1 log(1 + γτn )
hP
Combining the two (jump diffusion models)
dSt = St− [atdt + σtdwt + γtdNt]
implies
"Z
t+∆ St+∆ = St exp
t
1 2
as − σs ds +
2
Z
#
t+∆
σsdws
t
Nt+∆
Y
(1 + γn)
n=Nt
= St exp
hZ
t
t+∆ t+∆
1 2
σsdws
as − σs ds +
2
t
Z t+∆
i
+
log(1 + γt) dNt
Z
t
More risky assets
• A certain number K of risky assets
dSti = Stiaitdt + Sti
M
X
σti,j dwtj ;
i = 1, · · · , K
j=1
(dSt = (diag St) Atdt + (diag St) Σtdwt)
with wt = [wt1, · · · , wtM ]0 an M − Wiener process on a filtered
probability space (Ω, F, Ft, P ), (Ft = Ftw ).
→ In the classical Black-Scholes model ait and σti,j are
deterministic ⇒ St : a lognormal process.
• The coefficients may also be stochastic processes that are either:
i) adapted to Ftw , or
ii) Markov modulated (regime switching models)
dSt = (diag St) At(Zt)dt + (diag St) Σt(Zt)dwt
→ Zt an exogenous multivariate Markov factor process (volume
of trade, level of interest rates or, generically, the “state of the
economy”).
→ Zt may be directly observable or not.
• Price trajectories may exhibit a jumping behaviour, then
dSt = (diag St) Atdt + (diag St) Σtdwt + (diagSt−)Ψt−dNt
with Nt = (Nt1, · · · , NtH )0 a counting process and Ψi,j
t > −1
(Jump-diffusion models).
→ More general driving processes are possible (Levy, fractional
BM).
• On small time scales prices do not follow continuous trajectories,
but rather piecewise constant ones with jumps at random points
in time.
→ May be modeled by continuous trajectories sampled at the
jumps of a Poisson process.
Investment strategies (discrete time)
i
• Given Bn, Sn
(i = 1, · · · , K) let φ0n, φin be the number of riskless
resp. risky assets held in the portfolio in period n
→ φn = [φ0n, φ1n, · · · , φK
n ] predictable (determined on the basis
of the information Fn−1)
• The value of the corresponding portfolio is then
Vn = φ0nBn +
K
X
i=1
φinSni
Self financing property (with consumption cn in period n)
φ0nBn +
→ Vn+1 =
=
K
X
φinSni = φ0n+1Bn +
i=1
φ0n+1Bn+1
+
φ0n+1Bn +
→ ∆Vn
=
PK
+
φin+1Sni + cn
i=1
PK i
i
φ
S
n+1
n+1
i=1
i
i
φ
S
i=1 n+1 n
+φ0n+1(Bn+1
φ0n+1∆Bn
K
X
PK
− Bn) +
PK
i
i
i
φ
(S
−
S
n)
i=1 n+1 n+1
i
i
φ
∆S
− cn
n
n+1
i=1
With proportional transaction costs (rate γ i)
∆Vn = φ0n+1∆Bn +
K
X
i=1
φin+1∆Sni −
K
X
i=1
γ iSni | φin+1 − φin |
Alternative representation of the investment strategy
πni =
i
φin+1Sn
Vn ,
i = 1, · · · , K
with
Sn0 = Bn
determined on the basis of the informationFn.
Notice that
PK
i
i
φ
S
i=0 n+1 n = Vn − cn
→
PK
→
πn0
cn
i
π
=
1
−
i=0 n
Vn
=1−
PK
P
cn
i
π
−
i=1 n
Vn
K
i
π
i=0 n
= 1 for cn = 0
i
i
Taking ∆ = 1 and using Sn+1
= Sni ξn+1
:
Vn+1 = Vn +
φ0n+1∆Bn
= Vn +
φ0n+1Bnrn
h
= Vn (1 + rn) +
+
PK
+
PK
i
i
φ
∆S
n − cn
i=1 n+1
PK
i
i i
φ
S
i=1 n+1 n(ξn+1 − 1) − cn
i
π
i=1 n
i
i
ξn+1
− (1 + rn) − cn(1 + rn)
:= Gn(Vn, πn, cn, ξn+1)
→ Autonomous evolution of Vn, driven directly by ξn and without
reference to Sni .
Investment strategies/controls (continuous time)
Agents invest in the market according to an investment strategy,
e.g.
0
φ̄t = (φ0t , φ1t , · · · , φK
)
:=
(φ
t
t , φt )
with φit : number of shares of asset i held in the portfolio at time
t (i = 0 : riskless asset). φ̄t is taken to be predictable w. r. to Ft.
Vt = φ0t Bt +
K
X
i=1
is the corresponding value process.
φitSti
Equivalently
πt = (πt1, · · · , πtK )
πti
φitSti
Vt ,
(i = 1, · · · , K) the fraction of wealth invested in
with
=
PK i
the risky asset i at time t. (1 − i=1 πt : fraction invested in
the riskless asset).
Self-financing strategies/portfolios
Denoting by ct the consumption rate at time t, require
dVt = φ0t dBt +
K
X
φitdSti − ctdt
i=1
l
K
X
i
ct
dB
dS
dVt
t
t
0
i
= πt
+
πt i − dt
Vt
Bt
Vt
St
i=1
or, equivalently, considering price models with continuous
trajectories,
dVt = Vt [rtdt + πt(At − rt1)dt + πtΣtdwt] − ctdt
→ With πt instead of φt autonomous dynamics of Vt without
reference to Bt or St.
• Agents choose for their investments a subset of the available
assets (with prices Sti, i = 1, · · · , K).
→ In addition to actual portfolios one may then consider also
theoretical self-financing portfolios that include most of the
assets in the market.
→ Such theoretical portfolios may serve as index or benchmark
(e.g. S&P 500) with the goal of the investor being to track or
beat a given benchmark.
• With the index portfolio strategy expressed as fraction of wealth:
dIt = It [αtdt + σtdwt + ρtdvt]
with wt the (multivariate) Wiener driving the assets chosen
for actual investment and vt a Wiener independent of wt
and resulting from the disturbances that drive exclusively the
additional assets on the market.
→ A benchmark may also represent other economic quantities
such as a wage process in an insurance context.
Optimization/control problems
• Consider small investors, i.e. their investment decisions do not
affect the prices on the market.
• Two groups of “state variables” :
i) asset prices (and benchmark) are uncontrolled
ii) the portfolio value process is the only controlled state variable
(autonomous dynamics under self financing).
• The objective function depends generally on both types of state
variables.
Standard classical optimization problem
(maximization of expected utility from consumption and terminal
wealth)
Neglecting transaction costs but considering as additional control
variable ct that represents the rate of consumption at time t :

π,c
π,c
0
0
+
π
dV
=
V
[r
dt
+
π
(A
−
r
1)dt

t
t
t
t
t
tΣtdwt] − ctdt
t



(Z
)
T

π,c


max
E
U
(t,
c
)dt
+
U
(V
V
1
t
2

0
T )
π,c
0
with U1(·) and U2(·) utility functions from consumption and
terminal wealth respectively that satisfy the usual assumptions.
Insurance context
• The fundamental quantity, corresponding to Vt, is here the risk
process that, without investment or reinsurance, is given by
Rt = s + ct −
Nt
X
Xi := s + ct − Dt
i=1
with
Xi : claim sizes ;
c : premium intensity
→ Additional features :
? Investment
? Reinsurance
? Other
Investment
• One can invest in one or more risky assets. Assume one such
asset with price dynamics
dZt = aZtdt + bZtdwt
and let At be the amount invested in this asset (the rest in the
bank account with interest rate r).
A
• With θt = Z t denoting then the number of shares held in the
t
risky asset, one obtains for the risk process
dRtθ = c dt − dDt + θtdZt + r(Rtθ − θtZt) dt
Reinsurance
• There are various forms of reinsurance, here we mention excess-
of-loss reinsurance : the insurer pays min(X, b) and the reinsurer
pays (X −b)+. For this the insurer pays the reinsurance premium
h(b) to the insurer.
• One then obtains for the risk process
Rtb = s + ct −
Z
t
h(bs)ds −
0
Nt
X
i=1
min {bTi , Xi}
Objective
• A typical objective is the minimization of the ruin probability
P {Rt < 0 for some t | R0 = s}.
• One may consider other objective functions where πt denotes a
generic control at time t (may be θt or bt), i.e.
Z
max E
π
τ
U (Rtπ , πt) dt + U (Rτπ , πτ ) | R0 = s
0
with τ := inf{t ≥ 0 | Rtπ < 0}.
→ Is of the previous standard form (with a random horizon).
Hedging problem
• Given an (underlying) price process St and a future maturity T ,
let
HT ∈ FTS
(contingent claim)
→ It represents a liability depending on the future evolution of
the underlying S. This implies some risk, and the purpose is
to hedge this risk by investing in a self financing portfolio.
φ
Z t"
• Let Vt = V0 +
φ0s dBs +
0
N
X
#
φisdSsi be the value in t of a
i=1
self financing portfolio.
→ Determine, if possible, V0 and φ̄t s.t.
VTφ̄ = HT
a.s.
(equivalently VTπ = HT a.s.); i.e. such that one has perfect
duplication/replication of the claim.
→ If this is possible for any HT , then the market is said to be
complete.
→ If the market is not complete, or the initially available capital
is not sufficient for perfect replication, one has to choose a
hedging criterion. Two possible criteria are :
• Minimization of shortfall risk
(an asymmetric downside-type criterion)
π +
ES0,V0 L (HT − VT )
→
min
• Minimization of quadratic loss
(a symmetric risk criterion)
ES0,V0 (HT −
VTπ )2
→
min
General problem formulation
(over a finite horizon and including a benchmark)


dSt = (diag St) Atdt + (diag St) Σtdwt








dIt = It [αtdt + σtdwt + ρtdvt]




π,c
π,c
dV
=
V
[rtdt + πt(At − rt1)dt + πtΣtdwt] − ctdt

t
t




(Z
)


T



π,c
π

min
E
L
(I
,
V
,
π
,
c
)dt
+
L
(S
,
I
,
V

S
,I
,V
1
t
t
t
2
T
T
t
T)
0 0 0
 π,c
T −δ
• L1(·) and L2(·) are loss functions that may be of the following
form
+
L1(It, Vtπ,c, πt, ct) = (g(It) − η − Vtπ ) − ct
+
L2(ST , IT , VTπ ) = (H(IT , ST ) − VTπ )
for some functions g(·) and H(·) and for an η > 0.
→ Includes hedging of a contingent claim H(ST ).
One may consider different variants of the basic setup
corresponding to possible variants of a general stochastic control
problem as e.g. :
• The basic dynamics of the price vector St may be generalized as
described previously.
• If some of the assets are subject to default, the fixed horizon
may be replaced by a stopping time τ . This stopping time
may also become a control variable for the hedging problem of
American-type options.
• The horizon may become infinite for problems of life-time
consumption or when the objective is to maximize the growth
rate. Maximizing the risk sensitized growth rate leads to a risk
sensitive control problem.
• In the presence of transaction costs a convenient way to define a
trading strategy is by the total number of shares of the various
assets that are purchased or sold up to the current time t, i.e.
Lit and Mti respectively. Letting λi and µi denote the cost rate
for buying respectively selling asset i, the self financing condition
then leads to
N
X
i i
0
i
i
i
i
i
dVt = φt dBt +
φtdSt − St λ dLt + µ dMt
i=1
with φit = Lit − Mti. In this way one obtains a singular stochastic
control problem.
• The inclusion also of fixed transaction costs may lead to impulsive
control problems. This kind of problems may also arise when a
central bank intervenes to control the exchange rate..
Optimization, arbitrage, and martingale measures
• Arbitrage opportunity (OA) : existence of a self financing
portfolio φ s.t.
V0φ = 0 ,
VNφ ≥ 0 ,
P {VNφ > 0} > 0
• Consider for simplicity maximization of terminal utility
max
{φ self fin}
n
o
EV0=v U (VNφ)
→ If ∃ optimal solution φ∗ of this problem, then there cannot be
(OA)
Proof :
• Given φ∗, let φ be an arbitrage portfolio and put
∗
⇒
φ̄ = φ + φ
VNφ̄
=
φ∗
VN
+ VNφ (V0φ̄ = v)
φ
φ
• From the assumption on φ, VN ≥ 0 , P {VN > 0} > 0
⇒
VNφ̄
≥
φ∗
VN
with
P {VNφ̄
>
φ∗
VN }
• Since U (·) is monotonically increasing
⇒
E{U (VNφ̄)}
>
φ∗
E{U (VN )}
contradicting the assumed optimality of φ∗.
>0
• According to the 1st FTAP
AOA “ −→00
∃ MM Q
i.e. there exists a numeraire Nn (reference asset/portfolio) s.t.
E
Q
Sn
| Fm
Nn
Sm
=
,
Nm
m<n
→ For an at most denumerable Ω, if φ∗ is solution of
maxφ E{U (VNφ)} then, for the numeraire Nn = Bn,
φ∗
BN U (VN )
n
o
∗
φ
BN U 0(VN )
0
Q(ω) = P (ω)
E
Changing numeraire −→ change MM
Question : is there a numeraire s.t. for the corresponding Q it
holds
Q=P ?
→ A portfolio that, if used as numeraire, has the above property
is called numeraire portfolio
• Log-optimal portfolio
∗
φ s.t.
max EV0=v {log VNφ}
φ
=
φ∗
EV0=v {log VN }
→ a log-optimal portfolio is also growth-optimal in the sense
that it maximizes the “growth rate”
Theorem : A log (growth) optimal portfolio is a numeraire
portfolio.
Proof : (for Ω denumerable)
• It will be shown later that the log-optimal portfolio value is
Vn∗
=
φ∗
Vn
=
vBnL−1
n ;
Ln = E
dQ
| Fn
dP
, Q MM for num. B
• Let Q∗ be the MM for the numeraire VN∗ , then
dQ∗
VN∗ B0
=
= ∗
dQ
V0 BN
L∗N
∗
−→ Q
= Q
∗
VN
vBN
=P
(B0 = 1 , V0 = v)
∗
∗
VN
BN U 0(VN
)
∗ ) vBN
E {BN U 0(VN
}
BN (VN∗ )−1VN∗
= P
=P
−1
−1
E{BN v BN LN }vBN
Solution methodologies
• Like for general stochastic control problems, also for those arising
in finance a natural solution approach is based on Dynamic
Programming (DP).
• An alternative method, developed mainly in connection with
financial applications, is the so-called martingale method (MM).
Dynamic Programming (discrete time)
• Recalling Vn+1 = Gn(Vn, πn, cn, ξn+1) ; ξn i.i.d.
→ if πn = π(Vn), cn = c(Vn) are Markov controls then Vn is
Markov
• Objective :
(
max
(π0,c0),··· ,(πN ,cN )
EV0
N
X
n=0
)
U (Vn, πn, cn)
Dynamic Programming Principle (DP)
• If a process is optimal over an entire sequence of periods, then
it has to be optimal over each single period.
• Allows to obtain an optimal control sequence by a sequence of
optimizations over the individual controls.
Application of the DP principle
Using adaptedness of (πn, cn) and Markovianity of Vn
(for illustration the case of N = 2 and with only πn as controls)
max E{U (V0, π0) + U (V1, π1) + U (V2, π2)} = (DP)
π0,π1,π2
= max E{U (V0, π0) + U (V1, π1) + max U (V2, π2)} = (Markov)
π0,π1
π2
=maxπ0,π1 E {U (V0,π0)+[U (V1,π1)+E {maxπ2 U (V2,π2)|(V1,π1)}]}=(DP+M)
=maxπ0 E {U (V0,π0)+E {maxπ1 U (V1,π1)+E {maxπ2 U (V2,π2)|(V1,π1)}}|(V0,π0)}
Implementation of the DP principle
• Let (optimal cost-to-go)
(
Φn(v) := max E
πn,··· ,πN
N
X
)
U (Vm, πm) | Vn = v
m=n
→ the DP principle then leads to the backwards recursions (DPalgorithm)


ΦN (v) = max U (v, πN )


πN


 Φn(v) = max [U (v, πn) + E {Φn+1(G(Vn, πn, ξn+1)) | Vn = v}]
πn
→ leads to a sequence of individual maximizations; one obtains
automatically Markov controls (πn as function only of Vn = v)
• The DP algorithm can be used for numerical calculations if ξn is
finite-valued
• It can however also be used to obtain some explicit expressions
as will be illustrated for the following example (scalar S and, for
simplicity, only terminal utility and cn = rn = 0)


 Gn(Vn, πn, ξn+1) = Vn [1 + πn(ξn+1 − 1)]


max
π0,··· ,πN
E{U (VN )}
→ If U (v) = log v (log-utility) and ξn is binomial (ξn ∈ {u, d}),
then
Φn(v) = log v + kn
with
h
kn = (N − n) p log
p
q
+ (1 − p) log
q=
1−d
u−d
1−p
1−q
i
(d < 1 < u)
and
πn∗
p−q
(investing a constant ratio)
=
(u − d)q(1 − q)
Proof (by induction)
• True for n = N
• Assume true for n + 1,then
Φn(v) = max E{log v + log(1 + π(ξn+1 − 1)) + kn+1}
π
where
E{log(1+π(ξn+1−1))} = p log(1+π(n−1))+(1−p) log(1+π(d−1))
Imposing
p(u − 1)
(1 − p)(d − 1)
∂
E{log(1+π(ξn+1 −1))} =
+
=0
∂π
1 + π(u − 1) 1 + π(d − 1)
leads to the required π ∗; replacing the latter in the previous
expression allows then to conclude.
• A constant investment fraction results only for log-utility
• Taking U (v) = 1 − e−v one has, in the binomial case,

−v
Φ
(v)
=
1
−
k
e

n
n




 πn∗ =
1
p(1 − q)
log
v(u − d)
q(1 − p)
Dynamic Programming (continuous time)
Heuristic derivation of the HJB equation
(from discrete to continuous time)


 dSt = St[a dt + σ dwt] ;
Bt ≡ 1

 dVt = φtdSt − ctdt
→ with Zt := log St and πt :=
φtSt
Vt

2
σ

dZ
=
a
−

t
2 dt + σ dwt


dVt = (Vtaπt − ct)dt + Vtσπtdwt
0
→ Putting Yt := [Zt, Vt] ; Πt := [πt, ct] one obtains a control
problem of the following general form

dYt = At(Yt, Πt)dt + Bt(Yt, Πt)dwt ; t ∈ [0, T ]




(Z
)
T

Π


sup
E
U1(t, Yt, Πt)dt + U2(YT , ΠT )
Y0

Π
where Πt ∈ FtY .
0
• Apply an Euler-type discretization (step ∆)


Yt+∆







=
:=










/∆
 TX

sup EYΠ0 ∆
Ut(Yt, Πt)


Π
Yt + At(Yt, Πt)∆ + Bt(Yt, Πt)∆wt
Gt(Yt, Πt, ∆wt)
t=0
and recall that
Φt(y) = sup [∆ Ut(y, Π) + E {Φt+1 (Gt(y, Π, ∆wt)) | Yt = y}]
Π
Via a Taylor expansion and taking into account that
E(∆wt)2 ≈ ∆
sup [∆ Ut(y, Π) + E {Φt+1 − Φt(y) | Yt = y}]
Π
=
∂
∂t Φt(y)∆
h
∂
+ supΠ ∆Ut(y, Π) + ∆At(y, Π) ∂y
Φt(y)
2
i
∂
2
+∆
B
(y,
Π)
Φ (y) + o(∆) = 0
2 t
∂y 2 t
→ Dividing by ∆ and letting ∆ ↓ 0,









∂
∂t Φt(y)
h
∂
+ supΠ At(y, Π) ∂y
Φt(y)
i
∂
+ 21 Bt2(y, Π) ∂y
2 Φt (y) + Ut (y, Π) = 0
2







 ΦT (y) = sup UT (y, Π)
Π
Standard heuristic derivation (based on the DP principle)
(Z
Φt(y) =
sup
Πs, s∈[t,t+∆]
t+∆
E
Us(YsΠ, Πs)ds
t
Π
+Φt+∆(Yt+∆
)|
and then proceed analogously as before.
Yt = y
Solution procedure
i) Solve the maximization over Π depending on the yet unknown
Φt(y).
ii) Insert the maximizing value Π∗(t, y) and solve the resulting
PDE.
→ A “verification theorem” guarantees, under sufficient regularity
(classical solution), the optimality of the resulting Π∗(t, y) and
Φt(y).
→ In the absence of sufficient regularity : viscosity solution.
→ Explicit analytical solutions only in particular cases (e.g. linearquadratic Gaussian).
Standard classical optimization problem
(maximization of expected utility from consumption and terminal
wealth)
Neglecting transaction costs but considering as additional control
variable ct that represents the rate of consumption at time t :

π,c
π,c
0
0
dV
+
π
=
V
[r
dt
+
π
(A
−
r
1)dt

t
t
t
t
t
tΣtdwt] − ctdt
t



(Z
)
T

π,c


max
E
U
(t,
c
)dt
+
U
(V
V
1
t
2

0
T )
π,c
0
with U1(·) and U2(·) utility functions from consumption and
terminal wealth respectively that satisfy the usual assumptions.
• Put, for t ∈ [0, T ],
J π,c(t, v) := E π,c
(Z
t
)
T
U1(s, cs)ds + U2(VTπ,c) | Vt = v
and let (value function)
Φ(t, v) := sup J π,c(t, v)
π,c
HJB equation


∂Φ
∂Φ
0


(t, v) + sup [vrt − c + vπ (At − rt1)]
(t, v)


∂t
∂v
π,c



2
1 2 0
2∂ Φ
+ v ||π Σt||
(t, v) + U1(t, c) = 0
2

2
∂v






 Φ(T, v) = U (v) , Φ(t, 0) = 0
2
→

∂Φ
∗


 ct = I1 ∂v (t, v), t
(I1(·) inverse of U10 (·))


 πt∗ = − [ΣtΣ0t]−1 [At − rt1̄]
i−1
∂Φ
∂ Φ
(t,
v)
v
(t, v)
∂v
∂v 2
h
2
• After substituting (π ∗, c∗) for (π, c) one is left with a PDE :
explicit solutions can be obtained only in specific cases (mainly
in Insurance applications); regularity results are also required.
• Qualitative results are possible as e.g.
the “Mutual fund
theorem” : the optimal portfolio consists of an allocation between
two fixed mutual funds.
• The invertibility of ΣtΣ0t is equivalent to completeness of the
market (recall that, if an optimal solution exists, there cannot
be arbitrage but the market may be incomplete).
Approximations
• If analytical solutions are not possible : approximations (here an
outline of a methodology based on work by H.Kushner).
• Use the HJB equation only as an indication for finding an
appropriate time and space discretization (Vtδ ) of (Vt) such that
(Vtδ )
δ→0
⇒ Vt
in distribution
with (Vtδ ) a continuous time interpolation of a discrete time and
finite valued process.
π,c
• Letting Jδ
(t, v) be the corresponding expected remaining
cumulative utility at time t, assume furthermore that
| Jδπ,c(0, v) − J π,c(0, v) |≤ Gδ
with Gδ not depending on (π, c) and such that limδ→0 Gδ = 0
Then
i) | supπ,c Jδπ,c(0, v) − supπ,c J π,c(0, v) |≤ Gδ
ii) Let (π δ , cδ ) be the optimal strategy of the approximating
problem and let it denote also its interpolation in order to
apply it to the original problem. Then
| sup J
π,c
π,c
(0, v) − J
π δ ,cδ
(0, v) |≤ 2 Gδ
General underlying approximation philosophy
Approximate the original problem by a sequence of problems such
that the last one is explicitly solvable and show that the
corresponding solution (suitably extended to be applicable in the
original problem) is nearly optimal in the original problem.
→ The control computed from the approximating problem may
even be simpler to apply in practice.
Martingale method (discrete time)
Preliminaries
• Q is a martingale measure if, for a given numeraire Nn,
E
Q
Sn
| Fm
Nn
Sm
=
,
Nm
m<n
• With S̃n := Nn−1Sn
E Q{S̃n | Fm} = S̃m
⇔
E Q{∆nmS̃m | Fm} = 0
→ Usually Nn = Bn (locally riskless asset)
• The self financing condition is
φ0nBn +
K
X
φinSni = φ0n+1Bn +
i=1
K
X
φin+1Sni + cn
i=1
which, with Nn = Bn becomes
φ0n +
K
X
φinS̃ni = φ0n+1 +
i=1
→ Ṽn+1 = φ0n+1 +
K
X
φin+1S̃ni + c̃n
i=1
K
X
i
= Ṽn +
φin+1S̃n+1
K
X
φin+1∆S̃ni − c̃n
i=1
o
n
(for c̃n = 0) E Q Ṽn+1 | Fn = Ṽn
i=1
⇒
i.e. the discounted values of a self financing portfolio are
(Q, Fn)−martingales.
• Recall the hedging problem :
S
Given HN ∈ FN
, determine V0 = v and a self financing strategy
φ (no consumption) s.t.
VNφ = HN
a.s. ṼNφ = H̃N a.s.
• Since Ṽnφ is a Q−martingale for any martingale measure Q,
V0 = Ṽ0 = E Q{ṼNφ} = E Q{H̃N }
and this determines the initial wealth V0 = v.
Determining the hedging strategy corresponds to a martingale
representation problem.
i) Define M̃n := E Q{H̃N | Fn} which is a (Q, Fn)−martingale
(E Q{M̃n|Fm}=E Q{E Q{H̃N |Fn}|Fm}=E Q{H̃N |Fm}=M̃m)
ii) Determine φ̄n s.t., with V0 = v and for
Ṽnφ̄ = V0 +
n X
K
X
i
φ̄im+1∆S̃m
m=0 i=1
one has M̃n = Ṽn (representing the martingale M̃n in the form
of Ṽn)
→ φ̄n is then the hedging strategy
Martingale method (discrete time)
Methodology (only terminal utility; no consumption)
1. Given V0 = v, determine the set of reachable terminal wealths
VN , i.e.
n
o
Vv := V | V = VNφ for φ self financing and V0 = v
2. Determine the optimal terminal wealth VN∗
E{U (VN∗ )} ≥ E{U (VN )}
∀ V N ∈ Vv
3. Determine a self financing strategy φ∗ s.t.
φ∗
VN
= VN∗
(corresponds to hedging the “claim” HN = VN∗ )
• Solving i) : Vv is the set of all VN s.t.
E Q{ṼN } = v
∀ MM Q
→ If the set of all MM’s is a convex polyhedron with a finite
number of “vertices” Qj (j = 1, · · · , J), then the condition
becomes
Qj
E {ṼN } = v ; j = 1, · · · , J
• Solving ii) i.e.
max E{U (V )} =
V ∈Vv
max
j
{V |E Q {Ṽ }=v; j=1,·,J}
E{U (V )}
→ Use Lagrange multiplier method with
dQj
L :=
dP
j
so that
E
Qj
{Ṽ } = E Q{Ṽ Lj }
and one has


J


X
−1
max E U (V ) −
λj BN
V Lj
V


j=1
→
U 0(V ) =
J
X
j=1
−1 j
λj BN
L
• Putting I(·) = (U 0(·))−1 it follows that

VN∗ = I 
J
X

−1 j 
λj BN
L
j=1
with λj satisfying the system of budget equations



J


X
−1 ∗ j −1 j 
−1 j 
v = E BN VN L = E BN
LI
λj BN
L


j=1
for j = 1, · · · , J.
Example :
U (v) = log v
→ I(y) = y −1
→ In a complete market (a single MM Q) the budget equation
becomes
v = λ−1 ↔ λ = v −1
and
VN∗
BN
=v
L
dQ
with L =
dP
• In a binomial market model with νn denoting the total random
number of up-movements
νn N −νn
q
dQ
1−q
(νn) =
L=
dP
p
1−p
→ (for simplicity rn = 0 i.e. Bn = 1)
N −νn
νn p
1−p
∗
VN = v
q
1−q
and (recall E{νn} = N p)
1−p
p
∗
+ (1 − p) log
E{U (VN )} = log v + N p log
q
1−q
→ compare with DP; similarly for the strategies.
Martingale method (discrete time)
Methodology (terminal utility with consumption)
Definition: An investment/consumption strategy (φ, c) is
admissible if cN ≤ VN .
• Recalling that, allowing also for consumption, the self financing
condition reads as follows
Ṽn = V0 +
n−1
K
XX
m=0 i=1
i
φim+1∆S̃m
−
n−1
X
c̃m
m=0
we give also the following
Definition: An investment/consumption strategy (φ, c) is
attainable from the initial endowment V0 = v if (letting the set of
MM’s be a convex polyhedron with J vertices),
j
v = EQ
n
o
c̃0 + · · · + c̃N −1 + ṼN , ∀ j = 1, · · · , J
Procedure
i) Determine the set of attainable consumption processes and
terminal wealths.
ii) Determine the optimal attainable consumption and terminal
wealth.
iii) Determine an investment strategy that allows to consume
according to the optimal consumption process.
Solving i) : see definition of attainability.
(
Solving ii) :
max E
c,Vn
N
X
)
Uc(cn) + Up(VN − cN )
n=0
with the following budget equations where Nnj := Bn−1E{Lj | Fn}
j
v = EQ
nP
N −1
n=0
= E
nP
N −1
= E
nP
N −1
n=0
n=0
c̃n + ṼN
o
n hP
io
N −1
= E Lj
n=0 c̃n + ṼN
−1
E Bn−1cnLj | Fn + E BN
VN Lj | FN
j
cnNnj + VN NN
o
o
; ∀ j = 1, · · · , J
Having Uc(c) = −∞ for c < 0; Up(v) = −∞ for v < 0 guarantees
cn ≥ 0, cN ≤ VN → admissibility.
Lagrange multiplier technique


"
#
N
J
N
−1
X

X
X
j
max E
Uc(cn) + Up(VN − cN ) −
λj
cnNnj + VN NN


n=0
⇒
j=1

PJ
0
j

Uc(cn)
=
λ
N
;
j
n

j=1




Uc0 (cN )
= Up0 (Vn − cn)





PJ
 0
j
Up(Vn − cn) =
λ
N
j
n
j=1
n=0
n = 0, · · · , N − 1
⇒



 cn
= Ic
P
J
j
λ
N
j=1 j n
;
n = 0, · · · , N
P
P


J
J
j
j
 VN = Ip
λ
N
+
I
λ
N
c
j=1 j N
j=1 j N
with the budget equation





N
J
J
X

X
X
j
j 
v=E
Nnj Ic 
λj Nnj  + NN
Ip 
λj NN


n=0
for j = 1, · · · , J.
j=1
j=1
Martingale approach (continuous time)
Preliminaries: determining the hedging strategy in a complete
market (martingale representation)
(P ) dSt = (diag St) Atdt + (diag St) Σtdwt ,
Σt invertible
→ Want a measure Q ∼ P s.t.
(Q) dSt = (diag St) rt1dt + (diag St) ΣtdwtQ
→ S̃ti := Bt−1Sti satisfy dS̃t = diag S̃t ΣtdwtQ
i.e. S̃t is a (Q, Ft)−martingale (Q is a Martingale measure
(MM)).
→ The comparison of the two representations implies
dwtQ = dwt + θtdt
where
θt := Σ−1
t (At − rt1)
i.e.
Q is obtained from P by a Girsanov-type measure
transformation implying a translation of the Wiener process
wt by θt.
→ For the given model a MM exists and is unique.
( Z
)
Z
T
T
dQ
1
= exp −
L=
θt0 dwt −
θt0 θtdt
dP
2 0
0
• From the self financing condition
dVt = φ0t dBt +
N
X
φitdSti
i=1
putting Ṽt := Bt−1Vt, one has
dṼt = φtdS̃t = φt(diag S̃t)ΣtdwtQ
i.e., under Q, also Ṽt is a martingale with
Z t
Ṽt = Ṽ0 +
φs(diag S̃s)ΣsdwsQ
0
and the problem is to possibly find Ṽ0 and φt s.t. ṼT = BT−1HT
a.s.
(Q, Ft)−martingale (assume HT =
H(ST ) and put H̃T := BT−1HT )
• Consider the following
M̃t := E Q{H̃T | Ft} = E Q{H̃T | S̃t} := F (t, S̃t)
⇒ The problem is solved if we find Ṽ0 and φt s.t. Ṽt = M̃t a.s.
(need martingale representation for M̃t).
• By Ito’s rule
dM̃t = dF
h (t, S̃t)
i
= Ft(·) + 21 tr{(diag S̃t)ΣtΣ0t(diag S̃t)} Fss(·) dt
+ Fs(·)(diag S̃t)ΣtdwtQ
• Since M̃t is a martingale,

1
0

 Ft(t, s) + 2 tr{(diag s)ΣtΣt(diag s)} Fss(t, s) = 0


F (T, s) = H̃(s)
and one has the explicit martingale representation
Z t
M̃t = M̃0 +
Fs(t, S̃t)(diag S̃t)ΣtdwtQ
0
The problem is thus solved by choosing


 Ṽ0 = M̃0 = E Q{H̃T }

 φ = F (t, S̃ )
t
s
t
Basic idea of the martingale method
Two steps :
i) Determine the optimal value of the cost functional that, for
a given initial capital V0, can be reached by a self financing
portfolio (static optimization under a constraint)
ii) Determine the control/strategy that achieves this optimal value.
→ For step ii) use martingale representation
→ To solve the (static) problem in point i) more possibilities, e.g.:
• Method based on Lagrange multipliers;
• method based on convex duality.
Lagrange multiplier method
max E{U (V )}
V ∈Vv
with
Vv = V | E
Q
{BT−1V
}=v
leads then to
−1
−1
Q
max E{U (V )} − λE {BT V } = max E U (V ) − λLBT V
V
V
Example: U (v) = log v ; single MM Q , Bt = 1
In this case I(y) = y −1
λ = v −1
→
( Z
)
Z
T
T
v
dQ
1
→ VT∗ = with L =
θt0 dwt −
= exp −
θt0 θtdt
L
dP
2 0
0
where θt = Σ−1
t (At − r1), and therefore
E{log VT∗} = log v + E
1
= log v +
2
nR
T
0
Z
0
R
1 T
θt0 dwt + 2
0
θt0 θtdt
o
T
(A0t − 1rt) Σ−2
t (At − rt1) dt
• The optimal investment strategy is now determined as the
hedging strategy for the claim HT = VT∗ = vL−1.
→ Since Bt ≡ 1, all quantities are automatically already
discounted and so, under the unique MM Q one has
dSt = (diagSt) ΣtdwtQ
;
dVt = Vtπt0 ΣtdwtQ
• Determine now πt such that the Q−martingale Vt matches the
following Q−martingale Mt :
Mt := E Q{vL−1 | Ft}
→ need a representation of L−1 under Q
dQ
• From L = dP
n R
o
R
T
T
= exp − 0 θt0 dwt − 12 0 θt0 θtdt and using the
fact that dwtQ = dwt + θtdt, one has
L−1 =
→
dP
dQ
= exp
nR
T
= exp
nR
T
0
0
R
1 T
θt0 dwt + 2
0
o
θt0 θtdt
R
1 T
θt0 dwtQ − 2
0
θt0 θtdt
o

o
nR
R
t
t

−1
1
0
Q
−1
0
Q

θ
L
:=
E
{L
|
F
}
=
exp
θ
dw
−
t
 t
s
2 0 sθsds
0 s


 dL−1 = L−1θ0 dwQ
t
t
t
t
• One can now write
Mt := E Q{vL−1 | Ft} = vL−1
t
From
Q
Q
0
0
dMt = vL−1
θ
dw
=
M
θ
dw
t t
t
t
t
t

Q
0

dV
=
V
π
Σ
dw

t
t t t
t
and
one then has

 dM = M θ0 dwQ
t
t t
t
→
πt0 Σt = θt0
−1
→
πt = Σ−1
t θt
−1
• Recalling that θt := Σt (At−rt1) = Σt At, (Bt ≡ 1 ⇒ rt = 0),
one finally has that
πt = Σ−2
t At
which is constant if At and Σt do not depend on time.
Discussion of DP vs MM
• DP is based (in continuous time) on HJB : first one determines
the optimal control as a function of the (yet unknown) optimal
value; substituting this back into HJB one obtains a nonlinear
PDE that leads to the optimal value. In MM the opposite :
first one determines the optimal value without reference to the
control and then the optimal strategy is determined as a strategy
that leads to this optimal value.
• DP is a fully dynamic procedure by which, provided that the
state process is Markovian and the cost is additive over time, the
optimization over time is reduced to a parameter optimization.
MM is a more static procedure and, in fact, it does not require
Markovianity. In general, MM has however a narrower field of
applicability.
• The dynamic structure of DP makes it better suited to deal with
problems with partial/incomplete information.
• Explicit solutions are not easy to obtain by either of the methods.
For DP there exist approximation methods which is not so much
the case with MM.
Incomplete information/model uncertainty
To obtain an optimal solution for a financial problem one needs a
model. The model may not be perfectly known; on the other
hand, the solution may be rather sensitive to the model.
→ Problem of model uncertainty (model risk)
→ In what follows three possible approaches for hedging and utility
maximization under model uncertainty.
Min-max approach
It is a natural approach, but rather conservative in that it protects
against the worst case scenario.
• Letting P be a family of possible “real world probability
measures” (ambiguity set), consider the following criterion
related to the shortfall risk minimization for the hedging problem
inf sup
π
P ∈P
ESP0,V0
π +
L (HT − VT )
→ may be considered as upper value of a fictitious game
between the market and the agent.
Question : Does this game have a value, i.e. does the upper value
coincide with the lower max-min value
sup inf
P ∈P
π
ESP0,V0
π +
L (HT − VT )
?
Answer : (in general) yes !
→ This approach requires in general a large initial capital and it
does not easily allow to incorporate successive information that
becomes available by observing the market.
Adaptive approaches
(stochastic control under partial information;
stochastic adaptive control)
• Consider parametrized families of models and update successively
the knowledge about the parameters on the basis of observed
prices.
→ Bayesian point of view : updating the knowledge of the
parameters ≡ updating their distributions.
→ The unknown quantities may also be hidden processes ⇒
combined filtering and parameter estimation.
A. A first discrete time case
• Underlying market model (only one risky asset)
• Start from a classical price evolution model in continuous time
that we define under the physical measure P :
dSt = St[adt + Xtdwt]
with wt a Wiener process and where Xt is the non directly
observable volatility process (factor).
• For Yt := log St one then has
dYt =
1 2
a − Xt
2
dt + Xtdwt
• Passing to discrete time (deterministic time points with step δ),
let for n = 0, · · · , N


Xn : Markov chain with m states x1, · · · , xm




(generally resulting from a discretization of Xt).

√

1
2


Yn = Yn−1 + a − 2 Xn−1 δ + Xn−1 δεn



with εn i.i.d. ∼ N (0, 1). (Euler scheme).
→ The pair (Xn, Yn) is Markov with Xn is unobservable factor
(volatility), Yn are the observations (log-prices).
• More generally, consider
Yn = Gn(Xn−1, Yn−1, Xn, εn)
and assume that the distribution of Yn conditional on
(Xn−1, Yn−1, Xn) has a bounded and known density
y0
→
gn(Xn−1, Yn−1, Xn, y 0)
Portfolio optimization and hedging
• Consider an investor who can trade at any time n ≤ N a
number φn ∈ A ⊂ R of shares in the stock, investing the rest
(its monetary amount being denoted by βn) in a riskless asset
with constant interest rate r.
• The corresponding (self financing) wealth process then satisfies
φ
Vn+1
φ
Vn+1
=
Vnφerδ
Yn+1
+ φn e
Yn rδ
−e e
=
φn+1Sn+1+βn+1=(self-fin.)=φnSn+1+βneδr
=
(φnSn+βn)eδr +φn(Sn+1−Sneδr )=Vneδr +φn(eYn+1 −eYn eδr )
More generally, consider
φ
Vn+1
=F
Vnφ, φn, Yn, Yn+1
;
n = 0, · · · , N − 1
with φn adapted to FnY = σ{Y0, · · · , Yn}, (class A).
• Given a time horizon N , as control criterion consider
Jopt(V0) = inf J(V0, φ)
φ∈A
= inf E
φ∈A
(N −1
X
)
fn(Xn, Yn, Vnφ, φn) + `(XN , YN , VNφ)
n=0
which includes portfolio optimization and hedging in incomplete
markets. (By some abuse of notation we denote here by φ a
strategy φ = (φ0, · · · , φN −1) ∈ A).
→ It is a stochastic control problem under partial/incomplete
information.
→ When hedging a payoff h(YN ) at maturity N ,
fn(·) ≡ 0 and
• in case of mean-variance hedging
`(XN , YN , VN ) = (h(YN ) − VN )2
• in case of shortfall risk minimization
`(XN , YN , VN ) = (h(YN ) − VN )+
take
Transition to complete information
• A standard approach to optimization problems under partial
information is to transform them into complete information
ones by replacing the unobserved state variables Xn by their
conditional distributions given past and present observations of
Y (filter distribution).
• Let FnY be the filtration generated by Yj , ( j ≤ n), and let
Πin
i
:= P Xn = x |
FnY
;
i = 1, · · · , m
• By the Markovianity of Xn (denote by Pnij its transition
probability matrix in period n) and by Bayes’ formula one has
(filter dynamics)

Π0 = µ



0
H
(Y
,
Y
)
Πn−1

n
n−1
n

 Πn = H̄n(Πn−1, Yn−1, Yn) :=
| Hn(Yn−1, Yn)0Πn−1|
where µ is the (known) distribution of X0 and
Hnij (Yn−1, Yn) = gn(xi, Yn−1, xj , Yn) Pnij
• On the other hand let Qn(π, y, dy 0) be the law of Yn conditional
on (Πn−1, Yn−1) = (π, y) with density
y0
→
m
X
gn(xi, y, xj , y 0)Pnij π i
i,j=1
→ (Πn, Yn) is a sufficient statistic and an FnY −Markov process.
Solution approach
• By iterated conditional expectations and putting

m
X


i
i
ˆn(π, y, v, φ) =

f
f
(x
,
y,
v,
φ)π

n



ˆ y, v)

`(π,


=
i=1
m
X
`(xi, y, v)π i
i=1
one has
J(V0, φ) =
E{
= E
PN −1
φ
Y
E
f
(X
,Y
,V
,φ
)|F
{
}
n
n
n
n
n
n
n=0
n
oo
φ
Y
+E `(XN ,YN ,VN )|FN
(N −1
X
n=0
)
ˆ N , YN , V φ )
fˆn(Πn, Yn, Vnφ, φn) + `(Π
N
• By the Markovianity of Zn = (Πn, Yn) and the Dynamic
Programming approach, defining recursively the functions

ˆ y, v)

uN (π, y, v) = `(π,




h
ˆn(π, y, v, φ)+
u
(π,
y,
v)
=
inf
f

n

φ∈A


 +E {u
(Π
,Y
, F (v, φ, y, Y
)) | (Π , Y ) = (π, y)}]
n+1
n+1
n+1
n+1
n
n
(where φ here refers to the generic decision φ = φn in period n)
one has
u0(µ, Y0, V0) = Jopt(V0)
→ Requires the conditional law of Zn+1 given Zn.
→ Even if Xn is m−valued, Πn
m−dimensional, ∞−valued simplex
(
Nm =
i
i
takes values in the
π = (π )1≤i≤m | π ≥ 0,
)
X
πi = 1
i
→ To be able to perform actual computations one needs an
approximation leading to a finite-valued process Zn = (Πn, Yn).
Approximations
• The filter distribution Πn in the generic period n may be seen as
a sufficient statistic for (Y0, · · · , Yn) and one may express this
by writing Πn = Πn(Y0, · · · , Yn).
• A
basic traditional approximation approach consists in
approximating each Yj , (j ≤ n), by a discrete r.v. Ŷj and
then approximate Πn(Y0, · · · , Yn) by Πn(Ŷ0, · · · , Ŷn).
→ Problem: the number of possible values grows exponentially
with n (if Ŷj takes M values, then in period n one has M n
possible values of Πn(Ŷ0, · · · , Ŷn).
• Alternatively:
Given a maximum number K of acceptable
discrete values, we perform a quantization of the Markov process
Zn := (Πn, Yn) that leads to its best L2−approximation by a
discrete Markov process Ẑn = (Π̂n, Ŷn) where each Zn takes at
most K values.
→ This approximation of Zn then induces corresponding
approximations in the optimization problem.
• Pagès
G., Pham H. and J.Printemps (2004), “Optimal
quantization methods and applications to numerical problems
in finance”, Handbook of computational and numerical methods
in finance, (S.Rachev, ed.), Birkhäuser Verlag.
• Pham H., W.Runggaldier and A.Sellami (2005), “Approximation
by quantization of the filter process and applications to optimal
stopping problems under partial observation”, Monte Carlo
Methods and Applications, 11, pp. 57–82.
• Corsi,M., Pham H. and W.Runggaldier (2006), “Numerical
Approximation by Quantization of Control Problems in Finance
under Partial Observations”. To appear in : Mathematical
Modelling and Numerical Methods in Finance. Handbook of
Numerical Analysis, Vol XV. (A.Bensoussan, Q.Zhang, eds.).
B. A second discrete time case
• The multinomial case
• Recall
√
1 2
Yn = Yn−1 + a − Xn−1 δ + Xn−1 δεn := Yn−1 + ξn
2
with ξn (conditionally on X) i.i.d. Gaussian.
• Let now
Yn = Yn−1 + ξn
with
ξn i.i.d. multinomial
ξn ∈ {ξ 1, · · · , ξ M } with probability q = (q 1, · · · , q M )
→ Incomplete information: q unknown.
DP under complete and incomplete information
• Recall (complete information about q)







ΦN (v)=maxπN U (v,πN )
Φn(v)=maxπn [U (v,πn)+E{Φn+1(G(Vn,πn,ξn+1))|Vn=v}]






=maxπn [U (v,πn)+
PM
m=1 q
m
Φn+1(G(Vn,πn,ξ m))|Vn=v ]
• Not knowing q (knowing however FnY equiv. Y0n)
PM
n
Φn(v,Y0 )=maxπn U (v,πn)+ m=1 E{q m|Y0n}Φn+1(G(v,πn,ξ m))
[
→ Need only Bayesian updating of E {q m | Y0n}
]
C. A continuous time case
• Utility from terminal wealth; no consumption
• Given is the financial model (rt = 0 ⇒ Bt = const.)


 dSt = St [at(St, Xt)dt + σt(St)dwt]
dXt = Ft(Xt)dt + Rt(Xt)dMt

 dVt = Vt [πtat(Xt)dt + πtσtdwt]
(Recall that, by self financing,
dVt
Vt
t
= πt dS
St )
→ Mt a martingale independent of wt
Rt 2
→ σt is independent of Xt : in continuous time 0 σs ds can be
estimated by the empirical quadratic variation (dependence on
Xt ⇒ filter degenerates)
• Putting Zt := log St, consider the (specific) problem


dXt = Ft(Xt)dt + Rt(Xt)dMt (unobserved)









 dZt = At(Zt, Xt)dt + B(Zt)dwt (observed)
1

2

dVt = Vt πt At(Zt, Xt) + 2 Bt (Zt) dt + πtBt(Zt)dwt







µ

 sup E {VT } , µ ∈ (0, 1)
π
dVt
dSt
deZt =π
1 B 2 (Z ) dt+B (Z )dw
=π
=π
A
(Z
,X
)+
[(
t )
t
t
t]
t
t
t
t
t
t
2 t
Vt
St
eZ t
Reformulation of the incomplete information problem
(“separated problem”)
• Take as new “state”
Ψt = pt(x) = p(Xt | FtZ )| Xt=x
→ filter distribution of Xt given FtZ
• For φt = φt(Xt) let
pt(φ) := E φ(Xt) |
FtZ
Z
=
φ(x)dpt(x)
• Putting
Z
At(Zt, pt) := pt(At) =
At(Zt, x)dpt(x)
define the “innovations process” (a Wiener process in the
filtration FtZ ))
dw̄t := Bt−1(Zt) [dZt − pt(At)dt]
→ It implies a translation of the (P, Ft)−Wiener wt :
dw̄t = dwt + Bt−1(Zt) [At(Zt, Xt) − At(Zt, pt)] dt
and thus the implicit measure transformation P → P̄ with
dP̄
=
dP | FT
(Z
T
[At(Zt, pt) − At(Zt, Xt)] Bt−1(Zt)dwt
= exp
0
− 21
Z
0
T
)
2
[At(Zt, pt) − At(Zt, Xt)] Bt−2(Zt)dt
• Even if Xt is finite-dimensional, Ψt is in general ∞−dimensional.
• DP (HJB-eqn.) difficult for an ∞−dimensional state Ψt
→ In some cases Ψt = pt(x) = p(Xt | FtZ )| Xt=x is finitely
parametrized (finite-dimensional filter)
Examples
• Linear-Gaussian models


 dXt = FtXtdt + Rtdvt

 dZt = AtXtdt + Btdwt
{vt}, {wt} independent standard Wiener.
→ p(Xt | FtZ )| Xt=x ∼ N (mt, γt)

At

dm
=
F
m
dt
+
γ
t
t t
t Bt dw̄t




 γ̇t
= 2Ftγt − γt2
with dw̄t := Bt−1[dZt − Atmt dt]
2
At
Bt
+ Rt2
• Xt : finite-state Markov with states {s1, · · · , sK }, i.e.
dXt = ΛtXtdt + dMt
→ Put φi(x) =
1
0
if
if
pit
x = si
and let
x 6= si
:= P Xt =
si | FtZ
Furthermore, let
Dt(Zt) := diag(At(Zt, s1), · · · , At(Zt, sK ))
At(Zt) := [At(Zt, s1), · · · , At(Zt, sK )]0
0
then, with pt = [p1t , · · · , pK
]
t ,
dpt = Λtptdt+[Dt(Zt)−(A0t(Zt)pt) I]ptBt−2(Zt)(dZt −A0t(Zt)ptdt)
→ pt evolves on a finite-dimensional simplex.
Reformulation in case of a general finite-dimensional filter
• Let
pt(x) = p(Xt | FtZ )| Xt=x = p(x; ξt) ;
ξt ∈ Rp
and suppose that
dξt = βt(Zt, ξt)dt + δt(Zt, ξt) dw̄t
• Putting
Z
At(Zt, ξt) :=
At(Zt, x)dp(x; ξt)
on (Ω, F, Ft, P̄ ) with Wiener w̄t :


dξt = βt(Zt, ξt)dt + δt(Zt, ξt) dw̄t









 dZt = At(Zt, ξt)dt + Bt(Zt)dw̄t
1

2

dVt = Vt πt At(Zt, ξt) + 2 Bt (Zt) dt + πtBt(Zt)dw̄t







µ

 sup Ē {VT } , µ ∈ (0, 1)
π
→ It is the “separated problem” (equivalent complete information
problem). With Yt := [Zt, ξt, Vt] it is of the form of the general
complete information problem.
→ Other reformulations are possible e.g. as a risk sensitive control
problem (Nagai-R.07).
Hedging under incomplete information
(quadratic criterion)
For the quadratic hedging criterion
π 2
min ES0,V0 (HT − VT )
π
one has : if φ∗t (Xt, Zt) is the optimal strategy under full
Z
information then, under the partial information
F
the optimal
t
∗
strategy is basically the projection E φt (Xt, Zt) | FtZ .
→ For mathematically less tractable criteria one may thus first
obtain an optimal strategy corresponding to the quadratic
criterion and then evaluate by simulation its performance relative
to the original criterion, possibly adjusting it heuristically.
Robust approaches
• Investigate the sensitivity of the solution with respect to the
model.
• How reliable is the solution, obtained for a hypothetical model,
when applied to the real problem ?
Assume that for a “real world probability measure” P the problem
is


dXt = At(Xt, Zt, πt)dt + Σt(Xt, Zt, πt)dwt





 P
J (X0, π ∗) = inf J P (X0, π)
π(
)

Z

T


P

=
inf
E
c(Xt, πt)dt + C(XT )

X0

π
0
where one may e.g. think of Xt as Xt = [St, It, Vt]0 and of Zt as
a hidden process.
Assume that, not knowing the real measure P , one solves instead
the same problem for a hypothetical measure Q :

+ Σt(Xt, Zt, πt)dwt

 dXt = At(Xt, Zt, πt)dt(
)
Z
T
Q
Q
Q
J
(X
,
π
)
=
inf
E

0
X0

π
c(Xt, πt)dt + C(XT )
0
Problem : Find π Q and a bound (uniform in X0) on
J P (X0, π Q) − J P (X0, π ∗) ≥ 0
i.e. on the suboptimality of π Q, when applied to the real problem,
in terms of a measure of the difference between P and Q.
Discontinuous Market Model
Kirch (03), K/R (04)
• On (Ω, F, (Ft), P ) and for t ∈ [0, T ], let


 Bt
≡ 1 »
–
R t PM ji
ji
P
j
j
M
a
i
= St− i=1(e −1)dNt =S0 exp[ 0 i=1 a dNti]


=
j
dSt
P
j
ji i
S0 exp[ M
i=1 a Nt ] , j=1,··· ,N
Nti : Poisson without common jumps and (P, Ft)−intensity λit
aji : deterministic constants and so FtS = FtN
• A counting/point process Nt is a Poisson process if
i) N0 = 0;
ii) Nt is a process with independent increments;
iii) Nt − Ns is a Poisson random variable with a parameter Λs,t.
Rt
• Usually Λs,t = s λudu
→ λt : intensity of the Poisson (point) process
→ If Nt is a Poisson process with intensity λt = λ, then τn+1 − τn
are i.i.d. exponential r.v.’s with parameter λ.
• λt may be itself an adapted process and this corresponds to a
two-step randomization procedure :
i) draw at random a trajectory of λt;
ii) generate a Poisson process Nt having that λt as intensity.
• In this case one obtains a doubly stochastic Poisson process or a
Cox process.
• Just as the Wiener process is the basic building block for
processes with continuous trajectories, the Poisson process is
it for processes with jumping trajectories.
• The Wiener process is itself a martingale
• The Poisson process becomes a martingale by subtracting its
mean, i.e.
t
Z
Mt := Nt −
λsds is an Ft − martingale
0
t
Z
→
E {Nt − Ns | Fs} = E
λudu | Fs
s
Z
↔
E
∞
CsdNs
0
∀ Ft−predictable processes Ct.
Z
=E
0
∞
Csλsds
• Self-financing portfolio (no consumption)
dVt
=
Vt−
N
X
j
dS
πtj j t
St−
j=1
=
N
X
j=1
M
X
j
πt
aji
(e
− 1)dNti
i=1
πtj : fraction of wealth invested in S j at time t
PN
j aji
(assume j=1 πt (e − 1) ≥ −1 ; (i = 1, · · · , M ))




Z T
M
N
Y
X
j aji


exp
πt (e − 1) dNti
→ VT = V0
log 1 +
i=1
→
0
j=1

PM R T
PN
ji
j a
i

log
V
=
log
V
+
log
1
+
π
(e
−
1)
dN

T
0
t
i=1 0
j=1 t

hR
µ
i

Q
P
ji

T
N
j a
i
 V µ = V µ M exp
log
1
+
π
(e
−
1)
dN
t
t
0
T
i=1
j=1
0
Log-utility (Problem formulation)
a) πt ∈ Ft (full information also of λt)
sup E {log VT } = log V0
π
+ sup
π
M
X

Z
E

i=1
T

log 1 +
0
N
X

j aji
πt (e
− 1) dNti



j=1
= log V0
+ sup E
π

Z

0
M
T X
i=1

log 1 +
N
X
j=1

j aji
πt (e
− 1) λitdt



b) πt ∈ FtS ⊂ Ft
λ̂it := E{λit | FtS } = E{λi(Xt) | FtS }
;
sup E {log VT } = log V0
π
+ sup
π
M
X

Z
E

i=1
T

log 1 +
0
N
X

j aji
πt (e
− 1) dNti



j=1
= log V0
+ sup E
π

Z

0
M
T X
i=1

log 1 +
N
X
j=1

j aji
πt (e
− 1) λ̂itdt



Log-utility (Solution)
It suffices to perform, for each t ∈ [0, T ] and each ω ∈ Ω,
max
π
M
X

log 1 +
i=1
N
X

j aji
πt (e
− 1) λit
(resp. with λ̂it)
j=1
m
M
X
i=1
1+
i aji
λt(e − 1)
PN
`(ea`i
π
`=1 t
→ CE-type property
− 1)
= 0 ; j = 1, · · · , N
(resp. with λ̂it)
→ In the special case of M = 2 : a linear system of N equations
in the N unknowns hjt
PN
j aji
(constraint j=1 πt (e − 1) ≥ −1 ; (i = 1, · · · , M ))
→ Same result with DP (generator of λt does not depend on h)
→ In the full information case, using convex duality, the
maximization over the MMs leads to a linear system in the
parameters indexing the MMs whenever M = N + 1.
POWER UTILITY
Recall first the measure transformation for jump processes (scalar
case)
Theorem : Let Nt be a Poisson process
R t with (P, Ft)−intensity
λt. Let ψt ≥ 0 be Ft−predictable s.t. 0 ψsλsds < ∞ P −a.s. ∀t.
Let
dLt = Lt−(ψt − 1)(dNt − λtdt) ; i.e.
hR
i
R
t
t
Lt = exp 0 (1 − ψs)λsds + 0 log(ψs) dNs
= exp
hR
iQ
t
Nt
(1
−
ψ
)λ
ds
s s
n=1 ψTn
0
If E P {Lt} = 1, ∀t ≥ 0 then ∃ a measure Q ∼ P with
dQ|Ft = LtdP|Ft s.t. Nt has (Q, Ft)−intensity λtψt.
POWER UTILITY
Formulation of the problem as risk-sensitive control problem
a) πt ∈ Ft , predictable (full information also of λt)
E {VTµ} = V0µ E
=
V0µ
E
nQ
M
+
−
RT h
0
i=1 exp
i=1 exp
RT h
0
nQ
M
hR
T
0
hR
T
0
PN
ji
PN
ji
1− 1+
log 1 +
log 1 +
1− 1+
j a
π
j=1 t (e
j a
π
j=1 t (e
ji
PN
j a
π
j=1 t (e
j aji
j=1 πt (e
PN
io
µ
− 1) dNti
µ
− 1) dNti
µ i
− 1)
λitdt
µ i
io
− 1)
λitdt
⇒ E {VTµ}
=
V0µ
µ
i
io
n
hR P h
P
ji
T
M
N
j
a
i
π
(e
−
1)
−
1
λ
E π exp 0
1
+
tdt
i=1
j=1 t
→ Under P h ∼ P the intensity of Nti becomes

µ
N
X
πtj (eaij − 1) .
λit 1 +
j=1
The law of λt however remains the same.
j aji
−1)
j=1 πt (e
PN
→ Condition
≥ −1; (i = 1, · · · , M ) is sufficient
for ∃ of the R.N.-derivative having mean equal to 1.
b) πt ∈ FtS ⊂ Ft , predictable (being aji known, FtS = FtN for
Nt = (Nt1, · · · , NtM ))
→ Since πt ∈ FtS the same previous derivation is valid with the
Ft−intensities λit replaced by the FtS −intensities
λ̂it = E{λit | FtS }.
⇒ E {VTµ}
n
hR P h
µ
i
io
P
ji
T
M
N
j a
i
= V0µ E π exp 0
1
+
π
(e
−
1)
−
1
λ̂
t
tdt
i=1
j=1
and the P h−intensities are λ̂it 1 +
µ
j aij
π
− 1)
j=1 t (e
PN
POWER UTILITY (Solution under full information)
It suffices to perform, for each t and each ω ∈ Ω,
max
h
M
X

1 +
i=1
i aji
λt(e
i=1
1+
N
X
`=1
` a`i
πt (e
j aji
πt (e

− 1) − 1 λit
j=1
m
M
X
N
X
µ
(for µ ∈ (0, 1))
− 1)
(
!1−µ = 0,
− 1)
j = 1, · · · , N,
PN
j aji
− 1) ≥ −1
j=1 πt (e
Same result with DP
Define the value function
(
w(t, λ) := sup log E
"Z
)
C(πs, λs)ds | λt = λ
exp
π
#
T
t
where
C(πt, λt) :=
M
X
i=1

1 +
N
X
µ
j aji
πt (e

− 1) − 1 λit
j=1
and notice that, although the measure transformation P → P h
changes the intensities of Ntj , it does not change the law of λt.
The corresponding HJB equation is then given by



∂
∂t w(t, λ)
+ Ltw(t, λ) + supπ [C(π, λ)] = 0

 w(T, λ) = 0
with Lt the generator corresponding to (λt).
POWER UTILITY (Solution under incomplete information)
A. Transition to complete information.
Approach based on a “Zakai-type” equation ([Nagai/Peng])
• Let λit = λi(Xt) with Xt a K−state Markov process with
intensity matrix Q. (Notice also that (λi(Xt) ≤ λ̄).
d P̂
• Change of measure P → P̂ such that
= Lt with
d P |Ft
Lt =
M
Y
i=1
Z
exp
0
t
(λi(Xs) − 1)ds −
Z
t
log λi(Xs−) dNsi
0
→ the P̂ −intensities become λi(Xt) ≡ 1 and, under P̂ , Xt and
Nti are independent ∀i = 1, · · · , M .
We may then write
E{VTµ} =
V0µÊ
nQ
M
i=1 exp
hR T
0
log 1 +
µ
j aij
π
− 1)
j=1 t (e
PN
io
R
T
+ log λi(Xt−) dNti − 0 (λi(Xt) − 1)dt
n
h P R
io
R
T
T
M
i
i
i
= V0µÊ exp
log
Γ
(π
,
X
)
dN
−
(λ
(Xt) − 1)dt
t
t−
t
i=1
0
0
having put

Γi(πt, Xt) := λi(Xt) 1 +
N
X
j=1
µ
πtj (eaij − 1) ≤ Γ̄.
µ
• Motivated by the RHS for E{VT } consider the process
Ht := exp
M Z
hX
i=1
t
log Γi(πs, Xs−) dNsi −
t
Z
0
(λi(Xs) − 1)ds
i
0
for which Ê {Ht} ≤ eM T (Γ̄+1) := q̄ and put
S
qt(k) = Ê 1{Xt=k}Ht | Ft ≤ q̄ , k = 1, · · · , K
so that
E{VTµ} = V0µÊ{HT } = V0µÊ{Ê{HT | FTS }}
= V0µ
K
X
k=1
(
Ê{Ê{1{XT =k}HT | FTS }} = V0µÊ
K
X
k=1
)
qT (k)
(≤V0µK q̄)
Zakai equation
Using Ito’s formula on Ht and properties of finite-state Markov
chains (the law of Xt is the same under P and P̂ ) one obtains
dqt(k) = (Q0qt)(k)dt + qt(k)
M
X
(1 − λi(k))dt
i=1
+ qt−(k)
M
X
(Γi(ht, k) − 1) dNti
i=1
qt(k)
→ One has that PK
j=1 qt(j)
= P {Xt = k | FtS } so that qt(k) has
the interpretation of an unnormalized conditional probability.
→ qt(k) can be shown to be bounded.
B. The corresponding complete information pb and its solution
qt = [qt(1), · · · , qt(K)], the complete information
problem corresponding to the original portfolio optimization
problem can be synthesized as (recall FtS = FtN )
• Putting

(K
)

X


µ

qT (k)
max V0 Ê


π


k=1









M
X
j
0

dq
=
Q
+
[I
−
diag(λ
(k))] qtdt
t




j=1






m

X

j
j




+
diag(Γ
(π
,
k))
−
I
q
dN
t
t−

t


j=1
→ The dynamics of qt are under P̂ for which λj (Xt) ≡ 1, ∀j.
→ It is a problem of the type of piecewise deterministic control
problems and can thus be approached by one of the general
techniques for such problems. For our particular situation one
can however adapt an approach from [Kirch-R.,2004] leading to
an algorithm of the type of value iteration for infinite-horizon
MDP’s.
j
• At the generic jump time τ j of Nt one has
qτ j = diag(Γj (hτ j , k)) q(τ j )−
• Between two generic jump times of the multivariate jump process
Nt = (Nt1, · · · , NtM ), i.e. for t ∈ [τn, τn+1) one has the
deterministic evolution



M
X
dqt = Q0 + M I − diag 
λj (k) qtdt := Λ qtdt
j=1
so that, for t ∈ [τn, τn+1),
qt = exp[Λ (t − τ )] · qτ
with Λ as defined above.
• Recalling that under P̂ one has λj (Xt) ≡ 1, the following holds
Ê f (qτn+1 ) | qτn = q


Z T X
M

=
f diag(Γj (πt, k))eΛ(t−τn)q  e−M (t−τn)dt
τn
j=1
Λ(T −τn)
+f e
where, as before,
q e−M (T −τn)

Λ := Q0 + M I − diag 
M
X
j=1

λj (k)
• Since qt < q̄ 1, consider as state space
K
E = (q, t) | q ∈ R , 0 < qt < q̄ 1, t ∈ [0, T ]
• For J : E → R+ define the operator Ψ mapping J to ΨJ :
E → R+ by


Z T −t
M
X
−M s
j
Λs

(ΨJ)(q, t) =
e
max
J diag(Γ (π, k))e q, t + s  d
0
h
j=1
+ e−M (T −t)
K X
eΛ(T −t)q
k
k=1
→ The last term on the right is motivated by the objective
function.
→ The operator Ψ is such that
Ψ : C(E)
−→
C(E)
and it is a contraction operator with contraction constant
1 − e−M T .
• Given n ∈ N, let
J0 = 0
and, for j ≤ n,
J j = ΨJ j−1
and let (πtn)t∈[0,T ] be the strategy induced by computing
J n(q0, 0).
• Define
(
J ∗,n(q, t) = max Ê
π
K
X
)
qT (k) , τn > T | qt = q
.
k=1
→ It is the optimal value for the problem obtained from the
original one by replacing Ω with Ω ∩ {τn > T }.
→ One can show that J n(q, t) = J ∗,n(q, t) ,
∀(q, t) ∈ E.
(
• For J ∗(q, t) = max Ê
π
K
X
)
qT (k) | qt = q
one has
k=1
J n(q, t) ≤ J ∗(q, t) ≤ J n(q, t) + K q̄ P̂ {τn ≤ T }
with q̄ the bound on qt(k).
i) J ∗
Main theorem
(K
)
X
∗
= J (q, 0) = max Ê
qT (k) | q0 = q
is the unique
π
k=1
fixed point of the operator Ψ, i.e.
∗
J = ΨJ
∗
∗
n
MT
and ||J − J || ≤ e
−M T n
1−e
||J 1||
ii) The optimal π ∗ is


M
X
πt∗ = argmaxπ 
J ∗ diag(Γj (π, k))eΛ(t−τn)q, t 
j=1
h
with π admissible and Λ = Q0 + M I − diag
P
i
M
j
λ
(k)
j=1
iii) Let J˜n = J˜n(q, 0) be the value function of the original problem
corresponding to the strategy (πtn). Then
J n ≤ J˜n ≤ J n + K q̄ P̂ {τn ≤ T }
→ Since lim P̂ {τn ≤ T } = 0, it follows from the Theorem that
n→∞
lim J˜n = J ∗, i.e. the strategy (πtn), induced by computing the
n→∞
n−th iterate J n(q, 0) of Ψ, is, for n sufficiently large, nearly
optimal in the original problem.
→ To compute J n and the corresponding strategy (πtn), one thus
follows an approach of the type of value iteration for infinitehorizon MDP’s
→ Quantization (with convergence) to actually compute (πtn).
• Nagai H. and S. Peng (2002), “Risk-sensitive dynamic portfolio
optimization with partial information on infinite time horizon”,
Annals of Applied Probability, 12, pp. 173–195.
• Kirch M. and W.J.Runggaldier (2004), Efficient hedging when
asset prices follow e geometric Poisson process with unknown
intensities, SIAM J. Control and Optimiz.,43, pp. 1174–1195.
• Callegaro G.,
Di Masi G.B. and W.J.Runggaldier (2006),
“Portfolio optimization in discontinuous markets under
incomplete information”, Asia Pacific Financial Markets, 13/4
(2006), pp. 373–394.