Kriging of financial term-structures
Adaptive robust control of Markov decision process
Model uncertainty in finance :
Term-structure construction and hedging issues
Areski Cousin
ISFA, Université Lyon 1
Séminaire équipe TOSCA
INRIA Sophia-Antipolis, November 24, 2016
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
1/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Contents
1
Kriging of financial term-structures
2
Adaptive robust control of Markov decision process
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
2/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Motivation
Financial term-structures describes the evolution of some financial or
economic quantities as a function of time horizon.
Examples : interest-rates, bond yields, credit spreads, implied default
probabilities, implied volatilities.
Applications : valuation of financial and insurance products, risk
management
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
3/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
The term-structure construction problem
Several constraints have to be considered
Compatibility with market information : at a given date t0 , the curve
under construction T → P(t0 , T ) shall be compatible with observed prices
of some reference products.
Arbitrage-free construction : this translates into some specific shape
properties such as positivity, monotonicity, convexity or bounds on the
curve values
Additional conditions can be required : minimum degree of smoothness,
control of local convexity
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
4/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
The term-structure construction problem
1) Compatibility with market information :
At time t0 , we observe the market quotes S1 , . . . , Sn of n liquidly traded
instruments
The values of these products depend on the value of the curve at m input
locations X = (τ1 , . . . , τm )
The vector of output values P(t0 , X ) := (P(t0 , τ1 ), . . . , P(t0 , τm ))> satisfies a
linear system of the form
A · P(t0 , X ) = b,
where
A is a n × m real-valued matrix
b is a n-dimensional column vector
n < m =⇒ indirect and partial information on the curve values at τ1 , . . . , τm
2) No-arbitrage assumption :
T → P(t0 , T ) is typically a monotonic bounded function
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
5/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Range of arbitrage-free OIS discount curves
We observe the quoted par rates Si of an OIS with maturities Ti , i = 1, . . . , n
1) Compatibility with market quotes :
The curve T → P(t0 , T ) of OIS discount factors is such that
pi −1
Si
X
δk P(t0 , tk ) + (Si δpi + 1) P(t0 , Ti ) = 1, i = 1, ..., n
k=1
t1 < · · · < tpi = Ti : fixed-leg payment dates (annual time grid)
δk : year fraction of period (tk−1 , tk )
2) No-arbitrage assumption :
T → P(t0 , T ) is a decreasing function such that P(t0 , t0 ) = 1
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
6/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Range of arbitrage-free OIS discount curves
n = 14 liquidly traded maturities 1, 2, . . . , 10, 15, 20, 30, 40 years.
m = 40 points involved in the market-fit linear system
No-arbitrage bounds on OIS discount factors
Bounds for OIS discounting curves
1
0.9
OIS discount factors
0.8
0.7
0.6
0.5
0.4
0
5
10
15
20
25
timeïtoïmaturity
30
35
40
Input data : OIS swap rates as of May, 31st 2013.
Source : Cousin and Niang (2014)
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
7/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Range of arbitrage-free CDS-implied survival functions
We observe at time t0 the fair spreads Si of a CDS with maturities Ti ,
i = 1, . . . , n
1) Compatibility with market quotes :
The curve T → P(t0 , T ) of (risk-neutral) survival probabilities is such that
Si
pi
X
Z
Ti
δk D(t0 , tk )P(t0 , tk ) = −(1 − R)
D(t0 , u)dP(t0 , u), i = 1, ..., n
t0
k=1
t1 < · · · < tp = Ti : trimestrial premium payment dates, δk : year fraction
of period (tk−1 , tk )
D(t0 , T ) is the discount factor associated with maturity date T
R : expected recovery rate of the reference entity
2) No-arbitrage assumption :
T → P(t0 , T ) is a decreasing function such that P(t0 , t0 ) = 1
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
8/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Range of arbitrage-free CDS-implied survival functions
n = 4 liquidly traded maturities 3, 5, 7, 10 years.
m = 40 points involved in the market-fit linear system
No-arbitrage bounds on the issuer implied survival distribution function
Bounds for CDS implied survival probabilities
1
CDS implied survival probability
0.99
0.98
0.97
0.96
0.95
0.94
0.93
0.92
0.91
0
1
2
3
4
5
6
timeïtoïmaturity
7
8
9
10
Input data : CDS spreads of AIG as of December 17, 2007, R = 40%,
D(t, T ) = exp(−3%(T − t))
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
9/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
From spline interpolation to kriging
In practice, financial term-structures are constructed using deterministic
interpolation techniques.
Parametric approaches : Nelson-Siegel or Svensson models (used by most
central banks)
Non-parametric interpolation methods : shape-preserving spline techniques
(lack of interpretability but better ability to fit the data).
Could we propose an arbitrage-free interpolation method that additionally
allows for quantification of uncertainty ?
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
10/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Classical kriging
A function f is only known at a limited number of points x1 , . . . , xn
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
11/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Classical kriging
The (unknown) function f is assumed to be a sample path of a Gaussian
process Y
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
12/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Classical kriging
Definition : Gaussian process (GP) or Gaussian random field
A Gaussian process is a collection of random variables, any finite number of
which have (consistent) joint Gaussian distributions.
A Gaussian process Y (x), x ∈ Rd is characterized by its mean function
µ : x ∈ Rd −→ E(Y (x)) ∈ R.
and its covariance function
K : (x, x 0 ) ∈ Rd × Rd −→ Cov(Y (x), Y (x 0 )) ∈ R.
1D kriging kernel
Gaussian
Matérn 5/2
Matérn 3/2
Exponential
K(x, x 0 )
0 2
)
σ exp − (x−x
2θ 2
√
√
0
0 2
0
)
|
|
σ 2 1 + 5|x−x
+ 5(x−x
exp − 5|x−x
θ
θ
3θ 2
√
√
0
0
|
|
σ 2 1 + 3|x−x
exp − 3|x−x
θ
θ
0
|
σ 2 exp − |x−x
θ
Areski Cousin, ISFA, Université Lyon 1
2
Model uncertainty in finance
Class
C∞
C2
C1
C0
13/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Classical kriging
The estimation of f relies on the conditional distribution of Y given the
observed values yi = f (xi ) at points xi , i = 1, . . . , n.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
14/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Classical kriging
X = (x1 , . . . , xn )> ∈ Rn×d : some design points
y = (y1 , . . . , yn )> ∈ Rn : observed values of f at these points
Y (X ) = (Y (x1 ), . . . , Y (xn ))> : vector composed of Y at point X
The conditional process is still a Gaussian Process
Let Y be a GP with mean µ and covariance function K . The conditional
process Y | Y (X ) = y is a GP with mean function
η(x) = µ(x) + k(x)> K−1 (y − µ),
x ∈ Rd
and covariance function K̃ given by
K̃ (x, x 0 ) = K (x, x 0 ) − k(x)> K−1 k(x 0 ),
x, x 0 ∈ Rd
where µ = µ(X ) = (µ(x1 ), . . . , µ(xn ))> , K is the covariance matrix of Y (X )
and k(x) = (K (x, x1 ) , . . . , K (x, xn ))>
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
15/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to linear equality constraints
Recall that, in our term-structure construction problem, the (unknown) real
function f satisfies some linear equality constraints of the form
A · f (X ) = b,
(1)
where
A is a given matrix of dimension n × m
X = (x1 , . . . , xm )> ∈ Rm×d
f (X ) = (f (x1 ), . . . , f (xm ))> ∈ Rm
b ∈ Rn
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
16/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to linear equality constraints
X = (x1 , . . . , xm )> ∈ Rm×d : some design points
b = (b1 , . . . , bn )> ∈ Rn : right-hand side of the linear system
Y (X ) = (Y (x1 ), . . . , Y (xm )) : vector composed of Y at point X
The conditional process is still a Gaussian Process
Let Y be a GP with mean µ and covariance function K . The conditional
process Y | AY (X ) = b is a GP with mean function
−1
η(x) = µ(x) + (Ak(x))> AKA>
(b − Aµ),
x ∈ Rd
and covariance function K̃ given by
−1
K̃ (x, x 0 ) = K (x, x 0 ) − (Ak(x))> AKA>
Ak(x 0 ),
x, x 0 ∈ Rd
where µ = µ(X ) = (µ(x1 ), . . . , µ(xm ))> , K is the covariance matrix of Y (X ),
k(x) = (K (x, x1 ) , . . . , K (x, xm ))>
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
17/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to monotonicity constraints
New formulation of the problem : estimation of an unknown function f given
that
A · f (X ) = b
f ∈M
where M is the set of (say) non-increasing functions.
Problem : The conditional process is not a Gaussian process anymore.
How to cope with the infinite-dimensional monotonicity constraints ?
Which estimator could we propose for the term-structure ?
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
18/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to monotonicity constraints (1D case)
Proposed methodology : On an interval D = [x, x] of R, we construct a
finite-dimensional approximation of Y for which the monotonicity constraint is
easy to check.
Regular subdivision u0 < . . . < uN of D with a constant mesh δ
Set of increasing basis functions (φi )i=0,...,N defined on this subdivision
hi (x) := max 1 −
|x−ui |
,0
δ
Areski Cousin, ISFA, Université Lyon 1
φi (x) =
Rx
Model uncertainty in finance
x
hi (u)du
19/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to monotonicity constraints (1D case)
Proposition (Maatouk and Bay, 2014b)
Let Y be a zero-mean GP with covariance function K and with almost surely
differentiable paths.
The finite-dimensional process Y N defined on D by
Y N (x) = Y (u0 ) +
N
X
Y 0 (uj )φj (x)
j=0
uniformly converges to Y , almost surely.
Y N is non-decreasing (resp. non-increasing) on D if and only if
Y 0 (uj ) ≥ 0 (resp. Y 0 (uj ) ≤ 0) for all j = 0, . . . , N.
Let ξ := (Y (u0 ), Y 0 (u0 ), . . . , Y 0 (uN ))> , then ξ ∼ N (0, ΓN ) where


∂K
(u0 , uj )
K (u0 , u0 )
∂x 0

ΓN = 
∂K
∂2K
(ui , uj ) 0≤i,j≤N
(ui , u0 )
∂x
∂x∂x 0
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
20/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to monotonicity constraints (1D case)
For a given covariance function K , we assume that the unknown function f is a
sample path of the GP
Y N (x) = η +
N
X
ξj φj (x),
x ∈ D,
j=0
where ξ := (η, ξ0 , . . . , ξN )> ∼ N (0, ΓN ).
Kriging f is equivalent to find the conditional distribution of Y N given
A · Y N (X ) = b
linear equality condition
ξj ≤ 0, j = 0, . . . , N
monotonicity constraint
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
21/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to monotonicity constraints (1D case)
Or equivalently, to find the distribution of the truncated Gaussian vector
ξ ∼ N (0, ΓN ) given
A·Φ·ξ =b
linear equality condition
ξj ≤ 0, j = 0, . . . , N
monotonicity constraint
where Φ is a m × (N + 2) matrix defined as
1
for i = 1, . . . , m and j = 1,
Φi,j :=
φj−2 (xi ) for i = 1, . . . , m and j = 2, . . . , N + 2.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
22/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to monotonicity constraints (1D case)
Which estimator could we use for f ?
We consider the mode of the truncated gaussian process (most probable path) :
MKN (x | A, b) = ν +
N
X
νj φj (x),
j=0
where ν = (ν, ν0 , . . . , νN )> ∈ RN+2 is the solution of the following convex
optimization problem :
1 > N −1
ν = arg min
c
Γ
c ,
c∈C∩I(A,b)
2
with
C = ξ ∈ RN+2 : ξj ≤ 0, j = 0, . . . , N
I(A, b) = ξ ∈ RN+2 : A · Φ · ξ = b
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
23/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Extension to monotonicity constraints (1D case)
Efficient simulation of the truncated Gaussian vector
1) Simulate a truncated vector ξ given the linear equality constraint :
Z ∼ {ξ | B · ξ = b} ∼ N
N >
(BΓ )
N
BΓ B
>
−1
> −1
N
N >
N
b, Γ − BΓ
BΓ B
BΓ
N
where B = A · Φ.
2) Simulate
{Z | ξj ≤ 0, j = 0, . . . , N} ∼ {ξ | B · ξ = b and ξj ≤ 0, j = 0, . . . , N}
by an accelerated rejection sampling method (we use the method proposed in
Maatouk and Bay, 2014a)
P
3) The corresponding sample curves Y N (·) = η + N
j=0 ξj φj (·) satisfies the
constraints on the entire domain D.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
24/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Kriging of OIS discount curves
We compare two covariance functions : Gaussian and Matérn 5/2
Hyper-parameters θ and σ are estimated using cross-validation
1.0
discount factor
0.8
monotone GP
95% confidence interval
mode
Nelson−Siegel
Nelson−Siegel−Svensson
0.4
0.6
0.4
discount factor
0.8
monotone GP
95% confidence interval
mode
Nelson−Siegel
Nelson−Siegel−Svensson
0.6
1.0
Comparison with Nelson-Siegel and Svensson curve fitting
0
10
20
30
40
0
10
x
20
30
40
x
Discount curves. N = 50, 100 sample paths. Left : Gaussian covariance
function. Right : Matérn 5/2 covariance function. OIS data of 03/06/2010.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
25/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Kriging of OIS discount curves
0.035
0.030
0.025
0.020
spot rate
spot rate sample paths
95% confidence interval
spot rate mode
Nelson−Siegel
Nelson−Siegel−Svensson
0.010
0.015
spot rate sample paths
95% confidence interval
spot rate mode
Nelson−Siegel
Nelson−Siegel−Svensson
0.005
spot rate
0.005 0.010 0.015 0.020 0.025 0.030 0.035
Corresponding spot rate curves : − 1x log P(x)
0
10
20
30
40
0
10
x
20
30
40
x
Spot rate curves. Left : Gaussian covariance function. Right : Matérn 5/2
covariance function. OIS data of 03/06/2010. The black solid line is the most
likely spot rate curve − 1x log MKN (x | A, b).
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
26/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Kriging of OIS discount curves
d
log P (x)
Corresponding forward rate curves : − dx
Spot rate curves. Left : Gaussian covariance function. Right : Matérn 5/2
covariance function. OIS data of 03/06/2010. The black solid line is the most
d
likely forward rate curve − dx
log MKN (x | A, b).
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
27/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Kriging of OIS discount curves (2D)
The previous approach can be extended in dimension 2.
1.0
0.9
1.0
0.8
0.8
0.7
0.6
0.6
0.4
0.5
0
10
20
time−to−maturity 30
2 201
201011−10−06−1
1−0 5−1 0
1−3 0
1
201
0−1
2−3
0
201
0−1
1−2
201
0−0
9
8−0
201
3
0−0
dates
7−0
5
0.4
0.3
201
40 0−06−
02
0.2
Dicount curves. OIS discount factors as a function of time-to-maturities and
quotation dates.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
28/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Kriging of CDS-implied default distribution
Implied survival function of the Russian sovereign debt
CDS implied survival curves. N = 50, 100 sample paths. Left : Gaussian
covariance function. Right : Matérn 5/2 covariance function. CDS spreads as of
06/01/2005.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
29/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Kriging of CDS-implied default distribution (2D)
The previous approach can be extended in dimension 2.
1.00
0.95
1.0
0.90
0.9
0.85
0.8
0.80
0.7
0
1
2
3
4
5
6
7
protection maturities
8
9
201
2−0
8−2
201
3
1−0
7−1
201
4
0−0
6−2
200
1
9−0
5
−11
200
8−0
4
−04
200
7−0
3−2
200
0 dates
6−0
2−0
200
2
10 5−01−
06
0.75
0.70
0.65
Survival curves. CDS implied survival probabilities as a function of
time-to-maturities and quotation dates.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
30/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Perspectives
Impact of curve uncertainty on the assessment of related products and
their associated hedging strategies
What if the underlying market quotes are not reliable due to e.g. market
illiquidity (data observed with a noise) ?
Kriging of arbitrage-free volatility surfaces ?
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
31/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Kriging of arbitrage-free volatility surface
300
250
Call price
200
150
100
50
0
600
700
800
Strike
900
1000
0
Areski Cousin, ISFA, Université Lyon 1
0.5
1
1.5
2
2.5
3
Time-to-maturity
Model uncertainty in finance
32/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Contents
1
Kriging of financial term-structures
2
Adaptive robust control of Markov decision process
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
33/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
General setting and motivation
Robust control may be overly conservative when applied to the true
unknown system
We develop an adaptive robust methodology for solving a discrete-time
Markovian control problem subject to Knightian uncertainty
We focus on a financial hedging problem, but the methodology can be
applied to any kind of Markov decision process under model uncertainty
As in the classical robust case, the uncertainty comes from the fact that
the true law of the driving process is only known to belong to a certain
family of probability laws
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
34/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
General setting and motivation
T : terminal date of our finite horizon control problem
T = {0, 1, 2, . . . , T } : time grid
T 0 = {0, 1, 2, . . . , T − 1} : time grid without last date
S = {St , t ∈ T } : stochastic process that drives the random system
We assume that :
S is observable and we denote by FS = (FtS , t ∈ T ) its natural filtration.
The law of S is not known but it belongs to a family of parametrized
distributions P(Θ) := {Pθ , θ ∈ Θ}, Θ ⊂ Rd
The unknown (true) law of S is denoted by Pθ∗ and is such that θ∗ ∈ Θ
Model uncertainty occurs if Θ 6= {θ∗ }
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
35/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
General setting and motivation
We consider the following stochastic control problem
inf Eθ∗ (L(S, ϕ)) .
ϕ∈A
where
A is a set of admissible control processes : FS –adapted processes
ϕ = {ϕt , t ∈ T 0 }
L is a measurable functional (loss or error to minimize in our case)
Obviously, the problem cannot be dealt with directly since we do not know the
value of θ∗
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
36/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
General setting and motivation
Robust control problem : Başar and Bernhard (1995), Hansen et al. (2006),
Hansen and Sargent (2008)
inf sup Eθ (L(S, ϕ)) .
ϕ∈A θ∈Θ
(2)
Best strategy over the worst possible model parameter in Θ
If the true model is close to the best one, the solution to this problem
could perform very badly
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
37/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
General setting and motivation
Strong robust control problem : Sirbu (2014), Bayraktar, Cosso and Pham
(2014)
inf
ϕ∈A
sup
EQ (L(S, ϕ)) ,
(3)
Q∈Qϕ,ΨK
ΨK is the set of strategies chosen by a Knightian adversary (the nature)
that may keep changing the system distribution over time
Qϕ,ΨK represents all possible model dynamics resulting from ϕ and when
nature plays strategies in ΨK
Solution is even more conservative than in the classical robust case
No learning mechanism to reduce model uncertainty
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
38/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
General setting and motivation
Adaptive control problem : Kumar and Varaiya (1986), Chen and Guo (1991)
For each θ ∈ Θ solve :
inf Eθ (L(S, ϕ)) .
ϕ∈A
(4)
Let ϕθ be a corresponding optimal control
At each time t, compute a point estimate θbt of θ∗ , using a chosen, FtS
b
measurable estimator and apply control value ϕθt t .
Known to have poor performance for finite horizon problems
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
39/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Hedging under model uncertainty
Problem : Hedging a short position on an European-type option with maturity
T , payoff function Φ and underlying asset S with price dynamics
S0 = s0 ∈ (0, ∞),
St+1 = Zt+1 St ,
t ∈T0
where
Z = {Zt , t = 1, . . . , T } is a non-negative random process
Under each measure Pθ , Zt+1 is independent from FtS for each t ∈ T
The true law Pθ∗ of Z is not known.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
40/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Hedging under model uncertainty
Hedging is made using a self-financing portfolio composed of the underlying
risky asset S and of a risk-free asset (with constant value equal to 1).
The hedging portfolio has the following dynamics
V0 = v0 ,
Vt+1 = Vt + ϕt (St+1 − St ),
t = 0, . . . , T − 1
Exact replication is out of reach in our setting (v0 may be too small), so that
the nominal control problem (without uncertainty) is
inf Eθ∗ `[(Φ(ST ) − VT (ϕ))+ ] ,
ϕ∈A
where l is a loss function, i.e., an increasing function such that `(0) = 0
(shortfall risk minimization approach)
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
41/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
The methodology relies on recursive construction of confidence regions. We
assume that :
1) A point estimator θbt of θ∗ can be constructed recursively
θb0 = θ0 ,
θbt+1 = R(t, θbt , Zt+1 ),
t = 0, . . . , T − 1
where R(t, c, z) is a deterministic measurable function.
2) An approximate α-confidence region Θt of θ∗ can be constructed from θbt by
a deterministic rule :
Θt = τ (t, θbt )
where τ (t, ·) : Rd → 2Θ is a deterministic set valued function. The region Θt
should be such that Pθ∗ (θ∗ ∈ Θt ) ≈ 1 − α and limt→∞ Θt = {θ∗ } where the
∗
convergence is understood Pθ almost surely, and the limit is in the Hausdorff
metric.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
42/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
We consider the following (augmented) state process
Xt = (St , Vt , θbt ),
t∈T
d
with state space EX := R+ × R × R .
b is a Markov process with dynamics :
In our hedging problem, X = (S, V , θ)
St+1 = Zt+1 St ,
Vt+1 = Vt + ϕt St (Zt+1 − 1),
θbt+1 = R(t, θbt , Zt+1 )
We denote by
Q(B | t, x, a, θ) := Pθ (Xt+1 ∈ B | Xt = x, ϕt = a)
the time-t Markov transition kernel under probability Pθ when strategy a is
applied
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
43/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
Let us denote by
Ht := ((S0 , V0 , θb0 ), (S1 , V1 , θb1 ), . . . , (St , Vt , θbt )), t ∈ T ,
the history of the state process up to time t.
Note that, for any admissible trading strategy ϕ, Ht is FtS measurable and
Ht ∈ Ht := EX × EX × . . . × EX .
|
{z
}
t+1 times
We denote by
ht = (x0 , x1 , . . . , xt ) = (s0 , v0 , c0 , s1 , v1 , c1 , . . . , st , vt , ct )
a realization of Ht .
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
44/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
A robust control problem can be viewed as a game between a controller and
nature (the Knightian opponent).
The controller plays history-dependent strategies ϕ that belong to
A = {(ϕt )t∈T 0 | ϕt : Ht → A, t ∈ T 0 }
where ϕt is a measurable mapping.
Strong robust case : nature plays history-dependent strategies ψ that belong to
ΨK = {(ψt )t∈T 0 | ψt : Ht → Θ, t ∈ T 0 }
Adaptive robust case : nature plays history-dependent strategies ψ that belong
to
ΨA = {(ψt )t∈T 0 | ψt : Ht → Θt , t ∈ T 0 }
where Θt = τ (t, θbt ) is the α-confidence region of θ∗ at time t
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
45/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
Given that the controller plays ϕ and nature plays ψ, using Ionescu-Tulcea
theorem, we define the canonical law of the state process X on EXT as
Qϕ,ψ
(B1 , . . . , BT ) =
h
Z
Z 0
···
Q(dxT | T − 1, xT −1 , ϕT −1 (hT −1 ), ψT −1 (hT −1 ))
B1
BT
· · · Q(dx2 | 1, x1 , ϕ1 (h1 ), ψ1 (h1 )) Q(dx1 | 0, x0 , ϕ0 (h0 ), ψ0 (h0 )).
For a given strategy ϕ, we define
ϕ,ΨK
:= {Qϕ,ψ
h0 , ψ ∈ ΨK }
ϕ,ΨA
:= {Qϕ,ψ
h 0 , ψ ∈ ΨA }
Qh 0
and
Qh 0
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
46/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
The strong robust hedging problem :
inf
ϕ∈A
sup
EQ `[(Φ(ST ) − VT )+ ]
ϕ,ΨK
0
Q∈Qh
The adaptive robust hedging problem :
inf
ϕ∈A
sup
EQ `[(Φ(ST ) − VT )+ ]
ϕ,Ψ
Q∈Qh A
0
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
47/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
Without uncertainty
t
✓⇤
t
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
48/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
Robust
t
max(⇥)
✓⇤
min(⇥)
t
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
49/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
Strong robust
t
max(⇥)
✓⇤
min(⇥)
t
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
50/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
Adaptive robust
t
max(⇥)
max(⇥t )
✓⇤
min(⇥t )
min(⇥)
t
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
51/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Adaptive robust control methodology
Dynamic programming principle
Proposition
The solution ϕ∗ = (ϕ∗t (ht ))t∈T 0 of
inf
sup
ϕ∈A
EQ `[(Φ(ST ) − VT )+ ]
ϕ,ΨA
0
Q∈Qh
coincides with the solution of the following robust Bellman equation :
b ∈ EX ,
WT (x) = ` (Φ(s) − v )+ , x = (s, v , θ)
Z
Wt (x) = inf sup
Wt+1 (y )Q(dy | t, x, a, θ),
a∈A
b
θ∈τ (t,θ)
EX
b ∈ EX and t = T − 1, . . . , 0.
for any x = (s, v , θ)
Note that the optimal strategy at time t is such that ϕ∗t (ht ) = ϕ∗t (xt ).
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
52/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Example : uncertain log-normal model
We consider that the stock price is driven by an uncertain log-normal model
St+1 = Zt+1 St
P ∗
θ
N(µ∗ , (σ ∗ )2 ).
where Zt is an iid sequence such that ln Zt ∼
The MLE θbt = (b
µt , σ
bt2 ) of the unknown parameter θ∗ = (µ∗ , (σ ∗ )2 ) can be
expressed in the following recursive way :
t
1
µ
bt +
ln Zt+1 ,
t +1
t +1
t
t
=
σ
bt2 +
(b
µt − ln Zt+1 )2 ,
t +1
(t + 1)2
µ
bt+1 =
2
σ
bt+1
with µ
b1 = ln Z1 = ln
S1
S0
and σ
b12 = 0.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
53/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Example : uncertain log-normal model
Due to asymptotic normality of the MLE θbt = (b
µt , σ
bt2 ), we have
t
t
d
(b
µt − µ∗ )2 + 4 (b
σt2 − (σ ∗ )2 )2 −−−→ χ22
t→∞
σ
bt2
2b
σt
So that, if κα is the (1 − α)–quantile of the χ22 distribution,
n
o
t
t
Θt = τ (t, µ
b, σ
b2 ) := (µ, σ 2 ) ∈ R2 : 2 (b
µ − µ)2 + 4 (b
σ 2 − σ 2 )2 ≤ κα
σ
b
2b
σ
is an approximate α-confidence region of θ∗ , i.e., Θt is such that
Pθ∗ (θ∗ ∈ Θt ) ≈ 1 − α
[See Bielecki et al. (2016) for more details]
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
54/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Example : uncertain log-normal model
The adaptive robust control problem can be solved using the following dynamic
programming principle :
x = (s, v , µ
b, σ
b2 ) ∈ EX ,
WT (x) = ` (Φ(s) − v )+ ,
Z
Wt (x) = inf
Wt+1 (y )Q(dy | t, x, a; µ, σ 2 )
sup
a∈A (µ,σ 2 )∈τ (t,b
µ,b
σ2 )
EX
where x = (s, v , µ
b, σ
b2 ) ∈ EX = R+ × R × R × R+ , t = T − 1, . . . , 0
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
55/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Example : uncertain log-normal model
The integral in the previous slide can be written as
Z
Wt+1 se µ+σz , v + as(e µ+σz − 1), R(t, µ
b, σ
b2 , µ + σz) φ(z)dz
R
where φ is the density of the standard normal distribution and R is such that
t
1
t
t
2
R t, µ
b, σ
b2 , y =
µ
b+
y,
σ
b2 +
(b
µ
−
y
)
t +1
t +1 t +1
(t + 1)2
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
56/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Perspectives
Numerically solve Bellman equation for the considered hedging problem :
challenging issue due to the curse of dimensionality (optimal quantization,
approximate dynamic programming could be used)
Compare hedging performance with other approaches : control without
uncertainty, standard robust, adaptive robust, Bayesian adaptive robust
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
57/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
Thanks for your attention.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
58/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
References I
Bachoc, F. (2013).
Cross Validation and Maximum Likelihood estimations of hyper-parameters of Gaussian
processes with model misspecification.
Computational Statistics & Data Analysis, 66(0) :55 – 69.
Başar, T. and Bernhard, P. (1995).
H ∞ -optimal control and related minimax design problems.
Systems & Control : Foundations & Applications. Birkhäuser Boston, Inc., Boston, MA,
second edition.
A dynamic game approach.
Bay, X., Grammont, L., and Maatouk, H. (2016).
Generalization of the Kimeldorf-Wahba correspondence for constrained interpolation.
Bayraktar, E., Cosso, A., and Pham, H. (2014).
Robust feedback switching control : dynamic programming and viscosity solutions.
http ://arxiv.org/pdf/1409.6233v1.pdf.
Bielecki, T., Chen, T., and Cialenco, I. (2016).
Recursive Construction of Confdence Regions.
Preprint.
Chen, H. F. and Guo, L. (1991).
Identification and stochastic adaptive control.
Systems & Control : Foundations & Applications. Birkhäuser Boston, Inc., Boston, MA.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
59/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
References II
Corsi, M., Pham, H., and Runggaldier, W. (2007).
Numerical approximation by quantization of control problems in finance under partial
observations.
preprint.
Cousin, A., Maatouk, H., and Rullière, D. (2015).
Kriging of Financial Term-Structures.
Cousin, A. and Niang, I. (2014).
On the Range of Admissible Term-Structures.
Golchi, S., Bingham, D., Chipman, H., and Campbell, D. (2015).
Monotone Emulation of Computer Experiments.
SIAM/ASA Journal on Uncertainty Quantification, 3(1) :370–392.
Hansen, L. P., Sargent, T. J., Turmuhambetova, G., and Williams, N. (2006).
Robust control and model misspecification.
J. Econom. Theory, 128(1) :45–90.
Hansen, P. L. and Sargent, T. J. (2008).
Robustness.
Princeton University Press.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
60/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
References III
Iyengar, G. N. (2005).
Robust dynamic programming.
Mathematics of Operations Research, 30(2) :pp. 257–280.
Kumar, P. R. and Varaiya, P. (1986).
Stochastic systems : estimation, identification and adaptive control.
Prentice-Hall, Inc.
Maatouk, H. and Bay, X. (2014a).
A New Rejection Sampling Method for Truncated Multivariate Gaussian Random
Variables Restricted to Convex Sets.
To appear in Monte Carlo and Quasi-Monte Carlo Methods 2014, Springer-Verlag, Berlin
2016.
Maatouk, H. and Bay, X. (2014b).
Gaussian Process Emulators for Computer Experiments with Inequality Constraints.
in revision SIAM/ASA J. Uncertainty Quantification.
Runggaldier, W., Trivellato, B., and Vargiolu, T. (2002).
A Bayesian adaptive control approach to risk management in a binomial model.
In Seminar on Stochastic Analysis, Random Fields and Applications, III (Ascona, 1999),
volume 52 of Progr. Probab., pages 243–258.
Sirbu, M. (2014).
A note on the strong formulation of stochastic control problems with model uncertainty.
Electronic Communications in Probability, 19.
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
61/63
Kriging of financial term-structures
Adaptive robust control of Markov decision process
References IV
Areski Cousin, ISFA, Université Lyon 1
Model uncertainty in finance
62/63