Risk-averse Control of Diffusion Processes
Jianing Yao
(Advisor: Andrzej Ruszczyński)
Búzios Brazil, June 27th, 2016
XIV International Conference on Stochastic Programming
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Introduction
Motivation & Extension
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Extending to Continuous Time
Our Objective
To formulate a risk-averse control problem where the underlying process is a
continuous-time (continuous state) stochastic process.
There are four essential elements for modeling risk-averse control problem.
Questions to be addressed for extension
(1) What is the dynamics of the system?
(2) How risk is evaluated?
(3) What is the risk associated with certain policy (i.e., policy evaluation)?
(4) How about the computation of optimal value and control?
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Main Part
Four Elements for Extension
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
1. Dynamics of the System
Controlled Diffusion Process
2. Dynamic Risk Evaluation – Continuous-time Setting
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
3. Risk-Averse Modelling
Forward Backward Stochastic Differential Equation System
Weak Formulation
4. DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
1. Dynamics of the System
Controlled Diffusion Process
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
Background: Stochastic Differential Equation
Initial Setting
Given probability space (Ω, F , P; {Ft }t ∈T ), where T = [0, T ], and
Ft = σ{{W (s ); 0 ≤ s ≤ T } ∪ N} (N is collection of P -null sets in Ω, {W (s )}s ∈[0,T ] is
d -dimensional standard Brownian motion)
Let’s consider n-dimensional Stochastic differential Equation (SDE) :



 dX (s ) = b (s , X , W )ds + σ(s , X , W )dW (s ), s ∈ [0, T ]


 X (0) = ξ ∈ L 2 (Ω, F0 , P; Rn )
Here, b , σ are defined on [0, T ] × C([0, T ]; RN ) × C([0, T ]; Rd ).
Assumption Strong Solution (SDE)
b and σ are progressively measurable, satisfy Lipschitz condition and linear growth
condition (usual topology on continuous function).
Notice: The system (1) is Non-Markovian (w.r.t X ) !!!
Y. Jianing
Risk-averse Control of Diffusion Processes
(1)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
Background: Stochastic Differential Equation
Initial Setting
Given probability space (Ω, F , P; {Ft }t ∈T ), where T = [0, T ], and
Ft = σ{{W (s ); 0 ≤ s ≤ T } ∪ N} (N is collection of P -null sets in Ω, {W (s )}s ∈[0,T ] is
d -dimensional standard Brownian motion)
Let’s consider n-dimensional Stochastic differential Equation (SDE) :



 dX (s ) = b (s , X , W )ds + σ(s , X , W )dW (s ), s ∈ [0, T ]


 X (0) = ξ ∈ L 2 (Ω, F0 , P; Rn )
Here, b , σ are defined on [0, T ] × C([0, T ]; RN ) × C([0, T ]; Rd ).
Assumption Strong Solution (SDE)
b and σ are progressively measurable, satisfy Lipschitz condition and linear growth
condition (usual topology on continuous function).
Notice: The system (1) is Non-Markovian (w.r.t X ) !!!
Y. Jianing
Risk-averse Control of Diffusion Processes
(1)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
Background: Stochastic Differential Equation
Initial Setting
Given probability space (Ω, F , P; {Ft }t ∈T ), where T = [0, T ], and
Ft = σ{{W (s ); 0 ≤ s ≤ T } ∪ N} (N is collection of P -null sets in Ω, {W (s )}s ∈[0,T ] is
d -dimensional standard Brownian motion)
Let’s consider n-dimensional Stochastic differential Equation (SDE) :



 dX (s ) = b (s , X , W )ds + σ(s , X , W )dW (s ), s ∈ [0, T ]


 X (0) = ξ ∈ L 2 (Ω, F0 , P; Rn )
Here, b , σ are defined on [0, T ] × C([0, T ]; RN ) × C([0, T ]; Rd ).
Assumption Strong Solution (SDE)
b and σ are progressively measurable, satisfy Lipschitz condition and linear growth
condition (usual topology on continuous function).
Notice: The system (1) is Non-Markovian (w.r.t X ) !!!
Y. Jianing
Risk-averse Control of Diffusion Processes
(1)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
Dynamics: Controlled Diffusion Process I
Admissible Control Process
u(·) : [0, T ] × Ω 7→ U is called the control representing policy of the decision-maker.
Non-anticipative ⇔ u(·) is Ft -progressively measurable.
Furthermore, since Ft is the Brownian filtration, there exists a progressively
measurable process ψ such that
u(t , ω) = ψ(t , W (· ∧ t , ω))
(? ? ?)
Denote the class of admissible control as U.
As a special case of (1), for s ∈ [0, T ], we have Controlled Diffusion Process :



 dX (s ) = b (s , X (s ), ψ(s , W (· ∧ s )))ds + σ(s , X (s ), ψ(s , W (· ∧ s )))dW (s ),


 X0 = ξ ∈ L 2 (Ω, F0 , P; Rn )
Y. Jianing
Risk-averse Control of Diffusion Processes
(2)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
Dynamics: Controlled Diffusion Process I
Admissible Control Process
u(·) : [0, T ] × Ω 7→ U is called the control representing policy of the decision-maker.
Non-anticipative ⇔ u(·) is Ft -progressively measurable.
Furthermore, since Ft is the Brownian filtration, there exists a progressively
measurable process ψ such that
u(t , ω) = ψ(t , W (· ∧ t , ω))
(? ? ?)
Denote the class of admissible control as U.
As a special case of (1), for s ∈ [0, T ], we have Controlled Diffusion Process :



 dX (s ) = b (s , X (s ), ψ(s , W (· ∧ s )))ds + σ(s , X (s ), ψ(s , W (· ∧ s )))dW (s ),


 X0 = ξ ∈ L 2 (Ω, F0 , P; Rn )
Y. Jianing
Risk-averse Control of Diffusion Processes
(2)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Controlled Diffusion Process
Dynamics: Controlled Diffusion Process II
To formally answer the first question: the dynamic of the system is given by
Controlled Stochastic Differential Equation :
(
dX (s ) = b (s , X (s ), u(s ))ds + σ(s , X (s ), u(s ))dW (s ),
s ∈ [0, T ]
X0 = ξ ∈ L (Ω, F0 , P; R )
2
n
(3)
Notice: It is of Markovian type w.r.t. (X , u), because b , σ is deterministic w.r.t.
(x , u).
Assumption Strong Solution (CDP)
b and σ satisfy Lipschitz condition and linear growth condition w.r.t. (x , u).
Under above assumptions, the stochastic process {Xs }s ∈[0,T ] on
(Ω, F , P; {Ft }t ∈T ) is well-defined.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Expected Value Model v.s. Risk Measure Model – Time Consistency and
Translation Invariance
We shall be able to evaluate expectation/risk at any t and do the ’decomposition’:
Classical Stochastic Control
Define cost function c (·, ·, ·) : [0, T ] × Rn × U 7→ R, i.e., c (s , X (s ), u(s )) and possibly
final stage cost Φ(·) : Rn 7→ R, i.e., Φ(X (T )). By tower property of conditional
expectation,
"Z
T
#
c (s , X (s ), u(s ))ds + Φ(X (T )) Ft
E
t
"Z
=E
r
c (s , X (s ), u(s ))ds + E
t
"Z
T
#
#
c (s , X (s ), u(s ))ds + Φ(X (T )) Fr Ft t ≤ r ≤ T .
r
Remember, in discrete-time setting, similar structure is ensured by the construction of
risk transition mapping σt , i.e.,
ρπt ,T (Zt , ..., ZT )(ht ) = Zt (ht ) + σt ht , Qtπ (ht ), ρπt +1,T (Zt +1 , ..., ZT ))(ht , ·) .
Question: How to realize risk evaluation in continuous-time setting?
Y. Jianing
Risk-averse Control of Diffusion Processes
(4)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Expected Value Model v.s. Risk Measure Model – Time Consistency and
Translation Invariance
We shall be able to evaluate expectation/risk at any t and do the ’decomposition’:
Classical Stochastic Control
Define cost function c (·, ·, ·) : [0, T ] × Rn × U 7→ R, i.e., c (s , X (s ), u(s )) and possibly
final stage cost Φ(·) : Rn 7→ R, i.e., Φ(X (T )). By tower property of conditional
expectation,
"Z
T
#
c (s , X (s ), u(s ))ds + Φ(X (T )) Ft
E
t
"Z
=E
r
c (s , X (s ), u(s ))ds + E
t
"Z
T
#
#
c (s , X (s ), u(s ))ds + Φ(X (T )) Fr Ft t ≤ r ≤ T .
r
Remember, in discrete-time setting, similar structure is ensured by the construction of
risk transition mapping σt , i.e.,
ρπt ,T (Zt , ..., ZT )(ht ) = Zt (ht ) + σt ht , Qtπ (ht ), ρπt +1,T (Zt +1 , ..., ZT ))(ht , ·) .
Question: How to realize risk evaluation in continuous-time setting?
Y. Jianing
Risk-averse Control of Diffusion Processes
(4)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Expected Value Model v.s. Risk Measure Model – Time Consistency and
Translation Invariance
We shall be able to evaluate expectation/risk at any t and do the ’decomposition’:
Classical Stochastic Control
Define cost function c (·, ·, ·) : [0, T ] × Rn × U 7→ R, i.e., c (s , X (s ), u(s )) and possibly
final stage cost Φ(·) : Rn 7→ R, i.e., Φ(X (T )). By tower property of conditional
expectation,
"Z
T
#
c (s , X (s ), u(s ))ds + Φ(X (T )) Ft
E
t
"Z
=E
r
c (s , X (s ), u(s ))ds + E
t
"Z
T
#
#
c (s , X (s ), u(s ))ds + Φ(X (T )) Fr Ft t ≤ r ≤ T .
r
Remember, in discrete-time setting, similar structure is ensured by the construction of
risk transition mapping σt , i.e.,
ρπt ,T (Zt , ..., ZT )(ht ) = Zt (ht ) + σt ht , Qtπ (ht ), ρπt +1,T (Zt +1 , ..., ZT ))(ht , ·) .
Question: How to realize risk evaluation in continuous-time setting?
Y. Jianing
Risk-averse Control of Diffusion Processes
(4)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
F - consistent Nonlinear Expectation I
Nonlinear Expectation ρ0,T [ · ]
Monotonicity
Constant Preserving
∀ξ ∈ L 2 (Ω, Ft , P), ∃!η ∈ L 2 (Ω, Ft , P)
s.t. ρ0,T [ ξ1A ] = ρ0,T [ η1A ], ∀A ∈ Ft
η := ρt ,T [ ξ ], t ∈ [0, T ]
n
o
We can produce a system of ρs ,t
, it is called F-consistent Nonlinear
0≤s ≤t ≤T
Expectation.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
F - consistent Nonlinear Expectation II
Properties of F-consistent nonlinear expectation (by S., Peng)
If ρt ,T [ · ] is an F-consistent nonlinear expectation, then for all 0 ≤ t ≤ T and all
ξ, ξ0 ∈ L 2 (Ω, FT , P), it has the following properties:
(i) Monotonicity: If ξ, ξ0 ∈ L 2 (Ω, FT , P) and ξ ≥ ξ0 , then ρt ,T [ξ] ≥ ρt ,T [ξ0 ];
(ii) Generalized constant-preservation: If ξ ∈ L 2 (Ω, Ft , P), then ρt ,t [ ξ ] = ξ;
(iii) Time-consistency: ρs ,T [ξ] = ρs ,t [ ρt ,T [ ξ ] ], for all 0 ≤ s ≤ t ;
(iv) Local property: ρt ,T [ ξ1A + ξ0 1A C ] = 1A ρt ,T [ ξ ] + 1A C ρt ,T [ ξ0 ], for all A ∈ Ft .
ρt , T
T
Z
!
c (s , X (s ), u(s ))ds + Φ(X (T ))
t
Rr
=ρt ,r ρr ,T
t
c (s , X (s ), u(s ))ds
pull it out??
T
Z
!!
c (s , X (s ), u(s ))ds + Φ(X (T ) ,
+
t ≤ r ≤ T.
r
Question: Can we have more insights of ρt ,T ?
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
F - consistent Nonlinear Expectation II
Properties of F-consistent nonlinear expectation (by S., Peng)
If ρt ,T [ · ] is an F-consistent nonlinear expectation, then for all 0 ≤ t ≤ T and all
ξ, ξ0 ∈ L 2 (Ω, FT , P), it has the following properties:
(i) Monotonicity: If ξ, ξ0 ∈ L 2 (Ω, FT , P) and ξ ≥ ξ0 , then ρt ,T [ξ] ≥ ρt ,T [ξ0 ];
(ii) Generalized constant-preservation: If ξ ∈ L 2 (Ω, Ft , P), then ρt ,t [ ξ ] = ξ;
(iii) Time-consistency: ρs ,T [ξ] = ρs ,t [ ρt ,T [ ξ ] ], for all 0 ≤ s ≤ t ;
(iv) Local property: ρt ,T [ ξ1A + ξ0 1A C ] = 1A ρt ,T [ ξ ] + 1A C ρt ,T [ ξ0 ], for all A ∈ Ft .
ρt , T
T
Z
!
c (s , X (s ), u(s ))ds + Φ(X (T ))
t
Rr
=ρt ,r ρr ,T
t
c (s , X (s ), u(s ))ds
pull it out??
T
Z
!!
c (s , X (s ), u(s ))ds + Φ(X (T ) ,
+
t ≤ r ≤ T.
r
Question: Can we have more insights of ρt ,T ?
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
F - consistent Nonlinear Expectation II
Properties of F-consistent nonlinear expectation (by S., Peng)
If ρt ,T [ · ] is an F-consistent nonlinear expectation, then for all 0 ≤ t ≤ T and all
ξ, ξ0 ∈ L 2 (Ω, FT , P), it has the following properties:
(i) Monotonicity: If ξ, ξ0 ∈ L 2 (Ω, FT , P) and ξ ≥ ξ0 , then ρt ,T [ξ] ≥ ρt ,T [ξ0 ];
(ii) Generalized constant-preservation: If ξ ∈ L 2 (Ω, Ft , P), then ρt ,t [ ξ ] = ξ;
(iii) Time-consistency: ρs ,T [ξ] = ρs ,t [ ρt ,T [ ξ ] ], for all 0 ≤ s ≤ t ;
(iv) Local property: ρt ,T [ ξ1A + ξ0 1A C ] = 1A ρt ,T [ ξ ] + 1A C ρt ,T [ ξ0 ], for all A ∈ Ft .
ρt , T
T
Z
!
c (s , X (s ), u(s ))ds + Φ(X (T ))
t
Rr
=ρt ,r ρr ,T
t
c (s , X (s ), u(s ))ds
pull it out??
T
Z
!!
c (s , X (s ), u(s ))ds + Φ(X (T ) ,
+
t ≤ r ≤ T.
r
Question: Can we have more insights of ρt ,T ?
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
3. Risk-Averse Modelling
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Backward-Stochastic Differential Equation (BSDE) I
Backward Stochastic Differential Equation (S., Peng and E., Pardoux)
BSDE is a generalization of Martingale representation theorem,
Z T
Z T
Y (t ) = ξ +
g (s , Y (s ), Z (s )) ds −
Z (s ) dW (s ), 0 ≤ t ≤ T
t
t
where (ξ, g ) is the data (terminal condition and driver, resp.)
Case 1: g = 0
"
Y (t ) = E [ Y (t ) | Ft ] = E ξ −
Z
T
#
Z (s )dW (s ) Ft = E [ ξ | Ft ]
t
Case 2: g , 0
"
Y (t ) = E ξ +
Z
T
ξ
ξ
#
g (s , Y (s ), Z (s ))ds Ft
t
Y. Jianing
Risk-averse Control of Diffusion Processes
(5)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Backward-Stochastic Differential Equation (BSDE) I
Backward Stochastic Differential Equation (S., Peng and E., Pardoux)
BSDE is a generalization of Martingale representation theorem,
Z T
Z T
Y (t ) = ξ +
g (s , Y (s ), Z (s )) ds −
Z (s ) dW (s ), 0 ≤ t ≤ T
t
t
where (ξ, g ) is the data (terminal condition and driver, resp.)
Case 1: g = 0
"
Y (t ) = E [ Y (t ) | Ft ] = E ξ −
Z
T
#
Z (s )dW (s ) Ft = E [ ξ | Ft ]
t
Case 2: g , 0
"
Y (t ) = E ξ +
Z
T
ξ
ξ
#
g (s , Y (s ), Z (s ))ds Ft
t
Y. Jianing
Risk-averse Control of Diffusion Processes
(5)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Backward-Stochastic Differential Equation (BSDE) I
Backward Stochastic Differential Equation (S., Peng and E., Pardoux)
BSDE is a generalization of Martingale representation theorem,
Z T
Z T
Y (t ) = ξ +
g (s , Y (s ), Z (s )) ds −
Z (s ) dW (s ), 0 ≤ t ≤ T
t
t
where (ξ, g ) is the data (terminal condition and driver, resp.)
Case 1: g = 0
"
Y (t ) = E [ Y (t ) | Ft ] = E ξ −
Z
T
#
Z (s )dW (s ) Ft = E [ ξ | Ft ]
t
Case 2: g , 0
"
Y (t ) = E ξ +
Z
T
ξ
ξ
#
g (s , Y (s ), Z (s ))ds Ft
t
Y. Jianing
Risk-averse Control of Diffusion Processes
(5)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Backward-Stochastic Differential Equation (BSDE) II
Backward Stochastic Differential Equation (S., Peng and E., Pardoux)
Z T
Z T
Y (t ) = ξ +
g (s , Y (s ), Z (s )) ds −
Z (s ) dW (s ), 0 ≤ t ≤ T
t
(6)
t
As we shall notice: the solution of (6) is a pair process (Y , Z ) ∈ S2 [t , T ] × H 2,d [t , T ],
i
i
hRT
h
where S2 [t , T ]: E supt ≤s ≤T |Ys |2 < +∞, H 2,d [t , T ]: E t |Zs |2 ds < +∞.
Assumption 1: Strong Solution (BSDE)
g is jointly Lipschitz in (y , z ), ∀(yi , zi ) ∈ R × Rd , i = 1, 2, ∃K > 0 such that
|g (t , y1 , z1 ) − g (t , y2 , z2 )| ≤ K (|y1 − y2 | + |z1 − z2 |), a .s .;
g is ’bounded’, i.e., g (·, 0, 0) ∈ H 2 [0, T ].
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Backward-Stochastic Differential Equation (BSDE) II
Backward Stochastic Differential Equation (S., Peng and E., Pardoux)
Z T
Z T
Y (t ) = ξ +
g (s , Y (s ), Z (s )) ds −
Z (s ) dW (s ), 0 ≤ t ≤ T
t
(6)
t
As we shall notice: the solution of (6) is a pair process (Y , Z ) ∈ S2 [t , T ] × H 2,d [t , T ],
i
i
hRT
h
where S2 [t , T ]: E supt ≤s ≤T |Ys |2 < +∞, H 2,d [t , T ]: E t |Zs |2 ds < +∞.
Assumption 1: Strong Solution (BSDE)
g is jointly Lipschitz in (y , z ), ∀(yi , zi ) ∈ R × Rd , i = 1, 2, ∃K > 0 such that
|g (t , y1 , z1 ) − g (t , y2 , z2 )| ≤ K (|y1 − y2 | + |z1 − z2 |), a .s .;
g is ’bounded’, i.e., g (·, 0, 0) ∈ H 2 [0, T ].
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
3. Risk-Averse Modelling
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
g-evaluation and F-consistent nonlinear expectation
Surprising Result
We can view Backward Stochastic Differential Equation as a nonlinear operator
(depends on g ): ξ ∈ L 2 (Ω, FT , P) 7→ L 2 (Ω, Ft , P) . Let’s denote it ρgt [ ξ ], i.e.,
ρgt,T [ ξ ] := Yt . Surprisingly, { ρgt,T }t ∈T is an F-consistent nonlinear-expectation, as long
as driver g satisfies Assumption 1.
Remember: one degree of freedom to modify the nonlinearity through g : if g is
independent of y , i.e., g (t , y , z ) = g (t , z ), then
ρt ,T
T
Z
!
c (s , X (s ), u(s ))ds + Φ(X (T ))
t
Z
=ρt ,r
r
c (s , X (s ), u(s ))ds + ρr ,T
t
T
Z
!!
c (s , X (s ), u(s ))ds + Φ(X (T ) ,
t ≤ r ≤ T.
r
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Dynamic Convex/Coherent Risk measure
F-consistent nonlinear expectation + the independence condition ≈ dynamic
risk measure.
Assumption 2 ( Dynamic risk measure )
If g satisfies additional conditions: (i) independence of y ; (ii) convex in z ( or
homogeneous in z ) for all t ∈ [0, T ] and almost all ω ∈ Ω, we obtain
time-consistent dynamic risk convex (or coherent) risk measure.
Theorem (Time-consistent dynamic risk measure)
Suppose g satisfies Assumption 1 and Assumption 2 (except positive
g
homogeneity), then ρt ,T , for t ∈ T has the following properties, it is
(i)Normalized, (ii) Monotonic, (iii) Convex. Moreover, if g also satisfies
g
Positive Homogeneity, then ρt ,t (·) is positive homogeneous.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Dynamic Convex/Coherent Risk measure
F-consistent nonlinear expectation + the independence condition ≈ dynamic
risk measure.
Assumption 2 ( Dynamic risk measure )
If g satisfies additional conditions: (i) independence of y ; (ii) convex in z ( or
homogeneous in z ) for all t ∈ [0, T ] and almost all ω ∈ Ω, we obtain
time-consistent dynamic risk convex (or coherent) risk measure.
Theorem (Time-consistent dynamic risk measure)
Suppose g satisfies Assumption 1 and Assumption 2 (except positive
g
homogeneity), then ρt ,T , for t ∈ T has the following properties, it is
(i)Normalized, (ii) Monotonic, (iii) Convex. Moreover, if g also satisfies
g
Positive Homogeneity, then ρt ,t (·) is positive homogeneous.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Dynamic Convex/Coherent Risk measure
Theorem (Time-consistent dynamic risk measure)
Suppose g satisfies Assumption 1 and Assumption 2 (except positive homogeneity),
then ρgt,T , for t ∈ T has the following properties:
(i) Normalization: ρgt,T (0) = 0, 0 ≤ t ≤ T ;
(ii) Translation Invariance: for all ξ ∈ L 2 (Ω, FT , P) and η ∈ L 2 (Ω, Ft , P),
ρgt,T (ξ + η) = ρgt,T (ξ) + η,
a.s.;
(iii) Convexity: for all ξ, ξ0 ∈ L 2 (Ω, FT , P) and all λ ∈ L ∞ (Ω, Ft , P) such that
0 ≤ λ ≤ 1,
ρgt,T (λξ + (1 − λ)ξ0 ) ≤ λρgt,T (ξ) + (1 − λ)ρgt,T (ξ0 ),
a.s..
Moreover, if g also satisfies Positive Homogeneity, then for all ξ ∈ L 2 (Ω, FT , P) and
all λ ∈ L ∞ (Ω, Ft , P) such that λ ≥ 0, we have
ρgt,T (λξ) = λρgt,T (ξ),
Y. Jianing
a.s..
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
3. Risk-Averse Modelling
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Backward Stochastic Differential Equation
g-evaluation & Dynamic Risk Measure
Useful Results from BSDE
Useful Results from BSDE
t0
solution: (Y 0 , Z 0 ) deterministic
t
data: (g, ξ ), Ft 0 ⊂ Ft
t
T
solution: (Y , Z ) are Ft -adapted
For a fixed t0 ∈ [t , T ], denote Fst0 = σ{{Ws − Wt0 ; t0 ≤ s ≤ T } ∪ N}, the
following proposition is essential for our formulation later on:
Proposition (Deterministic)
Suppose Assumption 1 is satisfied and ∀(y , z ) ∈ R × Rd , in addition, g (·, y , z )
is Fst0 -adapted on the interval [t0 , T ] and ξ ∈ L 2 (Ω, FTt0 , P). Then the solution
(Y , Z ) of (6) is also Fst0 -adapted on [t0 , T ]. In particular, (Y , Z ) are
deterministic.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Expected Value Model v.s. Risk Measure Model – Objective
Let’s compare:
Classical Stochastic Control
The objective is:
"Z
T
#
c (s , X (s ), u(s ))ds + Φ(X (T ))
min E
u(·)∈U
0
Our Goal – Risk Measure
We want to REPLACE Expectation E[·] by Risk measure ρ0,T (·), i.e.,
!
Z T
min ρ0,T
c (s , X (s ), u(s ))ds + Φ(X (T ))
u(·)∈U
Y. Jianing
0
Risk-averse Control of Diffusion Processes
(7)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Expected Value Model v.s. Risk Measure Model – Objective
Let’s compare:
Classical Stochastic Control
The objective is:
"Z
T
#
c (s , X (s ), u(s ))ds + Φ(X (T ))
min E
u(·)∈U
0
Our Goal – Risk Measure
We want to REPLACE Expectation E[·] by Risk measure ρ0,T (·), i.e.,
!
Z T
min ρ0,T
c (s , X (s ), u(s ))ds + Φ(X (T ))
u(·)∈U
Y. Jianing
0
Risk-averse Control of Diffusion Processes
(7)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Expected Value Model v.s. Risk Measure Model – Policy Evaluation
We should be able to evaluate the risk given a policy. Suppose the we have a fixed
constant control u(·) ∈ U.
Classical Stochastic Control Model
For any time (t , x ) ∈ [0, T ] × Rn and constant control u, the control value function is of
the following form:
"Z T #
J (t , x ; u) = E
c s , X t ,x ;u (s ), u(s ) ds + Φ X t ,x ;u (T ) Ft
(8)
t
Risk-averse Control Model
In the risk-averse case, we define control value function associated with a constant
control as:
Z T !
g
u
t ,x ;u
t ,x ;u
V (t , x ) = ρt ,T
c s, X
(s ), u(s ) ds + Φ X
(T )
(9)
t
Question: How can we evaluate (9) ? Is it always well-defined ?
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Expected Value Model v.s. Risk Measure Model – Policy Evaluation
We should be able to evaluate the risk given a policy. Suppose the we have a fixed
constant control u(·) ∈ U.
Classical Stochastic Control Model
For any time (t , x ) ∈ [0, T ] × Rn and constant control u, the control value function is of
the following form:
"Z T #
J (t , x ; u) = E
c s , X t ,x ;u (s ), u(s ) ds + Φ X t ,x ;u (T ) Ft
(8)
t
Risk-averse Control Model
In the risk-averse case, we define control value function associated with a constant
control as:
Z T !
g
u
t ,x ;u
t ,x ;u
V (t , x ) = ρt ,T
c s, X
(s ), u(s ) ds + Φ X
(T )
(9)
t
Question: How can we evaluate (9) ? Is it always well-defined ?
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Expected Value Model v.s. Risk Measure Model – Policy Evaluation
We should be able to evaluate the risk given a policy. Suppose the we have a fixed
constant control u(·) ∈ U.
Classical Stochastic Control Model
For any time (t , x ) ∈ [0, T ] × Rn and constant control u, the control value function is of
the following form:
"Z T #
J (t , x ; u) = E
c s , X t ,x ;u (s ), u(s ) ds + Φ X t ,x ;u (T ) Ft
(8)
t
Risk-averse Control Model
In the risk-averse case, we define control value function associated with a constant
control as:
Z T !
g
u
t ,x ;u
t ,x ;u
V (t , x ) = ρt ,T
c s, X
(s ), u(s ) ds + Φ X
(T )
(9)
t
Question: How can we evaluate (9) ? Is it always well-defined ?
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
Forward Backward Stochastic Differential Equation System
Weak Formulation
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Decoupled FBSDE I
These questions are answered by Decouple Forward Backward Stochastic Differential
Equation (FBSDE).
Decoupled FBSDE (in risk-averse control model)
For any t ∈ [0, T ] and η ∈ L 2 (Ω, Ft , P; Rn ), we have the following system:
 t ,η;u

dX
(s ) = b (s , X t ,η;u (s ), u(s ))ds + σ(s , X t ,η;u (s ), u(s ))dW (s ),





− dY t ,η;u (s ) = f (s , X t ,η;u (s ), Z t ,η;u (s )ds − Z t ,η;u (s )dW (s ),





 X t ,η;u (t ) = η, Y t ,η;u (T ) = Φ(X t ,η;u (T )).
where f u (s , x , z ) = c u (s , x ) + g (s , z ). (dependence on u is carried over by X , Z )
Y. Jianing
Risk-averse Control of Diffusion Processes
(10)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Decoupled FBSDE II

Z T



t ,η;u


(s ) =
g (s , Z t ,η;u (s )) + c (s , X (s ), u(s ))ds − Z t ,η;u (s )dW (s ),
Y

t




 Y t ,η;u (T ) = Φ(X t ,η;u (T )).
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Decoupled FBSDE III
To have (9) to be well defined, we need following assumptions:
Assumption 3 Strong Solution (FBSDE)
f is jointly Lipschitz in (x , y , z ), ∀(xi , zi ) ∈ Rn × Rd , i = 1, 2, ∃K > 0 such that
|f (t , x1 , z1 ) − f (t , x2 , z2 )| ≤ K (|y1 − y2 | + |z1 − z2 |),
f is ’bounded’, i.e, f (·, 0, 0) ∈ H 2 [t , T ]
µ = b , σ, c , Φ satisfies linear growth condition, ∀x ∈ Rn , ∃K > 0, i.e.,
µ(x ) ≤ K (1 + |x |).
Unique Strong Solution
Under above assumption, (9) has a unique strong solution
(X , Y , Z ) ∈ S2,n [t , T ] × S2 [t , T ] × H 2,d [t , T ]
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Decoupled FBSDE III
To have (9) to be well defined, we need following assumptions:
Assumption 3 Strong Solution (FBSDE)
f is jointly Lipschitz in (x , y , z ), ∀(xi , zi ) ∈ Rn × Rd , i = 1, 2, ∃K > 0 such that
|f (t , x1 , z1 ) − f (t , x2 , z2 )| ≤ K (|y1 − y2 | + |z1 − z2 |),
f is ’bounded’, i.e, f (·, 0, 0) ∈ H 2 [t , T ]
µ = b , σ, c , Φ satisfies linear growth condition, ∀x ∈ Rn , ∃K > 0, i.e.,
µ(x ) ≤ K (1 + |x |).
Unique Strong Solution
Under above assumption, (9) has a unique strong solution
(X , Y , Z ) ∈ S2,n [t , T ] × S2 [t , T ] × H 2,d [t , T ]
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
Forward Backward Stochastic Differential Equation System
Weak Formulation
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Issues about Current (Strong) Formulation
In continuous-times setting, in general, we CANNOT expect the optimal
control is Markovian.
Discrete-time
Markov Control
v.s.
⇓
Continuous-time
Non-Markov Control
⇓ ??
Deterministic C.V. F
Deterministic C.V. F
The answer is obviously NO ! Given a general control u(·) ∈ U, for 0 ≤ t ≤ T ,
V u (0, x ) = ρ0,t
Z
t
c s , X 0,x ;u (s ), u(s ) ds
0
+ ρt , T
Z
T
c (s , X
0,x ;u
(s ), u(s ))ds + Φ X
t ,x ; u
(T )
!!
t
?
= ρ0,t
Z
t
c s , X 0,x ;u (s ), u(s ) ds + V u t , X t ,X
0,x ;u (t );u
(t )
!
0
Issues: V u (t , X (t )) is a random function!
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Issues about Current (Strong) Formulation
In continuous-times setting, in general, we CANNOT expect the optimal
control is Markovian.
Discrete-time
Markov Control
v.s.
⇓
Continuous-time
Non-Markov Control
⇓ ??
Deterministic C.V. F
Deterministic C.V. F
The answer is obviously NO ! Given a general control u(·) ∈ U, for 0 ≤ t ≤ T ,
V u (0, x ) = ρ0,t
Z
t
c s , X 0,x ;u (s ), u(s ) ds
0
+ ρt , T
Z
T
c (s , X
0,x ;u
(s ), u(s ))ds + Φ X
t ,x ; u
(T )
!!
t
?
= ρ0,t
Z
t
c s , X 0,x ;u (s ), u(s ) ds + V u t , X t ,X
0,x ;u (t );u
(t )
!
0
Issues: V u (t , X (t )) is a random function!
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Issues about Current (Strong) Formulation
In continuous-times setting, in general, we CANNOT expect the optimal
control is Markovian.
Discrete-time
Markov Control
v.s.
⇓
Continuous-time
Non-Markov Control
⇓ ??
Deterministic C.V. F
Deterministic C.V. F
The answer is obviously NO ! Given a general control u(·) ∈ U, for 0 ≤ t ≤ T ,
V u (0, x ) = ρ0,t
Z
t
c s , X 0,x ;u (s ), u(s ) ds
0
+ ρt , T
Z
T
c (s , X
0,x ;u
(s ), u(s ))ds + Φ X
t ,x ; u
(T )
!!
t
?
= ρ0,t
Z
t
c s , X 0,x ;u (s ), u(s ) ds + V u t , X t ,X
0,x ;u (t );u
(t )
!
0
Issues: V u (t , X (t )) is a random function!
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Weak Formulation – Admissible control system (XYZ’s Definition)
Definition (Admissible control system)
For 0 ≤ t ≤ t , U w [t , T ] is called an admissible control system if it satisfying the
following conditions:
(i) (Ω, F , P) is a complete probability space;
(ii) {W (s )}s ∈[0,T ] is an d -dimensional standard Brownian motion on (Ω, F , P) and
Ft = {Fs }s ∈[t ,T ] , where Fst = σ{{W (s ); t ≤ s ≤ T } ∪ N};
(iii) u : [t , T ] × Ω 7→ U is an {Fst }s ∈[t ,T ] -adapted process and u(·) ∈ H 2 [t , T ];
(iv) For any (t , x ) ∈ [0, T ] × Rn , control value function V : [0, T ] × Rn 7→ R:
!
Z T g
u
t ,x ;u
t ,x ;u
V (t , x ) = ρt ,T
c s, X
(s ), u(s ) + Φ(X
(T ))
t
Weak Formulation Objective
V (t , x ) =
Y. Jianing
inf
u(·)∈U w [t ,T ]
V u (t , x )
Risk-averse Control of Diffusion Processes
(11)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Weak Formulation – Admissible control system (XYZ’s Definition)
Definition (Admissible control system)
For 0 ≤ t ≤ t , U w [t , T ] is called an admissible control system if it satisfying the
following conditions:
(i) (Ω, F , P) is a complete probability space;
(ii) {W (s )}s ∈[0,T ] is an d -dimensional standard Brownian motion on (Ω, F , P) and
Ft = {Fs }s ∈[t ,T ] , where Fst = σ{{W (s ); t ≤ s ≤ T } ∪ N};
(iii) u : [t , T ] × Ω 7→ U is an {Fst }s ∈[t ,T ] -adapted process and u(·) ∈ H 2 [t , T ];
(iv) For any (t , x ) ∈ [0, T ] × Rn , control value function V : [0, T ] × Rn 7→ R:
!
Z T g
u
t ,x ;u
t ,x ;u
V (t , x ) = ρt ,T
c s, X
(s ), u(s ) + Φ(X
(T ))
t
Weak Formulation Objective
V (t , x ) =
Y. Jianing
inf
u(·)∈U w [t ,T ]
V u (t , x )
Risk-averse Control of Diffusion Processes
(11)
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Weak Formulation – Policy Evaluation
solution: (Y 0 , Z 0 ) is deterministic a.s.
P( · | Ftr )(ω)
(r , η(ω))
(t , x ) ∈ [0, T ] × Rn
data: (g, ξ ), Ftr ⊂ Ft
T
r
Idea: Under (Ω, F , P( · | Frt )(ω); Fr ), η is a.s. deterministic, also,
e (· ∧ s , ω) + W (r , ω) ,
u(s , ω) = ψ(ω, W (· ∧ s , ω)) = ψ s , W
Therefore, we get an almost surely deterministic control value function.
Policy Evaluation – Nested Structure
V (t , x ) =
u
ρgt,r
Z
r
c s, X
t ,x ; u
!
u
r ,X t ,x ;u (r );u
(s ), u(s ) ds + V r , X
(r )
t
Remark: a strong formulation can also obtained by partitioning outcome space.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Weak Formulation – Policy Evaluation
solution: (Y 0 , Z 0 ) is deterministic a.s.
P( · | Ftr )(ω)
(r , η(ω))
(t , x ) ∈ [0, T ] × Rn
data: (g, ξ ), Ftr ⊂ Ft
T
r
Idea: Under (Ω, F , P( · | Frt )(ω); Fr ), η is a.s. deterministic, also,
e (· ∧ s , ω) + W (r , ω) ,
u(s , ω) = ψ(ω, W (· ∧ s , ω)) = ψ s , W
Therefore, we get an almost surely deterministic control value function.
Policy Evaluation – Nested Structure
V (t , x ) =
u
ρgt,r
Z
r
c s, X
t ,x ; u
!
u
r ,X t ,x ;u (r );u
(s ), u(s ) ds + V r , X
(r )
t
Remark: a strong formulation can also obtained by partitioning outcome space.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Forward Backward Stochastic Differential Equation System
Weak Formulation
Weak Formulation – Policy Evaluation
solution: (Y 0 , Z 0 ) is deterministic a.s.
P( · | Ftr )(ω)
(r , η(ω))
(t , x ) ∈ [0, T ] × Rn
data: (g, ξ ), Ftr ⊂ Ft
T
r
Idea: Under (Ω, F , P( · | Frt )(ω); Fr ), η is a.s. deterministic, also,
e (· ∧ s , ω) + W (r , ω) ,
u(s , ω) = ψ(ω, W (· ∧ s , ω)) = ψ s , W
Therefore, we get an almost surely deterministic control value function.
Policy Evaluation – Nested Structure
V (t , x ) =
u
ρgt,r
Z
r
c s, X
t ,x ; u
!
u
r ,X t ,x ;u (r );u
(s ), u(s ) ds + V r , X
(r )
t
Remark: a strong formulation can also obtained by partitioning outcome space.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
4. DPE & HJB
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
4. DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Bellman Principle (Dynamic Programming Equation)
With the help of nested structure of control value function, we can easily
derive the Bellman Principle:
Theorem (Risk-averse Dynamic Programming Equation)
Under Assumption 3, for any (t , x ) ∈ [0, T ) × Rn and intermediate point
r ∈ [t , T ],
Z r !
g
t , x ;u
t ,x ;u
V (t , x ) =
inf
ρt ,r
c s, X
(s ), u(s ) ds + V r , X
(r ) .
u(·)∈U w [t ,T ]
t
We call it Risk-averse Dynamic Programming Equation.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Bellman Principle (Dynamic Programming Equation)
With the help of nested structure of control value function, we can easily
derive the Bellman Principle:
Theorem (Risk-averse Dynamic Programming Equation)
Under Assumption 3, for any (t , x ) ∈ [0, T ) × Rn and intermediate point
r ∈ [t , T ],
Z r !
g
t , x ;u
t ,x ;u
V (t , x ) =
inf
ρt ,r
c s, X
(s ), u(s ) ds + V r , X
(r ) .
u(·)∈U w [t ,T ]
t
We call it Risk-averse Dynamic Programming Equation.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
1. Dynamics of the System
2. Dynamic Risk Evaluation – Continuous-time Setting
3. Risk-Averse Modelling
4. DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Risk-averse Hamilton-Jacobi-Bellman Equation I
For α ∈ U , we define Laplacian operator Lα as follows: for w ∈ Cb1,2 ([0, T ] × Rn ),
n
n
X
X
1
[Lα ](t , x ) = ∂t w (t , x ) +
σσ> (t , x , α)ij ∂xi ,xj w (t , x ) +
bi (t , x , α)∂xi w (t , x )
2
i =1
i =1
Let’s postulate the following Risk-averse HJB equation associated with the controlled
system, for (t , x ) ∈ [0, T ) × Rn ,
(
)



α
α


c (t , x , α) + [L v ](t , x ) + g (t , [Dx v σ ](t , x )) = 0,
 min
α∈U
(12)




 v (T , x ) = Φ(T , x ), x ∈ Rn
Objective
Show the value function of risk-averse DPE is a viscosity solution of (12). And under
certain conditions, the solution of risk-averse HJB is the value function.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Risk-averse Hamilton-Jacobi-Bellman Equation I
For α ∈ U , we define Laplacian operator Lα as follows: for w ∈ Cb1,2 ([0, T ] × Rn ),
n
n
X
X
1
[Lα ](t , x ) = ∂t w (t , x ) +
σσ> (t , x , α)ij ∂xi ,xj w (t , x ) +
bi (t , x , α)∂xi w (t , x )
2
i =1
i =1
Let’s postulate the following Risk-averse HJB equation associated with the controlled
system, for (t , x ) ∈ [0, T ) × Rn ,
(
)



α
α


c (t , x , α) + [L v ](t , x ) + g (t , [Dx v σ ](t , x )) = 0,
 min
α∈U
(12)




 v (T , x ) = Φ(T , x ), x ∈ Rn
Objective
Show the value function of risk-averse DPE is a viscosity solution of (12). And under
certain conditions, the solution of risk-averse HJB is the value function.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Risk-averse Hamilton-Jacobi-Bellman Equation I
For α ∈ U , we define Laplacian operator Lα as follows: for w ∈ Cb1,2 ([0, T ] × Rn ),
n
n
X
X
1
[Lα ](t , x ) = ∂t w (t , x ) +
σσ> (t , x , α)ij ∂xi ,xj w (t , x ) +
bi (t , x , α)∂xi w (t , x )
2
i =1
i =1
Let’s postulate the following Risk-averse HJB equation associated with the controlled
system, for (t , x ) ∈ [0, T ) × Rn ,
(
)



α
α


c (t , x , α) + [L v ](t , x ) + g (t , [Dx v σ ](t , x )) = 0,
 min
α∈U
(12)




 v (T , x ) = Φ(T , x ), x ∈ Rn
Objective
Show the value function of risk-averse DPE is a viscosity solution of (12). And under
certain conditions, the solution of risk-averse HJB is the value function.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Risk-averse Hamilton-Jacobi-Bellman Equation II
Theorem (Risk-averse HJB)
Suppose Assumption 3 is satisfied and U is a compact set, then the value function
V (·, ·) is a viscosity solution of the equation (12).
Theorem (Verification Theorem)
Under the same assumption and let K ∈ Cb1,2 ([t , T ] × Rn ) satisfy (12). Then
K (t , x ) ≤ V u (t , x ) for any control u(·) ∈ U w [t , T ] and all (t , x ) ∈ [0, T ] × Rn .
Furthermore, if a control process u∗ (·) ∈ U w [0, T ] exists, satisfying for
almost all
∗
(s , ω) ∈ [0, T ] × Ω, together with the corresponding trajectory X 0,x ;u , the relation
( u (s ) ∈ argmin c s , X 0,x ;u (s ), α +
∗
α∈U
)
Lα K s , X 0,x ;u (s ) + g t , [Dx K · σα ](t , X 0,x ;u (s )) ,
∗
then K (t , x ) = V (t , x ) = V u (t , x ) for all (t , x ) ∈ [0, T ] × Rn .
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Discretization
Error Bounds for Using Piecewise Constant Control
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Error Bounds
Piecewise-constant Control
For any h ∈ (0, 1]. let Uht be a subset of U which are constant on intervals [t , t + h 2 ),
[t + h 2 , t + 2h 2 ], ..., [t + kh 2 , T ], where T − h 2 ≤ t + kh 2 ≤ T .
Value Function (Piecewise Constant Control)
Vh ( t , x ) =
inf V u (t , x )
u(·)∈Uht
By perturbation in time and space, regularization technique, we manage to show:
Error Bounds
Under some additional assumptions on the coefficients of FBSDE and cost
functional,
V (t , x ) − Vh (t , x ) ≤ Ne NT h 1/3
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Error Bounds
Piecewise-constant Control
For any h ∈ (0, 1]. let Uht be a subset of U which are constant on intervals [t , t + h 2 ),
[t + h 2 , t + 2h 2 ], ..., [t + kh 2 , T ], where T − h 2 ≤ t + kh 2 ≤ T .
Value Function (Piecewise Constant Control)
Vh ( t , x ) =
inf V u (t , x )
u(·)∈Uht
By perturbation in time and space, regularization technique, we manage to show:
Error Bounds
Under some additional assumptions on the coefficients of FBSDE and cost
functional,
V (t , x ) − Vh (t , x ) ≤ Ne NT h 1/3
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Approximation Idea
Comparison under
Original System
Perturbed System
Piecewise Constant
eh
Control, Vh ≈ V
eh , resulting V̂h ≈ V
eh
Mollification of V
Estimates Through Risk-averse HJB
DPE ⇒ HJB, HJB ⇒ DPE
V ≈ Vh
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Current Research
Research Going on
Solving Policy Evaluation – Evaluation of Risk: driver for different risk
measure, maximum principle, duality (almost done).
Solving Risk-averse Dynamic Programming Equation in Discrete-time Setting.
Application
To measure the exposure of financial derivatives. For example, Swaption is a
frequently traded instrument in interest market. Exchange floating rate
against a predetermined fixed rate leads to risk.
Risk-averse portfolio optimization problem, where the expectation is replaced
by specific dynamic risk measures. Under different risk-aversion, how investor
will manage their portfolios.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Current Research
Research Going on
Solving Policy Evaluation – Evaluation of Risk: driver for different risk
measure, maximum principle, duality (almost done).
Solving Risk-averse Dynamic Programming Equation in Discrete-time Setting.
Application
To measure the exposure of financial derivatives. For example, Swaption is a
frequently traded instrument in interest market. Exchange floating rate
against a predetermined fixed rate leads to risk.
Risk-averse portfolio optimization problem, where the expectation is replaced
by specific dynamic risk measures. Under different risk-aversion, how investor
will manage their portfolios.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
References
• N.V. Krylov., Controlled Diffusion Process, Stochastic Modeling and Applied
Probability, Springer, 2008
• S. Peng., Nonlinear expectations, nonlinear evaluations and risk measures. Lecture
Notes in Mathematics, Springer, 2004,
• H.Pham., Continuous-time Stochastic control with Financial Applications,
Stochastic Modeling and Applied Probability, Springer, 2010,
• F. Jingnan, A. Ruszczyński, Process-based Risk Measures for Observable and
Partially Observable Discrete-Time Control Systems, submitted,
• Ruszczyński, A., Jianing, Y., "Risk-averse Hamilton-Jacobi-Bellman Equation",
SIAM Control and Its Application, Conference Proceedings, 2015.
• Ruszczyński, A., Jianing, Y., "Risk-averse Control of Diffusion Process", SIAM
Control and Optimization, submitted, 2015.
• Yong. J., Zhou, X., Stochastic Control – Hamiltonian Systems and HJB Equations,
Springer , 1998.
Y. Jianing
Risk-averse Control of Diffusion Processes
Dynamics of the System
Dynamic Risk Evaluation – Continuous-time Setting
Risk-Averse Modelling
DPE & HJB
Risk-averse Dynamic Programming Equation
Risk-averse Hamilton-Jacobi-Bellman Equation
Thank You !
Y. Jianing
Risk-averse Control of Diffusion Processes