Introduction to risk-averse optimisation
Pantelis Sopasakis
IMT Institute for Advanced Studies Lucca
February 17, 2016
Outline
1 / 50
I. Introductory
X Scenario trees
X Conditional risk measures
2 / 50
Scenario trees
At every node α at stage k we need to take
a decision uαk in a non-anticipating way.
Node α
We assume that the disturbance is not
measured at time k.
Stage k
3 / 50
Scenario trees
With every node α at stage k < N
we associate a set of children,
child(α).
4 / 50
Filtration
Set child(α) can be identified with a subset of ΩN . Recall that this way
we may construct a filtration on the scenario tree.
5 / 50
Filtration
6 / 50
Filtration
7 / 50
Filtration
8 / 50
Filtration
9 / 50
Filtration
10 / 50
Conditional probability
For every set child(α) we define a
probability vector of conditional
probabilities.
Now child(α) becomes a probability
space...
11 / 50
The siblings probability space
On the probability space child(α) we
may define random variables
Z : child(α) → IR
12 / 50
The siblings probability space
These random variables are identified by
vectors
Z ∈ R| child(α)|
13 / 50
Conditional risk measures
For every such random variable Z we
may define a trivial risk measure as
ρα (Z) = Epα [Z]
14 / 50
Conditional risk measures
Of course we can define other conditional risk measures
ρα : R| child(α)| → IR.
15 / 50
Conditional risk measures
For instance we may use
ρα [Z] = inf {t + λ−1
α Epα [Z − t]+ },
t∈IR
with λα ∈ (0, 1).
16 / 50
Conditional risk measures
... or
ρα [Z] = Epα + cEpα [[Z − Epα [Z]]p ]1/p ,
with c ∈ [0, 1].
17 / 50
Conditional risk mappings
18 / 50
Conditional risk mappings
19 / 50
Conditional risk mappings
20 / 50
Conditional risk mappings
Define
Zk := {Z : ΩN → IR, Fk − measurable}
∼ {Z : Ωk → IR}
=
∼
= IRΩk ∼
= IR|Ωk | .
Of course
Z0 ∼
= IR.
Then a conditional risk mapping ρk+1 : Ωk+1 → IRΩk can be identified by
a mapping
ρk+1 : Zk+1 → Zk .
21 / 50
Conditional risk mappings – Examples
Conditional expectation. ρk+1 : Zk+1 → Zk with
ρk+1 [Z](α) = ρα [Z] = Epα [Z].
22 / 50
Conditional risk mappings – Examples
Conditional expectation. ρk+1 : Zk+1 → Zk with
ρk+1 [Z](α) = ρα [Z] = Epα [Z].
Conditional AV@Rα [·]. ρk+1 : Zk+1 → Zk with
ρk+1 [Z](α) = ρα [Z] = inf t + λ−1
α Epα [Z − t]+ .
t
22 / 50
Conditional risk mappings – Examples
Conditional expectation. ρk+1 : Zk+1 → Zk with
ρk+1 [Z](α) = ρα [Z] = Epα [Z].
Conditional AV@Rα [·]. ρk+1 : Zk+1 → Zk with
ρk+1 [Z](α) = ρα [Z] = inf t + λ−1
α Epα [Z − t]+ .
t
Conditional MUS. ρk+1 : Zk+1 → Zk and for cα ≥ 0
ρk+1 [Z](α) = ρα [Z] = Epα [Z] + cα Epα [Z − Epα [Z]]+ .
22 / 50
Regularity assumptions
Conditional risk mappings ρk+1 : Zk+1 → Zk must satisfy the following
1. Convexity: For Z1 , Z2 ∈ Zk+1 and λ ∈ (0, 1),
ρk+1 (λZ1 + (1 − λ)Z2 ) ≤ λρk+1 (Z1 ) + (1 − λ)ρk+1 (Z2 )
23 / 50
Regularity assumptions
Conditional risk mappings ρk+1 : Zk+1 → Zk must satisfy the following
1. Convexity: For Z1 , Z2 ∈ Zk+1 and λ ∈ (0, 1),
ρk+1 (λZ1 + (1 − λ)Z2 ) ≤ λρk+1 (Z1 ) + (1 − λ)ρk+1 (Z2 )
2. Monotonicity: For Z1 , Z2 ∈ Zk+1 with Z1 ≤ Z2 ,
ρk+1 (Z1 ) ≤ ρk+1 (Z2 )
23 / 50
Regularity assumptions
Conditional risk mappings ρk+1 : Zk+1 → Zk must satisfy the following
1. Convexity: For Z1 , Z2 ∈ Zk+1 and λ ∈ (0, 1),
ρk+1 (λZ1 + (1 − λ)Z2 ) ≤ λρk+1 (Z1 ) + (1 − λ)ρk+1 (Z2 )
2. Monotonicity: For Z1 , Z2 ∈ Zk+1 with Z1 ≤ Z2 ,
ρk+1 (Z1 ) ≤ ρk+1 (Z2 )
3. Translation equivariance: For Z ∈ Zk+1 and Y ∈ Zk ,
ρk+1 (Z + Y ) = Y + ρk+1 (Z)
23 / 50
Regularity assumptions
Conditional risk mappings ρk+1 : Zk+1 → Zk must satisfy the following
1. Convexity: For Z1 , Z2 ∈ Zk+1 and λ ∈ (0, 1),
ρk+1 (λZ1 + (1 − λ)Z2 ) ≤ λρk+1 (Z1 ) + (1 − λ)ρk+1 (Z2 )
2. Monotonicity: For Z1 , Z2 ∈ Zk+1 with Z1 ≤ Z2 ,
ρk+1 (Z1 ) ≤ ρk+1 (Z2 )
3. Translation equivariance: For Z ∈ Zk+1 and Y ∈ Zk ,
ρk+1 (Z + Y ) = Y + ρk+1 (Z)
4. Positive homogeneity: ρk+1 (αZ) = αρk+1 (Z) for all α ≥ 0.
23 / 50
Regularity assumptions
The mapping ρk+1 : Zk+1 → Zk is a conditional risk mapping if the
underlying risk measures ρα are coherent for all α ∈ Ωk .
24 / 50
Exercise
Let ρk : Zk → Zk−1 , for k ∈ N[0,N ] , be coherent risk measures. Verify that
ρ := ρ1 ◦ ρ2 ◦ · · · ◦ ρN , ρ : ZN → Z1 , is a coherent risk measure on ZN
(recall that ρ1 : Z1 → Z0 ∼
= IR).
25 / 50
Representation of conditional risk mappings
When all ρα at stage k are coherent, there is a Ak+1 (α) so that
ρα [Z] =
max
pα ∈Ak+1 (α)
Epα [Z].
Recall that Z is a function Z : child(α) → IR, or, equivalently a vector
Z ∈ IR| child(α)| . We will now extend ρα on IR|Ωk+1 | , i.e.,
ρα : IR|Ωk+1 | → IR.
Let’s see how this happens...
26 / 50
Extension of pα
27 / 50
A measure on Ωk+1
Having a probability measure ν on Ωk
and knowing the probability measures pα
for each α ∈ Ωk we may define a prob.
measure on Ωk+1 as
X
µ=
ν(α)pα .
α∈Ωk
28 / 50
Understanding and using µ
For every such measure µ on Ωk+1 notice that for Z ∈ Zk+1
Eµ [Z | Fk ](α) = Epα [Z]
for α ∈ Ωk . Note that
1. In the RHS above, pα is a prob. vector on Ωk+1
2. Given a measure ν on Ωk , we created a measure µ on Ωk+1
3. Fk provides all necessary info up to time k, therefore
4. Eµ [Z | Fk ] is a function of α
5. Eµ [Z | Fk ] is a random variable
29 / 50
Representation of conditional risk mapprings
We can now define the set of probability measures on Ωk+1



X
ν : prob. measure on Ωk 
.
Ck+1 = µ =
ν(α)pα α
p ∈ Ak+1 (α), ∀α ∈ Ωk 

α∈Ωk
Then
ρk+1 [Z] = max Eµ [Z | Fk ]
µ∈Ck+1
30 / 50
Interpretations of ρk+1
For all α ∈ Ωk , ρk+1 is defined sibling-wise as
(ρk+1 [Z])(α) =
max
p∈Ak+1 (α)
Ep [Z]
We also saw that ρk+1 [Z] = ρk+1 [Z](ν) is

ν : prob. measure on Ωk


X

µ=
ν(α)pα
ρk+1 [Z] = max Eµ [Z | Fk ] µ 
α∈Ωk


pα ∈ Ak+1 (α), ∀α ∈ Ωk







31 / 50
End of first section
X We revised the notion of a filtration
X RVs at stage k can be written as Fk -meas RVs on ΩN
X Zk ∼
= IR|Ωk |
X We introduced the very useful notion of risk mappings
X We derived a nice representation of conditional risk mappings
32 / 50
II. Risk-averse optimisation
X Problem statement
X Dynamic programming equations
33 / 50
Problem statement
Assume the system dynamics is given by
xk+1 = f (xk , uk , wk ),
where wk is a random variable. We introduce the risk-averse optimisation problem in nested form
h
inf `0 (x0 , u0 , ω) + ρ1
inf
`1 (x1 , u1 , ω) + . . .
u0 ∈U0 (x0 )
u1 ∈U1 (x1 ,ω)
+ ρN −1
inf
`N −1 (xN −1 , uN −1 , ω)+
uN −1 ∈UN −1 (xN −1 ,ω)
i
+ ρN [`N (xN , ω)]
34 / 50
Remark #1
The innermost term of the problem is ρN [`N (xN , ω)]. We require that
`N (xN , ω) is FN -measurable, i.e., `N ∈ ZN and ρN is a mapping
ρN : ZN → ZN −1 ,
thus ρN [`N (xN , ω)] ∈ ZN −1 .
35 / 50
Remark #2
The first term of the optimisation problem is `0 (x0 , u0 , ω). We require
that `0 is F0 -measurable, i.e., `0 ∈ Z0 . But, recall that
F0 = {ΩN , ∅},
Z0 ∼
= IR,
therefore `0 is deterministic.
36 / 50
Remark #3
Take a look at:
ρN −1
inf
uN −1 ∈UN −1 (xN −1 ,ω)
`N −1 (xN −1 , uN −1 , ω) + ρN [`N (xN , ω)]
Here xN depends on uN −1 : xN = f (xN −1 , uN −1 , wN −1 ). This way, the
above becomes a FN −2 -measurable RV which depends on xN −1 .
37 / 50
Remark #4
Overall, we require that functions `k (x, ·) are Fk -measurable and control
laws are causal (⇔ Fk -measurable).
38 / 50
Alternative formulations
Risk-averse problems can be written in the following ways
X Original nested formulation
X Dynamic programming
X Using the composite risk measure
39 / 50
Dynamic programming
Define
V0? (xN , ω) := ρN [`N (xN , ω)]
Here, `N (xN , ·) ∈ ZN and V0? (xN , ·) ∈ ZN −1 . Then
n
Vk? (xN −k , ω) := inf `N −k (xN −k , uN −k , ω)
uN −k
o
?
+ ρN −k+1 [Vk+1
(f (xN −k , uN −k , ω), ω)]
?
and `N −k (x, u, ·) ∈ ZN −k & Vk+1
(x, ·) ∈ ZN −k+1 , thus Vk? (x, ·) ∈ ZN −k .
40 / 50
Dynamic programming overview
1. At every iteration k of DP, Vk (x, ·) is an FN −k -meas. RV
2. f (x, u, ·) are Fk -meas
3. `k (x, u, ·) are Fk -meas and `N (x, ·) is FN -measurable
4. ρk : Zk → Zk−1
5. The last iteration of DP involved ρ1 : Z1 → Z0 ∼
= IR
41 / 50
Dynamic programming
We showed that the DP iteration is defined by
n
o
?
Vk? (x, ω) := inf `N −k (x, u, ω) + ρN −k+1 [Vk+1
(f (x, u, ω), ω)]
u
n
o
?
= inf `N −k (x, u, ω) + sup Eµ [Vk+1
(f (x, u, ω) | FN −k ](ω)
u
µ∈CN −k+1
and this is because for all Z ∈ ZN −k+1
ρN −k+1 [Z] =
sup
Eµ [Z | FN −k ]
µ∈CN −k+1
42 / 50
Composite risk measure
This representation is the counterpart of the product space representation (see stochastic MPC slides).
We shall introduce a risk measure ρ̄ so that the problem is written as
"N −1
#
X
min
ρ̄
`k (xk , uk , ω) + ρN (xN , ω)
u0 ,u1 ,...,uN −1
k=0
43 / 50
Composite risk measure
Since all ρk are conditional risk measures it holds that
ρN −1 [ZN −1 + ρN [ ZN ] ] = ρN −1 [ ρN [ZN −1 + ZN ] ]
44 / 50
Composite risk measure
Since all ρk are conditional risk measures it holds that
ρN −1 [ZN −1 + ρN [ ZN ] ] = ρN −1 [ ρN [ZN −1 + ZN ] ]
I
ZN ∈ ZN
44 / 50
Composite risk measure
Since all ρk are conditional risk measures it holds that
ρN −1 [ZN −1 + ρN [ ZN ] ] = ρN −1 [ ρN [ZN −1 + ZN ] ]
I
ZN ∈ ZN
I
ρN : ZN → ZN −1
44 / 50
Composite risk measure
Since all ρk are conditional risk measures it holds that
ρN −1 [ZN −1 + ρN [ ZN ] ] = ρN −1 [ ρN [ZN −1 + ZN ] ]
I
ZN ∈ ZN
I
ρN : ZN → ZN −1
I
ρN [ZN + Y ] = Y + ρN [ZN ] for Y ∈ ZN −1
44 / 50
Composite risk measure
Then the original nested formulation
h
inf
`1 (x1 , u1 , ω) + . . .
inf `0 (x0 , u0 , ω) + ρ1
u0 ∈U0 (x0 )
u1 ∈U1 (x1 ,ω)
+ ρN −1
inf
`N −1 (xN −1 , uN −1 , ω)+
uN −1 ∈UN −1 (xN −1 ,ω)
i
+ ρN [`N (xN , ω)]
becomes...
45 / 50
Composite risk measure
...becomes
inf
N
−1
X
(ρ1 ◦ ρ2 ◦ · · · ◦ ρN −1 ◦ ρN )[
`k (xk , uk , ω) + `N (xN , ω)],
{z
}
k=0
ρ
u0 ,u1 ,...,uN −1 |
subject to the system dynamics and the standard measurability assumption
on uk .
46 / 50
Composite risk measure
...becomes
inf
N
−1
X
(ρ1 ◦ ρ2 ◦ · · · ◦ ρN −1 ◦ ρN )[
`k (xk , uk , ω) + `N (xN , ω)],
{z
}
k=0
ρ
u0 ,u1 ,...,uN −1 |
subject to the system dynamics and the standard measurability assumption
on uk .
ρ is a risk measure on Z0 × · · · × ZN , that is
ρ(Z0 , Z1 , . . . , ZN ) = Z0 + ρ1 [Z1 + . . . + ρN −1 [ZN −1 + ρN [ZN ]]]
46 / 50
Composite risk measure
Since
N
−1
X
`k (xk , uk , ω) + `N (xN , ω)
k=0
as a function of ω is FN -measurable, we may naturally identify ρ with the
coherent risk measure
ρ̄(Z0 + Z1 + . . . + ZN ) = Z0 + ρ1 [Z1 + . . . + ρN −1 [ZN −1 + ρN [ZN ]]]
this is called composite risk measure.
47 / 50
Composite risk measure
The risk-averse optimisation problem becomes simply
"N −1
#
X
inf ρ̄
`k (xk , uk , ω) + `N (xN , ω) ,
u0 ,...,uN
k=0
subject to the system dynamics and measurability assumptions.
Note, however, that usually it is not possible to have a closed form for ρ̄.
48 / 50
End of second section
Just like their stochastic counterpart, risk-averse problems admit three
representations:
X Nested
X Dynamic programming
X Composite risk measure (product space)
X The last one can be cumbersome to work with
X We can also formulate the risk-averse problem using multi-period
conditional risk mappings (we’ll discuss that later)
49 / 50
References
1. A. Shapiro, D. Dentcheva, and A.P. Ruszczyński, Lectures on stochastic
programming: modeling and theory, SIAM 2009.
2. T. Asamov and A. Ruszczyński, Time-consistent approximations of risk-averse
multistage stochastic optimization problems, Mathematical Programming 153(2),
Springer editions, 2014.
50 / 50