Scenario-Free Approximations to Stochastic
Programming via Decision Rules
Daniel Kuhn
Department of Computing
Imperial College London
London SW7 2AZ, United Kingdom
March 9, 2011
Deterministic Optimisation
Deterministic optimisation model:
minimise
f (x , ξ)
subject to
x ∈ X (ξ),
x
where
• x are decision variables
• ξ are (precisely known) parameters
Real world is uncertain. Why not use ξ = E ξe ?
Deterministic Optimisation
Image Source: Sam L. Savage, Stanford University
alive (E [position]) = true,
but
E [alive (position)] = false!
Deterministic Optimisation
Portfolio optimisation:
wealth (E [stock returns])
vs.
E [wealth (stock returns)]
Stochastic Programming
One-stage stochastic program:
minimise
E [f (x (ξ), ξ)]
subject to
x (ξ) ∈ X (ξ) P-a.s.
x(·)
x (ξ) ∈ X (ξ)
ξ
f (x (ξ), ξ)
Stochastic Programming
Multi-stage stochastic program: several sequential decisions
• capacity expansion: several investment stages
• production planning: annual production plan (seasonalities)
• portfolio optimisation: rebalancing, asset & liability mgmt.
• ...
x1 (ξ1 )
x2 (ξ1 , ξ2 )
...
ξ2
ξ1
f1 (x1 (ξ1 ), ξ1 )
f2 (x2 (ξ1 , ξ2 ), x1 (ξ1 ), ξ1 , ξ2 )
Scenario-Based S.P.
Discretise distribution: into scenarios
X
ps f (x s , ξ s )
minimise
s
x
subject to
s∈S
x s ∈ X (ξ s )
∀s ∈ S.
x (ξ 1 )
x (ξ 2 )
x (ξ 2 )
x (ξ 3 )
ξ
f (x (ξ 2 ), ξ 2 )
Scenario-Based S.P.
Portfolio optimisation:
today
tomorrow
Scenario-Based S.P.
Multi-stage stochastic program:
Solution time:
Linear Decision Rules1
Xξ
minimize
E [f (X ξ, ξ)]
X2
subject to
X ξ ∈ X (ξ) P-a.s.
X1
X
X ξ ∈ X (ξ)
ξ2
ξ1
ξ
f (X ξ, ξ)
1 Ben-Tal
et al., Math. Programming, 2004.
Linear Decision Rules
Linear S.P. with fixed recourse:
minimize
x(·),s(·)
subject to
E c(ξ)⊤ x (ξ)
Ax (ξ) + s(ξ) = b(ξ) P-a.s.
s(ξ) ≥ 0 P-a.s.
=⇒ Optimize over all measurable functions x (·) and s(·)
Assumptions:
ξ1
• Linear data dependence:
c(ξ) = Cξ, b(ξ) = Bξ
Ξ
• Tractable uncertainty set:
supp P = Ξ, ξ ∈ Ξ ⇒ ξ1 = 1
• Finite second moments:
M = E(ξξ ⊤ )
ξ1 = 1
ξ2
ξ3
Linear Decision Rules
Linearize the decisions:
minimize
E ξ⊤C ⊤X ξ
subject to
AX ξ + Sξ = Bξ
Sξ ≥ 0 P-a.s.
X ,S
P-a.s.
=⇒ Optimize over all matrices X and S
Reexpress the semi-infinite constraints:
minimize tr MC ⊤ X
cone(Ξ)
ξ1
X ,S
Ξ
subject to AX + S = B
S ∈ (cone(Ξ)∗ )m
ξ3
=⇒ Conic program of polynomial size
in the input data
ξ2
Semi-Infinite Constraints
Lemma
For any z ∈ Rk the following statements are equivalent:
(i) z ⊤ ξ ≥ 0 ∀ ξ ∈ Ξ
(ii) z ∈ cone(Ξ)∗
(iii) ∃ λ ∈ Rl with λ ≥ 0, W ⊤ λ = z, and h⊤ λ ≥ 0
Proof.
⇐⇒
⇐⇒
⇐⇒
z ⊤ξ ≥ 0 for all ξ subjectto W ξ ≥ h
0 ≤ min z ⊤ ξ : W ξ ≥ h
ξ∈Rk 0 ≤ max h⊤ λ : W ⊤ λ = z, λ ≥ 0
λ∈Rl
∃ λ ∈ Rl with W ⊤ λ = z, h⊤ λ ≥ 0, λ ≥ 0
Example
Inventory problem:
• three factories produce single good, one warehouse
• limited per-period production and storage capacities
• demand uniformly distributed around known nominal demand
• nominal demand seasonal: E(ξt ) = 1,000 × 1 +
1
2
sin
h
π(t−1)
12
i
Bounding the Optimality Gap2
objective value
So far: obtained upper bound on optimal value via restriction from
measurable to linear decision rules
?
x
linear decision rules
measurable decision rules
Idea: obtain lower bound on optimal value to bound suboptimality
2 Kuhn
et al., Math. Programming, 2010.
Bounding the Optimality Gap
Step 1: dualize stochastic program in measurable decision rules
dual problem:
maximisation
primal problem:
minimisation
objective
value
duality theory: under mild regularity conditions,
optimal values of primal and dual problems coincide!
r
Result: dual stochastic program in measurable decision rules
Bounding the Optimality Gap
Step 2: restrict dual stochastic program to linear decision rules
bound on optimality gap
linear decision rules
dual gap primal gap
measurable decision rules
Result: dual stochastic program with linear decision rules allows us
to bound incurred suboptimality
Dual Linear Decision Rules
Min-max reformulation of linear S.P.:
minimize
x(·), s(·)
sup E c(ξ)⊤ x (ξ) + y (ξ)⊤ [Ax (ξ) + s(ξ) − b(ξ)]
y (·)
subject to s(ξ) ≥ 0
P-a.s.
=⇒ Minimize (supremum over all measurable dual decisions y (·))
over all measurable primal decisions x (·) and s(·)
Linearize the dual decisions:
minimize
x(·), s(·)
subject to
sup E c(ξ)⊤ x (ξ) + ξ ⊤ Y ⊤ [Ax (ξ) + s(ξ) − b(ξ)]
Y
s(ξ) ≥ 0 P-a.s.
Dual Linear Decision Rules
Dual LDR problem is equivalent to:
minimize E c(ξ)⊤ x (ξ)
x(·),s(·)
subject to E [Ax (ξ) + s(ξ) − b(ξ)] ξ ⊤ = 0
s(ξ) ≥ 0 P-a.s.
=⇒ constraint aggregation
And equivalent to:
minimize
X ,S
tr MC ⊤ X
subject to AX + S = B
SM ∈ cone(Ξ)m
=⇒ Conic program of polynomial size in the input data
Semi-Infinite Constraints
Lemma
For any z ∈ Rk the following statements are "essentially" equivalent:
(i) z ∈ cone(Ξ);
(ii) ∃ s(·) with E(s(ξ) ξ) = z and s(ξ) ≥ 0 P-a.s.
Sketch of Proof.
The feasible sets of the conditions (i) and (ii) represent pointed cones
in Rk ⇒ w.l.o.g. assume that z1 = 1. Thus,
• (i) ⇐⇒ z ∈ Ξ
• (ii) ⇐⇒ z is the mean of the distribution Q with density
dQ/dP(ξ) = s(ξ)
• As Q is supported on Ξ, (ii) is "essentially" equivalent to (i)
Example Problem Revisited
Inventory problem:
• three factories produce single good, one warehouse
• limited per-period production and storage capacities
• demand uniformly distributed around known nominal demand
• nominal demand seasonal: E(ξt ) = 1,000 × 1 +
1
2
sin
h
π(t−1)
12
i
Piecewise Linear Decision Rules3
Linear decision rules can fail:
minimize
E [x (ξ)]
subject to
x (ξ) ≥ kξk1
x(·)
P-a.s.
k
where ξ ∼ U [−1, 1] .
Remedy: use piecewise linear decision rules instead
3 Goh
& Sim, Oper. Research, 2010; Georghiou et al., Optimization Online, 2010.
Piecewise Linear Decision Rules
Piecewise linear decision rule ≡ linear decision rule in lifted space:
e
x (ξ)
x (ξ)
X2
X0
X1
ξ
ξb
ξ
Ξ
n o
x (ξ) = X0 + X1 min ξ, ξb − ξ
n o
+ X2 max ξ, ξb − ξb
}
X2
X0
X1
ξ
ξb
ξ − ξb
Ξ
e = X0 + X1 ξe1 + X2 ξe2
x (ξ)


n o
min ξ, ξb − ξ
n o

where ξe = 
max ξ, ξb − ξb
Increase Flexibility of Decision Rules
′
x (ξ) =
k
X
x i Li (ξ),
where L(ξ) = (L1 (ξ), . . . , Lk ′ (ξ))⊤
i=1
 
ξ1
L(ξ) = ξ2
ξ3


ξ1
max{ξ2, 0}



L(ξ) = 
 min{ξ2, 0} 
max{ξ3, 0}
min{ξ3, 0}


ξ1
 ξ 
 2 


L(ξ) =  ξ3 


2
(ξ2) 
(ξ3)2
Lifting and Retraction Operators
Lifting:
Retraction:
′
L : Rk → Rk ,
′
R : Rk → Rk ,
ξ 7→ ξ ′
ξ ′ 7→ ξ
(A1) L is continuous and satisfies L1 (ξ) = 1 for all ξ ∈ Ξ;
(A2) R is linear;
(A3) R ◦ L = Ik ;
ξ1
1
(A4) The component mappings of L are linearly independent


ξ1
ξ1
L
=  ξ2  ξ1′
ξ2
(ξ2)2
ξ3′
Ξ
ξ2
 ′
′
ξ1
ξ
R ξ2′  = 1′
ξ2
ξ3′
1
Ξ′ = L (Ξ)
ξ2′
Lifted Stochastic Program
• Probability distribution on lifted space
“
”
` ´
Pξ′ B ′ := Pξ {ξ ∈ Rk : L(ξ) ∈ B ′ }
min
s.t.
”
“
Eξ c (ξ)⊤ x(ξ)
x ∈ Lk ,n
Ax(ξ) ≤ b (ξ) Pξ -a.s.
(SP)
Lifting
−−−−→
Theorem
Problems SP and LSP are equivalent.
min
s.t.
′
∀B ′ ∈ B(Rk ).
”
“
⊤
Eξ′ c (Rξ ′ ) x(ξ ′ )
x ∈ Lk ′ ,n
Ax(ξ ′ ) ≤ b (Rξ ′ ) Pξ′ -a.s.
(LSP)
Decision Rule Approximations
min
s.t.
x (ξ) = X ′ L(ξ)
“
”
Eξ c (ξ)⊤ X ′ L (ξ)
′
min
n×k ′
X ∈R
AX ′ L (ξ) ≤ b (ξ) Pξ -a.s.
(N UB)
s.t.
x ′ (ξ ′ ) = X ′ ξ ′
“
”
⊤
Eξ′ c (Rξ ′ ) X ′ ξ ′
′
X ′ ∈ Rn×k
AX ′ ξ ′ ≤ b (Rξ ′ ) Pξ′ -a.s.
(LUB)
Theorem
• Problems N UB and LUB are equivalent
• Problems N LB and LLB are equivalent
x (ξ)
1
L(ξ) =
−1
0
1
ξ
max{ξ, 0}
min{ξ, 0}
′
ξ
R 1′ = ξ1′ + ξ2′
ξ2
1 x ′ ξ ′
ξ1′
1
0
ξ2′
−1
Tractability of LU B
• To apply LDRs to an instance of SP,
Ξ := Support (Pξ ) must be a convex polytope
′
• Ξ := Support (Pξ′ ) = L(Ξ) non-convex for non-linear L
=⇒ Find the convex hull of Ξ′
ξ1


ξ1
ξ
L 1 =  min{ξ2, 1} 
ξ2
max{ξ2, 0}
2
 ′
′ ξ1
ξ
R ξ2′  = ′ 1 ′
ξ1 + ξ2
′
ξ3
Ξ
0
2 ξ2
Theorem
• LUB generally intractable
• LLB generally intractable
conv(Ξ′)
ξ3′
2
′
1 ξ2
ξ1′
1
0
Liftings with Axial Segmentation
8
ξi ˘
>
>
¯
<
min ˘ξi , z1i ˘
⊥
¯
¯
Lij (ξ) :=
i
,0
> max ˘min ξi , zji ¯− zj−1
>
:
i
max ξi − zj−1
,0
if ri
if ri
if ri
if ri
= 1,
> 1, j = 1,
> 1, j = 2, . . . , ri − 1,
> 1, j = ri .
r
i
X
` ´
ξij′ .
Ri⊥ ξ ′ :=
j=1
• Lifting depends only on individual ξi ’s
ξ1
z11
z01
x(ξ) = X ′L(ξ)
z02
z12
z22
z32 ξ2
ξ1
ξ2
Liftings with General Segmentation
L(ξ) := G ◦ L⊥ ◦ F (ξ)
R(ξ ′ ) := F + ◦ R ⊥ ◦ G+ (ξ ′ )
• Piecewise linear w.r.t. linear combinations of ξi ’s
ξ3 := ξ1 + ξ2
ξ1
x(ξ)
ξ1
′
ξ32
ξ3 := ξ1 + ξ2
′
ξ31
ξ2
ξ2
Piecewise Linear Decision Rules
Example problem: capacity expansion of a power grid
• 10 regions with uncertain demand
• 5 power plants with known capacity, uncertain operating costs
• 24 transmission lines with known capacity
• goal: meet demand at lowest expected costs, via
• capacity expansion plan (here-and-now)
• plant operating policies (wait-and-see)
Stochastic Optimal Control
System dynamics:
x t+1 = At x t + Bt u t + Ct ξ t
y t = Dt x t + E t ξ t
ff
t = 1, . . . , T −1
m
x = Bu + Cξ
y = Dx + Eξ
Control problem:
i
h
minimize E u ⊤ Ju u + x ⊤ Jx x
u,x ,s
9
subject to x = Bu + Cξ
=
Fu u + Fx x + Fs s = h
P-a.s.
;
s≥0
Causal controllers:
u t = ϕt (y 1 , . . . , y t )
Controller Information Structure4
Purified observations:
• η t = y t − ȳ t
• ȳ t = obervation of noise-free system at time t
• η = Gξ where G = DC + E
Adapted controllers:
controllers adapted to y ≃ controllers adapted to η
⇒ Information structure is not decision-dependent
Affine controllers:
controllers affine in y ≃ controllers affine in η
⇒ Optimising over affine controllers is tractable
4 Ben-Tal
et al., Math. Programming, 2006; Goulart & Kerrigan, Int. J. Control, 2007
Primal Affine Controllers
Controllers affine in y:
i
h
minimize E u ⊤ Ju u + x ⊤ Jx x
• U is lower block
u,x,s,U
subject to
x = Bu + Cξ, u = Uy
Fu u + Fx x + Fs s = h
s≥0
Controllers affine in η:
h
i
minimize E u ⊤ Ju u + x ⊤ Jx x
9
>
=
triangular (causal)
P-a.s.
u,x,s,Q
subject to
x = Bu + Cξ, u = Qη
Fu u + Fx x + Fs s = h
s≥0
9
>
=
• u, x become non-linear
(rational) functions of U
>
;
• Q is lower block
P-a.s.
>
;
triangular (causal)
• u, x become linear
functions of Q
Tractable conic program:
`
´
minimize tr G⊤ Q ⊤ (Ju + B ⊤ Jx B)QGM + 2C ⊤ Jx BQGM + C ⊤ Jx CM
Q,S
subject to
(Fu + Fx B)QG + Fx C + Fs S − he0⊤ = 0
S ∈ (cone(Ξ)∗ )m
Dual Affine Controllers5
Dual controllers affine in ξ:
h
i
minimize sup E u ⊤ Ju u + x ⊤ Jx x + ξ⊤ Y ⊤ (Fu u + Fx x + Fs s − h)
u,x,s
Y
ff
subject to x = Bu + Cξ
P-a.s.
s≥0
Constraint aggregation:
ˆ
˜
minimize E u ⊤ Ju u + x ⊤ Jx x
u,x,s
subject to
x ˆ= Bu + Cξ P-a.s.
˜
E (Fu u + Fx x + Fs s − h)ξ⊤ = 0
s ≥ 0 P-a.s.
Tractable conic program:
`
´
minimize tr G⊤ Q ⊤ (Ju + B ⊤ Jx B)QGM + 2C ⊤ Jx BQGM + C ⊤ Jx CM
Q,S
subject to
(Fu + Fx B)QG + Fx C + Fs S − he0⊤ = 0
SM ∈ cone(Ξ)m
5 Hadjiyiannis
et al., Optimization Online, 2010
Double Integrator
120
Assumptions:
110
• System dynamics
»
1
0
» –
» –
–
1
0.2
0.1
xt +
ut +
ξ
1
1
0.1 t
Optimal Value
x t+1 =
100
Open−loop
Primal Affine
Dual Affine
Nominal
90
80
70
60
• Perfect state measurements: y t = x t
• i.i.d. noise ξt ∼ U ([0, 2])
50
40
2
4
6
T
8
10
u,x
subject to
subject to
− (5, 5)⊤ ≤ x t ≤ (5, 5)⊤
(1, 1) x t ≤ 5
(1, −1) x t ≤ 5
− 1 ≤ ut ≤ 1
9
>
>
=
>
>
;
P-a.s. ∀t
Second Component of x1
4
Control problem
i
h
minimize E u ⊤ u + x ⊤ x
Primal Affine
Dual Affine
3
2
1
0
−1
−2
−3
−4
−6
−4
−2
0
2
First Component of x1
4
6
Capacity Expansion in Power Systems
Multiple time scales:
Decades
construct
power plants &
transmission lines
Time to delivery
Years
Days
schedule
inflexible plants
Hours
schedule
flexible plants
Milliseconds
power flows
Capacity Expansion in Power Systems
Four-stage stochastic program:
Objective: minimize investment costs + expected operating costs
over next 50 years
Decades
Time to delivery
Years
Days
investment
decisions
Hours
Milliseconds
operating
decisions
(long−lead)
operating
decisions
(short−lead)
power flows
forecasts of:
fuel prices
demand, wind,
irradiation
realisations of:
fuel prices
demand, wind,
irradiation
line failures
Existing Infrastructure
Power system: 265 buses, 429 transmission lines
Extension Options
Possible additions: 217 power plants, 81 transmission lines
Existing Power Plants
Candidate Power Plants
Optimisation Model
minimize
P
cn un +
n∈Nc
subject to
P
dm vm + E
m∈Mc
gn Fl -measurable
gn Fs -measurable
fm F-measurable
un ∈ {0, 1}, vm ∈ {0, 1}
un = 1
vm = 1
0 ≤ gn ≤ g n un
gn ≤ ζn
|fmP
| ≤ ϕm f m vmP
gn −
fm +
n∈N(k )
m∈M− (k )
P
γn gn
n∈N
P
!
fm ≥ δk
m∈M+ (k )
∀n ∈ Nl
∀n ∈ Ns
∀m ∈ M
∀n ∈ N, ∀m ∈ M
∀n ∈ Ne
∀m ∈ Me
∀n ∈ N
∀n ∈ Nr
∀m ∈ M
∀k ∈ K
9
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
=
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
;
P-a.s.
Results
Solution time: 2.5 hours
Bibliography
1 A. Ben-Tal, S. Boyd, E. Guslitzer, A. Nemirovski. Extending the scope of robust
optimization: Comprehensive robust counterparts of uncertain problems.
Mathematical Programming 107(1–2):63–89, 2006.
2 A. Ben-Tal, A. Goryashko, E. Guslitzer, A. Nemirovski. Adjustable robust solutions
of uncertain linear programs. Mathematical Programming 99(2):351–376, 2004.
3 A. Georghiou, W. Wiesemann, D. Kuhn. Generalized decision rule
approximations for stochastic programming via liftings. Optimization Online,
2010.
4 J. Goh, M. Sim. Distributionally robust optimization and its tractable
approximations. Operations Research 58(4):902–917, 2010.
5 P. Goulart, E. Kerrigan. Output feedback receding horizon control of constrained
systems. International Journal of Control 80(1):8–20, 2007.
6 M. Hadjiyiannis, P. Goulart, D. Kuhn. An efficient method to estimate the
suboptimality of affine controllers. IEEE Transactions on Automatic Control, in
press, 2011.
7 D. Kuhn, W. Wiesemann, A. Georghiou. Primal and dual linear decision rules in
stochastic and robust optimization. Mathematical Programming, in press, 2010.