Optimal Decision Rules for Adjustable Robust Optimization via
the lens of Fourier-Motzkin Elimination
Jianzhe Zhen
1
Dick den Hertog
1
2
1
Melvyn Sim
2
Tilburg University
National University of Singapore
CMS, Bergamo
May 31, 2017
1 / 17
Overview
1
Fourier-Motzkin elimination (FME) approach for adjustable robust
optimization (ARO) problems.
2
Characterization of optimal decision rules (ODRs) for the primal and dual
formulations.
3
LP-based procedure to remove redundant constraints for ARO problems.
4
Numerical experiments on the primal and dual formulations.
2 / 17
Problem Definition
A two-stage ARO problem:
min
xPX,y
s.t.
c1 x
Apzqx ` Bypzq ě dpzq
I1 ,N2
yPR
@z P W
pP q
,
N1
1
X Ď RN1 , e.g., X “ RN
` , X “ Z
A and d are affine in z
z P RI1 resides in a general uncertainty set W
B P RM ˆN2 is constant (a.k.a., fixed recourse)
RI1 ,N2 denotes the space of functions from RI1 to RN2 that are bounded
on compact sets.
3 / 17
Definition of Feasible Set
Definition (Feasible set for two-stage ARO problems)
The feasible set of the here-and-now decisions x in pP q is:
!
)
X “ x P X | Dy P RI1 ,N2 : Apzqx ` Bypzq ě dpzq @z P W .
Goal: eliminate ypzq via FME.
4 / 17
Fourier-Motzkin Elimination for Two-stage ARO problems
1
Rewrite each constraint in X in the form: Dy P RI1 ,N2 :
ÿ
ÿ
bi1 y1 pzq ě di pzq´
aij pzqxj ´
bij yj pzq @z P W
jPrN1 s
@i P rM s;
jPrN2 szt1u
if bi1 ‰ 0, divide both sides by bi1 , we have: Dy P RI1 ,N2 :
y1 pzq ě fi pzq ` g 1i pzqx ` h1i y zt1u pzq
y1 pzq ď
0ě
fj pzq ` g 1j pzqx ` h1j y zt1u pzq
fk pzq ` g 1k pzqx ` h1k y zt1u pzq
@z P W
if bi1 ą 0,
@z P W
if bj1 ă 0,
@z P W
if bk1 “ 0.
5 / 17
Fourier-Motzkin Elimination for Two-stage ARO problems
1
Rewrite each constraint in X in the form: Dy P RI1 ,N2 :
ÿ
ÿ
bi1 y1 pzq ě di pzq´
aij pzqxj ´
bij yj pzq @z P W
jPrN1 s
@i P rM s;
jPrN2 szt1u
if bi1 ‰ 0, divide both sides by bi1 , we have: Dy P RI1 ,N2 :
y1 pzq ě fi pzq ` g 1i pzqx ` h1i y zt1u pzq
y1 pzq ď
0ě
2
fj pzq ` g 1j pzqx ` h1j y zt1u pzq
fk pzq ` g 1k pzqx ` h1k y zt1u pzq
@z P W
if bi1 ą 0,
@z P W
if bj1 ă 0,
@z P W
if bk1 “ 0.
Let Xzt1u be the feasible set after y1 is eliminated, and defined by:
Dy zt1u P RI1 ,N2 ´1 :
fj pzq ` g 1j pzqx ` h1j y zt1u pzq ě
fi pzq ` g 1i pzqx ` h1i y zt1u pzq
0 ě fk pzq `
Complexity: OpM 2
N2
g 1k pzqx
`
h1k y zt1u pzq
@z P W
if bj1 ă 0 and bi1 ą 0,
@z P W
if bk1 “ 0.
q.
5 / 17
Simple Example — 1
Example (Eliminate y1 via FME)
We eliminate y1 in the following linear inequalities via FME:
y1 pzq ´ z 1 x ď 0
@z P W
y1 pzq ´ y2 pzq ď 0
@z P W
´y1 pzq ` x3 ` 2y2 pzq ď 0
@z P W
´y2 pzq ď z2
Step 1.
@z P W.
Step 2.
1
y1 pzq ď z x
@z P W
x3 ` 2y2 pzq ď z 1 x
@z P W
y1 pzq ď y2 pzq
@z P W
x3 ` 2y2 pzq ď y2 pzq
@z P W
x3 ` 2y2 pzq ď y1 pzq
@z P W
´y2 pzq ď z2
@z P W.
´y2 pzq ď z2
@z P W.
6 / 17
Simple Example — 2
Example (Eliminate y2 via FME)
We now also eliminate y2 :
x3 ` 2y2 pzq ď z 1 x
x3 ` y2 pzq ď 0
´y2 pzq ď z2
Step 1.
1
y2 pzq ď pz 1 x ´ x3 q
2
y2 pzq ď ´x3
´z2 ď y2 pzq
@z P W
@z P W
@z P W.
Step 2.
@z P W
@z P W
´2z2 ď z 1 x ´ x3
´z2 ď ´x3
@z P W
@z P W.
@z P W.
Linear decision rule is optimal for y2 pzq.
7 / 17
Optimal Decision Rules — General
Theorem
There exist ODRs that are piecewise affine functions for y in pP q.
This theorem holds for Problem pP q with uncertain parameters that:
appear on both sides of the constraints
reside in a general uncertainty set
8 / 17
Optimal Decision Rules — General
Theorem
There exist ODRs that are piecewise affine functions for y in pP q.
This theorem holds for Problem pP q with uncertain parameters that:
appear on both sides of the constraints
reside in a general uncertainty set
which
generalizes the existing result of [Bemporad et al.(2003)]
stimulates a generalization of the recent methods in:
[Bertsimas and Georghiou (2015)]
[Ben-Tal et al.(2016)].
8 / 17
Dual Feasible Set [Bertsimas and de Ruiter (2016)]
Suppose pP q has a polyhedral uncertainty set
!
)
Wpoly “ z P RI1 | Dv P RI2 : P 1 z ď ρ .
Remember:
!
)
X “ x P X | Dy P RI1 ,N2 : Apzqx ` Bypzq ě dpzq @z P Wpoly .
9 / 17
Dual Feasible Set [Bertsimas and de Ruiter (2016)]
Suppose pP q has a polyhedral uncertainty set
!
)
Wpoly “ z P RI1 | Dv P RI2 : P 1 z ď ρ .
Remember:
!
)
X “ x P X | Dy P RI1 ,N2 : Apzqx ` Bypzq ě dpzq @z P Wpoly .
The dual feasible set:
$
ω 1 pA0 x ´ d0 q ě ρ1 λpωq
&
D
X “ x P X Dλ P RM,K : p1i λpωq “ pdi ´ Ai xq1 ω
%
λpωq ě 0
where
,
@ω P U
.
@ω P U, @i P rI1 s
@ω P U
!
)
1
Upoly “ ω P RM
` | B ω “ 0 .
Theorem ([Bertsimas and de Ruiter (2016)])
X “ X D.
9 / 17
Optimal Decision Rules — Simplex
Suppose
!
Wsimplex “ z P RI`1
)
11 z ď 1 .
Theorem
There exist LDRs that are optimal for y in (P) with Wsimplex .
10 / 17
Optimal Decision Rules — Simplex
Suppose
!
Wsimplex “ z P RI`1
)
11 z ď 1 .
Theorem
There exist LDRs that are optimal for y in (P) with Wsimplex .
“Proof”:
"
D
X “ xPX
M,1
Dλ P R
ω 1 pA0 x ´ d0 q ě λpωq
`
˘`
:
λpωq ě pdi ´ Ai xq1 ω
@ω P U
@ω P U, @i P rI1 s
*
.
This coincides with the recent finding in [Ben-Ameur et al.(2016),
Corollary 2], and generalizes [Bertsimas and Goyal (2012), Theorem 1].
Theorem ([Z. and den Hertog (2015)])
There exist polynomials of (at most) degree N1 and linear in each zi , i P rI1 s,
that are optimal decision rules for y in pP q.
10 / 17
Optimal Decision Rules — Box
Suppose
!
Wbox “ z P RI1
)
lďzďu .
Theorem
`
˘`
are ODRs for the
The convex two-piecewise affine functions pdi ´ Ai xq1 ω
adjustable variables λi , i P rI1 s, in (D).
“Proof”:
"
XD “ x P X
Dλ P RM,I1 :
¨ ¨ ¨ ě pu ´ lq1 λpωq
`
˘`
λi pωq ě pdi ´ Ai xq1 ω
@ω P U
@ω P U, @i P rI1 s
*
.
11 / 17
Redundant Constraint Identification
Suppose pP q has only rhs uncertainty, i.e.,
!
)
X “ x P X | Dy P RI1 ,N2 : Ax ` Bypzq ě dpzq @z P W .
Theorem
The l-th constraint is redundant in X if the following optimization problem
Zl: “ min
a1l x ` b1l y ´ dl pzq
s.t.
a1i x ` b1i y ě di pzq
x,y,z
N1
z P W, x P R
,y P R
has a nonnegative optimal objective value, i.e.,
pě 0 ñ Redundant!q
@i P rM sztlu
N2
.
Zl:
ě 0.
12 / 17
Inner Approximation via Decision Rules
Impose decision rules F I1 ,N2 ´|S| Ď RI1 ,N2 ´|S| on y zS :
!
)
XpzS “ x P X|Dy zS P F I1 ,N2 ´|S| : Gpzqx ` Hy zS pzq ě f pzq @z P W
!
)
“ x P X|Dy zS P F I1 ,N2 ´|S| , y S P RI1 ,|S| : Apzqx ` Bypzq ě dpzq @z P W .
Theorem
Xp Ď XpzS Ď XzrN2 s “ X , for S Ď rN2 s.
13 / 17
Numerical Example — A Two-stage RO Problem
Example (Lot-sizing on a network)
min
xPX,yij ,τ
s.t.
c1 x ` τ
ÿ
tij yij pzq ď τ
@z P W
i,jPrN s
xi `
ÿ
yji pzq ´
jPrN s
ÿ
yij pzq ě zi
@z P W,
i P rN s
jPrN s
yij pzq ě 0, yij P RN,1
@z P W,
i, j P rN s.
ci : storage cost of store i
x P X: stock allocations in X
z P W: the uncertain demands reside in a budget uncertainty set W
tij : transportation cost per unit from store i to store j
yij : units of good transported from store i to store j.
14 / 17
Numerical Results — Primal v.s. Dual
Dimensions of y and λ are N 2 and N ` 1, respectively.
15 / 17
Numerical Results — Primal v.s. Dual
Dimensions of y and λ are N 2 and N ` 1, respectively.
Table: Lot-sizing on a Network for N P t5, 10u.
N=5
P
D
N=10
P
D
#Elim.
RCI
Gap%
TTime
#Elim.
FME
Gap%
Time
#Elim.
RCI
Gap%
TTime
#Elim.
FME
Gap%
Time
1
30
3.3
0.1
1
11
3.3
0.1
1
110
6.0
0.1
1
21
6
0.1
11
37
2.9
12.9
2
10
2.8
0.1
12
100
6.0
14.8
5
43
4.6
0.2
15
75
1.7
58.3
3
13
2.3
0.1
17
133
6.0
52.6
7
135
2.6
0.4
19
101
0.7
223.2
4
19
1
0.1
19
180
5.9
100.7
8
261
1.8
0.9
22
116
0.1
394.3
5
33
0.2
0.1
21
276
5.8
987.7
9
515
0.8
1.9
25
127
0
550.3
6
272
0
0.1
22
343
5.7
1639.8
10
1025
0.2
4.6
–
–
–
–
–
–
100
*
*
*
11
149424
*
*
15 / 17
Numerical Results — Dual
Table: Lot-sizing on a Network for N P t15, 20, 30u.
N=15
D
N=20
D
N=30
D
#Elim.
FME
Red.%
Time
FME
Red.%
Time
FME
Red.%
Time
Red.%“
1
31
0
0.3
41
0
0.6
61
0
2.2
2
31
-0.1
0.3
41
-0.1
0.5
61
0
2.2
sol.´LDR
LDR
3
33
-0.4
0.3
43
-0.2
0.6
63
-0.1
2.6
4
39
-0.5
0.4
49
-0.3
0.8
69
-0.2
3.4
5
53
-0.7
0.5
63
-0.4
1.3
83
-0.2
5.8
6
83
-0.9
1
93
-0.5
2.5
113
-0.3
15.0
7
145
-1.6
1.5
155
-0.9
3.5
175
-0.4
54.6
8
271
-1.9
3.6
281
-1
10.1
301
-0.5
55.7
9
525
-2.2
8.9
535
-1.2
36.1
555
-0.6
214.5
10
1035
-2.8
25.2
1045
-1.5
67.7
1065
-0.8
522.2
11
2057
-3.4
125.1
2067
-1.8
206.2
2087
*
*
ˆ 100%.
16 / 17
Conclusions & Future Research
FME enables us to
gain new insights on ODRs
successively improve the solutions obtained from the existing methods.
FME cannot be applied to
non-fixed recourse Bpzq
integer recourse decisions ypzq P ZN2 .
Future research:
characterize ODRs for multistage problems
adapt the existing procedures to improve the computability of FME
approach.
Zhen, J., D. den Hertog, M. Sim. (2016) Adjustable Robust Optimization via
Fourier-Motzkin Elimination. Available on Optimization Online.
17 / 17
Bemporad, A., F. Borrelli, M. Morari (2003) Min-max control of
constrained uncertain discrete-time linear systems. IEEE Transactions
Automatic Control 48(9):1600–1606.
Ben-Ameur, W., G. Wang, A. Ouorou, M. Zotkiewicz (2016) Multipolar
Robust Optimization. Available at: arxiv.org/pdf/1604.01813.pdf.
Ben-Tal, A., O. El Housni, V. Goyal (2016) A tractable approach for
designing piecewise affine policies in dynamic robust optimization.
Available at: optimization-online.org/DB_FILE/2016/07/5557.pdf.
Bertsimas, D., F. de Ruiter (2016) Duality in two-stage adaptive linear
optimization: faster computation and stronger bounds. INFORMS Journal
on Computing 28(3):500–511.
Bertsimas, D., V. Goyal (2012) On the power and limitations of affine
policies in two-stage adaptive optimization. Mathematical Programming
Ser. A 134(2):491–531.
Bertsimas, D., A. Georghiou (2015) Design of near optimal decision rules
in multistage adaptive mixed-integer optimization. Operations Research
63(3):610–627.
17 / 17
Bertsimas, D., M. Sim, M. Zhang (2016) Distributionally adaptive
optimization. Available at:
www.optimization-online.org/DB_FILE/2016/03/5353.pdf.
Bertsimas, D. J. Tsitsiklis (1997) Introduction to Linear Optimization.
Athena Scientific.
Caron, R., J. McDonald, C. Ponic (1989) A degenerate extreme point
strategy for the classification of linear constraints as redundant or
necessary. Journal of Optimization Theory and Applications 62:225–237.
J. Fourier (1826) Reported in: Analyse des travaux de l’Académie Royale
des Sciences, pendant l’année 1824, Partie mathématique. Histoire de
l’Academie Royale des Sciences de l’Institut de France 7:47–55.
T. Motzkin (1936) Beiträge zur Theorie der linearen Ungleichungen,
University Basel Dissertation. Jerusalem, Israel.
Zhen, J. and D. den Hertog (2015) Computing the maximum volume
inscribed ellipsoid of a polytopic projection. INFORMS Journal on
Computing, to appear (available at:
optimization-online.org/DB_FILE/2015/01/4749.pdf).
17 / 17