Tutorial
Integer Programming for
Constraint Programmers
Ambros Gleixner and Stefan Heinz
Zuse Institute Berlin (ZIB)
Chris Beck, Timo Berthold, and Kati Wolter
DFG Research Center MATHEON
Mathematics for key technologies
CP 2011, 14/Sep/2011
ZIB – Zuse Institute Berlin
. Non-university research institute of the state of Berlin (Germany)
. Research Units:
I
Numerical Mathematics
I
I
I
Discrete Mathematics
I
I
I
Numerical analysis and modeling
Visualization and data analysis
Optimization
Scientific Information
Computer Science
I
I
Parallel and Distributed Systems
Supercomputing
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
http://www.zib.de
1 / 62
CPAIOR 2011
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
2 / 62
CPAIOR 2012
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
3 / 62
Outline
Integer Programming for Constraint Programmers
1
Introduction
2
Linear programming
3
Integer (linear) programming
4
Summary
5
Discussion
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
4 / 62
Problem description
Definition
The steel mill slab problem consists of assigning colored, sized orders
to slabs of certain different capacities such that the total loss is
minimized and at most two different colors are present in each slab.
3
2
1
Orders
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
5 / 62
Problem description
Definition
The steel mill slab problem consists of assigning colored, sized orders
to slabs of certain different capacities such that the total loss is
minimized and at most two different colors are present in each slab.
5
4
3
...
+
2
1
Orders
3
2
...
1
Slabs
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
5 / 62
Problem description
Definition
The steel mill slab problem consists of assigning colored, sized orders
to slabs of certain different capacities such that the total loss is
minimized and at most two different colors are present in each slab.
5
4
3
...
+
2
1
Orders
3
→
2
...
1
Slabs
0 2 0 0 0
Assignment
. Problem number 38 of the CSPLib (http://www.csplib.org/)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
5 / 62
(A) Constraint programming formulation
Given
. K set of possible capacities for the slabs
. C set of colors
. O set of orders, |O| = n
I
I
si size of order i
ci color of order i
Binary variables
. yij = 1 if order i is assigned to slab j
. zcj = 1 if color c is used in slab j
Observation
. We need at most n slabs
. Let S be the set of slabs
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
6 / 62
Leftover array
. Array storing the leftover depending on the load
L[i] = argmin{k − i | k ∈ K and k ≥ i}
for i = 0, . . . , Kmax
. Kmax := max{k | k ∈ K}
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
7 / 62
Leftover array
. Array storing the leftover depending on the load
L[i] = argmin{k − i | k ∈ K and k ≥ i}
for i = 0, . . . , Kmax
. Kmax := max{k | k ∈ K}
Example
1
L=
2
3
4
5
0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
7 / 62
Leftover array
. Array storing the leftover depending on the load
L[i] = argmin{k − i | k ∈ K and k ≥ i}
for i = 0, . . . , Kmax
. Kmax := max{k | k ∈ K}
Example
1
L=
0
2
3
4
5
0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
7 / 62
Leftover array
. Array storing the leftover depending on the load
L[i] = argmin{k − i | k ∈ K and k ≥ i}
for i = 0, . . . , Kmax
. Kmax := max{k | k ∈ K}
Example
L=
0
1
2
0
3
3
4
5
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
7 / 62
Leftover array
. Array storing the leftover depending on the load
L[i] = argmin{k − i | k ∈ K and k ≥ i}
for i = 0, . . . , Kmax
. Kmax := max{k | k ∈ K}
Example
L=
0
1
2
3
0
3
2
4
5
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
7 / 62
Leftover array
. Array storing the leftover depending on the load
L[i] = argmin{k − i | k ∈ K and k ≥ i}
for i = 0, . . . , Kmax
. Kmax := max{k | k ∈ K}
Example
L=
0
1
2
3
4
0
3
2
1
5
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
7 / 62
Leftover array
. Array storing the leftover depending on the load
L[i] = argmin{k − i | k ∈ K and k ≥ i}
for i = 0, . . . , Kmax
. Kmax := max{k | k ∈ K}
Example
L=
0
1
2
3
4
5
0
3
2
1
0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
7 / 62
(A) Constraint programming formulation
min
subject to
X X
L
si yij
j∈S
i∈O
X
yij = 1
∀i ∈ O
si yij ≤ Kmax
∀j ∈ S
j∈S
X
i∈O
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
yij , zcj ∈ {0, 1}
∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
8 / 62
(A) Constraint programming formulation
min
subject to
X X
L
si yij
j∈S
i∈O
X
yij = 1
∀i ∈ O
si yij ≤ Kmax
∀j ∈ S
Assignment
j∈S
X
i∈O
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
yij , zcj ∈ {0, 1}
∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
8 / 62
(A) Constraint programming formulation
min
subject to
X X
L
si yij
j∈S
i∈O
X
yij = 1
∀i ∈ O
si yij ≤ Kmax
∀j ∈ S
j∈S
X
Capacity
i∈O
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
yij , zcj ∈ {0, 1}
∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
8 / 62
(A) Constraint programming formulation
min
subject to
X X
L
si yij
j∈S
i∈O
X
yij = 1
∀i ∈ O
si yij ≤ Kmax
∀j ∈ S
j∈S
X
i∈O
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
Coloring
∀j ∈ S
c∈C
yij , zcj ∈ {0, 1}
∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
8 / 62
(A) Constraint programming formulation
min
subject to
X X
L
si yij
Leftover
j∈S
i∈O
X
yij = 1
∀i ∈ O
si yij ≤ Kmax
∀j ∈ S
j∈S
X
i∈O
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
yij , zcj ∈ {0, 1}
∀i ∈ O ∀c ∈ C ∀j ∈ S
. element constraint
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
8 / 62
(An) Integer programming formulation
Given
. K set of possible capacities for the slabs
. C set of colors
. O set of orders, |O| = n
I
I
si size of order i
ci color of order i
Binary variables
. xkj = 1 if capacity k is assigned to slab j
. yij = 1 if order i is assigned to slab j
. zcj = 1 if color c is used in slab j
Observation
. We need at most n slabs
. Let S be the set of slabs
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
9 / 62
(An) Integer programming formulation
min
XX
k xkj −
j∈S k∈K
subject to
X
X
si
i∈O
xkj = 1
∀j ∈ S
yij = 1
∀i ∈ O
k∈K
X
j∈S
X
si yij ≤
i∈O
X
k xkj
∀j ∈ S
k∈K
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
xkj , yij , zcj ∈ {0, 1}
∀k ∈ K ∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
10 / 62
(An) Integer programming formulation
min
XX
k xkj −
j∈S k∈K
subject to
X
X
si
i∈O
xkj = 1
∀j ∈ S
yij = 1
∀i ∈ O
Assignment
k∈K
X
j∈S
X
si yij ≤
i∈O
X
k xkj
∀j ∈ S
k∈K
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
xkj , yij , zcj ∈ {0, 1}
∀k ∈ K ∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
10 / 62
(An) Integer programming formulation
min
XX
k xkj −
j∈S k∈K
subject to
X
X
si
i∈O
xkj = 1
∀j ∈ S
yij = 1
∀i ∈ O
k∈K
X
j∈S
X
si yij ≤
i∈O
X
k xkj
∀j ∈ S
Capacity
k∈K
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
xkj , yij , zcj ∈ {0, 1}
∀k ∈ K ∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
10 / 62
(An) Integer programming formulation
min
XX
k xkj −
j∈S k∈K
subject to
X
X
si
i∈O
xkj = 1
∀j ∈ S
yij = 1
∀i ∈ O
k∈K
X
j∈S
X
si yij ≤
i∈O
X
k xkj
∀j ∈ S
k∈K
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
Coloring
∀j ∈ S
c∈C
xkj , yij , zcj ∈ {0, 1}
∀k ∈ K ∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
10 / 62
(An) Integer programming formulation
min
XX
k xkj −
j∈S k∈K
subject to
X
X
si
Leftover
i∈O
xkj = 1
∀j ∈ S
yij = 1
∀i ∈ O
k∈K
X
j∈S
X
si yij ≤
i∈O
X
k xkj
∀j ∈ S
k∈K
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
xkj , yij , zcj ∈ {0, 1}
∀k ∈ K ∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
10 / 62
####################
#
# data p a r s i n g
#
####################
# number o f c a p a c i t i e s
param n c a p a c i t y := r e a d DATAFILE a s ”1 n” u s e 1 ;
do p r i n t ” n c a p a c i t y = ” , n c a p a c i t y ;
# number o f c o l o r s
param n c o l o r s := r e a d DATAFILE a s ”1 n” s k i p 1 u s e 1 ;
do p r i n t ” n c o l o r s = ” , n c o l o r s ;
# number o f o r d e r n s
param n o r d e r s := r e a d DATAFILE a s ”1 n” s k i p 2 u s e 1 ;
do p r i n t ” n o r d e r s = ” , n o r d e r s ;
# get the capacity array
s e t tmp := { r e a d DATAFILE a s ”<n+>” u s e 1 } ;
s e t c a p a c i t i e s := i f c a r d ( tmp ) == n c a p a c i t y t h e n tmp u n i o n {0} e l s e tmp u n i o n {0} \ { n c a p a c i t y } end ;
do c h e c k n c a p a c i t y == c a r d ( c a p a c i t i e s ) −1;
do p r i n t c a p a c i t i e s ;
# index set for orders
s e t I := { 1 . . n o r d e r s } ;
# index set for colors
s e t C := { 1 . . n c o l o r s } ;
# get orders
s e t o r d e r s [< i > i n I ] := { r e a d DATAFILE a s ”<1n , 2 n>” s k i p 2+ i u s e 1 } ;
#do f o r a l l <i > i n I do p r i n t o r d e r s [ i ] ;
####################
#
# decision variables
#
####################
# s l a b v a r i a b l e s w h ich c a p a c i t i e s i s a s s i g n e d t o whi c h s l a b
var x [ I ∗ c a p a c i t i e s ] binary ;
# w h i ch o r d e r i s a s s i g n e d t o wh ic h s l a b
var y [ I ∗ I ] binary ;
# whi c h c o l o r i s u s e d by wh ich s l a b
var z [C ∗ I ] ;
####################
#
# objective function
#
####################
minimize l e f t o v e r :
sum <s , c> i n I ∗ c a p a c i t i e s : c ∗ x [ s , c ] − sum <o , s> i n I ∗ I : o r d ( o r d e r s [ o ] , 1 , 1 ) ∗ y [ o , s ] ;
####################
#
# constraints
#
####################
# e a c h s l a b g e t s e x a c t l y one c a p a c i t i e s
subto oneCapacity :
f o r a l l <s> i n I : sum <c> i n c a p a c i t i e s : x [ s , c ] == 1 ;
# e a c h o r d e r i s a s s i g n e d t o e x a c t l y one s l a b
subto oneSlab :
f o r a l l <o> i n I : sum <s> i n I : y [ o , s ] == 1 ;
# each s l a b i s not over loaded
subto Capacity :
f o r a l l <s> i n I : sum <c> i n c a p a c i t i e s : c ∗ x [ s , c ] − sum <o> i n I : o r d ( o r d e r s [ o ] , 1 , 1 ) ∗ y [ o , s ] >= 0 ;
# color linking constraints
subto l i n k i n g :
f o r a l l <o , s> i n I ∗ I : y [ o , s ] − z [ o r d ( o r d e r s [ o ] , 1 , 2 ) , s ] <= 0 ;
# color linking constraints
subto Color :
f o r a l l <s> i n I : sum <c> i n C : z [ c , s ] <= 2 ;
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
11 / 62
Linear programming relaxation
Integer program
Linear program relaxation
→
min{IP} ≥ min{LP}
max{IP} ≤ max{LP}
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
12 / 62
Linear programming relaxation
. Omit integrality condition
X
XX
k xkj −
si
min
i∈O
j∈S k∈K
subject to
X
xkj = 1
∀j ∈ S
yij = 1
∀i ∈ O
k∈K
X
j∈S
X
si yij ≤
i∈O
X
k xkj
∀j ∈ S
k∈K
yij ≤ zci j
X
zcj ≤ 2
∀i ∈ O ∀j ∈ S
∀j ∈ S
c∈C
xkj , yij , zcj ∈ [0, 1]
∀k ∈ K ∀i ∈ O ∀c ∈ C ∀j ∈ S
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
13 / 62
Root node solution
5
4
3
...
+
2
1
Orders
3
→
2
...
1
Slabs
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
0 0 0 0 0
Assignment
14 / 62
Root node solution
Capacities
x51 = 0.8
x01 = 0.2
x52 = 0.6
x02 = 0.4
x53 = 0.4
x03 = 0.6
Assignments
y11 = 1.0
y23 = 1.0
y31 = 1.0
y41 = 1.0
y52 = 1.0
x04 = 1.0
Colors
z11
z23
z31
z32
z33
z34
z35
z42
= 1.0
= 1.0
= 1.0
= 1.0
= 1.0
= 1.0
= 1.0
= 1.0
x06 = 1.0
. Remaining decission variables are zero
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
15 / 62
Root node solution
Capacities
x51 = 0.8
x01 = 0.2
x52 = 0.6
x02 = 0.4
x53 = 0.4
x03 = 0.6
x04 = 1.0
Assignments
y11 = 1.0
y23 = 1.0
y31 = 1.0
y41 = 1.0
y52 = 1.0
Colors
z11
z23
z31
z32
z33
z34
z35
z42
= 1.0
= 1.0
= 1.0
= 1.0
= 1.0
= 1.0
= 1.0
= 1.0
x06 = 1.0
. Remaining decission variables are zero
. Observation: Independently of the problem instance the root LP
value for this model is always zero.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
15 / 62
node
1
left
0
depth
0
frac
6
curdualbound
0.000000e+00
dualbound
0.000000e+00
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
primalbound
--
16 / 62
Search tree
1
2
3
4
#1
0
#2
#10
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#25
≥1
#5
#6
#22
≥1
0
#17
≥1
#28
≥1
#7
#29
0
#8
≥1
1
#9
Node 1
Lower bound: 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
17 / 62
Search tree
1
2
3
4
#1
0
#2
#10
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#25
≥1
#5
#6
#22
≥1
0
#17
≥1
#28
≥1
#7
#29
0
#8
≥1
1
#9
Node 2
Lower bound: 0
x52
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
17 / 62
Search tree
1
2
3
4
#1
0
#2
#10
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#25
≥1
#5
#6
#22
≥1
0
#17
≥1
#28
≥1
#7
#29
0
#8
≥1
1
#9
Node 3
Lower bound: 1
x52 , x53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
17 / 62
Search tree
1
2
3
4
#1
0
#2
#10
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#25
≥1
#5
#6
#22
≥1
0
#17
≥1
#28
≥1
#7
#29
0
#8
≥1
1
#9
Node 4
Lower bound: 0
x52 , x̄53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
17 / 62
Search tree
1
2
3
4
#1
0
#2
#10
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#25
≥1
#5
#6
#22
≥1
0
#17
≥1
#28
≥1
#7
#29
0
#8
≥1
1
#9
Node 5
Lower bound: 1
x52 , x̄53 , x51
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
17 / 62
node
1
1
2
3
4
*
5
left
0
2
3
4
5
2
depth
0
0
1
2
2
3
frac
6
6
4
6
2
-
curdualbound
0.000000e+00
0.000000e+00
0.000000e+00
1.000000e+00
0.000000e+00
1.000000e+00
dualbound
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
primalbound
-----1.000000e+00
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
18 / 62
Solution
5
4
3
...
+
2
1
Orders
3
→
2
...
1
Slabs
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
1 0 0 0 0
Assignment
19 / 62
Search tree
1
2
3
4
#1
0
#2
#10
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
0
#8
≥1
#9
Node 6
Lower bound: 0
x52 , x̄53 , x̄51
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
20 / 62
Search tree
1
2
3
4
#1
0
#2
#10
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
0
#8
≥1
#9
Node 7
Lower bound: 0
x52 , x̄53 , x̄51 , x̄55
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
20 / 62
Search tree
1
2
3
4
#1
0
#2
#10
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
0
#8
≥1
#9
Node 8
Lower bound: ≥ 1
x52 , x̄53 , x̄51 , x̄55 , x̄54
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
20 / 62
node
1
1
2
3
4
*
5
6
7
8
left
0
2
3
4
5
2
3
4
3
depth
0
0
1
2
2
3
3
4
5
frac
6
6
4
6
2
2
2
-
curdualbound
0.000000e+00
0.000000e+00
0.000000e+00
1.000000e+00
0.000000e+00
1.000000e+00
0.000000e+00
0.000000e+00
--
dualbound
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
0.000000e+00
primalbound
-----1.000000e+00
1.000000e+00
1.000000e+00
1.000000e+00
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
21 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 9
Lower bound: ≥ 1
x52 , x̄53 , x̄51 , x̄55 , x54
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 10
Lower bound: 0
x̄52
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 11
Lower bound: 0
x̄52 , x51
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 12
Lower bound: 0
x̄52 , x̄51
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 13
Lower bound: 0
x̄52 , x51 , x̄53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 14
Lower bound: 0
x̄52 , x51 , x̄53 , x̄54
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 15
Lower bound: 0
x̄52 , x̄51 , x̄54
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 16
Lower
Lower bound:
bound: ≥
≥10
1
x̄52 , x̄51 , xx̄54 , xx̄55
53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 17
Lower bound: ≥ 1
x̄52 , x51 , x̄53 , x54 , x̄55
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 18
Lower bound: 0
x̄52 , x̄51 , x54
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 19
Lower bound: 0
x̄52 , x̄51 , x54 , x̄53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 20
Lower bound: ≥ 1
x̄52 , x̄51 , x54 , x̄53 , x̄55
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 21
Lower bound: ≥ 1
x̄52 , x̄51 , x54 , x̄53 , x55
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 22
Lower bound: ≥ 1
x̄52 , x51 , x̄53 , x54
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 23
Lower bound: 0
x̄52 , x̄51 , x̄54 , x55
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 24
Lower bound: ≥ 1
x̄52 , x̄51 , x̄54 , x55 , x̄53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 25
Lower bound: ≥ 1
x̄52 , x51 , x̄53 , x54 , x55
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 26
Lower bound: ≥ 1
x̄52 , x̄51 , x54 , x53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 27
Lower bound: ≥ 1
x̄52 , x̄51 , x̄54 , x55 , x53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 28
Lower bound: ≥ 1
x̄52 , x51 , x53
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
Search tree
1
2
3
4
#1
0
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Node 29
Lower bound: ≥ 1
x52 , x̄53 , x̄51 , x55
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
22 / 62
node left depth frac curdualbound
dualbound
primalbound
1
0
0
6 0.000000e+00 0.000000e+00
-1
2
0
6 0.000000e+00 0.000000e+00
-2
3
1
4 0.000000e+00 0.000000e+00
-3
4
2
6 1.000000e+00 0.000000e+00
-4
5
2
2 0.000000e+00 0.000000e+00
-*
5
2
3
- 1.000000e+00 0.000000e+00 1.000000e+00
6
3
3
2 0.000000e+00 0.000000e+00 1.000000e+00
7
4
4
2 0.000000e+00 0.000000e+00 1.000000e+00
8
3
5
-- 0.000000e+00 1.000000e+00
9
2
5
-- 0.000000e+00 1.000000e+00
10
3
1
2 4.440892e-16 0.000000e+00 1.000000e+00
11
4
2
2 4.440892e-16 0.000000e+00 1.000000e+00
12
5
2
4 4.440892e-16 0.000000e+00 1.000000e+00
13
6
3
4 1.776357e-15 0.000000e+00 1.000000e+00
14
7
4
4 1.776357e-15 0.000000e+00 1.000000e+00
node left depth frac curdualbound
dualbound
primalbound
15
8
3
4 1.184238e-15 0.000000e+00 1.000000e+00
16
7
4
-- 0.000000e+00 1.000000e+00
17
6
5
-- 0.000000e+00 1.000000e+00
18
7
3
2 4.440892e-16 0.000000e+00 1.000000e+00
19
8
4
4 8.881784e-16 0.000000e+00 1.000000e+00
20
7
5
-- 0.000000e+00 1.000000e+00
21
6
5
-- 0.000000e+00 1.000000e+00
22
5
4
-- 0.000000e+00 1.000000e+00
23
6
4
2 1.184238e-15 0.000000e+00 1.000000e+00
24
5
5
-- 0.000000e+00 1.000000e+00
25
4
5
-- 0.000000e+00 1.000000e+00
26
3
4
-- 0.000000e+00 1.000000e+00
27
2
5
-- 0.000000e+00 1.000000e+00
28
1
3
-- 0.000000e+00 1.000000e+00
Ambros Gleixner
Programming-for Constraint
Programmers1.000000e+00
29 and Stefan
0 Heinz 4(ZIB) – Integer
1.000000e+00
23 / 62
Search tree
1
2
3
4
#1
Statistic with LP
0
Total nodes: 29
Max depth: 5
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
#9
≥1
24 / 62
Search tree
1
2
3
4
#1
Statistic with LP
0
Total nodes: 29
Max depth: 5
#10
#2
0
0
#12
#11
#4
#3
0
0
0
1
#15
#18
#13
0
0
0
#16
≥1
#24
≥1
#23
#19
0
0
#27
≥1
#20
≥1
#26
≥1
#21
≥1
#14
#22
≥1
0
#17
≥1
#28
≥1
#25
≥1
#6
#5
0
1
#7
#29
≥1
0
#8
≥1
#9
≥1
Statistic without LP
Total nodes: 2154
Max depth: 21
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
24 / 62
Summary
. A solution of a linear relaxation gives a proven dual bound
I
I
in case of minimization it is a lower bound
in case of maximization it is a upper bound
. Linear relaxation is a natural relaxation for an integer program
I
omitting integrality conditions
. Linear relaxation gives a global view w.r.t. all linear constraints
. Linear relaxation guides the search via fractional variables
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
25 / 62
Summary
. A solution of a linear relaxation gives a proven dual bound
I
I
in case of minimization it is a lower bound
in case of maximization it is a upper bound
. Linear relaxation is a natural relaxation for an integer program
I
omitting integrality conditions
. Linear relaxation gives a global view w.r.t. all linear constraints
. Linear relaxation guides the search via fractional variables
Coming up:
. How can a linear program be solved?
. How can an integer program be solved?
. For what is the linear programming relaxation used within an
integer programming solver?
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
25 / 62
Outline
Integer Programming for Constraint Programmers
1
Introduction
2
Linear programming
3
Integer (linear) programming
4
Summary
5
Discussion
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
26 / 62
General linear programs (LPs)
Continuous variables:
Linear constraints:
Linear objective:
xi ≥ 0, lbi ≤ xi ≤ ubi , xi free
a1 x1 + . . . + an xn S b
c1 x1 + . . . + cn xn
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
(columns)
(rows)
(→ min/max)
27 / 62
General linear programs (LPs)
Continuous variables:
Linear constraints:
xi ≥ 0, lbi ≤ xi ≤ ubi , xi free
(columns)
a1 x1 + . . . + an xn S b
Linear objective:
Computational standard form:
c1 x1 + . . . + cn xn
(rows)
(→ min/max)
min c 0 x | Ax = b, x ≥ 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
27 / 62
General linear programs (LPs)
Continuous variables:
xi ≥ 0, lbi ≤ xi ≤ ubi , xi free
Linear constraints:
(columns)
a1 x1 + . . . + an xn S b
Linear objective:
Computational standard form:
c1 x1 + . . . + cn xn
(rows)
(→ min/max)
min c 0 x | Ax = b, x ≥ 0
. feasible region is convex and
polyhedral
. problem may be
I
I
I
unbounded,
infeasible, or
optimal
. always optimal vertex solution
(if an optimal solution exists)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
27 / 62
General linear programs (LPs)
Continuous variables:
xi ≥ 0, lbi ≤ xi ≤ ubi , xi free
Linear constraints:
(columns)
a1 x1 + . . . + an xn S b
Linear objective:
Computational standard form:
c1 x1 + . . . + cn xn
(rows)
(→ min/max)
min c 0 x | Ax = b, x ≥ 0
. feasible region is convex and
polyhedral
. problem may be
I
I
I
unbounded,
infeasible, or
optimal
. always optimal vertex solution
(if an optimal solution exists)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
27 / 62
General linear programs (LPs)
Continuous variables:
xi ≥ 0, lbi ≤ xi ≤ ubi , xi free
Linear constraints:
(columns)
a1 x1 + . . . + an xn S b
Linear objective:
Computational standard form:
c1 x1 + . . . + cn xn
(rows)
(→ min/max)
min c 0 x | Ax = b, x ≥ 0
. feasible region is convex and
polyhedral
. problem may be
I
I
I
unbounded,
infeasible, or
optimal
. always optimal vertex solution
(if an optimal solution exists)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
27 / 62
General linear programs (LPs)
Continuous variables:
xi ≥ 0, lbi ≤ xi ≤ ubi , xi free
Linear constraints:
(columns)
a1 x1 + . . . + an xn S b
Linear objective:
Computational standard form:
c1 x1 + . . . + cn xn
(rows)
(→ min/max)
min c 0 x | Ax = b, x ≥ 0
. feasible region is convex and
polyhedral
. problem may be
I
I
I
unbounded,
infeasible, or
optimal
. always optimal vertex solution
(if an optimal solution exists)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
27 / 62
An incomplete history on linear programming
1827 J. Fourier: Variable elimination algorithm (“Fourier-Motzkin”)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
28 / 62
An incomplete history on linear programming
1827 J. Fourier: Variable elimination algorithm (“Fourier-Motzkin”)
1947 G. Dantzig: Primal Simplex algorithm
1954 C. Lemke and E. Beale: Dual Simplex algorithm
I
I
by far the most used algorithm to solve LPs
worst case exponential running time
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
28 / 62
LP algorithms
simplex algorithm
[Dantzig 1947]
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
29 / 62
An incomplete history on linear programming
1827 J. Fourier: Variable elimination algorithm (“Fourier-Motzkin”)
1947 G. Dantzig: Primal Simplex algorithm
1954 C. Lemke and E. Beale: Dual Simplex algorithm
I
I
by far the most used algorithm to solve LPs
worst case exponential running time
1975 L. Kantorovich and T.C. Koopmans:
Nobel prize for Economics
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
30 / 62
Leonid Kantorovich & Tjalling C. Koopmans
1975: Nobel price in Economic Science
“Optimal allocation of ressources”
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
31 / 62
An incomplete history on linear programming
1827 J. Fourier: Variable elimination algorithm (“Fourier-Motzkin”)
1947 G. Dantzig: Primal Simplex algorithm
1954 C. Lemke and E. Beale: Dual Simplex algorithm
I
I
by far the most used algorithm to solve LPs
worst case exponential running time
1975 L. Kantorovich and T.C. Koopmans:
Nobel prize for Economics
1979 L. Khachiyan: Ellipsoid Method
I
first polynomial time algorithm
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
32 / 62
LP algorithms
simplex algorithm
[Dantzig 1947]
ellipsoid method
[Khachiyan 1979]
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
33 / 62
LP algorithms
simplex algorithm
[Dantzig 1947]
ellipsoid method
[Khachiyan 1979]
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
interior point
[Karmarkar 1984]
33 / 62
An incomplete history on linear programming
1827 J. Fourier: Variable elimination algorithm (“Fourier-Motzkin”)
1947 G. Dantzig: Primal Simplex algorithm
1954 C. Lemke and E. Beale: Dual Simplex algorithm
I
I
by far the most used algorithm to solve LPs
worst case exponential running time
1975 L. Kantorovich and T.C. Koopmans:
Nobel prize for Economics
1979 L. Khachiyan: Ellipsoid Method
I
first polynomial time algorithm
1984 N. Karmarkar: Interior Point Method/Barrier Algorithm
≥ 1987 Primal-Dual Interior Point Algorithms
I
I
basis for state-of-the-art interior point implementations
for single, sparse LPs often faster than simplex
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
34 / 62
A transportation problem
7
1
13
1e
2e
4e
3e
9e
A
9
2
6
3
B
9
3e
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
35 / 62
A transportation problem
min 2xA,1 + 3xA,2 + 9xA,3 +
4xB,1 + 1xB,2 + 3xB,3
7
1
13
s.t.
1e
2e
4e
3e
9e
A
9
2
6
3
B
9
3e
xA,1 + xA,2 + xA,3
xB,1 + xB,2 + xB,3
xA,1 + xB,1
xA,2 + xB,2
xA,3 + xB,3
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
= 13
=9
=7
=9
=6
x ≥0
35 / 62
A transportation problem
min 2xA,1 + 3xA,2 + 9xA,3 +
4xB,1 + 1xB,2 + 3xB,3
7
1
13
s.t.
1e
2e
4e
3e
9e
A
9
2
6
3
B
9
3e
xA,1 + xA,2 + xA,3
xB,1 + xB,2 + xB,3
xA,1 + xB,1
xA,2 + xB,2
xA,3 + xB,3
= 13
=9
=7
=9
=6
x ≥0
Heuristic solution with objective value 53:
A→1=7
A→2=6
B →2=3
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
B →3=6
35 / 62
A transportation problem
min 2xA,1 + 3xA,2 + 9xA,3 +
4xB,1 + 1xB,2 + 3xB,3
7
1
s.t.
6
3
7
9
2
A
6
13
3
B
6
xA,1 + xA,2 + xA,3
xB,1 + xB,2 + xB,3
xA,1 + xB,1
xA,2 + xB,2
xA,3 + xB,3
= 13
=9
=7
=9
=6
x ≥0
9
Heuristic solution with objective value 53:
A→1=7
A→2=6
B →2=3
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
B →3=6
35 / 62
Dual multipliers: proofing solution quality
min
s.t.
2xA,1 +3xA,2 +9xA,3 +4xB,1 +1xB,2 +3xB,3
xA,1 +xA,2 +xA,3
xB,1 +xB,2
+xB,1
+xB,2
xA,1
xA,2
xA,3
= 13
+xB,3 = 9
=7
=9
+xB,3 = 6
x
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
≥0
36 / 62
Dual multipliers: proofing solution quality
min
2xA,1 +3xA,2 +9xA,3 +4xB,1 +1xB,2 +3xB,3
s.t. 2 ×
1×
xA,1 +xA,2 +xA,3
xB,1 +xB,2
+xB,1
+xB,2
xA,1
xA,2
2×
xA,3
= 13
+xB,3 = 9
=7
=9
+xB,3 = 6
x
≥0
+
⇒ min ≥
2xA,1 +2xA,2 +3xA,3 +1xB,1 +1xB,2 +3xB,3 = 47
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
36 / 62
Dual multipliers: proofing solution quality
min
2xA,1 +3xA,2 +9xA,3 +4xB,1 +1xB,2 +3xB,3
s.t. 3 ×
1×
−1×
xA,1 +xA,2 +xA,3
xB,1 +xB,2
+xB,1
+xB,2
xA,1
xA,2
2×
xA,3
= 13
+xB,3 = 9
=7
=9
+xB,3 = 6
x
≥0
+
⇒ min ≥
2xA,1 +3xA,2 +5xA,3
+1xB,2 +3xB,3 = 53
Solution is optimal!
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
36 / 62
Duality
Primal LP
Dual LP
min c Tx | Ax = b, x ≥ 0
max b Ty | y TA ≤ c T, y ∈
Rm Simple observation: For any x, y feasible,
c Tx ≥ y TAx = b Ty .
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
37 / 62
Duality
Primal LP
Dual LP
min c Tx | Ax = b, x ≥ 0
max b Ty | y TA ≤ c T, y ∈
Rm Simple observation: For any x, y feasible,
c Tx ≥ y TAx = b Ty .
. Weak Duality:
min c Tx | Ax = b, x ≥ 0 ≥ max b Ty | y TA ≤ c T, y ∈
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
Rm 37 / 62
Duality
Primal LP
Dual LP
min c Tx | Ax = b, x ≥ 0
max b Ty | y TA ≤ c T, y ∈
Rm Simple observation: For any x, y feasible,
c Tx ≥ y TAx = b Ty .
. Weak Duality:
min c Tx | Ax = b, x ≥ 0 ≥ max b Ty | y TA ≤ c T, y ∈
Rm . Strong Duality:
min c Tx | Ax = b, x ≥ 0 = max b Ty | y TA ≤ c T, y ∈
Rm Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
37 / 62
Vertex solutions
Consider n variables, m constraints:
min c Tx | Ax = b, x ≥ 0
with A ∈
Rm×n , b ∈ Rm , c ∈ Rn .
. If optimal: there always exists an optimal vertex solution.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
38 / 62
Vertex solutions
Consider n variables, m constraints:
min c Tx | Ax = b, x ≥ 0
with A ∈
Rm×n , b ∈ Rm , c ∈ Rn .
. If optimal: there always exists an optimal vertex solution.
. Vertices uniquely determined by n tight inequalities:
. In standard from that means
I
m equality constraints Ax = b
I
n − m tight bounds xi = 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
38 / 62
Vertex solutions
Consider n variables, m constraints:
min c Tx | Ax = b, x ≥ 0
with A ∈
Rm×n , b ∈ Rm , c ∈ Rn .
. If optimal: there always exists an optimal vertex solution.
. Vertices uniquely determined by n tight inequalities:
. In standard from that means
I
m equality constraints Ax = b
I
n − m tight bounds xi = 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
38 / 62
Basic solutions
Primal solution
. Fix n − m variables: xi = 0 for i ∈ N ⊆ {1, . . . , n}
. m variables remain: xi for i ∈ B = {1, . . . , n} \ N
. Solve linear system with m equations, m variables:
Ax = b
AB xB = b
xB = A−1
B b
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
39 / 62
Basic solutions
Primal solution
. Fix n − m variables: xi = 0 for i ∈ N ⊆ {1, . . . , n}
. m variables remain: xi for i ∈ B = {1, . . . , n} \ N
. Solve linear system with m equations, m variables:
Ax = b
AB xB = b
xB = A−1
B b
Dual multipliers
. Globally: find y such that y TA ≤ c T
. Locally: ignore fixed variables and solve
y TAB = cBT
y T = cBT A−1
B
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
39 / 62
Basic solutions
Primal solution
. Fix n − m variables: xi = 0 for i ∈ N ⊆ {1, . . . , n}
. m variables remain: xi for i ∈ B = {1, . . . , n} \ N
. Solve linear system with m equations, m variables:
Ax = b
AB xB = b
xB = A−1
B b
Dual multipliers
. Globally: find y such that y TA ≤ c T
. Locally: ignore fixed variables and solve
y TAB = cBT
y T = cBT A−1
B
Basic solution = discrete basis B + primal sol. x + dual mult. y
. In theory: could enumerate mn basic solutions.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
39 / 62
Improving steps
. xB is a function of xN :
AB xB + AN xN = b
x1 = 0
x̂
x2 = 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Improving steps
. xB is a function of xN :
AB xB = b − AN xN
x1 = 0
x̂
x2 = 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Improving steps
. xB is a function of xN :
xB =
A−1
B (b
− AN xN )
x1 = 0
x̂
x2 = 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Improving steps
. xB is a function of xN :
xB =
. x̂N = 0
A−1
B (b
− AN xN )
x̂B = A−1
B b
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
x1 = 0
x̂
x2 = 0
40 / 62
Improving steps
. xB is a function of xN :
xB =
A−1
B (b
− AN xN )
. x̂N = 0
x̂B = A−1
B b
. Unfix xi , i ∈ N :
x1 = 0
x̂
δ2
x2 > 0
A x
xB = x̂B − A−1
| B{z }i i
δi
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Improving steps
. xB is a function of xN :
xB =
A−1
B (b
− AN xN )
. x̂N = 0
x̂B = A−1
B b
. Unfix xi , i ∈ N :
x1 > 0
x̂
x2 = 0
δ1
A x
xB = x̂B − A−1
| B{z }i i
δi
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Improving steps
. xB is a function of xN :
xB =
A−1
B (b
− AN xN )
. x̂N = 0
x̂B = A−1
B b
. Unfix xi , i ∈ N :
x1 > 0
x̂
x2 = 0
δ1
A x
xB = x̂B − A−1
| B{z }i i
δi
. Objective change:
c Tx = c Tx̂ + ci xi − c Tδi xi
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Improving steps
. xB is a function of xN :
xB =
A−1
B (b
x1 > 0
x̂
x2 = 0
δ1
− AN xN )
. x̂N = 0
x̂B = A−1
B b
. Unfix xi , i ∈ N :
A x
xB = x̂B − A−1
| B{z }i i
δi
. Objective change:
c Tx = c Tx̂ + ci xi − c Tδi xi = c Tx̂ + (ci − y TAi ) xi
| {z }
reduced cost ri
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Improving steps
. xB is a function of xN :
xB =
A−1
B (b
x1 > 0
x̂
x2 = 0
δ1
− AN xN )
. x̂N = 0
x̂B = A−1
B b
. Unfix xi , i ∈ N :
A x
xB = x̂B − A−1
| B{z }i i
δi
. Objective change:
c Tx = c Tx̂ + ci xi − c Tδi xi = c Tx̂ + (ci − y TAi ) xi
| {z }
reduced cost ri
. Reduced cost of xi = obj. change per unit increase of xi
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Improving steps
. xB is a function of xN :
x3 = 0
xB = A−1
B (b − AN xN )
αδ1
. x̂N = 0
x̂B = A−1
B b
. Unfix xi , i ∈ N :
x2 = 0
A x
xB = x̂B − A−1
| B{z }i i
δi
. Objective change:
c Tx = c Tx̂ + ci xi − c Tδi xi = c Tx̂ + (ci − y TAi ) xi
| {z }
reduced cost ri
. Reduced cost of xi = obj. change per unit increase of xi
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
40 / 62
Primal simplex algorithm
Idea: Given a primal feasible starting basis B, i.e., xB = A−1
B b ≥ 0,
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
41 / 62
Primal simplex algorithm
Idea: Given a primal feasible starting basis B, i.e., xB = A−1
B b ≥ 0,
. maintain primal feasibility
. improve obj. value until reduced costs are ≥ 0 ⇔ dual feasible
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
41 / 62
Primal simplex algorithm
Idea: Given a primal feasible starting basis B, i.e., xB = A−1
B b ≥ 0,
. maintain primal feasibility
. improve obj. value until reduced costs are ≥ 0 ⇔ dual feasible
Init
. factorize basis matrix AB
“A−1
B ”
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
41 / 62
Primal simplex algorithm
Idea: Given a primal feasible starting basis B, i.e., xB = A−1
B b ≥ 0,
. maintain primal feasibility
. improve obj. value until reduced costs are ≥ 0 ⇔ dual feasible
Init
. factorize basis matrix AB
“A−1
B ”
. solve for xB and y
. compute reduced costs: ri = ci − y TAi for i ∈ N
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
41 / 62
Primal simplex algorithm
Idea: Given a primal feasible starting basis B, i.e., xB = A−1
B b ≥ 0,
. maintain primal feasibility
. improve obj. value until reduced costs are ≥ 0 ⇔ dual feasible
Init
. factorize basis matrix AB
“A−1
B ”
. solve for xB and y
. compute reduced costs: ri = ci − y TAi for i ∈ N
Repeat
. if ri ≥ 0 for all i ∈ N stop
OPTIMAL
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
41 / 62
Primal simplex algorithm
Idea: Given a primal feasible starting basis B, i.e., xB = A−1
B b ≥ 0,
. maintain primal feasibility
. improve obj. value until reduced costs are ≥ 0 ⇔ dual feasible
Init
. factorize basis matrix AB
“A−1
B ”
. solve for xB and y
. compute reduced costs: ri = ci − y TAi for i ∈ N
Repeat
. if ri ≥ 0 for all i ∈ N stop
else choose ri < 0, i ∈ N
OPTIMAL
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
(pricing)
41 / 62
Primal simplex algorithm
Idea: Given a primal feasible starting basis B, i.e., xB = A−1
B b ≥ 0,
. maintain primal feasibility
. improve obj. value until reduced costs are ≥ 0 ⇔ dual feasible
Init
. factorize basis matrix AB
“A−1
B ”
. solve for xB and y
. compute reduced costs: ri = ci − y TAi for i ∈ N
Repeat
. if ri ≥ 0 for all i ∈ N stop
OPTIMAL
else choose ri < 0, i ∈ N
. compute max. steplength α s.t. xB − αδi ≥ 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
(pricing)
(ratiotest)
41 / 62
Primal simplex algorithm
Idea: Given a primal feasible starting basis B, i.e., xB = A−1
B b ≥ 0,
. maintain primal feasibility
. improve obj. value until reduced costs are ≥ 0 ⇔ dual feasible
Init
. factorize basis matrix AB
“A−1
B ”
. solve for xB and y
. compute reduced costs: ri = ci − y TAi for i ∈ N
Repeat
. if ri ≥ 0 for all i ∈ N stop
OPTIMAL
else choose ri < 0, i ∈ N
. compute max. steplength α s.t. xB − αδi ≥ 0
. if α = ∞ stop
UNBOUNDED
else update B, x, y , r , A−1
B
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
(pricing)
(ratiotest)
41 / 62
Re-solving and hot starts
An IP solver classically solves many related LPs.
. modified objective function
. changed variable bounds or added constraints
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
42 / 62
Re-solving and hot starts
An IP solver classically solves many related LPs.
. modified objective function
I
I
I
last basic solution becomes suboptimal, but remains primal feasible
only reduced costs ri = ci − y TAi change
continue with primal simplex iterations
. changed variable bounds or added constraints
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
42 / 62
Re-solving and hot starts
An IP solver classically solves many related LPs.
. modified objective function
I
I
I
last basic solution becomes suboptimal, but remains primal feasible
only reduced costs ri = ci − y TAi change
continue with primal simplex iterations
. changed variable bounds or added constraints
I
I
I
last basic solution becomes primal infeasible
dual multipliers and reduced costs unchanged
continue with dual simplex
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
42 / 62
Re-solving and hot starts
An IP solver classically solves many related LPs.
. modified objective function
I
I
I
last basic solution becomes suboptimal, but remains primal feasible
only reduced costs ri = ci − y TAi change
continue with primal simplex iterations
. changed variable bounds or added constraints
I
I
I
last basic solution becomes primal infeasible
dual multipliers and reduced costs unchanged
continue with dual simplex
Dual simplex
. basic procedures as in primal simplex
. maintains dual feasibility and moves towards primal feasibility
. objective value increases towards optimum
. typically very few iterations to re-optimize
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
42 / 62
Dual simplex algorithm
Idea: Given a dual feasible starting basis B, i.e., r ≥ 0,
. maintain dual feasibility
. reduce primal infeasibility until x ≥ 0
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
43 / 62
Dual simplex algorithm
Idea: Given a dual feasible starting basis B, i.e., r ≥ 0,
. maintain dual feasibility
. reduce primal infeasibility until x ≥ 0
Init
. factorize basis matrix AB
“A−1
B ”
. solve for xB and y
. compute reduced costs: ri = ci − y TAi for i ∈ N
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
43 / 62
Dual simplex algorithm
Idea: Given a dual feasible starting basis B, i.e., r ≥ 0,
. maintain dual feasibility
. reduce primal infeasibility until x ≥ 0
Init
. factorize basis matrix AB
“A−1
B ”
. solve for xB and y
. compute reduced costs: ri = ci − y TAi for i ∈ N
Repeat
. if xi ≥ 0 for all i ∈ B stop
OPTIMAL
else choose xi < 0, i ∈ B
. compute max. steplength α
. if α = ∞ stop
INFEASIBLE
else update B, x, y , r , A−1
B
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
(pricing)
(ratiotest)
43 / 62
Dual simplex algorithm
Idea: Given a dual feasible starting basis B, i.e., r ≥ 0,
. maintain dual feasibility
. reduce primal infeasibility until x ≥ 0
Init
. factorize basis matrix AB
“A−1
B ”
. solve for xB and y
. compute reduced costs: ri = ci − y TAi for i ∈ N
Repeat while c Tx < z ∗ obj. limit
. if xi ≥ 0 for all i ∈ B stop
OPTIMAL
else choose xi < 0, i ∈ B
. compute max. steplength α
. if α = ∞ stop
INFEASIBLE
else update B, x, y , r , A−1
B
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
(pricing)
(ratiotest)
43 / 62
Summary
. Discrete and continuous:
I
I
I
vertices are uniquely determined by n equalities
if optimal: there always exists an optimal vertex solution
solution values are computed numerically
. Reduced costs quantify impact of (non-basic) variable on the
objective
. efficient hot starts for re-optimization
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
44 / 62
Summary
. Discrete and continuous:
I
I
I
vertices are uniquely determined by n equalities
if optimal: there always exists an optimal vertex solution
solution values are computed numerically
. Reduced costs quantify impact of (non-basic) variable on the
objective
. efficient hot starts for re-optimization
Further aspects:
. general bounds: lbi ≤ xi ≤ ubi
. feasible starting basis for simplex (“phase 1”), pricing strategies,
linear algebra tricks, . . .
. exponential worst-case complexity of simplex vs. performance in
practice
. interior point algorithms
. algorithms for specially structured LPs: networks, . . .
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
44 / 62
Outline
Integer Programming for Constraint Programmers
1
Introduction
2
Linear programming
3
Integer (linear) programming
4
Summary
5
Discussion
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
45 / 62
Linear programming
Linear program
Objective function:
. linear function
Feasible set:
. described by linear constraints
Variable domains:
. real values
min c Tx
s.t.
Ax = b
x∈
Rn≥0
. convex set
. “basic” solutions
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
46 / 62
Integer programming
Integer Program
Objective function:
. linear function
Feasible set:
. described by linear constraints
Variable domains:
. integer values
min c Tx
s.t.
Ax ≤ b
x∈
Z≥0
. not even connected
. N P-hard problem
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
47 / 62
An incomplete history on integer programming
Cutting plane algorithm
. R. E. Gomory, “Outline of an algorithm for integer solutions to
linear programs”. Bull. AMS 64, 1958, pp. 275–278.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
48 / 62
An incomplete history on integer programming
Cutting plane algorithm
. R. E. Gomory, “Outline of an algorithm for integer solutions to
linear programs”. Bull. AMS 64, 1958, pp. 275–278.
Branch-and-bound
. A. H. Land, A. G. Doig, “An automatic method of solving discrete
programming problems”. Econometrica 28, 1960, pp. 497–520
. R. J. Dakin, “A tree-search algorithm for mixed integer
programming problems”. The Computer Journal, Volume 8, 1965,
pp. 250–255
. J. D. C. Little, K. G. Murty, D. W. Sweeney, C. Karel, “An
algorithm for the traveling salesman problem”. Operations
Research 11, 1963, pp. 972–989.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
48 / 62
An incomplete history on integer programming
Cutting plane algorithm
. R. E. Gomory, “Outline of an algorithm for integer solutions to
linear programs”. Bull. AMS 64, 1958, pp. 275–278.
Branch-and-bound
. A. H. Land, A. G. Doig, “An automatic method of solving discrete
programming problems”. Econometrica 28, 1960, pp. 497–520
. R. J. Dakin, “A tree-search algorithm for mixed integer
programming problems”. The Computer Journal, Volume 8, 1965,
pp. 250–255
. J. D. C. Little, K. G. Murty, D. W. Sweeney, C. Karel, “An
algorithm for the traveling salesman problem”. Operations
Research 11, 1963, pp. 972–989.
Branch-and-cut
. Grötschel, Jünger, Reinelt (1984, 1985, 1987)
. Padberg, Rinaldi (1991)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
48 / 62
General cutting plane method
x2
FIP := {x ∈
Zn+ : Ax ≤ b}
3
FLP := {x ∈
Rn+
2
: Ax ≤ b}
∗
xIP
1
x1
1
2
3
4
min{c T x : x ∈ FIP }
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
49 / 62
General cutting plane method
Observation
. conv(FIP ) is a polyhedron
x2
3
. IP could be formulated as LP
∗
xIP
2
Problems with conv(FIP ):
. linear description not known
1
conv(FIP )
. large nr. of constraints needed
x1
1
2
3
4
min{c T x : x ∈ conv(FIP )}
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
49 / 62
General cutting plane method
Observation
x2
. conv(FIP ) is a polyhedron
3
. IP could be formulated as LP
∗
xIP
2
Problems with conv(FIP ):
. linear description not known
1
conv(FIP )
. large nr. of constraints needed
x1
1
2
3
4
min{c T x : x ∈ conv(FIP )}
FLP ⊇
T
F ⊇
T
conv(FIP )
T
min{c x : x ∈ FLP } ≤ min{c x : x ∈ F} = min{c x : x ∈ conv(FIP )}
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
49 / 62
General cutting plane method
x2
Algorithm
3
FLP
1. F ← FLP
2
2. Solve
min c Tx
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
1
2
3
4
min{c T x : x ∈ conv(FIP )}
5. Goto 2.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
FLP= F
1. F ← FLP
2
2. Solve
min c Tx
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
1
2
3
4
min{c T x : x ∈ conv(FIP )}
5. Goto 2.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
x∗
1. F ← FLP
FLP= F
2
2. Solve
min c Tx
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
1
2
3
4
min{c T x : x ∈ conv(FIP )}
5. Goto 2.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
x∗
1. F ← FLP
FLP= F
2
2. Solve
min c Tx
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
1
2
3
4
min{c T x : x ∈ conv(FIP )}
5. Goto 2.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
x∗
1. F ← FLP
FLP= F
2
2. Solve
min c Tx
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
1
2
3
4
min{c T x : x ∈ conv(FIP )}
5. Goto 2.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
2
2. Solve
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
1
2
3
4
min{c T x : x ∈ conv(FIP )}
5. Goto 2.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
2
2. Solve
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
1
2
3
4
min{c T x : x ∈ conv(FIP )}
5. Goto 2.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
2
2. Solve
x∗
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
5. Goto 2.
1
2
3
4
min{c T x : x ∈ conv(FIP )}
. Resolving is cheap since dual
feasible (hot start)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
2
2. Solve
x∗
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
5. Goto 2.
1
2
3
4
min{c T x : x ∈ conv(FIP )}
. Resolving is cheap since dual
feasible (hot start)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
2
2. Solve
x∗
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
5. Goto 2.
1
2
3
4
min{c T x : x ∈ conv(FIP )}
. Resolving is cheap since dual
feasible (hot start)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
2
2. Solve
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
5. Goto 2.
1
2
3
4
min{c T x : x ∈ conv(FIP )}
. Resolving is cheap since dual
feasible (hot start)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
2
2. Solve
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
5. Goto 2.
1
2
3
4
min{c T x : x ∈ conv(FIP )}
. Resolving is cheap since dual
feasible (hot start)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
x∗
2
2. Solve
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
5. Goto 2.
1
2
3
4
min{c T x : x ∈ conv(FIP )}
. Resolving is cheap since dual
feasible (hot start)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
x∗
2
2. Solve
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
5. Goto 2.
1
2
3
4
min{c T x : x ∈ conv(FIP )}
. Resolving is cheap since dual
feasible (hot start)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
General cutting plane method
x2
Algorithm
3
1. F ← FLP
∗
xIP
2
2. Solve
F
T
min c x
s.t.
x ∈F
1
conv(FIP )
x1
3. If x ∗ ∈ FIP : Stop
4. Add inequality to F that is ...
I valid for conv(F ) but
IP
I violated by x ∗ .
5. Goto 2.
1
2
3
4
min{c T x : x ∈ conv(FIP )}
. Resolving is cheap since dual
feasible (hot start)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
50 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
∅
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
∞
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
∞
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
∅
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
∞
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
∞
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
LP-based branch-and-bound (colorful picture)
Steps
1. Abort criterion
3. Solve relaxation
5. Feasibility check
2. Node selection
4. Bounding
6. Branching
x IP
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
51 / 62
Solving an integer program
x2
3
2
x∗
F
1
conv(FIP )
x1
1
2
3
4
min{c T x : x ∈ conv(FIP )}
cutting planes
x IP
branch-and-bound
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
52 / 62
Solving an integer program
x2
3
2
x∗
F
1
conv(FIP )
x1
1
2
3
4
min{c T x : x ∈ conv(FIP )}
cutting planes
x IP
branch-and-bound
. Both approaches solve an initial linear program.
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
52 / 62
Solving an integer program
x2
3
2
x∗
F
1
conv(FIP )
x1
1
2
3
4
min{c T x : x ∈ conv(FIP )}
cutting planes
x IP
branch-and-bound
. Both approaches solve an initial linear program.
. Classically using simplex algorithm for efficent hot start
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
52 / 62
Solving an integer program
x2
3
2
x∗
F
1
conv(FIP )
x1
1
2
3
4
min{c T x : x ∈ conv(FIP )}
cutting planes
x IP
branch-and-bound
. Both approaches solve an initial linear program.
. Classically using simplex algorithm for efficent hot start
I
A cutting plane or a bound change is an additional row (linear
constraint).
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
52 / 62
Branch-and-cut
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
53 / 62
Branch-and-cut
Branch-and-cut = Branch-and-bound + Cutting planes
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
53 / 62
Branch-and-cut
Branch-and-cut = Branch-and-bound + Cutting planes
. Use branch-and-bound as global solver paradigm.
. Perform limited cutting plane generation within each search node.
I
global versus local cuts
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
53 / 62
Branch-and-cut
Branch-and-cut = Branch-and-bound + Cutting planes
. Use branch-and-bound as global solver paradigm.
. Perform limited cutting plane generation within each search node.
I
global versus local cuts
. Pure cutting plane generation does not work in practice
I
I
numerical issues
convergence
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
53 / 62
Branch-and-cut
Branch-and-cut = Branch-and-bound + Cutting planes
. Use branch-and-bound as global solver paradigm.
. Perform limited cutting plane generation within each search node.
I
global versus local cuts
. Pure cutting plane generation does not work in practice
I
I
numerical issues
convergence
. Pure branch-and-bound fails in general
I
exponential search tree
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
53 / 62
Branch-and-cut
Branch-and-cut = Branch-and-bound + Cutting planes
. Use branch-and-bound as global solver paradigm.
. Perform limited cutting plane generation within each search node.
I
global versus local cuts
. Pure cutting plane generation does not work in practice
I
I
numerical issues
convergence
. Pure branch-and-bound fails in general
I
exponential search tree
. Branch-and-cut fails later
I
I
still exponential search tree
but shifts the exponential grow significantly
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
53 / 62
Solving an integer program
. How can a linear program be solved?
. How can an integer program be solved?
. For what is the linear programming relaxation used within an
integer programming solver?
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
54 / 62
Reduced cost propagation
. z ∗ : best objective value → c Tx ≤ z ∗
. x̂: LP optimum
. For variables xi with reduced cost ri 6= 0
I
I
I
I
variables are not in the basis
variables sitting on one of their bounds
ri > 0 → x̂i = lbi (lower bound)
ri < 0 → x̂i = ubi (upper bound)
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
∗
≤z
x
c
T
x̂
55 / 62
Reduced cost propagation
. z ∗ : best objective value → c Tx ≤ z ∗
. x̂: LP optimum
. For variables xi with reduced cost ri 6= 0
I
I
I
I
variables are not in the basis
variables sitting on one of their bounds
ri > 0 → x̂i = lbi (lower bound)
ri < 0 → x̂i = ubi (upper bound)
∗
≤z
x
c
T
x̂
Case 1 ri > 0:
c Tx̂ + ri (xi − lbi ) ≤ z ∗ ⇔ xi ≤
z ∗ − c Tx̂
+ lbi
ri
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
55 / 62
Reduced cost propagation
. z ∗ : best objective value → c Tx ≤ z ∗
. x̂: LP optimum
. For variables xi with reduced cost ri 6= 0
I
I
I
I
variables are not in the basis
variables sitting on one of their bounds
ri > 0 → x̂i = lbi (lower bound)
ri < 0 → x̂i = ubi (upper bound)
∗
≤z
x
c
T
x̂
Case 1 ri > 0:
c Tx̂ + ri (xi − lbi ) ≤ z ∗ ⇔ xi ≤
∗
z − c Tx̂
z ∗ − c Tx̂
+ lbi
+ lbi ⇒ xi ≤
ri
ri
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
55 / 62
Reduced cost propagation
. z ∗ : best objective value → c Tx ≤ z ∗
. x̂: LP optimum
. For variables xi with reduced cost ri 6= 0
I
I
I
I
variables are not in the basis
variables sitting on one of their bounds
ri > 0 → x̂i = lbi (lower bound)
ri < 0 → x̂i = ubi (upper bound)
∗
≤z
x
c
T
x̂
Case 1 ri > 0:
c Tx̂ + ri (xi − lbi ) ≤ z ∗ ⇔ xi ≤
∗
z − c Tx̂
z ∗ − c Tx̂
+ lbi
+ lbi ⇒ xi ≤
ri
ri
Case 2 ri < 0:
∗
z ∗ − c Tx̂
z − c Tx̂
c x̂ + ri (xi − ubi ) ≤ z ⇔ xi ≥
+ ubi ⇒ xi ≥
+ ubi
ri
ri
T
∗
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
55 / 62
Branching – Pseudo Cost
Estimating the objective
x3 = 7.4
c=2
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
ζ − (x3 ) =
4−2
7.4−7
=
2
0.4
=5
x3 = 7.4
c=2
x3 ≤ 7
c=4
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
ζ + (x3 ) =
8−2
8−7.4
=
6
0.6
= 10
x3 = 7.4
c=2
x3 ≤ 7
c=4
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
x3 ≥ 8
c=8
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
ζ1− (x3 ) = 5, ζ1+ (x3 ) = 10
x3 = 7.4
c=2
x3 ≤ 7
x3 ≥ 8
ζ1− (x3 )
ζ1+ (x3 )
c=4
c=8
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
I
ζ1− (x3 ) = 5, ζ1+ (x3 ) = 10
other values at other nodes
x3 ≤ 4
x3 ≤ 7
ζ2− (x3 ) = 3
x3 ≥ 8
ζ1− (x3 ) = 5
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
x3 ≥ 5
ζ2+ (x3 ) = 9
ζ1+ (x3 ) = 10
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
I
ζ1− (x3 ) = 5, ζ1+ (x3 ) = 10
other values at other nodes
x3 ≤ 4
. pseudocosts:
average objective gain
ψ − (x3 ) =
ζ1− (x3 )+...+ζn− (x3 )
n
=
x3 ≤ 7
5+3
2
=4
ζ2− (x3 ) = 3
x3 ≥ 8
ζ1− (x3 ) = 5
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
x3 ≥ 5
ζ2+ (x3 ) = 9
ζ1+ (x3 ) = 10
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
I
ζ1− (x3 ) = 5, ζ1+ (x3 ) = 10
other values at other nodes
. pseudocosts:
average objective gain
ψ − (x3 ) = 4, ψ + (x3 ) = 9.5
x3 = 5.2
c =0
. estimate increase of objective
by pseudocosts and fractionality:
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
I
ζ1− (x3 ) = 5, ζ1+ (x3 ) = 10
other values at other nodes
. pseudocosts:
average objective gain
ψ − (x3 ) = 4, ψ + (x3 ) = 9.5
x3 = 5.2
c =0
x3 ≤ 5
. estimate increase of objective
by pseudocosts and fractionality:
ψ − (x3 ) · frac(x3 )
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
I
ζ1− (x3 ) = 5, ζ1+ (x3 ) = 10
other values at other nodes
. pseudocosts:
average objective gain
ψ − (x3 ) = 4, ψ + (x3 ) = 9.5
x3 = 5.2
c =0
x3 ≤ 5
. estimate increase of objective
by pseudocosts and fractionality:
ψ − (x3 ) · frac(x3 ) = 4 · 0.2 = 0.8,
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
c ≈ 0.8
56 / 62
Branching – Pseudo Cost
Estimating the objective
. objective gain per unit:
I
I
ζ1− (x3 ) = 5, ζ1+ (x3 ) = 10
other values at other nodes
. pseudocosts:
average objective gain
ψ − (x3 ) = 4, ψ + (x3 ) = 9.5
. estimate increase of objective
by pseudocosts and fractionality:
ψ − (x3 ) · frac(x3 ) = 4 · 0.2 = 0.8,
and ψ + (x3 )(1 − frac(x3 )) = 7.6
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
x3 = 5.2
c =0
x3 ≤ 5
x3 ≥ 6
c ≈ 0.8
c ≈ 7.6
56 / 62
Heuristic – RENS
RENS – Relaxation Enforced Neighborhood Search
Idea: Search the vicinity of a relaxation solution
Algorithm
1. x̄ ← LP optimum;
2. Fix all integral variables:
xi := x̄i for all i : x̄i ∈ ;
3. Reduce domain of fractional variables
xi ∈ {bx̄i c; dx̄i e};
4. Solve the resulting sub-MIP;
Z
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
57 / 62
Heuristic – RENS
RENS – Relaxation Enforced Neighborhood Search
Idea: Search the vicinity of a relaxation solution
Algorithm
1. x̄ ← LP optimum;
2. Fix all integral variables:
xi := x̄i for all i : x̄i ∈ ;
3. Reduce domain of fractional variables
xi ∈ {bx̄i c; dx̄i e};
4. Solve the resulting sub-MIP;
Z
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
57 / 62
Heuristic – RENS
RENS – Relaxation Enforced Neighborhood Search
Idea: Search the vicinity of a relaxation solution
Algorithm
1. x̄ ← LP optimum;
2. Fix all integral variables:
xi := x̄i for all i : x̄i ∈ ;
3. Reduce domain of fractional variables
xi ∈ {bx̄i c; dx̄i e};
4. Solve the resulting sub-MIP;
Z
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
57 / 62
Heuristic – RENS
RENS – Relaxation Enforced Neighborhood Search
Idea: Search the vicinity of a relaxation solution
Algorithm
1. x̄ ← LP optimum;
2. Fix all integral variables:
xi := x̄i for all i : x̄i ∈ ;
3. Reduce domain of fractional variables
xi ∈ {bx̄i c; dx̄i e};
4. Solve the resulting sub-MIP;
Z
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
57 / 62
Heuristic – RENS
RENS – Relaxation Enforced Neighborhood Search
Idea: Search the vicinity of a relaxation solution
Algorithm
1. x̄ ← LP optimum;
2. Fix all integral variables:
xi := x̄i for all i : x̄i ∈ ;
3. Reduce domain of fractional variables
xi ∈ {bx̄i c; dx̄i e};
4. Solve the resulting sub-MIP;
Z
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
57 / 62
Heuristic – RENS
RENS – Relaxation Enforced Neighborhood Search
Idea: Search the vicinity of a relaxation solution
Algorithm
1. x̄ ← LP optimum;
2. Fix all integral variables:
xi := x̄i for all i : x̄i ∈ ;
3. Reduce domain of fractional variables
xi ∈ {bx̄i c; dx̄i e};
4. Solve the resulting sub-MIP;
Z
Crucial point: Does not need a feasible start solution
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
57 / 62
Outline
Integer Programming for Constraint Programmers
1
Introduction
2
Linear programming
3
Integer (linear) programming
4
Summary
5
Discussion
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
58 / 62
Summary
Linear relaxation
. gives a global view
. provides a proven dual bound for the original problem
I
quality guarantee
. can be used for more than getting a dual bound
I
propagation, branching, primal heuristic, ...
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
59 / 62
Summary
Linear relaxation
. gives a global view
. provides a proven dual bound for the original problem
I
quality guarantee
. can be used for more than getting a dual bound
I
propagation, branching, primal heuristic, ...
Additional remarks
. a linear relaxation does not have to represent all constraints
. numeric issues can arise due to continuous optimization
I
I
I
in general the numerics can be controlled
there exist critical instances
see also exact interger programming
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
59 / 62
Summary
Linear relaxation
. gives a global view
. provides a proven dual bound for the original problem
I
quality guarantee
. can be used for more than getting a dual bound
I
propagation, branching, primal heuristic, ...
Additional remarks
. a linear relaxation does not have to represent all constraints
. numeric issues can arise due to continuous optimization
I
I
I
in general the numerics can be controlled
there exist critical instances
see also exact interger programming
As in CP, the chosen model has a huge impact
on the performance of a solver
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
59 / 62
An incomplete list of IP Solvers
Non-commercial solvers
.
.
.
.
.
CBC (IBM)
GLPK
LPSOLVE
SCIP
SYMPHONY
https://projects.coin-or.org/Cbc
http://www.gnu.org/s/glpk/
http://lpsolve.sourceforge.net/
http://scip.zib.de
https://projects.coin-or.org/SYMPHONY
Commercial solvers
.
.
.
.
.
CPLEX (IBM)
GUROBI
MOPS
MOSEK
XPRESS (Fico)
http://www.cplex.com
http://www.gurobi.com
http://www.mops-optimizer.com
http://www.mosek.com
http://www.fico.com
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
60 / 62
Outline
Integer Programming for Constraint Programmers
1
Introduction
2
Linear programming
3
Integer (linear) programming
4
Summary
5
Discussion
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
61 / 62
Questions
Ambros Gleixner and Stefan Heinz (ZIB) – Integer Programming for Constraint Programmers
62 / 62
Tutorial
Integer Programming for
Constraint Programmers
Ambros Gleixner and Stefan Heinz
Zuse Institute Berlin (ZIB)
Chris Beck, Timo Berthold, and Kati Wolter
DFG Research Center MATHEON
Mathematics for key technologies
CP 2011, 14/Sep/2011