Integer Programming
Chapter 15
University of Chicago
Booth School of Business
Kipp Martin
November 9, 2016
1 / 101
Outline
Key Concepts
Problem Formulation Quality
Solver Options
Epsilon Optimality
Preprocessing
Cutting Planes
Chvátal-Gomory (C-G)
Constraint Aggregation Coefficient Reduction Cuts
Separation
Branch and Cut
COIN-OR Cgl
2 / 101
Key Concepts
I
the quality of the formulation is what counts, not the size –
quality usually measured by the lower bound – critical to
generate good formulations
I
problem reformulation
I
preprocess the problem
I
generating cutting planes
I
important to control solver options
I
getting the last bit of optimality can be tough
3 / 101
Problem Formulation Quality
Consider the following two sets of integer points.
Γ = {(x1 , x2 , y ) | x1 ≤ y , x2 ≤ y , x1 , x2 , y ∈ {0, 1}}.
Γ0 = {(x1 , x2 , y ) | x1 + x2 ≤ 2y , x1 , x2 , y ∈ {0, 1}}.
What are the points in each set and how do they differ?
4 / 101
Problem Formulation Quality
Consider the following two sets of integer points.
Γ = {(x1 , x2 , y ) | x1 ≤ y , x2 ≤ y , x1 , x2 , y ∈ {0, 1}}.
y
0
1
1
1
1
x1
0
0
1
0
1
x2
0
0
0
1
1
Γ0 has exactly the same set of points!
5 / 101
Problem Formulation Quality
Consider the following linear relaxations.
Γ = {(x1 , x2 , y ) | x1 ≤ y , x2 ≤ y , x1 , x2 , y ≥ 0, x1 , x2 , y ≤ 1}.
Γ0 = {(x1 , x2 , y ) | x1 + x2 ≤ 2y , x1 , x2 , y ≥ 0, x1 , x2 , y ≤ 1}.
What are the points in each set and how do they differ? What
about extreme points?
6 / 101
Problem Formulation Quality
Γ is the convex hull of integer points in Γ. Not so for Γ0
y
0
1
1
1
1
1/2
1/2
x1
0
0
1
0
1
0
1
x2
0
0
0
1
1
1
0
Important: Γ is a proper subset of Γ0 . It is a better relaxation.
Why?
7 / 101
Problem Formulation Quality
8 / 101
Problem Formulation Quality
THE BIG TAKE AWAY: Different polyhedra may contain exactly
same set of integer points!
THE BIG TAKE AWAY: Different polyhedra may contain exactly
same set of integer points!
THE BIG TAKE AWAY: Different polyhedra may contain exactly
same set of integer points!
WHAT IS THE BIG TAKE AWAY?
9 / 101
Problem Formulation Quality
Mixed Integer Finite Basis Theorem: If
l
Γ = {(x, y ) | Ax + By = b, x ≥ 0, y ∈ Z+
},
(1)
where A, B are rational matrices and b is a rational vector, then
conv(Γ) is a rational polyhedron.
Problem: in most real applications conv(Γ) ⊂ P where
P = {(x, y ) | Ax + By = b, x ≥ 0, y ≥ 0}
Objective: Make P smaller and more closely approximate conv(Γ).
10 / 101
Problem Formulation Quality
Mixed Integer Finite Basis Theorem: If
l
Γ = {(x, y ) | Ax + By = b, x ≥ 0, y ∈ Z+
},
(2)
where A, B are rational matrices and b is a rational vector, then
conv(Γ) is a rational polyhedron.
Problem: in most real applications conv(Γ) ⊂ P where
P = {(x, y ) | Ax + By = b, x ≥ 0, y ≥ 0}
Objective: Make P smaller and more closely approximate conv(Γ).
11 / 101
Problem Formulation Quality
Absolutely Critical: Understand the differences between
1. Γ
2. P or Γ
3. conv(Γ)
12 / 101
Reformulation (Core Carrier)
Core Carrier Revisited: Take the constraints
X
xij
≤ Myj ,
j ∈J
i∈Mj
and replace with the equivalent set:
xij
≤ yj ,
i ∈ Mj ,
j ∈J
THE BIG TAKE AWAY: The same set of integer points can have
many different representations in terms of linear inequalities.
13 / 101
Reformulation (Core Carrier)
Core Carrier Revisited (Continued) :
Rows
Columns
LP Value
Tight
35152
16300
876586
Loose
753
16300
791159
What happens when you try to solve?
14 / 101
Reformulation (Lot Sizing)
Dynamic Lot Sizing :
Variables:
xit – units of product i produced in period t
Iit – inventory level of product i at the end of the period t
yit – is 1 if there is nonzero production of product i during
period t, 0 otherwise
Parameters:
dit – demand product i in period t
fit – fixed cost associated with nonzero production of product
i in period t
cit – marginal production cost for product i in period t
hit – marginal holding cost charged to product i at the end of
period t
gt – production capacity in period t
15 / 101
Reformulation (Lot Sizing)
Objective: Minimize sum of marginal production cost, holding
cost, fixed cost
N X
T
X
(cit xit + hit Iit + fit yit )
i=1 t=1
Constraint 1: Do not exceed total capacity in each period
N
X
xit ≤ gt ,
t = 1, . . . , T
i=1
16 / 101
Reformulation (Lot Sizing)
Constraint 2: Inventory balance equations
Ii,t−1 + xit − Iit = dit ,
i = 1, . . . , N, t = 1, . . . , T
Constraint 3: Fixed cost forcing constraints
xit − Mit yit ≤ 0,
i = 1, . . . N, t = 1, . . . , T
17 / 101
Reformulation (Lot Sizing)
Dynamic Lot Sizing : A “standard formulation” is:
min
N X
T
X
(cit xit + hit Iit + fit yit )
i=1 t=1
s.t.
N
X
xit ≤ gt ,
t = 1, . . . , T
i=1
Ii,t−1 + xit − Iit = dit ,
xit − Mit yit ≤ 0,
xit , Iit ≥ 0,
yit ∈ {0, 1},
i = 1, . . . , N, t = 1, . . . , T
i = 1, . . . N,
t = 1, . . . , T
i = 1, . . . , N,
t = 1, . . . , T
i = 1, . . . , N,
t = 1, . . . , T .
18 / 101
Reformulation (Lot Sizing)
Dynamic Lot Sizing : An alternate formulation:
zitk is 1, if for product i in period t, the decision is to produce
enough items to satisfy demand for periods t through k, 0
otherwise.
T
X
zi1k
= 1
k=1
−
t−1
X
j=1
zij,t−1 +
T
X
zitk
= 0,
zitk
≤ yit ,
i = 1, . . . , N,
t = 2, . . . , T
k=t
T
X
i = 1, . . . , N,
t = 1, . . . , T
k=t
19 / 101
Reformulation (Lot Sizing)
Dynamic Lot Size (tvw200) :
Rows
Columns
LP Value
Tight
3208
7987
187503
Loose
3208
4600
116880
What happens when you try to solve? The optimal integer value is
187543 (I think – the lower bound is 187504).
20 / 101
Solver Options
Most solvers take options. This is particularly important in integer
programming. In GAMS you can communicate options to solvers
through an option text file.
First we tell GAMS which solver we want:
OPTION
MIP = CoinCbc;
Next we tell GAMS that we want the first option file (you can have
more than one, 2, 3, ...)
lot_size.optfile = 1;
21 / 101
Solver Options
Here is some code we where we put in GAMS options
file opt CoinCbc option file /coincbc.opt/;
put opt;
put ’optcr 0’ /
put ’reslim 100000’ /
put ’nodelim 5000000’ /
put ’cuts off’/
put ’knapsackcuts on’/;
putclose opt;
Make sure to put this code before the solve statement.
22 / 101
Solver Options
Here is what the options file does:
I
set the tolerance on integer optimality to zero (optcr 0)
I
set a time limit of 100000 seconds
I
set a node limit of 5000000
I
turned cutting plane generation off (cuts off)
I
turned knapsack cuts on (knapsackcuts on)
23 / 101
Epsilon Optimality
Key Take Away: Actually proving optimality in branch and bound
can be tough.
The closer you get, the harder it becomes to resolve that list bit of
integrality gap.
If, for example, you set optcr = .01, then branch and bound will
terminate (assume minimization here) when
.99*UB ≤ LB
24 / 101
Epsilon Optimality
Experiment: run the tight version of the lot sizing with ratio 0
and ratio .0005. Here is what happens for me:
optcr = .000
optcr = .005
UB
187543
187550
LB
187508
187503
Nodes
112000
42
Seconds
4304
21.84
Is it worth it?
25 / 101
Epsilon Optimality
Another Take Away: the data may not be that accurate to being
with.
Standard Oil story.
26 / 101
Preprocessing
By preprocessing we mean what is done to a formulation to make
it more amenable to solution before solving the linear programming
relaxation.
Objective: make the linear programming relaxation of the
mixed-integer program easy and tight. Try the following
I eliminate redundant constraints
I fix variables
I scale coefficients
I coefficient reduction
I rounding
I improve bounds on variables and constraints
I probing
We work with the following canonical form:
X
j∈I +
aj x j +
X
j∈C +
aj x j −
X
j∈I −
aj xj −
X
aj xj ≥ b
(3)
j∈C −
27 / 101
Preprocessing
Rounding:
If C + = C − = ∅, aj is integer for all j ∈ I + ∪ I − and
α = gcd(aj | aj ∈ I+ ∪ I− ) then the conical form of the constraint is
X
X
aj x j −
aj xj ≥ b
(4)
j∈I −
j∈I +
which is equivalent to
X
(aj /α)xj −
j∈I +
X
(aj /α)xj ≥ b/α.
(5)
(aj /α)xj ≥ db/αe.
(6)
j∈I −
then a valid rounding is
X
j∈I +
(aj /α)xj −
X
j∈I −
28 / 101
Preprocessing
Rounding Example: Consider the inequality (assume x1 , x2 are
general integer variables.
2x1 + 2x2 ≥ 3
A feasible solution is x1 = 1.5 and x2 = 0. Now let’s round,
α = gcd(a1 , a2 ) = gcd(2, 2) = 2
So an equivalent inequality is:
(1/2)(2x1 + 2x2 ) ≥ (1/2)3
→
x1 + x2 ≥ 1.5
Rounding up the right-hand-side gives: x1 + x2 ≥ 2
Is x1 = 1.5 and x2 = 0 feasible?
29 / 101
Preprocessing
Rounding Example: Let’s look at some geometry. Plot the
feasible regions:
Γ1 = {(x1 , x2 ) | 2x1 + 2x2 ≥ 3, x1 , x2 ≥ 0}
Γ2 = {(x1 , x2 ) | x1 + x2 ≥ 2, x1 , x2 ≥ 0}
What is the relationship between?
I
Γ1 ∩ Z 2 and Γ2 ∩ Z 2
I
Γ1 and Γ2
30 / 101
Preprocessing
Coefficient Reduction: if C − = I − = ∅ then it is valid to reduce
the coefficients on the integer variables to b. That is, (3) is
equivalent to
X
min{aj , b}xj +
X
aj xj ≥ b.
(7)
j∈C +
j∈I +
Additionally, if C − or I − is not empty then upper bounds on the
variables in these sets are used as follows. Define:
λ := b +
X
aj hj +
j∈C −
X
aj h j .
(8)
j∈I −
Then (3) is equivalent to
X
j∈I +
min{aj , λ}xj +
X
j∈C +
aj x j −
X
j∈I −
aj x j −
X
aj xj ≥ b.
(9)
j∈C −
31 / 101
Preprocessing
Tightening Bounds: The canonical form
X
aj x j +
X
aj x j −
aj xj −
j∈I −
j∈C +
j∈I +
X
X
aj xj ≥ b
j∈C −
implies for each k ∈ C +
X
ak xk ≥ b −
aj xj −
j∈C +
j6=k
X
aj xj +
j∈I +
X
X
aj xj +
j∈C −
aj x j .
j∈I −
The smallest the right hand side of can be is
b−
X
j∈C +
j6=k
aj hj −
X
j∈I +
aj h j +
X
j∈C −
aj lj +
X
aj lj .
j∈I −
32 / 101
Preprocessing
Therefore, it is valid to reset lk to
(b −
X
aj hj −
j∈C +
j6=k
X
aj h j +
X
aj lj +
j∈C −
j∈I +
X
aj lj )/ak
j∈I −
Using similar logic the upper bounds of variables indexed by
C − ∪ I − are adjusted by
(
X
j∈C +
aj lj +
X
j∈I +
aj lj −
X
j∈C −
j6=k
aj h j −
X
aj hj − b)/ak
j∈I −
33 / 101
Preprocessing
This is an iterative process.
The upper and lower bounds on variables are adjusted until there is
no improvement in an upper or lower bound.
Once the lower and upper bounds are calculated one can apply
coefficient reduction on the integer variable coefficients.
34 / 101
Preprocessing
Example: Dynamic Lot Sizing
min
N X
T
X
(cit xit + hit Iit + fit yit )
i=1 t=1
s.t.
N
X
xit ≤ gt ,
t = 1, . . . , T
i=1
Ii,t−1 + xit − Iit = dit ,
xit − Mit yit ≤ 0,
xit , Iit ≥ 0,
yit ∈ {0, 1},
i = 1, . . . , N, t = 1, . . . , T
i = 1, . . . N,
t = 1, . . . , T
i = 1, . . . , N,
t = 1, . . . , T
i = 1, . . . , N,
t = 1, . . . , T .
35 / 101
Preprocessing
Example: Dynamic Lot Sizing (Continued) Consider the Big M
constraints
xit − Myit ≤ 0
in canonical form
−xit + Myit ≥ 0
Recall
λ := b +
X
j∈C −
aj h j +
X
aj h j .
j∈I −
In this case, what is b, C − , I − , and λ?
What do we get after coefficient reduction?
36 / 101
Preprocessing
Example: Dynamic Lot Sizing (Continued) Let’s tighten the
bounds on the xit variables. Assume 5 time periods (we drop the
product subscript – all products are treated identically). In
canonical form:
−I4 − x5 ≥ −d5
−I3 − x4 + I4 ≥ −d4
−I2 − x3 + I3 ≥ −d3
−I1 − x2 + I2 ≥ −d2
−x1 + I1 ≥ −d1
What is a valid upper bound on x5 ? What about I4 ?
Work through to period 1.
37 / 101
Preprocessing
Feasibility: The canonical form is
X
aj x j +
X
aj x j −
aj xj −
j∈I −
j∈C +
j∈I +
X
X
aj xj ≥ b
j∈C −
if
X
j∈I +
aj h j +
X
j∈C +
aj h j −
X
j∈I −
aj lj −
X
aj lj < b.
j∈C −
the model instance is not feasible.
38 / 101
Preprocessing
Redundancy: The canonical form is
X
aj x j +
j∈I +
X
aj x j −
X
aj xj −
j∈I −
j∈C +
X
aj xj ≥ b
j∈C −
if
X
aj lj +
j∈I +
X
aj lj −
j∈C +
X
aj h j −
j∈I −
X
aj hj ≥ b.
j∈C −
the constraint is redundant and can be deleted. Solve (possibly
relaxations)
min
X
j∈I +
s.t.
aj xj +
X
j∈C +
aj xj −
X
j∈I −
aj xj −
X
aj xj
j∈C −
Ax ≥ b
x ≥0
39 / 101
Preprocessing
Probing and Variable Fixing: In a mixed 0/1 linear program
probing refers to fixing a binary variable xk to 0 or 1 and then
observing any resulting implications. Assume variable xk is binary.
If xk = 1, k ∈ I − and
X
aj h j +
X
aj h j −
aj lj −
j∈I − \{k}
j∈C +
j∈I +
X
X
aj lj < b + ak
j∈C −
then the model is infeasible which implies it is valid to fix variable
xk to 0. If xk = 0, k ∈ I + and
X
j∈I + \{k}
aj hj +
X
j∈C +
aj hj −
X
j∈I −
aj lj −
X
aj lj < b
j∈C −
then the model is infeasible which implies it is valid to fix variable
xk to 1.
40 / 101
Preprocessing
How to preprocess in Coin-OR Cbc. See (www.coin-or.org) for
free optimization solvers.
Declare a new solver interface – this is what will get preprocessed.
OsiSolverInterface *m_OsiSolverPre = NULL;
CglPreProcess process;
m_OsiSolverPre = process.preProcess(*solver->osiSolver,
false, 10);
Build the Cbc model with the preprocessed solver interface and
solve
CbcModel *model = new CbcModel( *m_OsiSolverPre);
model->branchAndBound();
41 / 101
Preprocessing
Unwind to get the original model back
process.postProcess( *model->solver() );
Results with p0033.osil
Without
With
Variables
33
24
Constraints
16
13
LP Relax
2520.6
2905.7
Nodes
614
184
42 / 101
Cutting Planes – Motivation
Preprocessing is nice, but is usually far from sufficient!
The typical integer program requires more than just preprocessing
in order to solve.
The solution of most real problems requires the use of cutting
planes.
43 / 101
Cutting Planes – Motivation
We can tighten an integer program by adding cutting planes or
just cuts. These are sometimes also called valid inequalities.
Consider the integer program:
min −x1 − x2
x1 − x2 ≤ 2
4x1 + 9x2 ≤ 18
−2x1 + 4x2 ≤ 4
x1 , x2 ≥ 0
x1 , x2
∈ Z
44 / 101
Cutting Planes – Motivation
In the figure below the black dots represent Γ (feasible points to
the integer program) and the red area represents the linear
programming relaxation Γ.
45 / 101
Cutting Planes
We are going to generate cuts – like the blue line.
46 / 101
Cutting Planes – Motivation
We have already seen an example of cuts. Consider
Γ = {(x1 , x2 , y ) | x1 + x2 ≤ 2y , x1 , x2 , y ∈ {0, 1}}.
Γ = {(x1 , x2 , y ) | x1 + x2 ≤ 2y , 0 ≤ x1 , x2 , y ≤ 1}.
We would say that x1 ≤ y and x2 ≤ y are valid cuts of Γ.
These cuts do not cut off any points in Γ but they do cut off
points in Γ.
47 / 101
Cutting Planes – Motivation
The feasible points in
Γ = {(x1 , x2 , y ) | x1 + x2 ≤ 2y , x1 , x2 , y ∈ {0, 1}}.
are
y
0
1
1
1
1
x1
0
0
1
0
1
x2
0
0
0
1
1
All, yes, every one of these points satisfy the cuts x1 ≤ y and
x2 ≤ y .
However points (1, 0, .5) and (0, 1, .5) which are in Γ do not
satisfy the cuts.
48 / 101
Cutting Planes – Theory
Big Picture: There is a finite basis theorem for integer programs
just like there is for linear programs. That is,
Γ = {x ∈ <n | Bx ≥ d, x ≥ 0, xj ∈ Z, j ∈ I }
is finitely generated. This means
conv(Γ) = {x ∈ <n | x =
q
X
i=1
q
X
zi x i +
r
X
zi x i ,
i=q+1
zi = 1, zi ≥ 0, i = 1, . . . , r }
(10)
i=1
What is the geometry of conv(Γ)? What kind of set is it?
49 / 101
Cutting Planes – Theory
Obviously, conv(Γ) is a polyhedron!
This implies there exists a system of inequalities (αi )> x ≥ αi0 ,
i = 1, . . . , q such that conv(Γ) = {x | αi x ≥ αi0 , i = 1, . . . , q}.
50 / 101
Cutting Planes – Theory
We want to add cuts and try and pare down the light grey area to
the dark grey area.
x2
3
2.5
2
1.5
1
0.5
-1
1
2
3
4
x1
51 / 101
Cutting Planes
Big Picture: Since conv (Γ) is a polyhedron, we could solve the
integer program by solving a linear program!
min c > x
s.t.
αi x
≥ αi0 , i = 1, . . . , q
So why don’t we just solve the linear program?
Why bother with branch and bound?
52 / 101
Cutting Planes
Γ = {x ∈ <n | Bx ≥ d, x ≥ 0, xj ∈ Z, j ∈ I }
It is critical to understand the difference between CONV (Γ) and Γ
(the linear relaxation).
Γ = {(x1 , x2 , y ) | x1 + x2 ≤ 2y , x1 , x2 , y ∈ {0, 1}}.
What is
I
Γ?
I
CONV (Γ)?
I
Γ?
what is the relationship?
53 / 101
CONV (Γ) Versus Γ
54 / 101
Cutting Planes
In practice we solve the problem min{c T x | x ∈ Γ} and get a point
x.
If life were fair, we would get x ∈ CONV (Γ) and be done.
Unfortunately, fair is the place where they display animals in
summer and has absolutely nothing to do with the way life actually
works!!! So we try to find a cut α> x ≥ α0 so that
α> x ≥ α0 ,
∀x ∈ CONV (Γ)
/ CONV (Γ) then
and if x ∈ Γ but x ∈
α> x < α0
55 / 101
Cutting Planes
We now address the problem of finding cutting planes.
So far, all we have really done this quarter is add (aggregate)
constraints together!
So why does anyone think this is a tough course!
We keep doing the same thing, only now we do some rounding.
That’s all – just add a few constraints and round a few coefficients.
Pretty easy stuff I would say! I am embarrassed! Please do not go
around telling the rest of University how trivial the material is that
I teach.
56 / 101
Cutting Planes
n
X
aij xj
= bi ,
i = 1, . . . , m.
j=1
Assign a set of multipliers ui , i = 1, . . . , m to the constraints and
generate the aggregate constraint
n X
m
X
(
ui aij )xj
=
j=1 i=1
m
X
ui bi .
i=1
Assume all of the xj ≥ 0 and integer, then
n
m
X
X
d(
ui aij )exj
j=1
i=1
≥
m
X
ui bi .
i=1
57 / 101
Cutting Planes
Since we assume all variables must be integer, the left-hand-side of
the aggregate constraint below will always be integer.
n
m
X
X
d(
ui aij )exj
j=1
≥
i=1
m
X
ui bi .
i=1
Therefore we can round the right-hand-side up and get the cut
n
m
X
X
d(
ui aij )exj
j=1
i=1
≥ d
m
X
ui bi e.
i=1
This cut is a Chvátal-Gomory (C-G) cut and the procedure used
to derive the cut is called Chvátal-Gomory rounding. It is very
famous in integer programming.
58 / 101
Cutting Planes
A slight variation: If we have ≥ instead of =,
n
X
aij xj
≥ bi ,
i = 1, . . . , m.
j=1
Assign a set of multipliers ui ≥ 0, i = 1, . . . , m to the constraints
and generate the aggregate constraint
n X
m
X
(
ui aij )xj
j=1 i=1
≥
m
X
ui bi .
i=1
and we can follow the same derivation. Why is ui ≥ 0 required?
59 / 101
Cutting Planes
Example One: Consider
2x1 + 3x2 + 4x3 ≥ 6
x3 ≤ 1
This is equivalent to
2x1 + 3x2 + 4x3 ≥ 6
−x3 ≥ −1
Assign multipliers u1 =
1
3
and u2 = 43 . The aggregate constraint is
2
x1 + x2 ≥
3
2
3
60 / 101
Cutting Planes
Example One (Continued): We have
2
x1 + x2 ≥
3
2
3
Round the coefficient on x1 ,
x1 + x2 ≥
2
3
Now round the left-hand-side, and get the valid cut
x1 + x2 ≥ 1
61 / 101
Cutting Planes
Example Two: Let’s go back to our friend
x1 + x2 ≤ 2y
x1 ≥ 0
x2 ≥ 0
y
≤ 1
Substitute x1 = 1 − z1 and x2 = 1 − z2 . This becomes
(1 − z1 ) + (1 − z2 ) ≤ 2y
z1 ≤ 1
z2 ≤ 1
y
≤ 1
62 / 101
Cutting Planes
Example Two (continued): This becomes
z1 + z2 + 2y
≥ 2
−z1 ≥ −1
−z2 ≥ −1
−y
≥ −1
Assign multipliers u1 = .5, u2 = .5, u3 = 0, u4 = 0. This gives the
aggregate constraint,
.5z2 + y
≥ .5
63 / 101
Cutting Planes
Example Two (continued): Round the coefficient of z2 to 1
z2 + y
≥ .5
Then round the right hand side
z2 + y
≥ 1
Substitute back z2 = 1 − x2 , and voila!
x2 ≤ y
What multipliers give the cut x1 ≤ y ?
64 / 101
Cutting Planes
Knapsack Cover Cuts: Example 1 and Example 2 are illustrations
of the famous knapsack cover cuts.
a1 x1 + a2 x2 + · · · + an xn ≥ b
xi
∈ {0, 1},
i = 1, . . . , n
Assume all ai ≥ 0, for i = 1, . . . , n. The index set C is a cover if
and only if
P
I aj ≥ b −
for all j ∈ C
k6∈C ak ,
P
I b−
k6∈C ak > 0
The implied knapsack cover cut is
X
xj
≥ 1
j∈C
65 / 101
Cutting Planes
Knapsack Cover Cuts: The knapsack cover cuts have proved
exceedingly effective in integer programming.
Example 1 Revisited:
2x1 + 3x2 + 4x3 ≥ 6
x1 , x2 , x3
∈ {0, 1}
Define C = {1, 2}. Then
b−
X
ak = 6 − a3 = 6 − 4 = 2
k6∈C
Since a1 = 2 ≥ 2 and a2 = 3 ≥ 2 the corresponding knapsack
cover cut is
x1 + x2 ≥ 1
66 / 101
Cutting Planes
Example 2 Revisited: After the substitution we had
z1 + z2 + 2y
≥ 2
There are two covers (assume 3 in the index of y ), C1 = {1, 3} and
C = {2, 3} that lead to the cuts
z1 + y ≥ 1 and z2 + y ≥ 1
which correspond to
x1 ≤ y
and x2 ≥ y
67 / 101
Cutting Planes
We now derive a cut using the updated Simplex tableau. The
equations corresponding to a basic feasible solution indexed by B
are:
xBi +
X
aij xj
= bi ,
i = 1, . . . , m
(11)
j∈N
where Bi , i = 1, . . . , m, indexes the basic variables and N indexes
the non basic variables. Applying C-G rounding gives
xBi +
X
daij exj
≥ db i e,
i = 1, . . . , m.
(12)
j∈N
Multiplying (11) by -1 and adding to (12) gives
X
f ij xj
≥ f i0
i = 1, . . . , m
(13)
j∈N
where f ij = daij e − aij and f i0 = db i e − b i .
68 / 101
Cutting Planes
This tableau based cut is called a Gomory cut. Named after
Ralph Gomory a famous computing guru at IBM.
Note: You can work with the cut in either the form
xBi +
X
daij exj
≥ db i e,
i = 1, . . . , m.
j∈N
or
X
f ij xj
≥ f i0
i = 1, . . . , m
j∈N
From the viewpoint of re-optimizing the tableau, the second form
is preferred. From viewpoint of explanation and derivation the first
form is preferred.
69 / 101
Cutting Planes
Gomory Cut Example: Start with the integer program
min −x1 − x2
x1 − x2 ≤ 2
4x1 + 9x2 ≤ 18
−2x1 + 4x2 ≤ 4
x1 , x2 ≥ 0
x1 , x2
∈ Z
70 / 101
Cutting Planes
Put the linear programming relaxation in standard form.
Gomory Cut Example: Start with the integer program
min −x1 − x2
x1 − x2 + x3 = 2
4x1 + 9x2 + x4 = 18
−2x1 + 4x2 + x5 = 4
x1 , x2 ≥ 0
Optimize using our good friend basicsimplex.m. See the
MATLAB file gomory.m
71 / 101
Cutting Planes
If you run gomory.m you get the following tableau.
-0.00
-0.00
-0.38
-0.15
-0.00
-3.54
---------------------------------------------------------------1.00
0.00
0.69
0.08
0.00
2.77
0.00
1.00
-0.31
0.08
0.00
0.77
0.00
0.00
2.62
-0.15
1.00
6.46
Generate a cut based on the fractional variable x2 = 0.77
x2 − .31x3 + 0.08x4 = 0.77
Round up
x2 + x4 ≥ 1
72 / 101
Cutting Planes
We have:
x2 + x4 ≥ 1
Since
4x1 + 9x2 + x4 = 18
we have
x4 = 18 − 4x1 − 9x2
and the cut becomes
x2 + 18 − 4x1 − 9x2 ≥ 1
4x1 + 8x2 ≤ 17
73 / 101
Cutting Planes
We can even tighten the Gomory cut!
4x1 + 8x2 ≤ 17
implies,
x1 + 2x2 ≤ 17/4 = 4
1
4
which implies
x1 + 2x2 ≤ 4
I love this stuff! Verify that this is valid.
74 / 101
Cutting Planes
Here is the cut x1 + 2x2 ≤ 4 plotted with the feasible region.
75 / 101
Cutting Planes
By generating the necessary cutting planes we could solve the
integer program as a linear program! The blue region represents
the convex hull of integer solutions.
76 / 101
Cutting Planes
Questions to Ponder:
I
If there is a fractional solution, will there be a knapsack cover
cut, that cuts this solution off?
I
If there is a fractional solution, will there be a Gomory cut ,
that cuts this solution off?
By the way, would you prefer to use Gomory or knapsack cover
cuts? Why?
77 / 101
Cutting Planes
Test problem P0458 with 177 rows and 548 variables.
No Cuts
Gomory + Knapsack
LP Value
591
8249
IP Value
8691
8691
Nodes
34826
154
Iterations
68343
7242
78 / 101
Cutting Planes
We have developed knapsack cover cuts and Gomory cuts. The
generation of valid cuts has been the subject of a huge amount of
research.
People have developed cuts for very special and for very general
structures.
See (www.coin-or.org) for free optimization solvers.
See the GAMS document coin.pdf (in folder docs/solvers), pages
22-23 for a list of cuts supported by the COIN-OR solver Cbc.
Most solvers have options for turning these various cuts on and off.
79 / 101
Cutting Planes
For COIN-OR Cbc
I
cuts off – turns all cut generation off
I
cuts on – turns on all the default cut generators (see
coin.pdf for default cut generators)
I
you can set cuts off and then turn on specific cuts
cuts off
gomorycuts on
knapsackcuts on
flowcovercuts on
80 / 101
Constraint Aggregation Coefficient Reduction Cuts
The C-G cut assumes all integer. However, if continuous variables
are present, then it is not valid to conclude that
n X
m
m
X
X
d
ui aij exj ≥
ui bi
j=1 i=1
i=1
→
n X
m
m
X
X
d
ui aij exj ≥ d
ui bi e.
j=1 i=1
i=1
In order to treat mixed-integer programs (i.e. some continuous
variables present) we use a technique developed by Martin and
Schrage.
It is very simple and just adds constraints and rounds variable
coefficients.
Despite the simplicity, it did help me earn tenure!
81 / 101
Constraint Aggregation Coefficient Reduction Cuts
Consider the pure integer case first.
n
X
aij xj
= bi ,
i = 1, . . . , m
j=1
Assign a set of multipliers ui , i = 1, . . . , m to these constraints and
generate the aggregate constraint
n X
m
X
(
ui aij )xj
=
j=1 i=1
m
X
ui bi .
i=1
If all of the variables xj are required to be nonnegative, then a valid
cut based on the aggregate constraint is
n
X
m
X
max {0, (
ui aij )}xj
j=1
i=1
≥
m
X
ui bi .
i=1
82 / 101
Constraint Aggregation Coefficient Reduction Cuts
We have:
n
X
m
X
max {0, (
ui aij )}xj
j=1
i=1
≥
m
X
ui bi .
i=1
P
If every xj is a nonnegative integer variable and if m
i=1 ui bi > 0,
then a valid constraint aggregation, coefficient reduction (CACR)
cut is
n
X
j=1
m
m
X
X
min (
ui bi , max {0, (
ui aij )})xj
i=1
i=1
≥
m
X
ui bi .
i=1
83 / 101
Constraint Aggregation Coefficient Reduction Cuts
The (CACR) cut has as a nice special case, pure 0-1 programming.
Any inequality in a pure 0/1 linear program can be written as
X
j∈I +
aj x j −
X
aj xj ≥ b
(14)
j∈I −
where aj > 0 for j ∈ I + ∪ I − and xj ∈ {0, 1} for all j ∈ I + ∪ I − .
Since xj is a binary variable the simple upper bound constraints
xj ≤ 1 for all j ∈ W ⊂ I + are valid side constraints.
Assign a multiplier of −aj to all of the side constraints xj ≤ 1 for
j ∈ W and a multiplier of +1 to the constraint (14).
84 / 101
Constraint Aggregation Coefficient Reduction Cuts
The aggregate constraint is
X
aj x j −
P
j∈W
aj xj ≥ λ := b −
j∈I −
j∈I + \W
If λ = b −
X
X
aj .
j∈W
aj > 0, then a valid (CACR) cut is
X
min{λ, aj }xj ≥ λ.
(15)
j∈I + \W
85 / 101
Constraint Aggregation Coefficient Reduction Cuts
Example: Generalized Assignment Problem
MIN
2 X11 + 11 X12
2 X32 + 5 X41 + 5 X42
SUBJECT TO
2)
X11 + X12
3)
X21 + X22
4)
X31 + X32
5)
X41 + X42
6)
3 X11 + 6
7)
2 X12 + 4
END
+ 7 X21 + 7 X22 + 20 X31 +
=
1
=
1
=
1
=
1
X21 + 5 X31 + 7 X41 <=
X22 + 10 X32 + 4 X42 <=
13
10
86 / 101
Constraint Aggregation Coefficient Reduction Cuts
Example: Generalize Assignment Problem (continued)
OBJECTIVE FUNCTION VALUE
1)
VARIABLE
X11
X21
X31
X32
X41
X42
20.32000
VALUE
1.000000
1.000000
.240000
.760000
.400000
.600000
REDUCED COST
.000000
.000000
.000000
.000000
.000000
.000000
Generate a cut.
87 / 101
Constraint Aggregation Coefficient Reduction Cuts
Example: Generalize Assignment Problem (continued)
There are two knapsack constraints. Convert the knapsack
constraints to canonical form.
Convert to canonical form by complementing the variables
zij = 1 − xij and then multiplying by -1. In canonical the knapsack
constraints become.
3z11 + 6z21 + 5z31 + 7z41 ≥ 8
2z12 + 4z22 + 10z32 + 4z42 ≥ 10
A minimal cover cut is z11 + z21 + z41 ≥ 1 which is
x11 + x21 + x41 ≤ 2 in the original variables. The current linear
programming relaxation violates this cut. Similarly, a minimal
cover cut for the second machine is x32 + x42 ≤ 1.
88 / 101
Constraint Aggregation Coefficient Reduction Cuts
Example: Generalize Assignment Problem (continued) Add these
cuts to the linear program and resolve.
OBJECTIVE FUNCTION VALUE
1)
VARIABLE
X11
X22
X31
X32
X41
23.20000
VALUE
1.000000
1.000000
.400000
.600000
1.000000
REDUCED COST
.000000
.000000
.000000
.000000
.000000
Are there any CACR cuts? See pages 546-549 of the textbook.
89 / 101
Constraint Aggregation Coefficient Reduction Cuts
Example: Lock box model (simple plant location). Consider a
constraint of the form
x1 + x2 ≤ 2y
Write this as
2y − x1 − x2 ≥ 0
Substitute, x1 = 1 − z1 , x2 = 1 − z2 and get the system
2y + z1 + z2 ≥ 2
−z1 ≥ −1
−z2 ≥ −1
90 / 101
Constraint Aggregation Coefficient Reduction Cuts
Example: Lock box model (simple plant location).
Aggregating the first and second constraint gives
2y + z2 ≥ 1
Coefficient reduction gives
y + z2 ≥ 1
which gives
x1 ≤ y
Similarly we get x2 ≤ y . In general, CACR applied to
n
X
xi ≤ My
i=1
gives,
xi ≤ y ,
∀i = 1, . . . , n
91 / 101
Constraint Aggregation Coefficient Reduction Cuts
CACR is trivially extended to the mixed-integer case.
If the integer variables are indexed by I and the continuous
variables by C , then after aggregating constraints, it follows that a
valid CACR cut is
X
j∈I
m
m
X
X
min (
ui bi , max {0, (
ui aij )})xj +
i=1
i=1
X
m
X
max {0, (
ui aij )}xj
j∈C
i=1
≥
m
X
ui bi . (16)
i=1
92 / 101
Constraint Aggregation Coefficient Reduction Cuts
Example: Dynamic lot sizing. Consider the constraint set
It−1 + xt − It
= dt
−xt + Myt
≥ 0
Assign a multiplier of 1 to the first constraint and 1 to the second
constraint. The aggregate constraint is
It−1 + Myt − It
≥ dt
Applying coefficient reduction gives
It−1 + dt yt ≥ dt
This is equivalent to xt ≤ It + dt yt .
93 / 101
Separation
Obviously, it is neither efficient nor possible to add all of the CACR
cuts.
We add them on-the-fly as needed.
We discover which CACR cuts to add on-the-fly using separation.
Separation Problem: “given x ∈ <n , is x ∈ conv(Γ), and if not,
find a cut α> x ≥ α0 such that α> x < α0 and α> x ≥ α0 for all
x ∈ conv(Γ).”
This is a huge idea in integer programming. It effectively allows
real problems to be solved!
94 / 101
Separation
Consider a special case – finding a CACR cut for the knapsack
problem. That is,
X
aj x j
≥ b
−xj
≥ −1,
j∈I +
j = 1, . . . , n
Assumptions:
I
We apply a multiplier of +1 to the knapsack constraint
I
We apply a multiplier of aj to constraint −xj ≥ −1
Given x ≥ 0, finding
P the “most violated cut” implies finding a
W ⊂ I+ P
so that j∈I + \W min{λ, aj }x j − λ is minimized and
λ = b − j∈W aj is strictly positive.
95 / 101
Separation
This is done by solving a binary knapsack problem. Let rj P
= 1 if
+
j ∈ I \W and 0 otherwise. Instead
P of requiring λ = b − j∈W aj ,
(15) is valid as long as λ ≤ b − j∈W aj . Then the separation
knapsack problem is
X
min{λ, aj }x j rj − λ
min
j∈I +
(KSEP)
s.t.
b−
X
aj +
j∈I +
X
aj rj ≥ λ
j∈I +
rj ∈ {0, 1}, j ∈ I + .
Note:
b−
X
j∈I +
aj +
X
j∈I +
aj rj = b −
X
aj
j∈W
96 / 101
Separation
Example: (Generalized Assignment Part Deux)
VARIABLE
X11
X21
X31
X32
X41
X42
VALUE
1.000000
1.000000
.240000
.760000
.400000
.600000
REDUCED COST
.000000
.000000
.000000
.000000
.000000
.000000
The constraint
3 X11 + 6 X21 + 5 X31 + 7 X41 <=
13
becomes
3 Z11 + 6 Z21 + 5 Z31 + 7 Z41 >=
8
97 / 101
Separation
Example: (Generalized Assignment Part Deux)
The separation knapsack with λ = 1 is
min 0r11 + 0r21 + .76r31 + .6r41 − 1
(KSEP)
s.t.
3r11 + 6r21 + 5r31 + 7r41 ≥ 14
r11 , r21 , r31 , r41
∈
{0, 1}
The optimal solution is r11 = 1, r21 = 1, r31 = 0 and r41 = 1.
The implied minimal cover cut is z11 + z21 + z41 ≥ 1 which is
x11 + x21 + x41 ≤ 2 in the original variables.
98 / 101
Branch and Cut
99 / 101
COIN-OR Cgl
COIN-OR Cgl (Cut generation library) Add cuts of various
types to Cbc
Basic idea – each type of cut is a class that we define in our code.
CglKnapsackCover cover;
CglSimpleRounding round;
CglGomory gomory;
add each cut to the model and solve.
model->addCutGenerator(&cover, 1, "Cover");
model->addCutGenerator(&round, 1, "Round");
model->addCutGenerator(&gomory, 1, "Gomory");
100 / 101
COIN-OR
COIN-OR: No size limitations. No license issue.
Two ways to build a model.
1. Use Modeling Language to call a solver.
2. Write a matrix generator and interact directly with solver.
101 / 101