AM 121: Intro to Optimization
Models and Methods
Fall 2016
Lecture 16: More cuts, Branch and
Cut, other tricks…
David C. Parkes
SEAS
Lesson Plan
•
•
•
•
Review Chvatal-Gomory cuts
Cover cuts
Branch-and-cut
Other tricks
– Preprocessing
– Primal heuristics
– SOS branching
Jensen & Bard: 8.5
1
• Definition. An inequality aTx ≤ b is valid for set X✓ Rn if
aTx ≤ b for all x2X.
• Defn. A cut is a valid inequality that separates the
current fractional solution x*.
fractional solution
members of set
X
valid, a cut
not valid
valid, not a cut
Chvatal-Gomory inequalities
• Integer, non-negative decision variables
• Consider X = P \ Z2, where P is given by
7x1 -2x2 ≤ 14
x2
≤3
2x1-2x2 ≤ 3
x≥0
• Valid to combine with non-neg weights u=(2/7, 37/63, 0):
2x1 + 1/63x2 ≤ 121/21
• Valid to use non-neg of x to round coefficients on LHS down
to nearest integer:
2x1 +
0x2 ≤ 121/21
• Valid to use integrality of LHS to round down RHS:
2x1
≤5
2
Example (Gomory’s algorithm)
• In addition to integer, non-negative decision
variables, needs integer coefficients and
integer RHs.
• Consider the IP
z = max 4x1 – x2
7x1 -2x2 ≤ 14
x2 ≤ 3
2x1-2x2 ≤ 3
x1, x2 ≥ 0, integer
• For row i with fractional RHS, the CG cut is
åj2B’ (aij- aij ) xj ≥ bi- bi
• z
+ 4/7 x3 + 1/7 x4
x1
+ 1/7x3 + 2/7 x4
x2
+ x4
-2/7 x3 + 10/7x4 + x5
• B={1, 2, 5}
= 59/7
= 20/7
=3
= 23/7
3
• For row i with fractional RHS, the CG cut is
åj2B’ (aij- aij ) xj ≥ bi- bi
• z
+ 4/7 x3 + 1/7 x4
x1
+ 1/7x3 + 2/7 x4
x2
+ x4
-2/7 x3 + 10/7x4 + x5
• B={1, 2, 5}
• Add cut 1/7x3 + 2/7x4 ≥ 6/7
• Add excess variable x6
= 59/7
= 20/7
=3
= 23/7
• Re-optimize. New optimal tableau:
z
+½x5 +3x6 = 15/2
x1
+x6 = 2
x2
–½x5 +x6 = ½
x3
–x5 – 5x6 = 1
x4 +½x5 + 6x6 = 5/2
• B={1,2,3,4} and x*=(2,1/2,1, 5/2, 0, 0)
• Add cut ½x5 ≥ ½
• Add excess variable x7
4
• Re-optimize. New optimal tableau:
z
+3x6 + x7 = 7
x1
+x6
= 2
x2
+x6 – x7 = 1
x3
–5x6 – 2x7 = 2
x4
+6x6 + x7 = 2
x5
– x7 = 1
• x*=(2, 1, 2, 2, 1, 0, 0)
x1≤2
x2
(20/7,3)
x2≤ 3
3
7x1-2x2≤14
2
2x1-2x2≤3
(2,1)
1
(2,1/2)
objective
0
0 x1-x2≤1
1
2
3 x1
5
Cover inequalities
• Defined for problems in which the feasible space is
a “0-1 Knapsack set”. Example:
• X={x2{0,1}7: 11x1+6x2+6x3+5x4+5x5+4x6+x7 ≤ 19}
0/1 variables
positive coefficients
positive RHS
• In general, we allow binary IPs with multiple such
“knapsack rows” but throughout our discussion we
will assume a single such row.
Cover inequalities
• Defined for problems in which the feasible space is
a “0-1 Knapsack set”. Example:
• X={x2{0,1}7: 11x1+6x2+6x3+5x4+5x5+4x6+x7 ≤ 19}
0/1 variables
positive coefficients
positive RHS
• Example cover inequalities for X are:
Using {0,1} property!
x1 + x2 +x3
≤2
x1 + x2
+ x6 ≤ 2
x1
+ x5 + x6 ≤ 2
x3 + x4 + x5 + x6 ≤ 3
6
Cover inequalities
• For X = {x2 {0,1}n : åjajxj ≤ b}, with aj ≥ 0, b ≥ 0.
• N={1,…,n}
• Defn. A set C✓ N is a cover if åj2C aj > b. A cover is
minimal if C \ {j} is not a cover for any j2C.
• Proposition. If C is a cover, the cover inequality
åj2C xj ≤ |C|-1 is valid.
• Proof. Consider x 2 {0,1}n with åj 2C xj = |C|. Have
åj ajxj ≥ åj 2C ajxj =åj 2C aj > b. Not in X.
Any x that violates cover is infeasible, not in X. So,
cover inequality is valid.
Back to the examples
• X={x2{0,1}7: 11x1+6x2+6x3+5x4+5x5+4x6+x7 ≤ 19}
0/1 variables
positive coefficients
positive RHS
• Example cover inequalities for X are:
C={1,2,3}: x1 + x2 +x3 ≤ |C|-1=2
C={1,2,6}: x1 + x2 + x6 ≤ |C|-1=2
7
Review: Cover inequalities
• For X = {x2 {0,1}n : åjajxj ≤ b}, with aj ≥ 0, b ≥ 0.
• Given a cover set C, the cover inequality is
åj2C xj ≤ |C|-1.
• Equivalently, can write a cover inequality as
åj2C (1 – xj) ≥ 1
• A cover set C has the property that åj2 C aj > b.
How can we use Cover
inequalities?
8
• Definition. An inequality aTx ≤ b is valid for set X✓ Rn if
aTx ≤ b for all x2X.
• Defn. A cut is a valid inequality that separates the
current fractional solution x*.
fractional solution
members of set
X
valid, a cut
not valid
valid, not a cut
Separation with Cover inequalities
• Can write cover inequality as
(“at least one not used”)
åj2C (1 – xj) ≥ 1
• Given frac x*, find valid C with åj 2C(1-x*j)<1
• Formulate an IP! Let zj denote whether j 2 C.
β= minåj 2N (1-x*j)zj
(valid cover)
s.t.
åj2N aj zj > b
z 2{0,1}n
• If β≥1, then x* satisfies all cover inequalities.
• If β<1, then we find a cut.
• Note: this IP is nonstandard (strict inequality.) But can
replace with åj2N aj zj ≥ b+1 when coeffs are integral.
9
Example use of a Cover Inequality
• X={x 2{0,1}6:45x1+46x2+79x3+54x4+53x5+125x6≤178},
fractional x*=(0, 0, 3/4, 1/2, 1, 0)
• Solve:
β= min z1 + z2 + ¼ z3 + ½ z4 + 0z5 + z6
s.t. 45z1+46z2+79z3+54z4+53z5+125z6>178
z 2{0,1}6
• z*=(0,0,1,1,1,0) with β=3/4
• Conclude that x3 + x4 + x5 ≤ 2 is a cut for x*.
• Why is this useful? Because we can try to solve the “auxiliary
problem” heuristically; give up if too hard.
Two Variations on Cover Inequalities
• Extended Cover Inequalities
• Lifted Cover Inequalities
10
Extended Cover Inequalities
•
•
•
•
X={x2{0,1}7: 11x1+6x2+6x3+5x4+5x5+4x6+x7≤19}
Cover inequality: x3 + x4 + x5 + x6 ≤ 3
Extended cover ineq: x1 + x2 + x3 + x4 + x5 + x6 ≤ 3
Still valid. Why? Any four variables have sum
coefficients > 19. (And so x 2 X must satisfy)
• Rule: Can extend cover set C to extended cover
set E(C), where
E(C)= C [ { j : aj ≥ ai for all i2C}.
• Extended cover inequality: åj2E(C) xj ≤ |C|-1 is valid,
and stronger than cover inequality.
Two Variations on Cover Inequalities
• Extended Cover Inequalities
• Lifted Cover Inequalities
11
Lifted Cover Inequalities
• X={x2{0,1}7: 11x1+6x2+6x3+5x4+5x5+4x6+x7≤19}
• Lifting: Given x2+x3+x4+x5+x6≤3, find a valid
inequality α1x1+x2+x3+x4+x5+x6≤3, for some α1>0.
• Valid for any soln with x1=0.
• What about solns with x1=1? Well, any x2X with
x1=1 satisfies: 6x2+6x3+5x4+5x5+4x6+x7≤8 (*)
• Idea: pick maximum α1 s.t. α1+x2+x3+x4+x5+x6≤3 for
all integer solutions satisfying (*)
• Set α1= 3 – maxx {x2+x3+x4+x5+x6:
6x2+6x3+5x4+5x5+4x6+x7≤8,
x 2{0,1}6 } = 3 – 1 = 2
• Lifted cover inequal: 2x1+x2+x3+x4+x5+x6 ≤ 3
• Theorem. Lifted cover inequalities are “facetdefining” for conv(X) when X is {0,1}-knapsack set.
• (Informal) An inequality is facet defining if it
defines the face of a polyhedron. Consider:
Not facet defining
Facet defining
Facet defining
Facet defining
12
Lifted Cover Inequalities
• Complete for IPs with constraints that are 0-1
knapsack sets.
• But, rather than use them to construct the
convex hull we will do something else…
• Let’s see this next. How to use lifted cover
inequalities and other approaches?
• We’ve seen two families of cuts:
– Chvatal-Gomory cuts for IPs with integer
coefficients and integer RHS values
– (Lifted, extended) cover cuts for IPs with {0,1}
vars and strictly positive coeffs and strictly
positive RHS values
• There are many others:
– Flow cover
– Mixed integer rounding
– Odd hole
–…
• How can we use these?
13
The Branch-and-Cut method
• Generalizes Branch-and-Bound.
• If cannot prune a node, can try to find a cut.
• Strengthen the formulation; re-solve the LP,
may now be able to prune by bound.
• If no cuts are found, then branch.
The Branch-and-Cut method
• Branch-and-Bound + cuts generated
throughout the tree.
• Cuts generated at root are “global,” cuts at
subproblems “local.” Place into a “cut pool.”
• Goal: reduce the number of search nodes by
improving bounds
Cuts valid (global cuts)
X1 ≤ 0
X1 ≥ 1
Cuts valid
(local cuts)
14
Example Application
• Winner determination problem in an auction,
e.g. selling lots of wine.
• 10 bidders, 5 different wines (≈100 cases
each).
• Bidders wants either wine 1, or wine 2, or
wine 3…
• Goal: maximize total revenue
Bidders
1
2
3
demand
value
demand
value
demand
value
Supply
Goods
A
B
C
91 1
21
110 16 25
54 53 …
65 69 …
3
87 …
19 93 …
91
87
D
…
…
109 88
E
64
15
Winner Determination Problem
• Bidders i, goods j with supply bj
• cij value to i for aij units of good j
• xij=1 if select bid from i on j as a winner
max åi åj cij xij
(Generalized Assignment)
s.t. åj xij ≤ 1 for all i = 1,…,m
åi aijxij ≤ bj for all j = 1,…,n
xij2 {0,1}
Has “0/1 knapsack set” structure. Can use Cover cuts.
Example Search Tree
554.1
14 lifted cover cuts generated.
LP bound improves from
595.6 to 554.1
S1
x73≤0
x73≥1
1 cut generated
548.9
537
S3
x13≤0
S2
x13≥1
< 545
545
545
S4
0 cuts. LP integral;
Fathom by integrality.
3 cuts generated. LP
537 integral. Fathom by
S5
0 cuts. Fathom by
bound.
integrality.
BnB without cuts requires
4206 search nodes.
Branch-and-cut: With even
more cut generation at root,
bound improves to 546 and
tree has 3 nodes!
16
Historical Context
• Use of cutting planes as a practical tool for solving
IPs was abandoned in 1960s and ‘70s!!!
• Revisited in early‘70s following NP-completeness
theory (interest in problem structure and facets.)
• Balas, Wolsey, Johnson, Padberg developed cover
inequalities and “lifting” in mid‘70s.
• Branch-and-cut developed in early ‘80s and
introduced into commercial solvers in ‘00s.
• Provides order of magnitude improvement in
solve times.
Other important components
• Preprocessing
• Primal heuristics
• SOS branching
17
Preprocessing
• Detect and eliminate redundant constraints
and variables, tighten bounds where possible
Preproc.: Isolating and Fixing
(Wolsey pp.104)
• Consider LP:
• max 2x1 + x2 – x3
5 x1 – 2x2 + 8x3 ≤ 15 (a)
8 x1 + 3x2 – x3 ≥ 9
(b)
x1 + x2 + x3 ≤ 6
(c)
0 ≤ x1 ≤ 3; 0 ≤ x2 ≤ 1; 1 ≤ x3
• Isolate x1 in (a) to obtain 5x1 ≤ 15+2x2 -8x3 ≤ 15+2-8=9, and
so x1 ≤ 9/5.
• Isolate x3 in (a) to obtain x3 ≤ 17/8. Not useful to isolate x2.
• Can also isolate x1 in (b); obtain x1 ≥ 7/8.
• Repeat: improve bound on x3 from (a) by using x1 ≥ 7/8;
obtain x3 ≤ 101/64
• Find constraint (c) is redundant, using x1 ≤ 9/5 and
x3 ≤ 101/64, together with x2 ≤ 1.
18
“Fixing” values of variables
The modified LP is:
• max 2x1 + x2 – x3
s.t. 5x1 – 2x2 + 8x3 ≤ 15
8x1 + 3x2 – x3 ≥ 9
7/8≤ x1≤ 9/5; 0 ≤ x2 ≤1; 1 ≤ x3 ≤ 101/64
• Fixing: In optimal solution, have x2 = 1; and x3 = 1.
• Why? x2 has positive coeff in obj, and increasing makes
constraints less tight. Similarly, best to decrease x3.
• Obtain:
max{2x1 : 7/8 ≤ x1 ≤ 9/5}. Easy to solve!
Other important components
• Preprocessing
• Primal heuristics
• SOS branching
19
Primal heuristics
• At a fractional search node, use a heuristic
approach to try to find an integral solution
• For example, round the solution or do a
directed “neighborhood search.”
feasible?
(1, 1, 1, 0)
(0, 0, 1, 0)
feasible?
(0.3, 0.7, 1, 0)
(1, 0, 1, 0)
feasible?
• May be able to improve incumbent.
Other important components
• Preprocessing
• Primal heuristics
• SOS branching
20
SOS branching
n
• Consider constraints åj=1 xj=1, xj2{0,1}
• Typical to branch on fractional xk* :
unbalanced
xk≤ 0
n-1 solutions
xk ≥ 1
1 solution
• “Special-ordered sets” branching. Pick
smallest k s.t. åkj=1 xj* ≥ ½
balanced
k
åj=1 xj ≤ 0
k
åj=1 xj ≥ 1
Example: x*=(0,0.2,0.4,0,0.4). Branch x1 +x2 +x3 ≤ 0, x1 +x2 +x3 ≥ 1
Summary: Advanced IP solving
• MIPs are a lot harder to solve than LPs!
• But IP technology is sophisticated:
– Dual simplex pivots
– Tight formulations
– Automated cut generation
– Primal heuristics
– Node and variable selection heuristics
• Problems with 100,000s variables and
10,000s constraints can be solved in
commercially viable times.
21