Branch-Cut-and-Price
Branch-Cut-and-Price: Otimalidade e Soluções
Quasi-Ótimas
Marcus Poggi
Departamento de Informática, PUC-Rio, Brazil
[email protected]
February 8, 2012
Branch-Cut-and-Price
Outlines
1
History and Basics
Column Generation
Integer Programming with Column Generation
Branch-Cut-and-Price
History and Basics
Column Generation
Why Column Generation ?
Linear Programming × Earlier Computers
Computers with Small Memory Space
Revised Simplex Algorithm to Solve LPs
n variables - m constraints
Branch-Cut-and-Price
History and Basics
Column Generation
Why Column Generation ?
Linear Programming × Earlier Computers
Computers with Small Memory Space
Revised Simplex Algorithm to Solve LPs
n variables - m constraints
One Pivot Time Complexity: O(m2 )
One Pivot Space Requirement: O(m2 )
Branch-Cut-and-Price
History and Basics
Column Generation
Why Column Generation ?
Linear Programming × Earlier Computers
Computers with Small Memory Space
Revised Simplex Algorithm to Solve LPs
n variables - m constraints
One Pivot Time Complexity: O(m2 )
One Pivot Space Requirement: O(m2 )
Basis Factorization: O(m3 )
Pricing Time Complexity: O(m.n)
Pricing Space Requirement: O(m)
Branch-Cut-and-Price
History and Basics
Column Generation
Why Column Generation ?
Linear Programming × Earlier Computers
Computers with Small Memory Space
Revised Simplex Algorithm to Solve LPs
n variables - m constraints
One Pivot Time Complexity: O(m2 )
One Pivot Space Requirement: O(m2 )
Basis Factorization: O(m3 )
Pricing Time Complexity: O(m.n)
Pricing Space Requirement: O(m)
To solve large LPs, reduce # constraints is a MUST!
Can’t fit too many constraints into the calculator.
If the problems fits, we can wait! :-)
Branch-Cut-and-Price
History and Basics
Column Generation
Problem: Multi-Commodity Network Flow
max
c 1 .x 1 +
I1 .x 1 +
N1 .x 1
c 2 .x 2 +
I2 .x 2 +
...
...
c K .x K
IK .x K
N2 .x 2
≤
=
=
u
b1
b2
=
0
bK
...
x1 , x2 ,
. . . , xK
NK .x K
≥
K commodities
Commodities share the network arcs
Arc capacity is shared (coupling constraints)
Branch-Cut-and-Price
History and Basics
Column Generation
Problem: Multi-Commodity Network Flow
max
c 1 .x 1 +
I1 .x 1 +
N1 .x 1
c 2 .x 2 +
I2 .x 2 +
...
...
c K .x K
IK .x K
N2 .x 2
≤
=
=
u
b1
b2
=
0
bK
...
x1 , x2 ,
. . . , xK
NK .x K
≥
K commodities
Commodities share the network arcs
Arc capacity is shared (coupling constraints)
Many commodities =⇒ Large LP
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
Title: A Suggested Computation for Maximal
Multi-Commodity Network Flows
Formulates each commodity flow from sources to sinks:
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
Title: A Suggested Computation for Maximal
Multi-Commodity Network Flows
Formulates each commodity flow from sources to sinks:
by enumerating all possible paths (chains) connecting them
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
Title: A Suggested Computation for Maximal
Multi-Commodity Network Flows
Formulates each commodity flow from sources to sinks:
by enumerating all possible paths (chains) connecting them
associating variables to flows through these paths
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
Title: A Suggested Computation for Maximal
Multi-Commodity Network Flows
Formulates each commodity flow from sources to sinks:
by enumerating all possible paths (chains) connecting them
associating variables to flows through these paths
Suggests pricing by computing shortest paths via
transshipment problem.
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
Title: A Suggested Computation for Maximal
Multi-Commodity Network Flows
Formulates each commodity flow from sources to sinks:
by enumerating all possible paths (chains) connecting them
associating variables to flows through these paths
Suggests pricing by computing shortest paths via
transshipment problem.
This was before Dijkstra’s algorithm (1959)
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
Title: A Suggested Computation for Maximal
Multi-Commodity Network Flows
Formulates each commodity flow from sources to sinks:
by enumerating all possible paths (chains) connecting them
associating variables to flows through these paths
Suggests pricing by computing shortest paths via
transshipment problem.
This was before Dijkstra’s algorithm (1959)
Column Generation!
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
How does it works ?
Assume there are two commodities.
Let Sk = {γ1k , γ2k , . . . , γpkk } be the set of all possible paths
joining a source and a sink of commodity k in the network.
Now let γrk is an incidence vector, each element is associated
to an arc in the network.
The FF formulation reads:
max
p1
P
r =1
p1
P
r =1
1
f ,
cr1 .fr1 +
γr1 .fr1 +
p2
P
r =1
p2
P
r =1
2
f
cr2 .fr2
γr2 .fr2
≤
u
≥
0
Column Generation is just solving this LP by the Revised
Simplex Method without listing all possible paths.
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
How does it works ?
Column Generation is just solving this LP by the Revised
Simplex Method without listing all possible paths:
Assume you have a (current) feasible basis. Let π the vector
of dual variables associated to this basis.
Note that π = {πa , a ∈ A}, with A being the arc set of the
network.
k
Likewise γrk = {γar
, a ∈ A}
The reduced cost of a variable frk is then: c kr = crk − π.γrk
Letting crk be constant, 1 in FF58, there is an improvement
path when there is c kr = 1 − π.γrk greater than zero.
I.e., there is a path from source to sink for some commodity
whose sum of the dual variables associated to its arcs is
smaller than one.
Branch-Cut-and-Price
History and Basics
Column Generation
Ford and Fulkerson Man. Sci. 1958, rec 1957
How does it works ?
Column Generation is just solving this LP by the Revised
Simplex Method without listing all possible paths:
Assume you have a (current) feasible basis. Let π the vector
of dual variables associated to this basis.
Note that π = {πa , a ∈ A}, with A being the arc set of the
network.
k
Likewise γrk = {γar
, a ∈ A}
The reduced cost of a variable frk is then: c kr = crk − π.γrk
Letting crk be constant, 1 in FF58, there is an improvement
path when there is c kr = 1 − π.γrk greater than zero.
I.e., there is a path from source to sink for some commodity
whose sum of the dual variables associated to its arcs is
smaller than one.
The Column Generation Pricing subproblem is then finding
the Shortest Path from a source to a sink where distances are
the dual variables associated to the arcs.
Branch-Cut-and-Price
History and Basics
Column Generation
A More General Approach: Dantzig-Wolfe Decomposition
c 1 .x 1 +
A1 .x 1 +
D1 .x 1
min
x1 ,
c 2 .x 2
A2 .x 2
D2 .x 2
x2
=
≤
≤
≥
b
d1
d2
0
Let Pk = {x|Dk .x ≤ dk }
Assume Pk is closed.
Let S k = {x1k , x2k , . . . , xpk } be the set of Pk extreme points.
We can write Pk as the convex combination of Sk :
Pk : {x|x =
p
X
r =1
xrk .λr ,
p
X
r =1
λr = 1, λr ≥ 0}
Branch-Cut-and-Price
History and Basics
Column Generation
A More General Approach: Dantzig-Wolfe Decomposition
Each λkj is associated to an extreme point of Pk
min
p
P
r =1
p
P
r =1
p
P
c 1 .xr1 .λ1r +
A1 .xr1 .λ1r +
p
P
r =1
p
P
c 2 .xr2 .λ2r
A2 .xr2 .λ1r
=
b
(π)
=
1
(γ1 )
=
1
(γ2 )
≥
0
r =1
λ1r
r =1
1
λ ,
p
P
r =1
2
λ
λ2r
Let Pk = {x|Dk .x ≤ dk }
Assume Pk is closed.
Let S k = {x1k , x2k , . . . , xpk } be the set of Pk extreme points.
Branch-Cut-and-Price
History and Basics
Column Generation
A More General Approach: Dantzig-Wolfe Decomposition
Let π, γ1 and γ2 be the dual variables, as above.
The Column Generation Subproblem is then:
min(c k − π.Ak ).x − γk
Dk .x ≤ dk
x ≤0
Branch-Cut-and-Price
History and Basics
Column Generation
To Remember
Formulation quality was measured by number of variables and
constrains!
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Main Resolution Method (until today):
LP Based Branch-and-Bound
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Main Resolution Method (until today):
LP Based Branch-and-Bound
P0: x1=1.67 x2=3.33
Z=68.33
x1  2
x1  1
P2: x1=2 x2=2.86
Z=68.29
P1: x1=1 x2=4
Z=65
x2  2
x2  3
P3: x1=2.6 x2=2
Z=68.2
5
x1  3
x1  2
P4: x1=2 x2=2
Z=58
4
P5: x1=3 x2=1.43
Z=68.14
3
x2  1
P6: x1=3.3 x2=1
Z=68.1
2
1
P6
1
2
3
4
5
x2  2
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Improvements to reach state-of-the-art:
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Improvements to reach state-of-the-art:
Better LP solving. Faster. Much faster.
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Improvements to reach state-of-the-art:
Better LP solving. Faster. Much faster.
Better Bounds.
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Improvements to reach state-of-the-art:
Better LP solving. Faster. Much faster.
Better Bounds.
Heuristics (everywhere).
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Improvements to reach state-of-the-art:
Better LP solving. Faster. Much faster.
Better Bounds ⇐=
Heuristics (everywhere).
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Better Bounds:
Cuts. (Polyhedral Theory, Lattice Free cuts, ...)
Decomposition - Column Generation, Lagrangian Relaxation
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
How does LP based B&B works:
Partition Space with constraint that eliminates current LP
solution
Regain primal feasibility (a constraint do not affect Dual
feasibility)
Then, Ready for next branch or cut
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Branching and Cutting have same impact on the current LP:
Only Primal feasibility is lost.
Dual Simplex Pivots to regain Primal Feasibility:
Selects (New) now not satisfied constraint (variable to exit
basis)
Determines most binding dual constraint (variable to enter
basis).
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Branching and Cutting have same impact on the current LP:
Only Primal feasibility is lost.
Dual Simplex Pivots to regain Primal Feasibility:
Selects (New) now not satisfied constraint (variable to exit
basis)
Determines most binding dual constraint (variable to enter
basis).
How is it with an Exponential Number of Constraints ? CUTTING
PLANE ALGORITHM
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming
Branching and Cutting have same impact on the current LP:
Only Primal feasibility is lost.
Dual Simplex Pivots to regain Primal Feasibility:
Selects (New) now not satisfied constraint (variable to exit
basis)
Determines most binding dual constraint (variable to enter
basis).
How is it with an Exponential Number of Constraints ? CUTTING
PLANE ALGORITHM
How is it with an Exponential Number of Columns ? COLUMN
GENERATION ALGORITHM
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming: B&B with CUTTING PLANE
Solving LP Relaxation
Solve LP relaxation
(subset of constraints)
Always a Valid Lower Bound
Solve LP after adding the
Cut-set constraint.
 violated cut-set?
Y
N
Stop
Best Lower Bound
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming: B&B with COLUMN GENERATION
Solving LP Relaxation
Solve LP relaxation
(subset of columns)
NOT a Valid Lower Bound
Solve LP after adding the
Column-set variable.
 Neg cost column?
Y
N
Stop
(First) Valid Lower Bound
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming: B&B with CUTTING × PRICING
Asymmetry:
Cutting: No Exact Cutting is Needed
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Integer Programming: B&B with CUTTING × PRICING
Asymmetry:
Cutting: No Exact Cutting is Needed (Except when cut-set is
part of the Integer Formulation)
Pricing: Exact Pricing is Mandatory on every B&B node.
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Seminal Column Generation Publications
Work of Gilmore and Gomory on the Cutting Stock Problem
(1961)
Christofides, Mingozzi and Toth on the CVRP (1981,
Lagrangean Relaxation)
Desrosiers, Soumis and Desrochers on the VRPTW (1984)
Late 80´s early 90´s
How to branch ? or more generally, How to Cut ?
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Column Generation on Integer Programs
Formalization
Vanderbeck (1994)
Vanderbeck and Wolsey (1996)
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Column Generation on Integer Programs
Formalization
Vanderbeck (1994)
Vanderbeck and Wolsey (1996)
Earlier Publications
Nemhauser and Park (Edge Coloring, 1991)
Mehrotra and Trick (Graph Coloring, 1996)
Vanderbeck (Lot Sizing, 1998)
Kim, Barnhart, Ware and Reinhardt (Network Design, 1999)
Kohl, Desrosiers, Madsen, Solomon and Soumis (VRPTW,
1999)
Felici, Gentile and Rinaldi (Supply Chain, 2000)
Belov and Scheithauer (Cutting Stock, 2000)
Barnhart, Hane and Vance (Integer Multiflow, 2001)
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: What are the main issues?
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: What are the main issues?
How to Branch
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: What are the main issues?
How to Branch
How long does it take to obtain a valid bound (Solve LP)
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: What are the main issues?
Formulation on the Original Variables
min
c 1 .x 1 +
A1 .x 1 +
D1 .x 1
x1 ,
c 2 .x 2
A2 .x 2
D2 .x 2
x2
=
≤
≤
≥0
b
d1
d2
Integers
Column Generation Formulation: λ Variables
min
p
P
c 1 .xr1 .λ1r +
r =1
p
P
r =1
p
P
r =1
1
λ ,
A1 .xr1 .λ1r +
p
P
c 2 .xr2 .λ2r
r =1
p
P
r =1
A2 .xr2 .λ1r
λ1r
p
P
r =1
2
λ
λ2r
=
b
(π)
=
1
(γ1 )
=
1
(γ2 )
≥
0
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: What are the main issues?
What happens when branch is on λ ?
What must be integer ?
What happens when a λr is set to Zero in a B&B ?
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: What are the main issues?
What happens when branch is on λ ?
What must be integer ?
What happens when a λr is set to Zero in a B&B ?
Each λr fixed to Zero may is a potential solution of the Pricing
subproblem.
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: What are the main issues?
What happens when branch is on λ ?
What must be integer ?
What happens when a λr is set to Zero in a B&B ?
Each λr fixed to Zero may is a potential solution of the Pricing
subproblem. =⇒ Changes the structure of the Pricing subproblem.
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: What are the main issues?
What happens when branch is on λ ?
What must be integer ?
What happens when a λr is set to Zero in a B&B ?
Each λr fixed to Zero may is a potential solution of the Pricing
subproblem. =⇒ Changes the structure of the Pricing subproblem.
Adds a no-good cut or combinatorial Benders cut
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: Robust or Non-Robust
ROBUST in the sense that the COMPUTATION TIME of the
PRICING is not (too) SENSITIVE
(throughout the B&B)
More precise way to consider a BCP robust.
The Number of Columns in the Formulation is Exponential
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: Robust or Non-Robust
ROBUST in the sense that the COMPUTATION TIME of the
PRICING is not (too) SENSITIVE
(throughout the B&B)
More precise way to consider a BCP robust.
The Number of Columns in the Formulation is Exponential
No matter how many cuts are added or branchings are
performed, the worst-case complexity of the pricing
subproblem is not exponential
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Branch-Cut-and-Price: Robust or Non-Robust
ROBUST in the sense that the COMPUTATION TIME of the
PRICING is not (too) SENSITIVE
(throughout the B&B)
More precise way to consider a BCP robust.
The Number of Columns in the Formulation is Exponential
No matter how many cuts are added or branchings are
performed, the worst-case complexity of the pricing
subproblem is not exponential
It may be polynomial, or pseudo-polynomial and/or
polynomial for fixed parameter.
Branch-Cut-and-Price
History and Basics
Integer Programming with Column Generation
Thanks