Integer program reformulation for
robust branch-and-cut-and-price
Marcus Poggi de Aragão
Informática
PUC-Rio
Eduardo Uchoa
Engenharia de Produção
Universidade Federal Fluminense
Outline of the talk
Robust branch-and-cut-and-price
Dantzig-Wolfe master X Explicit master
Tackling symmetries in the original formulation
Computational Results
– Capacitated Vehicle Routing Problem
– Capacitated Minimum Spanning Tree Problem
– Generalized Assignment Problem
Perspectives
What is a robust
branch-and-cut-and-price ?
A branch-and-bound algorithm where:
– each node solves a linear program with an
exponential number of both variables and
constraints,
– neither branching nor separation (cutting) ever
change the structure of the pricing subproblem.
Robust branch-and-cut-and-price
Since cut and column generation were
established as two of the most important techniques
in IP, one looks for ways of combining them.
Big (partially open) question:
“How to perform cutting without changing
the structure of the pricing subproblem ?”
Robust branch-and-cut-and-price
First (non-robust) BCP: Nemhauser and Park (1991),
edge-coloring problem.
In a non-robust BCP, the pricing subproblem eventually
needs to be solved by general IP techniques. This only
works for small instances.
An exception: Belov and Scheithauer (2000-03), on a
number of generalized cutting stock problems where column
generation gives very tight bounds. Mixed-integer Gomory
cuts may be enough to close the gaps in the root node.
Robust branch-and-cut-and-price
A number of researchers noted that cuts expressed in terms
of variables from an original formulation do not disturb the
pricing:
Vanderbeck (1998), lot-sizing.
Kim, Barnhart, Ware and Reinhardt (1999), a service
network design problem.
Kohl, Desrosiers, Madsen, Solomon and Soumis (1999),
VRPTW.
Felici, Gentile and Rinaldi (2000), a supply-chain problem.
Barnhart, Hane and Vance (2001), integer multiflow.
Robust branch-and-cut-and-price
This approach has a fundamental drawback as a general IP
technique: on many important problems the original formulation
sufers from variable symmetries. Cuts over those variables are
ineffective.
Implementing an efficient robust BCP raises many new
practical questions.
Up to now, robust BCP was only performed on problems
where BP alone already performs well. We feel that the
applicability of this technique goes far beyond that class of
problems.
IP Reformulation
Let (O) be the following IP with n variables:
(O)
min cx
Ax = b
s.t
Dx ≤ d
x∈ Nn
n
P
=
x
∈
N
| Dx ≤ d }
{
and suppose
is a finite set with elements x1, x2,...xp.
IP Reformulation
Let Q be a n X p matrix where each column
corresponds to one element of P
There is a one-to-one correspondence
between elements of P and the solutions of:
x = Qλ
s.t
1λ = 1
p
λ ∈ {0,1}
IP Reformulation
The traditional IP reformulation (Gilmore and
Gomory, 1961) consists of replacing x by its
equivalent expression in (O), a transformation similar
to the Dantzig-Wolfe reformulation for LP:
Z IP = min
s.t
(cQ)λ
( AQ)λ
1λ
λ
=
=
∈
b
1
{0,1}
p
Dantzig-Wolfe Master
Z DWM = min
s.t
( DWM )
(cQ)λ
( AQ)λ
1λ
= b
= 1
λ
≥ 0
Solving the DWM is equivalent to solving
Z DWM = min cx
s.t
Ax = b
x ∈ Conv { x ∈ N n | Dx ≤ d }
Dantzig-Wolfe Master
Columns on DWM are generated by solving
the following IP:
V ( µ ,ν ) = min(c − µ A) x −ν
s.t
Dx ≤ d
x∈ Nn,
where µ and ν are the associated dual variables.
Cutting on the Dantzig-Wolfe Master
Let λ be a fractional solution of DWM.
On the original formulation, this corresponds
to solution x = Q λ . A cut a i x ≤ bi violated
by x can be added to DWM in order to cut λ .
min
s.t
( cQ ) λ
( AQ ) λ
=
b
( a i Q )λ
≤
bi
1λ
=
1
λ
≥ 0
Incorporating the new row a i to A and the new
dual variable µ i to µ , the pricing subproblem
remains unchanged.
Alternative IP Reformulation
Rewrite (O) by adding variables x’:
min
s.t.
(O' )
x'
cx
−x =
0
Ax =
b
≤ d
∈ Nn
Dx '
x'
x
∈ Nn
Alternative IP Reformulation
Replacing x’ by its equivalent
expression, we obtain another
reformulation:
x
s.t
1
min
s.t. Qλ
Q
1
0,1
p
cx
−x =
0
Ax =
b
=
≥
1
0
1λ
λ
x
∈ Nn
Explicit Master
min
s.t.
(EM )
cx
Qλ
−x = 0
1λ
Ax = b
= 1
λ
≥ 0
x
≥ 0
Explicit Master
Columns on EM are generated by solving the
following IP:
V (π ,ν ) = min(−π x) −ν
s.t
Dx ≤ d
x∈ Nn,
Dual variables µ associated to Ax = b do not appear
in the subproblem.
Cutting on the Explicit Master
min
s.t.
(EM )
Additional cut
cx
Qλ
1λ
λ
−x
Ax
=
=
=
0
b
1
a i x = bi
≥ 0
x ≥ 0
New dual variable µi clearly do
not change the pricing.
Dantzig-Wolfe X Explicit Master
Lower bound
L
Lemma: Z DWM = Z EM
Sketch of the Proof: Let λ ∗ and ( µ ∗ ,ν ∗ ) be optimal
primal and dual solutions of DWM. Then (λ , x = Qλ )
∗
∗
and (π ∗ = ( µ ∗ A − c), µ ∗ ,ν ∗ ) are optimal primal and
dual solutions of EM.
∗
Dantzig-Wolfe X Explicit Master
LP size
Let Q′ ⊆ Q be the current n × p′ submatrix of priced
columns and let m be the current number of rows in A.
Rows
Cols
Non-zeros
DWM
m +1
p′
nz ( AQ′) + p′
EM m + n + 1 p′ + n nz ( A) + nz (Q′) + p′ + n
DWM usually leads to smaller LPs. However, upper
bounds on x variables have to be represented as rows
in DWM.
L
Dantzig-Wolfe X Explicit Master
Column generation convergence
As usually n > m, there are more dual variables
to handle in EM. This may lead to slower
convergence. It can be counterbalanced by
proper stabilization.
L
Is there any advantage on working
on the Explicit Master ?
BCP easier to code. (Not a really good
argument ... )
EM allows cheap fixing of x variables by
reduced costs.
EM allows better multiple column generation.
L
Fixing variables on DWM
After solving DWM, choose a variable x j from the
original formulation and solve:
V j0 ( µ ,ν ) = min(c − µ A) x + ν
s.t
Dx ≤ d
x ∈ N m, xj ≥1
If Z DWM + V j0 ( µ ,ν ) ≥ Z INC , then x j can be fixed to 0.
(This technique is known as fixing by Lagrangean bound).
Depending on the subproblem structure, each attempt
to fix a variable can be quite expensive.
L
Fixing variables on EM
After solving EM, just check if Z EM + c j ≥ Z INC .
The diference between EM and DWM is better
understood by having a look at the dual of EM.
Max
ν
+µb
π Q +ν 1
≤ 0
−π
+µ A ≤ c
L
Fixing variables on EM
The dual of EM is equivalent to
Max
ν
+µb
≤ 0
π Q +ν 1
−π
+ µ A +c = c
c
≥ 0
By solving DWM one always gets a solution of form
(π ∗ = ( µ ∗ A − c), µ ∗ ,ν ∗ ). This implies c = 0, no
variable is ever fixed.
In other words, solving DWM is equivalent to solving
EM with the dual space restricted by c = 0 constraints.
L
Fixing variables on EM
The fixing can much improved by perturbing the dual of EM
Max
ν
+ µ b +cε
π Q +ν 1
≤ 0
−π
+µ A
This is equivalent to
cx
min
s.t.
Qλ
−x = 0
Ax = b
1λ
= 1
≥ 0
λ
x
≥ ε
+c
= c
c
≥ 0
L
Multiple Column Generation
Let (O) be the following IP with n variables:
Min cx
Ax = b
(O)
Dx ≤ d
Fx ≤ f
x ∈ Nn
and suppose P1 = { x ∈ N n | Dx ≤ d } and P2 = { x ∈ N n | Fx ≤ f }
are finite sets.
Multiple Column Generation
Min
S.t. Q1λ1
1λ1
Q2 λ2
1λ2
λ1
λ2
cx
−x =
=
−x =
=
0
1
0
1
Min cx
S.t. Ax = b
x
x
∈ Conv { x ∈ N m | Dx ≤ d }
∈ Conv { x ∈ N m | Fx ≤ f }
Ax = b
≥ 0
≥ 0
x
≥ 0
EM allows multiple sets of constraints to be convexified
independently. On DWM this would be cumbersome.
Dantzig-Wolfe X Explicit Master
Implementing a BCP over the EM is interesting
when:
Fixing of variables is important and the
subproblem is too hard for effective Lagrangean
fixing.
Multiple column generation is performed.
L
The symmetry problem
On many important problems where BP is
applied, the corresponding original formulation is
highly symmetric. Some examples are:
–
–
–
–
–
bin packing
cutting stock problems
graph coloring
clustering problems
scheduling with some identical machines
L
The symmetry problem
Most of those problems share the same
characteristic:
L
– They search for an optimal feasible assignment of n
elements to m sets, and some of those sets are
undistinguishable.
– The original formulation corresponding to their usual
k
column generation reformulation have variables like xi ,
meaning that element i is assigned to set k.
– Cutting (and even branching) over those variables is
not effective.
How to construct a robust BCP for
those problems ?
We do not have a full answer to this, just an
approach that seems reasonable.
– Ryan and Foster (1981) proposed a branch rule widely
used on BPs for those problems. In one branch,
elements i and j must be in the same set; on the other
branch they must be in different sets.
– Valério de Carvalho (2001) suggested solving binpacking by BP considering it as a particular case of the
VRPTW.
L
How to construct a robust BCP for
those problems ?
L
Those ideas inspire a general approach:
Formulate the original problem over variables xij
meaning that elements i and j are assigned to the same
set. Apply IP reformulation.
2. Is the resulting pricing problem still tractable ? Go
ahead.
3. Find additional families of valid inequalities over the
original formulation.
1.
Example: vertex coloring problem
L
BP algorithm (Mehotra and Trick, 1996)
–
–
–
Pricing problem: maximum weight independent set over G.
Branch rule: Ryan and Foster.
k
Corresponding original formulation: xi variables, vertex i is
colored k. Not good for BCP.
Set-partition with a row for
each vertex and colunms
are independent sets.
Z EM =2.5
1
7
5
6
2
4
3
Example: vertex coloring problem
L
Possible BCP algorithm
–
–
–
Pricing problem: maximum edge weight clique over G .
Branch rule: over any constraints on variables xij .
Cuts: Odd holes, anti-odd holes, etc.
Set-partition with a row for
each vertex and colunms
are vertex-sets corresponding
to cliques in G, Z EM =2.5.
Adding odd-hole cut
x13 + x14 + x24 + x25 + x35 ≤ 2
Z EM =3
1
7
5
6
2
4
3
Computational Results
We are implementing robust BCPs for several
classical problems. We already have results on
the following problems:
CVRP
CMST
GAP
L
Capacitated Vehicle Routing Problem
L
BP is known to be effective on tightly constrained VRP
problems, like VRPTW (Desrosiers, Soumis and Desrochers,
1984).
BP does not work well on CVRP.
Best exact algorithms are sophisticated BCs. They can solve
instances with up to 50-100 vertices.
Capacitated Vehicle Routing Problem
Our BCP (Fukasawa, Poggi de Aragão, Reis, Uchoa)
Dantzig-Wolfe Master.
Columns correspond to q-routes without 2-cycles
(Christofides, Mingozzi, Toth, 1981). Pricing is cheap (O(n2C)).
Only capacity cuts.
Consistently solves instances with up to 100 vertices,
including almost all open instances from the literature.
Better results are expected soon by adding several other cuts
used by current BC codes.
Capacitated Minimum Spanning Tree
Problem
Two kinds of instances: unitary and non-unitary capacities.
Best exact algorithms:
Unitary case: BC over large “hop” formulations.
Non-unitary case: BC and Lagrangean relaxation. OR-Lib
instances with only 50 vertices can not be solved.
L
Capacitated Minimum Spanning Tree
Problem
Our BCP (Fukasawa, Poggi de Aragão, Porto, Uchoa)
Explicit Master.
Columns correspond to connected components from the
root. Pricing is expensive (O(n2C2)).
Capacity and root cutset cuts.
Results are good on the unitary case and very good on the
non-unitary case: all open instances with 50 vertices and
some with 100 vertices solved.
Generalized Assignment Problem
L
Most instances from the OR-Lib are now considered to be
easy and can be solved by commercial MIP packages. Only the
D series remains challenging.
Best exact algorithms: Lagrangean relaxation, BP and BC.
Generalized Assignment Problem
Our BCP (Pigatti, Poggi de Aragão, Uchoa)
Dantzig-Wolfe Master.
Pricing is cheap, knapsack problems.
Fischetti-Lodi cuts.
Good results. Could solve 3 out of 5 open instances from D
series. The use of other cuts from the literature is being
studied.
Real world instances with symmetry are being considered.
Perspectives
The implementation of robust BCP algorithms today is still
quite restricted to the “column generation community”. The
good results that are being obtained indicate that this situation
may soon change.
This certainly raises a number of interesting research topics.
Perspectives
Obtaining high performance from a BC or from a BP may
require experience to choose among several possible
strategies. On a BCP the number of possibilities increases
and little experience is currently available to make decisions.
For example:
Should I price to optimality before cutting ? Or cutting
should be done along with (or even before) pricing ?
Perspectives
For obvious reasons, polyhedral research concentrates on
problems where BC works. BCP gives motivation to studying
other polytopes.
For example:
What are the families of facets of the bin-packing polytope
over the xij variables ?
In a similar way, robust BCP motivates to devise new
column generation schemes on problems where BP alone
would not work.