MINLPs With a Special Structure: Lessons from
Combinatorial Optimization
Frauke Liers
FAU Erlangen-Nürnberg
MINO/COST Spring School on Optimization, March 12th, 2015
Outline
A Specific Class of MINLPs: Binary Quadratic Optimization Problems (BQO)
Applications for Binary Quadratic Optimization
Binary Quadratic Optimization and Maximum Cut
Brief Introduction to Polyhedral Combinatorics
Global Solution Approach: Branch-and-Cut
Improving the Relaxation: Lift-and-Project Methods
Other MINLPs where Linearization+Combinatorial Optimization Helps
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
2
Mixed-Integer Non-Linear Optimization Problems
Focus on Specific Classes of MINLPs
MINLPs can be very difficult, already for small instances.
• We focus on a class of MINLPs where we know: feasible solutions have
(combinatorial) structure, e.g., path, tour, flow, matching, etc.
• goal: solve linearized version of the optimization problem using global
methods
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
3
Even more Specific: Binary Quadratic Optimization
(BQO)
Optimization Model
max
Pm
c
x
x
+
i <j ij i j
i =1 ci xi
x ∈ {0; 1}m
x ∈P
P
P: some linear constraints
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
4
Even more Specific: Binary Quadratic Optimization
(BQO)
Optimization Model
max
Pm
c
x
x
+
i <j ij i j
i =1 ci xi
x ∈ {0; 1}m
x ∈P
P
P: some linear constraints
Why study quadratic optimization?
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
4
Outline
A Specific Class of MINLPs: Binary Quadratic Optimization Problems (BQO)
Applications for Binary Quadratic Optimization
Binary Quadratic Optimization and Maximum Cut
Brief Introduction to Polyhedral Combinatorics
Global Solution Approach: Branch-and-Cut
Improving the Relaxation: Lift-and-Project Methods
Other MINLPs where Linearization+Combinatorial Optimization Helps
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
5
Applications
One-dimensional Facility Layout
• cij : average daily traffic btw pairs of objects i , j
• zij : distance between objects i , j
P
• minimize ‘communication cost’ min 1≤i ,j ≤n cij zij
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
6
Applications
One-dimensional Facility Layout
• cij : average daily traffic btw pairs of objects i , j
• zij : distance between objects i , j
P
• minimize ‘communication cost’ min 1≤i ,j ≤n cij zij
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
6
Applications
One-dimensional Facility Layout
• cij : average daily traffic btw pairs of objects i , j
• zij : distance between objects i , j
P
• minimize ‘communication cost’ min 1≤i ,j ≤n cij zij
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
6
Applications
One-dimensional Facility Layout
• cij : average daily traffic btw pairs of objects i , j
• zij : distance between objects i , j
P
• minimize ‘communication cost’ min 1≤i ,j ≤n cij zij
zij = 1 + P
objects between i , j
= 1 + k lengths ∗ xik xkj
with linear ordering variables xij
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
6
Applications
Computer Vision: Determine ‘Highly Similar’ Subgraphs
[Torresani et al. (2008)]
[Szeliski (2010)]
G2 = (V2 , E2 )
b j
G1 = (V1 , E1 )
i 1
4
2k
3
Frauke Liers ·
FAU Erlangen-Nürnberg ·
l
c
a
d
MINLPs With Special Structure
max
X
i ∈V1 ,j ∈V2
xij +
X
cijkl xij xkl
i ,k ∈V1 ,j ,l ∈V2
x bipartite matching
cijkl < 0 if only (i , k )
or only (j , l ) exists
Spring School
7
Bipartite Crossing Minimization
Fix Nodes in a Layer
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
8
Bipartite Crossing Minimization
Fix Nodes in a Layer
xuv
(
1, if u drawn before v
=
0, otherwise
⇒ obj + xlk
minimal linear ordering (LO)
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
8
Bipartite Crossing Minimization
Nodes in Both Layers Permute
⇒ obj + xij xlk + xji xkl
(xji = 1 − xij
xlk = 1 − xkl )
minimal quadratic linear ordering
P
min
(i ,k ),(j ,l )∈E cijkl xij xlk
(QLO )
s.t.
x ∈ PLO
xij ∈ {0; 1}
PLO : linear ordering polytope
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
9
Binary Quadratic Optimization
Problems
are
• an interesting class of problems and applications: quadratic knapsack,
maximum cut, QAP, QMP, QLO, etc.
Even the simplest (i.e., unconstrained) BQO is NP-hard in general and difficult to solve in
practice...!
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
10
Outline
A Specific Class of MINLPs: Binary Quadratic Optimization Problems (BQO)
Applications for Binary Quadratic Optimization
Binary Quadratic Optimization and Maximum Cut
Brief Introduction to Polyhedral Combinatorics
Global Solution Approach: Branch-and-Cut
Improving the Relaxation: Lift-and-Project Methods
Other MINLPs where Linearization+Combinatorial Optimization Helps
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
11
Unconstrained BQO is equivalent to Maximum Cut
[Hammer (1965), De Simone (1988)]
Maximum Cut Problem
G = (V , E ), edge weights we .
• W ⊆ V . cut δ(W ) = {(u , v ) ∈ E | u ∈ W , v 6∈ W }
with maximum weight
P
• weight of a cut: e∈δ(W ) we
NP-hard in general, but poly-time solvable special
cases exist.
Equivalence
(Unconstrained) Binary quadratic optimization is equivalent to maximum cut.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
12
Unconstrained Binary Quadratic Optimization as
Maximum-Cut
Graph Construction
For given BQO problem, construct graph G = (V , E ):
• V = {r , v1, v2, . . . , vn }
• E = {(r , vi ) | i = 1, . . . , n} ∪ {(vi , vj ) | i , i = 1, . . . , n, i < j }
• edge weights:
• put linear cost ci on edge (r , vi )
• product cost cij : add 21 cij on edges (r , vi ) and (r , vj ), put − 21 cij on edge (vi , vj )
from a maximum cut in G = (V , E ), a maximum solution for BQO can be read
off:
• edge (r , vi ) corresponds to the linear term xi
• edge (vi , vj ) corresponds to xi + xj − 2xi xj
• if (r , vi ) is cut, set xi = 1. Otherwise xi = 0.
• if xj 6= xj , then edge (vi , vj ) is cut, otherwise not.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
13
Unconstrained Binary Quadratic Optimization as
Maximum-Cut
Example
2
max x1 + x2 + x3 + x1x2 + x1x3
x ∈ {0; 1}m
−0.5
−0.5
1
3
1.5
2
1
r
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
14
Unconstrained Binary Quadratic Optimization as
Maximum-Cut
Example
2
max x1 + x2 + x3 + x1x2 + x1x3
x ∈ {0; 1}m
−0.5
−0.5
1
3
1.5
2
1
r
The maximum cut is determined by
{r }. This corresponds to
x1 = x2 = x3 = 1.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
14
Solution Approaches
How to solve (constrained and unconstrained) binary quadratic problems?
• structural investigations (→ polyhedral combinatorics)
• exploit them algorithmically (→ separation routines)
• design global solution approaches (→ branch-and-cut)
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
15
Outline
A Specific Class of MINLPs: Binary Quadratic Optimization Problems (BQO)
Applications for Binary Quadratic Optimization
Binary Quadratic Optimization and Maximum Cut
Brief Introduction to Polyhedral Combinatorics
Global Solution Approach: Branch-and-Cut
Improving the Relaxation: Lift-and-Project Methods
Other MINLPs where Linearization+Combinatorial Optimization Helps
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
16
Introduction to Polyhedral Combinatorics
The Cut Polytope
(i , j ) ∈ E → 0 ≤ xij ≤ 1
(i , j ) ∈ cut → xij = 1
(i , j ) 6∈ cut → xij = 0
characteristic vectors of cuts
cut polytope PC (G) : convex hull of all characteristic cut vectors
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
17
Introduction to Polyhedral Combinatorics
The Cut Polytope
(i , j ) ∈ E → 0 ≤ xij ≤ 1
(i , j ) ∈ cut → xij = 1
(i , j ) 6∈ cut → xij = 0
characteristic vectors of cuts
cut polytope PC (G) : convex hull of all characteristic cut vectors
 
0
0
0
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
18
Introduction to Polyhedral Combinatorics
The Cut Polytope
(i , j ) ∈ E → 0 ≤ xij ≤ 1
(i , j ) ∈ cut → xij = 1
(i , j ) 6∈ cut → xij = 0
characteristic vectors of cuts
cut polytope PC (G) : convex hull of all characteristic cut vectors
 
0
1
1
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
19
Introduction to Polyhedral Combinatorics
The Cut Polytope
(i , j ) ∈ E → 0 ≤ xij ≤ 1
(i , j ) ∈ cut → xij = 1
(i , j ) 6∈ cut → xij = 0
characteristic vectors of cuts
cut polytope PC (G) : convex hull of all characteristic cut vectors
 
1
0
1
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
20
Introduction to Polyhedral Combinatorics
The Cut Polytope
(i , j ) ∈ E → 0 ≤ xij ≤ 1
(i , j ) ∈ cut → xij = 1
(i , j ) 6∈ cut → xij = 0
characteristic vectors of cuts
cut polytope PC (G) : convex hull of all characteristic cut vectors
 
1
1
0
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
21
Introduction to Polyhedral Combinatorics
The Cut Polytope

 
 
 

0
0
1
1
PC (K3) = conv{ 0  ,  1  ,  0  ,  1 } =
0
1
1
0
PC (K3) can be described by linear inequalities!
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
22
Polyhedral Combinatorics
Polyhedron
• For A ∈ Rm×n , b ∈ Rm , a polyhedron P ∈ Rn is a set P = {x ∈ Rn | Ax ≤ b}.
• If P is bounded, it is called polytope.
polyhedron
Frauke Liers ·
FAU Erlangen-Nürnberg ·
polytope
MINLPs With Special Structure
Spring School
23
Dimension of a Polyhedron P
Dimension of P
Let P = {x ∈ Rn | A=x = b=, A0x ≤ b0}.
• A=x = b= is a minimum equation system for P, if none of its equations are
redundant, and for arbitrary a>x ≤ β out of A0x ≤ b0 there exists x ∈ P with
a> x < β .
• If A=x = b= is a minimum equation system, the dimension dim(P ) of P is
dim(P ) = n − rang(A=)
• If dim(P ) = n, P is full-dimensional.
Note:
dim(P ) = number of affinely independent points in P − 1.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
24
Dimension of a Polyhedron P
Dimension of P
two-dimensional
one-dimensional
gx=f
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
25
Faces
Face of P
Let P = {x ∈ Rn | Ax ≤ b} =
6 ∅. Let d ∈ Rn with δ := max{d >x | x ∈ P } < ∞.
• {x | d >x = δ} is called a supporting hyperplane of P.
• A face of P is either P itself, or the intersection of finitely many supporting
hyperplanes.
• A face of the form {x } is called vertex of P.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
26
Faces
Characterization of Faces
Let P = {x ∈ Rn | Ax ≤ b} be a polyhedron, F ⊆ P. Then it is equivalent:
1. F is a face of P
2. There exists d ∈ Rn such that δ := max{d >x | x ∈ P } < ∞ and
F = {x ∈ P | d >x = δ}.
3. F = {x ∈ P | A0x = b0} for a subsystem A0x ≤ b0 from Ax ≤ b.
This also means: If F is a face of P, then F is a polyhedron as well.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
27
Facets
Facets
• A maximal face F 6= P is called facet of P, i.e., there does not exist a face
F 0 6= P of P with F 0 ⊃ F , F 0 6= F .
• An inequality d >x ≤ δ is facet defining for P, if ∀x ∈ P it is d >x ≤ δ , and
{x ∈ P | d >x = δ} is a facet of P.
• The dimension of a facet is dim(P ) − 1.
F1
F2
F
6
P
F3
F5
F4
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
28
Minimal Description
Non-Redundant Description of P
Let ∅ =
6 P ⊆ {x ∈ Rn | Ax = b} have dimension n − rang(A). Let A0x ≤ b0 a
minimal (i.e., non-redundant) inequality system such that
P = {x ∈ Rn | Ax = b, A0x ≤ b0}. Then each inequality from A0x ≤ b0 defines a
facet of P. Furthermore, each facet of P is defined through one of the
inequalities from A0x ≤ b.
TSP: Number of Facets
|V |
# facets
5
20
6
100
7
3437
8
194187
9
42104442
10 ≥ 51043900866
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
29
Back to the Cut Polytope PC (G)
Known Facts
For G = (V , E ):
• dim(PC (G)) = |E |
• If e ∈ E is not contained in a triangle, then xe ≥ 0 and xe ≤ 1 define facets.
• Let K be a cycle in G and F ⊆ K with |F | odd. Then the cycle inequality
x (F ) − x (K \ F ) ≤ |F | − 1
is valid for PC (G). It defines a facet if K does not contain a chord.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
30
Inner and Outer Description of a Polyhedron
Different Representations of a Polyhedron
A set P ⊆ Rn is a polyhedron, if and only if there exist sets R , S ∈ Rn such that it
is the sum of the convex hull of points in R and the conic hull of the points in S,
i.e.,
P = conv(R ) + cone(S ).
• The representation via R , S is called inner description.
• The representation as the intersection of a finite set of halfspaces is called
outer description.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
31
Outline
A Specific Class of MINLPs: Binary Quadratic Optimization Problems (BQO)
Applications for Binary Quadratic Optimization
Binary Quadratic Optimization and Maximum Cut
Brief Introduction to Polyhedral Combinatorics
Global Solution Approach: Branch-and-Cut
Improving the Relaxation: Lift-and-Project Methods
Other MINLPs where Linearization+Combinatorial Optimization Helps
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
32
Cutting-Plane Method
Cutting-Plane Method
0. start with a subset of the restrictions (e.g. with only the trivial 0 ≤ x ≤ 1).
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
33
Cutting-Plane Method
Cutting-Plane Method
0. start with a subset of the restrictions (e.g. with only the trivial 0 ≤ x ≤ 1).
1. solve LP, let x ∗ the found optimal solution
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
33
Cutting-Plane Method
Cutting-Plane Method
0. start with a subset of the restrictions (e.g. with only the trivial 0 ≤ x ≤ 1).
1. solve LP, let x ∗ the found optimal solution
2. decide ether there exist supporting hyperplanes aT x ≤ a0 not yet taken into
account that are valid for all characteristic vectors of cuts, so that aT x ∗ > a0.
If no: STOP (relaxation solved).
If yes: Find these inequalities, add them to LP , goto 1.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
33
Cutting-Plane Method
Cutting-Plane Method
0. start with a subset of the restrictions (e.g. with only the trivial 0 ≤ x ≤ 1).
1. solve LP, let x ∗ the found optimal solution
2. decide ether there exist supporting hyperplanes aT x ≤ a0 not yet taken into
account that are valid for all characteristic vectors of cuts, so that aT x ∗ > a0.
If no: STOP (relaxation solved).
If yes: Find these inequalities, add them to LP , goto 1.
The problem to solve in 2. is called separation problem.
Often, separation routines for facets are developed.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
33
Separation
Separation of Inequalities
Good news: For some problems and some classes of inequalities the
separation problem can be solved in polynomial time, despite the fact that
exponentially many candidate inequalities exist!
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
34
Separation
Separation of Inequalities
Good news: For some problems and some classes of inequalities the
separation problem can be solved in polynomial time, despite the fact that
exponentially many candidate inequalities exist!
(However, for some classes of inequalities, separation is NP-hard.)
Example: Cut Polytope
For the cut polytope, separation of cycle inequalities can be done in polynomial
time via all-pairs shortest-path calculations.
Depending on the ’power’ of the separation routines, it might happen in practice that no
more violated cutting plane can be generated, however the solution is not yet feasible.
What to do then? Combine it with branch-and-bound approach, call it branch-and-cut.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
34
Branch-and-Bound Algorithm
Branch-and-Bound
• (lb): lower bound for
optimum
• (ub): upper bound
• (lb) = (ub) ⇒ optimality
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
35
Branch-and-Bound
Basic Idea
In the following wlog: consider maximization problems.
• start solving the original problem
• bounds through feasible solutions and through relaxations
• in case bounds are equal: optimality proven
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
36
Branch-and-Bound
Basic Idea
In the following wlog: consider maximization problems.
•
•
•
•
start solving the original problem
bounds through feasible solutions and through relaxations
in case bounds are equal: optimality proven
otherwise: divide the problem into subproblems so that the combination of
the solutions in the subproblems can be combined to the solutions of the
original problem
• solve subproblem through
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
36
Branch-and-Bound
Basic Idea
In the following wlog: consider maximization problems.
•
•
•
•
start solving the original problem
bounds through feasible solutions and through relaxations
in case bounds are equal: optimality proven
otherwise: divide the problem into subproblems so that the combination of
the solutions in the subproblems can be combined to the solutions of the
original problem
• solve subproblem through
1. determination of an optimum solution, or
2. proof of its infeasibility, or
3. calculation of an upper bound that is not better than the currently best known
solution, or
4. subdividing the problem into further subproblems.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
36
Branch-and-Bound
Basic Idea
associate to the solution process in a natural way a branch-and-bound-tree:
•
•
•
•
the root is the original problem
a node represents some subproblem
a direct child of a node u represents a subproblem of u
tree leafs represent ‘solved’ problems
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
37
Branch-and-Cut Method
Branch-and-Cut
in each node of the branch-and-bound tree
1. do cutting-plane method
2. in case the found solution is not a characteristic vector of a feasible solution,
branch on some variable xe , 0 < xe < 1
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
38
Branch-and-Cut Method
Branch-and-Cut
in each node of the branch-and-bound tree
1. do cutting-plane method
2. in case the found solution is not a characteristic vector of a feasible solution,
branch on some variable xe , 0 < xe < 1
Branch-and-Cut is Branch-and-Bound with bounds determined by linear
optimization.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
38
Branch-and-Cut Method
Branch-and-Cut
in each node of the branch-and-bound tree
1. do cutting-plane method
2. in case the found solution is not a characteristic vector of a feasible solution,
branch on some variable xe , 0 < xe < 1
Branch-and-Cut is Branch-and-Bound with bounds determined by linear
optimization.
If branch-and-cut shall be used for (unconstrained or constrained) binary quadratic
optimization problems, they should be linearized...
Back to Binary Quadratic Optimization: Linearize...!
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
38
Linearizing Binary Quadratic Optimization Problems
Standard Linearization
max
Pm
c
x
x
+
i <j ij i j
i =1 ci xi
x ∈ {0; 1}m
x ∈P
P
Let a binary variable yij ∈ {0; 1} be given,
together with the inequalities
yij ≤ xi , yij ≤ xj , yij ≥ xi + xj − 1. Then yij = xi xj ,
for binary variables xi , xj .
Many linearizations exists for different classes of functions and variables. However, the
corresponding linear relaxations are usually weak...
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
39
Binary Quadratic Optimization
Back to Binary Quadratic Optimization
Maximum Cut Problem: G = (V , E ), edge weights we .
• W ⊆ V . cut δ(W ) = {(u , v ) ∈ E | u ∈ W , v 6∈ W } with maximum weight
P
• weight of a cut: e∈δ(W ) we
Equivalence
• Binary quadratic optimization is equivalent to maximum cut.
• The cut polytope and that of the linearized binary quadratic problem are
isomorphic.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
40
Constrained Quadratic Problems
Connection to Maxcut
max
P
i <j
Pm
cij xi xj + i =1 ci xi
x ∈ {0; 1}m
x ∈P
• clearly: faces of cut polytope remain valid
• sometimes even a face is cut out
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
41
Constrained Quadratic Problems
Connection to Maxcut
max
P
i <j
Pm
cij xi xj + i =1 ci xi
x ∈ {0; 1}m
x ∈P
• clearly: faces of cut polytope remain valid
• sometimes even a face is cut out
• quadratic assignment and, more generally, quadratic matching
• quadratic linear ordering: [Buchheim et al. (2010)]
• bipartite crossing minimization
• single-row facility layout
• betweenness
⇒ use maximum-cut approaches for these constrained cut problems? (yes for
ordering problems. [Buchheim,L,Oswald (2010)])
not always a face is cut out, see e.g., quadratic knapsack
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
41
Bounding Procedure
Bounding Procedure (Caprara (2008))
P
• rewrite quadratic objective of BQO problem as maxi ∈N ( j ∈N cij xj )xj
P
• if xi = 1, then it is j ∈N cij xj ≤ pi , where pi is optimum for the linear problem
max
j ∈N
cij xj
x ∈P
xi = 1
• a valid upper bound L on BQO problem is given by the linear problem
max
i ∈N
pi xi
x ∈P
• For the QAP, this yields the Gilmore-Lawler bound.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
42
Outline
A Specific Class of MINLPs: Binary Quadratic Optimization Problems (BQO)
Applications for Binary Quadratic Optimization
Binary Quadratic Optimization and Maximum Cut
Brief Introduction to Polyhedral Combinatorics
Global Solution Approach: Branch-and-Cut
Improving the Relaxation: Lift-and-Project Methods
Other MINLPs where Linearization+Combinatorial Optimization Helps
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
43
Lift-and-Project
Problem Statement
• given: K = {x ∈ Rn | Ax ≤ b}, with (w.l.o.g.) A ∈ Zm×n , b ∈ Zm
• task: linear description for the convex hull P = conv(K ∩ {0; 1}n ), i.e.,
0
0
A0 ∈ Zm ×n , b ∈ Zm , with P = {x ∈ Rn | Ax ≤ b, A0x ≤ b0}.
Obviously not possible in polynomial time, if P 6= NP.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
44
Lift-and-Project
Problem Statement
• given: K = {x ∈ Rn | Ax ≤ b}, with (w.l.o.g.) A ∈ Zm×n , b ∈ Zm
• task: linear description for the convex hull P = conv(K ∩ {0; 1}n ), i.e.,
0
0
A0 ∈ Zm ×n , b ∈ Zm , with P = {x ∈ Rn | Ax ≤ b, A0x ≤ b0}.
Obviously not possible in polynomial time, if P 6= NP.
Lift-and-Project
• goal: construct hierarchy P = K t ⊆ K t −1 ⊆ . . . ⊆ K 0 = K , with t < n ’small’,
and optimization over K t for fixed t can be done in polynomial time.
• only a few references:
• Balas, Ceria, Cornuéjols (1993)
• Lovász, Schrijver (1991)
• Sherali, Adams (1990)
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
44
Lift-and-Project
General Idea
1. Multiply Ax ≤ b with inequalities d >x ≤ δ that are valid for P. Yields nonlinear
relaxation of P with inequalities (δ − d >x )(bi − ai>x ) ≥ 0, i = 1, . . . , m.
2. Linearize by introducing new variables Y ∈ {0; 1}, with yij = xi xj and xi2 = xi .
Yields a polytope in the space of x- and y -variables.
3. Project polytope back in the space of the x-variables only, only implicitely.
Yields new linear description of P.
P2 ⊆ Rn+k . Projection P1 := {x ∈ Rn | ∃x 0 ∈ Rk : (x , x 0) ∈ P2}.
Example
Usually, difficult to calculate how the
linear inequalities from P2 translate
to those of P1 → keep x 0 variables,
however yields larger problems.
P
2
P1
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
45
Lift-and-Project
Construction of Sherali and Adams (Reformulation-Linearization Technique,
RLT)
Here explained for 0, 1 linear programs. Fix RLT-level k , 1 ≤ k ≤ n.
Reformulation phase:
1. For any disjoint pair S , T ⊆ {1, . . . , n}, let J (S , T ) denote Πi ∈S xi Πi ∈T (1 − xi ).
2. For each disjoint pair S , T ⊆ {1, . . . , n} satisfying |S | + |T | = k + 1, place
inequality J (S , T ) ≥ 0 in the system.
3. For each inequality aj>x ≤ bj , and each disjoint pair S , T ⊆ {1, . . . , n} with
|S | + |T | = k , place the inequality
J (S , T )(bj − aj>x ) ≥ 0
in the system.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
46
Lift-and-Project
Construction of Sherali and Adams (RLT)
Linearization phase:
1. Expand each polynomial inequality so that it becomes a weighted sum of
distinct monomials.
2. Make each monomial multilinear by using xir = xi ∀i = 1, . . . , n, 2 ≤ r ≤ k + 1.
3. For each S ⊆ {1, . . . , n} with 2 ≤ |S | ≤ min{k + 1, n}, introduce a new binary
variable yS representing Πi ∈S xi .
4. Linearize by replacing each multilinear terms with the corresponding y .
Remarks
• Hierarchy of relaxations, yields P for k = n.
• However, number of variables and constraints increases exponentially with
increasing K .
• In practice, often k = 1 is used.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
47
Outline
A Specific Class of MINLPs: Binary Quadratic Optimization Problems (BQO)
Applications for Binary Quadratic Optimization
Binary Quadratic Optimization and Maximum Cut
Brief Introduction to Polyhedral Combinatorics
Global Solution Approach: Branch-and-Cut
Improving the Relaxation: Lift-and-Project Methods
Other MINLPs where Linearization+Combinatorial Optimization Helps
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
48
Motivation: Gas or Water Networks
Challenging Nonlinear Network Optimization Problems
•
•
•
•
gas network is a digraph
nodes represent customers, entries and exists
arcs represent active elements such as pipes, valves, compressors, etc.
gas network transport energy minimization problem has
• technical constraints on pressure, flows, etc.,
• physical constrains such as gas dynamics, flow conservation
• legal constraints
• Given a balanced set of inflows and outflows, it consists in finding a minimum
cost feasible control of all active elements such that all constraints are
satisfied.
This yields complicated non-convex MINLPs!
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
49
The Setting
Linearized Relaxed Non-Linear Network Flow Problems
Network flow problem, flow values are restricted to lie in certain intervals
Main application: piecewise linear approximation, or better: relaxation → solve
MINLPs by means of MIP-techniques [e.g. Geißler et al., and many others...]
∆p
q
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
50
The Setting
Linearized Relaxed Non-Linear Network Flow Problems
• linearization/relaxation through incremental method, or multiple-choice
method, etc.
• Assume non-linearities are modeled as functions of the flow on a network arc
→ flow variable range divided into intervals
• zi : binary indicator for an interval
• consider feasible sets {(q , z ) ∈ Rm × {0; 1}n | li zi ≤ qi ≤ ui zi , Mq = d }.
• several flow intervals on an arc are possible. However, exactly one is active.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
51
Problem Setting
Problem Setting
min
s.t.
f (z )
li zi ≤ qi ≤ ui zi ∀i
(flow value in predefined interval)
(demands)
P Mq = d
i ∈Ia zi = 1 ∀arcs a (exactly one interval active per arc)
(q , z ) ∈ Rm × {0, 1}n
Formulation is locally ideal for one network arc.
[Vielma, Ahmed, Nemhauser, 2009]
Question: How can the formulation be strengthened for multiple
arcs/simple subnetworks?
eliminiate continous flow variables by considering projection of feasible set to
z-variables: P := {z ∈ Rm | ∃q ∈ Rm : li zi ≤ qi ≤ ui zi , Mq = d }
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
52
Structural Investigations
Two adjacent arcs at a degree-2 node
a
d =0
b
Construct compatibility graph:
a1 [1, 2]
b1 [1, 2]
a2 [1, 4]
b2 [3, 4]
a3 [3, 5]
b3 [3, 6]
...understanding the polyhedral structure → very strong relaxations. [L, Merkert
(2015)]
Frauke Liers · FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
53
Thanks.
Frauke Liers ·
FAU Erlangen-Nürnberg ·
MINLPs With Special Structure
Spring School
54