3.8 Strong valid inequalities
By studying the problem structure, we can derive strong valid inequalities which lead to
better approximations of the ideal formulation conv (X ) and hence to tighter bounds.
Two definitions related to polyhedra and their description:
Consider a polyhedron P = {x ∈ Rn+ : Ax ≤ b}.
Definition: Given two valid inequalities π t x ≤ π0 and µt x ≤ µ0 for P, π t x ≤ π0
dominates µt x ≤ µ0 if ∃ u > 0 such that uµ ≤ π and π0 ≤ uµ0 with (π, π0 ) 6= (uµ, uµ0 ).
Poiché uµt x ≤ π t x ≤ π0 ≤ uµ0 , chiaramente {x ∈ Rn+ : π t x ≤ π0 } ⊆ {x ∈ Rn+ : µt x ≤ µ0 }
Example: x1 + 3x2 ≤ 4 dominates 2x1 + 4x2 ≤ 9 since for (π, π0 ) = (1, 3, 4) and
(µ, µ0 ) = (2, 4, 9) we have 12 µ ≤ π and π0 ≤ 21 µ0 .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
1 / 27
Definition: A valid inequality π t x ≤ π0 is redundant in the description of P
if there exist k ≥ 2 valid inequalities π i x ≤ π0i for P with ui > 0, 1 ≤ i ≤ k, such that
(
k
X
i=1
ui π i )x ≤
k
X
ui π0i dominates π t x ≤ π0 .
i=1
Example:
P = {(x1 , x2 ) ∈ R2+ : −x1 + 2x2 ≤ 4, −x1 − 2x2 ≤ −3, −x1 + x2 ≤ 5/3, 1 ≤ x1 ≤ 3}
−x1 + x2 ≤ 5/3 is redundant because it is dominated by −x1 + x2 ≤ 3/2, which is
implied by −x1 + 2x2 ≤ 4 and −x1 ≤ −1 (with u1 = u2 = 12 )
Observation: When P = conv (X ) is not known explicitly it can be very difficult to check
redundancy. In pratice, we should avoid inequalities that are dominated by other already
available inequalities.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
2 / 27
3.8.1 Faces and facets of polyhedra
Consider a polyhedron P = {x ∈ Rn : Ax ≤ b}
Definitions
The vectors x 1 , . . . , x k are affinely independent if the k − 1 vectors
x 2 − x 1 , . . . , x k − x 1 are linearly independent.
The dimension of P, dim(P), is equal to the maximum number of affinely linearly
independent points of P minus 1.
P is full-dimensional if dim(P) = n, i.e., no equation at x = b is satisfied with
equality by all the points x ∈ P.
Illustrations:
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
3 / 27
For the sake of simplicity, we assume that P is full dimensional
Theorem: If P is of full dimension, P admits a unique minimal description
P = {x ∈ Rn : ati x ≤ bi , i = 1, . . . , m}
where each inequality is unique within a positive multiple.
Each inequality is necessary: deleting anyone of them we obtain a polyhedron that differs
from P.
Moreover, each valid inequality for P which is not a positive multiple of one of the
ati x ≤ bi is redundant (can be obtained as linear combination with non negative
coefficients of two or more valid inequalities).
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
4 / 27
Alternative characterization of necessary inequalities
Definitions
Let F = {x ∈ P : π t x = π0 } for any valid inequality π t x ≤ π0 for P. Then F is a
face of P and the inequality π t x ≤ π0 represents or defines F .
If F is a face of P and dim(F ) = dim(P) − 1, then F is a facet of P.
Consequences: The faces of a polyhedron are polyhedra and a polyhedron has a finite
number of faces.
Theorem: If P is full dimensional, a valid inequality is necessary for the description of P
if and only if it defines a facet of P.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
5 / 27
Example
Consider the polyhedron P ⊂ R2 described by:
x1
+
2x2 ≤ 4
(1)
−x1
−
(2)
−x1
+
2x2 ≤ −3
3
x2 ≤
2
x1 ≤ 3
(4)
x1 ≥ 1
(5)
(3)
Verify that P is full dimensional.
Establish which inequalities define facets of P or are redundant.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
6 / 27
How to show that a valid inequality is facet defining
Consider X ⊂ Z+n and a valid inequality π t x ≤ π0 for X
Assumption: conv (X ) is bounded and full dimensional
Two simple approaches to show that π t x ≤ π0 defines a facet of P = conv (X ):
1) Direct approach (definition): Find n points x 1 , . . . , x n ∈ X that satisfy the
inequality with equality (π t x = π0 ) and are affinely independent.
2) Indirect approach:
(i) Select t points x 1 , . . . , x t ∈ X , with t ≥ n, that satisfy π x = π0 . Suppose that they
all belong to a generic hyperplane µt x = µ0 .
(ii) Solve the linear system
n
X
µj xjk = µ0
for k = 1, . . . , t
j=1
with the n + 1 unknowns µ0 , µ1 , . . . , µn .
(iii) If the only solution is (µ, µ0 ) = λ(π, π0 ) with λ 6= 0, then the inequality π x = π0
defines a facet of conv (X ).
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
7 / 27
Example:
Consider X = {(x, y ) ∈ Rm × {0, 1} :
Pm
i=1
xi ≤ my , 0 ≤ xi ≤ 1 ∀i}
i) Verify that dim(conv (X )) = m + 1.
ii) Show with the indirect approach that the valid inequality xi ≤ y defines a facet of
conv (X ).
Consider the points (0, 0), (e i , 1) and (e i + e j , 1) for j 6= i which are feasible and satisfy
xi = y .
Since (0, 0) belongs to the hyperplane defined by
Pm
Since (e i , 1) belongs to the hyperplane defined by
µi = −µm+1 .
k=1
Pm
µk xk + µm+1 y = µ0 , then µ0 = 0.
k=1
Since (e i + e j , 1) belongs to the hyperplane defined by
µj = 0 for j 6= i.
µk xk + µm+1 y = µ0 , then
Pm
k=1
µk xk − µi y = µ0 , then
Thus the hyperplane is µi xi − µi y = 0 and the inequality xi ≤ y defines a facet of
conv (X ).
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
8 / 27
3.8.2 Cover inequalities for the binary knapsack problem
Consider X = {x ∈ {0, 1}n :
Pn
j=1
aj xj ≤ b} with b > 0 and N = {1, . . . , n}.
Assumptions: For each j with 1 ≤ j ≤ n, aj > 0 (if aj < 0 set xj0 = 1 − xj ) and aj ≤ b.
Definition: A subset C ⊆ N is a cover for X if
each j ∈ C , C \ {j} is not a cover.
P
j∈C
aj > b. A cover is minimal if, for
Example: For X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}
two minimal covers are: {1, 2, 3} and {3, 4, 5, 6}
Proposition: If C is a cover for X , the inequality
X
xj ≤ |C | − 1
j∈C
is valid for X , and is called a cover inequality.
Example: For X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}
some minimal cover inequalities: x1 + x2 + x3 ≤ 2, x1 + x2 + x6 ≤ 2, x1 + x5 + x6 ≤ 2
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
9 / 27
Can such cover inequalities be strengthened?
Proposition: If C is a cover for X , the extended cover inequality
X
xj ≤ |C | − 1
j∈E (C )
is valid for X , where E (C ) = C ∪ {j ∈ N : aj ≥ ai for all i ∈ C }.
Example: Consider X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}.
The extended cover inequality for C = {3, 4, 5, 6} is
x1 + x2 + x3 + x4 + x5 + x6 ≤ 3
which clearly dominates
x3 + x4 + x5 + x6 ≤ 3.
(6)
Observation: Since a1 = 11, ai ≥ 5 for i ∈ {3, 4, 5}, a6 = 4 and b = 19, if x1 = 1 at
most one of the other variables in (6) can take value 1 and the inequality
2x1 + x3 + x4 + x5 + x6 ≤ 3
is valid and in turn dominates (6).
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
10 / 27
Lifting procedure: to strengthen such valid inequalities to obtain facet-defining ones
Let j1 , . . . , jr be the indices of N \ C and set t = 1.
Let
Pt−1
i=1
αji xji +
P
j∈C
xj ≤ |C | − 1 be the inequality obtained at iteration t − 1.
At iteration t: Determine the maximum value of αjt such that
αjt xjt +
t−1
X
i=1
αji xji +
X
xj ≤ |C | − 1
j∈C
is valid for X by solving the (binary knapsack) problem
Pt−1
P
σt = max
αji xji + j∈C xj
Pi=1
P
t−1
s.t.
i=1 aji xji +
j∈C aj xj ≤ b − ajt
x ∈ {0, 1}|C |+t−1
and by setting αt = |C | − 1 − σt .
Terminate when t = r .
N.B.: σt = maximum amount of ”space” used up by the variables of indices
C ∪ {j1 , . . . , jt−1 } when xjt = 1.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
11 / 27
Example:
Consider X = {x ∈ {0, 1}7 : 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}.
Applying the lifting procedure to
x3 + x4 + x5 + x6 ≤ 3
considering in the order x1 , x2 and x7 , we obtain the valid inequality
2x1 + x2 + x3 + x4 + x5 + x6 ≤ 3.
Proposition: If C is a minimal cover and ai ≤ b ∀i ∈ N, the lifting procedure is
guaranteed to yield a facet-defining inequality of conv (X ).
Example cont.:
The valid inequality
2x1 + x2 + x3 + x4 + x5 + x6 ≤ 3
defines a facet of conv (X ).
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
12 / 27
Separation of cover inequalities
Separation problem: Given a fractional solution x ∗ with 0 ≤ xj∗ ≤ 1, 1 ≤ j ≤ n,
decide whether x ∗ satisfies all the cover inequalities or determine a one violated by x ∗ .
P
P
Since j∈C xj ≤ |C | − 1 can be written as j∈C (1 − xj ) ≥ 1, we have to answer the
question:
P
P
Does there exist a subset C ⊆ N such that j∈C aj > b and j∈C (1 − xj∗ ) < 1?
If z ∈ {0, 1}n is the incidence vector (binary characteristic vector) of the subset C ⊆ N,
it is equivalent to the question:
P
P
ζ ∗ = min{ j∈N (1 − xj∗ )zj : j∈N aj zj > b, z ∈ {0, 1}n } < 1?
Proposition:
(i) If ζ ∗ ≥ 1, x ∗ satisfies all the cover inequalities.
P
(ii) If ζ ∗ < 1 with optimal solution z ∗ , then j∈C xj ≤ |C | − 1
with C = {j : zj∗ = 1, 1 ≤ j ≤ n}, cuts x ∗ by a quantity 1 − ζ ∗ .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
13 / 27
Example:
Consider
max
s.t.
z = 5x1 + 2x2 + x3 + 8x4
4x1 + 2x2 + 2x3 + 3x4 ≤ 4
x1 , x2 , x3 , x4 ∈ {0, 1}
∗
= 9.25.
Optimal solution of the LP relaxation x ∗LP = (1/4, 0, 0, 1) with zLP
The separation problem amounts to the following binary knapsack problem:
ζ ∗ = min
s.t.
3
z + z2 + z3
4 1
4z1 + 2z2 + 2z3 + 3z4 > 4
z1 , z2 , z3 , z4 ∈ {0, 1}
where the > constraint can be replaced with 4z1 + 2z2 + 2z3 + 3z4 ≥ 5.
Optimal solution z = (1, 0, 0, 1) with ζ ∗ = 43 .
Thus the cover inequality
x1 + x4 ≤ 1
cuts away the current LP optimal solution x ∗LP by 1 − ζ ∗ = 41 .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
14 / 27
3.8.3 Strong valid inequalities for the Maximum
Independent Set problem
Definitions
- A subset S ⊆ V is an independent/stable set if {i, j} ∈
/ E for each pair of nodes
i, j ∈ S.
- A subset K ⊆ V is a clique if {i, j} ∈ E for each pair of nodes i, j ∈ K , .
- A subset S ⊆ V is a maximal independent/stable set (in the sense of inclusion) if
S ∪ {i} is not independent/stable for each i ∈ V \ S.
- A subset K ⊆ V is a maximal clique if K ∪ {i} is not a clique for each i ∈ V \ S.
Illustrations:
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
15 / 27
Maximum Independent Set problem (MIS):
(maximum stable set)
Given an undirected G = (V , E ), determine an independent set S ⊆ V of maximum size.
Proposition: MIS is NP-hard.
MIS has a wide range of interesting applications.
For each i ∈ V , let the binary variable xi indicate whether i ∈ S.
ILP formulation:
max
s.t.
Edoardo Amaldi (PoliMI)
P
i∈V
xi
(7)
xi + xj ≤ 1
∀ {i, j} ∈ E
(8)
xi ∈ {0, 1}
∀i ∈ V
(9)
Ottimizzazione
A.A. 2013-14
16 / 27
Definition:
The bounded polyhedron
PIS = conv ({x ∈ {0, 1}|V | : xi + xj ≤ 1, ∀ {i, j} ∈ E })
is the independent set polytope.
PIS = conv (X ) where X = {x ∈ {0, 1}|V | : x incidence vector of an independent set of G }.
Observation: dim(PIS ) = n = |V |
Indeed: since 0 and the n unit vectors e i , 1 ≤ i ≤ n, belong to X , there are n + 1 affinely
independent vectors in X .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
17 / 27
1) Clique inequalities
Property: For any clique K ⊆ V of G , the clique inequality
X
xi ≤ 1
i∈K
is valid for X .
Observation: The undominated clique inequalities are those corresponding to the
maximal cliques.
Consider two cliques K and K 0 with K ⊂ K 0 , then
P
i∈K xi ≤ 1.
Edoardo Amaldi (PoliMI)
Ottimizzazione
P
i∈K 0
xi ≤ 1 clearly dominates
A.A. 2013-14
18 / 27
Proposition: A clique inequality
X
xi ≤ 1
(10)
i∈K
defines a facet of PIS if and only if K is a maximal clique in G .
Proof:
Suppose K = {1, . . . , k}. Clearly the vectors e i with i = 1, . . . , k, satisfy (10) with
equality.
Since K is a maximal clique, for each i 6∈ K , there is a node j(i) ∈ K such that
{i, j(i)} 6∈ E .
Denote by x i the binary vector with components i and j(i) equal to 1 and all others to 0.
Then x i ∈ X and it satisfies (10) with equality for i = k + 1, . . . , n.
Since e 1 , . . . , e k , x k+1 , . . . , x n are linearly (affinely) independent, inequality (10) is facet
defining.
Conversely,
if K is not maximal ∃ i 6∈ K such that K ∪ {i} is a clique and thus
P
l∈K ∪{i} xi ≤ 1 is valid for PIS .
P
Since (10) is the sum of −xi ≤ 0 and l∈K ∪{i} xi ≤ 1, it is not facet defining.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
19 / 27
Separation problem for clique inequalities:
Given a fractional optimal solution x ∗ of the current linear relaxation of (7)-(9)
P
max
i∈V xi
s.t.
xi + xj ≤ 1
∀ {i, j} ∈ E
constraints (10) generated so far
xi ≥ 0
∀i ∈ V ,
∗
find a clique inequality that is violated by x or establish that no such inequality exists.
Separation procedure: Given x ∗ with 0 ≤ xi∗ ≤ 1 for all i ∈ V as above, assign a weight
xi∗ to each node i ∈ V and look for a clique K ∗ ⊆ V of maximum total weight.
P
P
∗
∗
Indeed, if
i∈K ∗ xi > 1, the clique inequality
i∈K ∗ xi ≤ 1 is violated by x ,
otherwise all those not yet introduced in the partial ILP formulation are satisfied by x ∗ .
Since the this separation problem is NP-hard, heuristics are used.
N.B.: The larger class of so-called orthonormal representation (OR) valid inequalities,
includes the clique inequalities and can be separated in polynomial time.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
20 / 27
2) Odd hole inequalities
Definition: A subset H ⊆ V is a hole of G if the induced subgraph H is a simple cycle,
i.e., the nodes in H are connected via a cycle and for each pair of non adjacent nodes i, j
we have {i, j} ∈
/ E.
Example: outer cycle of a wheel with n = 6 nodes
Proposition: For each hole with an odd number of nodes, the odd hole inequality
X
i∈H
xi ≤ b
|H|
|H| − 1
c=
2
2
is valid for X .
We cannot select more than half of the nodes of each hole; for odd holes the upper
bound is |H|−1
.
2
Observation: The inequalities corresponding to even holes and odd holes with 3 nodes,
are implied by (8) and (10).
An odd hole with 3 nodes is a clique; every even hole inequality can be obtained by
aggregating the inequalities (8) associated to all the edges of the hole with multipliers 12 .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
21 / 27
Separation problem: Given x ∗ optimal solution of the current linear relaxation, find an
odd hole inequality that is violated by x ∗ or establish that no such inequality exists.
Proposition: Odd hole inequalities can be separated by computing a shortest path
between n = |V | pairs of nodes in an ad hoc bipartite graph with appropriate edge costs.
Separation algorithm:
Given an optimal solution x ∗ as above, we look for a violated odd hole inequality.
Construct an auxiliary bipartite graph G 0 = (V 0 , E 0 ), including each node i ∈ V of G in
V1 and a copy i 0 in V2 .
For each edge {i, j} ∈ E , include in E 0 (G 0 ) two edge {i, j 0 } and {i 0 , j}, where i, j ∈ V1
and i 0 , j 0 ∈ V2 .
To each edge {i, j 0 } ∈ E 0 we assign a cost 1 − xi∗ − xj∗ .
This way, a cycle C in G 0 has cost
X
X ∗
X ∗
(1 − xi∗ − xj∗ ) = |C | − 2
xi = |V 0 (C )| − 2
xi ,
{i,j}∈C
i∈V 0 (C )
i∈V 0 (C )
where V 0 (C ) is the set of nodes of C in G 0 .
P
If such cost < 1, then i∈V (C ) xi ≤ |V (C2)|−1 is violated by x ∗ .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
22 / 27
To consider all possible odd holes, for each i ∈ V1 it suffices to look for a minimum cost
path pi∗ from node i ∈ V1 to its copy i 0 ∈ V2 .
- If pi∗ is a hole and its cost is < 1, we have a violated inequality.
- If pi∗ is not a hole and its cost < 1, then it contains an odd subcycle whose cost is
necessarily smaller than 1 (since the costs xi∗ ≥ 0), which corresponds to a violated
odd hole inequality.
- If all the paths found have a costs ≥ 1, x ∗ satisfy all the odd hole inequalities.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
23 / 27
Example:
G = (V , E ) is the wheel with 6 nodes and 10 edges (5 nodes in the outer cycle/hole).
i) List all the maximal clique inequalities.
∗
ii) Indicate the unique optimal solution xLP
of the linear programming relaxation
including all the maximal cliques inequalities.
iii) Determine the violated odd hole inequality.
iv) Lift this odd hole inequality so that it defines a facet of the Independent Set polytope
PIS .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
24 / 27
3.8.4 Equivalence between separation and optimization
Consider a family of LPs min{ c t x : x ∈ Po } with o ∈ O, where
Po = { x ∈ Rno : Ao x ≥ b o } polytope with rational (integer) coefficients and a very
large (e.g., exponential) number of constraints.
Examples:
1) Linear relaxation of asymmetric TSP with cut-set inequalities (O set of all graphs)
2) Maximum Matching problem: For each G = (V , E ), the matching polytope
X
conv ({x ∈ {0, 1}|E | :
xe ≤ 1, ∀i ∈ V })
e∈δ(i)
coincides (Edmonds) with
X
X
|S| − 1
|E |
xe ≤ 1, ∀i ∈ V ,
xe ≤
{x ∈ R+ :
, ∀S ⊆ V with |S| ≥ 3 odd}
2
e∈δ(i)
e∈E (S)
We adopt a cutting plane approach where constraints are only generated if needed.
Assumption: Even though the number of constraints mo of Po is exponential in no
(e.g., O(2no )), Ao and b o are specified in a concise way (as a function of a polynomial
number of parameters w.r.t. no ).
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
25 / 27
Optimization problem: Given a rational polytope P ⊆ Rn and a rational objective
vector c ∈ Rn , find a x ∗ ∈ P minimizing c t x over x ∈ P or establish that P is empty.
N.B.: we assume that P is bounded (polytope) just to avoid unbounded problems.
Separation problem: Given a rational polytope P ⊆ Rn and a rational vector x 0 ∈ Rn ,
establish that x 0 ∈ P or determine a rational vector π ∈ Rn such that πx < πx 0 for each
x ∈ P (indentify a cut that separate x 0 from P).
Theorem: (consequence of Grötschel, Lovász, Schriver 1988 theorem)
The separation problem for a family of polyhedra can be solved in polynomial time in n e
log U if and only if the optimization for that family can be solved in polynomial time in n
and log U, where U is an upper bound on all aij and bi .
Proof based on the Ellipsoid method, first LP polynomial algorithm (Khachiyan 1979).
For now theoretical tool: the resulting algorithm is not efficient but the equivalence may
guide the search for more practical polynomial-time algorithms.
Corollary: The linear relaxation of the ILP formulation for ATSP with cut-set inequalities
can be solved in polynomial time in spite of the exponential number of constraints.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
26 / 27
3.8.5 Remarks on cutting plane methods
Consider a generic discrete optimization problem
min{ c t x : x ∈ X ⊆ Rn+ }
with rational coefficients ci .
When designing a cutting plane method, we should be aware that:
It can be difficult to describe one or more families of strong (possibly facet defining)
valid inequalities for conv (X ).
The separation problem for a given family F may require a considerable
computational effort (if NP-hard we devise heuristics).
Even when finite convergence is guaranteed (e.g., with Gomory cuts), pure cutting
plane methods tend to be very slow.
To solve hard/large problems optimally, it is often necessary to embed strong valid
inequalities into a Branch-and-Bound framework, leading to a so-called Branch and Cut
method.
The subfield of Discrete Optimization studying the polyhedral structure of the ideal
formulations (conv (X )) is known as Polyhedral Combinatorics.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
27 / 27