3.2 Alternative, strong and ideal formulations
In linear optimization, good formulations are those with a small number of variables n
and constraints m, because the computational complexity grows polynomially in n and
m.
Since efficient algorithms (simplex and interior points methods) are available, the choice
of the formulation is important but it does not critically affect the possibility of solving
the problems.
The situation for ILP and MILP problems is completely different: extensive
computational experiments indicate that the choice of the formulation is crucial.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
1 / 27
3.2.1 Alternative and strong formulations
Definition: Given any MILP
zMILP = min
s.t.
c t1 x + c t2 y
A1 x + A2 y ≥ b
(1)
x ≥ 0, y ≥ 0 integer
(2)
its linear (programming) relaxation is the following Linear Program (LP):
zLP = min
s.t.
c t1 x + c t2 y
A1 x + A2 y ≥ b
(3)
x ≥ 0, y ≥ 0,
(4)
where the integrality restriction on the variables yj is omitted.
If in the MILP an integer variable yj is such that 0 ≤ yj ≤ uj , then in the linear
relaxation yj ∈ [0, uj ].
Illustration of the feasible region of the linear relaxation (MILP and ILP):
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
2 / 27
Let
XMILP = {(x, y ) ∈ Rn1 × Zn2 : A1 x + A2 y ≥ b, x ≥ 0, y ≥ 0}
denote the feasible region of the MILP and
XLP = {(x, y ) ∈ Rn1 × Rn2 : A1 x + A2 y ≥ b, x ≥ 0, y ≥ 0}
the feasible region of its linear relaxation.
Obviously: XMILP ⊆ XLP
Proposition: For any MILP with a minimization objective function, we have:
zLP ≤ zMILP , in other words zLP is a lower bound for zMILP ,
if the optimal solution x ∗LP of the linear relaxation is feasible for the original MILP
(ILP), it is also optimal for it.
For maximization problems, we clearly have zMILP ≤ zLP .
The same holds for ILP problems.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
3 / 27
Definition:
A polyhedron P = {(x, y ) ∈ Rn1 +n2 : A1 x + A2 y ≥ b, x ≥ 0, y ≥ 0} ⊆ Rn1 +n2 is a
formulation for a mixed integer set X ⊆ Rn1 × Zn2 if and only if X = P ∩ (Rn1 × Zn2 ).
Illustration of alternative formulations for an ILP:
P clearly depends on the constraints used in the formulation to define the feasible region
X.
N.B.: In modeling fixed costs, we did not consider X = {(0, 0), (x, 1) with 0 < x ≤ 1}
but the set X ∪ {(0, 1)}.
Observation: Any MILP/ILP admits an infinite number of alternative formulations that
are equivalent from the IP point of view (with the integrality restrictions) but whose
linear relaxations have different feasible regions.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
4 / 27
Examples:
1) Two alternative formulations for TSP: with cut-set or subtour-elimination constraints.
2) Original formulation for UFL:
Pn
Pm Pn
min
j=1 fj yj
i=1
j=1 cij xij +
Pn
x
=
1
s.t.
ij
Pmj=1
i=1 xij ≤ myj
∀i ∈ M
∀j ∈ N
yj ∈ {0, 1}
∀j ∈ N
0 ≤ xij ≤ 1
∀i ∈ M, j ∈ N
(5)
with n constraints (5) that link the corresponding variables xij and yj .
Alternative formulation for UFL:
Pn
Pm Pn
min
j=1 fj yj
j=1 cij xij +
i=1
Pn
x
=
1
s.t.
j=1 ij
xij ≤ yj
∀i ∈ M
∀i ∈ M, j ∈ N
yj ∈ {0, 1}
∀j ∈ N
0 ≤ xij ≤ 1
∀i ∈ M, j ∈ N
(6)
with mn constraints (6) that link the corresponding variables xij and yj .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
5 / 27
Definition:
Given a mixed integer set X ⊆ Rn1 × Zn2 and two formulations P1 and P2 for X ,
formulation P1 is stronger than formulation P2 if P1 ⊂ P2 .
Indeed, the lower bound on zMILP provided by the linear relaxation P1 is not smaller
(weaker) than that provided by P2
zMILP = min{c t1 x + c t2 y : (x, y ) ∈ X }
≥ min{c t1 x + c t2 y : (x, y ) ∈ P1 }
≥ min{c t1 x + c t2 y : (x, y ) ∈ P2 }.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
6 / 27
Examples:
1) Uncapacitated Facility Location (UFL)
Proposition: The linear relaxation of the MILP formulation with constraints
xij ≤ yj is
P
stronger than that of the MILP formulation with aggregate constraints m
i=1 xij ≤ myj .
Let
P1 =
n
(x, y ) ∈ Rmn+n :
Pn
j=1 xij
= 1, ∀i, xij ≤ yj , ∀i, ∀j, 0 ≤ xij ≤ 1, ∀i, ∀j, 0 ≤ yj ≤ 1, ∀j
o
P =
n2
o
Pn
Pm
(x, y ) ∈ Rmn+n :
j=1 xij = 1, ∀i,
i=1 xij ≤ myj , ∀j, 0 ≤ xij ≤ 1, ∀i, ∀j, 0 ≤ yj ≤ 1, ∀j
Obviously P1 ⊆ P2 (sum of m constraints xij ≤ yj for a given j yields
Pm
i=1
xij ≤ myj ).
It is easy to exhibit a point (x, y ) in P2 \ P1 :
Suppose m = kn for some integer k ≥ 2, and let each depot serve k clients:
xij = 1 for i = k(j − 1) + 1, . . . , k(j − 1) + k, j = 1, . . . , n, and xij = 0 otherwise,
yj = k/m for j = 1, . . . , n.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
7 / 27
2) Symmetric TSP (STSP)
STSP: Given an undirected graph G = (V , E ) and a cost ce for every edge
e = {i, j} ∈ E , determine a Hamiltonian cycle of G (i.e., a cycle that visits exactly once
each i ∈ V and returns to the starting node) of minimal total cost.
Two alternative formulations:
P
min
P e∈E ce xe
s.t.
x =2
P e∈δ(i) e
x
≤ |S| − 1
e
e∈E (S)
xe ∈ {0, 1}
and
min
s.t.
P
P e∈E ce xe
x =2
P e∈δ(i) e
e∈δ(S) xe ≥ 2
xe ∈ {0, 1}
i ∈V
S ⊂ V , S 6= ∅
e∈E
i ∈V
S ⊂ V , S 6= ∅
e ∈ E,
(DEG )
(SEC )
(DEG )
(CUT )
where δ(S) = {{i, j} ∈ E : i ∈ S, j ∈ V \ S}, δ(i) = δ({i}),
and E (S) = {{i, j} ∈ E : i ∈ S, j ∈ S}.
(DEG), (SEC) and (CUT) are, respectively, the degree, subtour-elimination and cut-set
constraints.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
8 / 27
Let Psec and Pcut be the polyhedra (feasible regions) of the linear relaxations of these
two formulations.
Proposition: The two formulations are equally strong (Psec = Pcut ).
Proof: We verify that: a) Psec ⊆ Pcut and b) Pcut ⊆ Psec .
For any subset S ⊂ V , the sum of the inequalities (DEG) with coefficients 1 over all
i ∈ S yields
X X
xe = 2|S|.
i∈S e∈δ(i)
P P
Note that in i∈S e∈δ(i) xe all the edges e = {i, j} with i ∈ S e j ∈ S occur twice
(once in the sum over δ(i) and once in that over δ(j)), while the edges e = {i, j} with
i ∈ S, j ∈ V \ S occur just once.
The aggregate equation can be rewritten as
X
X
2
xe +
xe = 2|S|.
e∈E (S)
Edoardo Amaldi (PoliMI)
e∈δ(S)
Ottimizzazione
A.A. 2013-14
9 / 27
Since
2
X
xe +
e∈E (S)
X
xe = 2|S|
e∈δ(S)
P
a) From (SEC) we have 2 e∈E (S) xe ≤ 2|S| − 2.
P
P
Thus 2|S| − e∈δ(S) xe = 2 e∈E (S) xe ≤ 2|S| − 2, that is
X
xe ≥ 2.
e∈δ(S)
P
b) (CUT) amounts to e∈δ(S) xe ≥ 2.
P
Thus 2|S| ≥ 2 e∈E (S) xe + 2, that is
X
xe ≤ |S| − 1.
e∈E (S)
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
10 / 27
3.2.2 Ideal formulations
Theorem (Meyer): Let X ⊆ Rn1 × Zn2 be the mixed integer feasible set (feasible
region) of an arbitrary MILP with rational coefficients, then conv (X ) is a rational
polyhedron. Moreover, the extreme points of conv (X ) belong to X .
For bounded sets X with integer points, intuitive and no need for the rational coefficients
assumption.
Consequence:
min{c t x : x ∈ X } = min{c t x : x ∈ conv (X )}
If we knew conv (X ) explicitly, we could solve the left-hand-side MILP/ILP by solving a
single Linear Program (by finding an optimal extreme point to the right-hand-side linear
optimization problem)!
Clearly the feasible region P of the linear relaxation of any formulation satisfies
X ⊆ conv (X ) ⊆ P.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
11 / 27
Definition: Let X ⊆ Rn1 × Zn2 be any mixed integer feasible set, the ideal (tight)
formulation for X is the polyhedron P ⊆ Rn1 +n2 with P = conv (X ).
Since, in practice, the ideal formulation is often of exponential size or difficult to
determine (also for bounded X ), we strive for formulations that closely approximate
conv (X ).
Examples:
1) Assignment problem
Natural ILP formulation:
min
s.t.
Pn
Pn
cij xij
i=1
Pn j=1
x
=
1
ij
i=1
Pn
j=1 xij = 1
∀j
xij ∈ {0, 1}
∀i, ∀j
∀i
Proposition: The polyhedron corresponding to the linear relaxation of this formulation
Pn
Pn
2
P = {x ∈ Rn :
j=1 xij = 1 ∀i, 0 ≤ xij ≤ 1 ∀i, j}
i=1 xij = 1 ∀j,
is an ideal formulation for the Assignment problem.
Proof later
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
12 / 27
2) Perfect Matching problem (PM)
PM: Given an undirected graph G = (V , E ) with an even number n = |V | of nodes and
a cost ce for each edge e = {i, j} ∈ E , determine a perfect matching, i.e., a subset of
edges without common nodes but incident to all the nodes, of minimum total cost.
Example:
Application: Each node represents a person, and each edge e = {i, j} ∈ E the fact that
persons i and j can be matched.
A natural ILP formulation:
min
s.t.
P
P
e∈E
ce xe
e∈δ(i) xe = 1
xe ∈ {0, 1}
∀i ∈ V
∀e ∈ E ,
where xe = 1 if edge e is selected, and xe = 0 otherwise.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
13 / 27
It easy to verify that all binary vectors x ∈ {0, 1}|E | corresponding to matchings satisfy
the class of inequalities:
X
∀S ⊂ V , |S| odd
xe ≥ 1
e∈δ(S)
Theorem (Edmonds):
P
P
PM = {x ∈ R|E | :
e∈δ(i) xe = 1 ∀i ∈ V ,
e∈δ(S) xe ≥ 1 ∀S ⊂ V , |S| odd,
0 ≤ xe ≤ 1 ∀e ∈ E }
is an ideal formulation for the Perfect Matching problem.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
14 / 27
3.2.3 Extended formulations
Alternative formulations can use additional and/or different variables.
Definition: The formulations including additional variables, are extended formulations.
Example: Uncapacitated Lot-Sizing problem (ULS)
A natural formulation:
xt = amount produced in period t, with 1 ≤ t ≤ n
st = amount in stock at the end of period t, with 0 ≤ t ≤ n
yt = 1 if production occurs in period t and yt = 0 otherwise, with 1 ≤ t ≤ n
min
s.t.
Pn
t=1
pt xt +
Pn
t=1
ht s t +
Pn
t=1 ft yt
st = st−1 + xt − dt
∀t
xt ≤ Myt
∀t
s0 = 0, sn = 50
∀t
s t , xt ≥ 0
∀t
yt ∈ {0, 1}
P
where M > 0 is large enough (e.g., M = nt=1 dt + sn − s0 )
Edoardo Amaldi (PoliMI)
Ottimizzazione
∀t
A.A. 2013-14
15 / 27
MILP extended formulation for ULS
Decision variables:
wit = amount produced in period i to satisfy the demand in period t, with
1≤i ≤t ≤n+1
yt = 1 if production occurs in period t and yt = 0 otherwise, with 1 ≤ t ≤ n
min
s.t.
Pn
i=1
Pn
t=i
P
t
Pn
cit wit +
i=1
Pn
t=1 ft yt
wit = dt
i=1 wi,n+1 = 50
∀t, 1 ≤ t ≤ n
(stock at the end)
wit ≤ dt yi
∀i, t, i ≤ t
wit ≥ 0
∀i, t, i ≤ t
yt ∈ {0, 1}
∀t
with aggregate costs (production and storage) cit = pi + hi + . . . + ht−1 .
N.B.: For each period i, we can add the constraint xi =
variable xi in terms of the ”new” variables wit .
Edoardo Amaldi (PoliMI)
Ottimizzazione
Pn
t=i
wit expressing the ”old”
A.A. 2013-14
16 / 27
Comparison between fromulations with different variables
For simplicity of notation, consider a formulation with only integer variables (ILP
problem)
min{c t x : x ∈ P1 ∩ Zn }
n
with P1 ⊆ R , and an extended formulation
′
min{c t (x, w ) : (x, w ) ∈ P2 ∩ (Zn × Rn )}
′
with P2 ⊆ Rn × Rn .
′
Definition: Given a polyhedron P ⊆ Rn × Rn , the orthogonal projection of P onto the
x-subspace Rn is the polyhedron
′
projx (P) = {x ∈ Rn : ∃ w ∈ Rn s.t. (x, w ) ∈ P }.
Example: orthogonal projection of a tridimensional polyhedron
′
Edoardo Amaldi
(PoliMI)
To compare
a formulation
P1 ⊆ Rn and Ottimizzazione
an extended formulation P2 ⊆ Rn ×A.A.
Rn2013-14
, we
17 / 27
One way to determine the orthogonal projection of polyhedra onto subspaces:
Fourier-Motzkin elimination method (1827)
First procedure to find a feasible solution of systems of linear inequalities.
Idea: At each iteration one variable is eliminated (an equivalent linear inequality system
is derived which does not contain that variable), the process ends when a system with a
single variable is obtained.
Given Ax ≤ b, suppose we wish to eliminate variable xi
The equivalent system without xi includes
all the inequalities of Ax ≤ b in which xi does not appear,
the inequalities resulting from all the possibile combinations of the upper and lower
bounds on xi implied by all the inequalities of Ax ≤ b containing xi .
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
18 / 27
Example:
x1
1
− x1
2
+x2
≥3
(7)
+x2
≥0
(8)
−x2
≥ −2
(9)
Eliminate x2 (project the polyhedron P of the feasible solutions of (7)-(9) into the
subspace of x1 ):
3 − x1 ≤ x2
1
x1 ≤ x2
2
x2 ≤ 2
and, considering all the pairs of inequalities ( ... ≤ x2 and x2 ≤ ... ), one obtains
3 − x1 ≤ 2
1
x1 ≤ 2,
2
namely 1 ≤ x1 ≤ 4, hence the projection [1, 4].
Eliminate x1 (project the polyhedron P into the subspace of x2 ):
one obtains 1 ≤ x2 ≤ 2 and hence the projection [1, 2].
Complexity: the number of constraints can grow exponentially in the number of
variables in the original system.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
19 / 27
Comparing formulations for ULS:
Consider the formulation P1 defined by
st = st−1 + xt − dt
∀t
xt ≤ Myt
∀t
s0 = 0, st ≥ 0, xt ≥ 0, 0 ≤ yt ≤ 1
∀t
and projx,s,y (P2 ), with P2 defined by
Pt
i=1 wit = dt
wit ≤ dt yi
P
xi = nt=i wit
wit ≥ 0
0 ≤ yt ≤ 1
(10)
∀t
∀i, t, i ≤ t
(11)
∀i
(12)
∀i, t, i ≤ t
∀t.
It is easy to verify that projx,s,y (P2 ) ⊂ P1 :
for instance, the point xt = dt , yt = dt /M for each t is an extreme point of P1 that does
not belong to projx,s,y (P2 ).
Proposition: P2 is the ideal formulation that describes the convex hull of all the feasible
(mixed integer) solutions of ULS.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
20 / 27
3.2.4 Stronger extended formulations
A way to strengthen a formulation is to look for an extended formulation involving
additional variables whose projection is a better approximation of the ideal formulation
(conv (X )).
Example:
Fixed charge network flow problem (FCNF):
Given a directed graph G = (V , A) with
a positive fixed cost fij , a unit cost cij and a capacity uij for each arc (i, j) ∈ A,
a demand
P bi for each node i ∈ V (bi < 0 for sources and bi > 0 for destinations)
with i∈V bi = 0,
determine a feasible flow of minimum total cost which satisfies all demands and capacity
constraints.
Example:
Proposition: FCNF is NP-hard.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
21 / 27
1) Natural MILP formulation:
Decision variables:
xij = amount of flow through arc (i, j), for all (i, j) ∈ A
yij = 1 if arc (i, j) is used and yij = 0 otherwise, for all (i, j) ∈ A
min
s.t.
P
P
(i,j)∈A (cij xij
+ fij yij )
∀i ∈ V
(13)
0 ≤ xij ≤ uij yij
∀(i, j) ∈ A
(14)
yij ∈ {0, 1}
∀(i, j) ∈ A
(h,i)∈δ − (i)
xhi −
P
(i,j)∈δ + (i)
xij = bi
where δ + (i) = {(i, j) ∈ A : j ∈ V } and δ − (i) = {(h, i) ∈ A : h ∈ V }
The linear relaxation of this natural formulation yields poor bounds because of the weak
coupling between the variables xij and yij , imposed by (14).
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
22 / 27
2) Multi-commodity extended MILP formulation:
Suppose w.l.o.g. that there is a single source node (say node s, with
Idea: P
bs = i∈V \{s} bi ), and decompose the flows according to their destinations.
Let K ⊆ V be the set of nodes with strictly positive demand.
Define one ”commodity” for each k ∈ K , with the flow variables xijk for all (i, j) ∈ A.
Define dik = −bk if i = s, dik = bi if i = k, and dik = 0 otherwise.
Decision variables:
xijk flow of commodity k in arc (i, j) destined to node k, ∀(i, j) ∈ A and ∀k ∈ K
yij = 1 if arc (i, j) is used and yij = 0 otherwise, ∀(i, j) ∈ A
P
P
k
min
k∈K xij ) + fij yij )
(i,j)∈A (cij (
P
P
k
k
k
∀i ∈ V , ∀k ∈ K
s.t.
(i,j)∈δ + (i) xij = di
(h,i)∈δ − (i) xhi −
0≤
xijk
≤ min{uij , bk }yij
yij ∈ {0, 1}
(15)
∀(i, j) ∈ A, ∀k ∈ K
(16)
∀(i, j) ∈ A
Significantly stronger (re)fromulation of FCNF (constraints (16) are tighter than (14))
but with |K | times more variables and constraints.
See Computer Lab session 1
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
23 / 27
3.2.5 Remarks on the strength and size of formulations
Definition: A compact formulation for a given problem is a formulation with a number
of variables and constraints that is polynomial w.r.t. the instance size.
Remark 1: A compact extended formulation can be much weaker than an alternative
exponential-size formulation.
Example: Asymmetric TSP (ATSP)
A compact extended formulation (Miller-Tucker-Zemlin) involving, for each i ∈ V , an
additional real variable ti representing the ”position” in which node i is visited:
min
s.t.
P
P
(i,j)∈A
cij xij
xij = 1
∀i ∈ V
i∈V :i6=j xij = 1
∀j ∈ V
Pj∈V :j6=i
tj ≥ ti + 1 − n(1 − xij )
xij ∈ {0, 1}
∀i ≥ 2, ∀j ≥ 2, i 6= j
(17)
∀(i, j) ∈ A,
where the O(n2 ) arc constraints (17) (also involving xij s) prevent all subtours that do
not contain node 1.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
24 / 27
The linear relaxation bound of this compact formulation is much weaker than that
provided by the formulations with subtour-elimination or cut-set constraints!
See Computer Lab session 2
By using Farkas Lemma, we can express this formulation only in terms of the xij variables:
P
min
(i,j)∈E cij xij
P
s.t.
xij = 1
∀i ∈ V
Pj∈V :j6=i
∀j ∈ V
i∈V :i6=j xij = 1
P
|C |
∀C ∈ C
(18)
(i,j)∈C xij ≤ |C | − |V |−1
xij ∈ {0, 1}
∀ (i, j) ∈ A,
where C denotes the set of all the circuits in the subgraph G ′ = (V \ {1} , A \ δ({1})).
It is easy to verify that constraints (18) are implied by the subtour-elimination
constraints (SEC) of Dantzig-Fulkerson-Johnson formulation:
X
xij ≤ |S| − 1
∀S ⊂ V \ {1} , |S| ≥ 2.
(i,j)∈E (S)
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
25 / 27
Example: G = (V , E ) complete with n = |V | ≥ 6
For S = {2, 3, 4, 5} the (SEC) constraint is
x23 + x32 + x34 + x43 + x45 + x54 + x52 + x25 + x35 + x53 + x24 + x42 ≤ 3
(19)
In the MTZ formulation, constraints (18) for all subtours involving nodes {2, 3, 4, 5} are
4
|V | − 1
4
x32 + x25 + x54 + x43 ≤ 4 −
|V | − 1
4
x23 + x35 + x54 + x42 ≤ 4 −
|V | − 1
4
x32 + x45 + x53 + x24 ≤ 4 −
|V | − 1
4
x34 + x25 + x53 + x42 ≤ 4 −
|V | − 1
4
x43 + x52 + x35 + x24 ≤ 4 −
,
|V | − 1
x23 + x34 + x45 + x52 ≤ 4 −
where (4 −
4
)
|V |−1
≥ 3 since |V | = 6.
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
26 / 27
Remark 2: A compact extended formulation can have a projection into the space of the
natural variables that is of exponential size.
Example: ATSP
Edoardo Amaldi (PoliMI)
Ottimizzazione
A.A. 2013-14
27 / 27