5. Integer Programming
Many practical problems require integer solutions. A linear programming problem in which some
or all of the variables are constrained to be integer-valued is known as an integer linear program
(IP or ILP). If all the variables are integer valued the problem is a pure integer LP. A problem
that contains both integer and continuous valued variables is a mixed integer LP (often abbreviated
to MIP).
In pure IP problems, the feasible region is contained in the integer lattice of points Zn : IP’s are
generally harder to solve than LP’s. However IP and MIP problems frequently arise in practice.
We will examine two solution techniques:
Method of cutting planes (Gomory)
Method of branch and bound (B&B)
5.1 Gomory’s cut (for pure IP)
The standard form of the pure IP problem (minimization case) is
Minimize
cT x
subject to
Ax = b
0 and integer
x
where c (n
1) ; A (m
n) ; b (m
1) are integer-valued.coe¢ cients.
We seek x 2 Zn . Geometrically, we might expect the optimal solution to a ILP to be “not
very far” from the optimal solution to the LP relaxation (de…ned as the LP problem obtained
by ignoring or relaxing the integer restriction on the variables). This is sometimes true, but often
not the case. However it seems a good idea, …rst to solve the relaxation by the simplex method.
Proposition 1 (Optimality of relaxation) Let xR be the solution to the LP relaxation. If xR
is integer (i.e. feasible for IP) then xR is optimal for the IP.
Proof. (minimization case)
Let FIP , FR be the feasible regions of the IP and its relaxation, respectively. Relaxing a LP
problem removes the integer constraint, so enlarges the feasible region. Hence FIP
FR . As the
objective functions for both problems (the IP and its relaxation) are the same, we have zR
zIP :
We have shown that the optimal value of the LP relaxation is a lower bound for the optimal value
of the IP.
If, however, xR is all integer then xR 2 FIP so zIP
Therefore zR = zIP and xR is optimal for IP:
1
zR since zIP = min fz (x) : x 2 FIP g :
Properties of a cutting plane
CP1 The cut removes the current LP solution from FR
CP2 It doesn’t cut away any integer feasible solutions. FIP is unchanged.
The equation of the cutting plane is derived from a source row in the …nal optimal tableau
for the LP relaxation. A new equation, involving an extra slack variable and having the required
two properties stated above, is added to the tableau making the tableau infeasible. Dual simplex
iterations are carried out to …nd a new LP solution. If this LP solution is integer then we stop
having solved the IP. Otherwise we generate a new cut and solve the resulting problem by dual
simplex iterations. This process can be shown to converge.
Algorithm
1. Solve the LP relaxation giving x (the current optimum) and z (the current optimal value)
2. If x 2 Zn then stop; x solves the ILP. Otherwise
3. (At least one variable in the basis is fractional valued). Corresponding rows of the tableau
are candidates for the source row
Choose a source row using some criterion e.g. mini jfi0
1
2j
(a heuristic) where fi0 denotes
the fractional part of the tableau right hand side y0 b . Construct a cut and add a row to
the tableau. Go to Step 1.
The cut equation (pure IP)
For pure IP problems, Gomory’s cut, sometimes termed a fractional cut, takes the form
X
fij xj
fi0
(5.1)
j2J
or equivalently, introducing a slack variable s
s
P
fij xj =
fi0
(5.2)
j2J
where
row i of the optimal tableau for the LP relaxation is the chosen source row, corresponding to
some basic variable xi having a fractional (non integer) value yi0 at the current optimum.
fij is the fractional part of the tableau entry yij ; j 2 J [ f0g
(J = index set of non-basic variables at the current optimum)
2
Result (Validity of cut equation)
The cut Eq.(5.2) can be shown to satisfy the required properties
Proof. Let xi be a fractional valued basic variable in the …nal tableau (for the LP relaxation).
The tableau equation has the canonical form
xi +
X
yij xj = yi0
j2J
which may be written
xi +
X
(nij + fij ) xj = ni0 + fi0
j2J
where nij is the integer part of yij and fij is the fractional part.
e.g. if yij = 3:6 then nij = 3 and fij = 0:6
if yij =
3:6 then nij =
4 and fij = 0:4
xi +
X
nij xj
X
ni0 = fi0
j2J
fij xj
(5.3)
j2J
- collecting all integer terms to the lhs.
At the current LP optimum x we have xj = 0 for all j 2 J so fi0 >
P
j2J
fij xj : Thus s < 0 in
Eq.(5.2) and x is infeasible for the problem with the cut added proving property P1.
At any feasible integer solution the lhs. of (5.3) is integer-valued. Therefore the rhs. must
also be integer-valued. Hence rhs.2 f0; 1; 2; ::g so rhs. of (5.3)
0; hence Eq.(5.2) is satis…ed at
every integer feasible solution, proving property P2.
Example
Maximize
s.t.
7x1 + 9x2
x1 + 3x2
7x1 + x2
x1 ; x2
6
35
0 and integer
The optimal tableau to the LP relaxation is
x2
x1
s2
s1
1
22
3
22
30
22
7
22
1
22
56
22
Using the x2 row as the source row criterion jfi0
3
7
2
9
2
63
1
2j
is tied leads to
x2
x1
s3
s2
s1
1
22
3
22
1
22
30
22
7
22
1
22
7
22
56
22
After a dual simplex iteration we …nd rhs.
7
2
9
2
1
2
63
0, therefore have found a new optimal LP solution
s2
s3
x2
0
1
3
x1
1
7
1
7
1
7
22
7
32
7
11
7
1
8
59
s1
which however is still fractional. We obtain a new cut using x1 as the source row.
s2
s3
x2
0
1
3
x2
3
x1
1
7
1
7
1
7
1
7
22
7
6
7
32
7
11
7
4
7
x1
4
s1
1
s2
4
1
8
59
s1
s4
s4
s3
55
and …nd an optimal integer feasible solution after one iteration. Therefore this solves the ILP. We
can retrace the steps of the above general proof for the 2nd cut in this example. The source row
x1 + 17 s2
1
7 s3
= 4 47
may be rewritten as
x1
s3
4=
4
7
1
7 s2
6
7 s3
At any feasible integer solution therefore, we have
4
7
1
7 s2
6
7 s3
0
which is the constraint represented by the s4 row in the penultimate tableau.
5.2 The mixed integer cut
Consider xi to be an integer variable of a MIP which, as before, is not integer valued in the optimal
tableau of the LP relaxation There must be such an xi otherwise the optimal LP solution is feasible
for MIP and we are done.
4
The equation corresponding to the i th tableau row is
xi +
X
yij xj = yi0 = ni0 + fi0
j2J
xi
X
ni0 = fi0
(5.4)
yij xj
(5.5)
j2J
At any feasible solution to MIP, xi is integer valued and the lhs. of Eq. 5.5 is integer.
Therefore either
Case A: rhs.
0; or
Case B: rhs.
1 must hold.
De…ne
J+
= fj : yij
0g and J
= fj : yij < 0g : Using the property that all xj
0 at a
feasible solution to MIP (we no longer have the integer property holding), we either have for Case
A:
X
yij xj
fi0
j2J
X
X
yij xj
j2J +
so, as the second term
0 (xj
0 and
yij xj
fi0
j2J
yij
0 for j 2 J ) we have
X
yij xj
fi0
(5.6)
j2J +
or, for Case B
X
X
yij xj
(1
fi0 )
yij xj
(1
fi0 )
yij xj
(1
fi0 )
j2J
yij xj +
j2J +
X
j2J
X
j2J
since xj
0 at any feasible solution to MIP. Therefore
fi0
1 fi0
X
yij xj
fi0
(5.7)
j2J
Combining the mutually exclusive inequalities (5.6) and (5.7) we obtain Gomory’s mixed integer cut
X
yij xj +
j2J +
{If either
or
with ;
X
fi0
1 fi0
0 then
+
5
yij xj
j2J
}
fi0 :
(5.8)
Introducing the slack variable s; we obtain
s
P
P
fi0
1 fi0
yij xj +
j2J +
yij xj =
fi0
(5.9)
j2J
Notice that all the coe¢ cients of terms on the lhs. are in fact
0 when the signs of yij are
taken into account.
Example (MIP cut)
Previous example with only x1 integer
Maximize
7x1 + 9x2
s.t.
x1 + 3x2
7x1 + x2
x1 ; x2
6
35
0 and x1 integer
The …nal tableau for the LP relaxation was
x2
x1
s2
s1
1
22
3
22
30
22
7
22
1
22
56
22
7
2
9
2
63
The only integer variable is x1 , so becomes the source row for the MIP cut. Formula (5.9) with
gives
3
22 s2
s3
1
22 s1
=
1
2
which we insert into the tableau
x2
x1
s3
Here fi0 =
1
2
and
fi0
fi0
1
=
s2
s1
1
22
3
22
3
22
30
22
7
22
1
22
1
22
56
22
7
2
9
2
1
2
63
1. Carry out a dual simplex iteration
s3
s1
10
3
x2
x1
4
s2
11
3
58
6
The rhs.
0; therefore is feasible for the relaxation. Stop dual simplex iterations. Check whether
the solution is feasible for MILP. x1 is integer, therefore it is. We conclude that the solution is
optimal for MILP. Stop.#
We can retrace the steps of the above general proof for this example. The x1 -row represents
x1 +
x1
Case A: rhs
4=
1
22 s1
1
2
3
22 s2
9
2
=
+
1
22 s1
0
1
2
Case B: rhs
3
22 s2
3
22 s2
+
1
22 s1
3
22 s2
0
1
22 s1
1
22 s1
1
1
2
1
1
2
3
22 s2
+
1
2
Hence
3
22 s2
1
22 s1
con…rming the cut.
7
1
2
5.3 Branch and bound
Consider the following example of a mixed integer linear program (MILP):
P0 :
Maximize
x1 + 2x2
s.t.
x1 + x2
3
2x1 + 5x2
x1 ; x2
8
0; x2 integer
Solving the LP relaxation e.g. graphically, we obtain (Ex.)
x1 = 2 13 ; x2 = 23 ; z = 3 32
Since x2 is the only integer variable, we "branch on x2 ", i.e. we consider two subproblems:
P1 :
Problem P0 with x2
0 added
P2 :
Problem P0 with x2
1 added
P1 is known as the "down" branch, P2 is the "up" branch. The solution of P0 (if feasible) can in
principle be found by solving both subproblems and choosing the solution with the higher value.
The respective feasible regions satisfy F0 = F1 [ F2 and F1 \ F2 = ;: For minimization problems
"higher" is replaced by "lower".
Let zi denote the value of the optimal integer solution to Pi (with zi =
1 if Pi is infeasible).
We have z0 = max (z1 ; z2 ). Solving the LP relaxations of P1 ; P2 give respectively
P1R :
x1 = 3; x2 = 0; z1R = 3 = z1
P2R :
x1 = 1 21 ; x2 = 1; z2R = 3 12 = z2
Note that z1R = z1 and z2R = z2 because (3; 0) ; 1 12 ; 1 are both feasible for P1 P2 respectively.
Therefore the optimal solution to MIP is x1 = 1 12 ; x2 = 1; z0 = 3 12 :
Now suppose that both x1 ; x2 were integer variables. Then z1R = z1 = 3 as (3; 0) is still feasible.
z2R = 3 21
z2 since the optimal value of the IP relaxation is an upper bound (for maximization)
for the integer optimal value. Since z2 is integer-valued in a pure IP problem if c is integer-valued,
then z2
3: The P2 branch therefore cannot improve on z1 = 3 which the value of the "incumbent"
(best integer feasible solution (3; 0) so far found). We say that the P2 branch is "fathomed".
Branching
We de…ne subproblems that iteratively partition the feasible region F into nonoverlapping
subregions F1 ; F2 ; :::
Each integer subproblem Pi requires in principle to be solved for zi .
If zi is not strictly better than the "incumbent", we may then discard or "fathom" this
branch.
8
Bounds
Solve the LP relaxation of Pi to give an upper bound ziR
[for minimization problems ziR forms a lower bound ziR
zi (for max problems)
zi ].
If ziR corresponds to an integer (feasible) solution then ziR = zi : No further branching takes
place for this subproblem. The branch is fathomed.
Any integer feasible solution zi is a lower bound to the unknown integer optimum, zIP : We
only keep the best, i.e. the incumbent.
The whole process can be viewed as "divide and conquer" or "implicit enumeration" because
we use bounds to fathom subproblems, thus increasing the e¢ ciency of the search, i.e. reducing
the computational e¤ort.
Fathoming Criteria
Let z0 denote the value of the incumbent. (Initially z0 =
becomes a terminal node (no further branching takes place) if
1): A subproblem is fathomed and
1. The LP relaxation is either infeasible or provides an integer solution, or
2. If ziR
z0 : [Since zi
ziR and z0
zIP we therefore have zi
zIP ]
Example (previously solved by Gomory’s cuts)
Maximize
7x1 + 9x2
s.t.
x1 + 3x2
7x1 + x2
x1 ; x2
6
35
0 and integer
Solving the LP relaxation gives …nal tableau
x2
x1
s2
s1
1
22
3
22
30
22
7
22
1
22
56
22
Suppose we again branch on x2 , adding either x2
7
2
9
2
(5.10)
63
3 (P1 :down) or x2
constraint is
x2 + s3 = 3
x2 +
s3
1
22 s2
1
22 s2
+
9
7
22 s1
7
22 s1
=
=
7
2
1
2
4 (P2 :up). The down
which becoms a new row in the tableau
x2
x1
s3
s2
s1
1
22
3
22
1
22
30
22
7
22
1
22
7
22
56
22
7
2
9
2
1
2
63
After 1 pivot we obtain an optimal (noninteger ) solution to the LP relaxation of P1
s3
s1
x2
0
1
3
x1
1
7
1
7
1
7
22
7
32
7
11
7
1
8
59
s2
with z1R = 59
(5.11)
z1 : Notice that the change in the value of the objective function (the down penalty)
is
i
D
= ui fi
(5.12)
zj
= min
cj
1
2
yij
j2J +
=4
where i refers to the x2 row in tableau (5.10)
J + indexes the strictly positive elements yij
fi is the fractional part of x2 in (5.10)
NB. The down penalty (5.12) is the decrease in z due to one iteration of the dual simplex
algorithm for the down subproblem.
The up branch constraint is
x2
x2 +
s3 +
1
22 s2
1
22 s2
+
+
s3 = 4
7
22 s1
7
22 s1
=
7
2
1
2
=
which we insert into the tableau (5.10) to give
x2
x1
s3
s2
s1
1
22
3
22
1
22
30
22
7
22
1
22
7
22
56
22
10
7
2
9
2
1
2
63
Notice that a dual simplex pivot is not possible as yij > 0; 8j in the s3 row, showing that the upi
U
subproblem is infeasible. In general, the up penalty
or decrease in z through one dual simplex
iteration for the up subproblem is
i
U
= vi (1
fi )
zj
= min
cj
1
2
yij
j2J
In this case J
(5.13)
(=unde…ned)
= fj : yij < 0g = ;, so P2 :is infeasible. Therefore z2R =
1 and P2 is fathomed.
Returning to tableau (5.11) for P1 with z1R = 59; we split P1 into two further subproblems by
adding either x1
4 (P3 :down) or x1
5 (P4 :up). Rather than calculate the new row that should
be added in each case we construct the following penalty table for branching based on the variable
x1
u
x1
7
v
f
56
4
7
D
= uf
U
= v (1
4
f)
(5.14)
24
This table tells us that the down subproblem is more promising, because the down penalty is only
4 whereas the up penalty is 24: i.e. the up subproblem will decrease by 24 in a single iteration of
the dual simplex algorithm. As the OF for dual simplex iterations decreases monotonically, these
penalties provide upper bounds for the relaxed solutions. Hence z3
55 while z4
35:
The tableau for P3 is
s3
s1
x2
0
1
3
x1
1
7
1
7
1
7
1
7
22
7
1
7
32
7
11
7
4
7
1
8
59
s2
s4
After 1 iteration we obtain a feasible tableau giving an optimal integer solution (4; 3) to the relaxation of P3 showing z3 = 55; Thus (4; 3) becomes the incumbent.and z0 = 55
zIP : Since z4
35,
P4 is fathomed.
s4
s1
x2
3
x1
4
s2
1
s3
4
7
9
55
P2 has previously been discarded. Therefore all nodes are fathomed, (4; 3) is the required optimal
solution and zIP = 55:
11
Use of Penalty Tables
In general when there is more than one integer variable that is fractional-valued in an optimal LP
tableau, we need to choose
1. the branching variable de…ning the subproblems
- if a variable leads to an infeasible branch, this is good
otherwise a suggested criterion is based on penalties
max min
i
i
D;
i
U
(5.15)
2. which subproblem to set aside
- examine …rst the branch giving smaller penalty in (5.15) i.e. arg min
i
D;
i
U
: Set aside
the other problem.
5.3.1 Branch and bound algorithm (max form)
A description of the BnB algorithm for the maximization case follows:
Notation:
L
list of active subproblems
x0
the incumbent (current best integer solution)
z0
value of the incumbent x0 ; z0 = cT x0 = z IP
IP i
a member of L - initial problem is IP 0
1. (Initialization)
Set L = IP 0 ; z IP =
1
2. (Termination)
If L = ; then STOP; the incumbent x0 with value z0 is optimal
If no such x0 exists, then IP is infeasible.
3. (Problem selection & relaxation)
Select and delete a problem IP i from L (based on penalty information). Solve the LP
relaxation of IP i : Let ziR denote the optimal value of the relaxation and let xiR be an optimal
solution if one exists (ziR = cT xiR or ziR =
1).
4. (Fathoming & pruning)
(a) If ziR
z0 go to Step 2.
12
(b) If ziR > z0 and xiR is integer feasible, update x0 = xiR ; z0 = ziR : Delete from L all
subproblems IP i with ziR
z0 : Go to Step 2.
5. (Branching)
Take IP i and branch into two subproblems IPDi and IPUi . Add these subproblems to L: Go
to Step 2.
5.3.2 The branch and bound tree
The progress of the algorithm can be summarized as a tree search where the root node of the tree
represents the original problem. The tree is constructed iteratively by branching on an existing
node for which the optimal solution of the relaxation is fractional. A branch into two subproblems
creates two additional child nodes, each of which needs to be examined.
The tree search ends when all nodes are fathomed.
13