Problem (pure IP)
Branch and Bound
(by Example)
z* := max 8x1 + 5x2
x
• We shall follow Winston (Section .3)
• We shall use Example 9 to drive the discussion
• You’ll have to cover Section .5 (Knapsack
problem) yourself (Assignment and test
question)
s.t.
x1 + x2 £ 6
9x1 + 5x2 £ 45
x1 ,x 2 ≥ 0, x1, x2 integer
• It is a 2 dimensional problem, we shall solve
the LP relaxations graphically.
Solution of IP by Inspection
9
9
9x1 + 5x2 £ 45
8
7
Optimal LP relaxation:
X=(3.75,2.25)
6
5
Z=8x1 + 5x2
8
4
7
6
5
4
3
3
Z=20
2
x1 + x2 £ 6
1
0
2
1
0
0
1
2
3
4
5
6
7
8
0
1
2
3
4
5
6
7
8
1
Solution of IP by Inspection
BB Strategy
9
8
7
Z=8x1 + 5x2
6
Optimal IP solution
Z*=40
x=(5,0)
5
4
3
2
• Upper Bound for z* : by LP relaxation
• Branching: Fractional decision variable
• Sub-problems enumeration: LIFO (Last in first
out)
1
0
0
1
2
3
4
5
6
7
8
BB In Action
Simple Ideology
•
•
•
•
Suppose that U is an upper bound for v.
Suppose that L is a lower bound for v.
Suppose that L=U.
Then … L=U = v.
How do we generate the bounds?
• Assume a maximization problem.
– U is generated by relaxation.
– L is generated … by a feasible solution.
• Initialization:
Solve the LP relaxation of the IP problem
z* := max 8x1 + 5x2
x
s.t.
x1 + x2 £ 6
9x1 + 5x2 £ 45
x1 ,x 2 ≥ 0, x1, x2 integer
2
By inspection
Sub-problems
9
Optimal solution to LP relaxation:
X=(3.75,2.25) , Z=165/4
8
7
6
5
4
3
2
1
0
0
1
2
3
4
5
6
7
8
• The solution of the LP relaxation is
x=(3.75,2.25)
• We shall (arbitrarily) branch on x1 (there are
“rules” for this as well.)
• That is, we shall consider the two cases
(regions) associated with x1 = 3.75 :
x1 >= 4
x1 <= 3
• Set then z’=165/4 to be the upper bound for z*.
Search Tree
t=1
x1 >= 4
Sub-problem 2
What do we do next?
Sub-problem 1
Z=165/4
x=(3.75,2.25)
x1 <= 3
Sub-problem 3
• We have to solve one of the two sub-problems
generated by the original problem.
• Which one?
• There are several “methods” for this purpose
(eg flip a coin …)
• Let us (arbitrarily) choose Sub-problem 2.
• The label t indicates the order in which the subproblems are “solved”.
3
How do we solve the new subproblem?
Search Tree
Sub-problem 1
Z=165/4
x=(3.75,2.25)
t=1
x1 >= 4
Sub-problem 2
x1 <= 3
Sub-problem 3
t=2
• The new sub-problem is an IP problem with
two variables.
• We can solve its LP relaxation graphically to
obtain an upper bound (UB).
• This problem is the same problem as its Mother (subproblem 1) , except that here x1 >= 4.
Sub problem 2
LP Relaxation of Sub-problem
z* := max 8x1 + 5x2
9
s.t.
7
x
8
Feasible Region of the
New LP relaxation
6
x1 + x2 £ 6
5
9x1 + 5x2 £ 45
3
4
x1 ≥ 4
2
x1 ,x 2 ≥ 0, x1, x2 integer
0
1
0
1
2
3
4
5
6
7
8
4
LP Relaxation of Sub-problem 2
Observation
9
8
Optimal solution of LP relaxation
of sub-problem 2. x=(4,1.8),
Z=41
7
6
5
4
3
2
1
0
0
1
2
3
4
5
6
7
8
• The optimal solution to the LP relaxation of the
new sub-problem does not satisfy the
integrality constraint.
• Thus we have to expand (branch) the search
tree.
• Since we use the LIFO policy, we expand the
tree on sub-problem 2.
Search Tree
t=1
Sub-problem 1
Z=165/4
x=(3.75,2.25)
x1 >= 4
t=2
Search Tree
Sub-problem 2
Z=41
x=(4,1.8)
t=1
x1 <= 3
Sub-problem 3
• Since x1 is an integer, we branch on x2. That is, we
consider x2 <=1 and x2 >= 2.
Sub-problem 1
Z=165/4
x=(3.75,2.25)
x1 >= 4
t=2
x1 <= 3
Sub-problem 2
Z=41
x=(4,1.8)
x2 >= 2
Sub-problem 4
Sub-problem 3
x2 <=1
Sub-problem 5
5
Search Tree
What is next?
• We have two new sub-problems.
• According to the LIFO policy, we now have to
solve one of these new problems.
• Which one?
• We arbitrarily choose Sub-problem 4.
Sub-problem 1
Z=165/4
x=(3.75,2.25)
t=1
x1 >= 4
t=2
x1 <= 3
Sub-problem 2
Z=41
x=(4,1.8)
x2 >= 2
Sub-problem 4
Sub-problem 3
x2 <=1
Sub-problem 5
Sub problem 4
Observation
z* := max 8x1 + 5x2
x
s.t.
x1 + x2 £ 6
9x1 + 5x2 £ 45
x1 ≥ 4
x2 ≥ 2
• This sub-problem is not feasible (violation of
the 2nd constraint):
9x1+5x2 <= 45
,
(x >= (4,2))
x1 ,x 2 ≥ 0, x1, x2 integer
6
Search Tree
LP Relaxation of Sub-problem 4
Sub-problem 1
Z=165/4
x=(3.75,2.25)
t=1
9
8
7
Feasible region is empty.
Thus, the problem is not feasible
6
5
x1 >= 4
4
t=2
3
2
1
x1 <= 3
Sub-problem 2
Z=41
x=(4,1.8)
x2 >= 2
0
0
1
2
3
4
5
6
7
8
t=3
Sub-problem 3
x2 <=1
Sub-problem 4
(not feasible)
Sub-problem 5
x
Sub problem 5
What’s next?
z* := max 8x1 + 5x2
x
• According to the LIFO policy, we backtrack
from sub-problem 4 and consider sub-problem
5.
s.t.
x1 + x2 £ 6
9x1 + 5x2 £ 45
x1 ≥ 4
x2 £ 1
x1 ,x 2 ≥ 0, x1, x2 integer
7
LP Relaxation of Sub-problem 5
LP Relaxation of Sub-problem 5
9
9
8
8
7
7
6
6
Feasible region
5
5
4
4
3
3
2
2
1
1
0
0
0
1
2
3
4
5
6
7
Optimal solution
x=(40/9,1)
Z=365/9
8
0
1
2
3
4
5
6
7
8
Search Tree
Observation
• The optimal solution of the LP relaxation of
sub-problem 5 does not satisfy the integrality
constraint.
• Thus, we have to expand the search tree at this
sub-problem.
• Since x2 is integer, we branch on x1. That is,
we consider the two cases x1 >=5 and x1<=4.
t=2
Sub-problem 1
Z=165/4
x1 >= 4 x=(3.75,2.25)
Sub-problem 2
Z=41
x=(4,1.8)
Sub-problem 3
x1 <= 3
t=1
x2 <=1
x2 >= 2
Sub-problem 4
(not feasible)
x
t=3
Sub-problem 5
x=(40/9,1)
Z=(365/9)
x1 >= 5
Sub-problem 6
t=4
x1 <= 4
Sub-problem 7
8
Sub problem 7
What’s next?
z* := max 8x1 + 5x2
x
• According to the LIFO policy, we have to
examine the last “unfathomed” sub-problem.
• We have a choice between sub-problem 6 and
sub-problem 7.
• We arbitrarily select sub-problem 7.
s.t.
x1 + x2 £ 6
9x1 + 5x2 £ 45
x1 ≥ 4
x1 £ 4
x1 ,x 2 ≥ 0, x1, x2 integer
Sub-problem 7
Observation
9
8
By inspection,
Optimal solution is at
x=(4,1), z=37
7
6
5
4
3
2
1
0
0
1
2
3
4
5
6
7
8
• Since the optimal solution to sub-problem 7
satisfies the integrality constraint, it constitutes
a feasible solution to the original IP problem.
Thus, we can use it as a “candidate”, and apply
its z value as a lower bound for z*.
• Thus, set LB=37.
• Note that sub-problem 7 has been fathomed.
9
Search Tree
t=2
Sub-problem 1
Z=165/4
x1 >= 4 x=(3.75,2.25)
Sub-problem 2
Z=41
x=(4,1.8)
x1 <= 3
• The only remaining unfathomed sub-problems
are sub-problem 3 and sub-problem 6.
• According to the LIFO policy, we now
consider sub-problem 6.
• We thus backtrack to this sub-problem.
t=1
x2 <=1
x2 >= 2
Sub-problem 4
(not feasible)
Sub-problem 5
x=(40/9,1)
Z=(365/9)
x
t=3
What’s next?
Sub-problem 3
t=4
x1 <= 4
x1 >= 5
Sub-problem 6
x
Sub-problem 7
X=(4,1)
Z=37, LB=37
t=5
Sub problem 6
z* := max 8x1 + 5x2
x
Sub-problem 6
9
8
s.t.
Optimal solution
x=(5,0), z=40
7
x1 + x2 £ 6
9x1 + 5x2 £ 45
x1 ≥ 4
x1 ≥ 5
x1 ,x 2 ≥ 0, x1, x2 integer
6
5
4
3
2
1
0
0
1
2
3
4
5
6
7
8
10
Search Tree
t=2
Sub-problem 1
Z=165/4
x1 >= 4 x=(3.75,2.25)
Sub-problem 2
Z=41
x=(4,1.8)
• The only unfathomed sub-problem is subproblem 3, so we have to explore it.
x2 <=1
Sub-problem 4
(not feasible)
Sub-problem 5
x=(40/9,1)
Z=(365/9)
x
t=3
t=4
x1 <= 4
x1 >= 5
t=6
x1 <= 3
t=1
x2 >= 2
What is next ?
Sub-problem 3
Sub-problem 6
x=(5,0)
Z=40 , LB=40
x
x
Sub-problem 7
x=(4,1)
Z=37 , LB=37
Sub problem 3
t=5
LP Relaxation of Sub-problem 3
z* := max 8x1 + 5x2
9
s.t.
7
x
8
6
x1 + x2 £ 6
5
9x1 + 5x2 £ 45
3
4
x1 £ 3
2
x1 ,x 2 ≥ 0, x1, x2 integer
0
1
0
1
2
3
4
5
6
7
8
11
LP Relaxation of Sub-problem 3
Observation
9
8
• The optimal solution to the LP relaxation of
sub-problem 3 satisfies the integrality
constraint.
• Hence, this solution is also optimal for subproblem 3.
• Thus, this problem has been fathomed.
Optimal solution
x=(3,3), Z=39
7
6
5
4
3
2
1
0
0
1
2
3
4
5
6
7
8
Search Tree
t=2
Sub-problem 1
Z=165/4
x1 >= 4 x=(3.75,2.25)
Sub-problem 2
Z=41
x=(4,1.8)
t=1
t=7
x2 <=1
x2 >= 2
Sub-problem 4
(not feasible)
x
t=3
Sub-problem 5
x=(40/9,1)
Z=(365/9)
Sub-problem 6
x=(5,0)
Z=40 , LB=40
t=4
x
Observation
• There are no more sub-problems to explore.
• Thus, the current candidate is optimal.
• Hence we conclude that x=(5,0) is the optimal
solution to the original IP problem.
x1 <= 4
x1 >= 5
t=6
Sub-problem 3
x=(3,3)
x1 <= 3
Z=39, LB=40
x
x
Sub-problem 7
x=(4,1)
Z=37 , LB=37
t=5
12
Important Comments
Basic Idea
• If a sub-problem had been fathomed, it is not
necessary to branch on it.
• A sub-problem can be fathomed if
– it is infeasible
– It yields an optimal solution satisfying the
integrality constraint
– Its optimal z-value is not greater (if opt=max) than
the current LB
• Observe that LP is used for two purposes:
– Determine whether a sub-problem is feasible
– Obtain a solution to a “relaxed” problem. This,
provides an upper bound (if opt=max)
• There could be more efficient ways for doing
this
• Go through Section 9.5 (knapsack problem) on
your own
• Extend the method to the “unbounded” version
of the knapsack problem.
Example
UB for knapsack problems
• Put as many items of the good items (in
increasing order of density) as possible, and use
“fractions” from the next best item, if
necessary, to fill-in the knapsack completely.
n
z* := max z =
x
 cj x j
j =1
s.t.
n
 a j xj £ b
j=1
x j Œ{0,1,2,3,...}, j = 1,2,...,n
• This is the standard unbounded knapsack problem.
13
n
• Suppose
c=(40,80,10,10,4,20,60)
a=(40,50,30,10,10,40,30)
b=100
• Then
z* := max z = Â c j x j
x
j=1
n
Âa x
j
j
£b
j=1
x j Π{0,1,2,3,...}, j = 1,2,...,n
j
1
2
3
4
5
6
7
Density cj/aj
†1
8/5
1/3
1
4/10
1/2
2
Rank
3-4
2
7
3-4
6
5
1
0-1 Knapsack problem
• Here the situation is more interesting:
• Put as many items as possible from the best
pile: x7=1. We are left with s=b-a7=100-30=70
units of space.
• We select an item from the next best item,
x2=1, so we are left with s-a2=70-50=20 units
of volume.
• We go to the next best item (j=1).
• Computing the upper bound:
– Put as many items as possible from pile 7 (with a
fraction if necessary from the next best item):
x7 = Integer (100/30) =3.
We are left with 100-30 x 3=10 units of volume
We fill the knapsack to the brim with the second best
item:
x2=10/50=1/5.
UB=x7 x c7 + x2 x c2 = 3 x 60 + 1/5 x 80 = 196
0-1 Knapsack problem
• We do not have enough volume for a full item
so we take a fraction x1=20/40=1/2
• UB = c7 + c2 + 0.5c1 = 60 + 80 + 0.5x40
= 160
14
Ties …..
Observation
• Had we used x4 instead of x1, …., we would
have had x4=1 and enough left space (s=20) for
2 items of type 6.
• UB = 80 +60 +! 10 + 2x20 = 190 >160
BB for TSP
Key Issues
(Winston, Example 11)
Distance matrix
city
1
2
3
4
5
1
0
132
217
164
58
2
132
0
290
201
79
3
217
290
0
113
303
• Note that these UBs are better than those
obtained from the LP relaxation assuming an
unbounded problem.
• The computation of these Ubs generates also
LBs (feasible 0-1 solutions).
4
164
201
113
0
196
5
58
79
303
196
0
• How do we compute lower bounds (opt=min)?
• How do we branch?
15
Recall …...
LP formulation of
The TSP
n
Assignment Problem
n
n
min   x(i, j)d(i , j)
x
xij := do we go from city
i to city j? Yes,No
x i =1 j =1
i =1 j =1
s.t.
n
 x(i, j) = 1, i = 1,2,...,n
j =1
n
 x(i, j) = 1, j = 1,2,...,n
Subtour elimination
constraint
i =1
n
 x(i, j) £ S - 1, "S à {1,2,...,n}
i , jŒS
x(i, j) Œ{0,1}
n
max   x(i, j)d(i, j)
xij := do we assign person i
to job j? Yes, No
s.t.
n
 x(i, j) = 1, i = 1,2,...,n
j=1
n
 x(i, j) = 1, j = 1,2,..., n
i =1
n
 x(i, j) £ S - 1, "S à {1,2,..., n}
i, j ŒS
x(i, j) Œ{0,1}
Thus …..
n n
Is a relaxation for
The TSP!
min   x(i, j)d(i, j)
s.t.
n
 x(i, j) = 1, i = 1,2,...,n
There are specialised
algorithms for AP, but
in emergency, can use
conventional LP
algorithms, eg Simplex
Method.
In short
x i =1 j =1
j=1
n
 x(i, j) = 1, j = 1,2,..., n
i =1
n
 x(i, j) £ S - 1, "S à {1,2,..., n}
i, j ŒS
x(i, j) Œ{0,1}
• Use AP as a relaxation for the TSP
• If there are no subtours, AP yields an optimal
solution for TSP.
• If there are subtours, Branch on them!
Eg. Suppose AP yields two subtours, say (1-2-1) (34-5-3). We can then branch so as to prevent the
first tour by setting
d(1,2) = M or d(2,1)=M
16
t=1
Example (continued)
x34=0
city
1
2
3
4
5
1
0
132
217
164
58
2
132
0
290
201
79
3
217
290
0
113
303
4
164
201
113
0
196
5
58
79
303
196
0
t=2
Sub-problem 1
Z=495
x15=x21=x34=x43=x52=1
Sub-problem 2
Z=652
x14=x25=x31=x43=x52=1
x43=0
Sub-problem 3
Z=652, UB=668
x13=x25=x34=x41=x52=1
t=5
x25=0
x52=0
x25=0
Sub-problem 4
Z=668,UB=668
x15=x24=x31
=x43=x52=1
Candidate
Sub-problem 5
Z=704
x14=x43=x32
=x25=x51=1
UB=668
Sub-problem 6
Z=704
x15=x34=x23
=x41=x52=1
UB=668
Sub-problem 7
Z=910
x13=x25=x31
=x42=x54=1
UB=668
t=6
t=7
t=3
t=4
x52=0
Optimal solution :x15=x24=x31=x43=x52=1
(from sub problem 4), z=668
17