Chapter 13 Backtracking
•
•
•
•
•
Introduction
The 3-coloring problem
The 8-queens problem
The general backtracking method
Branch and bound
13.1 Introduction
• In many real world problems, as in most of the
NP-hard problems, a solution can be obtained
by exhaustively searching through a large but
finite number of possibilities.
• The need arose for developing systematic
techniques of searching, with the hope of
cutting down the search space to possibly a
much smaller space.
• Backtracking is an organized exhaustive search
which often avoids searching all possibilities.
13.2 The 3-coloring problem
• Problem: Given an undirected graph G=(V,E), it is
required to color each vertex in V with one of
three colors, say 1,2 and 3, such that no two
adjacent vertices have the same color. We call
such a coloring legal; otherwise, if two adjacent
vertices have the same color, it is illegal.
• There are 3n possible colorings to color a graph
with n vertices. The set of all possible colorings
can be represented by a complete ternary tree
called the search tree.
• An incomplete coloring of a graph is called partial
if no two adjacent colored vertices have the same
color.
13.2 The 3-Coloring Problem
•
A coloring can be represented by an ntuple ( c1,c2,…,cn) such that
ci∈{1,2,3},1≤i≤n.
13.2 The 3-Coloring Problem
Fig. 13.1 The search three for all possible 3-colorings
for a graph with 3 vertices.
13.2 The 3-Coloring Problem
• Example 13.1
Consider the graph shown in Fig. 13.2(a), where
we are interested in coloring its vertices using the
colors {1,2,3}. Figure 13.2(b) shows part of the
search tree generated during the process of
searching for a legal coloring.
13.2 The 3-Coloring Problem
(a)
(b)
a
a=1
b=1
b=2
b
c
c=1
d
c=2
e
d=1
e=1
e=2
e=3
Fig. 13.2 An Example of using backtracking to
solve the problem 3-COLORING.
13.2
Algorithm 13.1 3-COLORREC
Input: An undirected graph G=(V,E)
Output: A 3-coloring c[1..n] of the vertices of G,
where each c[j] is 1,2 or 3.
1. for k←1 to n
2. c[k] ←0
3. end for
4. flag←false
5. graphcolor(1)
6. If flag then output c
7. else output “no solution”
Procedure graphcolor(k)
1. for color=1 to 3
2. c[k] ←color
3.
If c is a legal coloring then set flag←true
and exit
4.
else if c is partial then graphcolor(k+1)
5. end for
13.2
Algorithm 13.2 3-COLORITER
Input: An undirected graph G=(V,E).
Output: A 3-coloring c[1..n] of the vertices of G,
where each c[j] is 1,2 or 3.
1. for k←1 to n
2.
c[k] ←0
3. end for
4. flag←false
5. k←1
6. while k≥1
7.
while c[k]≤2
8.
c[k]←c[k]+1
9.
if c is a legal coloring then set flag←true and
exit from the two while loops.
10.
else if c is partial then k←k+1
{advance}
11. end while
12. c[k] ←0
13. k←k-1
{backtrack}
14. end while
15. If flag then output c
16. else output “no solution”
13.2 The 3-coloring problem
• Time complexity:
• We note that O(3n) nodes are generated
in the worst case. For each generated
node, O(n) work is required to check if
the current coloring is legal, partial, or
neither. Hence, the overall running time
is O(n3n) in the worst case.
13.3 The 8-Queens Problem
• Problem: How can we arrange 8 queens
on an 88 chessboard so that no two
queens can attack each other? Two
queens can attack each other if they are
in the same row, column or diagonal.
• The n-queens problem is defined
similarly, where in this case we have n
queens and an n n chessboard for an
arbitrary value of n>=1.
• We will study the 4-queens problem.
13.3 The 8-Queens Problem
• The algorithm:
• We used the term legal to mean a placement of n
queens that do not attack each other, and the term
partial to mean a placement of less than n queens
that do not attack each other.
• Initially, put n queens in n different rows. Two
queens placed at positions xi and xj are in the same
column if and only if xi=xj. Two queens are in the
same diagonal if and only if
xi-xj=i-j or xi-xj=j-i
13.3 The 8-Queens Problem
(a)
(b)
Fig. 13.3 Two configurations of the 4-queens problem
13.3
Algorithm 13.3 4-QUEENS
Input: none
Output: A vector x[1..4] corresponding to the
solution of the 4-queens problem.
1. for k←1 to 4
2. x[k] ←0
3. end for
4. flag←false
5. k←1
6. while k≥1
7. while x[k]≤3
8.
x[k] ←x[k]+1
9.
If x is a legal placement then set flag←true and
exit from the two while loops.
10.
else if x is partial then k←k+1
{advance}
11. end while
12. x[k] ←0
13. k←k-1
{backtrack}
14. end while
15. If flag then output x
16. else output “no solution”
13.3 The 8-Queens Problem
• Example 13.2 Applying the algorithm
produces the solution shown in Fig. 13.4.
13.3 The 8-Queens Problem
X1=2
X2=4
X3=1
X4=3
Fig. 13.4 An example of using backtracking to
solve the 4-queens problem
13.3 The 8-Queens Problem
• Time complexity
• The brute-force method can be improved
to test n! configurations instead of nn.
13.4 The General Backtracking Method
• a class of search problems whose solution consists of
a vector (x1,x2, ..,xi) satisfying some predefined
constraints.
• Here:
– i is some integer between 0 and n,
– n is a constant that dependent on the problem
formulation.
– each xi in the solution vector belongs to a finite
linearly ordered set Xi. Thus, the backtracking
algorithm considers the elements of the cartesian
product X1X2…Xn in lexicographic order.
• Initially, the algorithm starts with the empty vector.
It then chooses the least element of X1 as x1.
• If (x1) is a partial solution, the algorithm proceeds
by choosing the least element of X2 as x2.
• If(x1,x2) is a partial solution, then he least element
of X3 is included;
• otherwise x2 is set to the next element in X2.
• In general, suppose that the algorithm has detected
the partial solution (x1,x2, …xj). It then considers
the vector v=(x1,x2,…,xj,xj+1).
• We have the following cases:
(1) If v represents a final solution to the
problem, the algorithm records it as a
solution and either terminates in case
only one solution is desired or continues
to find other solutions.
(2) (The advance step). If v represents a
partial solution, the algorithm advances
by choosing the least element in the set
Xj+2.
(3) if V is neither a final nor a partial solution,
we have two subcases:
(a) if there are still more elements to choose
from in the set Xj+1, the algorithm sets xj+1
to the next member of Xj+1.
(b) (The backtrack step). If there are no more
elements to choose from in the set Xj+1,
the algorithm backtracks by setting xj to
the next member of Xj. If again there are
no more elements to choose from in the
set Xj, the algorithm backtracks by setting
xj-1 to the next member of Xj-1, and so on.
13.4 The general Backtracking Method
• Example 13.3
Consider a variant of the PARTITION
problem defined as follows. Given a set of n
integers X={x1,x2,…,xn} and an integer y, find
a subset Y of X whose sum is equal to y.
It is not hard to devise a backtracking
algorithm to solve this problem. Note that this
problem can be formulated in another way so
that the solution is a boolean vector of length
n in the obvious way.
13.4
Algorithm 13.4 BACKTRACKREC
Input: Explicit or implicit description of the
sets X1,X2,…,Xn.
Output: A solution vector v=(x1,x2,…,xi),0≤i≤n.
1. v← ()
2. flag←false
3. advance(1)
4. If flag then output v
5. else output “no solution”
Procedure advance(k)
1. for each x∈Xk
2. xk←x; append xk to v
3. If v is a final solution then set flag←true
and exit
4. else if v is partial then advance(k+1)
5. end for
13.4
Algorithm 13.5 BACKTRACKITER
Input: Explicit or implicit description of the
sets X1,X2,…,Xn.
Output: A solution vector v=(x1,x2,…,xi),
0<=i<=n
1. v←()
2. flag ←false
3. k ←1
4. while k>=1
5.
while xk is not exhausted
6.
xk ←next element in xk; append xk to v
7.
if v is a final solution then set flag ←true
and exit from the two while loops.
8.
else if v is partial then k ←k+1 {advance}
9.
end while
10.
reset xk so that the next element is the first.
11.
k ←k-1 {backtrack}
12. end while
13. if flag then output v
14. else output “no solution”
13.5 Branch and Bound
• Similar to backtracking
• Concerned with only maximization or
minimization of a given function.
• A bound is calculated at each node x on the
possible value of any solution given by
nodes that may later be generated in the
subtree rooted at x.
• If the bound calculated is worse than a
previous bound, the subtree rooted at x is
blocked, i.e., none of its children are
generated.
13.5 Branch and Bound
• E.g. The traveling salesman problem: Given
a set of cities and a cost function that is
defined on each pair of cities, find a tour of
minimum cost. Here a tour is a closed path
that visits each city exactly once.
• With each partial solution (x1,x2,…,xk), we
associate a lower bound y which is
interpreted as follows. The cost of any
complete tour that visits the cities
x1,x2,…,xk in this order must be at least y.
13.5 Branch and Bound
• A matrix is the reduction of the original
matrix if it reduces the cost matrix so that
each row or column contains at least one
entry that is equal to 0.
• Example see Fig 13.5 and Fig 13.6
• In general, if the edge included is
(ui,v1) and the path from the root
contains the two paths u1,u2, … ,ui
and v1,v2, … ,vj, then Mvj u1 is set
to ∞ , where M is the matrix at the
current node.
13.5 Branch and Bound
1 2 3 4 5
1 2 3 4 5
1
∞
17
7
35 18
1
∞
10
0
25 11
-7
2
9
∞
5
14 19
2
4
∞
0
6
14
-5
3
29 24
∞
30 12
3
17 12
∞
15
0
-12
4
27 21 25
∞
48
4
6
0
4
∞
27 -21
5
15 16 28 18
∞
5
0
1
13
0
A
B
Fig. 13.5 An instance matrix of the TRAVELING
Salesman and its reduction.
-3
∞
-15
1
2
3
4
5
1
∞
4
17
6
0
（3,5）
Bound=63
2
3 4
10 0 25
∞ 0 6
12 ∞ 15
0 4 ∞
1 13 0
B
5
11
14
0
27
∞
/（3,5）
Bound=63
1
2
3
1 ∞ 10 0
2 4 ∞ 0
4 6 0 4
5 0 1 ∞
C
4
25
6
∞
0
1
2
3
4
5
Bound=75
1
2
3 4 5
∞ 10 0 25 11
4 ∞ 0 6 14
17 12 ∞ 15 ∞
6 0 4 ∞ 27
0 1 13 0 ∞
D
-12
Bound=63
1
2
3
1 ∞ 10 0
2 4 ∞ 0
4 6 0 4
5 0 1 ∞
Bound=63
1
2
4
1 ∞ 0 15
4 6 0 ∞
5 0 ∞ 0
E
(2,3)
C
4
25
6
∞
0
/(2,3)
Bound=67
-10
1
2
3 4
1 ∞ 10 0 25
2 0 ∞ ∞ 2
4 6 0 4 ∞
5 0 1 ∞ 0
F
-4
Bound=67
1
2
3 4
1 ∞ 10 0 25
2 0 ∞ ∞ 2
4 6 0 4 ∞
5 0 1 ∞ 0
-4
F
Bound=67 (1,3)
1
2
4
2 0 ∞ 2
4 6 0 ∞
5 ∞ 1 0
G
/(1,3)
Bound=77
1
2
3 4
1 ∞ 0 ∞ 15
2 0 ∞ ∞ 2
4 6 0 4 ∞
5 0 1 ∞ 0
H
-10
Bound=67
1
2
4
2 0 ∞ 2
4 6 0 ∞
5 ∞ 1 0
G
(4,2)
Bound=67
1
4
2 0 ∞
5 ∞ 0
I
/(4,2)
Bound=74
1
2
4
2 0 ∞ 2
4 6 ∞ ∞
5 ∞ 0 0
J
-1
-6
Bound=67
2
5
1
4
0 ∞
∞ 0
I
(2,1)
Bound=67
4
0
5
K
(5,4)
solution
/(5,4) Bound=67
4
∞
M
5
/(2,1)
2
5
Bound=67
1
4
∞ ∞
∞ 0
L
• The solution is:
(1,3) (3,5) (5,4) (4,2) (2,1)