CS 758/858: Algorithms
Backtracking
http://www.cs.unh.edu/~ruml/cs758
Local Search
Wheeler Ruml (UNH)
Class 25, CS 758 – 1 / 20
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
Local Search
Wheeler Ruml (UNH)
Backtracking
Class 25, CS 758 – 2 / 20
Hardness
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
NPC: SAT, vertex cover, clique, subset sum, . . .
greedy: local choice is optimal
DP: poly number of options to track
search: exponential number of options, often combinations
Local Search
Wheeler Ruml (UNH)
Class 25, CS 758 – 3 / 20
Combinatorial Optimization
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
decision 1
option 1
option 2
decision 2
decision 2
option 1
option 2
option 1
option 2
Local Search
(1,1)
(1,2)
(2,1)
(2,2)
A tree representation of alternatives in a small combinatorial
problem.
Wheeler Ruml (UNH)
Class 25, CS 758 – 4 / 20
Backtracking
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
depth-first search
child ordering
lower bounds
branch-and-bound
Local Search
Wheeler Ruml (UNH)
Class 25, CS 758 – 5 / 20
Depth-first Search
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
DFS (node)
1
If is-leaf(node)
2
Visit(node)
3
else
4
For i from 0 to num-children
5
DFS(child(node, i))
Local Search
Wheeler Ruml (UNH)
Class 25, CS 758 – 6 / 20
Depth-first Search Order
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
Local Search
Wheeler Ruml (UNH)
Class 25, CS 758 – 7 / 20
Problems Are Hard
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
Local Search
13,509 US cities (W. Cook)
Wheeler Ruml (UNH)
Class 25, CS 758 – 8 / 20
Problems Are Hard
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
. . . 13,508
. . . 13,507
Local Search
..
.
13,505
Wheeler Ruml (UNH)
. . . 13,506
Class 25, CS 758 – 9 / 20
Problems Are Hard
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
Local Search
(S. LaValle)
Wheeler Ruml (UNH)
Class 25, CS 758 – 10 / 20
Improved Discrepancy Search
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
Local Search
ILDS (node, allowance, remaining)
1
If is-leaf(node)
2
Visit(node)
3
else
4
If allowance > 0
5
ILDS(child(node, 1), allowance − 1, remaining − 1)
6
If remaining > allowance
7
ILDS(child(node, 0), allowance, remaining − 1)
start with ILDS(root, iteration, max-depth)
Wheeler Ruml (UNH)
Class 25, CS 758 – 11 / 20
Discrepancy Search Order
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
Local Search
The second pass of ILDS visits all leaves with one discrepancy in
their path from the root.
Wheeler Ruml (UNH)
Class 25, CS 758 – 12 / 20
Break
Backtracking
■ Hardness
■ Optimization
■ Backtracking
■ Depth-first Search
■ DFS Order
■ Problems
■ ILDS
■ ILDS Order
■ Break
■
■
■
■
asst 10 recall
asst 13
last year’s final
time for surveys on Thursday: bring device!
Local Search
Wheeler Ruml (UNH)
Class 25, CS 758 – 13 / 20
Backtracking
Local Search
■ Local Search
■ Local Search
■ Max Cut
■ Suboptimality
■ EOLQs
Local Search
Wheeler Ruml (UNH)
Class 25, CS 758 – 14 / 20
Local Search
Backtracking
Local Search
■ Local Search
■ Local Search
■ Max Cut
■ Suboptimality
■ EOLQs
2.6
4.4
2.3
3.9
1.5
6.2
1.6
2.1
A graph representing an improvement-based search.
Wheeler Ruml (UNH)
Class 25, CS 758 – 15 / 20
Local Search
Backtracking
Local Search
■ Local Search
■ Local Search
■ Max Cut
■ Suboptimality
■ EOLQs
hill climbing
simulated annealing
large neighborhood search
genetic algorithms
particle swarm optimization
Wheeler Ruml (UNH)
Class 25, CS 758 – 16 / 20
Max Cut
Backtracking
Local Search
■ Local Search
■ Local Search
■ Max Cut
■ Suboptimality
■ EOLQs
maximize weight of edges crossing the cut w(A, B)
decision version is NP-complete
simple local search:
move vertex u from A to B iff
X
X
wuv >
wuv
v6=u∈A
v∈B
it’s possible to bound suboptimality of local minima under this
neighborhood!
Wheeler Ruml (UNH)
Class 25, CS 758 – 17 / 20
Suboptimality of Local Search
Backtracking
Local Search
■ Local Search
■ Local Search
■ Max Cut
■ Suboptimality
■ EOLQs
for any u in A,
X
wuv ≤
v6=u∈A
X
add:
2
X
(u,v)∈A
Wheeler Ruml (UNH)
wuv = w(A, B)
u∈A,v∈B
summing over all u in B,
X
2
wuv ≤
(u,v)∈B
wuv
v∈B
summing over all u in A,
X
2
wuv ≤
(u,v)∈A
X
X
wuv = w(A, B)
u∈A,v∈B
wuv + 2
X
wuv ≤ 2w(A, B)
(u,v)∈B
Class 25, CS 758 – 18 / 20
Suboptimality of Local Search
Backtracking
Local Search
■ Local Search
■ Local Search
■ Max Cut
■ Suboptimality
■ EOLQs
divide by 2:
X
wuv +
(u,v)∈A
X
wuv ≤ w(A, B)
(u,v)∈B
eg, more weight crossing than within partitions
let W be sum of all weight in graph.
add crossing weight to both sides:
W ≤ 2w(A, B)
W/2 ≤ w(A, B)
note optimal is at most W
Wheeler Ruml (UNH)
Class 25, CS 758 – 19 / 20
EOLQs
Backtracking
Local Search
■ Local Search
■ Local Search
■ Max Cut
■ Suboptimality
■ EOLQs
For example:
■
■
■
What’s still confusing?
What question didn’t you get to ask today?
What would you like to hear more about?
Please write down your most pressing question about algorithms
and put it in the box on your way out.
Thanks!
Wheeler Ruml (UNH)
Class 25, CS 758 – 20 / 20