TOPICS IN ALGORITHMS
http://eclass.aueb.gr/courses/INF171/
Spring 2017
I. ΜILIS
Traveling Salesman Problem
(TSP)
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
1
Traveling Salesman Problem (TSP)
TSP
Instance: A complete directed weighted graph G=(V,E), integer B
Question (Decision): Is there a permutation of V, <v1,v2,…,vn>
such that Σi=1 to n w(vi, v(i mod n) + 1) £ B ?
i.e., is there a Hamiltonian Cycle in G (a tour)
of cost £ B ?
Question (Optimization): Find a tour of minimum cost
Brute force approach: O(n!)
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
2
HC ≤p TSP
G=(V,E)
G' = (V, E')
E' = V ´ V
1, if (u,v) Î E
w(u,v) = w(v,u) =
{
2, otherwise
B= |V|
G has a HC
All its edges have cost 1 in G'
G' has a tour of cost B
G' has a tour of cost £ B
It uses only edges of cost 1 (cost = B)
G has a HC
∆-TSP: A special case of TSP where the triangle inequality holds,
i.e., w(i,j) + w(j,k) ≥ w(i,k), 1 £ i,j,k £ n
QUESTIONS: Is ∆-TSP NP-complete ? Is TSP(1,2) NP-complete ?
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
3
Coping with NP-complete problems
1. Small instances: an O(exp) algorithm may be satisfactory
2. Special cases: may be O(poly), e.g. 2-SAT
3. Exponential algorithms
Pseudo-polynomial algorithms, Dynamic Programming,
Backtracking, Branch-and-Bound, Local search
4. Approximation algorithms : O(poly) algorithms that produce
solutions within a guaranteed factor away from the optimum solution
5. Randomized algorithms: O(poly) algorithms
Monte Carlo: Deterministic complexity; whp a correct solution
Las Vegas: Expected complexity; always a correct solution
5. Heuristic algorithms: any O(poly) approach without a proven
guarantee of performance but valid in practical situations
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
4
DP for TSP
Subproblem/Optimal Substructure
Assume w.l.o.g. that we start at node 1
Assume that 1 ->…S1 …->i->…S2 …-> 1 is an optimal tour
Then the i->…S2 …-> 1 Hamiltonian Path in V-S1
must be also optimal
1
S1
i
S2
1
Subproblem:
g(i,S) = the cost of the shortest i->……..-> 1 HP in S È {i,1}, S Ì V
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
5
DP for TSP
Let g(i,S) = the cost of the shortest i->……..-> 1 HP in S È {i,1}, S Ì V
S – {j}
i
j
1
S
g (i, S ) = min{ w(i, j ) + g ( j, S - { j}) }
jÎS
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
6
DP for TSP
Our aim is to find
g (1, V - {1}) = min{w(1, i ) + g (i,V - {1, i})}
2£i £ n
1
i
S = V – {1,i}
1
How ?
By finding g(i, V-{1,i} ) for all choices of i
This can be done recursively by using
g (i, S ) = min{w(i, j ) + g ( j, S - { j})}
jÎS
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
7
DP for TSP
Obviously, g(i,Æ) = w(i,1)
Find g(i, S) for all |S| = 1
Find g(i, S) for all |S| = 2
...
Find g(i, S) for all |S| =n-2
Find g(1, V-{1}) --- |S| =n-1
We need to compute g(i,S)
for ALL S’s of EACH size |S|= 1,2,…,n-2
for ALL i Î V - {1} – S
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
8
DP for TSP
Example
1
2
4
3
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
9
DP for TSP
|S|=0:
S=Æ
g(2,Æ)=5,
g(3,Æ)=6,
g(4,Æ)=8
|S|=1:
S = {3}
g(2,{3}) = w23 + g(3,Æ) = 9 + 6 = 15
g(4,{3}) = 15
g(2,{4}) = 18
g(3,{4}) = 20
g(3,{2}) = 18
g(4,{2}) = 13
S = {4}
S = {2}
|S|=2:
S={3,4}
S={2,4}
S={2,3}
|S|=3 :
S={2,3,4}
g(2,{3,4}) = min{ w23 + g(3,{4}), w24 + g(4,{3}) } = 25
g(3,{2,4}) = min{ w32 + g(2,{4}), w34 + g(4,{2}) } = 25
g(4,{2,3}) = min{ w42 + g(2,{3}), w43 + g(3,{2}) } =23
g(1,{2,3,4}) = min{ w12 + g(2,{3,4}),
w13 + g(3,{2,4}),
w14 + g(4,{2,3}) } = min{ 35, 40, 43 } = 35
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
10
DP for TSP
for i = 2 to n do g(i,Æ) = w(i,1) ;
for k = 1 to n–2 do
// for all sizes of S
for each S Í V–{1} s.t. |S|=k do
for each i Î V–{1}-S
// for all possible
S’s of size k
// for all i’s…
g(i,S):= min{w(i,j) + g(j,S–{j})} ;
jÎS
find g(1, V–{1}) ;
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
11
DP for TSP
Complexity
N = # of g(i,S)’s to be computed:
For each value of |S| there are n – 1 choices for i
æ n - 2ö
ç
÷
The number of S’s with |S| = k not including 1 and i is ç k ÷
è
ø
n-2
æ n - 2ö
÷÷ = (n - 1 )2n -2
N = å (n - 1 )çç
k =0
è k ø
T(n) = N ∙ time to compute g(I,S) = N ∙ n
T(n)=O(n22n )
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
12
Coping with NP-complete problems
1. Small instances: an O(exp) algorithm may be satisfactory
2. Special cases: may be O(poly), e.g. 2-SAT
3. Exponential algorithms
Pseudo-polynomial algorithms, Dynamic Programming,
Backtracking, Branch-and-Bound, Local search
4. Approximation algorithms : O(poly) algorithms that produce
solutions within a guaranteed factor away from the optimum solution
5. Randomized algorithms: O(poly) algorithms
Monte Carlo: Deterministic complexity; whp a correct solution
Las Vegas: Expected complexity; always a correct solution
5. Heuristic algorithms: any O(poly) approach without a proven
guarantee of performance but valid in practical situations
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
13
Branch-and-Bound
Lower bound:
1 n
å (min {wi , j } + min {w j ,i })
j ¹i
2 i =1 j ¹i
(the half of the sum of minimum elements of each row and each column)
Σ0
A
B
C
D
A
x
3
2
7
2
B
4
x
3
6
3
C
1
1
x
3
1
D
1
6
6
x
1
1
1
2
3
LB = 14/2 = 7
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
14
Branch-and-Bound
Σ1 AC in tour è CA, AB, AD, BC, DC not in tour (why ?)
A
B
C
D
A
x
x
2
x
2
B
4
x
x
6
4
C
x
1
x
3
1
D
1
6
x
x
1
1
1
2
3
LB = 15/2 = 7.5
7 Σ0
Σ2 AC not in tour
A
B
C
D
A
x
3
x
7
3
B
4
x
3
6
3
C
1
1
x
3
1
D
1
6
6
x
1
1
1
3
3
LB = 16/2 = 8
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
AC
__
AC
7.5 Σ1
8 Σ2
15
Branch-and-Bound
Σ3
Σ4
AC in tour è CA, AB, AD, BC, DC not in tour
CB in tour è CD, DB, BA not in tour (why?)
A
B
C
D
A
x
x
2
x
2
B
x
x
x
6
6
C
x
1
x
x
1
D
1
x
x
x
1
1
1
2
6
LB = 20/2 = 10
AC in tour è CA, AB, AD, BC, DC not in tour
CB not in tour
A
B
C
D
A
x
x
2
x
2
B
4
x
x
6
4
C
x
x
x
3
3
D
1
6
x
x
1
1
6
2
3
LB = 22/2 = 11
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
and so on …
16
Branch-and-Bound
7 Σ0
AC
__
AC
7.5 Σ1
Σ2 8
CB
__
CB
__
AB
10 Σ3
11 Σ4
11.5 Σ5
Σ6 9
BC
solution ACBDA
cost = 10
AB
solution ABCDA
cost = 10
10 Σ7
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
__
BC
Σ8 13.5
17
Branch-and-Bound
Parameters
§ Maintain a set S of active states, Initially S={Σ0}
§ In each step extract state Σ from S (Σ is the state to be expanded)
§ UB is an global upper bound of the optimum solution
§ LB(Σ) is a lower bound on all solutions represent by state Σ
General method (global UB, LB(Σ))
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
18
Branch-and-Bound
AlgoBB1
{ S={Σ0} ;
UB= +¥
while S ¹Æ do
{
get a node Σ from S;
// which node ? FIFO/LIFO/Best LB
S:= S - {Σ} ;
for all possible extensions Σj of Σ do
{
create Σj and find LB(Σj);
if LB(Σj) £ UB then
if Σj is terminal then
{ UB:= LB(Σj);
optimum:= Σj }
else add Σj to S
}
}
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
}
19
Coping with NP-complete problems
1. Small instances: an O(exp) algorithm may be satisfactory
2. Special cases: may be O(poly), e.g. 2-SAT
3. Exponential algorithms
Pseudo-polynomial algorithms, Dynamic Programming,
Backtracking, Branch-and-Bound, Local search
4. Approximation algorithms : O(poly) algorithms that produce
solutions within a guaranteed factor away from the optimum solution
5. Randomized algorithms: O(poly) algorithms
Monte Carlo: Deterministic complexity; whp a correct solution
Las Vegas: Expected complexity; always a correct solution
5. Heuristic algorithms: any O(poly) approach without a proven
guarantee of performance but valid in practical situations
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
20
Approximation algorithms
O(poly) algorithms obtaining a solution of cost C
within a factor ρ of the optimum cost OPT
Max
Minimization problems: C/OPT £ ρ = 1 + ε, ε > 0
Min
ΟPT
Maximization problems: C/OPT £ ρ = 1 – ε , ε > 0
|OPT –C| / OPT £ ε , ε > 0
ε: relative error (ε ´ 100) %
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
21
Approximations:
Good, better, best and more …
Non - constant approximation : C/OPT £ f(n)
Constant (ρ-)approximation : C/OPT £ ρ (a constant, e.g. 3/2)
Polynomial Time Approximation Schemes (PTAS)
§ C/OPT £ 1 + ε, for any ε >0
§ O(poly (|I|) ), O(exp (1/ε) ), e.g. O( n3/ε )
Fully Polynomial Time Approximation Schemes (FPTAS)
§ C/OPT £ 1 + ε, for any ε >0
§ O(poly (|I|) ), O(poly (1/ ε) ) !!! e.g. O( (1/ε)2n3 )
Additive approximation
§ C £ OPT+ f(n) or C £ OPT+ k (a constant), e.g. C £ OPT+1 !
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
22
TSP is not approximable
Is there any f(n)-approximation algorithm for TSP ?
NO !
Th. For any (polynomial time computable) function f(n), TSP can not be
approximated within a factor of f(n), unless P=NP.
Proof:
If there is an f(n)-approximation algorithm, A , for TSP,
then, there is an O(poly) algorithm for HC, unless P=NP !
Let a graph G=(V,E), |V|=n
Consider the graph G' = (V, E'), E' = V ´ V
with weights
1,
if (u,v) Î E
w(u,v) =
n f(n), otherwise
{
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
23
TSP is not approximable
Proof (cont.):
Running A on G' returns a tour of cost C
a) if C= n, then G is Hamiltonian
b) if C > n, then the tour contains at least one edge of cost n f(n)
ð C ³ nf(n) + (n-1) > n f(n)
but,
C /C* £ f(n)
ð C* ³ C / f(n) > n f(n) / f(n)
ð C* > n
ð G is not Hamiltonian !
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
24
Δ-TSP: ρ=2
There is a 2-approximation algorithm for Δ-TSP
Find a minimum spanning tree, T, of G, of cost C(T)
1
7
5
9
4
3
6
8
10
Let H* be an optimal tour of cost C(H*)
Let e be an edge of H* and T' be the rest of H* (this is a chain/tree)
C(H*) = w(e) + C(T') ³ w(e) + C(T) ³ C(T) ð C(T) £ C(H*)
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
25
Δ-TSP: ρ=2
1
1
7
5
9
4
3
9
2
4
6
8
10
7
5
6
3
8
10
Double the edges of T and let T’’ be the obtained (multi)graph
All vertices of T’’ are of even degree
Find an Euler cycle, W, in T’’
Euler cycle W: 1, 2, 3, 2, 4, 6, 5, 7, 5, 6, 8, 10, 9, 10, 8, 6, 4, 2, 1
W traverses each edge of T twice: C(W) = 2 C(T) £ 2 C(H*)
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
26
Δ-TSP: ρ=2
1
1
7
5
9
2
4
3
9
2
4
6
8
10
7
5
6
3
8
10
Find a tour H by shortcutting W:
H: 1, 2, 3, 2, 4, 6, 5, 7, 5, 6, 8, 10, 9, 10, 8, 6, 4, 2, 1
C(H) £ C(W) , because of the triangle inequality
C(H) £ C(W) £ 2 C(H*) ð
C(H) / C(H*) £ 2
QUESTION: What is the complexity of this algorithm ?
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
27
Δ-TSP: Tightness of ρ=2
Example
Complete graph Kn
Red edges: w =2
Other edges: w=1
Optimal tour
C(H*) = n
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
28
Δ-TSP: Tightness of ρ=2
Minimum MST
Solution
C(H) = (n-2)*2 + 2*1 = 2n-2
Hence, C(H) / C(H*) = (2n-2) / nà 2
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
29
Matching problems
Matching in a graph G=(V,E):
A subset M ÍE of edges s.t. no two edges in M have a vertex in common.
§ Maximal: it is not a subset of another matching,
i.e. it can not be extended
§ Maximum: a matching of maximum cardinality, |M|,
i.e. a maximum maximal matching
§ Perfect: a matching of cardinality |M|=n/2
(it is defined only for graphs with even # of vertices)
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
30
Matching problems
Examples
a matching in a
bipartite graph
v8
v7
a matching in a
general graph
v6
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
31
Matching problems
Matching problems
• Maximal matching: find a matching where no more edges can be added
• Maximum matching: find a matching with a maximum number of edges
• Perfect matching: find a matching where every vertex is matched
(if |V| is even and if one exists)
• Maximum weight matching: given a weighted graph, find a matching
with maximum possible total weight
• Minimum weight perfect matching: given a weighted graph, find a
perfect matching with minimum cost
All matching problems are polynomial, even their weighted versions
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
32
Δ-TSP: ρ=1.5
There is a 1.5-approximation algorithm for Δ-TSP [Chistofides 1976]
Find a minimum spanning tree, T, of G, of cost C(T)
1
7
5
9
4
3
6
8
10
Let H* be an optimal tour of cost C(H*)
Let e be an edge of H* and T' be the rest of H* (this is a chain/tree)
C(H*) = w(e) + C(T') ³ w(e) + C(T) ³ C(T) ð C(T) £ C(H*)
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
33
Δ-TSP: ρ=1.5
1
7
5
9
2
4
3
6
8
10
Find the set of vertices of T of odd degree (S)
S contains an even number of vertices – why ?
Consider the graph GS induced by S
Find a minimum weight perfect matching, M, in GS
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
34
Δ-TSP: ρ=1.5
odd degree vertices
H*
Consider the odd degree vertices in S in the order that they appear in H*
Consider the red, M1, and the black, M2, matchings between them
Then C(H*) ≥ C(M1) + C(M2), by the triangle inequality
Also C(H*) ≥ C(M1) + C(M2) ≥ C(M) + C(M), since M is optimal
Hence, C(M) ≤ C(H*) / 2
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
35
Δ-TSP: ρ=1.5
1
7
5
1
9
2
4
3
9
2
4
6
8
10
7
5
6
3
8
10
Add the edges of M to T and let T’’ be the obtained (multi)graph
All vertices of T’’ are of even degree
Find an Euler cycle, W, in T’’
Euler cycle W: 1, 2, 3, 6, 8, 10, 9, 7, 5, 6, 4, 2, 1
W traverses each edge of T once:
C(W) = C(T) + C(M) £ C(H*) + C(H*) / 2 = 1.5 C(H*)
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
36
Δ-TSP: ρ=1.5
1
1
7
5
9
2
4
3
9
2
6
8
5
10
3
7
4
6
8
10
Find a tour H by shortcutting W:
H: 1, 2, 3, 6, 8, 10, 9, 7, 5, 6, 4, 2, 1
C(H) £ C(W) , by of the triangle inequality
C(H) £ C(W) £ 1.5 C(H*) ð
C(H) / C(H*) £ 1.5
QUESTION: What is the complexity of this algorithm ?
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
37
Δ-TSP: Tightness of ρ=1.5
b1
b2
1
b3
bn-1
1
bn
T, S, M, T’’, H
...
C(H)=n+n+n=3n
a2
a1
b1
1
b2
a3
an-1
an
b3
bn-1
bn
...
an+1
H*
C(H)=n+(n-1) +2 =2n+1
a1
a2
a3
...
an-1
an
an+1
C(H) / C(H*) à 3/2
AUEB / DoI / TOPICS IN ALGORITHMS / Spring 2017 / I. MILIS / 05 - TSP
38