CS 4407
Algorithms
Lecture 8:
Circumventing Intractability, using
Approximation and other Techniques
Prof. Gregory Provan
Department of Computer Science
University College Cork
CS 4010
1
Lecture Outline
Introduction to Approximation Algorithms
Motivation
Approximation Algorithm definitions
Examples
Vertex cover
Traveling Salesman
Summary
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Motivation
By now we’ve seen many NP-Complete problems.
We conjecture none of them has polynomial time
algorithm.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Motivation
Is this a dead-end? Should we give up
altogether?
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Motivation
Or maybe we can settle for good approximation
algorithms?
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Today’s Learning Objectives
Objectives:
– To formalize the notion of approximation.
– To demonstrate several such algorithms.
Overview:
– Optimization and Approximation
– Vertex Cover, Traveling Salesman
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Optimization
Many of the problems we’ve encountered
so far are really optimization problems.
I.e - the task can be naturally rephrased as
finding a maximal/minimal solution.
For example: finding a maximal clique in a
graph.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Approximation
An algorithm that returns an answer C which is
“close” to the optimal solution C* is called an
approximation algorithm.
“Closeness” is usually measured by the ratio
bound ρ(n) the algorithm produces.
Which is a function that satisfies, for any input
≤ρ(n).
size n, max{C/C*,C*/C}≤ρ
≤ρ
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Approximation Algorithms
Approximation algorithms: provide sub-optimal answers
that are still reasonable
Approximation algorithms tend to use greedy approaches
– In some cases, these algorithms rely on “heuristics” to help
search through the possible answers for the best one (which is
not really the best one, but might be good enough)
• A heuristic is generally defined as a “rule of thumb”, some basic idea
applied to problem solving
• Often, heuristics are used in Artificial Intelligence to reduce an
intractable search to something tractable
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Definitions
A feasible solution is an object of the right type but not necessarily an
optimal one
– Example: a feasible solution to graph coloring is one that colors all vertices
such that no two adjacent vertices have the same color, but such a solution
may not have the minimum number of colors
– Example: a feasible solution to bin packing will fill items into bins such that
no bin exceeds 1 in size, but many not provide the minimum number of bins
Value function
– returns the value of the parameter for the feasible solution that is used to
determine optimality (number of colors, number of bins, etc)
Optimum value
– the value returned by the value function when tested on the optimal solution
(or as provided by the optimal algorithm)
We judge an approximation algorithm by the value provided by the value
function compared to the optimum value
– This is a value >= 1 (where 1 means as good as the optimal algorithm)
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Examples
Vertex cover
– Real-world motivation
– Approximation algorithm
Traveling salesman
– Real-world motivation
– Approximation algorithm
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Network Power
Say you have a network, with links between some components
Each link requires power supply, hence, you need to supply
power to a set of nodes that cover all links
Obviously, you’d like to connect the smallest number of nodes
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
VERTEX-COVER
Instance: an undirected graph G=(V,E).
Problem: find a set C⊆
⊆V of minimal size s.t. for any
(u,v)∈
∈E, either u∈
∈C or v∈
∈C.
Example:
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Minimum VC NP-hard
Proof: It is enough to show the decision problem
below is NP-Complete:
Instance: an undirected graph G=(V,E) and a number k.
Problem: to decide if there exists a set V’⊆
⊆V of size k s.t
∈E, u∈
∈V’ or v∈
∈V’.
for any (u,v)∈
This follows immediately from the following
observation.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Minimum VC NP-hard
Observation: Let G=(V,E) be an undirected graph. The
complement V\C of a vertex-cover C is an
independent-set of G.
Proof: Two vertices outside a vertex-cover cannot be
connected by an edge. CS 4407, Algorithms
University College Cork,
Gregory M. Provan
VC - Approximation Algorithm
C←φ
E’ ← E
while E’ ≠ φ
– do let (u,v) be an arbitrary edge of E’
– C ← C ∪ {u,v}
– remove from E’ every edge incident to
either u or v.
return C.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Demo
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Polynomial Time
C←φ
E’ ← E O(n2)
while E’ ≠ φ do
– let (u,v) be an arbitrary edge of E’
O(1)
– C ← C ∪ {u,v}
O(n2)
– remove from E’ every edge incident to either
O(n)
u or v
return C
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Correctness
The set of vertices our algorithm returns is clearly a
vertex-cover, since we iterate until every edge is
covered.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Vertex-cover problem (Example 2)
Near Optimal
size=6
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Optimal
Size=3
20
How Good an Approximation is it?
Observe the set of edges our algorithm chooses
no common vertices! ⇒ any VC contains 1 in each
our VC contains both, hence at most twice as large
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
The vertex-cover problem: Bounds
This is a polynomial-time
2-aproximation algorithm. (Why?)
– Because:
Optima
l
• APPROX-VERTEX-COVER is O(V+E)
• |C*| ≥ |A|
Selected
|C| = 2|A|
Edges
|C| ≤ 2|C*|
Selected
Vertices
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
22
Example 2: Traveling Salesman
The Mission: A Tour Around the World
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
The Problem: Traveling Costs Money
1795$
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Introduction
Objectives:
– To explore the Traveling Salesman Problem.
Overview:
–
–
–
–
TSP: Formal definition & Examples
TSP is NP-hard
Approximation algorithm for special cases
Inapproximability result
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
TSP
Instance: a complete weighted undirected graph
G=(V,E) (all weights are non-negative).
Problem: to find a Hamiltonian cycle of minimal cost.
3
1
3
4
5
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
2
10
Polynomial Algorithm for TSP?
What about the greedy
strategy:
At any point, choose the closest
vertex not explored yet?
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
The Greedy $trategy Fails
10
∞
2
∞
5
12
∞
1
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
3
0
The Greedy $trategy Fails
10
∞
2
∞
5
12
∞
1
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
3
0
TSP is NP-hard
The corresponding decision problem:
Instance: a complete weighted undirected graph
G=(V,E) and a number k.
Problem: to find a Hamiltonian cycle whose cost is
at most k.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
TSP is NP-hard
Theorem: HAM-CYCLE ≤p TSP.
Proof: By the straightforward efficient reduction
illustrated below:
verify!
1
1
2
1
HAM-CYCLE
TSP
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
1 2
k=|V|
What Next?
We’ll show an approximation
algorithm for TSP,
with approximation factor 2
for cost functions that satisfy a certain
property.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
The Triangle Inequality
Definition: We’ll say the cost function c
satisfies the triangle inequality, if
∀u,v,w∈
∈V : c(u,v)+c(v,w)≥
≥c(u,w)
u
w
v
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Approximation Algorithm
1. Grow a Minimum Spanning Tree (MST) for G.
2. Return the cycle resulting from a preorder
walk on that tree.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Demonstration and Analysis
The cost of a
minimal Hamiltonian
cycle ≥ the cost of a
MST
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Demonstration and Analysis
The cost of a
preorder walk is
twice the cost of
the tree
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Demonstration and Analysis
Due to the triangle
inequality, the
Hamiltonian cycle is
not worse.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
The Bottom Line
optimal
HAM
cycle
≥
MST
= ½·
preorder
walk
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
≥ ½·
our
HAM
cycle
Example
T
d
a
e
b
c
f
g
b
c
f
e
g
c
h
d
a
b
a
d
b
f
e
b
c
d
a
e
h
T
T
d
a
f
h
g
H
c
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
h
f
h
optimal
solution
e
g
H*
g
Formal Algorithm
35.2.1 The TSP with triangle inequality
APPROX_TSP_TOUR( G,c)
1 Select a vertex r ∈V [ G ] to be a root vertex
2 grow a MST T for G from root r using
MST_PRIM(G,c,r)
3 Let L be the list of vertices visited in a preorder
walk of T.
4 return the ha miltonian cycle H that visit the vertices in
the order L.
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Formal Proof
Theorem 35.2. APPROX_TSP_TOUR is an approximation
algorithm with ratio bound of 2 for TSP with triangular
inequality.
Proof.
c( T ) ≤ c( H *)
c( W ) = 2 c( T ) ≤ 2 c( H *)
c( H ) ≤ c( W )
⇒ c( H ) ≤ 2 c( H *)
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
What About the General Case?
We can show TSP cannot be
approximated within any constant
factor ρ≥1
ρ≥
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Formal Proof Sketch of TSP Bounds
35.2.2 The general TSP
Theorem
35.3
NP ≠ P
If
and
ρ ≥ 1 , there is no
polynomial time approximation algorithm with ratio bound
ρ for some general TSP.
Proof.
Let G = ( V , E ) be an instance of HC.
Let G ' = ( V , E ' ) be the complete graph of V.
 1
Set c ( u , v ) = 
 ρ | V |+ 1
if ( u , v ) ∈ E
o th e rw ise
CS 4407, Algorithms
University College Cork,
Gregory M. Provan
Summary
As it turns out, we can sometimes find
efficient approximation algorithms for NPhard problems.
We’ve seen two such algorithms:
– for VERTEX-COVER (factor 2)
– for TSP (factor 2).
In some cases (TSP) finding close
approximations is NP-hard
CS 4407, Algorithms
University College Cork,
Gregory M. Provan