Design and Analysis of
Approximation Algorithms
Ding-Zhu Du
Text Books
• Ding-Zhu Du and Ker-I Ko, Design and
Analysis of Approximation Algorithms
(Lecture Notes).
Chapters 1-8.
Schedule
•
•
•
•
•
•
•
•
•
Introduction
Greedy Strategy
Restriction
Partition
Guillotine Cut
Relaxation
Linear Programming
Local Ratio
Semi-definite Programming
Rules
• You may discuss each other on 5
homework assignments. But, do not copy
each other.
• Each homework is 10 points. 4 top scores
will be chosen.
• Midterm Exam (take-home) is 30 points.
• Final Exam (in class) is 30 points.
• Final grade is given based on the total
points (A ≥ 80; 80 > B ≥ 60 ; 60 > C ≥ 40).
Chapter 1 Introduction
• Computational Complexity (background)
• Approximation Performance Ratio
• Early results
Computability
• Deterministic Turing Machine
• Nondeterministic Turing Machine
• Church-Turing Thesis
Deterministic Turing Machine
(DTM)
tape
head
Finite Control
a
l
p
h
a
B
e
The tape has the left end but infinite to the
right. It is divided into cells. Each cell
contains a symbol in an alphabet Γ. There
exists a special symbol B which represents
the empty cell.
a
• The head scans at a cell on the tape and
can read, erase, and write a symbol on the
cell. In each move, the head can move to
the right cell or to the left cell (or stay in
the same cell).
a
• The head scans at a cell on the tape and
can read, erase, and write a symbol on the
cell. In each move, the head can move to
the right cell or to the left cell (or stay in
the same cell).
• The finite control has finitely many states
which form a set Q. For each move, the
state is changed according to the
evaluation of a transition function
δ : Q x Γ → Q x Γ x {R, L}.
b
a
p
q
• δ(q, a) = (p, b, L) means that if the head
reads symbol a and the finite control is in
the state q, then the next state should be p,
the symbol a should be changed to b, and
the head moves one cell to the left.
a
p
b
q
• δ(q, a) = (p, b, R) means that if the head
reads symbol a and the finite control is in
the state q, then the next state should be p,
the symbol a should be changed to b, and
the head moves one cell to the right.
s
• There are some special states: an initial
state s and an final states h.
• Initially, the DTM is in the initial state and
the head scans the leftmost cell. The tape
holds an input string.
x
h
• When the DTM is in the final state, the DTM
stops. An input string x is accepted by the
DTM if the DTM reaches the final state h.
• Otherwise, the input string is rejected.
• The DTM can be represented by
M = (Q, Σ, Γ, δ, s)
where Σ is the alphabet of input symbols.
• The set of all strings accepted by a DTM
$M$ is denoted by L(M). We also say that
the language L(M) is accepted by M.
• The transition diagram} of a DTM is an
alternative way to represent the DTM.
• For M = (Q, Σ, Γ, δ, s), the transition diagram of
M is a symbol-labeled digraph G=(V, E)
satisfying the following:
V = Q (s =
E={p
a/b,D
,h=
)
q | δ(p, a) = (q, b, D)}.
1/1,R
0/0,R; 1/1,R
0/0,R
s
0/0,R
p
q
B/B,R
h
1/1,R
M=(Q, Σ, Γ, δ, s) where Q = {s, p, q, h},
Σ = {0, 1}, Г = {0, 1, B}.
δ
s
p
q
0
1
(p, 0, R) (s, 1, R)
(q, 0, R) (s, 1, R)
(q, 0, R) (q, 1, R)
L(M) = (0+1)*00(0+1)*.
B
(h, B, R)
Nondeterministic Turing
Machine (NTM)
tape
head
Finite Control
a
l
p
h
a
B
e
The tape has the left end but infinite to the
right. It is divided into cells. Each cell
contains a symbol in an alphabet Γ. There
exists a special symbol B which represents
the empty cell.
• The finite control has finitely many states
which form a set Q. For each move, the
state is changed according to the
evaluation of a transition function
δ : Q x Γ → 2^{Q x Γ x {R, L}}.
Church-Turing Thesis
• Computability is Turing-Computability.
Multi-tape DTM
Input tape
(read only)
Storage
tapes
Output tape
(possibly, write only)
Time of TM
• TimeM (x) = # of moves that TM M takes on
input x.
• TimeM(x) < infinity iff x ε L(M).
Space
• SpaceM(x) = # of cell that M visits on the
work (storage) tapes during the
computation on input x.
• If M is a multitape DTM, then the work
tapes do not include the input tape and the
write-only output tape.
Time Bound
M is said to have a time bound t(n) if for
every x with |x| < n,
TimeM(x) < max {n+1, t(n)}
Complexity Class
• A language L has a (deterministic) timecomplexity t(n) if there is a multitape TM M
accepting L, with time bound t(n).
• DTIME(t(n)) = {L(M) | DTM M has a time
bound t(n)}
• NTIME(t(n)) = {L(M) | NTM M has a time
bound t(n)}
c
• P = U C>0 DTIME(n )
• NP = UC>0 NTIME(nc )
NP Class
Earlier Results on Approximations
• Vertex-Cover
• Traveling Salesman Problem
• Knapsack Problem
Performance Ratio
Approx (input )
r  sup input
for MIN
Opt (input )
Opt (inpu )
r  sup input
for MAX.
Approx (input )
Constant-Approximation
• c-approximation is a polynomial-time
approximation satisfying:
1 < approx(input)/opt(input) < c for MIN
or
1 < opt(input)/approx(input) < c for MAX
Vertex Cover
• Given a graph G=(V,E), find a minimum
subset C of vertices such that every edge
is incident to a vertex in C.
Vertex-Cover
• The vertex set of a maximal matching
gives 2-approximation, i.e.,
approx / opt < 2
Traveling Salesman
• Given n cities with a distance table, find a
minimum total-distance tour to visit each
city exactly once.
Traveling Salesman with triangular
inequality
• Traveling around a minimum spanning
tree is a 2-approximation.
Traveling Salesman with Triangular
Inequality
• Minimum spanning tree + minimum-length
perfect matching on odd vertices is 1.5approximation
Minimum perfect matching on odd vertices
has weight at most 0.5 opt.
Lower Bound
1+ε
1
1+ε
1
1+ε
1+ε
1+ε
2n  (1   )n
3

as n  
2  (2n  1)(1   )
2
Traveling Salesman without
Triangular Inequality
Theorem For any constant c> 0, TSP has no capproximation unless NP=P.
Given a graph G=(V,E), define a distance table
on V as follows:
if (u, v)  E
1,
d (u, v)  
| V | c, if (u, v)  E
Contradition Argument
• Suppose c-approximation exists. Then we
have a polynomial-time algorithm to solve
Hamiltonian Cycle as follow:
C-approximation solution < cn
if and only if
G has a Hamiltonian cycle
Knapsack
max c1 x1  c 2 x 2    cnxn
s. t. s1 x1  s 2 x 2    snxn  S
xi  {0, 1}.
Assume si  S for any i. Hence ci  opt for any i.
c1 c 2
cn
Sort
  .
s1 s 2
sn
k 1
k
Choose k such that
s  S  s.
i
i 1
k
Output cG  max{  ci, ck  1}.
i 1
i
i 1
2-approximation

c
i  opt  
c
i
i 1
i 1
k
k 1
cG  opt  cG  ck  1  2cG .
PTAS
• A problem has a PTAS (polynomial-time
approximation scheme) if for any ε > 0, it
has a (1+ε)-approximation.
Knapsack has PTAS

• Classify: for i < m, ci < a= 1   cG,
for i > m+1, ci > a.
• Sort c1 c 2
cm
s1

s2

sm
.
• For I  {m  1,, n},
if
 s  S , then set c( I )  0;
i
iI
If
 s  S , then choose maximum
i
k  m such that
iI
k
s  S s
i
i 1
i
iI
and set
k
c( I )   ci   ci.
iI
i 1
Output c( Ioutput)  max I c( I ).
Proof.
Let I *  {i {m  1,..., n} | xi  1 in opt}.

If 
If
s
i  S 
s
i , then c ( I *)  opt .
i 1
iI *
m
m
i 1
si  S  iI * si, then opt  c( I *)  ck  1
 c( I *)  a.
Hence, c( Ioutput)  opt  c( Ioutput)  a
 c( Ioutput) 
opt
 1 
c( Ioutput)

1 
opt.
Time
Note opt  2cG. If | I |  2(1   )/ , then

c
i  ( 2(1   ) /  ) a  2cG  opt .
iI
Therefore, we need only consider t hose I with
| I | 2(1   ) /  .
This gives time O(n
2 (1 ) / 
)n
O (1 /  )
.
Fully PTAS
• A problem has a fully PTAS if for any ε>0,
it has (1+ε)-approximation running in time
poly(n,1/ε).
Fully FTAS for Knapsack
c(i, j ) is a subset of indeces in {1, ..., i} such that


ic ( i , j )
ci  j ,

ic ( i , j )
si  S and
s
i  min {
s
i |
c
i  j,
ic ( i , j )
iI
iI
I  {1,..., i}}.
c(i, j )  nil if no such subset exists.
Pseudo Polynomial-time Algorithm
for Knapsak
• Initially,
for j  1, ... csum  c1    cn,
  if j  0,

c(1, j )  {1} if j  c1,
nil otherwise

for i  1 to n do
for j  0 to csum do
Case 1. c(i  1, j  ci )  nil ,
Set c(i, j )  c(i  1, j );
Case 2. c(i  1, j  ci )  nil , but
si  kc (i 1, j ci ) sk  S .
Set c(i, j )  c(i  1, j ).
Case 3. c(i  1, j  ci )  nil , si  kc ( i 1, j ci ) sk  S ,
and c(i  1, j )  nil .
Set c(i, j )  c(i  1, j  ci )  {i}.
Case 4. c(i  1, j  ci )  nil , si  kc ( i 1, j ci ) sk  S ,
and c(i  1, j )  nil .
If si  kc (i 1, j ci ) sk  kc ( i 1, j ) sk
then set c(i, j )  c(i  1, j )
else set c(i, j )  c(i  1, j  ci )  {i}.
Output opt  max{ j | c(i, j )  nil }.
Time
•
•
•
•
•
outside loop: O(n)
Inside loop: O(nM) where M=max ci
Core: O(n log (MS))
3
Total O(n M log (MS))
Since input size is O(n log (MS)), this is a
log M
pseudo-polynomial-time due to M=2
 n(h  1) 
Set c' i  ci

M


Consider
max c'1 x1  c'2 x 2    c' nxn
s.t. s1 x1  s 2 x 2    snxn  S
xi  {0, 1}.
h
Its optimal solution x can be computed in time
O (n (h  1) log( n(h  1) S )).
4
Let c  c1 x 1  c 2 x 2    cnx n. We show
h
h
h
1 h
opt  (1  )c
h
h
Suppose opt  c1 x * 1  c 2 x * 2    cnx * n.
c h  c1 x h 1    cnx h n
M
 (c'1 x 1    c' nx n)
n(h  1)
M
 (c'1 x * 1    c' nx * n )
n(h  1)
c1n(h  1)
cnn(h  1)
M
 ((
 1) x * 1    (
 1) x * n )
M
M
n(h  1)
M
1
 opt 
 opt (1 
)
h 1
h 1
h
h
Thanks, End
Lecture 3 Complexity of
Approximation
•
•
•
•
L-reduction
Two subclass of PTAS
Set-cover ((ln n)-approximation)
c
Independent set (n -approximation)