Strong N P Hardness Approximation Primal-Dual
Discrete Optimization 2010
Lecture 10
More about N P-complete Problems
Marc Uetz
University of Twente
[email protected]
Lecture 10: sheet 1 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Outline
1
Pseudo-Polynomial Time Algorithms and Strong N P-hardness
2
Approximation Algorithms
3
Primal-Dual Algorithms
Lecture 10: sheet 2 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Knapsack Problem
Given
n items with weights wj ≥ 0 and value vj ≥ 0
a knapsack with total (weight) capacity W
Problem
find subset S of items with weight ≤ W and maximal value
Theorem
The Knapsack problem is N P-hard (Reduction from Partition).
Lecture 10: sheet 3 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Solving the Knapsack Problem
weights wj :
1
values vj :
2
v1 = 2
3
v2 = 3
v3 = 4
capacity W = 4
Construct directed graph G = (V , A): nodes ujk , k = wj , . . . , W ,
edges (uik , uj` ) with lengths vj ⇔ i < j and ` − k = wj
item
size
1
2
1
2
3
4=W
u11
u12
u13
u14
v1
s
v2
v2
u22
u23
u24
t
v3
v3
u33
3
Lecture 10: sheet 4 / 33
v2
Marc Uetz
u34
Discrete Optimization
length 0
!"#$%&'%()*%"'(+,%-.*/-%0(1#23-%-#2
Strong N P Hardness Approximation Primal-Dual
Solving the Knapsack Problem
!Observations
!"#$%&'(!"#$%&'%(%$)(*+,#-'. )*(&!"%(+#$
/+-(0123(12#43(5+(.67+(%$5"-$%$8(,60+(*%5+*.
%$5"-$%$8(,60+(*%5+*.
.9)'(-'%-(0123(".(%,(9&&
9&&
any (s, t)-path traverses
some ‘rows’ of the digraph,
thereby
569,0
:6$(-'+(.'6$-+.-(&%-'(:$67(.(-6(2(12#4;.3<
collecting
the corresponding items
P
! ='+(,60+(*%5+*.(0123(%$+(.'6$-+.-(&%-'(*+,#-'.(":(%,0(6,*8(":
":(%,0(6,*8(":
by definition, all these items fit in the knapsack,
wi ≤ W
01>3(! 01,3 ? ),%- :6$(%**(12@>3#A
length of path = total value of items collected along the path
!
⇒ longest (s, t)-path = optimal solution Knapsack
!
$ paths "in digraphs?
How to compute
longest
!
" "
!
* as shortest paths only with 4-inequality
+
Same
(&)
%&'
reversed: d(s, w ))≥ d(s,
$ v ) + c)vw
(digraph is acyclic,
so
we
even have# a linear
!
$
time algorithm)
# #"
!
Lecture 10: sheet 5 / 33
Marc Uetz
Discrete Optimization
)2>
2
>
Strong N P Hardness Approximation Primal-Dual
Solving the Knapsack Problem
Knapsack is N P-hard
Longest Path is polynomially solvable, so ∈ P
And we have a transformation
T : Knapsack → Longest Path
Hence, can we conclude that P=N P
and claim 1.000.000 $ from the Clay Mathematics Institute?
Lecture 10: sheet 6 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
No 1.000.000 $ (yet). . .
Encoding length of Knapsack ∈ O( n log U ), where
U := max{W , n, wj , vj , j = 1, . . . , n} = maximal input number
Transformation, in general, yields digraph with Ω( W ) nodes
So in general, not even the size of the digraph is polynomial, and
transformation T : Knapsack → Longest Path is not poly-time
Example: Say, knapsack size W = 2n , and all wj , vj ≤ W , then
|Knapsack| ∈ O( n log W ) = O( n2 )
but (as long as there is one item with wj ∈ O( 1 )), we have that
|Digraph| ∈ Ω( 2n )
(and the longest path computation takes Ω( 2n ) time, too)
Lecture 10: sheet 7 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Solving the Knapsack Problem
Yet, we have an algorithm to (optimally) solve Knapsack, what is
it’s computation time?
Again, let U be the largest number appearing in the input
Longest path computation ∈ O( n0 + m0 ) (with n0 and m0 the
number of nodes & arcs in the digraph)
n0 ∈ O( nU ), and m0 ∈ O( (nU)2 )
hence computation time ∈ O( nU + (nU)2 ) ∈ O( (nU)2 )
and encoding length Knapsack is ∈ Ω( n + log U )
As long as U ∈ poly( n ), this is poly-time algorithm for Knapsack
In other words: Knapsack no longer N P-hard if U ∈ poly( n )
(unless P = N P)
Lecture 10: sheet 8 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Pseudo Polynomial Time Algorithms
Let I be instance of problem P (either decision or optimization),
|I | its (binary) encoding length, and number (I ) := the largest
number appearing in instance I
Definition
We say that algorithm A is a pseudo polynomial time algorithm for
P if there exists a polynomial function q : R × R → R such that
A solves P
tA (I ) ≤ q(|I |, number (I )) for all instances I ∈ P
Example: Our previous algorithm for the Knapsack problem
Note: Pseudo poly-time is equivalent to poly-time w.r.t. unary
encoding of instances of the problem (|I |unary ≥ number (I ))
Lecture 10: sheet 9 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Strongly / Unary N P-complete Problems
Let P be a problem, |I | the (binary) encoding length of any
instance I ∈ P, and f : R → R be some nonnegative function,
denote by Pf the restricted version of problem P
Pf = {I ∈ P | number (I ) ≤ f (|I |)}
Definition
Problem P is strongly / unary N P-complete (N P-hard) if there
exists a polynomial function p such that Pp is N P-complete
(N P-hard)
In words: P remains N P-hard even if ‘all numbers are restricted to
be small’, i.e. polynomial in |I |
Knapsack is N P-complete, but not strongly N P-complete
Lecture 10: sheet 10 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Strongly N P-complete problems: Examples
(1) Problems that have ‘no large numbers’ anyhow, so where
number (I ) ≤ |I |, for example. . .
Clique
encoding length Clique is ∈ Ω( n + m )
and number (I ) = m (assuming a connected graph)
hence Cliquep =Clique is N P-complete, e.g. for p(x) = x
(as number (I ) ≤ |I | for any I ∈ Clique)
So, Clique is strongly N P-complete
Lecture 10: sheet 11 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Strongly N P-complete problems: Examples
(2) Problems where N P-completeness does not depend on the
presence of large numbers, for example. . .
0 − 1 Integer Programming Feasibility (IPF)
given A ∈ Zm×n , b ∈ Zm , ∃x ∈ {0, 1}n with Ax ≤ b?
encoding length A, b is certainly ∈ Ω( n + m )
transformation SAT → 0 − 1 IPF, only requires coefficients 0
and ±1 in A and largest coefficient in b is m − 1
Hence, restricted version 0 − 1 IPF, where all coefficients in
A, b are at most m, is still N P-complete
Hence, 0 − 1 IPFp is N P-complete, e.g. for p(x) = x
(number (I ) ≤ m ≤ |I | suffices for transformation from SAT)
So, 0 − 1 IPF is strongly N P-complete
Lecture 10: sheet 12 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Strong N P-completeness
Theorem
A strongly N P-complete problem P cannot have a pseudo
polynomial time algorithm, unless P = N P.
Proof: Assume the contrary, so P has pseudo poly-time algorithm
A, we know ∃ polynomial q with
tA (I ) ≤ q(|I |, number (I )) ∀ I ∈ P
P is strongly N P-complete, have polynomial p, such that Pp is
still N P-complete (Pp = instances with number (I ) ≤ p(|I |)).
Now for all instances I ∈ Pp , computation time of A is
tA (I ) ≤ q(|I |, p(|I |)) ,
so polynomial in |I |. Hence P= N P.
Lecture 10: sheet 13 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Outline
1
Pseudo-Polynomial Time Algorithms and Strong N P-hardness
2
Approximation Algorithms
3
Primal-Dual Algorithms
Lecture 10: sheet 14 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Dealing with N P-hard problems
So your favourite problem P turns out to be N P-hard. . .
Algorithmic Choices
1
Exact Algorithms
compute optimum solution, no bound on computation time
(Branch & Bound, Cutting Planes, Branch & Cut, . . . )
2
Heuristic & Local Search
compute some solution, no bound on computation time
(Constructive heuristics, and then: Simulated Annealing, Tabu
Search, Genetic Algorithms, Ant Colony Optimization, . . . )
3
Approximation Algorithms
compute solution with (worst-case) performance guarantee
do it quickly, i.e., polynomial bound on computation time
Lecture 10: sheet 15 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Performance Guarantees
!"#$%#&'()"*+,'#'(-"".
Definition
Given
a combinatorial minimization problem P, and α ≥ 1, then
!"#$%$&$'%()*$+"%),%)'-&$.$/,&$'%)0.$%$.$/,&$'%1)-2'34".)56),%),47'2$&8.)9)
algorithm
A has performance
guarantee α for
P if
8,:),)0.;4&$-4$<,&$+"1)-"2#'2.,%<")7;,2,%&"")
-"2#'2.,%<")7;,2,%&"")!
! #'2)-2'34".)5
$#)
1
2
A returns
a solution for all instances
I ∈ P =)! 5
!9)2"&;2%:),):'4;&$'%)90=1)!
>))#'2),44)$%:&,%<":
if A(I
) = value of As solution for I , and OPT (I ) = optimal
! $#)90=1)$:)&8"):'4;&$'%)+,4;")'#)&8"):'4;&$'%)2"&;2%"?)3@)96
solution
value, then
,%?)A5B0=1)$:)&8"):'4;&$'%)+,4;")'#),%)'-&$.,4):'4;&$'%6
90=1)"
" ! ! A5B0=1
90=1)
A(I ) ≤
αOPT
(I ) for#'2),44)$%:&,%<":)=)!
all instances I ∈5P
!#! OPT(I)
OPT(I)
A(I)
0C".,2D(),??$&$+")-"2#'2.,%<")7;,2,%&"")!())90=1)
" A5B0=1)E)!
=)! 51 (I )
A5B0=1)E)
Note: For
maximization problems 0 ≤ α ≤ 90=1)"
1 and
A(I ) ≥! "αOPT
Lecture 10: sheet 16 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Approximation Algorithms
Definition
Algorithm A is an α-approximation algorithm for problem P if
A has a performance guarantee α for P
A is a polynomial time algorithm
Lecture 10: sheet 17 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Worst-case Ratios
Definition
We say α is the worst case ratio for algorithm A on minimization
(maximization) problem P if
A has performance guarantee α
for all β < α (β > α), there exist instances I such that
A(I ) > βOPT (I )
A(I ) < βOPT (I )
In words: The performance guarantee of A on P is indeed not
better than α, or ‘analysis of A is tight’
Lecture 10: sheet 18 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
The Vertex Cover Problem (Again)
Vertex Cover
Given undirected graph G = (V , E ) find subset C ⊆ V of nodes of
minimal cardinality such that e ∩ C 6= ∅ for all e ∈ E
Vertex Cover is (strongly) N P-hard (∃ transformation from SAT)
!""#$%&'$()*(+*,(-.*/0.1'.23*4(5.1
Applications: telecommunication
(when placing antennae), or for
example: In street network, place minimum # tollbooths such that
!"#$#%&'((&#"(&)*'+,#-.$/(#01"1020#$0*2"&#*3#&*..#4**&5%#%2/5#&5$&#
any path
can be charged correctly
$"6#-$&5#&'$7(..(8 /$"#4(#/5$'9(8#/*''(/&.6:
Lecture 10: sheet 19 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Greedy Algorithm for Vertex Cover
Given G = (V , E ), the following seems perfectly reasonable. . .
Algorithm Greedy
let W = ∅
while (there are edges left in G )
pick v ∈ V with largest degree d(v ) := |δ(v )|, W ← W ∪ v
eliminate v and all incident edges from G
return vertex cover W
Computation time is O( n2 ), what about the (guaranteed)
performance?
Lecture 10: sheet 20 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Analysis Greedy Algorithm
Let Hm =
Pm
1
i=1 i
be the mth harmonic number (Hm ≈ log m)
Theorem
G = (V , E ), |E | = m, Greedy is Hm -approximation algorithm
Proof: Say in iteration k we pick vk covering d(vk ) uncovered edges, we
say each of these edges e has coste := d(v1 k ) . Then the solution value of
Xm
Greedy can be written
coste
e=1
Let us assume the edges are covered in the order e1 , . . . , em . Then, before
the iteration (say iteration k) where edge ej was covered, there are at
least m − j + 1 uncovered edges left. OPT can cover these edges with
≤ |OPT | nodes, thus there is a node v left with degree d(v ) ≥ m−j+1
|OPT | ,
so edge ej , covered in iteration k, has costej ≤
Greedy =
m
X
coste ≤ |OPT |
e=1
Lecture 10: sheet 21 / 33
m
X
e=1
Marc Uetz
|OPT |
m−j+1 .
Therefore,
1
= Hm |OPT |
m−e +1
Discrete Optimization
!"#$%&'&(!$)*+',-.(/
Strong N P Hardness Approximation Primal-Dual
Greedy Algorithm for Vertex Cover
Greedy may compute vertex cover containing A, of size 8, where
|OPT | = 5
!"#$%&'()*+,*-($$./.*0*1*!*!
2*$%*0*1*!*! 3*" 404*1*56
Marc Uetz
Discrete Optimization
Lecture 10: sheet 22 / 33
Strong N P Hardness Approximation Primal-Dual
Greedy Algorithm for Vertex Cover
!"#$%&'&(!$)*+',-.(/(01*",'"2345
Greedy may compute vertex cover containing A, of size 14, where
|OPT
|=6
!"#$%&'()*+,*-($$./.*0*1*!*!
2**" 303*1+45
67'*$8'&)7)*&.*0*1*2*9&'(*303*1*:*;
Lecture 10: sheet 23 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Greedy Algorithm!"#$%&'&(!$)*+',-.(/(01*",'"2345
for Vertex Cover
More generally:!"#$%&'()*+,*-($$./.*0*1*!*!
Layers B and C 2**"
= 303*1+45
perfect matching of 2k nodes
67'*$8'&)7)*&.*0*1*2*9&'(*303*1*:*;
Layer A:
b k2 c nodes, each connected to 2 nodes of B
b k3 c nodes, each connected to 3 nodes of B, etc. . .
. . . 1 node connected to all nodes of B
So Greedy may pick vertex cover containing A, total size
k + b k2 c+ b k3 c + · · · + 1 ≥ k( 12 + 13 + · · · + k1 ) = k(Hk − 1),where
|OPT | = k
Lecture 10: sheet 24 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Greedy Algorithm for Vertex Cover
Theorem
The worst-case ratio of Greedy is Θ( log n ) for Vertex Cover.
Proof:
Have seen that Greedy is an Hm -approximation, and have seen
instances where Greedy ≥ (Hk − 1)OPT , for k = k(n) as in the
example before, so if α is the worst-case ratio for Greedy,
Hk − 1 ≤ α ≤ H m
Finally, Hm ∈ Θ( log m ) ∈ O( log(n2 ) ) = O( log n )
and Hk − 1 ∈ Ω( log k ) = Ω( log(k 2 ) ) ∈ Ω( log n ).
(in previous example, n ≤ 2k + k log k ≤ k 2 )
Lecture 10: sheet 25 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Design of Approximation Algorithms
Let P be a (combinatorial) minimization problem, α ≥ 1
The Main Challenge
what we want is poly-time algorithm A with
A(I ) ≤ αOPT (I ) for all instances I ∈ P
but the trouble is: mostly we don’t know OPT (I )
So what we typically need: some lower bound LB(I ) with
LB(I ) ≤ OPT (I ), and A(I ) ≤ αLB(I )
Lecture 10: sheet 26 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Another Algorithm for Vertex Cover
Now we want to make use of a clever lower bound on |OPT |. . .
Algorithm A
let W = ∅
while (there are edges left in G )
pick an arbitrary edge e = {v , w }
let W ← W ∪ {v , w }
eliminate v and w and all incident edges from G
Computation time is O( n + m ), what about A’s performance?
Lecture 10: sheet 27 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Analysis Algorithm A
Theorem
On any G , algorithm A is a 2-approximation for Vertex Cover
Proof:
Observe that OPT ≥ |M| for any matching M in G
(size of any matching is a lower bound on OPT )
The edges selected by algorithm A are a matching, say M A
A selects both endpoints of any edge, so
A = 2|M A | ≤ 2OPT
Lecture 10: sheet 28 / 33
Marc Uetz
Discrete Optimization
1$%0-(2/0#(%/-,$($3(.(3$%(4$5#(2$6#%
Strong N P Hardness Approximation Primal-Dual
Analysis Algorithm A
#'().(,0(/(.:/;;%$<,&/-,$4(/*+$%,-"&7
Theorem
-/42#?0@(>(1"#%#(;#%3$%&/42#(,0(/%A,-%/%,*9
The worst-case ratio of algorithm A is 2.
B8!?>@7
Proof:
#'
Algorithm A selects all nodes, half of them would suffice
E?).@(F(.4(F(.!B8!7
Lecture 10: sheet 29 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Outline
1
Pseudo-Polynomial Time Algorithms and Strong N P-hardness
2
Approximation Algorithms
3
Primal-Dual Algorithms
Lecture 10: sheet 30 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Crucial Idea Behind Algorithm A
Recall, the dual of Vertex Cover is the Matching problem
Vertex Cover
min
X
v ∈V
xv
s.t. xv + xw ≥ 1
∀ {v , w } ∈ E
x ∈ {0, 1}
Matching = Dual of Vertex Cover
X
max
ye
e∈E
s.t.
X
e∈δ(v )
ye ≤ 1
∀v ∈V
x ∈ {0, 1}
Lecture 10: sheet 31 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Crucial Idea!"#$%&'()"*+,Behind Algorithm
A
./,01*("2$*3$'$45'/$.6610')"
OPT
zLP
!"#$!%$! & '%
()*)$$$'% +$', " -$$$# ./0%1,2$! 3
'% ! 041-2$$$$$$# %$! &
LP Relaxation (LP)
!"#$!%$! & '%
()*)$$$'% +$', " -$$$# ./0%1,2$! 3
'% " 4$$$$$$$$$$$$$# %$! &
zLP = zD
LP Dual (D)
!5'$!.$! 3 6.
D
z ()*)$$$$!.$! "7%8 6. ! -$$$# %$! &
6. " 4$$$$$$$$$$$$$$$$$# .$! 3
OPT
Any
the LP
dual
lower
bound
on OPT !
Anysolution
(!) feasibleof
solution
of (D)
is a is
lower
bound
on OPT
A and
matching
is just
solution
of the
LP dual
any matching
M issome
nothing
but a feasible
solution
for (D)
Lecture 10: sheet 32 / 33
Marc Uetz
Discrete Optimization
Strong N P Hardness Approximation Primal-Dual
Crucial Idea Behind Algorithm A
For any minimization problem P, a feasible solution for the dual is
a valid lower bound on OPT
So if we can compute a feasible solution x for P, and a feasible
solution y for P’s dual, and prove x and y are ‘close’, we’re done
Here, algorithm A computes feasible solutions x = VC for primal
and y = M for dual, respectively, such that |VC | ≤ 2|M|
OP T (LP )
|M |
OP T (V C)
|V C|
Question: is there a systematic way to achieve that?
Lecture 10: sheet 33 / 33
Marc Uetz
Discrete Optimization
2|M |