NP-Completeness
Reference: Computers and Intractability: A
Guide to the Theory of NP-Completeness
by Garey and Johnson,
W.H. Freeman and Company, 1979.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
1
General Problems, Input Size and
Time Complexity
• Time complexity of algorithms :
polynomial time algorithm ("efficient algorithm") v.s.
exponential time algorithm ("inefficient algorithm")
f(n) \ n
10
30
50
n
0.00001 sec
0.00003 sec
0.00005 sec
n5
0.1 sec
24.3 sec
5.2 mins
2n
0.001 sec
17.9 mins
35.7 yrs
Young
CS 530 Ad. Algo. D&A
NP-Completeness
2
1
“Hard” and “easy’ Problems
• Sometimes the dividing line between “easy” and “hard”
problems is a fine one. For example
– Find the shortest path in a graph from X to Y. (easy)
– Find the longest path in a graph from X to Y. (with no
cycles) (hard)
• View another way – as “yes/no” problems
– Is there a simple path from X to Y with weight <= M? (easy)
– Is there a simple path from X to Y with weight >= M? (hard)
• First problem can be solved in polynomial time.
• All known algorithms for the second problem (could)
take exponential time .
Young
CS 530 Ad. Algo. D&A
NP-Completeness
3
• Decision problem: The solution to the
problem is "yes" or "no". Most optimization
problems can be phrased as decision
problems (still have the same time
complexity).
Example : Assume we have a decision
algorithm X for 0/1 Knapsack problem with
capacity M, i.e. Algorithm X returns “Yes”
or “No” to the question “is there a solution
with profit  P subject to knapsack capacity
 M?”
Young
CS 530 Ad. Algo. D&A
NP-Completeness
4
2
We can repeatedly run algorithm X for
various profits(P values) to find an optimal
solution. Example : Use binary search to get
the optimal profit,
maximum of lg pi runs.
(where M is the capacity of the knapsack optimization problem)
Min Bound
Optimal Profit
Max Bound
Search for the optimal solution
0
pi
|___________________|_________________|
Young
CS 530 Ad. Algo. D&A
NP-Completeness
5
The Classes of P and NP
• The class P and Deterministic Turing
Machine
• Given a decision problem X, if there is a
polynomial time Deterministic Turing Machine
program that solves X, then X is belong to P
• Informally, there is a polynomial time
algorithm to solve the problem
Young
CS 530 Ad. Algo. D&A
NP-Completeness
6
3
• The class NP and Non-deterministic Turing
Machine
• Given a decision problem X, if there is a
polynomial time Non-deterministic Turing
machine program that solves X, then X belongs
to NP
• Given a decision problem X. For every instance
I of X, (a) guess solution S for I, and (b) check
“is S a solution to I?”. If (a) and (b) can be done
in polynomial time, then X belongs to NP.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
• Obvious : P  NP, i.e. A
problem in P does not need
“guess solution”. The correct
solution can be computed in
polynomial time.
7
NP
P
• Some problems which are in NP, but may not in P :
• 0/1 Knapsack Problem
• PARTITION Problem : Given a finite set of positive
integers Z.
Question : Is there a subset Z' of Z such that
Sum of all numbers in Z' = Sum of all numbers in Z-Z' ?
i.e.  Z' =  (Z-Z')
Young
CS 530 Ad. Algo. D&A
NP-Completeness
8
4
• One of the most important open problem in
theoretical compute science : Is
P=NP ?
Most likely “No”. Currently, there are many
known problems in NP, and there is no
solution to show anyone of them in P.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
9
NP-Complete Problems
•
Stephen Cook introduced the notion of NPComplete Problems. This makes the
problem “P = NP ?” much more interesting
to study.
•
The following are several important things
presented by Cook :
Young
CS 530 Ad. Algo. D&A
NP-Completeness
10
5
1. Polynomial Transformation ("  ")
• L1  L2 : There is a polynomial time
transformation that transforms arbitrary
instance of L1 to some instance of L2.
• If L1  L2 then L2 is in P implies L1 is in
P (or L1 is not in P implies L2 is not in P)
• If L1  L2 and L2  L3 then L1  L3
Young
CS 530 Ad. Algo. D&A
11
2.
Focus on the class of NP – decision problems only.
Many intractable problems, when phrased as decision
problems, belong to this class.
3.
L is NP-Complete if L NP and for all other L'  NP,
L'  L
•
L
NP-Completeness
•
If a problem in NP-complete can be solved in polynomial time
then all problems in NP can be solved in polynomial time.
If a problem in NP cannot be solved in polynomial time then
all problems in NP-complete cannot be solved in polynomial
time.
NP
Young
CS 530 Ad. Algo. D&A
Note that an NP-complete problem is
one of those hardest problems in NP.
NP-Completeness
12
6
•
•
So, if an NP-complete problem is in P then P=NP
if P != NP then all NP-complete problems are in
NP-P
Question : how can we obtain the first NP-complete
problem L?
4. Cook Theorem : SATISFIABILITY is NPComplete. (The first NP-Complete problem)
Instance : Given a set of variables, U, and a
collection of clauses, C, over U.
Question : Is there a truth assignment for U
that satisfies all clauses in C?
Young
CS 530 Ad. Algo. D&A
NP-Completeness
13
Example :
U = {x1, x2}
C1 = {(x1, ¬ x2), (¬ x1, x2)}
= (x1 OR ¬ x2) AND (¬ x1 OR x2)
if x1 = x2 = True  C1 = True
C2 = (x1, x2) (x1, ¬ x2) (¬ x1)  not satisfiable
“¬ xi ” = “not xi ” “OR” = “logical or ” “AND” = “logical and ”
This problem is also called “CNF-Satisfiability” since the expression
is in CNF – Conjunctive Normal Form (the product of sums).
Young
CS 530 Ad. Algo. D&A
NP-Completeness
14
7
• With the Cook Theorem, we have the
following property :
Lemma : If L1 and L2 belong to NP, L1 is
NP-complete, and L1  L2 then L2 is NPcomplete.
i.e. L1, L2 NP and for all other L'  NP,
L'  L1 and L1  L2  L'  L2
Young
CS 530 Ad. Algo. D&A
NP-Completeness
15
• So now, to prove a problem L to be NPcomplete problem , we need to
• show L is in NP
• select a known NP-complete problem L'
• construct a polynomial time transformation f
from L' to L
• prove the correctness of f (i.e. L’ has a solution
if and only if L has a solution) and that f is a
polynomial transformation
Young
CS 530 Ad. Algo. D&A
NP-Completeness
16
8
• P: Problems solvable by deterministic algorithms in
polynomial time
• NP: Problems solved by non-deterministic algorithms
in polynomial time
• A group of problems, including
NP-Complete
all of the ones we have discussed
(Satisfiability, 0/1 Knapsack,
Longest Path, Partition) have an
NP
additional important property:
If any of them can be solved
P
in polynomial time, then
they all can!
• These problems are called NP-complete problems.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
17
• Some NP-complete problems :
•
•
•
•
SATISFIABILITY
0/1 Knapsack
PARTITION
Two-Processor Non-Preemptive Schedule
Length
• CLIQUE : An undirected graph G=(V, E) and
a positive integer J  |V|
Question : Does G contain a clique (complete
subgraph) of size J or more?
Young
CS 530 Ad. Algo. D&A
NP-Completeness
18
9
Proving NP-Completeness
Results
• Example 1 : Show that the PARTITION problem
is NP-complete.
Given a known NPC problem - Sum of Subset
Problem (SS), show that PARTITION problem is
NPC.
SS Problem
Instance : Let A = {a1, a2, …, an} be a set of n
positive numbers.
Question : Given M, is there a subset A'  A such
that A' = M
Young
CS 530 Ad. Algo. D&A
NP-Completeness
19
PARTITION Problem
Instance : Given a finite set of m positive
integers Z.
Question : Is there a subset Z'  Z such that
 Z' =  (Z-Z')
• PARTITION is in NP
guess a subset Z'
verify  Z' =  (Z-Z')?
Young
CS 530 Ad. Algo. D&A
NP-Completeness
O(m)
O(m)
Total O(m)
20
10
• SS  PARTITION
Given an arbitrary instance of SS, i.e. A = {a1, a2, …, an} and
M,
Construct an instance of PARTITION as follows :
B={b1, b2, …, bn, bn+1, bn+2} of m = n+2 positive numbers
where
bi
bn+1
= ai
for 1  i  n
=M+1
bn+2
= A + (1 – M)
Note :  bi = 2 A + 2. Also, the transformation can be
done in polynomial time (based on input size of A & M)
Young
CS 530 Ad. Algo. D&A
NP-Completeness
21
To show the transformation is correct : The SS
problem has a solution if and only if the
PARTITION problem has a solution.
 If SS problem has a solution, then the
PARTITION problem has a solution
assume A' is the solution for SS problem then
Z' = A'  {bn+2} and Z-Z' =A-A'  {bn+1}
 Z' = M + A + (1 – M) = A + 1 =  (Z-Z')
Young
CS 530 Ad. Algo. D&A
NP-Completeness
22
11
 If the PARTITION problem has a solution then
the SS problem has a solution
if Z' is the solution then  Z' = A + 1
 exactly one of bn+2 or bn+1  Z'
 if bn+2  Z' then Z' – { bn+2 } = A' and  A' = M
if bn+1  Z', then use Z - Z' to obtain A'
Young
CS 530 Ad. Algo. D&A
NP-Completeness
23
• Example 2 : Show that the Traveling
Salesman (TS) Problem is NP-complete.
Given a known NPC problem - Hamiltonian
Circuit (HC), show that TS problem is NPC.
Hamiltonian Circuit (HC) problem
Instance : Give an undirected graph G=(V, E)
Question : Does G contain a Hamiltonian
circuit, i.e. a sequence < v1, v2, …, vn > of all
vertices in V which is a simple cycle.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
24
12
Traveling Salesman (TS) Problem
Instance : Give an undirected complete graph
G=(V, E) with distance d(i,j)  0 for each edge (i,j)
for i  j and a positive integer B.
Question : Is there a tour of all cities (a simple
cycle with all vertices) having total distance no
more than B.
• TS is in NP
guess a tour, i.e. sequence of all vertices
verify that it is a cycle covering all
vertices and total distance  B
Young
CS 530 Ad. Algo. D&A
NP-Completeness
O(|V|)
O(|V|)
25
• HC  TS
Given arbitrary instance of HC, i.e. G=(V, E).
Construct an instance of TS as follows :
G’ = (V , E’ ), where (u,v)  E’
for all u, v  V and u  v
d(u,v) = 0 if (u,v)  E
d(u,v) = 1 if (u,v)  E
and B = 0
Note : The transformation can be done in polynomial
time (based on input size of V and E)
Young
CS 530 Ad. Algo. D&A
NP-Completeness
26
13
To show the transformation is correct : The HC
problem has a solution if and only if the TS problem
has a solution.
 If HC problem has a solution, then TS problem has a
solution
Assume a < v1, v2, …, vn > is the solution for HC
It is a simple cycle which contains all vertices
Each edge (u,v) in this cycle has d(u,v) = 0
Total distance is 0
Solution for TS




 If TS problem has a solution then HC problem has a
solution
Obvious to see.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
27
• Example 3 : Show that the Vertex Cover (VC)
Problem is NP-complete.
Given 3SAT problem is NPC, show that VC
problem is NPC.
3SAT Problem
Instance : Given a set of variables U = {u1, u2, …,
un} and a collection of clauses C = {c1, c2, …, cm}
over U such that | ci | = 3 for 1  i  m.
Question : Is there a truth assignment for U that
satisfies all clauses in C?
Note : 3SAT problem is a restricted problem of
SATISFIABILITY problem.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
28
14
Vertex Cover (VC) Problem
Instance : Given an undirected graph G=(V, E)
and a positive integer K  |V|
Question : Is there a vertex cover of size K or
less for G, i.e. a subset V’  V such that |V’| 
K and, for each (u,v)  E, at least one of u or
v  V’.
• VC is in NP
guess a set of vertices V’  V
verify that |V’|  K and,
for each (u,v)  E, u  V’ or v  V’
Young
CS 530 Ad. Algo. D&A
NP-Completeness
O(|V|)
O(|V|+|E|)
29
• 3SAT  VC
Given arbitrary instance of 3SAT, i.e. U = {u1, u2,
…, un} and C = {c1, c2, …, cm}. Construct an
instance of VC as follows :
 G = (V , E ) and K = n+2m
 V = Vu  Vc
Vu = {u1t, u1f, u2t, u2f, …, unt, unf} and
Vc = {a11, a12, a13}  {a21, a22, a23}  …  {am1, am2, am3}
Young
CS 530 Ad. Algo. D&A
NP-Completeness
30
15
 E = Eu  Ec  Euc
Eu = {(u1t, u1f) , (u2t, u2f) , …, (unt, unf)}
Ec = {(a11, a12) , (a12, a13), (a13, a11)}  …  {(am1, am2)
, (am2, am3), (am3, am1)}
Assume ci = (xi, yi, zi) for 1  i  m,
find the corresponding vertices, xi, yi, zi, in Vu
Euc = {(x1, a11), (y1, a12), (z1, a13)}  …  {(xm, am1),
(ym, am2), (zm, am3)}
Young
CS 530 Ad. Algo. D&A
NP-Completeness
31
 |V| = 2n+3m and |E| = n+3m+3m
 The transformation can be done in polynomial time
(based on input size n and m)
Eu
Example : U = {u1, u2, u3 , u4} and C = {{u1, ¬ u3, ¬ u4},
{u1, u2, ¬ u4}}
u1t u1f
u2t
u2f
u3t u3f
u4t u4f
Euc
E
Ec
a22
a12
a11
Young
CS 530 Ad. Algo. D&A
a13
a21
NP-Completeness
a23
32
16
Major Property : if there is a vertex cover
set V’  K = n+2m, then
a) |V’| = n+ 2m, and
b) V’ must include exactly 1 vertex in {uit, uif}
for 1  i  n from Vu and at exactly 2 vertices
in {ai1, ai2, ai3} for 1  i  m from Vc,
i.e. n vertices from Vu and 2m vertices from Vc
 Look at edges in Eu and Ec, a vertex cover set V’ must
include
at least 1 vertex {uit, uif} for 1  i  n, and
at least 2 vertices from {ai1, ai2, ai3} for 1  i  m.
Since |V’|  K  |V’| = K
Young
CS 530 Ad. Algo. D&A
NP-Completeness
33
To show the transformation is correct : The 3SAT
problem has a solution if and only if the VC problem
has a solution.
 If VC problem has a solution then 3SAT problem has a
solution
From the above property, V’ contains n vertices from
Vu and 2m vertices from Vc
From Vu , the truth assignment for {u1, u2, …, un} in
3SAT is
ui = T if uit  V’
ui = F if uif  V’ for 1  i  n
Young
CS 530 Ad. Algo. D&A
NP-Completeness
34
17
To see that this is a solution for 3SAT :
we must show that for each ci = (xi, yi, zi), there
is at least one variable i {xi, yi, zi} which set
ci to TRUE, 1  i  n
From the above property, exactly 2 vertices from {ai1,
ai2, ai3} in V’, for 1  i  m
only cover 2 edges from {(xi, ai1), (yi, ai2),
(zi, ai3)} from Euc
assume the edge (xi, ai1) is not covered by 2
vertices from {ai1, ai2, ci3} then xi  V’ since V’ is
a vertex cover set
xi set the clause ci to True for 1  i  n
Young
CS 530 Ad. Algo. D&A
NP-Completeness
35
 If the 3SAT problem has a solution, then the VC
problem has a solution
The vertex cover set V’ with exactly n+2m vertices can be obtained as
follows :
From the truth assignment for {u1, u2, …, un} in 3SAT, we get n
vertices from Vu,
i.e. uit  V’ if ui = T; otherwise uif  V’ for 1  i  n
This covers all edges in Eu and at least one edge in {(xi, ai1), (yi, ai2),
(zi, ai3)} for 1  i  m
From Vc, include 2 vertices from each {ai1, ai2, ai3} into V’, for 1  i
 m. These 2 vertices cover all edges {(ai1, ai2) , (ai2, ai3), (ai3, ai1)}
and also cover edges in Euc that are not covered previously.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
36
18
• Example 4 : Show that the Square Packing (SP)
Problem is NP-complete.
Motivation : truck loading, processor allocation problem in a partitionable
mesh connected system, the design of VLSI chips and etc.
Square Packing Problem
Given a packing square S and a set of packed
squares L = {s1, s2, ..., sn}.
Question : is there and orthogonal packing of L into
S?
Note : orthogonal packing  the sides of squares are
parallel to the vertical and horizontal axes
Young
CS 530 Ad. Algo. D&A
NP-Completeness
37
3-Partition Problem
Given a list A = {a1, a2, ..., a3z} of 3z positive integers
such that sum of all numbers is zB and B/4 < ai < B/2
for each 1  i  3z.
Question : Can A be partitioned into z groups such
that the sum of all numbers in each group is B.
Note : each group must have exactly 3 numbers
Proof : Refer to the following research paper
Leung, Tam, Wong, Young and Chin, “Packing Squares into a Square”,
Journal of Parallel and Distributed Computing, 1990.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
38
19
Coping with NP-Completeness Problem
• NP-hard Problem
Note : Refer to Chapter 5 of Garey and
Johnson
If L' L and L' is an NP-complete problem
then L is called NP-hard problem.
All NPC problems are NP-hard.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
39
There are some NP-hard decision problems that are
not in NP.
Example : Kth Largest Subset Problem is not in NP
Instance : Given a set of positive integers A = {a1,
a2, …, an}, and two non-negative numbers, B  A
and K  2|A|.
Question : Are there at least K distinct subsets A’ 
A such that each subset has total sum >= B.
Note : PARTITION problem  Kth Largest Subset
Problem
Young
CS 530 Ad. Algo. D&A
NP-Completeness
40
20
• Pseudo-polynomial time algorithm
Note : Refer to Chapter 4 of Garey and
Johnson
Some NP-complete problem may be solved
in "polynomial" time (based on input size
and magnitude).
Example : PARTITION problem
Young
CS 530 Ad. Algo. D&A
NP-Completeness
41
Dynamic Programming Algorithm :
•
•
•
•
assume B = (sum of n integers)/2
construct a table of size (approx.) n x B
fill in the table row by row
for each row, add a new element
mark sum of all possible subsets
if there is a subset with sum = B, stop.
Time Complexity : O(nB)
Young
CS 530 Ad. Algo. D&A
NP-Completeness
42
21
• NP-completeness in the strong sense
Note : Refer to Chapter 4 of Garey and Johnson
If L is NP-complete problem in the strong sense,
then L cannot be solved by a
pseudo-polynomial time algorithm unless P=NP
L is NP-complete in the strong sense if L'  L , L'
is NP-complete in the
strong sense and L is in NP
Example : 3-partition problem is NPC in the
strong sense.
Young
CS 530 Ad. Algo. D&A
NP-Completeness
43
• Solving more restricted problems
If we restricted the problem L to problem L'
i.e. L' is a special/restricted case of L,
then we may solve L' in polynomial time.
For example : PARTITION problem
if we assume that each input integer ai  n, where
n is the number of input integers, then the pseudo
polynomial time algorithm becomes polynomial
time algorithm, i.e. O(n3)
Young
CS 530 Ad. Algo. D&A
NP-Completeness
44
22