Ch 10 - NP-completeness
•
•
•
•
•
•
Tractable and intractable problems
Decision/Optimization problems
Deterministic/NonDeterministic algorithms
Classes P and NP
Polynomial reductions
NP-complete problems
– Cook-Levin theorem
– CLIQUE (via SAT  CLIQUE reduction)
– TSP (via HC  TSP reduction)
1
Classifying problems
• Classify problems as tractable or
intractable.
• Problem is tractable if there exists at least
one polynomial bound algorithm to solve it.
• An algorithm is polynomial bound if its worst
case growth rate can be bound by a
polynomial p(n) in the size n of the problem
p(n)  an n  ...  a1n  a0 where k is a constant
k
2
Intractable problems
• Problem is intractable if it is not tractable.
• All algorithms that solve such a problem
are not polynomial bound.
• It has a worst case growth rate f(n) which
cannot be bound by a polynomial p(n) in
the size n of the problem.
• For intractable problems the bounds are:
f (n)  c , or n
n
log n
, etc.
3
Why is this classification useful?
• If problem is intractable, no point in trying
to find an efficient algorithm
• All algorithms will be too slow for large
inputs.
4
Hard practical problems
• There are many practical problems for
which no one has yet found a polynomial
bound algorithm.
• Examples: traveling salesperson, 0/1
knapsack, graph coloring, bin packing etc.
• Most design automation problems such as
testing and routing.
• Many networks, database and graph
problems.
5
The theory of NP completeness
• The theory of NP-completeness enables
showing that these problems are at least
as hard as NP-complete problems
• Practical implication of knowing problem is
NP-complete is that it is probably
intractable ( whether it is or not has not
been proved yet)
• So any algorithm that solves it will
probably be very slow for large inputs
6
Decision Problems vs. Optimization
Problems
• Decision Problem: Yes/no answer
• Optimization Problem: Maximization or
minimization of a certain quantity
• When studying NP-Completeness, it is
easier to deal with decision problems than
optimization problems
7
Element Uniqueness Problem
• Decision Problem: Element Uniqueness
– Input: A sequence of integers S
– Question: Are there two elements in S that are
equal?
• Optimization Problem: Element Count
– Input: A sequence of integers S
– Output: An element in S of highest frequency
• What is the algorithm to solve this problem? How
much does it cost?
8
Coloring a Graph
• Decision Problem: Coloring
– Input: G=(V,E) undirected graph and k, k > 0.
– Question: Is G k-colorable?
• Optimization Problem: Chromatic Number
– Input: G=(V,E) undirected graph
– Output: The chromatic number of G,(G)
• i.e. the minimum number (G) of colors
needed to color a graph in such a way that no
two adjacent vertices have the same color.
9
Traveling Salesman
• Given a finite set C={c1,...,cm} of cities, a
distance function d(ci, cj) of nonnegative numbers
and a bound B
• Decision Problem
– Is there a tour of all the cities (in which each
city is visited exactly once) with total length at
most B?
• Optimization Problem
– Find the length of the minimum distance tour
which includes every city exactly once
10
Cliques
• Definition: A clique of size k in G, for some
+ve integer k, is a complete subgraph of G
with k vertices.
• Decision Problem
– Input:
– Question:
• Optimization Problem
– Input:
– Output:
11
The relation between
• If we have a solution to the optimization
problem we can compare the solution to
the bound and answer “yes” or “no”.
• Therefore if the optimization problem is
tractable so is the decision problem
• If the decision problem is “hard” the
optimization problem is also “hard”
– If the optimization was easy then the
decision problem is easy.
12
Deterministic Algorithms
• Definition: Let A be an algorithm to solve
problem. A is called deterministic if,
when presented with an instance of the
problem , it has only one choice in each
step throughout its execution.
– If we run A again and again, is there a
possibility that the output may change?
• What type of algorithms did we have so
far?
13
The Class P
• Definition: The class of decision problems
P consists of those whose yes/no solution
can be obtained using a deterministic
algorithm that runs in polynomial time of
steps, i.e. O(nk), where k is a non-negative
integer and n is the input size.
14
Examples
• Sorting: Given n integers, are they sorted
in non-decreasing order?
• Set Disjointness: Given two sets of
integers, are they disjoint?
• Shortest path:
• 2-coloring:
– Theorem: A graph G is 2-colorable if and only
if G is bipartite if and only if G has no odd
length cycle
15
Non-Deterministic Algorithms
• A non-deterministic algorithm A on input x consists
of two phases:
– Guessing: An arbitrary “string of characters y” is
(called certificate) generated.
• It may or may not correspond to a solution
• Not be in proper format of a solution
• Differ from one run to another
– Verification: A deterministic algorithm verifies
• The generated “string of characters y” is a
solution
16
Non-Deterministic Algorithms
• Definition: Let A be a nondeterministic
algorithm for a problem . We say that A
accepts an instance I of  if and only if on
input I, there exists a guess that leads to a
yes answer.
– Does it mean that if an algorithm A on a given
input I leads to an answer of no for a certain
guess, that it does not accept it?
17
The Class NP
• Definition: The class of decision problems NP
consists of those decision problems for which
there exists a nondeterministic algorithm that
runs in polynomial time.
– The running time of a nondeterministic
algorithm, is the sum of the running times of
the guessing and verification phases.
– Both phases need run in polynomial time.
– In particular, the verification phase.
18
Example - TSP
• Show that the traveling salesman problem
belongs to the class of NP problems
• Given a problem instance x for TSP and a
certificate y
– Check that y is indeed a cycle that includes
every vertex exactly once
– Verify that the length of the cycle is at most B
• Is the verification algorithm polynomial?
19
P and NP Problems
• What is the difference between P problems and
NP Problems?
• We can solve problems in P using deterministic
algorithms that run in polynomial time
• We can check or verify the solution of NP
problems in polynomial time using a
deterministic algorithm
• What is the set relationship between the classes
P and NP?
20
Summary: P and NP
• Summary so far:
– P = problems that can be solved in poly time
– NP = problems for which a solution can be verified
in polynomial time
– P  NP
– Unknown whether P = NP (most suspect not)
• We’ve seen problems that belong to NP that may not
belong to P
– Hamiltonian path/cycle, TSP problems are in NP
– Cannot solve in polynomial time
– Easy to verify solution in polynomial time
21
NP-Complete Problems
• We will see that NP-Complete problems are the
“hardest” problems in NP:
– If any one NP-Complete problem can be
solved in polynomial time…
– …then every NP-Complete problem can be
solved in polynomial time…
– …and in fact every problem in NP can be
solved in polynomial time (showing P = NP)
– Thus: solve hamiltonian-cycle in O(n100) time,
you’ve proved that P = NP. Retire rich &
famous.
22
Reduction
• The crux of NP-Completeness is reducibility
– Informally, a problem A can be reduced to
another problem B if any instance of A can be
“easily rephrased” as an instance of B , the
solution to which provides a solution to the
instance of A
• This rephrasing is called transformation
– Intuitively: If A reduces to B, A is “no harder to
solve” than B
• Total cost: cost of transformation + cost to solve B
23
Reduction: A reduces to B
A solver
IA
Transformer
IB
B solver
OB
OA
• An example:
– A: Given a set of Booleans, is at least one TRUE?
– B: Given a set of integers, is their sum positive?
– Transformation: (x1, x2, …, xn) = (y1, y2, …, yn)
where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE
24
NP-Hard and NP-Complete
• If A is polynomial-time reducible to B, we denote
this A p B
• Definition of NP-Hard and NP-Complete:
– If all problems X  NP are reducible to A, then
A is NP-Hard
– We say A is NP-Complete if A is NP-Hard
and A  NP
• If A p B and A is NP-Complete, B is also
NP-Complete
25
Why Prove NP-Completeness?
• Though nobody has proven that P ≠ NP, if you
prove a problem NP-Complete, most people
accept that it is probably intractable
• Therefore it can be important to prove that a
problem is NP-Complete
– Don’t need to come up with an efficient
algorithm
– Can instead work on approximation
algorithms
26
Proving NP-Completeness
• What steps do we have to take to prove a
problem A is NP-Complete?
– Pick a known NP-Complete problem B
– Reduce B to A
• Describe a transformation that maps
instances of B to instances of A, s.t. “yes”
for A = “yes” for B
• Prove the transformation works
• Prove it runs in polynomial time
– Oh yeah, prove A  NP
27
Examples of NP-Complete problems
• Cook’s theorem
– Satisfiability is NP-complete
• This was the first problem shown to be NPcomplete
• Other problems
– Graph coloring (= register allocation)
– Hamiltonian path
– Traveling salesman
– Knapsack
28
Conjunctive Normal Form (CNF)
• A logical (Boolean) variable is a variable
that may be assigned the value true or
false (p, q, r and s are Boolean variables)
• A literal is a logical variable or the negation
of a logical variable (p and q are literals)
• A clause is a disjunction of literals
( (pqs) and (q  r) are clauses)
29
Conjunctive Normal Form (CNF)
• A logical (Boolean) expression is in
Conjunctive Normal Form if it is a
conjunction of clauses.
• The following expression is in conjunctive
normal form:
(pqs) (q  r) (p  r) (r  s)
(psq)
30
The Satisfiability problem
• Is there a truth assignment to the n variables of
a logical expression in Conjunctive Normal Form
which makes the value of the expression true?
– For the answer to be “yes”, all clauses must
evaluate to true
– Otherwise the answer is “no”
• p=T, q=F, r=T and s=T is a truth assignment for:
(pqs) (q  r) (p  r) (r  s)
(psq)
31
The Satisfiability problem
• Theorem: Satisfiability is an NP-Complete
problem.
• Proof:
– Satisfiability  NP: Given a truth assignment
of a Boolean formula consisting of n literals, it
is clear that we can evaluate it in poly time.
– Satisfiability  NP-Hard: The proof involves
reducing every problem in NP to Satisfiability
in polynomial time. See Garey & Johnson.
32
Proving NP-Completeness
SAT
3-CNF-SAT
Clique
Hamiltonian Cycle
Vertex-Cover
Traveling Salesman
Subset-Sum
33
NP completeness of CLIQUE
• SAT reduces to CLIQUE
– Given any input to SAT, we create a
corresponding input to CLIQUE that will help
us solve the original SAT problem
– Specifically, for a SAT formula with K clauses,
we construct a CLIQUE input that has a clique
of size K if and only if the original Boolean
formula is satisfiable
34
NP completeness of CLIQUE
• SAT reduces to CLIQUE
– Associate a person to each variable
occurrence in each clause
35
NP completeness of CLIQUE
• SAT reduces to CLIQUE
– Associate a person to each
variable occurrence in each clause
– ”Two people” know each other
except if:
• they come from the same clause
• they represent t and t’ for some
variable t
36
NP completeness of CLIQUE
• SAT reduces to CLIQUE
– Two people know each other except if:
• they come from the same clause
• they represent t and t’ for some variable t
– Clique of size 4  satisfiable
assignment
• set variable in clique to ”true”
• (x, y, z) = (true, true, false)
37
NP completeness of CLIQUE
• SAT reduces to CLIQUE
– Two people know each other except if:
• they come from the same clause
• they represent t and t’ for some variable t
– Clique of size 4  satisfiable assignment
– Satisfiable assignment  clique of size 4
• (x, y, z) = (false, false, true)
• choose one true literal from
each clause
38
CLIQUE is NP-complete
• CLIQUE is NP-complete
– CLIQUE is in NP
– SAT is in NP-complete
– SAT reduces to CLIQUE
• Hundreds of problems can be shown to be
NP-complete that way…
39
NP completeness of CLIQUE
• Show that the traveling salesman problem is
NP-complete, assuming that the Hamiltonian
cycle problem is NP-complete.
– Prove that TSP  NP (already done)
• Guess: a sequence of cities
• Verify: that it is a tour of cost less than B
– Reduce the undirected Hamiltonian cycle
problem to the TSP
• So if we had a TSP-solver, we could use it
to solve the Hamilitonian cycle problem in
polynomial time
40
HamCycle p TSP?
• Transform input for HamCycle into input
for TSP
– HamCycle: Given unweighted graph G1,
does it have a Hamiltonian cycle?
– TSP: Given weighted graph G2 and B, is
there a Hamiltonian cycle with total cost
less than B?
• Must convert unweighted graph to
weighted
41
HamCycle p TSP?
• Convert unweighted graph to weighted
– Add edges to G1 to make a complete graph
G2
– Add weights as follows:
• wt(i,j) is 0 if edge i,j is in G1 (original graph
for HamCycle)
• wt(i,j) is 1 if edge i,j is not in G1
• G1 has ham. cycle iff G2 has TSP with B=0
– Can you see why?
– Is this transformation polynomial?
42
The Main Question
• Is P = NP?
– Is the original DECISION problem as easy as
VERIFICATION?
• Most important open problem in theoretical
computer science.
43
The Big Picture
• Summarizing: it is not known whether NP
problems are tractable or intractable
• But, there exist provably intractable problems
– Even worse – there exist problems with
running times unimaginably worse than
exponential!
• More bad news: there are provably noncomputable (undecidable) problems
– There are no (and there will not ever be!)
algorithms to solve these problems
44