Algorithms
NP-Complete Problems
CS 8833
Algorithms
Polynomial time algorithms
Most algorithms we have studied so far
are polynomial time in the size of their
inputs n.
 Worst case running time is O(nk) for
some constant k.
 Problems that are solvable by
polynomial time algorithms are said to
be tractable.

CS 8833
Algorithms
What’s so great about
polynomial time?
1. Although we would consider a problem that is
(n100) intractable, this type of problem is
rarely encountered. The exponents on the
polynomial are usually small.
2. For most reasonable models of computation,
a problem that can be solved in polynomial
time in one model can be solved in
polynomial time in another.
CS 8833
Algorithms
Closure properties of
polynomial algorithms

Closed under addition
f(n) + g(n)

Closed under multiplication
f(n)*g(n)

Closed under composition
f(g(n))
CS 8833
Algorithms
Why is polynomial time an important
goal?



An algorithm is efficient (or polynomial time)
if there exists a polynomial p(n) such that the
algorithm can solve any instance of size n in
time O(p(n)).
A problem is tractable if it has an efficient
algorithm.
This is valid only if we avoid algorithms :
– with extremely large constants
– with large values for k
O(nk)
– with very large operands.
CS 8833
Algorithms
Do all problems have polynomial
time solutions?

NO!
Some problems are not solvable
– Turing’s Halting problem is an example
– Cannot be solved by any computer no matter how
much time is given


Some problems can be shown to require
super-polynomial time to solve.
There is a a large class of important problems
for which we do not know the answer. These
are the NP complete problems.
CS 8833
Algorithms
Some Classifications
Unsolvable–Turing problem
 Solvable

– Provably intractable (infeasible)
– Probably intractable (NP-complete)
– Tractable (feasible)
» hard (> n2)
» easy (most of the problems we have studied so
far)
CS 8833
Algorithms
NP Completeness

The Question!!
– Does P = NP?

This is an unanswered but very
important question.
– What are P, NP, and NP-complete?
– Why is the question important?
– What do we think the answer is?
– What if the answer is what we think it is?
CS 8833
Algorithms
NP Complete Problems
The complexity status of these
problems is unknown
 There is no established lower bound
 But, it is known that if one of these
problems can be solved in polynomial
time, all of them can.
 Very important equivalence class

CS 8833
Algorithms
What Do We Think?


Most computer scientists believe that the NP
complete problems are intractable.
Why?
– Because there are a lot of these problems
– Many of them are very important problems
– People have been trying to find efficient algorithms
for them for a long time and no one has
succeeded.
CS 8833
Algorithms
Many P and NP-Complete
Problems are Closely Related
Shortest path problem and longest path
problem
 Euler tour and Hamiltonian Cycle

– Euler tour of connected, directed graph
visits each edge exactly once
– Hamiltonian cycle begins and ends at
same vertex and visits each vertex exactly
once
CS 8833
Algorithms
2-SAT and 3-SAT
2-Conjunctive Normal Form (CNF)
Satisfiability
 3-Conjunctive Normal Form (CNF)
Satisfiability

CS 8833
Algorithms
The Satisfiability Problem
Given a Boolean formula containing
variables whose value are 0 or 1,
connected by the Boolean connectives
, , and .
 A Boolean formula is satisfiable if there
is an assignment of values to variables
that causes the formula to evaluate to 1

CS 8833
Algorithms
k-CNF
Informal definition: A formula is in kconjunctive normal form if it is the AND
of clauses of ORs of exactly k variables
or their negations.
 Example of 2-CNF formula

(x1  x2)  (x1  x3)  (x2  x3)

What is a satisfying assignment of
values to variables?
CS 8833
Algorithms
CNF-Satisfiability
A polynomial time algorithm exists for to
determine if a 2-CNF formula is
satisfiable.
 3-CNF is NP-complete

CS 8833
Algorithms
Three Classes of Problems

P
(Polynomial)
– Problems solvable in polynomial time
– Can be solved in time O(nk) where k is a constant

NP (Non-deterministic Polynomial)
– Problems that are verifiable in polynomial time
– Given a “certificate” of a solution, we can verify that the
solution is correct in polynomial time
» For Hamiltonian path the certificate is a sequence of vertices
» For 3-SAT, the certificate is an assignment of values to
variables

NPC (NP Complete)
CS 8833
Algorithms
NP Complete
Informal definition: A problem is in the
class NPC if it is in NP and is as “hard”
as any problem in NP.
 If any problem in the class NP can be
solved in polynomial time, then all can
be solved in polynomial time

CS 8833
Algorithms
Proofs of NP-Completeness
Different from other proofs we have
done
 Rather than show that a problem has an
efficient algorithm, we will be
demonstrating that it is hard.

CS 8833
Algorithms
Theory of NP-Completeness

Limited to one type of problem decision problems
– Each element of S (the set of solutions) is an
element of {yes,no} (or {0,1})
– Seems like a severe restriction, but it is not
– Usually dealing with optimization problems that
are easily restated as a decision problem.
– Can show that if the decision problem is easy,
then the optimization problem is easy.
CS 8833
Algorithms


Optimization Problem vs Decision Problem
Shortest path problem as an optimization
problem
Given a graph G= (V,E), two vertices u,v  V,
what is the shortest path that exists in G
between u and v?
Shortest path problem as a decision problem
Given a graph G= (V,E), two vertices u,v  V
and a nonnegative integer k, does a path
exist in G between u and v whose length is
at most k?
CS 8833
Algorithms
General Approach
To change an optimization problem to a
decision problem, we can usually just
impose a bound on the value to be
optimized.
– minimization problem example:
» length of shortest path at most k
– maximization problem example:
» length of longest path at least k
CS 8833
Algorithms
General statements
Usually, if we can solve the optimization
problem quickly, we can solve the
decision problem quickly.
 Also, if we can provide evidence that
the decision problem is hard, we also
provide evidence that the optimization
problem is hard.

CS 8833
Algorithms
The Class P

The complexity class P is the set of
concrete decision problems that are
solvable in polynomial time.
CS 8833
Algorithms
Nondeterministic Polynomial
(NP)
Let an oracle guess an answer to the
problem (certificate).
 If we can find an algorithm that can
verify that the answer is correct in
polynomial time, then this problem is in
the class NP
 P is clearly a subset of NP

CS 8833
Algorithms
P = NP?
NP
NP-complete
NP
NP
NP-complete
P
P
CS 8833
Algorithms
Example: HAM-CYCLE
Hamiltonian Cycle Problem
– A Hamiltonian cycle of an undirected graph
G=(V,E) is a simple cycle that contains
each vertex in V.
– The Hamiltonian-cycle problem asks if a
given graph contains a Hamiltonian cycle.
CS 8833
Algorithms
HAM-CYCLE NP

HAM-CYCLE is an element of NP
because we can verify an answer
– Given a list of vertices, does the list
represent a Hamiltonian cycle?
– We can find a polynomial time algorithm to
determine if this list is a Hamiltonian cycle.
» Is the first vertex the same as the last vertex?
» Are all vertices visited?
» For each vertex in the list, is there an edge to
the next vertex in the list?
» Is any vertex other than the first repeated
CS 8833 Algorithms
Showing Problems to be NPComplete
Restate problem as a decision problem
 Demonstrate that the decision problem
is in the class NP
 Show that a problem known to be NPComplete can be reduced to the current
problem in polynomial time.

CS 8833
Algorithms
NP-Completeness and
Reducibility

NP-complete problems
– informally they are the hardest problems in
the class NP
– no polynomial time algorithm is known to
solve any of them
– but every NP-complete problem is
reducible to every other NP-complete
problem in polynomial time
CS 8833
Algorithms
Reducibility
Set of instances of
Problem A
Polynomial
time
Set of instances of
Problem B
Decision problem A is polynomial-time reducible to
decision problem B if a polynomial time algorithm can be
developed which changes each instance of problem A to
an instance of problem B such that if the answer for an
instance of B is yes, the answer to the corresponding
instance of A is yes.
CS 8833
Algorithms
Algorithm from Reduction for
Solving Problem A

Polynomialtime reduction
algorithm

Polynomialtime algorithm
to decide B
YES
NO
A First NP-Complete Problem
This technique for showing that a
problem is in the class NP-Complete
requires that we have one NP-Complete
problem to begin with
 Circuit satisifiability was the first
problem to be shown to be in NPComplete.

CS 8833
Algorithms
Lemma 34.3
If Q1, Q2 are problems such that
Q1 p Q2, then Q2  P implies Q1 P.
 Proof:

Suppose A1 is a polynomial time algorithm
that reduces instances of Q1to instances of
Q2 and that A2 is a polynomial time
algorithm for problem Q2. Then the
concatenation of A1 and A2 is a polynomial
time algorithm for Q1.
CS 8833
Algorithms
An Instance of
Problem Q1
Reduction
(polynomial)
Polynomial
algorithm for
Q1
An Instance of
Problem Q2
Polynomial
algorithm for
Q2
Solution for
any Instance
(YES,NO)
NP-Complete

A problem that satisfies property 2, but
not necessarily property 1 is NP-hard.
CS 8833
Algorithms
Theorem 34.4
If any NP-complete problem is
polynomial-time solvable, then P = NP.
 If any problem in NP is not polynomialtime solvable, then all NP-complete
problems are not polynomial-time
solvable.

CS 8833
Algorithms
Proof of Theorem 34.4
Proof of Statement 1: If any NP-complete problem is
polynomial-time solvable, then P = NP.
Suppose that Q P and Q NPC.
Q
P
Q
NP
Q’
NPC
Then, for any Q’ NP, we know that Q’ pQ by
property 2 of NP-completeness. By Lemma 34.3, Q’
P.
NPC
P/NP
CS 8833
Algorithms
Proof of Theorem 34.4
Proof of Statement 2: If any problem in NP is not
polynomial-time solvable, then all NP-complete
problems are not polynomial-time solvable.
Suppose that there exists an Q NP such that Q
P. Let Q’ NPC be any NP-complete problem and
suppose that Q’  P.
Q
Q’
NPC
NPC
P
By Lemma 34.3, Q’ pQ and so Q P. Contradiction
CS 8833
Algorithms
Importance of Theorem




The theorem make the focus of the P=NP
question the NP-complete problems
Most theoretical computer scientists believe
that P NP.
But, if a polynomial time algorithm exists for
any NPC problem, then P = NP.
But, no one has been able to do this so far.
CS 8833
Algorithms
NP-Completeness Proofs

Suppose that we can show that one problem
(Q) is in the class NP-complete. Then we can
show that another problem Q’ is in the class
NP-complete by showing that
– Q’ is in NP
– Q can be reduced to Q’ in polynomial time.
» Describe an algorithm for the reduction
» Prove that the answer to Q’ is yes iff the answer to Q is
yes
» Prove that the algorithm for the reduction runs in
polynomial time
CS 8833
Algorithms
Circuit Satisifiability





Cook proved that the CIRCUIT-SAT problem
is NP-complete.
3-SAT was also shown to be NPC by Cook.
It is quite difficult to show that a particular
problem in NP can be reduced to every other
problem in NP.
But once one problem is know to be NPcomplete, the others are easier.
Many other problems have since been proven
to be NP-complete.
CS 8833
Algorithms
Sketch of NP-Completeness
Proof for CIRCUIT-SAT
Very long and difficult in the details.
 Used to provide the basis for most other
proofs of NP-Completeness.
 Problem domain is combinatorial
circuits.

CS 8833
Algorithms
Combinatorial Circuit
Components
CS 8833
Algorithms
Circuit-Satisfiability Problem
Given a boolena combinatorial circuit
composed of AND, OR, and NOT gates, is it
satisfiable?
Demonstrating Circuit
Satisfiability is NP Complete
This proof was done from first
principles.
 It did not depend on the existence of
any other NPC problems.
 A problem Q is NP-Complete if

1. It is an element of the class NP
2. Q’ pQ for every Q’ in NP
CS 8833
Algorithms
Step 1
Prove CIRCUIT-SAT  NP


Develop an algorithm A that can verify a solution in
polynomial time.
Inputs for A
– A standard representation of a combinatorial circuit
– An assignment of input values to the circuit



Algorithm A determines the output of the
combinatorial circuit. Output is 0 or 1.
This can be done in polynomial time (linear for a
clever algorithm.)
Therefore CIRCUIT-SAT NP
CS 8833
Algorithms
Step 2: Reduction


We need to show that every other problem in NP can
be reduced to CIRCUIT-SAT.
The basic argument is:
– A combinatorial circuit can be used to implement a computer
»
»
»
»
Program
Memory
Program counter
Working storage, etc.
– You can show that every problem in NP can be mapped onto
operations that can be represented as a sequences of states
of combinatorial circuits.
– You show that this can be done in polynomial time.
– It is not simple to show all of this, but in the end you can
show that any problem in NP can be reduced to CIRCUITSAT in polynomial time
CS 8833
Algorithms
Implications of CIRCUITSATNPC
CIRCUIT-SAT is the “seed” problem in
NPC
 Once we know that one problem is in
NPC, we can use it to demonstrate that
other problems are in NPC using a
simpler procedure.

CS 8833
Algorithms
CIRCUIT-SAT
SAT
3-SAT
CLIQUE
VERTEXCOVER
HAM-CYCLE
TSP
SUBSET-SUM
EXAMPLE



Suppose someone has proven that the
problem of determining if a graph G has a
Hamiltonian cycle has been shown to be NPcomplete.
And we want to show that the Traveling
Salesman Problem (TSP) is NP-complete.
Two steps
– show that TSP is in the class NP
– show that HAM reduces to TSP in
polynomial time
CS 8833
Algorithms
Traveling Salesman Problem

TSP Optimization Problem
A salesman must visit n cities. He must visit each
city exactly one time and finish with the city at
which he starts. Model possible routes as an
undirected graph with weights on the edges.
c(I,j) is cost of traveling from city I to j

TSP Decision Problem
Change it to a decision problem by making the
problem to find a route of length at least k.
CS 8833
Algorithms
Step 1
Show that TSP is in the class NP.
If we are given a set of n+1 vertices that we claim
constitute a tour of length at most k, then we can
» check to see that each vertex occurs exactly
one time
» check to see that beginning and ending vertices
are the same
» check to see that edges exist between adjacent
vertices in the graph
» see if the sum of the edges is  k
CS 8833
Algorithms
Step 1 continued
We can show that each of these steps
can be done in polynomial time
 Therefore TSP is an element of NP

CS 8833
Algorithms
Step 2
Now we need to show that any HAM problem
can be reduced to a TSP problem.
HAM
Reduction
TSP
Known NPC Problem
CS 8833
Algorithms
Problem we want to
show to be NPC
The Two Problems


HAM
– A Hamiltonian cycle of an undirected graph
G=(V,E) is a simple cycle that contains each
vertex in V.
– The Hamiltonian-cycle problem asks if a given
graph contains a Hamiltonian cycle.
TSP
– A TSP circuit in of a undirected graph with edge
weights visits each vertex exactly one time and
begins and ends at the same vertex.
– The TSP problem asks if a given graph contains a
TSP circuit of length less than k.
CS 8833
Algorithms
The Reduction from A to B
Consider an instance of problem A.
Show how that instance can be
changed to an instance of B.
 Prove that the answer to B is yes iff the
answer to Q is yes
 Prove that the algorithm for the
reduction runs in polynomial time.

CS 8833
Algorithms
The Reduction
Creating an Instance
Now we need to show that any HAM problem
can be reduced to a TSP problem.
– Let G=(V,E) be an instance of HAM
– Construct an instance of TSP as follows:
» Form the complete graph G’=(V,E’)
» Define the cost of the edges as
c(i,j) = 0 if (i,j) is in E, 1 otherwise
» We look for a tour of length 0
CS 8833
Algorithms
0
0
0
1
0
G’
G
TSP
HAM
CS 8833
Algorithms
0
Step 2 continued
Now we must show that graph G has a
Hamiltonian cycle iff G’ has a tour of cost at
most 0.
a)G has Hamiltonian cycle implies G’ has tour of
cost at most n.
» Suppose that graph G has a Hamiltonian cycle h.
» Each edge in h belongs to E and thus there is an
edge from vi to vi+1 with a cost of 0 in G’
» Thus, h is a tour in G’ with a cost of 0.
CS 8833
Algorithms
Step 2 continued
b) G’ has a tour of cost at most 0 implies G
has as Hamiltonian cycle.
» Suppose that G’ has a tour h’ of cost at most
0.
» Since the cost of the edges is 0 or 1, none of
the edges can have a cost of 1if the cost of
the tour is at most 0
» Therefore all edges are in E.
» So h visits each vertex exactly once, contains
only edges in E, and is therefore a
Hamiltonian cycle in G.
CS 8833
Algorithms