CPSC 320: NP-Completeness
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Up to now:
We have been learning “recipes” to design
efficient algorithms.
Every problem we looked at can be solved in
O(n), or O(n log n), or O(n2), or maybe O(n3).
In some sense, every one of these problems was
easy to solve.
Definition: a problem is solvable in polynomial
time if there is an algorithm for it that runs in
O(nk) time for some integer k  0.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Now:
We will look at problems that (we think) are hard
to solve.
It doesn't mean the problems are difficult to
describe.
Only that we don't know of any efficient algorithms
for them.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Examples:
Find a way to schedule n exams in k time-slots
without any exam hardship.
Find the cheapest route for a traveling business
person to visit n cities and come back to his/her
starting point.
Find k students, in a class of size n, that (already)
all know one another.
Divide a group of n people into two teams of equal
total strength to play tug-of-war.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Two types of problems:
Optimization problem: we want to find the solution
s that maximizes or minimizes a function f(s).
Decision problem: we are given a parameter k,
and we need to decide if there is a solution s for
which
f(s) ≤ k [minimization] or
f(s) ≥ k [maximization].
So the answer is True or False.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example: Clique
Definition: Given a graph G = (V, E), a clique of
G is a subset V' of V such that every two vertices
of V' are joined by an edge in G.
Optimization Problem: Given G = (V, E), find the
largest clique of G.
Decision Problem: Given G = (V, E) and an
integer k, does G have a clique with at least k
vertices?
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
The two types of problem are very similar in terms
of complexity:
If we have a solution to the optimization problem,
then it gives us an answer to the decision problem.
If we can solve the decision problem efficiently,
then we can use binary search on k to find the
answer to the optimization problem.
We will look at decision problems only.
Problem 1: Clique (see previous slides)
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Problem 2: Satisfiability (SAT)
Given
A set of boolean variables X1, X2, ..., Xn.
A set of clauses C1, C2, ..., Cm where for each i,
Ci = Yi,1 ν Yi,2 ν ... ν Yi,t
i
where each Yi,j is either Xk or ~Xk.
We want to know if there is a way to set each Xk
to true or false, so that every clause evaluates to
true.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Problem 3: 3-Satisfiability (3SAT)
The same, but every clause contains exactly three
variables (or complements of variables).
Example:
C1: X1 v X3 v ~X4
C2: ~X1 v X2 v X4
C3: X2 v ~X3 v ~X4
C4: ~X1 v ~X2 v ~X3
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Problem 4: Graph k-colourability
Given: a graph G = (V, E) and a positive integer k.
We want to know if it is possible to
colour every vertex with one of k different colors
so that no edge has both its endpoints with the
same color.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
This problem has many practical applications, for
instance exam scheduling:
Each vertex is a course.
An edge means a student is taking both courses.
Each colour represents a time-slot.
An edge with two endpoints of the same colour
means an exam conflict for one or more students!
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Now we can get to the heart of the topic: we look
at two sets:
P: the set of all problems that can be solved in
polynomial time.
NP: the set of all problems such that, if the
answer is true, then there is a “proof” of that fact
that can be verified in polynomial time.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Examples:
For SAT or 3SAT:
The proof is an assignment of true/false values to
the variables.
We can verify that all clauses are satisfied fairly
quickly.
For Graph k-colorability:
The proof is the colour assigned to each vertex.
We can count the different colours and verify there
are at most k of them.
We can verify that every edge has endpoints of
different colours.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Is P = NP ?
Nobody knows
Most people think the answer is No.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
We have
Easy problems.
Problems that we think are hard.
Cook's theorem: if SAT can be solved in polynomial
time, then every problem in NP can be solved in
polynomial time.
A problem that belongs to NP and has this property is
called NP-complete.
Why do we want to know if a problem is NP-complete?
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Suppose your boss asked you to design an
algorithm to solve a problem. You would like to
avoid this:
Illustration removed to avoid copyright issues
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Ideally, you would like to do this:
Illustration removed to avoid copyright issues
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
But since you can't (we still don't know if P = NP),
at least you might be able to do this, and avoid
getting fired:
Illustration removed to avoid copyright issues
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
How do we prove that a problem P is NP-Complete?
First we prove that
easy:
P
belongs to NP. This is (usually)
Show what kind of “proof” for Yes answers can be
checked in polynomial time.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
How do we prove that a problem P is NP-Complete
(continued)?
P
Then we prove that if we can solve
in polynomial
time, then every other problem in NP can be solved in
polynomial time. This is (usually) a bit more involved.
The proof of Cook's theorem is ugly (6 pages in Garey &
Johnson).
Luckily, now that we know that SAT is NP-complete, we
can use a much simpler method.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Polynomial-time reduction:
Pick a known NP-complete problem
P
NPC
.
Give a polynomial-time algorithm that transforms
P
instances of NPC into instances of
same Yes/No answer.
Instance of
PNPC
P
Transformation (poly-time)
with the
Instance of
P
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
If you could solve P efficiently, then you could
solve an instance of PNPC as follows:
Transform it into an instance of
Solve the instance of
answer.
P
P.
and return the same
Since PNPC is NP-complete, this means you could
solve every problem in NP in polynomial time!
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example 2: proving that 3SAT is NP-Complete:
Clearly 3SAT is in NP:
the proof for a Yes answer is the values we need to
give the variables.
We can verify that every variable is assigned
exactly one value, and that every clause evaluates
to true.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example: proving that Graph 3-colorability is NPComplete:
<Done on the board>
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
What do we do if we find the problem we need to
solve is NP-Complete?
Solving it exactly most likely takes exponential
time.
Or we can use an approximation algorithm: an
algorithm for which we have a bound on how bad
the solution is.
So we don't get the optimal solution, but we know
how far off we are from the optimal.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example: Traveling salesperson with the triangle
inequality.
Given:
a graph G = (V, E)
a weight function w: E → R+ such that w(x,z) ≤
w(x,y) + w(y,z) for all vertices x, y, z.
Find a simple cycle that contains every vertex of
G, and whose total weight is minimum.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Traveling salesperson problem with the triangle
inequality (continued):
Algorithm:
Compute a minimum spanning tree of G.
Starting from a vertex v, go around the tree twice.
The second (or more) time a vertex w is visited,
remove it from the tour.
Let W be the weight of the tour produced by this
algorithm.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Traveling salesperson problem with the triangle
inequality (continued):
Let
MST(G) be the weight of the minimum spanning
tree of G.
OPT(G) be the weight of the minimum TSP tour.
Fact 1: MST(G) ≤ OPT(G)
Proof: Take the minimum weight TSP tour. We get
a spanning tree T by removing one edge. So
MST(G) is ≤ the weight of T, which is ≤ OPT(G).
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example: the traveling salesperson problem with
the triangle inequality (continued):
Facts 2: W ≤ 2 MST(G)
Proof: the initial tour (the one that goes around the
tree twice) as weight exactly twice that of MST(G)
(it follows every edge twice). Because of the
triangle inequality the shortcuts can only reduce
the weight of the tour.
Corollary: W ≤ 2 OPT(G).
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example 2: proving that 3SAT is NP-Complete:
Now we show every instance I of SAT can be
transformed into an instance I' of 3SAT. Consider
a clause C.
If C contains exactly three variables, then we copy
C to I'.
If C is Y1 v Y2, then we add two clauses
Y1 v Y2 v Z
Y1 v Y2 v ~Z
to I' where Z is a new variable. Observe that Y1 v
Y2 is true if and only if both these clauses are true
(no matter what value we give Z).
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Appendix
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example: proving that 3SAT is NP-Complete:
Consider a clause C (continued):
If C is Y, then we add four clauses
Y v Z1 v Z2
Y1 v Z1 v ~Z2
Y1 v ~Z1 v Z2
Y1 v ~Z1 v ~Z2
to I'. Observe that Y is true if and only if all four
clauses are true, no matter what values we give Z1
and Z2.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example: proving that 3SAT is NP-Complete
(continued):
Consider a clause C (continued):
If C contains more than 3 variables, we proceed as
in the following example. Suppose C = Y1 v Y2 v Y3
v Y4 v Y5 v Y6 v Y7. Then we add the following to I':
Y1 v Y2 v Z1
~Z1 v Y3 v Z2
~Z2 v Y4 v Z3
~Z3 v Y5 v Z4
CPSC 320 – 2014W1
~Z4 v Y6 v Y7
CPSC 320: NP-Completeness
Example: proving that 3SAT is NP-Complete
(continued):
If all the Y's are false, then how could we satisfy all
5 clauses?
Y1 v Y2 is false, so Z1 must be true.
~Z1 v Y3 is false, so Z2 must be true.
...
~Z3 v Y5 is false, so Z4 must be true.
But then ~Z4 v Y6 v Y7 is false.
So we can not satisfy all five clauses if all the Y's
are false.
CPSC 320 – 2014W1
CPSC 320: NP-Completeness
Example: proving that 3SAT is NP-Complete
(continued):
If Yj is true then we can make all the clauses true
by setting Zk true for k < j-1, and false for k  j-1.
For instance, suppose Y4 is true: the red term in
each clause is true (other terms might also be true
if more than one Y is true).
Y1 v Y2 v Z1
~Z1 v Y3 v Z2
~Z2 v Y4 v Z3
~Z3 v Y5 v Z4
CPSC 320 – 2014W1
~Z4 v Y6 v Y7
CPSC 320: NP-Completeness
Example: proving that 3SAT is NP-Complete
(continued):
Therefore the answer to the instance I of SAT is
“Yes” if and only if the answer to the instance I' of
3SAT is “Yes”.
Moreover we can generate I' in linear time.
CPSC 320 – 2014W1