5/4/2016
Computer Science 101
Problem: “Given a graph, is it connected?”
Each particular graph is an instance.
Some instances are yes-instances; some are no-instances.
Chapter 10
Part 2: NP Complete Problems
The size of the instance, n, is the number of bits needed to
specify it
An algorithm is polynomial-time if it uses at most knc
steps, for some constants k,c
P is the class of all problems that have polynomial-time
algorithms
Definition of P
EXPTIME (or EXP)
P. Problems for which there is a poly-time algorithm.
Problem
Description
Algorithm
Yes
No
MULTIPLE
Is x a multiple of y?
Grade school
division
51, 17
51, 16
RELPRIME
Are x and y relatively prime?
Euclid (300 BCE)
34, 39
34, 51
PRIMES
Is x prime?
AKS (2002)
53
51
EDITDISTANCE
Is the edit distance between
x and y less than 5?
Dynamic
programming
niether
neither
acgggt
ttttta
LSOLVE
Is there a vector x that
satisfies Ax = b?
Gauss-Edmonds
elimination
0 1 1  4 

  
2 4 2  ,  2 


0 3 15 
 
36 
• Problems that can be solved in exponential time (O(2f(N))) for some polynomial function f(N))
 1 0 0  1
 1 1 1 , 1

  
0 1 1 1
3
NP
• Problems whose solutions can be verified in polynomial time
• For these problems, an algorithm takes the problem plus a potential solution and verifies that it’s indeed a solution in polynomial time O(Nk) for some constant k.
• Or equivalently, problems that can be solved by nondeterministic algorithms in polynomial time.
Formal Definition of NP
Certification algorithm intuition.
Certifier views things from "managerial" viewpoint.
Certifier doesn't solve the problem on its own;
rather, it checks a proposed solution t is indeed a solution.


Def. Algorithm C(s, t) is a certifier for a decision problem X if for
every instance s, s has a yes answer iff there exists a certificate t
such that C(s, t) = yes.
"witness"
NP. Problems for which there exists a poly-time certifier.
C(s, t) is a poly-time algorithm and the size of
t  a polynomial of the size of s.
Remark. NP stands for nondeterministic polynomial-time, which is an
equivalent definition of the same class of problems.
For an NP problem, an instance is a yes-instance iff there
exists a certificate t such that C(s, t) = yes.
6
1
5/4/2016
Certifiers and Certificates: Satisfiability
Certifiers and Certificates: Composite
SAT. Given a formula , is there a satisfying assignment?
COMPOSITES. Given an integer s, is s composite?
Certificate. An assignment of truth values to the n boolean variables.
Certificate. A nontrivial factor t of s. Note that such a certificate
exists iff s is composite. Moreover |t|  |s|.
Certifier.
Certifier. Check that each clause in  has at least one true literal.
boolean C(s, t) {
if (t  1 or t  s)
return false
else if (remainder(s, t) == 0)
return true
else
return false
}
Ex.
 x1
 x2  x3  
 x1
 x2  x3  
 x1
 x2  x4   x1  x3  x4 
instance s
x1  1, x2  1, x3  0, x4  1
certificate t
Instance. s = 437,669.
437,669 = 541  809
Certificate. t = 541 or 809.
Conclusion. COMPOSITES is in NP.
Conclusion. SAT is in NP.
Every problem in P is also in NP.
7
8
P, NP, EXP
Certifiers and Certificates: Hamiltonian Cycle
HAM-CYCLE. Given an undirected graph G = (V, E), does there exist a
P. Decision problems for which there is a poly-time algorithm.
EXP. Decision problems for which there is an exponential-time algorithm.
NP. Decision problems for which there is a poly-time certifier.
simple cycle C that visits every node?
Certificate. A permutation of the n nodes.
Claim. P  NP.
Pf. Consider any problem X in P.
By definition, there exists a poly-time algorithm A(s) that solves X.
That is, s is a yes-instance iff A(s) = yes.
For any certificate t, define certifier as C(s, t) = A(s). Then C(s, t)
takes poly-time and returns yes iff s is a yes-instance ▪
Certifier. Check that the permutation contains each node in V exactly
once, and that there is an edge between each pair of adjacent nodes in
the permutation.

Conclusion. HAM-CYCLE is in NP.

Claim. NP  EXP.
Pf. Consider any problem X in NP.
By definition, there exists a poly-time certifier C(s, t) for X.
To solve input s, run C(s, t) on all strings t with |t|  p(|s|), where |t|
and |s| are the sizes of s and t, respectively.
Return yes, if C(s, t) returns yes for any of these. ▪


instance s
certificate t

9
Does
P=NP?
10
The Main Question: P Versus NP
Does P = NP? [Cook 1971, Edmonds, Levin, Yablonski, Gödel]
Is solving a problem as easy as verifying its solution (upto to a polytime difference)?
Clay $1 million prize.


NP
EXP
EXP
P
P = NP
If P  NP
If P = NP
would break RSA cryptography
(and potentially collapse economy)
If yes: Efficient algorithms for 3-COLOR, TSP, FACTOR, SAT, …
If no: No efficient algorithms possible for 3-COLOR, TSP, SAT, …
The (literally) $1,000,000 question
Consensus opinion on P = NP? Probably no.
12
2
5/4/2016
Review: Polynomial-Time Reduction
NP-Hard and NP-Complete
Purpose. Classify problems according to relative difficulty.
NP-hard. A problem Y is NP-hard if, for every problem X in NP, X  p Y.
NP-complete. A problem Y is NP-complete, if it is NP-hard and in NP.
Design algorithms. If X  P Y and Y can be solved in polynomial-time, then
X can also be solved in polynomial time. That is, if Y is easy, so is X.
Theorem. Suppose Y is an NP-complete problem. Then Y is in P iff P = NP.
Pf.  If P = NP then Y can be solved in poly-time since Y is in NP.
Pf.  Suppose Y is in P, i.e., can be solved in poly-time.
Let X be any problem in NP. Since X  p Y, we can solve X in
poly-time. This implies NP  P.
We already know P  NP. Thus P = NP. ▪
Establish intractability. If X  P Y and X cannot be solved in polynomialtime, then Y cannot be solved in polynomial time. That is, if X is hard, so
is Y.


Establish equivalence. If X  P Y and Y  P X, we use notation X  P Y.
Fundamental question. Do there exist "natural" NP-complete problems?
up to cost of reduction
13
14
NP-hard
Hamilton cycle
Graph 3-coloring
Satisfiability
Maximum clique
…
NPcomplete
SAT is NP‐Complete
Halting problem
…
• The Cook‐Levin theorem: SAT is NP‐complete
• For details, see:
http://en.wikipedia.org/wiki/Cook‐Levin_theorem
• Need to show:
NP
Graph connectivity
Primality testing
Matrix determinant
Linear programming
…
– SAT is in NP
– All other problems in NP can be reduced to SAT
– For any NP problem, there is a certifier C(s, t) that takes poly‐time in |s| to check if s is a yes‐instance using certificate t.
– An instance s is a yes‐instance iff there exists a certificate t
such that C(s, t) = yes. Factoring
Graph isomorphism
Minimum circuit size
…
P
3-SAT is NP-Complete
SAT is NP‐Complete
• Need to show:
– For any NP problem, there is a certifier C(s, t) that takes poly‐
time in |s| to check if s is a yes‐instance using certificate t.
– An instance s is a yes‐instance iff there exists a certificate t such that C(s, t) = yes. – Suppose C(s, t) is implemented in today’s computer and it takes m=k*size(s)c steps to execute. – At any step i, today’s computer can be represented by a combinatorial circuit, which can be viewed as a logic formula Fi.
– Input (s, t) can be Boolean variables for F0 and outputs of Fi‐1 are inputs of Fi for i=1, …, m.
– s is a yes‐instance iff ᴲt C(s, t) = yes iff (F0, F1, …, Fm) is satisfiable.
Theorem. 3-SAT is NP-complete.
Pf. Suffices to show that SAT  P 3-SAT since 3-SAT is in NP.
Let K be any circuit representing the formula of SAT.
Create a 3-SAT variable xi for each circuit element i.
Make circuit compute correct values at each node:
x2  x3 , x2  x3
– x2 ≡  x3
 add 2 clauses:
– x1 ≡ x4  x5  add 3 clauses:
x1  x4 , x1  x5 , x1  x4  x5
– x0 ≡ x1  x2  add 3 clauses:
x0  x1 , x0  x2 , x0  x1  x2





Hard-coded input values and output value.
– x5 = 0  add 1 clause:
x5
x0
– x0 = 1  add 1 clause:
Final step: turn clauses of length < 3 into
clauses of length exactly 3 by introducing new
variables. ▪
output
x0

x1
x2


x5
x4
0
s
x3
?
?
t
3
5/4/2016
3-SAT Reduces to Independent Set
3-SAT Reduces to Independent Set
Claim. 3-SAT  P INDEPENDENT-SET.
Pf. Given an instance  of 3-SAT, we construct an instance (G, k) of
INDEPENDENT-SET that has an independent set of size k iff  is
satisfiable.
Claim. G contains independent set of size k = || iff  is satisfiable.
Pf.  Let S be independent set of size k.
S must contain exactly one vertex in each triangle.
and any other variables in a consistent way
Set these literals to true.
Truth assignment is consistent and all clauses are satisfied.


Construction. G = (V, E)
V = { ci,j | ci is a clause of , 1<= j <= 3}, it contains 3 vertices for
each clause, one for each occurrence of literal.
E connects 3 literals in a clause in a triangle.
E connects literal to each of its negations.


Pf  Given satisfying assignment, select one true literal from each
triangle. This is an independent set of size k. ▪


x2
x1
x1
G
x1
G
x2
k=3
x2
x1
x3
 
 x1
x1
 x2  x 3  
x3
 x1
 x2  x3  
x2
 x1
x4
x2
 x2  x 4 
k=3
x3
 
 x1
x1
 x2  x 3  
x3
 x1
 x2  x3  
x2
 x1
x4
 x2  x 4 
19
Review
20
NP-Complete Problems
Basic reduction strategies.
Simple equivalence: INDEPENDENT-SET  P VERTEX-COVER.
Special case to general case: VERTEX-COVER  P SET-COVER.
Encoding with gadgets: 3-SAT  P INDEPENDENT-SET.

• More than 3,000 known NP-complete
problems


• Art gallery problem—Find the minimum number of guards needed
• Bin packing—Pack objects in the fewest bins possible
• Bottleneck TSP—Find a Hamiltonian path with minimum largest
link cost
• Chromatic number (or vertex coloring)—Given a graph, find the
smallest number of colors needed to color the graph’s nodes.
(The graph is not necessarily planar.)
• Clique—In a graph, find the largest clique (mutually connected
nodes)
• Clique cover problem—Given a number k, find a way to partition a
graph into k cliques.
Transitivity. If X  P Y and Y  P Z, then X  P Z.
Pf idea. Compose the two algorithms.
Ex: 3-SAT  P INDEPENDENT-SET  P VERTEX-COVER  P SET-COVER.
21
NP-Complete Problems, Part 2
NP-Complete Problems, Part 3
• Dominating set—Given a graph, find a set of nodes S so that
every other node is adjacent to one of the nodes in the set S
• Hamiltonian cycle—Determine whether there is a path through a
graph that visits every node exactly once and then returns to its
starting point
• Hamiltonian path (HAM)—Determine whether there is a path
through a graph that visits every node exactly once
• Knapsack—Given a knapsack with a capacity and a set of objects
with weights and values, find the set of objects with the largest
possible value that fits in the knapsack.
• Longest path—Find the longest path that doesn’t revisit any
nodes
• Maximum independent set—Find the largest set of nodes where
no two nodes in the set are connected by a link
• Maximum leaf spanning tree—Find a spanning tree that has the
maximum possible number of leaves
• Minimum leaf spanning tree—Find a spanning tree that has the
minimum possible number of leaves
• Minimum degree spanning tree—Find a spanning tree with the
minimum possible degree
• Partitioning—Given a set of integers, find a way to divide the
values into two sets with the same total value
4
5/4/2016
NP-Completeness
NP-Complete Problems, Part 4
Observation. All problems below are NP-complete and polynomial
reduce to one another!
• Satisfiability (SAT)—Given a boolean expression containing
variables, find an assignment of true and false to the variables
to make the expression true
• Subset sum—Given a set of integers, find a subset with a given
total value
• Three-satisfiability (3-SAT)—Given a boolean expression in
3CNF, find an assignment of true and false to the variables to
make the expression true
• Traveling salesman problem (TSP)—Given a list of cities and the
distances between them, find the shortest possible route that
visits all the cities and returns to the starting city
• Vertex cover—Find a minimal set of vertices so that every link
in the graph touches one of the selected vertices
by definition of NP-completeness
CIRCUIT-SAT
3-SAT
INDEPENDENT SET
DIR-HAM-CYCLE
GRAPH 3-COLOR
SUBSET-SUM
VERTEX COVER
HAM-CYCLE
PLANAR 3-COLOR
SCHEDULING
SET COVER
TSP
26
Subset Sum
Subset Sum
SUBSET-SUM. Given a set of natural numbers w1, …, wn and an integer
W, is there a subset that adds up to exactly W?
Construction. Given 3-SAT instance  with n variables and k clauses,
form 2n + 2k decimal integers, each of n+k digits, as illustrated below.
Ex: { 1, 4, 16, 64, 256, 1040, 1041, 1093, 1284, 1344 }, W = 3754.
Yes. 1 + 16 + 64 + 256 + 1040 + 1093 + 1284 = 3754.
Claim.  is satisfiable iff there exists a subset that sums to W.
Pf. No carries possible.
x
y
z
C1
C2
C3
1
0
0
0
1
0
100,010
x
1
0
0
1
0
1
100,101
C2  x  y  z
y
0
1
0
1
0
0
10,100
y
0
1
0
0
1
1
10,011
C3  x  y  z
z
0
0
1
1
1
0
1,110
z
0
0
1
0
0
1
1,001
0
0
0
1
0
0
100
0
0
0
2
0
0
200
0
0
0
0
1
0
10
0
0
0
0
2
0
20
0
0
0
0
0
1
1
0
0
0
0
0
2
2
1
1
1
4
4
4
111,444
x
Remark. With arithmetic problems, input integers are encoded in
binary. Polynomial reduction must be polynomial in the size of binary
encoding.
C1  x  y  z
Claim. 3-SAT  P SUBSET-SUM.
Pf. Given an instance  of 3-SAT, we construct an instance of SUBSETSUM that has solution iff  is satisfiable.
dummies to get clause
columns to sum to 4
W
27
Set Partition
28
Bin Packing
PARTITION. Given a set of natural numbers w1, …, wn , is there a subset
BIN-PACKING. Given a set S of real numbers w1, …, wn , 0 < wi  1, and
that adds up to exactly half sum of all wi?
integer K, is there a partition of S into K subsets such that each
subset adds up no more than 1?
Claim. PARTITION  P SUBSET-SUM.
Claim. PARTITION  P BIN-PACKING.
Pf. PARTITION is a special of SUBSET-SUM.
Pf. For any instance S of PARTITION, we construct S’, an instance of
BIN-PACKING: S’ = { 2x/sum(S) | x in S } and k = 2.
Claim. SUBSET-SUM  P PARTITION.
Pf. For any instance (S, W) of SUBSET-SUM, we construct an instance
S’ of PARTITION, S’ = S U { x, y }, where x = sum(S)+W and y =
2sum(S)-W.
Claim: S can be partitioned into two equal sum subsets iff S’ can be
packed into k=2 bins.
Claim: (S, W) has a SUBSET-SUM solution iff S’ can be partitioned into
two equal sum subsets.
29
30
5
5/4/2016
Asymmetry of NP
The Knapsack Problem
Input
Capacity K
n items with weights wi and values vi
Goal
Output a set of items S such that
– the sum of weights of items in S is at most K
– and the sum of values of items in S is maximized
Asymmetry of NP. We only need to have poly-time verifiable proofs of
yes instances.


Ex 1. SAT vs. UNSAT.
Can prove a CNF formula is satisfiable by giving such an assignment.
How could we prove that a formula is not satisfiable?



Ex 2. HAM-CYCLE vs. NO-HAM-CYCLE.
Can prove a graph is Hamiltonian by giving such a Hamiltonian cycle.
How could we prove that a graph is not Hamiltonian?


Remark. SAT is NP-complete, but how do we classify UNSAT?
not even known to be in NP
32
NP and co-NP
NP = co-NP ?
NP. Decision problems for which there is a poly-time certifier.
Ex. SAT, HAM-CYCLE, COMPOSITES.
Fundamental open question. Does NP = co-NP?
Do yes instances have succinct certificates iff no instances do?
Consensus opinion: no.


Def. Given a decision problem X, its complement X is the same problem
with the yes and no answers reversed.
Equivalence: Since X  P X and X  P X, we have X  P X.
Theorem. If NP  co-NP, then P  NP.
Pf idea.
P is closed under complementation.
If P = NP, then NP is closed under complementation.
In other words, if P = NP, then NP = co-NP.
This is the contrapositive of the theorem.
co-NP. Complements of decision problems in NP.
Ex. TAUTOLOGY, NO-HAM-CYCLE, PRIMES.
1.
Ex. X = { 0, 1, 4, 6, 8, 9, 10, 12, 14, 15, … } -- composite numbers
Ex. X = { 2, 3, 5, 7, 11, 13, 17, 23, 29, … } -- primes




Three possibilities:
NP  co-NP and P  NP.
2.
NP = co-NP and P  NP.
3.
NP = co-NP = NP.
NP-hardness: the complement of every NP-hard problem is NP-hard.
33
34
6