Design and Analysis
of Algorithms
Lecture 11
Yoram Moses
June 3, 2010
http://www.ee.technion.ac.il/courses/046002
1
Nondeterministic Polynomial Time
(NP)
2
Shortest Path:
Search, Existence, Verification

Search problem:




Existence problem:




Input: (G,w,s,t): a directed graph G with weight function w,
a source s, and a sink t.
Goal: find a shortest path from s to t. (or reject if none exists)
Complexity: our solution runs in O(VE) = O(n2)
(Notice n = size of input = O(V+E))
Input: (G,w,s,t,k): G,w,s,t are as before + a number k.
Goal: decide whether there is a path from s to t of length ≤ k.
Complexity: our solution runs in O(VE) = O(n2)
Verification problem:



Input: (G,w,s,t,k,p): G,w,s,t,k as before. p is a path in G.
Goal: decide whether p is a simple path from s to t of length ≤ k.
Complexity: O(V) = O(n).
3
Max Flow:
Search, Existence, Verification

Search problem:




Existence problem:




Input: (G,c,s,t): a directed graph G with capacity function c, a
source s, and a sink t.
Goal: find a maximum flow in G. (or reject if none exists)
Complexity: O(VE2) = O(n3)
Input: (G,c,s,t,k): G,c,s,t as before + a number k.
Goal: decide whether there is a flow in G with value ≥ k.
Complexity: O(VE2) = O(n3)
Verification problem:



Input: (G,c,s,t,k,f): G,c,s,t,k as before. f is a function from edges of
G to real numbers.
Goal: decide whether f is a legal flow with value ≥ k.
Complexity: O(E) = O(n).
4
Hamiltonian Cycle:
Search, Existence, Verification

Search problem:




Existence problem:




Input: an undirected graph G.
Goal: find a Hamiltonian cycle in G (or reject if none exists).
Complexity: O(VxV!) = O(n2n log n)
Input: an undirected graph G.
Goal: decide whether G has a Hamiltonian cycle.
Complexity: O(VxV!) = O(n2n log n)
Verification problem:



Input: (G,p): an undirected graph G and a sequence of nodes p.
Goal: decide whether p is a Hamiltonian cycle in G.
Complexity: O(V) = O(n).
5
3-Coloring:
Search, Existence, Verification

Search problem:




Existence problem:




Input: G: an undirected graph
Goal: find a 3-Coloring of G. (or reject if none exists)
Complexity: O(E 3V) = O(n2n log 3)
Input: G: as before.
Goal: decide whether G has a 3-Coloring.
Complexity: O(E 3V) = O(n2n log 3)
Verification problem:



Input: (G,): G as before and  : V {1,2,3}.
Goal: decide whether  is a 3-Coloring of G.
Complexity: O(E) = O(n).
6
Search and Existence vs.
Verification
Conclusion: in many natural examples:
 Search
and existence are computationally
equivalent
 Verification is easier
Sometimes it’s just a little easier (Shortest Path,
Max flow)
 Sometimes it’s a lot easier (Hamiltonian cycle,
3-Coloring)

7
Verification Relations

Language: L  {0,1}*

Definition: A verification relation for L is a relation
R  {0,1}*  {0,1}* s.t. for all x  {0,1}*:
x  L  there is at least one y  {0,1}* s.t. (x,y)  R.
 x  L  there is
no
y  {0,1}* s.t. (x,y)  R.


y is called the “certificate” for x


A.k.a. its “witness” or “proof”
Remarks:
Every input x  L has at least one certificate y.
 If (x,y)  R, then y is a certificate for x.
 An input x  L may have several certificates.
 A language L has many verification relations.

8
Verification Relations: Examples

Shortest path:
x
= (G,w,s,t,k), y = a path p
 Language: {(G,w,s,t,k): G has an s-t path of length ≤ k}
 Certificate: s-t path of length ≤ k
 Verification relation:
{((G,w,s,t,k),p): p is an s-t path of length ≤ k in G}

Hamiltonian cycle:
x
= undirected graph G, y = a path p
 Language: G that has a Hamiltonian cycle
 Certificate: a Hamiltonian cycle in G
 Verification relation: {(G,p): p is a Hamiltonian cycle in G}
9
Nondeterministic Polynomial Time
Definition: A binary relation R is polynomially bounded, if
there exists some c > 0 s.t. for every (x,y)  R, |y| ≤ |x|c.
Definition: L is polynomial-time verifiable, if it has a
verification relation R, which satisfies both:


R is polynomially bounded, and
R is polynomial-time decidable.
Definition: The class NP (Nondeterministic Polynomial
Time) is the set of all polynomial-time verifiable
languages.
10
NP: Examples

Examples of languages in NP:



Decision Shortest Path, Decision Max Flow, Decision LP
Hamiltonian Cycle, TSP, 3-Coloring, SAT, k-SAT, Clique
Examples of languages not known to be in NP:

HC-complement: given a graph G, decide whether G has no
Hamiltonian cycles.
11
Nondeterministic Algorithms

Definition: A nondeterministic algorithm is an algorithm N
that, on input x,



First, N “nondeterministically” guesses a “witness” y.
Then, N runs a deterministic “verification” algorithm on (x,y).
Note: N may make different nondeterministic guesses in different
runs on the same input x.
Nondeterministic Algorithm N
x
Nondeterministic
guess
y
Verification(x,y)
yes/no
12
Decision by Nondeterministic
Algorithms

Definition: A nondeterministic algorithm N is said to decide language L if:
For every input x  L, there is at least one guess y s.t. N accepts (x,y).
 For every input x  L, the verification algorithm N rejects (x,y), for all
guesses y.


A polynomial-time nondeterministic algorithm is one in which



The guesses (y’s) are of polynomial size (in |x|), and
The verification algorithm runs in polynomial time.
Lemma: L  NP iff L is decidable by a polynomial-time nondeterministic
algorithm.
13
An NP Algorithm for Clique
Nondeterministic guess (input: x = (G,k))
1. for i = 1,…,k
2.
vi  nondeterministic guess of a node in V=V(G)
3. output y = (v1,…,vk)
Verification algorithm (input: (x,y))
1. If x is not a valid encoding of a graph G and an integer k, reject.
2. If y is not a valid encoding of k nodes v1,…,vk in G, reject.
3. If v1,…,vk are not distinct, reject.
4. for i 1,…,k-1 do
5.
for j  i+1,…,k do
6.
if {vi,vj}  E reject.
7. accept
14
P vs. NP
Lemma: P  NP
 Biggest open problem of computer science:
is P = NP?
P = NP?

Two possibilities:
NP
P = NP

P
Current belief: P  NP
 Search
& Existence strictly harder than Verification.
15
Time Hierarchy


f: N  N: a complexity measure.
Time(f(n)) = all languages decidable in time O(f(n)).
Lemma:
Let f(n),g(n) be two complexity measures. If there exists a
constant c, s.t. for all n > c, f(n) ≤ g(n), then
Time(f(n))  Time(g(n)).
Theorem (Time Hierarchy)
Let f(n),g(n) be two complexity measures. If there exists a
constant c, s.t. for all n > c, f(n) ≤ g(n)1/2, then
Time(f(n))  Time(g(n)).
16
P, NP, and EXP

Definition:
Lemma: P  EXP but P  EXP
Lemma: NP  EXP (exercise)
Open problem: is NP = EXP?
 3 Possibilities:
EXP
NP =
P
EXP
= NP
P
EXP
NP
P
P
17
NP-Completeness (NPC)



Problems in NP not known to be in P:
Hamiltonian Cycle, Clique, SAT, k-SAT (k ≥ 3),
k-Coloring (k ≥ 3), TSP, …. (many others)
All of these are “NP-Complete”
NP-Complete Problems:



Belong to NP
If any of them belongs to P, then NP = P.
Two possibilities:
NP =
NPC =
P
NP
NPC
P
18
NP-Hardness (NPH)
Definition: A language L is NP-hard if
L’ ≤p L holds for all L’  NP.

NPH = class of all NP-hard problems.
Lemma: If any NP-hard problem belongs to P,
then NP = P.
 If
one NPH problem is easy, then all of NP is easy.
Lemma: If L  NPH and L ≤p L’, then
L’  NPH.
19
NP-Completeness
Definition: A language L is NP-complete if both




L  NP and
L is NP-hard
NPC = class of NP-complete problems
NPC = NP  NPH
Theorem:


If some NPC language is in P, then P = NP.
(P  NPC    NP = P = NPC).
If some NPC language is not in P, then no NPC language is in P.
(NPC  P  P  NPC =   NP  P).
20
NP-Completeness
NPC: “hardest” problems in NP
 Behave as a “single block”: either all in P
or all outside P

Lemma: If L1,L2  NPC, then both
L1 ≤p L2 and L2 ≤p L1.
21
Proving NP-Completeness

How to prove that a given language L is NPC?
 Show
 Show

that L  NP, and
that L’ ≤p L holds for every L’  NP.
Easier alternative:
 Show that L  NP, and
 Find some NPC problem

L’ and show L’ ≤p L.
How do we obtain the first NPC problem?
 Using the first alternative
 Cook-Levin theorem: Circuit-SAT
is NP-complete.
22
NP-Completeness: the Full Recipe

To show that L is NPC:
 Prove

L is in NP
Show a polynomial time nondeterministic algorithm
for L
 Select
an NPC problem L’
 Show a polynomial-time reduction f from L’ to L
Prove that x  L’ iff f(x)  L
 Show a polynomial-time algorithm to compute f

23
Example: Clique is NPC
Clique is in NP (seen today)
 3-SAT is NPC (will show this later on)
 3-SAT ≤p Clique (seen in previous lecture)
 Therefore: Clique is also NP-Complete!

24
End of Lecture 11
25