NP-completeness
Class of hard problems
Outline
Introduction
 Problems and Languages



Turing machines and algorithms
P and NP
Satisfiability problem
 Reduction
 Other NP-complete problems

Jaruloj Chongstitvatana
NP-complete Problems
2
Problems


A problem can be defined as a mapping from
the instance to the answer.
Examples:

Shortest path problem
{<graph G, node u, node v>}  {a shortest path P}

0/1 Knapsack problem
{<set S of objects, capacity C>}  {the maximum
value}

Closest-pair-of-points problem
{<set P of points>}  {<point q, point r>, which is
the closest pair of points in P}.

Satisfiable problem
{<boolean formula>}  {T, F}
Jaruloj Chongstitvatana
NP-complete Problems
3
Abstract problems
A set of instances
A set of solutions
Jaruloj Chongstitvatana
NP-complete Problems
4
Decision Problems

A decision problem is a problem whose
answer is either “yes” or “no”.

Example: Satisfiable problem
{<boolean formula>}  {T, F}

Any problem can be described in term of
a decision problem.
Jaruloj Chongstitvatana
NP-complete Problems
5
Reformulate Problems as Decision Problems


Shortest path problem

{<graph G, node u, node v, int k>}  {T, F}

The answer is T iff there is no path between
u and v, which is shorter than k.
0/1 Knapsack problem


{<set S of objects, capacity C, value V>} 
{T, F}
The answer is T iff there is no selection of
objects fitted in the sack which yields higher
value than V.
Jaruloj Chongstitvatana
NP-complete Problems
6
Encoding of Decision Problems

An encoding of a problem is a mapping from the
set of instances of the problem to a set of strings.

Example: shortest path

The encoding of <graph G, node u, node v, path p>
in the Shortest path problem is a string which is used
to describe G, u, v, and p.



G can be described by a string of its adjacency matrix.
u and v can be described by two strings, indicating the
node labels.
p can be described by a sequence of node in the path.
Jaruloj Chongstitvatana
NP-complete Problems
7
Abstract/ Concrete problems
A set of instances
Mapping
(encoding)
Abstract
problem
Mapping
(encoding)
Concrete
problem
A set of solutions
Jaruloj Chongstitvatana
NP-complete Problems
8
Encoding of Problems

Is X a prime ?
f:{X |X>0 and is a binary number}{Y,N}
f(x) = Y
f(x) = N

if X is prime
otherwise
Is a 3-CNF expression X satisfiable?
f:{X | X is a 3-CNF expression}  {Y, N}
f(x) = Y
f(x) = N
Jaruloj Chongstitvatana
if X is satisfiable
otherwise
NP-complete Problems
9
Why encoding is important?
Encoding determines the instance size.
 Example: an integer n




n encoded as one number
=> instance size = 1
n encoded as binary number
=> instance size = log2 n
n encoded as unary number
=> instance size = n
Jaruloj Chongstitvatana
NP-complete Problems
10
Problems and Languages
Languages and Decision Problems

Let  be an alphabet.

A language over  is a set of strings created from
symbols in .

B={0,1}* is the set of all binary strings.

{ is a palindrome} is a language over B.

A yes-instance (or no-instance) is an instance
whose answer is “yes” (or “no”).

An instance must be either a yes- or a noinstance.
Jaruloj Chongstitvatana
NP-complete Problems
12
Languages and Decision Problems

A decision problem can be encoded in a
language {| is an encoding of a yesinstance}.

The shortest path problem can be encoded as
{e(G, u, v, p>)| p is the shortest path between u
and v in the graph G}.
Jaruloj Chongstitvatana
NP-complete Problems
13
Turing Machines
Turing machines (TM)

A Turing machine is a machine which takes
a string on its input tape, and decides
whether to accept or to reject the string.

A Turing machine contains

A finite-state control with

A start state

A halt state


Transition function: state x symbol  state x symbol x
{R,L}
A tape with a Read/Write head which can move
left and right.
Jaruloj Chongstitvatana
NP-complete Problems
15
Deterministic/ Nondeterministic Turing machines

Difference


What is nondeterminism?


A transition is deterministic or nondeterministic.
Given a state of a machine/program, there is
more than one possible next step.
Examples


Deterministic: functions.
Nondeterministic: relations.
Jaruloj Chongstitvatana
NP-complete Problems
16
Example of a DTM
s
/@,R
p1
q1
@/,R
p2
0/0,R
1/1,R
0/,R
p4
1/,R
q2
Jaruloj Chongstitvatana
NP-complete Problems
/1,L
/,L
p3
h
17
How a DTM works
s
/,L
/@,R
/,R
0/0,L
1/1,L
p4
0/,L
p1
1/,R
0/,R
p2
0/0,R
1/1,R
/,L
p3
Jaruloj Chongstitvatana
q1


 
0 0
0 1
0 0 
0 
 1
 
@
@/,R
q2
/1,L
h
On the input 0001000,
the TM halts.
NP-complete Problems
18
How a DTM works
s
/,L
/@,R
/,R
0/0,L
1/1,L
p4
0/,L
p1
1/,R
0/,R
p2
0/0,R
1/1,R
/,L
p3
Jaruloj Chongstitvatana
q1
 
0 0
0 0
0 0 
0 
 
 
@
@/,R
q2
/1,L
h
On the input 0000000,
the TM hangs.
NP-complete Problems
19
Example of NTM

Let L={ww| w{0,1}*}
r0
s
/@,R
v
q0
0/0,L
1/1,L
/@,L
0/0,R
1/1,R
p
t0
/,R
u
@/,L
h
0/0,R
1/1,R
0/0,L
1/1,L
/,L
q1
t1
r1
Jaruloj Chongstitvatana
NP-complete Problems
0/0,R
1/1,R
20
Accept/Decide
Let T be a Turing machine.
 For a string  over ,


T accepts  iff T halts on  with output “1”.
T rejects  iff T halts on  with output “0”.

T hangs on  iff T dose not halts on .


For a language L over ,

T accepts L iff, for any string  in L, T accepts .

T decides  iff
 for any string  in L, T accepts , and
 for any string  inL, T rejects .
Jaruloj Chongstitvatana
NP-complete Problems
21
Time complexity

A language L is accepted/decided in O(f(n))
by a Turing machine T if, for any length-n
string  in L,  is accepted/decided in k·f(n)
by a Turing machine T, for some constant k.

A language L is accepted/decided in
polynomial time by a Turing machine T if, L
is accepted/decided in O(nk) by a Turing
machine T, for some constant k.
Jaruloj Chongstitvatana
NP-complete Problems
22
Church-Turing Thesis
If there is a Turing machine deciding a
problem  in O(f), then there is an
algorithm solving  in O(f).
 Turing machine  algorithm

Jaruloj Chongstitvatana
NP-complete Problems
23
Complexity classes
Class of P

If  is a problem in P, given any instance I
of ,


there is an algorithm which solves I in O(nk), or
there is a Turing machine which accepts e(I) in
O(nk) steps,
where n is the size of e(I) and k is a
constant.

What does this mean?

If  is in P,  is easy because it can be solved in
polynomial time.
Jaruloj Chongstitvatana
NP-complete Problems
25
Class of NP

If  is a problem in P, given any instance I
and any certificate C of ,



there is an algorithm which verifies, in O(nk),
that C is the answer of I, for any C and I.
there is a Turing machine which verifies, in
O(nk), that C is the answer of I, for any C and I.
there is a nondeterministic Turing machine which
accepts e(I), in O(nk), for any I.
where n is the size of e(I) and k is a
constant.
What does this mean?

If  is in NP,  is not necessarily easy because it
can only be verified in polynomial time.
Jaruloj Chongstitvatana
NP-complete Problems
26
Easy problems / Hard problems

Can be solved in
polynomial time


Prove by showing the
algorithm.
Cannot be solved in
polynomial time



Can verify the answer
in polynomial time.
Can usually be solved
by an exponential-time
algorithm.

Jaruloj Chongstitvatana
Prove by showing that
there is no polynomialtime algorithm.
Brute-force algorithm:
try every possible
answers.
NP-complete Problems
27
Class of co-NP

L is in co-NP ifL is in NP.
NP
Jaruloj Chongstitvatana
P
co-NP
NP-complete Problems
28
Reduction
Is X more difficult than Y?
Reduction

Let L1 and L2 be languages over 1 and 2,
respectively. L1 is (polynomial-time) reducible to
L2, denoted by L1L2, if there is a TM M computing
a function f: 1*2* such that wL1  f(w)L2 in
polynomial time.

Let P1 and P2 be problems. P1 is (polynomial-time)
reducible to P2, denoted by P1P2, if there is an
algorithm computing a function f: 1*2* such that
w is a yes-instance of P1  f(w) is a yes-instance of
P2 in polynomial time.
Jaruloj Chongstitvatana
NP-complete Problems
30
Meaning of Reduction

P1 is reducible to P2
if  DTM computing, in polynomial time, a function
f: 1*2* such that w is a yes-instance of P1 
f(w) is a yes-instance of P2.
If you can map yes-instances of problem A to yesinstances of problem B, then
 we can solve A in polynomial time if we can solve B
in polynomial time.
 it doesn’t mean we can solve B in polynomial time
if we can solve A in polynomial time.
Jaruloj Chongstitvatana
NP-complete Problems
31
Example of Reductuion
Satisfiability problem (SAT)


A B C A~BC
~A~C 
Given a Boolean
expression , is 
satisfiable?
0 0 0
0
1
0
(A~BC)(~A~C)B
~C is not satisfiable.
0 0 1
0 1 0
1
0
1
1
0
0
0 1 1
1
1
0
1 0 0
1
1
0
1 0 1
1
0
0
1 1 0
0
1
0
1 1 1
1
0
0
Jaruloj Chongstitvatana
NP-complete Problems
33
3CNF-SAT Problem
Given an expression  in 3CNF, is 
satisfiable?
 Given an expression  in 3CNF, is there a
truth assignment of propositions in  which
makes  true?
 Example:



=(PQR)(PQR)(PSR)(PSR)
 is satisfiable when


P=t, Q=f, R=f, S=t or
P=f, Q=f, R=f, S=t
Jaruloj Chongstitvatana
NP-complete Problems
34
Clique or Complete Subgraph

A graph G=(V,E) has a clique if there is W V such
that for all nodes a and b in W, there is an edge
between a and b.
C
B
A
E
F
D
Jaruloj Chongstitvatana
NP-complete Problems
35
Clique Problem


Given a graphG=(V,E) and an integer k, is there a
clique of size k in G?
Example: Is there a clique of size 3, 4, or 5, in the
graph below?
C
B
A
E
F
D
Jaruloj Chongstitvatana
NP-complete Problems
36
3CNF-SAT  Clique
Proof:
To prove that 3CNF-SAT  Clique,
 Find a function f such that
a
3CNF expression  is satisfiable 
a graph G has a clique of size k, where
<G,k>=f() and
 there is a TM computing f in polynomial time.
f can be defined as follows:
Let = (111213)(212223)…(n1n2n3)
f() = <G,n>, where G=(V, E),
 V={ij|
0<i<n+1, 0<j<4}, and
 E={(ij, km)| i≠k and ij ≠~km}
Jaruloj Chongstitvatana
NP-complete Problems
37
Building a graph from a 3CNF exp.
(PQR)(PSR)(PSR)
~Q
P
R
~P
P
~S
S
~R
Jaruloj Chongstitvatana
~R
NP-complete Problems
38
3CNF-SAT  Clique (cont’d)
Show  is satisfiableG has clique of size n
 Let G has a clique of size n.
 From the definition of G, there is no edge
between vertices

in the same disjunctive clause, and
 representing a literal and its negation.
A link between two nodes means that the two
literals can be true at the same time, and they are
in different disjunctive clauses.


Jaruloj Chongstitvatana
NP-complete Problems
39
Satisfiability and Clique
(PQR)(PSR)(PSR)
~Q
P
R
~P
P
~S
S
~R
Jaruloj Chongstitvatana
~R
NP-complete Problems
40
Satisfiability and Clique
(PQR)(PQR)(PQR)(PQR)
Q
P
R
~P
P
Q
Q
R
~R
~P
~R
Q
Jaruloj Chongstitvatana
NP-complete Problems
41
Satisfiability and Clique
(PQR)(PQR)(PQR)(PQR)
Q
P
R
~P
P
Q
Q
R
~R
~P
~R
Q
Jaruloj Chongstitvatana
NP-complete Problems
42
3CNF-SAT  Clique (cont’d)

There is a TM computing f in polynomial time.
Jaruloj Chongstitvatana
NP-complete Problems
43
Properties of Reductions
Reflexivity of reduction
Theorem:
Let L be a language over . L  L.
Proof:
Let L be a language over .
Let f be an identity function from **.
Then, there is a TM computing f.
Because f is an identity function, wL 
f(w)=wL.
By the definition, L  L.
Jaruloj Chongstitvatana
NP-complete Problems
45
Property of reduction
Theorem:
Let L1 and L2 be languages over .
If L1L2, thenL1L2.
Proof:
Let L1 and L2 be languages over .
Because L1L2, there are a function f such that
wL1  f(w)L2, and a TM T computing f in
polynomial time.
wL1  f(w)L2.
By the definition,L1L2.
Jaruloj Chongstitvatana
NP-complete Problems
46
Transitivity of reduction
Theorem:
Let L1, L2 and L3 be languages over .
If L1  L2 and L2  L3, then L1  L3.
Proof: Let L1, L2 and L3 be languages over .
There is a function f such that wL1  f(w)L2, and a TM
T1 computing f in polynomial time because L1 L2.
There is a function g such that wL2  g(w)L3, and a
TM T2 computing g in polynomial time because L2  L3.
wL1  f(w)L2  g(f(w))L3, and T1T2 computes
g(f(w)) in polynomial time .
By the definition, L1  L3.
Jaruloj Chongstitvatana
NP-complete Problems
47
Using reduction to prove P/NP
Theorem: If L2 is in P/NP, and L1L2, then L1 is also P/NP.
Proof:
Let L1 and L2 be languages over , L1L2, and L2 be in P/NP.
Because L2 is in P/NP, there is a DTM/NTM T2 accepting L2 in
polynomial time.
Because L1L2, there is a DTM T1 computing a function f such
that wL1  f(w)L2 in polynomial time.
Construct a DTM/NTM T=T1T2. We show that T accepts L1
in polynomial time.
 If wL1, T1 in T computes f(w)L2 and T2 in T accepts f(w), in
polynomial time.

If wL1, T1 in T computes f(w) L2 and T2 in T does not
accept f(w) in polynomial time.
Thus, L1 is also in P/NP.
Jaruloj Chongstitvatana
NP-complete Problems
48
Using reduction to prove co-NP.
Theorem:
If L2 is in co-NP, and L1L2, then L1 is also in co-NP.
Proof:
Let L1 and L2 be languages over , L1L2, and L2 be
in co-NP.
Because L2 is in co-NP,L2 is in NP.
Because L1L2,L1L2. Then,L1 is in NP.
Thus, L1 is co-NP.
Jaruloj Chongstitvatana
NP-complete Problems
49
NP-completeness
NP-completeness

A language (or problem) L is NP-hard if, for
any language L' in NP, L' ≤ L.

A language (or problem) L is NP-complete if
it is in NP and is NP-hard.
Jaruloj Chongstitvatana
NP-complete Problems
51
Why interested in NP-complete problem
Implication on the problem “is P=NP ?”
 If an NP-complete problem is in P, then
P=NP.
 Why?




If L is NP-complete, any language in NP is
reducible to L.
Since L is in P, any language in NP is reducible to
a language in P.
Then, any language in NP is in P.
Jaruloj Chongstitvatana
NP-complete Problems
52
Lemma
If L is a language such that L’ ≤ L for some
L’ ∈ NPC, then L is NP-hard.
Moreover, if L ∈ NP, then L ∈ NPC.
Proof
Since L’ is NP-complete, for all L’’ ∈ NP, we
have L’’ ≤ L’ .
Because L’ ≤ L, L’’ ≤ L by transitivity.
Then, L is NP-hard.
If L ∈ NP, we also have L ∈ NPC.
Jaruloj Chongstitvatana
NP-complete Problems
53
Ground Reduction
SAT is NP-complete
SAT is NP-complete

To prove that SAT is NP-complete:



Prove SAT is in NP
Prove any language in NP is reducible to SAT.
SAT is in NP: Informal proof

Given a certificate which is a truth assignment of
propositions which makes the expression true,
there is an algorithm that can verify the answer
in polynomial time.
Jaruloj Chongstitvatana
NP-complete Problems
55
SAT is NP-complete (cont’d)
Prove any language in NP is reducible to SAT.
Proof:
Let L be any language in NP.
Then, there is an NTM T which accepts L in
polynomial time p(n), where n is the input size.
To show that L ≤ SAT, we find a function f such that
x L  f(x)  SAT.
That is, to find a function f such that x L  f(x) is
satisfiable.
Jaruloj Chongstitvatana
NP-complete Problems
56
SAT is NP-complete (cont’d)
Proof (cont’d):
That is, to find a function f such that T accepts x  f (x) is
satisfiable.
Let f (x)=g1(x)g2(x)g3(x)g4(x)g5(x)g6(x)g7(x).
f (x) describes the sequence of configuration changes which
leads to the halt state of T.
f(x) contains propositions Qi,j, Hi,k, and Si,k,j .
Qi,j: T is in state qj after i moves.
Hi,k: T’s tape head is on square k after i moves.
Si,k,j: Symbol j is on square k after i moves.
Jaruloj Chongstitvatana
NP-complete Problems
57
SAT is NP-complete (cont’d)
Proof (cont’d):
g1(x): the initial configuration of T at the beginning.
g2(x): accepting configuration of T after N steps.
g3(x): possible transitions of T.
g4(x): hanging configuration.
g5(x): describe that T is in exactly one state at a
time.
g6(x): describe that each square of the tape contains
exactly one symbol.
g7(x): describe that change can only made on the
tape square on which the tape head is located.
Jaruloj Chongstitvatana
NP-complete Problems
58
SAT is NP-complete (cont’d)
Proof (cont’d):
g1(x): describe the initial configuration of T at
the beginning.
n
Q0,0  H0,0  S0,0,0   S0,k, ik 
k=1
N
 S0,k, 0
k=n+1
∆ 0 1 1 0 1 ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆
q0
Jaruloj Chongstitvatana
NP-complete Problems
59
SAT is NP-complete (cont’d)
Proof (cont’d):
g2(x): describe accepting configuration of T
after N steps.
QN,a
ha
Jaruloj Chongstitvatana
NP-complete Problems
60
SAT is NP-complete (cont’d)
Proof (cont’d):
g3(x): describe possible transitions of T.
For each transition
δ(qj, l)= (qt1, z1, D1), δ(qj, l)= (qt2, z2,
D2)
(Qi,j Hi,k Si,k,l)((Qi+1,t1  Hi+1,d1(k)  Si+1,k,z1)
V (Qi+1,t2  Hi+1,d2(k)  Si+1,k,z2))
Jaruloj Chongstitvatana
NP-complete Problems
61
SAT is NP-complete (cont’d)
Proof (cont’d):
g4(x): describe hanging configuration.
For each transition δ(qj, l)= ø
(Qi,j  Hi,k  Si,k,l)  (Qi+1,j  Hi+1,k  Si+1,k,l)
Jaruloj Chongstitvatana
NP-complete Problems
62
SAT is NP-complete (cont’d)
Proof (cont’d):
g5(x): T is in exactly one state at a time.
For each time i and states qj0, qj1, qj2, …, qj0 ≠
qj1 ≠ qj2 ≠ … :
(Qi,j0 v Qi,j1 v Qi,j2 v …)

At least one state
(~Qi,j0v~Qi,j1)(~Qi,j1v~Qi,j2)(~Qi,j0v~Qi,j2)…

Never two state at the same time
Jaruloj Chongstitvatana
NP-complete Problems
63
SAT is NP-complete (cont’d)
Proof (cont’d):
g6(x): state that each square of the tape
contains exactly one symbol.
For each time i, square k, and symbols 0, 1,
2, …, 0 ≠ 1 ≠ 2 ≠ … :
(Si,k,0 v Si,k,1 v Si,k,2 v …)

At least one symbol in a cell
(~Si,k,0v~Si,k,1)  (~Si,k,1v~Si,k,2) 
(~Si,k,0v~Si,k,2) …

Never two symbols in a cell at the same time
Jaruloj Chongstitvatana
NP-complete Problems
64
SAT is NP-complete (cont’d)
Proof (cont’d):
g7(x): state that change can only made on
the tape square on which the tape head is
located.
For each i, k, and l :
(~Hi,k  Si,k,l)  (Si+1,k,l)
Hi,k v ~Si,k,l v Si+1,k,l
Jaruloj Chongstitvatana
NP-complete Problems
65
SAT is NP-complete (cont’d)
Proof (cont’d):
From the definition of f, f(x) is true iff T
accepts x.
f(x) can be computed in polynomial time.
Thus, SAT is NP-complete.
Jaruloj Chongstitvatana
NP-complete Problems
66
3CNF-SAT is NP-complete
First, show 3CNF-SAT is in NP.
Let  be a 3CNF expression, and
truth assignments C be a certificate for .
There is an algorithm to verify that  is true
under the assignments C, by substituting
the value of each literal and evaluating the
expression.
This algorithm takes polynomial time of the
length of .
Jaruloj Chongstitvatana
NP-complete Problems
67
3CNF-SAT is NP-complete
Second, show SAT ≤ 3CNF-SAT.
Let  be a Boolean expression.
We will construct a 3CNF expression f() such that
f() is satisfiable iff  is satisfiable.
To create f():
(A) Create an abstract syntax tree of  so that
internal nodes are logical operators , v, ~, , or
 , and leaf nodes are literals (x or ~x).
Then, create , from the abstract syntax tree which
is satisfiable iff  is satisfiable.
Jaruloj Chongstitvatana
NP-complete Problems
68
3CNF-SAT is NP-complete
 = ((p  q)  (((~s v r)  (q  ~r)) v ~s)
:y1
:y2
p
 (y1  (y2  y3))
 (y2  (p  q ))
 (y3  (y4 v ~s))
 (y4  (y5  y6))
 (y5  (~s v r ))
 (y6  (q  ~r))
 is satisfiable iff  is satisfiable.
v:y3
q
~s
v:y5
 = y1
Jaruloj Chongstitvatana
:y4
~s
NP-complete Problems
:y6
r
q
~r
69
3CNF-SAT is NP-complete
(B) Convert  into CNF.
P  Q is transformed to (P  Q)  (Q  P).
PQ
is transformed to ~P v Q.
~(P v Q) is transformed to ~P  ~Q.
~(P  Q) is transformed to ~P v ~Q.
 = (y1  (y2  y3)) =(y1  (y2  y3))  ((y2  y3)  y1)
=(~y1 v (y2  y3))  (~y2 v ~y3 v y1)
= (~y1 v y2)  (~y1 v y3)  (y1 v ~y2 v ~y3)
= (~y1 v y2 v y3)  (~y1 v y2 v ~y3)  (~y1 v y2 v y3)
 (~y1 v ~y2 v y3)  (y1 v ~y2 v ~y3)

Jaruloj Chongstitvatana
NP-complete Problems
70
3CNF-SAT is NP-complete
f() = , and f() is satisfiable iff  is satisfiable.
Next, we need to show that f is computable in polynomial time.
In step (A):
the abstract syntax tree can be created in polynomial time of
the length of ,
the expression  can be created also in polynomial time of the
number of node in the tree and the length of , which is also
a polynomial time of the length of .
In step (B):
 can be created, using Boolean logic identities, in polynomial
time of the length of  , which is also a polynomial time of the
length of .
That is, SAT ≤ 3CNF-SAT.
Jaruloj Chongstitvatana
NP-complete Problems
71
CLIQUE is NP-complete
Proof:
3CNF-SAT is NP-complete.
3CNF-SAT ≤ CLIQUE.
CLIQUE is NP-complete.
Jaruloj Chongstitvatana
NP-complete Problems
72