Design and Analysis
of Algorithms
Lecture 12
Yoram Moses
June 10, 2010
http://www.ee.technion.ac.il/courses/046002
1
Reductions among
NP-Complete Problems
2
Reminder: NP

NP = all polynomial-time verifiable languages.

Language: L  {0,1}*
Verification relation for L: R  {0,1}*  {0,1}*





If x  L, then there is a “certificate” y s.t. (x,y)  R.
If x  L, then there is no y s.t. (x,y)  R.
R is polynomially bounded if there is a constant c > 0 s.t.
|y| ≤ |x|c for every (x,y)  R.
L is polynomial-time verifiable if it has a verification relation
R that is both


polynomially bounded, and
polynomial-time decidable.
3
Reminder: P vs. NP
Lemma: P  NP
 Biggest open problem of th. computer science:
is P = NP?
P = NP?

Two possibilities:
NP
P = NP

P
Current belief: P  NP
4
Reminder: NP Completeness

A language L is NP complete if both



L  NP and
L is NP-hard
A language L is NP hard if L’ ≤p L holds for all L’  NP.
Lemma:
If L1 is NP hard and L1 ≤p L2, then L2 is NP hard.
Lemma:
If L1 ≤p L2 and L2  P, then L1  P.
Corollary: If one NP hard language is in P, then NP = P.
5
NP-Completeness: the Full Recipe

To show that L is NPC:
 Prove

L  NP
Show L is polynomial-time verifiable
 Select
an NPH problem L’
 Show a polynomial-time reduction f
from L’ to L:
Prove that x  L’ iff f(x)  L
 Show a polynomial-time algorithm to compute f

6
A First NPC Problem
We need a “first NPC problem” to start with.
Theorem [Cook-Levin]: Circuit-SAT is NP Complete.

Proof: next lecture.
Our goal: using reductions, show that many more problems
are NP Complete.
7
Our Reduction Tree
Homework:
Circuit-SAT
SAT

Partition

k-Coloring, k ≥ 3

Others…
3-SAT
Vertex
Cover
Subset
Sum
Clique
Hamiltonian
Cycle
TSP
8
Boolean Circuits

Boolean circuit:
 A directed
acyclic graph (DAG)
 Nodes are also called gates
 n input gates (in-degree = 0)

Each input gate is labeled by a distinct Boolean variable
(denoted by x1,…,xn).
 A single
output line
 Internal gates are labeled by AND / OR / NOT
 AND,OR gates with in-degree ≥ 2
 NOT gate with in-degree = 1
9
Boolean Circuits: Example

x1

x2



output

x3
input
gates

internal
gates
10
Input Assignments

Input assignment: a vector   {0,1}n
a value (i  {0,1} ) to each input variable xi
in x1,…,xn.
 Assigns

Circuit evaluation: a mapping C: {0,1}n  {0,1}
every assignment  to an output bit (1 or 0).
 C() = evaluation of the circuit (output value) on 
 Evaluation is done in topological order, starting from
the input gates.
 Maps
11
Circuit Evaluation: Example
1
1
x1

1

0
0
1
1
x2
1
1

1
0
x3
input
gates
0

1


1
1
1

1
output
1
internal
gates
12
Satisfying Assignments

Satisfying assignment: An input
assignment  for which C() = 1.

Satisfiable circuit: A circuit that has at least
one satisfying assignment. There is a way
to make it “output” 1.
13
Example of an Unsatisfiable Circuit

x1

x2



output

x3
input
gates

internal
gates
14
Circuit-SAT
Circuit-SAT = language of all satisfiable circuits.
Lemma: Circuit-SAT  NP.
 Proof:

Define a relation R: (C,)  R iff



Fact 1: R is a verification relation for Circuit-SAT:



If C  Circuit-SAT, then it has a satisfying assignment ’, and hence
(C,’)  R for this particular ’
If C  Circuit-SAT, then no assignment  satisfies C, and hence
(C,)  R for all 
Fact 2: R is polynomially bounded


C is a valid encoding of a Boolean circuit, and
 is a valid encoding of a satisfying input assignment for C
If  is an assignment for C, then || ≤ |C|
Fact 3: R is polynomial-time decidable

Evaluating C on  takes linear time
15
Cook-Levin Theorem
Theorem [Cook-Levin]:
Circuit-SAT is NP-Hard.

Proof idea:
Show that L ≤p Circuit-SAT for every language L  NP.

For now, we’ll assume the theorem is true:

Circuit-SAT is our “first” NPH problem
16
Boolean Formulae

Boolean formula: a logical expression with





CNF formula: formula of the form:  = C1  C2  …  Cm,
where each clause Cj is an OR of literals.



n Boolean variables x1,…,xn
Logical operators: , , , , , etc.
Parentheses
Ex:  = ((x1  x2)  ((x1  x3)  x4))  x2
Literal: a variable xi or its negation xi
Example: (x1  x2  x3)  (x1  x4)  (x5)
k-CNF formula: a CNF formula in which each clause has
exactly k literals.

Example of a 3-CNF formula: (x1  x2  x3)  (x1  x4  x5)
17
SAT

Truth assignment: a vector   {0,1}n


Assigns a value to each of the n Boolean variables x1,…,xn.
Formula evaluation: a mapping : {0,1}n  {0,1}
Maps every assignment  to a bit (1 or 0).
 () = evaluation of  on 


Example:
 = ((x1  x2)  ((x1  x3)  x4))  x2
  = (0,0,1,1)
 () = ((0  0)  ((0  1)  1))  0 = (1  (1  1))  1 =
(1  0)  1 = 1




Satisfying assignment: an assignment  s.t. () = 1.
Satisfiable formula: a formula that has at least one satisfying
assignment.
SAT = language of all satisfiable Boolean formulae.
18
SAT is NP-Complete
Theorem: SAT is NP-Complete.
Lemma 1: SAT  NP:
 Verification
relation:
R= {(,):  is a satisfying assignment for }.
Lemma 2: Circuit-SAT ≤p SAT

Need to find a mapping f from Boolean circuits to
Boolean formulae s.t.
 For
every circuit C:
C is satisfiable iff  = f(C) is satisfiable
 f is polynomial-time computable
19
Reduction from Circuit-SAT to SAT:
1st Attempt
Main idea: recursive construction of a formula  that
represents C


Associate with each gate v  C a unique variable xv
Start from the output line o


Set  = xo
For each xv   s.t. v is an internal gate, let



v1,…,vk be the in-neighbors of v
op be the label of v
Then replace each xv in  by (op(xv1,…,xvk))
20
Example

x1

x2




x3

21
Example

x1

x2



x10

x3

 = x10
22
Example

x1

x2

x3



x8
x9

x10
x7
 = (x7  x8  x9)
23
Example

x1

x2
x6

x3



x8
x9

x10
x7
 = (x7  x8  (x6  x7))
24
Example
x5

x1

x2
x6

x3



x8
x9

x10
x7
 = (x7  (x5  x6)  (x6  x7))
25
Example
x5

x1

x2
x6

x3



x8
x9

x10
x7
x4
 = ((x1  x2  x4)  (x5  x6)  (x6  (x1  x2  x4)))
26
Example
x5

x1

x2
x6

x3



x8
x9

x10
x7
x4
=
((x1  x2  x3)  ((x1  x2)  (x3)) 
((x3)  (x1  x2  x3)))
27
Circuit-SAT ≤p SAT: 1st Attempt

The above mapping is indeed a reduction
= () for every assignment 
so C is satisfiable iff  is satisfiable
 C()


BUT - this reduction is not polynomial-time
computable
 Common
sub-formulae are generated again and
again.

The formula may have exponential size
28
Bad Example
x1



…

n AND gates
Corresponding formula:
 = (x1  x1  …  x1)
2n occurrences of x1
29
Reduction from Circuit-SAT to SAT:
2nd Attempt
Main idea: associate each internal gate with one
“basic” formula, and take AND over all gates


Associate with each gate v  C a unique
variable xv and the output with xO
For each internal gate v  C let
 v1,…,v be the in-neighbors of v
k
 op be the label of v
 Then  = (x  (op(xv ,…,xv )))
v
v
1
k
30
Example
x5

x1

x2
x6

x3



x8
x9

x10
x7
x4
 = x10  (x4  x3)  (x5  (x1  x2))

(x6  x4)
 (x7  (x1  x2  x4)) 
(x8  (x5  x6))  (x9  (x6  x7))

(x10  (x7  x8  x9))
31
Circuit-SAT ≤p SAT: 2nd Attempt

Let C have n input gates and m internal gates.

Then  = f(C) has m+n variables and m+1 clauses.
Lemma 1: f is polynomial-time computable.
Proof: easy
32
Circuit-SAT ≤p SAT: 2nd Attempt
Lemma 2: f is a reduction from Circuit-SAT to SAT.

 {0,1}n:
assignment for C
g() =   {0,1}m+n: assignment for 
v = evaluation of v on .
   {0,1}m+n:
assignment for 
h() =   {0,1}n:
assignment for C
v  v
Lemma 3: For any circuit C,
 If
 If
C() = 1, then (g()) = 1.
() = 1, then C(h()) = 1.
Conclusion: C is satisfiable iff  is satisfiable.
33
Next: 3SAT is NP-Complete
Theorem: 3SAT is NP-Complete.
Lemma 1: 3SAT  NP:
relation: (y,), where  is a satisfying
assignment for y.
 Verification
Lemma 2: SAT ≤p 3SAT

Need to find a mapping f from general Boolean
formulae to 3CNF Boolean formulae s.t.
every general formula ,  is satisfiable iff y = f() is
satisfiable
 f is polynomial-time computable
 For
34
Reduction from SAT to 3SAT:
Step 1: Generate a binary parse tree for .
((x1  x2)  ((x1  x3)  x4))  x2
 =


x2


x1

x2

x1
x4
x3
35
Reduction from SAT to 3SAT:
Step 2: Add a variable for each internal node in the tree
((x1  x2)  ((x1  x3)  x4))  x2
 =
 y1
y2

 y3
x1
x2
x2

 y5
 y6
x1
y4
x3
x4
36
Reduction from SAT to 3SAT:
Step 3: For each internal node v, create a clause Cv:
Let

op be label(v)
z be var(v)
v1,…,vk be children(v) (k ≤ 2)
z1,…,zk be var(v1),…,var(vk)

Then Cv = (z  op(z1,…,zk))



37
Reduction from SAT to 3SAT:

Step 3: For each internal node v, create a clause Cv
 = ((x1  x2)  ((x1  x3)  x4))  x2

y2

 y3
x1
x2

x2 

y4


 y5


 y6
x1
y1
x3
C1 = (y1  (y2  x2))
C2 = (y2  (y3  y4))
C3 = (y3  (x1  x2))
C4 = (y4  y5)
C5 = (y5  (y6  x4))
C6 = (y6  (x1  x3))
x4
38
Reduction from SAT to 3SAT:
Step 4: Add a clause C0 = y1 and AND all clauses.
 = ((x1  x2)  ((x1  x3)  x4))  x2

y2

 y3
x1
x2
y=
x2

y4
 y5
 y6
x1
y1
x3
x4
y1 
(y1  (y2  x2)) 
(y2  (y3  y4)) 
(y3  (x1  x2)) 
(y4  y5) 
(y5  (y6  x4)) 
(y6  (x1  x3))
39
Reduction from SAT to 3SAT:
Step 5: Transform each clause Ci into CNF.

Write the truth table of Ci.


Write the DNF of Ci.




Note: the truth table has at most 8 rows
DNF: D1  …  Dm, where each Dj is an AND of literals
Take OR on the “0” entries of the table
Each clause has at most 3 literals
Use De-Morgan’s law to obtain CNF of Ci.


At most 8 clauses
Each clause has at most 3 variables
40
Reduction from SAT to 3SAT:
Example: C1 = (y1  (y2  x2))
C1 =
y1 y2
(y1  y2  x2) 
1 1
(y1  y2  x2) 
1 1
(y1  y2  x2) 
1 0
(y1  y2  x2)
1 0
C1 =
0 1
(y1  y2  x2) 
0 1
(y1  y2  x2) 
0 0
(y1  y2  x2) 
(y1  y2  x2)
0 0
x2
(y1  (y2  x2))
1
0
0
1
1
0
1
0
0
1
0
1
0
0
1
1
41
Reduction from SAT to 3SAT:
Step 6: Transform each clause Ci that has < 3 literals into a
clause with 3 literals.


Use two dummy variables p and q.
If Ci = (z  w) (z,w: literals), then replace Ci by:
Ci,1  Ci,2 , where


Ci,1 = (z  w  p) and Ci,2 = (z  w  p)
If Ci = (z) (z: a single literal), then replace Ci by:
Ci,1  Ci,2  Ci,3  Ci,4 , where


Ci,1 = (z  p  q)
Ci,3 = (z  p  q)
and Ci,2 = (z  p  q) and
and Ci,4 = (z  p  q)
42
SAT ≤p 3SAT




Suppose that  has n variables
Parse tree has m ≤ 2n – 2 internal nodes
y has at most 4  8  (m + 1) = O(n) clauses
y has m + n = O(n) variables
Lemma 1: f is polynomial-time computable
Proof: easy
Lemma 2: f is a reduction
Proof: exercise
Conclusion: If SAT is NPC, then so is 3SAT
43
End of Lecture 12
44