MaxClique
Inapproximability
Seminar on HARDNESS OF APPROXIMATION PROBLEMS
by Dr. Irit Dinur
Presented by Rica Gonen
MaxClique
Inapproximability using PCP
• This talk will present the MaxClique
inapproximability proved by Subhash Khot.
• The inapproximability result uses
– a PCP verifier for 3SAT
– to determine the size of a clique in graph G
with N’ vertices
– where G is the product graph of a randomized
reduction from 3SAT to graph G.
Talk Outline
• MaxClique inapproximability – results’ history.
• Khot result - general structure.
• Technical background.
• Khot verifier - construction and proof.
• MaxClique inapproximability using Khot
verifier.
MaxClique
Inapproximability –
Results’ History.
• On the positive side
– The best (known) polynomial time approximation achieves
an approximation ratio of  n 
1o 1
– It is of the form n  
O

  log n 2 


• On the negative side
– The first step towards proving strong inapproximability
was taken by Feige et al.
– They showed a connection between Probability
Checkable Proof Systems and inapproximability of
MaxClique.
– The discovery of the PCP theorem by Arora et al implied
that MaxClique is inapproximable within a factor n c for
some constant c > 0 unless P = NP.
• On the negative side (cont)
– Bellare and Sudan defined an important parameter of
PCP called amortized free bit complexity and showed
that
– Theorem: if NP has probabilistically checkable proofs
where the verifier uses logarithmic randomness and f
amortized free bits, then MaxClique is inapproximable in
1

1 f
polynomial time with a factor n
unless NP=ZPP.
for any constant  > 0
– They constructed PCPs with 3 amortized
free bids and
1
obtained a hardness factor of n 4  for clique.
– It was improved by Bellare et all to n
1

3
.
• On the negative side (cont)
– Hastad proved an n1 inapproximability factor. He obtain
a PCP verifier that achieves amortized free bit
complexity f for arbitrarily small constant f >0.
– Khot paper aim at getting the best subconstant value of 
in Hastad’s result.
Khot’s Result –
General Structure.
Approximating MAX-Clique is NP-hard
PCP Theorem: 3SAT  PCP  r , q 
c, s
We will reduce 3SAT to MAX-Clique.
 = { j1, ..., jl } of clauses over variables y1,...,ym of range 2V,
Each of j1, ..., jl depends on at most 3 variables.
The verifier V uses r  log m random
bits
and
chooses a
*
*
*
uniformly random clause j j  yi  yk  yl of  (* denotes
the variable or its compliment), and query q bits of the
proof.
The proof  corresponds to an assignment of the variables
appearing in j j and accepts iff the values satisfy j j
If  is SAT then
 Pr Ver _ acc   c
r
If  is unSAT then
 Pr Ver _ acc   s
r
Approximating MAX-Clique is NP-hard
We will construct a graph, G , that:
• has a clique of size at least c  there exists an
assignment, satisfying the constraints y1,...,ym that
verifier V accepts with probability  c
• has a clique of size at most s  there does not
exists an assignment, satisfying the constraints
y1,...,ym that verifier V accepts with probability  s
( 2 , 2 )-co-partite Graph G=(RQ, E)
r
q
• Comprise 2r independent sets of size < 2 q
i  R, j1 , j2  Q ( i, j1 ,  i, j2 )  E
2^r
Clique
Instance: an ( 2r , 2 q)-co-partite
graph G=(RQ, E)
Problem: distinguish between
– Good: CL(G) = c
– Bad: every set w  V s.t. |w|>
s is not a clique.
2^r
3SAT  MAX-Clique
Construct a graph G that has 1 independent set ji  ,
randomly chosen by the verifier V,
in which 1 vertex  assignment for jI accepted by the verifier V.


j1
y1
T
y2
jj
yi
T
F
yF
T
m-1
jl
ym
2^r
3SAT  MAX-Clique
Two vertices are connected iff the assignments they
represent are consistent
j1
T
y1
y2
T
jj
T
F
yi
T
F
T
ym-1
jl
ym
T
2^r
3SAT  MAX-Clique
Lemma:
If  is SAT, clique of size at least c  verifier V accepts
with probability  c
 Consider an assignment A that was accepted by the
verifier V with probability  c . It satisfies at least c
clauses. For each clause i consider A's restriction to ji‘s
variables
The corresponding c vertexes form a clique in G
 Any clique of size c in G implies an assignment
satisfying c clauses accepted by the verifier V with
probability  c.
3SAT  MAX-Clique
Lemma:
If  is unSAT, clique of size at most s  verifier V accepts
with probability  s
 Consider an assignment A that was accepted by the
verifier V with probability  s . It satisfies at most s
clauses. For each clause i consider A's restriction to ji‘s
variables
The corresponding s vertexes form a clique in G
 Any clique of size s in G implies an assignment
satisfying s clauses accepted by the verifier V with
probability  s .
Hence: MAX-Clique is NP hard, and MAXClique is NP-hard to approximate!
Khot Constructs a PCP verifier for 3SAT
• Theorem B.1  (Engebretsen and Holmerin):
if there is a PCP verifier for 3SAT using r random
bits, f free query bits, completeness c and
soundness s, then there is a randomized reduction
rf
from 3SAT to a graph G with
vertices
r
log1/ s 
'
N 2
such that:
– If the 3sat formula is satisfiable with
probability 2/3, G has a clique of size at least c
– If the formula is unsatisfiable with probability
2/3, maximum clique size in G is at most 2r.
r 2r
log 1/ s 
Khot wants to use Hadamard code in
the PCP verifier
• Hastad’s result uses long code encodings.
• Long coed encodes a u-bit string by a
2
2u-bit
string.
• To improve Hastad’s result Khot uses Hadamard code.
• Hadamard code encodes a u-bit string by a
allows randomness efficient checking.
2u-bit string and
• Hadamard codes are define using linear functions
• Khot needs an underlying NP-hard problem that features
linear constraints.
• He defines such a problem called Max-3Lin( ).
The hardness of Max-3Lin( )
Stated according to Hastad theorem:
• Theorem 2.1:
There exist a polynomial time reduction from a SAT
formula  with n variables to a system L modulo 2
with N variables such that:
– If is satisfiable, there exist an assignment to
the variables in L that satisfies 1- fraction of
equations.
– If is unsatisfiable, no assignment can satisfy
more than 1/2+ fraction of the equations.
– Every equation contains exactly 3 variables and every variable appears in
exactly 7 equations.
• Moreover, the reduction can achieve 
for
some constant  >0 if we allow the running time
of the reduction and N to be slightly
superpolynomial , i.e. n O (log log n )
= 1/  log N 

Khot’s PCP Verifier construction
makes use of the Raz Verifier.
• Khot’s PCP Verifier expects a (Hadamard)
encoding of the proof supplied to the Raz
Verifier.
• Details about the Raz Verifier in the Technical
background section.
Technical background
The Raz Verifier
• The Raz Verifier is given an instance L of Max3Lin().
• It expects two proofs P and Q.
• For every set U of u variable, P(U) is a u-bit string
giving the values of those variables in some global
assignment.
• For every set W of u equations, Q(W) is a 3u-bit
string giving the values of the 3u variables
appearing in these u equations.
The Raz Verifier works as follows:
u
x
• It randomly picks variables U= i i 1
• It picks equations W= Ci i 1 where equation Ci is
chosen randomly from equations containing variable xi
u
• The verifier accepts iff
u
– Q(W) satisfies all the equations  Ci i 1
(the linear constraints test)
– P(U)=  (Q(W)), where  is the projection from
3u-bit strings to u-bit
strings i.e. the values of
u
the variables  xi i 1 in P(U) and Q(W) are the
same.
(the projection test).
Completeness of the Raz Verifier
• Completeness is  1     1   u .
u
• If there is an assignment that satisfies 1  
fraction of equations, both P and Q are consistent
with this assignment.
• With probability 1    , all the equations
u
C
 i i 1 will be satisfied and the verifier will accept.
u
Soundness of the Raz Verifier
• When at most 1/2+ fraction of equations in L are
satisfiable, the soundness can be upper bounded
by Raz’s Parallel Repetition Theorem.
• Theorem 2.2:
There exist an absolute constant Clin< 1 such that
soundness of the Raz Verifier for
u
Max-3Lin() is at most Clin
Hadamard Codes
u
p

F
• Hadamard(p) : Hadamard code of
2 is the
2u -bit string  p  a aF .
ap
–  p  a    1 returns 1 or -1 for every vector a.
u
2
u
– F2 has
2u vectors.
Khot verifier construction and proof.
The Construction of
Khot’s PCP Verifier (Vlin)
• Vlin is given an instance L of Max-3Lin( ).
• Vlin expects proofs P’,Q’ in Hadamard codes.
– P’,Q’ are encodings of proofs P,Q
– P,Q are supplied to the Raz Verifier.
• Vlin picks a set U of u variables at random.
k
• Vlin picks k sets Wj  independently.
j 1
– Each set is picked such that it has u equations and every
variable appears at least ones in the set.
The Construction of
Khot’s PCP Verifier (Vlin) (cont)
• Let A be Hadamard code of P(U)
• Let B j be Hadamard code of Q W j  .
– Tables B j are assumed to be folded over respective
linear constraints.
3u
• Vlin pickes a1 ,..., ak  F2u and b1 ,..., bk  F2 randomly.
• Vlin accepts iff for 1  i, j  k
A  ai  B j  b j   B j  j 1  ai   b j 
–  j is the projection function between W j and U.
Ignoring the linear constraints test
• If Khot’s PCP verifier accepts the encoded proofs
with a good probability, then these proofs can be
decoded to construct proofs (P,Q) which the Raz
Verifier accepts with a good probability.
• Folding ensures that the decoding procedure
satisfies the linear constraints on W.
• Thus linear constraint test can be ignored.
Folding
• Let the string x=Q(W) read by the Raz Verifier
satisfy the linear constrains modulo 2,
h1  x   1 ,..., hu  x   u
3u
– h1 ,..., hu  F2
–  1 ,...,  u  F2
• Let B be Hadamard code of x.
• Let H be linear subspace spanned by the vectors
h1 ,..., hu
• Let h  H , h   i hi
• For b  vb i i hi , 1 ,..., u  F2
• (1): B '  b   B  vb    1i i i
– vb denotes the lexicographically smallest
vector in the set of vectors b  H
• B’ is a folding of B over the linear constrains.
Folding (cont)
• Decoding of a table B gives  with probability B̂2 .
• Folding ensures that any  given by this decoding
procedure satisfies the linear constraints on W.
• Lemma 2.4:
if Bˆ '  0 , then  must satisfy the linear
constraints, i.e. hi     i i .
Folding (cont)
• It will be required that the supposed Hadamard
codes be folded over the respective constraints.
• This requirement can be enforced using the
following access mechanism.
– When the verifier wants to read B(b),
– it reads B(vb) instead
– And “calculates” the value of B(b) from (1).
Vlin Analysis
• Theorem 3.1:
The Verifier Vlin for Max-3Lin( ) instance L with N
variables
– Uses r=ulogN +O(ku) random bits.
– Queries 2k  k 2 bits from the proof with f=2k
free bits.
– Has completeness at least 1   ku
– Has soundness 2
k2
  provided Clinu   2
Analysis of random bits and
queries
• Vlin picks u variables which are ulogN bits at random
• Vlin picks uk equations which are O(ku) bits at random.
 
• Queries A  ai  k bits and B j b j k bits giving 2k free
bits
• Queries k 2 bits projection test.
Vlin completeness
• Assume that there is an assignment that satisfies
1-  fraction of equations.
• Proofs P,Q are consistent with this assignment.
• And encoded with correct Hadamard codes to
construct proofs P’,Q’.
• With probability 1-  a single equation can be
satisfied.
• With probability 1     1   ku, all the ku
k will be satisfied.
equations in the sets
ku
W 
j
j 1
Vlin completeness (cont)
• With correct Hadamard codes,
• A  ai    1
ai P  u 
B j  b j    1
  a   b    1
• Bj 
1
j
i
 
b jQ W j
j
 
b jQ W j
 j 1  ai Q W j 
  1
– Since a projection function maintains the parity
  
1
   a
 1
•  1
 
 a     1
 1  a 
  1 
  1  a 

      a 
    1b QW    1a  QW   B  b   A  a 
j
j
i
b j Q W j  j 1  ai Q W j
j
j
    P U 
• Since  j Q W j
i
j
j
in a correct proof.
MaxClique
inapproximability using
Khot verifier.
Inapproximability for MaxClique
• Theorem 1.2:
It is not possible to approximate MaxClique in
polynomial time within a factor n for some
constant   0 unless
2 log n 1

NP  ZPTIME 2
 log n O1

Vlin
• To prove Theorem 1.2
– Use verifier Vlin from Theorem 3.1
• With superconstant values of u,k
• And subconstant value of 
• As given by Theorem 2.1
max-3Lin
– And apply Theorem B.1.
max-3sat maxclique
max-3Lin
max-3sat
Inapproximability for MaxClique (cont)
• The gap between maximum clique sizes in
N'
Theorem B.1 is
2 log N '1
– For some constant   0
– N’ is the size of the graph produced by the
reduction in Theorem B.1.
If you are still awake…
Soundness and Technical
Background
Fourier Transforms
• Let A : F2u  1, 1
• Function A is called linear if A  x  y   A  x   A  y  .
• for every   F2u, there is a function  defined by
  a    1
– There are
functions.
a
a  F2u
2u vectors in F2u and therefore 2u linear
• Define an inner product on this space as
1
 A1 , A2  u  A1  a   A2  a 
2 aF2u
Fourier Transforms (cont)
• The set of all linear functions forms an
orthonormal basis for this vector space w.r.t. the
above inner product.
– Orthogonal:
a
• A1  a     a    1 1
1
     
a1  2 
• A1 a A2 a  1
• 1  2 is a constant
• In summation over a half elements are odd and half even
a  

• Half elements  1 1 2 =-1
a   
• Half elements  1  1 2  =1
– Orthonormal:
• A1  a  is either 1 or -1. Both squared =1
• 2u functions results in summing 2u 1s
•
It follows that any function A can be
uniquely expressed as A   Aˆ  where Â are
its Fourier coefficients. 
Fourier Transforms (cont)
• A projection function  : F23u  F2u is a function
that maps vectors in F23u to some fixed u
coordinates.
• For   F2 , let  1  a  denote the unique vector c  F23u
such that   c   a and coordinates of c other than
those projected by are 0.
u
Vlin Soundness
• To prove the soundness, by Theorem 2.2, it is
sufficient to show that
k2
– If the soundness is 2  
– Then there exist proofs P,Q
– Which the Raz Verifier accepts with
probability  2 ( Clinu   2).
– According to Theorem 2.2 the soundness of
the Raz Verifier is at most Clinu
– Therefore the soundness of Vlin can not exceed
k2
2 
Vlin Soundness
• According to Samorodnitsky and Trevisan the
acceptance probability of the verifier is given by:
• (2)
1
2
k2

S  k  k 
TS
TS  EU ,W1 ,...,Wk :
a1 ,..., ak ,b1 ,...,bk


1
  A  ai  B j  b j  B j  j  ai   b j  
 i , j S

Vlin Soundness (cont)
• If this probability is  2   , there exist a
nonempty set S   k    k  such that TS   .
2
– The summation at (2) has 2k elements.
– Assume to the contrary that there is no set
S   k    k  such that TS   .
– Then for every set S   k    k , TS  .
k
T

2

– Meaning  S
k 2
2
S  k  k 
– It follows that
1
2
k2

S  k  k 
TS  
– Contradicting our assumption
1
2
k2

S  k  k 
TS  2
k 2

Vlin Soundness (cont)
• Lemma 3.2 by Samorodnitsky and Trevisan
enables us to assume that S is of the form [2]x[d]
– 1 d  k
– Lemma 3.2:
2
T


if TS   for some non-empty set S, then 2d 
for some 1  d  k.
• Two cases of d are considered, even and odd.
The case when S=[2]x[d] and d is even


1
• TS  EU ,W1 ,...,Wk :   A  ai  B j  b j  B j  j  ai   b j  
a1 ,..., ak ,b1 ,...,bk   i , j S




1
1
 E   B j  j  a1   b j B j  j  a2   b j  
(3)
 jd 

a1P  u 
– Since A  a1    1
the power does not depend on j
– (-1) to the power of a1 P  u  even number of times and
therefore equals 1.
a2 P  u 
– Similarly A a2  1
   
The case when S=[2]x[d] and d is even
• Using the following Fourier expansions
B j  j 1  a1   b j    Bˆ j  j   j  j 1  a1   b j  
j
Bˆ      a       b  


j
j
1
j
j
1
j
j
j
– Since A  x  y   A  x   A  y 
  Bˆ j  j      a1     j  b j 
j
j

(4)
j
– Since a projection function maintains the parity
  
 a     1
   a

1
        a 
1
 1  a 
  1 
  1  a 

• Similarly
B j  j 1  a2   b j    Bˆ j j     a2     j  b j  (5)
j
j

j
The case when S=[2]x[d] and d is even
• Substituting (4)and (5) in (3)
TS


1
1
 E   B j  j  a1   b j B j  j  a2   b j  
 jd 



 E    Bˆ j  j      a1     j  b j   Bˆ j j     a2    j  b j  
j
j
j
j
j
 jd   j



 E    Bˆ j  j Bˆ j j       a1     j  b j       a2    j  b j 
j
j
j
j
  j , j , jd  jd 



ˆ
ˆ
   B j  j B j j E         a1         a2      j  j  b j   (6)
j j
j
j j
j
 j , j , j d  j d 
j d 


The case when S=[2]x[d] and d is even
• Taking expectation over b , only if  j   j , j , the
terms in (6) are non-zero. j
2  jbj
  j  j b j
–

b 

 1
 1  1

j d 
 j  j
  
 
j

j d 
j d
–  j   j   a1  and     a2  are the orthonormal
j
– If  j  , j
j
basis vectors.
       a1         a2   0
j
j
j
j
j
j
– (6) will equal 0
 
• Taking expectation over a1 , only if  jd   j  j  0 ,
the terms in (6) are non-zero.
– If 
   0 then
j d  j  j 
 j  j   j a1
0a1
       a1    1
  1  1
j j
j
 
– Similarly if 
 j 0
j d  j
The case when S=[2]x[d] and d is even
• It is concluded that:
2

 TS  EU ,W ,...,W
•
1
k
d


2
ˆ


B


j j
  j : jd   j   j 0 j 1

(7)
The case when S=[2]x[d] and d is odd
TS  EU ,W1 ,...,Wk :
a1 ,..., ak ,b1 ,...,bk


1
  A  ai  B j  b j  B j  j  ai   b j  
 i , j S



1
1
 E  A  a1  A  a2   B j  j  a1   b j B j  j  a2   b j  
j d 


• Using Fourier expansions of A, B1 ,..., Bd and
similarly to the case where d is even
 2  TS  EU ,W ,...,W
1
k
d


2
ˆ2
ˆ

 (8)
A
B


j j
 ,  j :  jd   j   j 

j 1
Define proofs for the Raz Verifier
• Remainder of the steps in the soundness proof.
• Define proofs for the Raz Verifier as follows:
– For the set W, pick  with probability B̂ 2

– Q W   
• Remainder – folding ensures that this  satisfies the linear
constraints on W.
 
d
– For a set U, pick sets W j
at random
d
j 2
d
– And pick  j
with probability
Bˆ 2
 
j 2
j 2
– If d is even,
• Define P U    dj  2  j   j 
– If d is odd,
• Pick

 with probability
Â2
 
• Define P U     j  2  j  j
d
j j
Vlin Soundness
• The acceptance probability of the Raz Verifier on
these proof is the expressions in (7) and (8).
• There exist at least one choice of proofs P,Q
which is accepted by the Raz Verifier with
probability at least  2 .
Thank You!
Inapproximability for MaxClique (cont)
• Construct a PCP verifier for 3SAT as follows:
– Using Theorem 2.1 transform a given 3SAT formula to an
instance L of Max-3Lin()

1
 log N 

O loglog n
N n
– Using Theorem 3.1 construct a PCP verifier for L

1
u
1
3 / 4
log
N


2
 log N 
 /4
k   log N 
 2
r   log N 
f  2k

1 3  / 4
s  22
c  1/ 2
– Apply Theorem B.1
R  r  log N 
k2
 /4
k 2