Approximability
& Proof Complexity
Ryan O’Donnell
Carnegie Mellon
Approximability
& Proof Complexity
of optimization problems
Ryan O’Donnell
Carnegie Mellon
Minimum Vertex-Cover
MIN-VC(G)
= min {|S| : S ⊆ V, S touches all edges of E}
Minimum Vertex-Cover
MIN-VC(G)
= min {|S| : S ⊆ V, S touches all edges of E}
“2-approximating” Min-VC
• Choose any maximal matching M ⊆ E
• MIN-VC(G) ≥ |M|
• Let S = {all endpoints in M}.
It’s a vertex-cover (why?)
satisfying
|S| = 2|M| ≤ 2MIN-VC(G)
“Factor 2-certifying” Min-VC
• Choose any maximal matching M ⊆ E
• Output “MIN-VC(G) ≥ |M|”
A “factor α-certification” algorithm:
1. Output bound is always correct.
2. Bound is always within factor α of truth.
Linear programming (LP) relaxation
k = minimize: ∑v∈V Xv
subject to:
0 ≤ Xv ≤ 1
for all v∈V
Xu + Xv ≥ 1 for all (u,v)∈E
Output “MIN-VC(G) ≥ k”
Matching algorithm, LP algorithm are both
factor 2-certification algorithms.
Are they also 1.99-certification algorithms?
No.
MIN-VC(Kn) = n−1
maximal |M| = n/2
LP value = n/2
Kn
Matching algorithm, LP algorithm are both
factor 2-certification algorithms.
IsAre
there
theyany
also
poly-time
1.99-certification
1.99-certification
algorithms?
alg?
No.
MIN-VC(Kn) = n−1
maximal |M| = n/2
LP value = n/2
Kn
Is there any poly-time 1.99-certification alg?
We don’t know. Best we know is:
∃ 1.36-certification alg (unless P=NP)
[Dinur-Safra’02]
Approximability &
Proof Complexity
Refutes statements encoded…
Resolution
Cutting planes
Nullstellensatz/
Polynomial Calculus
by boolean disjunctions
by integer inequalities
by polynomial equations
[BIKPP’96,CEI’96]
Positivstellensatz/
Sum-of-Squares (SOS)
by polynomial inequalities
[Grigoriev-Vorobjov’99]
ZFC (“Frege”)
in ordinary math language
Positivstellensatz
[Krivine’64,Stengle’73,Schmüdgen’91,Putinar’93,Wörmann’98]
assuming a mild
technical condition
is infeasible if
and only if
there exist “certifying polynomials” Q0, …, Qm,
each a sum of squares, s.t. we have the identity
−1 = Q0 + Q1P1 + Q2P2+ ••• +QmPm
“MIN-VC(G) > k”
⇔
Xv2 ≥ Xv
Xv2 ≤ Xv
Xu+Xv ≥ 1
for all v∈V
for all (u,v)∈E
infeasible
∑v Xv ≤ k
⇔ ∃ certifying SOS polynomials Q such that
−1 = Q0 + Q1 (k−∑ Xv) + ∑ Quv (Xu+Xv−1) + •••
Positivstellensatz / SOS
proof system
Suggested by Grigoriev and Vorobjov in 1999.
The complexity of an SOS proof is the
maximum degree of any QiPi or Q0.
SOSd denotes the proof system
restricted to degree d.
(No longer complete.)
Example proof
Theorem:
The following system is infeasible:
{X2 ≤ 1, Y2 ≤ 1, Z2 ≤ 1,
XY+YZ+ZX ≤ −2}.
ZFC proof: Let f(X,Y,Z) = XY+YZ+ZX.
Suppose X2 ≤ 1; i.e., X ∈ [−1,1].
Since f is linear in X, it’s maximized if X = ±1.
Similarly for Y and Z.
Suffices to show f(±1,±1,±1) > −2.
If all three inputs same, f is 3.
If not all three inputs same, f is −1.
Example proof
Theorem:
The following system is infeasible:
{X2 ≤ 1, Y2 ≤ 1, Z2 ≤ 1,
XY+YZ+ZX ≤ −2}.
SOSd=4 proof:
+
Exercise
Show SOSd=4 certifies MIN-VC(Kn) ≥ n−1
(i.e., refutes MIN-VC(Kn) ≤ n−2).
SOSd is ‘automatizable’!
Theorem:
[Lasserre’00,Parrilo’00, cf. N.Shor’87]
If a polynomial inequality system can
be refuted in the SOSd proof system,
the certifying Qi’s can be found in
nO(d) time (using semidefinite programming).
The strongest(?) automatizable
proof system that we know
SOSd is stronger than:
Width-d Resolution
Degree-d Nullstellensatz
Basic LP relaxations
Basic SDP relaxations
d/2 rounds of Lovász-Schrijver LP/SDP hierarchy
d/2 rounds of Sherali-Adams LP/SDP hierarchy
(Doesn’t seem to be stronger than degree-d Polynomial Calculus.)
A very powerful poly-time algorithm
for Vertex-Cover certification:
Output the largest k ∈ [n] such that
SOSd=1000 certifies “MIN-VC(G) > k”.
Could this be a 1.99-certification algorithm?
I.e., is it true that whenever MIN-VC(G) = β,
∃ degree-1000 Qi’s certifying Min-VC(G) ≥ β/1.99?
Partial history of upper
and lower bounds for SOSd
SOSd upper bounds, 2001-2011
Nothing that we didn’t already
know by other means.
E.g., SDP is a .878-certification alg for Max-Cut
∵ SDP ≤ SOSd=4 ∴ SOSd=4 also .878-certifies Max-Cut
SOSd lower bounds, 1999-2009
Tseitin Tautologies / 3Lin(mod 2)
Consider a random system of O(n)
3-variable equations modulo 2.
[Grigoriev’99]:
(indep. [Schoenbeck’08])
With very high probability…
• No assignment sats > 51% of equations
• Unless d = Ω(n), SOSd cannot refute
“the system is totally satisfiable”.
‘Knapsack’
“If n is odd and X1, …, Xn satisfy Xi2 = 1,
then X1 + ••• + Xn cannot be 0.”
(Essentially equivalent: “MAX-CUT(Kn) ≥
[Grigoriev’01]:
See also [Laurent’02],
[Cheung’05]
”)
Not provable in SOSd
unless d ≥ n+1.
Open Problem: Give a pleasant proof that
d needs to be at least, say, 6.
[Tulsiani’09] Rule of Thumb
(A corollary of the 3Lin(mod 2) lower bound.)
(Not rigorously proven, but seems true in all cases.)
For any factor-α certification problem
which we know is NP-hard,
there exists instances which require
degree-nΩ(1) SOS proofs.
A very powerful poly-time algorithm
for Vertex-Cover certification:
Output the largest k ∈ [n] such that
SOSd=1000 certifies “MIN-VC(G) > k”.
Could this be a 1.99-certification algorithm?
Integrality Gaps
[GK’95] SDP does not 2-certify Min-VC:
Frankl-Rödl graphs
[FS’00] SDP does not .879-certify Max-Cut:
Noisy-sphere graphs
[KV’05] SDP+∆ does not .879-certify Max-Cut
or solve Unique-Games:
KV noisy-hypercube graphs
[DKSV’06] SDP+∆ does not O(1)-certify
Sparsest-Cut (Balanced-Separator):
DKSV noisy-hypercube
graphs
[KS’09,RS’09] Sherali-Adams+, degree-O(1),
does not .879-certify Max-Cut
or solve Unique-Games:
KV noisy-hypercube graphs
[BCGM’11] Sherali-Adams+, degree-6,
(and probably degree-O(1))
does not 2-certify Min-VC
Frankl-Rödl graphs
These tricky instances
aren’t so hard for SOS!
[BBHKSZ’12]:
SOSd=4 solves the KV
Unique-Games instances!
[OZ’13]:
SOSd=4 solves the DKSV
Balanced-Separator instances.
[OZ’13]:
SOSd=O(1) .95-certifies the KV
Max-Cut instances.
[BBHKSZ’12]:
SOSd=4 solves the KV
Unique-Games instances!
[OZ’13]:
SOSd=4 solves the DKSV
Balanced-Separator instances.
[DMN’13]: SOSd=O(1) solves the KV
Max-Cut instances.
• [KV’05]:
used ZFC to show
• [KS’09,RS’09]: SA+d=O(1) only certify
• [DMN’13]
SOSd certifies
“MAX-CUT(KV) ≈ 85%”
“MAX-CUT(KV) ≥ 75%”
“MAX-CUT(KV) ≥ 85% − od(1)”
[BBHKSZ’12]:
SOSd=4 solves the KV
Unique-Games instances!
[OZ’13]:
SOSd=4 solves the DKSV
Balanced-Separator instances.
[DMN’13]:
SOSd=O(1) solves the KV
Max-Cut instances.
[KOTZ’13]: SOSd=O(1) solves “most of”
the Frankl-Rödl Min-VC
instances.
The whole result is just that
one particular algorithm does
well on one particular instance?
I have 3 responses. 
Response 1: an old joke
Q:
Why did the complexity theorist
work on algorithms?
A:
To get lower bounds on his lower bounds.
We basically no longer know any “hard instances”.
Response 2: Evidence for
algorithmic optimism?
[BBHKSZ+’13] points out that as far as we know,
SOSd=4 solves the Unique-Games
problem (i.e., refutes the UGC).
Perhaps SOSd is the killer algorithm
for combinatorial optimization.
Response 3: New proofs
Proving known theorems in restricted proof
systems can lead to new insights and proofs.
[Razborov’93]:
New Switching Lemma proof
[BBHKSZ’12,
OZ’13,KOTZ’13]:
Hypercontractivity insights
[BHM’12,KOTZ’13]: New Frankl-Rödl Thm. proof
[MN’13,DMN’13]: New Maj. is Stablest proof
Let’s take stock
• Approximation Algs ≤ Proof Complexity:
“Is there an efficient algorithm for
100-coloring a 3-colorable graph?”
Let’s take stock
• Approximation Algs ≤ Proof Complexity:
“Given a graph that is not 100-colorable,
how hard is it to prove that it’s not 3-colorable?”
• SOSd is a quirky yet strong proof system,
automatizable in time nO(d).
• SOSd=O(1) solves all the trickiest instances
we know of Unique-Games, Max-Cut.
Three closing thoughts
regarding proof complexity
1. Frankl-Rödl graphs and SOS lower bounds
2. The Dynamic SOS proof system
3. My favorite algorithm for Unique-Games
Frankl-Rödl graphs
FRm(γ):
V = {0,1}m
E = {(x,y) : Δ(x,y) ≥ (1−γ) m}
[Frankl-Rödl’87]:
MIN-VC(FRm(γ)) ≥
[KOTZ’13]:
(1−o(1))
2m
SOSd=O(1/γ) can prove this.
But perhaps SOSd=O(1) cannot handle
A simpler open problem
Theorem:
(a corollary of [Harper’66]’s Vertex-Isoperimetric Inequality)
Let A, B ⊆ {0,1}m satisfy dist(A,B) ≥
Then |A|, |B| aren’t both large:
Conjecture: SOSd=4 cannot prove this.
Dynamic SOS
• Lines of the proof are of form P(X1, …, Xn) ≥ 0.
• From P ≥ 0 and Q ≥ 0 can derive
P + Q ≥ 0 and P • Q ≥ 0.
• Can always derive R2 ≥ 0.
• To refute a system {P1 ≥ 0, …, Pm ≥ 0},
derive −1 ≥ 0.
Complexity: max degree of any line
Dynamic SOS
Facts
[Grigoriev-Hirsch-Pasechnik’01]:
• Dynamic SOSd=3 refutes Knapsack
• Dynamic SOSd=5 refutes any unsatisfiable
3XOR instance
Open problem 1 [GHP’01]:
Suggest an explicit unsatisfiable boolean
formula which SOSd=O(1) might not refute.
Open problem 2:
Give negative evidence re automatizability.
Unique-Games
[Khot’02] conjectured that for the “UG” CSP, it’s
NP-hard to distinguish ϵ-satisfiable instances
from (1−ϵ)-satisfiable instances.
[BBHKSZ’12]: Perhaps SOSd=4 can actually do it.
⇒ UGC is false (assuming NP ≠ P)
Perhaps SOSd=log(n) can do it.
⇒ UGC is false (assuming NP ⊈ TIME[nlog n])
Why be so concerned about automatizability?
Unique-Games
[Khot’02] conjectured that for the “UG” CSP, it’s
NP-hard to distinguish ϵ-satisfiable instances
from (1−ϵ)-satisfiable instances.
My favorite UG algorithm:
Given an ϵ-satisfiable instance,
nondeterministically guess a poly-length
ZFC proof that instance is ≤ (1−ϵ)-satisfiable.
If this algorithm works, UGC is false.
(assuming NP ≠ coNP)
ありがとうございました！
Thank you!