AnintroductiontoExtendedFormulationscapturingtheexpressivepowerofLPsandSDPs
SebastianPokutta
GeorgiaInstituteofTechnology
SchoolofIndustrialandSystemsEngineering(ISyE)
AlgorithmsandRandomnessCenter(ARC)
OptimizationWithoutBorders
DedicatedtoYuriNesterov’s 60th Birthday
LesHouches,2/2016
IntroductionandMotivation
Projections candrasticallyincreasethecomplexityofadescription.
Canweinvert theprocesstofind
smallformulations forcomplicatedproblems?
2
IntroductionandMotivation
Thisphenomenon iswellknown(but notunderstood):
1. Quantifier elimination
2. ThequestofPvs.NPisexactlyofthistype:𝒙 ∈ 𝑳 ⇔ ∃𝒚: 𝒇 𝒙, 𝒚 = 𝟏,
3. Fourier-Motzkin elimination leadstoexponential blow-up
Extendedformulations =quantifierelimination backwards
Naturalquestions.
1. Maybeany 0/1polytope haspoly-sizeextendedformulations?
2. ∃ problems inP,howeverdonotadmitsmallformulations?
3. ∃ problems thatadmitgood approximations viasmallSDPsbutnotLPs?
3
IntroductionandMotivation
Howitallstarted…
In86/87,Swartclaimedhecouldprove𝑷 = 𝑵𝑷
How?
Bygiving a(purported) poly-sizelinearprogramfortheTSPproblem.
Theorem. [Yannakakis 88/91]EverysymmetricLPfortheTSPhassize21(3) .
Swart’s LPwassymmetricandofsize𝑝𝑜𝑙𝑦(𝑛) =>itwaswrong.
4
IntroductionandMotivation
However,thatwasnottheendbutthebeginning.
1. [Kaibel,Pashkovich, Theis 10]symmetrycanmakeahuge difference.
2. [Yannakakis 11](20yearsafterhisinitialproof):
3. Hugeinterestinfinallyruling outallLPsof polynomial sizeforTSP.
4. [Fiorini, Massar,P.,Tiwary,deWolf12]TheTSPpolytope hasnosmallLPs.
Thenotion ofLP-complexity(#inequalities) isindependent ofPvs.NP.
=>verystrongindicationsfor Pvs.NP
5
ProblemsandLPs
Disclaimer.SimilarforSDPs,howeverforsimplicity confinetoLPs.
6
ApproximationProblems
Anapproximationproblem𝑷 (maxorminproblem):
𝑆:setoffeasiblesolutions
𝐹:setofconsidered objectivefunctions (for simplicity:nonnegative)
𝜅:completenessguarantee,𝜅(𝑓) ∈ ℝ foreach𝑓 ∈ 𝐹
𝜏:soundness guarantee,𝜏(𝑓) ∈ ℝ foreach𝑓 ∈ 𝐹
Goal.Whenever 𝑓 ∈ 𝐹 withmax 𝑓 𝑠 ≤ 𝜏 𝑓
E∈F
Find:approximatesolution withval ≤ 𝜅(𝑓)
(maxproblem)
Example (exactmin VertexCover):Givenagraph𝐺
𝑆:allvertexcoversofgraph𝐺 (i.e.,subsetsofnodes coveringalledges)
𝐹: allnonnegative weightvectorsonvertices
𝜅, 𝜏:define 𝜅 𝑓 = 𝜏(𝑓) ≔ min 𝑓(𝑠)
E∈F
7
LPscapturingApproximationProblems
Modelof[Chan, Lee,Raghavendra,Steurer13]and[Braun,P.,Zink14]
AnLPformulationofanapproximationproblem𝑷 = 𝑺, 𝑭, 𝜿, 𝝉 isanLP𝐴𝑥 ≤ 𝑏
with𝑥 ∈ ℝT andrealizations,where𝐹U = {𝑓 ∈ 𝐹 ∣ max 𝑓 𝑠 ≤ 𝜏 𝑓 }
a)Feasiblesolutions: forevery𝑠 ∈ 𝑆 wehavex Y ∈ ℝT with
𝐴𝑥 E ≤ 𝑏 forall𝑠 ∈ 𝑆,
(relaxation𝑐𝑜𝑛𝑣(𝑥 E ∣ 𝑠 ∈ 𝑆))
b)Objectivefunctions: forevery𝑓 ∈ 𝐹U wehaveanaffine 𝑤 ] : ℝT → ℝ with
𝑤] 𝑥E = 𝑓 𝑠
forall𝑠 ∈ 𝑆,
(linearization thatisexacton𝑆)
c)Achieving (κ, τ)-approximation: forevery𝑓 ∈ 𝐹U
𝑓a = max 𝑤 ] 𝑥 𝐴𝑥 ≤ 𝑏 ≤ 𝜅(𝑓)
8
FormulationComplexity
Approximation
Problem
𝑺, 𝑭, 𝜿, 𝝉
Slackmatrixof
problem
𝑀U (𝑓, 𝑠) = 𝜅(𝑓) − 𝑓(𝑠)
LPfactorization
𝑀U = 𝑇 ⋅ 𝑈 + 𝜇 ⋅ 𝟏
(restr.NMF)
Factorizationtheorem. Let 𝑃 = 𝑆, 𝐹, 𝜅, 𝜏 beaproblem and𝑀 slackmatrixof𝑃
𝑓𝑐 𝑃 = 𝑟𝑎𝑛𝑘pq (𝑀U )
Formulationcomplexity.
•
•
•
•
•
Independent ofPvs.NP
Independent ofaspecific polyhedralrepresentation
DonotliftgivenrepresentationbutconstructtheoptimalLPfromfactorization
Infact:LPistrivial.Construct optimalencodingfromfactorization
Restrictednotionofnonnegativematrixfactorizationtosupport approximations
OptimalLP.
𝑥≥0
withencodings
feasiblesolutions: 𝑥 E ≔ 𝑈E objectivefunctions: 𝑤 ] (𝑥) ≔ 𝜅(𝑓) − 𝜇 𝑓 − 𝑇] ⋅ 𝑥
9
OptimalLPs
non-optimal
forallf
optimal
forsomef
𝑥 ≥0
𝑤 ]t (𝑥) = 𝜅(𝑓u ) − 𝜇 𝑓u − 𝑇]t 𝑥
𝑐𝑜𝑛𝑣{𝑥E ∣ 𝑠 ∈ 𝑆}
𝑤 ]r (𝑥) = 𝜅(𝑓s ) − 𝜇 𝑓s − 𝑇]r 𝑥
10
Lowerboundingtechniques
11
Asimplelowerboundingtechnique
Therectanglecoveringbound.
Consider hypothetical
𝑆 = 𝑇𝑈
(nonnegative rank-𝑟 factorization)
= ∑wxs,…,z 𝑇 w 𝑈w
(sum of𝑟 nonneg. rank-1 matrices)
Takesupport
supp 𝑆 = ⋃wxs,…,z supp 𝑇 w 𝑈w
= ⋃wxs,…,z supp 𝑇 w ×supp(𝑈w )
(union of𝑟 rectangles)
Rectanglecoveringnumber. rc 𝑀 = min 𝑟 ∃𝑟-sizecoverofsupp(𝑀)}
⇒ rk … 𝑀 ≥ rc(𝑀)
12
Thecorrelationpolytope– anexample
[Razborov 92]establishedinthecontextofnondeterministic communication:
Theorem. 𝑟𝑐 UDISJ‹ = 2Œ3
Thisimplies[Fiorini, Massar,P.,Tiwary,deWolf12]
2Œ3 = 𝑟𝑐 UDISJ‹ ≤ 𝑟𝑘… UDISJ‹ ≤ 𝑟𝑘… 𝑀3 ≤ fc(COR 𝑛 )
Hardnessofapproximation via
1. Communication complexity[Braun,Fiorini, P.,Steurer12]and
2. Information Theory[Braverman,Moitra13]and[Braun,P.13].
Theorem. AnyLPapproximatingCOR(𝑛) within afactor𝑛s‘Œ isofsize2’(s)Œ3 .
13
Thematchingproblem– amuchmorecomplicatedcase
ViaageneralizationofRazborov’s technique:
Theorem. [Rothvoss14]AnyLPformulation ofthematchingpolytope isof
exponentialsize.
Thisisveryspecialandimportant:
1. Matchingcanbesolved inpolynomial time
2. YetanyLPcapturingitisofexponentialsize
=>Separatesthepowerof Pfrompolynomial sizeLPs
Withinformation theory: ruling outtheexistenceofFPTAS-typeLPformulations
Theorem. [Braun,P.14]Forsome𝜀 > 0 anyLPapproximatingtheMatching
Œ
Polytope withinafactor1 + 3 isofexponential size.
14
Recentresultsfor
SDPextendedformulations
15
WhyareSDPEFssomuchhardertounderstand?
…becausetheyaresomuchstronger:
1. [Braun,Fiorini, P.,Steurer12]:∃ (bounded) spectrahedron
1. indimension 𝑛u ,i.e.,smallSDP-EF
2. anyLPthatapproximatesitwithinafactorof𝑛s‘Œ isof size21(3)
2. [Chain,Lee,Raghavendra,Steurer13]:SeparationviaMaxCut
1. TheGoemans-Williams SDPgivesanapproximationof0.87
2. Nopolynomial-size LPcandobetterthan0.5
3. [Yannakakis 91]:Stablesetpolytope overperfectgraphs
1. BasicSDPgivesperfectEFof theproblem
2. Nopolynomial sizeLPisknown; bestknown 𝑛˜(™š› 3)
16
KnownSDPEFlowerbounds
RecentSDPEFlowerbounds:
1. [Brïet,Dadush, P.13]:Viacounting argument
1. Thereexist0/1polytopes thatdonotadmitpoly-sizeSDPEFs
2. However, argumentisonly existentialinnature
2. [Lee,Raghavendra,Steurer14]:Bounds via(quantum) learning
1. ReuseLasserregapinstancesandlowerbounds
2. ShowthatahypotheticalsmallSDPEFcanbeusedtolearnagood small
LasserrebasedSDPEF->contradiction
3. [Braun,Brown-Cohen, Huq, P.,Raghavendra,Roy,Weitz,Zink 15]:Y.forSDPs
1. Matchinghasno smallsymmetricSDPs
2. Among allsymmetricSDPsof size𝑂(𝑛w ) forTSP𝑘-levelLasserrearebest
4. Noother direct/explicitlowerbounding techniquesareknown
17
Reductionsfor
forextendedformulations
(reusewhatyouknow)
18
ReductionsbetweenProblems
Incomplexitytheory.Howtoshowthataproblem 𝑃 isNP-hard(toapproximate)?
P
Q
Reductionf
Your
Problem
Polynomial-time
computablemap
NP-hard
Problem
Intuition. 𝑓 embeds instancesof𝑄 into𝑃.Nowif𝑃 wouldbeeasy,thentogether
with𝑓,theproblem 𝑄 would beeasy.Contradiction tohardnessof𝑄.
Note. Reduction canbealsoused inreversetoestablishupper bounds onthe
complexityof𝑄 if𝑃 isknown tobeeasy.
19
ReductionsforEFs
NowforEFs.Howtoshowthataproblem 𝑃 doesnotadmitsmallLPs?
P
Q
Reductionf
Your
Problem
Computationally unbounded
(wecareonlyfor#ineqs)
BUTother restrictions…
LP-hard
Problem
Intuition. 𝑓 embeds instancesof𝑄 into𝑃.However,somehow thishastopreservethe
structureofbeing anLPorSDP…
Note. Reduction canbealsoused inreverseagain.Thiscanleadtointerestingresults.
20
ReductionsforLPsandSDPs
Affinereductions. 𝑃s = 𝑆s , 𝐹s , 𝜅s, 𝜏s reducesto𝑃u = 𝑆u ,𝐹u , 𝜅u ,𝜏u viatwomaps
1. Rewritefeasiblesolutions:
∗: 𝑆s → 𝑆u with 𝑠s ↦ 𝑠s∗ ∈ 𝑆u
2. Rewriteobjectivefunctions:
∗: 𝐹s → 𝐹u with𝑓s ↦ 𝑓s∗ ∈ 𝐹u
Sothatthefollowing holds:
𝜅s 𝑓s − 𝑓s 𝑠s = [𝜅u 𝑓s∗ − 𝑓s∗ 𝑠s∗ ] ⋅ 𝑀s (𝑓s , 𝑠s ) + 𝑀u (𝑓s , 𝑠s)
max𝑓s∗ ≤ 𝜏u 𝑓s∗
(completeness)
ifmax𝑓s ≤ 𝜏s 𝑓s
(soundness)
Important. ∗ mapssolutions andfunctions independently ofeachother.
𝑀s , 𝑀u ∈ 𝑅…z×T capturelow-ranknon-affine partoffunction
Theorem. [Braun,P.,Roy15]If𝑃s reducesto𝑃u ,then(essentially)
fc 𝑃s ≤ 𝑟𝑎𝑛𝑘pq/F§q 𝑀s ⋅ fc(𝑃u ) + 𝑟𝑎𝑛𝑘pq/F§q (𝑀u )
Enablescertainformsofgap-amplification.
21
Reductions:afewexamples(LP)
[BPZ15]
MaxCut
1
+𝜀
2
[BPZ15]
[BPR15]
UniqueGames
𝜔(1)
[BPR15]
Max3Sat*’
7
+𝜀
8
StableSet
𝜔(1)
Max2Sat*’
3
+𝜀
4
VertexCover*
2−𝜀
SparsestCut*
btw(supply)
2−𝜀
q-VertexCover*
q− 𝜀
1/QF-CSP
𝜔(1)
[BPR15]
Matching
𝜀
1+
𝑛
[BPR15]
Matching(deg ≤ 3)
Œ
1 + 3t
Sparsestcut
btw(demand)
𝜔(1)
Bal.separator
btw(demand) 𝜔(1)
Bal.separator
btw(supply)
2−𝜀
*Optimal
‘Havebeenalsoobtaineddirectly
22
Reductions:afewexamples(SDP)
SparsestCut
btw(demand)
𝑂(1)
Max-3-XOR/0
2−𝜀
MaxCut
15
+𝜀
16
SparsestCut
btw(supply)
16/15 − 𝜀
VertexCover
7
−𝜀
6
SparsestCut
𝑂(1)
[Sch08]+[LRS14]
Lasserre
reduction
Max-k-CSP
1/𝑞 w
(Ω(𝑛) rounds)
[BCV+12]
Independent Set
𝑛 (s‘®)
(Ω(𝑛 ®)rounds)
=>Lasserre is suboptimalfor
IndependentSet
OpenProblems
1. (general) SDPformulation complexityofthematchingproblem
2. Lowerbounding techniques forSDPEFs
3. LP-hardnessofapproximation forcomplexproblems suche.g.,TSP
4. Extensioncomplexityoveralternativecones:
1. Hyperbolic programming (inP)
2. Copositive programming (NPhard)
5. Understanding thedifferencebetweenhierarchiesandgeneralEFs
24
Thankyou!
25