Geometric Embeddings, Graph Partitioning, and
Expander flows: A survey of recent results
• Sanjeev Arora
Princeton
( touches upon:
S. A., Satish Rao, Umesh Vazirani, STOC’04;
S. A., Elad Hazan, and Satyen Kale, FOCS’04;
S. A., James Lee, and Assaf Naor, unpublished
+ papers that are not mine)
Outline:
• graph partitioning problems: intro and history
• new approximation algorithm + analysis (“Structure
Theorem”)
[A., Rao, Vazirani]
• applications of “S. T.” to other NP-hard problems
• Outline of proof of “S. T.”
• Uses of “S. T.” in Geometric embeddings
• Introduction to expander flows
• Using expander flows to design O(n2) algorithms for
graph partitioning [A., Hazan, Kale]
• Open problems
Sparsest Cut
G = (V, E)
S
(G) = min
| E(S, Sc)|
SµV
S
|S| < |V|/2
c- balanced separator
c(G) = min
Both NP-hard
| E(S, Sc)|
SµV
c |V| < |S| < |V|/2
|S|
|S|
Why these problems are important
• Arise in analysis of random walks, PRAM simulation,
packet routing, clustering, VLSI layout etc.
•
Underlie many divide-and-conquer graph algorithms
(surveyed by Shmoys’95)
•
Discrete analogs of isoperimetric constant; useful in
study of Riemannian manifolds and 2nd eigenvalue of
Laplacian (Cheeger’70)
•
Graph-theoretic parameters of inherent interest (cf.
Lipton-Tarjan planar separator theorem)
Previous approximation algorithms
1) Eigenvalue approaches (Cheeger’70, Alon’85, Alon-Milman’85)
2c(G) ¸ L(G) ¸ c(G)2/2
c(G) = minS µ V E(S, Sc)/ E(S)
2) O(log n) -approximation via LP (multicommodity flows)
(Leighton-Rao’88)
• Approximate max-flow mincut theorems
• Region-growing argument
3) Embeddings of finite metric spaces into l1
(Linial, London, Rabinovich’94,
AR’94)
• Geometric approach; more general result
(but still O(log n) approximation)
New results of [ARV’04]
1. O( log n ) -approximation to sparsest cut and conductance
2. O( log n )-pseudoapproximation to c-balanced separator
(algorithm outputs a c’-balanced separator, c’ < c)
3. Existence of expander flows in every graph
(approximate certificates of expansion)
Disparate approaches from previous slide get “unified”
Semidefinite relaxations for c-balanced separator
(and how to round the solutions)
Semidefinite
LP Relaxations for c-balanced separator
Min (i, j) 2 E Xij
0c) ·
Xij ·a 1(semi) metric
Motivation: Every cut (S, S
defines
Xij 2 {0,1}
1
0
1
1
0
Xij + Xj k ¸ Xik
 i< j Xij ¸ c(1-c)n2
There exist unit vectors v1, v2, …, vn 2 <n
such that
Xij = |vi - vj|2 /4
Semidefinite relaxation (contd)
Min (i, j) 2 E
Unit l22 space
|vi –vj|2/4
|vi|2 = 1
|vi –vj|2 + |vj –vk|2 ¸ |vi –vk|2
8 i, j, k
i < j |vi –vj|2 ¸ 4c(1-c)n2
Unit l22 space
Unit vectors v1, v2,… vn 2 <d
Vi
Vj
|vi –vj|2 + |vj –vk|2 ¸ |vi –vk|2
Vk
8 i, j, k
Angles are non obtuse
s
s
s
Taking r steps of length s
s
only takes you squared distance rs2
(i.e. distance r s)
Example of l22 space: hypercube {-1, 1}k
|u – v|2
= i |ui – vi|2
= 2 i |ui – vi|
= 2 |u – v|1
In fact, every l1 space is also l22
Conjecture (Goemans, Linial): Every l22 space is l1 up to
distortion O(1)
Structure Theorem for l22 spaces
<d
Two subsets S and T are -separated
if for every vi 2 S, vj 2 T |vi –vj|2 ¸ 
¸
Thm: If i< j |vi –vj|2 = (n2) then there exist two sets S, T
of size (n) that are  -separated for  = ( 1 )
log n
Main thm ) O( log n)-approximation
v1, v2,…, vn 2 <d is optimum SDP soln;
SDPopt = (i, j) 2 E |vi –vj|2
S, T :  –separated sets of size (n)

Do BFS from S until you hit T. Take the level
of the BFS tree with the fewest edges and
output the cut (R, Rc) defined by this level
(i, j) 2 E |vi –vj|2
¸
|E(R, Rc)| £ 
) |E(R, Rc)| · SDPopt /
· O(
log n SDPopt)
Other new
log n
-approximation algorithms
• MIN-2-CNF deletion and several graph deletion problems. [Agarwal,
Charikar, Makarychev, Makarychev’04]
• MIN-LINEAR ARRANGEMENT [Charikar, Karloff, Rao’04]
• General SPARSEST CUT [A., Lee, Naor ’04]
•
Min-ratio VERTEX SEPARATORS and Balanced VERTEX
SEPARATORS [ Feige, Hajiaghayi, Lee, ’04]
Next 10-12 min: Proof-sketch of Structure Thm
( algorithm to produce  -separated S, T of size (n);
 = 1/ log n )
T
S
Projection onto a random line
<d
v
u
e
<u, v> ??
Pru[ projection exceeds 2
1
1
d
d
log n ] < 1/n2
d
2
-t /2
Algorithm to produce two  –separated sets
<d
u
Check if Su and Tu have size (n)
If any vi 2 Su and vj 2 Tu satisfy
|vi –vj|2 · ,
delete them and
Tu
Su
0.01
d
repeat until no such vi, vj remain
If Su, Tu still have size (n), output them
Main difficulty: Show that whp only o(n) points get deleted
“Stretched pair”: vi, vj such that |vi –vj|2 ·  and
| h vi –vj, u i | ¸ 0.01
d
Obs: Deleted pairs are stretched and they form a matching.
“Matching is of size o(n) whp” : naive argument fails
“Stretched pair”: vi, vj such that |vi –vj|2 ·  and
| h vi –vj, u i | ¸ 0.01
d
O( 1

) PrU [ vi, vj get stretched] = exp( - 1 )

) £ standard deviation
= exp( - log n )
E[# of stretched pairs] = O( n2 ) £ exp(- log n )
Suppose matching of (n) size exists with probability (1)…
….stretched pairs are almost everywhere you look!
Vj
Ball (vi , )
u
Vi
0.01
d
Generating a contradiction: the walk on stretched pairs
Vj
vfinal



Vi
0.01
0.01
d
d
|vfinal - vi| <
r
r steps
0.01
r
d
| <vfinal – vi, u>| ¸ 0.01r = O( r ) x standard dev.
u

d
Contradiction (if r large enuff)!!
Measure concentration (P. Levy, Gromov etc.)
<d
A : measurable set with (A) ¸ 1/4
A
A
A : points with distance ·  to A
(A) ¸ 1 – exp(-2 d)
Reason: Isoperimetric inequality for
spheres

Embeddings of finite metric spaces into
geometric spaces
<k (with l2 norm)
Finite metric space (X, d)
y
x
f
d(x,y)
distortion of f is minimum C>1 such that
d( x, y) · |f(x ) – f( y)|2 ·
C d( x, y)
8 x, y
Thm (Bourgain’85): For every n-point metric space, a map
exists with distortion O(log n)
map;
Qs: [LLR’94]:
ImproveEfficient
O(log algorithm
n) when toX find
is athe
geometric
space; say l1 ?
Proof that O(log n) cannot be improved in general
Status report of this area
Best lowerbound
l1 into l2
Best upperbound
log0.5 n
[Enflo’69]
log n
[Bourgain’85]
l22 into l1
Exactly the integrality
gap of SDP for general
SPARSEST CUT
[LLR’94, AR’94]
2
l2 into l2
1.16
[Zatloukal’04]
log0.75 n
Superconstant
[Khot,
Vishnoi’04]
[Chawla,Gupta,Racke ’04]
log0.5 n
log0.5 n log log n
[Enflo’69]
[A., Lee, Naor’04]
Frechet’s recipe to embed metric space (X, d) into Rk
Pick k suitable subsets A1, A2, …, Ak of X
Map x 2 X to (d(x, A1), d(x, A2), … , d(x, Ak))
x
In recent results, Ai’s are chosen using [ARV] Structure
Theorem and “Measured descent” idea of
[Krauthgamer, Lee, Naor, and Mendel’04]
Ai
Expander flows
(approximate certificates of expansion)
Expander flows: Motivation
“Expander”
G = (V, E)
S
Idea: Embed a D-regular
(weighted) graph such that
S
8 S w(S, Sc) = (D |S|)(*)
(certifies expansion = (D) )
Weighted Graph w satisfies (*)
iff
L(w) = (1) [Cheeger]
Our Thm: If G has expansion , thenCf.
a D-regular
expander
flow
Jerrum-Sinclair,
Leighton-Rao
exists in it where D= 
(embed a complete graph)
log n
Example of expander flow
n-cycle
Take any 3-regular expander on n nodes
Put a weight of 1/3n on each edge
Embed this into the n-cycle
Routing of edges does not exceed any capacity ) expansion =(1/n)
New Result
(A., Hazan, Kale;FOCS’04)
O(n2) time algorithm that given any graph G finds for some D >0
• a D-regular expander flow
• a cut of expansion O( D log n )
)
(D) · (G) ·O(D log n )
Ingredients: Approximate eigenvalue computations;
Approximate flow computations (Garg-Konemann; Fleischer)
Random sampling (Benczur-Karger + some more)
Idea: Define a zero-sum game whose optimum solution is an expander
flow; solve approximately using Freund-Schapire approximate solver.
Expander flows: LP view
·1
·D
G
LP feasible )  ¸ (D)
Thm [ARV]: 9 0 s.t. the LP is
feasible with D = /√log n
OPEN PROBLEMS
• Better approximation factor than O( log n )?
(For general SPARSEST CUT, log log n “lowerbound” )
• Better distortion bound for embedding l22 into l1? ( log n upperbound
v/s loglog n lowerbound.)
• Combinatorial approximation algorithms for other problems ?
(similar to one for SPARSEST CUT from [A., Hazan, Kale] )
• O(m) time algorithm for SPARSEST CUT instead of O(n2)?
(not known even for Leighton-Rao’88 O(log n) approximation)
• Other applications of expander flows?
(Useful in results about Banach spaces [Naor, Rabani, Sinclair’04])
Looking forward to more progress…
Thanks !
Open problems (circa April’04)
2) time;
O(n
• Better running time/combinatorial algorithm?
[A., Hazan, Kale]
• Improve approximation ratio to O(1); better rounding??
(our conjectures may be useful…)
• Extend result to other expansion-like problems (multicut,
general sparsest cut; MIN-2CNF deletion)
Integrality gap is (log n) [Charikar]
• Resolve conjecture about embeddability of l22 into l1; of
l1 into l2
log3/4 n distortion; [Chawla,Gupta, Racke]
•
Any applications of expander flows?
Yes [Naor,Sinclair,Rabani]
Better embeddings of lp into lq [Lee]
Various new results
O(n2) time combinatorial algorithm for sparsest cut
(does not use semidefinite programs)
[A., Hazan, Kale’04]
New results about embeddings: (i) lp into lq [J. Lee’04]
(ii) l22 and l1 into l2 [CGR’04]
(approx for general sparsest cut)
Clearer explanation of expander flows and their connection to
embeddings [NRS’04]
Formal statement : 9 0 >0 s.t. foll. LP is feasible for d = (G)
log n
Pij = paths whose endpoints are i, j
8i
j p 2 Pij fp = d
8e 2 E p 3 e fp · 1
8S µ V i 2 S j 2 Sc p 2 Pij fp ¸ 0 d |S|
fp ¸ 0
8 paths p in G
(degree)
(capacity)
(demand graph is
an expander)
A concrete conjecture (prove or refute)
G = (V, E);  = (G)
For every distribution on n/3 –balanced cuts {zS} (i.e., S
zS =1)
there exist (n) disjoint pairs (i1, j1), (i2, j2), ….. such that for each k,
•
distance between ik, jk in G is O(1/ )
• i k, j k
are across (1) fraction of cuts in {zS}
(i.e., S: i 2 S, j 2 Sc zS = (1) )
Conjecture ) existence of d-regular expander flows for d = 
log n
log n
log n