UChicago REU 2013 Apprentice Program
Summer 2013
Lecture 10: July 15, 2013
Instructor: Madhur Tulsiani
1
Scribe: David Kim
Eigenvalues of the adjacency matrix
All graphs in this lecture will be undirected.
Let G = (V, E) be a graph with n vertices. Let A be the adjacency matrix of G. So A is a symmetric
real matrix and therefore its eigenvalues are real; let µ1 ≥ µ2 ≥ . . . µn be these eigenvalues.
Exercise 1.1
Pn
= 0.
Exercise 1.2
Pn
= 2|E|.
i=1 µi
2
i=1 µi
(Hint: trace Tr(A))
(Hint: Tr(A2 ))
Exercise 1.3 (∀i)(|µi | ≤ degmax ) where degmax is the maximum degree.
Exercise 1.4 µ1 ≥ degavg where degavg = 2|E|/n is the average degree. (Hint: Rayleigh quotient)
Exercise 1.5 (∀i)(µ1 ≥ |µi |).
Exercise 1.6 (a) If G is connected then µ1 > µ2 .
(b) Prove that the converse is false: µ1 > µ2 does not imply connectedness.
Exercise 1.7 If G is bipartite then the adjacency spectrum is symmetric about the origin:
(∀i)(µn−i+1 = −µi ).
Exercise 1.8 (a) If G is connected and µn = −µ1 then G is bipartite.
(b) Prove that the converse is false: without the connectedness assumption, µn = −µ1 does not
imply that the graph is bipartite.
2
The Laplacian
Let G = (V, E) be a graph with n vertices. Let A be the adjacency matrix of G. Let D be the
diagonal matrix with Dii = deg(i).
Definition 2.1 (The Laplacian) Given G as above, the Laplacian associated with G is the symmetric n × n matrix L := D − A.
1
Next we take a look at the Laplacian quadratic form. Let x ∈ Rn be a function on the vertices of
G = (V, E). Then
xT Lx = xT (D − A)x
= xT Dx − xT Ax
n
X
X
=
2xi xj
deg(i)x2i −
i=1
=
{i,j}∈E
n
X
X
i=1 {i,j}∈E
X
=
X
x2i −
2xi xj
{i,j}∈E
(x2i + x2j − 2xi xj )
{i,j}∈E
X
=
(xi − xj )2
{i,j}∈E
So the Laplacian is positive semidefinite and therefore its eigenvalues are nonnegative. In fact, the
smallest eigenvalue is 0, as any non-zero constant vector is an eigenvector of eigenvalue 0. Thus the
eigenvalues of the Laplacian (the “Laplacian eigenvalues”) of a graph G can be ordered as follows:
0 = λ1 ≤ λ2 ≤ · · · ≤ λn
Exercise 2.2 A graph is connected if and only if its second smallest Laplacian eigenvalue is positive: λ2 > 0.
Definition 2.3 (The Normalized Laplacian) The normalized Laplacian of a graph G is N :=
D−1/2 LD−1/2 , where D and L are as above.
Note that we have
N = D−1/2 LD−1/2
= D−1/2 (D − A)D−1/2
= I − D−1/2 AD−1/2
Now consider a d-regular graph G. Then
L = D − A = dI − A
and if we let µ1 ≥ · · · ≥ µn be the eigenvalues of A, then µi = d − λi for i = 1, . . . , n as
Lvi = λi vi ⇔ Avi = (d − λi )vi . So we have the following spectrum for A:
d = µ1 ≥ · · · ≥ µn ≥ −d
The first inequality follows from λ1 = 0 and µ1 = d − λ1 (and alternatively from degmax = degavg =
d, by Exx. 1.3 and 1.4). The last inequality follows from Ex. 1.3.
2
For the normalized Laplacian, we have in the d-regular case:
1
N =I− A
d
If we let ν1 ≥ · · · ≥ νn be the eigenvalues of N , then νi = 1 − d1 µi =
are the eigenvalues of L and A, respectively)
3
λi
d
for i = 1, . . . , n. (λi and µi
Conductance of a Graph
A cut in the graph G = (V, E) is a partition of the vertices into (S, S) where S ⊆ V and S = V \ S.
We write E(S, S) to denote the set of edges connecting a vertex in S to a vertex in S (the edges
“leaving S”).
P
The volume of S ⊆ V is vol(S) = i∈S deg(i). The volume counts the pairs (v, e) where v ∈ S, e ∈
E and v, e are incident. Note that if G is d-regular then vol(S) = d|S|.
Definition 3.1 (Edge Expansion of a subset) The edge expansion of a subset S ⊆ V is
ϕ(S) :=
|E(S, S)|
.
vol(S)
This is the proportion of the indicences (v, e) where e leaves S among all incidences counted by the
volume.
Definition 3.2 (Conductance of a graph) The conductance of a graph G is
ΦG :=
min
ϕ(S)
0<vol(S)≤|E|
The conductance is also called the edge expansion of G.
Assume G is d-regular. In this case the minimum in the definition of the conductance is over all
subsets S ⊆ V such that 0 < |S| ≤ n/2.
If we represent S ⊆ V by its incidence vector x ∈ {0, 1}n then
P
2
1P
|E(S, S)|
{i,j}∈E (xi − xj )
i,j Ai,j |xi − xj |
2
P 2
P 2
ϕ(S) =
=
=
d|S|
d i xi
d xi
and ΦG is obtained by x ∈ {0, 1}n , x 6= 0, 1 (|S| ≤ n/2) which minimizes the above expression.
But this is exactly the expression of the Rayleigh quotient, RN (x), associated with the normalized
Laplacian of G. Thus, the eigenvalues of the Laplacian, optimized over x ∈ Rn , x 6= 0, can be seen
as a continuous relaxation of the conductance.
Cheeger’s Inequality bounds the conductance of a graph in terms of the second smallest eigenvalue
of the normalized Laplacian.
3
4
Cheeger’s Inequality
Let λi denote the i-th smallest eigenvalue of the normalized Laplacian of the graph G:
0 = λ1 ≤ · · · ≤ λn ≤ 2
Theorem 4.1 (Cheeger’s Inequalities)
p
λ2
≤ ΦG ≤ 2λ2
2
4.1
Proof of the First Inequality
We first prove the easy direction:
Proof:
λ2
2
≤ ΦG We give the proof for the case when G is d-regular.
Let (S, S) be a cut such that |S| ≤ n2 .
As before, N denotes the normalized Laplacian and RN (x) its Rayleigh quotient.
Claim 4.2 ∃x ∈ Rn , x 6= 0, x ⊥ 1, such that RN (x) ≤ 2 · ϕ(S).
→ Define x ∈ Rn as follows:
 1


|S|
xi =
1

−
|S|
if i ∈ S
if i ∈ S
Then x ⊥ 1 since
n
X
1
1
1 x=
xi = |S| ·
+ |S| · −
=0
|S|
|S|
i=1
T
Also, the Rayleigh quotient evaluates to
P
2
xT N x
1
1 |E(S, S)| · 1/|S| + 1/|S|
{i,j}∈E (xi − xj )
RN (x) = T
= ·
= ·
x x
d
xT x
d |S| · (1/|S|2 ) + |S| · (1/|S|2 )
as only the edges going across the cut (S, S) contribute to the sum in the numerator. Thus we have
RN (x) =
|E(S, S)| |S| + |S|
·
≤ 2 · ϕ(S)
d|S|
|S|
since |S| ≤ n/2. The claim also holds for S and the corresponding x for which ϕ(S) = ΦG . Thus,
λ2 = min
v⊥1
vT N v
xT N x
≤
≤ 2 · ϕ(S) = 2 · ΦG
vT v
xT x
4
4.2
Proof of the Second Inequality
√
For the other direction of the inequality, we will prove a slightly weaker bound: ΦG ≤ 2 λ2 . Again,
we give the proof for the case when G is d-regular.
Proof:
First we prove the following main lemma:
Lemma 4.3 (Spectralp
Partitioning) Given x ∈ Rn such that x 6= 0, x ⊥ 1, we can find |S| ≤
n/2, such that ϕ(S) ≤ 2 RN (x) .
Our proof is algorithmic; it is a linear time algorithm known as spectral partitioning.
Observation 1. Multiplying x by a constant c 6= 0 does not change the Rayleigh quotient, i.e.,
RN (x) = RN (cx).
Observation 2. Adding c1 to x decreases the Rayleigh quotient.
(proof) Let y = x + c1. Then RN (y) ≤ RN (x), since y T N y = xT N x (the numerator stays same),
while y T y = (x + c1)T (x + c1) = xT x + ||c1||2 , since x ⊥ 1 (the denominator increases). Now shift and scale x to get z ∈ [−1, 1]n such that zd n2 e = 0, i.e., the median is at 0 (think of
z ∈ Rn as points, z1 ≤ · · · ≤ zn , on the real line). We can then obtain a cut (S, S) by cutting the
line, say at t ∈ R, and taking S = {1, . . . , i} for all zi ≤ t. Note that in this way, there are at most
n cuts possible.
Claim 4.4 One of these cuts is good enough for us.
This is a linear-time algorithm, since the algorithm calculates only these cuts, and given the value
of the cut for (S, S) where S = {1, . . . , i}, to find the value of the next cut, S = {1, . . . , i + 1},
one only needs to look at the edges incident to vertex i + 1; hence each edge will be looked at only
twice.
So given z ∈ [−1, 1]n , we use a probabilistic argument to show that one of these cuts is good enough:
• choose t ∈ [0, 1] uniformly at random
• let S = {i : zi2 ≥ t}.
We use P [A] to denote the probability of event A and E[X] to denote the expected value of the
random variable X.
Now P [i ∈ S] = zi2 . We will show that
E[|E(S, S)|] p
≤ 2RN (z)
d · E[|S|]
p
in other words, ∃t ∈ [0, 1] s.t. S = {i : zi2 ≥ t} gives ϕ(S) ≤ 2RN (z).
5
Let z (1) denote the positive part of z ∈ Rn and z (2) denote negative part of z, defined by
(1)
= max{zi , 0}
(2)
= min{zi , 0}.
zi
zi
We observe that the set S = {i : zi2 ≥ t} for both z (1) and z (2) will be one of the n cuts seen by
the algorithm, and we can show that either RN (z (1) ) or RN (z (2) ) ≤ 2RN (z). Applying the results
to z (1) or z (2) proves the lemma.
Claim 4.5 At least one of RN (z (1) ) and RN (z (2) ) must be ≤ 2RN (z) .
Observation 3. (z (1) )T (z (1) ) + (z (2) )T (z (2) ) = z T z.
Observation 4. (z (1) )T N (z (1) ) ≤ z T N z, (z (2) )T N (z (2) ) ≤ z T N z.
(proof) We have
(z (1) )T N (z (1) ) =
1 X
(1)
(1)
·
(zi − zj )2
d
{i,j}∈E
1 X
zT N z = ·
(zi − zj )2
d
{i,j}∈E
(1)
If both zi , zj ≥ 0, then (zi − zj )2 = (zi
(1)
(1)
− zj )2 . If both zi , zj < 0, then (zi
(1)
(1)
erwise, zi and zj have opposite signs, and one of zi or zj
An analogous argument gives (z (2) )T N (z (2) ) ≤ z T N z. (1)
− zj )2 = 0. Oth(1)
is 0, hence (zi − zj )2 ≥ (zi
By Observation 3, one of (z (1) )T (z (1) ) and (z (2) )T (z (2) ) must be ≥
RN (z (2) ) must be ≤ 2RN (z).
zT z
2 .
(1)
− zj )2 .
Therefore, RN (z (1) ) or
Claim 4.6 Given z ∈ [−1, 1]n , let us choose t ∈ [0, 1] uniformly at random. Let S = {i : zi2 ≥ t}.
Then
E[|E(S, S)|] p
≤ 2RN (z)
d · E[|S|]
h
i
p
Once this has been verified, we conclude that E |E(S, S)| − d 2RN (z)|S| ≤ 0 and therefore
|E(S, S)| p
∃t ∈ [0, 1] s.t.
≤ 2RN (z).
d|S|
(proof) For i = 1, . . . , n, define Yi s.t.
(
1 if i ∈ S
Yi :=
0 o.w.
6
Then we have
E[|S|] =
n
X
E[Yi ] =
i=1
n
X
zi2
i=1
and also
X
E[|E(S, S)|] =
|zi2 − zj2 |
{i,j}∈E
X
=
|zi − zj ||zi + zj |
{i,j}∈E
s X
≤
|zi − zj |2
{i,j}∈E
s X
≤
=
|zi + zj |2
{i,j}∈E
|zi − zj |2
{i,j}∈E
s X
s X
s X
(2zi2 + 2zj2 )
{i,j}∈E
s X
2
|zi − zj | 2d
zi2
i
{i,j}∈E
where the first inequality follows from Cauchy-Schwarz, the second from zi2 +zj2 ≥ 2|zi zj |. Therefore,
we have
E[|E(S, S)|] p
≤ 2R(z).
dE[|S|]
This completes the proof of
z (1)
z (2) ;
Now apply Claim 3.6 to
and
by Claim 3.5, one of them will show the existence of a good
cut satisfying Claim 3.3 and Claim 3.4. (Note that at least one of z (1) and z (2) is nonzero. Also,
having set the median to 0, we do not need to worry about |S| ≤ n/2.) That is, assuming WLOG
(1)
that RN (z (1) ) ≤ 2RN (z), ∃S = {i : (zi )2 ≥ t}
q
p
p
ϕ(S) ≤ 2RN (z (1) ) ≤ 2 RN (z) ≤ 2 RN (x)
To finish, apply the main lemma (spectral partitioning) to the second eigenvector v2 of the second
smallest eigenvalue λ2 of the normalized Laplacian. Then the output S will satisfy:
p
p
ΦG ≤ ϕ(S) ≤ 2 RN (v2 ) = 2 λ2 .
4.3
Proof of the Tight Bound
√
We are getting the 2 outside the root in 2 λ2 ; we will try to get it inside now.
√
To prove ΦG ≤ 2λ2 , we prove the following lemma:
7
Lemma 4.7 Given x ⊥ 1, ∃S ⊆ V s.t. |S| ≤ n/2 and ϕ(S) ≤
p
2RN (x).
Proof: Again, shift and scale x to z s.t. zdn/2e = 0 and z12 + zn2 = 1. Choose t ∈ [z1 , zn ], where t
is a random variable with density 2|t|, defined only between z1 and zn .
Let S = {i : zi ≤ t}. Then for a ≥ z1 and b ≤ zn ,
Z
b
P [t ∈ [a, b]] =
2|r|dr = sgn(b) · b2 − sgn(a) · a2
a
Claim 4.8
p
E[|E(S, S)|]
≤ 2R(z)
dE[min(|S|, |S|)]
(pf.) For i = 1, . . . , n let Yi be such that
(
1 if (i ∈ S and |S| ≤ n/2) OR (i ∈ S and |S| ≤ n/2)
Yi =
0 o.w.
Then we have
E[min(|S|, |S|)] =
X
E[Yi ] =
i
E[|E(S, S)|] =
X
X
zi2
i
P [t ∈ [zi , zj )]
{i,j}∈E
=
X
| sgn(zj ) · zj2 − sgn(zi ) · zi2 |
{i,j}∈E
≤
X
|zj − zi |(|zj | + |zi |)
{i,j}∈E
One can easily check | sgn(zj ) · zj2 − sgn(zi ) · zi2 | ≤ |zj − zi |(|zj | + |zi |) holds for all cases of the signs.
The rest of the proof is same as the previous proof using Cauchy-Schwarz. Having established the lemma, using v2 , the eigenvector to the second eigenvalue λ2 of the normalized Laplacian, the set S of the above analysis immediately satisfies
p
p
ΦG ≤ ϕ(S) ≤ 2RN (v2 ) = 2λ2
5
Some Generalizations of Cheeger’s Inequality
√
√
Here we found sets S1 , S2 s.t. S2 = S1 , ϕ(S1√) ≤ 2λ2 , ϕ(S2 ) ≤ 2λ2 . We can also find sets
S1 , . . . , Sk that are disjoint s.t. ϕ(Si ) ≤ O(k 2 ) λ√k for each i = 1, . . . , k. More recent results also
show that we can find S1 , . . . , S k s.t. ϕ(Si ) ≤ O( λk log k) (cannot do better than log k).
2
Look up: Dirichlet Boundary Conditions
8