Chapter 17
Graphs and Graph Laplacians
17.1
Directed Graphs, Undirected Graphs, Incidence
Matrices, Adjacency Matrices, Weighted Graphs
Definition 17.1. A directed graph is a pair G = (V, E),
where V = {v1, . . . , vm} is a set of nodes or vertices, and
E ✓ V ⇥V is a set of ordered pairs of distinct nodes (that
is, pairs (u, v) 2 V ⇥ V with u 6= v), called edges. Given
any edge e = (u, v), we let s(e) = u be the source of e
and t(e) = v be the target of e.
Remark: Since an edge is a pair (u, v) with u 6= v,
self-loops are not allowed.
Also, there is at most one edge from a node u to a node
v. Such graphs are sometimes called simple graphs.
733
734
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
v1
e1
v2
e5
e3
e2
e4
v5
e6
v3
e7
v4
Figure 17.1: Graph G1 .
For every node v 2 V , the degree d(v) of v is the number
of edges leaving or entering v:
d(v) = |{u 2 V | (v, u) 2 E or (u, v) 2 E}|.
We abbreviate d(vi) as di. The degree matrix D(G), is
the diagonal matrix
D(G) = diag(d1, . . . , dm).
For example, for graph G1, we have
0
2 0 0 0
B0 4 0 0
B
D(G1) = B
B0 0 3 0
@0 0 0 3
0 0 0 0
1
0
0C
C
0C
C.
0A
2
Unless confusion arises, we write D instead of D(G).
17.1. DIRECTED GRAPHS, UNDIRECTED GRAPHS, WEIGHTED GRAPHS
735
Definition 17.2. Given a directed graph G = (V, E),
for any two nodes u, v 2 V , a path from u to v is a
sequence of nodes (v0, v1, . . . , vk ) such that v0 = u, vk =
v, and (vi, vi+1) is an edge in E for all i with 0  i 
k 1. The integer k is the length of the path. A path
is closed if u = v. The graph G is strongly connected if
for any two distinct node u, v 2 V , there is a path from
u to v and there is a path from v to u.
Remark: The terminology walk is often used instead of
path, the word path being reserved to the case where the
nodes vi are all distinct, except that v0 = vk when the
path is closed.
The binary relation on V ⇥ V defined so that u and v are
related i↵ there is a path from u to v and there is a path
from v to u is an equivalence relation whose equivalence
classes are called the strongly connected components of
G.
736
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
Definition 17.3. Given a directed graph G = (V, E),
with V = {v1, . . . , vm}, if E = {e1, . . . , en}, then the
incidence matrix B(G) of G is the m ⇥ n matrix whose
entries bi j are given by
8
>
<+1 if s(ej ) = vi
bi j =
1 if t(ej ) = vi
>
:
0
otherwise.
Here is the incidence matrix
0
1 1 0
B 1 0
1
B
1 1
B=B
B0
@0 0 0
0 0 0
of the graph G1:
1
0 0 0 0
1 1 0 0C
C
0 0 0 1C
C.
1 0
1 1A
0
1 1 0
Again, unless confusion arises, we write B instead of
B(G).
Remark: Some authors adopt the opposite convention
of sign in defining the incidence matrix, which means that
their incidence matrix is B.
17.1. DIRECTED GRAPHS, UNDIRECTED GRAPHS, WEIGHTED GRAPHS
a
v1
737
v2
e
b
c
d
v5
f
v3
g
v4
Figure 17.2: The undirected graph G2 .
Undirected graphs are obtained from directed graphs by
forgetting the orientation of the edges.
Definition 17.4. A graph (or undirected graph) is a
pair G = (V, E), where V = {v1, . . . , vm} is a set of
nodes or vertices, and E is a set of two-element subsets
of V (that is, subsets {u, v}, with u, v 2 V and u 6= v),
called edges.
Remark: Since an edge is a set {u, v}, we have u 6= v,
so self-loops are not allowed. Also, for every set of nodes
{u, v}, there is at most one edge between u and v.
As in the case of directed graphs, such graphs are sometimes called simple graphs.
738
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
For every node v 2 V , the degree d(v) of v is the number
of edges incident to v:
d(v) = |{u 2 V | {u, v} 2 E}|.
The degree matrix D is defined as before.
Definition 17.5. Given a (undirected) graph G = (V, E),
for any two nodes u, v 2 V , a path from u to v is a sequence of nodes (v0, v1, . . . , vk ) such that v0 = u, vk = v,
and {vi, vi+1} is an edge in E for all i with 0  i  k 1.
The integer k is the length of the path. A path is closed
if u = v. The graph G is connected if for any two distinct
node u, v 2 V , there is a path from u to v.
Remark: The terminology walk or chain is often used
instead of path, the word path being reserved to the case
where the nodes vi are all distinct, except that v0 = vk
when the path is closed.
The binary relation on V ⇥ V defined so that u and v are
related i↵ there is a path from u to v is an equivalence relation whose equivalence classes are called the connected
components of G.
17.1. DIRECTED GRAPHS, UNDIRECTED GRAPHS, WEIGHTED GRAPHS
739
The notion of incidence matrix for an undirected graph
is not as useful as in the case of directed graphs
Definition 17.6. Given a graph G = (V, E), with V =
{v1, . . . , vm}, if E = {e1, . . . , en}, then the incidence
matrix B(G) of G is the m ⇥ n matrix whose entries bi j
are given by
(
+1 if ej = {vi, vk } for some k
bi j =
0
otherwise.
Unlike the case of directed graphs, the entries in the
incidence matrix of a graph (undirected) are nonnegative.
We usually write B instead of B(G).
The notion of adjacency matrix is basically the same for
directed or undirected graphs.
740
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
Definition 17.7. Given a directed or undirected graph
G = (V, E), with V = {v1, . . . , vm}, the adjacency matrix A(G) of G is the symmetric m ⇥ m matrix (ai j )
such that
(1) If G is directed, then
8
>
<1 if there is some edge (vi, vj ) 2 E
ai j =
or some edge (vj , vi) 2 E
>
:
0 otherwise.
(2) Else if G is undirected, then
(
1 if there is some edge {vi, vj } 2 E
ai j =
0 otherwise.
As usual, unless confusion arises, we write A instead of
A(G).
Here is the adjacency matrix
0
0 1
B1 0
B
A=B
B1 1
@0 1
0 1
of both graphs G1 and G2:
1
1 0 0
1 1 1C
C
0 1 0C
C.
1 0 1A
0 1 0
17.1. DIRECTED GRAPHS, UNDIRECTED GRAPHS, WEIGHTED GRAPHS
741
If G = (V, E) is a directed or an undirected graph, given
a node u 2 V , any node v 2 V such that there is an edge
(u, v) in the directed case or {u, v} in the undirected case
is called adjacent to v, and we often use the notation
u ⇠ v.
Observe that the binary relation ⇠ is symmetric when G
is an undirected graph, but in general it is not symmetric
when G is a directed graph.
If G = (V, E) is an undirected graph, the adjacency matrix A of G can be viewed as a linear map from RV to
RV , such that for all x 2 Rm, we have
X
(Ax)i =
xj ;
j⇠i
that is, the value of Ax at vi is the sum of the values of
x at the nodes vj adjacent to vi.
742
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
The adjacency matrix can be viewed as a di↵usion operator .
This observation yields a geometric interpretation of what
it means for a vector x 2 Rm to be an eigenvector of A
associated with some eigenvalue ; we must have
X
xi =
xj , i = 1, . . . , m,
j⇠i
which means that the the sum of the values of x assigned
to the nodes vj adjacent to vi is equal to times the value
of x at vi.
Definition 17.8. Given any undirected graph G = (V, E),
an orientation of G is a function : E ! V ⇥ V assigning a source and a target to every edge in E, which means
that for every edge {u, v} 2 E, either ({u, v}) = (u, v)
or ({u, v}) = (v, u). The oriented graph G obtained
from G by applying the orientation is the directed graph
G = (V, E ), with E = (E).
17.1. DIRECTED GRAPHS, UNDIRECTED GRAPHS, WEIGHTED GRAPHS
743
Proposition 17.1. Let G = (V, E) be any undirected
graph with m vertices, n edges, and c connected components. For any orientation of G, if B is the incidence matrix of the oriented graph G , then c =
dim(Ker (B >)), and B has rank m c. Furthermore,
the nullspace of B > has a basis consisting of indicator vectors of the connected components of G; that is,
vectors (z1, . . . , zm) such that zj = 1 i↵ vj is in the ith
component Ki of G, and zj = 0 otherwise.
Following common practice, we denote by 1 the (column)
vector whose components are all equal to 1. Observe that
B >1 = 0.
According to Proposition 17.1, the graph G is connected
i↵ B has rank m 1 i↵ the nullspace of B > is the onedimensional space spanned by 1.
In many applications, the notion of graph needs to be
generalized to capture the intuitive idea that two nodes u
and v are linked with a degree of certainty (or strength).
744
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
Thus, we assign a nonnegative weight wi j to an edge
{vi, vj }; the smaller wi j is, the weaker is the link (or
similarity) between vi and vj , and the greater wi j is, the
stronger is the link (or similarity) between vi and vj .
Definition 17.9. A weighted graph is a pair G = (V, W ),
where V = {v1, . . . , vm} is a set of nodes or vertices, and
W is a symmetric matrix called the weight matrix , such
that wi j
0 for all i, j 2 {1, . . . , m}, and wi i = 0 for
i = 1, . . . , m. We say that a set {vi, vj } is an edge i↵
wi j > 0. The corresponding (undirected) graph (V, E)
with E = {{vi, vj } | wi j > 0}, is called the underlying
graph of G.
Remark: Since wi i = 0, these graphs have no self-loops.
We can think of the matrix W as a generalized adjacency
matrix. The case where wi j 2 {0, 1} is equivalent to the
notion of a graph as in Definition 17.4.
We can think of the weight wi j of an edge {vi, vj } as a
degree of similarity (or affinity) in an image, or a cost in
a network.
An example of a weighted graph is shown in Figure 17.3.
The thickness of an edge corresponds to the magnitude
of its weight.
17.1. DIRECTED GRAPHS, UNDIRECTED GRAPHS, WEIGHTED GRAPHS
745
Encode Pairwise Relationships as a Weighted Graph
15
Figure 17.3: A weighted graph.
For every node vi 2 V , the degree d(vi) of vi is the sum
of the weights of the edges adjacent to vi:
d(vi) =
m
X
wi j .
j=1
Note that in the above sum, only nodes vj such that there
is an edge {vi, vj } have a nonzero contribution. Such
nodes are said to be adjacent to vi, and we write vi ⇠ vj .
The degree matrix D is defined as before, namely by D =
diag(d(v1), . . . , d(vm)).
746
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
The weight matrix W can be viewed as a linear map from
RV to itself. For all x 2 Rm, we have
X
(W x)i =
wij xj ;
j⇠i
that is, the value of W x at vi is the weighted sum of the
values of x at the nodes vj adjacent to vi.
Observe that W 1 is the (column) vector (d(v1), . . . , d(vm))
consisting of the degrees of the nodes of the graph.
17.1. DIRECTED GRAPHS, UNDIRECTED GRAPHS, WEIGHTED GRAPHS
747
Given any subset of nodes A ✓ V , we define the volume
vol(A) of A as the sum of the weights of all edges adjacent
to nodes in A:
vol(A) =
X
vi 2A
d(vi) =
m
XX
wi j .
vi 2A j=1
Degree of a node:
di = j Wi,j
Volume of a set
vol(A) =
di
i
A
Degree matrix:
Dii = j Wi,j
18
Figure 17.4: Degree and volume.
Observe that vol(A) = 0 if A consists of isolated vertices,
that is, if wi j = 0 for all vi 2 A. Thus, it is best to
assume that G does not have isolated vertices.
19
748
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
Given any two subset A, B ✓ V (not necessarily distinct), we define links(A, B) by
X
links(A, B) =
wi j .
vi 2A,vj 2B
Since the matrix W is symmetric, we have
links(A, B) = links(B, A),
and observe that vol(A) = links(A, V ).
The quantity links(A, A) = links(A, A), where A = V
A denotes the complement of A in V , measures how
many links escape from A (and A), and the quantity
links(A, A) measures how many links stay within A itself.
17.1. DIRECTED GRAPHS, UNDIRECTED GRAPHS, WEIGHTED GRAPHS
749
The quantity
cut(A) = links(A, A)
is often called the cut of A, and the quantity
assoc(A) = links(A, A)
is often called the association of A. Clearly,
cut(A) + assoc(A) = vol(A).
Weight of a cut:
cut(A,B) = i A, j
B
Wi,j
20
Figure 17.5: A Cut involving the set of nodes in the center and the nodes on the perimeter.
We now define the most important concept of these notes:
The Laplacian matrix of a graph. Actually, as we will see,
it comes in several flavors.
750
17.2
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
Laplacian Matrices of Graphs
Let us begin with directed graphs, although as we will
see, graph Laplacians are fundamentally associated with
undirected graph.
The key proposition below shows how BB > relates to the
adjacency matrix A. We reproduce the proof in Gallier
[15] (see also Godsil and Royle [17]).
Proposition 17.2. Given any directed graph G if B
is the incidence matrix of G, A is the adjacency matrix of G, and D is the degree matrix such that Di i =
d(vi), then
BB > = D A.
Consequently, BB > is independent of the orientation
of G and D A is symmetric, positive, semidefinite;
that is, the eigenvalues of D A are real and nonnegative.
17.2. LAPLACIAN MATRICES OF GRAPHS
751
The matrix L = BB > = D A is called the (unnormalized) graph Laplacian of the graph G.
For example, the graph Laplacian of
0
2
1 1 0
B 1 4
1 1
B
1
L=B
B 1 1 3
@0
1 1 3
0
1 0
1
graph G1 is
1
0
1C
C
0C
C.
1A
2
The (unnormalized) graph Laplacian of an undirected
graph G = (V, E) is defined by
L=D
A.
Observe that each row of L sums to zero (because
B >1 = 0). Consequently, the vector 1 is in the nullspace
of L.
752
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
Remark: With the unoriented version of the incidence
matrix (see Definition 17.6), it can be shown that
BB > = D + A.
The natural generalization of the notion of graph Laplacian to weighted graphs is this:
Definition 17.10. Given any weighted graph G = (V, W )
with V = {v1, . . . , vm}, the (unnormalized) graph Laplacian L(G) of G is defined by
L(G) = D(G)
W,
where D(G) = diag(d1, . . . , dm) is the degree matrix of
G (a diagonal matrix), with
di =
m
X
wi j .
j=1
As usual, unless confusion arises, we write L instead of
L(G).
17.2. LAPLACIAN MATRICES OF GRAPHS
753
The graph Laplacian can be interpreted as a linear map
from RV to itself. For all x 2 RV , we have
X
(Lx)i =
wij (xi xj ).
j⇠i
It is clear that each row of L sums to 0, so the vector 1 is
the nullspace of L, but it is less obvious that L is positive
semidefinite. One way to prove it is to generalize slightly
the notion of incidence matrix.
Definition 17.11. Given a weighted graph G = (V, W ),
with V = {v1, . . . , vm}, if {e1, . . . , en} are the edges of
the underlying graph of G (recall that {vi, vj } is an edge
of this graph i↵ wij > 0), for any oriented graph G obtained by giving an orientation to the underlying graph
of G, the incidence matrix B of G is the m⇥n matrix
whose entries bi j are given by
8 p
>
<+ wij if s(ej ) = vi
p
bi j =
wij if t(ej ) = vi
>
:
0
otherwise.
754
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
For example, given the weight
0
0 3
B3 0
W =B
@6 0
3 3
matrix
1
6 3
0 3C
C,
0 3A
3 0
the incidence matrix B corresponding to the orientation
of the underlying graph of W where an edge (i, j) is oriented positively i↵ i < j is
0
1
1.7321 2.4495 1.7321
0
0
B 1.7321
0
0
1.7321
0 C
B
C.
B=@
0
2.4495
0
0
1.7321 A
0
0
1.7321 1.7321 1.7321
The reader should verify that BB > = D
true in general, see Proposition 17.3.
W . This is
It is easy to see that Proposition 17.1 applies to the underlying graph of G.
17.2. LAPLACIAN MATRICES OF GRAPHS
755
For any oriented graph G obtained from the underlying
graph of G, the rank of the incidence matrix B is equal
to m c, where c is the number of connected components
of the underlying graph of G, and we have (B )>1 = 0.
We also have the following version of Proposition 17.2
whose proof is immediately adapted.
Proposition 17.3. Given any weighted graph G =
(V, W ) with V = {v1, . . . , vm}, if B is the incidence
matrix of any oriented graph G obtained from the
underlying graph of G and D is the degree matrix of
W , then
B (B )> = D W = L.
Consequently, B (B )> is independent of the orientation of the underlying graph of G and L = D W is
symmetric, positive, semidefinite; that is, the eigenvalues of L = D W are real and nonnegative.
756
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
Another way to prove that L is positive semidefinite is to
evaluate the quadratic form x>Lx.
Proposition 17.4. For any m ⇥ m symmetric matrix
W = (wij ), if we let L = D W where D is the degree
matrix associated with W , then we have
m
X
1
x>Lx =
wi j (xi
2 i,j=1
x j )2
for all x 2 Rm.
Consequently, x>Lx does not depend on the diagonal
entries in W , and if wi j 0 for all i, j 2 {1, . . . , m},
then L is positive semidefinite.
Proposition 17.4 immediately implies the following facts:
For any weighted graph G = (V, W ),
1. The eigenvalues 0 = 1  2  . . .  m of L are
real and nonnegative, and there is an orthonormal
basis of eigenvectors of L.
2. The smallest eigenvalue 1 of L is equal to 0, and 1
is a corresponding eigenvector.
17.2. LAPLACIAN MATRICES OF GRAPHS
757
It turns out that the dimension of the nullspace of L
(the eigenspace of 0) is equal to the number of connected
components of the underlying graph of G. This is an
immediate consequence of Proposition Proposition 17.1
and the fact that L = BB >.
Proposition 17.5. Let G = (V, W ) be a weighted
graph. The number c of connected components K1, . . .,
Kc of the underlying graph of G is equal to the dimension of the nullspace of L, which is equal to the multiplicity of the eigenvalue 0. Furthermore, the nullspace
of L has a basis consisting of indicator vectors of the
connected components of G, that is, vectors (f1, . . . , fm)
such that fj = 1 i↵ vj 2 Ki and fj = 0 otherwise.
Proposition 17.5 implies that if the underlying graph of G
is connected, then the second eigenvalue 2 of L is strictly
positive.
758
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
Remarkably, the eigenvalue 2 contains a lot of information about the graph G (assuming that G = (V, E) is an
undirected graph).
This was first discovered by Fiedler in 1973, and for this
reason, 2 is often referred to as the Fiedler number .
For more on the properties of the Fiedler number, see
Godsil and Royle [17] (Chapter 13) and Chung [9].
More generally, the spectrum (0, 2, . . . , m) of L contains a lot of information about the combinatorial structure of the graph G. Leverage of this information is the
object of spectral graph theory.
It turns out that normalized variants of the graph Laplacian are needed, especially in applications to graph clustering.
17.2. LAPLACIAN MATRICES OF GRAPHS
759
These variants make sense only if G has no isolated vertices, which means that every row of W contains some
strictly positive entry.
In this case, the degree matrix D contains positive entries,
so it is invertible and D 1/2 makes sense; namely
D
1/2
= diag(d1
1/2
, . . . , dm1/2),
and similarly for any real exponent ↵.
Definition 17.12. Given any weighted directed graph
G = (V, W ) with no isolated vertex and with V =
{v1, . . . , vm}, the (normalized) graph Laplacians Lsym
and Lrw of G are defined by
Lsym = D 1/2LD
Lrw = D 1L = I
1/2
=I D
D 1W.
1/2
WD
1/2
Observe that the Laplacian Lsym = D 1/2LD 1/2 is a
symmetric matrix (because L and D 1/2 are symmetric)
and that
Lrw = D 1/2LsymD1/2.
760
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
The reason for the notation Lrw is that this matrix is
closely related to a random walk on the graph G.
Since the unnormalized Laplacian L can be written as
L = BB >, where B is the incidence matrix of any oriented graph obtained from the underlying graph of G =
(V, W ), if we let
Bsym = D
1/2
B,
we get
>
Lsym = BsymBsym
.
In particular, for any singular decomposition Bsym =
U ⌃V > of Bsym (with U an m ⇥ m orthogonal matrix,
⌃ a “diagonal” m ⇥ n matrix of singular values, and V
an n ⇥ n orthogonal matrix), the eigenvalues of Lsym are
the squares of the top m singular values of Bsym, and the
vectors in U are orthonormal eigenvectors of Lsym with
respect to these eigenvalues (the squares of the top m
diagonal entries of ⌃).
Computing the SVD of Bsym generally yields more accurate results than diagonalizing Lsym, especially when
Lsym has eigenvalues with high multiplicity.
17.2. LAPLACIAN MATRICES OF GRAPHS
761
Proposition 17.6. Let G = (V, W ) be a weighted
graph without isolated vertices. The graph Laplacians,
L, Lsym, and Lrw satisfy the following properties:
(1) The matrix Lsym is symmetric, positive, semidefinite. In fact,
!2
m
1X
xi
xj
>
p
x Lsymx =
wi j p
x 2 Rm .
2 i,j=1
di
dj
(2) The normalized graph Laplacians Lsym and Lrw have
the same spectrum
(0 = ⌫1  ⌫2  . . .  ⌫m), and a vector u 6= 0 is an
eigenvector of Lrw for i↵ D1/2u is an eigenvector
of Lsym for .
(3) The graph Laplacians, L, Lsym, and Lrw are symmetric, positive, semidefinite.
(4) A vector u 6= 0 is a solution of the generalized
eigenvalue problem Lu = Du i↵ D1/2u is an eigenvector of Lsym for the eigenvalue i↵ u is an eigenvector of Lrw for the eigenvalue .
762
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS
(5) The graph Laplacians, L and Lrw have the same
nullspace. For any vector u, we have u 2 Ker (L)
i↵ D1/2u 2 Ker (Lsym).
(6) The vector 1 is in the nullspace of Lrw , and D1/21
is in the nullspace of Lsym.
(7) For every eigenvalue ⌫i of the normalized graph
Laplacian Lsym, we have 0  ⌫i  2. Furthermore,
⌫m = 2 i↵ the underlying graph of G contains a
nontrivial connected bipartite component.
(8) If m 2 and if the underlying graph of G is not
a complete graph, then ⌫2  1. Furthermore the
underlying graph of G is a complete graph i↵ ⌫2 =
m
m 1.
(9) If m 2 and if the underlying graph of G is connected then ⌫2 > 0.
(10) If m 2 and if the underlying graph of G has no
isolated vertices, then ⌫m mm 1 .
17.2. LAPLACIAN MATRICES OF GRAPHS
763
A version of Proposition 17.5 also holds for the graph
Laplacians Lsym and Lrw .
This follows easily from the fact that Proposition 17.1
applies to the underlying graph of a weighted graph. The
proof is left as an exercise.
Proposition 17.7. Let G = (V, W ) be a weighted
graph. The number c of connected components K1, . . .,
Kc of the underlying graph of G is equal to the dimension of the nullspace of both Lsym and Lrw , which
is equal to the multiplicity of the eigenvalue 0. Furthermore, the nullspace of Lrw has a basis consisting
of indicator vectors of the connected components of
G, that is, vectors (f1, . . . , fm) such that fj = 1 i↵
vj 2 Ki and fj = 0 otherwise. For Lsym, a basis of the
nullpace is obtained by multiplying the above basis of
the nullspace of Lrw by D1/2.
764
CHAPTER 17. GRAPHS AND GRAPH LAPLACIANS