Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Spectral Graph Theory,
Linear Solvers, and
Applications
Gary Miller
Carnegie Mellon University
joiny work with Yiannis Koutis and David Tolliver
Theory and Practice of Computational Learning
June 9, 2009
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Linear Systems
3x 2y −z = 3
2x −5y 4z = 7
−x 1/2y 2z = 2
Fundamental Constraint
System
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Matrix Form


 
3 2 −1
x
3
 2 −5 4   y  =  7 
−1 1/2 2
z
2
Gary L. Miller

Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Solving the General Case
•
Dense Case: O(n3 ) or O(n2.81 ) Strassen.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Solving the General Case
•
•
Dense Case: O(n3 ) or O(n2.81 ) Strassen.
Sparse Case: Still O(n2.81 ).
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
An Easy Case
•
Upper and Lower Triangular Systems
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
An Easy Case
•
•
Upper and Lower Triangular Systems
O(m) time where m = number of nonzeros
entries.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
An Easy Case
•
•
•
Upper and Lower Triangular Systems
O(m) time where m = number of nonzeros
entries.
Goal: Find more easy cases.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Symmetric Matrices
•
Assume A is symmetric, A = AT .
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Symmetric Matrices
•
•
Assume A is symmetric, A = AT .
Assume A is positive definite, xT Ax > 0 for
x 6= 0
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Direct Methods
Gaussian Elimination Matrices
•
Goal: algorithms that minimize work and
space.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Direct Methods
Gaussian Elimination Matrices
•
•
Goal: algorithms that minimize work and
space.
Trick: View nonzero entries as an undirected
graph and view pivoting as a graph
operation.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Pivoting
Viewing pivoting as a graph operation.
•
Let G = (V, E) be a graph and v a vertex.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Pivoting
Viewing pivoting as a graph operation.
•
•
Let G = (V, E) be a graph and v a vertex.
P IV OT (v) :
• Make a clique out of neighbors of v.
• Remove v.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Pivoting
Viewing pivoting as a graph operation.
•
•
Let G = (V, E) be a graph and v a vertex.
P IV OT (v) :
• Make a clique out of neighbors of v.
• Remove v.
•
Fill: New edges formed.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Pivoting
Viewing pivoting as a graph operation.
•
•
Let G = (V, E) be a graph and v a vertex.
P IV OT (v) :
• Make a clique out of neighbors of v.
• Remove v.
•
•
Fill: New edges formed.
Work: All edges “touched”.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Good Pivot Strategies
1970s and 1980s
• Planar systems: O(n3/2 ) work and O(n log n)
fill/space.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Good Pivot Strategies
1970s and 1980s
• Planar systems: O(n3/2 ) work and O(n log n)
fill/space.
• 3D Systems: O(n2 ) work and O(n3/2 )
fill/space.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Good Pivot Strategies
1970s and 1980s
• Planar systems: O(n3/2 ) work and O(n log n)
fill/space.
• 3D Systems: O(n2 ) work and O(n3/2 )
fill/space.
• O(n3/2 ) space is a problem for ML size
problems.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Pure Iterative Methods
Solving Ax = b.
• Basic method: x(i+1) = (I − A)x(i) + b
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Pure Iterative Methods
Solving Ax = b.
• Basic method: x(i+1) = (I − A)x(i) + b
• Convergence/Rate is determined by ||I − A||.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Pure Iterative Methods
Solving Ax = b.
• Basic method: x(i+1) = (I − A)x(i) + b
• Convergence/Rate is determined by ||I − A||.
• Accelerated Methods: Chebyshev Iteration,
Conjugate Gradient.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Pure Iterative Methods
Solving Ax = b.
• Basic method: x(i+1) = (I − A)x(i) + b
• Convergence/Rate is determined by ||I − A||.
• Accelerated Methods: Chebyshev Iteration,
Conjugate Gradient.
• CG is still too slow.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Preconditioned Iterative Methods
Solving B −1 Ax = B −1 b = b0 .
• Basic method: x(i+1) = (I − B −1 A)x(i) + b0
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Preconditioned Iterative Methods
Solving B −1 Ax = B −1 b = b0 .
• Basic method: x(i+1) = (I − B −1 A)x(i) + b0
• Computing the term z = B −1 Ax(i)
(i)
• y = Ax
Forward Multiply
• Bz = y
Solve the preconditioner system
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Preconditioned Iterative Methods
Solving B −1 Ax = B −1 b = b0 .
• Basic method: x(i+1) = (I − B −1 A)x(i) + b0
• Computing the term z = B −1 Ax(i)
(i)
• y = Ax
Forward Multiply
• Bz = y
Solve the preconditioner system
• Goal: Minimize the number of iteration while
minimizing the cost of the solve.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Classic Preconditioners
•
Jacobi: B = Diagonal(A).
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Classic Preconditioners
•
•
Jacobi: B = Diagonal(A).
Gauss-Seidel: B = U pperT riangular(A).
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Classic Preconditioners
•
•
•
Jacobi: B = Diagonal(A).
Gauss-Seidel: B = U pperT riangular(A).
SSOR: B = (L + ω1 D) ω1 D(L + ω1 D)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Classic Preconditioners
•
•
•
•
Jacobi: B = Diagonal(A).
Gauss-Seidel: B = U pperT riangular(A).
SSOR: B = (L + ω1 D) ω1 D(L + ω1 D)
Still too slow and unreliable.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian
• G = (V, E, w) weighted undirected graph, wij > 0.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian
• G = (V, E, w) weighted undirected graph, wij > 0.
• Weighted incidence matrix:
Aij =
Gary L. Miller
wij
0
if eij ∈ E
otherwise
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian
• G = (V, E, w) weighted undirected graph, wij > 0.
• Weighted incidence matrix:
Aij =
• Degree of vi : di =
P
j
wij
0
if eij ∈ E
otherwise
wij
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian
• G = (V, E, w) weighted undirected graph, wij > 0.
• Weighted incidence matrix:
Aij =
• Degree of vi : di =
•
P
j
wij
0
if eij ∈ E
otherwise
d1
0
wij

..

D=
0
Gary L. Miller



.
dn
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian
• G = (V, E, w) weighted undirected graph, wij > 0.
• Weighted incidence matrix:
Aij =
• Degree of vi : di =
P
j
•
wij
0
if eij ∈ E
otherwise
d1
0
wij

..

D=
0



.
dn
• Laplacian: L = D − A
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Classic Applications of the Laplacian
• View each edge a conductor with conductance wij .
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Classic Applications of the Laplacian
• View each edge a conductor with conductance wij .
• Let V be a column vector of voltages
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Classic Applications of the Laplacian
• View each edge a conductor with conductance wij .
• Let V be a column vector of voltages
• If LV = c the c residual current needed to maintain the
given voltages.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian’s and the Heat
Equations
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian’s and Random
Walks
Transition Matrix: D−1 LG
Fundamental Eigenvectors: Õ(n + m) (Spielman Teng)
Trick: Inverse Powering only requires O(log n) iterations.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Laplacian’s and Spring Mass Systems
•
G = (V, E, w) weighted graph and wij is
viewed a spring constant.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Laplacian’s and Spring Mass Systems
•
•
G = (V, E, w) weighted graph and wij is
viewed a spring constant.
M is a diagonal matrix of mass constants
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Laplacian’s and Spring Mass Systems
•
•
•
G = (V, E, w) weighted graph and wij is
viewed a spring constant.
M is a diagonal matrix of mass constants
Fact: Modes of vibration of Spring-Mass
system G, M are:
Eigen-pairs of LG x = λM x.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Spring Mass System
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian’s and Maximum
Flow
Graph Maximum Flow: Õ((m + n)3/2 ) (Daitch Spielman)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Graph Laplacians
Applications
Graph Laplacian’s and Convex
Programming
Nonuniform TV Denoising: Õ((m + n)3/2 ) (Koutis M Sinop
Tolliver)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Spanning Tree Preconditioners
•
Vaidya ’93: Use Maximum Weight Spanning
Tree (MST) plus a few edges.
j
b
a
i
h
d
c
e
Gary L. Miller
f
g
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Spanning Tree Preconditioners
•
Advantages: Easy to find and Easy to solve
their systems.
j
b
a
i
h
d
c
e
Gary L. Miller
f
g
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Spanning Tree Preconditioners
•
Problem: Small edge weights differences
can make MST bad.
j
b
a
i
h
d
c
e
Gary L. Miller
f
g
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Low Stretch Spanning Trees
•
EEST ’05: Use low stretch spanning trees
plus a few edges.
j
b
a
i
h
d
c
e
Gary L. Miller
f
g
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Low Stretch Spanning Trees
•
Advantages: Better condition number.
j
b
a
i
h
d
c
e
Gary L. Miller
f
g
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Low Stretch Spanning Trees
•
Problem: Super linear time to find.
j
b
a
i
h
d
c
e
Gary L. Miller
f
g
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Low Stretch Spanning Trees
•
Richter-M ’04: There are no good spanning
trees even for square mesh.
j
b
a
i
h
d
c
e
Gary L. Miller
f
g
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Steiner Tree Preconditioners
•
Gremban-M ’94: Use Steiner trees.
j
b
a
i
h
d
c
e
f
g
f
g
a
b
e
h
c
Gary L. Miller
i
j
d
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Steiner Tree Preconditioners
•
Advantages: Better condition number for
graphs like square mesh.
j
b
a
i
h
d
c
e
f
g
f
g
a
b
e
h
c
Gary L. Miller
i
j
d
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Steiner Tree Preconditioners
•
Advantages: Good experimental results.
j
b
a
i
h
d
c
e
f
g
f
g
a
b
e
h
c
Gary L. Miller
i
j
d
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Steiner Tree Preconditioners
•
Problem: Hard to construct in general and
analyze.
j
b
a
i
h
d
c
e
f
g
f
g
a
b
e
h
c
Gary L. Miller
i
j
d
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Steiner Forest Preconditioners
•
Koutis-M ’07: Use Steiner Forest.
j
b
a
i
h
d
c
e
f
g
f
g
a
b
e
h
c
Gary L. Miller
i
j
d
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Steiner Forest Preconditioners
•
Advantages: Easy to find and works well
with recursive solvers.
j
b
a
i
h
d
c
e
f
g
f
g
a
b
e
h
c
Gary L. Miller
i
j
d
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Steiner Forest Preconditioners
•
Advantages: Good experimental results.
j
b
a
i
h
d
c
e
f
g
f
g
a
b
e
h
c
Gary L. Miller
i
j
d
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Steiner Forest Preconditioners
•
Problem: Analysis only for planar systems.
j
b
a
i
h
d
c
e
f
g
f
g
a
b
e
h
c
Gary L. Miller
i
j
d
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Recursive Methods
•
Vaidya ’93: Idea: Tree have a lot of degree
1-2 degree nodes. Pivot on these nodes and
then find a preconditioner for this graph.
a
b
c
d
Gary L. Miller
e
f
g
h
i
j
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Recursive Methods
•
Vaidya ’93: Idea: Tree have a lot of degree
1-2 degree nodes. Pivot on these nodes and
then find a preconditioner for this graph.
The reduced graph.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
The Error
•
Recurrence: u(i+1) = Gu(i) + b, for us
G = I − A.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
The Error
•
•
Recurrence: u(i+1) = Gu(i) + b, for us
G = I − A.
Error: e(i) = u(i) − ū, where ū = Gū + b
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
The Error
•
•
•
Recurrence: u(i+1) = Gu(i) + b, for us
G = I − A.
Error: e(i) = u(i) − ū, where ū = Gū + b
Fact: e(i) = Gi e(0)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
The Error
•
•
•
•
Recurrence: u(i+1) = Gu(i) + b, for us
G = I − A.
Error: e(i) = u(i) − ū, where ū = Gū + b
Fact: e(i) = Gi e(0)
We need: limi→∞ Gi = 0
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Condition Number
•
Ax = λx
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Condition Number
•
•
Ax = λx
DEF: λ Eigenvalue and x Eigenvector.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Condition Number
•
•
•
Ax = λx
DEF: λ Eigenvalue and x Eigenvector.
Λ(A) = {0 ≤ λ1 ≤ · · · λn }.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Condition Number
•
•
•
•
Ax = λx
DEF: λ Eigenvalue and x Eigenvector.
Λ(A) = {0 ≤ λ1 ≤ · · · λn }.
Condition Number: κ(A) = λn /λ1
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Convergence Rates
•
Basic Method: Convergence Rate
= O(1/κ(A))
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Convergence Rates
•
•
Basic Method: Convergence Rate
= O(1/κ(A))
p
Conjugate Gradient: O(1/ κ(A))
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Convergence Rates
•
•
•
Basic Method: Convergence Rate
= O(1/κ(A))
p
Conjugate Gradient: O(1/ κ(A))
Conjugate Gradient: ≈ 1/diameter(A).
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Generalized Condition Number
•
Goal: Bound condition number of B −1 A
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Generalized Condition Number
•
•
Goal: Bound condition number of B −1 A
Note: B −1 Ax = λx iff Ax = λBx
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Generalized Condition Number
•
•
•
Goal: Bound condition number of B −1 A
Note: B −1 Ax = λx iff Ax = λBx
DEF: λ is a generalized eigenvalue.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Generalized Condition Number
•
•
•
•
Goal: Bound condition number of B −1 A
Note: B −1 Ax = λx iff Ax = λBx
DEF: λ is a generalized eigenvalue.
Condition Number: κ(B −1 A) = λn /λ1
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
The Support
•
Positive Semi-Definite: ∀x xT Ax ≥ 0
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
The Support
•
•
Positive Semi-Definite: ∀x xT Ax ≥ 0
Support of A by B:
σ(A/B) = min{τ : τ B − A is PSD}
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
The Support
•
•
•
Positive Semi-Definite: ∀x xT Ax ≥ 0
Support of A by B:
σ(A/B) = min{τ : τ B − A is PSD}
Fact: κ(A, B) = σ(A, B) · σ(B, A).
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Estimating Support
•
•
G and H are graphs and V (G) = V (H)
Path Embedding: φ : E(G) then paths(H)
s.t. φ(eij ) = Vi · · · Vj .
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Estimating Support Example
EG: G ≡ K4 and H ≡ 4-cycle
Gary L. Miller
Spectral≡
Graph
Congestion ≡ 3 and
Dilation
2 Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
History of Planar Solvers
•
1950’s O(n2 ) (Conjugate Gradient)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
History of Planar Solvers
•
•
1950’s O(n2 ) (Conjugate Gradient)
1970’s O(n1.5 ) (Nested Dissection) (LRT)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
History of Planar Solvers
•
•
•
1950’s O(n2 ) (Conjugate Gradient)
1970’s O(n1.5 ) (Nested Dissection) (LRT)
1990’s O(n1.2 ) (Combinatorial
Preconditioners) (Vaidya)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
History of Planar Solvers
•
•
•
•
1950’s O(n2 ) (Conjugate Gradient)
1970’s O(n1.5 ) (Nested Dissection) (LRT)
1990’s O(n1.2 ) (Combinatorial
Preconditioners) (Vaidya)
2000’s O(n log2 n) (Low stretch spanning
trees) (ST)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
History of Planar Solvers
•
•
•
•
•
1950’s O(n2 ) (Conjugate Gradient)
1970’s O(n1.5 ) (Nested Dissection) (LRT)
1990’s O(n1.2 ) (Combinatorial
Preconditioners) (Vaidya)
2000’s O(n log2 n) (Low stretch spanning
trees) (ST)
2006’s O(n) (separator based
preconditioners) (KM)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
General Laplacian Solver
Õ(n + m) (Spielman Teng)
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Two dimensional images
20
18
Running Time (secs)
16
14
12
10
8
Our solver
MATLAB’s direct solver
6
4
2
0
0
0.2
0.4
0.6
0.8
1
1.2
Number of Pixels (in millions)
Gary L. Miller
1.4
1.6
1.8
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Combinatorial Preconditioners
Recursive Preconditioned Methods
Analysis
Run Times
Three dimensional images
12
Running Time (seconds)
10
Out of memory
8
6
Our solver
direct solver
4
2
0
0
0.1
0.2
0.3
0.4
0.5
0.6
Number of Pixels (in millions)
Gary L. Miller
0.7
0.8
0.9
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Symmetric Diagonally Dominate
Systems
•
Goal: Show how to solve SDD using regular
Laplacians.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Symmetric Diagonally Dominate
Systems
•
•
Goal: Show how to solve SDD using regular
Laplacians.
Let G = (V, E, w) such that ij ∈ E:wij 6= 0
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Symmetric Diagonally Dominate
Systems
•
•
•
Goal: Show how to solve SDD using regular
Laplacians.
Let G = (V, E, w) such that ij ∈ E:wij 6= 0
Weighted incidence matrix: A.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Symmetric Diagonally Dominate
Systems
•
•
•
•
Goal: Show how to solve SDD using regular
Laplacians.
Let G = (V, E, w) such that ij ∈ E:wij 6= 0
Weighted incidence
Pmatrix: A.
Degree of vi : di = j |wij |
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Symmetric Diagonally Dominate
Systems
•
•
•
•
•
Goal: Show how to solve SDD using regular
Laplacians.
Let G = (V, E, w) such that ij ∈ E:wij 6= 0
Weighted incidence
Pmatrix: A.
Degree of vi : di = j |wij |
Generalized Laplacian: L = D − A
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Symmetric Diagonally Dominate
Systems
•
•
•
•
•
•
Goal: Show how to solve SDD using regular
Laplacians.
Let G = (V, E, w) such that ij ∈ E:wij 6= 0
Weighted incidence
Pmatrix: A.
Degree of vi : di = j |wij |
Generalized Laplacian: L = D − A
Note: Every SDD is a Generalized
Laplacian.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Basic Properties: Generalized
Laplacian
•
T
x
P Lx =
P
2
2
wij >0 wij (xi − xj ) −
wij <0 wij (xi + xj )
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Basic Properties: Generalized
Laplacian
•
•
T
x
P Lx =
P
2
2
wij >0 wij (xi − xj ) −
wij <0 wij (xi + xj )
Thus L is positive semidefinite.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Basic Properties: Generalized
Laplacian
•
•
•
T
x
P Lx =
P
2
2
wij >0 wij (xi − xj ) −
wij <0 wij (xi + xj )
Thus L is positive semidefinite.
Claim: Rank = n − 1 if G is connected.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Solving Generalized Laplacian by
Change of Variables
•
First Idea: Find a change of variables it get a
regular Laplacian.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Solving Generalized Laplacian by
Change of Variables
•
•
First Idea: Find a change of variables it get a
regular Laplacian.
Note: multiplying the ith column and row by
−1 preserves Laplacian.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Solving Generalized Laplacian by
Change of Variables
•
•
•
First Idea: Find a change of variables it get a
regular Laplacian.
Note: multiplying the ith column and row by
−1 preserves Laplacian.
This is just Flipping xi and bi to −xi and −bi .
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Generalized Laplacians Example
V1
−1
+1
V3
−1
V2

  
x1
b1
2 +1 −1
 +1 2 +1   x2  =  b2 
x3
b3
−1 +1 2

Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Change of Variables for Laplacians
V1
−1
+1
−1 +1
V3
−1
V2


 

2 −1 +1
−x1
−b1
 −1 2 +1   x2  =  b2 
x3
b3
+1 +1 2
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Orientable Generalized Laplacians
•
DEF: G = (V, E, w) is orientable is ∃
sequence of flips s.t. w > 0.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Orientable Generalized Laplacians
•
•
DEF: G = (V, E, w) is orientable is ∃
sequence of flips s.t. w > 0.
DEF: LG is orientable if G is.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Orientable Generalized Laplacians
•
•
•
DEF: G = (V, E, w) is orientable is ∃
sequence of flips s.t. w > 0.
DEF: LG is orientable if G is.
Note: Orientability is linear testable, greedy.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Orientable Generalized Laplacians
•
Claim: If G is connected and not orientable
then L is SPD.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Orientable Generalized Laplacians
•
•
Claim: If G is connected and not orientable
then L is SPD.
Proof: Suppose xT Lx = 0 and x 6= 0
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Orientable Generalized Laplacians
•
•
•
Claim: If G is connected and not orientable
then L is SPD.
Proof: Suppose xT Lx = 0 and x 6= 0
Pick a spanning tree T of G and orient it and
flipping x.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Orientable Generalized Laplacians
•
•
•
•
Claim: If G is connected and not orientable
then L is SPD.
Proof: Suppose xT Lx = 0 and x 6= 0
Pick a spanning tree T of G and orient it and
flipping x.
WLOG: x is the all-ones vector, a contra!,
since G still has negative edges.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Two-Fold Covers
•
DEF: The 2-fold cover Ḡ = (V̄ , W̄ , w̄) of
G = (V, W, w) is:
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Two-Fold Covers
•
•
DEF: The 2-fold cover Ḡ = (V̄ , W̄ , w̄) of
G = (V, W, w) is:
V̄ = {V1 , V¯1 , . . . , Vn , V¯n }
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Two-Fold Covers
•
•
•
DEF: The 2-fold cover Ḡ = (V̄ , W̄ , w̄) of
G = (V, W, w) is:
V̄ = {V1 , V¯1 , . . . , Vn , V¯n }
Ē: If wij > 0
add edges < Vi , Vj > and < V̄i , V¯j > with
weight wij
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Two-Fold Covers
•
•
•
•
DEF: The 2-fold cover Ḡ = (V̄ , W̄ , w̄) of
G = (V, W, w) is:
V̄ = {V1 , V¯1 , . . . , Vn , V¯n }
Ē: If wij > 0
add edges < Vi , Vj > and < V̄i , V¯j > with
weight wij
Ē: If wij < 0
add edges < Vi , V¯j > and < V̄i , Vj > with
weight −wij
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Example: Two-Fold Cover
V1
−1
+1
V3
−1
V2
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Two-Fold Covers and Solving
Generalized Laplacians
•
Let L = L(G) and L̄ = Ḡ
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Two-Fold Covers and Solving
Generalized Laplacians
•
•
Let L = L(G) and L̄ = Ḡ
L̄ is a regular Laplacian which we can solve
quickly.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Two-Fold Covers and Solving
Generalized Laplacians
•
•
Let L = L(G) and L̄ = Ḡ
Note:
x
b
Lx = b then L̄
=
−x
−b
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Two-Fold Covers and Solving
Generalized Laplacians
•
•
Let L = L(G) and L̄ = Ḡ
Note:
x
b
Lx = b then L̄
=
−x
−b
•
L̄
x
y
=
b
−b
Gary L. Miller
then L(x/2 − y/2) = b
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
SDD systems
•
The 2-fold trick can be run without the factor
of two in space and time.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
SDD systems
•
•
The 2-fold trick can be run without the factor
of two in space and time.
There should be uses of negative weights in
recommendation problems.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
SDD systems
•
•
•
The 2-fold trick can be run without the factor
of two in space and time.
There should be uses of negative weights in
recommendation problems.
Naive approach does not seem to work right.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Spectral Graph Partitioning
•
Idea: Pick a few low frequency eigenvectors.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Spectral Graph Partitioning
•
•
Idea: Pick a few low frequency eigenvectors.
Use these vectors to embed the graph in Rd
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
The Blue Sky Problem
Shi Malik applied to an image:
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
The Blue Sky Problem
Shi Malik solution:
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
The Blue Sky Problem
Spectral Rounding applied to Image:
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Spectral Rounding
Edge Reweighting
Algorithm:
• Solve Lf = λ2 Df .
• Reweight graph edges getting L0 and D 0 .
• Solve L0 f = λ2 D 0 f
• Repeat while λ2 6= 0.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
SR: The reweighting scheme
•
•
•
•
•
View D, f , and λ2 as a function of L
Subject to Lf = λ2 Df and f T Df = 1.
∂λ2
= (fi − fj )2 − λ2 (fi2 + fj2 )
We get: ∂e
ij
Take a “small” step in the direction of the
gradient.
If an edge goes negative set it to zero.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Medical Examples of SR
Breast Tumors
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
Medical Examples of SR
Breast Tumors
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications
Introduction
Graph Based Methods
Iterative Methods for Laplacians
Symmetric Diagonally Dominate Systems
Spectral Rounding
Open Questions
•
•
•
•
Find fast methods for any SPD system.
Find spectral methods that find better cut by
using more than one eigenvector.
Find solvers that work in the L2 norm.
A implementable solver with near linear time
guarantees.
Gary L. Miller
Spectral Graph Theory, Linear Solvers, and Applications