Lecture 22:
Matrix Operations and All-pair
Shortest Paths II
Shang-Hua Teng
Matrix Multiplication
b11b12b1 p 
a11a12a1n 


a a a 
b
b

b
21 22
2p 
A   21 22 2 n ,B  
,

 


 





bn1bn 2bnp 
am1 am 2amn 
n a1i  bi1 n a1i  bi 2  n a1i  bip 
i 1
i 1
 i 1

n
n
n


a

b
a

b

a

b



2
i
i
1
2
i
i
2
2
i
ip
i 1
i 1

A  B   i 1
 



 n

n

i =1 ami  bi1
i 1 ami  bip 
Add and Multiply
• Rings: ,,
• Commutative, Associative
• Distributive
• Other rings
, min,  
{0,1},, 
Matrix Multiplication Can be
Defined on any Ring
Adjacency Matrix wit h
{0,1},, 
Connectivi ty
Weighted Matrix wit h
, min,  
Shortest Paths
Two Graph Problems
• Transitive closure: whether there exists a path between
every pair of vertices
– generate a matrix closure showing all transitive closures
– for instance, if a path exists from i to j, then closure[i, j] =1
• All-pair shortest paths: shortest paths between every
pair of vertices
– Doing better than Bellman-Ford O(|V|2|E|)
• They are very similar
Transitive Closure
• Given a digraph G, the transitive
closure of G is the digraph G*
such that
– G* has the same vertices as G
– if G has a directed path from u
to v (u  v), G* has a directed
edge from u to v
• The transitive closure provides
reachability information about a
digraph
D
E
B
C
G
A
D
E
B
C
A
G*
Transitive Closure and Matrix
Multiplication
• Let A be the adjacency matrix of a graph G
• What is A2 , where the matrix multiplica tion
is defined by {0,1},, ?
1
2
A 
2
ij
4
 ( Aik  Akj )
k
1
2
4
-3
3
3
0

0
0

1

1 0 0

0 1 1
0 0 0


0 1 0
A
Floyd-Warshall, Iteration 2
v7
BOS
ORD
v4
JFK
v2
v6
SFO
DFW
LAX
v1
v3
MIA
v5
Transitive Closure and Matrix
Multiplication
A 
2
ij
 1 iff there exists a path with two edges
that connects i to j
3
What is A ?
4
What is A ?
What is A  A2  ...  An 1 ?
  time algorithm
OV
4
Let A
2 
A Better Idea
 A A
In general, let A
2
k 
 A  A  ...  A
2
What is Ak  ?
Need to compute A
 n 1
?
What is A2  A2  ?
k  k 
What is A A ?

3

O V log V time algorithm
k
Even Better idea: Dynamic
Programming; Floyd-Warshall
• Number the vertices 1, 2, …, n.
• Consider paths that use only vertices numbered 1, 2,
…, k, as intermediate vertices:
Uses only vertices numbered 1,…,k
(add this edge if it’s not already in)
i
j
Uses only vertices
numbered 1,…,k-1
k
Uses only vertices
numbered 1,…,k-1
Floyd-Warshall’s Algorithm
A is the original matrix, T is the transitive matrix
TA
for(k=1:n)
for(j=1:n)
for(i=1:n)
T[i, j] = T[i, j] OR
(T[i, k] AND T[k, j])
• It should be obvious that the
complexity is (n3) because of
the 3 nested for-loops
• T[i, j] =1 if there is a path from
vertex i to vertex j
Floyd-Warshall Example
v7
BOS
ORD
v4
JFK
v2
v6
SFO
DFW
LAX
v1
v3
MIA
v5
Floyd-Warshall, Iteration 1
v7
BOS
ORD
v4
JFK
v2
v6
SFO
DFW
LAX
v1
v3
MIA
v5
Floyd-Warshall, Iteration 3
v7
BOS
ORD
v4
JFK
v2
v6
SFO
DFW
LAX
v1
v3
MIA
v5
Floyd-Warshall, Iteration 4
v7
BOS
ORD
v4
JFK
v2
v6
SFO
DFW
LAX
v1
v3
MIA
v5
Floyd-Warshall, Iteration 5
v7
BOS
ORD
v4
JFK
v2
v6
SFO
DFW
LAX
v1
v3
MIA
v5
Floyd-Warshall, Iteration 6
v7
BOS
ORD
v4
JFK
v2
v6
SFO
DFW
LAX
v1
v3
MIA
v5
Floyd-Warshall, Conclusion
v7
BOS
ORD
v4
JFK
v2
v6
SFO
DFW
LAX
v1
v3
MIA
v5
All Pair Shortest Paths
• Using matrix multiplication
• Using Dynamic Programming
Shortest Path and Matrix Multiplication
• Let W be the weighted matrix of a graph G
• What is W 2 , where the matrix multiplica tion
is defined by , min,  ?
1
2
W 
2
ij
4
 min (Wik  Wkj )
k
1
2
4
3
3
3
0




2

 

0  3 4
 0 

 3 0 
1
A
Shortest Paths and Matrix Multiplication
W 
2
ij
 is the length of the shortest path with two
edges that connects i to j
3
What is W ?
4
What is W ?
What is W min W 2 min ... min W n 1 ?
  time algorithm
OV
4
Let D
2 
A Better Idea
 W min W
In general, let D
k 
2
 W min W min ... min W
2
What is D k  ?
Need to compute D
 n 1
?
What is D 2  D 2  ?
k 
k 
What is D D ?

3

O V log V time algorithm
k
Even Better idea: Dynamic
Programming; Floyd-Warshall
• Number the vertices 1, 2, …, n.
• Consider paths that use only vertices numbered 1, 2,
…, k, as intermediate vertices:
Uses only vertices numbered 1,…,k
(add this edge if it’s not already in)
i
j
Uses only vertices
numbered 1,…,k-1
k
Uses only vertices
numbered 1,…,k-1
Floyd-Warshall Algorithm
• Dij(k) = length of shortest path from i to j with
intermediate vertices from {1, 2, ..., k}:
• (i, j)= Dij(n)
• Dynamic Programming: recurrence
– Dij(0) = Dij
– Dij(k) = min {Dij(k-1) , Dik(k-1) + Dkj(k-1) }
Dik(k-1)
k
Dkj(k-1)
i
Dij(k-1)
intermediate nodes in {1, 2, ..., k}
j
The Floyd-Warshall algorithm
d
wij
{ min( d
(k )
ij =
( k 1)
ij
, dik( k 1)  d kj( k 1) )
if k = 0
if k  1
Floyd-Warshall(W)
(n3)
1 n  rows[W]
2 D(0) = W
3 for k 1 to n
4
do for i 1 to n
5
do for j  1 to n
dij( k )  min( dij( k 1) , dik( k 1)  dkj( k 1) )
6
7 return D(n)
2
3
4
7
8
1
3
calculate D(0) ,D(1) ,D(2), D(3) ,D(4) and
2
D(5)
1
-4
-5
5
4
26
The matrix  can be constructed within Floyd-Warshall as follows:
 ij(0)={
Nil
if i = j or wij = 
i
if i  j and wij < 
 ij( k 1)
if dij( k 1)  dik( k 1)  dkj( k 1)
( k 1)
kj
if dij( k 1)  dik( k 1)  dkj( k 1)
{
 ij(k ) =
2
3
1
7
2
-4
5
4
8
1
3
-5
calculate (0) , (1) , (2), (3) , (4)
and (5)
4
27