Algorithms and Theory of Computation
Lecture 9: Dynamic Programming
All Pair Shortest Paths
Xiaohui Bei
MAS 714
September 6, 2016
Nanyang Technological University
MAS 714
September 6, 2016
1 / 20
Shortest Path and Negative Cycle Problems
Negative Cycle Problem
Given a directed weighted graph G = (V, E) with arbitrary edge lengths,
find a negative cycle (if one exists).
Shortest Path Problem
Given a directed weighted graph G = (V, E) with arbitrary edge lengths
and no negative cycles, find a shortest path from s to t.
Nanyang Technological University
MAS 714
September 6, 2016
2 / 20
Shortest Paths: Dynamic Programming
Sub-problem idea: paths of fewer hops/edges.
d(v, k): shortest path from s to v using at most k edges
dist(s, v) = d(v, n − 1)
Recursion for d(v, k)
1
If such shortest path P uses ≤ k − 1 edges: d(v, k) = d(v, k − 1).
2
If such shortest path P uses exactly k edges: let (u, v) be the last
edge, then P selects the best s ; u path using ≤ k − 1 edges.
d(v, k) = min
d(v, k − 1)
minu∈V (d(u, k − 1) + `(u, v))
Base case: d(s, 0) = 0 and d(v, 0) = ∞ for all v 6= s.
Nanyang Technological University
MAS 714
September 6, 2016
3 / 20
Bellman-Ford Algorithm
Algorithm: Bellman-Ford(G, s):
foreach u ∈ V do
d(u, 0) = ∞;
d(s, 0) = 0;
for k = 1 to n − 1 do
foreach v ∈ V do
d(v, k) = d(v, k − 1);
foreach edge (u, v) ∈ E do
d(v, k) = min{d(v, k), d(u, k − 1) + `(u, v)};
foreach v ∈ V do
dist(s, v) = d(v, n − 1);
Running time: O(mn)
Nanyang Technological University
Space: O(m + n2 )
MAS 714
September 6, 2016
4 / 20
Bellman-Ford Algorithm
Algorithm: Bellman-Ford(G, s):
foreach u ∈ V do
d(u) = ∞;
d(s) = 0;
for k = 1 to n − 1 do
foreach v ∈ V do
foreach edge (u, v) ∈ E do
d(v) = min{d(v), d(u) + `(u, v)};
foreach v ∈ V do
dist(s, v) = d(v);
Running time: O(mn) Space: O(m + n)
Exercise: Show that this algorithm achieves same results.
Nanyang Technological University
MAS 714
September 6, 2016
5 / 20
Bellman-Ford: Negative Cycle Detection
Algorithm: Bellman-Ford(G, s):
foreach u ∈ V do
d(u) = ∞;
d(s) = 0;
for k = 1 to n − 1 do
foreach v ∈ V do
foreach edge (u, v) ∈ E do
d(v) = min{d(v), d(u) + `(u, v)};
/* One more iteration to check if distances change
foreach v ∈ V do
foreach edge (u, v) ∈ E do
if d(v) > d(u) + `(u, v) then
return “Negative Cycle”
Nanyang Technological University
MAS 714
September 6, 2016
*/
6 / 20
Correctness: Detecting Negative Cycle
Lemma
G has a negative cycle reachable from s if and only if there exists some
vertex v such that d(v, n) < d(v, n − 1).
We only prove the only if direction.
Nanyang Technological University
MAS 714
September 6, 2016
7 / 20
Correctness: Detecting Negative Cycle
Lemma
Suppose G has a negative cycle C reachable from s. Then there is some
node v ∈ C such that d(v, n) < d(v, n − 1).
Proof.
Suppose not. Let C = v1 → v2 → · · · → vh → v1 be a negative cycle
reachable from s.
By assumption, we know there is no change in nth iteration, which means:
d(vi ) ≤ d(vi−1 ) + `(vi−1 , vi ) for 2 ≤ i ≤ h and
d(v1 ) ≤ d(vh ) + `(vh , v1 ).
Adding up all these inequalities implies 0 ≤ `(C). Contradiction.
Nanyang Technological University
MAS 714
September 6, 2016
8 / 20
Single-Source Shortest Path Recap
Single-Source Shortest Path Problems
Given a directed weighted graph G = (V, E) with edge lengths (or
weights). For edge e = (u, v), `(e) = `(u, v) is its length.
1
Given vertices s, t, find a shortest path from s to t.
2
Given vertex s, find shortest paths from s to all other vertices.
Dijkstra’s algorithm for non-negative edge lengths.
Running time O((m + n) log n) with heaps and O(m + n log n) with
advanced priority queues.
Bellman-Ford algorithm for arbitrary edge lengths.
Running time O(mn).
Nanyang Technological University
MAS 714
September 6, 2016
9 / 20
All-Pair Shortest Path
All-Pair Shortest Path Problem
Given a (undirected or directed) weighted graph G = (V, E) with edge
lengths (or weights). For edge e = (u, v), `(e) = `(u, v) is its length.
1
Find shortest paths for all pair of vertices.
Apply single-source algorithm n times, once for each vertex.
1
2
Non-negative lengths. O(nm log n) with heaps and
O(nm + n2 log n) using advanced priority queues.
Arbitrary edge lengths: O(n2 m).
I
Θ(n4 ) if m = Ω(n2 ).
Can we do better?
Nanyang Technological University
MAS 714
September 6, 2016
10 / 20
Recursion on Index of Intermediate Vertices
dist(i, j, k): length of shortest path from vi to vj that uses only vertices
v1 , . . . , vk as intermediate vertices.
dist(i, j, k) = min
dist(i, j, k − 1)
dist(i, k, k − 1) + dist(k, j, k − 1)
Base case: dist(i, j, 0) = `(vi , vj ).
Correctness: If a shortest path from vi to vj goes through vk , then vk
occurs only once on the path - otherwise there is a negative cycle.
Detect negative cycle: if dist(k, k, n) < 0 then there is a negative cycle
containing k.
Nanyang Technological University
MAS 714
September 6, 2016
11 / 20
Floyd-Warshall Algorithm
Algorithm: Floyd-Warshall(G):
for i = 1 to n do
for j = 1 to n do
dist(i, j, 0) = `(vi , vj );
for k = 1 to n do
for i = 1 to n do
for j = 1 to n do
dist(i, j, k) = min
dist(i, j, k − 1)
dist(i, k, k − 1) + dist(k, j, k − 1)
for i = 1 to n do
if dist(i, i, n) < 0 then
return there is a negative cycle in G.
Nanyang Technological University
MAS 714
September 6, 2016
12 / 20
Floyd-Warshall Algorithm
Algorithm: Floyd-Warshall(G):
for i = 1 to n do
for j = 1 to n do
dist(i, j) = `(vi , vj );
for k = 1 to n do
for i = 1 to n do
for j = 1 to n do
dist(i, j) = min
dist(i, j)
dist(i, k) + dist(k, j)
for i = 1 to n do
if dist(i, i) < 0 then
return there is a negative cycle in G.
Nanyang Technological University
MAS 714
September 6, 2016
13 / 20
Floyd-Warshall Algorithm Analysis
Running Time: O(n3 )
Space: O(n3 )
Simple algorithm without fancy data structures
=⇒ constant factor is small.
Correctness: via induction and recursive definition.
Finding the Paths
Can we find the paths in addition to the distances?
Exercise.
Nanyang Technological University
MAS 714
September 6, 2016
14 / 20
Unweighted Graphs
Unweighted All-Pair Shortest Path Problem
Given a directed unweighted graph G = (V, E).
1
Find shortest paths for all pair of vertices.
An even easier problem:
Unweighted All-Pair Connectivity Problem
Given a directed unweighted graph G = (V, E).
1
Decide connectivity for all pair of vertices.
Nanyang Technological University
MAS 714
September 6, 2016
15 / 20
All-Pair Connectivity Problem
Algorithm: Floyd-Warshall(G):
for i = 1 to n do
for j = 1 to n do
dist(i, j) = 1 if (vi , vj ) ∈ E, dist(i, j) = 0 otherwise;
for k = 1 to n do
for i = 1 to n do
for j = 1 to n do
dist(i, j) = dist(i, j) | (dist(i, k) & dist(k, j));
Nanyang Technological University
MAS 714
September 6, 2016
16 / 20
Powers of the Adjacency Matrix
Matrix Multiplication:
Algorithm: Multi(A, B):
for i = 1 to n do
for j = 1 to n do
P
C[i, j] = k A[i, k] ∗ B[k, j];
Let both A and B be our adjacency matrix M, then C is the square of M,
and its entries are
X
C[i, j] =
M[i, k] ∗ M[k, j].
k
If we think of M[u, v] as the number of paths of length 1 from u to v,
then C[i, j] becomes the number of paths of length exactly 2 from i to j.
Nanyang Technological University
MAS 714
September 6, 2016
17 / 20
Powers of the Adjacency Matrix
If we think of M[u, v] as the number of paths of length 1 from u to v,
then C[i, j] becomes the number of paths of length exactly 2 from i to j.
This is because we try all intermediate vertices k and add up the number
of paths of length 1 from i to k times the number of paths of length 1
from k to j.
This extends to higher powers of M.
Mt [u, v] is the number of paths of length exactly t from u to v.
these may not be simple paths
Connectivity Problem: Compute Mn−1 .
Nanyang Technological University
MAS 714
September 6, 2016
18 / 20
Matrix Multiplication
Given a n × n matrix M, compute Mn−1 .
Naive approach: n − 1 matrix multiplications.
Smart approach: recursive squaring
Mn/2 · Mn/2
n
M =
M(n−1)/2 · M(n−1)/2 · M
if n is even
if n is odd
T (n) = T (n/2) + O(1) =⇒ T (n) = Θ(log n) matrix multiplications in
total!
Best algorithm for matrix multiplication needs time O(nγ )
2 ≤ γ ≤ 2.3728639.
Overall Running Time: O(nγ log n)
Nanyang Technological University
MAS 714
September 6, 2016
19 / 20
Shortest Path Problems Summary
Single-Source Shortest Paths
No negative edge lengths
Arbitrary edge lengths
Dijkstra
Bellman-Ford
O(n log n + m)
O(mn)
All-Pair Shortest Paths
No negative edge lengths
No negative cycles
Unweighted
Nanyang Technological University
n×Dijkstra
n×Bellman-Ford
Floyd-Warshall
Matrix multiplication
MAS 714
O(n2 log n + mn)
O(n2 m) = O(n4 )
O(n3 )
O(n2.3728639 log n)
September 6, 2016
20 / 20