Lecture Notes for Chapter 24:
Single-Source Shortest Paths
Shortest paths
How to Þnd the shortest route between two points on a map.
Input:
•
•
Directed graph G = (V, E)
Weight function w : E → R
Weight of path p = v0 , v1 , . . . , vk k
w(vi−1 , vi )
=
i=1
= sum of edge weights on path p .
Shortest-path weight u to v:
(
'
p
;
v
if there exists a path u ; v ,
min
w(
p)
:
u
δ(u, v) =
∞
otherwise .
Shortest path u to v is any path p such that w( p) = δ(u, v).
Example: shortest paths from s
[d values appear inside vertices. Shaded edges show shortest paths.]
t
3
x
9
6
t
3
3
3
2
s 0
5
x
9
6
1
4
2
7
3
5
y
6
11
z
2
s 0
5
1
4
2
7
3
5
y
6
11
z
This example shows that the shortest path might not be unique.
It also shows that when we look at shortest paths from one vertex to all other
vertices, the shortest paths are organized as a tree.
Can think of weights as representing any measure that
24-2
Lecture Notes for Chapter 24: Single-Source Shortest Paths
accumulates linearly along a path,
we want to minimize.
•
•
Examples: time, cost, penalties, loss.
Generalization of breadth-Þrst search to weighted graphs.
Variants
Single-source: Find shortest paths from a given source vertex s ∈ V to every
vertex v ∈ V .
Single-destination: Find shortest paths to a given destination vertex.
Single-pair: Find shortest path from u to v. No way known that’s better in
worst case than solving single-source.
All-pairs: Find shortest path from u to v for all u, v ∈ V . We’ll see algorithms
for all-pairs in the next chapter.
•
•
•
•
Negative-weight edges
OK, as long as no negative-weight cycles are reachable from the source.
If we have a negative-weight cycle, we can just keep going around it, and get
w(s, v) = −∞ for all v on the cycle.
But OK if the negative-weight cycle is not reachable from the source.
Some algorithms work only if there are no negative-weight edges in the graph.
We’ll be clear when they’re allowed and not allowed.
•
•
•
Optimal substructure
Lemma
Any subpath of a shortest path is a shortest path.
Proof Cut-and-paste.
u
pux
x
pxy
y
pyv
v
Suppose this path p is a shortest path from u to v.
Then δ(u, v) = w( p) = w( pux ) + w( px y ) + w( p yv ).
px y
Now suppose there exists a shorter path x ; y.
Then w( px y ) < w( px y ).
Construct p :
u
pux
x
p'xy
y
pyv
v
Lecture Notes for Chapter 24: Single-Source Shortest Paths
24-3
Then
w( p ) = w( pux ) + w( px y ) + w( p yv )
< w( pux ) + w( px y ) + w( p yv )
= w( p) .
So p wasn’t a shortest path after all!
(lemma)
Cycles
Shortest paths can’t contain cycles:
•
•
•
Already ruled out negative-weight cycles.
Positive-weight ⇒ we can get a shorter path by omitting the cycle.
Zero-weight: no reason to use them ⇒ assume that our solutions won’t use
them.
Output of single-source shortest-path algorithm
For each vertex v ∈ V :
•
d[v] = δ(s, v).
•
•
•
•
Initially, d[v] = ∞.
Reduces as algorithms progress. But always maintain d[v] ≥ δ(s, v).
Call d[v] a shortest-path estimate.
π [v] = predecessor of v on a shortest path from s.
•
•
•
If no predecessor, π [v] = NIL .
π induces a tree—shortest-path tree.
We won’t prove properties of π in lecture—see text.
Initialization
All the shortest-paths algorithms start with I NIT-S INGLE -S OURCE .
I NIT-S INGLE -S OURCE (V, s)
for each v ∈ V
do d[v] ← ∞
π [v] ← NIL
d[s] ← 0
Relaxing an edge (u, v)
Can we improve the shortest-path estimate for v by going through u and taking
(u, v)?
24-4
Lecture Notes for Chapter 24: Single-Source Shortest Paths
R ELAX (u, v, w)
if d[v] > d[u] + w(u, v)
then d[v] ← d[u] + w(u, v)
π [v] ← u
u
4
3
v
10
4
RELAX
4
7
4
3
6
RELAX
6
For all the single-source shortest-paths algorithms we’ll look at,
•
•
start by calling I NIT-S INGLE -S OURCE ,
then relax edges.
The algorithms differ in the order and how many times they relax each edge.
Shortest-paths properties
Based on calling I NIT-S INGLE -S OURCE once and then calling R ELAX zero or
more times.
Triangle inequality
For all (u, v) ∈ E, we have δ(s, v) ≤ δ(s, u) + w(u, v).
Proof Weight of shortest path s ; v is ≤ weight of any path s ; v. Path
s ; u → v is a path s ; v, and if we use a shortest path s ; u, its weight is
δ(s, u) + w(u, v).
Upper-bound property
Always have d[v] ≥ δ(s, v) for all v. Once d[v] = δ(s, v), it never changes.
Proof Initially true.
Suppose there exists a vertex such that d[v] < δ(s, v).
Without loss of generality, v is Þrst vertex for which this happens.
Let u be the vertex that causes d[v] to change.
Then d[v] = d[u] + w(u, v).
So,
d[v] <
≤
≤
⇒ d[v] <
δ(s, v)
δ(s, u) + w(u, v) (triangle inequality)
d[u] + w(u, v)
(v is Þrst violation)
d[u] + w(u, v) .
Lecture Notes for Chapter 24: Single-Source Shortest Paths
24-5
Contradicts d[v] = d[u] + w(u, v).
Once d[v] reaches δ(s, v), it never goes lower. It never goes up, since relaxations
only lower shortest-path estimates.
No-path property
If δ(s, v) = ∞, then d[v] = ∞ always.
Proof d[v] ≥ δ(s, v) = ∞ ⇒ d[v] = ∞.
Convergence property
If s ; u → v is a shortest path, d[u] = δ(s, u), and we call R ELAX (u, v, w), then
d[v] = δ(s, v) afterward.
Proof After relaxation:
d[v] ≤ d[u] + w(u, v)
(R ELAX code)
= δ(s, u) + w(u, v)
= δ(s, v)
(lemma—optimal substructure)
Since d[v] ≥ δ(s, v), must have d[v] = δ(s, v).
Path relaxation property
Let p = v0 , v1 , . . . , vk be a shortest path from s = v0 to vk . If we relax,
in order, (v0 , v1 ), (v1 , v2 ), . . . , (vk−1 , vk ), even intermixed with other relaxations,
then d[vk ] = δ(s, vk ).
Proof Induction to show that d[vi ] = δ(s, vi ) after (vi−1 , vi ) is relaxed.
Basis: i = 0. Initially, d[v0 ] = 0 = δ(s, v0 ) = δ(s, s).
Inductive step: Assume d[vi−1 ] = δ(s, vi−1 ). Relax (vi−1 , vi ). By convergence
property, d[vi ] = δ(s, vi ) afterward and d[vi ] never changes.
The Bellman-Ford algorithm
•
•
•
Allows negative-weight edges.
Computes d[v] and π [v] for all v ∈ V .
Returns TRUE if no negative-weight cycles reachable from s, FALSE otherwise.
24-6
Lecture Notes for Chapter 24: Single-Source Shortest Paths
B ELLMAN -F ORD (V, E, w, s)
I NIT-S INGLE -S OURCE (V, s)
for i ← 1 to |V | − 1
do for each edge (u, v) ∈ E
do R ELAX (u, v, w)
for each edge (u, v) ∈ E
do if d[v] > d[u] + w(u, v)
then return FALSE
return TRUE
Core: The Þrst for loop relaxes all edges |V | − 1 times.
Time: (V E).
Example:
r
–1
–1
s 0
3
1
2
2
4
1 x
–3
2
z
5
–2
y
Values you get on each pass and how quickly it converges depends on order of
relaxation.
But guaranteed to converge after |V | − 1 passes, assuming no negative-weight
cycles.
Proof Use path-relaxation property.
Let v be reachable from s, and let p = v0 , v1 , . . . , vk be a shortest path from s
to v, where v0 = s and vk = v. Since p is acyclic, it has ≤ |V | − 1 edges, so
k ≤ |V | − 1.
Each iteration of the for loop relaxes all edges:
•
•
•
First iteration relaxes (v0 , v1 ).
Second iteration relaxes (v1 , v2 ).
kth iteration relaxes (vk−1 , vk ).
By the path-relaxation property, d[v] = d[vk ] = δ(s, vk ) = δ(s, v).
How about the TRUE/FALSE return value?
•
Suppose there is no negative-weight cycle reachable from s.
At termination, for all (u, v) ∈ E,
d[v] = δ(s, v)
≤ δ(s, u) + w(u, v) (triangle inequality)
= d[u] + w(u, v) .
So B ELLMAN -F ORD returns TRUE .
Lecture Notes for Chapter 24: Single-Source Shortest Paths
24-7
Now suppose there exists negative-weight cycle c = v0 , v1 , . . . , vk , where
v0 = vk , reachable from s.
k
Then
(vi−1 , vi ) < 0 .
•
i=1
Suppose (for contradiction) that B ELLMAN -F ORD returns TRUE .
Then d[vi ] ≤ d[vi−1 ] + w(vi−1 , vi ) for i = 1, 2, . . . , k.
Sum around c:
k
k
d[vi ] ≤
(d[vi−1 ] + w(vi−1 , vi ))
i=1
i=1
=
k
d[vi−1 ] +
k
i=1
w(vi−1 , vi )
i=1
Each vertex appears once in each summation
k
w(vi−1 , vi ) .
0≤
k
i=1
d[vi ] and
k
i=1
d[vi−1 ] ⇒
i=1
This contradicts c being a negative-weight cycle!
Single-source shortest paths in a directed acyclic graph
Since a dag, we’re guaranteed no negative-weight cycles.
DAG-S HORTEST-PATHS (V, E, w, s)
topologically sort the vertices
I NIT-S INGLE -S OURCE (V, s)
for each vertex u, taken in topologically sorted order
do for each vertex v ∈ Adj[u]
do R ELAX (u, v, w)
Example:
6
s
0
2
t
2
1
x
7
6
–1
y
5
–2
z
3
4
2
Time: (V + E).
Correctness: Because we process vertices in topologically sorted order, edges of
any path must be relaxed in order of appearance in the path.
⇒ Edges on any shortest path are relaxed in order.
⇒ By path-relaxation property, correct.
24-8
Lecture Notes for Chapter 24: Single-Source Shortest Paths
Dijkstra’s algorithm
No negative-weight edges.
Essentially a weighted version of breadth-Þrst search.
•
•
Instead of a FIFO queue, uses a priority queue.
Keys are shortest-path weights (d[v]).
Have two sets of vertices:
•
•
S = vertices whose Þnal shortest-path weights are determined,
Q = priority queue = V − S.
D IJKSTRA (V, E, w, s)
I NIT-S INGLE -S OURCE (V, s)
S←∅
Q←V
i.e., insert all vertices into Q
while Q = ∅
do u ← E XTRACT-M IN (Q)
S ← S ∪ {u}
for each vertex v ∈ Adj[u]
do R ELAX (u, v, w)
•
•
Looks a lot like Prim’s algorithm, but computing d[v], and using shortest-path
weights as keys.
Dijkstra’s algorithm can be viewed as greedy, since it always chooses the “lightest” (“closest”?) vertex in V − S to add to S.
Example:
x
8
10
s 0
2
3
4
5
6 z
1
5
y
Order of adding to S: s, y, z, x.
Correctness:
Loop invariant: At the start of each iteration of the while loop, d[v] =
δ(s, v) for all v ∈ S.
Initialization: Initially, S = ∅, so trivially true.
Termination: At end, Q = ∅ ⇒ S = V ⇒ d[v] = δ(s, v) for all v ∈ V .
Lecture Notes for Chapter 24: Single-Source Shortest Paths
24-9
Maintenance: Need to show that d[u] = δ(s, u) when u is added to S in each
iteration.
Suppose there exists u such that d[u] = δ(s, u). Without loss of generality,
let u be the Þrst vertex for which d[u] = δ(s, u) when u is added to S.
Observations:
•
•
•
u = s, since d[s] = δ(s, s) = 0.
Therefore, s ∈ S, so S = ∅.
There must be some path s ; u, since otherwise d[u] = δ(s, u) = ∞ by
no-path property.
So, there’s a path s ; u.
p
This means there’s a shortest path s ; u.
Just before u is added to S, path p connects a vertex in S (i.e., s) to a vertex in
V − S (i.e., u).
Let y be Þrst vertex along p that’s in V − S, and let x ∈ S be y’s predecessor.
p2
s
S
p1
u
x
y
p
p
Decompose p into s ;1 x → y ;2 u. (Could have x = s or y = u, so that p1
or p2 may have no edges.)
Claim
d[y] = δ(s, y) when u is added to S.
Proof x ∈ S and u is the Þrst vertex such that d[u] = δ(s, u) when u is added
to S ⇒ d[x] = δ(s, x) when x is added to S. Relaxed (x, y) at that time, so by
(claim)
the convergence property, d[y] = δ(s, y).
Now can get a contradiction to d[u] = δ(s, u):
y is on shortest path s ; u, and all edge weights are nonnegative
⇒ δ(s, y) ≤ δ(s, u) ⇒
d[y] = δ(s, y)
≤ δ(s, u)
≤ d[u]
(upper-bound property) .
Also, both y and u were in Q when we chose u, so
d[u] ≤ d[y] ⇒ d[u] = d[y] .
Therefore, d[y] = δ(s, y) = δ(s, u) = d[u].
Contradicts assumption that d[u] = δ(s, u). Hence, Dijkstra’s algorithm is
correct.
24-10
Lecture Notes for Chapter 24: Single-Source Shortest Paths
Analysis: Like Prim’s algorithm, depends on implementation of priority queue.
•
•
If binary heap, each operation takes O(lg V ) time ⇒ O(E lg V ).
If a Fibonacci heap:
•
•
•
Each E XTRACT-M IN takes O(1) amortized time.
There are O(V ) other operations, taking O(lg V ) amortized time each.
Therefore, time is O(V lg V + E).
Difference constraints
Given a set of inequalities of the form x j − xi ≤ bk .
•
•
x’s are variables, 1 ≤ i, j ≤ n,
b’s are constants, 1 ≤ k ≤ m.
Want to Þnd a set of values for the x’s that satisfy all m inequalities, or determine
that no such values exist. Call such a set of values a feasible solution.
Example:
x1 − x2
x1 − x3
x2 − x4
x3 − x4
x4 − x1
≤
≤
≤
≤
≤
5
6
−1
−2
−3
Solution: x = (0, −4, −5, −3)
Also: x = (5, 1, 0, 2) = [above solution] + 5
Lemma
If x is a feasible solution, then so is x + d for any constant d.
Proof x is a feasible solution ⇒ x j − xi ≤ bk for all i, j, k
⇒ (x j + d) − (xi + d) ≤ bk .
(lemma)
Constraint graph
G = (V, E), weighted, directed.
•
•
•
•
V = {v0 , v1 , v2 , . . . , vn }: one vertex per variable + v0
E = {(vi , v j ) : x j − xi ≤ bk is a constraint} ∪ {(v0 , v1 ), (v0 , v2 ), . . . , (v0 , vn )}
w(v0 , v j ) = 0 for all j
w(vi , v j ) = bk if x j − xi ≤ bk
Lecture Notes for Chapter 24: Single-Source Shortest Paths
0
v1
0
0
–3
v0 0
0
0
24-11
v2
–4
5
–1
6
–3
v4
–2
–5
v3
Theorem
Given a system of difference constraints, let G = (V, E) be the corresponding
constraint graph.
1. If G has no negative-weight cycles, then
x = (δ(v0 , v1 ), δ(v0 , v2 ), . . . , δ(v0 , vn ))
is a feasible solution.
2. If G has a negative-weight cycle, then there is no feasible solution.
Proof
1. Show no negative-weight cycles ⇒ feasible solution.
Need to show that x j − xi ≤ bk for all constraints. Use
x j = δ(v0 , v j )
xi = δ(v0 , vi )
bk = w(vi , v j ) .
By the triangle inequality,
δ(v0 , v j ) ≤ δ(v0 , vi ) + w(vi , v j )
x j ≤ xi + bk
x j − xi ≤ bk .
Therefore, feasible.
2. Show negative-weight cycles ⇒ no feasible solution.
Without loss of generality, let a negative-weight cycle be c = v1 , v2 , . . . , vk ,
where v1 = vk . (v0 can’t be on c, since v0 has no entering edges.) c corresponds
to the constraints
x2 − x1 ≤ w(v1 , v2 ) ,
x3 − x2 ≤ w(v2 , v3 ) ,
..
.
xk−1 − xk−2 ≤ w(vk−2 , vk−1 ) ,
xk − xk−1 ≤ w(vk−1 , vk ) .
[The last two inequalities above are incorrect in the Þrst three printings of the
book. They were corrected in the fourth printing.]
If x is a solution satisfying these inequalities, it must satisfy their sum.
So add them up.
24-12
Lecture Notes for Chapter 24: Single-Source Shortest Paths
Each xi is added once and subtracted once. (v1 = vk ⇒ x1 = xk .)
We get 0 ≤ w(c).
But w(c) < 0, since c is a negative-weight cycle.
Contradiction ⇒ no such feasible solution x exists.
How to Þnd a feasible solution
1. Form constraint graph.
•
•
•
n + 1 vertices.
m + n edges.
(m + n) time.
2. Run B ELLMAN -F ORD from v0 .
•
O((n + 1)(m + n)) = O(n 2 + nm) time.
3. If B ELLMAN -F ORD returns FALSE ⇒ no feasible solution.
If B ELLMAN -F ORD returns TRUE ⇒ set xi = δ(v0 , vi ) for all i.
(theorem)