Introduction to Algorithms:
Shortest Paths I
Introduction to Algorithms
Shortest Paths I
• Properties of shortest paths
• Dijkstra’s Algorithm
• Correctness
• Analysis
• Breadth-First Search
CS 421 - Computer Science II
2
Paths in Graphs
Consider a digraph G = (V, E) with edge-weight
function w : E  ℛ.
The weight of path
p = v1  v2  …  vk
is defined to be
k 1
w( p)   w(vi , vi1 ).
i1
CS 421 - Computer Science II
3
Paths in Graphs
Example:
p:
𝑣1
4
𝑣2
–2
𝑣3
–5
𝑣4
1
𝑣5
w(p) = 4 + (-2) + (-5) + 1
= -2
CS 421 - Computer Science II
4
Shortest Paths
A shortest path from u to v is a path of
minimum weight from u to v. The shortestpath weight from u to v is defined as
(u, v) = min{w(p) : p is a path from u to v}.
Note: (u, v) = , if no path from u to v exists.
CS 421 - Computer Science II
5
Well-Definedness of
Shortest Paths
If a graph G contains a negative-weight cycle,
then some shortest paths may not exist.
…
Example:
<0
u
v
Then, (u, v) = -.
CS 421 - Computer Science II
6
Optimal Substructure
Theorem. A sub-path of a shortest path is a
shortest path.
CS 421 - Computer Science II
7
Optimal Substructure
Theorem. A sub-path of a shortest path is a
shortest path.
Proof. Suppose that p is the shortest path
between the vertices u and v. And that x and y
lie on that shortest path.
p:
u
y
x
CS 421 - Computer Science II
v
8
Optimal Substructure
Assume there’s another sub-path between x
and y with a lower weight. Then that path
would sub-path of a shortest path between
u and v. But p is the shortest path. Which is
a contradiction. Therefore, a sub-path of a
shortest path is also a shortest path.
p:
u
x
y
CS 421 - Computer Science II
v
9
Triangle Inequality
Theorem. For all u, v, x  V, we have
(u, v)  (u, x) + (x, v).
Proof.
(u, v)
u
(u, x)
v
(x, v)
x
CS 421 - Computer Science II
10
Single-Source Shortest Paths
Problem. From a given source vertex s  V, find
the shortest-path weights (s, v) for all v  V.
If all edge weights w(u, v) are non-negative, all
shortest-path weights must exist. (May be ).
IDEA: Greedy.
1. Maintain a set S of vertices whose shortest- path
distances from s are known.
2. At each step add to S the vertex v  V – S
whose distance estimate from s is minimal.
3. Update the distance estimates of vertices
11
adjacent to v. CS 421 - Computer Science II
Dijkstra’s Algorithm
d[s]  0
for each v  V – {s}
do d[v]  
S
⊳Q is a priority queue with V – S
QV
while Q  
do u  EXTRACT-MIN(Q)
S  S  {u}
for each v  Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
CS 421 - Computer Science II
12
Dijkstra’s Algorithm
d[s]  0
for each v  V – {s}
do d[v]  
S
⊳Q is a priority queue with V – S
QV
while Q  
do u  EXTRACT-MIN(Q)
S  S  {u}
relaxation step
for each v  Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
Implicit DECREASE-KEY
13
Example of Dijkstra’s Algorithm
Graph with
non-negative
edge weights:
10
A
B
1 4
3
CS 421 - Computer Science II
C
2
8
2
D
7 9
E
14
Example of Dijkstra’s Algorithm
Graph with
non-negative
edge weights:
10
0 A
Q: A B C D E
d:
0

 

B
1 4
3

C

2
8
2

D
7 9
E

S: {}
CS 421 - Computer Science II
15
Example of Dijkstra’s Algorithm
“A”  EXTRACT-MIN(Q):
10
0 A
Q: A B C D E
d:
0

 

B
1 4
3

C

2
8
2

D
7 9
E

S: { A }
CS 421 - Computer Science II
16
Example of Dijkstra’s Algorithm
Relax all edges leaving A:
10
0 A
Q: A B C D E
d:
0
10
3

10
B
1 4
3

C
3
2
8
2

D
7 9
E

S: { A }
CS 421 - Computer Science II
17
Example of Dijkstra’s Algorithm
“C”  EXTRACT-MIN(Q):
10
0 A
Q: A B C D E
d:
0
10
3

10
B
1 4
3

C
3
2
8
2

D
7 9
E

S: { A, C }
CS 421 - Computer Science II
18
Example of Dijkstra’s Algorithm
Relax all edges leaving C:
10
0 A
Q: A B C D E
d:
0
7
3 11
7
B
1 4
3
5
C
3
2
8
2
11
D
7 9
E
5
S: { A, C }
CS 421 - Computer Science II
19
Example of Dijkstra’s Algorithm
“E”  EXTRACT-MIN(Q):
10
0 A
Q: A B C D E
d:
0
7
3
11
7
B
1 4
3
5
C
2
8
2
3
11
D
7 9
E
5
S: { A, C, E }
CS 421 - Computer Science II
20
Example of Dijkstra’s Algorithm
Relax all edges leaving E:
10
0 A
Q: A B C D E
d:
0
7
3 11
7
B
1 4
3
5
C
2
8
2
3
11
D
7 9
E
5
S: { A, C, E }
CS 421 - Computer Science II
21
Example of Dijkstra’s Algorithm
“B”  EXTRACT-MIN(Q):
10
0 A
Q: A B C D E
d:
0
7
3
11
7
B
1 4
3
5
C
11
D
2
8
7 9
2
3
E
5
S: { A, C, E, B }
CS 421 - Computer Science II
22
Example of Dijkstra’s Algorithm
Relax all edges leaving B:
10
0 A
Q: A B C D E
d:
0
7
3
9
7
B
1 4
3
5
C
9
D
2
8
7 9
2
3
E
5
S: { A, C, E, B }
CS 421 - Computer Science II
23
Example of Dijkstra’s Algorithm
“D”  EXTRACT-MIN(Q):
10
0 A
Q: A B C D E
d:
0
7
3
9
7
B
1 4
3
5
C
9
D
2
8
7 9
2
3
E
5
S: { A, C, E, B }
CS 421 - Computer Science II
24
Example of Dijkstra’s Algorithm
10
0 A
Q: A B C D E
d:
0
7
3
9
7
B
1 4
3
5
C
9
D
2
8
7 9
2
3
E
5
S: { A, C, E, B }
CS 421 - Computer Science II
25
Dijkstra’s Algorithm –
Finding Shortest Path
Algorithm determines weight of shortest path
between source vertex s but how find the actual
shortest path?
Shortest Path Tree: union of all shortest paths.
For each vertex v  V and the last edge into
that vertex that was relaxed (u, v), the union of
all of those edges is a shortest path tree.
Has the property that there’s a unique path
between s and all v  V and it’s the shortest
path.
CS 421 - Computer Science II
26
Analysis of Dijkstra’s Algorithm
|V |
times
d[s]  0
for each v  V – {s}
do d[v]  
S
QV
while Q  
do u  EXTRACT-MIN(Q)
S  S  {u}
for each v  Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
CS 421 - Computer Science II
27
Analysis of Dijkstra’s Algorithm
while Q  
do u  EXTRACT-MIN(Q)
|V |
S  S  {u}
times
for each v  Adj[u]
degree(u)
do if d[v] > d[u] + w(u, v)
times
then d[v]  d[u] + w(u, v)
Handshaking Lemma  (E) implicit DECREASE-KEY’s.
CS 421 - Computer Science II
28
Analysis of Dijkstra’s Algorithm
while Q  
do u  EXTRACT-MIN(Q)
|V |
S  S  {u}
times
for each v  Adj[u]
degree(u)
do if d[v] > d[u] + w(u, v)
times
then d[v]  d[u] + w(u, v)
Time = (V·TEXTRACT-MIN + E·TDECREASE-KEY)
Note: Same formula as in the analysis of Prim’s
minimum spanning tree algorithm.
CS 421 - Computer Science II
29
Analysis of Dijkstra (cont'd)
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY
Q
TEXTRACT-MIN TDECREASE-KEY
array
O(V)
binary
heap
O(lg V)
Fibonacci O(lg V)
amortized
heap
O(1)
Total
O(V2)
O(lg V) O((V+E )lg V)
O(1)
O(E + V lg V)
amortized worst case
CS 421 - Computer Science II
30
Unweighted Graphs
Suppose that w(u, v) = 1 for all (u, v)  E.
Can Dijkstra’s algorithm be improved?
• Use a simple FIFO queue instead of a priority
queue.
CS 421 - Computer Science II
31
Unweighted Graphs
Breadth-first search
while Q  
do u  DEQUEUE(Q)
for each v  Adj[u]
do if d[v] = 
then d[v]  d[u] + 1
ENQUEUE(Q, v)
Analysis: Time = O(V + E).
CS 421 - Computer Science II
32
Example of Breadth-First Search
a
f
h
d
b
g
e
i
c
Q:
CS 421 - Computer Science II
33
Example of Breadth-First Search
0
a
f
h
d
b
g
e
i
c
0
Q: a
CS 421 - Computer Science II
34
Example of Breadth-First Search
0
a
1
f
h
d
1
b
g
e
i
c
1 1
Q: a b d
CS 421 - Computer Science II
35
Example of Breadth-First Search
0
a
1
f
h
d
1
b
g
e
2
c
i
2
1 2 2
Q: a b d c e
CS 421 - Computer Science II
36
Example of Breadth-First Search
0
a
f
1
h
d
1
b
g
e
2
c
i
2
2 2
Q: a b d c e
CS 421 - Computer Science II
37
Example of Breadth-First Search
0
a
f
1
h
d
1
b
g
e
2
c
i
2
2
Q: a b d c e
CS 421 - Computer Science II
38
Example of Breadth-First Search
0
a
f
1
h
d
1
2
3
b
c
g
e
i
2
3
3 3
Q: a b d c e g i
CS 421 - Computer Science II
39
Example of Breadth-First Search
4
0
a
f
1
h
d
1
2
3
b
c
g
e
i
2
3
3 4
Q: a b d c e g i f
CS 421 - Computer Science II
40
Example of Breadth-First Search
0
a
1
4
4
f
h
d
1
2
3
b
c
g
e
i
2
3
4 4
Q: a b d c e g i f h
CS 421 - Computer Science II
41
Example of Breadth-First Search
0
a
1
4
4
f
h
d
1
2
3
b
c
g
e
i
2
3
4
Q: a b d c e g i f h
CS 421 - Computer Science II
42
Example of Breadth-First Search
0
a
1
4
4
f
h
d
1
2
3
b
c
g
e
i
2
3
Q: a b d c e g i f h
CS 421 - Computer Science II
43
CS 421 - Computer Science II
44
Correctness Dijkstra’s — Part I
Lemma.
Initializing d[s]  0 and d[v]   for all v 
V – {s} establishes d[v]  (s, v) for all v  V,
and this invariant is maintained over any
sequence of relaxation steps.
CS 421 - Computer Science II
45
Correctness Dijkstra’s — Part I
Proof. Suppose not. Let v be the first vertex for
which d[v] < (s, v), and let u be the vertex that
caused d[v] to change: d[v] = d[u] + w(u, v).
Then,
d[v] < (s, v)
 (s, u) + (u, v)
 (s,u) + w(u, v)
 d[u] + w(u, v)
supposition
triangle inequality
sh. path  specific path
v is first violation
Contradiction.
CS 421 - Computer Science II
46
Correctness Dijkstra’s — Part II
Lemma.
Let u be v’s predecessor on a shortest
path from s to v.
Then, if d[u] = (s, u) and edge (u, v) is relaxed,
we have d[v]  (s, v) after the relaxation.
CS 421 - Computer Science II
47
Correctness Dijkstra’s — Part II
Proof. Observe that (s, v) = (s, u) + w(u, v).
Suppose that d[v] > (s, v) before the relaxation.
(Otherwise, we’re done.)
Then, the test d[v] > d[u] + w(u, v) succeeds,
because
d[v] > (s, v) = (s, u) + w(u, v) = d[u] + w(u, v),
and the algorithm sets
d[v] = d[u] + w(u, v) = (s, v).
CS 421 - Computer Science II
48
Correctness Dijkstra’s — Part III
Theorem. Dijkstra’s algorithm terminates with
d[v] = (s, v) for all v  V.
CS 421 - Computer Science II
49
Correctness Dijkstra’s — Part III
Proof. It suffices to show that d[v] = (s, v) for every
v  V when v is added to S. Suppose u is the first
vertex added to S for which d[u]  (s, u). Let y be the
first vertex in V – S along a shortest path from s to u,
and let x be its predecessor:
u
S, just before
adding u.
s
CS 421 - Computer Science II
x
y
50
Correctness Dijkstra’s — Part III
S
s
u
x
y
Since u is the first vertex violating the claimed
invariant, we have d[x] = (s, x). When x was
added to S, the edge (x, y) was relaxed, which
implies that d[y] = (s, y)  (s, u)  d[u]. But,
d[u]  d[y] by our choice of u. Contradiction.
CS 421 - Computer Science II
51
Correctness of BFS
while Q  
do u  DEQUEUE(Q)
for each v  Adj[u]
do if d[v] = 
then d[v]  d[u] + 1
ENQUEUE(Q, v)
Key idea:
The FIFO Q in breadth-first search mimics the
priority queue Q in Dijkstra.
• Invariant: v comes after u in Q implies that
d[v] = d[u] or d[v] = d[u] + 1.
CS 421 - Computer Science II
52