Introduction to
Algorithms
Jiafen Liu
Sept. 2013
Today’s Tasks
• Shortest Paths
– Properties of shortest paths
– Dijkstra’s algorithm
– Correctness
– Analysis
– Breadth-first search
Paths in graphs
• Consider a digraph G= (V, E) with edgeweight function w: E→R. The weight of
path p = v1→v2→…→vk is defined to be
• Example:
w(p)=?
Shortest paths
• A shortest path from u to v is a path of
minimum weight from u to v.
• The shortest path weight from u to v is
defined as
δ(u, v) = min{w(p): p is a path from u to v}.
• In what case there is no shortest path?
– Note: δ(u, v) = ∞ if no path from u to v exists.
– There is a negative cycle.
• In other cases, usually a finite set.
Well-definedness of shortest paths
• If a graph G contains a negative-weight
cycle, then some shortest paths may not
exist.
• Example:
Thought of solving
• Dynamic Programming
• Greedy Algorithms
• Characteristics：
– Optimal Substructure
– Overlapping Subproblems
– Greedy choice property
Optimal substructure
• What is the optimal substructure for this
problem?
• Theorem. A subpath of a shortest path is
a shortest path.
• How to prove?
– Cut and paste
What’s the subproblems？
• Shortest path going from u to v, maybe
involves computing shortest path from u to
some intermediate point x, and then from
x to v.
• Different choosing of x1,x2…xn consists
overlapping subproblems.
• How many subproblems are there?
• |v|2
Triangle inequality
• Important for us to make greedy choice
• Theorem:
For all u, v, x∈V, we have
• Figure Illustration:
Single-source shortest paths
• Problem. From a given source vertex s∈V, find
the shortest-path weights δ(s, v) for all v∈V.
• If all edge weights w(u, v) are nonnegative, all
shortest-path weights must exist.
• IDEA: Greedy.
1.Maintain a set S of vertices whose shortest-path
distances from s are known. (s∈S)
2.At each step add the vertex v∈V–S whose
distance estimate from s is minimal to S .
3.Update the distance estimates of vertices
adjacent to v.
Dijkstra’s algorithm
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Correctness —Part I
• Lemma. Initializing d[s]←0 and d[v]←∞ for
all v∈V–{s} establishes d[v] ≥ δ(s,v) for all
v∈V, and this invariant is maintained over
any sequence of relaxation steps.
• Any suggestions about how to prove?
– Induction
• Proof.
• Base case: check the initializing term
• Induction: suppose that this fails to hold at
some point.
Correctness —Part I
• Suppose not. Let v be the first vertex for
which d[v] < δ(s, v), and let u be the
vertex that caused d[v] to change: d[v] =
d[u] + w(u, v). Then how to induct a
contradition?
supposition
d[v]< δ(s, v)
≤ δ(s, u) + δ(u, v) triangle inequality
sh. path ≤ specific path
≤ δ(s,u) + w(u, v)
d[v] = d[u] + w(u, v)
v is first violation
• Contradiction.
Correctness —Part II
• Lemma. Let u be v’s predecessor on a
shortest path from s to v: s→…u→v.
if d[u]=δ(s, u) and if we relax edge(u, v),
Then, we have d[v] = δ(s, v) after the
relaxation.
• Proof.
• s→…u→v is the shortest path from s to v,
so δ(s, v) = w(s→… →u)+ w(u, v)
= δ(s, u)+ w(u, v).
Correctness —Part II
• Suppose that d[v] > δ(s, v) before the
relaxation.
– By Correctness I, d[v] ≥ δ(s,v)
– Otherwise d[v] = δ(s,v) , we’re done.
• Then test d[v] > d[u] + w(u, v) succeeds
– because d[v] > δ(s, v)
= δ(s, u)+ w(u, v)
= d[u] + w(u, v)
• Then the algorithm sets d[v] = d[u]+ w(u,v)
= δ(s, v).
Correctness —Part III
• Theorem. Dijkstra’s algorithm terminates
with d[v] = δ(s, v) for all v∈V.
• Something to be addressed:
– Once we add a vertex to S, we never change
its weight
• Proof.
• It suffices to show that d[v] = δ(s, v) for
every v∈V when v is added to S.
Correctness —Part III
• Suppose for contradiction: Suppose u is
the first vertex added to S for which d[u] ≠
δ(s, u).
– d[u] > δ(s, u).
• Let p be a shortest path from s to u.
• Now look at the first place here where the
path p exits S
Correctness —Part III
• Let y be the first vertex in V–S along a
shortest path from s to u, and let x be its
predecessor.
• Since u is the first vertex violating the
claimed invariant, we have d[x] = δ(s, x) .
Correctness —Part III
• When x was added to S, the edge (x, y)
was relaxed, which implies that
d[y] = δ(s, y) ≤ δ(s, u) < d[u]
• But, d[u] ≤ d[y] by our choice of u.
Contradiction.
Analysis of Dijkstra
Analysis of Dijkstra
• Note: Same formula as in the analysis of Prim’s
minimum spanning tree algorithm.
Unweighted graphs
• Suppose that w(u, v) =1 for all (u, v) ∈E .
Can Dijkstra’s algorithm be improved?
– Use a simple FIFO queue instead of a priority
queue.
– Analysis: Time = O(V+ E).
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Example
Correctness of BFS
• IDEA: The FIFO Q in breadth-first search
mimics the priority queue Q in Dijkstra.
• Invariant: v comes after u in Q implies that
d[v] = d[u] or d[v] = d[u] + 1.
Step Further I
Bellman-Ford algorithm
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Example of Bellman-Ford
Step Further II
• All-pairs shortest paths
• Input: Digraph G= (V, E), where V= {1, 2,
…, n}, with edge-weight function w: E→R.
• Output: n ×nmatrix of shortest-path
lengths δ(i, j)for all i, j∈V.
• Floyd alglrithm
Any idea of this problem?
• Dynamic programming again!
• Consider the n×n adjacency matrix A=
(aij) of the digraph, and define
dij(m)=weight of a shortest path from i to j that
uses at most m edges
• Claim: We have
and for m= 1, 2, …, n–1,
Proof of Claim
Matrix multiplication
• Compute C= A·B, where C, A, and B are
n×n matrices
• Time = Θ(n3) using the standard algorithm.
• What if we map “+”→“min” and “·”→“+”?