Topics
Graph algorithms
Disjoint sets datastructure
Minimum Spanning Trees
–
–
Shortest paths
–
–
Kruskal
Prim
single source
all shortest pahts
Flow networks
1
Disjoint Sets
Algorithms
Disjoint-Set-Data-Structure: maintains a collection {S1, S2, …, Sk}
of dynamic disjoint sets.
Each set is identified by a representative, which is a member of
the set.
Let x be an object:
Make-Set(x): makes a singleton set with x.
Union(x, y): takes two disjoint sets with representatives x and y
respectively and creates their union. Usually, either x or y will be
the new representative. The old sets are destroyed.
Find-Set(x): returns a pointer to the representative of the unique
set containing x.
3
Disjoint sets for
connected component example
a
b
e
c
d
g
f
h
j
i
Edge Processed
2
Collection of disjoint sets
initial sets
{a}
{b}
{c}
{e}
{f}
{g}
{h}
{i}
{j}
(b, d)
{a}
{b, d}
{c}
{d}
{e}
{f}
{g}
{h}
{i}
{j}
(e, g)
{a}
{b, d}
{c}
{e, g}
{f}
{h}
{i}
{j}
(a, c)
{a, c}
{b, d}
{e, g}
{f}
{h}
{i}
{j}
(h, i)
{a, c}
{b, d}
{e, g}
{f}
{h, i}
{j}
(a, b)
{a, b, c, d}
{e, g}
{f}
{h, i}
{j}
(e, f)
{a, b, c, d}
{e, f, g}
{h, i}
{j}
(b, c)
{a, b, c, d}
{e, f, g}
{h, i}
{j}
5
CONNECTED-COMPONENTS(G)
for each vertex v ∈V[G]
do MAKE-SET(v)
for each edge (u, v) ∈E[G]
do if FIND-SET(u) ≠ FIND-SET(v)
then UNION(u, v)
SAME-COMPONENT(u, v)
if FIND-SET(u) = FIND-SET(v)
then return TRUE
else return FALSE
4
Disjoint set using the forest of uptrees
Make-Set(x) creates a up-tree with one node.
Find-Set(x) follows parent pointers from x up to the
root.
Union(x, y) causes the root of one tree to point to the
root of the other.
6
1
Disjoint-set heuristics
Disjoint-set heuristics
Unless we are careful, this will not improve on the
naive implementation via linked lists: the tree could be
just a chain of n nodes.
We introduce two heuristics:
1. Union by Rank: make the root of the tree with
fewer nodes point to the root of the tree with more
nodes. Rather than maintaining an exact count
associated with each node, we will maintain an upper
bound to the exact count, called the rank: it will be an
upper bound on the height of the node. In a Union, the
root with the smaller rank will be made to point at the
root with the larger rank.
2. Path Compression: during a Find() operation,
compress the path along the chain of nodes so that, at
the end, all the nodes in the chain point to the root.
Path Compression does not change any ranks.
7
8
Up-Tree: Array representation
Algorithm for Disjoint-Set Forest
Up-tree of size n can be represented by an array of
size n.
MAKE-SET(x)
1. p[x]←x
2. rank[x]←0
p
0
2
4
0
4
2
4
2
6
0
1
2
3
4
5
6
7
8
p[i] = parent of i
p[root] = root
4
2
1
5
FIND-SET(x)
LINK(x,y)
1. if x≠ p[x]
1. if rank[x]>rank[y]
2.
then p[x] ←FIND-SET(p[x])
2. then p[y] ←x
3. return p[x]
3. then p[x] ←y
4.
if rank[x]=rank[y]
5.
then rank[y]++
0
6
7
UNION(x,y)
1. LINK(FIND-SET(x),FIND-SET(y))
3
Worst case running time for m MAKE-SET, UNION, FIND-SET operations is:
O(mα(n)) where α(n)≤4. So nearly linear in m.
8
9
10
Recall - graph definition
Graphs: examples
A graph G = (V,E) is a pair, where V = {v1, .. vn} is the
vertex set (nodes) and E = {e1, .. em} is the edge set. An
edge ek = (vi ,vj) connects (is incident to) two vertices vi and
vj of V.
Edges can be undirected or directed (unordered or
odered):
eij: vi — vj or eij: vi —> vj
The graph G is finite when |V| and |E| are finite.
The size of graph G is |G| = |V| + |E|.
Let V = {1,2,3,4,5,6}
1
2
3
1
2
3
4
5
6
4
5
6
Directed graph
11
Undirected graph
12
2
Recall - graphs representation
Minimum Spanning Tree (MST)
Two standard ways of representing graphs:
1. Adjacency list: for each vertex v there is a linked list
Lv of its neighbors in the graph. Size of the
representation: Θ(|V|+|E|).
2. Adjacency matrix: a |V| ×|V| matrix in which an edge
e = (u,v) is represented by a non-zero (u,v) entry.
Size of the representation: Θ(|V|2).
Adjacency lists are better for sparse graphs.
Adjacency matrices are better for dense graphs.
Given a set of nodes and possible connections with
weights between them, find the subset of connections
that connects all the nodes and whose sum of weights
is the smallest.
Examples:
–
–
telephone switching network
electronic board wiring
The nodes and subset of
connections form a tree!
13
14
Spanning trees
Generic MST Algorithm
Definition: Let G=(V,E) be a weighted connected
undirected graph. A spanning tree of G is a subset T
⊆ E of edges, such that the sub-graph G’=(V,T) is
connected and acyclic.
The minimum spanning tree (MST) is a spanning tree
that minimizes the sum:
w(T ) =
∑ w(u, v )
Generic-MST(G,
Generic-MST(G, w)
w)
11 A←∅
A←∅ //
// Contains
Contains edges
edges that
that belong
belong to
to aa MST
MST
22 while
while AA does
does not
not form
form aa spanning
spanning tree
tree do
do
33
Find
an
edge
Find an edge (u,v)
(u,v) that
that is
is safe
safe for
for AA
44
A←A∪{(u,v)}
A←A∪{(u,v)}
55 return
return AA
Safe edge – edge that does not destroy A’s property
The
The algorithm
algorithm manages
manages aa set
set of
of edges
edges AA maintaining
maintaining
the
the following
following loop
loop invariant
invariant
Prior
Prior to
to each
each iteration,
iteration, AA is
is aa subset
subset of
of some
some
minimum
minimum spanning
spanning tree.
tree.
At
At each
each step,
step, an
an edge
edge is
is determined
determined that
that can
can be
be added
added
to
to AA without
without violating
violating this
this invariant.
invariant. Such
Such an
an edge
edge
is
is called
called aa Safe
Safe Edge.
Edge.
(u ,v )∈T
15
16
Kruskal's Algorithm
Kruskal's Algorithm
Edge based algorithm
Add the edges one at a time, in increasing weight
order
The algorithm maintains A – a forest of trees. An
edge is accepted it if connects vertices of distinct
trees
We need a data structure that maintains a partition,
i.e.,a collection of disjoint sets
The algorithm adds the cheapest edge that connects
two trees of the forest
MST-Kruskal(G,w)
01
02
03
04
05
MakeSet(S,x): S ← S ∪ {{x}}
Union(Si,Sj): S ← S – {Si,Sj} ∪ {Si ∪ Sj}
– FindSet(S, x): returns unique Si ∈ S, where x ∈ Si
–
–
06
07
08
09
17
A ← ∅
for each vertex v ∈ V[G] do
Make-Set(v)
sort the edges of E by non-decreasing weight w
for each edge (u,v) ∈ E, in order by nondecreasing weight do
if Find-Set(u) ≠ Find-Set(v) then
A ← A ∪ {(u,v)}
Union(u,v)
return A
18
3
Kruskal Example
Kruskal Example (2)
19
20
Kruskal Example (3)
Kruskal Example (4)
21
22
Kruskal Running Time
Prim Algorithm
Initialization O(V) time
Sorting the edges Θ(E lg E) = Θ(E lg V) (why?)
|E| ≤ |V2|
O(E) calls to FindSet, O(V) calls to MakeSet, Union
Disjoint Set operations are in t(v) ≤ log V
Total time: O(E lg V)
23
Vertex based algorithm
Grows one tree T, one vertex at a time
A cloud covering the portion of T already computed
Label the vertices v outside the cloud with key[v] –
the minimum weigth of an edge connecting v to a
vertex in the cloud, key[v] = ∞, if no such edge exists
24
4
Prim Example
Prim Example (2)
25
26
Prim Example (3)
Prim Algorithm
MST-Prim(G,w,r)
01
02
03
04
05
06
07
08
09
10
11
Q ← V[G] // Q – vertices out of T
for each u ∈ Q
key[u] ← ∞
key[r] ← 0
π[r] ← NIL
while Q ≠ ∅ do
u ← ExtractMin(Q) // making u part of T
for each v ∈ Adj[u] do
if v ∈ Q and w(u,v) < key[v] then
π[v] ← u
key[v] ← w(u,v) /*decrease value of key */
27
28
Analysis of Prim’s algorithm
Summary: MST
Complexity: Depends on the implementation of the
minimum priority queue. With a binary min-heap, we
have:
–
Building the initial heap takes O(|V|).
– Extract-Min takes O(lg |V|) per vertex
total O(|V| lg |V|)
– The for loop is executed O(|E|).
– Membership test is O(1). Decreasing a key is O(lg |V|).
–
MST is a tree of all nodes with minimum total cost
Two greedy algorithms for finding MST:
Overall, the running time is
O(|V| lg |V| + |E| lg |V|) = O(|E| lg |V|).
–
29
Kruskal’s algorithm: edge-based. Runs in O(|V| |E|).
Prim’s algorithm: vertex-based. Runs in O(|E| lg |V|).
Complexity of Kruskal’s algorithm can be improved with
Union-Find ADT to O(|E| lg |V|),
Complexity of Prim’s algorithm can be improved with
Fibonacci heaps to O(|V| lg |V| + |E|).
Randomized algorithm takes O(|V| + |E|) expected time.
30
5