Design and Analysis of Algorithms
!"## $%&'(()#* &$(+!$ ,*"&'(
-"*
%##++(&%# .#
.$'/(0 1$/-1%#%
. . !"
#
2542
http://www.cp.eng.chula.ac.th/faculty/spj
Outline
Advanced Data Structures
Disjoint Set
Make-Set, Union, Find-Set
Disjoint-Set Forests
Union by Rank
Path Compression
Analysis
http://www.cp.eng.chula.ac.th/faculty/spj
Disjoint Sets : ADT
• Make-Set( x )
: π( 1 )
1
• Union( s1, s2 ) : π( 1 )
• Find-Set( x )
Disjoint-Set Forests
: ν( log * n ) amortized cost
2 9 4
7
5
6
{1, 2, 9, 4, 6}
set #1
http://www.cp.eng.chula.ac.th/faculty/spj
8
3
{3} {5, 7, 8}
set #3 set #8
http://www.cp.eng.chula.ac.th/faculty/spj
Disjoint-Set : Find-Set
Disjoint Set : Union
1
2 9 4
6
{1, 2, 9, 4, 6}
set #1
1
Find-Set( 6 ) = Find-Set( 2 )
= Find-Set( 1 )
=1
3
8
7
2 9 4
5
6
π( h )
Union( 1, 8 )
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Disjoint Set : Union
3
8
7
5 1
Disjoint Set : Two Heuristics
Union : Smart Union
Find-Set : Path Compression
2 9 4
6
Union( 1, 8 )
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Disjoint Set : Smart Union
Smart Union
Union by height
Union by size
1
8
2 9 4
3 7 11 5 10
6
1
2 9 4 8
6 3 7 11 5 10
Union by height
http://www.cp.eng.chula.ac.th/faculty/spj
8
1 3 7 11 5 10
2 9 4 Union by size
6
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
Smart Union : Worst-Case
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Smart Union : Worst-Case
Smart Union : Worst-Case
height = π( log n )
Union : π(1), Find-Set : π( log n )
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Path Compression
Path Compression
7
3
1
7
1
2
Find-Set( 1 ) = Find-Set( 2 ) = Find-Set( 3 ) = 7
p[3] = 7
http://www.cp.eng.chula.ac.th/faculty/spj
2
3
Find-Set( 1 ) = Find-Set( 2 ) = Find-Set( 3 ) = 7
p[2] = p[3] = 7
http://www.cp.eng.chula.ac.th/faculty/spj
Path Compression
Union by Rank
7
1
With path compression
2
3
Union by height : inefficient to recompute heights
Union by size : perfectly compatible
Union by rank
Find-Set( 1 ) = Find-Set( 2 ) = Find-Set( 3 ) = 7
p[1] = p[2] = p[3] = 7
http://www.cp.eng.chula.ac.th/faculty/spj
Make-Set
Make-Set( x )
{
p[x] = x
rank[x] = 0
}
rank = estimated height
no height updates during path compression
http://www.cp.eng.chula.ac.th/faculty/spj
Union by Rank
Union( s1, s2 )
{
if rank[s1] > rank[s2]
p[s2] = s1
else
p[s1] = s2
if rank[s1] = rank[s2]
rank[s2]++
}
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Union by Rank
x4
5
y
Union(x, y)
rank[x] < rank[y]
5 y
x5
Union(x, y)
x4
x5
5
6
Find-Set
y
y
Find-Set( x )
{
if x = p[x] return x
p[x] = Find-Set( p[x] )
return p[x]
}
No rank is updated during path compression
rank[x] = rank[y]
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Union by Rank & Path Compression
Ackermanns Function and Inverse
n elements (n Make-Sets, at most n-1 Unions)
m Make-Set, Union, and Find-Set operations
m = Ω( n )
worst case = Ο( m α(m, n) )
α(m, n) is the inverse of Ackermann s function
a very very very slow function
A( 1, j ) = 2 j
for j ≥ 1
A( i, 1 ) = A( i-1, 2 )
for i ≥ 2
A( i, j ) = A( i-1, A(i, j-1) ) for i, j ≥ 2
α( m, n ) = min{ i ≥ 1 : A( i, m/n ) > lg n }
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Ackermanns Function
i
j
1
2
3
4
1
21
22
23
24
2
22
22
3
2
2
2
2
22
}
2 16
2
}
2 3
2
2
2
}
2 4
22
}
2 16
2 22
}
Analysis
2
}
n elements (n Make-Sets, at most n-1 Unions)
m Make-Set, Union, and Find-Set operations
m = Ω( n )
union by rank, path compression
worst case = O( m α(m, n) ) = O( m log * n )
2 16
2 22
2 22
2
}
}
lg* 2
2
2m  n

= n +1
proof this
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Properties of Ranks : #1
Properties of Ranks : #2
For all tree roots x, size( x ) ≥ 2rank( x )
% Basis : rank = 0, 20 = 1 = only one root node
% Induction : Let+s Union( a, b )
rank is increased by 1 only if rank(a) = rank(b)
size( a ∪ b ) = size( a ) + size( b )
≥ 2rank[ a ] + 2rank[ b ]
= 2rank[ a ]+1 = 2rank[ a ∪ b ]
The ranks of the nodes on a path from a leaf to a
root increase monotonically
http://www.cp.eng.chula.ac.th/faculty/spj
y2
x1 0
0
1
0
z3
0
1
0
2
0
4
1 0
0
x1
0
Find-Set( x )
2y
3 z
2
0 1 0 1 0
0
0
4
1 0
0
http://www.cp.eng.chula.ac.th/faculty/spj
Properties of Ranks : #3
The number of nodes of rank r is at most n/2r
each node of rank r is the root of a subtree of at least
2r nodes (property #1)
all subtrees are disjoint
∴ there are at most n / 2r disjoint subtrees of rank r
there are at most n / 2r nodes of rank r
Properties of Ranks
For all tree roots x, size(x) ≥ 2rank(x)
The ranks of the nodes on a path from a leaf to a
root increase monotonically
The number of nodes of rank r is at most n/2r
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Analysis
Actual cost of Find-Set( x ) is the number of nodes
of the path from x to the root
Accounting method : deposit 1 baht for every node
on the path from x to the root
For m operations, how many bahts did we spend ?
Rank Grouping
Partition ranks into groups
group 0 contains only elements of rank 0
rank r goes to group G(r)
the largest rank in any group g is F(g), F = G -1
the number of ranks in any group g > 0 is F(g)-F(g-1)
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Rank Grouping : Example
eas
in
g
Find-Set( x )
+1 7
incr
call
y
em
ono
toni
s ar
Rank
0
1
2, 3, 4
5, 6, ..., 9
10, 11, , 16
(i-1)2+1, , i2
k #
G( r ) = r0.5
Group
0
1
2
3
4
i
Ran
F( g ) = g2
Money Deposit
+1
4
+1
x 2
1
9
8
+1
+1
5
group 1
0
group 0
http://www.cp.eng.chula.ac.th/faculty/spj
+1
group 2
+1 +1 +1
central
account
http://www.cp.eng.chula.ac.th/faculty/spj
Money Deposit
After path compression
x
0
9
2 4 5
1 +1 +1
Money Deposit Rules
Deposit one baht at the central account if
7 8
+1
All of the deposited nodes
along the access path get
a new parent of higher
+1 +1 +1
rank.
v is the root
the parent of v is the root
the parent of v is in a different group from v
Otherwise deposit one baht into the node
central
account
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Central Account
Central Account Deposit
9
After m operations, the total deposit at the central
account is at most m( G(n) + 2 )
Each operation deposits at most ( G(n) + 2 ) bahts.
8
7
5
3 groups : G(n) = 2
4
2
Find-Set( x )
http://www.cp.eng.chula.ac.th/faculty/spj
Central Account
Each operation deposits at most ( G(n) + 2 ) bahts.
The total deposit at the central account, after m
operations, is at most m( G(n) + 2 )
http://www.cp.eng.chula.ac.th/faculty/spj
x
+2 bahts
1
G(n) bahts
0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Accounts
Let node v be in group g.
How many bahts can be deposited into node v ?
If one baht is deposited into a node v, v will get a
new parent of higher rank.
http://www.cp.eng.chula.ac.th/faculty/spj
Money Deposit
9
Find-Set( x )
Money Deposit
9
Find-Set( x )
8
8
7
7
5
x
5
4
+1
4
+1
x 2 +1
2
1
1
0
0
http://www.cp.eng.chula.ac.th/faculty/spj
Money Deposit
9
After path compression
0
x
2
1
+1 +1
4
5
7
8
+1
All of the deposited nodes
along the access path get
a new parent of higher
rank.
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Node Accounts
Let node v be in group g.
How many bahts can be deposited into node v ?
If one baht is deposited into a node v, v will get a
new parent of higher rank.
How many times can v be moved before its parent
get pushed out of group g ?
Answer : the number of ranks in group g
= F(g) - F(g-1) - 1
( F(g) is the largest rank # in group g )
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
Node Account Deposit : Example
1
0
1
0
1 0 0 0 0 0 0 0 0 0 0 0 0
0
1
0
1 0 0 0 0 0 0 0 0 0 0
0
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
Node Account Deposit : Example
1
0
1
0
1
0
1
0
1 0 0 0 0 0 0 0 0
0
1
0
1
0
1
0
1 0 0 0 0 0 0
0
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
Node Account Deposit : Example
1
0
1
0
1
0
1
0
1
0
1
0
1 0 0 0 0
0
http://www.cp.eng.chula.ac.th/faculty/spj
1
0
1
0
1
0
1
0
1
0
1
0
0 0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
Node Account Deposit : Example
2
1 0
0
1
0
1
0
1
0
1
0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
2
1 0
0
1
0
2
0
1
0
1
0
1
0
1
0
2
1 0
0
1
0
2
0
1
0
2
0
2
1 0
0
http://www.cp.eng.chula.ac.th/faculty/spj
1
0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
2
1 0
0
1
0
2
0
1
0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
1
0
2
0
1
0
1
0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
0
2
1 0
0
1
0
3
0
1
0
2
0
2
1 0
0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
+1
1 2
0 0
1
0
3
0
2
0
1
0
Node Account Deposit : Example
2
1 0
0
+1
1 2
0 0
1
0
3
0
1
0
http://www.cp.eng.chula.ac.th/faculty/spj
Node Account Deposit : Example
+1
1 2
0 0
1
0
3
0
1
0
Node Account Deposit : Example
+2
Group
0
1
2
http://www.cp.eng.chula.ac.th/faculty/spj
The Number of Nodes in a Group
The number of nodes, V(g),
in group g > 0 is at most
n/2F(g-1)
F (g)
∑
V (g) ≤
r = F ( g −1) +1
∑
r = F ( g −1) +1
=
n
2
F ( g −1) +1
1
2r
∞
1
∑2
s =0
Rank
0
1, 2, 3
4, 5, ..., 8
1
0 2
0
1
0
3
0
1
0
2
0
4
1 0
0
Any node in group 1 can be deposited
at most 3 -1 = 2 bahts.
http://www.cp.eng.chula.ac.th/faculty/spj
Total Node Deposit
The maximum number of bahts deposited to all
nodes in group g is at most n F(g) / 2F(g-1)
n
2r
∞
≤n
3
1 0
0
http://www.cp.eng.chula.ac.th/faculty/spj
4
1 0
0
2
0
2
0
s
2n
2 F ( g −1) +1
n
= F ( g −1)
2
=
http://www.cp.eng.chula.ac.th/faculty/spj
the number of nodes in group g > 0 is at most n/2F(g-1)
each node gets at most F(g)-F(g-1)-1 bahts
F(g) - F(g-1) - 1 < F(g)
G(n)
The total node deposit is at most g=1
Σ n F(g) / 2F(g-1)
http://www.cp.eng.chula.ac.th/faculty/spj
Total Deposit
Total Deposit
G(n)
G(n)
m ( G(n)+2 ) + n Σ
F(g) / 2F(g-1)
g=1
m ( G(n)+2 ) + n Σ
F(g) / 2F(g-1)
g=1
Let F( g ) = g, G( r ) = r
Central account deposit
n
Total node deposit
m(n+2)+nΣ
g / 2(g-1) < m ( n + 2 ) + 2n
g=1
= O( m n )
What are G(n) and F(g) ?
Very loose upper bound, worse than O( m log n )
http://www.cp.eng.chula.ac.th/faculty/spj
http://www.cp.eng.chula.ac.th/faculty/spj
Total Deposit
G(n)
m ( G(n)+2 ) + n Σ
F(g) / 2F(g-1)
g=1
Let F( g ) = 2F(g-1), G( r ) = 1 + lg* n
1+lg*n
m ( 1+lg*n + 2 ) + n g=1
Σ 2(g-1) / 2(g-1)
= m ( 2+lg*n ) + n( 1+lg*n )
m = Ω(n)
= O( m log* n )
http://www.cp.eng.chula.ac.th/faculty/spj
Conclusion
n elements (n Make-Sets, at most n-1 Unions)
m Make-Set, Union, and Find-Set operations
m = Ω( n )
union by rank, path compression
worst case = O( m log* n )
: almost linear
amortized cost = O( log* n )
: almost constant
http://www.cp.eng.chula.ac.th/faculty/spj