Randomized Algorithms
CS648
Lecture 17
Miscellaneous applications of Backward analysis
1
MINIMUM SPANNING TREE
2
Minimum spanning tree
d
17
h
v
19
6
3
22
1
a
10
b
7
16
12
u
4
3
9
2
c
11
5
x
y
18
15
13
3
Minimum spanning tree
d
17
h
v
19
6
3
22
1
a
10
b
16
7
Algorithms:
• Prim’s algorithm
• Kruskal’s algorithm
• Boruvka’s algorithm
12
u
4
3
9
2
c
11
5
x
y
18
15
13
Less known but it is the first algorithm for MST
4
Minimum spanning tree
𝑮 = (𝑽, 𝑬) : undirected graph with weights on edges
𝒏 = |𝑽|,
𝒎 = |𝑬|.
Deterministic algorithms:
Prim’s algorithm
1. O((𝒎 + 𝒏) log 𝒏) using Binary heap
2. O(𝒎 + 𝒏 log 𝒏) using Fibonacci heap
Best deterministic algorithm:
O(𝒎 + 𝒏 𝐥𝐨𝐠 ∗ 𝒏) bound
–
Too complicated to design and analyze
– Fails to beat Prim’s algorithm using Binary heap
5
Minimum spanning tree
When finding an efficient solution of a problem appears hard,
one should strive to design an efficient verification algorithm.
MST verification algorithm: [King, 1990]
Given a graph 𝑮 = (𝑽, 𝑬) and a spanning tree 𝑻, it takes O(𝒎 + 𝒏) time to
determine if 𝑻 is MST of 𝑮.
Interestingly, no deterministic algorithm for MST could use this algorithm to
achieve O(𝒎 + 𝒏) time.
6
Minimum spanning tree
𝑮 = (𝑽, 𝑬) : undirected graph with weights on edges
𝒏 = |𝑽|,
𝒎 = |𝑬|.
Randomized algorithm:
Karger-Klein-Tarjan’s algorithm [1995]
1. Las Vegas algorithm
2. O(𝒎 + 𝒏) expected time
This algorithm uses
• Random sampling
• MST verification algorithm
• Boruvka’s algorithm
• Elementary data structure
7
Minimum spanning tree
𝑮 = (𝑽, 𝑬) : undirected graph with weights on edges
𝒏 = |𝑽|,
𝒎 = |𝑬|.
Randomized algorithm:
Karger-Klein-Tarjan’s algorithm [1994]
1. Las Vegas algorithm
2. O(𝒎 + 𝒏) expected time
• Random sampling :
How close is MST of a random sample of edges to MST of original graph ?
The notion of closeness is formalized in the following slide.
8
Light Edge
d
17
h
v
19
6
3
22
1
a
10
b
7
16
12
u
4
3
9
2
31
c
11
5
x
y
18
15
13
Definition: Let 𝒂 ⊂ 𝑬. An edge 𝒆 ∈ 𝑬\𝒂 is said to be light with respect to 𝒂 if
MST({𝒆} ∪ 𝒂) ≠ MST(𝒂)
Question:
If 𝒂 ⊂𝑟 𝑬 and |𝒂|= 𝒌,
how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ?
Answer: ??< 𝒏 (𝒎 − 𝒌)
𝒌
9
USING BACKWARD ANALYSIS FOR
MISCELLANEOUS APPLICATIONS
10
PROBLEM 1
SMALLEST ENCLOSING CIRCLE
11
Smallest Enclosing Circle
12
Smallest Enclosing Circle
Question: Suppose we sample 𝒌 points randomly uniformly from a set of
𝒏 points, what is the expected number of points that remain outside the
smallest circle enclosing the sample?
𝒏
For 𝒌= 𝟐, the answer is < 𝟏𝟎
13
PROBLEM 2
SMALLEST LENGTH INTERVAL
14
Sampling points from a unit interval
Question: Suppose we select 𝒌 points from interval [0,1], what is expected
length of the smallest sub-interval ?
𝟏
• for 𝒌 = 1, it is ??
𝟒
• for 𝒌 = 2, it is ??
𝟏
𝟗
• General solution :
𝟏
𝒌+𝟏
??
𝟐
0
This bound can be derived using two methods.
• One method is based on establishing a relationship between uniform
distribution and exponential distribution.
• Second method (for nearly same asymptotic bound) using Backward
analysis.
1
PROBLEM 3
MINIMUM SPANNING TREE
16
Light Edge
d
17
h
v
19
6
3
22
1
a
10
b
7
16
12
u
4
3
9
2
31
c
11
5
x
y
18
15
13
Definition: Let 𝒂 ⊂ 𝑬. An edge 𝒆 ∈ 𝑬\𝒂 is said to be light with respect to 𝒂 if
MST({𝒆} ∪ 𝒂) ≠ MST(𝒂)
Question:
If 𝒂 ⊂𝑟 𝑬 and |𝒂|= 𝒌,
how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ?
17
USING BACKWARD ANALYSIS FOR
THE 3 PROBLEMS :
A GENERAL FRAMEWORK
18
A General framework
Let 𝒁 be the desired random variable in any of these problems/random
experiment.
• Step 1: Define an event 𝜺 related to the random variable 𝒁.
• Step 2: Calculate probability of event 𝜺 using standard method based on
definition. (This establishes a relationship between )
• Step 3: Express the underlying random experiment as a Randomized
incremental construction and calculate the probability of the event 𝜺 using
Backward analysis.
• Step 4: Equate the expressions from Steps 1 and 2 to calculate E[𝒁].
19
PROBLEM 3
MINIMUM SPANNING TREE
20
A BETTER UNDERSTANDING OF
LIGHT EDGES
21
Minimum spanning tree
d
17
(𝑽, 𝑬)
h
v
19
6
3
22
1
a
10
b
16
42
u
4
2
31
3
19
5
11
c
x
y
18
15
7
13
Random sampling
16
d
(𝑽, 𝑹)
h
v
19
4
31
22
1
b
a
5
c
10
u
15
11
x
y
18
22
Minimum spanning tree
d
17
(𝑽, 𝑬)
h
v
19
6
3
22
1
a
10
b
16
42
u
4
2
31
3
19
5
11
c
18
13
16
d
(𝑽, 𝑹)
v
19
b
a
5
31
c
10
u
MST(𝑹)
4
22
1
y
15
7
h
x
15
11
x
y
18
23
Minimum spanning tree
d
17
(𝑽, 𝑬)
h
v
19
6
3
22
a
1
10
b
42
u
4
31
3
1
v
10
b
7
y
18
15
16
d
3
a
x
23
19
6
5
11
c
17
h
19
2
7
(𝑽, 𝑹)
16
42
u
4
3
2
c
MST(𝑹)
19
11
5
x
𝑬\𝑹
y
15
23
24
Minimum spanning tree
d
17
(𝑽, 𝑬)
h
v
19
6
3
22
a
1
10
b
42
u
4
31
3
1
v
10
b
7
y
18
15
16
d
3
a
x
23
19
6
5
11
c
17
h
19
2
7
(𝑽, 𝑹)
16
42
u
4
3
2
c
MST(𝑹)
19
11
5
x
𝑬\𝑹
y
Light
15
23
25
First useful insight
Lemma1: An edge 𝒆 is light with respect to 𝒂 ⊂ 𝑬 if and only if
𝒆 belongs to MST( 𝒆 ∪ 𝒂).
26
Minimum spanning tree
d
17
(𝑽, 𝑬)
h
v
19
6
3
22
a
1
10
b
42
u
4
31
3
1
v
10
b
7
y
18
15
16
d
3
a
x
23
19
6
5
11
c
17
h
19
2
7
(𝑽, 𝑹)
16
42
u
4
3
2
c
MST(𝑹)
19
11
5
x
𝑬\𝑹
y
Light
15
23
heavy
27
Minimum spanning tree
d
(𝑽, 𝑬)
v
h
6
3
a
1
4
3
2
5
x
MST(𝑬)
y
c
u
b
7
Is there any relationship
among MST(𝑬), MST(𝑹)
and Light edges from 𝑬\𝑹 ?
d
17
(𝑽, 𝑹)
h
1
v
19
6
3
a
10
b
7
16
42
u
4
3
2
c
MST(𝑹)
19
11
5
x
𝑬\𝑹
y
Light
15
23
heavy
28
Second useful insight
Lemma2: Let
• 𝒂 ⊂ 𝑬 and
• 𝑳 be the set of all edges from 𝑬\𝒂 that are light with respect to 𝒂.
Then
MST(𝑬) = MST(𝑳 ∪ 𝐌𝐒𝐓(𝒂))
This lemma is used in the design of randomized algorithm for MST as
follows (just a sketch):
• Compute MST of a sample of 𝒎/𝟐 edges (recursively). Let it be T’.
• There will be expected 𝒏 edges light edges among all unsampled edges.
• Recursively compute MST of 𝑳 ∪ T’ edges which are less than 2𝒏 on
expectation.
29
Light Edge
d
17
h
v
19
6
3
22
1
a
10
b
7
16
12
u
4
3
9
2
31
c
11
5
x
y
18
15
13
Definition: Let 𝒂 ⊂ 𝑬. An edge 𝒆 ∈ 𝑬\𝒂 is said to be light with respect to 𝒂 if
MST({𝒆} ∪ 𝒂) ≠ MST(𝒂)
Question:
If 𝒂 ⊂𝑟 𝑬 and |𝒂|= 𝒌,
how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ?
We shall answer the above question using the Generic framework.
But before that, we need to get a better understanding of the
corresponding random variable.
30
𝒌
𝑹
MST(𝑹)
𝑬\𝑹
31
𝒌
𝑹
𝑬\𝑹
MST(𝑹)
Light
32
𝒌
𝑹
𝑬\𝑹
MST(𝑹)
Light
heavy33
𝒁: random variable for the number of light edges in 𝑬\𝑹 when 𝑹 is a random
sample of 𝒌 edges.
𝑺: set of all subsets of 𝑬 of size 𝒌.
𝒇 𝒂 : number of light edges in 𝑬\𝑹 when 𝑹 = 𝒂.
𝟏
|𝑺|
𝐄[𝒁] = ??
𝒇 𝒂
𝒂∈𝑺
Can you express 𝐄[𝒁] in
terms of 𝒇 𝒂 and 𝑺 only ?
34
Step 1
Question:
Let 𝑹 be a uniformly random sample of 𝒌 edges from 𝑬.
What is the prob. 𝒑 that an edge selected randomly from 𝑬\𝑹 is a light edge ?
Two methods to find 𝒑
35
Step 2
Calculating 𝒑 using definition
36
Step 2
Calculating 𝒑 using definition
𝒌
𝒂
Light edges
=𝒇 𝒂
𝑬\𝒂
MST(𝒂)
Light
heavy37
Step 2
Calculating 𝒑 using definition
𝑺: set of all subsets of 𝑬 of size 𝒌.
The probability 𝒑 is equal to
=
=
=
𝟏
?
|𝑺|
?𝒇 𝒂
𝒂∈𝑺
𝟏
𝒎−𝒌
𝟏
𝒎 − 𝒌 |𝑺|
𝟏
𝒎−𝒌
𝒇 𝒂
𝒂∈𝑺
𝐄[𝒁]
38
Step 3
Expressing the entire experiment as Randomized Incremental Construction
A slight difficulty in this process is the following:
• The underlying experiment talks about random sample from a set.
• But RIC involves analyzing a random permutation of a set of elements.

Question: What is relation between random sample from a set and a random
permutation of the set ?
Spend some time on this question before proceeding further.
39
random sample and random permutation
Random permutation of 𝑬
𝑬\𝑨
𝑨
𝒓
𝒎−𝒓
Observation:
𝑨 is indeed a uniformly random sample of 𝑬
40
Step 3
The underlying random experiment as Randomized Incremental Construction:
• Permute the edges randomly uniformly.
• Find the probability that 𝒊th edge is light relative to the first 𝒊 − 𝟏 edges.
Question: Can you now calculate probability 𝒑 ?
Spend some time on this question before proceeding further.
41
Step 3
Random permutation of 𝑬
𝒆𝟏𝒆𝟐
…
𝒆𝒊
𝑬𝒊−𝟏
42
Step 3
Random permutation of 𝑬
𝒆𝟏𝒆𝟐
…
𝑬𝒊−𝟏
𝒆𝒊
𝑬\𝑬𝒊−𝟏
𝑿𝒊 :
a random variable taking value 1 if 𝒆𝒊 is a light edge with respect to MST(𝑬𝒊−𝟏 ).
43
Step 3
Random permutation of 𝑬
𝒆𝟏𝒆𝟐
…
𝒆𝒊
𝑬𝒊−𝟏
𝑬\𝑬𝒊−𝟏
𝑿𝒊 :
a random variable taking value 1 if 𝒆𝒊 is a light edge with respect to MST(𝑬𝒊−𝟏 ).
Question: What is relation between 𝒑 and 𝑿𝒊 ’s?
Answer: ??𝒑 = 𝐏(𝑿𝒌+𝟏 = 𝟏).
44
Calculating 𝐏(𝑿𝒊 = 𝟏).
Random permutation of 𝑬
𝒆𝟏𝒆𝟐
…
𝑬𝒊−𝟏
𝒆𝒊
Forward analysis
𝑺𝒊−𝟏 : set of all subsets of 𝑬 of size 𝒊 − 𝟏.
𝐏(𝑿𝒊 = 𝟏) =
𝒂∈𝑺𝒊−𝟏 𝐏(𝑿𝒊 = 𝟏 𝑬𝒊−𝟏 = 𝒂 ∙ 𝐏(𝑬𝒊−𝟏 = 𝒂)
 depends upon 𝒂
MST(𝒂)
45
Calculating 𝐏(𝑿𝒊 = 𝟏).
Random permutation of 𝑬
𝒆𝟏𝒆𝟐
…
𝑬𝒊
𝒆𝒊
Backward analysis
𝑺𝒊 : set of all subsets of 𝑬 of size 𝒊.
𝐏(𝑿𝒊 = 𝟏)=
𝒂∈𝑺𝒊−𝟏 𝐏(𝑿𝒊 = 𝟏 𝑬𝒊 = 𝒂 ∙ 𝐏(𝑬𝒊 = 𝒂)
46
𝐏(𝑿𝒊 = 𝟏 𝑬𝒊 = 𝒂
Random permutation of 𝑬
𝒆𝟏𝒆𝟐
…
𝑬𝒊
𝒆𝒊
Backward analysis
Use Lemma 2.
𝐏(𝑿𝒊 = 𝟏 𝑬𝒊 = 𝒂
= ??
𝐏 𝒊th edge 𝐛𝐞𝐥𝐨𝐧𝐠𝐬 to MST(𝒂)
|MST(𝒂)|
= ??
≤
𝒏−1
𝒊
𝒊
MST(𝒂)
47
Calculating 𝐏(𝑿𝒊 = 𝟏)
Random permutation of 𝑬
𝒆𝟏𝒆𝟐
…
𝑬𝒊
𝒆𝒊
Backward analysis
𝑺𝒊 : set of all subsets of 𝑬 of size 𝒊.
𝐏(𝑿𝒊 = 𝟏)=
𝒂∈𝑺𝒊−𝟏 𝐏(𝑿𝒊 = 𝟏 𝑬𝒊 = 𝒂 ∙ 𝐏(𝑬𝒊 = 𝒂)
≤
=
=
𝒂∈𝑺𝒊−𝟏
𝒏−1
𝒊
𝒏−1
𝒊
𝒏−1
𝐏(𝑬𝒊 = 𝒂)
𝒊
𝒂∈𝑺𝒊−𝟏
𝐏(𝑬𝒊 = 𝒂)
48
Combining the two methods for calculating 𝒑
Using method 1:
𝟏
𝒑 =
𝐄[𝒁]
𝒎−𝒌
Using method 2:
𝒑 = 𝐏(𝑿𝒌+𝟏 = 𝟏)
𝒏−1
<
𝒌+𝟏
Hence:
𝐄[𝒁] < 𝒏−1
(𝒎 − 𝒌)
𝒌+𝟏
𝒏
𝒌
≤ (𝒎 − 𝒌)
49
Theorem: If we sample 𝒌 edges uniformly randomly from an
undirected graph on 𝒏 vertices and 𝒎 edges, the number of light
𝒏
edges among the unsampled edges will be less than (𝒎 − 𝒌)
𝒌
on expectation.
50