Mathematics for Computer Science
MIT 6.042J/18.062J
Great Expectations
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.1
Leader Election
• Everyone flips a coin
• Person who flips H becomes the leader
T
T
T
T
T
• If none or multiple Hs, repeat the process.
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.2
In Class Experiment
Question: How long does it take
to elect a leader?
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.3
Analysis
How long does it take to elect a leader?
1. What is the probability of electing a leader
in a given round?
2. What is the expected number of rounds
before a leader is successfully elected?
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.4
Success in a Given Round
Given n people,
Pr{leader is chosen in a given round}
= Pr{exactly one head}
1
n 
2
n 1
HTT
THT
TTH
1
 
2
1 head
n positions
for the head
Copyright © Radhika Nagpal, 2002. All rights reserved.
n-1 tails
April 24, 2002
L11-2.5
Success in a Given round
Given n people,
Pr{leader is chosen in a given round}
= Pr{exactly one head}
n
 n
2
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.6
How long does it take to elect a
leader?
What is the probability of electing a leader in a
given round?
n
– Answer: pR = n
2
What is the expected number of rounds before a
leader is successfully elected?
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.7
Mean Time to Failure/Success
If given round succeeds with pR,
then the expected number of rounds
till the first success:
E[#Rounds] = 1/pR
n
= 2 /n
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.8
Improvement: Use a Biased Coin
Let Pr{heads} = b,
Pr{tails} = 1-b.
n 1
Pr{exactly one head}  n 1  b  b
What happens when b is large ?
when b is small ?
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.9
Calculating the Optimal Bias
• Maximizing Pr{success in a given round}:


d
n 1
n 1  b  b  0
db
Optimal bias b = 1/n
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.10
Probability of Success in a Round
Pr{success in a given round}
 1
 1  
 n
n 1
pR ≈ 1/e
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.11
Mean Time to Failure/Success
If given round succeeds with pR=1/e,
then the expected number of rounds till the
first success:
E[#Rounds] = 1/pR
≈e
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
(≈ 2.7 rounds)
L11-2.12
Exercise
How can you simulate a biased coin
using a fair coin?
Say want Pr{heads} = b = 1/8?
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.13
Leader Election and Ethernet
• n computers have to talk on the same
coaxial cable, who gets to go first?
Metcalfe and
Boggs,1976
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.14
Exponential Backoff
Optimal bias p = 1/n, but what if you
don’t know what n is?
Be optimistic, use p=1
– If that fails, p = 1/2
– If that fails, p = 1/4
– If that fails, p= 1/8
–…
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.15
Analysis of Process
Expected Time =
Time taken per round · E[#Rounds]
What if time per round
is not fixed length?
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.16
Analysis of Process
Expected Time =
E[Time per round] · E[#Rounds]
If time per round is
not fixed length
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
This is
Wald’s Theorem
L11-2.17
Example
Roll a die until get a 6, what is the expected sum?
S ::= R1 + R2 + R3 … RQ
• Ri = value of the dice on roll i E[Ri]=3.5
• Q = number of rolls until a 6. E[Q]=6
(stopping rule)
Wald’s theorem: E[S] = E[Ri]  E[Q]
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.18
Herbert Simon
The Architecture of Complexity, 1962
Copyright © Radhika Nagpal, 2002. All rights reserved.
April 24, 2002
L11-2.19