Mathematics for Computer Science
MIT 6.042J/18.062J
Deviation of
Repeated Trials
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.1
Jacob Bernoulli (1659 – 1705)
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.2
Bernoulli Trials
Rn = number of heads in n
tosses of coin with bias p
E[Rn] = np
Var[Rn] = np(1-p)
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.3
Average of Bernoulli Trials
Rn
An ::=
n
= fraction of heads in n tosses
of a coin with bias p
E[An] = ?
Var[An] = ?
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.4
Expected Value and Variance
1
 Rn  1
E[ An ]  E    E[ Rn ]  np  p
n
n n
 Rn  1
Var[ An ]  Var    2 Var[ Rn ]
n n
1
 2 np (1  p )
n
p (1  p )

n
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.5
Average of Bernoulli Trials
Rn
An ::=
n
= fraction of heads in n tosses
of a coin with bias p
E[An] = p
p

(
1

p
)
Var[An] =
n
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
same irrespective of n
decreases as n increases!
L12-3.6
Average of Fair Coin Tosses
An = fraction of heads in n tosses
of a coin with bias p = ½
E[An] = ½
p  (1  p) 1

Var[An] =
n
4n
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.7
Binomial Distribution
E[An]=1/2
Pr{An= k}
0
Copyright © Radhika Nagpal, 2002. All
An
rights reserved.
May 1, 2002
Std dev
1
L12-3.8
Different Number of Coin Tosses
1000 coin tosses
100 coin tosses
10 coin tosses
E[An]
1
0
Std dev[An]
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.9
Implication
The more times I toss the coin, the
more likely it is that the average
will be close to the mean
• How likely?
• How close?
• Use Chebyshev
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.10
Probability of Deviation
What is the probability of deviating
more than x from the expected value?
Pr{| An  E[ An ] |  x}
Result of an
experiment
Copyright © Radhika Nagpal, 2002. All
Expected
value
rights reserved.
May 1, 2002
Deviation of
experiment
from the
expectation

Var[ An ]
x2
CHEBYSHEV:
The probability
is limited
L12-3.11
Probability of Deviation
Pr{| An  p |  x}
Probability of deviating
more than x from the
expected value
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002

p (1  p )
2
nx
Is less than
this bound
L12-3.12
In Class Exercise
Compute a bound on the probability that I get
fraction of heads within 5% of expected value?
If I toss
• 100 fair coins
• 1000 fair coins
• 10,000 fair coins
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.13
Solution
Pr{| An  p |  x} 
Pr{ ≥ 5% deviation} 
p (1  p )
2
nx
1
100

2
4(0.05) n
n
100
Pr{< 5% deviation} ≥ 1 
n
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.14
In Class Exercise
Compute a bound on the probability that I get
fraction of heads within 5% of expected value?
If I toss
• 100 fair coins
• 1000 fair coins
• 10,000 fair coins
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
Pr{< 5% deviation}
≥0
≥ 90%
≥ 99%
L12-3.15
Other Questions
• How many tosses do I need in order to get
within 5% of the mean, with 99%
probability? (find n)
• What if it is a biased coin?
(p≠½)
• What if I don’t know the bias? (Problem Set)
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.16
Polling/Sampling
What percentage of fish in the Charles River
have more than acceptable levels of some
toxin?
Procedure: catch n fish, test each one, and
compute the percentage of contaminated fish
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.17
Questions
If I catch 100 fish, what is the probability that
my procedure yields a value more than 5%
from the actual value?
If I want to be 95% confident of that my
results are within 5%, how many samples do
I need to take?
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.18
Model as Coin Tosses
Say that actual % contaminated fish was p.
Each sample amounts to a coin toss, where
Pr{contaminated} = p.
Collecting n samples: tossing n coins
Compute An = fraction of contaminated fish
Deviation is |An − p|
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.19
Probability of Deviation
Pr{| An  p |  x}
Average of
n samples
Actual
percentage
Error I am
willing to
tolerate
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002

p (1  p )
2
nx
“Confidence
Level”
L12-3.20
Confidence Level
Pr{ ≥ 5% deviation}
Problem: What is p??
Worst case: p =1/2!
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002

p(1  p)
(0.05) 2 n
(1  12 )

2
(0.05) n
1
2
L12-3.21
In Class Problem 1
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.22
6.170 Poll
• How many students do I need to poll in
order to determine the fraction of students
taking 6.170 within 4% tolerance with 95%
confidence.
• Estimate so far: poll 3000 students!
• ???
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.23
Better Polling
Pr{| An  p |  x}
BUT, if I know the PDF then I can
calculate this exactly!
= Binomial Distribution
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.24
Binomial PDF
n k
nk
Pr{Rn  k}    p (1  p)
k 
k 1
Pr{Rn  k}   Pr{Rn  i}
i 0
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.25
Probability of Deviation
Pr{| An  p |  x}
 Pr{ An  p  x}  Pr{ An  p  x}
 Pr{Rn  ( p  x)n}  Pr{Rn  ( p  x)n}
 Pr{Rn  ( 12  x)n}  Pr{Rn  ( 12  x)n}
 2  Pr{Rn  ( 12  x)n}
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.26
Binomial Approximation
Pr{Rn  n}  Fn ,1/ 2{n}
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.27
Probability of Deviation
Pr{| An  p |  x}  2  Fn ,1/ 2{( 12  x)n}
Average of
Actual
n samples percentage
Error I am
willing to
tolerate
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
“Confidence
Level”
L12-3.28
In Class Problem 2
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.29
Weak Law of Large Numbers
lim  P r{ An     }  1
n
Copyright © Radhika Nagpal, 2002. All
rights reserved.
May 1, 2002
L12-3.30