Mathematics for Computer Science
MIT 6.042J/18.062J
Deviation from
the Mean
Albert R Meyer, May 12, 2009
lec 14W.1
Don’t expect the Expectation!
Toss 101 fair coins.
E[#Heads] = 50.5
Albert R Meyer, May 12, 2009
lec 14W.2
Don’t expect the Expectation!
Pr{exactly 50.5 Heads} == 0
?
Pr{exactly 50 Heads} < 1/13
Pr{50.5 §1 Heads}
< 1/7
Albert R Meyer, May 12, 2009
lec 14W.3
Don’t expect the Expectation!
Toss 1001 fair coins.
E[#Heads]
= 500.5
Pr{#H = 500}
< 1/39
Pr{#H = 500.5§1 } < 1/19
smaller
Albert R Meyer, May 12, 2009
lec 14W.4
Within a % of the mean?
of 1001
Toss 1001 fair coins.
Pr{#H = 500 § 1%}
= Pr{#H = 500 § 10}
0.49
not so bad
Albert R Meyer, May 12, 2009
lec 14W.5
Giving Meaning to the Mean
Let  ::= E[R]. What is
Pr{R far from }?
Pr{|R - μ|  x}
R’s average deviation ?
E[ |R − µ| ] ?
Albert R Meyer, May 12, 2009
lec 14W.6
Two Dice with Same Mean
Fair Die
• E[D1] = 3.5
Loaded Die throwing only 1 & 6:
• E[D2] = (1+6)/2 = 3.5 also!
Albert R Meyer, May 12, 2009
lec 14W.8
Two Dice with Same Mean
deviation from the mean
Fair
1.5 on average
Pr{D = i}
Loaded
1
0
1
2.5
0
i: 0 1 2 3 4 5 6 7
Albert R Meyer, May 12, 2009
lec 14W.9
Dice have Different Deviations
Fair Die:
E[ |D1
| ] = 1.5
E[ |D2
| ] = 2.5
Loaded Die:
Albert R Meyer, May 12, 2009
lec 14W.10
Giving Meaning to the Mean
The mean alone is not a good
predictor of R’s behavior. We
generally need more about its
distribution, especially probable
deviation from its mean.
Albert R Meyer, May 12, 2009
lec 14W.11
Example: IQ
IQ measure was constructed so
that
average IQ = 100.
What fraction of the people
can possibly have an IQ ≥ 300?
Albert R Meyer, May 12, 2009
lec 14W.15
IQ Higher than 300?
Fraction f with IQ ≥ 300
adds ≥ 300f to average,
so 100 = avg IQ ≥ 300f:
f
100/300 = 1/3
Albert R Meyer, May 12, 2009
lec 14W.16
IQ Higher than 300?
At most 1/3 of people
have IQ ≥ 300
E[IQ]
Pr{IQ  300} 
300
Albert R Meyer, May 12, 2009
lec 14W.17
IQ Higher than x?
In general,
100
Pr{IQ  x} 
x
Albert R Meyer, May 12, 2009
lec 14W.18
IQ Higher than x?
Besides mean = 100,
we used only one fact about
the distribution of IQ:
IQ is always nonnegative
Albert R Meyer, May 12, 2009
lec 14W.19
Markov Bound
If R is nonnegative, then
Pr{R  x} 
E R 
x
for x > 0.
Albert R Meyer, May 12, 2009
lec 14W.20
Markov Bound
•Weak
•Obvious
•Useful anyway
Albert R Meyer, May 12, 2009
lec 14W.22
IQ ≥ 300, again
Suppose we are given that IQ
is always ≥ 40?
Get a better bound on fraction
f with IQ ≥ 300, by considering
(IQ – 40)
since this is now ≥ 0.
Albert R Meyer, May 12, 2009
lec 14W.25
IQ ≥ 300, again
f contributes 300f to the
average of (IQ-40), so
60 = E[IQ-40] ≥ 300f
f ≤ 60/300 = 1/5
Better bound from Markov by
shifting R to have 0 as minimum
Albert R Meyer, May 12, 2009
lec 14W.26
Improving the Markov Bound
Pr{|R−µ| ≥ x}
2
2
= Pr{(R−µ) ≥ x }
by Markov:
E[(R - µ) ]

2
x
variance of R
2
Albert R Meyer, May 12, 2009
lec 14W.27
Chebyshev Bound
Var[R]
Pr{|R - μ | x} 
2
x
Albert R Meyer, May 12, 2009
lec 14W.28
Variance and Standard Deviation
σR ::= Var[R]
PDFR


Albert R Meyer, May 12, 2009
lec 14W.30
Standard Deviation
Pr{|R -μ| x} 

2
x
2
R probably not many σ’s from :
further than
σ
2σ
3σ
4σ
Pr ≤ 1
Pr ≤ 1/4
Pr ≤ 1/9
Pr ≤ 1/16
Albert R Meyer, May 12, 2009
lec 14W.32
Calculating Variance
Var[aR  b]  a Var[r]
2
Var[R]  E[R ]  E[R] 
2
2
simple proofs applying linearity
of E[] to the def of Var[]
Albert R Meyer, May 12, 2009
lec 14W.35
Calculating Variance
Pairwise Independent Additivity
Var[R1 + R2 +
+ Rn ]
= Var[R1 ] + Var[R2 ] +
+ Var[Rn ]
providing R1,R2,…,Rn are
pairwise independent
again, a simple proof applying
linearity of E[] to the def of Var[]
Albert R Meyer, May 12, 2009
lec 14W.38
Team Problems
Problems
1 5
Albert R Meyer, May 12, 2009
lec 14W.39