Mathematics for Computer Science
MIT 6.042J/18.062J
Introduction to
Random Variables
Albert R Meyer, December 4, 2009
lec 13F.1
Random Variables
Informally: an RV is a number
produced by a random process:
• #hours to next system crash
• #faulty chips in production run
• avg # faulty chips in many runs
• #heads in n coin flips
Albert R Meyer, December 4, 2009
lec 13F.12
Intro to Random Variables
Example: Flip three fair coins
C ::= # heads (Count)
1 if all Match,
M :: 
0 otherwise.
Albert R Meyer, December 4, 2009
lec 13F.13
Intro to Random Variables
Specify events using values of variables
• [C = 1] is event “exactly 1 head”
Pr{C = 1} = 3/8
• Pr{C
1} = 7/8
• Pr{C·M > 0} = Pr{M>0 and C>0}
= Pr{all heads} = 1/8
Albert R Meyer, December 4, 2009
lec 13F.14
What is a Random Variable?
Formally,
R:
Sample space
Albert R Meyer, December 4, 2009
(usually)
lec 13F.15
Independent Variables
random variables R,S
are independent iff
[R = a], [S = b]
are independent
events for all a, b
Albert R Meyer, December 4, 2009
lec 13F.16
Independent Variables
alternate version:
Pr{R = a AND S = b} =
Pr{R = a} · Pr{S = b}
Albert R Meyer, December 4, 2009
lec 13F.18
Binomial Random Variable
Bn,p::= # heads in n independent flips.
Coin may be biased. So 2 parameters
n ::= # flips, p ::= Pr{head}.
C is binomial for 3 flips: C is B3,1/2
for n=5, p=2/3
Pr{HHTTH} =
Pr{H}⋅Pr{H}⋅Pr{T}⋅Pr{T}⋅Pr{H}
(by independence)
Albert R Meyer, December 4, 2009
lec 13F.26
Binomial Random Variable
Bn,p::= # heads in n independent flips.
Coin may be biased. So 2 parameters
n ::= # flips, p ::= Pr{head}.
C is binomial for 3 flips: C is B3,1/2
3
2
for n=5, p=2/3 Ê ˆ Ê ˆ
2˜˜ Á1 ˜˜
Á
Á
³
Pr{HHTTH} = Á
˜
˜
Á
Á
2
2
Á1˜˜ Á 1˜˜
3
³
3
³
Ë3¯ ³Ë3¯
3
Albert R Meyer, December 4, 2009
3
³
2
3
lec 13F.27
Binomial Random Variable
Bn,p::= # heads in n independent flips.
Coin may be biased. So 2 parameters
n ::= # flips, p ::= Pr{head}.
Pr{each sequence w/i H’s, n-i T’s} =
p 1- p
i
n- i
( )
Albert R Meyer, December 4, 2009
lec 13F.28
Binomial Random Variable
Bn,p::= # heads in n independent flips.
Coin may be biased. So 2 parameters
n ::= # flips, p ::= Pr{head}.
Pr{get i H’s, n-i T’s} =
Ênˆ˜
Á
i
˜˜ p 1 - p
Á
Á
˜
Á
Ëi ˜¯
n- i
( )
Albert R Meyer, December 4, 2009
lec 13F.29
Binomial Random Variable
Bn,p::= # heads in n independent flips.
Coin may be biased. So 2 parameters
n ::= # flips, p ::= Pr{head}.
Pr{
B n,p = i
}=
Ênˆ˜
Á
i
˜˜ p 1 - p
Á
Á
˜
Á
Ëi ˜¯
n- i
( )
Albert R Meyer, December 4, 2009
lec 13F.30
Density & Distribution
The Probability Density Function
of random variable R,
so
PDFR(a) ::= Pr{R = a}
Ênˆ˜
n- i
Á
i
˜
PDFB i = Á
˜˜ p 1 - p
Á
n,p
Á
Ëi ˜¯
()
( )
Albert R Meyer, December 4, 2009
lec 13F.31
Uniform Distribution
R is uniform iff PDFR is constant.
R ::= outcome of fair die roll.
Pr{R=1} = Pr{R=2} = ··· = Pr{R=6} = 1/6
S ::= 4-digit lottery number
Pr{S = 0000} = Pr{S = 0001} = ···
= Pr{S = 9999} = 1/10000
Albert R Meyer, December 4, 2009
lec 13F.33
Mathematics for Computer Science
MIT 6.042J/18.062J
Great
Expectations
Albert R Meyer, December 4, 2009
lec 13F.39
Carnival Dice
choose a number from 1 to 6,
then roll 3 fair dice:
win $1 for each match
lose $1 if no match
Albert R Meyer, December 4, 2009
lec 13F.44
Carnival Dice
Example: choose 5, then
roll 2,3,4: lose $1
roll 5,4,6: win $1
roll 5,4,5: win $2
roll 5,5,5: win $3
Albert R Meyer, December 4, 2009
lec 13F.45
Carnival Dice
Is this a
fair game?
Albert R Meyer, December 4, 2009
lec 13F.46
Carnival Dice
# matches probability
$ won
0
125/216
-1
1
75/216
1
2
15/216
2
3
1/216
3
Albert R Meyer, December 4, 2009
lec 13F.48
Carnival Dice
so every 216 games, expect
0 matches about 125 times
1 match about 75 times
2 matches about 15 times
3 matches about once
Albert R Meyer, December 4, 2009
lec 13F.49
Carnival Dice
So on average expect to win:
NOT fair!
125  ( 1)  75  1  15  2  1  3
216
17

 8 cents
216
Albert R Meyer, December 4, 2009
lec 13F.51
Carnival Dice
You can “expect” to lose 8 cents
per play. But you never actually
lose 8 cents on any single play,
this is just your average loss.
Albert R Meyer, December 4, 2009
lec 13F.52
Expected Value
The expected value of
random variable R is
the average value of R
--with values weighted
by their probabilities
Albert R Meyer, December 4, 2009
lec 13F.53
Expected Value
The expected value of
random variable R is
E[R]::= ∑v Pr{R = v}
17
so E[$win in Carnival] = 
216
Albert R Meyer, December 4, 2009
lec 13F.54
Expected Value
Alternative definition:
E[R] 
 R()  Pr{}
S
both forms are useful
Albert R Meyer, December 4, 2009
lec 13F.55
Expected Value
also called
mean value, mean, or
expectation
Albert R Meyer, December 4, 2009
lec 13F.59
Expected #Heads
n independent flips of a
coin with bias p for Heads.
How many Heads expected?
n
E Bn,p    k  Pr k Heads ?
  k 0
n
nk
 n k
::  k   p 1  p
k0  k
 
Albert R Meyer, December 4, 2009
lec 13F.61
Expected #Heads
n independent flips of a
coin with bias p for Heads.
How many Heads expected?
n
E Bn,p    k  Pr k Heads ?
  k 0
n
 n k nk
::  k   p q
k0  k
Albert R Meyer, December 4, 2009
lec 13F.62
Expected #Heads
we know how to get a closed
formula for this sum, but we’ll
see simpler approaches soon.
 n k nk
::  k   p q
k0  k
n
Albert R Meyer, December 4, 2009
lec 13F.63
Law of Total Expectation
conditional expectation:
E[R | A] ::  v  pr{R  v | A}
E[R]  E[R |A]  Pr{A}
 E[R |A]  Pr{A}
good for reasoning by cases
Albert R Meyer, December 4, 2009
lec 13F.68
Expected #Heads
Let e(n) ::= expected #H’s in n flips.
= 1 + e(n-1)
if 1st flip H
= e(n-1)
if 1st flip T
by Total Expectation:
e(n) = [1 + e(n-1)] p + e(n1) q
= np  E Bn,p 
e(n) = e(n-1) + p
Albert R Meyer, December 4, 2009
lec 13F.69
Mean Time to “Failure”
E[# flips until first head]?
p
H
q
p
H
q
p
B
q
H
Albert R Meyer, December 4, 2009
lec 13F.72
Mean Time to “Failure”
E[# flips until first head]?
p
H
q
B
B
now use Total Expectation
Albert R Meyer, December 4, 2009
lec 13F.73
Mean Time to “Failure”
E[# flips until first head]?
p
H
E=
E[# |1st is H]
1
q
B
B
p + E[# |1st is T]
Albert R Meyer, December 4, 2009
1+E
q
lec 13F.74
Mean Time to “Failure”
E[# flips until first head]?
p
H
E =1
q
B
B
p + [E+1]
Albert R Meyer, December 4, 2009
(1-p)
lec 13F.75
Mean Time to “Failure”
E[# flips until first head]
1

p
Albert R Meyer, December 4, 2009
lec 13F.76
Linearity of Expectation
A,B random variables, a,b
constants
E[aA + bB] =
aE[A] + bE[B]
Albert R Meyer, December 4, 2009
lec 13F.79
Expectation of indicator I
E[I] = 1⋅Pr{I=1} +
0⋅Pr{I=0}
= Pr{I=1}
Albert R Meyer, December 4, 2009
lec 13F.81
Expected #Heads
E  #H's   E H1 + H2 +
+ Hn 
where Hi is indicator
for Head on ith flip
E[Hi] = p
Albert R Meyer, December 4, 2009
lec 13F.82
Expected #Heads
E  #H's   E H1 + H2 +
so by linearity
 E H1  + E H2  +
 np
Albert R Meyer, December 4, 2009
+ Hn 
+ E Hn 
lec 13F.83
Chinese Banquet
Say n people sit around a
spinner (a “lazy-Susan”) with
n different dishes.
Spin randomly.
How many people do we expect
will get same dish as initially?
Albert R Meyer, December 4, 2009
lec 13F.86
Chinese Banquet
Let Ri be indicator for ith
person getting initial dish.
#people get initial dish =
R1 + R2 + … + Rn
Pr{Ri=1} = 1/n
Albert R Meyer, December 4, 2009
lec 13F.87
Chinese Banquet
E[# initial dishes] = E[∑
Ri]
so by linearity = ∑ E[Ri]
= ∑ Pr{Ri=1} = ∑ 1/n
= n(1/n) = 1
Albert R Meyer, December 4, 2009
lec 13F.88
Chinese Banquet
Ri’s are totally dependent
...all are 1 or all are 0
but linearity still holds
Albert R Meyer, December 4, 2009
lec 13F.89
Expectation & Independence
for independent R,S
E[R⋅S] = E[R]⋅E[S]
Albert R Meyer, December 4, 2009
lec 13F.90
Team Problems
Problems
1―4
Albert R Meyer, December 4, 2009
lec 13F.92