Mathematics for Computer Science
MIT 6.042J/18.062J
Random Walks
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.1
Random Walks
Broad Class of Distributions:
• Markov Chains
• Hidden Markov Models (HMMs)
• Martingales
• Gambler’s Ruin
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.2
Random Walks
What do they have in common?
– State space
– Probability of transition from one state to
another
Analytic results depend on the structure
of the probability space, often by
appealing to symmetries
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.3
Random Walks
Applications
• Physics – Brownian Motion
• Algorithms – web search, clustering
• Finance – Stocks, options
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.4
Random Walks
Different Representations:
Time or “steps”
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.5
1-Dimensional Walk:
The Gambler’s Ruin
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.6
The Gambler’s Ruin
T
$$$
n
# of bets
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.7
The Gambler’s Ruin
Parameters:
• n::= initial capital (stake)
• T::= gambler’s target
• p::= Pr{win $1 bet}
• q::= 1– p
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.8
The Gambler’s Ruin
Three general cases:
• Biased against
• Biased in favor
• Unbiased (Fair)
Copyright © Albert R. Meyer, 2002. All
rights reserved.
p < 1/2
p > 1/2
p = 1/2
May 3, 2002
L13-3.9
Unbiased Analysis
Let w::= Pr{reach target}
E[$$] = w·(T – n) + (1 – w)·(–n)
= wT – n
But game is fair, so E[$$ won] = 0
w = n/T
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.10
Unbiased Analysis
Consequences
n=500, T=600
Pr{win $100} = 500/600  0.83
n=1,000,000, T=1,000,100
Pr{win $100}  0.9999
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.11
Biased Against
Betting red in US roulette
p = 18/38 = 9/19 < 1/2
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.12
Biased Against
Astonishing Fact!
Pr{win $100 starting with $500}
< 1/37,000 !
(Was 5/6 in the unbiased case.)
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.13
Biased Against
More amazing still!
Pr{win $100 starting with $1M} < 1/37,000
Pr{win $100 starting w/ any $n stake}
< 1/37,000
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.14
More analysis
In unbiased case, why was
E[amount won] = 0?
Define Random variables, Gi
0 if game ends in  i bets

th
Gi ::  1 if gambler wi ns i bet
 1 if gambler loses i th bet

Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.15
More analysis

$$ won   Gi
i 1
E[$$ won]
Copyright © Albert R. Meyer, 2002. All
rights reserved.
= E[Gi]
=  E[Gi]
=0
=0
May 3, 2002
L13-3.16
More analysis
WAIT!
ALARM!
This is just like
the bet-doubling (St. Peterburg) paradox
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.17
More analysis
We must verify that



E
G
 i
i 1
converges.
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.18
More analysis
1
Gi  
0
if  i bets are made,
game ends before ith bet.
EGi  = Pr{game takes  i bets}
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.19
More analysis
In-class Problem 1:
i
Pr{game takes  i bets}  (1 + )
so


i 1
i 1
i
 EGi    (1   )  
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.20
More analysis
i
Pr{game takes  i bets}  (1 + )
so Pr{game takes forever}= 0.
Already assumed in:
Pr{loss} = 1 – w = 1 – Pr{win}
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.21
In-class Problems
1, 3, 2.
Copyright © Albert R. Meyer, 2002. All
rights reserved.
May 3, 2002
L13-3.22