CS70: Jean Walrand: Lecture 22.
How to model uncertainty?
Why Probability?
I
I
3. Sample Spaces.
I
I
I
I
I
I
I
I
Weather
Stock market
Noise and corruption of signals your cell phone receives
Diseases, etc.
I
I
I
Examples of man-made randomness:
I
I
I
Play rock-paper-scissors, or tennis, etc.
WiFi algorithm: randomized
Randomized Quicksort
I
Some applications (they are everywhere ...):
Probability.
I
The probability of getting a straight is around 1 in 250.
Straight: Consecutive cards, suit doesn’t matter, e.g.,
I
I
“Heads or Tails” in coin tossing (or coin flipping).
I
I
The probability of rolling snake eyes is 1/36. How many
snake eyes? one pip and one pip. 1 ∗ 1 = 1.
The probability that a poll of a 1000 people will report at
least 50% support for a candidate with 60% support is
80%.
Models knowledge about uncertainty
Discovers best way to use that knowledge in making
decisions
Terminology
I
I
Play games of chance
Design randomized algorithms.
Probability
I
Applications
Buy stocks
Detect signals (transmitted bits, speech, images, radar,
diseases, etc.)
Control systems (Internet, airplane, robots, self-driving
cars, etc.)
How to best use ‘artificial’ uncertainty?
I
I
Coin flip: 50% chance of ‘tails’ [subjectivist]
Many coin flips: About half yield ‘tails’ [frequentist]
How to best make decisions under uncertainty?
I
Examples of unpredictability:
I
4. Examples.
Model key feature of life: Unpredictability
Man-made randomness: Randomized algorithms
Unpredictable does not mean “nothing is known”
I
Two aspects of probability:
I
1. Why Probability?
2. Applications.
Key insight
I
One side of a coin is called the “heads” side and the other
the “tails” side.
Thus, one says that one gets one heads or one tails in a
coin flip, or that the coin flip is heads or tails.
Rolling one die or two dice.
I
Singular is die and plural is dice.
Probability
Probability Space.
1. A “random experiment”:
I
If you flip a fair coin and get 50 heads, you will get heads
the next time with probability ...1/2.
I
The probability that the next person through the door is
younger than 21 is 80%.
I
Amanda Knox is innocent with 70% probability.
They are statements about a probability space.
(Except perhaps the last one or two.)
Random experiment constructed by us, or the world.
Probability Space: Formalism.
In a uniform probability space each outcome ω is equally
1
probable: Pr [ω] = |Ω|
for all ω ∈ Ω.
(a) Flip a biased coin;
(b) Flip two fair coins;
(c) Deal a poker hand.
2. A set of possible outcomes: Ω.
(a) Ω = {H, T };
(b) Ω = {HH, HT , TH, TT }; |Ω| = 4;
(c) Ω = { A♠ A♦ A♣ A♥ K ♠, A♠ A♦ A♣ A♥ Q♠, . . .}
|Ω| = 52
5 .
Probability Space: formalism.
Ω is the sample space.
ω ∈ Ω is a sample point. (Also called an outcome.)
Sample point ω has a probability Pr [ω] where
I
I
0 ≤ Pr [ω] ≤ 1;
∑ω∈Ω Pr [ω] = 1.
3. Assign a probability to each outcome: Pr : Ω → [0, 1].
(a) Pr [H] = p, Pr [T ] = 1 − p for some p ∈ [0, 1]
(b) Pr [HH] = Pr [HT ] = Pr [TH] = Pr [TT ] = 14
(c) Pr [ A♠ A♦ A♣ A♥ K ♠ ] = · · · = 1/ 52
5
Probability Space: Formalism
Simplest physical model of a uniform probability space:
Probability Space: Formalism
Simplest physical model of a non-uniform probability space:
p!
!
p3
3
2
p2
Examples:
I
Flipping two fair coins, dealing a poker hand are uniform
probability spaces.
I
Flipping a biased coin is not a uniform probability space.
A bag of identical balls, except for their color (or a label). If the
bag is well shaken, every ball is equally likely to be picked.
1
Fraction p1
of circumference
The roulette wheel stops in sector ω with probability pω .
(Imagine a perfectly constructed wheel, that you spin hard
enough, with low friction... .)
Probability of exactly one heads in two coin flips?
Idea: Sum the probabilities of the outcomes with one heads.
Event
Probability of exactly one heads in two coin flips?
Leads to a definition!
p!
An event, E, is a subset of outcomes.
Pr [E] = ∑ω∈E Pr [ω].
!
Sample Space, Ω = {HH, HT , TH, TT }.
Uniform probability space:
Pr [HH] = Pr [HT ] = Pr [TH] = Pr [TT ] = 14 .
Event, E, “exactly one heads”: {TH, HT }.
p3
3
1
2
p1
p2
E
Event E = ‘wheel stops in sector 1 or 2 or 3’ = {1, 2, 3}.
Pr [E] =
Pr [E] = p1 + p2 + p3
|E|
2
1
∑ Pr [ω] = |Ω| = 4 = 2 .
ω∈E
Probability of a straight?
Construct straight:
Calculation.
Recall: Straight := Consecutive cards, suit does not matter.
Outcomes: Ω = “poker hands”.
Uniform probability space
Pr [ω] =
1
1
= 52 .
|Ω|
5
Event E = { a straight }.
|E|
∑ Pr [ω] = |Ω| .
ω∈E
Pr [E] =
10 ∗ 45
52
5
First choose the smallest value of the cards:
{A, . . . , 10} : 10 ways
and then five choices of suit: 5 choices, 4 ways for each
|E| = 10 × 4 × 4 × 4 × 4 × 4 = 10 × (45 )
Pr [E] =
∑ Pr [ω] =
ω∈E
10 ∗ 45
52
5
irb(main):004:0*> 52*51*50*49*48/((5*4*3*2)*10*4**5)
=> 253
Thus,
Pr [straight] ≈
1
.
253
Is a flush more likely than a straight?
A flush is a hand that contains five cards of the same suit.
Outcomes: Ω = “poker hands”.
Uniform probablity space
Pr [ω] =
1
1
= 52 .
|Ω|
20 coin tosses
Pr [E] =
5
Event E = { a flush }.
|E|
∑ Pr [ω] = |Ω| .
ω∈E
|E|?
Construct flush:
First choose the suit – 4 ways
13
and then choose five cards from 13:
5 ways
13
|E| = 4 ×
5
13
Plug in.
|E| 4 5
= 52 .
Pr [ω] =
|Ω|
5
Summary
How to model uncertainty?
Key ideas:
I
Random experiment
I
Probability space
I
I
I
Sample space Ω
Probability: Pr (ω)
Event: E ⊆ Ω; Pr [E].
Sample space: Ω = set of 20 fair coin tosses = {H, T }20 .
Calculation.
4
13
5
52
5
|Ω| = 2 × 2 × · · · × 2 = 220 .
I
I
I
irb(main):001:0> 52*51*50*49*48/(4*13*12*11*10*9)
=> 504
Hence,
Thus, a straight is about twice as likely as a flush. (1/253 vs.
1/504.)
ω1 := HHHHHHHHHHHHHHHHHHHH, or
ω2 := HHTHTTHHTTHTHHTTHTHT ?
Answer: Both are equally likely: Pr [ω1 ] = Pr [ω2 ] =
I
1
.
Pr [flush] ≈
504
What is more likely?
What is more likely?
1
|Ω| .
(E1 ) Twenty Hs out of twenty, or
(E2 ) Ten Hs out of twenty?
Answer: Ten Hs out of twenty.
Why? There are many sequences of 20 tosses with ten Hs;
1
2|
only one with twenty Hs. ⇒ Pr [E1 ] = |Ω|
Pr [E2 ] = |E
|Ω| .
20
|E2 | =
= 184, 756.
10