Algorithms
CSCI 235, Fall 2016
Lecture 9
Probability
1
Permutations
How many ways can you take k items from a set of n items?
Order matters (ab is counted separately from ba)
No duplicates allowed (don't count aa)
Example: S = {a, b, c, d}. How many ways can we take 2 items?
ab
ac
ad
ba
bc
bd
ca
cb
cd
da
db
dc
Total pairs = 12
4 ways to pick first
3 ways to pick second
In general:
The number of permutations of k items from a set of n is:
n!
n(n -1)(n - 2)...(n - k + 1) =
(n - k)!
2
Combinations
Combinations are like permutations, except order doesn't matter.
ab and ba are considered the same.
Combination of 4 things taken 2 at a time:
ab
ac bc
ad bd cd
Each set of k items has k! possible permutations.
number of combinations = number of permutations divided by k!
ænö
n!
ç ÷=
è k ø k!(n - k)!
3
k-tuples and k-selections
k-tuple (or k-string): Like a permutation, but can have duplicates.
aa
ab
ac
ad
ba
bb
bc
bd
ca
cb
cc
cd
da
db
dc
dd
n choices for 1st item
n choices for 2nd item, etc.
Total possible k-tuples for k items
taken from a group of n items is nk
k-selection: Like combination, but can have duplicates.
aa
ab
ac
ad
bb
bc
bd
Formula:
cc
cd
(n -1 + k)!
k!(n -1)!
dd
4
Sample Spaces
A sample space is the set of all possible outcomes for an
experiment.
Experiment A: flip 3 coins
Sample space = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
Number of possibilities?
Experiment B: Flip a coin until you get heads.
Sample space = {H, TH, TTH, TTTH, . . .}
5
Diagramming sample spaces
Experiment A:
(flip 3 coins)
Experiment B:
(flip until heads)
6
Events
An event is a subset of the sample space:
Experiment A (flip 3 coins):
Event A1: First flip is head = {HHH, HHT, HTH, HTT}
Event A2: Second flip is tail = {HTH, HTT, TTH, TTT}
Event A3: Exactly 2 tails = {HTT, THT, TTH}
Event A4: Two consecutive flips are the same=
{HHH, HHT, HTT, THH, TTH, TTT}
Experiment B (flip until get heads):
Event B1: First flip is head = {H}
Event B2: First flip is tail = {TH, TTH, TTTH ...}
Event B3: Even number of flips = {TH, TTTH, TTTTTH}
7
Definitions
If A and B are events in sample space S, then
"A and B" is translated A Ç B
"A or B" is translated A È B
"not A" is translated
S-A
Two events are mutually exclusive if
AÇ B =Æ
Examples:
1. Of the four events in Experiment A, which pairs are
mutually exclusive?
2. Of the three events in Experiment B, which pairs are
mutually exclusive?
8
Probability Distribution
A probability distribution Pr{ } on sample space S is any mapping
from events of S to [0...1] such that the following axioms hold:
1) Pr{A} >= 0 for any event A
2) Pr{S} = 1
3) If A Ç B = Æ then Pr{A È B} = Pr{A} + Pr{B}
Example:
Flip two coins: S= {HH, HT, TH, TT}
Event A = {HT}
Event B = {TH}
What is
Pr{A È B}?
9
Some useful theorems
1) Pr{Æ} = 0
2) Pr{S - A} = 1- Pr{A}
4) Pr{AÈB} = Pr{A} + Pr{B} - Pr{AÇB}
Example: Flip two coins.
Event A = {HH, HT, TH}
Event B = {HT, TH, TT}
What is Pr{A È B}?
10
Working with probability trees
If events at distinct stages of a probability tree are
independent, then the probability of a leaf is the product of
the probabilities on the path to the leaf.
Experiment A (flip 3 weighted coins): H1=1/3, T1 = 2/3; H2=1/4, T2=3/4;
H3=1/5, T3 = 4/5
H2
H1
T1
T2
H2
T2
H3
T3
H3
T3
H3
T3
H3
T3
H1H2H3
H1H2T3
H1T2H3
H1T2T3
T1H2H3
T1H2T3
T1T2H3
T1T2T3
11
Another example
Experiment B (flip a weighted coin until heads): Hi = 1/3, Ti = 2/3
H1
H1
T1H2
H2
T1T2H3 T1T2T3H4
H3
H4
T1
T2
T3
T4 . . .
Probability of S = ?
12