Slides 8: Applying Probability
• Define problem of interest
– in terms of ‘random variables’ and/or ‘composite events’
• Use real world knowledge, symmetry
– to associate probabilities in [0,1] with ‘elementary events’
– all probabilities are conditional on real world knowledge
• Use consistent probability rules
– to associate probabilities with random variables / composite events
– multiplication and addition rules
1
Probability
• Week 7 Probability Rules
– Basic in text chapter 2.2
– Conditional probability / Bayes rule in chapter 6
– Fuller treatment in Chapters 7 and 8
• Week 8 Discrete Probability Distribution
– Chapter 4 (see lab on queuing) and Chapter 9
• Week 9 Continuous probability distribution
– Chapter 5 Normal distribution and Chapter 10
• Note we give more emphasis to ‘event identities’. The book in Chapter 7
uses more math shortcuts (binomial coefficients) and Σ notation than we
will use. Best immediate preparation is Q1-Q12 in Chapter 1 which you can
formulate via EXCEL before attempting probability solution.
2
Events, Random Variables, Sample Space
and Probability Rules
• Event A
– Simplest Random Variable
– Values of A are TRUE/FALSE
• Random Variable Y
– Values of Y are y1 , y2 , . . . , yk (sample space; exhaustive list)
– Events such as Y = y
3
Event Algebra
• Event Identities: Re-express compound events in and/or combinations of
elementary events
– Coin H or T
– Cards Ace ⇒ Ace Spades, or Ace Hearts, or Ace Clubs, or Ace
Diamonds.
– Red and not diamonds ⇒ 2 hearts or ... or Ace hearts
• Not A = Ā
• D = A or B
• D̄ = Ā and B̄
• D = (A and B̄) or (Ā and B) or (A and B)
• A = (A and B̄) or (A and B)
• φ = A and Ā Disjoint / Mutually Exclusive
• Ω = φ̄ = A or Ā Exhaustive
4
Event Identities
• Re-express in terms of and/or combinations of elementary events and/or
simple compound events (often there is more than one way).
• ‘A out-right winner of league’
– Use as elementary events outcomes of games A/B etc. and as relatively
simple compound events the scores NA etc.
– ‘At least one Queen in two cards’
– ‘Max of 3 dice is 3’ and ‘Max of 3 dice is ≤3’
– ‘Sum of 3 is 4’
5
Event Identities
• Coins: Elementary events H1 , T1 , H2 , . . .
• Define F (3) = First head on 3rd toss
• F (3) = T1 , T2 , H3
• Define F (> 3) = First head on 4th or higher toss
• F (> 3) = (T1 , T2 , T3 , H4 ) or (T1 , T2 , T3 , T4 , H5 ) or . . .
• Alternatively F (> 3) = N OT (F (1) or F (2) or F (3))
6
Event Identities: Coin Toss
7
Event Identities: Password
8
Event Identities
• E =No common birth date in class.
• Define Yi as birth date of student i (takes values in 1, 2, . . . , 365)
• E = (Y1 any date n1 ) AN D( Y2 any date n2 except n1 )
AN D( Y3 any date n3 except n1 , n2 ) . . .
9
Event Identity
• Define composite event Bn = Ball in box n
• Elementary events L1 , R1 , L2 , R2 , . . .
• B2 = (L1 , L2 , L3 , L4 )
• B3 =?
10
Event Identities for Random Variables
• Random Variables
• T3 = Time to third taxi.
• Y (t) = Number of taxis arriving in next t minutes.
• Event Identity (T3 > t mins) = (Y (t) < 3)
11
Probability Rules
• P (A) = P (A is true) ∈ [0, 1], P (Ω) = 1
• P (A or B) = P (A) + P (B) if mutually exclusive
• Event Identity (A or Ā) = Ω
• Whence P (A or Ā) = P (A) + P (Ā) = 1
• Also P (φ) = 0
• P (A and Ā) = 0
• Addition rule P (A or B) = P (A) + P (B) − P (A and B)
• So P (A and Ā) = P (A or Ā) − P (A) − P (Ā) = 0
• P (A and B) = 0 if A and B mutually exclusive
• P (A or B . . . or Z) = P (A) + P (B) + · · · P (Z) if mutually exclusive.
• Plus real world knowledge
12
Coins/Dice/Cards
• Coin Toss: Define H = Heads, T = Tails = H̄
• H and T = φ; H or T = Ω
• 1 = P (H or T ) = P (H) + P (T ) since mutually exclusive.
• Real world knowledge: Symmetry ⇒ P (H) = P (T ) ⇒ P (H) = P (T ) = 1/2
• Similar for one dice, define 6= throws 6, compute P (6) and P (6̄)
• Similar for one card, define Q= draws queen, compute P (Q) and P (Q̄)
13
Applying Probability Rules
• Event Identity A = (A and B) or (A and B̄)
• P (A) = P (A and B) + P (A and B̄) since disjoint.
• Similarly P (B) = P (A and B) + P (Ā and B) since disjoint.
• Event Identity (A or B) = (A and B̄) or (Ā and B) or (A and B)
• Thus P (A or B) = P (A and B̄) + P (Ā and B) + P (A and B)
• Hence P (A or B) = P (A) + P (B) − P (A and B) (Generalizarion of
addition rule)
14
Example
• Define A = Team A at least joint winner, similarly B and C.
• Given symmetry P (A) = P (B) = P (C)
• P (A) =?
• P (A or B) =?
• P (A or B or C) =?
15
Event Identities: Password
• Elementary events and associated probabilities
• P (Dup) via addition rules.
16
Conditional Probability
• P (A) requires Real World Knowledge, short hand for P (A|RW K)
• P (A|B) read probability of A given B, now RWK includes fact B is true,
P (A|B, RW K)
• Conditional Simulation:
– Sequence of simulations
– simulation rules at each stage are influenced by random outcomes of
previous stages
17
Probability Rules for Conditional
Probability and Independence
• P (A and B) = P (A|B)P (B)
• Equivalently, P (A|B) = P (A and B)/P (B)
• Book uses AB to denote A and B
• Important special case P (A and B) = P (A)P (B) when statistically
independent.
• Example Dice rolls: Define 6i = 6 on i-th roll, what is P (61 and 62 )?
• Example Cards: Define Qi = Queen on i-th roll, what is P (Q1 and Q2 )
with and without replacement?
18
Decomposing with Conditional
Probabilities
• Probability second card is a Queen denoted P (Q2 )
• Event Identity: Q2 = First card is anything and Q2 = ((Q1 or Q̄1 )and Q2 )
• So Q2 = ((Q̄1 and Q2 ) or (Q1 and Q2 ))
• So:
P (Q2 )
=
P ((Q̄1 and Q2 ) or (Q1 and Q2 ))
=
P (Q̄1 and Q2 ) + P (Q1 and Q2 )
=
P (Q2 |Q̄1 )P (Q̄1 ) + P (Q2 |Q1 )P (Q1 )
4 48
3 4
4
+
=
51 52 51 52
52
=
Exercise P (Q3 )?
19
(1)
Event Identities: Password
• Elementary events and associated probabilities
• P (N ot Dup) via product rules.
20
Applying Conditional Probability Rules
• Define Q2 Queen on second card, similar Q1
• Seek P (Q1 |Q2 ) given regular deck
• P (A|B) = P (A and B)/P (B)
21
Applying Conditional Probability Rules
• Mini-League: Define A if A outright winner, similar B and C
• Given symmetry P (A) = P (B) = P (C) = 1/4
• P (A|one team is outright winner)?
• P (A|team C is outright winner)?
• P (A|team C scored no wins)?
• P (A|no information about outright winner)?
• Write down event identities explicitly.
• Justify use of + or × explicitly.
22
Bayes’ Rule and Thinking Backwards
• Inverting the direction of conditionality.
• P (this evidence|at crime scene) or P (at crime scene|this evidence)
• P (B|A) =
P (A|B)P (B)
P (A)
• Alternatively
P (B|A)
P (B̄|A)
=
=
P (A|B)P (B)
P (A|B̄)P (B̄)+P (A|B)P (B)
P (A|B)
P (A|B̄)
×
P (B)
P (B̄)
• First ratio posterior odds, last is prior odds
• See Tjims chapter 8.2.
23
Bayes’ Rule and Thinking Backwards
• Murder committed either X or unknown Y .
• In absence of information P (X) = P (Y ) = 1/2
• Evidence E: Blood group A at crime scene.
• X has blood A so P (E|X) = 1
• Y blood group not known, but know P (E) = P (E|Y ) = 1/10
• P (X|E) =
P (E|X)P (X)
P (E)
• P (E) = P (E|X)P (X) + P (E|Y )P (Y )
• Exercise: Use Odds Rule Format
24
Probabilities Distributions and Random
Variables
• Main use of Probability
• Output of a simulation exercise (thought experiment)
• Columns define random variables Y
– Discrete: countable list of possible values
– Continuous values
– True/False values: random variable is an ‘event’
• Discrete random variable fully described by 2 lists: possible values y of Y ,
and associated probabilities P (Y = y)
25
Applying Probability Rules:
Independent Case
• Dice: Define M = maximum score on two independent rolls
• Seek probability distribution of M
• Two lists: Possible (sample space) and probabilities
• Define elementary events; use event identity and probability rules.
26
Applying Probability Rules:
Independent Case
• Mini-League: Define NB = number of wins by B
• A twice as good as B and C; B and C evenly matched.
• Games independent (simulate using 3 random numbers)
• Seek probability distribution of NB
• Two lists: possible sample space for NB and probabilities.
• Similar for joint distribution of NA and NB .
27
Applying Probability Rules:
Independent Case
• Games are independent, scores are not!
• If the events NA = 0 and NB = 0 are independent, then:
P (NA = 0, NB = 0)(= 0) = P (NA = 0)P (NB = 0)(= 0.112 × 0.333)
• Two random variables are independent if
• P (NA = nA and NB = nB ) = P (NA = na )P (NB = nB ) for every possible
pair (nA , nB ).
28
Conditional Distributions
• Exercise: Mini-League with A more skilled.
• What is probability distribution for NA ?
• Know NC = 0 what is probability distribution for NA ?
• What is E[NA |NC = 0], E[NA |NC = 1], Cov[NA , NB |NC = 2]?
• NOTE: Probabilities must sum to 1!
29
Probability and Random Variables
• Random Variable Y :
– Univariate of multivariate
– Name
– (Discrete) Probability Distribution: List of possible values Y , list of
probability for events Y = y (sample space must sum to 1).
• Events:
– Composite events
– AND/OR combinations of elementary events
• Probabilities:
– Satisfy rules (multiplicative/additive)
– Conditioned on ‘knowledge’
30