CONDITIONAL PROBABILITY AND BAYES THEOREM
HOT NOTES FOR STATISTICS
Abstract. Define conditional probability and the multiplication rule, and
show how Bayes Theorem works.
1. Conditional Probability
Condition probability, written Pr(A | B) is the probability of event A, given the
knowledge that event B has occurred. For example, in this compilation of Indiana
bicycle accident injuries, where H indicates wearing a helmet and I indicates a
head injury,
head injury
yes
no
total
wearing helmet
yes no total
17 218
235
130 428
558
147 646
793
235
, while the conditional
the marginal probability of a head injury is
793
probability of a head injury, given the fact that the subject was not wearing
218
a protective helmet, is
In the first case, all cyclists are considered (the
646
denominator is the grand total); in the second case only this without helmets are
considered (a column total).
Mathematically, the conditional probability for this event is defined as
Pr(I | ¬H) =
Pr(I ∩ ¬H
Pr(¬H)
2. Multiplication Rule
Pr(I ∩ ¬H) = Pr(¬H) × Pr(I | ¬H)
1
2
HOT NOTES FOR STATISTICS
2.1. Bayes Theorem. Using the multiplication rule and the Law of Total
Probability, the definition of conditional probability can be expanded to give Bayes
Theorem
Pr(B | A) Pr(A)
Pr(A | B) =
Pr(B | A) Pr(A) + Pr(B | ¬A) Pr(¬A)
For example, from the head injury data, we see that
Pr(¬H | I)
=
=
Pr(I | ¬H) Pr(¬H)
Pr(I | ¬H) Pr(¬H) + Pr(I | H) Pr(H)
218/646 × 646/793
218
=
218/646 × 646/793 + 17/147 × 147/793
235
3. A Diagnostic Example
Consider an HIV test with the following sensitivity, specificity, and prevalence:
sensitivity
Pr(+ | HIV )
0.999
specificity Pr(− | ¬HIV )
0.995
prevalance
Pr(HIV )
0.0025
The possible test outcomes are shown in this decision tree
Using Bayes’ Theorem, we can find the posterior predictive value, Pr(HIV | +)
as
PPV
=
=
sensitivity × prevalence
sensitivity × prevalence + (1 − specificity × (1 − prevalence)
0.999 × 0.0025
= 0.3337
0.999 × 0.0025 + 0.005 × 0.0075
Department of Management Science and Statistics, UTSA
E-mail address: [email protected]