Topic 1 – Probability
Module 1.2 – Probability Theory
Total Probability Rule & Bayes Rule
Suppose we have events X1, X2, …, Xk where Xi∩Xj = ∅ for i ≠ j, and an event Y where
Y ⊆ X1∪X2∪…∪Xk where ⊆ denotes subset. (As an example, visualize the X’s as the
states of the US and Y as a region of the US, e.g., the South West).
TOTAL PROBABILITY RULE:
If Y ⊆ X1∪X2∪…∪XK and Xi∩Xj = ∅ for i ≠ j,
then P(Y) = P(Y∩X1) + P(Y∩X2) + … + P(Y∩XK).
By the general multiplication rule, applied k times, we have
If Y ⊆ X1∪X2∪…∪XK and Xi∩Xj = ∅ for i ≠ j,
then P(Y) = P(Y/X1)P(X1) + P(Y/X2)P(X2) + … + P(Y/XK)P(XK).
BAYES RULE (SIMPLE FORM):
If Y ⊆ X1∪X2∪…∪XK and Xi∩Xj = ∅ for i ≠ j,
P(X i )P(Y /X i )
then P(X i /Y) =
P(Y)
By the total probability rule, we have:
€
BAYES RULE (EXTENDED FORM):
If Y ⊆ X1∪X2∪…∪XK and Xi∩Xj = ∅ for i ≠ j,
P(X i )P(Y /X i )
then P(X i /Y) = K
∑ P(X j )P(Y /X j )
j=1
€
Bayes rule is the rule by which we revise probabilities; it will play a crucial role in
decision analysis and the economic theory of information.
For example, suppose we have
P(X1) = .2
P(X2) = .3
P(X3) = .5
Xi∩Xj = ∅ for i ≠ j; i, j = 1, 2, 3
Y ⊆ X1∪X2∪X3
P(Y/X1) = .4
P(Y/X2) = .6
P(Y/X3) = .8
Then, by the total probability rule, we have
P(Y) = P(Y/X1)P(X1) + P(Y/X2)P(X2) + P(Y/X3)P(X3) = (.4)(.2)+(.6)(.3)+(.8)(.5) = .66.
€
By Bayes rule, we have
P(X1 )P(Y /X1 ) (.2)(.4) .08
P(X1 /Y) =
=
=
P(Y)
.66
.66
P(X 2 /Y) =
P(X 2 )P(Y /X 2 ) (.3)(.6) .18
=
=
P(Y)
.66
.66
P(X 3 /Y) =
P(X 3 )P(Y /X 3 ) (.5)(.8) .40
=
=
P(Y)
.66
.66
€
€
Note that the denominator, .66, is simply the sum of the numerators, .08, .18, and .40.
€
PROBLEM:
Suppose we have
P(X1) = .2
P(X2) = .3
P(X3) = .5
Xi∩Xj = ∅ for i ≠ j; i, j = 1, 2, 3
Y ⊆ X1∪X2∪X3
P(Y/X1) = .1
P(Y/X2) = .5
P(Y/X3) = .9
Find P(Y), P(X1/Y), P(X2/Y), and P(X3/Y)