Mr. Brooks
Conditional Probability
1
To relate the conditional probability expressions below to a more intuitive
mental image, let’s invoke the example of a conference containing both men (M)
and women (W), with some subgroup of them who are Italian (I).
Number in the conference: 1,000
Number of women: 600; number of men: 400
Number of Italians: 200
Number of Italian men: 150; number of Italian women: 50
We will use the symbol: “¬” to indicate “not”, so women (W) are not men (¬M),
and “¬I” means “not Italians”.
So, in probability parlance:
P(M) = 400/1000 = 2/5
P(W) = P(¬M) = 1 – P(M) = 3/5
P(I) = 200/1000 = 1/5
Prob. of Italian men:
P(M,I) = 150/1000 = 3/20
Prob. of Italian women:
P(¬M,I) = 50/1000 = 1/20
Prob. of non-Italian men:
P(M, ¬I) = 250/1000 = ¼
Prob. of non-Italian women:
P(¬M, ¬I) = 550/1000 = 11/20
The sum of these 4 probabilities = 1 (because each person fits into exactly one of
these groups).
Rules
Assume entities (like people) with potential properties X and Y. For instance X
may be being bald, and Y may be the ability to wiggle the ears – there’s
presumably no relation between the two properties.
P(X, ¬Y) is, then, the probability of being bald, and at the same time, not being
able to wiggle the ears.
Definition: P(X|Y) is the probability of having property X, given (assuming)
that you already have property Y.
Example: For instance, in the conference above: P(I|M) is the probability that
you are Italian given that we know that you are a man. In other words, among
only the men, it is the probability that you are Italian. Since there are 200
Italians, and of them, 150 are men, P(I|M) = 150/200 = ¾.
Note: This should not be confused with P(I,M) which is the probability of picking
a person from the entire conference (not just the men) with both properties,
manitude and Italianity, and that is 150/1000 = 3/20.
Mr. Brooks
1
2
P(X) + P(¬X) = 1
P(Y|X) + P(¬Y|X) = 1
3
4
P(X,Y) = P(Y,X)
P(X,Y) = P(X|Y)*P(Y) = P(Y|X)*P(X)
5
P(X) = P(X,Y) + P(X, ¬Y)
Conditional Probability
P(X) = P(X|Y)*P(Y) + P(X|¬Y)*P(¬Y)
6
7
𝑃(𝑌|𝑋) ∗ 𝑃(𝑋)
𝑃(𝑌)
𝑃(𝑌|𝑋) ∗ 𝑃(𝑋)
𝑃(𝑋|𝑌) =
𝑃(𝑌|𝑋) ∗ 𝑃(𝑋) + 𝑃(𝑌|¬𝑋) ∗ 𝑃(¬𝑋)
𝑃(𝑋|𝑌) =
2
You are either bald or not.
Among the bald, the prob. that a random person can
wiggle plus the prob. that she cannot, must be 100%
This follows from the definition of P(X,Y)
The prob. of being bald and wiggling, is
a) the prob. of wiggling, P(Y) times the prob. of being
bald given that you can wiggle P(X|Y).
b) the prob. of being bald, P(X), times the prob. of
wiggling given that you’re bald P(Y|X).
The prob. of being bald is equal to the prob. of being
[bald and able to wiggle] plus [bald and not being able
to wiggle]
The second expression is an expansion of the first,
using Rule #4.
This is Bayes Theorem simply derived from Rule #4
This is also Bayes Theorem, with the denominator
expanded using Rule #5
Mr. Brooks
Conditional Probability
The Aids Example:
There is a very good test, with a 90% accuracy:
a) If you have Aids, there’s a 90% probability that the test will come back
positive.
b) If you don’t have Aids, there’s a 90% probability that the test will come back
negative.
P(+|A) = 0.9
P(-|¬A) = 0.9
P(A) = 0.01
->
->
P(-|A) = 0.1 (false negative)
P(+|¬A) = 0.1 (false positive)
(1% of the population has Aids)
You are interested in P(A|+), which is the probability that if your test is
positive, then you actually have Aids.
Using Rule #4:
𝑃(𝐴|+) ∗ 𝑃(+) = 𝑃(+|𝐴) ∗ 𝑃(𝐴)
Dividing by P(+) or Rule #6 or Bayes Theorem:
𝑃(𝐴|+) =
𝑃(+|𝐴) ∗ 𝑃(𝐴)
𝑃(+)
Using Rule #7:
𝑃(𝐴|+) =
Filling in actual values:
𝑃(𝐴|+) =
𝑃(+|𝐴) ∗ 𝑃(𝐴)
𝑃(+|𝐴) ∗ 𝑃(𝐴) + 𝑃(+|¬𝐴) ∗ 𝑃(¬𝐴)
0.9 ∗ 0.01
0.009
1
=
=
≈ 8%
0.9 ∗ 0.01 + 0.1 ∗ 0.99 0.108 12
So, your initial estimate of getting Aids is 90%, but the real value is about 8%,
over 10 times smaller.
3