Data Mining
Winter 2014/15
Lab
Mouna Kacimi
3. Bayesian Networks
1. Conditional Independence
Coughing is a natural reflex that protects your lungs, however if it becomes frequent than it indicates the presence of a disease. There can be many causes for
frequent Coughing including: Lung Cancer and Bronchitis.
1. Build a Bayesian Network that models the above problem by considering
the following random variables:
L: Lung Cancer
B: Bronchites
C: Cough
2. Show that L and B are independent using the topological property Bayesian
networks (i.e., Each node is conditionally independent of its non-descendants
given its parents).
Answer: By definition, L is independent of B given its parents. Since L
does not have any parent then it is independent of B
3. Assume now that Smoking makes people more subject to Bronchitis. Additionally, Family History is one of the main reasons of Lung Cancer. Update
your network assuming that:
S: Smoking
F: Family History
4. Can you show that L and B are independent using the topological property
of the Bayesian Network?
Answer: No, because we can say that L and B are conditionally independent given S and F but we cannot claim their independence.
5. Compute P(L and B). Are L and B still independent?
P
P (L, B) = s,f ∈T rue,F alse P (S = s, F = f, L, B)
P (L, B) =
P
s,f ∈T rue,F alse
P (S = s)P (F = f )P (B|S = s)P (L|F = f )
According to the Bayesian Rule we have:
P (S = s)P (B|S = s) = P (S = s|B)P (B)
and
P (F = f )P (L|F = f ) = P (F = f |L)P (L)
Thus
P (L, B) =
P
s,f ∈T rue,F alse
P (L, B) = P (L)P (B)
P
P (S = s|B)P (B)P (F = f |L)P (L)
s,f ∈T rue,F alse
P (S = s|B)P (F = f |L)
P (L, B) = P (L)P (B)
L and B are independent
6. Use the d-separation Algorithm to show that L and B are independent.
What do you observe?
Answer: Considering E = {}, L and B are independent (third case of the
algorithm) We observe that d-separation gives more information than the
topological property of the network
7. Are L and B are conditionally independent given C? Use the d-separation
Algorithm to prove that.
Answer: L and B are not conditionally independent because we have E =
{C} and C has 2 incoming arrows. So, the path is not blocked which means
that L can somehow explain B.
8. Assume now that Smoking can also lead to Lung Cancer which can be diagnosed doing an X ray. If the X ray is positive then, the person has Lung
Cancer. Update your network. We assume:
X: Positive X ray
9. Given evidence E, which node pairs are conditionally independent?
(a) E = {}
(b) E = {S}
(c) E = {L}
(d) E = {L, B}
2. Inference
P(A)=0.4 A B P(C|B)=0.2 P(C|~B)=0.3 P(B|A)=0.8 P(B|~A)=0.5 D C P(D|B)=0.8 P(D|~B)=0.5 Given the network above, calculate marginal and conditional probabilities P (C).
1. Apply the method of inference by enumeration (Blind Approach)
note: the network contains probabilities of the form P (X|Y ), so we can deduce any probability of complementary events, i.e. P (X|Y ) = 1 − P (X|Y )
Computation of P (C)
P
P
P
P (C) = A0 ∈A,A B 0 ∈B,B D0 ∈D,D P (A0 , B 0 , C, D0 )
P (C) =
P
A0 ∈A,A
P
B 0 ∈B,B [P (A
0
, B 0 , C, D) + P (A0 , B 0 , C, D)]
P
P (C) = A0 ∈A,A [P (A0 , B, C, D) + P (A0 , B, C, D) + P (A0 , B, C, D) +
P (A0 , B, C, D)]
P (C) = P (A, B, C, D)+P (A, B, C, D)+P (A, B, C, D)+P (A, B, C, D)+
P (A, B, C, D) + P (A, B, C, D) + P (A, B, C, D) + P (A, B, C, D)
We have:
P (A, B, C, D) = P (A)P (B|A)P (C|B)P (D|B)
.
= 0.4 × 0.8 × 0.8 × 0.8 = 0.2048
P (A, B, C, D) = P (A)P (B|A)P (C|B)P (D|B)
.
= 0.4 × 0.8 × 0.8 × 0.2 = 0.0512
P (A, B, C, D) = P (A)P (B|A)P (C|B)P (D|B)
.
= 0.4 × 0.2 × 0.7 × 0.5 = 0.028
P (A, B, C, D) = P (A)P (B|A)P (C|B)P (D|B)
.
= 0.4 × 0.2 × 0.7 × 0.5 = 0.028
P (A, B, C, D) = P (A)P (B|A)P (C|B)P (D|B)
.
= 0.6 × 0.5 × 0.8 × 0.8 = 0.192
P (A, B, C, D) = P (A)P (B|A)P (C|B)P (D|B)
.
= 0.6 × 0.5 × 0.8 × 0.2 = 0.048
P (A, B, C, D) = P (A)P (B|A)P (C|B)P (D|B)
.
= 0.6 × 0.5 × 0.7 × 0.5 = 0.105
P (A, B, C, D) = P (A)P (B|A)P (C|B)P (D|B)
.
= 0.6 × 0.5 × 0.7 × 0.5 = 0.105
P (C) = 0.2048 + 0.0512 + 0.028 + 0.028 + 0.192 + 0.048 + 0.105 + 0.105
P (C) = 0.762
2. Apply the method of variable elimination
P (C) =
P
A0 ∈A,A
P
B 0 ∈B,B
P
D0 ∈D,D
P (A0 , B 0 , C, D0 )
P (A0 )P (B 0 |A0 )P (C|B 0 )P (D0 |B 0 )
P (C) =
P
P
P
P (C) =
P
P
P (A0 )P (B 0 |A0 )P (C|B 0 )
A0 ∈A,A
A0 ∈A,A
B 0 ∈B,B
B 0 ∈B,B
D0 ∈D,D
P
D0 ∈D,D
P (D0 |B 0 )
At this point, we eliminate variable D since the sum over D, D is equal
to 1 (probability of an event and its negation sums up to 1). As a result we
have:
P (C) =
P
A0 ∈A,A
P
B 0 ∈B,B
P (A0 )P (B 0 |A0 )P (C|B 0 )
We observe that P (C|B 0 ) is a constant with respect to
it should be taken out as follows:
P (C) =
P
B 0 ∈B,B
[
P
A0 ∈A,A
P
A0 ∈A,A
and thus
P (A0 )P (B 0 |A0 )]P (C|B 0 )
P
We let fA (B 0 ) = A0 ∈A,A P (A0 )P (B 0 |A0 ), then we compute the value of
fA (B 0 ) for each value of B 0 :
For B 0 = B we have fA (B) = P (A)P (B|A) + P (A)P (B|A)
= 0.4 × 0.8 × 0.6 × 0.5 = 0.62
.
For B 0 = B we have fA (B) = P (A)P (B|A) + P (A)P (B|A)
= 0.4 × 0.2 × 0.6 × 0.5 = 0.38
.
At this point, we eliminate variable A since we summed up all its values.
Thus:
P (C) =
P
B 0 ∈B,B
fA (B 0 )P (C|B 0 )
P (C) = fA (B)P (C|B) + fA (B)P (C|B)
P (C) = 0.62 × 0.8 + 0.38 × 0.7 = 0.762