HW3: Probability and Bayes Nets Due 04/26/2012 Please indicate the CEC ID of all group members clearly on the submission. Only one submission per group is required. Group size is not limited. Problem 1: Probability Attach events to the binary random variables X, Y, and Z that are consistent with the following patterns of common-­‐sense reasoning. You may use different events for the different parts of the problem. Answers will vary. Examples below. (a) Accumulating evidence: P(X=1) < P(X=1|Y =1) < P(X=1|Y =1,Z=1) X = “it is raining”, Y = “the forecast predicts rain”, Z = “the sky is cloudy” (b) Explaining away: P(X=1|Y =1) > P(X=1), P(X=1|Y =1,Z=1) < P(X=1|Y =1) X = “sinus congestion”, Y = “flu”, Z = “allergies” (c) Conditional independence: P(X=1,Y =1) ≠ P(X=1)P(Y =1) P(X=1,Y =1|Z=1) = P(X=1|Z=1)P(Y =1|Z=1) X = “baseball game is cancelled”, Y = “outdoor picnic is cancelled”, Z = “it is raining” (g)
P (C|A, B, D, E, F, H)
=
P (C)
(h)
P (A|B, C, D, F, G, H, I)
=
P (A)
(i)
P (D, E|I)
=
P (D|I) P (E|I)
=
P (A, B|E) P (C, D|G)
Problem 2: Conditional Independence P (A, B, C, D|E, G)
(j)
For the belief network shown below, indicate whether the following statements of conditional independence are true (T) or false (F). A
B
E
C
F
H
D
G
I
(a) P(A|F) = P(A) True a. A ind. F. True because there is no valid chain from A to F. (b) P(B|C,I) = P(B|I) False a. B cond. ind. of C given I. False because 3 amigos chain B-­‐F-­‐I-­‐G-­‐C are all valid triples. 3
(c) P(F,G) = P(F) P(G) True a. F ind. of G. True because no valid chain from F to G. (d) P(E,F,G) = P(E) P(F) P(G) False a. E, F, G all independent. False because E,F are dependent (chain through B. (e) P(C,D) = P(C) P(D) True a. C independent of D. True because CDG is not a valid 3 amigos triple (because you do not observe G). (f) P(C,D|I) = P(C|I) P(D|I) False a. C independent of D given I. False because the observation of I, downstream from G, makes CGD a valid 3amigos triple. (g) P(C|A,B,D,E,F,H) = P(C) True a. C independent of A,B,D,E,F,H. There is no valid chain to any of these. (h) P(A|B,C,D,F,G,H,I) = P(A) False a. A independent of B,C,D,F,G,H,I. False because A-­‐E-­‐H is an active triple. (i) P(D,E|I) = P(D|I) P(E|I) False a. D independent of E given I. False. The long chain: D-­‐G-­‐I-­‐F-­‐B-­‐E are all active 3amigos triples, so there is a dependence between D and E. (j) P(A,B,C,D|E,G) = P(A,B|E) P(C,D|G) True a. (A,B) independent of (C,D) given E and G. True, because, given E,G, There is no valid chain from any of the group A,B to any of the group C,D. Independence of “groups” of nodes like this will not be on the final. Problem 3: Markov Blankets For the graphical model shown in Problem 2, indicate the Markov blanket for the following variables. (a) A {B,E} (b) B {A,E,F} (c) D {C,G} (d) E {A,BF,H} (e) G {C,D,F,I} (f) I {F,G} Problem 4: Variable Elimination and Likelihood Weighting Your friendly neighborhood TA, like most other TAs, is intent on world domination. His first step, obviously, was to build a graphical model, as shown in the figure below. The variables are: Graduate (G), Free Food (FF), The Force (TF), Knowledge (K), Money (M), Power (R) and World Domination (WD). All the variables are binary valued {T,F}. The conditional probability tables in the figure denote P(A = T | B=T …) and P(A = T | B=F …). (a) How likely is the TA to take over the world, if he manages to graduate? Compute P(WD = T|G = T) by variable elimination. (Report the ordering used and the factors produced after eliminating each variable for the query.) Eliminate ¬G from M M P(M|G=T) T F 0.6 0.4 Join WD and R WD M R P(WD=T,R|M) = P(WD|M,R)*P(R|M) T T T 0.7*0.7 = 0.49 T T F 0.5*0.3 = 0.15 T F T 0.6*0.1 = 0.06 T F F 0.005*0.9 = 0.0045 Marginalize out R WD M P(WD=T|M) = ∑R P(WD=T,R|M) T T 0.49 + 0.15 = 0.64 T F 0.06 + 0.0045 = 0.0645 Join WD and M WD M P(WD=T,M|G=T) = P(WD=T|M)*P(M|G=T) T T 0.64*0.6 = 0.384 T F 0.0645*0.4 = 0.0258 Marginalize out M WD P(WD=T|G=T) = ∑MP(WD=T,M|G=T) T 0.384 + 0.0258 = 0.4098 P(WD=T|G=T) = 0.4098 (b) We know the TA passes an important exam (K = T) and starts attending lots of lunch seminars around campus (FF = T) but is still a powerless TA (R = F). From the following samples, estimate the probability of imminent World Domination (WD = T) using Likelihood Weighting. The weights represent the likelihood of the evidence (K, FF, R) given the other variables in the sample. TF, ¬G, ¬M, ¬WD W1 = P(K|¬G) P(FF|TF) P(¬R|¬M) = 0.6 x 0.8 x 0.9 = 0.432 ¬TF, ¬G, ¬M, ¬WD w2 = P(K|¬G) P(FF|¬TF) P(¬R|¬M) = 0.6 x 0.6 x 0.9 = 0.324 TF, G, ¬M, WD w3 = P(K|G) P(FF|TF) P(¬R|¬M) = 0.7 x 0.8 x 0.9 = 0.504 ¬TF, ¬G, M, WD w4 = P(K|¬G) P(FF|¬TF) P(¬R|M) = 0.6 x 0.6 x 0.3 = 0.108 ¬TF, G, M, ¬WD w5 = P(K|G) P(FF|¬TF) P(¬R|M) = 0.7 x 0.6 x 0.3 = 0.126 P(WD|K,FF,¬R) = (∑wi,WD=T)/(∑wi) = (0.504+0.108) / (0.432+0.324+0.504+0.108+0.126) = 0.410 Problem 5: HMMs and Particle Filtering You are an interplanetary search and rescue expert who has just received an urgent message: a rover on Mercury has fallen and become trapped in Death Ravine, a deep, narrow gorge on the borders of enemy territory. You zoom over to Mercury to investigate the situation. Death Ravine is a narrow gorge 6 miles long, as shown below. There are volcanic vents at locations A and D, indicated by the triangular symbols at those locations. The rover was heavily damaged in the fall, and as a result, most of its sensors are broken. The only ones still functioning are its thermometers, which register only two levels: hot and cold. The rover sends back evidence E = hot when it is at a volcanic vent (A and D), and E = cold otherwise. There is no chance of a mistaken reading. The rover fell into the gorge at position A on day 1, so X1 = A. Let the rover's position on day t be Xt ∈ {A,B,C,D,E,F}. The rover is still executing its original programming, trying to move 1 mile east (i.e. right, towards F) every day. However, because of the damage, it only moves east with probability 0.5, and it stays in place with probability 0.5. Your job is to figure out where the rover is, so that you can dispatch your rescue-­‐bot. (a) Three days have passed since the rover fell into the ravine. The observations were (E1 = hot , E2 = cold , E3 = cold). What is P (X3 | hot1, cold2, cold3), the probability distribution over the rover's position on day 3, given the observations? X1 A B … P(X1 | E1=hot) 1.0 0.0 0.0 We compute the following value for each location X2 ∈ {A,B,…} and normalize: P(X2|E1=hot,E2=cold) ∝ P(E2=cold | X2) ∑X1 P(X2|X1) P(X1|E1=hot) X2 P(X2 | E1=hot, E2=cold) A 0.0 B 1.0 … 0.0 We compute the following value for each location X3 ∈ {A,B,…} and normalize: P(X3|E1=hot,E2=cold,E3=cold) ∝ P(E3=cold | X3) ∑X2 P(X3|X2) P(X2|E1=hot,E2=cold) X3 P(X3 | E1=hot, E2=cold, E3=cold) A 0.0 B 0.5 C 0.5 … 0.0 You decide to attempt to rescue the rover on day 4. However, the transmission of E4 seems to have been corrupted, and so it is not observed. (b) What is the rover's position distribution for day 4 given the same evidence, P (X4 | hot1, cold2, cold3)? P(X4 | E1=hot, E2=cold, E3=cold) = P(X4 | X3) X4 P (X4 | E1=hot, E2=cold, E3=cold) A 0.0 B 0.25 C 0.5 D 0.25 (c) All this computation is taxing your computers, so the next time this happens you decide to try approximate inference using particle filtering to track the rover. If your particles are initially in the top configuration shown below, what is the probability that they will be in the bottom configuration shown below after one day (after time elapses, but before evidence is observed)? P(new configuration | old configuration) = 0.5 * 0.5 * 0.5 = 0.125