Homework 2
1
40-957 Special Topics in Artificial Intelligence:
Probabilistic Graphical Models
Homework 2
Due data: 1393/1/17
Directions: To submit your assignment, make a zip file containing a pdf
file of your writeup (write it by hand and then scan it) and your code,
then submit it to "[email protected]" by email with the subject
"HW2 Submission-student number" (for example: "HW2 Submission90207018")
Late submission policy: late submission carries a penalty of 10% off
your assignment per day after the deadline.
1 Problem 1 (15 Pts)
Consider the undirected graphical model of figure 1 (This 3 × 3 grid is a small snapshot of
the graph which are often used in image processing and computer vision) and the elimination
algorithm for inference on this model:
1.1 (5 Pts) Find the set of factors after each elimination step and plot the graph
obtained in each step when the chosen elimination order is 5, 4, 8, 6, 2, 9, 3,
7, 1.
1.2 (5 Pts) Answer the above question when the elimination order is 1, 7, 3, 9, 2,
4, 6, 8, 5.
1.3 (5 Pts) When each variable takes K values, find the number of entries in
the largest intermediate factor for both of the above orders. Is there another
elimination ordering that needs lower computational complexity?
2 Problem 2 (25 Pts)
In this problem, you are going to predict the price of an asset in the future! The file
PriceData.mat holds the price of an asset through T = 100 time points. The value of the
price is an integer between 1 and 100. We know the following scenario about the asset:
The price of the asset changes from time t to t + 1 with probability transition matrix
pbull(t, t + 1) if the market is in state ”bull” and with probability transition matrix
2
Figure 1 Graphical model of problem 1.
pbear(t, t + 1) if the market is in state "bear". If the market is in a "bull" state it will remain
in that state with probability 0.7 and if the market is in a "bear" state it will remain in that
state with probability 0.8. We also know that at time point 1, the probability of the market
being to "bull" or "bear" state is uniformly distributed. Moreover, the distribution of the
price of the asset at time point 1 is uniform.
2.1 (5 Pts) Draw the direceted grapgical representation of the problem using the
Plate notation.
2.2 (10 Pts) Based on your model and using the Blief Propagation algorithm
on this special graph, calculate the propability of the price of the asset at
time T + 1 given all the observed
( prices from time 1 to T). Mathematically
speaking, you must compute P price(T + 1)|price(1 : T ) .
2.3
(10 Pts) Based on the price of the asset through T time provided in the file
test_price_data.mat, find the most probable market states for this sequences
of prices using the max-product algorithm on this special graph.
3 Problem 3 (25 Pts)
In this problem, you are going to implement the sum-product algorithm for the ALARM
data set [1] (see figure 2). This data set has been designed for hospital patient monitoring
and consists of 37 discrete random variables taking 2 to 4 states indexed by positive integers.
For your convenience, the MATLAB code has been provided that you can use to load the
Bayesian network.
Homework 2
3
Figure 2 ALARM Bayesian Network [1].
3.1 (15 Pts) Implement the Blief Propagation algorithm for this graph. Iterate the
message updates (one iteration updates all messages once) until the absolute
difference between the old message and the new message is less than 10−5 .
Does the algorithm fail to converge after 500 iterations? If so, explain the
reason for that.
3.2 (10 Pts) Consider a sub-graph of the ALARM graph containing the
variables VENTTUBE, PULMEMBOLUS, KINKEDTUBE, INTUBATION,
VENTLUNG, PAP, SHUNT, PRESS. To construct a Bayes Net (BN) based
on this sub-graph, assume we observe the variables DISCONNECT, and
VENTMACH with values 2 and 4 respectively. Construct the BN by
conditioning on the observed variables and marginalizing all other variables.
Now, compute the means of the two variables PULMEMBOLUS, and
VENTTUBE by conditioning on observing PRESS = 4 and SHUNT = 2 using
the sum-product algorithm. Compare your answer to the means obtained by
running the sum-product on the whole graph (part 3.1).
4 Problem 4 (20 Pts)
Consider the MRF of figure 3.
4
Figure 3 Graphical models of problem 4.
4.1
(5 Pts) Based on the elimination order 1, 10, 5, 2, 6, 3, 9, 8, 4, 7, find the
corresponding triangulated graph.
4.2 (10 Pts) Since finding the best elimination order (or the best triangulation)
is NP-hard, we can use heuristics to find an elimination order (See Section
9.4.3.2 of Koller and Friedman). Consider the min-fill heuristic: greedily
choose to eliminate next the node that results in the smallest fill-in (i.e.,
the smallest number of edges that need to be added to the graph due to its
elimination). Based on the min-fill heuristic, find the triangulated graph and
the corresponding junction tree.
4.3 (5 Pts) Consider a directed graphical model that is a poly-tree. What is the best
elimination order to find a junction tree on this graph? What are the separator
sets in this junction tree? Is there a relation between the factor graph of the
poly-tree and this resulted junction tree?
5 Problem 5 (15 Pts)
5.1 (6 Pts) Consider the junction tree algorithm with the Shafer-Shenoy updates.
Show that it is equivalent to the sum-product algorithm on clique trees
described in Section 10.2 of the text book.
5.2 (3 Pts) What is the meaning of a message in this algorithm?
5.3 (6Pts) In a clique tree, consider an arbitrary clique Ci in this tree and its
parent Cj (the upward neighborclique). Let βi be the potential at the clique Ci
after the first pass of the sum-product message passing algorithm. Show that
βi is equivalent to the original unnormalized conditional measure P̃ϕ (Ci −
Sij |Sij ) =
P̃ϕ (Ci )
P̃ϕ (Sij )
(ϕ shows the set of the original potentials).
References
[1] I.A. Beinlich, H.J. Suermondt, R.M. Chavez, and G.F. Cooper. The alarm monitoring
system: a case study with two probabilistic inference techniques for belief networks.
Homework 2
5
Technical Report KSL-88-84, Stanford University. Computer Science Dept., Knowledge
Systems Laboratory, 1989.