Sharif University of Technology
Probabilistic Graphical Models (CE-956)
Dr. Soleymani
Homework 2: Exact inference (100 + 5 Pts)
Theory: 24 Esfand 24:00
Programming: 14 Farvardin 24:00
1: Variable elimination (20 Pts)
In this problem we aim to run variable elimination on figure 1.
(a). (3Pts) Draw the equivalence MRF for this BN.
(b). (9 Pts) What is the order of elimination if we use the below heuristics (you can refer to Section
9.4.3.2 of the Koller’s book to study these heuristics):
• Min-neighbors
• Min-weight
• Min-fill
If there are multiple choices, eliminate the variable with the lowest index. Suppose the domain
cardinality of X1 , X3 , ..., X13 is k and that of X2 , X4 , ..., X14 is 2k
(c). (8 Pts) Which of the heuristics has the best complexity and which one has the worst complexity?
(explain about it and determine the required number of operations in each case)
2: Inference (15 Pts)
(a). (5 Pts) Consider a directed graphical model that is a poly-tree. What is the best elimination
order to find a junction tree on this graph? What are the separator sets in this junction tree? Is
there a relation between the factor graph of the poly-tree and this resulted junction tree?
(b). (10 Pts) Consider the graphical model in Figure 2:
1. Explain which of the exact inference algorithms is proper to find marginal distribution on
each of the variables?
2. When all of the Zi s are observed, explain which of the exact inference algorithms is preferred
to find the marginal distribution on each of the remaining variables?
3. When all of the Xi s are observed, explain which algorithm is preferred to find the marginal
distribution on each of the remaining variables?
1
2
Figure 1: Question 1
Figure 2: Question 2.b
4. When all of the Yi s are observed, can we find the marginal distribution on each of the remaining variables using the sum-product algorithm on factor trees? Explain it.
3: Factor graph and Belief propagation - (30 Pts)
Draw the factor graph of the model in Figure 3, and determine the factors. Suppose the variables
are 3-valued (0, 1, 2) and the filled variables are observed, (X6 = 1 and X9 = 2). Each factor is
defined based on its entries as bellow, (Sort the entries based on their indexes)
f (v1 ) = 1
f (v1 , v2 , v3 ) = 2v1 + 3v2 ∗ v3 + 1.4
f (v1 , v2 , v3 , v4 ) = 2v1 + 2v2 ∗ v3 ∗ v4 + 2.1
f (v1 , v2 , v3 , v4 , v5 ) = 3v1 + v2 ∗ v3 ∗ v4 + 2v5 + 1.3
1. (15 Pts)Implement sum-product message passing on this graph to find the marginal distribution on each of the remaining variables.
3
Figure 3: Question 3 part 2
2. (11 Pts) Implement max-product message passing on this graph which computes max-marginal
on these variables and determine the Most Probable Configuration (MPC) of them.
3. (4 Pts) Can we use the most probable value of each random variable using the obtained
marginal distributions on these variables obtained in part (a) to find an MPC?
4: Junction Tree (40 Pts)
Consider Figure 4:
(a). (2 Pts) Draw the moralized graph.
(b). (5 Pts) Consider the elimination order Xt,2 , Xt,3 , ..., Xt,N −1 , Xt,N , Xt,1 for t = 1...T . Determine
the fill-in edges and the maximal cliques created in the process of elimination.
(c). (15 Pts) According to table 2-8, implement the Hugin algorithm (in Matlab) for N = 6 and
T = 10 to find marginal distributions on the maximal cliques.
(d). (8 Pts) Implement a function named CheckConsistancy for checking local consistency, this
function can use to show that the implementation is correct.
(e). (10 Pts) Using the UGM toolbox, implement the Junction tree and compare the obtained
results with your results in Part (c). (you can download the UGM at ’’https://www.cs.ubc.ca/
~schmidtm/Software/UGM.html’’
T able2
x1,1
1 2 3
P (x1,1 ) 14 12 14
4
T able3 : P (xt,1 |xt−1,1 ); t ∈ [2, T ]
xt,1 |xt−1,1 1 2
3
1
2
3
1
6
6
6
2
2
1
2
6
6
6
3
2
2
3
6
6
6
T able4 : P (xt,2 |xt,1 ); t ∈ [1, T ]
xt,2 |xt,1 1 2
3
1
2
3
1
5
5
5
1
2
1
2
5
5
5
3
1
1
3
5
5
5
T able5 : P (x1,N |x1,N −1 )
x1,N |x1,N −1 1 2 3
3
1
3
1
7
7
7
2
5
1
2
7
7
7
2
1
3
3
7
7
7
T able6 : P (xT,i |xT,i−1 ); i ∈ [3, N − 1]
xT,i |xT,i−1 1 2
3
2
1
1
1
5
7
6
2
4
3
2
5
7
6
1
2
2
3
5
7
6
T able7 : P (xt,i |xt+1,i−1 , xt,i−1 ); t ∈ [1, T − 1], i ∈ [3, N − 1]
xt,i |xt+1,i−1 , xt,i−1 1,1 1,2 1,3 2,1 2,2 2,3 3,1 3,2 3,3
2
1
1
1
1
3
2
4
2
1
5
5
5
5
5
5
5
6
6
2
3
2
3
1
1
1
1
3
2
5
5
5
5
5
5
5
6
6
1
1
2
1
3
1
2
1
1
3
5
5
5
5
5
5
5
6
6
T able8 : P (xt,N |xt,N −1 , xt−1,N ); t ∈ [2, T ]
xt,N |xt,N −1 , xt−1,N 1,1 1,2 1,3 2,1 2,2 2,3 3,1
2
1
1
1
2
2
1
1
7
6
5
6
6
5
5
2
3
3
2
1
1
1
2
7
6
5
6
6
5
5
3
2
1
3
3
2
3
3
7
6
5
6
6
5
5
3,2
3,3
4
7
2
7
1
7
2
5
2
5
1
5
5
Figure 4: Question 4