Tutorial 8
Optimal Clique Trees
A rundown of the paper - Optimal Junction Trees by Finn V. Jensen and Frank Jensen
Tal Shor
Basic definitions
• Junction tree or Clique graph/tree – is a graph where each node is a
maximal clique in a main graph.
• It maintains the so-called junction tree property – for each pair of
maximal cliques 𝑈, 𝑉 with intersection 𝑆, all cliques on paths between
them, contain 𝑆.
• We give each link in the junction tree a label called separator, and it’s
value is exactly 𝑆
Minimal spanning tree - recap
For a graph G with a weight function on the links, find a subset of links,
creating the minimal weighted tree.
Prim’s algorithm - 𝑆 = 𝑁 . Pick the minimal weighted link (𝑈,𝑉) such
that 𝑈 ∈ 𝑆 and 𝑉 ∉ 𝑆, add 𝑉 to 𝑆.
Kruskal’s algorithm – Choose successively a link of minimal weight not
producing a cycle.
Maximal spanning tree
Student: If minimal spanning tree is P, it must be NP-Hard, no?
Tutor: It’s basically the same as MST – just pick the edge with the
maximal weight instead of the minimal.
The proof’s logic (Visually)
Max spanning tree of
1
2
3
=
Min spanning tree of
-1
-2
-3
=
Min spanning tree of
10000-1
10000-3
10000-2
Theorems
Theorem 1
A Spanning tree for the junction graph of G is a junction tree
if and only if it is a spanning tree of maximal weight.
Theorem 2
Any minimal cost junction tree can be constructed by
successively choosing a link of maximal weight not introducing cycles,
and if several links may be chosen then a link of minimal cost is
selected.
Junction tree algorithm example
V
S
S,L,B
L
B
T
S,L,B
A,B
L
Σ𝐷 ⇒
A
A,T,L
A
B
A,B,D
A
A
A,L,B
⇐ Σ𝐵
Σ𝑇 ⇒
A,T,L
⇐ Σ 𝑇,𝐿
T
Σ𝑋 ⇒
D
X,A
Σ𝑆 ⇒
⇐ Σ𝐴
⇐ Σ𝐿
A,L
A,B,D
X
A,L,B
L,B
V,T
X,A
Σ𝐴,𝐿 ⇒
⇐ Σ𝑉
V,T
Theorem 1 proof
Maximal spanning tree ⇒ Junction tree
• 𝑇 - spanning tree of maximal weight where 𝑇1 ⊆ 𝑇2 … ⊆ 𝑇𝑛 are Prim’s subtrees. We assume that 𝑇 is not a junction tree
• 𝑚 - a stage such that 𝑇𝑚 can be extended to a junction tree (𝑇′),
yet T𝑚+1 can’t
• 𝑈, 𝑉 - the edge added in step m such that 𝑉 ∈ 𝑇𝑚+1
with the label 𝑆 ⇒ 𝑈, 𝑉 ∉ 𝑇′
Theorem 1 proof (2)
• 𝑇′ must contain a path between U and V such that in it, there is link
𝑈 ′ , 𝑉 ′ where 𝑈 ′ ∈ 𝑇𝑚 , 𝑉 ′ ∉ 𝑇𝑚 with 𝑆 ′ as it’s label.
• 𝑆 ⊆ 𝑆′ and 𝑆 ≥ |𝑆 ′ | ⇒ 𝑆 = 𝑆′
• Remove (𝑈 ′ , 𝑉 ′ ) from 𝑇′ and you get a junction tree that extends
𝑇𝑚+1 contradicting the assumption
Theorem 1 proof (3)
¬Maximal spanning tree ⇒ ¬Junction tree
• 𝑇 is a non maximal spanning tree while 𝑇1 , … , 𝑇𝑛 = 𝑇 ′ are as before.
• (𝑈, 𝑉) the first node where 𝑇′ and 𝑇 (have to) diverge. It is labeled 𝑆.
Find the (𝑈 ′ , 𝑉 ′ ) link again (𝑆′)
• By definition of MST - 𝑆 ′ < |𝑆| ⇒ 𝑆 > 𝑆 ′ ≥ |𝑈 ∩ 𝑉| therefore 𝑇
is not a junction tree
Optimal Junction trees
• Using the junction tree propagation (passing the marginalized factor of the
clique) - we pass messages throughout our junction tree.
• We do so in order to get the marginal probability of the joint probability
distribution for every variable
• This message passing can be organized so that it is sufficient to pass a
single message in each direction of every edge.
• We can associate a local measure 𝐶(𝑈, 𝑉) to the links (𝑈, 𝑉) where this
cost indicates the time/space consumption of the two passes.
Find an optimal junction tree
• A junction graph might have several junction tree.
• We now find a junction tree that minimizes some 𝐶(𝑇) = ∑𝐶(𝑈, 𝑉)
• Unlike before – we focus on Kruskal’s algorithm. Let 𝑤1 , … , 𝑤𝑛 be the
unique, sorted (decreasing) weights of G such that on stage 𝑖 all
possible links of weights 𝑤1 , … , 𝑤𝑖 have been chosen.
Find an optimal junction tree (2)
• Stage 𝑖 + 1 can be viewed as such: we add all the links of weight 𝑤𝑖+1
and break all the cycles by removing links of that weight.
Find an optimal junction tree (3)
• Any thinning will result in a forest of partial maximal spanning trees.
And every thinning, at every stage, results in the same connected
components therefore the thinning has no impact on the next stage.
• Therefore we can max a secondary priority (cost) so that we end up
with a maximal weight spanning tree of minimal cost.
Theorems
Theorem 1
A Spanning tree for the junction graph of G is a junction tree
if and only if it is a spanning tree of maximal weight.
Theorem 2
Any minimal cost junction tree can be constructed by
successively choosing a link of maximal weight not introducing cycles,
and if several links may be chosen then a link of minimal cost is
selected.
Corollary
All junction trees over the same triangulated graph have the
same separators (also counting multiplicity)
Proof
When we remove a link (𝑈, 𝑉) with label 𝑆 of
weight 𝑤𝑖+1 in the thinning process, then all separators in the
remaining paths between U and V must contain S.
U
𝑤𝑖+1
𝑤𝑖+1
Since they weigh the same – all the separators are exactly 𝑆.
V
Reminder - Theorem 7
• Definition: The following are equivalent:
1. G is chordal
2. The edges of G can be directed acyclically so that every pair of
converging arrows comes from two adjacent vertices.
3. G has a perfect vertex elimination scheme.
4. There is a tree T with maximal cliques of G as vertices. Any two
cliques containing v are either adjacent in T or connected by a path
of cliques that contain v.
16