Optimization of Pearl’s Method of
Conditioning and Greedy-Like
Approximation Algorithm for the Vertex
Feedback Set Problem
Authors: Ann Becker and Dan Geiger
Presented by: Igor Kviatkovsky
Outline
•
•
•
•
•
Introduction
The Loop Cutset (LC) problem
Weighted Vertex Feedback Set (WVFS) problem
Reduction from LC to WVFS
MGA (Modified Greedy Algorithm)
– 2-approximation for WVFS
Introduction
• Pearl’s method of conditioning is one of the known
inference methods for Bayesian networks
• Find a set of vertices such that once the corresponding
variables are instantiated, the remaining network is
singly–connected
– Pearl’s UPDATE-TREE procedure can be applied
A
a
B
C
D
E
I
J
K
L
M
Fixing value of A & B
b
a
b
C
D
E
I
J
K
L
M
Introduction
A
B
C
D
E
I
J
K
L
M
• Fixing value of A & B & L
breaks all loops. But can we
choose less variables to
break all loops?
• Are there better variables to
choose than others?
• Motivation(Geiger&Becker):
– choose the vertices which
break more loops
– vertices with higher degree are
more likely to break more
loops (vertice J with degree 3
breaks 3 loops!)
The Loop Cutset Problem
• Definitions:
• The underlying graph G of a directed graph D
is the undirected graph formed by ignoring
the directions of the edges in D.
• A loop in D is a subgraph of D whose
underlying graph is a cycle.
• A vertex v is a sink with respect to a loop Γ if
the two edges adjacent to v in Γ are directed
into v .
• Every loop must contain at least one vertex
that isn’t a sink with respect to that loop.
sink
The Loop Cutset Problem
• Each vertex that isn’t a sink with respect to a
loop Γ is called an allowed vertex with
respect to Γ.
• A loop cutset of a directed graph D is a set of
vertices that contain at least one allowed
vertex with respect to each loop in D.
• A minimum loop cutset of a weighted directed
graph D is the one for which the weight is
minimum.
The Loop Cutset Problem
A
• Example
B
C
D
E
I
J
K
L
M
– L is a sink with respect to the
loop ACILJDA
– L is an allowed vertex with
respect to the loop JLMJ
– {A,B,L}, {C,J} are valid loop
cutsets of the graph
– Suppose equal size of
variables’ domains (=r)
• The number of instances
associated with {C,J} is r2
• The number of instances
associated with {A,B,L} is r3
• {C,J} is of lower weight than
{A,B,L}
Weighted Vertex Feedback Set
(WVFS)
• Let G(V,E) be an undirected graph.
• Let w: V R+ be a weight function on the vertices of
G.
• A vertex feedback set of G is a subset of vertices F
in V such that each cycle in G passes through at
least one vertex in F.
• A weight of a set of vertices X is w(X)=ΣvЄX w(v).
• A minimum vertex feedback set of weighted graph G
is vertex feedback set F* with the minimum weight.
• WVFS problem is finding minimum vertex feedback
set of a given weighted graph G having a weight
function w.
Reduction from LC to WVFS
• Given a weighted directed graph (D,w), splitting
weighted undirected graph Ds with a weight
function ws is constructed
– Split each vertex v in D into two vertices vin and vout in
Ds , connect vin and vout .
– All incoming edges to v become undirected incident
edges with vin .
– All outcoming edges from v become undirected
incident edges with vout .
– Set ws(vin)=∞ and ws(vout)=w(v).
V
Γ2
V
C2
W(v)
Vin
Vout
∞
Γ1
∞
W(v)
W(v)
Vin
Vout
C1
Algorithm LC
•
•
Ψ(X) is a set obtained by replacing each vertex
vin or vout in X by the respective source vertex v
in D
Algorithm LC
– Input: A Bayesian network D.
– Output: A loop cutset of D.
1. Construct the splitting graph Ds with weight function
ws .
2. Apply MGA on (Ds , ws) to obtain a vertex feedback
set F.
3. Output Ψ(F).
•
One-to-one and onto correspondence between
loops in D and cycles in Ds
–
MGA 2-approximation for WVFS yields LC 2approximation!
Algorithm GA (Greedy
Algorithm)
•
•
Input: A weighted undirected graph G(V,E,w)
Output: A vertex feedback set F.
– F
Ø; i 0;
– Repeatedly remove all vertices with degree 0 or
1 from V and their adjacent edges from E and
insert the resulting graph into G1 .
– While Gi isn’t the empty graph, do:
1.
2.
3.
4.
5.
Pick a vertex vi for which w(vi)/d(vi) is minimum in Gi
F
F U {vi}
V
V \{vi}
i
i+1
Repeatedly remove all vertices with degree 0 or 1 from
V and their adjacent edges from E and insert the
resulting graph into Gi .
Algorithm MGA (Modified GA)
–
–
–
F ' Ø; i 0;
Repeatedly remove all vertices with degree 0 or
1 from V and their adjacent edges from E and
insert the resulting graph into G1 .
While Gi isn’t the empty graph, do:
1.
2.
3.
4.
5.
Pick a vertex vi for which w(vi)/d(vi) is minimum in Gi
F’ F’ U {vi}
V
V \{vi}
i
i+1
Repeatedly remove all vertices with degree 0 or 1 from
V and their adjacent edges from E and insert the
resulting graph into Gi . For every edge e=(u1,u2)
removed in this process do:
– C(e) w(vi)/d(vi);
– w(u1) w(u1) – C(e);
– w(u2) w(u2) – C(e);
MGA (Phase 2)
• Phase 2
–F
F‘
– For i = |F ’| to 1 do
• If every cycle in G that intersects with {vi} also
intersects with F \{vi} then,
–F
F \{vi}
• After applying phase 2 on the vertex feedback set, all
redundant vertices are removed and we get a minimal
vertex feedback set
• Before applying phase 2
– 4-approximation
• After applying phase 2
– 2-approximation
MGA - Motivation
• In each iteration remove all the isolated nodes
and all the chains
– Isolated nodes and chains don’t break singlyconnectedness
• Pick a node with the minimum cost w(vi)/d(vi) to
vertex feedback set
– Small w(vi) means, it’s worthy taking node vi into the
vertex feedback set since it has a low weight
– Large d(vi) means, it’s worthy taking node vi into the
vertex feedback set since it might break a high
number of loops (has a high degree), even if it’s
weight w(vi) is large
MGA - Motivation
• Subtract the effective cost (cost paid per edge)
from the neighboring nodes weights.
• Example:
w2=2.5
w1=3
w1=1
c=0.5
w1=10
c=1
v2
v3
c=5
C(e)=0.5
G1
v1
c=2.5/2
w2=10
v2
v3
G2
v1
v5
v4
c=5
w1=10
F ’=Φ
c=10/3
w2=10
v5
w2=9.5
F ‘={v1}
c=9.5/2
C(e)=1.25
C(e)=0.5
v1
w1=10
C(e)=0.5
c=5
v4
c=5
C(e)=0.5
v2
C(e)=1.25
v5
v3
G3
C(e)=1.25
C(e)=1.25
v4
5
F ‘={v1,v2}
F=F
’
Performance and Summary
• Theoretical approximation ratio of MGA is 2
(the worst case)
• Average approximation ratio of MGA on
randomly generated graphs is 1.22
• Before running the computations of
conditioning method, the optimal complexity
of its running time can be computed with
high precision using MGA
Extra slides – MGA Analysis and
approximation ratio proof
MGA - Analysis
• F* - a minimum vertex feedback set of G(V,E,w).
• Vertices in F ‘ are {v1,v2,…,vt} when vi are indexed in
the order they inserted into F ’.
• wi(v) and di(v) are weight and degree respectively of
vertex v in Gi .
• Vi is a set of vertices of Gi .
• Let Fi  F  Vi and Fi*  F *  Vi
• Theorem 6 (proved later):
 d (v)  2  d (v)
vFi
i
i
vFi*
MGA - Analysis
• Let Γ1(v) be the set of edges in G1 for which at least one
endpoint is v.
• From the description of the algorithm for every vertex v
in G1 ,  C (e)  w(v) , if vЄF  C (e)  w(v)
e1 ( v )
e1 ( v )
Theorem 3
w( F )  2  w( F * )
• Theorem 3: Algorithm MGA always outputs a
vertex feedback set F whose weight is no
more than twice the weight of a minimum
vertex feedback set F*.
• Proof:
– Let i=j+1, then:
w j (v j )
w j (vi )
wi (vi )  c j
wi (vi )
cj 


 cj 
 ci
d j (v j ) d j (vi ) di (vi )  1
di (vi )
Theorem 3 (Proof)
w( F )  2  w( F * )
– By grouping edges according to iterations they
are assigned a weight:
i 1

e1 ( vi )
C (e) ci  di (vi )   c j  (d j (vi )  d j 1 (vi ))
j 1
– Let
i  1 if vi  F and i  0 if vi  F

C (e)  w(vi )
t
i 1


w( F )  i  w(vi ) i   C (e) i  ci  di (vi )   c j  (d j (vi )  d j 1 (vi )) 
i 1
i 1
e1 ( vi )
i 1
j 1


e1 ( vi )
t
t
t
t
t
i 1
i 2
j i
w( F )  c1 i  d1 (vi )   (ci  ci 1 ) j  di (v j )
t
– By definition:

j i
j
 d i ( v j )   d i (v )
vFi
Theorem 3 (Proof)
t
w( F )  c1  d1 (v )   ( ci  ci 1 )  d i ( v ) 
vF1
i 2
Theorem 6:
w( F )  2  w( F * )
 d (v)  2  d (v)
vFi
vFi
i
i
vFi*
t
 2c1  d1 ( v )   2( ci  ci 1 )  d i ( v ) 
i 2
vF1*
vFi*
t 1
t
  2ci  d i ( v )   2ci
i 1
t
  2ci
i 1
i 1
vFi*

vFi* \ Fi*1
t
d i (v )   2ci
i 1
d
i 1
(v ) 
vFi*1

vFi*1
t 1
d i (v )   2ci
i 1

d i 1 (v ) 
vFi*1
Grouping edges by
iterations they are
assigned a weight
 t 1 


 2    ci  d i ( v ) ci  (d i (v ) d i 1 (v ))   ct  d t (v )  


vFi*1
vFt*
 i 1  vFi* \ Fi*1


 2

vF * e1 ( v )
C (e)  2  w(v)  2  w( F * ).
vF *
Theorem 6
• Definitions:
– Let F  V \ F and F *  V \ F *
– Let dX(v) be the number of edges whose one
endpoint is v and the other is a vertex in X.
v
dX(v)=3
X
• Theorem 6: Let G be a weighted graph for which
every vertex has a degree strictly greater than 1,
F be a minimal vertex feedback set of G and F* be
an arbitrary vertex feedback set of G (possibly a
minimum weight one), then:
 d (v )  2  d (v )
vF
vF *
 d (v )  2  d (v )
Theorem 6 (Proof)
vF
vF *
• To prove the theorem l.h.s. is divided to 2 terms and
an upper bound for each term is provided.
 d (v)  2 F   (d (v)  2)
vF
vF
• Lemma 7: Let G,F and F* be defined as above. Then,
2 F   d (v )  2 F  F *  2 F  F *
vF
• Proof:
≥0
– For every set of vertices B in G
 d (v )  2 F  F
vF
*


vF  B
d (v ) 
 d (v )  2 F  F
 B  2 (F  F *) \ B
vF \ B
– Since d(v) ≥ 2 for each v in G
 d (v )  2 F \ B  2 ( F  F * ) \ B
vF \ B
*
F
F*
B
2 F   d (v )  2 F  F *  2 F  F *
vF
Theorem 6 (Lemma 7)
– Since dB(v) ≤ d(v),
2F 

vF  B

vF  B

d B ( v )  2 F  F *  B  2 F  F *  B   d (v )  2 F  F *  2 F  F *
vF
d B (v )  2 F  F *  B  2 F  F *  B  2 F  F *  B  2 F  F *  B 

vF  B

 

d B (v )  2 F  F *  B  F  F *  B  2 F  F *  B  F  F *  B 
  d B (v)  2 F  B  2 F *  B   (d B (v)  2)  2 F *  B
vF  B
vF  B
– We have to prove that the following hold for some set of vertices
B
2F 

vF  B
(d B (v)  2)  2 F *  B
F
F*
B
2F 

vF  B
(d B (v)  2)  2 F *  B
Lemma 7
– Let’s define a set B for which this inequality can be
proven.
– F is minimal
• Each vertex in F can be associated with a cycle in G that
contains no other vertices of F.
• We define a graph H that consists of the union of these
cycles (one cycle per each vertex).
• Definition: A linkpoint is a vertex with degree 2.
A branchpoint is a vertex with degree larger than 2.
linkpoint
branchpoint
• Every vertex in F is a linkpoint in H.
• Let B the vertices of H.
F
F*
B
2F 

vF  B
Lemma 7
(d B (v)  2)  2 F *  B
– The proof is constructive
• We apply the following procedure on H and showing that
there are terms on the r.h.s. that contribute 2 for each vєF ,
and weren’t used for other vєF .
– Pick a vertex vєF and follow the two paths p1 and p2 in H
from it until the first branchpoint on each path is found.
– There are 3 cases to consider:
• Case 1:
– From definition of H:
– dB(b1)-2 ≥ 1, dB(b2)-2 ≥ 1 b1 , b2  F
– There are terms to contribute 2 to r.h.s.
p1
b1
v
p2
b2
2F 
vF  B
Lemma 7
(d B (v)  2)  2 F *  B
p1
• Case 2:
v
b1
p2
– If dB(b1)≥ 4
» dB(b1)-2 ≥ 2
– If dB(b1)= 3
» dB(b1)-2 = 1
» dB(b2)-2 ≥ 1

p3
p1
v
b2
b1
p2
– There are terms to contribute 2 to r.h.s.
p
• Case 3:
v
p
– Isolated cycle
– There exists a vertex in F* , that
resides on no other cycle of H.
» There are maximum F *  B such
cases
– There are terms to contribute 2 to
r.h.s.
1
2
b1
Lemma 7
2F 

vF  B
(d B (v)  2)  2 F *  B
– Remove the paths p1 and p2 from H
obtaining a graph in which still each
vertex in F resides on a cycle that
contains no other vertices of F.
– Continue till F is exhausted.
Lemma 7
• Lemma 8: Let G,F and F* be defined as above.
Then,
 (d (v)  2)  
vF
vF  F *
d F * (v ) 

(d (v)  2) 
vF  F *

vF  F *
(d F * (v)  2)
• Proof: Note that
(2-2)+(3-2)=1
 (d (v)  2)  
vF
vF  F *
=1
d F * (v ) 

vF  F *
F*
F
v2
v3
v5
v4
v1
=1
(d (v)  2) 

vF  F *
=-1
(d F * (v)  2)
 (d (v)  2)  
vF
vF  F
*
d F * (v) 

( d ( v )  2) 
vF  F
*

vF  F
*
Lemma 8
 (d F (v)  2)   (d F (v)  2)  0
• So if we prove,
vF  F
vF  F
then we prove the lemma
• The graph induced by F * is a forest, and since the
number of edges in a forest is smaller than the
number of vertices, the following holds
*
*
*
*
*
d
(
v
)
/
2

F
*
 F
vF *
 (d
vF *

vF  F *
F*
( v )  2) 0
( d F * ( v )  2) 

vF  F *
( d F * ( v )  2)  0.
( d F * ( v )  2)
Theorem 6 (Proof)
 d (v )  2  d (v )
vF
• To complete the proof , we use bounds obtained
from lemmas 7 and 8:
 d (v )  2 F   (d (v )  2) 
vF
vF
  d (v )  2 F  F *  2 F  F * 
vF


( d ( v )  2) 

d (v ) 
vF  F *

vF  F *

d
vF *
F*

vF  F *
vF  F *
(v ) 

vF  F *
d F * (v ) 
( d F * ( v )  2) 
d F * (v ) 

vF  F *
 d (v )  2  d (v ).
vF *

vF *
d F * (v )   d (v ) 
vF
vF *