AND/OR Search for
• Mixed Networks
• #CSP
Robert Mateescu
ICS280 Spring 2004 - Current Topics in Graphical Models
Professor Rina Dechter
Mixed Networks
Belief Networks
Constraint Networks
A
A
F
B
E
C
D
F
B
E
C
D
Variables : A, B, C , D, E , F
Variables : A, B, C , D, E , F
Domains : DA  DB  DC  DD  DE  DF  {0,1}
Domains : DA  DB  DC  DD  DE  DF  {0,1}
CPTS : P( A), P( B | A), P(C | A), P( D | B, C )
Relations : R ( ABC ), R ( AFC ), P ( BCD ), R ( A, E )
P( E | A, B), P( F | A)
Mixed Networks
A
A
F
B
E
F
C
B
A
D
C
E
D
F
B
E
C
D


PB x
P
x
|
x



, if x  

PΜ x   B
PB x  
0, otherwise



Variables : A, B, C , D, E , F
Domains : DA  DB  DC  DD  DE  DF  {0,1}
CPTs : P( A), P( B | A), P(C | A), P( D | B, C )
P( E | A, B), P( F | A)
Relations : R( ABC ), R( AFC ), P( BCD ), R( A, E )



Tasks for Mixed Networks
• Tasks for Belief Networks:
– Belief updating: evaluating the posterior probability of each
singleton proposition given some evidence;
– Most probable explanation (MPE): finding a complete assignment
to all variables having maximum probability given the evidence;
– Maximum a posteriori hypothesis (MAP): finding the most likely
assignment to a subset of hypothesis variables given the evidence;
• Tasks for Constraint Networks:
– Consistent: Decide if network is consistent
– Find solutions: Find one, some or all solutions
• Tasks for Mixed Networks:
– Belief updating, MPE, MAP
– Constraint Probability Evaluation (CPE): Find the probability of
the constraint query
Auxiliary Network
Variables : A, B, C , D, E , F
Variables : A, B, C , D, E , F
Domains : DA  DB  DC  DD  DE  DF  {0,1}
Domains : DA  DB  DC  DD  DE  DF  {0,1}
CPTS : P( A), P( B | A), P(C | A), P( D | B, C )
Relations : R ( ABC ), R ( AFC ), P ( BCD ), R ( A, E )
P( E | A, B), P( F | A)
A
A
F
B
F
C
B
A
X
E
F
D
Q
C
B
E
C
D
Y
Z
1, if t  R
P( X aux | t S )  
0, otherwise
E
D
Mixed Graph
• The mixed graph is the union of the belief network
graph and the constraint network graph
• The moral mixed graph is the union of the moral
graph of the belief network and the graph of the
constraint network
• Given a mixed graph GM = (GB,GR) of a mixed
network M(B,R) where GB is the directed graph of
B, and GR is the undirected constraint graph of R,
the ancestral graph of Y \in X in GM is the union of
GR and the ancestral graph of Y in GB
dm-separation
• Definition: Given a mixed graph, GM and given
three subsets of variables W, Y and Z which are
disjoint, we say that W and Y are dm-separated
given Z in the mixed graph GM, denoted <W,Z,Y>dm,
iff in the ancestral mixed graph of WUYUZ, all the
paths between W and Y are intercepted by variables
in Z.
• Theorem: Given a mixed network M = M(B,R) and its
mixed graph GM, then GM is a minimal I-map
relative to dm-separation
AND/OR Search for Mixed Networks
A
A
F
B
B
C
E
E
C
D
D
Mixed network
F
DFS (legal) tree
AND/OR Search Space
A
B
OR
A
E
AND
0
C
1
D
OR
B
AND
AND
0
C
1
E
0
1
D
0
F
1
0
1
E
OR
AND
B
0
OR
0
0
D
1
0
0
0
1
0
F
1
0
1
D
0
D
1
1
E
C
1
F
1
F
0
1
0
1
0
E
0
1
D
F
1
C
F
1
0
0
D
1
0
1
F
1
0
C
0
1
D
1
0
F
1
0
D
1
0
F
1
0
1
Forward: Expanding an AND node
A
B
OR
A
E
AND
0
C
1
D
OR
B
AND
AND
0
C
1
E
0
1
D
0
F
1
0
1
E
OR
AND
B
0
OR
0
0
D
1
0
0
0
1
0
F
1
0
1
D
0
D
1
1
E
C
1
F
1
F
0
1
0
1
0
E
0
1
D
F
1
C
F
1
0
0
D
1
0
1
F
1
0
C
0
1
D
1
0
F
1
0
g-value(ANDnode) = ∏ Pi (or 1, if product is empty)
e.g. g-value(<B,1>) = P(B=1|A=0)
D
1
0
F
1
0
1
Forward: Expanding an OR node
A
B
OR
A
E
AND
0
C
1
D
OR
B
AND
AND
0
C
1
E
0
1
D
0
F
1
0
1
E
OR
AND
B
0
OR
0
0
D
1
0
0
0
1
0
F
1
0
1
D
0
D
1
1
E
C
1
F
1
F
0
1
0
1
0
E
0
1
D
F
1
C
F
1
0
g-value(ORnode) = 0
0
D
1
0
1
F
1
0
C
0
1
D
1
0
F
1
0
D
1
0
F
1
0
1
Backward: Propagating g-values
A
B
OR
A
E
AND
0
C
1
D
OR
B
AND
AND
0
C
1
E
0
1
D
0
F
1
0
1
E
OR
AND
B
0
OR
0
0
D
1
0
0
0
1
0
F
1
0
1
D
0
D
1
1
E
C
1
F
1
F
0
1
0
1
0
E
0
1
D
F
1
C
F
1
0
0
D
1
0
OR nodes perform summation
AND nodes perform product
1
F
1
0
C
0
1
D
1
0
F
1
0
D
1
0
F
1
0
1
The main idea
• Search algorithms can exploit a wide range of
constraint propagation techniques
• When generating AND nodes, consistency is
checked according to the desired level of
constraint propagation
AND/OR Search (1)
A
F
OR
AND
A
g(A)=0
B
E
C
D
OR
A
AND
B
OR
AND
OR
AND
E
C
D
F
AND/OR Search (2)
A
F
OR
AND
A
0
g(<A,0>)=P(A=0)
B
g(A)=0
1
E
C
D
g(<A,1>)=P(A=1)
OR
A
AND
B
OR
AND
OR
AND
E
C
D
F
AND/OR Search (3)
A
F
OR
AND
A
0
g(<A,0>)=P(A=0)
B
g(A)=0
1
E
C
D
g(<A,1>)=P(A=1)
OR
B
g(B)=0
A
AND
B
OR
AND
OR
AND
E
C
D
F
AND/OR Search (4)
A
F
OR
A
AND
0
B
g(A)=0
g(<A,0>)=P(A=0)
1
E
C
D
g(<A,1>)=P(A=1)
OR
B
g(B)=0
A
AND
0
g(<B,0>)=P(B=0|A=0)
1
g(<B,1>)=P(B=1|A=0)
B
OR
AND
OR
AND
E
C
D
F
AND/OR Search (5)
A
F
OR
A
AND
0
B
g(A)=0
g(<A,0>)=P(A=0)
1
E
C
D
g(<A,1>)=P(A=1)
OR
B
g(B)=0
A
AND
0
g(<B,0>)=P(B=0|A=0)
1
g(<B,1>)=P(B=1|A=0)
B
OR
AND
OR
AND
E
g(E)=0
C
g(C)=0
E
C
D
F
AND/OR Search (6)
A
F
OR
A
AND
0
B
g(A)=0
g(<A,0>)=P(A=0)
1
E
C
D
g(<A,1>)=P(A=1)
OR
B
g(B)=0
A
AND
0
g(<B,0>)=P(B=0|A=0)
1
g(<B,1>)=P(B=1|A=0)
B
OR
AND
E
g(E)=0
C
g(C)=0
0
g(<E,0>)=P(E=0|A=0,B=0)
OR
AND
E
C
D
F
AND/OR Search (6)
A
F
OR
A
AND
0
B
g(A)=0
g(<A,0>)=P(A=0)
1
E
C
D
g(<A,1>)=P(A=1)
OR
B
g(B)=0
A
AND
0
g(<B,0>)=P(B=0|A=0)
1
g(<B,1>)=P(B=1|A=0)
B
OR
E
C
g(C)=0
E
C
g(E)=P(E=0|A=0,B=0)
AND
OR
AND
0
D
F
AND/OR Search (8)
A
F
OR
A
AND
0
B
g(A)=0
g(<A,0>)=P(A=0)
1
E
C
D
g(<A,1>)=P(A=1)
OR
B
g(B)=0
A
AND
OR
AND
OR
AND
g(<B,1>)=P(B=1|A=0)
g(<B,0>)= P(B=0|A=0) * P(E=0|A=0,B=0)
0
E
0
1
C
g(C)=0
B
E
C
D
F
Constraint propagation
A
B
C
C
>
B
A
F
I
K
D
E
G
H
G
H
All domains are {1,2,3,4}
I
F
>
E
>
D
K
Constraint checking only
A
B
A
C
>
OR
2
3
4
B
B
D
E
>
1
OR
B
2
AND
3
D
OR
AND
3
OR
G
H
AND
4
4
C
E
D
4
D
E
B
C
2
3
4
3
4
3
4
4
F
F
F
D
D
F
F
D
4
3
4
4
3
4
4
4
4
G
I
I
G
K
K
K
G
K
4
4
G
H
I
F
>
AND
K
Forward checking
A
B
A
C
>
OR
2
3
4
B
B
D
E
>
1
OR
B
2
AND
3
D
OR
AND
3
OR
G
H
AND
4
4
C
E
D
4
D
E
B
C
2
3
4
3
4
3
4
4
F
F
F
D
D
F
F
D
4
3
4
4
3
4
4
4
4
G
I
I
G
K
K
K
G
K
4
4
G
H
I
F
>
AND
K
Maintaining arc-consistency
A
B
A
C
>
OR
2
3
4
B
B
D
E
>
1
OR
B
2
AND
3
D
OR
AND
3
OR
G
H
AND
4
4
C
E
D
4
D
E
B
C
2
3
4
3
4
3
4
4
F
F
F
D
D
F
F
D
4
3
4
4
3
4
4
4
4
G
I
I
G
K
K
K
G
K
4
4
G
H
I
F
>
AND
K
AND/OR vs. OR Space
Linear space algorithms
Linear space algorithms
AND/OR search vs. Bucket Elimination
#CSP
• #CSP – Counting the solutions of a CSP problem
is very similar to the CPE task in mixed networks
• If the belief part in a mixed network is empty, we
can translate the AND/OR search for mixed
networks to an AND/OR search for CSPs.
• OR nodes are initialized with g-value 0
• AND nodes are initialized with g-value 1
Parent set and parent separator set
Parent set
Parent
separator set
A
A
B
B
A
AB
C
AB
ABC
D
BC
D
E
AB
E
F
AC
F
E
C
D
F
Caching in AND/OR search
• If space is available, parts of the search tree can be
cached, transforming the search space into a
search graph
• In principle, we can cache at OR level and/or at
AND level
• Caching at AND level => use parent separator set
• Caching at OR level => use parent set
size(parent separator set) ≤ size(parent set) + 1
ORcache, N20, K3, Nc20 ,Pc4, inst20
Time
10%
20%
30%
40%
50%
60%
70%
AUXILIARY BE
0.10110
0.10155
0.10115
0.10025
0.10000
0.08970
0.08805
i=0
AND/OR FC
AND/OR RFC
OR FC
OR RFC
0.00650
0.00350
0.00505
0.00400
0.01250
0.01005
0.01200
0.01255
0.02450
0.02555
0.02755
0.02800
0.06555
0.07660
0.08670
0.09870
0.22940
0.27490
0.52620
0.56040
1.09355
1.33295
5.49720
5.72635
5.81740
6.94850
65.68775
67.94275
i=3
AND/OR FC
AND/OR RFC
OR FC
OR RFC
0.00550
0.00300
0.00555
0.00400
0.01210
0.01305
0.01250
0.01000
0.02555
0.02550
0.02750
0.02810
0.06410
0.07810
0.08765
0.09820
0.22925
0.27850
0.52405
0.56400
1.09505
1.33705
5.48500
5.72880
5.79485
6.90190
65.83190
67.98520
i=6
AND/OR FC
AND/OR RFC
OR FC
OR RFC
0.00500
0.00500
0.00450
0.00300
0.01250
0.01100
0.01250
0.01050
0.02405
0.02750
0.02960
0.03200
0.06455
0.07555
0.08860
0.09805
0.21370
0.25930
0.49920
0.53625
0.91375
1.09625
4.66985
4.87520
4.33875
5.08375
49.77530
51.24910
i=9
AND/OR FC
AND/OR RFC
OR FC
OR RFC
0.00455
0.00450
0.00550
0.00450
0.01155
0.00950
0.01355
0.01150
0.02500
0.02600
0.02950
0.03020
0.06405
0.07310
0.08160
0.09415
0.17240
0.20530
0.40010
0.43620
0.48865
0.58830
2.98980
3.15515
1.22135
1.46265
23.39555
24.25300
ORcache, N20, K3, Nc20 ,Pc4, inst20
# of nodes
10%
20%
30%
40%
50%
60%
70%
i=0
AND/OR FC
AND/OR RFC
OR FC
OR RFC
225
154
225
154
518
387
519
387
1192
1052
1203
1062
3552
3407
3810
3664
16003
15737
28079
27801
106651
106617
414463
414428
735153
735153
6533674
6533674
i=3
AND/OR FC
AND/OR RFC
OR FC
OR RFC
225
154
225
154
518
387
519
387
1192
1052
1203
1062
3552
3407
3810
3664
16003
15737
28079
27801
106651
106617
414463
414428
735153
735153
6533674
6533674
i=6
AND/OR FC
AND/OR RFC
OR FC
OR RFC
224
154
225
154
512
384
519
387
1162
1028
1203
1062
3306
3175
3764
3618
12765
12562
24700
24422
70273
70238
294525
294491
436554
436554
3931078
3931078
i=9
AND/OR FC
AND/OR RFC
OR FC
OR RFC
224
153
225
154
499
371
518
387
1093
962
1192
1052
2883
2761
3604
3461
8873
8705
18729
18457
28038
28003
166912
166877
79946
79946
1516976
1516976