From AND/OR Search to AND/OR BDDs Rina Dechter Information and Computer Science, UC-Irvine, and Radcliffe Institue of Advanced Study, Cambridge Joint work with Robert Mateescu and John Cobb Verification/constraints workshop, 2006 Combining Decision Diagrams f = A*E + B*F A B g = C*G +D*H C B D E D E G F G H 0 1 0 1 A B B C D C D E OBDD f * g = F C D E D E F D E E F G G H 0 D 1 E Combining AND/OR BDDs f = A*E + B*F A B g = C*G +D*H C B D E D E G F G H 0 1 0 AND A B B AOBDD f * g = E C D D E G F 0 1 G H 1 0 1 OBDDs vs AOBDD AND A AOBDD B C B E f *g= D D E G F G H 0 1 0 1 A B B OBDD C D C D E D E F C D E F D D E E F G G H 0 1 E Outline Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR graphs to AOMDDs Compilation of AOMDDs AOMDDs and earlier BDDs Constraint Networks A Example: map coloring Variables - countries (A,B,C,etc.) Values - colors (red, green, blue) Constraints: A B, A D, D E, etc. Constraint graph A B red red green green yellow yellow green yellow red yellow green red A E A E D B F C Semantics: set of all solutions Primary task: find a solution D B F G G C Propositional Satisfiability = {(¬C), (A v B v C), (¬A v B v E), (¬B v C v D)}. Graphical models A graphical model (X,D,C): X = {X1,…Xn} variables D = {D1, … Dn} domains C = {F1,…,Ft} functions (constraints, CPTS, cnfs) Fi : P( F | A, C ) Fi : F A C A F B C Examples: •Constraint networks •Belief networks •Cost networks •Markov random fields •Influence diagrams E D All these tasks are NP-hard identify special cases approximate Tree-solving is easy CSP – consistency (projection-join) Belief updating (sum-prod) P(X) mXY (X ) mXZ (X ) mYX (X ) mZX (X ) P(Y|X) P(Z|X) mYT (Y ) mYR (Y ) mZL (Z ) mZM (Z ) mTY (Y ) mRY (Y ) mLZ (Z ) mMZ (Z ) P(T|Y) MPE (max-prod) P(R|Y) P(L|Z) P(M|Z) #CSP (sum-prod) Trees are processed in linear time and memory Transforming into a Tree By Inference (thinking) Transform into a single, equivalent tree of sub-problems By Conditioning (guessing) Transform into many tree-like subproblems. Inference and Treewidth ABC G D A DGF B BDEF F C EFH E M K H L FHK J HJ KLM Inference algorithm: Time: exp(tree-width) Space: exp(tree-width) treewidth = 4 - 1 = 3 treewidth = (maximum cluster size) - 1 Conditioning and Cycle cutset H G D F O E C K M J A O E C P B K L Cycle cutset = {A,B,C} H F B O E P K L C N D F M J H G N D P B L G N D F M A J H G N M J O E C P K L Search over the Cutset E Graph Coloring problem M D • Inference may require too much memory L C A B • Condition (guessing) on some of the variables K H F G J A=yellow E D A=green M E D M L C L C B K B K H F H F G J G J Search over the Cutset (cont) E Graph Coloring problem M D • Inference may require too much memory L C A B • Condition on some of the variables K H F G J A=yellow B=red E D B=blue M E D B=green M E L C E J E E L K K H F J M C H G D L K F J M C H G D B=yellow L K F G M C H F D B=blue L K H J M C K G D B=red L C F A=green H F G J G J Inference vs. Conditioning By Inference (thinking) ABC A C B BDEF F E H J L FHK KLM Variable-elimination Directional resolution Join-tree clustering By Conditioning (guessing) A=yellow E D M L C A B K H F G J Exponential in treewidth Time and memory EFH M K HJ DGF G D A=green B=blue B=green B=red B=blue E D M E D M E D M E D M L C L C L C L C K K K K H H H H F F F F G J G J G J G J Exponential in cycle-cutset Time-wise, linear memory Backtracking Branch and bound Depth-first search Outline Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR graphs to AOMDDs Compilation of AOMDDs AOMDDs and earlier BDDs Classic OR Search Space A F B E A 1 0 E 1 0 C F D 0 B D Ordering: A B E C D F C 1 0 0 1 1 0 0 0 1 0 1 1 0 0 1 0 1 0 1 1 0 0 0 1 0 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 0 0 0 1 0 1 1 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 AND/OR Search Space A A A F B F B C B C C E E D E Primal graph D D F DFS tree OR A AND 0 1 OR B B AND 0 OR AND E 0 OR AND C 1 E 0 1 D 0 F 1 0 1 0 0 D 1 0 F 1 0 0 0 F 1 0 1 D 1 E C 1 0 D 1 1 0 1 0 1 0 E 0 1 D F 1 C F 1 0 0 D 1 0 1 F 1 0 C 0 1 D 1 0 F 1 0 D 1 0 F 1 0 1 A A F B AND/OR vs. OR OR B C E E C D D F A AND 0 1 OR B B AND 0 OR AND 0 C 1 OR 0 E 0 1 D AND F 1 0 1 E 0 0 D 1 0 F 1 0 0 1 0 F 1 0 D 1 0 C 0 1 D 1 E C 1 AND/OR 1 1 0 0 1 D F 0 1 E F 1 0 0 D 1 0 1 F 1 0 C 0 1 D 1 F 0 1 0 D 1 0 F 1 0 1 AND/OR size: exp(4), OR size exp(6) A 0 B 0 E F 1 0 C D 1 1 0 0 1 1 0 0 0 1 0 1 1 0 0 1 0 1 0 1 1 0 0 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 0 0 0 1 OR 1 0 1 1 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 AND/OR vs. OR with Constraints OR No-goods (A=1,B=1) (B=0,C=0) F B E C D D F A 0 1 OR B B AND 0 OR 0 C 1 OR 1 D AND 0 E 0 F 1 0 0 D 1 0 F 1 0 1 A 0 0 F 1 0 D 1 0 C 0 1 D 1 E C 1 AND/OR 0 1 E AND 1 1 0 0 1 D F 0 1 E F 1 0 0 D 1 0 1 0 1 0 C 1 0 0 1 1 0 1 0 1 1 0 1 0 1 1 D 1 F 0 1 0 D 1 0 0 0 0 0 F 1 0 1 1 0 E C 1 F 0 B F B C E AND D A A 0 1 1 0 0 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 0 0 0 1 OR 1 0 1 1 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 AND/OR vs. OR with Constraints OR F B 1 OR B B AND 0 OR E 0 C E 1 1 0 D AND 0 1 0 0 0 F 1 0 1 D 1 E C 1 F A C D F AND/OR 0 D 1 0 C 1 1 D F 1 0 0 1 F 1 0 1 0 B 1 0 0 1 1 C F E 0 1 OR D D B A 0 E C E AND AND A A No-goods (A=1,B=1) (B=0,C=0 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1 1 0 0 0 1 OR 0 0 1 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1 DFS algorithm (#CSP example) A A F B B C solution E E C D D OR F 11 A AND 0 5 1 6 OR B 5 B 6 AND 0 OR AND 0 1 0 D 2 2 F 1 D 1 0 4 1 2 F 0 D 1 F 1 D 1 F 0 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 0 1 1 0 0 1 0 1 0 0 1 0 0 D 0 1 1 F 1 D 2 1 0 1 0 1 0 1 1 1 0 1 1 1 0 Value of node = number of solutions below it 0 1 1 F D 0 1 2 C 1 0 2 2 1 E 1 C 4 E 1 Marginalization C 1 E OR node: operator (summation) 1 0 0 1operator 0 2 1 2 0 1 node: 0 1 1 0 0 AND Combination (product) 1 0 2 C 1 OR AND 1 2 E 1 4 1 1 1 0 1 1 F 0 1 1 0 1 D 0 0 1 1 F 1 0 1 1 1 0 Pseudo-Trees (Freuder 85, Bayardo 95, Bodlaender and Gilbert, 91) 4 1 1 6 2 3 m <= w* log n 2 7 5 (a) Graph 4 1 2 3 7 1 7 3 5 3 5 4 2 7 5 6 6 4 6 (b) DFS tree depth=3 (c) pseudo- tree depth=2 (d) Chain depth=6 Tasks and value of nodes V( n) is the value of the tree T(n) for the task: Consistency: v(n) is 0 if T(n) inconsistent, 1 othewise. Counting: v(n) is number of solutions in T(n) Optimization: v(n) is the optimal solution in T(n) Belief updating: v(n), probability of evidence in T(n). Partition function: v(n) is the total probability in T(n). Goal: compute the value of the root node recursively using dfs AND/OR search tree and algorithms are search of the AND/OR tree. ([Freuder & Quinn85], [Collin, Dechter & Katz91], [Bayardo & Miranker95],[Darwiche 2001], [Bacchus et. Al, 2003]) Space: O(n) Time: O(exp(m)), where m is the depth of the pseudo-tree Time: O(exp(w* log n)) BFS is time and space O(exp(w* log n) F From AND/OR Tree B B C E K C H A AND 0 1 OR B B AND 0 OR AND E D G OR A J A E 0 C 1 OR 0 1 E 0 1 D F 0 D 1 F 0 0 1 D F D C 1 G J H K E 0 1 D F F 1 E C D F C 0 D 1 F 0 1 D F D F AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 OR G G J J G G J J G G J J G G J J G G J J G G J J G G J J G G J J AND 0 1 0 OR H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K AND 01010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 To an AND/OR Graph F B B C E K A 0 1 OR B B AND 0 OR OR C H AND AND E D G OR A J A E 0 C 1 D 0 1 E 0 1 F D 0 F 1 D 0 0 1 F D C 1 D F AND 0 1 0 1 OR G G J J AND 0 1 0 OR H H H H K K K K AND 0101010101010101 1 0 1 0 1 F G J H K 1 E C D E 0 1 F D 0 F C 1 0 D 1 F D F AND/OR Search Graphs Any two nodes that root identical subtrees/subgraphs (are unifiable) can be merged Minimal AND/OR search graph: of R relative to tree T is the closure under merge of the AND/OR search tree of R relative to T, where inconsistent subtrees are pruned. Canonicity: The minimal AND/OR graph relative to T is unique for all constraints equivalent to R. Context based caching Caching is possible when context is the same context = = parent-separator set in induced pseudo-graph current variable + ancestors connected to subtree below A J A F B C E D G H context(B) = {A, B} B K E C context(C) = {A,B,C} D F G J H K context(D) = {D} context(F) = {F} Caching F B AND OR 0 OR 0 E 0 1 D F F G J H K B C 1 D 1 1 E C A 0 OR E H B AND AND K D G context(B) = {A, B} context(C) = {A,B,C} context(D) = {D} 0 context(F) = {F} B C E OR A J A 0 D F E C 1 0 0 1 D 1 F D C 1 0 1 D F E F C 0 D 1 F 0 1 D F D F AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 OR G G J J G G J J G G J J G G J J G G J J G G J J G G J J G G J J AND 0 1 0 OR H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K H H H H K K K K AND 01010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 A J A F Caching B E context(D)={D} K A 0 1 OR B B 0 OR AND OR 0 C 1 D 0 1 E E 0 1 F D 0 F C H AND AND E D G context(F)={F} OR B C 1 D 0 0 1 F D C 1 D F AND 0 1 0 1 OR G G J J AND 0 1 0 OR H H H H K K K K AND 0101010101010101 1 0 1 0 1 F G J H K 1 E C D E 0 1 F D 0 F C 1 0 D 1 F D F C A D F B E All Four Search Spaces A B C D E F 0 0 1 0 0 1 1 0 0 0 1 A 1 0 1 1 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 B 1 0 1 0 0 1 C 1 0 1 0 0 D 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 0 0 1 E 01 01 0 1 0 101 01 0 1 0 1 0 1 0 1 01 010 1 0 1 01 01 0 1 0 1 01 010 1 0 1 01 01 01 01 0 1 0 101 01 0 1 0 1 1 1 0 0 1 0 1 1 0 0 1 1 0 0 F 0 1 1 1 Full OR search tree Context minimal OR search graph 126 nodes 28 nodes OR 1 A AND 0 1 OR B B AND 1 0 C OR C E 1 0 C E C E E AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 OR AND D D F F D D F F D D F F D D F F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 Full AND/OR search tree 54 AND nodes OR AND OR AND OR AND OR AND A 0 1 B B 1 0 C C E 1 0 C E 0 1 0 1 0 D D F F 01 D C E E 1 0 1 D F F 01 0 1 0 1 0 1 0 1 Context minimal AND/OR search graph 18 AND nodes Properties of minimal AND/OR graphs Complexity: Minimal AND/OR R relative to pseudo-tree T is O(exp(w*)) where w* is the tree-width of R along T. Minimal OR search graph is O(exp(pw*)) where pw* is pathwidth w* ≤ pw*, pw* ≤ w*log n Canonicity: The minimal AND/OR search graph is unique (canonical) for all equivalent formulas (Boolean or Constraints), consistent with its pseudo tree. Searching AND/OR Graphs AO(j): searches depth-first, cache i-context j = the max size of a cache table (i.e. number of variables in a context) i=0 i=w* j Space: O(n) Space: O(exp w*) Time: Time: O(exp(w* log n)) Space: O(exp(j) ) Time: O(exp(m_j+j ) O(exp w*) Context-Based Caching A B C E D F context(A) = {A} context(B) = {B,A} context(C) = {C,B} context(D) = {D} context(E) = {E,A} context(F) = {F} Cache Table (C) C Value 0 5 1 2 0 2 1 0 Space: O(exp(2)) A D B F E Primal graph A 5 0 7 1 5 B 7 B 6 0 3 C B 0 0 1 1 C 5 1 4 C E 6 0 E 4 C 4 1 E 5 0 2 1 2 0 0 1 D D D D 4 C E Outline Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR search graphs to AOMDDs Compilation of AOMDDs AOMDDs and earlier BDDs OR Search Graphs vs OBDDs A A 0 0 0 0 1 1 1 1 B 0 0 1 1 0 0 1 1 C 0 1 0 1 0 1 0 1 f(ABC) 0 0 0 1 0 1 0 1 0 A B B 0 0 C 1 0 1 C 1 0 C 1 0 1 Full AND/OR search tree A A 0 1 1 0 C B 1 1 0 redundant 1 1 1 Minimal AND/OR search graph Backtrack free AND/OR search tree 1 B 1 C C C 1 0 1 B B 1 0 C C B C B 1 A 0 1 1 An OBDD AND/OR Search Graphs; AOBDDs A F(A,B,C,D)= (0,1,1,1), (1,0,1,0), (1,1,1,0) B C D AOBDD(F) A A 0 0 1 1 B B redundancy B 0 1 1 1 D C D D C D 1 1 0 1 1 0 AND/OR Search Graphs; AOBDDs A F(A,B,C,D)= (0,1,1,1), (1,0,1,0), (1,1,1,0) B C D AOBDD(F) A A 0 0 1 1 B B redundancy B 0 1 1 1 D C D D C D 1 1 0 1 1 0 AOBDD Conventions A B D C A 0 A 0 Point dead-ends to terminal 0 Point goods to terminal 1 1 1 B 0 1 B C 1 0 C D 1 D 1 0 D 1 0 1 D 1 0 0 1 AOBDD Conventions A B D C Introduce Meta-nodes A A 0 0 1 1 B 0 B 1 1 C 0 C D 1 D 1 0 D 1 0 1 D 1 0 0 1 Combining AOBDD (apply) A 0 0 0 0 1 1 1 1 B 0 0 1 1 0 0 1 1 C 0 1 0 1 0 1 0 1 f(ABC) 0 0 0 1 0 1 0 1 A B C A 0 0 0 0 1 1 1 1 B 0 0 1 1 0 0 1 1 D 0 1 0 1 0 1 0 1 A 0 1 0 * 1 C 0 A B B D D C A 1 0 B 0 A A B 0 g(ABC) 0 0 0 1 1 0 1 0 1 1 0 = 1 D 0 0 D 1 0 B 0 1 C 0 1 1 1 0 D 1 0 D 1 0 1 1 Example: (f+h) * (a+!h) * (a#b#g) * (f+g) * (b+f) * (a+e) * (c+e) * (c#d) * (b+c) A m7 A D m7 B A A 0 0 1 B 0 1 0 C 0 C 1 0 1 0 0 1 C 1 0 0 1 0 1 0 F 1 0 0 F 1 0 B 1 D D G E 1 0 0 1 G 1 0 H 1 0 C H 1 0 0 1 m6 E A H C F D E G pseudo tree F 1 D 0 F primal graph B 0 B 1 F 1 C A B F C 1 1 B B G E G H 1 m3 A 0 B A 0 B 0 C 0 C 0 0 0 1 0 1 1 0 0 1 1 0 0 m5 D m4 C D 1 D E 0 1 C8 D C 1 0 G 1 0 0 1 F 1 0 1 B F 1 0 H G 1 0 1 1 0 A A 0 0 1 H 1 0 F 1 0 1 0 1 C6 C7 0 0 1 C G 0 1 0 1 C3 0 1 F A 1 0 H B H 1 0 1 C4 H m1 0 G E G 1 1 A 1 G 0 G 0 F 0 B 1 m2 1 A 1 B 1 0 C5 E E E 0 F 0 0 1 1 0 0 A 1 C A 1 0 1 F 0 1 D 0 B 1 C 0 0 F A B 1 B 1 E D C9 0 C D 1 A 1 C 1 1 0 H 1 0 1 F 0 1 0 1 G C1 C2 F H H Example (continued) A 0 1 B 0 C 0 B 1 0 C 1 0 C 1 0 C 1 0 F 1 0 1 F 1 0 F 1 0 F 1 0 1 A D 0 D 1 0 E 1 0 G 1 0 0 G 1 0 H 1 0 H 1 0 1 1 m7 m7 A A 0 0 1 B 0 1 0 C 0 C 1 0 1 0 0 1 C 1 0 0 1 0 0 F 1 A B 1 F C 1 1 B B 0 1 F 1 0 F 1 0 B 1 B D 0 D G E 1 0 0 1 G 1 0 H 1 0 C H 1 0 D 0 m6 1 0 m3 1 F 1 E G H AOBDD vs. OBDD D A G C B F E A H B C primal graph A 0 C 0 D 0 0 C 1 D 0 C 1 0 D 1 0 D F 1 0 E 1 F 1 0 F 1 0 F 1 0 D E 1 0 G 1 0 G 1 0 H 1 0 0 1 G F G AOBDD B 18 nonterminals F G D 0 1 47 arcs G A H H C 1 E F OBDD 27 nonterminals G H F G A E D E F D D D H 1 B C C E F 1 G 0 C B 1 C 1 C 1 B 0 B H 54 arcs 0 Complexity Complexity of apply: Complexity of apply is bounded quadratically by the product of input AOBDDs restricted to each branch in the output pseudo-tree. Complexity of VE-AOBDD: is exponential in the tree-width along the pseudo-tree. AOMDDs and tree-BDDs Tree-BDDs (McMillan1994) are : O(| g | 2 w22 w ) AND/OR BDDS are w O(| g | k ) Related work Related work in Search Related to compilation schemes: Backjumping + learning (Freuder 1988,Dechter 1990,Bayardo and Mirankar 1996) Recursive-conditioning (Darwiche 2001) Value elimination (Bacchus et. Al. 2003) Search over tree-decompositions (Trrioux et. Al. 2002) Minimal AND/OR – related to tree-OBDDs (McMillan 94), d-DNNF (Darwiche et. Al. 2002) Case-factor diagrams (Mcallester, Collins, Pereira, 2005) Tutorial references: Dechter, Constraint processing, Morgan Kauffman, 2003 Dechter, Tractable Structures of Constraint Satisfaction problems, In Handbook of constraint solving, forthcoming. Conclusion AND/OR search should always be used. AND/OR BDDs are superior to OBDDs. A search algorithm with good and no-good learning generates an OBDD or an AND/OR Bdds. Dynamic variable ordering can be incorporated With caching, AND/OR search is similar to inference (variableelimination) The real tradeoff should be rephrased: Time vs space rather than search vs inference, or search vs model-checking. Search methods are more sensitive to this tradeoff.
© Copyright 2025 Paperzz