T A K I N G A D V A N T A G E OF STABLE SETS OF V A R I A B L E S
IN C O N S T R A I N T SATISFACTION P R O B L E M S
Eugene C. Freuder and Michael J. Quinn*
Department of Computer Science, University of New Hampshire, Durham, NH 03824
ABSTRACT
B i n a r y constraint s a t i s f a c t i o n p r o b l e m s i n v o l v e f i n d i n g values for
variables subject to constraints between pairs of variables. A l g o r i t h m s
that take advantage of the s t r u c t u r e of c o n s t r a i n t connections can be
more efficient t h a n simple b a c k t r a c k search. Some pairs of variables
may have no direct constraint between t h e m , even if they are linked
i n d i r e c t l y t h r o u g h a chain of constraints i n v o l v i n g other variables. A
set of variables w i t h no direct constraint between any pair of t h e m
forms a stable set in a constraint graph representation of a p r o b l e m .
We describe an a l g o r i t h m designed to t a k e advantage of stable sets
of variables, and give e x p e r i m e n t a l evidence t h a t it can o u t p e r f o r m
not only simple b a c k t r a c k i n g , but also f o r w a r d checking, one of the
best v a r i a n t s of backtrack search. P o t e n t i a l a p p l i c a t i o n s to p a r a l l e l
processing are noted. Some light is shed on the question of how and
when a constraint satisfaction p r o b l e m can be advantageously d i v i d e d
into subproblems.
1
D I R E C T I N D E P E N D E N C E OF VARIABLES
C o n s t r a i n t satisfaction problems i n v o l v e f i n d i n g values for v a r i ables subject to constraints, or relations a m o n g the variables. Often
these c o n s t r a i n t s are restricted to being b i n a r y relations between t w o
variables, we shall consider b i n a r y c o n s t r a i n t s here. S t a n d a r d backHack search can be used to solve such p r o b l e m s . In general t h e upper
b o u n d on the c o m p l e x i t y of the search is e x p o n e n t i a l in the n u m b e r
of variables
In this paper we propose t a k i n g advantage of the " r e l a t i v e i n dependence' 1 of variables to ameliorate the search c o m p l e x i t y . We
t e r m t w o variables d i r e c t l y independent if there is no direct constraint,
between t h e m , even if they may be i n d i r e c t l y related by a chain of
constraints passing t h r o u g h i n t e r m e d i a t e variables.
The basic insight is i l l u s t r a t e d by the example in Figure 1. The
p r o b l e m involves three variables, each w i t h three possible values, and
a constraint graph ( F i g u r e l a , l b ) L i n k s in the graph represent cons t r a i n t s ; nodes represent variables Note that there is no direct constraint between variables y and z. If we approach the p r o b l e m w i t h
a s t r a i g h t f o r w a r d b a c k t r a c k i n g a l g o r i t h m , we might have to examine
almost 3 x (3 x 3) or 27 possible triples of values before h i t t i n g upon
a solution in the r i g h t m o s t branch of the search tree. More t h a n 27
tests on pairs of values w o u l d be performed (Figure ]c).
Now let. us take i n t o account the relat ive independence of variables
y and z. H a v i n g chosen a value for x, we can go ahead and choose
a value for y and a value for z independently There w i l l be at most
6 values to consider (3 for each) before we succeed or f a i l If we f a i l ,
we repeat the process for the next x value. At most we w i l l p e r f o r m
3x (3 + 3) or 18 tests on pairs of values (Figure I d ) .
In general we can p a r t i t i o n c o n s t r a i n t graphs i n t o sets of " m u t u a l l y i n d e p e n d e n t " variables, where there is no direct constraint between any pair of variables in the set. Such a set is called a stable set
in graph theory. Consider the constraint graph shown in Figure 2, for
the p r o b l e m of labeling the cube. T h e scene labeling p r o b l e m Waltz.,
1975 provides an a p p l i c a t i o n d o m a i n here, but those u n f a m i l i a r w i t h
it can regard the cube as an abstract constraint graph The graph in
F i g u r e 2b is s i m p l y a r e d r a w i n g of the constraint graph in Figure 2a.
Observe t h a t the variables F and C are not j o i n e d by a c o n s t r a i n t ,
nor are B and D, not A and E. There are no edges between variables
at the same level in Figure 2b. Once a value has been chosen for (7,
the values for F and C may be chosen m u t u a l l y independently, then
values for D and D, and so o n . ( F i g u r e 2b is a generalized f o r m of an
"ordered c o n s t r a i n t g r a p h " [Freuder, 1 9 8 2 . )
M a n y interesting i m p r o v e m e n t s on basic b a c k t r a c k i n g have been
made H a r a l i c k and E l l i o t t , 1980; Gaschnig, 1978]. Generally these
i m p r o v e m e n t s operate at a "microlevel," i n v o l v i n g the relationships
between i n d i v i d u a l values for variables, e.g., value a for v a r i a b l e x
is inconsistent w i t h value b for variable y. Our concerns are at a
" m a c r o l e v e l , " i n v o l v i n g the relationships between the variables t h e m selves, e.g., there is no constraint between variables x and y. A more
detailed e x p e r i m e n t a l comparison w i t h one of the best of the backtrack
v a r i a t i o n s is made later
E. Freuder and M. Quinn
II PSEUDO-TREE SEARCH
ALGORITHM
G i v e n a constraint graph p a r t i t i o n e d i n t o stable .sets, we w o u l d
like to take advantage of direct independence not only on one m a x i m a l
stable set (or an a p p r o x i m a t i o n t h e r e o f ) , but on all the stable sets. For
e x a m p l e , we w o u l d like to use the stable sets i d e n t i f i e d in Figure 2 as
"levels" in a generalized backtrack a l g o r i t h m that employed the basic
" a d d i t i v e " insight of Section 1 at each level.
T h i s can be done, up to a p o i n t , in a s t r a i g h t f o r w a r d way Proceeding d o w n t h r o u g h the levels, each v a r i a b l e at a given level can be
considered i n d e p e n d e n t l y . However, a p r o b l e m arises in b a c k t r a c k i n g .
If t w o variables at one level have a constraint path d o w n to a single
variable at a lower level (as in Figure 2 b ) , all c o m b i n a t i o n s of values
for the t w o higher level variables may need to be tried before f i n d i n g
one c o m p a t i b l e w i t h the lower level v a r i a b l e T h u s , we again have a
" m u l t i p l i c a t i v e " r a t h e r t h a n an " a d d i t i v e " effect.
D e f i n i t i o n : A pseudo-tree is a rooted tree T
(V. E) augmented
w i t h zero or more a d d i t i o n a l edges /,'' such t h a t all edges between
some vertex v and vertices closer to the root be along the p a t h in T
f r o m v to the r o o t T h e level of a variable in a pseudo-tree is its level
in the u n d e r l y i n g r o o t e d tree
A pseudo-tree s t r u c t u r e d constraint graph supports an a l g o r i t h m
that avoids the " m u l t i p l i c a t i v e backtracking" , problem cited above.
It also makes it easy to determine what variables need to be m o d ified d u r i n g b a c k t r a c k i n g
T h i r d l y , the pseudo-tree p r o p e r t y makes
the c o n s t r a i n t satisfaction problem more amenable to s o l u t i o n on a
multiprocessor, because a processor could t r y to assign values to a
variable and all its descendenis w i t h o u t h a v i n g to c o m m u n i c a t e w i t h
other processors Freuder and Q u i n n , 1985
Pseudo-Tree Search A l g o r i t h m
1077
L e m m a : G i v e n a pseudo-tree T and A variables at level m in T,
each v a r i a b l e capable of t a k i n g on b values, the search a l g o r i t h m backtracks at most bk times to level rn before b a c k t r a c k i n g to level rn
1.
Proof: T h e first t i m e the search a l g o r i t h m reaches level m, the
value of every variable at that level is i n i t i a l i z e d as low as possible.
T h e search a l g o r i t h m backtracks f r o m level rn - 1 in T when there
exists at that level a variable v that cannot be given a value that does
not v i o l a t e c o n s t r a i n t s w i t h the values of ancestor variables in T. By
the pseudo-tree p r o p e r t y , all of the variables c o n s t r a i n i n g v lie along a
simple path f r o m v to the root of T. In order to t r y all c o m b i n a t i o n s of
values that m i g h t lead to an a l l o w a b l e value of v, o n l y v a i i a b l e s along
this path need to have their values changed
Hence every t i m e the
search backtracks to level rn, exactly one of the variables w at level m
must have its value m o d i f i e d . The value of w cannot be decremented:
all lowei values were previously held by w and led to a b a c k t r a c k i n g
lower in the tree. Because it is only necessary to reset the value of w
when one of its ancestors has its value i n c r e m e n t e d , the value of no
variable at level m w i l l lowei u n t i l the search b a c k t r a c k s f r o m level m.
Since there are k variables, each w i t h b possible values, the m a x i m u m
n u m b e r of values variables at level m can take on before one variable
has no more possible values is bk,
T h e o r e m : G i v e n a pseudo-tree T w i t h rn levels, each level / cont a i n i n g A:, variables capable of t a k i n g on b, values, the worst-case t i m e
c o m p l e x i t y of our a l g o r i t h m is
C o r o l l a r y ; G i v e n a pseudo-tree T our a l g o r i t h m can find one assignment of values to variables that satisfies the c o n s t r a i n t s , if one
exists, but it cannot be used to find all solutions that satisfy the constraints.
Ill
DERIVING
PSEUDO-TREE
M E T A P R O B L EMS
A constraint graph can be s u b d i v i d e d i n t o s u b p r o b l e m s , where
each s u b p r o b l e m involves satisfying a subset of the o r i g i n a l variables
T h e p r o b l e m of satisfying all the subproblems s i m u l t a n e o u s l y m a y then
be regarded as a metproblem, w i t h the s u b p r o b l e m s as metavariables
It has long been recognized t h a t p a r t i t i o n i n g a constraint, satisfact i o n p r o b l e m i n t o subproblems may s i m p l i f y the p r o b l e m . However,
l i t t l e guidance is available as to how and when to s u b d i v i d e problems. One c r i t e r i o n for considering a p r o b l e m s u b d i v i s i o n is the felicity of the s t r u c t u r e of the r e s u l t i n g m e t a p r o b l e m . For example, if the
m e t a p r o b l e m has a tree s t r u c t u r e d constraint graph on the m e t a v a r i ables, then it c an be solved in t i m e linear in the n u m b e r of m e t a v a r i ables ; M a c k w o r t h and Freuder, to appear
Here we present a m e t h o d
lor p r o d u c i n g a m e t a p r o b l e m where the metavariables are organized
i n t o stable sets f o r m i n g levels in a pseudo-tree constraint s t r u c t u r e .
Recall that the previous section presented an a l g o r i t h m for pseudotree-struct ured p r o b l e m s , w i t h a c o m p l e x i t y b o u n d e x p o n e n t i a l in the
n u m b e r of pseudo-tree levels.
The o r i g i n a l p r o b l e m is p a r t i t i o n e d i n t o a m e t a p r o b l e m as follows:
1
F i n d a cut set in the constraint g r a p h ; i.e., a set of vertices
whose r e m o v a l divides the graph i n t o t w o or more unconnected
subgraphs. T h e cut set S corresponds to S variables in the
m e t a p r o b l e m ; thesemetavariable*will
f o r m the first \S\ levels of
the pseudo-tree T h e unconnected subgraphs t e m p o r a r i l y become
metavariables and are children of the variable in S deepest in the
"tree "
2
A p p l y this process recursively to each of the c h i l d r e n , t e r m i n a t i n g
when the- subgraph being examined cannot be split f u r t h e r .
The a l g o r i t h m w i l l not necessarily produce an " o p t i m a l " pseudotree
We w o u l d like to be certain of efficiently t r a n s f o r m i n g a given
constraint g r a p h i n t o a pseudo-tree s t r u c t u r e w h i c h takes o p t i m a l (or
nearly o p t i m a l ) advantage of direct independence, T h i s remains an
area for f urt her w o r k .
IV
T h e f o l l o w i n g lemma proves that the pseudo-tree search a l g o r i t h m
has a c o m p l e x i t y b o u n d e x p o n e n t i a l in the n u m b e r of levels in the
pseudo-tree, rather t h a n in the n u m b e r of variables in the p r o b l e m In
the next section we present an a l g o r i t h m for t r a n s f o r m i n g an a r b i t r a r y
c o n s t r a i n t satisfaction problem i n t o a n equivalent " m e t a p r o b l e m " w i t h
a pseudo-tree constraint graph s t r u c t u r e .
EXPERIMENTAL
RESULTS
To e x p e r i m e n t a l l y verify the efficiency of this a l g o r i t h m , we have
used it to t r y to color graphs w i t h three colors
T h e performance
of our a l g o r i t h m is c o n t r a s t e d w i t h t w o others. T h e first a l g o r i t h m
performs s i m p l e b a c k t r a c k i n g , where the order in w h i c h the variables
are searched is r a n d o m l y chosen. T h e second a l g o r i t h m uses f o r w a r d
checking H a r a l i c k and E l l i o t t . 1980], w h i c h H a r a l i c k and Elliot showed
to be superior to several other v a r i a n t s of b a c k t r a c k i n g under certain
e x p e r i m e n t a l c o n d i t i o n s T h e order in w h i c h variables are considered
corresponds to a level-by-level traversal of the pseudo-tree.
1078
E. Freuder and M. Quinn
The form of the test graphs is shown in Figure 3. Notice that
we have created constraint graphs that have the pseudo-tree property,
which avoids the previously-mentioned problem of efficiently transforming constraint graphs into pseudo-trees. We have tested the three
algorithms on a large number of graphs. For each problem size ranging
from 4 to 36 variables, we have generated 100 random pseudo-trees.
We have run the three search algorithms on all 900 graphs, measuring
the number of constraint checks made by each algorithm. Each result
has been put into one of two categories, depending upon whether the
graph is three-colorable or not. This is because unsuccessful searches
require many more constraint checks on average. Figure 4 displays the
mean number of constraint checks performed by the three algorithms
for successful and unsuccessful searches of the various graphs. Not only
does our algorithm outperform standard backtracking, it also performs
substantially fewer constraint checks than the forward checking algorithm on both colorable and uncolorable graphs. (We have also computed the median number of constraint checks performed by the three
algorithms Our algorithm outperforms standard backtracking and
forward checking on unsuccessful searches and standard backtracking
on successful searches. However, the median number of constraint
checks performed by forward checking on successful searches is a few
percent lower than the number performed by our algorithm.)
Certainly our search algorithm will not outperform other algorithms, such as forward checking, in all situations. The constraints
must fit a certain pattern in order to benefit from our exploitation of
direct independence. For example, if our methodology were applied
to the eight-queens problem (placing eight queens on a chess board
so that no queen can attack another), the pseudo-tree would have
eight levels, and the search algorithm would degenerate into standard
backtracking.
In fact, much of the experimental work on backtrack search has
been done in the context of the eight-queens problem. This is actually
a very specialised type of problem: the constraint graph is complete,
all constraints are the same, and all variables have the same domain.
Direct independence in a constraint graph is in a sense the opposite of
completeness; it involves subsets of variables, which, far from forming
complete subgraphs, form stable sets, where no variable is connected
to any other.
V
CONCLUSIONS
We have presented an algorithm tailored to solve constraint satisfaction problems on constraint graphs that have the pseudo-tree property. We have also described an algorithm for turning any constraint
graph into a constraint'graph with the pseudo-tree property Experimental evidence indicates there exist constraint graphs for which our
algorithm outperforms not only standard backtracking, but also forward checking.
ACKNOWLEDGMENTS
This paper is based in part upon work supported by the National
Science Foundation under Grant MCS 8003307.
REFERENCES
[1] Freuder, E.C. 1982. A sufficient condition for backtrack-free
search. J. ACM 29, 1, pp. 24-32.
[2] Freuder, E.C, and Quinn, M.J. 1985. Parallelism in an algorithm
that takes advantage of stable sets of variables in constraint satisfaction problems. Tech Rep. 85-21, Dept. of Computer Science,
Univ of New Hampshire.
|3j Gaschnig, J. 1978. Experimental case studies of backtrack vs.
Waltz-type vs. new algorithms for satisficing-assignment problems. Proc. 2nd National Conf. of Canadian Society for Computational Studies of Intelligence, Toronto, Ontario, July 19-21 pp
268-277.
[4] Haralick, R , and Elliott, G. 1980. Increasing tree search efficiency
for constraint satisfaction problems. Artificial Intelligence 14, pp.
263-313.
|5I Waltz,D 1975. Understanding line drawings of scenes with shadows In The Psychology of Computer Vision, P.H. Winston, ed.,
McGraw-Hill, New York, pp. 19-91.