THE HEURISTIC SEARCH AND THE GAME OF CHESS
A STUDY OF QUIESCENCE, S A C R I F I C E S , AND PLAN ORIENTED PLAY
L a r r y R. H a r r i s
Department of Mathematics
Dartmouth College
H a n o v e r , New H a m p s h i r e
03755
Abstract
Thus
the appeal of the
heuristic search,
an
attempt
to
combine
a
sufficiently
powerful
search
mechanism
with
a
knowledgable h e u r i s t i c .
T h i s paper
describes the
results of
applying
the
formal
heurisitic
search
algorithm
t o t h e game
of chess,
and t h e
impact
of
this
work
on the
theory
of
heuristic
search.
It
is not
that
the
a p p l i c a t i o n o f the h e u r i s t i c search can b y
i t s e l f solve
the problems at the h e a r t of
t h e computer chess,
but that representing
these problems w i t h i n the formalism of the
h e u r i s t i c s e a r c h w i l l f u r t h e r t h e i r common
solution.
A separate
search h e u r i s t i c is
p r o p o s e d t h a t d o e s o f f e r a common s o l u t i o n
to the problems of quiescence, s a c r i f i c e s ,
and p l a n o r i e n t e d p l a y .
The h e u r i s t i c s e a r c h a p p r o a c h h a s n o t
proven
itself
competitively superior
to
the
alpha-beta
technique
either
winning,
d r a w i n g , and
l o s i n g games
with
Columbia,
Northwestern,
and
Tech
II
respectively.
This
substantiates
the
current
f e e l i n g in the AI
community t h a t
search s t r a t e g y i t s e l f plays a lower order
r o l e than o t h e r modules of the system.
However,
certain
aspects
of
this
approach
are worthy
of study
since they
may
prove to
provide a
b e t t e r mechanism
f o r d e a l i n g w i t h the problems at the h e a r t
of
the
computer chess
dilemma;
namely,
quiescence checking,
s a c r i f i c e s , and p l a n
oriented
play.
W e w i l l s p e c i f y t h e means
by
w h i c h the h e u r i s t i c s e a r c h can e x p r e s s
these
problems,
and
propose
a
common
solution,
whereas
the
current
minimax
p a r a d i g m seems r o a d b l o c k e d .
Introduction
The
purpose
of
this
paper
is
to
describe
the
results
of
applying
the
formal h e u r i s t i c
search a l g o r i t h m
to the
game
o f c h e s s , and d i s c u s s
the impact of
this
work
on
the
theory
of
heuristic
search.
We
hope to
demonstrate that
a
symbiotic relationship
e x i s t s between the
two areas i n t h a t t h e problems e n c o u n t e r e d
when
p l a y i n g computer chess can be b e t t e r
solved
within
the
formalism
of
the
heuristic
s e a r c h and
that the
theory of
heuristic
search
can
be
furthered
by
gaining
insight
from t h i s
very
complex
application.
Definitions
The h e u r i s t i c s e a r c h makes u s e o f t h e
following functions
defined over
the set
o f l e g a l board
configurations referred to
as nodes,
g(n)
- t h e n u m b e r o f moves f r o m t h e r o o t
t o t h e node n .
h(n)
= t h e m i n i m u m number
o f moves f r o m
node n t o a g o a l .
f(n)
= h(n)+g(n)
The
minimum number of
moves f r o m t h e r o o t t o a g o a l v i a
node n.
Computer c h e s s has b - e n d o m i n a t e d b y
programs
using
the
alpha-beta
minimax
s e a r c h ( G r e e n b l a t t ) and
more r e c e n t l y by
programs
using
an
exhaustive
search
(Northwestern,
Kiassa,
Tech
II).
The
t r e n d o f s u c c e s s f u l p r o g r a m s has t h u s been
t o a more b r u t e f o r c e a p p r o a c h r a t h e r t h a n
developing
more f o r m a l
s o l u t i o n s to
the
d i f f i c u l t problems t h a t a r i s e .
Approaches
using
more
sophisticated
representation
and u t i l i z a t i o n o f c h e s s k n o w l e d g e s u c h a s
Zobrist
and B e r l i n e r
have been u n a b l e t o
perform competitively
w i t h programs u s i n g
the
alpha-beta
technique.
The
actual
assessment of
new a p p r o a c h e s
is hampered
by the f a c t
t h a t a chess program hangs by
i t s w e a k e s t l i n k , a n d p o o r p l a y may n o t b e
the
f a u l t o f t h e new
approach.
However,
approaches
that
attempt
to
use
more
sophisticated heuristics
in preference to
a g e n e r a l t r e e s e a r c h a r e more s u s c e p t i b l e
to
occasional serious errors in play.
It
is
by no
means c l e a r
t h a t in
chess the
tradeoff
of search e f f o r t f o r accuracy in
evaluation
can
be
consistently
made
without
significant
loss
of
precision.
S i n c e some o f t h e s e f u n c t i o n s a r e n o t
known
during
the actual
search we
must
e s t i m a t e them.
The e s t i m a t i n g f u n c t i o n i s
denoted
by a
" " .
Thus,
the e s t i m a t i n g
function
of h(n) is
denoted f i ( n ) and i s
typically referred
to as
the " h e u r i s t i c "
or
the board e v a l u a t i o n f u n c t i o n .
A goal
is a winning p o s i t i o n .
At
each s t e p of the h e u r i s t i c search
the
most p r o m i s i n g l i n e o f
p l a y , t h e one
with
the best
f
value, is
expanded.
In
this
way
the
heuristic
h
orders
the
search.
The
h e u r i s t i c search
proceeds as f o l l o w s .
334
f o r a game
tree
1)
2)
3)
4)
5)
Initialize
the
search
tree
to
the
current
board r e f e r r e d to as the r o o t .
Select
a
node
for
expansion
by
following
t h e { - p o i n t e r s t o a t i p node
p.
H a l t if p is a t e r m i n a l node.
Expand
p,
linking all
sons i n t o
the
search t r e e .
calculate
fi
and g
f o r each
s o n , use
these to c a l c u l a t e f.
Back
up the values of
the "best" son,
setting
the
f-pointer
for
each
node
along the path to p.
6)
go t o (2) .
The a l g o r i t h m t e r m i n a t e s i n s t e p (2) o r
when
time
or
space
constraints
are
exhausted,
in which
case the
move i s
made t o t h e 1 s t l e v e l s o n w i t h t h e b e s t
fi value.
" B e s t " i s d e f i n e d a s e i t h e r t h e maximum o r
minimum
value depending
on i t s
level in
the
search
tree.
In
step
(5)
the
information
obtained at the t i p
nodes i s
backed
up i n t o t h e
search tree
where i t
can be compared to o t h e r l i n e s of p l a y .
Most
theoretical
work d e a l i n g
with
the h e u r i s t i c search deals w i t h the e f f e c t
o f r e s t r i c t i o n s o n t h e h e u r i s t i c and t h e i r
resulting
impact
on
the
search.
For
example, i f f i ( n ) < h ( n ) f o r a l l nodes n then
the
search
is
guaranteed
to
find
the
minimal
cost
goal.
By
applying
restrictions
that allow for error
in the
heurstic
we
can d e t e r m i n e
theoretically
how
the
h e u r i s t i c search
behaves
under
conditions
of
error
( H a r r i s ' ).
When
d e a l i n g w i t h chess we a r e f o r c e d to use an
error
prone h e u r i s t i c
or else
no search
would
be necessary.
Thus, it is c r i t i c a l
that the
search technique
remain s t a b l e ,
even when
misled by the h e u r i s t i c .
It is
shown
that
the
accuracy of
the
search
degrades
at worst only linearly
with the
error in the h e u r i s t i c .
The
Search
Heuristic
3
We
begin
our
discussion
of
the
problems
of
quiescence, sacrifices,
and
plan oriented play
by introducing another
forward estimating function
called 3.
As
we
w i l l
argue
later,
the notion
of
a
separate search h e u r i s t i c ,
one t h a t h e l p s
guide
the
search
independent of
the fi
evaluator,
has
application
in
problem
domains
other than chess.
We
w i l l argue
t h e need to
order the search on the basis
of information other than t h a t represented
by
g or
fi.
The o r d e r
of expansions w i l l
now b e d e t e r m i n e d b y f = g + h + 3 .
For
example,
in tactical
positions
the
fi
estimator
is
of
l i t t l e
use
in
searching for
the best l i n e of play since
the
n o t i o n s fi t y p i c a l l y measures, such as
material
advantage
and
pawn
structure,
become t e m p o r a r i l y u n i m p o r t a n t r e l a t i v e t o
an
effective
tactical
threat.
The
3
estimator tries
to forecast
the t a c t i c a l
line
and
bias
the
search
in
this
direction.
Note t h a t
we are
not saying
t h a t 3 can a c c u r a t e l y e s t i m a t e the r e s u l t s
o f t h e t a c t i c s - - i f t h i s c o u l d b e done w e
would simply put the r e s u l t d i r e c t l y in fi.
We
assume
only
that
d
will
be
oversensitive
and
signal
when
tactics
might p r e v a i l .
I n these cases w e leave i t
to
the search
to investigate
and v e r i f y
the 3
prediction.
We hope to s i m p l y b i a s
the search in the proper d i r e c t i o n .
The
need to s e p a r a t e fi &
3 is c l e a r
from
o u r use of h to
s e l e c t a c t u a l moves
to
b e made o n
the board.
In t h i s
way 3
guides
the s e a r c h , b u t does
not d i r e c t l y
affece
move
decisions,
unless
the
d
prediction
is manifested by an improved h
after
a successful search of the t a c t i c a l
line.
T h u s w e a s s o c i a t e moves w i t h h i g h f i
as
b e i n g " g o o d moves t o
m a k e " , and moves
with
high
3
as
being
"good
moves
to
investigate
further."
Quiescence
Checking
In
order
to
accurately
predict
t a c t i c a l l i n e s 3 m u s t b e s e n s i t i v e t o much
more
than o v e r t
c a p t u r e s and
checks, it
must
consider
pins,
forks,
discovered
attacks,
back
rank
attacks,
pawn
p r o m o t i o n , and
o t h e r phenomena
l i k e l y to
bring
about
an
a b r u p t change
in
piece
advantage.
In t h i s sense the 3 measure is
a quiescence check par e x c e l l e n c e .
When 3
is
near zero
then the h
esciiuace
can be
considered
accurate
since there
are
no
pending t a c t i c a l threats.
I n t h i s way t h e
"horizon
effect"
(Berliner )
can
be
avoided
by stopping
the search
when a l l
t i p nodes o f t h e s e a r c h t r e e a r e q u i e t ,
in
which
case
the search
tree is
said to
have t e r m i n a t e d .
Thus
w e a r e u s i n g t h e same e s t i m a t o r
that
finds
tactical
lines
to
define
quiescence
in a
much more
sophisticated
manner
than
others
(Greenblatt )
have
suggested.
Typically
only overt captures
and
checks are
i n c l u d e d in
a quiescence
check, 3 i n c l u d e s these even to the degree
of
signaling high values 1
ply after the
check
or
capture
to
force
continued
evaluation
of a forced l i n e .
But 3 a l s o
includes
the
tactical ploys
that
often
precede sudden f l u c t u a t i o n s i n fi.
We
must
consider the
problem
that
w i t h t h i s loose
d e f i n i t i o n of quiescence,
the
tree
may n e v e r
terminate.
In
this
regard
we
note that
3 features
are not
a d d i t i v e ; that i s , they do not cancel o u t .
If
both
players
have
roughly
equally
p r o m i s i n g t h r e a t s , t h e 3 f u n c t i o n musu n o t
sum t o z e r o , a s t h i s b o a r d i s a n y t h i n g b u t
a
q u i e t s i t u a t i o n t h a t can
be a c c u r a t e l y
rated.
By computing separate d values for
each of the
two p l a y e r s we can a v o i d t h i s
problem
and
can
also
define
tree
termination
i n a way t h a t
i s more l i k e l y
to
occur.
By s e p a r a t i n g
each
player' s
t h r e a t s we can d y n a m i c a l l y r a t e a board as
s t a b l e when t h e
son w i t h t h e b e s t f i v a l u e
has
one
sided
threats that
would
only
improve
the
rating.
Using
the
same
THE HEURISTIC SEARCH AND THE GAME OF CHESS
A STUDY OF QUIESCENCE, S A C R I F I C E S , AND PLAN ORIENTED PLAY
Larry R. H a r r i s
Department of Mathematics
Dartmouth College
H a n o v e r , New H a m p s h i r e
03755
Abstract
Thus
the appeal of the
heuristic search,
an
attempt
to
combine
a
sufficiently
powerful
search
mechanism
with
a
knowledgable h e u r i s t i c .
This paper
describes the
results of
applying
the
formal
heurisitic
search
algorithm
t o t h e game
of chess,
and t h e
impact
of
this
work
on the
theory
of
heuristic
search.
It
i s not
that
the
a p p l i c a t i o n o f the h e u r i s t i c search can b y
i t s e l f solve
the problems at the h e a r t of
the computer chess,
but that representing
these problems w i t h i n the formalism of the
h e u r i s t i c s e a r c h w i l l f u r t h e r t h e i r common
solution.
A separate
search h e u r i s t i c is
p r o p o s e d t h a t d o e s o f f e r a common s o l u t i o n
to the problems of quiescence, s a c r i f i c e s ,
and p l a n o r i e n t e d p l a y .
The h e u r i s t i c s e a r c h a p p r o a c h h a s n o t
proven
itself
competitively superior
to
the
alpha-beta
technique
either
winning,
d r a w i n g , and
l o s i n g games
with
Columbia,
Northwestern,
and
Tech
II
respectively.
This
substantiates
the
current
f e e l i n g in the AI
community t h a t
search s t r a t e g y i t s e l f plays a lower order
r o l e than o t h e r modules of the system.
However,
certain
aspects
of
this
approach
are w o r t h y
of study
since they
may
prove to
provide a
b e t t e r mechanism
f o r d e a l i n g w i t h the problems at the h e a r t
of
the
computer chess
dilemma;
namely,
quiescence checking,
s a c r i f i c e s , and p l a n
oriented
play.
W e w i l l s p e c i f y t h e means
by
w h i c h the h e u r i s t i c search can e x p r e s s
these
problems,
and
propose
a
common
solution,
whereas
the
current
minimax
p a r a d i g m seems r o a d b l o c k e d .
Introduction
The
purpose
of
this
paper
is
to
describe
the
results
of
applying
the
formal h e u r i s t i c
search a l g o r i t h m
to the
game
o f c h e s s , and d i s c u s s
the impact of
this
work
on
the
theory
of
heuristic
search.
We
hope to
demonstrate that
a
symbiotic relationship
e x i s t s between the
two a r e a s i n t h a t t h e p r o b l e m s e n c o u n t e r e d
when
p l a y i n g computer chess can be b e t t e r
solved
within
the
formalism
of
the
heuristic
s e a r c h and
t h a t the
theory of
heuristic
search
can
be
furthered
by
gaining
insight
from t h i s
very
complex
application.
Definitions
The h e u r i s t i c s e a r c h makes u s e o f t h e
following functions
defined over
the set
o f l e g a l board
configurations referred to
as nodes.
g(n)
= t h e n u m b e r o f moves f r o m t h e r o o t
t o t h e node n .
h(n)
- t h e m i n i m u m number
o f moves f r o m
node n t o a g o a l .
f(n)
= h(n)+g(n)
The
minimum number of
moves f r o m t h e r o o t t o a g o a l v i a
node n.
Computer chess has
b ' e n dominated by
programs
using
the
alpha-beta
minimax
s e a r c h ( G r e e n b l a t t ) and
more r e c e n t l y by
programs
using
an
exhaustive
search
(Northwestern,
Kiassa,
Tech
II).
The
t r e n d o f s u c c e s s f u l programs has t h u s been
t o a more b r u t e f o r c e a p p r o a c h r a t h e r t h a n
developing
more f o r m a l
solutions to
the
d i f f i c u l t problems t h a t a r i s e .
Approaches
using
more
sophisticated
representation
and u t i l i z a t i o n o f chess k n o w l e d g e s u c h a s
Zobrist
and B e r l i n e r
have been u n a b l e t o
perform competitively
w i t h programs u s i n g
the
alpha-beta
technique.
The
actual
assessment of
new a p p r o a c h e s
is hampered
by the f a c t
t h a t a chess program hangs by
i t s w e a k e s t l i n k , and p o o r p l a y may n o t b e
the
f a u l t o f t h e new
approach.
However,
approaches
that
attempt
to
use
more
sophisticated heuristics
in preference to
a g e n e r a l t r e e s e a r c h a r e more s u s c e p t i b l e
to
occasional serious errors in play.
It
is
by no
means c l e a r
t h a t in
chess the
tradeoff
of search e f f o r t f o r accuracy in
evaluation
can
be
consiSwently
made
without
significant
loss
of
precision.
S i n c e some o f t h e s e f u n c t i o n s a r e n o t
known
during
the a c t u a l
search we
must
e s t i m a t e them.
The e s t i m a t i n g f u n c t i o n i s
denoted
by a
" ".
Thus,
the e s t i m a t i n g
function
of h(n) is
d e n o t e d f i ( n ) and i s
typically referred
to as
the " h e u r i s t i c "
or
the board e v a l u a t i o n f u n c t i o n .
A goal
is a winning p o s i t i o n .
At
each s t e p of the h e u r i s t i c search
the
most p r o m i s i n g l i n e o f
p l a y , t h e one
with
the best f
value, is
expanded.
In
this
way
the
heuristic
fi
orders
the
search.
The
h e u r i s t i c search
proceeds as f o l l o w s .
334
f o r a game
tree
1) I n i t i a l i z e
the
search
tree
to
the
current
board r e f e r r e d to as the r o o t .
2) S e l e c t
a
node
for
expansion
by
following
t h e f - p o i n t e r s t o a t i p node
p.
H a l t if p is a t e r m i n a l node.
3 ) Expand
p,
linking all
sons i n t o
the
search t r e e .
4) c a l c u l a t e
fi
and g
f o r each
s o n , use
these to c a l c u l a t e f.
5 ) Back
up the values of
the "best" son,
setting
the
?-pointer for
each
node
along the path to p.
of t h e t a c t i c s -- if t h i s c o u l d be done we
would simply put the result d i r e c t l y in fi.
We
assume
only
that
3
will
be
oversensitive
and
signal
when
tactics
might p r e v a i l .
I n these cases w e leave i t
to
the search
to investigate
and v e r i f y
the 3
prediction.
We hope to s i m p l y b i a s
the search in the proper d i r e c t i o n .
The
need to separate fi &
d
is c l e a r
from
our use o f f i t o
s e l e c t a c t u a l moves
to
b e made o n
the board.
In t h i s
way 3
guides
the s e a r c h , b u t does
not d i r e c t l y
affece
move
decisions,
unless
the
d
prediction
is manifested by an improved fi
after
a successful search of the t a c t i c a l
line.
T h u s w e a s s o c i a t e moves w i t h h i g h f i
as
b e i n g ''good moves to
m a k e " , and moves
with
high
3
as
being
"good
moves
to
investigate
further."
6)
go t o (2) .
The a l g o r i t h m t e r m i n a t e s i n s t e p (2) o r
when
time
or
space
constraints
are
exhausted,
in which
case the
move i s
made t o t h e 1 s t l e v e l s o n w i t h t h e b e s t
fi value.
" B e s t " i s d e f i n e d a s e i t h e r t h e maximum o r
minimum
value depending
on i t s
level in
the
search
tree.
In
step
(5)
the
information
obtained at the t i p
nodes i s
backed
up into the
search tree
where it
can b e compared t o o t h e r l i n e s o f p l a y .
Quiescence
In
order
to
accurately
predict
t a c t i c a l l i n e s 3 m u s t b e s e n s i t i v e t o much
more
than overt
c a p t u r e s and
checks, it
must
consider
pins,
forks,
discovered
attacks,
back
rank
attacks,
pawn
p r o m o t i o n , and
o t h e r phenomena
l i k e l y to
bring
about
an
a b r u p t change
in
piece
advantage.
In t h i s sense the 3 measure is
a quiescence check par e x c e l l e n c e .
When d
is
near zero
then the fi estimate
can be
considered
accurate
since there
are
no
pending t a c t i c a l t h r e a t s .
I n t h i s way t h e
"horizon
effect"
(Berliner )
can
be
avoided
by stopping
the search
when a l l
t i p nodes o f the s e a r c h t r e e a r e q u i e t , i n
which
case
the search
tree is
said to
have t e r m i n a t e d .
Most
theoretical
work d e a l i n g
with
the h e u r i s t i c search deals w i t h the e f f e c t
o f r e s t r i c t i o n s o n t h e h e u r i s t i c and t h e i r
resulting
impact
on
the
search.
For
example, i f f i ( n ) < h ( n ) f o r a l l nodes n t h e n
the
search
is
guaranteed
to
find
the
minimal
cost
goal.
By
applying
restrictions
that allow for error
in the
heurstic
we
can d e t e r m i n e
theoretically
how
the
h e u r i s t i c search
behaves
under
conditions
of
error
(Harris ' ).
When
d e a l i n g w i t h chess w e a r e f o r c e d t o use a n
error
prone h e u r i s t i c
or else
no search
would
be necessary.
Thus, it is c r i t i c a l
that the
search technique
remain s t a b l e ,
even when
misled by the h e u r i s t i c .
It is
shown
that
the
accuracy of
the
search
degrades
at worst only linearly
w i t h the
error in the h e u r i s t i c .
The
Search
Heuristic
Checking
Thus
w e a r e u s i n g t h e same e s t i m a t o r
that
finds
tactical
lines
to
define
quiescence
in a
much more
sophisticated
manner
than
others
(Greenblatt )
have
suggested.
Typically
only overt captures
and
checks are
i n c l u d e d in
a quiescence
check, 3 i n c l u d e s these even to the degree
of
signaling high values 1
ply after the
check
or
capture
to
force
continued
evaluation
of a f o r c e d l i n e .
But 3 a l s o
includes
the
t a c t i c a l ploys
that
often
precede sudden f l u c t u a t i o n s i n f i .
3
We
begin
our
discussion
of
the
problems
of
quiescence, sacrifices,
and
plan oriented play
by introducing another
forward estimating function
called 3.
As
we
w i l l
argue
later,
the notion
of
a
separate search h e u r i s t i c ,
one t h a t h e l p s
guide
the
search
independent of
the fi
evaluator,
has
application
in
problem
domains
other than chess.
We
w i l l argue
t h e need to
order the search on the basis
of information other than that represented
by
g or fi.
The o r d e r
of expansions w i l l
now b e d e t e r m i n e d b y f = g + f i + d .
We
must
consider the
problem
that
with this loose
d e f i n i t i o n of quiescence,
the
tree
may n e v e r
terminate.
In
this
regard
we
note that
3 features
are not
additive;
that i s , they do not cancel o u t .
If
both
players
have
roughly
equally
p r o m i s i n g t h r e a t s , the 3 f u n c t i o n must n o t
sum t o z e r o , a s t h i s b o a r d i s a n y t h i n g b u t
a
q u i e t s i t u a t i o n t h a t can
be a c c u r a t e l y
rated.
By computing separate 3 values f o r
each of the
two p l a y e r s we can a v o i d t h i s
problem
and
can
also
define
tree
termination
i n a way t h a t
i s more l i k e l y
to
occur.
By separating
each
player's
t h r e a t s we can d y n a m i c a l l y r a t e a board as
s t a b l e when t h e
son w i t h t h e b e s t f i v a l u e
has
one
sided
threats that
would
only
improve
the
rating.
Using
the
same
For
example,
in tactical
positions
the
fi
estimator
is
of
l i t t l e
use
in
searching for
the best l i n e of play since
the
n o t i o n s fi t y p i c a l l y measures, such as
material
advantage
and
pawn
structure,
become t e m p o r a r i l y u n i m p o r t a n t r e l a t i v e t o
an
effective
tactical
threat.
The
3
estimator tries
to forecast
the t a c t i c a l
line
and
bias
the
search
in
this
direction.
Note t h a t
we are
not saying
t h a t 3 can a c c u r a t e l y e s t i m a t e the r e s u l t s
335
Critical
philosophy
as a l p h a - b e t a
c u t o f f s we
can
backup
the
quiet
rating
when-ever
one-sided
attacks
won't
change
the
ordering
even
if
they
were
to
prove
successful.
In
this
way t h e
root
can
eventually
be rated as q u i e t
even though
many t i p n o d e s o f t h e s e a r c h t r e e a r e n o t .
This
d e f i n i t i o n also
makes s e n s e
from a
search
p o i n t of view
since we
would not
wish
to
waste time
searching a
l i n e of
play
whose outcome
would not
change t h e
order
of our r a t i n g s , while
there remain
nodes i s t h e s e a r c h t r e e whose r a t i n g s a r e
unstable.
Sacrifices
Another flaw
of computer
chess p l a y
is
the
i n a b i l i t y to
find or
adequately
defend from e f f e c t i v e s a c r i f i c e s .
We w i l l
show
how
the search
heuristic 3
can be
used
in solving
t h i s problem
also.
The
f o l l o w i n g two boards
are extreme examples
of
s a c r i f i c i a l play, but as
such c l e a r l y
demonstrate
the
problems computers
face
when
required
to search
these l i n e s
of
play.
Board I is a
famous 1 9 - p l y m a t i n g
sequence
described
in
Berliner-.,
that
begins
w i t h a queen s a c r i f i c e .
The l i n e
is
1.
Q - R 5 c h , NxQ
2.
PxPch, K-N3,
3.
B-B2ch,
K-N4,
4.
R~B5ch,
K-N3
5.
R-B6ch,
K-N4 6 . R - N 6 c h ,
K-R5
7. R-K4ch,
N-B5,
8.
RxNch,
K-R4, 9. P-N3, any
10.
R-R4mate.
Quiescence checking
in the h e u r i s t i c
search
environment is on a scale r e l a t i v e
to the e n t i r e search t r e e .
The n o d e s w i t h
high
3 w i l l b e expanded
f i r s t u n t i l they
become q u i e t r e l a t i v e t o o t h e r s e c t i o n s o f
the
tree.
In fact,
one c o u l d
view the
search
process
as
attempting
to
m o n o t o n i c a l l y reduce the backup 3 v a l u e of
the
r o o t u n t i l i t becomes
close to zero;
meaning
that
the dynamic
r a t i n g of
the
root
is accurate.
If the
v a l u e does n o t
reach zero before
i t i s t i m e t o move t h e n
the
s i t u a t i o n i s too complex f o r a c c u r a t e
play,
but
we
can
at
least
make
a
r e a s o n a b l e move
based on
the i n f o r m a t i o n
gained in reducing the 3 value of the r o o t
a s much a s p o s s i b l e .
In the a l p h a - b e t a minimax environment
the
queiscence
check
must
be
made
independent
of other f a c t o r s in the t r e e ,
making
relative
comparison
impossible.
The d e c i s i o n m u s t
b e made a t t h e p o i n t o f
maximum p l y w h e t h e r o r n o t t o c o n t i n u e t h e
expansion.
Since
the
algorithm
is
recursive
there is
no information
about
other
areas
in the
tree.
As
such, the
d e c i s i o n m u s t b e made o n a b s o l u t e g r o u n d s ,
requiring a conservative
a p p r o a c h t o make
the
algorithm terminate
w i t h i n the
time
constraint
in
all
cases.
Thus,
most
quiescence
checks
in
the
alph-beta
environment do not include a l l the f a c t o r s
necessary
to determine whether a board is
really quiet.
Board I
We
are
faced
w i t h the
problem
of
finding
such
moves
when
they
are
s u c c e s s f u l , but
also avoiding
s u c h moves
when t h e y a r e n o t .
This l a t t e r concern is
even
more i m p o r t a n t s i n c e a p r o g r a m c o u l d
play
good
chess w i t h o u t
the a b i l i t y
to
make
19-ply
mates,
but
not
if
it
continually
enters
such s a c r i f i c e s
when
they are not s u c c e s s f u l .
T h e s a c r i f i c i a l move m i g h t b e d e f i n e d
a s a good
move t o i n v e s t i g a t e , b u t o n e t o
avoid
making u n t i l
its full
e f f e c t s are
searched o u t .
I n t h e p r o p o s e d scheme t h i s
would
b e a move w i t h
an i n c r e m e n t a l l o s s
in fi representing
the s a c r i f i c e d
piece,
and
an a r t i f i c i a l l y h i g h 3 to t e m p o r a r i l y
overcome t h e l o s s
in fi. This high d w i l l
bias the search to i n v e s t i g a t e the l i n e in
s p i t e of the m a t e r i a l l o s s .
Any
search,
minimax
or
heuristic,
t h a t uses a s i n g l e v a l u e t o compare b o a r d s
and
make m o v e s , c a n n o t s e p a r a t e
the good
f r o m t h e bad s a c r i f i c e s .
The r e a s o n i n g i s
t h a t a 5 - p l y minimax search c o u l d n o t f i n d
t h e mate in a s i n g l e s e a r c h s i m p l y because
of the depth
o f the sequence.
But, it is
entirely
possible
to
play
the
mating
sequence u s i n g a s e r i e s of 5 - p l y s e a r c h e s .
The
problem a r i s e s
in the
f i r s t search,
where we
are forced to decide whether the
queen
sacrifice
is
successful
or
not,
w i t h o u t being allowed to f u l l y explore the
336
requiring
substantial
mobilization
of
attacking
pieces
combined
with
limited
mobility
of the attacked piece before the
d rating
will
suggest
the
further
consideration
of
a sacrifice.
Both
of
these
conditions
exist
in
the
above
situation
and
would
allow
for
the
sacrificial
line
to
be
searched
even
beyond
t h e 8 t h move w h i c h i s
not a check
for white.
sequence.
If
the
heuristic
rates
the
s a c r i f i c e a s w o r t h w h i l e t h e n t h e move w i l l
b e made w i t h o u t
a thorough i n v e s t i g a t i o n .
This of c o u r s e , is v e r y dangerous as Board
II exemplifies.
Board I I i s i d e n t i c a l
to
Board
I e x c e p t t h a t b l a c k ' s KRP
is at R4
i n s t e a d o f R3.
The q u e e n s a c r i f i c e i s n o t
now
successful
because
of
this
small
change, s i n c e t h e k i n g can e v e n t u a l l y h i d e
and a v o i d f u r t h e r c h e c k s .
The l i n e i s : 1 .
QxPch,
NxQ
2.
PxPch,
K-N3
3.
any,
K-R3 e n d i n g t h e c h e c k s .
S u r e l y it must be
a
s e n s i t i v e h e u r i s t i c to
be sure
of the
outcome o f a queen s a c r i f i c e i n s i t u a t i o n s
such
as t h e s e .
Since
the stakes
are so
h i g h , most
programs w i l l p l a y i t s a f e and
w i l l avoid the s a c r i f i c e . This is a r e s u l t
of
the
minimax
forcing
a
premature
decision
without
allowing
a
thorough
i n v e s t i g a t i o n of the sequence.
Plan
Oriented
Play
Computer
chess
programs
are
most
often
c r i t i c i z e d because o f t h e i r l a c k o f
using
a
plan to
guide t h e i r
play.
The
p r o g r a m may know t o d e v e l o p i t s p i e c e s a n d
castle
early,
thus playing
the
opening
f a i r l y w e l l , only to flounder early in the
midgame w a i t i n g
f o r something
to happen.
A g a i n s t human
opponents the
game c h a n g e s
rapidly
enough s o
t h a t the
program must
react
t o t h r e a t s and can
formulate short
range a t t a c k s so the lack of a p l a n is not
always apparent.
When
two computers p l a y
each o t h e r
the p r o b a b i l i t y
of stagnation
increases markedly.
Nowhere,
however, is
the lack
of a
plan
more c r i t i c a l t h a n i n
endgame p l a y .
In
these cases
a winning
pawn a d v a n t a g e
can
be l o s t s i m p l y
because a
long range
plan
is
needed t o
promote
it.
Let
us
consider
some e x a m p l e s
of such
endgames
a n d how
the h e u r i s t i c
search environment
provides
a
means
of
using
long
range
goals.
Our c o n c e r n h e r e i s
not the very
d i f f i c u l t problem o f suggesting what p l a n s
are
worthy
of consideration
in a
given
s i t u a t i o n , but the d i f f i c u l t i e s that occur
when
we
try
to a c c u r a t e l y
measure
the
effectiveness of a given p l a n .
Board
A
board
suggested by
Berliner
is
shown
in
Board I I I .
In
t h i s case
the
white
king
must s i m p l y
flank the
black
pawn
w a l l to support
the advance
of the
white
pawn a t B 6 .
In t h i s
case the p l a n
itself
is quite clear, but
t h e manner i n
w h i c h i t musu b e c a r r i e d o u t i s h a r d f o r a
computer
to
find.
The
problem
appears
difficult
to a
computer program
because
the
w h i t e k i n g must f i r s t
move away f r o m
the
desired
position
supporting
the
advance of the
pawn a t B 6 .
We can e x p e c t
that
most chess h e u r i s t i c s w i l l
rate the
moves
K-K3 o r P-B7 i n i t i a l l y b e t t e r t h a n
the
r e q u i r e d K-B2.
Given t h i s
error in
II
The
o n l y way t o a v o i d t h i s t w o - s i d e d
dilemma
is
to
separate
the
search
ordering
f r o m t h e move o r d e r i n g .
Once w e
have
allowed
this distinction
we
could
e x p l o r e the
queen s a c r i f i c e
more d e e p l y .
If
the s a c r i f i c e i s successful
this w i l l
be
manifested
by
an improved
h
value.
Until
the fi value
of
the s a c r i f i c e
is
rated
as the
best in the search
tree we
w i l l not be
t e m p t e d t o make t h e m o v e , b u t
this
does
not r e s t r a i n
the search
from
investigating it further.
Once a g a i n , t h e
notion
of a separate search h e u r i s t i c , to
g u i d e t h e s e a r c h t o a r e a s o f t h e move t r e e
that
need
further
investigation
before
t h e y c a n b e a c c u r a t e l y a s s e s s e d , makes f o r
a
clean solution
to this
very d i f f i c u l t
problem.
For as
long as
move d e c i s i o n s
and s e a r c h
c r i t e r i a a r e based on a s i n g l e
value,
sacrificial
lines
of
play
like
those
exemplified
above
cannot
be
accurately
solved.
Clearly
some
tuning
of
the
d
heuristic
is
required
to
avoid
i n v e s t i g a t i n g every
seemingly i n t e r e s t i n g
possible
sacrifice.
This
is
done
by
337
the
h e u r i s t i c how
h e u r i s t i c searches
would the
react?
minimax and
search
technique.
A
similar
problem
arises
when t h e r e a r e m u l t i p l e p l a n s t h a t
could
apply in a
s i t u a t i o n and
the best
plan
requires
more
moves
to
become
effective.
Consider the
s i t u a t i o n in
Board I V .
Here,
Black
could formulate
2 plans
to
exploit
his
pawn
advantage.
He
could
a g g r e s s i v e l y a t t a c k t h e weak w h i t e pawn a t
R2
by moving K-B3.
Black
could also t r y
to
create a
p a s s e d pawn
in two
ways by
m o v i n g e i t h e r K-Q3 o r K - B 4 .
The f i r s t t w o
moves
lead to
a
draw, but
t h i t is
not
evident for 6 ply.
The 3 r d p l a n w i n s , b u t
is
not c l e a r l y evident for
about 15 p l y .
Thus a program
t h a t searches 5 p l y on a l l
3 lines
w i l l be unable to d i s t i n g u i s h the
good
plan
and
is
likely
to
draw
the
position.
Board
III
Berliner
gives
a detailed
scenario
for the minimax.
The move
P-B7 w i l l s o o r
lose
the
pawn
so
the
rating
drops
accordingly.
B u t t h e r a t i n g f o r t h e move
K-K3
increases
as
the
king
progresses
d i r e c t l y towards i t s g o a l .
The r a t i n g f o r
K-B2
will
be
r e l a t i v e l y low
since
the
distance
from
the king
and i t s
goal is
widened.
At
the
bottom
of
the
5~ply
search
t r e e the minimax w i l l
compare t h e
relative
closeness
of
the king
to
the
passed
pawn o b t a i n e d b y K - B 2 .
The c h o i c e
is q u i t e c l e a r
and t h e m i n i m a x b a c k s u p a
descendant
of
K-K3
not
realizing
the
f u t i l i t y o f t h e move.
How
then can
the h e u r i s t i c
search,
operating
with
the
same
error
prone
heuristic,
find
the
winning
move?
It
should
not
be
surprising
that
the
heuristic
search
can s o l v e
the
problem
since
it
has
long
been used
to
solve
s i m i l a r -puzzles
such
as
the
9-puzzle
(Nilsson ).
It
is just
for
situations
such as t h i s in which the h e u r i s t i c search
has
been
most
successful.
The
search
scenario proceeds as f o l l o w s .
Board
IV
When s e a r c h i n g w i t h m u l t i p l e p l a n s i n
mind
is is
n o t enough t o r a t e
by simply
taking
the
best
value obtained
by
any
plan.
This
would tend to b i a s the search
towards
the
plan
that
has
the
most
immediate
payoff.
This,
of course,
can
prove
to be short sighted, often ignoring
the
better
p l a n For
example, if
we are
required
t o r a t e Board I V
u s i n g o n l y one
value,
then
we are
forced to
determine
statically
which plan is b e s t .
When t h i s
determination
is in error, as
w i l l often
be
the case, the search
would
be b i a s e d
away f r o m i n v e s t i g a t i n g t h e b e s t p l a n .
It
is
actually
necessary to
keep
separate
ratings
o f each
p l a n and
either perform
the
search s e q u e n t i a l l y f o r each
plan or
perform
i t i n p a r a l l e l using the m u l t i p l e
heuristic values.
The f o r m e r
case w o u l d
seem p r o h i b i t i v e w i t h r e g a r d t o t i m e .
The
latter
case
would
require
backing
up
multiple information
f o r each
plan. This
would
s e v e r e l y h i n d e r t h e a b i l i t y t o make
alpha-beta cutoffs since
the c u t o f f would
have t o o c c u r f o r a l l p l a n s .
On the other
hand,
the h e u r i s t i c
search could
simply
continue
the
expansion
of
both
plans
taking into
account the g f a c t o r s so t h a t
s m a l l "humps" to be c r o s s e d , and a b i a s in
d
to investigate a l l
plans to
the deptl.
The moves
P-B7
and
K-K3
will
undoubtedly be
expanded f i r s t
because of
their
high ratings r e l a t i v e to
K-B2.
In
fact
the
move
K-B2
would
remain
unexpanded
f o r q u i t e some t i m e
until all
other
moves
demonstrate
their
lack
of
merit.
Soon P-B7 w i l l l o s e
t h e pawn a n d
be
rated
below
K-B2.
However,
the
descendants
o f K-K3 w i l l a t f i r s t i m p r o v e
as
the
king progresses.
When t h e
king
reaches
KB4
the king
w i l l be
forced to
retreat.
retreat.
This
r e t r e a t plus the
buildup of
g
values f o r these l i n e s w i l l
soon
diminish
their
ratings
until
the
point that
K-B2 i s e x p a n d e d .
I n t h e p r e v i o u s e x a m p l e t h e r e was o n e
obvious plan
and were i t n o t f o r " g e t t i n g
over
the
hump"
of
t h e pawn
wall,
the
solution
would
be
e a s i l y found
by
any
338
necessary
potential.
to
accurately
assess
up
the t r e e .
t h a t avoid the
their
If
we describe t h i s technique in the
terminology
of the
h e u r i s t i c search,
we
can
try
to
apply
it to
other
problem
domains.
Whenever
the
heuristic
rises
suddenly (gets worse)
a f t e r the expansion
o f a n o d e , t h e n w e h a v e d e t e c t e d some s o r t
of
catastrophe;
since
the
path
was
considered
to be
the b e s t we had,
or we
wouldn't
have been
expanding i t .
If we
c a n now i s o l a t e t h e r e a s o n f o r t h i s s u d d e n
r i s e i n f i , t h i s i n f o r m a t i o n c o u l d b e used
to reduce the search space.
Thus,
as
plan
oriented
play
is
introduced
into
computer
chess,
it
is
c l e a r t h a t p a r t o f i t s i m p l e m e n t a t i o n must
be
in b i a s i n g the
s e a r c h deep
enough t o
accurately
measure
the
success
of
the
plan.
The s e a r c h h e u r i s t i c d i s o n e means
b y w h i c h t h i s can b e c a r r i e d o u t .
Impact
upon
the
General
Heuristic
From t h i s
p o i n t o n l y moves
catastrophe are considered.
Search
We
have
seer
the
utility
of
the
search h e u r i s t i c
a* in c h e s s .
The u s e o f
t h i s h e u r i s t i c came a b o u t b e c a u s e t h e m o s t
e f f e c t i v e order of expansion d i f f e r e d from
the
order determined by the
g+fi m e a s u r e .
That
is,
some n o d e s
were expanded,
not
because
o f h i g h g+fi r a t i n g s ,
b u t because
their
fi
rating could
not be
accurately
measured.
In
these
cases
the
search
itself
must
b e used
to help
accurately
r a t e the node.
Hopefully,
continued
work
on
high
performance
problem s o l v i n g
systems such
as
chess
will
continue
to
contribute
searching techniques
t h a t can
be applied
e f f e c t i v e l y across problem domains.
References
1 . G r e e n b l a t t , R.D e t
a l , "The G r e e n b l a t t
Chess P l a y i n g
Program", Prodeedings of
the
1967
Fall
Joint
Computer
C o n f e r e n c e , p p . 801-R10
2. Zobrist,
A.L.,
Carlson,
F.R.,
"An
Advice
Taking
Chess
Computer",
S c i e n t i f i c A m e r i c a n , June 1973, v . 2 2 8 ,
N o . 6 , p . 92
3. B e r l i n e r ,
H.,
"Chess
as
Problem
Solving:
The D e v l o p m e n t
of a
Tactics
A n a l y z e r " Ph.D. T h e s i s , C a r n e g i e - M e l l o n
University
4. Harris,
L.R.,"A
Model
for
Adaptive
Problem
Solving
Applied
to
Natural
Language A c q u i s i t i o n " , T e c h n i c a l R e p o r t
TR-133, C o r n e l l U n i v e r s i t y
5. H a r r i s ,
L.R.
"The
Heuristic
Search
Under C o n d i t i o n s
of Error", A r t i f i c i a l
Intelligence,
F a l l 1974, v o l 5 , no. 3 ,
p . 217
6 . B e r l i n e r , H.,"Some Necessary C o n d i t i o n s
for
a
Master
Chess
Program",
Proceedings of
the Third I n t e r n a t i o n a l
Joint
Conference
on
Artificial
I n t e l l i g e n c e , 19 73
7. N i l s s o n , N. ,Problem
S o l v i n g Methods in
Artificial
Intelligence/
New
York:McGraw H i l l
8. Pohl,
I.,
"Avoidance
of
(Relative)
Catastrophe
e t c . " , Proceedings
of the
T h i r d I n t e r n a t i o n a l J o i n t Conference on
A r t i f i c i a l I n t e l l i g e n c e , 197 3
Considerations
such a s
this already
appear
in the l i t e r a t u r e of the h e u r i s t i c
search,
although
in
a
different
terminology.
In
a p p l y i n g the
heuristic
search
to
solve the
Traveling
Salesman
Problem,
Pol
uses
fi
with
two f a c t o r s
t h a t are
r e a l l y search
heuristics.
Both
of
t h e s e f a c t o r s t e n d t o more e f f e c t i v e l y
order the
sequence o f
expansions.
Since
they
are based on search
c r i t e r i a (depth
in
the tree)
and n o t
based on
t h e node
i t s e l f , w e c o u l d l a b e l them a s p a r t o f t h e
search h e u r i s t i c d.
This
relabeJing
allows
us
to
d i s t i n g u i s h the h e u r i s t i c f a c t o r s r e l a t i n g
t o t h e node i t s e l f ,
fi,
and t h o s e r e l a t i n g
to
the
search process
3.
Once
t h i s is
done
w e can s t a n d a r d i z e t h e d e f i n i t i o n o f
f t o b e f = g + f i + 3
and
recognize
that
t h i s "biasing of ?" is an important aspect
of
the
algorithm
itself
and
will
be
necessary
i n many
pr oblem domains
where
high performance is r e q u i r e d .
One
spinoff
of
work
on
a
high
performance
chess program
has been
this
recognition
of
t h e need
for a
separate
search
heuristic.
There
are
other
insights
to
the
general
technique
of
h e u r i s t i c s e a r c h e s t h a t can b e g a i n e d f r o m
high
performance
chess.
We
use
the
formalism
of
the
heuristic
search
to
a b s t r a c t the
ideas
of
chess programmers
and,
if
possible,
a p p l y them
in
other
search domains.
An
example
of
this
is
Berliner's
"Refutation
Description",
in
which
he
reduces
the
search
space
by
passing
information up the t r e e .
It is applied in
c h e s s when
a c e r t a i n l i n e of play thought
to
be
g o o d , ends
in a
catastrophe.
At
this
point
details of
the
catastrophe,
such
as
the
attacker,
attack
path,
attackee,
and f l i g h t p a t h a r e passed back
339