ANAPHORA RESOLUTION:
SHORT-TERM MEMORY AND FOCUSING
RAYMONDE GUINDON
Microeleotronlcs
and Computer T e c h n o l o g y C o r p o r a t i o n
(MCC)
9430 R e s e a r c h Blvd.
Austin, Texas ?8759.
ABSTRACT
INTRODUCTION
A n a p h o r a r e s o l u t i o n is the
process
of
determining
the
referent
of ~uaphors.
such
as d e f i n i t e
noun
phrases
and
pronouns, in a discourse.
Computational
linguists, in m o d e l i n g
the
process
of
anaphora
resolution,
have p r o p o s e d the
n o t i o n of
focusing.
Focusing
is
the
process,
engaged
in by
a
reader, of
selecting
a subset
of
the
discourse
items
and
maJ£ing them h i g h l y a v a i l a b l e
for further
computations.
This
paper
provides
a c o g n i t i v e basis for a n a p h o r a
r e s o l u t i o n and focusing.
Human
memory
is
divided
into
a short-term,
an
operating,
and
a
long-term
memory.
Short-term
memory
can
only
contain a
small number of meaning
units
and
its
retrieval
time
is
fast.
Short-term
m e m o r y is d i v i d e d into
a cache
and
a
buffer.
The cache contains a subset of
meaning units e x p r e s s e d in the
previous
sentences
and
the
buffer
holds
a
r e p r e s e n t a t i o n of ~he incoming sentence.
Focusing
is
r e a l i z e d in ~he cache that
contains a subset of
the
most
topical
units
and
a
subset of the mos~ recent
units
in the
text.
The
information
stored in the cache is used to i n t e g r a t e
~he incoming s e n t e n c e with the p r e c e d i n g
discourse.
Pronouns
should be u s e d to
refer
to units
in
focus.
Operating
memory
contains
a very large number of
units but its r e ~ r l e v a l
time
is
slow.
It contains the previous tex~ u n i t s that
are not in the cache.
It c o m p r i s e s
the
tex~
units not in focus.
D e f i n i t e noun
phrases should be used to refer to unite
not in focus.
Two e m p i r i c a l studies are
d e s c r i b e d that d e m o n s t r a t e the c o g n i t i v e
basis
for focusing, the use of d e f i n i t e
noun phrases to refer to a n t e c e d e n t s not
in
focus,
and
the
use of p r o n o u n s to
refer to a n t e c e d e n t s in focus.
The goal of
thls
research
is
to
show
the
relation
between
the
psychological
work
on
anaphora
resolution
based
on
the
notion
of a
l i m i t e d s h o r t - t e r m or w o r k i n g memory and
the c o m p u t a t i o n a l l i n g u i s t i c s work b a s e d
on the notion of focusing.
This r a p p r o c h e m e n t is important for
the f o l l o w i n g reasons:
I)
From
a
theoretical
viewpoint.
cognitive
evidence
increases
the
v a l i d i t y of the c o m p u t a t i o n a l notion
of
focus.
2) F o c u s i n g c o r r e s p o n d s to
one
of
the
reader's
c o m p r e h e n s i o n p r o c e s s e s and it
needs to be i n c o r p o r a t e d in the model of
the
user
in
language
understanding
systems
to
adequately
resolve
a m ~ i g u i t l e s in the user's u t t e r a n c e s and
to h a n d l e language generation.
FOCUSING
IN C O M P U T A T I O N A L
LINGUISTICS
A c c o r d i n g to Grosz ( 1 9 7 7 ) . who
was
i n t e r e s t e d in ~he r e s o l u t i o n of d e f i n i t e
noun phrases, focusing is
the
process.
engaged
in
by
participants
in
a
discourse, of h i g h l i g h t i n g a
subset
of
their shared reality.
Grosz. Joshi. and
w e i n s t e i n (1983) d i s t i n g u i s h b e t w e e n two
levels
of
focus,
global
focus
and
centerimg.
G l o b a l f o c u s i n g is
a major
factor
in m a i n t a i n i n g global c o h e r e n c e
and in the
interpretation
of d e f i n i t e
noun
phrases.
Centering
is a m a j o r
factor in m a i n t a i n i n g
local
coherence
and in the i n t e r p r e t a t i o n of pronouns.
Grosz. Joshi. and W e i n s t e i n further
define
the
notion
of centering.
Each
sentence has two types of centers
whose
purpose
is t o integrate the sentence t o
the
discourse.
The
backward-looking
center links the current sentence to the
preceding
discourse.
The
set
of
f o r w a r d - l o o k l n g centers p r o v i d e s th~ set
of e n t i t i e s to
which
further
anaphors
m~y
refer.
The b 6 o k w ~ r d - l o o k l n g c e n t e r
corresponds, roughly.- to Sidner's
focus
and
the
forward-looklng
centers
to
Sidner's potentla~l fool.
~8
•
L
oomputatlon~l
costs
assoolated
with
inferential
p r o c e s s i n g when testln~ the
a p p l i c a b i l i t y of the o o - s p e c l f l e r to the
anaphor.
One
principle
derived
by Grosz,
Joshl,
and
W e i n s ~ e i n is the following:
if the b ~ o k w a r d - l o o k i n g
center
of the
ourren~
utterance
is
the
same as the
b a o k w a r d - l o o k l n g cen~er of the
previous
utterance, a p r o n o u n should be used.
In
other
words,
if
there
are
no
~oplc
shifts,
continue
to
refer to the same
entity by using a pronoun.
COGNITIVE
STUDIES OF A N A P H O R A
RESOLUTION
A
few
representative
empirical
studies
of
anaphora
resolution
are
d e s c r i b e d below.
All
the
experimental
par~dlgms
used
share
the
following
assumptions:
1) h u m a n
memory
is
func~ionally
or
s~ruc~urally
divided
into at least two
types of memories, a s h o r t - t e r m
memory
with
small
storage
capacity
but very
fast
retrieval
time
and
a
long-term
memory
with very large s~orage capacity
but slow retrieval time:
2) a topic
shift
transfers
the
units
currently
in
short-term
memory
to
l o n ~ - t e r m memory:
3) ~n
anaphor
transfers
its
referent
from
long-term
memory
to
short-term
m e m o r y (i.e.
r e i n s t a t e s its
referent),
if
it
was
not
already
in s h o r t - t e r m
memory.
However,
violations
ofthis
principle
have
been p r e s e n t e d in Grosz
(1977) and n o t e d in Grosz,
Joshl,
and
Welns~eln
(198~).
They have shown that
pronouns are sometimes used to refer
to
entities
m e n t i o n e d many sentences back,
even though the b a c k w a r d - l o o k l n g
center
of
intervening
sentences
has
been
changed by topic shifts.
Sidner (19V9.
1983)
has
proposed
the
notion
of
focus in the context of
interpreting
anaphors,
especially
pronouns.
In
Sidner's
theory,
an
anaphor neither refers ~o
another
word
nor
co-refers
to another
word.
but
rather c o - s p e c i f i e s a cognitive
elemen~
in
the
reader's
mind.
Moreover.
a
theory
of an&phora
resolution
must
predict
the pattern of reader's correct
and incorrect choices
of
co-specifiers
and
~he
failures
~o u n d e r s t a n d . This
view makes explicit the c o n s i d e r a t i o n of
the
reader's
mental
model
and
inferential capa~bili~ies
The first
assumption
is
crucial.
Other
things
being equal, c o m p u t a t i o n s
involving
retrieval
from
short-term
memory
will
be
faster
than
those
involving
retrieval
from
long-term
memory.
Turning
to
the
second
assumption, topic shifts have been found
to
be
induced
wlth
a varify
of
l i n g u l s % i c devices.
One of the
devices
is
the
introduction
of
intervening
sentences between the referent
and
its
anaphor.
The i n t e r v e n i n g sentences are
u n r e l a t e d zo the referent but related to
the overall text.
A n o t h e r device is the
s p e c i f i c a t i o n of a temporal
or
Spatial
parameter
that
is
outside
the normal
range of a situation.
When d e s c r i b i n g a
dinner,
the
phrase "Five hours later,"
signals ~h~t the topic
of
conversation
is no longer the dinner.
A n o t h e r device
is the use of an anaphor,
frequently
a
deflnlte
noun
phrase,
to
refer to an
antecedent tha~
is
not
currently
the
topic
of c o n v e r s a t i o n
bu~
is
in the
"background".
Finally, there is the use
of key phrases to signal a d i v e r s i o n in
the flow of discourse,
such
as
"Let's
turn
to.".
as d o c u m e n t e d
in Relchman
(1978, 1984).
A sEetch
of
Sidner's
focusing
process
follows.
First.
an
initial
focus
is
selected
on
the
basis
of
syntactic
features
and
thematic roles
indicating
toplc~lity
in
the
flrs~
sentence.
Other elements i n t r o d u c e d in
the sentence
are
stored
as
potential
loci
for
later
sentences.
When
an
anaphorlc
expression
is
encountered.
this
focus
is tested as a c o - s p e c l f l e r
for ~he
anaphor.
It
has
to
satisfy
syntaotlo
r e s ~ r l o ~ i o n s on c o - r e f e r e n c e s
(L~snlk,
1976),
semantic
seleo~ional
restrlo~ions (Katz and Fodor, 1963), and
pragmatic
plausibility
oons~raln~s
expressed
in
the
remainder
of
the
sentence.
If
the
focus
fails
~s
a
co-speclfier
for
the
~n~phor,
the
potential fool are tried
in turn.
At
the
same
time,
the
new
elements
i n t r o d u c e d in the sentence are stored as
potential
loci
for
later
sentences.
Third,
the
focus
is updated
to
the
selected
co-speclfler
for the anaphor.
If the focus has changed, a topic
s~ift
has
occurred.
The
second
and
third
s~eps are cyclically applied after
each
sentence.
The
general
pattern
for
the
material
used
in
these e x p e r i m e n t s is
the following.
A~ the b e g i n n i n g of
the
tex~
appears
a
sentence
containing a
referent
(e.g.
biologist).
For
example,
"The
mission
included
a
biologist".
Then,
if
~he
referent
should
not
be
in
focus,
the
nex~
sentence or sentences indloate
a topic
shift
as
described
aJ3ovs
(e.g.
The
advantage
of using
a
focus
mechanism
is
tha~
it
priorltlzes and
r e s t r l c ~ s the search for a co-speclfier,
and
as
a consequence,
reduces
the
~9
unrelated
intervening
sentences).
If
the
referent
should
be in
focus, no
devices for topic shifts are used.
The
following
sentence
then
contains
an
an&phor (e.g.
scientist,
he)
to the
focused
or non-focused
referent (e.g.
biologist).
For example, "The scientlst
collected
samples
from
the cultures".
Another example is shown in Table
1 of
this paper.
These results show
not
only
that
&naphora
resolution
is easier when the
referent is nearer the ~naphor but
also
that
one
intervenln E sentence
may be
sufflolent to produce a topic shift.
C l a r e and Sengul
(1979)
used
the
sentence reading time technique to study
a n a p h o r a resolution.
In this technique.
subjects control the onset and offset of
the
presentation
of a
sentence
by
pressing
a button.
The
subjects are
i n s t r u c t e d to press the button to see
a
new
sentence
as
soon
as
they
have
u n d e r s t o o d the
current
sentence.
The
a s s u m p t i o n behind this technique is that
additional
processing
required
for
comprehension
will
increase
sentence
reading time.
C a r p e n t e r and Just (1977) used
eye
traoklng
with
other
converging
techniques to study anaphora resolution.
Wlth
eye tracking, one can monitor very
p r e c i s e l y the trajectory
of the
eyes,
with
their
forward
and
regressive
movements,
and
the
duration
of
eye
fixations on small segments of the t e ~ .
The
assumption
behind
using
this
technique
is
that
eye
movements
are
closely
related
to
higher
level
cognitive
activities
such
as
comprehension.
Therefore.
one
can
expect longer fixation durations on text
segments requiring additional processing
to
be
comprehended
and one can expect
the eye movement pattern to
mirror
the
selective
pickup
of
important
i n f o r m a t i o n in the text.
Clare and
Sengul
(1979)
measured
the
reading
time
of
a
sentence
containing
an
anaphor.
They
distinguished
between two models of the
effect of recency of a referent
on the
speed
of
~naphora
resolution.
In the
first
model,
called
the
"continuity
model",
entities
mentioned
in the
d i s c o u r s e are searched backward from the
last
one.
One
should
expect
m o n o t o n i c a l l y increasing re~din~ time as
the searched entity is farther back.
In
the
second
model,
called
the
"discontinuity
model",
entities
mentioned
in the
current
or
last
sentence
are
kept in short-term memory
and accessed first.
All
the
entities
that are further back are more likely to
be
in
long-term
memory
(and
not
in
shor~-term memory) and accessed second.
They
performed
a
series
of
experiments
testln~
the
effect
of
recency of a referent on the time course
of
anaphora
resolution.
Indirectly.
they tested the effect of recency on the
availability
of
an
item in short-term
memory.
They presented texts where
the
number of sentences between the referent
and the anaphor was varied from zero
to
three.
The subjects read each sentence
and. after the sentence, had
to decide
whether
it
was
consistent
or
inconsistent
with
the
previous
sentences.
The
consistency
Judgment
times
and
the
eye
fixations
were
recorded.
The
consistency
Judgment
task, used as converging
evidence
with
the
eye movement technique, is believed
to induce the subjects to integrate each
new
sentence
and
should
pars,llel the
d i f f i c u l t y of ~ p h o r a
resolution.
The
overall
reading
time
of the ~n&phorlo
sentence
was
measured
using
the
eye
tracking technique.
Each of these tasks
should be faster if the referent was
in
short-term
memory
than if the referent
was in long-term memory.
Subjects
rea~
short
paragraphs
where a referent could be separated from
the anaphor by zero ~o
two
intervenin~
sentences.
The
readln~
time
of
~he
sentence containing the anaphor was fast
when the referent was in the immediately
preceding sentence but ~ a l l x
~
when
it
was
two
or three sentences before.
This finding supports the
discontinuity
model.
Entities in the last processing
cycle are more
likely
to
be
kept
in
short-term
memory
than
entities
in
previously
processed
cycles.
Once
a
tex~
entity
is
not in short-term, the
number of intervening sentences does not
affect the speed of an~phora resolution.
Lesgold, Roth, and
Curtis
(1979),
who
related
the
linguistic
notion of
foregrounding
(Chafe,
1972)
to
~he
psychological
notion
of
short-term
memory,
performed
a
series
of
experiments
similar
~o those of Clark
~nd Sengul
(1979),
using
more
varied
ways
to
produce
topic
shifts,
and
replicated the above findings.
Response times for the
consistency
Judgments
and
reading
times
of the
anaphorlc
sentences
increased
as the
n-mher
of
intervening
sentences
increased.
The
sharpest
difference
appeared
between
zero
and
one
intervening
sentence.
Gaze
durations
within
the
anaphorlo
sentence
were
shorter when there were
no
intervenlng
sentences th~n in the other conditions.
220
the e n d of the
clause,
as
opposed
to
when
the
anaphor
is
encountered, the
referent had time to
be
reinstated
in
shor~-term
memory
and
be
highly
available.
This is an important
point.
The a c t i v a t i o n p r o c e d u r e was not on-llne
since the old-new r e c o g n i t i o n ocoured at
the
~n~
of
the sentence as opposed to
M~ll~ the
sentence
was
read
and
the
a n a p h o r encountered.
M c K o o n and g a t o l i f f (1980) used
an
activation
procedure
based
on Chang
(1980).
A desoriptlon
of the
baslo
paradigm
and
its
underlying
loglo
follows.
When one reads a text, only
a
small
part
of
the text i n f o r m a t i o n is
s t o r e d in s h o r t - t e r m m e m o r y and most
of
the
information
is stored in l o n g - t e r m
memory.
This is due to the
very
small
s t o r a g e c a p a c i t y of s h o r t - t e r m m e m o r y (7
t2 chunEs;
Miller,
1956).
Given
that
retrieval
time
in s h o r t - t e r m m e m o r y is
much
faster
than
retrieval
time
in
l o n g - t e r m memory, it will tame longer to
r e m e m b e r s o m e t h i n g from the text if
the
memory
is
stored
in
long-term m e m o r y
than in s h o r t - t e r m memory.
A n o t h e r i n i t i a l l y s u r p r i s i n g effect
was
that
~he old-new r e c o g n i t i o n times
for the r e f e r e n t s
were
slower
in
the
zero
intervening
sentences
when
the
a n a p h o r was a r e p e t i t i o n of the referent
itself
than
when
the
anaphor was the
category
name.
This
last
result
suggests
that
it is not a p p r o p r i a t e to
use a d e f i n i t e noun phrase, e s p e c i a l l y a
repetition
of the referent, to refer to
a a n t e c e d e n t in s h o r t - t e r m memory.
In their
study,
subjects
read
a
paragraph
sentence
by
sentence.
I m m e d i a t e l y after the last sentence, the
subjects
were
presented
with a single
w o r d and the subjects
had
to
remember
w h e t h e r the w o r d had appeared p r e v i o u s l y
in
the
text
or
not
(an
old-new
recognition).
If
the
tested word was
still in s h o r t - t e r m memory, the
old-new
recognition
time
should be faster than
if it was in l o n g - t e r m memory.
As
explained
previously,
intervening
sentences
are not the only
d e v i c e s that transfer
text
units
from
short-term
to
long-term
memory.
Stereotypical
situations
have
spatial
and
temporal
parameters
with
legal
ranges of values.
If
one
specifies
a
spatial
or ~emporal value outside these
ranges, a s c e n a r i o - s h i f t
occurs.
For
example.
Anderson
(in
Sanford
and
Garrodo 1981)
constructed
texts
about
stereotypical
situations
such as going
to a restaurant.
In one sentence of the
text,
there
was
a reference
to
a
c h a r a c t e r related to the script,
say
a
waiter.
AZ
the
b e g i n n i n g of the next
sentence,
there
was
a mention
of
a
temporal
or
spatial parameter, such as
"One hour later" or "Five hours
la~er".
In the
flrs~
case
the
parameter
is
within the range d e f i n i n g the scrip~, in
the
second case it is not.
The rest of
~he sentence c o n t a i n e d an anaphor to the
previously
mentioned
character,
the
walter.
M e a s u m l n g ~he reading
time
of
the
a n a p h o r l o sentence. A n d e r s o n showed
longer reading time when the spatial
or
temporal p a r a m e t e r w~s outside the range
of
the
script
th~n
inside.
This
suggests
that
the
referent
was
t r a n s f e r e d from short-term to
long-term
m e m o r y by the s c e n a r l o - s h l f t and it took
longer ~o retrieve the
referent
during
a n a p h o r a resolution.
To
test
this
hypothesis,
the
paragraphs
were
constructed
in the
f o l l o w i n g manner.
The
referent
(e.g.
burglar)
was s e p a r a t e d from the anaphor
by
either
zero
or
~wo
in~ervenlng
sentences.
The anaphor appeared in the
last sentence
of
the
paragraph.
The
last
sentence
was
p r e s e n t e d in one of
three versions:
i) the subject
of
the
sentence
was
a
repetition
(i.e.
burglar) of the referent
in the
first
sentence
(anaphorio-identioal);
2) the
subject was the
name
of
the
category(e.g.
criminal)
in which the referent
b e l o n g e d (anaphorlc- category);
3)
the
subject was a noun (e.g.
ca~) u n r e l a t e d
~o the referent (non-anaphoric).
During
the
e x p e r i m e n t a l trials, the "referent"
(i.e.
burglar)
was
presented
immediately
after the last sentence for
an o l d - n e w recognition.
A s s u m i n g that an anaphor
activates
its
referent
by m a k i n g it a v a i l a b l e in
short-term
memory,
one
can
expect
s l g n i f l o a n t l y faster old-new r e c o g n i t i o n
times
for
"burglar"
in
the
a n a p h o r l c - c a ~ e g o r y oondi~lon than in the
non-anaphorlo
condition.
This
p r e d i c t i o n was observed.
The
results
from
all
these
experiments
support
the notion tha~ an
anaphor activates its referent by malting
it highly available in s h o r t - t e r m memory
and ~hat
topic
shifts
transfer
units
from
short-term
memory
to
long-term
memoz'y.
However. none of these studles~
except
some
eye
movement
siudles.
provide data on ~
a n a p h o r a resolution
occurs
during the reading of a sentence
~nd when i~ ooou2s in
relation
to the
Surprisingly,
the
number
of
intervening
sentences
did
not have an
effect.
This
suggests
that
the
two
i n t e r v e n i n g sentences did not remove the
referent from
short-term
memory
(i.e.
"backgrounds"
the
referent).
It
is
p r o b a b l y not the case.
Rather.
i~
is
llkely
that
by testing the referent at
2~
lexioal,
pragmatic
syntactic.
analyses.
semantic,
COGNITIVE
BASIS FOR FOCUSING
and
Cache ~
.
During the input
of
a
sentence
into the buffer ~nd the
c o n c u r r e n t i n t e g r a t i o n of
the sentence
into the cache, a subset of the s e m a n t i c
u n i t s h e l d in the STM is s e l e c t e d to
be
held
over
in the
cache
for the next
cycle.
F o l l o w i n g E l n t s c h and
van
Dijk
(1978),
the
cache
management strategy
s e l e c t s a subset T of the
most
topical
items
and a subset R of the most recent
items to be held over in the cache.
The
selection
strategy
aims
at m ~ x l m i z i n ~
the p r o b a b i l i t y that an anaphor
in
the
nex~
sentence
will refer to a semantic
unit
held
in
the
cache.
Cache
management
is
applied
after
each
s e n t e n c e or clause.
A sketch of a cognitive
model
of
a n a p h o r a r e s o l u t i o n is offered here.
It
has b e e n
heavily
influenced
by
the
short-term~long-term
memory
model
of
Kintsch
and
van
DiJk
(19~8)
and
e s p e c i a l l y its leading edge strategy.
~tructure
~f ~
memg/~
Analogically, human memory
can
be
conceptualized
as
a
three
level
s t r u c t u r e similar to the m e m o r y of most
mini
and
main
frame
computers.
It
consists of a small,
very
fast
memory
called
short-term
memory
(STM);
a
relatively
larger
main
or
operating
memory
(OM):
and
a vast
store
of
general w o r l d knowledge called l o n g - t e r m
m e m o r y (LTM).
Pronouns
and
d e f i n i t e noun phrases
r e s o l v e d using different strategies,
will d e s c r i b e four cases:
The total STM is only large
enough
to
c o n t a i n 7t2 chunks of i n f o r m a t i o n at
any
one
time
(Simon,
1974;
Miller.
1956).
The
resources
for
STM
are
dynamically
allocated
to
one
of
two
uses.
First, par~ of the STM is used to
store the incoming sentence
or
clause.
This
is
a temporary
storage
of
the
sentence
or
clause
before
further
processing and is called the STM buffer.
The second part of STM is c a l l e d the STM
cache.
It
is
used to hold over. from
one sentence or clause to the next.
the
information
necessary
to provide local
and global
coherence.
It
contains
a
subset
of
the previous text items that
are topical and a subset of
those
that
are
recent.
Retrieval
times
from
s h o r t - t e r m m e m o r y are v e r y fast.
are
we
i.
The anaphor is a d e f i n i t e
noun
phrase
and the referent is not
in focus, that
is,
i~
is
in
operating memory,
2.
The anaphor is a d e f i n i t e
noun
phrase
and
the referent is in
focus, that is. it
is
in
the
cache.
3.
The anaphor is
a
the
referent
is
(in fOCUS).
4.
The anaphor is
a pronoun
and
the
referent
is
in o p e r a t i n g
m e m o r y (not in focus).
pronoun
and
in the cache
It
is
hypothesized
that
the
explicitness
of sm anaphor is a signal.
used
by
the
readier,
which
denotes
whether
the referent is in the cache or
in o p e r a t i n g memory.
Conceptually. operating
memory
is
the
subset
of
the
w o r l d k n o w l e d g e in
long-term
memory
which
is
deemed
relev~n~
to
the
prooesslng
of the
current
par~
of the
text.
It
also
contains
the
growing
m e m o r y structure
o o r r s s p o n d i n ~ to the tex~ read
so
f~r.
I~
contains
the
less topical and less
recent
information
from
the
text.
Retrieval times are much longer than for
short-term memory.
If the ~naphor is a d e f i n i t e
noun
phrase,
operating
memory
is
searched
immediately.
If the
referent
is
in
operating
memory
it is then r e i n s t a t e d
into
the
cache.
A topic
shift
has
occured.
If the anaphor is a definite
noun
phrase
and
the
referent
is
in focus
(i.e.
in
the
cache),
anaphora
r e s o l u t i o n will be hindered.
The reader
searches
operating
memory
while
the
referent
is
in
short-term
memory.
Correspondingly. this violates a rule of
cooperative
communication:
use
a
d e f i n i t e noun
phrase
to
refer
to
an
~ntecedent
not
in focus.
The d e f i n i t e
noun phrase signals a topic shift, while
in fact. the same entity is being talked
about.
The
time
course
of
anaphora
resolution
is greatly d e t e r m i n e d by the
current content of shor~-term memory and
of operating memory.
Moroever, pronouns
and definite noun phrases
are
resolved
using different s~rategies.
999
TWO
ON-LINE
RESOLUTION
If the anaphor is a pronoun,
the
cache
is
searched
for
a plausible
referent.
If found, mnaphora resolution
is
completed.
Because cache management
is
based
on t o p i o a l l t y
and
recency,
pronouns
can refer to ~he main ~opio of
~he text even when the
main
~opio
has
no~
been
mentioned
directly
for mamy
sentences.
Unless
there
is a global
~opic shift, the main topic in the cache
remains u n c h a n g e d throughout ~he text.
STUDIES
OF
ANAPHORA
The
presented
studies
test
the
notion
tha~
focus
is
cognltively
realized
in
the
reader's
limited
short-term
memory.
They
also
test
Grosz. Joshl. and W e l n s t e l n ' s claim that
d e f i n i t e noun phrases, and not pronouns,
s h o u l d be used
to
refer
~o
items
no
longer
in
focus and ~hat pronouns, ~nd
not d e f i n i t e
noun
phrases,
should
be
used
to
refer
to
items
in
focus.
Moreover,
if
one
assumes
that
the
content
of
short-term
memory
is
dynamically
updated
on
~he
basis
of
recency
~nd topicality, one can explain
why p r o n o u n s can be
used
to
refer
Zo
recent
items
~nd
also
to
topical
n o n - r e c e n ~ items.
If the anaphor is a pronoun but
no
referent
is
found
in the cache, it is
then
necessary
to
search
operating
memory.
If a
referen~
is
found
in
o p e r a t i n g memory, it is reinstated
into
the
c~che.
A ~opic shift has occured.
Using a pronoun ~o refer ~o
information
in
operating
memory
is d e ~ r l m e n t a l ~o
a m a p h o r a resolution.
The
reader
first
searches
the
cache. ~hen ~he operating
memory, and ~hen has
~o
relnst~te
~he
referent into the cache.
A
new
technique,
called
Q~zli~
~ i Q n ,
was
developed specifically
to p r o v i d e the e m p i r i c a l da~a for
these
studies.
The
on-line
activation
t e c h n i q u e can be compared
to
"closely"
t r a c i n g the e x e c u t i o n of ~ program.
COMPARISONS
In
the
on-line
activation
technique,
passages are p r e s e n t e d using
rapid serial visual p r e s e n t a t i o n (RSVP),
one word
a~
a time.
In ~dditlon to
reading each text. the p a r t i c i p a n t s were
also given the ~ask to recognize whether
some s p e c i a l l y m a r k e d
words,
presented
surreptitiously
wi~hln
~he
~ext,
had
a p p e m r e d before
in
the
tex~
or
not.
Some
of
~hese
special
words
were
p r e s e n t e d before in the text and
others
were
not.. We will call ~hese specially
m a r k e d words zest words.
This
task
is
c a l l e d am old-new recognition task.
A clear relation exists between ~he
notion
of
focusing
proposed
in
c o m p u t a t i o n a l l i n g u i s t i c s and the
model
of human memory and d i s c o u r s e processing
proposed in cognitive psychology.
The Q ~ h ~
is
used
to
store
the
items
in f.~Q~.
Given the small number
of items stored in the cache, a sketchy
anaphor
such as ~ ~
is sufflclen~
to retrieve
the
referent.
The
cache
management
strategy
in human .memory is
aimed at m a x i m i z i n g ~he p r o b a b i l i t y that
the
cache
contains
the
information
relevant
to
the
next
cycle
of
computation.
The
cache, by containing
topical
and
recen~
i~ems,
allows
to
maintain global and local coherence.
The
passages
contained
anaphors
referring
~o a n t e c e d e n t s
which
were
either in focus or
not
in
focus.
An
antecedent
was
removed
from
focus by
introducing
a topic
shift,
with
~he
restrlc~ion
that the antecedent was not
the main topic
of
the discourse.
An
example
ter~
is
presented in table I.
Note that only one
of the a l t e r n a t i v e
sentences
5a,
5b.
or 50 was presented
for each text to the participants of the
study.
Q ~
m~y~
is used
~o
store
items
that
are
not in f ~ A ~ .
Because
the
set
of
items
is
large,
an
informative
descrlp~ion
of the Item ~o
be searched
for
is needed.
D~fi~i~
~ou~ ~ h / ~ e s a/e used ~o indlc~te to the
reader ~ha~ the i ~ e m is no~
in
focus.
thus
In operating memory.
Other things
being equal, it will tame more
time
to
retrieve
an
item from operating memory
than from the cache.
The referent
will
need
to
be
reinstated into the cache.
This will produce a topic
shift.
The
reinstated
referent
is then
highly
a v a i l a b l e and
can
be
referred
to by
using a pronoun.
In each text. one of the test words
was
the
referent
of
the anaphor.
At
some point before or after
the
anaphor
was
presented
on the CRT, its referent
was p r e s e n t e d
for
old-new
recognition
and
recognition
times
and errors were
collected.
The d e l a y between the
onse~
of the anaphor and the onset of the test
word
is
called
the
stimulus
onset
a s y n c h r o n y (SOA).
The ~naphor is acting
as a prime, which
should
activate
the
referent.
The old-new recognition time
for the
referent
test
word
indicates
223
whether
the referent
in o p e r a t i n g memory.
is in the cache or
TABLE
2
SCHEMA OF THE P R O C E D U R E
TABLE 1
SOAs
E X A M P L E OF TEXTS WITH A N T E C E D E N T S
IN FOCUS AND NOT IN FOCUS
Antecedent:
Anaphor:
in Focus
1The
assistant
was
preparing
s o l u t i o n s for a c h e m i s t r y experiment.
2The e x p e r i m e n t would take
at
least
four hours.
3There w o u l d then be a ten hour
wait
for the r e a c t i o n to complete.
4He m e a s u r e d
the
temperature
of
a
s o l u t i o n u s i n g a thermometer.
5a- The thin instrument was
not
giving
the e x p e c t e d re~ding.
5b- A broken instrument was
not
giving
the e x p e c t e d reading.
5c- The c o m p u z e r terminal was not giving
the e x p e c t e d reading.
Antecedent
~50 msec
1250 msec
Time
T1
The
The
The
T2
thin
thin
thin
T~ "thermometer* i n s t r u m e n t instrument
T4
i n s t r u m e n t *thermometer"
was
T5
was
was
not
T8
not
not
giving
T7
giving
giving
*thermometer"
thermometer
instrument
Antecedent
Before
The p r e d i c t i o n s
I.
If a referent is not in
focus,
due
to
a topic
shift,
the
o o o u r e n o e of the a n a p h o r should
reinstate t h e referent into the
cache,
leading
to
faster
old-new
r e c o g n i t i o n times.
In
terms
of
the
experimental
conditions,
there
should be a
decrease in old-new r e c o g n i t i o n
time
at
the 350 and 1250 msec
conditions
in
the
S+R÷
condition
(i.e.
after
the
anaphor), but not in
the
S+Rand
S-R- conditions, w h i c h are
not anaphorio.
2.
The
use
of
a definite
noun
phrase
to
refer
to
an
a n t e c e d e n t in the
cache
(i.e.
in focus) should be d e t r i m e n t a l
to a n a p h o r a
resolution.
IZ
should
slow
down
the
r e c o g n i t i o n of the referent
as
old
or
new.
In terms of the
e x ~ e r l m e n t a l conditions, if the
referent
is
in
focus,
the
old-new
recognition
times
in
the
350
and
1250
msec
SOA
conditions
should
be
slower
than
in
the
before
SOA
coD~Litlon.
not in Focus
IThe
assistant
was
preparing
solutions for a c h e m i s t r y experiment.
2- He m e a s u r e d
the
temperature
of a
solution using a thermometer.
3- The e x p e r i m e n t would take
at
least
four hours.
4There w o u l d then be a ten hour
w~it
for the r e a c t i o n to complete.
5a- The thin instrument was
not
givlng
the e x p e c t e d reading.
5b- A b r o k e n instrument was
not
giving
the e x p e c t e d reading.
50- The c o m p u t e r terminal was not giving
the e x p e c t e d reading.
In addition, there were three types
of primes, as shown in sentences 5a, 5b,
8~d 5o in T a b l e i.
The prime
could
be
either
semantically
related
and
referential
(S+R÷)
~
in
5a,
s e m a n t i c a l l y r e l a t e d and not r e f e r e n t i a l
(S+R-)
as
in
5b,
or
semantically
unrelated
and not referential (S-R-) as
in 5c.
In the S÷R÷ condition, the prime
is the an~phor.
The two conditions S÷Rand
S-Rwere
control
conditions
to
s e p a r a t e the effect of semantic priming,
due ~o semantic ~ s s o c i a t i o n between
the
a n a p h o r and the referent, on the old-new
r e c c g n l t l o n for r e f e r e n t s .
were:
Method
ELT.TJ,~,pA~
There
p ~ r t i o i p a n t s in this study.
were
36
~/~I~
There
were
36
e x p e r i m e n t a l texts.
They c o n t a i n e d as a
referent an instance of a cl~ss
(e.g.
thermometer)
to be used later as a test
word, and a~ an an~phor the
class
name
(e.g.
instrument).
In this study, the
a n ~ p h o r w~s a d e f i n i t e noun phrase.
An
e x a m p l e of the m a t e r i a l was p r e s e n t e d in
Table
i.
There
were
three
p~imlng
oondltlons,
S+R+.
S+R-,
and
S-R-,
A schema of the procedure is
shown
in Table
2.
The
words surrounded by
stars a~e the test words.
224
exemplified
respectively
5 a , 5b, a n d 50.
by
focus
and
that
f o c u s is r e a l i z e d i n t o
the cache.
T h e y a l s o s u p p o r t the n o t i o n
that
an
anaphor
reinstates a referent
not in f o c u s a n d d o e s so by t r a n s f e r r i n g
the r @ f e r e n t to the cache.
sentences
During
the
presentation
of
each
text.
two
or
three
test
words
were
p r e s e n t e d , one e x p e r i m e n t a l a n d
one
or
two
fillers.
The
filler
words
were
p r e s e n t e d at
semi-random
locations
in
the
text.
In
the
entire
experiment
t h e r e w a s an e q u a l n u m b e r of old and new
t e s t words.
1345.
L
A
T
E
N
C
I
E
S
~r~re
The
experiment
was
computer-controlled
using
real-tlme
r o u t i n e s on the V A X / V M S
11/780
of
the
Computer
Laboratory
for I n s t r u c t i o n in
P s y c h o l o g i c a l R e s e a r c h at the U n i v e r s i t y
of
Colorado.
Each
p a r t i c i p a n t sat in
f r o n t of a CRT s c r e e n
with
a
keyboard
which
had
a "yes" b u t t o n on the right.
for old test words, a n d a "no" b u t t o n on
the left. for n e w test words.
The t e x ~ s
were presented
using
RSVP.
with
each
word
presented
in
the
center
of the
s c r e e n for 300 msec.
The
participants
w e r e a s k e d to r e c o g n i z e w h e t h e r the test
words
were
old
or
new.
as
fast
as
p o s s i b l e but w i t h o u t m a k i n g m i s t a k e s .
and
1265
m
Not in Focus
1225
~
In Focus
1185
1145
1105
(msec)
1065
1025
S+R+S*R-S-RPRIMING
FIGURE
I.
R e c o g n i t i o n l a t e n c i e s az e a c h
focus and priming condition.
An a p r i o r i c o m p a r i s o n d e m o n s t r a t e s
that
using
a
definite
n o u n p h r a s e ~o
refer
to
an
item
in
focus
hinders
anaphora
resolution.
What
seems
~o
h a p p e n is a s u r p r i z e
effect
caused
by
the
violation
of
a
linguistic
usage
r e l a t i n g the f o r m of the a n a p h o r to
the
fOCUS
S~atus
of
its
referent.
The
r e c o g n i t i o n t i m e for
the
referent,
in
the
focus
c o n d i t i o n , was l o n g e r at the
350 m s e c a n d 1250 m s e c SOAs t h a n in
the
before
SOA.
~(35) - -4.1.
R ~ 0.001.
MSe - 24 by s u b j e c t s , and
~(35) - -2.9,
, 0.008.
MSe - 31
by items.
T h i s is
s h o w n in F i g u r e 2.
D~i~
There were
36
experimental
texts
and
18
experimental conditions.
The first m a n i p u l a t i o n was the
focusing
of
the
referent:
in
f o c u s or not in
focus.
The s e c o n d m a n i p u l a t i o n was
the
SOA:
i m m e d i a t e l y b e f o r e ~he prime. 350
m s e c after. 1250 m s e c after.
The
third
manipulation
was
priming:
S+R+. S+R-.
S-R-.
The
design
was
completely
within-subject,
w i t h two t e x t s r a n d o m l y
a s s i g n e d to e a c h e x p e r i m e n t a l
condition
usin~
two
randomly
sampled
18
by 18
Latin
Squares.
Each
participant
was
randomly
a s s i g n e d to a row of the L a t i n
Squares.
~sul~a
1305.
L
A
T
E
N
C
I
E
D~.scusl~
The
predicted
interaction
of
focusing
and p r i m i n g is s h o w n in F i g u r e
I:
the
prime
in
the
S+R+
condition
(i.e.
the
anaphor)
reinstates
the
r e f e r e n t into t h e
cache,
focusing
it.
while
~he r e f e r e n t is not r e l n s t a z e d in
the
non-referentlal
conditions.
E ( 2 . 7 0 ) = 3.6. ~ , 0.04. MSe = 213~21 by
subjects
and
E ( 2 . 7 0 ) = 2.5,
~ , O.Og,
MSe = 277568
by
items.
A
priori
comparisons
show
that
the
difference
b e t w e e n the r e c o g n i t i c n t i m e s in the two
f o c u s c o n d i t i o n s in ~he
S¢R+
condition
is
much
smaller
than in the o t h e r two
p r i m i n g c o n d i t i o n s , S-R- a n d S-R-. w h i c h
do
mot
differ
between
themselves,
~ ( 3 5 ) ° 2.6.
~ , 0.01.
MSe - 87
by
subjects,
and
~(35) = 2.14.
; , 0.02,
MSe = 114
by
items.
These
resul~s
support
the n o t i o n s that i ~ e m s in f o c u s
are m o r e a c c e s s i b l e t h a n
items
not
in
1345.
1305126512251185-
(msec) 110510651025
F I G U R E 2.
225
•
before 350 1250
SOA
(~sec)
Recognition
la~encies
a~ e a c h
SOA for a r e f e r e n t in focus.
In another study
(Gulndon,
1982),
using
~he
same
on-llne
~c~iva~ion
technique,
the
~c~ivation
of
an
antecedent
by a pronoun was ~raced.
In
this study, it was fo%L~d tha~
referring
~o an anteceden~ not in focus by using a
pronoun
was
detrimental
to
anaphora
resolution.
The
delay between reading
the
anaphor
and
reins~atlng
the
an~eceden~
was
as
long
as 2400 msec.
The a c t l v a ~ i o n of an anteceden~
no~
in
focus
by
a pronoun
takes a long ~ime
because ~he reader is
induced:
I) to
search
the
cache unsuocesfully;
2) to
search operating memory with a "sketchy"
pronoun:
3)
to relnstaZe the referent
into
the
cache.
Activation
was
immediate
for ~he antecedents in focus.
As opposed ~o the previous
s~udy
where
referring
to a focused referen~ using a
d e f i n i t e noun phrase
hindered
anaphora
resolution,
no such effect was observed
when using a pronoun.
This is expected
since
pronouns signal tha~ ~he referent
is in the cache.
the
basis
an~iysls.
of
discourse
or
dialogue
REFERENCES
CarpenZer,
P.A.
~
Just.
M.A.
lnZegraZlve
processes
in
comprehension.
In D.
LaBer~e
~.j.
Samuels
(Eds.),
~i~
P/.OD~ses
i~ E.~,~&L'LD~. Hillsdale,
N.g.:
Erlbaum, 1977.
Chafe, w.
Discourse structure and human
knowledge.
In J.B.
Carroll ~ R.O.
Freedle
"(Eds.),
L ~ n ~
co~mmhmna~mn ~n~ ~hm a m g ~ l ~ i m n
~ i ~ .
1972.
Washington:
of
Winston.
Chang, F.
AcZive
memory
processes
in
sentence
comprehension:
Clause
effects
and
pronominal
reference.
~m~
~n~ G ~ n i ~ n ,
1980, 8, 58 64.
SUMMARY
The
notion
of
focusing
and
the
notion
that
the
form
of
the anaphor
signals whether ~he referen~ is in focus
or no~ have cognitive support.
Items in
focus are items in the
cache
which
is
dynamically
updated
~o
contain
~he T
most ~opical and the R most recen~ items
in the ~ex~.
Because the cache con~alns
few items, pronouns should
be
used
~o
refer
to
items in focus.
O~her things
being equal, anaphora resolution will be
easier
if the
antecedent is in focus,
because ~he
retrieval
~imes
from
the
cache
are
much
faster ~han those from
the operating memory.
Antecedents
not
in focus are in operating memory.
I~ems
no~ in focus are in operating memory.
A
definite noun phrase, because it is more
descriptive ~han a pronoun,
should
be
used to re~rieve the ~nteceden~ from ~he
large set of i~ems in operating
memory.
However,
because
~he reErleval ~ime is
slow
in
opera~in~
memory,
anaphora
resolution
is more dlfflcul~ for i~ems
~ha~
are
no~
in
focus.
The
relns~a~emen~
of am an~eceden~ into ~he
cache effects a ~oplc shift.
Clark, H.H., ~ Sengul. C.J.
In
search
of referents for nouns and pronouns.
~m~=~
~n~ C_Q~nl~i~n. 1 9 7 9 , Z, 35
~I.
van DiJk, T.A. ~ Kintsch. W.
~X~gi~S
~f
~l~cou=g2
G~m~x~h~si~a.
New
YorE:
A c a d e m i c Press, 1963.
af
focus in ~ i ~ l a ~ u m ~ n ~ n G i n ~ .
Technical
No~e
151.
Artificial
InZeiligence Center, SRI, 1977.
Grosz, B.J.. Joshi,
S.
Providing
~
~
nmm
A.K.,
~
a unified
WeinsZein.
account of
in
~lggm~na~.
~
Technical
No~e
~92,
In~elligence Center, SRI,
Gulndon,
R.
Q~=Ii~
~oc~ing
~i~s~Sm~3
searcheS.
manuscript.
Universl~y
Boulder, 1962.
Guindon,
The
on-llne
activation
~echnique
was
developed
specifically
to provide
empirical data on the no~ion
of
focus.
The
~dvan~age
of
this
technique over
conventional memory experiments is that
one
can
~est
precisely
the
~emporal
properties
of various
analyses
and
processes
occurring durln~ sentence and
ter~ comprehension.
This technique
can
be used to distinguish between different
models of anaphora resolution when ~hese
models
are
no~ easily distinguished on
R.
~hz ~ff~ct
Artificial
1983.
of
~f
Unpublished
of Colorado.
re~en~
~n~
Doctoral
Dissertation.
University
of Colorado, Boulder. 198~.
Just, M.A. ~ Carpenter. P.A.
A theory
of
rea~ing:
From eye fixations to
comprehension.
~chologi~l
Ke.J.l~, IgBO. ~Z, 329 - 3S4.
E~z,
226
J.J. ~ Fodor, J.A.
The
of
a sem~nZlc
~heory.
1963, ~ ,
170 - 210.
structure
L~uEu~g~.
Ein%scho W. ~ van DiJk, T.A.
Toward
a
model
of %ex~
comprehension
and
production.
~
Review,
1978. 85, 363 - 394.
LasniE.
H.
LinglL~
Remarks
on co-reference.
An~l~.~is, 1976, ~, 1-22.
Lesgold. A.M..
Ro~h,
S.F..
~ Curtis,
M.E.
Foregrounding
effects
in
discourse comprehension.
~rnal
of
Ver~a~
Le~Luin~
~ n s l o n ,
1979,
~
i~,
Ver~l
281-
308.
McEoon.
G.
~
Ra%Ollff.
R.
The
comprehension
processes
and memory
s%ruc~ures
involved
in
anaphorlc
references.
Journ~l
of
Y~I
L~mnlng
~n~ Y~=L~I ~;~,.V.i.~,
1980.
19, 6 6 8 - 6 8 2 .
Miller. G.A.
The magical n,~mher
seven.
plus
or minus ~wo:
Some llmi~s on
our
capacity
for
processing
informaZion.
~l~gl~l
E~Xi~.
1956, ~ ,
81 - 9 7 .
Reichman. R.
C~g~i~ix~
Reichman,
Conversational
~i.~,
19?8,
R.
Ex~ended
in%erface.
1884.
Sanford,
~,
person-machlne
& ~ l
Zn~elllEenoe.
157 - 218.
A.J.
~
~n~fi~n~in~
~i~J~n
York:
1961.
Wiley,
~heory
coherency,
~, 283-327.
of
Garrod.
I.~g,~&g~.
definite
S.C.
New
anaphor&
~fih~nsion
i= English
~L~g~urse.
Technical
repor~
537.
MIT
Ar%ificlal In~elllgenoe
Laboratory,
C~mhrldge MA, 1979.
Sidner.
C.
comprehension
In M.
Brady
(Eds.).
~iH~g~.
1983.
Simon,
E.A.
~Ql~fi,
Focusing
in
%he
of definite anaphora.
and
R.
C.
Berwlck
com~tQn~l
Cambridge:
~
MIT
of
Press.
Eow
big
is
a
chunk?
1974. IE~, 482 - 488.
ACKNOWLEDGMENT
Thls research was performed as par%
of
the auZhor's doctoral dissertation while
a~ ~he University of Colorado.
She
is
ex%remely
grateful
for ~he help of her
d i s s e r t a t i o n oommlZ~ee. Wal~er
EinZsch.
Peter
Polson,
Alice
Healy.
Richard
Olson.
AndrzeJ
Ehrenfeuch~.
Bur%on
Wagner
has
provided
many
insightful
comments on this paper.
MCC
is kindly
~hanked
for
~he
technlcal
suppor%
p r o v i d e d while oomposlng ~hls paper.
227