Cognitive Brain Research 15 (2002) 31–45
www.bres-interactive.com
Interactive report
Attentional control of the processing of neutral and emotional stimuli
Luiz Pessoa a , *, Sabine Kastner b , Leslie G. Ungerleider a
a
Laboratory of Brain and Cognition, National Institute of Mental Health, 49 Convent Drive, Building 49, Room 1 B80, Bethesda,
MD 20892 -4415, USA
b
Department of Psychology, Center for the Study of Brain, Mind and Behavior, Princeton University, Princeton, NJ, USA
8 September 2002
Abstract
A typical scene contains many different objects that compete for neural representation due to the limited processing capacity of the
visual system. At the neural level, competition among multiple stimuli is evidenced by the mutual suppression of their visually evoked
responses and occurs most strongly at the level of the receptive field. The competition among multiple objects can be biased by both
bottom-up sensory-driven mechanisms and top-down influences, such as selective attention. Functional brain imaging studies reveal that
biasing signals due to selective attention can modulate neural activity in visual cortex not only in the presence but also in the absence of
visual stimulation. Although the competition among stimuli for representation is ultimately resolved within visual cortex, the source of
top-down biasing signals likely derives from a distributed network of areas in frontal and parietal cortex. Competition suggests that once
attentional resources are depleted, no further processing is possible. Yet, existing data suggest that emotional stimuli activate brain regions
‘automatically,’ largely immune from attentional control. We tested the alternative possibility, namely, that the neural processing of
stimuli with emotional content is not automatic and instead requires some degree of attention. Our results revealed that, contrary to the
prevailing view, all brain regions responding differentially to emotional faces, including the amygdala, did so only when sufficient
attentional resources were available to process the faces. Thus, similar to the processing of other stimulus categories, the processing of
facial expression is under top-down control.
Published by Elsevier Science B.V.
Theme: Neural basis of behavior
Topic: Cognition
Keywords: fMRI; Attention; Executive function; Emotion; Amygdala
1. Introduction
As memorably put by William James, attention allows
‘‘possession of the mind, in clear and vivid form, of one
out of what seems several simultaneously possible objects
or trains of thought. Focalization, concentration of consciousness are of its essence. It implies withdrawal from
some things in order to deal effectively with others’’ [38]
(p. 403). Since the time of James, we have greatly
advanced his penetrating insights and now have a fairly
good understanding of how attention modulates brain
processes, as revealed by the recording of single-cell
activity, event-related potentials, and hemodynamic events
*Corresponding author. Tel.: 11-301-496-5625x275; fax: 11-301402-0046.
E-mail address: [email protected] (L. Pessoa).
0926-6410 / 02 / $ – see front matter
PII: S0926-6410( 02 )00214-8
Published by Elsevier Science B.V.
in neuroimaging studies. Although the regions affected by
attention have been extensively studied in the past decade,
much less is known about the brain areas that control
attention. The areas that express the effects of attention are
typically sensory processing areas (e.g. visual cortex). By
contrast, the areas that control this expression appear to be
prefrontal and parietal regions. However, less is known
about the mechanisms by which these regions control the
flow of processing within the regions affected by attention.
Because processing capacity is limited, selective attention to one part of the visual field comes at the cost of
neglecting other parts. Thus, several investigators have
proposed that there is competition for neural resources
[7,20–23,32,34]. In the present paper, we will adopt the
framework advanced by the biased competition model of
attention, as developed by Desimone and Duncan [21].
According to this model, illustrated in Fig. 1, the competi-
32
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
stimuli undergo competition for neural resources, as do
neutral ones, or whether they comprise a special category
of stimuli that may be processed ‘automatically’.
2. Biased competition model of attention
2.1. Sensory interactions among multiple visual stimuli
Fig. 1. Biased competition model of visual attention. In a typical visual
scene that contains many different objects, there exists competition
among those objects for neural representation due to the limited processing resources of the visual system. The competition among stimuli can be
biased in several ways. One way is by bottom-up sensory-driven
mechanisms, such as stimulus salience. Another way is by attentional
top-down feedback, which is hypothesized to be generated in areas
outside the visual cortex. Stimuli that win the competition for neural
representation will have further access to memory systems for mnemonic
encoding and retrieval and to motor systems for guiding action and
behavior.
tion among stimuli for neural representation, which occurs
within visual cortex itself, can be biased in several ways.
One way is by bottom-up sensory-driven mechanisms,
such as stimulus salience. For example, stimuli that are
colorful or of high contrast will be at a competitive
advantage. But, another way is by attentional top-down
feedback, which is generated in areas outside the visual
cortex. For example, directed attention to a particular
location in space facilitates processing of stimuli presented
at that location. In this way, even objects that are not
physically salient may win the competition and influence
on-going behavior. The stimulus that wins the competition
for neural representation will have further access to
memory systems for mnemonic encoding and retrieval and
to motor systems for guiding action and behavior. It is
interesting to note that the architecture shown in Fig. 1 was
originally suggested not only to explain attentional phenomena, but also to account for how representations of
sensory events are coded by the brain in a stable, durable
manner [32,33].
In the following sections, we will first describe the
evidence for competition among multiple visual stimuli for
neural representation and then describe how attention
biases the competition, thereby filtering out unwanted
information. Next, we will discuss top-down attentional
control, focusing on behavioral studies of patients with
brain lesions and neuroimaging studies of normal individuals. Finally, we will address how attention interacts
with the processing of stimuli with emotional valence.
Specifically, we will discuss whether emotion-laden
What are the neural correlates for competitive interactions among multiple objects in the visual field? Single-cell
recording studies in the monkey have shed light on this
question by comparing responses to a single visual
stimulus presented alone in a neuron’s receptive field (RF)
with the responses to the same stimulus when a second one
is presented simultaneously within the same RF [67,85].
When the monkey attended to stimulus outside the RF, it
has been shown that the responses to the paired stimuli
within the RF were a weighted average of the responses to
the individual stimuli when presented alone. For example,
if a single effective stimulus elicited a high firing rate and
a single ineffective stimulus elicited a low firing rate, the
response to the paired stimuli was reduced compared to
that elicited by the single effective stimulus. This result
indicates that two stimuli present at the same time within a
neuron’s RF are not processed independently, for, if they
were, then the responses to the two stimuli when presented
together would have summed. Rather, the reduced response to the paired stimuli suggests that the two stimuli
within the RF interacted with each other in a mutually
suppressive way. This sensory suppressive interaction
among multiple stimuli has been interpreted as an expression of competition for neural representation. Sensory
suppression among multiple stimuli present at the same
time in the visual field has been found in several areas of
the visual cortex, including extrastriate areas V2, V4, the
middle temporal (MT) and medial superior temporal
(MST) areas, and the inferior temporal cortex
[63,67,81,85,87,91].
Based on hypotheses derived from these monkey physiology studies, we examined competitive interactions
among multiple stimuli in the human cortex using functional magnetic resonance imaging (fMRI) [42]. In these
studies, hemodynamic changes, as measured by fMRI,
were used as indirect measures of neural activity [4,49,72].
Complex, colorful visual stimuli, known to evoke robust
responses in ventral stream visual areas of the monkey
brain, were presented eccentrically in four nearby locations
of the upper right quadrant of the visual field, while
subjects maintained fixation. Fixation was ensured by
having subjects count the occurrences of Ts or Ls at
fixation in a stream of Ts and Ls, an attentionally
demanding task. The stimuli were presented under two
different presentation conditions, sequential and simultaneous. In the sequential presentation condition (Fig. 2A), a
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
33
Fig. 2. Experimental design. Four complex images (2328) were presented in nearby locations at an eccentricity of 6–108 from a fixation point (where the
‘L’ is located) to the upper right quadrant in two presentation conditions: sequential (A) and simultaneous (B). Presentation time was 250 ms followed by a
blank period of 750 ms, on average, in each location. A stimulation period of 1 s is shown, which was repeated in blocks of 18 s. Integrated over time, the
physical stimulation parameters in the two presentation conditions were identical in each location. But competitive interactions among stimuli could only
take place in the simultaneous, not in the sequential condition.
single stimulus appeared in one of the four locations, then
another appeared in a different location, and so on, until
each of the four stimuli had been presented in the four
different locations. In the simultaneous presentation condition (Fig. 2B), the same four stimuli appeared in the
same four locations, but they were presented together.
Thus, integrated over time, the physical stimulation parameters were identical in each of the four locations in the two
presentation conditions. However, sensory suppression
among stimuli within RFs could take place only in the
simultaneous, not in the sequential presentation condition.
Sequential and simultaneous conditions were presented
in blocks interleaved with blank periods, during which
time subjects continued to count the occurrences of Ts or
Ls but no stimuli were presented. As predicted by our
hypothesis that stimuli presented together interact in a
mutually suppressive way, simultaneous presentations
evoked weaker responses than sequential presentations in
all activated visual areas, which included V1, V2, V4, TEO,
V3A and the MT complex (hereafter called area MT). This
is illustrated for V2 and V4 in Fig. 3A. Importantly, the
difference in activations between sequential and simultaneous presentations was smallest in V1 and increased in
magnitude towards ventral extrastriate areas V4 (Fig. 3A)
and TEO, and dorsal extrastriate areas V3A and MT. This
increase in magnitude of the sensory suppressive effects
across visual areas suggests that the sensory interactions
were scaled to the increase in RF size of neurons within
these areas. That is, the small RFs of neurons in V1 and V2
would encompass only a small portion of the visual
display, whereas the larger RFs of neurons in V4, TEO,
V3A and MT would encompass all four stimuli. Therefore,
suppressive interactions among the stimuli within RFs
could take place most effectively in these more anterior
extrastriate visual areas (see also [43]).
2.2. Attention biases competition: filtering of unwanted
information
Single-cell recording studies in the monkey have demonstrated that spatially directed attention can bias the
competition among multiple stimuli in favor of one of the
stimuli by modulating competitive interactions. In particular, in extrastriate areas V2 and V4 it has been shown
that spatially directed attention to an effective stimulus
within a neuron’s RF eliminates the suppressive influence
of a second, ineffective stimulus presented within the same
RF. For example, if attention is directed to the effective
stimulus, it is as if the ineffective stimulus were not in the
RF. When a monkey directs attention to one of two
competing stimuli within a RF, the response is similar to
the response to that stimulus presented alone [85]. These
findings imply that attention may resolve the competition
among multiple stimuli by counteracting the suppressive
influences of nearby stimuli, thereby enhancing information processing at the attended location. This may be an
important mechanism by which attention filters out unwanted information from cluttered visual scenes [20,21].
Our recent fMRI studies suggest that a similar mechanism operates in the human visual cortex [42]. We studied
the effects of spatially directed attention on multiple
competing visual stimuli in a variation of the paradigm we
used to examine competitive interactions among simultaneously presented stimuli, described above. In addition to
the two different visual presentation conditions, sequential
and simultaneous (Fig. 2), two different attentional con-
34
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
Fig. 3. Sensory suppression and attentional modulation in human visual cortex. (A) Sensory suppression in V2 and V4. As shown by the time series of
fMRI signals, simultaneously presented stimuli (SIM) evoked less activity than sequentially presented stimuli (SEQ). (B) Attentional modulation of
sensory suppression. The sensory suppression effect was replicated in the unattended condition of this experiment, when the subjects’ attention was
directed away from the stimulus display (unshaded time series). Spatially directed attention (blue shaded time series) increased responses to simultaneously
presented stimuli to a larger degree than to sequentially presented ones (see arrows).
ditions were tested, such that the eccentrically presented
stimuli were either unattended or attended. During the
unattended condition, attention was directed away from the
visual display by having subjects count Ts or Ls at
fixation, exactly as in our original study. In the attended
condition, subjects were instructed to maintain fixation and
attend covertly to the peripheral stimulus location closest
to fixation in the display and to count the occurrences of
one of the four stimuli presented there, which was indicated before the scan started.
The same areas in striate and extrastriate cortex were
activated during both the unattended and attended condition, namely, V1, V2, V4, TEO, V3A, and MT. However,
in the attended condition, the extent of activation increased
significantly in V4, TEO, V3A, and MT. As illustrated in
Fig. 3B, for area V4, directing attention to the location
closest to fixation in the display enhanced responses to
both the sequentially and the simultaneously presented
stimuli. More importantly, and in accordance with our
prediction from monkey physiology, directed attention led
to greater increases of fMRI signals to simultaneously
presented stimuli than to sequentially presented stimuli.
Additionally, the magnitude of the attentional effect scaled
with the magnitude of the suppressive interactions among
stimuli, with the strongest reduction of suppression occurring in ventral extrastriate areas V4 (Fig. 3B) and TEO,
suggesting that the effects scaled with RF size. These
findings support the idea that directed attention enhances
information processing of stimuli at the attended location
by counteracting suppression induced by nearby stimuli,
which compete for limited processing resources. In essence, unwanted distracting information is effectively filtered out.
3. Top-down attentional control
In humans, studies of patients suffering from attentional
deficits due to brain damage, as well as functional brain
imaging studies of healthy individuals performing attentionally demanding tasks, have given insights into a
distributed network of higher-order areas in frontal and
parietal cortex that appears to be involved in the generation
and control of attentional top-down feedback signals, as
proposed by the biased competition model. Furthermore,
there exists an anatomical substrate for such top-down
influences (Fig. 4), inasmuch as tract-tracing studies in
monkeys have demonstrated direct feedback projections to
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
35
form a distributed network for spatially directed attention
[62,78].
3.2. Functional brain imaging studies
Fig. 4. Anatomical substrate for top-down influences. Cortical visual
processing, as outlined on a lateral view of a monkey brain, originates in
the primary visual cortex (V1) and proceeds both ventrally and dorsally
to temporal and parietal regions, respectively, before converging in
prefrontal cortex. Red arrows indicate potential feedback connections that
might provide the anatomical substrate for top-down attentional effects.
Abbreviations: PFC, prefrontal cortex; PG, inferior parietal cortex; TE,
anterior inferior temporal area; TEO, temporal–occipital area.
extrastriate visual areas V4 and TEO from parietal cortex
(area PG) and to inferior temporal cortex (area TE) from
prefrontal cortex (PFC), as well as indirect feedback
projections to areas V4 and TEO from prefrontal cortex via
parietal cortex [9,98,105].
3.1. Lesion studies
Lesions of the right cerebral hemisphere may produce
the syndrome of visuospatial neglect: although visual
sensation is intact, patients fail to detect stimuli in the side
of space opposite the lesion, and they are not consciously
aware of contralesional objects or parts of objects [62,80].
For example, a patient will read from one side of a book,
apply make-up to only one half of her face, or eat from
only one side of a plate. Patients with visuospatial neglect
typically exhibit extinction. Detection reaction time in the
contralesional field is not significantly slowed if a valid
cue is given. When, however, a cue draws attention to the
ispilesional field and the target subsequently appears in the
opposite, contralesional field, then detection time is slowed
dramatically. This pattern of results (i.e. extinction) is
often interpreted as a deficit in one of the proposed
elementary operations of attention [78], namely, disengagement.
Visuospatial neglect may follow unilateral lesions at
very different sites, including the parietal lobe, especially
its inferior part and the temporo-parietal junction [99],
regions of the frontal lobe [17,36], the anterior cingulate
cortex [39], the basal ganglia [17] and the thalamus, in
particular, the pulvinar [104]. Studies in monkeys have
implicated the same brain regions [28,53,61,77,103,108].
The finding that lesions of many different areas may cause
visuospatial neglect has led to the notion that these areas
3.2.1. Meta-analysis
Neuroimaging studies of visual attention have helped to
elucidate how the processing of visual information is
enhanced for attended compared to unattended information. In addition to examining activations within visual
cortex, it is also informative to examine whether other
brain regions are routinely recruited by attentional tasks.
Interestingly, a fronto-parietal network of regions consisting of areas in the superior parietal lobule (SPL), the
frontal eye field (FEF), and the supplementary eye field
(SEF) has been consistently activated in a variety of tasks
involving
visuospatial
attention
[12,15,16,26,29,48,71,88,100]. In addition, but less consistently, activations in the inferior parietal lobule (IPL),
the lateral prefrontal cortex in the region of the middle
frontal gyrus (MFG), and the anterior cingulate cortex
have been reported. Fig. 5 illustrates the results of a
meta-analysis of foci of activation from ten studies across
several independent laboratories. A common feature
among the visuospatial tasks in these experiments is that
subjects were asked to maintain fixation at a central
fixation point and to direct attention covertly to peripheral
target locations in order to detect a stimulus
[12,15,29,48,71,88], to discriminate it [26,44,100] or to
track its movement [16]. Thus, there appears to be a
general spatial attention network that operates independently of the specific requirements of the visuospatial task.
3.2.2. Isolating ‘ cue’ and ‘ target’ activity
The approach of analyzing the cortical response to a cue
as a tool to study control mechanisms of visuospatial
attention was first used in event-related potentials by
Harter and colleagues [33a]. Subsequently, neuroimaging
studies of visual attention used a similar approach to
elucidate how the processing of visual information is
increased for attended compared to unattended information
(e.g. Ref. [14]). In many of these studies, however, it was
not possible to separate signals associated with visual cues
that prime the subject to expect potential subsequent visual
targets from signals associated with the attended targets
themselves, because cues and targets follow each other in
rapid succession. More recent neuroimaging studies, however, have attempted to explicitly investigate top-down
modulation in attentional paradigms by disentangling cueand target-related activity by, for instance, introducing a
longer interval between the two [35,44]. In this way, the
effects of attention in the presence and in the absence of
visual stimulation can be assessed. The reasoning is that
purely target-related activity should be observed in visual
processing areas that respond to the specific stimulus (e.g.
MT to moving stimuli). At the same time, expectation- or
36
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
Fig. 5. Meta-analysis of studies investigating the spatial attention network. Talairach (peak) coordinates of activated areas in parietal and frontal cortex
from the following studies are indicated: (1) Corbetta et al. (1993) [15]; (2) Fink et al. (1997) [26]; (3) Nobre et al. (1997) [71]; (4) Vandenberghe et al.
(1997) [100]; (5) Corbetta et al. (1998) [12]; (6) Culham et al. (1998) [16]; (7) Kastner et al. (1999) [42]; (8) Rosen et al. (1999) [88]; (9) Corbetta et al.
(2000) [13]; (10) Hopfinger et al. (2000) [37]. Abbreviations: ACC, anterior cingulate cortex; FEF, frontal eye field; IPS, intraparietal sulcus; MFG, middle
frontal gyrus; SEF, supplementary eye field; SPL, superior parietal lobule. Based on Kastner and Ungerleider [46].
cue-related activity that is uncontaminated by ensuing
target-related activity should reflect mainly top-down
signals and be observed in attentional control regions of
the brain. We will next describe one of our studies and then
review a few related ones.
We explored attentional biasing signals in the human
visual cortex in the absence of visual stimulation by adding
a third experimental condition to the design used to
investigate competitive interactions and their modulation
by attention [44]. In addition to the two visual presentation
conditions, sequential and simultaneous, and the two
attentional conditions, unattended and attended, an expectation period preceding the attended presentations was
introduced. The expectation period was initiated by a
marker (the cue) presented briefly next to the fixation point
11 s before the onset of the stimuli. At the appearance of
the marker, subjects covertly shifted attention to the
peripheral target location in anticipation of a target
stimulus that would appear there. In this way, the effects of
attention in the presence and absence of visual stimulation
could be identified (Fig. 6).
Directed attention in the absence of visual stimulation
activated the same distributed network of areas as directed
attention in the presence of visual stimulation and consisted of the FEF, the SEF, and the SPL. A time course
analysis of the fMRI signals revealed that there was an
increase in activity in these frontal and parietal areas
during the expectation period (in the absence of visual
input), with no further increase in activity evoked by the
attended stimulus. Rather, there was sustained activity
throughout the expectation period and the attended visual
presentations. These results suggest that the activity reflected the attentional operations of the task per se and not
the effects of attention on visual processing. This conclusion is supported by the finding that, in the unattended
condition, no significant visually evoked activity was
observed in these frontal and parietal regions.
Additional evidence for a fronto-parietal network of
regions involved in attentional control comes from three
additional imaging studies. By using a spatial attention
Posner-type task, Corbetta et al. [13] showed that the
intraparietal sulcus (IPS) was uniquely active when attention was directed toward and maintained at a relevant
location (preceding target presentation), suggesting that the
IPS is a top-down source of biasing signals observed in
visual cortex. The same investigators [92] found, when
studying attention to motion, cue-related activity in the
intersection of the precentral sulcus and the superior
frontal sulcus (likely including the FEF), as well as in
several sites in the IPS. Finally, Hopfinger et al. [37]
obtained evidence for a wider attentional control network,
including the left superior frontal gyrus, bilateral midfrontal gyrus, bilateral SPL, bilateral IPS, as well as bilateral
superior temporal gyrus (this latter region was also observed by Corbetta et al. [13]).
Taken together, our study and the ones just cited provide
evidence that a distributed fronto-parietal attentional network may be the source of feedback that generates the
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
37
Fig. 6. A fronto-parietal network subserving biased competition and spatially directed attention. Axial slice through frontal and parietal cortex (see small
brain inset for the plane of the section). Left: Visual stimulation did not activate frontal or parietal cortex reliably when attention was directed elsewhere in
the visual field. Middle: When the subject directed attention to a peripheral target location and performed an object discrimination task, a distributed
fronto-parietal network was activated including the supplementary eye field (SEF), the frontal eye field (FEF) and the superior parietal lobule (SPL). Right:
The same network of frontal and parietal areas was activated when the subject directed attention to the peripheral target location in the expectation of the
stimulus onset, that is, in the absence of any visual input whatsoever. This activity therefore may not reflect attentional modulation of visually evoked
responses, but rather attentional control operations themselves. L: left hemisphere; R: right hemisphere.
top-down biasing signals modulating activity in visual
cortex. Because, in our study, the magnitude of the activity
in the parietal and frontal areas was comparable during
directed attention in the absence and in the presence of
visual stimulation, it appears that this activity may be
independent of the particular visual task, be it detection or
discrimination. This would explain the finding that functional brain imaging studies using different visuospatial
attention tasks have described very similar attentional
networks. The network revealed by imaging studies exhibits great overlap with the set of regions implicated in
visuospatial neglect in studies of patients with brain lesions
[62,80].
Corbetta and Shulman [15a] have recently proposed two
anatomically segregated but interacting networks for spatial attention. According to their scheme, a dorsal frontoparietal system is involved in the generation of attentional
sets associated with goal-directed stimulus–response selection. Key nodes within this largely bilateral network
include the IPS / SPL and the FEF. A second, ventral
system, which is strongly lateralized to the right hemisphere, is proposed to detect behaviorally relevant stimuli
and to work as an alerting mechanism for the first system
when these stimuli are detected outside the focus of
processing. This latter network is thought to involve the
temporo-parietal junction (at the intersection of the inferior
parietal lobule and the superior temporal gyrus) and the
middle and inferior frontal gyri. Overall, the dorsal and
ventral networks can be thought of as subserving, respectively, endogenous and exogenous spatial attention functions. Although previous studies explicitly comparing these
two functions revealed largely overlapping networks (e.g.
Refs. [48,88], because of the interacting nature of the two
networks, more subtle event-related designs may be required to reveal the anatomical specificities of the two
systems.
3.2.3. Increases of baseline activity
What is the evidence that the top-down biasing signals
generated in frontal and parietal areas produce a change
within visual cortex so that visually evoked activity there
is enhanced? Single-cell recording studies have shown that
spontaneous (baseline) firing rates are 30–40% higher for
neurons in areas V2 and V4 when a monkey is cued to
attend covertly to a location within the neuron’s RF before
the stimulus is presented there; that is, in the absence of
visual stimulation [60]. A similar effect was demonstrated
in dorsal stream area LIP [11]. This increased baseline
activity, termed the ‘baseline shift’, has been interpreted as
a direct demonstration of a top-down signal that feeds back
from higher-order control areas to lower-order processing
areas. In the latter areas, stimuli at attended locations
38
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
would be biased to ‘win’ the competition for processing
resources at the expense of stimuli appearing at unattended
locations [7,20–23,34]. Such a shift in baseline activity
would ‘sensitize’ neurons with RFs at the attended location, so that when a stimulus subsequently appeared at that
location there would be enhanced visually evoked activity.
In our study in which we introduced an expectation
period, described above [44], in addition to the observed
frontal and parietal activations, we found that visual
processing areas were also activated. This activity was
related to directing attention to the target location in the
absence of visual stimulation, in anticipation of the target
stimulus that would appear there. Notably, the increase in
activity during these expectation periods was topographically specific, inasmuch as it was only seen in areas with a
representation of the attended spatial location, specifically,
in areas V1, V2, V4, and TEO. The increase of baseline
activity during the expectation period was followed by a
further increase of activity evoked by the onset of the
stimulus presentations.
The baseline increases found in human visual cortex
[10,44,45,92] may be subserved by increases in spontaneous firing rate similar to those found in the single-cell
recording studies [11,60], but summed over large populations of neurons. The increases evoked by directing
attention to a target location in anticipation of a behaviorally relevant stimulus at that attended location are
thus likely to reflect a top-down feedback bias in favor of
the attended location in human visual cortex.
4. Attention and the processing of emotion-laden
stimuli
Thus far, we have considered the neural basis for the
control of the processing of neutral stimuli, that is, those
without emotional valence. But, in addition to neutral
stimuli, others can be considered as either positive (associated with appetitive behaviors) or negative (associated with
withdrawal behaviors). We now turn to issues raised by
studying the processing of visual stimuli with emotional
content and, in particular, how stimulus valence and
attention interact.
4.1. Is attention necessary for the processing of faces
with emotional expression?
To what extent are unattended objects processed by the
visual system? Psychophysical evidence suggests that
processing outside the focus of attention is attenuated and
may even be eliminated under some conditions [54,75].
For example, ‘change blindness’ studies show that subjects
may fail to report even very large changes in complex
scenes, provided the changes are not associated with
stimulus transients that capture attention [84,93]. Even
low-level tasks commonly thought to be ‘preattentive’,
such as orientation pop-out, may require attention to be
successfully performed [41]. Likewise, at the neural level,
the stimulus-evoked fMRI response is essentially eliminated when subjects are engaged in a competing task with
high attentional load. For example, moving stimuli did not
elicit fMRI activation in area MT when subjects performed
a concurrent, highly demanding linguistic task [82], and
activations associated with words were not observed when
subjects performed a concurrent, highly demanding object
working memory task [83]. Taken together, these studies
suggest that perception and its underlying neural substrate
may be abolished if attentional resources are completely
consumed by a competing task.
A major exception to this critical role of attention may
be the neural processing of emotion-laden stimuli, which
are reported to be processed automatically, namely, without attention [73,74,101]. For example, subjects exhibit
fast, involuntary autonomic responses to emotional stimuli,
such as aversive pictures or faces with fearful expressions
[30,109]. Other behavioral studies suggest that the visual
processing of facial expression occurs not only automatically [94] but may even take place without conscious
awareness [73]. This conclusion is also supported by
imaging studies of the neural processing of emotional
stimuli in the amygdala, a structure that is known to be
important in the processing of emotion, particularly fear
[1,51,57]. Such studies report that the amygdala is activated not only when normal subjects view fearful faces,
but even when these stimuli are masked and subjects
appear to be unaware of their occurrence [69,110]. The
view has thus emerged that the amygdala is specialized for
the fast detection of emotionally relevant stimuli in the
environment, and that this can occur without attention and
even without conscious awareness.
In a recent study, we tested the alternative possibility,
namely, that the neural processing of stimuli with emotional content is not automatic and instead requires some
degree of attention, similar to the processing of other
stimulus categories. We hypothesized that the failure to
modulate the processing of emotional stimuli by attention
in previous studies was due to a failure to fully engage
attention by a competing task. In other words, activation in
the amygdala by emotional stimuli should resemble activation in MT; if the competing task is of high load, activation
should be reduced or absent. We therefore employed fMRI
and measured activations in the amygdala and other brain
regions that responded differentially to faces with emotional expressions (fearful and happy) compared to neutral
faces and then examined how these responses were modulated by attention.
We measured fMRI responses evoked by pictures of
faces with fearful, happy, or neutral expressions when
attention was focused on them (attended condition), and
compared the responses evoked by the same stimuli when
attention was directed to oriented bars (unattended condition). In alternating blocks of trials, subjects performed a
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
Fig. 7. Experimental paradigm to study the interaction of stimulus
valence and attention. While subjects fixated the faces, they indicated in
alternating blocks of trials, either whether the face was male or female
(‘attended’ trials), or whether the bars were or were not of similar
orientations (‘unattended’ trials); the dashed lines indicate the display
regions attended on alternating blocks (not shown on actual displays). On
each trial, the faces were fearful, happy, or neutral. Stimuli are not drawn
to scale.
39
gender discrimination task on the faces and a same / different orientation discrimination task on the eccentric bars
(Fig. 7). On all trials, subjects fixated centrally on the
faces, attending only covertly to the eccentrically presented
bars on the unattended trials. In designing our bar orientation task, we chose one that was sufficiently demanding to
exhaust all attentional resources on that task and leave
little or none available to focus on the faces, even though
they were viewed foveally during the bar orientation task.
Behavioral performance indicated that the bar orientation
task was indeed extremely difficult, inasmuch as subjects
were correct only on 64% of the trials.
As shown in previous studies [65,66,73,101,103], fearful
faces produced greater activation than neutral faces in the
amygdala, as revealed by the analysis of attended trials
only; this effect was bilateral. Attended compared to
unattended faces evoked significantly greater activations
bilaterally for all facial expressions (Fig. 8). Importantly,
there was a significant interaction between stimulus valence and attention, that is, in the left amygdala the
differential response to stimulus valence was observed
only in the attended condition (Fig. 8B); in the right
amygdala we observed a trend. For the unattended condition, responses to all stimulus types were equivalent and
not significantly different from zero. Thus, amygdala
responses to emotional stimuli are not automatic and
instead require attention.
Fig. 8. Attention and valence effects in the amygdala. (A) Arrows point to the amygdala. Attended faces compared to unattended faces evoked significantly
greater activations for all facial expressions. The level of the coronal section is indicated on the small whole-brain inset. (B) Estimated responses for the
left and right amygdala regions of interest as a function of attention and valence. FA, fearful attended; FU, fearful unattended; HA, happy attended; HU,
happy unattended; NA, neutral attended; NU, neutral unattended.
40
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
Outside of the amygdala, differential effects of valence
as a function of attention were also observed in a number
of other brain regions. These regions comprised several
visual processing areas, including the calcarine fissure, the
middle occipital and fusiform gyri, and the right superior
temporal sulcus (STS), as well as the insula, nucleus
accumbens, the anterior cingulate gyrus, and orbitofrontal
cortex (including ventromedial and orbital territories). As
in the amygdala, in these brain regions, enhanced responses to fearful and / or happy faces compared to neutral
faces was not automatic but instead required attention.
Our findings are in direct contrast to those by Vuilleumier et al. [101] who also studied the effects of
attention and valence on face processing, using a variation
of the task used by Wojciulik et al. [112]. Subjects fixated
a central cue and matched either two faces or two houses
presented eccentrically. In their study, they failed to see
evidence that attention modulated responses in the
amygdala, regardless of stimulus valence. What is the
explanation for their negative findings? The most likely
explanation is that the attentional manipulation in the
Vuilleumier et al. [101] study was not as effective as ours.
For example, behavioral performance for the bar orientation task in our study and house matching in the Vuilleumier et al. [101] study was 64 and 86% correct,
respectively, indicating that our competing task was a
more demanding one (see Ref. [76] for further discussion).
To summarize, contrary to the prevailing view, we
found that all brain regions responding differentially to
faces with emotional content, including the amygdala, did
so only when sufficient attentional resources were available to process those faces. Indeed, when all attentional
resources were consumed by another task, responses to
faces were eliminated, consistent with Lavie’s [54] proposal that if the processing load of a target task exhausts
available capacity, stimuli irrelevant to that task will not be
processed. Therefore, it does not appear that faces with
emotional expressions are a ‘privileged’ category of objects immune to the effects of attention. Instead, facial
expressions must compete for neural representation, just as
neutral stimuli do; this is illustrated within the context of
the biased competition model of attention in Fig. 9.
4.2. Emotional stimuli can bias competition for
processing resources
Although our results indicate that attentional resources
are required for processing stimulus valence, they do not
imply that humans are unable to respond to potential
threats outside the focus of attention or that the amygdala
only responds to attended stimuli. Indeed, even ignored
items of neutral valence can attract attention and interfere
with on-going processing (e.g. Refs. [54,55,102,114]).
Moreover, numerous studies have demonstrated that negative stimuli are a more effective source of involuntary
interference to on-going tasks than neutral and positive
Fig. 9. Biased competition model of visual attention and the processing
of emotion-laden stimuli. Facial expressions must compete for neural
representation (see arrow labeled ‘stimulus valence’), just as neutral
stimuli do.
ones [35,97,101,111], and more readily recruit attention
[5,24,79,89]. An interesting example comes from the study
by Vuilleumier et al. [101], mentioned above, in which it
was found that when subjects performed the house-matching task (i.e. the faces were unattended), RTs were slower
if the (ignored) faces were fearful than if they were
neutral, suggesting that fearful faces recruited attention
even when they were task-irrelevant and ignored.
It therefore appears that emotional (especially negative)
stimuli can bias the competition for processing resources,
such that they are at a competitive advantage compared to
neutral stimuli. If so, then just as attention enhances
activity within visual cortex to items at attended locations,
so too should emotional pictures evoke stronger responses
in visual cortex than neutral ones. This is indeed the case.
We and others have found that both posterior, visual
processing areas, such as the occipital gyrus, and more
anterior, ventral temporal regions, such as the fusiform
gyrus, exhibit differential activation when emotional and
neutral pictures are contrasted [6,50,52,66,69,96]. In our
study of the processing of facial expressions, we observed
evidence for emotional modulation such that emotional
compared to neutral stimuli produced greater activation
throughout ventral occipitotemporal cortex. Remarkably,
we also obtained evidence for emotional modulation of
activity in and around the calcarine fissure (V1 / V2). It
therefore appears that, like attentional modulation of
activity in visual cortex, emotional modulation can extend
to very early processing areas.
In sum, just as attention can favor the processing of
attended items, so too does emotional valence. Attended
items are associated with faster reaction times and with
increased neural activity. In a similar fashion, the processing of emotion-laden stimuli is prioritized and leads to
stronger activation in visual processing regions. Thus, we
hypothesize that the increased activation produced by
emotional stimuli reflects processes of emotional modulation by which the processing of this stimulus category is
favored as compared to the processing of neutral stimuli.
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
4.3. What is the source of the biasing signal for
emotional stimuli?
In the past decade or so, the amygdala has been shown
to be a critical node in a circuit mediating the processing
of stimulus valence, notably fear. Because of its widespread projections to cortical sensory processing areas,
including direct inputs to visual cortical areas as far back
as V1 [2], it has been suggested that the amygdala may be
the source of modulation of activity evoked by emotional
stimuli. Morris et al. [70] found that amygdala signals
covary with signals from visual areas in a conditiondependent manner. Such changes in ‘functional connectivity’ highlight changes in the coupling between brain
regions. In their study, the correlation between amygdala
and visual cortical activity increased when subjects viewed
fearful faces compared to happy ones (see also Ref. [90]).
Similar findings have been observed when subjects view
unpleasant compared to neutral words [95].
In our study of processing of facial expressions, we also
examined the coupling between amygdala activity and
activity in other brain regions. We anticipated that ventral
occipitotemporal cortex would show increased coupling
with the amygdala during attended compared to unattended
trials. The results of this analysis indeed revealed that the
middle occipital and the fusiform gyri exhibited increased
coupling with the amygdala on attended trials, a finding
consistent with a modulatory role for the amygdala. Fig. 10
illustrates the site of the fusiform enhanced coupling and,
for comparison, at the same slice level, the contrast of
fearful and neutral faces during attended trials, as this
modulation is hypothesized to originate in the amygdala.
Note the striking similarity between the two maps. Interestingly, we also found increased coupling between the
amygdala with the calcarine fissure on attended trials,
which is consistent with projections to very early visual
areas, including V1 and V2, from the amygdala (ref).
Fig. 10. Coupling of activity between the amygdala and other brain
regions. (A) Increases in the coupling between amygdala and visual
cortical activity (V1 / V2 and fusiform gyrus; see arrows) on attended
compared to unattended trials. (B) Contrast of fearful and neutral faces
during attended trials, demonstrating a significant effect of valence in the
fusiform gyrus. A threshold of P,0.00001 (uncorrected) was employed
for both A and B (which show the y5252 plane).
41
Increased coupling with the amygdala was not restricted to
occipitotemporal regions, but also included the STS, the
ventromedial prefrontal cortex and the orbitofrontal cortex,
as well as parietal and other frontal regions. While the
results from our study and others are thus consistent with a
modulatory role for the amygdala, the type of analysis
employed (based on activity covariation) cannot determine
the direction of the interaction. Therefore, the evidence
cited above must be considered indirect.
More direct evidence that the amygdala is a source of
emotional modulation comes from a recent study by
Anderson and Phelps [3], who showed that patients with
bilateral amygdala lesions did not show an advantage at
detecting word stimuli with aversive content compared to
neutral content, in stark contrast to the behavior of normal
subjects. The authors thus suggested that signals originating in the amygdala might help increase perceptual
sensitivity to these stimuli. In other words, the amygdala
would bias the processing of visual stimuli in favor of ones
with valence, which could potentially signal important
events in the environment.
Emotional modulation can potentially be implemented in
one of two ways. First, it could rely on the direct feedback
projections from the amygdala to visual processing areas
[2], as illustrated in Fig. 11. Alternatively, the amygdala
could modulate activity within these processing areas via
its projections to frontal sites that control the allocation of
attentional resources. One such site is the dorsolateral
prefrontal cortex [2] along the middle frontal gyrus, which
is involved in attentional processes (e.g. see axial slice at
Fig. 11. Emotional modulation. The amygdala receives highly processed
visual input from inferior temporal areas TEO and TE. At the same time,
the amygdala projects to several levels of visual processing, including as
early as V1, which allows it to influence visual processing according to
the valence of the stimulus. Note that the amygdala is also interconnected
with, among other regions, the orbitofrontal cortex, another brain
structure important for the processing of ‘stimulus significance.’ Brain
regions: green5occipitotemporal visual processing areas; orange5
posterior superior temporal sulcus; red5amygdala (note that the amygdala
is not visible from a lateral view of the brain; instead it is situated
subcortically near the brain’s medial surface); blue5orbitofrontal cortex
(note that important orbitofrontal regions are situated along the midline,
and hence are not visible from a lateral view of the brain).
42
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
z5140 in Fig. 5), and has recently been shown to
integrate cognitive and emotional information [31].
Another site is the rostral anterior cingulate cortex, which
has been proposed to be an affective division of the
anterior cingulate cortex [8]. The cingulate cortex is
proposed to integrate inputs from various sources and to
contribute to the modulation of processing in other brain
regions [8]; see Ref. [113] for a different proposal. Of
course, the possibilities of direct and indirect emotional
modulation are not mutually exclusive. Finally, although
we have emphasized the role of the amygdala in the
attribution of stimulus valence, several brain regions,
including the orbitofrontal and ventromedial prefrontal
cortices, might act in concert with the amygdala in
determining the behavioral and social significance of
incoming stimuli. Thus, these frontal regions may be
additional sources of emotional modulation.
4.4. How does visual information reach the amygdala:
slow-cortical and fast-subcortical pathways for the
processing of emotional stimuli
If the amygdala conveys valence to sensory stimuli,
what is the pathway by which it receives its sensory
inputs? There is evidence that two pathways exist. One of
them involves the cortical pathway that starts at early
sensory regions, progresses through several intermediate
stages, and finally delivers highly processed sensory
information to the amygdala. For example, for the visual
system, signals reach V1 and proceed through a series of
occipitotemporal regions, culminating in projections from
inferior temporal area TE to the amygdala [2]. A similar
organization is presumed to exist for other sensory systems; for evidence concerning the somatosensory system,
see Ref. [27].
The existence of a parallel, subcortical pathway to the
amygdala for auditory processing has been demonstrated in
studies of fear conditioning to acoustic stimuli in rats and
guinea pigs [56,106]. It has been shown, for instance, that
the medial geniculate body has direct projections to the
amygdala [59], and that interruption of these connections
interferes with conditioning (see Ref. [58]). It has thus
been proposed that, in general, acoustic signals are transmitted via a ‘fast,’ subcortical route, in addition to the
‘well processed’ signals that the amygdala receives via
cortical projections [57].
Several investigators have proposed that a fast, subcortical pathway also exists for the processing of visual stimuli,
and, in particular, the processing of face stimuli
[18,19,70,73]. This view is supported by studies that, as
mentioned above, suggest that the processing of facial
expression occurs not only ‘automatically,’ but even
without conscious awareness. Based on behavioral and
imaging results, Morris et al. [70] have proposed that a
retino-collicular–pulvinar–amygdala pathway provides the
neural substrate for the automatic processing of facial
expression. Recently, affective blindsight (see below) has
also been taken as evidence for the subcortical processing
of faces.
However, as stated by Vuilleumier et al. [101], a truly
automatic pathway should not depend on attentional
resources. Yet, in our study, we found a strong interaction
between stimulus valence and attention, such that differential responses to emotional and neutral faces only occurred
when subjects attended to the faces. Moreover, our results
provide strong evidence that both occipitotemporal and
amygdala responses were eliminated when processing
resources were exhausted. In fact, it is unclear how a
proposed subcortical pathway would support the processing of the detailed form information required for face
perception. For example, the responses of superior colliculus neurons are unable to discriminate the high spatial
resolutions demonstrated by responses of neurons in the
geniculostriate system [64,86].
And yet, studies with blindsight patient GY (who has a
right hemianopia following left occipital lobe damage)
reveal that he is able to discriminate emotional facial
expressions presented in his blind hemifield [18,19], a
phenomenon called affective blindsight [18]. Moreover, in
a recent fMRI study, amygdala responses were elicited in
GY to the presentation of fearful and fear-conditioned
faces in his blind hemifield [68], suggesting that information reached the amygdala subcortically. One difficulty
with the interpretation of GY’s results is that he suffered
an occipital lesion at an early age (8 years old), which may
have produced experience-dependent changes in collicular
function. Experiments with monkeys indicate that the
number of collicular cells with enhanced responses to
visual targets increases significantly following striate cortex lesions [65]. According to Morris et al. [68], similar
plasticity in GY’s superior colliculus may explain the
gradual improvements in his blindsight performance that
have been observed during repeated testing over the years
[107]. Other possible, and less interesting, explanations for
GY’s discrimination of facial expressions could involve
improper fixation and residual vision in his ‘blind’
hemifield [25].
Based on the findings reviewed above and our own
study, we suggest that in the normal brain the critical
pathway for the processing of facial expressions is not
subcortical but rather proceeds from V1 to extrastriate
areas, including the fusiform gyrus and STS, and then to
the amygdala. If attentional resources are depleted, however, face stimuli, regardless of valence, will fail to reach
the amygdala and will fail to be tagged with emotional
expression. Consequently, valence information will not be
conveyed. This is exactly what we observed when subjects
were engaged in a high-load, competing task: neither the
amygdala nor regions to which it projects showed a
valence effect. Thus, contrary to simple auditory stimuli,
for which subcortical processing is likely to be sufficient,
for detailed form information required for face perception,
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
a cortical pathway seems to be necessary. In fact, there is
evidence that in conditioning paradigms involving finer
acoustic discrimination, where one conditioned stimulus is
paired with shock and another is not, cortical lesions
interfere with conditioning [40].
Thus, according to our proposal, the initial volley of
activation over occipitotemporal cortex when emotional
faces are viewed would be equivalent to that produced by
neutral faces. Later, after feedback from other structures
such as the amygdala converge onto occipitotemporal
cortex, the responses would be selective for the valence of
the stimulus. This view is consistent with results from
event-related potential studies, which reveal that valencemodulated components in occipitotemporal cortex occur
between 250 and 600 ms after stimulus onset, far exceeding the so-called N170 face-selective responses (170 ms
post stimulus).
4.5. Attention and awareness
We have proposed that attention is required for the
expression of valence. How does one reconcile this view
with the finding that amygdala responses are evoked by
masked faces of which subjects are presumably unaware
[69,110]? In such instances, it is possible that attentional
resources to the masked stimuli were not sufficiently
reduced by a competing task. Typically, in masking
paradigms subjects are required to direct attention to the
location of the stimulus in the absence of any competing
task, suggesting that attentional resources are available to
allow for subliminal responses to the emotional stimuli.
Thus, while attention appears to be necessary for the
processing of faces with emotional expressions, it may not
ensure that they reach awareness.
Acknowledgements
We wish to thank David Sturman for assistance in the
preparation of the manuscript and an anonymous reviewer
for valuable suggestions. The authors’ research presented
here was supported by the National Institute of Mental
Health Intramural Research Program. Portions of this
paper have appeared previously [46,47].
References
[1] J. Aggleton (Ed.), The Amygdala: A Functional Analysis, Oxford
University Press, Oxford, 2000.
[2] D.G. Amaral, J.L. Price, A. Pitkanen, S.T. Carmichael, Anatomical
organization of the primate amygdaloid complex, in: J. Aggleton
(Ed.), The Amygdala: Neurobiological Aspects of Emotion, Memory, And Mental Dysfunction, Wiley-Liss, New York, 1992, pp.
1–66.
[3] A.K. Anderson, E.A. Phelps, Lesions of the human amygdala impair
43
enhanced perception of emotionally salient events, Nature 411
(2001) 305–309.
[4] P.B. Bandettini, E.C. Wong, R.S. Hinks, R.S. Tikofsky, J.S. Hyde,
Time course EPI of human brain function during task activation,
Magn. Reson. Med. 25 (1992) 390–397.
[5] B.P. Bradley, K. Mogg, S.C. Lee, Attentional biases for negative
information in induced and naturally occurring dysphoria, Behav.
Res. Ther. 35 (1997) 911–927.
[6] H.C. Breiter, N.L. Etcoff, P.J. Whalen, W.A. Kennedy, S.L. Rauch,
R.L. Buckner, M.M. Strauss, S.E. Hyman, B.R. Rosen, Response
and habituation of the human amygdala during visual processing of
facial expression, Neuron 17 (1996) 875–887.
[7] C. Bundesen, A theory of visual attention, Psychol. Rev. 97 (1990)
523–547.
[8] G. Bush, P. Luu, M.I. Posner, Cognitive and emotional influences in
anterior cingulate cortex, Trends Cogn. Sci. 4 (2000) 215–222.
[9] C. Cavada, P.S. Goldman-Rakic, Posterior parietal cortex in rhesus
monkey: II. Evidence for segregated cortico-cortical networks
linking sensory and limbic areas with the frontal lobe, J. Comp.
Neurol. 287 (1989) 422–445.
[10] D. Chawla, G. Rees, K.J. Friston, The physiological basis of
attentional modulation in extrastriate visual areas, Nat. Neurosci. 2
(1999) 671–676.
[11] C.L. Colby, J.R. Duhamel, M.E. Goldberg, Visual, presaccadic, and
cognitive activation of single neurons in monkey lateral intraparietal
area, J. Neurophysiol. 76 (1996) 2841–2852.
[12] M. Corbetta, E. Akbudak, T.E. Conturo, A.Z. Snyder, J.M. Ollinger,
H.A. Drury, M.R. Linenweber, S.E. Petersen, M.E. Raichle, D.C.
Van Essen, G.L. Shulman, A common network of functional areas
for attention and eye movements, Neuron 21 (1998) 761–773.
[13] M. Corbetta, J.M. Kincade, J.M. Ollinger, M.P. McAvoy, G.L.
Shulman, Voluntary orienting is dissociated from target detection in
human posterior parietal cortex, Nat. Neurosci. 3 (2000) 292–297.
[14] M. Corbetta, F.M. Miezin, S. Dobmeyer, G.L. Shulman, S.E.
Petersen, Selective and divided attention during visual discriminations of shape, color, and speed: functional anatomy by positron
emission tomography, J. Neurosci. 11 (1991) 2382–2402.
[15] M. Corbetta, F.M. Miezin, G.L. Shulman, S.E. Petersen, A PET
study of visuospatial attention, J. Neurosci. 13 (1993) 1202–1226.
[15a] M. Corbetta, G.L. Shulman, Control of goal-directed and stimulusdriven attention in the brain, Nat. Rev. Neurosci. 3 (2002) 201–
215.
[16] J.C. Culham, S.A. Brandt, P. Cavanagh, N.G. Kanwisher, A.M.
Dale, R.B. Tootell, Cortical fMRI activation produced by attentive
tracking of moving targets, J. Neurophysiol. 80 (1998) 2657–2670.
[17] A.R. Damasio, H. Damasio, C.H. Chang, Neglect following damage
to frontal lobe or basal ganglia, Neuropsychologia 18 (1980) 123–
132.
[18] B. de Gelder, G. Pourtois, M. van Raamsdonk, J. Vroomen, L.
Weiskrantz, Unseen stimuli modulate conscious visual experience:
evidence from inter-hemispheric summation, NeuroReport 12
(2001) 385–391.
[19] B. de Gelder, J. Vroomen, G. Pourtois, L. Weiskrantz, Non-conscious
recognition of affect in the absence of striate cortex, NeuroReport 10
(1999) 3759–3763.
[20] R. Desimone, Visual attention mediated by biased competition in
extrastriate visual cortex, Philos. Trans. R. Soc. Lond. 353 (1998)
1245–1255.
[21] R. Desimone, J. Duncan, Neural mechanisms of selective attention,
Annu. Rev. Neurosci. 18 (1995) 193–222.
[22] J. Duncan, Cooperating brain systems in selective perception and
action, in: T. Inui, J.L. McClelland (Eds.), Attention and Performance XVI, MIT Press, Cambridge, MA, 1996, pp. 549–578.
[23] J. Duncan, Converging levels of analysis in the cognitive neuroscience of visual attention, Philos. Trans. R. Soc. Lond. B 353
(1998) 1307–1317.
[24] J.D. Eastwood, D. Smilek, P.M. Merikle, Differential attentional
44
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
guidance by unattended faces expressing positive and negative
emotion, Percept. Psychophys. 63 (2001) 1004–1013.
[25] R. Fendrich, C.M. Wessinger, M.S. Gazzaniga, Speculations on the
neural basis of islands of blindsight, Prog. Brain Res. 134 (2001)
353–366.
[26] G.R. Fink, R.J. Dolan, P.W. Halligan, J.C. Marshall, C.D. Frith,
Space-based and object-based visual attention: shared and specific
neural domains, Brain 120 (1997) 2013–2028.
[27] D.P. Friedman, E.A. Murray, J.B. O’Neill, M. Mischkin, Cortical
connections of the somatosensory fields of the lateral sulcus of
macaques: evidence for a cortocolimbic pathway for touch, J. Comp.
Neurol. 252 (1986) 323–3447.
[28] D. Gaffan, J. Hornak, Visual neglect in the monkey: representation
and disconnection, Brain 120 (1997) 1647–1657.
[29] D.R. Gitelman, A.C. Nobre, T.B. Parrish, K.S. LaBar, Y.H. Kim,
J.R. Meyer, M. Mesulam, A large-scale distributed network for
covert spatial attention: Further anatomical delineation based on
stringent behavioural and cognitive controls, Brain 122 (1999)
1093–1106.
[30] J. Globisch, A.O. Hamm, F. Esteves, A. Ohman, Fear appears fast:
temporal course of startle reflex potentiation in animal fearful
subjects, Psychophysiology 36 (1999) 66–75.
[31] J.R. Gray, T.S. Braver, M.E. Raichle, Integration of emotion and
cognition in the lateral prefrontal cortex, Proc. Natl. Acad. Sci. USA
99 (2002) 4115–4120.
[32] S. Grossberg, How does a brain build a cognitive code?, Psychol.
Rev. 87 (1980) 1–51.
[33] S. Grossberg, The link between brain learning, attention, and
consciousness, Conscious. Cogn. 8 (1999) 1–44.
[33a] M.R. Harter, L. Anllo-Vento, F.B. Wood, Event related potentials,
spatial orienting, and reading disabilities, Psychophysiology 26
(1989) 404–421.
[34] M.R. Harter, C.J. Aine, Brain mechanisms of visual selective
attention, in: R. Parasuraman, D.R. Davies (Eds.), Varieties of
Attention, Academic Press, Orlando, FL, 1984, pp. 293–321.
[35] K.M. Hartikainen, K.H. Ogawa, R.T. Knight, Transient interference
of right hemispheric function due to automatic emotional processing, Neuropsychologia 38 (2000) 1576–1580.
[36] K.M. Heilman, E. Valenstein, Frontal lobe neglect in man, Neurology 22 (1972) 660–664.
[37] J.B. Hopfinger, M.H. Buonocore, G.R. Mangun, The neural mechanisms of top-down attentional control, Nat. Neurosci. 3 (2000)
284–291.
[38] W. James, Principles of Psychology, Holt, New York, 1890.
[39] K.W. Janer, J.V. Pardo, Deficits in selective attention following
bilateral anterior cingulotomy, J. Cogn. Neurosci. 3 (1991) 231–
241.
[40] T.W. Jarrell, C.G. Gentile, L.M. Romanski, P.M. McCabe, N.
Schneiderman, Involvement of cortical and thalamic auditory regions in retention of differential bradycardiac conditioning to
acoustic conditioned stimuli in rabbits, Brain Res. 412 (1987)
285–294.
[41] J.S. Joseph, M.M. Chun, K. Nakayama, Attentional requirements in
a preattentive feature search task, Nature 387 (1997) 805–807.
[42] S. Kastner, P. De Weerd, R. Desimone, L.G. Ungerleider, Mechanisms of directed attention in the human extrastriate cortex as
revealed by functional MRI, Science 282 (1998) 108–111.
[43] S. Kastner, P. De Weerd, M.A. Pinsk, M.I. Elizondo, R. Desimone,
L.G. Ungerleider, Modulation of sensory suppression: Implications
for receptive field sizes in the human visual cortex, J. Neurophysiol.
86 (2001) 1398–1411.
[44] S. Kastner, M.A. Pinsk, P. De Weerd, R. Desimone, L.G. Ungerleider, Increased activity in human visual cortex during directed
attention in the absence of visual stimulation, Neuron 22 (1999)
751–761.
[45] S. Kastner, M.A. Pinsk, R. Desimone, L.G. Ungerleider, Activity in
human visual cortex is differentially modulated during expectation
of color or face stimuli, NeuroImage 11 (2000) S14.
[46] S. Kastner, L.G. Ungerleider, Mechanisms of visual attention in the
human cortex, Annu. Rev. Neurosci. 23 (2000) 315–341.
[47] S. Kastner, L.G. Ungerleider, The neural basis of biased competition
in human visual cortex, Neuropsychologia 39 (2001) 1263–1276.
[48] Y.H. Kim, D.R. Gitelman, A.C. Nobre, T.B. Parrish, K.S. LaBar,
M.M. Mesulam, The large-scale neural network for spatial attention
displays multifunctional overlap but differential asymmetry, NeuroImage 9 (1999) 269–277.
[49] K.K. Kwong, J.W. Belliveau, D.A. Chesler, I.E. Goldberg, R.M.
Weisskoff, B.P. Poncelet, D.N. Kennedy, B.E. Hoppel, M.S. Cohen,
R. Turner et al., Dynamic magnetic resonance imaging of human
brain activity during primary sensory stimulation, Proc. Natl. Acad.
Sci. USA 89 (1992) 5675–5679.
[50] R.D. Lane, P.M. Chua, R.J. Dolan, Common effects of emotional
valence, arousal and attention on neural activation during visual
processing of pictures, Neuropsychologia 37 (1999) 989–997.
[51] R.D. Lane, L. Nadel (Eds.), Cognitive Neuroscience of Emotion,
Oxford University Press, Oxford, 1999.
[52] P.J. Lang, M.M. Bradley, J.R. Fitzsimmons, B.N. Cuthbert, J.D.
Scott, B. Moulder, V. Nangia, Emotional arousal and activation of
the visual cortex: an fMRI analysis, Psychophysiology 35 (1998)
199–210.
[53] R. Latto, A. Cowey, Visual field defects after frontal eye-field
lesions in monkeys, Brain 30 (1971) 1–24.
[54] N. Lavie, Perceptual load as a necessary condition for selective
attention, J. Exp. Psychol. Hum. 21 (1995) 451–468.
[55] N. Lavie, Y. Tsal, Perceptual load as a major determinant of the
locus of selection in visual-attention, Percept. Psychophys. 56
(1994) 183–197.
[56] J.E. LeDoux, In search of an emotional system in the brain: Leaping
from fear to emotion and consciousness, in: M.S. Gazzaniga (Ed.),
The Cognitive Neurosciences, MIT Press, Cambridge, MA, 1995,
pp. 1049–1061.
[57] J.E. LeDoux, The Emotional Brain, Simon and Schuster, New York,
1996.
[58] J.E. LeDoux, P. Cicchetti, A. Xagoraris, L.M. Romanski, The lateral
amygdaloid nucleus: sensory interface of the amygdala in fear
conditioning, J. Neurosci. 10 (1990) 1062–1069.
[59] J.E. LeDoux, D.A. Ruggiero, D.J. Reis, Projections to the subcortical forebrain from anatomically defined regions of the medial
geniculate body in the rat, J. Comp. Neurol. 242 (1985) 182–213.
[60] S.J. Luck, L. Chelazzi, S.A. Hillyard, R. Desimone, Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of
macaque visual cortex, J. Neurophysiol. 77 (1997) 24–42.
[61] J.C. Lynch, J.W. McLaren, Deficits of visual attention and saccadic
eye movements after lesions of parietooccipital cortex in monkeys,
J. Neurophysiol. 61 (1989) 74–90.
[62] M.M. Mesulam, A cortical network for directed attention and
unilateral neglect, Ann. Neurol. 10 (1981) 309–325.
[63] E.K. Miller, P.M. Gochin, C.G. Gross, Suppression of visual
responses of neurons in inferior temporal cortex of the awake
macaque by addition of a second stimulus, Brain Res. 616 (1993)
167–202.
[64] M. Miller, P. Pasik, T. Pasik, Extrageniculostriate vision in the
monkey. VII. Contrast sensitivity functions, J. Neurophysiol. 43
(1980) 1510–1526.
[65] C.W. Mohler, R.H. Wurtz, Role of striate cortex and superior
colliculus in visual guidance of saccadic eye movements in monkeys, J. Neurophysiol. 40 (1977) 74–94.
[66] J. Moll, R. de Oliveira-Souza, P.J. Eslinger, I.E. Bramati, J. MouraoMiranda, P.A. Andreiuolo, L. Pessoa, The neural correlates of moral
sensitivity: a functional magnetic resonance imaging investigation of
basic and moral emotions, J. Neurosci. 22 (2002) 2730–2736.
[67] J. Moran, R. Desimone, Selective attention gates visual processing
in the extrastriate cortex, Science 229 (1985) 782–784.
[68] J.S. Morris, B. DeGelder, L. Weiskrantz, R.J. Dolan, Differential
extrageniculostriate and amygdala responses to presentation of
L. Pessoa et al. / Cognitive Brain Research 15 (2002) 31–45
[69]
[70]
[71]
[72]
[73]
[74]
[75]
[76]
[77]
[78]
[79]
[80]
[81]
[82]
[83]
[84]
[85]
[86]
[87]
[88]
[89]
[90]
emotional faces in a cortically blind field, Brain 124 (2001) 1241–
1252.
J.S. Morris, K.J. Friston, C. Buchel, C.D. Frith, A.W. Young, A.J.
Calder, R.J. Dolan, A neuromodulatory role for the human amygdala
in processing emotional facial expressions, Brain 121 (1998) 47–57.
J.S. Morris, A. Ohman, R.J. Dolan, A subcortical pathway to the
right amygdala mediating ‘unseen’ fear, Proc. Natl. Acad. Sci. USA
96 (1999) 1680–1685.
A.C. Nobre, G.N. Sebestyen, D.R. Gitelman, M.M. Mesulam, R.S.J.
Frackowiak, C.D. Frith, Functional localization of the system for
visuospatial attention using positron emission tomography, Brain
120 (1997) 515–533.
S. Ogawa, D.W. Tank, R. Menon, J.M. Ellermann, S.G. Kim, H.
Merkle, K. Ugurbil, Intrinsic signal changes accompanying sensory
stimulation: functional brain mapping with magnetic resonance
imaging, Proc. Natl. Acad. Sci. USA 89 (1992) 5951–5955.
A. Ohman, Automaticity and the amygdala: nonconscious responses
to emotional faces, Curr. Dir. Psychol. Sci. 11 (2002) 62–66.
A. Ohman, F. Esteves, J.J.F. Soares, Preparedness and preattentive
associative learning: Electrodermal conditioning to masked stimuli,
J. Psychophysiol. 9 (1995) 99–108.
H. Pashler, The Psychology of Attention, MIT Press, Cambridge,
MA, 1998.
L. Pessoa, M. McKenna, E. Gutierrez, L.G. Ungerleider, Neural
processing of emotional faces requires attention, Proc. Natl. Acad.
Sci. USA 99 (2002) 11458–11463.
S.E. Petersen, D.L. Robinson, J.D. Morris, Contributions of the
pulvinar to visual spatial attention, Neuropsychologia 25 (1987)
97–105.
M.I. Posner, S.E. Peterson, The attention system of the human brain,
Annu. Rev. Neurosci. 13 (1990) 25–42.
F. Pratto, O.P. John, Automatic vigilance: the attention-grabbing
power of negative social information, J. Pers. Soc. Psychol. 61
(1991) 380–391.
R. Rafal, Neglect, in: R. Parasuraman (Ed.), The Attentive Brain,
MIT Press, Cambridge, MA, 1998, pp. 489–525.
G.H. Recanzone, R.H. Wurtz, U. Schwarz, Responses of MT and
MST neurons to one and two moving objects in the receptive field,
J. Neurophysiol. 78 (1997) 2904–2915.
G. Rees, C.D. Frith, N. Lavie, Modulating irrelevant motion
perception by varying attentional load in an unrelated task, Science
278 (1997) 1616–1619.
G. Rees, C. Russell, C.D. Frith, J. Driver, Inattentional blindness
versus inattentional amnesia for fixated but ignored words, Science
286 (1999) 2504–2507.
R.A. Rensink, Change detection, Annu. Rev. Psychol. 53 (2002)
245–277.
J.H. Reynolds, L. Chelazzi, R. Desimone, Competitive mechanisms
subserve attention in macaque areas V2 and V4, J. Neurosci. 19
(1999) 1736–1753.
H.R. Rodman, C.G. Gross, T.D. Albright, Afferent basis of visual
response properties in area MT of the macaque. I. Effects of striate
cortex removal, J. Neurosci. 9 (1989) 1510–1526.
E.T. Rolls, M.J. Tovee, The responses of single neurons in the
temporal visual cortical areas of the macaque when more than one
stimulus is present in the receptive field, Exp. Brain Res. 103 (1995)
409–420.
A.C. Rosen, S.M. Rao, P. Caffarra, A. Scaglioni, J.A. Bobholz, S.J.
Woodley, T.A. Hammeke, J.M. Cunningham, T.E. Prieto, J.R.
Binder, Neural basis of endogenous and exogenous spatial orienting:
a functional MRI study, J. Cogn. Neurosci. 11 (1999) 135–152.
D.R. Roskos-Ewoldsen, R.H. Fazio, On the orienting value of
attitudes: Attitude accessibility as a determinant of an object’s
attraction of visual attention, J. Pers. Soc. Psychol. 63 (1992)
198–211.
P. Rotshtein, R. Malach, U. Hadar, M. Graif, T. Hendler, Feeling or
features. Different sensitivity to emotion in high-order visual cortex
and amygdala, Neuron 32 (2001) 747–757.
45
[91] T. Sato, Interactions of visual stimuli in the receptive fields of
inferior temporal neurons in awake monkeys, Exp. Brain Res. 77
(1989) 23–30.
[92] G.L. Shulman, J.A. Ollinger, E. Akbudak, T.E. Conturo, Areas
involved in encoding and applying directional expectations to
moving objects, J. Neurosci. 21 (1999) 9480–9496.
[93] D. Simons, D.T. Levin, Change blindness, Trends Cogn. Sci. 1
(1997) 261–267.
[94] G. Stenberg, S. Wilking, M. Dhal, Judging words at face value:
Interference in a word processing task reveals automatic processing
of affective facial expressions, Cogn. Emot. 12 (1998) 755–782.
[95] M.H. Tabert, J.C. Borod, C.Y. Tang, G. Lange, T.C. Wei, R.
Johnson, A.O. Nusbaum, M.S. Buchsbaum, Differential amygdala
activation during emotional decision and recognition memory tasks
using unpleasant words: an fMRI study, Neuropsychologia 39
(2001) 556–573.
[96] S.F. Taylor, I. Liberzon, R.A. Koeppe, The effect of graded aversive
stimuli on limbic and visual activation, Neuropsychologia 38 (2000)
1415–1425.
[97] J. Tipples, D. Sharma, Orienting to exogenous cues and attentional
bias to affective pictures reflect separate processes, Br. J. Psychol.
91 (2000) 87–97.
[98] L.G. Ungerleider, D. Gaffan, V.S. Pelak, Projections from inferior
temporal cortex to prefrontal cortex via the uncinate fascicle in
rhesus monkeys, Exp. Brain Res. 76 (1989) 473–484.
[99] G. Vallar, D. Perani, The anatomy of unilateral neglect after righthemisphere stroke lesions: A clinical / CT-scan correlation study in
man, Neuropsychologia 24 (1987) 609–622.
[100] R. Vandenberghe, J. Duncan, P. Dupont, R. Ward, J. Poline, G.
Bormans, J. Michiels, L. Mortelmans, G.A. Orban, Attention to
one or two features in left or right visual field: a positron emission
tomography study, J. Neurosci. 17 (1997) 3739–3750.
[101] P. Vuilleumier, J.L. Armony, J. Driver, R.J. Dolan, Effects of
attention and emotion on face processing in the human brain: An
event-related fMRI study, Neuron 30 (2001) 829–841.
[102] R. Ward, J. Duncan, K. Shapiro, The slow time-course of visual
attention, Cogn. Psychol. 30 (1996) 79–109.
[103] R.T. Watson, K.M. Heilman, J.C. Cauthen, A.K. Frederick, Neglect
after cingulectomy, Neurology 23 (1973) 1003–1007.
[104] R.T. Watson, K.M. Heilman, Thalamic neglect, Neurology 29
(1979) 690–694.
[105] M.J. Webster, J. Bachevelier, L.G. Ungerleider, Connections of
inferior temporal areas TEO and TE with parietal and frontal
cortex in macaque monkeys, Cereb. Cortex 4 (1994) 470–483.
[106] N.M. Weinberger, Retuning the brain by fear conditioning, in: M.S.
Gazzaniga (Ed.), The Cognitive Neurosciences, MIT Press, Cambridge, MA, 1995, pp. 1071–1089.
[107] L. Weiskrantz, Blindsight: implications for the conscious experience of emotion, in: R.D. Lane, L. Nadel (Eds.), Cognitive
Neuroscience of Emotion, Oxford University Press, New York,
2000, pp. 277–295.
[108] K. Welch, P. Stuteville, Experimental production of unilateral
neglect in monkeys, Brain 81 (1958) 341–347.
[109] A. Wells, G. Matthews, Attention and Emotion: A Clinical Perspective, Lawrence Erlbaum, Hove, UK, 1994.
[110] P.J. Whalen, S.L. Rauch, N.L. Etcoff, S.C. McInerney, M.B. Lee,
M.A. Jenike, Masked presentations of emotional facial expressions
modulate amygdala activity without explicit knowledge, J. Neurosci. 18 (1998) 411–418.
[111] M. White, Anger recognition is independent of spatial attention, NZ
J. Psychol. 25 (1996) 30–35.
[112] E. Wojciulik, N. Kanwisher, The generality of parietal involvement
in visual attention, Neuron 23 (1999) 747–764.
[113] H. Yamasaki, K.S. LaBar, G. McCarthy, Dissociable prefrontal
brain systems for attention and emotion, Proc. Natl. Acad. Sci.
USA 99 (2002) 215–222.
[114] S. Yantis, D.N. Johnson, Mechanisms of attentional priority, J. Exp.
Psychol. Hum. 16 (1990) 812–825.