Neuropsychologia 50 (2012) 3663–3672
Contents lists available at SciVerse ScienceDirect
Neuropsychologia
journal homepage: www.elsevier.com/locate/neuropsychologia
Counterfactuals in action: An fMRI study of counterfactual sentences
describing physical effort
Mabel Urrutia a, Silvia P. Gennari b,n, Manuel de Vega a
a
b
Department of Cognitive Psychology, University of La Laguna, Tenerife, Spain
Department of Psychology, University of York, York, UK
a r t i c l e i n f o
abstract
Article history:
Received 28 February 2012
Received in revised form
4 August 2012
Accepted 4 September 2012
Available online 10 September 2012
Counterfactual statements such as if Mary had cleaned the room, she would have moved the sofa convey
both actual and hypothetical actions, namely, that Mary did not clean the room or move the sofa, but
she would have done so in some possible past situation. Such statements are ubiquitous in daily life and
are involved in critical cognitive activities like decision-making and evaluation of alternative outcomes.
Here, we investigate the brain mechanisms and the nature of the semantic representations involved in
understanding the complex meaning of counterfactual statements. We used fMRI to examine brain
responses to counterfactual statements describing actions of high and low physical effort and compared
them to similar factual statements describing the same actions. Results indicated that the inferior
parietal lobule, known to support planning of object-directed actions, responded more strongly to higheffort than low-effort statements. Moreover, counterfactual statements, compared to factual ones,
recruited a distinctive neural network partially overlapping with action execution networks. This
network included medial pre-motor and pre-frontal structures, which underpin selection and inhibition
of alternative action representations, and parahippocampal and temporal regions, involved in retrieving
episodic memories. We argue that counterfactual comprehension recruit action-related networks
encoding and managing alternative representations of behaviors.
& 2012 Elsevier Ltd. All rights reserved.
Keywords:
Counterfactuals
Language comprehension
Supplementary motor area
1. Introduction
Language is often used to describe actions in the world. In this
function, language appears to recruit semantic representations that
have been systematically linked to aspects of action planning and
observation during the course of learning. Numerous behavioral
studies report that words and sentences quickly influence motor
responses, suggesting that language recruits motor-related information even in tasks in which attention is not specifically directed
to motor content (Bub, Masson, & Cree, 2008; Caligiore, Borghi,
Parisi, & Baldassarre, 2010; Glenberg & Kaschak, 2002; Scorolli,
Borghi, & Glenberg, 2009; Tucker & Ellis, 2004; Zwaan & Taylor,
2006). Moreover, neuroimaging studies have shown that understanding action words and sentences recruits brain regions overlapping with those involved in performing and planning actions
(Aziz-Zadeh, Wilson, Rizzolatti, & Iacoboni, 2006; Hauk, Johnsrude,
& Pulvermuller, 2004; Pulvermuller, 2005; Tettamanti et al., 2005,
2008). For example, simple sentences referring to actions that imply
varying degrees of physical effort (e.g., the delivery man has forgotten
n
Corresponding author. Tel.: þ44 1904 322877; fax: þ44 1904 323181.
E-mail address: [email protected] (S.P. Gennari).
0028-3932/$ - see front matter & 2012 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.neuropsychologia.2012.09.004
the piano, the delivery man is pushing the piano, the delivery man is
pushing the chair) modulate activity in pre-motor regions also
involved in action execution, thus suggesting that language recruits
action representations as a function of fine-grained action properties (Moody & Gennari, 2010). Taken together, these findings
indicate that language comprehension activates highly specific
and detailed semantic representations of actions that are partially
shared with those engaged in performing actions.
Many instances of language use however may not fully recruit
such detailed action representations because they do not refer to
actual actions and thus are less likely to engage motor simulations. Yet factual language use has been the focus of most
previous studies. Here, we investigate counterfactual language
uses such as those in counterfactual statements, e.g., if Mary had
cleaned the room, she would have moved the sofa, where linguistic
references do not apply to real world events. The meaning of
counterfactuals is rather paradoxical: counterfactuals are implicit
negations (the described events did not happen), but at the same
time they invite comprehenders to consider the alternative events
as if they had happened (Byrne, 2002; Byrne & Tasso, 1999;
De Vega, Urrutia, & Riffo, 2007; Roese & Olson, 1995). Thus, the
above counterfactual statement implies multiple statements: that
Mary did not actually clean the room nor move the sofa but that
3664
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
she would have done so in some past alternative of the real world
(Kratzer, 1989; Lewis, 1973). Such statements are ubiquitous in
everyday life and are arguably involved in many high order
cognitive activities such as decision-making, action planning
(through the evaluation of alternative past outcomes), understanding of our own and others’ mental states and expressing
feelings such as regret or relief (Byrne, 2002).
Our goal in this work is to begin to elucidate the neural
processing mechanisms and the nature of the semantic representations involved in understanding counterfactual actions. Specifically,
we examine the extent to which sensory-motor action representations are recruited when understanding counterfactual actions, as
compared to factual ones, and more generally, how the complex
meaning of counterfactuals is processed. Previous research has not
directly investigated these issues, as it has focused instead on
reasoning, decision making, truth-value judgments, or other language processing issues (Baird & Fugelsang, 2004; Barbey, Krueger,
& Grafman, 2009; Nieuwland, 2012; Nieuwland & Martin, 2012).
Nevertheless, previous related studies provide important clues
regarding counterfactual sentence processing and the kinds of
sensory-motor representations that they may recruit.
One possibility is that action representations in motor-related
regions are activated for counterfactuals just as for factual sentences. Previous behavioral research indeed shows that reading
counterfactual action sentences such as that exemplified above
interferes with the planning of an action response (De Vega &
Urrutia, 2011) like simple action sentences do (Glenberg & Kaschak,
2002). This possibility is also consistent with related neuroimaging
studies reporting motor and pre-motor activations for metaphorical
language uses, e.g., grasp the idea, which suggest that non-literal
language nevertheless engages pre-motor brain regions overlapping
with action execution networks (Boulenger, Hauk, & Pulvermüller,
2009; Santana & de Vega, 2011; Wilson & Gibbs, 2007).
Another possibility is that the multiple representations triggered
by counterfactual sentences elicit stronger activations than factual
sentences, because these representations compete with each other,
and thus engage pre-frontal inhibition or control processes. In text
comprehension, for example, counterfactual actions, are less accessible in memory upon later mention in the discourse (De Vega et al.,
2007) and are poorly integrated with the ongoing discourse
representation, as reflected by desynchronization of gamma activity
in EEG (Urrutia, De Vega, & Bastiaansen, 2012). Patients with
Parkinson disease and pre-frontal cortex lesions (often medially
located) are also impaired in counterfactual generation and reasoning, and this impairment correlates with poor performance in a
variety of executive function measures such as response inhibition
in Stroop tasks (Beck, Riggs, & Gorniak, 2009; Gómez-Beldarrain,
Garcı́a-Monco, Astigarraga, González, & Grafman, 2005; McNamara,
Durso, Brown, & Lynch, 2003). Moreover, studies in social cognition
have argued that counterfactual reasoning and thinking engage prefrontal control processes because these tasks, like theory of mind
tasks, require the decoupling of alternative from actual states (Baird
& Fugelsang, 2004; Barbey et al., 2009; Fritz & Frith, 2003;
Hanakawa et al., 2002; Krueger, Barbey, & Grafman, 2009; Ursu &
Carter, 2005).
Supporting this possibility, related language and action research
also suggests that counterfactuals may engage competition
between alternative interpretations specifically in medial prefrontal regions. Imaging studies investigating discourse comprehension for example report the recruitment of medial pre-frontal
regions around supplementary motor area (SMA) in understanding
connected sentences, due to increased demands in maintaining
multiple representations across sentences (characters’ goals, locations, actions, etc.) (Ferstl, Neumann, Bogler, & von Cramon, 2008;
Ferstl & von Cramon, 2001; Goel, 2007; Knauff, Mulack, Kassubek,
Salih, & Greenlee, 2002; Speer, Reynolds, Swallow, & Zacks, 2009;
Yarkoni, Speer, & Zacks, 2008). Similarly, TMS, neuroimaging and
electrophysiological studies indicate that pre-SMA and SMA, are
involved in resolving competition among alternative physical
actions in go/no-go tasks, task switching and stop tasks (Chen,
Muggleton, Tzeng, Hung, & Juan, 2009; Rushworth, Hadland, Paus,
& Sipila, 2002; Simmonds, Pekar, & Mostofsky, 2008). Among the
neurons recorded in the macaque pre-SMA during task switching
for example, there are neurons responding to all alternative task
responses that fire before an action is performed, suggesting that
they exert a modulatory role in inhibiting one response and
selecting another (Isoda & Hikosaka, 2007).
To examine these possibilities, we presented Spanish-speaking
participants with factual and counterfactual statements referring
to actions that varied in their degree of effort—high vs. low effort
(2 by 2 design). These actions critically involved the use of
hands, although complex actions also involved other body
parts. Example sentences and their English translations are
provided in Table 1. Participants saw the sentences and were
occasionally asked comprehension questions to guarantee that
they understood their meaning. A separate motor task was also
conducted to evaluate the overlap between action execution and
language. Previous results examining simple sentences have
found sensitivity to physical effort in pre-motor cortex (Moody
& Gennari, 2010) and sensitivity to object manipulation in
parietal structures (Chao & Martin, 2000; Just, Carpenter,
Maguire, Diwadkar, & McMains, 2001; Just, Newman, Keller,
McEleney, & Carpenter, 2004; Rueschemeyer, van Rooij,
Lindemann, Willems, & Bekkering, 2009). We therefore expected
that if actions in counterfactual sentences are processed as
actions in factual sentences, sensitivity to effort but no difference
between counterfactual and factual statements should be
observed in action-related pre-motor and parietal regions overlapping with the action execution. Moreover, if counterfactual
understanding involves competition between multiple action
representations, as discussed above, they should elicit stronger
responses than factual statements in medial prefrontal regions, in
addition to motor-based regions contributing to action
representations.
Table 1
Example sentences in the reading task.
Conditions
Example sentence
Factual
Low-effort
High-effort
Counterfactual
Low-effort
High-effort
Como Pedro decidió pintar la sala, está moviendo la foto.
‘‘Since Pedro decided to paint the room, he is moving the photograph.’’
Como Pedro decidió pintar la sala, está moviendo el sofá.
‘‘Since Pedro decided to paint the room, he is moving the sofa.’’
Si Pedro hubiera decidido pintar la sala, habrı́a movido la foto.
‘‘If Pedro had decided to paint the room, he would have moved the photograph.’’
Si Pedro hubiera decidido pintar la sala, habrı́a movido el sofá.
‘‘If Pedro had decided to paint the room, he would have moved the sofa.’’
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
3665
2. Methods
2.4. Design and procedure
2.1. Participants
Before scanning, each participant was briefly trained in the language comprehension task and in the execution task. The reading task preceded the motor task
as described below.
Eighteen healthy right-handed Spanish native speakers with normal or
corrected-to-normal vision (13 women, 5 men; mean age ¼31 years) participated
in the experiment. All participants attended graduate or postgraduate courses in
the University of York, UK. All gave written informed consent and were paid for
their participation.
2.2. Materials
The materials consisted of 120 Spanish sentences, arranged in 30 items. For
each item, four sentence versions were constructed, one for each of the conditions
of the factorial design: counterfactual high-effort, counterfactual low-effort,
factual high-effort, and factual low-effort sentences. See Appendix for the list of
stimuli. The item set also included 30 filler two-clause sentences containing
psychological actions (e.g., examine, think) to add action-content variability to the
stimulus set and minimize attention to the action repetition across factual and
counterfactual sentences. These filler items were not included in the main analysis
contrasts. Experimental sentences contained two clauses: antecedence and consequent. The two effort versions of each item used the same action verbs and only
differed in the object involved in the action (effort manipulation: move the
sofa/the photograph). These verbs were translations from Moody and
Gennari’s stimulus verbs, which were shown to describe hand actions in a rating
study, although other body parts were also involved in the high-effort sentences
(Moody & Gennari, 2010). The verbs and object nouns across conditions were
matched for length and frequency according to a Spanish word frequency
dictionary (Davis & Perea, 2005). Thus, comparisons of high-effort objects and
low-effort objects did not show any statistical difference in log frequency (higheffort: M ¼ 1.02; SD ¼0.48; low-effort: M¼ 1.00, SD ¼ 0.42; t(29)¼0.23, p4 .05), or
length (high-effort: M ¼ 5.83, SD ¼ 1.57; low-effort: M ¼5.88, SD ¼1.69;
t(29)¼ 0.09, p 4.05). Factual and counterfactual sentences had the same lexical
content but differed in one word: the verb auxiliary habrı́a (would have),
which is a high frequency function word indicating the counterfactual interpretation (see Table 1). This length difference is inevitable, as there is no way to
compare these two sentence types in natural language without also including
length differences. However, this difference is unlikely to influence activity in the
regions targeted here, as length differences have been found in occipital visual
areas (Indefrey et al., 1997).
2.2.1. Stimulus pre-tests
To guarantee that the manipulation of physical effort was appropriate and that
our stimulus sentences were equally plausible across conditions, we conducted
two norming studies. The first study asked participants to indicate in a scale of
1 to 7 the degree of effort required to perform the action described. Ninety
nineSpanish speakers from the Universiy of La Laguna (Spain) participanted in this
study. The stimulus sentences only contained the consequent clause in its factual
format, for example, Pedro is moving the sofa. Sentences were arranged into
different lists so that each participant only saw one version of each item. Pair-wise
comparisons indicated that there was a significant difference between high
and low-effort sentences (high-effort: M¼ 5.43, SD ¼ 0.69; low-effort: M ¼2.7;
SD ¼ 0.76; t(29)¼ 24.70, p o 0.0001).
The second norming study asked participants to indicate in a scale of 1 to
7 how plausible the stimulus sentences were. Another group of 36 native Spanish
speakers, students at the University of La Laguna (Spain) completed a questionnaire. Sentences were arranged into different lists so that each participant
only saw one version of each item. Pair-wise comparisons indicated that there
was no significant difference between sentence conditions (low-effort factual:
M ¼4.96, SD ¼ 0.8; high-effort factual: M ¼ 5.24, SD ¼0.97; low-effort counterfactual: M ¼ 4.45, SD ¼1.19 and high-effort counterfactual: M ¼ 4.87, SD ¼1.02, all
ps4.05).
2.3. fMRI data acquisition parameters
Images were obtained with a 3 T GE Signa Excite MRI scanner. Functional
images were obtained in an interleaved fashion using a gradient-echo EPI
sequence (TR 2000 ms, TE 50 ms, flip angle 90 degrees a matrix 64 64, field of
view 24 cm) with 625 volumes, corresponding to 32 axial slices of thickness
3.5 mm for the linguistic task and 171 volumes for the motor task. The voxels
dimensions for the main tasks were 3.5 4.06 4.06 mm. Functional images
excluded a portion of the cerebellum. Whole-brain high-resolution structural
images (of 1 mm isotropic voxels) were obtained for each participant while
resting. The experimental session lasted approximately 40–45 min.
2.4.1. Reading comprehension task
Participants were instructed to fully understand the meaning of the sentences
and occasionally answer comprehension questions by pressing the YES or NO
response key when a question appeared (catch trials). This task used an event
related design. In each event, participants read a factual or counterfactual
sentence. Each sentence stimulus was presented in the screen in white 34 pt font
on a black background lasting for 4 s and was followed by a fixation cross of
randomly variable duration (average 5600 ms). The order and timing of the
stimulus sentences was optimized using the Optseq tool (http://surfer.nmr.mgh.
harvard.edu/optseq/), which schedules events for rapid presentation in eventrelated designs by calculating the most efficient random order of conditions and
inter-trial intervals. The order of event types across the experiment was thus such
that every condition followed any other condition with equal probability (about
20% of the total number of trials). The four versions of an item (high-effort factual,
high-effort counterfactual, low-effort factual and low-effort counterfactual) were
counterbalanced across items so that 16 possible presentation orders of the
4 versions of an experimental item were equally represented, although they were
never close in time across the experiment. (For example, if the presentation order
for item 1’s versions was ‘‘high-effort counterfactual, low-effort factual, low effortcounterfactual, high-effort factual’’, item 2’s versions would follow the reverse
order, item 3’s versions would follow a different order and so on for all item). This
minimizes order effects because the content of an item and its versions cannot be
predicted across the experiment. Moreover, the average activity across all items
for a given condition was obtained from cases presented at different times across
the scanning session and in different orders relative to the other conditions of the
same item. Thus, the activity elicited say, by counterfactual high-effort sentences
cannot be attributed to the fact that these sentences were read in a particular
order relative to either their factual or low-effort counterparts.
Comprehension questions (catch trials) were used to check that participants
indeed read the sentences for meaning. These trials occurred in 17% of the total
number of trials (27 catch trials in total) and were treated as separate events in
the analyses (see below). Questions always referred to the preceding sentence and
were written in capital letters for easy identification. The inter-trial times between
sentences and questions were determined by the scheduler as indicated above, so
that their activity could be estimated separately from other trial types. The
questions queried either the antecedent or consequent clause with similar
frequency to encourage attention to the entire sentence (e.g., for the factual
versions in Table 1, the questions could be Was Pedro moving the sofa? or Did Pedro
decide to paint the room? for ‘‘yes’’ answers, and Was Mary moving the sofa? or Did
Pedro clean the room? For ‘‘no’’ answers; counterfactual versions had similar
questions in either positive or negative format to yield negative or positive
answers—e.g., Was Pedro not moving the sofa?. The questions were evenly
distributed across sentence types (counterfactuals, factuals and fillers) to guarantee that participants could not predict whether a question would be asked. Thus,
across the experiment, nine questions referred back to counterfactual statements,
nine to factual ones and nine to filler sentences. Across the experiment and across
conditions, there were roughly equal numbers of ‘‘yes’’ and ‘‘no’’ responses. Taken
together, the features of this design therefore make it very unlikely that the
activity elicited by the 17 catch trials had an unexpected influence on the brain
activity elicited by the reading trials, or by a specific trial type (condition). The
reading task lasted 22 min.
2.4.2. Action execution task
This task was structured in 12 blocks presented randomly (6 blocks per effort
condition). Between blocks there were resting periods of 18 s each. In each block
of the action execution task, participants were asked to squeeze a soft toy ball
repeatedly, in response to the symbol ‘‘@’’ presented in the center of the screen on
a black background. This symbol was presented in different colors, prompting
participants to apply effort in the squeezing task or not to apply any force (holding
without squeezing), which was used as baseline comparison. A block started with
a colored symbol that kept flashing every 2 s on the screen for 12 s. Every time
participants saw the symbol, they either squeezed the ball or held it, depending on
the block condition. This task lasted 6 min.
2.5. Data analysis
Preprocessing and statistical analysis were performed using FSL tools (the
software library of the Oxford Centre for Functional MRI of the Brain (FMRIB);
www.fmrib.ox.ac.uk/fsl). First-level and higher-level analyses were separately
carried out for each task, using FEAT (FMRI Expert Analysis Tool). Pre-processing
steps included motion correction (Jenkinson, Bannister, Brady, & Smith, 2002),
slice-timing correction, brain extraction, spatial smoothing using a Gaussian
3666
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
kernel of full-width-half-maximum (FWHM) 8 mm, and high-pass temporal
filtering (Gaussian-weighted least-squares straight line fitting, with
sigma¼ 25.0 s.). Head movement in the scanner did not exceed 0.9 mm across
participants and was 0.4 mm on average. Time-series analysis was carried out
using FILM (FMRIB’s Improved Linear Model) with local autocorrelation correction
(Woolrich, Ripley, Brady, & Smith, 2001). Time series were modeled with
covariates representing the trial structure and conditions convolved with a
hemodynamic response function (gamma function). Each event type was modeled
as starting at the beginning of the stimulus presentations and lasted as long as the
stimulus stayed on the screen. There were four explanatory variables (EVs) for
each condition (high-effort counterfactual, high-effort factual, low-effort counterfactual and low-effort factual) and two additional EVs for catch trials and filler
trials. For the experimental trials, a total of four contrasts were computed,
obtaining the main effect of each sentence condition (high-effort-counterfactual,
low-effort-counterfactual, high-effort-factual, low-effort-factual) relative to rest
(e.g., 1, 0, 0, 0, 0, 0). For the action execution data, a contrast was computed
comparing the effort condition to the no-effort condition (i.e., squeezing vs.
holding the toy ball).
Registration to high-resolution individual space and standard MNI space was
carried out using FLIRT (FMRIB’s linear image registration tool) for each individual
participants and both data sets (language and execution data) (Jenkinson et al.,
2002; Jenkinson & Smith, 2001). Mixed-effect statistics were used for higher-level
analyses. z (Gaussianised T/F) statistic images were thresholded at clusters
determined by z 42.3 and a (corrected) cluster significance threshold of p ¼0.05
(Worsley, Evans, Marrett, & Neelin, 1992). The lower-level contrast parameter
estimates that were obtained for each participant in each of the four sentence
conditions were entered into the higher-level analysis. Main effects (counterfactual 4factual, high4low-effort) and interactions were then calculated, using
conditions as repeated factors. These higher-level analyses were conducted within
the left and right cerebral cortex separately from which the occipital pole and
auditory cortex (as defined the Harvard–Oxford cortical structural atlas and the
Juelich histological atlas) had been removed in order to target expected regions
and increase statistical power.
2.5.1. Region of interest analysis
A region of interest (ROI) was employed to evaluate the hypothesis that the
inferior parietal lobe would be sensitive to the effort manipulation. An anatomically defined ROI of inferior parietal structures (labeled used as ‘‘supramarginal
gyrus’’ in the Harvard–Oxford Atlas) was used to this end. This ROI targeted
portions of the parietal lobe – the intraparietal sulcus (IPS) and the gyri inferior to
it – that have been previously reported to be active for language understanding
and object-directed action planning but were not fully included in the (statistically corrected) action execution results. This is because our motor task was not
designed to specifically detect object-related information but rather hand-action
planning and execution. To detect object-directed action information, an execution task in which participants manipulate objects of varying size and weight
should have been used, as in our linguistic effort manipulation. However, task
duration and practical constraints precluded the use of large objects and various
handgrip types as described in the sentences (e.g., moving a sofa).
Nevertheless, a large body of evidence particularly concerning object manipulation amply justifies the independent targeting of inferior parietal structures,
including the IPS: Apraxic patients with parietal lesions have specific impairments
in planning object-directed hand actions (Binkofski et al., 1998; Buxbaum, 2001;
Buxbaum, Johnson-Frey, & Bartlett-Williams, 2005; Buxbaum, Kyle, Grossman, &
Coslett, 2007). Electrophysiological recordings in monkeys also suggest that
posterior parietal regions, which are homologue with the human IPS, encode
transformations from object features such as shape, size and weight into hand
motor plans (Grefkes, Weiss, Zilles, & Fink, 2002; Jeannerod & Decety, 1995;
Murata, Gallese, Luppino, Kaseda, & Sakata, 2000; Rizzolatti, Ferrari, Rozzi, &
Fogassi, 2006; Rizzolatti, Fogassi, & Gallese, 1997). Numerous neuroimaging and
TMS studies investigating action planning and/or conceptual representations have
also shown that inferior parietal structures such as the IPS respond strongly to
planning tool actions, viewing or naming tools, reading descriptions of actions and
evaluating the manipulability of objects (Chao & Martin, 2000; Culham & Valyear,
2006; Gre zes & Decety, 2001; Johnson-Frey, Newman-Norlund, & Grafton, 2005;
Kellenbach, Brett, & Patterson, 2003; Moody & Gennari, 2010; Noppeney, Josephs,
Kiebel, Friston, & Price, 2005; Tettamanti et al., 2005).
3. Results
3.1. Counterfactual vs. factual language
The higher-level analysis of the reading task within each
hemisphere (see above Section 2.5) revealed that counterfactual
sentences elicited stronger activity than factual sentences in a
cluster centered in the left superior frontal gyrus (BA6) (cluster
corrected, z¼2.5, p ¼.05). This cluster extends ventrally to the
Counterfactual > Factual Sentences
L
L
4.8
2.5
High-effort > Low-effort Sentences
L
2.3
L
3.0
Fig. 1. Language results superimposed on the action performance results (cluster
corrected, zZ2.3, p¼.05). The area in red shows the significant cluster for the language
task, the area in green indicates the action execution results and that in blue indicates
the overlap between the two tasks. The area in yellow in the bottom images shows a
view of the parietal lobe region of interest. (For interpretation of the references to color
in this figure legend, the reader is referred to the web version of this article.)
cingulate gyrus (BA 24, 32) and anteriorly to the medial frontal
gyrus, including aspects of the supplementary motor area (SMA),
as defined by the Harvard–Oxford Atlas. Importantly, this cluster
partially overlapped with the action execution results from our
execution task in pre-motor regions. The significant cluster is
shown in red in the top panel of Fig. 1 and the overlap with the
execution task is shown in blue.
Additional activations for the counterfactual–factual contrast
were also found in both left and right parahipocampal gyrus
(BA19) as well as the right inferior temporal gyrus (BA37) (see
middle panel of Fig. 1). Parahipocampal and inferior temporal
structures are involved in lexical and episodic memory retrieval
(Paller & Wagner, 2002) and have been associated with the
generation of alternative episodic memories to be evaluated in
consequential and future thinking (Baird & Fugelsang, 2004;
Schacter & Addis, 2009; Schacter, Addis, & Buckner, 2007). No
interactions were observed in this analysis. (A separate analysis
using the contrast factual4counterfactual sentences revealed no
significant differences). These results thus suggest that more
cognitive effort is involved in processing counterfactual than
factual statements, engaging posterior structures aiding knowledge and episodic retrieval, as well as pre-motor and medial
structures known to be involved in action selection and
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
3667
Table 2
Summary of results (active regions).
Contrast
Location
Counterfactual–
factual
L superior frontal gyrus
High effort–low effort
MNI coordinates Nearest
(x, y, z)
BA
6, 4, 62
6, 6, 60
L cingulate gyrus
4, 12, 42
L medial frontal gyrus
10, 36, 48
R inferior temporal gyrus 53, 56, 10
R parahippocampal gyrus 28, 52, 10
L parahippocampal gyrus 24, 56, 8
Inferior parietal lobule
40, 44, 40
Supramarginal gyrus
42, 44, 38
6
24
32
8
37
19
19
40
40
Fig. 3. Effect size of each condition compared to rest within the significant cluster
found in the inferior parietal lobule. Error bars indicate standard errors.
Fig. 2. Effect size of each condition compared to rest within the significant cluster
found in the superior frontal gyrus. Error bars indicate standard errors.
evaluation of alternatives (Nachev, Wydell, O’Neill, Husain, &
Kennard, 2007; (Krueger & Grafman, 2008; Mostofsky &
Simmonds, 2008; Nachev et al., 2007). Table 2 reports the
coordinates of the active regions.
To examine the activation levels for each condition within the
significant clusters of activation, we extracted t-values for each
subject derived from the contrast of each sentence condition
relative to rest baseline, which represent normalized indices of
effect sizes (Postle et al., 2000). These t-values (effect sizes) were
then used as dependent variables in random-effect group analyses
(Woods, 1996). Effect sizes for each condition and within the
superior frontal gyrus are shown in Fig. 2. For the superior frontal
gyrus cluster, a repeated measures ANOVA with the effect size as
dependent variable and sentence type (factual vs. counterfactual)
and effort (high vs. low effort) as factors revealed a main effect of
sentence type (F(1, 17)¼20.25, po.0001), with no other effect or
interaction. The same pattern was obtained in the right inferior
temporal gyrus (F(1, 17)¼9.89, p¼.006), and the left and right
parahippocampal gyri (F(1, 17)¼7.89, p¼.003; F(1, 17)¼19.13,
po.0001). These results thus confirm the sentence type effect in
the significant clusters of Fig. 1 (top and middle panel) and
furthermore indicate no further sub-threshold interactions or
effects within these clusters.
3.2. Effort effects in language
The higher-level analysis of the reading task within each hemisphere reported above (see Section 2.5) revealed no significant
cluster sensitive to the effort main effect. As can be inferred from
the analysis above (see Fig. 2), activity in pre-motor cortex was only
sensitive to the factuality manipulation. To specifically test the
hypothesis that activity within parietal structures varies as a
function of the effort conveyed in the sentence, we used an
anatomically defined ROI of inferior parietal structures containing
regions that were not fully included in the execution task (see
Section 2.5.1). This analysis revealed a cluster of voxels that was
sensitive to the effort manipulation: high effort vs. low effort
(cluster corrected, z¼2.3, p¼.05), and is shown in red and blue in
the bottom panel of Fig. 1. As for the factuality effect, we examined
whether the effort-sensitive language activity overlapped with that
of action execution by overlaying the execution results (shown in
green in Fig. 1) and the language results within the inferior parietal
ROI. The cluster in blue represents the overlap between the two,
and the areas in yellow shows a view of the parietal ROI. This
confirms that a portion of the parietal lobe that was engaged in
action execution was more sensitive to high-effort than low-effort
sentences, although other neighboring regions outside the execution results were also engaged. Indeed, the location of this parietal
cluster was more medial, including the inferior portion of the IPS
(see coordinates in Table 2), than in other language studies
contrasting action verbs (Noppeney et al., 2005; Tettamanti et al.,
2005, 2008; Moody & Gennari, 2010). This is consistent with a large
body of evidence in object manipulation (see above) and the fact
that our study compared the same action across different objects
(moving the sofa/the photograph) rather than across different verbs,
thus detecting object-specific action properties.
To examine this effort effect in more detail, and in particular,
to see whether the effort effect obtains for both counterfactual
and factual sentences, we extracted the effect sizes for each
sentence condition relative to rest baseline in each subject’s data,
and conducted further statistics across subjects. A repeated
measures ANOVA with effect size as the dependent variable and
sentence type (factual vs. counterfactual) and effort (high vs. low
effort) as factors revealed a main effect of effort (F(1, 17) ¼6.54,
p¼.02), with no other effect or interaction (Fig. 3). Planned
contrasts between high-effort vs. low-effort conditions revealed
similar effects. The contrasts between high-effort vs. low-effort
counterfactuals and between and high-effort vs. low-effort factual
sentences were significant in one-tailed tests (t(1,17)¼1.73,
p¼.05; t(1,17)¼1.94, p ¼.035). These results and the lack of
interaction suggest that the effort manipulation affected both
factual and counterfactual sentences alike, although numerically,
there was a larger effect in factual sentences. These results thus
suggest that counterfactual sentences elicit a similar pattern of
activity to factual sentences in parietal structures also recruited
for action execution.
3668
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
4. Discussion
The present study aimed to examine how the complex meaning of counterfactuals is processed and represented in the brain,
with special attention to the sensory-motor character of the
representations involved and the potential conflict between
multiple (factual and counterfactual) action representations. We
found that in the parietal cortex, but not the pre-motor cortex,
both factual and counterfactual sentences describing high-effort
actions elicited stronger responses than those describing loweffort actions. Moreover, counterfactual sentences elicited stronger activity than their factual counterparts in a large pre-motor
and pre-frontal medial cluster, including the SMA and extending
ventrally towards the cingulate gyrus and anteriorly along the
medial superior frontal gyrus. A counterfactual vs. factual effect
was also found in right inferior temporal and bilateral parahippocampal structures. The involvement of superior medial structures and the SMA suggests that control processes, rather than the
activation of action properties in pre-motor cortex, take place in
counterfactual comprehension, since the primary function of
these regions is inhibition and management of multiple action
representations (Mostofsky & Simmonds, 2008).
The present results suggest that action representations are
computed in parietal cortex whether or not the overall discourse
implies that the actions occur in the actual world. These action
representations are further processed by medial pre-frontal
structures, which receive afferent connections from the parietal
lobe (Mostofsky & Simmonds, 2008) and may manage the multiple (actual and possible) action representations in counterfactual
statements. This pattern of results indicates that whereas actionrelated activations appear similar to those of simple action
sentences in the parietal cortex, prefrontal medial structures,
which are commonly activated in discourse processing and social
cognition tasks, play an additional role in understanding counterfactuals. We discuss these results and their implications in more
detail below.
4.1. Effort modulation in factual and counterfactual sentences
The effort modulation in inferior parietal regions near the IPS,
which overlapped with regions active during action execution,
suggests that object-directed motor plans and object manipulation knowledge were recruited in language comprehension. Note
that the amount of effort implied by the sentences can only be
derived from the properties of the objects being talked about (e.g.,
photograph vs. sofa in Table 1), as the action referred to by the
verb is the same (e.g., moving). The recruitment of object-related
information is in line with numerous neuroimaging, electrophysiological and TMS studies. Inferior parietal structures, including
the IPS, have been shown to serve planning and execution of
actions, the understanding of action sentences, conceptual judgments about manipulable objects, judgments of someone’s beliefs
about an object’s weight, and processing of spatial relations
between objects (Damasio et al., 2001; Kellenbach et al., 2003;
Noppeney et al., 2005; Rueschemeyer et al., 2009). Specifically,
the IPS is active in planning object-oriented actions and is
proposed to encode transformations from object features such
as shape, size and weight into hand motor plans, thus supporting
the selection of appropriate motor programs for action (Jeannerod
and Decety, 1995; Rizzolatti et al., 1997; Binkofsky et al., 1998;
Murata et al., 2000; Grefkes et al., 2002; Johnson-Frey et al., 2005;
Rizzolatti et al., 2006). Lesion studies confirm this view, as
patients with lesion in inferior parietal regions have difficulty
with hand-object interactions (Buxbaum, 2001; Buxbaum et al.,
2005, 2007).
The focus on object-specific actions in the present study
explains that the language activity did not fully overlap with
the action execution results: the execution task was designed to
reveal hand-related execution areas and, due to practical limitations, did not include actions with objects of different size and
weight, which frequently involve other body parts, as the linguistic stimuli did. The focus on object-specific actions also explains
the differences in activation sites found across several language
studies. Studies reporting more lateral activation sites in inferior
parietal cortex have targeted contrasts across different actions,
i.e., verbs (Tettamanti et al., 2005; Noppeney et al., 2005) whereas
studies reporting IPS activity or more medial aspects of the
inferior parietal lobule have contrasted object manipulations
elicited by visual or linguistic stimuli (Chao & Martin, 2000; Just
et al., 2004, 2001; Binder, Desai, Graves & Conant, 2009). This is
consistent with Moody and Gennari’s effort results in that the
direct contrast between high- and low-effort sentences was not
significant in lateral parietal regions but the contrast across
actions was (is pushing the piano vs. has forgotten the piano).
More generally, the fact that both factual and counterfactual
sentences were similarly modulated by physical effort in parietal
structures suggests that the greater number of alternative representations implied by counterfactual statements did not modulate these regions. This is consistent with previous claims arguing
that the inferior parietal lobule encodes schematic action information linked to specific object properties but does not manage
competition between alternative representations (Glover, 2004).
Instead, such competitive processes are thought to involve dorsolateral, ventrolateral or medial prefrontal cortex, depending on
the nature of the stimulus and task (Miller & Cohen, 2001;
Simmonds et al., 2008; Thompson-Schill, 2003; ThompsonSchill, Bedny, & Goldberg, 2005). These pre-frontal regions have
been shown to cooperate with parietal regions during action
planning or execution in electrophysiological studies (Fuster,
2001; Quintana & Fuster, 1999), thus suggesting that a similar
functional network may be engaged during language
comprehension.
One issue that remains unexplained so far is the unexpected
absence of an effort effect in pre-motor cortex, even though our
study specifically targeted motor actions following previous
action and language studies (Chouinard, Leonard, & Paus, 2005;
Cramer et al., 2002; Moody & Gennari, 2010). Along the lines of
our discussion of the parietal results above, it is possible that the
effort contrast was not observed in this study because the stimuli
only included differences in the manipulated object (e.g., moving
the sofa vs. moving the photograph), whereas previous language
stimuli (Moody & Gennari, 2010) included a main effect of action
(forget the piano vs. push the piano) together with the effort effect
(forget the piano opush the chairopush the piano), thus identifying voxels sensitive to all contrasts. This is consistent with the
fact that object manipulations and object size and weight in
general are associated with the parietal cortex and not the pre-motor
cortex, as reviewed above (Just et al., 2001, 2004; Rueschemeyer
et al., 2009), whereas contrasts between actions (body-parts or
physical vs. non-physical actions) give rise to pre-motor effects
(Aziz-Zadeh et al., 2006; Hauk et al., 2004; Rüschemeyer, Brass, &
Friederici, 2007; Tettamanti et al., 2005). Thus, if pre-motor cortex
responds more strongly to action contrasts than object contrasts, this
may explain the absence of an effect in these regions in the
present study.
Another possible explanation for the lack of this effect is that
discourse stimuli involve more complex interactions across the
processing network, which may overshadow small effects in premotor cortex. Maintaining information across the antecedent and
consequent clauses, establishing a relationship between them and
inferring their implications for the actual world surely involve
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
additional processes and brain structures that are not present in
simple subject–verb–object sentences used in previous studies.
The proximity of pre-motor structures to the medial cluster found
here may have therefore led to lack of detection sensitivity when
more blood flow is required in a nearby region. This is consistent
with discourse processing studies, which tend to report activity in
medial structures (Speer et al., 2009; Yarkoni et al., 2008; Ferstl &
Von Cramon, 2001, Knauf et al., 2002; Goel, 2007), and with the
fact that increased BOLD response in a cluster is often accompanied with decreased BOLD response in surrounding regions—a
consequence of metabolic processes or the complex dynamics of
neural assemblies (Shmuel et al., 2002; Pasley et al., 2007).
Clearly, further research is needed to distinguish between these
alternative explanations.
4.2. Factuality modulations in pre-motor and prefrontal regions
The most interesting result obtained in this study is the
difference between counterfactual and factual statements in
medial pre-motor and pre-frontal structures. Although it is
possible that some portions of the observed medial structures
include regions encoding motor schemas, the recruitment of the
SMA and medial pre-frontal structures is typically taken to reflect
modulatory mechanisms, rather than the mere activation of
motor features (e.g., Miller & Cohen, 2001; Fuster, 2001). As
mentioned in the introduction, the SMA and medial prefrontal
structures are involved in modulating (inhibiting and selecting)
competing motor plans, often in conjunction with parietal structures (Mostofky & Simmonds, 2008). The critical role of these
regions in response selection and inhibition is demonstrated by
lesion studies and electrophysiological recordings (Isoda &
Hikosaka, 2007; Keysers & Gazzola, 2010; Mukamel, Ekstrom,
Kaplan, Iacoboni, & Fried, 2010; Nachev et al., 2007). Neurons
recorded in the human SMA during action observation and
execution for example exhibit anti-mirror properties, i.e., they
increase their firing during execution but decrease it during
observation, implying inhibitory mechanisms that suppress
unwanted imitation (Mukamel et al., 2010). Such inhibitory
processes may appear as increased activation in the BOLD signal,
as in many fMRI studies, because they consume as much energy
as excitatory processes (Keysers & Gazzola, 2010).
The multiple meanings of counterfactual sentences are therefore very likely to be responsible for the activations observed in
medial prefrontal structures. Compared to factual sentences
describing the same events, counterfactual sentences imply two
alternative representations of an event, one that did not actually
occur, and another that would have occurred, given the appropriate circumstances. Such representations require more cognitive effort, as they must be kept separate to fully understand the
counterfactual interpretation. This is consistent with evidence
suggesting that cognitive tasks requiring the management of
alternative representations engage medial pre-frontal regions, as
in discourse processing, and furthermore, with the fact that
prefrontal patients and young children are impaired in counterfactual reasoning (Beck et al., 2009; Gómez-Beldarrain et al.,
2005; Hanakawa et al., 2002; Owen, McMillan, Laird, &
Bullmore, 2005; Petrides, 2000; Tanaka, Honda, & Sadato, 2005).
Moreover, the fact that portions of the cingulate gyrus were
included within the main cluster of medial activations is also
consistent the role of this region in response conflict monitoring
(Botvinick, Cohen, & Carter, 2004). Thus, the modulatory role of
medial structures in behavior is also recruited for language
comprehension and other cognitive tasks when multiple representations need to be managed.
The multiple meaning of counterfactuals also explains the
additional parahippocampal and right temporal activations
3669
observed in this study. These structures are involved in lexical
and episodic memory retrieval (Paller & Wagner, 2002; St. George,
Kutas, Martinez, & Sereno, 1999) and the generation of alternative
episodic memories to be evaluated in consequential and future
thinking (Baird & Fugelsang, 2004; Nadel & Moscovitch, 2001;
Schacter & Addis, 2009; Schacter et al., 2007; Squire, Stark, &
Clark, 2004). This confirms the claim that counterfactuals recruit
episodic memories aiding the implicit evaluation of alternative
courses of action (Baird & Fugelsang, 2004; Barbey et al., 2009).
Critically for our current findings, human neurons recorded in the
parahippocampal gyrus during action observation and execution
display a pattern of activity similar to those found in medial prefrontal regions (Mukamel et al., 2010), suggesting that these
regions cooperate during action inhibition/selection, perhaps by
activating or deactivating memories relevant to the actions
at hand.
4.3. Implications for processing models and theories of semantic
representations
Our results have implications for theories of counterfactual
thinking and language processing. A model of counterfactual
thinking proposed by Baird and Fugelsang (2004), for example,
claims that several brain regions have specific roles in this
process: the basal ganglia are responsible for generating counterfactual alternatives; hippocampal structures generate relevant
memories feeding the construction of alternatives; the parietal
cortices generate mental images of possible scenarios; the cingulate gyrus evaluates alternatives and the dorsolateral prefrontal
cortex selects between the representations generated. Our results
confirm activations in the parietal lobule, parahippocampal
structures and the cingulate cortex. However, no activation
specific to counterfactuals was found in dorsolateral prefrontal
cortex and the basal ganglia.
Numerous language and working memory studies, including
lesion and TMS studies, have also indicated that the left inferior
frontal gyrus (LIFG) play a critical role in processing semantic
complexity and maintaining multiple representations in memory
(D’Esposito, Postle, Jonides, & Smith, 1999; Gennari, MacDonald,
Postle, & Seidenberg, 2007; Manenti, Cappa, Rossini, & Miniussi,
2008; Novick, Trueswell, & Thompson-Schill, 2005; ThompsonSchill et al., 2005). Consequently, the LIFG are pivotal in virtually
all models of language processing (Tyler & Marslen-Wilson,
2008). However, our results indicate that although the LIFG
supported linguistic processes common to all sentence types
(indeed, comparing all sentences to rest reveals activity in this
region), the LIFG did not appear to play a role in processing
counterfactuals over and above its contribution to processing
factual sentences.
One reason for the absence of factual vs. counterfactual
modulations in LIFG is that different processing networks are
engaged as a function of the semantic content of the stimuli and
task demands. Language studies indeed indicate that sentence
comprehension flexibly recruits largely distributed networks, in
which various posterior regions along the ventral and dorsal
pathways cooperate with pre-frontal cortex, most typically, the
LIFG. Sentences such as I appreciate sincerity describing abstract
semantic content, for example, establish connectivity between
regions different from those describing concrete actions, and
these in turn differ from those describing spatial relations
(Damasio et al., 2001; Ghio & Tettamanti, 2010; Just et al.,
2004). Thus, the fact that our counterfactual sentences describe
physical actions and that their understanding engages possible
and actual behaviors may explain the activation of control
processes near action networks, rather than typical language
networks such as the LIFG. In fact, the medial pre-frontal regions
3670
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
are ideally located to mediate between more posterior motorrelated structures and other frontal regions (Mostofky &
Simmonds, 2008) so that representations of actions are best
manipulated in this region, rather than in the more distant IFG.
Our results also have implications for understanding the
processing of negative action sentences, since counterfactuals
are implicitly negated statements. The statement If Pedro had
painted the room, he would have moved the sofa clearly indicates
that Pedro did not paint the room and did not move the sofa.
Some studies have found that negative sentences, compared to
affirmative ones, elicit decreased activations in motor cortex or in
the left-fronto-parietal action network typically involved in
understanding action sentences (Liuzza, Candidi, & Aglioti,
2011; Tettamanti et al., 2008; Tomasino, Weiss, & Fink, 2010).
The contrast between these results and ours may stem from many
factors. One is that previous studies have use simple sentences
(e.g., I press the button now vs. I do not press the button now) or
requests (e.g., move vs. do not move), which require much simpler
computations to understand. Therefore, the managing of alternative representations of the world, some factual ones (e.g., Pedro
did not paint the room) and some possible ones (e.g., he could
have moved the sofa) do not enter into consideration. Another
factor is that these studies investigated the explicit processing of
the word not – this is the only difference between conditions – in
the context of sentences referring to the self. In contrast, our
stimulus sentences do not contain explicit negation or reference
to the self and require linking semantic elements across the
antecedent and consequent (e.g., Pedro and he), as in other
discourse contexts. These differences in meaning and cognitive
demands explain why counterfactuals do not elicit suppressed
responses in motor-related regions: the effort effect does not
differ for factual and counterfactual sentences because the
difference in action content (moving the sofa vs. moving the
picture) must still be generated (and understood) for both possible
and actual events. Medial pre-frontal regions then manage the
difference between the possible and implicitly negated events to
keep them distinct. This is consistent with other studies investigating negation in the context of complex sentences such as
subordinate constructions, which do not find suppressed
responses and instead, find increased connectivity of the language
network with the SMA (Bahlmann, Mueller, Makuuchi, &
Friederici, 2011).
More generally, our results have implications for theories of
semantic representations in language comprehension. Sensorymotor approaches have argued that semantic representations are
grounded in, or organized around, sensory-motor systems that
underpin perception and action (Barsalou, Simmons, Barbey, &
Wilson, 2003; Martin & Chao, 2001). Most previous studies
within this approach however, have focused on the fact that
various properties of actions are shared across language and
action, suggesting that motor schemas for action are recruited
by language, as reviewed in the introduction. This is perhaps
unsurprising because words are bound to become associated
with the actions they refer to during the course of language
learning (Pulvermuller, 2005). The present results however
go further in suggesting that not only action representations but
also modulatory processes underpinning action selection and
inhibition are shared across domains. This commonality may
stem from evolutionary or ontogenetic processes in which behavioral control functions are co-opted for language and cognition
more generally, particularly if sensory-motor functions are taken
to have primacy over cognitive ones. Although this remains an
unresolved matter, our results extend the link between language
and action to modulatory mechanisms that are opportunistically
recruited if management of alternative action representations is
required.
5. Conclusions
In sum, a distinctive neural network partially overlapping with
action execution was shown to be involved in processing counterfactual statements describing effortful physical actions. Specifically, parietal structures, known to support object directed action
planning, were sensitive to the physical effort conveyed by the
sentence regardless of their degree of factuality. Furthermore, a
large cluster of pre-motor and pre-frontal medial areas responded
more strongly to counterfactual than factual statements, indicating the cognitive cost of managing and/or inhibiting alternative
actions representations. These results are consistent with
sensory-motor theories of meaning arguing that linguistic meaning and higher cognitive abilities are linked to our experience of
actions. Critically, the relevant experience not only includes
action representations but also their selection and manipulation,
supporting both action performance and counterfactual comprehension. This work thus constitutes an important step in understanding the neural underpinnings of counterfactual statements.
Acknowledgments
This research was supported by the Spanish MICIIN Grant
SEJ2007-66916 to Manuel de Vega, the researcher-training program FPI SEJ2004-02360 to Mabel Urrutia, and the Neurocog
Project sponsored by the ACIISI (Canary Islands) and by the ERDF
(European Union). We thank Aziz Asghar, Maribel Pulgarı́n
Montoya and researchers at the York Neuroimaging center for
their help in data collection and analyses.
Appendix A. Supplementary material
Supplementary data associated with this article can be found
in the online version at http://dx.doi.org/10.1016/j.neuropsycho
logia.2012.09.004.
References
Aziz-Zadeh, L., Wilson, S. M., Rizzolatti, G., & Iacoboni, M. (2006). Congruent
embodied representations for visually presented actions and linguistic phrases
describing actions. Current Biology, 16(18), 1818–1823.
Bahlmann, J., Mueller, J. L., Makuuchi, M., & Friederici, A. D. (2011). Perisylvian
functional connectivity during processing of sentential negation. Frontiers in
Psychology, 2(104), 1–10.
Baird, A., & Fugelsang, J. (2004). The emergence of consequential thought:
evidence from neuroscience. Philosophical Transactions of The Royal Society of
London: Series B, 359, 1797–1804.
Barbey, A. K., Krueger, F., & Grafman, J. (2009). Structured event complexes in the
prefrontal cortex support counterfactual representations for future planning.
Philosophical Transactions of the Royal Society of London: Biological Sciences, 364,
1291–1300.
Barsalou, L. W., Simmons, W. K., Barbey, A. K., & Wilson, C. D. (2003). Grounding
conceptual knowledge in modality-specific systems. Trends in Cognitive
Sciences, 7(2), 84–91.
Beck, S. R., Riggs, K. J., & Gorniak, S. L. (2009). Relating developments in children’s
counterfactual thinking and executive functions. Thinking and Reasoning, 15,
337–354.
Binder, J. R., Desai, R. H., Graves, W., & Conant, L. L. (2009). Where is the semantic
system? A critical review and meta-analysis of 120 functional neuroimaging
studies. Cerebral Cortex, 19, 2767–2796.
Binkofski, F., Dohle, C., Posse, S., Stephan, K. M., Hefter, H., Seitz, R. J., & Freund, H. J.
(1998). Human anterior intraparietal area subserves prehension: a combined
lesion and functional MRI activation study. Neurology, 50(5), 1253–1259.
Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and
anterior cingulate cortex: an update. Trends in Cognitive Sciences, 8(12),
539–546.
Boulenger, V., Hauk, O., & Pulvermüller, F. (2009). Grasping ideas with the motor
system: semantic somatotopy in idiom comprehension. Cerebral Cortex, 19, 8.
Bub, D. N., Masson, M. E., & Cree, G. S. (2008). Evocation of functional and
volumetric gestural knowledge by objects and words. Cognition, 106(1), 27–58.
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
Buxbaum, L. J. (2001). Ideomotor apraxia: a call to action. Neurocase, 7(6),
445–458.
Buxbaum, L. J., Johnson-Frey, S. H., & Bartlett-Williams, M. (2005). Deficient
internal models for planning hand-object interactions in apraxia. Neuropsychologia, 43(6), 917–929.
Buxbaum, L. J., Kyle, K., Grossman, M., & Coslett, H. B. (2007). Left inferior parietal
representations for skilled hand-object interactions: evidence from stroke and
corticobasal degeneration. Cortex, 43(3), 411–423.
Byrne, R. M. J. (2002). Mental models and counterfactual thoughts about what
might have been. Trends in Cognitive Sciences, 6(10), 426–431.
Byrne, R. M. J., & Tasso, A. (1999). Deductive reasoning with factual, possible, and
counterfactual conditionals. Memory and Cognition, 27, 309–323.
Caligiore, D., Borghi, A. M., Parisi, D., & Baldassarre, G. (2010). TRoPICALS: a
computational embodied neuroscience model of compatibility effects. Psychological Review, 117(4), 1188–1228.
Chao, L. L., & Martin, A. (2000). Representation of manipulable man-made objects
in the dorsal stream. NeuroImage, 12(4), 478–484.
Chen, C. Y., Muggleton, N. G., Tzeng, O. J., Hung, D. L., & Juan, C. H. (2009). Control
of prepotent responses by the superior medial frontal cortex. NeuroImage,
44(2), 537–545.
Chouinard, P. A., Leonard, G., & Paus, T. (2005). Role of the primary motor and
dorsal premotor cortices in the anticipation of forces during object lifting.
Journal of Neuroscience, 25(9), 2277–2284.
Cramer, S. C., Weisskoff, R. M., Schaechter, J. D., Nelles, G., Foley, M., Finklestein, S. P.,
& Rosen, B. R. (2002). Motor cortex activation is related to force of squeezing.
Human Brain Mapping, 16(4), 197–205.
Culham, J. C., & Valyear, T. K. (2006). Human parietal cortex in action. Current
Opinion in Neurobiology, 16, 205–212.
D’Esposito, M., Postle, B.R., Jonides, J., & Smith, E.E. (1999). The neural substrate
and temporal dynamics of interference effects in working memory as revealed
by event-related functional MRI. Proceedings of the National Academy of
Sciences of the United States of America, 96(13), 7514–7519.
Damasio, H., Grabowski, T. J., Tranel, D., Ponto, L. L., Hichwa, R. D., & Damasio, A. R.
(2001). Neural correlates of naming actions and of naming spatial relations.
NeuroImage, 13(6), 1053–1064.
Davis, C. J., & Perea, M. (2005). BuscaPalabras: A program for deriving orthographic
and phonological neighborhood statistics and other psycholinguistic indices in
Spanish. Behavior Research Methods, 37, 665–671.
De Vega, M., & Urrutia, M. (2011). Counterfactual sentences activate embodied
meaning. An action-sentence compatibility effect study. Journal of Cognitive
Psychology, 23(8), 962–973.
De Vega, M., Urrutia, M., & Riffo, B. (2007). Cancelling updating in the comprehension of counterfactuals embedded in narratives. Memory and Cognition, 35(6),
1410–1421.
Ferstl, E. C., Neumann, J., Bogler, C., & von Cramon, D. Y. (2008). The extended
language network: a meta-analysis of neuroimaging studies on text comprehension. Human Brain Mapping, 29(5), 581–593.
Ferstl, E. C., & von Cramon, D. Y. (2001). The role of coherence and cohesion in text
comprehension: an event-related fMRI study. Cognitive Brain Research, 11(3),
325–340.
Fritz, U., & Frith, C. D. (2003). Development and neurophysiology of mentalizing.
Philosophical Transactions of the Royal Society of London B, 358, 459–473.
Fuster, J. M. (2001). The prefrontal cortex—an update: time is of the essence.
Neuron, 30(2), 319–333.
Gennari, S. P., MacDonald, M. C., Postle, B. R., & Seidenberg, M. S. (2007). Contextdependent interpretation of words: evidence for interactive neural processes.
NeuroImage, 35(3), 1278–1286.
Ghio, M., & Tettamanti, M. (2010). Semantic domain-specific functional integration
for action-related vs. abstract concepts. Brain and Language, 112(3), 223–232.
Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin and Review, 9(3), 558–565.
Glover, S. (2004). Separate visual representations in the planning and control of
action. Behavioral and Brain Sciences, 27, 3–78.
Goel, V. (2007). Anatomy of deductive reasoning. Trends in Cognitive Sciences,
11(10), 435–441.
Gómez-Beldarrain, M., Garcı́a-Monco, J. C., Astigarraga, E., González, A., & Grafman, J.
(2005). Only spontaneous counterfactual thinking is impaired in patients with
prefrontal cortex lesions. Cognitive Brain Research, 24, 723–726.
Grefkes, C., Weiss, P. H., Zilles, K., & Fink, G. R. (2002). Crossmodal processing of
object features in human anterior intraparietal cortex: an fMRI study implies
equivalencies between humans and monkeys. Neuron, 35, 173–184.
Gre zes, J., & Decety, J. (2001). Functional anatomy of execution, mental simulation,
observation, and verb generation of actions: a meta-analysis. Human Brain
Mapping, 12(1), 1–19.
Hanakawa, T., Honda, M., Sawamoto, N., Okada, T., Yonekura, Y., Fukuyama, H., &
Shibasaki, H. (2002). The role of rostral Brodmann area 6 in mental
operation tasks: an integrative neuroimaging approach. Cerebral Cortex, 12,
1157–1170.
Hauk, O., Johnsrude, I., & Pulvermuller, F. (2004). Somatotopic representation of
action words in human motor and premotor cortex. Neuron, 41(2), 301–307.
Indefrey, P., Kleinschmidt, A., Merboldt, K.-D., Krüger, G., Brown, C., Hagoort, P., &
Frahm, J. (1997). Equivalent responses to lexical and nonlexical visual stimuli
in occipital cortex: a functional magnetic resonance imaging study. NeuroImage, 5, 78–81.
Isoda, M., & Hikosaka, O. (2007). Switching from automatic to controlled action by
monkey medial frontal cortex. Nature Neuroscience, 10(2), 240–248.
3671
Jeannerod, M., & Decety, J. (1995). Mental motor imagery: a window into the
representational stages of action. Current Opinion in Neurobiology, 5(6),
727–732.
Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002). Improved optimisation
for the robust and accurate linear registration and motion correction of brain
images. NeuroImage, 17(2), 825–841.
Jenkinson, M., & Smith, S. M. (2001). A global optimisation method for robust
affine registration of brain images. Medical Image Analysis, 5(2), 143–156.
Johnson-Frey, S. H., Newman-Norlund, R., & Grafton, S. T. (2005). A distributed left
hemisphere network active during planning of everyday tool use skills.
Cerebral Cortex, 15(6), 681–695.
Just, M. A., Carpenter, P. A., Maguire, M., Diwadkar, V., & McMains, S. (2001).
Mental rotation of objects retrieved from memory: an fMRI study
of spatial processing. Journal of Experimental Psychology: General, 130,
493–504.
Just, M. A., Newman, S. D., Keller, T. A., McEleney, A., & Carpenter, P. A. (2004). Imagery
in sentence comprehension: an fMRI study. NeuroImage, 21(1), 112–124.
Kellenbach, M. L., Brett, M., & Patterson, K. (2003). Actions speak louder than
functions: the importance of manipulability and action in tool representation.
Journal of Cognitive Neuroscience, 15(1), 30–45.
Keysers, C., & Gazzola, V. (2010). Social neuroscience: mirror neurons recorded in
humans. Current Biology, 20(8), 27.
Knauff, M., Mulack, T., Kassubek, J., Salih, H. R., & Greenlee, M. W. (2002). Spatial
imagery in deductive reasoning: a functional MRI study. Cognitive Brain
Research, 13(2), 203–212.
Kratzer, A. (1989). An investigation of the lumps of thought. Linguistic and
Philosophy, 12, 607–653.
Krueger, F., Barbey, A. K., & Grafman, J. (2009). The medial prefrontalcortex
mediates social event knowledge. Trends in Cognitive Sciences, 13(3), 103–109.
Krueger, F., & Grafman, J. (2008). The human prefrontal cortex stores structured
event complexes. In: T. Shipley, & J. M. Zacks (Eds.), Understanding events:
From perception to action (pp. 617–638). New York: Oxford University
Press.
Lewis, D. K. (1973). Counterfactuals. Oxford, UK: Basil Blackwell.
Liuzza, M. T., Candidi, M., & Aglioti, S. M. (2011). Do not resonate with actions:
sentence polarity modulates cortico-spinal excitability during action-related
sentence reading. Plos One, 6(2), e16855.
Manenti, R., Cappa, S. F., Rossini, P. M., & Miniussi, C. (2008). The role of the
prefrontal cortex in sentence comprehension: an rTMS study. Cortex, 44,
337–344.
Martin, A., & Chao, L. L. (2001). Semantic memory and the brain: structure and
processes. Current Opinion in Neurobiology, 11(2), 194–201.
McNamara, P., Durso, R., Brown, A., & Lynch, A. (2003). Counterfactual cognitive
deficit in persons with Parkinson’s disease. Journal of Neurology, Neurosurgery,
and Psychiatry, 74, 1065–1070.
Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex
function. Annual Review of Neuroscience, 24, 167–202.
Moody, C. L., & Gennari, S. P. (2010). Effects of implied physical effort in sensorymotor and pre-frontal cortex during language comprehension. NeuroImage, 49,
782–793.
Mostofsky, S. H., & Simmonds, D. J. (2008). Response inhibition and response
selection: two sides of the same coin. Journal of Cognitive Neuroscience, 20(5),
751–761.
Mukamel, R., Ekstrom, A. D., Kaplan, J., Iacoboni, M., & Fried, I. (2010). Singleneuron responses in humans during execution and observation of actions.
Current Biology, 20(8), 750–756.
Murata, A., Gallese, V., Luppino, G., Kaseda, M., & Sakata, H. (2000). Selectivity for
the shape, size, and orientation of objects for grasping in neurons of monkey
parietal area AIP. Journal of Neurophysiology, 83(5), 2580–2601.
Nachev, P., Wydell, H., O’Neill, K., Husain, M., & Kennard, C. (2007). The role of the
pre-supplementary motor area in the control of action. NeuroImage, 36,
T155–T163.
Nadel, L., & Moscovitch, M. (2001). The hippocampal complex and long-term
memory revisited. Trends in Cognitive Neuroscience, 5, 228–230.
Nieuwland, M. S. (2012). Establishing propositional truth-value in counterfactual
and real-world contexts during sentence comprehension: differential sensitivity of the left and right inferior frontal gyri. NeuroImage, 59, 3433–3440.
Nieuwland, M. S., & Martin, A. E. (2012). If the real world were irrelevant, so to
speak: the role of propositional truth-value in counterfactual sentence
comprehension. Cognition, 122, 102–109.
Noppeney, U., Josephs, O., Kiebel, S., Friston, K. J., & Price, C. J. (2005). Action selectivity
in parietal and temporal cortex. Cognitive Brain Research, 25(3), 641–649.
Novick, J. M., Trueswell, J. C., & Thompson-Schill, S. L. (2005). Cognitive control and
parsing: reexamining the role of Broca’s area in sentence comprehension.
Cognitive, Affective and Behavioral Neuroscience, 5(3), 263–281.
Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005). N-back working
memory paradigm: a meta-analysis of normative functional neuroimaging
studies. Human Brain Mapping, 25, 46–59.
Paller, K. A., & Wagner, A. D. (2002). Observing the transformation of experience
into memory. Trends in Cognitive Sciences, 6, 93–102.
Pasley, B. N., Inglis, B. A., & Freeman, R. D. (2007). Analysis of oxygen metabolism
implies a neural origin for the negative BOLD response in human visual cortex.
NeuroImage, 36(2), 269–276.
Petrides, M. (2000). Dissociable roles of mid-dorsolateral prefrontal and anterior
inferotemporal cortex in visual working memory. Journal of Neuroscience,
20(19), 7496–7503.
3672
M. Urrutia et al. / Neuropsychologia 50 (2012) 3663–3672
Postle, B. R., Zarahn, E., & D’Esposito, M. (2000). Using event-related fMRI to assess
delay-period activity during performance of spatial and nonspatial working
memory tasks. Brain research protocols, 5(1), 57–66.
Pulvermuller, F. (2005). Brain mechanisms linking language and action. Nature
Reviews Neuroscience, 6(7), 576–582.
Quintana, J., & Fuster, J. M. (1999). From perception to action: temporal integrative
functions of prefrontal and parietal neurons. Cerebral Cortex, 9(3), 213–221.
Rizzolatti, G., Ferrari, P. F., Rozzi, S., & Fogassi, L. (2006). The inferior parietal
lobule: where action becomes perception. Novartis Foundation Symposium, 270,
129–140 discussion 140-125.
Rizzolatti, G., Fogassi, L., & Gallese, V. (1997). Parietal cortex: from sight to action.
Current Opinion in Neurobiology, 7(4), 562–567.
Roese, N., & Olson, J. (1995). What might have been the social psychology of
counterfactual thinking. Mahwah, New Jersey: Lawrence Erlbaum Associates.
Rueschemeyer, S. A., van Rooij, D., Lindemann, O., Willems, R. M., & Bekkering, H.
(2009). The function of words: distinct neural correlates for words denoting
differently manipulable objects. Journal of Cognitive Neuroscience, 22(8),
1844–1851.
Rüschemeyer, S. A., Brass, M., & Friederici, A. D. (2007). Comprehending prehending: neural correlates of processing verbs with motor stems. Journal of
Cognitive Neuroscience, 19(5), 855–865.
Rushworth, M. F., Hadland, K. A., Paus, T., & Sipila, P. K. (2002). Role of the human
medial frontal cortex in task switching: a combined fMRI and TMS study.
Journal of Neurophysiology, 87(5), 2577–2592.
Santana, E., & de Vega, M. (2011). Metaphors are embodied, and so are their literal
counterparts. Frontiers in Psychology, 2, 90.
Schacter, D. L., & Addis, D. R. (2009). On the nature of medial temporal lobe
contributions to the constructive simulation of future events. Philosophical
Transactions of the Royal Society B, 364, 1245–1253.
Schacter, D. L., Addis, D. R., & Buckner, R. L. (2007). The prospective brain:
remembering the past to imagine the future. Nature Reviews Neuroscience, 8,
657–661.
Shmuel, A., Yacoub, E., Pfeuffer, J., Van de Moortele, P. F., Adriany, G., Hu, X., &
Ugurbil, K. (2002). Sustained negative BOLD, blood flow and oxygen consumption response and its coupling to the positive response in the human brain.
Neuron, 36(6), 1195–1210.
Scorolli, C., Borghi, A. M., & Glenberg, A. (2009). Language-induced motor activity
in bi-manual object lifting. Experimental Brain Research, 193(1), 43–53.
Simmonds, D. J., Pekar, J. J., & Mostofsky, S. H. (2008). Meta-analysis of Go/No-go
tasks demonstrating that fMRI activation associated with response inhibition
is task-dependent. Neuropsychologia, 46, 224–232.
Speer, N. K., Reynolds, J. R., Swallow, K. M., & Zacks, J. M. (2009). Reading stories
activates neural representations of visual and motor experiences. Psychological
Science, 20(8), 989–999.
Squire, L. R., Stark, C. E. L., & Clark, R. E. (2004). The medial temporal lobe. Annual
Review of Neuroscience, 27, 279–306.
St. George, M., Kutas, M., Martinez, A., & Sereno, M. I. (1999). Semantic integration
in reading: engagement of the right hemisphere during discourse processing.
Brain, 122, 1317–1325.
Tanaka, S., Honda, M., & Sadato, S. (2005). Modality-specific cognitive function of
medial and lateral human Brodmann area 6. The Journal of Neuroscience, 25(2),
496–501.
Tettamanti, M., Buccino, G., Saccuman, M. C., Gallese, V., Danna, M., Scifo, P., &
Perani, D. (2005). Listening to action-related sentences activates frontoparietal motor circuits. Journal of Cognitive Neuroscience, 17(2), 273–281.
Tettamanti, M., Manenti, R., Della Rosa, P., Falini, A., Perani, D., Cappa, S., & Moro, A.
(2008). Negation in the brain: modulating action representations. NeuroImage,
43, 358–367.
Thompson-Schill, S. L. (2003). Neuroimaging studies of semantic memory:
inferring ‘‘how’’ from ‘‘where’’. Neuropsychologia, 41(3), 280–292.
Thompson-Schill, S. L., Bedny, M., & Goldberg, R. F. (2005). The frontal lobes and the
regulation of mental activity. Current Opinion in Neurobiology, 15(2), 219–224.
Tomasino, B., Weiss, P. H., & Fink, G. R. (2010). To move or not to move:
imperatives modulate action-related verb processing in the motor system.
Neuroscience, 169(1), 246–258.
Tucker, M., & Ellis, R. (2004). Action priming by briefly presented objects. Acta
Psychologica, 116(2), 185–203.
Tyler, L. K., & Marslen-Wilson, W. (2008). Fronto-temporal brain systems supporting spoken language comprehension. Philosophical Transactions of the Royal
Society of London—Series B: Biological Sciences, 363(1493), 1037–1054.
Urrutia, M., De Vega, M., & Bastiaansen, M. (2012). Understanding counterfactuals
in discourse modulates ERP and oscillatory gamma rhythms in the EEG. Brain
Research, 1455, 40–55.
Ursu, S., & Carter, C. S. (2005). Outcome representations, counterfactual comparisons and the human orbitofrontal cortex: implications for neuroimaging
studies of decision-making. Cognitive Brain Research, 23(1), 51–60.
Wilson, N., & Gibbs, R. (2007). Real and imagined body movement primes
metaphor comprehension. Cognitive Science, 31(4), 721–731.
Woods, R. P. (1996). Modeling for intergroup comparisons of imaging data.
NeuroImage, 4(3), S84–94.
Woolrich, M. W., Ripley, B. D., Brady, J. M., & Smith, S. M. T. (2001). Temporal
autocorrelation in univariate linear modelling of fMRI data. NeuroImage, 14(6),
1370–1386.
Worsley, K. J., Evans, A. C., Marrett, S., & Neelin, P. (1992). A three-dimensional
statistical analysis for CBF activation studies in human brain. Journal of
Cerebral Blood Flow and Metabolism, 12, 900–918.
Yarkoni, T., Speer, N. K., & Zacks, J. M. (2008). Neural substrates of narrative
comprehension and memory. NeuroImage, 41(4), 1408–1425.
Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: motor resonance in
language comprehension. Journal of Experimental Psychology: General, 135, 1–11.