Unusual Actions 1
Unusual Actions do not always trigger the Mentalizing Network
Lisa Ampe, Ning Ma, Nicole Van Hoeck, Marie Vandekerckhove, Frank Van Overwalle
Vrije Universiteit Brussel, Belgium
We are very grateful to Floris de Lange and his colleagues Marjolein Spronk, Roel Willems, Ivan Toni,
and Harold Bekkering for providing us with their stimulus material. We would like to thank Floris de
Lange in particular, for his valuable feedback on the experimental design.
This research was supported by a PhD research Fellowship to the first author from the Research
Foundation – Flanders (FWO). Address for correspondence: Frank Van Overwalle, Department of
Psychology, Vrije Universiteit Brussel, Pleinlaan 2, B - 1050 Brussel, Belgium; or by e-mail:
[email protected].
Running Head: Unusual Actions
[Lisa_UnusualIntentions-2500]
16 October, 2012
Unusual Actions 2
Abstract
Past fMRI research has demonstrated that to understand other people’s behavior shown visually,
the mirror network is strongly involved. However, the mentalizing network is also recruited when a
visually presented action is unusual and/or when perceivers think explicitly about the intention. To
further explore the conditions that trigger mentalizing activity, we replicated one of such studies (de
Lange, Spronk, Willems, Toni, and Bekkering, 2008) under the minimal instruction to “view” pictures
of unusual actions, without giving any “intention” instruction as in the original study. Contrary to earlier
research, merely viewing unusual actions did not activate mentalizing areas. Instead, the dorsal anterior
cingulate cortex was activated. We conclude that unusual actions are not sufficient by themselves to
trigger mentalizing. In order to activate the mentalizing network without an intention instruction, a
richer action context informative of the implausibility of the action might be a prerequisite.
Unusual Actions 3
As social beings, people constantly try to make sense of the social world around them and to
understand other people’s actions. On the brain level, there is a growing consensus that action
understanding may involve two independent networks (cf. meta-analysis by Van Overwalle & Baetens,
2009). First, a mirror network that matches observed behavior to one’s own behavioral repertoire and
the goals associated with them, and so allows to understand the goals of others at a basic, perceptual
level. Second, a mentalizing network that supports action and goal understanding at a more abstract
level through inferential processes, also known as theory of mind.
However, recent evidence suggests that when observable actions are unusual, the mirror network
alone seems insufficient for understanding observable actions, and mentalizing areas are also recruited
(Van Overwalle & Baetens, 2009). For example, when people observe actions that are erroneous,
pretended or faked, or implausible given physical constraints (Brass, Schmitt, Spengler, & Gergely,
2007; Liepelt, von Cramon, & Brass, 2008). The mentalizing network is also activated during action
observation when people think consciously about the intention of an action (Buccino et al., 2007; de
Lange, Spronk, Willems, Toni, & Bekkering, 2008; Spunt, Falk, & Lieberman, 2010; Spunt, Satpute, &
Lieberman, 2011). However, a few studies did not find mentalizing activity given unusual actions (de
Lange et al., 2008; Jastorff, Clavagnier, Gergely & Orban, 2011; Manthey, Schubotz, & von Cramon,
2003), even when the conditions were quite similar to the other studies mentioned above. For instance,
de Lange et al. (2008) presented an actor holding various objects at unusual places of the head (e.g.,
holding a cup at one’s ear instead of in front of one’s mouth) and mentalizing areas were recruited only
when instructions were given to judge the intention of the action, not when the means by which the
action was carried out had to be judged (e.g., the hand grip holding the cup).
Given these conflicting results, the question remains which specific conditions need to be met for
unusual actions to trigger the mentalizing network. To explore this issue further, we replicated de Lange
et al. (2008) under spontaneous “viewing” rather than explicit instructions to attend to the intention or
means of the action. Indeed, it is possible that an instruction to judge the means in the study by de Lange
et al. (2008), rather than acting as a neutral condition, may actually have focused the participants’
attention away from the implied intention in atypical behaviors. To avoid such distraction, a “viewing”
Unusual Actions 4
instruction seems more neutral and may allow for spontaneous mentalizing about intentionality. Based
on earlier research, we expect under such spontaneous instruction to replicate the original findings of de
Lange et al. (2008). Specifically, we predict activity in the mentalizing network for unusual action
intentions (i.e., unusual object locations). In addition, for unusual means (e.g., unusual hand grip), we
also predict activity in the extrastriate body area (EBA) and anterior intraparietal sulcus (aIPS; part of
the mirror network).
Method
Participants
Twenty healthy participants (16 women and 4 men, mean age = 21.7, age range = 18 – 27 years),
all students at the Ghent University or at the Vrije Universiteit Brussel took part in the experiment.
Participants reported no history of neurological, major medical, or psychiatric disorders. Other exclusion
criteria were left handedness, regular taking of medication or drugs and contraindications to fMRI such
as pregnancy, claustrophobia, metallic implants, etc. All participants had normal or corrected-to-normal
vision. Three additional participants were excluded from further analysis after the experiment; one
because of too much body movement in the scanner (more than 10% outlier scans), one because of poor
attention to the stimuli (only 24 % catch trials correct), and one because of a scanner crash. The
experiment was approved by the Medical Ethics Committee at the University Hospital of Ghent and
Brussels. Informed consent was obtained from the participants and they received 10 euro for their
participation.
Procedure and Stimulus Material
The stimuli and the design were replicated from de Lange et al. (2008). In brief, four types of
actions were shown: normal actions (e.g., an actor bringing a coffee cup to her mouth), actions with an
unusual intention (e.g., an actor bringing a coffee cup to her ear), actions performed with unusual means
(e.g., an actor bringing a coffee cup to her mouth while holding the cup with a power grip) and actions
with both an unusual intention and performed with unusual means (e.g., an actor bringing a coffee cup to
Unusual Actions 5
her ear while holding the cup with a power grip). Participants saw 168 pictures (14 objects each shown
while the actor looked in 3 directions [left – front – right] for 4 conditions) in a randomized order, each
for a duration of 3 s followed by a blank screen for 2 s and a random jitter (0 – 1 s). After a set of 20
pictures, there was a pause of 35 s (i.e., blank screen with pause message).
In contrast to de Lange et al. (2008), there were no explicit instructions to attend to the intention
nor to the means of the action. Instead, participants were only instructed to watch the pictures carefully.
Moreover, while de Lange et al. (2008) used blocks of 6 - 7 stimuli (from the same condition) preceded
by the instruction, the present study used a random presentation of stimuli. Given the lack of a specific
instruction, we included 10% additional pictures as catch trials (i.e., an actor interacting with an earring)
to which participants had to react with a button press. All participants in the analysis had 94 % or more
of the catch trials correct. A pilot study (n = 76) prior to the experiment confirmed that for the 14 objects
used under the scanner, actions with an unusual intention (M = 6.70, SD = 1.74) were perceived as more
unusual on a 0-10 scale than actions with an unusual means (M = 4.71, SD = 1.63; t(13) = 3.12, p <
.001) or normal actions (M = 1.83, SD = 1.02; t(13) = 9.60, p < .001). Nevertheless, the majority of
participants in the pilots were able to come up with at least one goal interpretation of each action (51 %
– 100 %), although the proportion of goals provided was lower for actions with an unusual intention (M
= 77 %, SD = 11 %) than for actions with an unusual means (M = 92 %, SD = 7 %; t(13) = 4.04, p <
.001) or normal actions (M = 98 %, SD = 3 %; t(13) = 6.38, p < .001).
Imaging Procedure
Images were collected with a 3 Tesla Magnetom Trio MRI scanner system (Siemens Medical
Systems, Erlangen, Germany), using an 8-channel radiofrequency head coil. Stimuli were projected onto
a screen at the end of the magnet bore that participants viewed by way of a mirror mounted on the head
coil. Stimulus presentation was controlled by E-Prime 2.0 (www.pstnet.com/eprime; Psychology
Software Tools) under Windows XP. Immediately prior to the experiment, participants completed a
brief practice session. Foam cushions were placed within the head coil to minimize head movements. A
high-resolution T1-weighted structural scan (MP-RAGE) was first collected, followed by one functional
Unusual Actions 6
run of about 770 volume acquisitions (30 axial slices; 4mm thick; 1mm skip) of which the first 54 scans
for practice trials were omitted. Functional scanning used a gradient-echo echoplanar pulse sequence
(TR = 2 s; TE = 33 ms; 3.5 × 3.5 x 4.0 mm in-plane resolution).
Image Processing and Statistical Analysis
The fMRI data were preprocessed and analyzed using SPM8 (Statistical Parametric Mapping;
The Wellcome Trust Centre for NeuroImaging, London, UK). For each functional run, data were
preprocessed to remove sources of noise and artifact. Functional data were corrected for differences in
acquisition time between slices for each whole-brain volume, realigned within and across runs to correct
for head movement, and co-registered with each participant’s anatomical data. Functional data were then
transformed into a standard anatomical space (2 mm isotropic voxels) based on the ICBM 152 brain
template (Montreal Neurological Institute), which approximates Talairach and Tournoux atlas space.
Normalized data were then spatially smoothed (6 mm full-width-at-half-maximum [FWHM]) using a
Gaussian kernel. Additional artifact analysis was performed on the realigned data, using the Artifact
Detection Tool software package (ART; http://www.nitrc.org/projects/artifact_detect/), to detect and
correct for excessive movement artifacts. Outlier scan movements (identified by assessing between-scan
differences with Z-threshold: 3.0, scan to scan movement threshold: 0.5 mm; rotation threshold: 0.02
radians) were omitted in the statistical analysis by including in the general linear model a single
regressor for each outlier (i.e., bad scan). No correlations between motion and experimental design or
global signal and experimental design were identified. Six directions of motion parameters from the
realignment step as well as outlier time points (defined by ART) were included as nuisance regressors.
We used a default high-pass filter of 128s and serial correlations were accounted for by the default autoregressive AR(1) model.
Statistical analyses involved a first-level single participant event-related design with a regressor
for each condition time-locked at the presentation of the picture, and additional movement and nuisance
artifact regressors, and applying a canonical response function (duration set to 0) using the general linear
model of SPM8. Contrasts of interest were performed at the individual first-level using simple t-tests,
Unusual Actions 7
and then at the group second-level on the parameter estimates (regressors) associated with each firstlevel contrast. This analysis proved to be more sensitive than a contrast analysis at the second level,
using the parameter estimates of each condition. Given the lack of specific instructions, a lenient wholebrain threshold of p ≤ 0.005 (uncorrected) was used for all comparisons with a cluster extent of 10
voxels (see also Ma et al., 2011). Statistical comparisons between conditions of interest are reported
after correction for multiple comparisons using the non-parametric test statistic developed by Slotnick,
Moo, Segal, and Hart (2003), which requires a cluster extent of 49 voxels for a corrected p < .05 (at a
whole brain uncorrected threshold of p < .005).
Results
A Unusual Intention > Normal contrast revealed significant activation only in the dorsal anterior
cingulate cortex [dACC]; Table 1, Figure 1A). The Unusual Means > Normal contrast showed
significant activation in the predicted areas, including the bilateral extrastriate body area (EBA)
embedded in the temporal cortex and the bilateral anterior intraparietal sulcus (aIPS) embedded in the
parietal lobule. In addition, there was activation in the left postcentral gyrus and the bilateral middle
occipital gyrus. The contrast combining both Intention and Means violations > Normal showed no
effects.
To explore whether the lack of activation in mentalizing areas in the Unusual Intention > Normal
contrast was due to low sensitivity of the whole-brain analysis, we also conducted a region of interest
(ROI) analysis with small volume correction of key mentalizing regions. We constructed ROIs with
spheres of 8 mm around coordinates derived from the meta-analyses by Van Overwalle (2009) and Van
Overwalle and Baetens (2009) for the bilateral TPJ (±50, -55, 25) and mPFC (0, 50, 20). These analyses
revealed no significant effect (Figure 1B; all clusters had 0 voxels at p < .005 uncorrected). We also
extracted the percentage signal change from the same ROIs with a 15 mm radius. There was no
significant effect of condition in any ROI (all Fs < 1).
Unusual Actions 8
Discussion
Recent evidence suggests that observing an unusual action (intention) triggers the mentalizing
network instead of, or in addition to, the mirror network. The aim of this study was to investigate
whether mentalizing activation for unusual actions also occurs under minimal conditions that do not
explicitly invite observers to infer the intention of an actor and do not draw attention away from the
whole action (to avoid focusing on intention-irrelevant aspects of the action). To begin with our main
hypothesis, we did not find mentalizing activity when people observed actions with an unusual intention
(i.e., unusual object locations) under minimal viewing conditions. Surprisingly, we also failed to detect
mirror activity in this condition, contrary to de Lange et al. (2008). Instead we found significant
activation in the dACC. As this latter area is involved in conflict detection and monitoring (Botvinick,
Cohen, & Carter, 2004), this suggests that participants noticed that there was a violation of the typical
behavioral execution in this type of actions.
With respect to unusual means (e.g., unusual hand grip), in line with de Lange et al. (2008), we
found that observing actions performed with unusual means versus normal actions activated the EBA, an
area associated with the visual processing of the human body (Downing, Jiang, Shuman, & Kanwisher,
2001; Peelen & Downing, 2007). We also observed activations in the right aIPS which is part of the
mirror network (Rizzolatti & Craighero, 2004; Van Overwalle & Baetens, 2009) and in temporal and
parietal areas that are part of a “functional circuit” involved in practical (and to some extent conceptual)
tool-use knowledge (for a review, see Lewis, 2006).
There might be several reasons why mentalizing activity for unusual actions was not observed
under minimal instructions. One interesting explanation is the poor context in which the action was
embedded which might have reduced the social relevance to the observer and hence his or her
spontaneous motivation to make mentalizing inferences. In our study, apart from the object held by the
same person in front of the same white background, there were no contextual clues that could inform the
observer about the meaning or the unusualness of the action. Most other relevant studies depicted
unusual actions in a much richer context. For instance, in Brass et al. (2007), a light switch was operated
with a knee (rather than finger) while the actor’s hands were empty (implausible action) versus while
Unusual Actions 9
both hands were occupied by a stack of books (plausible action; see also Liepelt et al., 2008). Note that
participants paid sufficient attention to the material itself in the present study, because most of them
detected the inconsistency and showed activity in the relevant brain area (dACC).
Another explanation is that perhaps participants could easily generate plausible alternative goals
for the unusual actions. Our pilot study show that, remarkably, the majority of participants had no
difficulty in finding a goal for unusual actions (on average 77 % gave a plausible goal; e.g., in the case
where the person is holding a camera to her ear, participants came up with goals such as “listening to the
camera to check whether it still works”). Under the scanner, this ease may have triggered little additional
mentalizing (or even mirror) activity in comparison with normal actions. Perhaps, this ease of goal
explanation might have been true also for actions with combined unusual intention and means (although
we have no pilot data on this condition), which may explain why these elicited no activation compared
to normal actions.
Still another explanation for the lack of activation in mentalizing brain areas is the modification
of de Lange et al.’s (2008) blocked design (blocks of 6 - 7 stimuli from the same condition) into the
present event-related design using a randomized stimulus order. Although this event-related design
might have reduced the statistical sensitivity for detecting mentalizing activity relative to a blocked
design, a more focused analysis on key regions of interest failed to reveal even a single significant
voxel. Nevertheless, decreased statistical power remains a viable explanation for the present null results,
as well as reduced psychological sensitivity and interest to unusualness due to a random stimulus
presentation, aggravated by the bare social context discussed earlier.
In summary, unusual action intentions do not always trigger the mentalizing network. We
interpret this null result as suggesting that when there are not enough contextual clues explaining the
unusualness of an action, people only notice that there is something unusual about the action, but do not
spontaneously mentalize about why the actor behaves strangely. In order to trigger the mentalizing
network, either a richer action context or an explicit instruction to attend to the intention might be a
prerequisite. Further research is necessary to explore the respective contributions of the mirror and
mentalizing networks in understanding unusual behavior.
Unusual Actions 10
References
Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate
cortex: An update. Trends in Cognitive Sciences, 8(12), 539-546.
Brass, M., Schmitt, R. M., Spengler, S., & Gergely, G. (2007). Investigating action understanding:
Inferential processes versus action simulation. Current Biology, 17(24), 2117-2121.
de Lange, F. P., Spronk, M., Willems, R. M., Toni, I., & Bekkering, H. (2008). Complementary systems
for understanding action intentions. Current Biology, 18(6), 454-457.
Jastorff, J., Clavagnier, S., Gergely, G., & Orban, G. A. (2011). Neural mechanisms of understanding
rational actions: Middle temporal gyrus activation by contextual violation. Cerebral Cortex, 21,
318-329.
Lewis, J. W. (2006). Cortical networks related to human use of tools. The Neuroscientist, 12(3), 211231.
Liepelt, R., von Cramon, D. Y., & Brass, M. (2008). How do we infer others’ goals from nonstereotypic actions? The outcome of context-sensitive inferential processing in right inferior
parietal and posterior temporal cortex. NeuroImage, 43(4), 784-792.
Ma, N., Vandekerckhove, M., Van Overwalle, F., Seurinck, R., & Fias, W. (2011). Spontaneous and
intentional trait inferences recruit a common mentalizing network to a different degree:
Spontaneous inferences activate only its core areas. Social Neuroscience, 6(2), 123-138.
Manthey, S., Schubotz, R. I., & von Cramon, D. Y. (2003). Premotor cortex in observing erroneous
action: An fMRI study. Cognitive Brain Research, 15(3), 296-307.
Peelen, M. V., & Downing, P. E. (2007). The neural basis of visual body perception. Nature, 8, 636-648.
Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27,
169-192.
Slotnick, S. D., Moo, L. R., Segal, J. B., & Hart, J. (2003). Distinct prefrontal cortex activity associated
with item memory and source memory for visual shapes. Cognitive Brain Research, 17(1), 7582.
Spunt, R. P., Falk, E. B., & Lieberman, M. D. (2010). Dissociable neural systems support retrieval of
how and why action knowledge. Psychological Science, 21(11), 1593-1598.
Spunt, R. P., Satpute, A. B., & Lieberman, M. D. (2011). Identifying the what, why, and how of an
observed action: An fMRI study of mentalizing and mechanizing during action observation.
Journal of Cognitive Neuroscience, 23(1), 63-74.
Van Overwalle, F. (2009). Social cognition and the brain: A meta-analysis. Human Brain Mapping, 30,
829-858.
Van Overwalle, F., & Baetens, K. (2009). Understanding others’ actions and goals by mirror and
mentalizing systems: A meta-analysis. NeuroImage, 48(3), 564-584.
Unusual Actions 11
Table 1: Contrasts of Unusual Intention and Unusual Means against Normal action
MNI coordinates
Anatomical Label
Unusual Intention > Normal
dorsal ACC
Unusual Means > Normal
L Postcentral Gyrus
R Inferior Parietal Lobule (incl. aIPS)
L Superior Parietal Lobule (incl. aIPS)
R Middle Occipital Gyrus
R Inferior Temporal Gyrus (incl. EBA)
R Inferior Temporal Gyrus
L Middle Occipital Gyrus
L Middle Temporal Gyrus (incl. EBA)
Unusual Intention & Means > Normal
x
y
z
Voxels
Max t
-2
0
8
0
30
36
57
3.42
3.11
-44 -40
-34 -38
-46 -30
66
72
48
463
5.18a
4.21
3.84
34
34
34
-28
-32
-44
-42
-40
-62
-60
52
72
64
66
58
488
4.05a
3.92
3.77
3.78
3.29
34
52
44
-92
-72
-56
16
-8
-10
1561
5.88a
5.26
4.91
-46 -68
-46 -60
-2
6
394
3.37a
4.20
53
No suprathreshold clusters
Note. Coordinates in MNI (Montreal Neurological Institute) stereotactic space of local
maxima within each cluster. The reported clusters survive a whole brain uncorrected
threshold of p < .005 and are significant after correction for multiple comparisons
according to the Slotnick test statistic (cluster size > 49). Regions denoted by a are also
significant after FDR correction for multiple comparisons at cluster level (p < .05). Only
subpeaks with a different anatomy are relabeled. ACC = Anterior Cingulate Cortex; EBA =
Extrastriate Body Area; aIPS = anterior IntraParietal Sulcus, L = Left, R = Right, incl. =
including.
Unusual Actions 12
Figure 1. A. Unusual Intention and Unusual Means. Whole brain view with uncorrected threshold of p
Unusual Actions 13
< .005 and cluster volume > 49 voxels. Circles denote regions of interest (ROI). B. Percent signal
change for core mentalizing ROIs. dACC = dorsal Anterior Cingulate Cortex; mPFC = medial
PreFrontal Cortex (ROI); SPL = Superior Parietal Lobule; aIPS = anterior IntraParietal Sulcus; TPJ =
Temporo-Parietal Junction (ROI); MOG = Middle Occipital Gyrus; EBA = Extrastriate Body Area.