Stop, Crying, Stop Crying : When 1 + 1 = 1

Master de Sciences Cognitives
Spécialité : Neurosciences Cognitives
Memoire
Stop, Crying, Stop Crying :
When 1 + 1 = 1
Using fMRI to investigate how the Brain merges single words
into sentences
Y-Lan Boureau
Directeur de stage : Christophe Pallier
Directeur d’unité : Stanislas Dehaene
Unité Inserm 562 - CEA - SHFJ
4 pl. du Général Leclerc
91401 Orsay
· 2006 ·
Abstract
A characteristic of natural languages is that meanings of single words in a sentence do not simply
add up into the meaning of the sentence. Instead, they combine in a non-trivial way dictated by the
rules of syntax.
Intuitions of investigators looking for an “area of syntax” in the brain have been guided by the pattern
of syntactic impairments in aphasic patients with a lesion in Broca’s area.
Indeed, imaging studies manipulating complexity in sentences by making use of a large array of syntactic transformations, and comparing these complex sentences to simple ones, have found increased
activation in Broca’s area, but also in left posterior temporal areas.
However, they might have missed core components of syntax (also present in simple sentences) by
subtracting them away.
Studies comparing activation to sentences and to non-sentence stimuli may be better suited to capture
core components of syntax. They generally pointed to an activation of areas in the left posterior and
anterior temporal lobe, but not the frontal lobe.
We conducted two fMRI experiments to investigate the transition from individual words to sentences.
An earlier fMRI study had found adaptation in the left temporal lobe when listening to sentences with
repeated semantic content. The first experiment sought to refine these results by disentangling adaptation to syntax-dependent meaning (meaning of a word after it has been attributed a thematic role)
and to individual, syntax-independent word meaning. Confusing early results call for modifications of
the initial design.
The second experiment tracked the phrase-building process by incrementally adding syntactic structure to strings of twelve words (from 12 isolated words to one 12-word phrase in five steps).
In pilot subjects, areas displaying a progressive increase in activation were found in the left angular/posterior middle temporal gyrus and the left anterior middle temporal gyrus.
Some evidence suggests that among these, the posterior areas subserve attribution of thematic roles to
verb arguments, whereas the anterior areas are in charge of merging small phrases into bigger phrases.
Potential ways to modify the experiment to bring about a dissociation between these processes are
discussed.
Acknowledgments
I am extremely grateful to Anne-Dominique Devauchelle, Christophe Pallier, Stanislas Dehaene and
Luigi Rizzi for being such pleasant people to work with.
I would also like to thank Antoinette Jobert for helping me to run the fMRI experiment.
Contents
1 Introduction
1
2 Some background in linguistics and imaging of syntax
2.1 A light primer on linguistics . . . . . . . . . . . . . . . .
2.1.1 Phonology, semantics, syntax, pragmatics . . . .
2.1.2 More about syntax . . . . . . . . . . . . . . . . .
2.1.3 Merging and transforming . . . . . . . . . . . . .
2.2 Imaging literature on syntax . . . . . . . . . . . . . . . .
2.2.1 Scope . . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 The bias towards Broca’s area . . . . . . . . . .
2.2.3 Sentences vs. sentences . . . . . . . . . . . . . .
2.2.4 Sentences vs. non-sentences . . . . . . . . . . . .
2.2.5 Priming of syntactic structure . . . . . . . . . . .
2.3 So where do we stand ? . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
. 3
. 3
. 4
. 4
. 7
. 7
. 7
. 9
. 13
. 16
. 17
3 Disentangling word form, single word meaning, and thematic roles
3.1 The first study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.1 Methods and results . . . . . . . . . . . . . . . . . . . . . . . . .
3.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 The second study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.2 Materials and methods . . . . . . . . . . . . . . . . . . . . . . . .
3.2.3 Behavioral results . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.4 Confusing imaging results . . . . . . . . . . . . . . . . . . . . . .
3.2.5 Possible other explanations . . . . . . . . . . . . . . . . . . . . .
3.3 Further directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
19
19
19
21
21
22
22
23
24
24
26
4 Core syntax at the phrase level
4.1 Reconciling Devauchelle et al. and Noppeney &
4.2 Tracking constituents . . . . . . . . . . . . . . .
4.3 Material and methods . . . . . . . . . . . . . .
4.3.1 Subjects . . . . . . . . . . . . . . . . . .
4.3.2 Design . . . . . . . . . . . . . . . . . . .
4.3.3 Stimuli . . . . . . . . . . . . . . . . . .
4.3.4 Task . . . . . . . . . . . . . . . . . . . .
4.3.5 Procedure . . . . . . . . . . . . . . . . .
4.3.6 fMRI Scanning Technique . . . . . . . .
4.3.7 Data Analysis . . . . . . . . . . . . . . .
4.4 Results . . . . . . . . . . . . . . . . . . . . . . .
4.4.1 Behavioral results . . . . . . . . . . . .
4.4.2 Network activated by all conditions . . .
4.4.3 Linear contrast . . . . . . . . . . . . . .
4.5 Discussion . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
27
27
28
29
29
29
29
29
31
31
31
32
32
32
32
34
iii
Price
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4.5.1
4.5.2
Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Patterns of activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5 General discussion and further directions
5.1 Adaptation to semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Looking for core syntax and constituent-building areas . . . . . . . . . . . . . . . . . . .
5.3 Further directions concerning the core-syntaxe study . . . . . . . . . . . . . . . . . . . .
5.3.1 Controlling for mere sequence effects . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.2 Decreasing variability between subjects by introducing tighter control on stimuli
5.3.3 Differentiating between agency and phrase building . . . . . . . . . . . . . . . . .
5.3.4 Digging constituency further: priming by constituents vs. adjacent strings of
words . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
41
41
41
42
42
42
42
Bibliography
48
Abbreviations table
49
iv
43
Chapter 1
Introduction
When you listen to someone talking in your mother tongue, words flow like a breeze and blend seamlessly into a meaningful message. Actually language is so deeply rooted into your brain that even
when you are trying to focus on something else, and wish you would not be distracted by what people
around you are saying, you simply cannot help parsing sentences and turning noise into meaning.
Yet the hard time people have trying to learn foreign languages should suffice to remind us that
using and understanding language is far from being an inherently easy task.
The contrast between the computational difficulty of parsing language and the experienced effortless use of natural language points towards the existence of specialized modules in the human brain.
Indeed, human brain mapping’s debut more than one century ago was all about language, and the loss
of the capacity to produce it following damage to Broca’s area (Broca, 1861).
Tremendous progress has been made in brain imaging since then, yet language still defies understanding. Evidence often proves either inconclusive or even contradictory. Studies have progressively
drifted from trying to find a hypothetical area for language, to trying to break down language into its
more simple components, and mapping those on the brain. Thus, the study of language is now often
restricted to the study of one particular aspect of it, to various degrees of specificity, ranking from the
study of semantics or syntax, to that of highly specialized subcomponents of language, as for instance
the study of passivization(Grodzinsky & Friederici, 2006).
However, trying to disentangle autonomous processes in language is quite a tough task. Parts of
language like phonology, semantics, and syntax are supposed to be working together and support one
another towards a common goal; there is evidence (Tyler & Marslen-Wilson, 1977; Altmann & Steedman, 1988) that language processing is not made up of autonomous networks that operate serially on
the input they receive, but of highly interconnected processors that interact all the time, with information flowing relentlessly back and forth from one processor to the other so that they all converge as
efficiently as possible on the final interpretation of language input. Communication through language
has to be redundant in order to be robust, to allow for understanding in a various number of nonoptimal conditions, for instance in a noisy environment(Miller et al., 1951). So language subprocessors
should not be expected to operate independently from one another. They all concur to the same goal.
Thus, all endeavours to tease apart pieces of language always exert a kind of violation on the natural
processing of language. Given the amount of cross-talk between levels of language processing, supposed
to be highly parallel in nature (Scott & Johnsrude, 2003), it appears quite unlikely that it might be
possible to shut down a specific subcomponent of language processing without influencing the others.
Yet such an approach cannot be dispensed with as there is no all-purpose language area in the brain,
but rather, a network of areas that subserve processes used in language (Grodzinsky & Friederici, 2006)
Let us now focus more specifically on syntax. Trying to pinpoint where it is implemented in the
1
Chapter 1: Introduction
2
brain implies both discriminating syntactic processes from non-syntactic ones - that is, distinguishing
syntax from semantics, pragmatics, and phonology - , and, within syntax, teasing apart different types
of operations.
Studies of syntactic processes in the brain have tackled the problem of mapping the syntactic brain
from various angles that have yielded somewhat conflicting results. However, bearing in mind that
syntax is no unitary process should help make sense of these results.
The first part of this report provides a short review of the literature and examines if a unified
framework can be sketched. Background in linguistics is provided as well.
I shall then move on to present the work that I have done, which consists in two fMRI experiments.
The first part, conducted conjointly with Anne-Dominique Devauchelle, comes as a logical following
to an initial fMRI study that was carried out by Devauchelle et al. (2006). This study compared
adaptation to syntactic and semantic features by monitoring decreases in activation following repetition
of sentences that did or did not share semantic ans syntactic features.
No adaptation to syntactic features was found, but robust adaptation to semantic features was seen
in the left middle temporal gyrus (L-MTG). However, it was unclear if the habituation was to word
form, individual word meanings (syntax-independent meaning), or attribution of the same thematic
roles (syntax-dependent meaning, a.k.a. “who did what to whom”), as the same thematic situation
was presented in various syntactic guises.
The study I shall present was designed to separate habituation to each of these three factors by
varying them as independently as possible. But the results we obtained have led us to think that the
experimental design failed to meet the requirements for efficient parameters estimation. The design is
currently being revised.
The second experiment looked directly at activations, to capture the phrase-building process that
underlies what we shall call “core syntax”, allowing to build larger phrases from small phrases.
We used an incremental design that we shall explain in far more detail below, the underlying intuition being that if some neural tissue engages in merging words into phrases, then its activation should
be higher when building one big phrase than when building separate smaller trees.
Finally, I propose directions for further research.
Chapter 2
Some background in linguistics and
imaging of syntax
In this chapter, I review the broad directions imaging research has taken to tackle syntax. Studies are
often interested with only one aspect of language; but when trying to break down language into its
parts, the first questions that arise are, what those parts are, and how specific they should be.
Before starting our review, we shall start off by giving some background in linguistics and see what
distinct components can be isolated in language.
2.1
2.1.1
A light primer on linguistics
Phonology, semantics, syntax, pragmatics
Understanding language implies converting auditory input into a meaningful message. However, this
is not a one-step conversion and a number of levels can be distinguished. The division of labour we
provide here is quite schematic for clarity’s sake, but the reader should keep in mind that all those
levels interact and cooperate into a common goal.
The first step (with which phonology is concerned) is to turn the auditory flow, an input that is
analogical in nature (variation in air pressure), into a sequence of phonemes, that is, into a digital
output (phonemes are a finite set of features that is specific to every single language); for instance, a
continuum between sounds “ba” and “pa” will not be perceived as such by a native French speaker,
but rather as a discontinuous leap from “ba” to “pa”, in a winner-take-all fashion(Liberman et al.,
1967).
The next step is then to turn that sequence of phonemes into words; that is, the sequence of
phonemes is compared against a dictionary until a match is found, and the corresponding meaning is
retrieved. Finally, all words combine into a sentence.
Semantics is the study of meaning, at every level, from word (or even parts of words) to sentences;
however, meanings of words do not simply add up when words are stringed together to form sentences:
the way their meanings combine is dictated by syntax. Thus, syntax can be viewed as a catalogue of
sophisticated ways to combine meanings, or even meaningless utterances. We shall enter into more
details about syntax in the next paragraph.
The last step is, integrating the message into a context and a network of expectations, guesses,
and implicit rules of communication (e.g., the expectation that someone who speaks to you intends
to convey an informational message and not some irrelevant sentence that merely “has a meaning”).
The study of relevance and interpretation of language in context is the goal of pragmatics (Sperber &
3
Chapter 2: Some background in linguistics and imaging of syntax
4
Wilson, 1986).
2.1.2
More about syntax
We are going to dwell somewhat longer on syntax, as it is sometimes unclear what syntax is, and what
syntax does, and it is crucial for our work to know what we are after when we are looking for the neural
correlates of syntax. Syntax is what makes language so powerful, and what makes it such a fine tool
when compared to syntactically poor ways to communicate like pidgins. Perhaps the contribution of
syntax to language (as compared to isolated words, or to mere juxtaposition of words, which anyway
is already a kind of syntax) can be seen as a shift from a 1D world to a 3D one, with the intermediate
2D world corresponding to basic, elementary juxtapositional syntax.
What steps could be distinguished in an increasingly complex interpretation of a string of words ?
As an example, take the sentence “it is the boy that the girl hits”. In a syntax-deprived world, you
would have isolated representations for the words “boy”, “girl”, “hit”, that would not even blend into
a common meaning. The lowest possible level of integration would be, representing the words in a
completely disconnected way, so that hearing three words one right after the other, or ten days apart,
would make no difference.
Then word meanings can combine with one another linearly. There is a girl, a boy, and there is
hitting, but importantly, the boy and the girl are the ones who are involved in the hitting.
A big progress is then to actually assign roles to participants; this is syntax already, and syntax tells
you whether it is the girl hitting the boy, or the other way round. We shall refer to that level of
syntactic processing as “core syntax”, as a short-hand.
But there is more to syntax than simple role assignment, which is already a considerable achievement per se. Syntax provides people with an exquisite way of shifting topics while still retaining the
same distribution of roles between participants; you know you should focus on the boy when hearing
“it is the boy that the girl hits”, which would not be the case if you were simply told “the girl hits
the boy”. That more sophisticated aspect of syntax allows to convey information in a far more refined
way; but it depends heavily on correct functioning of basic syntax.
Let us now turn to linguistic theories of syntax to see if they make that kind of distinction between
core syntax and the postulated more refined syntax.
2.1.3
Merging and transforming
Theories of syntax refer heavily to Chomsky’s work. Chomskyan theories come broadly in three guises.
The coverage provided here will be somewhat simplistic (the interested reader is referred to Chomsky
(1957, 1986); Haegeman (1991); Chomsky (1995); Lasnik (2002) for more detailed presentations).
The three distinct stages of Chomskyan theories of syntax are usually known as the Standard Theory (Chomsky, 1957), the Government and Binding theory (Chomsky, 1986; Haegeman, 1991) , and
the Minimalist Program (Chomsky, 1995; Lasnik, 2002).
I shall not enter into much detail; the crucial point I want to make is that all those three theories
have retained throughout a division between two distinct layers of syntax: that in charge of building
phrases (D-structure, X-bar theory, or MERGE), and that in charge of moving and binding those
phrases in a kind of second-order treatment of sentences (the sophisticated, but somewhat less indispensable part).
5
2.1 A light primer on linguistics
Core syntax
The first stage of syntactic processing involves building phrases out of isolated words. An example is
given below. The phrase “a black cat” is called a determiner phrase because it behaves as a whole set
of such phrases that generally contain a determiner and a noun (but not necessarily: “John” is a DP
as well).
DP
H
H
D
H
NP
HH
NP
AP
a
A’
N’
A
N
black
cat
Here, the important point to make is that the words “a”, “black” and “cat” are no longer isolated
entities, but are tied together in a tree and can be manipulated as a single entity (“a black cat” behaves
like “John”).
A node in a tree is called a constituent.
The meaning of the phrase depends crucially on the tree that has been built. For example:
1. (a)
S
H
H
DP
PP
The girl
HH
said
H
VP
HH
H
HH
CP
PPP
P
PP
PP
that she was coming on Thursday
Chapter 2: Some background in linguistics and imaging of syntax
2. (b)
S
HHH
6
H
H
H
DP
PP
The girl
H
VP
H
H
H
H
H
H
VP
H
H
said
H
H
CP
PPP
P
PP
PP
P
on Thursday
P
that she was coming
This example shows that semantic and syntactic interpretation are tightly interleaved at this level.
It roughly corresponds to building Chomsky’s D-structure until the Government and Binding theory,
but the term has been abandoned in the Minimalist framework. In the Minimalist framework, this
level seems to map on MERGE; however, we shall keep the term “core syntax” because of unsufficient
knowledge of the refinements of Chomsky’s MERGE.
Thematic roles assignment A predicate (where the predicate can be a verb, an adjective, a noun,
or a preposition) comes with arguments, that can be described as the participants that have some part
to play in the event being described by the predicate.
The roles that the arguments are assigned by the predicate are usually referred to as thematic roles,
or θ-roles. Common examples of θ-roles are: the theme, which is roughly the topic of the discussion
(e.g. Mary in the sentence “I called Mary yesterday”), the agent, which is causing the event to happen
or has some conscious control on the event (e.g. Mary in “Mary called her mother”), the experiencer,
which is undergoing the event (e.g. Mary in “Mary likes flowers”).
Assignment of θ-roles is crucial for the understanding of action, particularly when it comes to
knowing who did what to whom. The mapping of θ-roles on the arguments of a predicate is at the
core of the interface between syntax and semantics.
Second level syntax
We shall call second level syntax the part of the syntax that moves around the constituents built by
core syntax (or smaller pieces as well). In the standard theory, transformations would be part of that
second level syntax. For us, second level syntax will be all the more refined parts of syntax that do
something different than merely merging two words. In the minimalist framework, that would probably
correspond to move.
What syntax are we after ?
Looking for an area involved in syntactic processing in the brain appears to be too vague: the two
levels we have presented should at least be distinguished. That point should remain present in the
reader’s mind now that we are about to tackle our review of research on imaging of language, as it
can explain why lots of different studies uniformly labelled “looking for syntax” yielded various results.
7
2.2
2.2.1
2.2 Imaging literature on syntax
Imaging literature on syntax
Scope
Phonology, semantics and syntax have been studied in a large number of imaging studies; surprisingly
few attempts have been made to study pragmatics.
Semantics and syntax are intimately connected as we shall see in more depth ; sentences are often
syntactically ambiguous and semantics lead people to the right choice (remember the classical example
“time flies like an arrow”, that will never be spontaneously interpreted as a twin sentence to “fruit
flies like a banana”). Moreover, syntax can be viewed as nothing more than an auxiliary to semantics,
with its existence as a separate entity (in Jabberwocky speech for example 1 ) being a mere side-effect.
The boundaries between semantics and syntax are somewhat tricky to draw; for instance, does the
attribution of thematic roles regard syntax, or semantics ? Our review will quite often drift from
syntax towards semantics and back again.
The connection with pragmatics is not an easy one either. Let us take the example of attribution
of θ-roles once again. How much is that influenced by our pragmatic knowledge of the world ? When
reading a sentence like “the cat chases the mouse”, do we really need syntax to understand who did
what to whom ?
Taking a closer look at the interaction between syntax and pragmatics might be an interesting topic
for research.
2.2.2
The bias towards Broca’s area
Before looking at papers, we shall remind the reader with a few facts that should be kept present when
interpreting the results of research.
Broca’s famous lectures (Broca, 1861) about his patient who was incapable of producing grammatically structured utterances was the first unambiguous demonstration that a localized chunk of brain
tissue could subserve a particular function; at the time, Broca’s area was supposed to be involved in
speech production.
Later on, Wernicke’s patients completed the picture with what was perceived as a comprehension
deficit. The binary opposition was then one between speech production and speech comprehension,
with Broca’s aphasics being unable to produce speech while Wernicke’s aphasics could not understand
it. Following quite logically, conduction aphasia, a condition with impaired repetition, was associated
with damage to the pathway connecting Broca’s to Wernicke’s areas - namely, the arcuate fasciculus
(Geschwind, 1965, 1979).
It was not until the 1970s (Caramazza & Zurif, 1976) that that position was revised; according to
the new - now classic - assignment, Broca’s area was the one involved in syntax, whereas Wernicke’s
area received the charge of semantics. Indeed, Broca’s aphasics were described as producing mostly
speech without function words, whereas Wernicke’s aphasics were freely substituting words with irrelevant ones, while keeping the syntactic structure of sentences intelligible.
At first, it could look like some kind of progress. The opposition had moved to more abstract
grounds; but it was still a binary one, that tried to squeeze language into a clear-cut, neat framework. However challenged this view has been (e.g. Grodzinsky (2000)), it still survives happily in
the textbooks and the minds. Yet a careful study of lesions shows that there is neither a necessary
nor a sufficient causal link between a lesion in Broca’s area, and Broca’s aphasia, or purely syntactic
impairments. The same holds for Wernicke’s area and fluent aphasia.
1 Jabberwocky speech is grammatical speech which uses nonwords instead of existing content words. It is named after
Lewis Caroll’s poem in Alice in Wonderland
Chapter 2: Some background in linguistics and imaging of syntax
8
Figure 2.1: Broca and the brain of his patient Mr Tan
Figure 2.2: Broca’s and Wernicke’s areas and the arcuate fasciculus: the incomplete network for
language
9
2.2 Imaging literature on syntax
We shall review a number of papers that point towards syntax recruiting areas in the temporal
lobe, perhaps even more crucially than in the frontal lobe. Other papers argue that semantic tasks
are most often mapped to the anterior frontal cortex (e.g. Brodmann area 47, considered by some
authors to be part of Broca’s area. See Bookheimer (2002)). However, the classical mapping of syntax
on Broca’s area and semantics on Wernicke’s area has strongly biassed researchers into looking for a
confirmation of that mapping, rather than looking for a more adequate one. Thus, some studies (e.g.
Kang et al. (1999); Embick et al. (2000)) did not even include anterior temporal areas in the part of
the brain they imaged, while they had been hypothesized to be critically involved in syntax by earlier
papers(Mazoyer et al., 1993) .
It is important to bear those facts in mind when reading about papers looking for syntax. We shall
now move on to the review proper, that will fall into three parts: we first review papers that have
sought to disentangle syntax from semantics by comparing sentences to other sentences. We then turn
to studies that compared activation to sentences to activation to non-sentences stimuli. Finally, we
review what has been done using habituation paradigms.
2.2.3
Sentences vs. sentences
Broca’s area and complexity
Several studies (Just et al., 1996; Stromswold et al., 1996; Ben-Shachar et al., 2003, 2004)have been
successful in finding activation in Broca’s area in relation to syntactic complexity (see fig.2.3). The
idea is generally to oppose sentences with increased complexity, to simpler sentences, while balancing
for the number of words and other factors; thus, syntax can be isolated from semantics as there is
arguably no difference in semantic contents of syntactically complex and syntactically simple sentences.
Studies have used various ways to increase complexity, involving the presence of object-relative vs.
subject relatives (Ben-Shachar et al., 2004), center-embedded vs. right-branching relatives(Stromswold
et al., 1996), non-canonical order vs. canonical order(Ben-Shachar et al., 2003). Increased complexity
is generally assessed by behavioral reading-time measures or ratings, or rely on previous such measures.
Manipulations of syntactic complexity have consistently led to activation in Broca’s area (see Kaan
& Swaab (2002) for a review, cf. fig.2.3).
However, it should be noted that hearing simple sentences did not seem to activate primarily Broca’s
area (and activated temporal areas instead), and a great deal of complexity had to be introduced to
light up the area (see Kaan & Swaab (2002) for a review, cf. fig.2.5 ). This is a first clue pointing to
an identification of that type of syntax as the second level, not core, one.
Ben-Shachar et al. (2003, 2004) have provided a nice account of activation in Broca’s area, relating
it not to a general and somewhat vague increase in overall complexity (Just et al., 1996) or working memory load (Kaan & Swaab, 2002), but explicitly to transformations, in a way that relates to
Grodzinsky’s hypothesis about Broca’s aphasia being linked to trace deletion, based on the observation
that Broca’s aphasics’ deficit roughly boiled down to an incapacity to link antecedent to their traces
(Grodzinsky, 2000). Ben-Shachar et al. (2003, 2004) propose that Broca’s area is in charge of moving
antecedents to their traces, and indeed they observe increased activation in Broca’s area in sentences
that contain movement (e.g. cleft sentences and object relatives) relatively to sentences where movement is much smaller. Care is taken that a number of potential generators of complexity (number of
content words, number of embeddings, number of verb arguments) are balanced across conditions.
Ben-Shachar et al. (2003) also found higher activation in the posterior superior temporal sulcus bilaterally with transformational sentences, and in the left posterior superior temporal sulcus in relation
to the number of arguments required by a verb. We shall return to this later.
Chapter 2: Some background in linguistics and imaging of syntax
10
Figure 2.3: Map of activations found by studies that compared complex sentences and simple sentences.
From Kaan & Swaab (2002)
11
2.2 Imaging literature on syntax
Figure 2.4: Variation of activation in Broca’s area according to SOV or OSV order depends on the
nature of the verb and on the θ-role assignment to its arguments. An account based solely on syntactic
transformations as in Ben-Shachar et al. (2004) would predict higher activation in OSV noncanonical
order than SOV no matter what. From Bornkessel et al. (2005)
Broca’s area and linearization
However, a paper by Bornkessel et al. (2005) represents a recent challenge to this proposal. In Benschachar et al.’s view, the role of Broca’s area should be specified purely in terms of grammatical
functions; that is, a moved subject has to get back to where the subject is expected to be, and the
same goes for the object.
If that conception is true, then the activation in Broca’s area should be greater in sentences where
arguments have been displaced in a non-canonical order, than in sentences with standard order - SOV
(Subject Object Verb) if the language in question is one with standard SOV order, no matter what
their meaning is.
Bornkessel have studied sentences in German with various word orders. German case-marking
being explicit (so that an accusative masculine name, e.g. den Mond, can be distinguished from a
nominative (der Mond) or a dative (dem Mond) one) makes word order relatively free; however, there
is still a preference for SOV (the canonical order).
Indeed, activation in Broca’s area is higher when the object is presented first, and the verb is an
active one like schlagen (to hit), as was predicted by Benschachr’s interpretation. Importantly however,
the pattern is reversed when the verb is what is called an object-experiencer verb, e.g. gefallen (to
please), with Broca’s area being significantly more activated with SOV order than OSV (see fig.2.4).
Thus, Broca’s area seems to be recruited when moving arguments to conform to a standard θ-roles
hierarchy that would rule that experiencer should appear before theme, rather than a purely syntactic
SOV hierarchy. The fact that activations for object-experiencer verbs lie between the levels of activations associated with processing active verbs might reflect that both hierarchies come into play.
Importantly, this study also found increased activity of left pSTS when an early attribution of
θ-roles based on case-marking had to be revised when the verb appeared.
Another study (Grewe et al., 2005) shows that Broca’s area fails to be activated more when sentences in German are presented with OSV order than standard SOV order, but the object is not a
determiner phrase, but a pronoun instead. The authors argue that this is because there is a rule stating that a pronoun should generally precede non-pronominal arguments; thus, Broca’s area seems to
conform to a number of hierarchical linearization rules and not only the purely syntactic ones proposed
by Ben-Shachar et al. (2003).
Finally, Friederici et al. (2006) have found a parametrical increase of activation in Broca’s area as
a function of the number of moved verb arguments in German, which lends support to both a view
Chapter 2: Some background in linguistics and imaging of syntax
12
of Broca’s area subserving syntactic transformations, and an account in terms of reordering elements
according to various hierarchies.
Syntax as opposed to semantics
Kang et al. (1999) have tried to isolate syntax from semantics by comparing minimal verb phrases
(only two words) that contained either a syntactic (e.g. grew heard) or a semantic (e.g. ate suitcases)
violation to normal phrases (e.g. ate apples). They found higher activation in Broca’s area (Left BA
44) related to the syntactic condition, while BA 45,10 and 46 showed laterality differences, with the
semantic condition being associated with right-lateralization, and the syntactic condition with leftlateralization.
Unfortunately, they did not include the anterior temporal lobe in their imaging sessions, so that
the involvement of areas in the temporal lobe below the AC-PC line cannot be discussed.
Dapretto & Bookheimer (1999) have used another approach to disentangle semantics and syntax.
Their paradigms relied on the task to focus subjects’ attention either on semantics or syntax: subjects
had to make a judgement as to whether two sentences had the same meaning or were different; in the
semantic condition, sentences differed by a content word (e.g. “The lawyer questioned the witness” vs.
“The attorney questioned the witness”), and in the syntactic condition, they differed by their word
order (e.g. “West of the bridge is the airport” vs. “The bridge is west of the airport”). Behavioral
measured ensured that both tasks were equally difficult.
They found comparatively higher activation in a part of Broca’s area (Brodmann area BA 44, part
opercularis) in the syntactic condition, and in the lower portion of the left inferior frontal gyrus (LIFG, BA 47, part orbitalis), taken by some authors to be part of Broca’s area too) in the semantic
condition.
It should be noted that the activation of Broca’s area in the syntactic condition is consistent with
previously presented hypotheses that Broca’s area is recruited when there is some kind of movement
from a non-canonical order to the canonical one, the latter being specified either syntactically or thematically.
The activation of the L-IFG in the semantic task is consistent with a number of studies that have
implicated that area in tasks involving lexical processing (see Bookheimer (2002) for a review on semantic processing).
The two studies by Dapretto & Bookheimer (1999) and Kang et al. (1999) have been criticized by
Embick et al. (2000) on the ground that they used tasks that “lacked an experimental history”. Embick et al. (2000) get back to a classical violation detection task, where subjects had to report whether
sentences contained one or two violations, that were either grammatical (e.g. inverting a noun and its
determiner) or spelling errors.
The authors report a significantly greater activation in Broca’s area than in Wernicke’s area or in the
angular gyrus/supramarginal gyrus in the syntactic condition. It should be noted that the grammatical
errors introduced were errors in word order, which makes these results consistent with the linearization
hypothesis stated earlier.
As is the case with the study by Kang et al. (1999), the authors were primarily concerned with attributing roles to Broca’s and Wernicke’s areas, and used an ROI approach that only imaged slices
above the AC-PC plane. Therefore, no data on the lower temporal lobe are available from their study.
Problems with this approach
A problem with those kinds of approaches is that they are bound to look for a subpart of syntax
that has as few links to semantics as possible. Remembering the distinction between core syntax
and second-level syntax, we can ask what part of syntax they are tracking. Given that core-syntax is
tightly linked to semantics as it is about combining meanings and attributing thematic roles, it is likely
13
2.2 Imaging literature on syntax
Figure 2.5: Map of activations found by studies that compared sentences and non-sentence stimuli.
From Kaan & Swaab (2002)
that studies are often biassed towards second-level syntax. Indeed, the contrasts used often oppose
complex sentences to less complex ones - that is, sentences to sentences, so that any common factor
gets substracted out. Core syntax might indeed be one of those missed common factors.
2.2.4
Sentences vs. non-sentences
When comparing sentences to non-sentence stimuli, activation is usually found in the temporal, not
frontal lobe, as we shall now review. See fig.2.5 from Kaan & Swaab (2002).
An early study: stories vs. words
The most natural approach to tackle phrase-level syntax is to compare activation to sentences, to
activation to word lists. Indeed, an early study by Mazoyer et al. (1993) has used PET to compare
activation to sentences and word lists, while orthogonally varying semantic content (the words inside
the sentences could be congruent words, incongruent words, or pseudowords).
Processing of meaningful stories (semantically congruent condition) activated the left middle temporal
gyrus, the left and right temporal poles, and a superior prefrontal area in the left frontal lobe, in
addition to regions activated by single-word comprehension.
Importantly, only the bilateral temporal poles were consistently activated more in the sentence
condition than in the word list condition, regardless of the semantic condition, pointing towards a role
of that area in core syntax.
Chapter 2: Some background in linguistics and imaging of syntax
14
Sentences vs. environmental sounds
Support to this hypothesis has come from a study by Humphries et al. (2001), who compared activation
to sentences describing a short scene to activation to environmental sounds that matched as closely as
possible the semantic content of the sentence (e.g. “the tires hissed and the car crashed” compared to
the corresponding sounds).
They found increased activation for sentences in anterior superior temporal lobe bilaterally, and in left
posterior superior temporal regions.
Breaking down the sentence: semantics, prosody
The evidence presented in the previous sections point towards an involvement of anterior temporal
regions bilaterally, and left posterior superior temporal regions, in the processing of sentences.
There is a marked difference between sentences and word lists at the level of syntax, but sentences
and word lists also differ at the levels of semantics and prosody. Several studies have tried to tease
apart those effects.
Friederici et al. (2000), using a design that crossed sentence conditions (sentences vs. word lists)
and word type conditions (content words vs. pseudowords), found an increase of activation in the
planum polare bilaterally and in the deep portion of the left frontal operculum in the sentence conditions.
Vandenberghe et al. (2002) tried to disentangle syntax and semantics by using semantically unrelated words (e.g. “The branches of the traffic harmed the shopkeeper’s further tea” as opposed to
“The baby is staying with his grandmother”), and crossing this semantic condition with a syntactic
one (sentences vs. scrambled order sequences). They found increased activation of the left anterior
temporal pole to sentences compared to scrambled versions of sentences (coordinates: -44, -6, -24), and
a similar but weaker response in the left anterior superior temporal sulcus (coordinates: -64,-10,-12)
and the left posterior middle temporal gyrus (coordinates: -50,-58,-12) Only a subset of the voxels
in these areas, situated in the most anterior part of the temporal pole, showed an interaction with
semantics (that is, the increase in activation due to sentence structure was higher when the words
were semantically realted). No increased activation was observed in the left inferior frontal cortex
during reading sentences compared to scrambled word sequence, or with the inverse comparison. A
higher activation was observed in the left anterior temporal pole during reading semantically random
sentences and their scrambled versions compared to normal sentences and their scrambled versions.
Humphries et al. (2006) took that approach one step further by adding pseudowords conditions
to the semantically congruent and incongruent conditions, while still crossing this factor with the
syntactic structure one, thus yielding six different conditions.
Effects of syntactic structure were observed in the left anterior STS and left angular gyrus, which
showed greater activation to sentences than to word lists. On the other hands, widespread, bilateral
temporal lobe areas and the angular gyrus displayed greater activation to semantically congruent stimuli than either incongruent or pseudoword stimuli, probably reflecting increased semantic integration.
The fact that the left angular gyrus responded both to semantic and syntactic structure might
add evidence to the view that superior posterior temporal regions are involved in processing semantic
informations in the context of sentences.
The left anterior temporal lobe was “relatively insensitive to manipulations of semantic structure”,
which suggests that this area might be more purely syntactical than the other areas involved in sentence
processing.
15
2.2 Imaging literature on syntax
Figure 2.6: From Humphries et al. (2006)
Chapter 2: Some background in linguistics and imaging of syntax
16
Figure 2.7: From Humphries et al. (2006)
2.2.5
Priming of syntactic structure
Some caveats
A problem to studies relying on violation detection is that they rely on the assumption that an area
naturally associated with syntax will work harder when a violation occurs; however, the prediction is
quite tricky as to what direction the effect will be, with top-down processes being so important.
If the subject tries to bypass the violation and makes his best at parsing the sentence, then indeed
higher activation should be expected. But if the subject disengages from the task as soon as a violation
is detected, then lower activation will be predicted. This is especially the case in experimental settings
where the subject might learn to expect violations and to ignore them.
Thus, areas that are detected using violation techniques might reflect error detection rather than increased normal processing.
The same holds for jabberwocky and semantically incongruous sentences (e.g. “the truck in the cloud
ate too many chimneys”): it might still be possible - but effortful - to parse them and to try and build
a representation of their meaning. Then a subject would have the choice between either engaging,
or giving up altogether. There would then be a step-like discontinuity in activation, with activation
climbing and then suddenly dropping off when the difficulty crosses some threshold.
Habituation paradigms, that seek to identify areas in the brain that show decreased activation
following repetition of the feature of interest, do not have that problem and might be better suited to
pinpoint normal processing of language.
Two studies have used habituation paradigms (where activation in an area involved in a task gets
lower when material sharing features related to the task is presented repeatedly) to track syntactic
processing.
Noppeney & Price (2004) used four different syntactic structures in a context of local parsing ambiguity, and found a decrease in activation in left anterior temporal cortex (coordinates: -42, 3, -27),
pointing toward an involvement of that area in syntactic structure processing.
17
2.3 So where do we stand ?
Devauchelle et al. (2006) have compared habituation to to syntactic and semantic features No
habituation to syntax was seen. However, habituation to semantics was observed in the left medium
temporal gyrus and sulcus; we shall enter into more detail on these two studies in the next chapter.
2.3
So where do we stand ?
Broca’s area does not seem to subserve core-syntax
Importantly, activation in the left frontal lobe has been reported with studies involving sentences
with increased syntactic complexity (Keller et al., 2001; Caplan et al., 1998, 1999, 2000; Just et al.,
1996; Stromswold et al., 1996), syntactic violations (Moro et al., 2001; Embick et al., 2000; Kuperberg
et al., 2000; Ni et al., 2000; Kang et al., 1999), and implicitly induced syntactic analysis (Dapretto &
Bookheimer, 1999), but not when sentences do not involve complex syntactic analysis (Schlosser et al.,
1998; Friederici et al., 2000; Kuperberg et al., 2000). See fig.2.3 and 2.5
Moreover, Broca’s aphasia has been reported to lead to specific impairments to assigning a referent
to a pronoun (Vasic et al., 2006) or an antecedent to a trace (Grodzinsky, 2000), and there is evidence
that activity in Broca’s area increases parametrically with the number of displaced verb arguments
(Friederici et al., 2006), and shows sensitivity to how canonical the order of θ-roles presentation is
(Bornkessel et al., 2005).The evidence reviewed points to an involvement of Broca’s area not in core
syntax, but in second-level syntax.
Importance of the temporal lobe
Left anterior temporal regions In contrast to what has been reported of Broca’s area, activation of
the left anterior temporal lobe has been found consistently when contrasting sentences to environmental
sounds or word-lists regardless of their semantic content and of the prosody used (Mazoyer et al., 1993;
Humphries et al., 2001, 2005, 2006; Vandenberghe et al., 2002; Stowe et al., 1999). See fig. 2.5.
Left posterior superior temporal regions We have also reviewed evidence that the activity in
left pSTG and pSTS increases with verb argument complexity (Ben-Shachar et al., 2003), and errors
in the early attribution of θ-roles in case-marked languages (Bornkessel et al., 2005). This evidence has
lead authors to postulate a role for left pSTS in the interface between syntax and semantics (Vandenberghe et al., 2002; Bornkessel et al., 2005; Friederici et al., 2006; Humphries et al., 2006): posterior
temporal areas would subserve attribution of θ-roles and reinterpretation of individual words in the
context of the sentence.
Additional support for this hypothesis come from the lesion studies that show that Wernicke’s
aphasics have an impaired performance on complex verbs (Shapiro et al., 1993); according to a study
by Pinango & Zurif (2001), Wernicke’s aphasics show a deficit in semantics at the level of sentence
beyond their deficit in lexical-semantic retrieval.
Indeed, that view fits well with evidence from event-related brain potential studies, that have identified a late centro-parietal positivity (labelled P600) around 600 msec after the onset of a critical word
that introduces a syntactic violation or a ’garden-path’ (Friederici et al., 1993; Osterhout et al., 1994).
Patients with lesions in the posterior portion of the left temporal lobe including the pSTG have been
reported to show a selective absence of the P600 (Friederici & Kotz, 2003).
As proposed in Bornkessel et al. (2005), a view of left posterior superior temporal regions attributing agency fits well with a conception of posterior STS playing an important role in the identification
of agency more generally (Frith & Frith, 1999), based on observations that this region engages in the
processing of biological and implied motion, moral judgements, and tasks involving theory of mind
Chapter 2: Some background in linguistics and imaging of syntax
18
(reviewed in Frith & Frith (2003)). Indeed, all these tasks attribute a thematic representation to an
object or person.
Chapter 3
Disentangling word form, single
word meaning, and thematic roles
Here, I shall first present briefly the study that was run by Devauchelle et al. (2006); I shall then
explain in more detail the second experiment that logically stemmed from the first one.
3.1
The first study
The aim of the study was to distinguish areas responsible for syntactic and semantic processing, using
a habituation paradigm.
Four different repetition conditions were devised to disentangle syntax and semantics; repeated
sentences shared either:
1. both semantic and syntactic features (repetition of the exact same sentence four times)
2. semantic features (the same words were repeated and had the same θ-roles, but syntax was
varied)
3. syntactic features (the sentences shared the same syntactic tree)
4. neither syntactic nor semantic features (the sentences had different content words and different
syntactic trees)
An example is given below, see fig.3.1.
3.1.1
Methods and results
Methods
Briefly, during a session, subjects were presented with twenty blocks of four sentences that fell into one
of the four above mentionned conditions (see fig.3.2). There were six different sessions per subjects.
Visual and auditory modalities alternated across sessions.
To ensure that subjects attended to the stimuli, they were instructed to press a button in a few
sentences.
Results
Habituation to syntax No adaptation to syntactic structure was observed in either perceptual
modality.
Habituation to semantics Adaptation was observed in left inferior frontal and middle temporal
regions, as can be seen on fig.3.3.
19
Chapter 3: Disentangling word form, single word meaning, and thematic roles
20
Figure 3.1: Example of stimuli used by Devauchelle et al. (2006) to probe for adaptation to syntax
and semantics
Figure 3.2: Design of Devauchelle et al. (2006)
21
3.2 The second study
Figure 3.3: Results of Devauchelle et al. (2006): no adaptation was found to syntactic structure, but
there was adaptation to semantic features
3.1.2
Discussion
Absence of habituation to syntax
The absence of habituation to syntax is at odd with Noppeney & Price (2004), who did find adaptation
in the left anterior temporal gyrus while also using a habituation paradigm. I shall discuss that
discrepancy in the next chapter as an introduction to my own study.
Adaptation to semantics
Let us try and examine how similar sentences were in the “semantically similar” condition.
The similarity of the sentences can be broken down into three parts:
1. similarity of the word form
2. similarity of individual content words meaning
3. similarity of θ-roles mapping
It is impossible to conclude from the first experiment to which of those three sets of repeated features
brain areas adapt, and if some of the activated brain areas adapt more selectively to one particular set
of features. Therefore, a second study was designed to disentangle these mixed influences.
3.2
The second study
In order to disentangle single word semantics, word form, and θ-roles mapping, we 1 devised an experiment that varied each of those three factors as independently as possible (full orthogonality could
not be achieved because the three factors are correlated; e.g., it is generally not possible to find two
1 This
experiment was run conjointly by Anne-Dominique Devauchelle and myself
Chapter 3: Disentangling word form, single word meaning, and thematic roles
22
Same single-word meaning
Same θ-roles
Different θ-roles
Le flic a assommé le bandit.
Le bandit a assommé le flic.
Le policier a matraqué le délinquant.
Le délinquant a matraqué le policier.
C’est le bandit que le flic a assommé.
C’est le flic que le bandit a assommé.
C’est le délinquant que le policier a matraqué.
C’est le policier que le délinquant a matraqué.
Le bandit a été assommé par le flic.
Le flic a été assommé par le bandit.
Le délinquant a été matraqué par le policier.
Le policier a été matraqué par le délinquant.
C’est par le flic que le bandit a été assommé.
C’est par le bandit que le flic a été assommé.
C’est par le policier que le délinquant a été matraqué. C’est par le délinquant que le policier a été matraqué.
Different single-word meaning
Les convives ont acclamé l’hôte.
C’est l’hôte que les convives ont acclamé.
L’hôte a été acclamé par les convives.
C’est par les convives que l’hôte a été acclamé.
Table 3.1: Full array of conditions
words that share the same word form but have different individual meanings).
More specifically, here is how variations were obtained in each of the dimensions (single-word
semantics, θ-roles, word form):
1. single-word semantics: words were changed for words of an unrelated semantic field (e.g. “the
priest buys a bag” vs. “the girl hits the boy”)- in this case, neither of the other two dimensions
was preserved.
Therefore, variations along the two remaining dimensions were embedded in the “same singleword semantics” conditions and could be crossed orthogonally in a 2x2 factorial design:
2. θ-roles assignement: subject and object of a transitive verb were swapped (e.g. “the lawyer hits
the girl” vs. “the girl hits the lawyer”)
3. word form: words were changed for synonyms (e.g. “the miss strikes the attorney” vs. “the girl
hits the lawyer”).
Moreover, these variations were further crossed with variations in the syntactic structure, with
sentences being either simple (subject DP, transitive verb, object DP), cleft, passivized, or passivized
and cleft, so that habituation to syntax could also be examined.
A full grid of conditions is provided table 3.1.
3.2.1
Design
3.2.2
Materials and methods
Subjects 6 healthy, right-handed, French native speakers participated to the study after having
given written informed consent.
Design 180 pairs of sentences (a primer sentence and a target sentence) were presented.
Repetition was reduced from four in Devauchelle et al. (2006) to only two sentences, because results from Devauchelle et al.’s experiments had shown that habituation was already present at the first
23
3.2 The second study
repetition.
The target sentence was always a noncleft active one. The primer sentence could be:
• noncleft active, noncleft passive, cleft active or cleft passive, with equal 25% probability
• semantically related (80% trials) or not related to the target sentence (no similarity at the level
of single-word meaning, 20% trials)
The 80% semantically related primer sentences were further divided into four groups by crossing
two factors (first factor: word form; second factor: θ-role assignment):
• half “same words” sentences (identical individual content words), half “different words” sentences
(synonyms)
• half “same θ-roles assignment” sentences (thematic subject and object were not moved), half
“different θ-roles assignment” sentences (subject and object were swapped)
Stimuli 180 blocks of 20 sentences corresponding to the 20 different primer types were generated.
Each subject was presented with exactly one pair of sentences from each single block. Conditions were
assigned to trials in a pseudo-randomized way.
As to plausibility, sentences were:
• 25% plausible and reversible (that is, they remained plausible when subject and object were
swapped; e.g. “le docteur a interrogé le patient”)
• 25% plausible and not reversible (e.g. “Le garçon a ciré ses chaussures”)
• 25% implausible and reversible (e.g. “le spaghetti a adoré les noisettes”)
• 25% implausible and not reversible (e.g. “le paquet a livré le facteur”)
All target sentences were built according to the same syntactic structure: DP + present perfect
verb + DP and had three content words.
Task Subjects were instructed to press one button when the sentence presented was judged plausible,
and the other when the sentence was judged implausible. The assignment of buttons to plausibility
conditions was reversed after two sessions. Subjects underwent a small training outside the scanner to
ensure they understood the task.
Procedure There were four sessions of 12 minutes for each subject, corresponding to the presentation
of 45 pairs of sentences. Sentences were presented using the rapid serial visual presentation (RSVP)
technique: words were projected one at a time for 270 msec on a screen in white letters against a black
background. After each sentence, the screen went black for a time that varied randomly between 4
and 6 sec for subjects 1 to 4 , 2 and 8 seconds for subjects 5 and 6, mean 5 sec. The wider interval
of variation was introduced in the last session to decrease statistical correlation of stimuli which could
have prevented satisfying modelization.
3.2.3
Behavioral results
Response time data across the four pilot subjects showed that there was a strong behavioral priming
effect of individual word meaning, θ-role attribution, word form (see fig.3.4). There was no effect of
syntactic structure of the primer sentence.
Chapter 3: Disentangling word form, single word meaning, and thematic roles
24
Figure 3.4: Effect of semantic priming on response time
3.2.4
Confusing imaging results
The initial plan was to compare habituation for different categories of semantic priming. However,
analysis of fMRI time series yielded weird activation and contrast maps. Not only did we fail to find
a decrease in activation, but the results of overall activation did not even seem to make a lot of sense.
This has led us to think that the design of the experiment was probably flawed, with regressors
being too correlated to allow for a reliable estimation of the model. However, I shall briefly go through
possible alternative explanations for the absence of results.
3.2.5
Possible other explanations
The stimuli Synonyms are never true synonyms. Therefore, if the adaptation found in the first
study was due to adaptation to individual word meaning, then that adaptation could disappear when
synonyms were used as primers instead of the real word, as their meaning only approximates that of
the target word.
Yet activation should have been reduced all the same in half the cases, when the very same words
were used as primer and target, and that was not found to be the case.
Moreover, the behavioral analysis shows that priming by synonyms was efficient in terms of reducing
reaction times.
Are the stimuli too short ? The target sentences were quite simple (affirmative active sentences
that seldom contained more than six words), which was not the case in Devauchelle et al. (2006).
25
3.2 The second study
Figure 3.5: Effect of semantic priming on response time
Chapter 3: Disentangling word form, single word meaning, and thematic roles
26
Perhaps these simple sentences led to small activations, the variations of which could more easily be
drowned in task-related variations.
Semantic incongruency The task used was a plausibility task. In order to make judgements about
plausibility as effortless as possible, sentences were often grossly meaningless, with unrelated semantic
words (e.g. “le paon a mangé le nuage”). Thus, adaptation to θ-roles might have been prevented
by the semantic incongruency of words, that might have discouraged the subject to try and attribute
θ-roles.
However, adaptation should have been observed for the sentences that were plausible, and that
was not the case (that is, when plausible sentences used the very same words, to avoid any problems
caused by the choice of synonyms on the one hand, and semantically incongruent words on the other
hands. This should have replicated the former study by Devauchelle et al. (2006)).
The task The subjects did not have to perform any particular task in the first study, apart from
pressing a button once in a while when instructed to. No task was used either in Noppeney & Price
(2004). Habituation paradigms in syntax might be particularlry sensitive to tasks.
We have used a plausibility judgment task, that has been used previously in imaging studies of language (Stromswold et al., 1996). Feedback from subjects showed that the task was indeed not that easy.
The task itself might have disturbed adaptation processes. If the task of making a plausibility
judgement is intellectually effortful, then subjects might have tried to reduce it to its minimum by
turning it into a linguistic similarity judgement:
1. They have made a judgement for the primer sentence
2. As they do not want to take great pains again making a judgement about the target sentence,
they compare it first against the first one to see if they have the same meaning (because if they
do, there is no longer any need to make a judgement)
3. If meanings do not match, they make a decision about plausibility again.
This could explain why we found areas that appeared to be more activated with target sentences
than primer sentences.
In particular, a possibility might be that the culprit for failure to model successfully the data is an
interaction between design and task. Indeed, we used a rapid event paradigm, that has been shown
(Josephs & Henson, 1999) to allow for good detection of differences in activation, but poor detection
of activations against baseline with the kind of SOA we used, especially as our design did not contain
a “silence” condition against which the other conditions could be compared.
The task may have led to intractably high variability from one trial to the other, so that small
differences related to semantic processing might have been drowned in parasit differences related to the
task, especially as the sentences used were often short and simple (this is true of every target sentence);
all conditions involving language, no “sanity check” could be performed to check that activated areas
seemed to make some sense.
3.3
Further directions
A slightly different version of the experiment is to be run soon. The task will be removed to replicate
the conditions of the first study by Devauchelle et al. (2006), and perhaps some silence added to allow
for more reliable statistical estimation of sheer activations. The findings of the first study should at
least be replicated.
Given the presence the behavioral priming effect, it is unlikely that priming is simply not working in
this case.
Chapter 4
Core syntax at the phrase level
I shall first start by comparing studies by Noppeney & Price (2004) and Devauchelle et al. (2006), and
from then move on to the pilot experiment I have run.
4.1
Reconciling Devauchelle et al. and Noppeney & Price
Both of the studies I am going to discuss now looked for habituation to syntactic structures; in one
case, no habituation was found, while in the other, an area was identified in the temporal lobe.
Noppeney & Price (2004)’s study compared activation to the presentation of sentences that were
syntactically ambiguous.
Two types of syntactic ambiguities were used: clause boundary ambiguity and reduced relative/main clause ambiguity (see examples table 4.1).
Subjects were simply asked to read the sentences silently, and informed that their eye-movements
would be monitored to ensure that they paid attention.
Sentences were presented by blocks of five sentences in either a primed or an unprimed condition;
in the primed condition, the five sentences were of the same syntactic type, while in the unprimed
condition, the types of the sentences were randomized.
The primed condition resulted in decreased left anterior temporal activation, providing evidence
that this area is involved in syntactic processing.
These results show that syntactic structure can indeed lead to habituation, ruling out an explanation
of Devauchelle et al.’s results based solely on an inadequacy of the habituation procedure.
However, there could be at least two explanations to this discrepancy:
• the number of different syntactic structures used was four in Noppeney & Price (2004), and
about twenty in Devauchelle et al. (2006)
• the study used a context of local ambiguity, which is not the case in Devauchelle et al.
Let us focus on the second explanation.
Devauchelle et al. manipulated syntactic structure in a way that made large use of the possibilities
offered by second level syntax, and the syntactic contrasts used, involving movement, cleft sentences,
passivization, etc, were all contrasts relevant to second-level (not core) syntax, and had not much to
say about core-level syntax.
Conversely, Noppeney & Price’s experiment used sentences where the syntactic contrast bore specifically on the core-level tree being built. Moreover, ambiguity was maintained throughout the experiment, which might have focused attention of the subjects specifically on the object of ambiguity - that
27
Chapter 4: Core syntax at the phrase level
28
Clause boundary ambiguity
1. Preferred reading: late closure.
e.g.: Before the director left the stage the play began.
2. Nonpreferred reading: early closure.
e.g.: after the headmaster had left the school deteriorated rapidly.
Reduced relative/main clause ambiguity
1. Preferred reading: simple active.
e.g.: The artist left his sculptures to the British Museum.
2. Nonpreferred reading: reduced relative.
e.g.: The child left by his parents played table football.
Table 4.1: Examples of sentences with the four different syntactic forms, Noppeney & Price (2004)
is, the tree being built -, as in a kind of challenge where the goal was to resolve the ambiguity (which
is this case amounted to guessing the underlying syntactic tree).
Thus, core-syntax processes might have been amplified by that context of ambiguity, which might
have made the adaptation easier to capture.
4.2
Tracking constituents
As reviewed in the background chapter, previous studies have provided some evidence that the core
process of syntax - the process that underlies all syntax, that is, chaining together two isolated words
to build a phrase - might be carried out in the anterior portion of the left temporal cortex.
The approach taken in the study I shall now describe stems from the intuition that if an area is involved in core syntactic processing, that is, is somehow in charge of building constituents from smaller
chunks of language, then it should vary quite monotonically with the number of words that build up
into a tree. Roughly speaking, the bigger the tree to build, the harder a tree-building area has to work.
More explicitly, it seems reasonable to expect that a node is harder to build when the leaves it
merges are “heavier”; that is, building a DP from D “the” and NP “cat” would be easier than building
a DP from D “the” and NP “cat that my mother found on the street”, so that we hope that activation
of this area for building the whole phrase “the cat that my mother found on the street” would be
higher than the sum of the activations for “the cat” and “my mother found a cat on the street”.
So, building the tree for the sentence: “the black cat chases the white mouse” is expected to lead
a hypothetical syntax area to activate more than the sums of activities engendered by processing
“the black cat”, “the white mouse”, “she chases him”, even if the overall number of nodes and types
of nodes (two DPs from a D, an AP and an NP, and a TP from two DPs and a tensed verb) are the same.
Building on this intuition, we devised an experiment where a constant number of words could be
combined into syntactic trees of progressively larger sizes. So to speak, we tried to fill the gap between
sentence and isolated word processing.
What’s twelve words ? Leaving aside all non-homogeneous assignments, twelve words can be a
twelve word constituent, two six word constituents, three four word constituents, four three word constituents, six two word constituents, or twelve isolated words. An area involved in building phrases
from smaller phrases should display a progressive increase in activation when constituents get bigger.
29
4.3 Material and methods
This is exactly what we looked for in this study.
Relying merely on the number of words in a constituent to predict activation might seem a little
too vague, as there are a number of other factors that come into play when it comes to evaluating
complexity of a structure. For instance, it might not seem reasonable to predict that it is harder to
build a representation for the phrase “the very very very very very very very very very very big car”,
than for the phrase “please let her help me fold these sheets.” Care was therefore taken that the most
accessible factors were controlled: thus, the number of content words did not vary across conditions
(indeed, stimuli were generated without obvious biasses, as the same content words were used across
conditions to ensure that the design was well balanced); passivization and negation were avoided, as
well as object relatives. Furthermore, only monosyllabic words were used, so that the syllabic input
was carefully controlled.
4.3
4.3.1
Material and methods
Subjects
This study is still in its early stages of development, so only two healthy, right-handed, native French
speakers with no history of neurological damage have participated in it so far, after giving written
informed consent.
4.3.2
Design
144 twelve-word series were presented to subjects in a slow event design in two runs of 16 minutes.
The series of words could be of six different conditions, according to whether they were formed of 1,
2, 3, 4, 6, or 12 word constituents (see table 4.2).
Single trials were presented in a pseudorandomized order to balance transition probabilities between
conditions.
4.3.3
Stimuli
Stimuli were generated by blocks of six sentences. Each sentence contained six monosyllabic content
words, and one dependent clause. Content words were then rearranged across sentences to obtain
blocks of 6 12 word series corresponding to each of the five remaining conditions. Function words were
not necessarily conserved, nor agreement or tense features.
In order to avoid subjects rearranging shuffled content words to get the original sentence back, content words were shuffled across sentences rather than keeping always the same content words together
in the same 12 word string.
All sentences were active, affirmative ones, with no object relative and a word order that tried to
be as canonical as possible. All the words used were monosyllables.
4.3.4
Task
Subjects were instructed to press a button whenever they saw a proper name. They were not explicitly
asked to understand the meaning of the sentences; however, syntactic parsing is a highly automatized
task, so we expected to see syntactic activation nonetheless. In order to avoid the task being a mere
“capital letter” spotting task, we presented all words in lower-case font, regardless of whether they
were a proper name or not.
Chapter 4: Core syntax at the phrase level
Conditions
One word
mur nous boire la des chez scène bas toile laisses très en
je belle nous moite sans tu brique par chez eau soie lâchent
sans rêche pour tombe longent sous frais un je mains chien des
pour rends cour chez soif je fibre des une geste par glisse
noir ils joie sans fais hors dans sueur elle jambe des donne
suie des peu elles toi bruit maude ils hors crains chanvre pour
Two words
le mur la scène des bas pour boire trop belle lâche ça
en brique une eau tout moite en soie donne ça une laisse
longe la trop frais ma main ils tombent le chanvre un chien
la cour la soif rends les ils glissent en toile le faire
tout noir une joie la sueur mes jambes une fibre un geste
la suie si simple crains le sans bruit pour maude trop rêche
Three words
longe la cour tu as soif glisse sans bruit ces deux chiens
noir de suie tant de joie sur sa jambe lâche la laisse
mur en brique trop de sueur sans un geste une fibre rêche
des mains moites tout ce chanvre bas de soie elle est fraı̂che
boire cette eau une belle toile en faire trop il y tombe
une simple scène donnes en moi craindre pour soi maude les rend
Four words
le mur en brique tout moite de sueur le chanvre est rêche
de la belle toile il longe la cour geste de la main
est noir de suie donnent de la fibre il craint le bruit
boire de cette eau le bas de soie il lâche la laisse
la soif est fraı̂che glisse sur sa jambe un chien qui tombe
ils en font trop une joie si simple maude rend la scène
Six words
le mur qui longe la cour le bas se soie qui tombe
la brique est noire de suie ses jambes glissent sans un bruit
boire de cette eau si fraı̂che le chanvre est une fibre rêche
par une simple soif de joie ça donne de la belle toile
la sueur de ses mains moites maude qui lui fait un geste
ils craignent de rendre la scène il lâche la laisse du chien
Twelve words
le mur en brique qui longe la cour est noir de suie
boire cette eau fraı̂che quand on a soif est une joie simple
il craint que la sueur rende ses mains moites sur la scène
le bas de soie qui tombe glisse sur sa jambe sans bruit
le chanvre est une fibre rêche qui donne de la belle toile
il lâche la laisse de son chien quand maude fait un geste
Table 4.2: Sample block
30
31
4.3 Material and methods
A proper name was inserted in 1 over 6 sentences (twice in every condition) and could occur anywhere in the sentence.
4.3.5
Procedure
Rapid serial visual presentation was used, to control for visual input and eye movements, while not
preventing syntactic processing. No clue whatsoever informed the subjects as to the structure of the
twelve word series he was presented; that is, no punctuation mark was inserted to trace boundaries
between constituents. This choice was made to ensure that conditions differred minimally visually.
A fixation cross was displayed for two seconds at the beginning of each trial to make sure that the
subjects were prepared to pay attention, and avoid any “surprise effect” at the first word.
Twelve word series were then presented one word at a time, with a word duration of 300 msec. Then
the screen went black for 8400 msec, so that the duration of a complete trial was 14 sec.
Subjects underwent a small training session outside the scanner to ensure they understood the
task, and to prepare them to the unfamiliar nature of the stimuli.
4.3.6
fMRI Scanning Technique
Imaging data were collected on a Bruker 3T scanner with standard head coil. Functional images
used EPI (TR=2.4 sec, TE=30 msec, Matrix=64x64 ; Voxel size=3.5x3x3mm; Number of slices = 40.
The anatomical scan used a 3D gradient echo sequence (TI=2530 msec, TE=3.3 msec, TR=8.6 msec;
voxels=1.33x1.2x1.2 mm).
4.3.7
Data Analysis
Data preprocessing and analysis were performed using SPM2 software. All volumes were realigned
to the first volume and resliced using a sinc interpolation. The time series (fMRI) in each voxel was
high-pass filtered to 1/60 Hz and globally normalized with proportional scaling.
A General Linear Model was estimated for each of the two subjects, using standard HRF as a basis
function. The model consisted of 13 activation conditions: 6 conditions for reading twelve-word strings
that made up constituents of one, two, three, four, six or twelve words, where no response had been
given, 1 condition corresponding to the pressing the button, and 6 conditions shadowing the first six
conditions for trials where a response had been emitted.
Another model with five FIR regressors per conditions was also estimated, to have a better feel of
the shape of thre responses. For this model, baseline activation was taken as the fixation of the cross
and the 2 first seconds after presentation of the first word of a trial.
A third model using Fourier basis functions set was also calculated, in order to look for any effect
of conditions on the phase of the response. Indeed, the 6 response-less conditions had a very different
phase pattern than the 6 with-response conditions; but no convicing difference could be found across
the 6 conditions of interest, so I shall not show data on the phase here.
In all models, nuisance covariates included the realignment parameters (to account for motion artefacts).
A linear contrast, postulating a linear increase across conditions, was used to locate areas showing
the expected monotonic pattern of activation (1 word < 2 word < 3 word < 4 word < 6 word < 12
word condition). We report activations at a significance threshold of p < 0.001, uncorrected.
Chapter 4: Core syntax at the phrase level
32
Figure 4.1: Areas activated by reading words
4.4
4.4.1
Results
Behavioral results
Subjects were asked to press a button whenever they saw a proper name. A proper name was inserted
in 1 over 6 sentences and could occur anywhere in the sentence. Thus, a subject had to detect 24
proper names on the whole.
Subjects performed significantly above chance. The second subject performed well with only two
false alarms and no misses. The first one was somewhat less reliable, and missed 4 proper names, while
pressing for two false alarms (e.g. pressing for “gorge”).
4.4.2
Network activated by all conditions
An activation of bilateral occipital, temporal areas was seen following presentation of words. Word
reading relative to baseline resulted in predominantly left-lateralized activation in the left superior
temporal sulcus, middle temporal gyri spreading into the temporal poles bilaterally. Activation was
also observed in the left inferior frontal gyrus and parietal lobe.
4.4.3
Linear contrast
The linear contrast was chosen to detect areas presenting an increase across conditions. It should be
noted that the contrast was linear according to condition numbers, not numbers of words, because we
expected the activation to increase more steeply in the low indice conditions.
The areas that were selected by this contrast were highly left lateralized.
The regions found for the two subjects could not be said to really overlap, however both subjects presented roughly the same pattern of areas responding to the linear contrast: one area in the
L-AG/L-MTG, one area more anteriorly in the L-MTG, and some patches of activation in the L-aMTG.
The more anterior areas displayed increasing activations, whereas the areas in the L-AG displayed
deactivation for the low-indice conditions, and activations not significantly different from baseline in
33
4.4 Results
Figure 4.2: Voxels responding to the linear contrast, p<0.001, uncorrected
Chapter 4: Core syntax at the phrase level
Table 4.3: MNI coordinates - clusters > 5 voxels
Subject 1
Coordinates
Region
x
y
z
T score Extent
Left angular gyrus
-48 -60 27
4.71
22
Left middle temporal gyrus
-42 -54 15
4.78
8
-54
6
-27
4.15
9
Left middle frontal gyrus
-45
6
51
4.08
12
Left precentral gyrus
Left superior frontal gyrus
Left medial superior frontal gyrus
Left inferior frontal gyrus
pars opercularis
Left inferior temporal gyrus
Right precentral gyrus
34
Subject 2
Coordinates
x
y
z
T score
-51 -66 21
3.56
-45 -57 12
3.99
-57 -15 -6
4.86
-39
-27
-9
-57
-6
57
42
9
63
24
51
24
3.96
4.11
4.30
4.16
7
6
9
8
-33
15
3
-21
-39
72
4.06
3.99
6
16
the higher-order conditions.
One of the two subjects also displayed a monotonic increase in activation in Broca’s area (L-IFG).
Other activations were found in various locations in one subject but they are not discussed as they
were not present in the other subject.
4.5
4.5.1
Discussion
Areas
The results could seem somewhat disappointing, as they show very patchy response to the linear
contrast. However, the most activated voxels were found in areas of the temporal lobe or the temporoparietal junction that were repeatedly implicated in the processing of sentence vs. isolated words
(Mazoyer et al., 1993; Vandenberghe et al., 2002; Stowe et al., 1998, 1999; Noppeney & Price, 2004;
Humphries et al., 2001, 2005, 2006; Kaan & Swaab, 2002) and the interface between syntax and
semantics (left posterior superior temporal areas, left temporo-parietal junction, left angular gyrus),
as I have reviewed in the second chapter (Ben-Shachar et al., 2003; Pinango & Zurif, 2001; Shapiro
et al., 1993; Shapiro & Levine, 1990; Bornkessel et al., 2005; Humphries et al., 2006).
4.5.2
Extent
9
6
12
Patterns of activation
Left inferior frontal gyrus As reviewed in the second chapter, a number of studies suggest that
Broca’s area is involved in linearizing displaced constituents and dealing with sophisticated parts of
the syntax.
There is not much room for movement and sophistication in a single word world, and even a twoword world. It seems reasonable to say that the more the words, the higher the probability that some
part of sophisticated syntax will sneak in. The activation found in Broca’s area for one of the subjects
fits that account, with a step-like increase followed by a smoother curve.
Left anterior temporal regions As can be seen on figs. 4.3 and 4.4, patterns of activation seemed
to be different in the left angular gyrus (L-AG) and anterior middle temporal gyrus (L-aMTG).
Activation in the middle anterior regions seemed to follow a quite smooth curve while the number of
words in a constituent increased, whereas activation in the more posterior regions seemed more of an
“all-or-none” type.
35
4.5 Discussion
Figure 4.3: Estimated parameters of the HRF model for the first subject
Chapter 4: Core syntax at the phrase level
Figure 4.4: Estimated parameters of the HRF model for the second subject
36
37
4.5 Discussion
Figure 4.5: Regions found in the study by Vandenberghe et al. (2002) (adapted from Vandenberghe
et al. (2002))
Chapter 4: Core syntax at the phrase level
38
These patterns do make some sense in the light of recent proposals regarding the roles of posterior and anterior temporal regions in language. As we have reviewed earlier, there is considerable
evidence linking the left anterior temporal regions to the processing of sentences, irrespective of their
semantic content (Mazoyer et al., 1993; Vandenberghe et al., 2002; Humphries et al., 2001, 2006) . If
this area is in charge of building trees, and it is not surprising that activation should increase gradually.
We had foreseen a steeper increase in the low indice conditions (which is why we chose a contrast
that predicted the same increase when going from 2 to 3 word phrases, or when going to 6 to 12 word
phrase). Rather, the activation seemed to plateau around the 6 word condition.
An explanation could be that the 12 word condition was the only one that did not contain an
unexpected discontinuity (unexpected because the subject had no clue about the condition they were
in until a word that could not be integrated into the tree appeared), thus allowing smooth reading. It
would be interesting to see if the plateau would be observed at 12 word phrases too, in a paradigm
where conditions would go from isolated words to, say, 18 word phrases.
Left posterior temporal regions/Left angular gyrus The posterior temporal regions have been
postulated to play a crucial part in the interface between syntax and semantics, particularly in the
attribution of agency and mapping of θ-roles on arguments (Ben-Shachar et al., 2003; Pinango & Zurif,
2001; Shapiro et al., 1993; Shapiro & Levine, 1990; Bornkessel et al., 2005; Humphries et al., 2006)..
If this is true, what activation pattern would we predict ?
In order for arguments to exist, there has to be space for them; that is, at least one verb and
one argument, and probably one word more to allow for a determiner. A steep increase in activation
of an area involved in argument processing should be expected, with the discontinuity corresponding
to the constraint that there need to be room for the argument. Indeed, only activations (that is,
deactivations) observed in conditions 1 and 2 differ significantly from baseline activation in the L-AG
in subject 1, and this is almost true of subject 2 as well (the 4 word phrase condition hardly reaches
significance).
Interestingly, the observed difference in activity for conditions 1 and 2 are deactivations, not activations.
This should not come as a surprise, as the left angular gyrus has been shown to be part of a network
of area that is spontaneously active during so-called “rest” (Binder et al., 1999). I shall get back to
this shortly.
A role for the L-AG and posterior MTG in attributing syntax-dependent meaning to words is
particularly interesting given how close these areas are to the left temporo-parietal junction (L-TPJ),
which has been postulated to mediate attribution of beliefs (Saxe, 2006). See fig. 4.7.
Attributing θ-roles to arguments of a verb might be seen as a very elementary preliminary to
attributing mental states to people; in both cases, a representation has to be reinterpreted in the light
of the context it is embedded in.
Of particular interest is the evidence that deaf children that do not sign natively have impaired
theory of mind when compared to native ASL signers, even when the task does not involve language
(de Villiers & de Villiers, 2000; Woolfe et al., 2002; Peterson & Siegal, 1995).
Thus, if we consider areas L-pSTS, L-AG and L-TPJ as a whole:
1. People with lesions in posterior temporal regions have impaired argument processing (Shapiro
et al., 1993; Shapiro & Levine, 1990) and impaired reinterpretation of words according to the
sentence they are in (Pinango & Zurif, 2001)
39
4.5 Discussion
Figure 4.6: Regions postulated to play a crucial role in understanding others’ minds. From Saxe et al.
(2004)
Figure 4.7: Activation in the bilateral TPJ is higher for stories involving Theory of Mind and attribution of false beliefs than stories about objects or descriptions of people or objects. Adapted from Saxe
& Kanwisher (2003)
Chapter 4: Core syntax at the phrase level
40
2. pSTS shows increased activation with verbs that require more arguments (Ben-Shachar et al.,
2003) and when a false thematic mapping has to be reinterpreted (Bornkessel et al., 2005)
3. People with lesions in the L-TPJ have impaired theory of mind (Samson et al., 2004)
4. bilateral TPJ is activated when attributing false beliefs to people (Saxe et al., 2004; Saxe &
Kanwisher, 2003)
5. These areas are spontaneously active at rest, so that they display deactivation in perceptual
tasks (Binder et al., 1999)
6. Late signers of ASL have impaired theory of mind
Let us try and tie this evidence together: there might be some kind of gradient of increasingly
sophisticated attribution of specific semantic content from posterior superior temporal lobe to the
temporo-parietal junction; attribution of agency and intention to persons requires the capacity to represent thematic functions, that is, to adapt the semantic content of a representation to a context of
interaction.
Attributing a θ-role to a noun is like creating a tight link between that noun and the verb which
takes it as argument, and this link is somewhat independent from the rest of the world.
Perhaps representing someone’s beliefs relies on this kind of ability to build a priviledged and “protected” (that is, isolated from the rest of the world) relationship between two entities. If I say, “the cat
chases the dog, and the dog runs away”, I have in fact two dogs: an accusative dog and a nominative
dog. I am able to replicate my representation of the dog as many times as needed with a particular
θ-role for each single verb of which it is an argument, and the θ-role attributed by a verb will not leak
out of the verb-argument pair.
This capacity to represent one single object with bilateral relationships (to different entities) that
do not interfere with one another might underlie the capacity to attribute beliefs to people: the marble
can be both inside the basket for Sally and inside the box for Anne, just as the dog is at the same
time Accusative and Nominative.
So how can we tell the story ?
During rest, we represent a thousand things that have all kind of interactions, and these areas
are active. If given a perceptual task that has nothing to do with attribution of states, then the
area deactivates. If the task requires some attribution of thematic roles, the area is as active as in
rest. Finally, if the task involves a great deal of reasoning about intentions and beliefs, then the areas
activate above baseline for real.
Chapter 5
General discussion and further
directions
5.1
Adaptation to semantics
Starting from the results of Devauchelle et al. (2006) which gave evidence for semantic habituation in
the left middle temporal gyrus, we ran an experiment to disentangle adaptation due to word form,
meaning of individual words, and attribution of θ-roles in the sentence.
We obtained results that varied widely (“wildly” might even be better suited here) between subjects and seemed to indicate that something was wrong with the design or the task, which prevented
statistical modelling from yielding any informative results.
Nice behavioral priming results showed that the stimuli used were promising; moreover, the results
of the statistical analysis did not look like “normal” results which would simply fail to show the type of
adaptation we were longing for, but rather, like something went completely wrong in the way, probably
at the level of the temporal presentation of events.
Despite a number of modifications to the design between the first four subjects and the remaining
two (e.g. increasing the jitter between conditions to decrease the correlation between conditions), the
results remained puzzling and call for more substantial modifications.
While we are still trying to find out what could lead to the initial puzzling results, we are planning
to clean the experiment from a number of possible sources of trouble, by removing the task, introducing
silent conditions, and perhaps even turning to a slow event related paradigm.
5.2
Looking for core syntax and constituent-building areas
The second study aimed to investigate the neural correlates of constituent building during silent wordlist reading.
Subjects were submitted to increasingly large constituents while controlling for the number of words,
content words, and the number of syllables (as only monosyllabic words were used).
Results of the two pilot subjects pointed towards involvement of areas strongly left-lateralized, in the
temporal lobe. Although the areas that appeared do not fully overlap, the results nonetheless showed
some consistency across subjects, in that the linear contrast we tested pointed in both cases towards
the recruitment of three clusters in the left angular/posterior middle temporal gyrus, anterior middle
temporal gyrus medially and in a more anterior part.
This consistency is particularly interesting in view of accumulating evidence of an involvement of these
41
Chapter 5: General discussion and further directions
42
areas in the normal processing of sentences, with various interactions with single-word level semantics
(Humphries et al., 2006, 2005, 2001; Noppeney & Price, 2004; Vandenberghe et al., 2002; Mazoyer
et al., 1993).
5.3
5.3.1
Further directions concerning the core-syntaxe study
Controlling for mere sequence effects
One might argue that the activation found is due not to processing of constituents, but with processing
of sequences of adjacent words. The paradigm used for the pilot study does not allow to separate
constituency and adjacency. An additional control condition will be introduced, where chunks of
sentences made up of an increasing number of adjacent words are presented.
5.3.2
Decreasing variability between subjects by introducing tighter control on stimuli
Results of the two pilot subjects show that the parametric increase of syntactic structure leads to
monotonic activity in similar areas across subjects, yet the overlap is not really satisfying.
The approach to generate stimuli was to eliminate a number of potentially confusing factors, while
keeping complete freedom within the framework of some constraints (number of content words, number
of syllables, no object relative, no passive, no negation).
This strategy can be viewed as an “opt-out” one, as it starts from the whole pool of grammatical
sentences and excludes some of them according to the given criteria. It allowed for wider coverage of
possible syntactic trees, but this might have been at the expense of consistency in activations across
subjects.
Shifting from this “opt-out”strategy to an “opt-in” one, where we start from scratch and decide
that such and such are the only sentence patterns that will appear in the stimuli, would allow for a
tighter control of stimuli, and would ensure that the syntactic structures presented are matched as
closely as possible across subjects.
We plan to introduce a tighter filter of possible syntactic structures, but have not decided yet how
wide the coverage of possible sentences will be.
Conversely, we consider abandoning the “monosyllabic word” constraint, as it does not seem obvious that the benefits of retaining it match its cost. Removing it from the design would allow a better
control of the other, more critical constraints.
5.3.3
Differentiating between agency and phrase building
The areas we found to respond to a linear contrast according to the number of words in maximal
constituents have received increasing interest from investigators looking for syntax, as reviewed in
chapter 2.
Here is the proposed mapping 1 that might guide ideas:
1. left posterior superior temporal areas are involved in retrieving meaning of single words as modified by the sentence context in which they appear. This corresponds to activation related to
treatment of argument properties of predicate, θ-roles attribution
1 It
should be noted that studies that agree on the involvement of areas in the left temporal lobe vary in terms of
lateralization. We keep the safer “left-lateralized” assumption, especially as our data were strongly left-lateralized.
43
5.3 Further directions concerning the core-syntaxe study
2. left anterior middle temporal areas might be involved in concatenating individual meanings of
words into the meaning of a sentence
3. more anterior parts of the left anterior middle temporal gyrus and the left temporal pole seem
to subserve sentence building even with pseudowords or semantically incongruent words
Keeping that framework in mind, it is tempting to modify the experimental design to see if the
activation we find in these areas is separable.
As we have proposed in the discussion, the step-like increase in activation in the left posterior superior temporal areas could reflect the surge of θ-roles mapping on arguments, while the smoother curves
in more anterior parts of the temporal lobe would correlate with integrating syntactic and semantic
structures into larger ones.
Thus an exciting perspective would be to separate these activations by manipulating independently
arguments and tree depth; a way to achieve that separation without changing much of the design would
be to split some conditions (e.g., four word and six word conditions) between an “argument heavy”
and an “argument light” condition; for instance, contrasting “ma soeur parle à son fils le merle chante
un bel air” and “le fils de ma grande soeur le beau merle tout en noir”. If the proposed hypotheses
are true, then we would expect activation of the posterior superior temporal, but not anterior middle
areas to display an interaction of argument-type and number-of-word conditions.
A more refined design could involve comparisons between phrases where content DPs (as opposed
to pronouns) are arguments and phrases where the same DPs appear but in an A bar position, thus
receiving no θ-role, while making the underlying syntactic tree deeper, e.g.: “the dog bit my uncle”
vs. “my uncle’s dog bit them”.
If the hypothesis that the left posterior superior temporal areas subserve reinterpretation of meaning in the context of a sentence is true, then it should be less activated in the case where the internal
argument of the verb is a pronoun than in the case where it is a content DP.
On the other hand, an area recruited to integrate individual meanings into sentences and syntactic
phrases into larger phrases should activate more when the tree is deeper, that is, in the second phrase.
Thus, splitting conditions according to this kind of considerations might shed light on the role of
these various areas, and help tease apart processes involved in sentence processing at large.
5.3.4
Digging constituency further: priming by constituents vs. adjacent
strings of words
Investigating the influence of priming a sentence by some of it constituents would be another way to
identify areas involved in processing constituents, while keeping the framework of a parametric increase
in the size of constituents.
The idea would be to compare activation to target sentences (e.g. “le bas de soie qui tombe glisse
sur sa jambe sans bruit”) that would be primed by a string of words in one of four conditions:
1. constituents of the target sentence (e.g. “sans bruit sur sa jambe bas de soie”)
2. constituents of an unrelated sentence (e.g. “le soir la peau douce bout des doigts”)
3. nonconstituent strings of adjacent words of the target sentence (e.g. “soie qui tombe glisse sur
sa bas de” )
4. nonconstituent strings of adjacent words of an unrelated sentence (e.g.“lui parlent mère quand
ils il rêve de”)
Chapter 5: General discussion and further directions
44
The prediction would be that in a tree-building area, activation to sentences that were primed by
their constituents would be lower compared to sentences that were primed to adjacent strings of words.
It would be interesting also to add another condition, with primer being made up of adjacent words
that do not form a constituent, yet are the beginning of one, e.g. priming “je crois que tu as tort”
by “crois que tu” (adjacent), “je crois que” (adjacent + beginning of constituent), “tu as tort” (contituent). If an area is recruited only to perform the merging operation, then it should not show more
adaptation to the adjacent + beginning of constituent condition; conversely, if an area is recruited to
keep on-line chunks of sentences that still need to be integrated, then this area might display more
adaptation to the adjacent + beginning of constituent condition.
Concluding remarks
In this report, I have presented a review of the literature on syntax processing in the brain. In order
to have a clearer view of the results investigators have obtained when looking for syntax in the brain, I
have proposed that it is necessary to make a distinction between at least two levels of syntax, with the
more basic level being in charge of building phrases and the more sophisticated taking those phrases
and playing around with them.
From the review of the literature, it appears that left frontal language areas seem to come into play
whenever a sequence of words is presented, that does not follow some kind of canonical order stipulated
in terms of syntax, θ-roles and perhaps other hierarchies as well.
Left posterior superior temporal areas have been repeatedly reported to show increased activation
in relation to interface between semantics and syntax; that is, when the meaning of a word has to
be refined according to its appearance in a precise sentence. This includes argument processing and
θ-roles attribution.
Still in the left temporal lobe, more anterior areas that were often neglected when studying language
are beginning to attract more attention. Among those, it seems that more posterior areas show a
greater sensitivity to semantics, while more anterior areas extending to the anterior temporal pole
seems to respond primarily to sentence structure regardless of their semantic content.
I have presented results of two pilot subjects of an experiment which tried to trace down syntax in the
brain by using a parametric increase in the size of constituents that were presented. The areas that
emerged from the analyses in both subjects are situated in the left temporal lobe and could correspond
to the proposed mapping of sentence processing on left posterior superior and anterior middle temporal
areas.
Finally, I have presented further directions to explore that could help tease apart components of
sentence processing.
Bibliography
Altmann, G. & Steedman, M. (1988). Interaction with context during human sentence processing.
Cognition, 30.
Ben-Shachar, M., Hendler, T., Kahn, I., Ben-Bashat, D. & Grodzinsky, Y. (2003). The neural reality
of syntactic transformations: evidence from functional magnetic resonance imaging. Psychol Sci ,
14, 433–440.
Ben-Shachar, M., Palti, D. & Grodzinsky, Y. (2004). Neural correlates of syntactic movement: converging evidence from two fMRI experiments. Neuroimage, 21, 1320–1336.
Binder, J.R., Frost, J.A., Hammeke, T.A., Bellgowan, P.S., Rao, S.M. & Cox, R.W. (1999). Conceptual
processing during the conscious resting state. a functional mri study. J Cogn Neurosci , 11, 80–95.
Bookheimer, S. (2002). Functional MRI of language: new approaches to understanding the cortical
organization of semantic processing. Annu Rev Neurosci , 25, 151–188.
Bornkessel, I., Zysset, S., Friederici, A.D., von Cramon, D.Y. & Schlesewsky, M. (2005). Who did what
to whom? The neural basis of argument hierarchies during language comprehension. Neuroimage,
26, 221–233.
Broca, P. (1861). Remarques sur le siège de la faculté du langage articulé, suivies d’une observation
d’aphémie. Bulletin de la Société Anatomique, XXXVI, 330–357.
Caplan, D., Alpert, N. & Waters, G. (1998). Effects of syntactic structure and propositional number
on patterns of regional cerebral blood flow. J Cogn Neurosci , 10, 541–552.
Caplan, D., Alpert, N. & Waters, G. (1999). PET studies of syntactic processing with auditory sentence
presentation. Neuroimage, 9, 343–351.
Caplan, D., Alpert, N., Waters, G. & Olivieri, A. (2000). Activation of Broca’s area by syntactic
processing under conditions of concurrent articulation. Hum Brain Mapp, 9, 65–71.
Caramazza, A. & Zurif, E.B. (1976). Dissociation of algorithmic and heuristic processes in language
comprehension. Brain and Language, 3, 572–582.
Chomsky, N. (1957). Syntactic Structures. Mouton, La Haye.
Chomsky, N. (1986). Knowledge of Language Its Nature Origin and Use. Praeger, New-York.
Chomsky, N. (1995). The Minimalist Program. MIT Press.
Dapretto, M. & Bookheimer, S.Y. (1999). Form and content: dissociating syntax and semantics in
sentence comprehension. Neuron, 24, 427–432.
de Villiers, J. & de Villiers, P. (2000). Children’s Reasoning and the Mind . Hove.
Devauchelle, A.D., Oppenheim, C., Rizzi, L., Dehaene, S. & Pallier, C. (2006). An fMRI study of
syntactic and semantic adaptation. In Cognitive Neuroscience Society Annual Meeting.
45
BIBLIOGRAPHY
46
Embick, D., Marantz, A., Miyashita, Y., O’Neil, W. & Sakai, K.L. (2000). A syntactic specialization
for Broca’s area. Proc Natl Acad Sci U S A, 97, 6150–6154.
Friederici, A.D. & Kotz, S.A. (2003). The brain basis of syntactic processes: functional imaging and
lesion studies. Neuroimage, 20 Suppl 1, S8–17.
Friederici, A.D., Pfeifer, E. & Hahne, A. (1993). Event-related brain potentials during natural speech
processing: effects of semantic, morphological and syntactic violations. Brain Res Cogn Brain Res,
1, 183–192.
Friederici, A.D., Meyer, M. & von Cramon, D.Y. (2000). Auditory language comprehension: an eventrelated fMRI study on the processing of syntactic and lexical information. Brain Lang, 74, 289–300.
Friederici, A.D., Fiebach, C.J., Schlesewsky, M., Bornkessel, I.D. & von Cramon, D.Y. (2006). Processing Linguistic Complexity and Grammaticality in the Left Frontal Cortex. Cereb Cortex .
Frith, C.D. & Frith, U. (1999). Interacting minds–a biological basis. Science, 286, 1692–1695.
Frith, U. & Frith, C.D. (2003). Development and neurophysiology of mentalizing. Philos Trans R Soc
Lond B Biol Sci , 358, 459–473.
Geschwind, N. (1965). Disconnexion syndromes in animals and man. i. Brain, 88, 237–294.
Geschwind, N. (1979). Specializations of the human brain. Sci Am, 241, 180–199.
Grewe, T., Bornkessel, I., Zysset, S., Wiese, R., von Cramon, D.Y. & Schlesewsky, M. (2005). The
emergence of the unmarked: a new perspective on the language-specific function of Broca’s area.
Hum Brain Mapp, 26, 178–190.
Grodzinsky, Y. (2000). The neurology of syntax: language use without Broca’s area. Behav Brain Sci ,
23, 1–21; discussion 21–71.
Grodzinsky, Y. & Friederici, A.D. (2006). Neuroimaging of syntax and syntactic processing. Curr Opin
Neurobiol , 16, 240–246.
Haegeman, L. (1991). Introduction to government and binding theory. B. Blackwell Cambridge, Mass.,
USA.
Humphries, C., Willard, K., Buchsbaum, B. & Hickok, G. (2001). Role of anterior temporal cortex in
auditory sentence comprehension: an fMRI study. Neuroreport, 12, 1749–1752.
Humphries, C., Love, T., Swinney, D. & Hickok, G. (2005). Response of anterior temporal cortex to
syntactic and prosodic manipulations during sentence processing. Hum Brain Mapp, 26, 128–138.
Humphries, C., Binder, J.R., Medler, D.A. & Liebenthal, E. (2006). Syntactic and semantic modulation
of neural activity during auditory sentence comprehension. J Cogn Neurosci , 18, 665–679.
Josephs, O. & Henson, R.N. (1999). Event-related functional magnetic resonance imaging: modelling,
inference and optimization. Philos Trans R Soc Lond B Biol Sci , 354, 1215–1228.
Just, M.A., Carpenter, P.A., Keller, T.A., Eddy, W.F. & Thulborn, K.R. (1996). Brain activation
modulated by sentence comprehension. Science, 274, 114–116.
Kaan, E. & Swaab, T. (2002). The brain circuitry of syntactic comprehension. Trends Cogn Sci , 6,
350–356.
Kang, A.M., Constable, R.T., Gore, J.C. & Avrutin, S. (1999). An event-related fMRI study of implicit
phrase-level syntactic and semantic processing. Neuroimage, 10, 555–561.
Keller, T.A., Carpenter, P.A. & Just, M.A. (2001). The neural bases of sentence comprehension: a
fMRI examination of syntactic and lexical processing. Cereb Cortex , 11, 223–237.
47
BIBLIOGRAPHY
Kuperberg, G.R., McGuire, P.K., Bullmore, E.T., Brammer, M.J., Rabe-Hesketh, S., Wright, I.C.,
Lythgoe, D.J., Williams, S.C. & David, A.S. (2000). Common and distinct neural substrates for
pragmatic, semantic, and syntactic processing of spoken sentences: an fMRI study. J Cogn Neurosci ,
12, 321–341.
Lasnik, H. (2002). The minimalist program in syntax. Trends Cogn Sci , 6, 432–437.
Liberman, A.M., Cooper, F.S., Shankweiler, D.P. & Studdert-Kennedy, M. (1967). Perception of the
speech code. Psychological Review , 74, 431–461.
Mazoyer, B.M., Dehaene, S., Tzourio, N., Frak, V., Murayama, N., Cohen, L., Lévrier, O., Salamon, G., Syrota, A. & Mehler, J. (1993). The cortical representation of speech. Journal Cognitive
Neuroscience, 5, 467–479.
Miller, G.A., Heise, G.A. & Lichten, W. (1951). The intelligibility of speech as a function of the context
of the test materials. JEP , 41, 329–335.
Moro, A., Tettamanti, M., Perani, D., Donati, C., Cappa, S.F. & Fazio, F. (2001). Syntax and the
brain: disentangling grammar by selective anomalies. Neuroimage, 13, 110–118.
Ni, W., Constable, R.T., Mencl, W.E., Pugh, K.R., Fulbright, R.K., Shaywitz, S.E., Shaywitz, B.A.,
Gore, J.C. & Shankweiler, D. (2000). An event-related neuroimaging study distinguishing form and
content in sentence processing. J Cogn Neurosci , 12, 120–133.
Noppeney, U. & Price, C.J. (2004). An FMRI study of syntactic adaptation. J Cogn Neurosci , 16,
702–713.
Osterhout, L., Holcomb, P.J. & Swinney, D.A. (1994). Brain potentials elicited by garden-path sentences: evidence of the application of verb information during parsing. J Exp Psychol Learn Mem
Cogn, 20, 786–803.
Peterson, C.C. & Siegal, M. (1995). Deafness, conversation and theory of mind. J Child Psychol
Psychiatry, 36, 459–474.
Pinango, M.M. & Zurif, E.B. (2001). Semantic operations in aphasic comprehension: implications for
the cortical organization of language. Brain Lang, 79, 297–308.
Samson, D., Apperly, I.A., Chiavarino, C. & Humphreys, G.W. (2004). Left temporoparietal junction
is necessary for representing someone else’s belief. Nat Neurosci , 7, 499–500.
Saxe, R. (2006). Uniquely human social cognition. Curr Opin Neurobiol , 16, 235–239.
Saxe, R. & Kanwisher, N. (2003). People thinking about thinking people. the role of the temporoparietal junction in ”theory of mind”. Neuroimage, 19, 1835–1842.
Saxe, R., Carey, S. & Kanwisher, N. (2004). Understanding other minds: linking developmental psychology and functional neuroimaging. Annu Rev Psychol , 55, 87–124.
Schlosser, M.J., Aoyagi, N., Fulbright, R.K., Gore, J.C. & McCarthy, G. (1998). Functional MRI
studies of auditory comprehension. Hum Brain Mapp, 6, 1–13.
Scott, S.K. & Johnsrude, I.S. (2003). The neuroanatomical and functional organization of speech
perception. Trends Neurosci , 26, 100–107.
Shapiro, L.P. & Levine, B.A. (1990). Verb processing during sentence comprehension in aphasia. Brain
Lang, 38, 21–47.
Shapiro, L.P., Gordon, B., Hack, N. & Killackey, J. (1993). Verb-argument structure processing in
complex sentences in Broca’s and Wernicke’s aphasia. Brain Lang, 45, 423–447.
BIBLIOGRAPHY
48
Sperber, D. & Wilson, D. (1986). Relevance: Communication and Cognition. Harvard University Press,
Cambridge MA.
Stowe, L.A., Broere, C.A., Paans, A.M., Wijers, A.A., Mulder, G., Vaalburg, W. & Zwarts, F. (1998).
Localizing components of a complex task: sentence processing and working memory. Neuroreport,
9, 2995–2999.
Stowe, L.A., Paans, A.M., Wijers, A.A., Zwarts, F., Mulder, G. & Vaalburg, W. (1999). Sentence comprehension and word repetition: a positron emission tomography investigation. Psychophysiology,
36, 786–801.
Stromswold, K., Caplan, D., Alpert, N. & Rauch, S. (1996). Localization of syntactic comprehension
by positron emission tomography. Brain Lang, 52, 452–473.
Tyler, K.T. & Marslen-Wilson, W. (1977). The on-line effects of semantic context on syntactic processing. Journal of Verbal Learning and Verbal Behavior , 16, 683–692.
Vandenberghe, R., Nobre, A.C. & Price, C.J. (2002). The response of left temporal cortex to sentences.
J Cogn Neurosci , 14, 550–560.
Vasic, N., Avrutin, S. & Ruigendijk, E. (2006). Interpretation of pronouns in VP-ellipsis constructions
in Dutch Broca’s and Wernicke’s aphasia. Brain Lang, 96, 191–206.
Woolfe, T., Want, S.C. & Siegal, M. (2002). Signposts to development: theory of mind in deaf children.
Child Dev , 73, 768–778.
Abbreviations table
L (e.g. in L-IFG)
R
Anatomy
Left
Right
a (e.g. in L-aSTS)
p
Anterior
Posterior
IFG
STS
STG
MTG
TPJ
AG
fMRI
Inferior Frontal Gyrus
Superior Temporal Sulcus
Superior Temporal Gyrus
Middle Temporal Gyrus
Temporo-parietal Junction
Angular Gyrus
Functional magnetic resonance imaging
SOV
OSV
DP
NP
ASL
Linguistics
Subject Object Verb
Object Subject Verb
Determiner Phrase
Noun Phrase
American Sign Language
49

Download Report

Stop, Crying, Stop Crying : When 1 + 1 = 1

Paperzz.com

Your Paperzz