Brain embodiment of syntax and grammar:
Discrete combinatorial mechanisms spelt out in neuronal circuits
Friedemann Pulvermüller
Medical Research Council, Cognition and Brain Sciences Unit, Cambridge CB2 7EF, UK
This paper has been accepted for publication and is in press in the journal
Brain and Language
running head: Brain Embodiment of Syntax
Key words: associative memory, brain, discrete combinatorial rule, grammar, neuronal
assembly, pushdown store, sequence detector, syntax, tree structure
234 words in abstract, 7,724 words in text, 155 references, 3 tables, 4 figures
Address for correspondence:
Friedemann Pulvermüller, Ph.D.
MRC Cognition and Brain Sciences Unit
15 Chaucer Road
Cambridge CB2 2EF, UK
phone: +44 1223 355294 ext. 670
fax: +44 1223 359062
e-mail: [email protected]
Acknowledgements: I would like to thank Valentino Braitenberg, Andreas Knoblauch,
Marion Ormandy, Yury Shtyrov, Mark Steedman, Kambiz Tavabi, and two anonymous
referees for their comments and help at different stages of this work. Supported by the
Medical Research Council (UK) (U1055.04.003.00001.01, U1055.04.003.00003.01) and by
the European Community under the “New and Emerging Science and Technologies”
Programme (NEST-2005-PATH-HUM contract 043374, NESTCOM).
2
BRAIN EMBODIMENT OF SYNTAX
ABSTRACT
Neuroscience has greatly improved our understanding of the brain mechanisms of abstract
lexical and semantic processes. The brain mechanisms underlying words and concepts are
distributed neuronal assemblies reaching into sensory and motor systems of the cortex and, at
the cognitive level, this mechanism is mirrored by the sensorimotor grounding of form and
meaning of symbols. Recent years have seen the emergence of evidence for similar brain
embodiment of syntax. Neurophysiological studies have accumulated support for the
linguistic notion of abstract combinatorial rules manifest as functionally discrete neuronal
assemblies. Concepts immanent to the theory of abstract automata could be grounded in
observations from modern neuroscience, so that it became possible to model abstract
pushdown storage – which is critical for building linguistic tree structure representations – as
ordered dynamics of memory circuits in the brain. At the same time, neurocomputational
research showed how sequence detectors already known from animal brains can be neuronally
linked so that they merge into larger functionally discrete units, thereby underpinning abstract
rule representations that syntactically bind lexicosemantic classes of morphemes and words
into larger meaningful constituents. Specific predictions of brain-based grammar models
could be confirmed by neurophysiological and brain imaging experiments using MEG, EEG
and fMRI. Neuroscience and neurocomputational research offering perspectives on
understanding abstract linguistic mechanisms in terms of neuronal circuits and their
interactions therefore point programmatic new ways to future theory-guided experimental
investigation of the brain basis of grammar.
3
BRAIN EMBODIMENT OF SYNTAX
1. Introduction
In spite of great progress in the neurosciences in understanding the mechanisms of
language and conceptual thought, there is still one domain that appears to be largely immune
to brain evidence. This domain, syntax, is far removed from concrete events and is built on
highly sophisticated abstraction. Even its most basic phenomena are best described in terms of
abstract formula and it therefore may appear doubtful – if not impossible – that its core
principles can be translated into the profane language of nerve cells and circuits. After all,
linguists have spent some effort to prove that it is impossible to build a finite neuronal
automaton processing relevant aspects of syntax (Chomsky, 1963) and the project of unifying
language and the brain at a theoretical level has therefore been long abandoned by most
language researchers. Paradoxically so, because the human brain is the very instrument whose
invention is equivalent with that of language. Important clues about the mechanisms of syntax
and grammar may therefore come from brain research in the effort to understand specifically
human abstract abilities. However, at present, the mainstream linguistic position seems to be
that neuroscience is still far from addressing the relevant questions (see, for example,
Jackendoff, 2002). To cite Chomsky: “The ERP (event-related potential) observations are just
curiosities, lacking a theoretical matrix” (Chomsky, 2000) (p. 25). Is this position wellfounded? This review will give the answer.
Linguists would not state that modern neuroscience ignores the topic of syntax.
Neuroimaging and also neuropsychology have in fact been immensely successful in defining
the brain regions activated during syntactic processing and even the regions whose lesion
leads to syntactic failure (Caplan, 2006; Friederici, 2002; Grodzinsky & Friederici, 2006;
Hagoort, 2005; Ullman, 2001). However, this work had so far little impact on linguistics. One
reason for this may be that much neuroscience research focused on the question of WHERE in
4
BRAIN EMBODIMENT OF SYNTAX
the brain syntax is localized, demonstrating that the relevant regions are mainly in left
perisylvian language cortex, but also involve adjacent areas, the insula and even widespread
areas in both the left and right hemispheres; and also subcortical structures are relevant,
including basal ganglia and thalamus. The where question as such is not considered to be
extremely exciting by many linguists, and skepticism towards modern variants of phrenology
(the 19th Century science of where aspects of the mind are located) is shared by
neuroscientists with a theoretical interest (for example, Knight, 2007).1 In contrast, the
question of WHEN physiological indicators of syntax would first occur does spark more
interest, especially in psycholinguists. The precise timing of EEG and MEG responses
provides clues about the time course of syntactic information access in the context of lexical
and semantic processing; this has implications for psycholinguistic theory (Friederici, 2002;
Hagoort, 2005, 2008; Pulvermüller & Shtyrov, 2006). However, the most interesting
functional question of HOW syntax is represented and computed in brain circuits, has, for a
long time, not been addressed at all.
At the mechanistic level of nerve cells and their connections, theoretical work on
syntax associated with empirical efforts is still relatively sparse. Substantial work focuses on
neural network architectures that may solve specific syntactic problems (Botvinick & Plaut,
2006; Chang, Dell, & Bock, 2006; Chater & Manning, 2006; Elman, 1990; Hanson &
Negishi, 2002) and some studies of neural networks even relate syntactic and serial order
computations to cortical areas and subcortical structures (Beiser & Houk, 1998; Dominey,
1
Note, furthermore, that the sluggish haemodynamic response revealed by fMRI is naturally difficult to relate to
rapid linguistic processes such as sentence comprehension. In many cases, experiments use secondary tasks and,
therefore, features of these tasks, rather than the comprehension of critical linguistic structures, may be reflected in
brain activation. Here is a citation from a recent publication which exemplifies and illustrates this problem: “Many
participants reported after the experiment that they had employed a strategy of internally generating a second
sentence of the form used in the comprehension task in order to prepare for the possible response. Thus, this kind of
task preparation may have engendered the frontoparietal activation observed here” (Bornkessel, Zysset, Friederici,
von Cramon, & Schlesewsky, 2005) (p. 230).
5
BRAIN EMBODIMENT OF SYNTAX
1997; Dominey, Hoen, Blanc, & Lelekov-Boissard, 2003; Voegtlin & Dominey, 2005).
However, only a few proposals exist that relate syntax to defined mechanisms, for example to
cell populations sequentially linked into synchronously firing “synfire” chains establishing
serial order, functionally discrete neuronal assemblies that structure the sequential activation
of other neuronal populations, and the ordered dynamics of whole brain areas brought about
by the various excitatory and inhibitory neuronal populations they house (Bienenstock, 1996;
Braitenberg, 1996; Hayon, Abeles, & Lehmann, 2005; Pulvermüller, 1993, 2003a;
Smolensky, 1999; Wennekers, Garagnani, & Pulvermüller, 2006; Wennekers & Palm, 1996;
Wickens, Hyland, & Anson, 1994). This paper will focus on the question what we know and
can reasonably assume about the mechanistic level of syntax in the brain. It will review views
on the embodiment of grammar in neuronal circuitry and discuss implications of neuroscience
experiments for linguistic theory.
2. Neuroscience evidence for phonological and semantic circuits
In neuroscience, recent progress led to a better understanding of the brain mechanisms
of phonology and semantics. The brain areas for phoneme processing were identified and
even fine-grained local differences in processing speech sounds with different features could
be mapped (Diesch, Eulitz, Hampson, & Ross, 1996; Obleser et al., 2006; Uppenkamp,
Johnsrude, Norris, Marslen-Wilson, & Patterson, 2006). Old models – according to which
language comprehension engages the posterior language area in the superior temporal cortex
of the dominant left hemisphere (“Wernicke’s area”) and the left inferior frontal language area
(“Broca’s area”) acts as a centre for speech production (Lichtheim, 1885; Wernicke, 1874) –
could be overthrown, especially by evidence for the involvement of inferiorfrontal and even
motor areas in language comprehension and, vice versa, the activation of temporal regions in
6
BRAIN EMBODIMENT OF SYNTAX
production (Fadiga, Craighero, Buccino, & Rizzolatti, 2002; Fadiga, Fogassi, Pavesi, &
Rizzolatti, 1995; Paus, Perry, Zatorre, Worsley, & Evans, 1996; Pulvermüller et al., 2006;
Pulvermüller, Shtyrov, & Ilmoniemi, 2003; Wilson, Saygin, Sereno, & Iacoboni, 2004;
Zatorre, Evans, Meyer, & Gjedde, 1992). Motor systems activation even reflects, and
contributes to, the perception of phonological features in the understanding of speech
(D’Ausilio et al., 2009; Pulvermüller et al., 2006). The picture emerging now is this:
Phonemes and morphemes, the smallest units that distinguish or carry meaning, are
represented and processed in the brain by distributed neuronal ensembles spread-out over
fronto-temporal language areas (Pulvermüller, 1999). These circuits are lateralized to the left
dominant hemisphere, although significant portions of their neurons may be housed in the
right non-dominant hemisphere (Grodzinsky & Friederici, 2006; Hickok & Poeppel, 2000;
Marslen-Wilson & Tyler, 2007; Pulvermüller & Mohr, 1996).
The links between signs and their meaning critically depend on long-distance corticocortical connections. Words referring to objects or actions correspond to lateralized frontotemporal circuits linking up with object or action representations in visual and other sensory
areas or motor systems of the brain (Barsalou, 2008; Martin, 2007; Pulvermüller, 2005). Clear
evidence for this view comes from neuroimaging studies and from work with patients. Frontal
areas activate specifically to words semantically related to actions, for example action verbs
such as ‘write’ or ‘speak’. Words semantically related to visually defined objects activate
specific parts of the inferior-temporal stream of object processing (Chao, Haxby, & Martin,
1999; Pulvermüller & Hauk, 2006; Simmons et al., 2007) and even words meaning odours
may activate specifically areas involved in olfaction (González et al., 2006). Hence, the brain
appears to classify words according to their referential semantic features and to the modalities
involved in processing this information. Semantic mappings – or semantic topographies – can
7
BRAIN EMBODIMENT OF SYNTAX
be so sharp that they even distinguish action words according to the body part with which the
referent action is preferably executed (Aziz-Zadeh, Wilson, Rizzolatti, & Iacoboni, 2006;
Buccino et al., 2005; Hauk, Johnsrude, & Pulvermüller, 2004; Pulvermüller, Härle, &
Hummel, 2000; Tettamanti et al., 2005). Critically, action-perception links may even
contribute to abstract semantic processing (Boulenger, Hauk, & Pulvermüller, 2009; Glenberg
& Kaschak, 2002; Glenberg, Sato, & Cattaneo, 2008).
The mentioned studies revealing activation loci – and therefore addressing the
WHERE question – have implications for cognitive theory. As motor-articulatory and
acoustic systems of the brain are co-activated in processing of language sounds, the results
endorse theories postulating a link of articulatory and acoustic-phonetic information at both
the neural and cognitive levels (Braitenberg & Pulvermüller, 1992; Fry, 1966; Galantucci,
Fowler, & Turvey, 2006; Liberman & Mattingly, 1985; Pulvermüller et al., 2006). At the
semantic level, the co-activation of left fronto-temporal language regions with modalityspecific areas reflecting semantic aspects of language materials supports neurocognitive
theories of embodied cognition that view the conceptual and semantic store as a result of
information exchange with the world in the context of perceptions and actions (Barsalou,
1999, 2008; Glenberg & Kaschak, 2002; Pulvermüller, 1999; Warrington & McCarthy, 1987).
Interestingly, these activations reflecting aspects of the meaning of words and sentences
emerge extremely rapidly, within 1/5 second after input information allows for unique
recognition of the critical word. This implies that, at the cognitive level, these activations
reflect language comprehension and meaning access – rather than late processes of
reprocessing or reanalysis (Hauk, Shtyrov, & Pulvermüller, 2008; Pulvermüller, 2005;
Pulvermüller, Shtyrov, & Ilmoniemi, 2005). Claims that the multimodal links in cortex might
not be essential for semantic and linguistic processing (Mahon & Caramazza, 2008) are
8
BRAIN EMBODIMENT OF SYNTAX
confronted with observations from the clinic: Word processing degrades in a category specific
manner if action- or visually-related brain areas are lesioned and a pronounced processing
deficit for action words and concepts accompanies degeneration of the motor system in Motor
Neuron Disease (Bak, O'Donovan, Xuereb, Boniface, & Hodges, 2001; Damasio, Grabowski,
Tranel, Hichwa, & Damasio, 1996; Neininger & Pulvermüller, 2003; Tranel, Damasio, &
Damasio, 1997). In Semantic Dementia, where frontotemporal areas are most severely
affected, the same subcategories of action- and visually-related words, which normally most
strongly activate these areas, are most severely impaired (Pulvermüller et al., 2009). These
lexico-semantic deficits accompanying circumscribed brain dysfunction endorse a critical role
of sensorimotor areas in the processing of form and meaning of symbols (Pulvermüller et al.,
2009).
3. Abstract combinatorial rules in the human brain: neurophysiological results
Notwithstanding the progress made in the phonological and the semantic domains, one
may judge that the core of our linguistic powers, the specifically human ability to variably
combine discrete meaningful units into rule-conforming higher-order structures, has so far not
been enlightened by neuroscience research (cf. Chomsky, 2000; Jackendoff, 2002). Still, in
the neurophysiological domain, effort has been spent to find brain indexes that possibly
indicate syntactic or grammatical processes. A good number of studies compare brain
responses to ungrammatical strings with those to grammatical, well-formed ones. This
comparison is particularly important to linguistic theory, as most theories of syntax provide
parses for grammatical strings, but separate out ungrammatical strings by parser blockage, a
distinction sometimes thought to be best assessable empirically using the grammaticality
9
BRAIN EMBODIMENT OF SYNTAX
judgment task. Neurophysiology may provide the tool for looking at the brain’s
grammaticality judgements directly.
A range of components of the event-related brain potential and field, including the
early left-anterior negativity (ELAN, Friederici, Pfeifer, & Hahne, 1993; ELAN, Neville,
Nicol, Barss, Forster, & Garrett, 1991), a late posterior positivity (P600, Hagoort, Brown, &
Groothusen, 1993; P600, Osterhout & Holcomb, 1992), and the early left-lateralised syntactic
Mismatch Negativity (sMMN, Shtyrov, Pulvermüller, Näätänen, & Ilmoniemi, 2003), are
enhanced to ungrammatical as compared with grammatical word strings (for review, see
Friederici, 2002; Hagoort, 2005; Kuperberg, 2007; Pulvermüller, Shtyrov, & Hauk, 2009).
Cortical localization indicates that two areas, the inferior frontal gyrus and the superiortemporal cortex, house the neuron populations that contribute to these physiological responses
potentially linked to grammatical sequence computation (Friederici, Wang, Herrmann, Maess,
& Oertel, 2000; Pulvermüller & Shtyrov, 2003; Shtyrov, Pulvermüller, Näätänen, &
Ilmoniemi, 2003). Interestingly, these same areas are also involved in processing nonlinguistic sequences (Fazio et al., 2009; Maess, Koelsch, Gunter, & Friederici, 2001).
However, it is difficult to decide whether the brain response to an ungrammatical
string reflects its syntactic features or rather the conditional probability with which words
follow each other. As every ungrammatical string is usually also a very rarely occurring item,
it is also possible that the degree of expectedness – or string probability – is reflected. This
point, which equally applies to comparisons between simple (and therefore frequent) and
more complex (rarer) sentence types (see discussion below), is especially virulent, as the
major brain responses discussed as possible reflections of ungrammaticality and syntactic
processing load – ELAN, P600 and sMMN – are similar in latencies and distributions to wellknown components of the event-related potential that reflect the probability with which
10
BRAIN EMBODIMENT OF SYNTAX
stimuli occur within an experiment. The ELAN, falls into the time range of the N100 and
N200 components, which are modulated by attention, the P600 resembles closely the
attention-related P300, and the sMMN is a left-lateralized variant of the Mismatch Negativity,
or MMN, which reflects change detection. Respectable arguments have been brought up to
defend the position that components elicited by ungrammatical sentences are different from
their nonlinguistic sisters – for example, that the syntactic P600 is different from the P300
(Osterhout & Hagoort, 1999) – but this does not prove a specifically syntactic role of the
respective component. It may still be that sequential probabilities in language use, rather than
the presence or absence of an underlying discrete combinatorial rule, determine the emergence
of a syntactic ELAN, P600 or MMN. Direct evidence for a neuronal correlate of rules is
therefore not provided by the work reviewed so far. Although the number of occurrences of
grammatical and ungrammatical strings can be balanced exactly within a given experiment,
the general occurrence probability of a sentence, sentence type, or relevant section thereof, is
usually higher than that of a similar asyntactic string. As both grammaticality and sequential
probability may have a correlate in the brain response, it is necessary to ask whether these
effects can be separated.
Important information about grammatical processes in the brain may come from
studies of different types of correct sentences differing in their grammatical structure, word
order or memory requirements. However, as performance shapes grammar (Hawkins, 1994), it
is naturally difficult to separate effects of grammatical structure per se from those of string
frequency. As already mentioned, if sentences with “canonical” and “noncanoncal” word
order are contrasted, the influence of string type frequencies may play a role. For example,
noncanoncical passive or object-relative sentences activated left inferiorfrontal/premotor and
superiortemporal areas more strongly than canonical active or subject-relative sentences
11
BRAIN EMBODIMENT OF SYNTAX
(Caplan et al., 2002; Just, Carpenter, Keller, Eddy, & Thulborn, 1996; Kinno, Kawamura,
Shioda, & Sakai, 2008). Similarly, the posterior inferiorfrontal area was more strongly active
for subordinate clauses with an initial accusative noun phrase (NP) referring to nonanimate
objects than for similar structures with an initial dative noun phrase including an animate
noun (Grewe et al., 2006); note that the former seem to be rarer than the latter and,
interestingly, grammaticality ratings reflect this frequency difference (Keller, 2001; Kempen
& Harbusch, 2003). Together with data on grammaticality violations mentioned above, these
results on grammatical sentence types are compatible with the idea that syntactic circuits are
housed in inferiorfrontal cortex, with an additional potential contribution of superiortemporal
areas. Interestingly, as Keller’s data demonstrate, a link between gradually more or less
canonical syntactic structure and grammaticality ratings exists (Keller, 2001). However, this
fact still leaves us with the critical question of how to separate string probability and
grammaticality per se.2
In a recent experiment using the sMMN as the dependent variable, an attempt was
made to dissociate rule- and probability-related processes. All critical sequences of words
were presented with equal probability in the experiment, but both grammaticality and the
probability with which these strings are used in language did vary. Critical words were
identical in different conditions so that lexical or semantic factors could not account for any
differences. Determiner-noun agreement was used as the syntactic feature of interest, as it
represents a syntactic link without semantic implication. As in previous work, grammatical
strings frequently used in language were compared with rare ungrammatical ones. However,
in a third condition grammatical but extremely rare strings were presented. The fourth logical
2
Further important issues are critical in the investigation of words, phrases and sentences with different
morphological and grammatical structure. It is important to match stimulus sentences for psycholinguistic variables
such as the length and standardized lexical frequency of their critical words, their sequential probability at their
12
BRAIN EMBODIMENT OF SYNTAX
possibility – ungrammatical strings that occur frequently – is not available, because a
descriptive grammar classifies frequently occurring strings as regular. In this sense, Bever and
colleagues pointed out that ungrammatical strings used in everyday language may initiate rule
changes (Bever, Carroll, & Hurtig, 1976). Because of their changing or even unclear
grammatical status, “frequent ungrammatical” strings were not included in the experiment.
For the third and critical condition, rare but grammatical strings, two competing
predictions are available. Cognitive scientists and computational modelers have claimed that
rules (in the sense of discrete combinatorial units) do not exist in neural systems, brains
included, and that probability mapping is the mechanism that explains the human ability to
string words in sentences (Elman et al., 1996; McClelland & Patterson, 2002). A probability
mapping approach without discrete neuronal units predicts that a rare grammatical string
elicits a brain response comparable to that of similarly rare ungrammatical strings. However,
if the brain uses a discrete rule mechanism to distinguish between grammatical and
ungrammatical strings, this mechanism should be involved to the same degree in the
processing of rare and common grammatical word strings with the same syntactic structure,
but it should fail to process ungrammatical word sequences. This does not rule out a possible
physiological difference between brain processes sparked by rare and common grammatical
strings.3 Therefore, probability mapping and rule access theories lead to different predictions
on the brain response to rare grammatical strings of words.
The syntactic enhancement of the MMN was found to rare ungrammatical strings
compared with common grammatical ones, as had been reported in a range of previous
experiments (Hasting, Kotz, & Friederici, 2007; Menning et al., 2005; Pulvermüller, Shtyrov,
position in the stimulus sentence or sentence type, their cloze probabilities and other features of the context along with
lexical properties (Pulvermüller, 2007; Pulvermüller & Shtyrov, 2006).
13
BRAIN EMBODIMENT OF SYNTAX
Hasting, & Carlyon, 2008; Shtyrov, Pulvermüller, Näätänen, & Ilmoniemi, 2003). Relevant
cortical generators are being sparked early (60-200 ms) in inferiorfrontal and superiortemporal
cortex. Critically, syntactic but rare phrases led to an MMN similar to that of common
grammatical strings (Figure 1). The sMMN seems to behave as if it was an automatic
grammaticality index (Pulvermüller, Shtyrov, Hasting, & Carlyon, 2008). The
neurophysiological rule correlate revelaed by the sMMN provides support for the rule access
theory (Pulvermüller & Assadollahi, 2007) and is consistent with the existence of discrete
combinatorial neuronal processors generating the discrete neurophysiological responses.
Rules appear to be effective in the brain’s syntactic computations.
Figure 1 about here
4. Recursion, pushdown storage and syntactic linkage
A simple sentence, such as (1) in Table 1, can be assembled from words, morphemes
and phrases by successively applying rewriting rules, for example (2a)-(e). Together with an
additional rule, (3), (2a) can be applied repeatedly, so that the string becomes longer and the
same syntactic relationship is established between newly added items. This describes the wellknown recursivity (or recursiveness) of grammar. Generally, a function is recursive if the
function itself is applied within its own definition. Rules (2a) and (3) together therefore result
in recursivity.
Insert Table 1 about here
Linguists have argued that languages spoken by humans all have the feature of
recursivity, allowing us, in principle, to build infinitely many sentences out of a limited
3
As the brain is so sensitive to conditional probabilities (Donchin, 1981; Kutas & Hillyard, 1984; Näätänen, Gaillard, &
Mäntysalo, 1978; Näätänen, Tervaniemi, Sussman, Paavilainen, & Winkler, 2001), it is not surprising to find a physiological
correlate for them in an experiment investigating word sequences.
14
BRAIN EMBODIMENT OF SYNTAX
vocabulary and set of rules. Languages include a few 10,000s of morphemes (Pinker, 1994)
and current grammar theories use in the range of 100 rule formula to describe relevant
fragments of their grammars (see, for example, Gazdar, Klein, Pullum, & Sag, 1985).
Linguists sometimes argue that, in principle, indefinitely many sentences can thus be built
from a limited set of elements, although this statement assumes that strings can be endless,
and under this assumption even the duck language is indefinitely recursive, as sequences of
“quack”s could also be considered to be, in principle, not limited in length. Clearly, indefinite
recursiveness cannot therefore be a critical feature of human language.
However, recursion of a structurally interesting type, especially the variant making it
possible to embed sentences into each other – as illustrated by examples (1), (4) and (5) above
– might be characteristic of humans (Hauser, Chomsky, & Fitch, 2002). The duck’s endless
quacking is easily generated by recursive activity in a finite state automaton (Kleene, 1956;
McCulloch & Pitts, 1943), whereas the complex sentences are not (Chomsky, 1963). It is
therefore of the essence to ask what algorithms and, ultimately, brain mechanisms such
special recursion is based on.
Chomsky showed that the generation of center-embedded strings of the type discussed
requires a context-free grammar (including rule pairs like (2a) and (3)) and that this type of
grammar can be mechanistically implemented as one kind of abstract automaton (Chomsky,
1963). The automaton type does not need the great computational capacity of a Turing
machine – which uses a store, in which it can, at any point in time, freely access all its
memory entries. Chomsky showed that a machine with reduced processing power and
memory access, which reads from and writes into its store in a predefined order, is
advantageous for processing language, especially certain types of complex sentences
(Hopcroft, Motwani, & Ullman, 2001). The last entry into the store has to be retrieved first,
15
BRAIN EMBODIMENT OF SYNTAX
the second second, and the first entry entered is retrieved first. As this ordering is similar to
the one generated by putting items on a stack and later removing them again (from top to
bottom), the terms “pushdown” or “stack automaton” are sometimes used. In the above
example, with each application of the rules (2a) and (3), a new VP symbols is put on the stack,
“pushing down” any symbols that might already sit on the stack. In retrieval, the last VP
entered pops up first, calling for immediate completion of the phrase started last and so on.
This iterative procedure results in a centre-embedded structure (cf. sentences (6)-(8)).
Availability of multiple stacks and the possibility to move items between stacks further
increases the generation power of the automaton (Joshi & Schabes, 1997).
5. Is there a neurophysiological basis for pushdown storage?
Memory researchers found nerve cells that store specific memory contents (Fuster,
1997, 2003). If a monkey has to keep in memory that a stimulus (characterized, for example,
by a specific colour) has been presented and the animal has to find a matching stimulus later,
cells which are specific to memory content (the relevant colour) become strongly active.
Many of these cells show strong activation initially and subsequently loose activity
continuously – following approximately an exponential or power function. Interestingly, these
memory cells would stay at an enhanced state of activity until the sample stimulus appears
and the memory content is being retrieved. It has been argued that the mechanism underlying
the continuous activity of memory cells is the reverberation of neuronal excitation in
distributed neuronal assemblies or memory circuits (Fuster, 1997, 2003; Fuster & Jervey,
1981). Circuits of this type may take a role in object or action representation and also in
controlling sequences of processing events (e.g., Sigala, Kusunoki, Nimmo-Smith, Gaffan, &
Duncan, 2008).
16
BRAIN EMBODIMENT OF SYNTAX
The neurophysiological mechanisms revealed by memory cell activity can provide the
brain basis of pushdown storage. Given there are several memory circuits whose dynamics are
governed by the same activation-deactivation function, these circuits can be activated in
sequence and will each start loosing activity thereafter. As the first circuit will therefore have
lost activity when the second is sparked, and both of these will have (further) declined when a
third one activates, the hierarchy of circuit activation levels reflects the temporal order of
inputs. The circuit activated last will be at the highest level and the first-activated will be at
the lowest, resembling a pushdown stack. A read-out mechanism only accessing the most
highly active circuit reads the last memory entry first and the first entry last, thus providing
typical pushdown memory retrieval (Figure 2, Braitenberg & Pulvermüller, 1992;
Pulvermüller, 1993). Multiple pushdown stacks can, in principle, be located in different brain
areas. Neuroscience research therefore suggests a brain correlate of abstract devices critical
for representing and processing complex sentences.
Figure 2 about here
Interestingly, single linguistic representations may be used several times in the
processing of a sentence. This is a requirement for recursion. Note that, in the generation of
example sentence (5), the N and V symbols are each used three times and three “copies” of
the VP symbol – for the complements of the three initial NPs – remain in the pushdown store
for some time. Linguists have claimed that neural networks cannot cope with this problem of
multiple processing of the same element, which was dubbed the “problem of two”, “type
token problem”, or “problem of multiple instantiation” (Jackendoff, 2002; Marcus, 2001;
Sougné, 1998). This critique seems, however, to be aimed at neural networks of a certain kind
(e.g., Elman et al., 1996; e.g., Rumelhart & McClelland, 1987). Applied to the brain itself, it
17
BRAIN EMBODIMENT OF SYNTAX
would lead to the apparent paradox that the language organ of humans cannot process human
language – an irrational position.
Whereas a large class of neural networks is unable to cope with the problem of
multiple instantiation, solutions are offered by specific brain-inspired network architectures
(Shastri & Ajjanagadde, 1993; Sougné, 1998, 2001; van der Velde & de Kamps, 2006). One
mechanism for the multiple processing of individual representations is provided by neuronal
circuits that can hold several distinct activations at a time. Such distinct activations can be
precisely timed synchronous waves traveling independently in the same discrete circuit, for
example in reverberatory synfire chains (for details and simulations, see Hayon, Abeles, &
Lehmann, 2005). There is evidence for synfire chains from neuroscience experiments (Abeles,
Bergman, Margalit, & Vaadia, 1993; Plenz & Thiagarajan, 2007), although the proposal of a
co-existence of two or three traveling waves of activity in the same representation is presently
based on computer simulations. Together, multiple circuit activation and neuronal pushdown
storage provide critical neural mechanisms that may underpin the binding of linguistic
constituents in syntactic strings (Pulvermüller, 2003a). However, both proposals are at a
theoretical, neurocomputational level, still awaiting direct experimental support.
As the outlined brain basis of abstract algorithmic syntactic processes is consistent
with current knowledge about brain function, it would be false to state that linguistic
representations and processes are far removed from brain mechanisms. It will be critical for
future investigations of syntax to have available a brain model of syntax that predicts brain
activation based on the dynamics of syntactic circuits. Availability of such models is
necessary for bridging the brain-mind gap in the language sciences.
Abstract algorithmic rules can be applied iteratively, without an upper limit for the
number of iterations. This limitlessness had been an argument for the superiority of grammar
18
BRAIN EMBODIMENT OF SYNTAX
systems to neural networks, which always have restricted numbers of elements and were
therefore believed not to be capable of generating context-free grammars equivalent to
pushdown automata, let alone mildly context-sensitive ones and devices employing multiple
stacks. Linguists had previously argued that neural architectures can be modified to
accommodate a pushdown mechanism (e.g., Petri, 1970; Schnelle, 1996). The present
proposal is also open to limitless iterations. Both the number of simultaneously active circuits
(holding activity at different levels) and the number of simultaneously reverberating activation
waves in a given circuit can, in principle, be increased without limit. This allows free choice
of the numbers of embedding levels and instantiations of a given category. However, a
realistic neuronal model of syntax should account for limits also, as it aims to realistically
mirror brain mechanisms, not abstract ones. As a matter of fact, strings with two or three
levels of centre-embedding are documented in language use, but are extremely rare (Karlsson,
2007) and, if used at all, frequently misunderstood (Bach, Brown, & Marslen-Wilson, 1986).
Noise leading to merging of simultaneously traveling waves in the same circuit accounts for
difficulties caused by word repetitions, for example the fact that sentence (9) is more difficult
to process than (4) – although these sentences have the same syntactic structure (see also
Marcus, 2008). It seems important to offer accounts of these and similar limitations within a
neuronal language theory (Christiansen & Chater, 1999).
Taken together, these considerations related to system limitations and noise suggest
that the neuronal pushdown store and multiple activations of the same neuronal assembly are
fragile mechanisms that should be used sparsely in grammar theories. While a neuronal model
of grammar may exploit them in the processing of sentence embedding, a less fragile and
complementary system would be advantageous for computing syntactic relationships within
19
BRAIN EMBODIMENT OF SYNTAX
simple sentences and clauses. Is there evidence for more elementary mechanisms of syntactic
linkage?
6. From sequence detectors in animals to syntactic linkage
Morphemes, words and larger constituents within a sentence are related to each other.
These relationships are not restricted to elements that directly follow each other in the
sequence. Linear models of syntactic links, describing pairs, triplets or, more generally, ktuplets of adjacent morphemes, are therefore insufficient for syntactic analysis. Sentence (10)
(Figure 3) illustrates some local and distant syntactic links in a sentence.
Figure 3 about here
There are next-neighbor syntactic links between the transitive verb and its complements, the
head noun of the subject and the object realized as pronoun. Also, the verb and its affix can be
analyzed as morphologically linked. Non-local syntactic relationships include agreement
between subject noun and verb suffix (Kate … -s) and verb-particle linkage (build … up).
Describing these relationships in a two-dimensional tree structure is not easily possible,
because projection lines would cross if the links between connected but distant constituents
were added to the tree (dashed lines in (10b), Figure 3). Linguistic theories did therefore
introduce additional mechanisms – for example percolation of grammatical features through
the tree (for an introduction, see Haegeman, 1991) – to capture noun-verb agreement or
movement to account for particle displacement. However, after a slight revision of the
mechanism for syntactic linkage, these additional mechanisms may not be necessary.
Syntactic links between meaningful language units, words or larger constituents,
including the ones described by rewriting rules ((2), (3)), agreement and other non-local
syntactic relationships, have been proposed to be neurobiologically grounded in discrete
20
BRAIN EMBODIMENT OF SYNTAX
combinatorial neuronal assemblies, DCNAs, that link together pairs of constituents
(Pulvermüller, 2003a). The DCNAs are discrete as their activation takes one of several
discrete states, they are combinatorial as they link together, or bind, specific lexical and
morphological categories, and they are higher-order neuronal representations of linguistic
units above the level of lexical units. Apart from linking categories together, typical DCNAs
establish a temporal order between the category members they bind to.4 In the neuronal
framework, each of the syntactic links indicated by lines in (10a) is carried by one DCNA.
Is there reason for assuming DCNAs? The central nervous system of higher mammals
(Barlow & Levick, 1965; Hubel, 1995) and already that of invertebrates (Reichardt & Varju,
1959; Varju & Reichardt, 1967) includes neuronal units, so-called motion detectors, that
respond specifically to sequences. The sequence detectors link to two other neuronal units, A
and B, and respond specifically if these “input” units are activated in a given order, A–then–
B, but not, or much less, to the reverse order of activations, B–then–A. The mechanisms
resulting in sequence-sensitivity are diverse and may involve non-linear computations (Borst,
2007; Egelhaaf, Borst, & Reichardt, 1989), but may also result from an additive integration
process (Hubel, 1995). As sequence sensitive units are so common in nervous systems, it
would be surprising if they were unavailable for processing combinatorial information in
sentences. Sequence detectors for morphemes and words may provide a mechanism for
dynamically linking constituents pair-wise, thus realizing syntactic links that are effective in
processing a sentence (Pulvermüller, 2003b). The linkage includes the previous activation of
first and second word in a string (e.g., noun-verb), the resultant ignition of the order-sensitive
DCNA and a binding process functionally linking the latter with the active lexical
representations. The binding mechanism is best thought of as synchronized oscillatory activity
4
DCNAs that do not impose temporal order (thus acting, in principle, as AND units for two constituents) are
21
BRAIN EMBODIMENT OF SYNTAX
at high frequencies (gamma band), as oscillatory dynamics appear to reflect lexical (Canolty
et al., 2007; Pulvermüller, Birbaumer, Lutzenberger, & Mohr, 1997) and sentence processing
(Bastiaansen & Hagoort, 2006; Weiss et al., 2005). A critical question remains how abstract
syntactic processing units operating on classes of lexical items can be built from sequence
detectors (Section 7).
Note that the non-local nature of the syntactic binding mechanism5, which may apply
to adjacent elements or units that are separated in time, does not make it necessary to postulate
a principal difference between phrase structure relationships between adjacent words in a
sentence, as captured by tree structures, and agreement mechanisms between distant units, nor
does it run into difficulty in the processing and representation of discontinuous constituents,
such as particle verbs (see example sentence (10)). These few remarks on the perspectives of
neurobiological grammar mechanisms must remain sketchy, but a neuronal grammar model
integrating neuronal pushdown storage (Section 5) and syntactic linkage by way of DCNAs
has been laid out elsewhere (Pulvermüller, 2003a). In this approach, the elementary clauseand sentence-internal level of processing is dealt with by syntactic sequence detectors and the
more complex simultaneous and recursive processing of more than one sentence, for example
in the case of embedding, is dealt with by the pushdown memory component.
7. Discrete combinatorial neuronal assemblies: How they emerge
Are syntax and grammar largely inborn or rather learned? As mentioned in Section 3,
neuronal network modelers view probability mapping in simple recurrent networks as
thought to underlie free constituent order and certain forms of scrambling (Pulvermüller, 2003a).
5
The term “binding” has a range of different usages in linguistics. Binding sometimes specifically refers to the referential
properties of nominals (Chomsky, 1981), or to a range of different syntactic relationships that lead to unification of
constituents (Hagoort, 2005). Here, the term “syntactic binding” is used in the latter, widest sense, further assuming that a
neurophysiological binding mechanisms by oscillatory synchrony provides the neuronal mechanism for it (Pulvermüller,
Birbaumer, Lutzenberger, & Mohr, 1997).
22
BRAIN EMBODIMENT OF SYNTAX
sufficient for growing syntax in automata (see Section 3, Elman et al., 1996; see Section 3,
McClelland & Patterson, 2002; Rumelhart & McClelland, 1987). The linguistic notion of a
discrete combinatorial rule is, in this view, ill-defined and syntactic regularities can be learned
by associative learning principles. Some linguists, on the other hand, deny the relevance of
learning for building syntactic representations. In their view, the genetic code provides the
critical machinery and learning plays a peripheral role – comparable to the labeling of the
keys of a typewriter whose mechanistic system is already fully in place (Chomsky, 1980).
However, research into statistical language learning shows that a lot of structural information
about syntax and grammar can be extracted from corpora and therefore does not require preprogrammed knowledge. For example, information about the lexical classification of words
can be extracted from the local context in which they appear using statistical (Briscoe &
Carroll, 1997; Lin, 1998; Townsend & Bever, 2001) or neural network methods (Elman,
1990; Honkela, Pulkki, & Kohonen, 1995). If genuinely grammatical information can be
extracted by the learner from corpora and language input, the postulate of inborn information
about lexical categories violates the parsimony principle and the preferable solution is to
attribute the formation of lexico-syntactic categories to learning.
On the other hand, an unconstrained associative learning perspective not providing any
neural basis for discrete combinatorial rules appears to be insufficient too, as the rulelessness
assumption seems to be in conflict with current neuroscience evidence (Section 3). In essence,
the linguistic approach appears correct in postulating rules, whereas neural networks
perspectives rightly exploit statistical information about correlation of lexical items. The
critical question now becomes whether a putative brain correlate of discrete combinatorial
rules might emerge in brain-like networks due to biologically plausible correlation learning.
23
BRAIN EMBODIMENT OF SYNTAX
The concept of DCNAs for syntactic binding (cf. footnote 5) offers a new perspective
on this debate. Neuronal modeling encourages precise questions about which specific types of
wirings are a prerequisite of grammar and about the precise role of associative learning. A
possible mechanism is this: Elementary sequence detectors (Section 6) are sensitive to ordered
pairs of words/morphemes, and would, by way of reciprocal connections and synchronization,
link functionally (or merge) with two lexical circuits, thereby building a higher-order syntactic
unit. Obviously, “binding units” for specific word pairs are insufficient for processing the
structure of a sentence, because classes of lexical elements, not individual items, need to be
linked together. A fruitful idea here is that a range of binding units for word pairs become
linked among each other. The emerging complex aggregates of sequence detectors thereby
become sensitive to any lexical element included in a particular syntactic class or lexical
category followed by any other element belonging to a different lexical category (Knoblauch
& Pulvermüller, 2005; Pulvermüller, 2003b).
In a simulation project exploring syntactic learning, we used a pre-structured autoassociative memory with built-in sequence detectors for every possible pair sequence of
words. The “grammar area” of the network did not only include these pre-wired sequence
detectors, but, in addition, initially very weak links between all pairs of sequence detectors.
This was done in an attempt to incorporate an important feature of the cortex (O'Reilly, 1998).
Based on the statistics of cortical connections, it has been argued that the cortex is, in fact, an
auto-associative memory (Braitenberg & Schüz, 1998) and a network with similar auto-rich
auto-associative connectivity (Palm, 1993; Willshaw, Buneman, & Longuet-Higgins, 1969)
was therefore chosen as a basis of the modeling. Binding of sequence detectors into circuits
operating on classes of lexical items was indeed observed in these simulations. This binding
was critically related to the availability of sequence detectors and auto-associative links
24
BRAIN EMBODIMENT OF SYNTAX
between them, which could be strengthened by associative learning. As Figure 4 shows,
sequence detectors in the network’s grammar area (black dots) became strongly linked not
only to their respective lexical representations (red lines to grey dots at the left and top) but
also among each other (black lines). The links between the elementary sequence detectors
sensitive to word pairs were selectively strengthened because sequence detectors were
regularly co-activated in the process of string learning and word substitution between strings.
This co-activation of sequence detectors led to the emergence of neuronal aggregates
comprising sequence detectors sensitive to similar contexts. Because of the strong internal
connections within the neuronal aggregates, they acted as a higher-order discrete functional
unit for syntactic binding.
Figure 4 and Table 2 about here
Note that the network explains generalization by a simple mechanism. Any possible
first word is bound into the syntactic binding circuit, as any possible second item is, too.
Activity will therefore spread from any active neuronal unit included in the first lexical class
to all items included in class 2, regardless of whether the specific sequence had been learned
before or not. Even though only some of the possible noun-verb pairs had been learned,
sequence learning and substitution between a subset of the nouns and verbs in their respective
contexts led to priming between any noun and verb included into corresponding combinatorial
subsets (Table 2). Subsequent to priming by possible first string elements, full activation of
any of the possible second elements led to DCNA ignition and to syntactic binding of the
constituent pair involved.
25
BRAIN EMBODIMENT OF SYNTAX
In one sense, one can argue that, after learning, syntactic binding units formed in the
network provided a mechanistic correlate of an abstract binary rule.6 Interestingly,
combinatorial information taken from the British National Corpus, BNC, led to subclassification of nouns and verbs into lexico-semantic sub-categories and sub-rules. Figure 4
shows separable sub-rules/DCNAs for nouns denoting human subjects and verbs referring to
specifically human actions, and nouns referring to flying objects and verbs related to flying.
These syntactic-semantic sub-classifications depended on the threshold with which the
network operates: At a low activation threshold, the two DCNA merged into one yielding
syntactic priming between all nouns and verbs. At a higher threshold, the syntactic-semantic
rules were kept separate. The functionally discrete rule representations could also overlap to a
degree neuroanatomically, that is, in the sequence detectors they consisted of, while still
staying functionally separate (for further discussion, see Pulvermüller & Knoblauch, 2009).
Grades of canonicity and grammaticality of constructions can be represented by DCNAs with
different connection strengths. Note that the neuronal binding mechanism illustrated for single
words can also operate on larger syntactic constituents.
In sum, DCNAs that unify lexico-semantic categories and develop as outlined above
can act as syntactic binding circuits in establishing grammatical relationships in simple
sentences (see Figure 3). Although not all syntactic phenomena are easily modeled by these
emerging aggregates of sequence detectors, they do provide a candidate neuronal mechanism
for one type of grammar rule (Knoblauch & Pulvermüller, 2005; Pulvermüller, 2003b;
Pulvermüller & Knoblauch, 2009).
6
Here is a description of the type of rule covered: Let a and b be two syntactic categories, where members of a
usually precedes b members. The rule SD(ab) licences the n*m sequences of elements of a and b, and does so even though
26
BRAIN EMBODIMENT OF SYNTAX
8.
Syntax embedded in semantics: Relationship to Cognitive and Construction Grammar
Whereas most grammar frameworks list syntactic rules, the lexicon and semantics as
separate components, the brain-mechanistic framework views DCNAs as directly connected
to word (morpheme, and other lexical) representations, which, in turn, connect with their
semantic networks. Syntax and semantics are therefore functionally connected and the
“syntactic” DCNAs, after ratifying a given order of constituents, can mediate the activation
equilibrium between lexical-semantic circuits. Also, as illustrated in Section 7, DCNAs bind
together lexical-semantic classes of constituents (see Table 2, Figure 4). Lexical-semantic
word categories are manifest in the sets of DCNAs they connect with. These units, in turn,
determine the grammatical strings in which the lexical units regularly occur. This
amalgamated nature of syntax and semantics puts the neurobiological perspective in the
vicinity of lexical grammar theories in the cognitive linguistic tradition viewing grammar as
an integration machinery for form and meaning (e.g., Bresnan, 2001; Lakoff, 1987;
Langacker, 1991).
The neurobiological-mechanistic approach assumes (a) basic morpheme
representations linking form and meaning and, in addition, (b) storage of representations for
morpheme sequences by way of sequence detectors and discrete combinatorial neuronal
assemblies, DCNAs (see also Feldman, 2006; Feldman & Narayanan, 2004; see also
Pulvermüller, 2003a). This view fits well with proposals in the Construction Grammar
framework that word sequences can be represented, or “listed”, as higher-order lexical items
(Goldberg, 2006; Lakoff, 1987). The present proposal takes this idea further and views rules
as emergent properties of multiple listings of sequences, bound together as a consequence of
the recombination of string segments. In addition, a distinction is proposed between the
the input is sparse, that is, the number of learned strings is substantially below n*m. Such a rule becomes recursive if the
27
BRAIN EMBODIMENT OF SYNTAX
separate listing, or representation, of 1/ morpheme sequences and 2/ single higher-order
lexical items, complex words or idioms. Different neurophysiological indexes may reflect
linguistic binding at the lexical and syntactic levels (Shtyrov, Pihko, & Pulvermüller, 2005;
Shtyrov, Pulvermüller, Näätänen, & Ilmoniemi, 2003).
As mentioned, syntactic DCNAs unify word-related circuits that bind information
about word form and meaning (Section 2), and therefore, there is direct interaction between
syntactic binding and the dynamics of semantic circuits. This may provide a neurobiological
perspectives on semantic compositionality, as the stronger or weaker links from the syntactic
binding circuit to specific lexical units have a direct impact on semantic activation. This can
be illustrated using compound words. For the compound noun swine flue, the DCNA
connection would activate flue more strongly than swine, so that the full set of semantic
features of the lexicosemantic representation of the main part of the compound (or head) is
kept activate upon initial excitation. The (modifier) noun swine, whose circuit is connected to
the DCNA through weaker links, would receive less priming, thus modeling the contextspecific attribute character of this item. When recognizing flue swine, however, the opposite
dynamics account for the understanding of “swine with flue as an attribute”. In languages like
French, where modifiers tend to follow heads, the links of the DCNAs established by learning
would impose the reverse activation hierarchy. In case the meaning of compounds and larger
structures is not easily derived from the semantics of their parts, semantic circuits can be
linked to sequence detectors and lexical representations of strings and constructions (Section
2). This may be relevant for implementing the meaning of complex forms that are
semantically opaque and cannot be derived compositionally (cash cow). It is needless to say
that the various semantic relationships that can hold between the different parts of a
pushdown mechanism of section 5 is functional.
28
BRAIN EMBODIMENT OF SYNTAX
compound expression may require the involvement of additional semantic circuits and
schemas (Feldman, 2006; Lakoff, 1987). The mechanisms outlined here may help to
neurobiologically underpin the tight link between syntax and semantics postulated in the
context of Cognitive and Construction Grammar.
9. Strong auto-associative links as critical step in language evolution?
The present approach assumes a division of labor between DCNAs and neuronal
pushdown mechanisms (Table 3). Subtypes of syntactic mechanisms, supporting more local
vs. non-local processing, may relate to different brain areas (Opitz & Friederici, 2007),
although not all studies support such a spatial dissociation (Indefrey et al., 2001). Independent
of any difference in cortical localization, the two proposed mechanisms supported by DCNAs
and pushdown store are fundamentally different at the functional level. However, in spite of
their functional differences, they may rely on similar brain-structural requirements. Critical
for the development of DCNAs is a high degree of auto-associative connectivity in a
“grammar area”, between sequence specific units. A high degree of auto-associative
connectivity may also be critical for the constant ignition and decline of activation in different
memory circuits, which is essential for the neuronal pushdown store (Braitenberg &
Pulvermüller, 1992). If connections within circuits are extremely strong, activity decrease
may become the consequence of habituation phenomena shared by large quantities of
excitatory neurons and thus also shared by different circuits. It may therefore be that rich
auto-associative connectivity between the main language areas in left-perisylvian cortex is a
requirement for the human capacity to process syntax and grammar. Apart from local
neuroanatomical specificity in this area (e.g., Harasty, Seldon, Chan, Halliday, & Harding,
2003; Hutsler & Galuske, 2003; e.g., Jacobs et al., 1993), the surprisingly strongly developed
29
BRAIN EMBODIMENT OF SYNTAX
connections between left-superiortemporal and -inferiorfrontal areas in humans (but not other
primates) may be of relevance here (Catani, Jones, & Ffytche, 2005; Rilling et al., 2008; Saur
et al., 2008). These strong links may enable the human cortex to build especially strongly
connected syntactic circuits that form the basis of a pushdown store, as they may allow the
binding of string-specific sequence detectors into generalizing DCNA, thereby providing a
basic rule mechanism. Models and data therefore seem to converge on the conclusion that a
high degree of auto-associative connectivity in left-perisylvian language cortex might have
been the critical evolutionary step towards human language.
Insert Table 3 about here
10. Summary and Conclusion
This paper reviewed some abstract properties of neuronal networks in general and the
human brain in particular. First, the so-called rule debate was mentioned and new evidence
from neurophysiological experiments for the existence of abstract syntactic rule
representations in the human brain was discussed. The claim that rules exist as brain
mechanisms calls for concrete mechanistic and brain-inspired models of syntactic and
grammatical processing. A brain model of recursion and embedding was reviewed and related
to abstract structural approaches to grammar. The interim conclusions were that abstract
automata and brains provide equivalent pushdown mechanisms for recursive processes and
that progress can be achieved in tailoring the limitations of these devices to those of real
brains. Sequence detectors and sequence detector aggregates, DCNAs, were discussed as a
brain basis for syntactic linkage of morphemes and larger constituents and the interplay
between circuits possibly built-in into specific structures of the human brain and associative
learning was put in the context of neuronal rule formation. The two proposed brain-internal
30
BRAIN EMBODIMENT OF SYNTAX
grammar mechanisms were related to each other (Table 3) and it was hypothesized that the
emergence of strong auto-associative connectivity in the perisylvian language cortex was the
critical step in human language evolution. Throughout, relationships between grammar
theories and neuronal approaches were mentioned.
31
BRAIN EMBODIMENT OF SYNTAX
References
Abeles, M., Bergman, H., Margalit, E., & Vaadia, E. (1993). Spatiotemporal firing patterns in the frontal cortex
of behaving monkeys. Journal of Neurophysiology, 70, 1629-1638.
Aziz-Zadeh, L., Wilson, S. M., Rizzolatti, G., & Iacoboni, M. (2006). Congruent embodied representations for
visually presented actions and linguistic phrases describing actions. Curr Biol, 16(18), 1818-1823.
Bach, E., Brown, C., & Marslen-Wilson, W. (1986). Crossed and nested dependencies in German and Dutch: a
psycholinguistic study. Language and Cognitive Processes, 1, 249-262.
Bak, T. H., O'Donovan, D. G., Xuereb, J. H., Boniface, S., & Hodges, J. R. (2001). Selective impairment of verb
processing associated with pathological changes in Brodmann areas 44 and 45 in the Motor Neurone
Disease-Dementia-Aphasia syndrome. Brain, 124, 103-120.
Barlow, H., & Levick, W. R. (1965). The mechanism of directionally selective units in rabbit's retina. Journal of
Physiology, 178, 477-504.
Barsalou, L. W. (1999). Perceptual symbol systems. Behav Brain Sci, 22(4), 577-609; discussion 610-560.
Barsalou, L. W. (2008). Grounded cognition. Annu Rev Psychol, 59, 617-645.
Bastiaansen, M., & Hagoort, P. (2006). Oscillatory neuronal dynamics during language comprehension. Prog
Brain Res, 159, 179-196.
Beiser, D. G., & Houk, J. C. (1998). Model of cortical-basal ganglionic processing: encoding the serial order of
sensory events. J Neurophysiol, 79(6), 3168-3188.
Bever, T. G., Carroll, J. M., & Hurtig, M. (1976). Analogy or Ungrammatical sequences that are utterable and
comprehensible are the origin of new grammars in language acquisition and linguistic evolution. In T.
G. Bever, J. J. Katz & D. T. Langendoen (Eds.), An integrated theory of linguistic ability. New York:
T.Y. Crowell Press.
Bienenstock, E. (1996). On the dimensionality of cortical graphs. Journal of Physiology (Paris), 90, 251-256.
Bornkessel, I., Zysset, S., Friederici, A. D., von Cramon, D. Y., & Schlesewsky, M. (2005). Who did what to
whom? The neural basis of argument hierarchies during language comprehension. Neuroimage, 26(1),
221-233.
Borst, A. (2007). Correlation versus gradient type motion detectors: the pros and cons. Philos Trans R Soc Lond
B Biol Sci, 362(1479), 369-374.
Botvinick, M. M., & Plaut, D. C. (2006). Short-term memory for serial order: a recurrent neural network model.
Psychol Rev, 113(2), 201-233.
Boulenger, V., Hauk, O., & Pulvermüller, F. (2009). Grasping ideas with the motor system: Semantic
somatotopy in idiom comprehension. Cereb Cortex, in press.
Braitenberg, V. (1996). Il gusto della lingua. Meccanismi cerebrali e strutture grammaticali. Merano: Alpha &
Beta.
Braitenberg, V., & Pulvermüller, F. (1992). Entwurf einer neurologischen Theorie der Sprache.
Naturwissenschaften, 79, 103-117.
Braitenberg, V., & Schüz, A. (1998). Cortex: statistics and geometry of neuronal connectivity (2 ed.). Berlin:
Springer.
Bresnan, J. (2001). Lexical Functional Syntax. Oxford: Blackwell Publishers Ltd.
Briscoe, E., & Carroll, J. (1997). Automatic extraction of subcategorization from corpora. Proceedings of the 5th
ACL Conference on Applied Natural Language Processing, Washington, DC, 5, 1-9.
Buccino, G., Riggio, L., Melli, G., Binkofski, F., Gallese, V., & Rizzolatti, G. (2005). Listening to action-related
sentences modulates the activity of the motor system: a combined TMS and behavioral study. Brain Res
Cogn Brain Res, 24(3), 355-363.
Canolty, R. T., Soltani, M., Dalal, S. S., Edwards, E., Dronkers, N. F., Nagarajan, S. S., Kirsch, H. E., Barbaro,
N. M., & Knight, R. T. (2007). Spatiotemporal dynamics of word processing in the human brain. Front.
Neurosci., doi:10.3389/neuro.01.1.1.014.2007.
Caplan, D. (2006). Aphasic deficits in syntactic processing. Cortex, 42(6), 797-804.
Caplan, D., Vijayan, S., Kuperberg, G., West, C., Waters, G., Greve, D., & Dale, A. M. (2002). Vascular
responses to syntactic processing: event-related fMRI study of relative clauses. Hum Brain Mapp, 15(1),
26-38.
Catani, M., Jones, D. K., & Ffytche, D. H. (2005). Perisylvian language networks of the human brain. Annals of
Neurology, 57(1), 8-16.
Chang, F., Dell, G. S., & Bock, K. (2006). Becoming syntactic. Psychol Rev, 113(2), 234-272.
32
BRAIN EMBODIMENT OF SYNTAX
Chao, L. L., Haxby, J. V., & Martin, A. (1999). Attribute-based neural substrates in temporal cortex for
perceiving and knowing about objects. Nature Neuroscience, 2(10), 913-919.
Chater, N., & Manning, C. D. (2006). Probabilistic models of language processing and acquisition. Trends in
Cognitive Sciences, 10(7), 335-344.
Chomsky, N. (1963). Formal properties of grammars. In R. D. Luce, R. R. Bush & E. Galanter (Eds.), Handbook
of mathematical psychology. Volume 2 (pp. 323-418). New York, London: Wiley.
Chomsky, N. (1980). Rules and representations. New York: Columbia University Press.
Chomsky, N. (1981). Lectures on government and binding. Dordrecht: Foris.
Chomsky, N. (2000). New horizons in the study of language and mind. Cambridge: Cambridge University Press.
Christiansen, M. H., & Chater, N. (1999). Toward a connectionist model of recursion in human linguistic
performance. Cognitive Science, 23(2), 157-205.
D’Ausilio, A., Pulvermüller, F., Salmas, P., Bufalari, I., Begliomini, C., & Fadiga, L. (2009). The Motor
Somatotopy of Speech Perception. Current Biology, in press.
Damasio, H., Grabowski, T. J., Tranel, D., Hichwa, R. D., & Damasio, A. R. (1996). A neural basis for lexical
retrieval. Nature, 380, 499-505.
Diesch, E., Eulitz, C., Hampson, S., & Ross, B. (1996). The neurotopography of vowels as mirrored by evoked
magnetic field measurements. Brain Lang, 53(2), 143-168.
Dominey, P. F. (1997). An anatomically structured sensory-motor sequence learning system displays some
general linguistic capacities. Brain Lang, 59(1), 50-75.
Dominey, P. F., Hoen, M., Blanc, J. M., & Lelekov-Boissard, T. (2003). Neurological basis of language and
sequential cognition: evidence from simulation, aphasia, and ERP studies. Brain Lang, 86(2), 207-225.
Donchin, E. (1981). Surprise! ... Surprise? Psychophysiology, 18, 493-513.
Egelhaaf, M., Borst, A., & Reichardt, W. (1989). Computational structure of a biological motion-detection
system as revealed by local detector analysis in the fly's nervous system. Journal of the Optical Society
of America (A), 6, 1070-1087.
Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179-211.
Elman, J. L., Bates, L., Johnson, M., Karmiloff-Smith, A., Parisi, D., & Plunkett, K. (1996). Rethinking
innateness. A connectionist perspective on development. Cambridge, MA: MIT Press.
Fadiga, L., Craighero, L., Buccino, G., & Rizzolatti, G. (2002). Speech listening specifically modulates the
excitability of tongue muscles: a TMS study. European Journal of Neuroscience, 15(2), 399-402.
Fadiga, L., Fogassi, L., Pavesi, G., & Rizzolatti, G. (1995). Motor facilitation during action observation: a
magnetic stimulation study. J Neurophysiol, 73(6), 2608-2611.
Fazio, P., A., C., Craighero, L., D’Ausilio, A., Roy, A., Pozzo, T., Calzolari, F., Granieri, E., & Fadiga, L.
(2009). Encoding of human action in Broca's area. Brain, in press.
Feldman, J. (2006). From molecule to metaphor: A neural theory of language. Cambridge, MA: MIT Press.
Feldman, J., & Narayanan, S. (2004). Embodied meaning in a neural theory of language. Brain and Language,
89(2), 385-392.
Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences,
6(2), 78-84.
Friederici, A. D., Pfeifer, E., & Hahne, A. (1993). Event-related brain potentials during natural speech
processing: effects of semantic, morphological and syntactic violations. Cognitive Brain Research, 1(3),
183-192.
Friederici, A. D., Wang, Y., Herrmann, C. S., Maess, B., & Oertel, U. (2000). Localization of early syntactic
processes in frontal and temporal cortical areas: a magnetoencephalographic study. Hum Brain Mapp,
11(1), 1-11.
Fry, D. B. (1966). The development of the phonological system in the normal and deaf child. In F. Smith & G.
A. Miller (Eds.), The genesis of language (pp. 187-206). Cambridge, MA: MIT Press.
Fuster, J. M. (1997). Network memory. Trends in Neurosciences, 20(10), 451-459.
Fuster, J. M. (2003). Cortex and mind: Unifying cognition. Oxford: Oxford University Press.
Fuster, J. M., & Jervey, J. P. (1981). Inferotemporal neurons distinguish and retain behaviorally relevant features
of visual stimuli. Science, 212, 952-955.
Galantucci, B., Fowler, C. A., & Turvey, M. T. (2006). The motor theory of speech perception reviewed.
Psychon Bull Rev, 13(3), 361-377.
Gazdar, G., Klein, E., Pullum, G., & Sag, I. (1985). Generalized phrase structure grammar. Cambridge, MA:
Harvard University Press.
Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3),
558-565.
33
BRAIN EMBODIMENT OF SYNTAX
Glenberg, A. M., Sato, M., & Cattaneo, L. (2008). Use-induced motor plasticity affects the processing of abstract
and concrete language. Curr Biol, 18(7), R290-291.
Goldberg, A. E. (2006). Constructions at work: The nature of generalisation in language. Oxford: Oxford
University Press.
González, J., Barros-Loscertales, A., Pulvermüller, F., Meseguer, V., Sanjuán, A., Belloch, V., & Ávila, C.
(2006). Reading cinnamon activates olfactory brain regions. Neuroimage, 32(2), 906-912.
Grewe, T., Bornkessel, I., Zysset, S., Wiese, R., von Cramon, D. Y., & Schlesewsky, M. (2006). Linguistic
prominence and Broca's area: the influence of animacy as a linearization principle. Neuroimage, 32(3),
1395-1402.
Grodzinsky, Y., & Friederici, A. D. (2006). Neuroimaging of syntax and syntactic processing. Curr Opin
Neurobiol, 16(2), 240-246.
Haegeman, L. (1991). Introduction to government and binding theory. Cambridge, MA: Basil Blackwell.
Hagoort, P. (2005). On Broca, brain, and binding: a new framework. Trends Cogn Sci, 9(9), 416-423.
Hagoort, P. (2008). The fractionation of spoken language understanding by measuring electrical and magnetic
brain signals. Philos Trans R Soc Lond B Biol Sci, 363(1493), 1055-1069.
Hagoort, P., Brown, C., & Groothusen, J. (1993). The syntactic positive shift (SPS) as an ERP-measure of
syntactic processing. Language and Cognitive Processes, 8(4), 439-483.
Hanson, S. J., & Negishi, M. (2002). On the emergence of rules in neural networks. Neural Comput, 14(9), 22452268.
Harasty, J., Seldon, H. L., Chan, P., Halliday, G., & Harding, A. (2003). The left human speech-processing
cortex is thinner but longer than the right. Laterality, 8(3), 247-260.
Hasting, A. S., Kotz, S. A., & Friederici, A. D. (2007). Setting the stage for automatic syntax processing: the
mismatch negativity as an indicator of syntactic priming. J Cogn Neurosci, 19(3), 386-400.
Hauk, O., Johnsrude, I., & Pulvermüller, F. (2004). Somatotopic representation of action words in the motor and
premotor cortex. Neuron, 41, 301-307.
Hauk, O., Shtyrov, Y., & Pulvermüller, F. (2008). The time course of action and action-word comprehension in
the human brain as revealed by neurophysiology. J Physiol Paris, 102(1-3), 50-58.
Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of language: what is it, who has it, and how did
it evolve? Science, 298(5598), 1569-1579.
Hawkins, J. (1994). A performance theory of order and constituency (Vol. 73). Cambridge: Cambridge
University Press.
Hayon, G., Abeles, M., & Lehmann, D. (2005). A model for representing the dynamics of a system of synfire
chains. J Comput Neurosci, 18(1), 41-53.
Hickok, G., & Poeppel, D. (2000). Towards a functional neuroanatomy of speech perception. Trends in
Cognitive Sciences, 4(4), 131-138.
Honkela, T., Pulkki, V., & Kohonen, T. (1995). Contextual relations of words in grimm tales analyzed by selforganizing map. proceedings of international conference on artificial neural networks. In F. FogelmanSoulie' & P. Gallinari (Eds.), Proceedings of the International Conference on Neural Networks
(ICANN) 1995 (pp. 3-7). Paris: EC2 et Cie.
Hopcroft, J. E., Motwani, R., & Ullman, J. D. (2001). Introduction to automata theory, languages, and
computation. New York: ACM.
Hubel, D. (1995). Eye, brain, and vision (2 ed.). New York: Scientific American Library.
Hutsler, J., & Galuske, R. A. (2003). Hemispheric asymmetries in cerebral cortical networks. Trends Neurosci,
26(8), 429-435.
Indefrey, P., Brown, C. M., Hellwig, F., Amunts, K., Herzog, H., Seitz, R. J., & Hagoort, P. (2001). A neural
correlate of syntactic encoding during speech production. Proceedings of the National Academy of
Sciences,USA, 98(10), 5933-5936.
Jackendoff, R. (2002). Foundations of language: Brain, meaning, grammar, evolution. Oxford, UK: Oxford
University Press.
Jacobs, B., Batal, H. A., Lynch, B., Ojemann, G., Ojemann, L. M., & Scheibel, A. B. (1993). Quantitative
dendritic and spine analyses of speech cortices: a case study. Brain and Language, 44, 239-253.
Joshi, A. K., & Schabes, Y. (1997). Tree-adjoining gammars. In A. Salomon & E. Rozenberg (Eds.), Handbook
of formal languages and automata. Berlin: Springer-Verlag.
Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F., & Thulborn, K. R. (1996). Brain activation modulated
by sentence comprehension. Science, 274(5284), 114-116.
Karlsson, F. (2007). Constraints on multiple centre embedding of clauses. Journal of Linguistics (2007), 43 :
365-392 Cambridge University Press 43, 365-392.
34
BRAIN EMBODIMENT OF SYNTAX
Keller, F. (2001). Gradience in grammar: experimental and computational aspects of degrees of grammaticality.
University of Edinburgh, Edinburgh.
Kempen, G., & Harbusch, K. (2003). An artificial opposition between grammaticality and frequency: comment
on Bornkessel, Schlesewsky, and Friederici (2002). Cognition, 90(2), 205-210; discussion 211-203,
215.
Kinno, R., Kawamura, M., Shioda, S., & Sakai, K. L. (2008). Neural correlates of noncanonical syntactic
processing revealed by a picture-sentence matching task. Hum Brain Mapp, 29(9), 1015-1027.
Kleene, S. C. (1956). Representation of events in nerve nets and finite automata. In C. E. Shannon & J.
McCarthy (Eds.), Automata studies (pp. 3-41). Princeton, NJ: Princeton University Press.
Knight, R. T. (2007). Neuroscience. Neural networks debunk phrenology. Science, 316(5831), 1578-1579.
Knoblauch, A., & Pulvermüller, F. (2005). Sequence detector networks and associative learning of grammatical
categories. In S. Wermter, G. Palm & M. Elshaw (Eds.), Biomimetic neural learning for intelligent
robots (pp. 31-53). Berlin: Springer.
Kuperberg, G. R. (2007). Neural mechanisms of language comprehension: challenges to syntax. Brain Res, 1146,
23-49.
Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic
association. Nature, 307, 161-163.
Lakoff, G. (1987). Women, fire, and dangerous things. What categories reveal about the mind. Chicago:
University of Chicago Press.
Langacker, R. (1991). Foundations of Cognitive Grammar. 2 vols. Stanford: Stanford University Press.
Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1-36.
Lichtheim, L. (1885). On aphasia. Brain, 7, 433-484.
Lin, D. (1998). Automatic retrieval and clustering of similar words. Proceedings of the17th International
Conference on Computational Linguistics and 36th Annual Meeting of the Association for
Computational Linguistics, Montreal, 17, 768-773.
Maess, B., Koelsch, S., Gunter, T. C., & Friederici, A. D. (2001). Musical syntax is processed in Broca's area: an
MEG study. Nat Neurosci, 4(5), 540-545.
Mahon, B. Z., & Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal
for grounding conceptual content. J Physiol Paris, 102(1-3), 59-70.
Marcus, G. F. (2001). The algebraic mind: integrating connectionism and cognitive science. Cambridge, MA:
MIT Press.
Marcus, G. F. (2008). Kluge: The haphazard construction of the human mind. Boston, MA: Houghton Mifflin
Co.
Marslen-Wilson, W. D., & Tyler, L. K. (2007). Morphology, language and the brain: the decompositional
substrate for language comprehension. Philos Trans R Soc Lond B Biol Sci, 362(1481), 823-836.
Martin, A. (2007). The representation of object concepts in the brain. Annu Rev Psychol, 58, 25-45.
McClelland, J. L., & Patterson, K. (2002). Rules or connections in past-tense inflections: what does the evidence
rule out? Trends Cogn Sci, 6(11), 465-472.
McCulloch, W. S., & Pitts, W. H. (1943). A logical calculus of ideas immanent in nervous activity. Bulletin of
Mathematical Biophysics, 5, 115-133.
Menning, H., Zwitserlood, P., Schoning, S., Hihn, H., Bolte, J., Dobel, C., Mathiak, K., & Lutkenhoner, B.
(2005). Pre-attentive detection of syntactic and semantic errors. Neuroreport, 16(1), 77-80.
Näätänen, R., Gaillard, A. W., & Mäntysalo, S. (1978). Early selective-attention effect on evoked potential
reinterpreted. Acta Psychologica, 42, 313-329.
Näätänen, R., Tervaniemi, M., Sussman, E., Paavilainen, P., & Winkler, I. (2001). 'Primitive intelligence' in the
auditory cortex. Trends in Neurosciences, 24(5), 283-288.
Neininger, B., & Pulvermüller, F. (2003). Word-category specific deficits after lesions in the right hemisphere.
Neuropsychologia, 41(1), 53-70.
Neville, H., Nicol, J. L., Barss, A., Forster, K. I., & Garrett, M. F. (1991). Syntactically based sentence
processing classes: evidence from event-related brain potentials. Journal of Cognitive Neuroscience, 3,
151-165.
O'Reilly, R. C. (1998). Six principles for biologically based computational models of cortical cognition. Trends
Cogn Sci, 2(11), 455-562.
Obleser, J., Boecker, H., Drzezga, A., Haslinger, B., Hennenlotter, A., Roettinger, M., Eulitz, C., & Rauschecker,
J. P. (2006). Vowel sound extraction in anterior superior temporal cortex. Hum Brain Mapp, 27(7), 562571.
35
BRAIN EMBODIMENT OF SYNTAX
Opitz, B., & Friederici, A. D. (2007). Neural basis of processing sequential and hierarchical syntactic structures.
Hum Brain Mapp, 28(7), 585-592.
Osterhout, L., & Hagoort, P. (1999). A superficial resemblance does not necessarily mean you are part of the
family: Counterarguments to Coulson, King and Kutas (1998) in the P600/SPS-P300 debate. Language
and Cognitive Processes, 14, 1-14.
Osterhout, L., & Holcomb, P. J. (1992). Event-related brain potentials elicited by syntactic anomaly. Journal of
Memory and Language, 31, 785-806.
Palm, G. (1993). On the internal structure of cell assemblies. In A. Aertsen (Ed.), Brain theory: spatio-temporal
aspects of brain function (pp. 261-270). Amsterdam: Elsevier.
Paus, T., Perry, D. W., Zatorre, R. J., Worsley, K. J., & Evans, A. C. (1996). Modulation of cerebral blood flow
in the human auditory cortex during speech: role of motor-to-sensory discharges. European Journal of
Neuroscience, 8(11), 2236-2246.
Petri, C. A. (1970). Kommunikation mit Automaten. Dissertation: Universität Bonn.
Pinker, S. (1994). The language instinct. How the mind creates language. New York: Harper Collins Publishers.
Plenz, D., & Thiagarajan, T. C. (2007). The organizing principles of neuronal avalanches: cell assemblies in the
cortex? Trends Neurosci, 30(3), 101-110.
Pulvermüller, F. (1993). On connecting syntax and the brain. In A. Aertsen (Ed.), Brain theory - spatio-temporal
aspects of brain function (pp. 131-145). New York: Elsevier.
Pulvermüller, F. (1999). Words in the brain's language. Behavioral and Brain Sciences, 22, 253-336.
Pulvermüller, F. (2003a). The neuroscience of language. Cambridge: Cambridge University Press.
Pulvermüller, F. (2003b). Sequence detectors as a basis of grammar in the brain. Theory in Biosciences, 122, 87103.
Pulvermüller, F. (2005). Brain mechanisms linking language and action. Nature Reviews Neuroscience, 6(7),
576-582.
Pulvermüller, F. (2007). Word processing in the brain as revealed by neurophysiological imaging using EEG and
MEG. In G. Gaskell (Ed.), Handbook of Psycholinguistics (pp. 119-140). Oxford: Oxford University
Press.
Pulvermüller, F., & Assadollahi, R. (2007). Grammar or serial order?: Discrete combinatorial brain mechanisms
reflected by the syntactic Mismatch Negativity. Journal of Cognitive Neuroscience, 19(6), 971-980.
Pulvermüller, F., Birbaumer, N., Lutzenberger, W., & Mohr, B. (1997). High-frequency brain activity: its
possible role in attention, perception and language processing. Progress in Neurobiology, 52(5), 427445.
Pulvermüller, F., Cooper-Pye, E., Dine, C., Hauk, O., Nestor, P., & Patterson, K. (2009). The word processing
deficit in Semantic Dementia: All categories are equal but some categories are more equal than others.
Journal of Cognitive Neuroscience in press.
Pulvermüller, F., Härle, M., & Hummel, F. (2000). Neurophysiological distinction of verb categories.
Neuroreport, 11(12), 2789-2793.
Pulvermüller, F., & Hauk, O. (2006). Category-specific processing of color and form words in left frontotemporal cortex. Cerebral Cortex, 16(8), 1193-1201.
Pulvermüller, F., Huss, M., Kherif, F., Moscoso del Prado Martin, F., Hauk, O., & Shtyrov, Y. (2006). Motor
cortex maps articulatory features of speech sounds. Proceedings of the National Academy of Sciences,
USA, 103(20), 7865-7870.
Pulvermüller, F., & Knoblauch, A. (2009). Discrete combinatorial circuits emerging in neural networks: A
mechanism for rules of grammar in the human brain? Neural Netw, 22(2), 161-172.
Pulvermüller, F., & Mohr, B. (1996). The concept of transcortical cell assemblies: a key to the understanding of
cortical lateralization and interhemispheric interaction. Neuroscience and Biobehavioral Reviews, 20,
557-566.
Pulvermüller, F., & Shtyrov, Y. (2003). Automatic processing of grammar in the human brain as revealed by the
mismatch negativity. Neuroimage, 20, 159-172.
Pulvermüller, F., & Shtyrov, Y. (2006). Language outside the focus of attention: the mismatch negativity as a
tool for studying higher cognitive processes. Progress in Neurobiology, 79(1), 49-71.
Pulvermüller, F., Shtyrov, Y., Hasting, A., & Carlyon, R. P. (2008). Syntax as a reflex: Neurophysiological
evidence for early automaticity of grammatical processing. Brain and Language, 104(1), 244-253.
Pulvermüller, F., Shtyrov, Y., & Hauk, O. (2009). Understanding in an instant: Neurophysiological evidence for
mechanistic language circuits in the brain. Brain and Language, in press.
Pulvermüller, F., Shtyrov, Y., & Ilmoniemi, R. J. (2003). Spatio-temporal patterns of neural language
processing: an MEG study using Minimum-Norm Current Estimates. Neuroimage, 20, 1020-1025.
36
BRAIN EMBODIMENT OF SYNTAX
Pulvermüller, F., Shtyrov, Y., & Ilmoniemi, R. J. (2005). Brain signatures of meaning access in action word
recognition. Journal of Cognitive Neuroscience, 17(6), 884-892.
Reichardt, W., & Varju, D. (1959). Übertragungseigenschaften im Auswertesystem für das Bewegungssehen.
Zeitschrift für Naturforschung, 14b, 674-689.
Rilling, J. K., Glasser, M. F., Preuss, T. M., Ma, X., Zhao, T., Hu, X., & Behrens, T. E. (2008). The evolution of
the arcuate fasciculus revealed with comparative DTI. Nat Neurosci, 11(4), 426-428.
Rumelhart, D. E., & McClelland, J. L. (1987). Learning the past tense of English verbs: implicit rules or parallel
distributed processing. In B. MacWhinney (Ed.), Mechanisms of language acquisition. Hillsdale, NJ:
Erlbaum.
Saur, D., Kreher, B. W., Schnell, S., Kummerer, D., Kellmeyer, P., Vry, M. S., Umarova, R., Musso, M.,
Glauche, V., Abel, S., Huber, W., Rijntjes, M., Hennig, J., & Weiller, C. (2008). Ventral and dorsal
pathways for language. Proc Natl Acad Sci U S A, 105(46), 18035-18040.
Schnelle, H. (1996). Die Natur der Sprache. Die Dynamik der Prozesse des Sprechens und Verstehens (2 ed.).
Berlin, New York: Walter de Gruyter.
Shastri, L., & Ajjanagadde, V. (1993). From simple associations to systematic reasoning: a connectionist
representation of rules, variables and dynamic bindings using temporal synchrony. Behavioral and
Brain Sciences, 16, 417-494.
Shtyrov, Y., Pihko, E., & Pulvermüller, F. (2005). Determinants of dominance: Is language laterality explained
by physical or linguistic features of speech? Neuroimage, 27(1), 37-47.
Shtyrov, Y., Pulvermüller, F., Näätänen, R., & Ilmoniemi, R. J. (2003). Grammar processing outside the focus of
attention: an MEG study. Journal of Cognitive Neuroscience, 15(8), 1195-1206.
Sigala, N., Kusunoki, M., Nimmo-Smith, I., Gaffan, D., & Duncan, J. (2008). Hierarchical coding for sequential
task events in the monkey prefrontal cortex. Proc Natl Acad Sci U S A, 105(33), 11969-11974.
Simmons, W. K., Ramjee, V., Beauchamp, M. S., McRae, K., Martin, A., & Barsalou, L. W. (2007). A common
neural substrate for perceiving and knowing about color. Neuropsychologia, 45(12), 2802-2810.
Smolensky, P. (1999). Grammar-based connectionist approaches to language. Cognitive Science, 23(4), 589-613.
Sougné, J. (1998). Connectionism and the problem of multiple instantiation. Trends in Cognitive Sciences, 2,
183-189.
Sougné, J. (2001). Binding and multiple instantiation in a distributed network of spiking nodes Connection
Science, 13(2), 99-126.
Tettamanti, M., Buccino, G., Saccuman, M. C., Gallese, V., Danna, M., Scifo, P., Fazio, F., Rizzolatti, G.,
Cappa, S. F., & Perani, D. (2005). Listening to action-related sentences activates fronto-parietal motor
circuits. Journal of Cognitive Neuroscience, 17(2), 273-281.
Townsend, D. J., & Bever, T. G. (2001). Sentence comprehension. Cambridge, MA: MIT Press.
Tranel, D., Damasio, H., & Damasio, A. R. (1997). A neural substrate for the retrieval of conceptual knowledge.
Neuropsychologia, 35, 1319-1327.
Ullman, M. T. (2001). A neurocognitive perspective on language: the declarative/procedural model. Nat Rev
Neurosci, 2(10), 717-726.
Uppenkamp, S., Johnsrude, I. S., Norris, D., Marslen-Wilson, W., & Patterson, R. D. (2006). Locating the initial
stages of speech-sound processing in human temporal cortex. Neuroimage, 31(3), 1284-1296.
van der Velde, F., & de Kamps, M. (2006). Neural blackboard architectures of combinatorial structures in
cognition. Behav Brain Sci, 29(1), 37-70; discussion 70-108.
Varju, D., & Reichardt, W. (1967). Übertragungseigenschaften im Auswertesystem für das Bewegungssehen II.
Zeitschrift für Naturforschung, 22b, 1343-1351.
Voegtlin, T., & Dominey, P. F. (2005). Linear recursive distributed representations. Neural Netw, 18(7), 878895.
Warrington, E. K., & McCarthy, R. A. (1987). Categories of knowledge: further fractionations and an attempted
integration. Brain, 110, 1273-1296.
Weiss, S., Mueller, H. M., Schack, B., King, J. W., Kutas, M., & Rappelsberger, P. (2005). Increased neuronal
communication accompanying sentence comprehension. Int J Psychophysiol, 57(2), 129-141.
Wennekers, T., Garagnani, M., & Pulvermüller, F. (2006). Language models based on Hebbian cell assemblies. J
Physiol Paris, 100, 16-30.
Wennekers, T., & Palm, G. (1996). Controlling the speed of synfire chains (Vol. 1112/1996). Berlin: Springer.
Wernicke, C. (1874). Der aphasische Symptomencomplex. Eine psychologische Studie auf anatomischer Basis.
Breslau: Kohn und Weigert.
Wickens, J., Hyland, B., & Anson, G. (1994). Cortical cell assemblies: a possible mechanism for motor
programs. J Mot Behav, 26(2), 66-82.
37
BRAIN EMBODIMENT OF SYNTAX
Willshaw, D. J., Buneman, O. P., & Longuet-Higgins, H. C. (1969). Non-holographic associative memory.
Nature, 222(197), 960-962.
Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004). Listening to speech activates motor areas
involved in speech production. Nat Neurosci, 7(7), 701-702.
Zatorre, R. J., Evans, A. C., Meyer, E., & Gjedde, A. (1992). Lateralization of phonetic and pitch discremination
in speech processing. Science, 256, 846-849.
38
BRAIN EMBODIMENT OF SYNTAX
TABLE LEGENDS
Table 1: Example sentences and syntactic rewriting rules for illustrating recursion and
embedding. For explanation, see text. N–noun, NP–noun phrase, S–sentence, V–verb, VP–
verb phrase, Æ-is rewritten as, curly brackets index sets of lexical categories.
Table 2: The merging of sequence detectors into a neuronal aggregate (see Figure 4) leads to
priming between syntactic-semantic groups of lexical elements. The table shows the
physiological effect of such priming as revealed by a network simulation. The numbers give
the activation values of second string elements, verbs, (at the top) after activation of first
string elements, nouns (left). Note that, after activation of first string elements (nouns)
involved in substitutions in similar contexts (verbs), activation levels of context elements also
involved in the substitutions were enhanced to similar degrees (green areas). The neuronal
aggregates provide a generalized functional connection between subsets of nouns and verbs
also linking new pairs the network had never encountered together.
Table 3: Dual syntactic mechanism model: Different roles of syntactic discrete combinatorial
neuronal assemblies (DCNAs) and neuronal pushdown store in providing the mechanistic
brain basis for syntax and grammar.
39
BRAIN EMBODIMENT OF SYNTAX
FIGURE CAPTIONS
Figure 1: The syntactic Mismatch Negativity (MMN) occurred to rare ungrammatical strings
(black curve and bar), but not to similarly rare grammatical sentences (in red) or to frequently
occurring grammatical strings (in green and blue). Event-related magnetic MMN responses
are shown at the top and their statistics at the bottom. The syntactic MMN therefore reflected
grammaticality, but not string probability (Pulvermüller & Assadollahi, 2007).
Figure 2: Sequential activation of several memory circuits with graceful activity decline
(colored curves) yields storage of the activation order in the hierarchy of activity levels of the
assemblies. At a later time point (dashed line), the circuit sparked first will be at the lowest
activity level and the assembly activated last will still be at the highest level. This mechanism
can act as a neuronal implementation of a pushdown store (modified from Pulvermüller,
1993).
Figure 3: Syntactic links between constituents of a simple example sentence, (10). Diagram
(a) represents links implemented by syntactic binding circuits (DCNAs). Diagram (b) presents
a syntactic tree structure resulting from application of rewriting rules, plus syntactic
relationships not easily captured by trees (broken lines).
Figure 4: Combinatorial information immanent to noun-verb sequences and network result of
learning this information in an auto-associative memory including sequence detectors for
word pairs. The diagram on the left presents the matrix of co-occurrences and substitutions of
20 nouns and verbs obtained from the British National Corpus, BNC (in number of word pair
40
BRAIN EMBODIMENT OF SYNTAX
occurrences per 100 million words). The diagram on the right shows the network of lexical
circuits and sequence detectors and the connections strengthened by learning the string set
obtained from BNC. Grey circles in the periphery represent neural units corresponding to the
words in the diagram on the left. The central matrix shows sequence detectors corresponding
to word pair sequences. Filled black circles indicate sequence detectors whose respective
word pair was in the input, leading to strengthening of connections between sequence detector
and word representations (in red). Black lines show strengthened auto-associative links
between pairs of sequence detectors. At the top left and bottom right, two discrete
combinatorial neuronal assemblies, DCNAs, have formed. Depending on the threshold of
activation, these DCNAs either bind all nouns to verbs, or provide specific syntactic-semantic
linkage of action verbs and nouns related to living entities, and of flight-related verbs and
flying-object nouns (after Pulvermüller & Knoblauch, 2009).
41
BRAIN EMBODIMENT OF SYNTAX
Table 1
42
BRAIN EMBODIMENT OF SYNTAX
Table 2:
43
BRAIN EMBODIMENT OF SYNTAX
Table 3
44
BRAIN EMBODIMENT OF SYNTAX
Figure 1
45
BRAIN EMBODIMENT OF SYNTAX
Figure 2
46
BRAIN EMBODIMENT OF SYNTAX
Figure 3
47
BRAIN EMBODIMENT OF SYNTAX
Figure 4