Language and Imagination
Eric Reuland
Utrecht institute of Linguistics OTS
[email protected]
http://eric.reuland.nl/httpdocs/Welcome.html
Aim and structure of this talk
Claim: The evolutionary sources of language are intrinsically connected with
ZKDWRQHPD\EURDGO\FDOO³,PDJLQDWLRQ´
Aim: To provide a functional analysis of the requirements working memory
systems must meet in order to subserve GLIIHUHQWµPRGHV¶RILPDJLQDWLRQ
and to elucidate how developments in working memory systems contribute
to facilitating the emergence of language, by focusing on possible
discontinuities.
Structure:
‡ Initial discussion of imagination
‡ Tool making and demands on memory resources
‡ Art and demands on memory resources
‡ Language and its prerequisites
‡ Non-linguistic diagnostics for language (?)
‡ The gap between design features and what we know about the structure of 2
language.
Imagination: some reflections
The cave on the first picture and most of the subsequent
ones is Blombos Cave in South Africa. It is famous
because it was found to contain some of the oldest
traces of human imagination. Imagination in a most
modern sense: results of imagination freed from the
use in the direct struggle for life with the natural
environment.As an illustration, the next page shows
some of the products of such free imagination.
3
3URGXFWVRI³IUHH´LPDJLQDWLRQ
Beads from Blombos Cave, South
Africa, Middle Stone Age, 70 ka BP
Piece of ochre with
engravings (id.)
4
An early reflection
Reflecting on imagination is not new.
In 1967 the mathematician and humanist Jacob
Bronowski wrote an essay on imagination that
captured what is to my mind the core insight
that imagination rests on the manipulation of
mental images.
However, as we will see in the end, the notion
of an image will have to extended beyond what
what Bronowski had in mind.
5
What is imagination?
Jacob Bronowski ( The Reach of Im agination, 1967):
³Imagination is a specifically human gift. To imagine is the characteristic
act, not of the SRHW¶V mind, or the SDLQWHU¶V or the VFLHQWLVW¶V, but of the
mind RIPDQ´
More concretely:
‡ ³7RLPDJLQH PHDQVWRPDNHLPDJHVDQGPRYHWKHPDERXWLQVLGHRQH¶V
head in new arrangements´
More prosaically:
‡ Imagination rests on the manipulation of mental representations.
Here is where the first reference to a crucial human trait comes in:
‡ µ0HQWDOLPDJHV¶may result from operations on µPHQWDOLPDJHV¶
‡ It is enough to allow operations on mental images to apply to mental
images that resulted from such operations to have recursion in the
relevant sense.
.6
What role does imagination play?
S. Mithen ( The Evolution of Imagination: An Archaeological
Perspective, 2001):
‡ 0RVWEDVLFIRUP³6LPSO\HQYLVLRQLQJDOWHUQDWLYHFRXUVHVRIDFWLRQZKLFK
IRRGWRFKRRVHLQWKHVXSHUPDUNHWZKLFKSHUVRQWRDVNRQDGDWH´
‡ ³7KHDELOLW\WRWDNHWKHRUGLQDU\«DQGXVLQJLWWRH[SORUHH[SUHVVRU
GLVFRYHUVRPHWKLQJTXLWHH[WUDRUGLQDU\«´
‡ ³,PDJLQDWLRQDERXWZRUOGVWKDWZHFDQRQO\LQKDELWLQRXUPLQGV² worlds
in which the laws of nature are regularly broken or simply do not exist.
Examples are the worlds created in mythology and science fiction, the
ZRUOGVRIVXSHUQDWXUDOEHLQJVIURP+LHURQ\PXV%RVFKWR6DOYDGRU'DOL´
0LWKHQ¶V question: How could such an ability to think about the unreal have
evolved?
Such questions come up in various forms in evolutionary discussions. But the
moment we realize that imagination results from operations on mental
representation it follows that no deep problem is involved.
7
A precondition for imagination
If we have manipulation of images we need a space in which
these manipulations can be carried out. Without such a space
imagination is impossible. And limits on this space will limit
imagination, and the use we can make of its results. This
immediately connects the study of imagination to the study of
mental work spaces in our brain: working memory (WM).
Baddeley, one of the pioneers in this field, provides a state-ofthe art summary of the theory of working memory in his 2007
book. Coolidge and Wynn (2005) and subsequent work
develop an interesting perspective on human cognitive
evolution in these terms Æ my interest in the role of WM-use
8
and access as preconditions for language.
Towards an understanding
Crucial for the current perspective on imagination is that mental
representations should be analyzable into component parts.
As the basic tool for our discussion here and elsewhere, I will
take a functional analysis.
Given that a system can perform certain operations on
representations, what are the minimal properties it must have?
Such a functional analysis abstracts away from particular kind
of processes taking place in the brain.
Of course, in the end such links must be established, but this
leads us beyond the scope of the present essay.
9
Summary of what it takes
Functional analysis:
A memory system for mental representations needs:
‡ A storage system;
‡ The capacity to maintain representations through
time;
‡ The capacity to access and maintain alternative
representations simultanuously;
‡ An articulation/analysis of representations allowing
manipulation/operations on sub-parts.
10
From imagination to realization
It is not only the case that we can imagine things, but
we also try to realize and use what we imagine as the
examples on the next slide show.
All these examples are based on the ability to form
complex representations or ± when they are realized ±
corresponding objects, by combining more
elementary components (shafts, heads, colors, lines,
words, melodies, etc.)
But, as we will see further on, there are also limitations
on our ability to use what we imagine.
11
From imagination to realization
We imagine
‡ a tool and make it
‡ a painting and paint it
‡ a poem and write it
‡ a song and sing it
««
‡ rules and impose them
These all show the conditions on imagination as we have it:
‡ Allowing the formation of complex representations by
combining more elementary representations.
12
Various realizations
Tools are made of component parts, and paintings of lines,
surfaces and colors.
Poems are made of words, and songs of words, rythms, and
melodies.
Rules are also made of elements that are words in some sense, but
their status differs from poems or songs. Whereas poems,
songs and also paintings are complex objects that are endproducts and carry their purpose in themselves (at least in our
culture), rules share with tools that they are not selfsufficient.
Tools and rules are instruments to some end, and their use
involves conditions that go beyond the cognitive capacities
that devised them, as one can see. But before we do so, OHW¶V
13
first move to some more elementary considerations.
Some elementary considerations
Inputs from sensory systems cause bio-chemical changes in the internal state of organisms (in a
part that often, but not always, corresponds to a brain). Such changes may be very elementary,
as for instance in the case of snails or SODQDULD¶V.
In mammals, on the other hand, we find a topographically organized representation of the visual
field (Van Essen et al 1984, as an example of an early study). For instance, in the case of
PDFDTXHVWKDWKDYHEHHQµVDFULILFHG¶DIWHUUHFHLYLQJDSDUWLFXODUYLVXDOLQSXWZHFDQVHHWKH
effects of such changes as a visible pattern in their brains like an imprint of that input.
There need not be deep philosophical issues in the conception of a representation. It is enough
that we can observe bio-chemical differences in a brain on the basis of some input that persist
for some time (from seconds to years), and call these representations.
There are vast differences across organisms in the details of the input that can be encoded, and
note, that any object has infinitely more properties than can be encoded. So, representations
are not true or false, they are just more or less detailed, and perhaps more or less useful in the
quest for survival. To say that an organism has imagination, is to say that it has operations
affecting initial representations, yielding brain states of a similar type that did not directly
result from a sensory input.
For the present discussion I will take my starting point in some initial observation about dogs,
then move to chimpanzees and after that to early humans.
15
A crude mode of internal
representation
Bronowski (1967), citing Hunter (1910):
Dog memory experiment:
‡ One of three tunnels would open when a light came on ± the
dog was rewarded with food if went down that tunnel. The
dogs were quite successful at this task.
‡ Adding a complication: The dogs had to wait a while after the
light had gone out again.
‡ A dog forgets which of the three tunnels has been lit in a
matter of seconds.
‡ Similar results have been obtained more recently both for dogs and for cats
(see Journal of Animal Cognition 2003, 2006).
17
A huge step
We now take a huge step from dogs to chimpanzees.
Chimpanzees are well-known for their ability to use
simple tools, and for our persisting attemps to teach
them human language. Although they have a
surprising ability to handle abstract symbols they
have not been able to acquire langage in our sense
(perhaps not so surprising since they DUHQ¶W us).
Here I will primarily limit discusssion to certain aspects
of their tool making.
19
Comparative approach: Complexity
Tool use by chimpanzees:
20
Some notes on chimpanzees
What chimps can do is impressive - from our perspective - as compared to other
animals¶Vtandards. Stories about their capacities in captivity abound in the
literature.
Here I will focus on one instance of tool use in the wild. Sanz and Morgan
(2007) report on their use of tools in catching termites. They use branches to probe
into termite nests. Sticking them in, and getting them out full of termites clinging
to them. However, some types of branches work better than others. For an optimal
result they modify branches by chewing on them, and thus producing branches
that fit the requirements.
Sanz and Morgan studied a group of chimpanzees in the wild by closely
following them during a couple of months. For our understanding of their
memory capacity the following remark is significant:³We also observed that
termite-fishing probes may be conserved and transported between nests within a
single da\´ But not, by implication, overnight. This indicates an important
limitation in their ability to maintain internal representations in their working
memory (WM) through time. If such limitations also apply to other aspects of
WM one would expect that that they could use two tools for a task, and make
simple tools, but that making composite tools would go beyond their WM
21
capacity. What it takes can be summarized as follows:
A bit more sophistication
Tool use by chimps (Sanz & Morgan 2007):
‡
‡
‡
‡
Mental representation of goal
Mental representation of shape fitting the goal
Finding suitable object(s)
Sequential adaptation of objects (branches) to fitting shape
WM properties:
‡ Representation of envisioned shape for as long as it takes
‡ Snapshot comparisons with tool as it develops
‡ Representations restricted - to (relative) here and (relative)
now: Tools not preserved for more than a day, no composite 22
tools.
On tool manufacture by chimps
Tool manufacture of this type requires that one is able
to access both the current shape and the envisaged
shape of the tool under construction.
However, it is not the case that both representations
need to be available simultaneously. It is sufficient to
be able to to access the envisaged shape snapshotwise. BriefµJOimpses¶Wo check whether the tool is
being modified into the right direction.
A related issue is the role of WM in further tasks:
The mental representation of others in the form of a
³7heory of Mind´ (usually abbreviated asµ7oM¶).
23
Theory of mind
Having a theory of mind, that is, forming beliefs of how another person is
perceiving us, and that¶V person possible intentions towards us, is very
characteristic of human behavior.
Whether chimps have a theory of mind is highly disputed among
primatologists. Some, such as the well-known primatologist Frans de Waal
ascribe them quite high level abilities in this respect, others, for instance
Simone Pika (Humboldt Research Group Comparative Gestural Signalling),
are far more skeptical in this regard, and argue that for their apparently
stunning feats there are simpler explanations available thatGRQ¶W rely on
highly developed cognitive functions.
There is, however, agreement on an upper bound. There is no evidence of
any behavior in this domain that goes beyond that of a 4-year old human
child.
See the next slide for aµ7oM stRU\¶Dnd an assessment of its limits.
24
What can be represented:
³7KHRU\RIPLQG´
Many stories of attempted deception (but all disputed)
Byrne (1998):
‡ A[nother] chimpanzee inhibited its normal tendency to begin eating a coveted food item when it
VDZWKHGRPLQDQWFKLPSDQ]HHQHDUE\«7KHGRPLQDQW¶VUHDFWLRQVKRZHGWKDWWKHGHFHSWLRQ
was not successful: it hid and peeped out from behind a tree. Presumably thinking that the
dominant animal had instead left, the subordinate chimpanzee picked up the food, and was
promptly relieved of it. (117)
Tomasello et al. (2003):
‡ &KLPSDQ]HHVFDQPHQWDOO\UHSUHVHQWZKDWFDQEHVHHQEXWFDQQRWUHSUHVHQWRWKHUV¶EHOLHIV
Analysis: Representing others¶beliefs requires accessing belief states per se. That is, the most direct
source for imagining (=representing) belief states, is RQH¶V own belief state, which is
tantamount to a representation of one self, and then moving on WR³if I were in your
position«´or ³OHWPHSXWmy self in your position, and consider the result from my current
position´
In this sense representing some other LQGLYLGXDO¶V belief state involves a recursive step.
This is then what chimps must be lacking:
‡ recursion in the domain of mental operations on belief states.
Or more strongly:
25
‡ recursive operations on mental representations as such.
Towards modernity
One of the crucial steps towards modernity in the
human lineage is the use of fire.
It is tempting to consider this step just as an act of willpower, or as an independent evolutionary step. A
functional analysis shows that it is neither.
To be able to use fire requires quite a bit more of
working memory than is needed for the use of simple
tools. But what it takes can be easily described in
terms of enhanced WM capacity, plus perhaps one
more intangible property: Controlling fear.
See the next slide for a summary of the steps required. 26
A step towards modernity:
The use of fire
From fire to cave
- Envision a new arrangement ± a fire in the cave
- Envision fire in a form that can be handled (burning logs)
- Envision transfer of burning wood to cave
- Crucial for effective implementation: suppression of fear ± accessing RQH¶V
mental state and manipulating it (as if it were temporarily set apart).
27
- Operating on a mental state and returning a mental state: the basis of recursion.
Tool making
The next page illustrates some development in stone
tool making. From rather basic tools in the Oldowan
period to more refined ± sharper, better controlled
flaking - tools in the Acheulien. The stone tools were
manufactured by knapping techniques. Despite some
increase in refinement, relatively little changed over a
very long period of time. It is only at the end of the
Middle Pleistocene(130 ka BP) that that one finds
traces of hafting: attaching a stone head to a wooden
stick to produce a spear (or an arrow), see Coolidge
and Wynn¶s sketch of Neanderthal hafting. Mazza et
al. (2006) report signs of tar hafting in Italy at the end
of the Middle Pleistocene.
30
Developments in tool making
From Oldowan
2.5 -- 1.2 Ma BP
Homo Habilis
to
Acheulien
1.6 Ma ± 200 Ka BP
Homo Erectus
to End
Middle Pleistocene
130 Ka BP
Homo
Neanderthalensis
For extended time periods very little changes
31
Towards a functional analysis of
early tool making
Also for early tool making quite intricate requirements must be met. The maker must have
an eye for material, possible fracture lines, and the right angle for hitting (I am indebted to
Stan Ambrose for once giving me a demonstration of what it takes.)
The next page provides a functional analysis of the requirements on early tool making, based
on Wadley (2010). Although it definitely involves a much higher degree of eye-hand
coordination, and jumping back and forth between mental representation, what it takes can
still be seen as an a gradual extension of abilities of hominin ancestors. The few changes
over long time periods also indicate that developments in early tool making just involve ³a
bit better´of the same.
The question is to what extent hafting involves a fundamental change in mental abilities.
Simple hafting involves manipulating three object/representations, and a goal. What favors
the idea that it represents a leap is that it developed late after a long stationary period.
)XQFWLRQDODQDO\VLVKRZHYHUGRHVQ¶WVKRZthat it requires fundamentally different mental
resources than stone knapping.
Furthermore, as Wadley (2010) points out an apprentice could still learn it by watching.
For Wadley (2010) the main question is whether simple hafting involves recursion in some 32
sense. Under her analysis the next step, complex hafting, in any case, does.
Requirements for early tool making
The knapping technique for stone tools requires mental systems
allowing:
‡ Maintaining attention through time
‡ Manipulating two objects (the knapping stone and the tool
being made) simultanuously.
‡ Keeping active simultanuously two mental representations: the
goal, and the tool under construction (including properties of
its material)
± for the knapping stone just snapshots will do (LVQ¶W it broken).
± process still sequential: after each hit determine if it meets your needs
Invention of simple (tar) hafting: a goal and three objects
(including stickiness as a property of tar)
Transfer of knowledge can still simply take place by watching.
33
From early hominins to modernity:
Progress in tool making
Wadley (2010):
Africa - 70 ka BP: complex adhesives
‡ Compound glue: based on plant gum, combined with
ochre powder (clay containing hydrated iron oxide),
which has no gluelike attributes.
‡ ³,WFRXOGQRWEHSUHGLFWHGZLWKRXWFRQVLGHUDEOH
imagination, that the use of items with nonadhesive
SURSHUWLHVFRXOGFUHDWHVXFFHVVIXOJOXH´
‡ Process involves a chemical reaction: loose, dry
powder + sticky, wet gum + heat = hard, dry
34
concreted adhesive.
Requirements for complex tools
Wadley (2010): Complex adhesives
‡ Memory system must be able to maintain ultimate
goal through a range of subgoals
‡ Planning requires combining materials in ways not
determined by their observable properties
± requires abstraction from the here and now
± requires operations on representations
± abstract objects in the memory system are like any other
object available for manipulation
‡ Wadley: Transmission of such knowledge involves
more than watching: requires description/language
35
A first leap
According to Wadley (2010) complex tool making
involves recursion. In my present terms, it requires
the ability hold representations that have no direct
connection to the sensory system in WM for a
considerable time, and to manipulate and combine
such representations recursively.
Even if there are limits in practice on what can be held
in WM simultaneously, there are no in-built limits on
it in the system.
This reflects an evolutionary leap opening the way for
creating images with serious degrees of freedom,
36
reflected in pictures like the following one.
Imagination
A big leap: manipulating
representations
The creativity of imagination
:HFDQWDNH(VFKHU¶V2WKHU:RUOGEHDGVRUWKHHQJUDYHGRFKUHIURP
Blombos cave, and the Hohlenstein-Stadel figurine all as indicative of a
fundamental leap. They are all direct consequences of an ability to analyze
representations into subparts and recombine them.
This brings us back to 0LWKHQ¶V question, and similar questions about the
µDGDSWLYHYDOXH¶RIFHUWDLQWUDLWVRIKXPDQFRJQLWLRQ,WFDQQRWEH
sufficiently stressed that given such a free combinatorics there is no
SULQFLSOHGGLIIHUHQFHEHWZHHQµXVHIXO¶DQGµXVHOHVV¶FUHDWLRQVWRROVYHUVXV
beads or figurines). Having the one, entails having the other.
Combinatory principles applying to simple representations to imagine complex
tools, will equally well apply to objects, lines, shapes, and colors, to yield
other complex representations.
What is useless at first will subsequently find a use. The driving force behind
WKLVQHHGEHQRWKLQJPRUHWKDQ³SOD\´QRWGHHSHUWKDQZKDWRQHVHHVLQ
kittens, bears or foals in a meadow: exploring their potential.
38
Leaping towards modernity
Beads from Blombos Cave, South
Africa, 70 ka BP
Löwenmensch from
Hohlenstein-Stadel, Europe,
30 ka BP
39
Combining away
Thus, it is of no use to ask what such applications of recursion are good for from an evolutionary
perspective. Once you are able to combine elements, you get the rest for free. A combinatory
V\VWHPGRHVQ¶WVWRSKDOI-way.
Consider, for instance, the number system. 3HDQR¶V axioms define the set of natural numbers and
their properties as follows:
1. 0 is a natural number.
2. For every natural number x, x = x (reflexivity)
3. For all natural numbers x and y, if x = y, then y = x. (symmetry)
4. For all natural numbers x, y and z, if x = y and y = z, then x = z. (transitivity)
5. For all a and b, if a is a natural number and a = b, then b is also a natural number. (closure
under equality)
6 For every natural number n, S(n) is a natural number. (closure under the successor function).
8QGHUWKHVHD[LRPVWKHQDWXUDOQXPEHUVDUHDOOµJLYHQ¶LUUHVSHFWLYHRIWKHLUQRWDWLRQRUWKHLUVL]H
That is, a natural number that requires only 3 digits to write down in a decimal notation is no
more given than a natural number that would require a trillion of digits to write down. One
cannot discover new natural numbers, although one could in principle identify particular
expressions as representing a natural number that has never been represented before (or not in
that particular manner). And, of course, even if they are all given, as are their properties, this
GRHVQ¶WHQWDLOWKDWDOOWKHLUSURSHUWLHVKDYHEHHQLGHQWLILHGDVDQ\VWXGHQWRIPDWKHPDWLFVLV40
only too well aware of).
The leap towards modernity
Summarizing the minimal requirements on mental
representations needed for µmodernity¶
‡ Hafting: combining on the basis of derived properties
‡ Strings of beads: a combination of objects created on the basis
of abstract qualities
‡ Löwenmensch: combining elements into a representation of a
non-existing object
Commonality: A mental system that can hold complex
representations formed by (free) combination and
accesses properties beyond the initial appearance of
an object.
42
An intermediate step or all the way
towards modernity?
Given the availability of operations on representations it is
important to note that:
¾ What such operations can achieve depends on the elements in
the analysis of representations.
¾ Distinguishing parts entails the possibility to play with
recombination of such parts.
¾ Distinguishing borders between parts entails the possibility to
recognize lines,which entails the ability to play with shapes,
ultimately leading to drawing.
¾ Identifying surfaces between borders leads to the notion of
colours, which in turn facilitates playing with colours, and
ultimately leads to painting.
43
First generation imagination
Assuming that an analysis of representations
into lines and colours reflects an older/
independent evolutionary event:
‡ An increase in working memory capacity
making it possible to hold and access complex
representations formed by free (re-)
combination of component parts is sufficient to
yield a range of artistic achievements.
44
What 1st generation imagination
can do
Specifically, given such a system, the prerequisites are
met to imagine paintings, from the famous bison in
the Altamira cave +/- %3WR(VFKHU¶V2WKHU
world from the first half of the previous century.
2UIURP+LHURQ\PXV%RVFK¶V+HOOWKHULJKWSDQHORI
³7KH*DUGHQRI(DUWKO\'HOLJKWV´IURPWKHODVW
decade of the 16th FHQWXU\WRRQHRI6DOYDGRU'DOL¶V
paintings of an egg, realized in an uncannily similar
style almost 500 years later, all illustrated on the next
slide.
45
What it can do!
What it cannot do!
A system restricted to representations derivative
of objects cannot represent:
‡ That every hunter who saw it hit some bear that chased him.
‡ That the FKLHI¶V father told him yesterday that WRPRUURZ¶V hunt
will probably be much better than his last hunt the year
before.
‡ That the priest will probably like to have her first food at
sunrise tomorrow morning.
In order to express these we need a further step:
Language
47
Evolution of language
There is a lot of discussion about the evolutionary origins of
language and why it evolved. Much of it uses notions based on
adaptation and evolutionary advantage that are doubtful, such as
freeing the hands of mothers taking care of their children (Falk
2004), a substitute for grooming (Dunbar 1998), etc. (See Fodor and
Piatelli-Palmarini 2011 for some thought-provoking criticisms of the
notion of adaptation, and the fierce reactions it evoked).
E-coli bacteria, jellyfish, spiders, mice, deer, macaques, all survived
so far quite nicely without language. Rather, reproduction goes with
mutation, some clusters of properties are compatible with survival,
others areQ¶W. Hence from one original population representing a
³gene pool´ different lines emerge showing different clusterings of
48
µGHviations¶Oeading to different paths towaUGµVpecialization¶
Diagnostics for Language?
How can we determine whether some ancestor of ours had language? We find
the following claim by archaeologists: The humans who inhabited Blombos
&DYHLQ6RXWK$IULFDKDGµPRGHUQ¶RUµIXOO\V\QWDFWLFDOODQJXDJH¶VRPH
75,000 years ago (Henshilwood et al. 2004, 17±18).
Botha (2010) however notes that the leap from µshowing symbolic behavior¶WR
µhaving fully syntactical language¶- if that FRPHVGRZQWRµhaving
language in our sense - is not warrranted. And, as we will see, this
criticism is correct. Symbolic behaviour is a prerequisite but not enough.
For an understanding of the evolution of language we must, for a starter:
‡ Develop a functional analysis of what is required for language as we know
it (see Reuland 2010);
‡ Determine in what respects the mental resources needed for language go
beyond what is needed for complex tools or ornaments.
± But parsimoniously, nature makes leaps, but not very big ones
‡ Determine how language bears on imagination.
The next slide shows the prerequisites for this endeavor.
49
Language: Prerequisites
Thought system (not specific to language):
It includes an articulation (of the internal
representation) of the world:
‡ Concepts (spear, fire, bear, hunt, haft, kill, warm, on,..)
‡ Events, temporal structure and participants
± -- (will) haft (the) head (onto the) spear
± The hunter killed the bear
± The hunter lit the fire
Realization system (not specific to language):
An articulation of events in an external medium into
50
discrete repeatable units: sounds, gestures«
Defining language
First approximation:
A language is a systematic mapping of forms in some medium
(sounds, gestures) and interpretations (instructions to
change/update internal representations).
In this form the definition is very broad, and covers much that we
would not want to call language, at least not human language.
Second approximation:
A language is a systematic mapping of forms in some medium
(sounds, gestures) and interpretations (instructions to
change/update internal representations), based on a finite
initial set of arbitrary form-meaning combinations (thus taking
the Saussurean sign as a starting point as sketched on the next
51
slide - although this will turn out to be not quite sufficient).
Relating forms and interpretations
Elementary linguistic units (first approximation):
Expressions in a medium that can stand for a
concept/eventuality in an arbitrary form-meaning
combination:
³Saussurean
The expression
concept of
³arbre´VWDQGVfor
an arbitrary
the concept/
sign´FRUH
mental picture
Arbre
notion in a
of a tree.
naive conception
of language)
A current sketch
The schema of the language system on the next slide (from
Chomsky 1995) enables one to formulate issues sharply, and is
so general that it should not be controversial.
‡ Comb LVWKHµcombinatory system of human language¶LQ
Chomsky (1995) referred to Comb as the Computational
System of Human Language (CHL), or just µsyntax¶
‡ Syntax in the current sense is understood as the computational
system effecting the form-meaning mapping, that is,
connecting the Form interface with the Meaning interface,
mediated by the lexicon.
53
Language as a systematic mapping
of forms onto interpretations
Form interface Meaning interface
SensoriÅ CombÆ Interpretation
Motor system
system (system
of thought)
Lexicon
- dedicated
- dedicated
A systematic form-meaning
mapping
The elementary form-meaning mappings are represented in the Lexicon. This
does not imply anything about the size of these minimal combination.
These may vary from elements such as ±ed standing for PAST when
combined with a verb, articles such as the conveying definiteness, to
composite verbs such as transform, to large units such as kick the bucket
standing for DIE. In fact, without change in principle, elements could
include templates/constructions LQWKHVHQVHRIµconstruction grammar¶
(Croft 2001).
Much of the current discussion in linguistic theory centers on the question how
rich this combinatory syntactic system is. A leading hypothesis in Chomsky
(1995) and subsequent work is that it is in fact very limited: just the
operations of Combine (Merge) and Compare (check), followed by
property sharing (Agree) if there are properties (features) to be shared.
A further important issue is to what extent the systems underlying language are
specific to language, or whether language is just the result of general
55
systems applying to linguistic units.
/DQJXDJH³VWDQGLQJIRU´
For the emergence of language, the core question is: What is the innovation underlying
language, and to what extent does it go beyond what is needed for complex tools?
With complex tools we have a combinatory system, a WM able to maintain mental
representations through time and an ability to monitor progress.
The sounds and gestures realizing language are part of the external world, but as such
their representations are part of the ZRUOG¶V internal representation in the mind.
Turning to the simple case of a sign, what we need is that one mental
representation - the form ´arbre´ ± can stand for another mental representation,
namely the concept TREE, and that the latter is handled by handling the former.
The notion of standing for need not go much beyond what is needed for tool making.
The mental representation used as a reference point in monitoring progress can be
said to stand for a goal. But this reference point is still connected to something
concrete, for instance a tool as it has been used earlier. One can still think of this
earlier tool as acting as a reinforcement to keep the goal representation active.
If so, a core innovation for language may well be what enables arbitrariness: The ability
to keep representations active without outside reinforcement. In a nutshell:
Internally produced stimili are treated on the same footing as external stimuli,
56
making them as effective in keeping the representation active.
Requirements on memory system
This leads to the following requirements on a (working) memory system:
The ability to hold representations of abstract properties:
‡
satisfied by Blombos beads
The ability to hold representations of two or more objects simultanuously.
‡
satisfied by the manufacture of complex tools.
The ability to use and access internally produced representations on the same footing as representations
produced in response to external stimuli (keeping internally produced representations sufficiently active).
‡
A requirement for effective imagination
The ability to treat one representation D³form´reflecting an instruction to the motor system for realization) as
standing for another representation (reflecting ³FRQWHQW´
‡
quite possibly an independent innovation (but note how marginal in view of raisin+ochre +heat=glue)
The ability to (efficiently) switch back and forth between the form and content modes of representation
‡
an independent innovation
. A specific instantiation of this is the key innovation for language: A manipulation RI³form´representations
can be the input for the formation of content representations.
‡
Piggybacking on the motor-system , this effectively leads to recursive combinability and interpretability of
linguistic forms.
57
Reaching modernity
Baddeley (2007, synthesizing previous work) presents an influential functional analysis
of a memory system for mental representations, with four main components (see
also the next slide):
‡ Visuo-Spatial sketch pad
‡ Phonological loop
‡ Episodic buffer
‡ Central executive
Coolidge and Wynn (2005) interpret this from an evolutionary perspective. They
propose the following factor as the crucial step towards modernity:
Enhanced working memory (either a general increase in capacity, or an increase in
the capacity of one of its subsystems, for instance WKHµphonological ORRS¶LQline
with our preceding discussion, a very effective enhancement would be the capacity
to store and manipulate representations both in their form and content mode, that is,
to have coupled representions enabling efficient switch back and forth between
these modes).
They provide a systematic overview of potentially relevant mutations over the last 200
Kyears and their possible effects on the neural basis of our cognitive system (for
58
the details I refer to their paper).
%DGGHOH\¶V0HPRU\0RGHO
59
From Coolidge and Wynn 2005
Structure building unleashed
Given such a change in WM, allowing ³coupling´OHW¶V consider its effect on the capacity
for language in more detail.
The story goes in two steps. First of all, a core property of language is that its elements
can be recursively combined into larger units that can be combined in turn into yet
larger units. Informally: Comb may apply to its own output.
Recursion unleashes combinability into its full potential (Hauser, Chomsky, Fitch 2005)
(see Coolidge et al. 2011 for further instructive discussion).
¾ To have anything like language in our sense, such combination rules must be
associated with interpretation rules that act on the resulting configuration.
¾ This is illustrated by two such configurations with their interpretations:
- Modification configuration: [[Adjective brown] [Noun bear]]
- Interpretation rule: sharing a property.
- Predication configuration: [[Noun PhraseHector] [Verb Prase killed the bear]]
- Interpretation rule: ascribing a property (of being a participant in an event) .
See the next slide for a further illustration of modification and predication informally
sketched in terms of sets and their membership.
60
Language is an intriguing
phenomenon
M odification: Take the intersection of the set denoted by
the adjectival modifier (intriguing) and the set denoted
by the common noun (phenomenon).
Language
Phenomena
Intriguingt
V
intriguing phenomena
Predication: The subject has the property denoted by the
predicate (here reflected by the ascription of set
membership).
Mental time travel
As we saw earlier, discussing the limits of 1st generation imagination, language does
not only allow us to speak about the here and now, but also about past events and
possibly future events, thus positioning such states of affairs on a dimension of
time.
Our imagination not only allows us to position such states of affairs on a dimension of
time, but also to position our selves ± our awareness - in such states of affairs,
HIIHFWLQJPHQWDOµWLPHWUDYHO¶Tulving (2002) identifies three factors ± three clues
in his words ± WKDWSRLQWWRZKDWKHFDOOVµWKUHHFHQWUDOFRPSRQHQWVRID
neurocognitive PLQGEUDLQV\VWHPWKDWPDNHVPHQWDOWLPHWUDYHOSRVVLEOH¶VHQVH
of subjective time, autonoetic DZDUHQHVVDQGVHOI7KLVV\VWHPKHFDOOVµHSLVRGLF
PHPRU\¶DVDQH[WHQVLRQRIµVHPDQWLFPHPRU\¶7KHGLIIHUHQFHEHWZHHQµHSLVRGLF
PHPRU\¶DQGµVHPDQWLFPHPRU\¶LVWKDWHSLVRGLFPHPRU\LVRUJDQL]HGE\WLPH
How to understand these in a parsimonious framework? What is the role of language
with respect to the emergence of episodic memory? Is it a facilitator? Does it take a
free ride? Or it is all part and parcel of the same development?
62
Time travel and language
The crucial common factor is the possibility to access (internal)
representations and manipulate them. For the emergence of a time axis,
nothing more is needed than a WM able to retain that two events occurred
in a sequence, and the subsequent organization of temporal sequentiality
into a recursive structure. For a self nothing more is needed than the ability
WRDFFHVVDQGPDQLSXODWHUHSUHVHQWDWLRQVRIRQH¶VRZQEHOLHIVWDWHV
Subjective time MXVWUHTXLUHVSXWWLQJRQH¶VVHOIDVDQREVHUYHURIDSDVWRU
future event. Autonoetic awareness is nothing more than the ability to
LVRODWHDQGDFFHVVRQH¶VRZQEHOLHIVWDWHVMXVWDVµQRUPDO¶LQGHSHQGHQW
objects. All reduce to (recursively) operating on internal representations.
And, again, once an individual has one she has them all.
The interesting question is how such operations depend on language. One
intriguing possibility is that it is the availability of linguistic expressions
that facilitates the operations involved in perspective change and time
travel. However it is not trivial to subject this to empirical investigation.
63
Diagnostics for language
It is unlikely that we would find in the archaeological record direct evidence
for coupled representations as a diagnostic of the capacity for language.
Perhaps, though, the following two properties could serve as proxies:
i.
ii.
Arbitrary stand-for relations;
Self-representation.
Both are indicators of the availability of recursive operations on internal
representations.
Note: recursion in the relevant sense is a yes²no property of a system. So, one
instance serves for all!
64
Propositionality
Clearly, WM provides a sketchpad with only limited space even after
µH[WHQVLRQ¶Extension can have the effect, however, that a new style WM
can hold structures of a qualitatively different type. Crucial for human
reasoning is the ability to maintain and manipulate propositional structures:
structures that can be true or false (see Reinhart 2006 for discussion).
Modification as in brown bear just yields a more precise description of an
individual. Applying brown to bear retains an expression of the same type;
applying a further modifier, such as dead just induces an incremental
change.
The expression The brown bear is dead is of a radically different sort than the
expression the brown bear or the dead brown bear .
Propositional structures minimally contain predicates and arguments, but the
crucial factor linking them is predication, and the assertion that the relation
expressed by the predication holds. To achieve this not only arguments and
predicates, but also force of assertion, and a temporal dimension must all be
simultaneously be represented and be available for the computation.
66
Nesting dependencies
Characteristic of language is the prevalence of dependencies. Just like the
interpretation of a composite expression depends on the interpretation of its
component parts, the form of one expression may depend on the form of
another expression. For instance, in many languages the form of the
inflected verb depends on formal properties of the subject. In English, for
example, we find I walk, but John walks.
As objects of verbs, we find not only noun phrases, but also full sentences. As
modifiers of nouns we find adjectives, but also full relative clauses. As
modifiers of clauses we find not only adverbs, but also, again, adverbial
clauses. These in turn contain noun phrases and modifiers, etc. So we find
sentences within sentences, noun phrases within noun phrases, etc.
The next slide illustrates recursion in full swing. The arrows indicate where
further material could still be inserted, as in the light gray birds from the
country that fails to protect them or are rapidly arriving in great numbers.
Note the dependency between subject and inflected verb that has to be kept in
memory over quite a distance.
67
Recursion in human language
Structure
S
The birds1
S
are1 arriving
that the man2
S
is2 recording
I3 was3 watching S
when «
Dependencies galore
‡ We find a range of different types of dependencies in natural language. To
mention just a few:
‡ There are agreement dependencies such as those between subjects and
inflected verbs, and between adjectives and nouns, or Case-dependencies
between verbs and their arguments (especially in languages with rich case
marking systems).
‡ There are semantic dependencies between verbs and their arguments (for
instance John worries is fine, but the rock worries is odd).
‡ There are dependencies between quantifiers and elements in their scope
(consider a flag flew from every building ± how many flags?), and between
pronouns and their antecedents ( Mary put the book beside her).
‡ Many of these dependencies may co-occur in one simplex or complex
sentence, and our processing system has no problem handling them.
‡ One type of dependency is often singled out in discussions: namely nested
dependencies, in which a category of a certain type is embedded in a
69
category of the same type, etc. See the next slide.
Recursion DQGµFlattening¶
Since Chomsky and Miller (1963) it has become well-known that certain
nested dependencies provide a problem for the human processing system,
as illustrated in (1), which as anyone will see is difficult:
(1) #The rat [the cat [the dog chased] hurt] died.
This is occasionally used to argue that the human language system cannot
really handle recursion. Such a conclusion is unjustified, however. The
flattened counterpart of (1) in which the embedded relatives occur on the
right presents no particular processing difficulties:
(2) The rat died that the cat hurt whom the dog chased.
Crucially, interpretively nothing has changed. The phrase WKDWWKHFDWDWH«
still modifies the rat, the phrase whom the dog chased still modifies the
cat and the interpretive system has to accommodate all that, and put the
dislocated phrases right back in their proper place for interpretation
purposes. Consequently, this flattening requirement can only reflect a
property of a sub-part of the language processor ± specifically involving
the memory system of forms - but cannot reflect a property of the
70
grammatical system per se.
Beyond signs: function words
Coming back to the nature of minimal units, not all minimal language forms fit the
conception of a Saussurean sign. Some lexical elements are function words and
represent instructions for interpretation rather than concepts.
This reflects what appears to be a further core property of human language:
desymbolization. We see desymbolization in many categories. Here it is illustrated
in the use of determiners and prepositions. Consider :
‡ The determiner the in:
[[Det the] [[Adjective brown] [Noun bear]]] arrived on the scene
Interpretation rule: value the expression by a unique individual in the context of
utterance.
‡ The preposition of in :
[[Det the] [[Noun hunt] [Prep of [the bear]]] was succesful
Interpretation rule: ascribe to the bear the role of a patient in the (hunting) event.
In none of these cases does the lexical item represent a canonical concept.
Such instructions for interpretation can be viewed as operations on forms or pairs of
forms, effectively higher order interpretations. This comes down to expanding the
domain of interpretations: internally generated (linguistic) objects can be treated as71
values, on a par with mental objects resulting from perception of the outside world.
Recursion in a range of dimensions
On the basis of a range of considerations we see that the formation rules of
linguistic objects apply to their own output. We observe this in a range of
domains, including syntactic structure, semantic structure and word
formation:
Structure: combinability applies to elementary and composite syntactic objects.
Word formation: any piece of composite meaning can be squeezed into a word
Arguments: anything from basic objects (cave), to properties of objects (warmth), events (work,
hunt), properties of events (success), complex events (beautification), etc. can form a possible
argument.
Predicates: anything from a basic event description (eat), to a property description (beautify), to a
noun (carpet), can be made into a possible predicate.
Interpretation: anything from basic concepts to higher level instructions for
interpretation may serve as a possible interpretation.
‡
‡
Thus, we have meanings as mental objects reflecting the result of perception, but also as
instructions applying to the formal representation of such objects, or instructions applying to
such instructions, at infinitum.
The hunter hoped he spotted a dear µWKHhunter¶DQGµhe¶refer to the same individual)
72
Every hunter hoped he spotted a dear µhe¶LVbound by the expression µevery hunter¶
Language: Imagination Unleashed
The net effect of this recursive combinability in all domains, is that
imagination is unleashed, and escapes from the easily imaginable. The
immediate sources are the following properties:
‡ Syntactic categories are blind to type of concept
The category Noun or Noun Phrase does not distinguish between the abstract
(beauty) and the concrete ( rock, between individual (John), or mass
(water), or even event ( attack).
‡ Interpretation rules are blind to sense or nonsense
Modification treats brown bear on the same footing as square circle.
Predication treats abstract nouns on the same footing as concrete nouns.
Modification:
[[Adjective brown] [Noun bear]]
[[Adjective square] [Noun circle]]
Interpretation rule: blindly imposes property sharing.
Predication: [[Noun PhraseHector] [Verb Prase killed the bear]]
[[Noun Phrase beauty] [Verb Prase killed the hunt]]
73
Interpretation rule: blindly ascribes a property.
Freedom from the expected
We form nouns and verbs as illustrated below: abstract or concrete, simplex or
compound, behaving as dictated by their category, not by what is
represented.
‡ Nouns: rock, spear, bear, circle, beauty, courage, zero, pi, humanities, hunt,
walk, annihilation, antagonizationNLOOLQJ«
‡ Verbs: arrive, walk, hunt, make, beautify, antagonize, rock, spear, zero,
DQQLKLODWHDQWDJRQL]HNLOO«
Interpretation rules are not restricted by plausibility or by expectations. This
leads us to the very foundation of human creativity, and puns (with profound
implications for linguistic methodology):
‡ The interplay between what the rules give us, and what we expect to find.
(as illustrated by the following notice near a waterfall in Chiapas, Mexico)
74
Language as a systematic mapping
of forms onto interpretations
The interplay of combination and interpretation allows us to first create novel
combinations of forms, and then:
‡ apply fixed rules to determine the interpretation of these combinations (and
note, if the effects of the rules were not fixed, they would not allow us to
escape from the conventional). ( See also Hinzen (2008:xiii³7he point of
ODQJXDJH«LV«DFWXDOO\WRIUHHRXUPLQGIURPWKHFRQWURORIWKHH[WHUQDO
VWLPXOXV«´WKDQNVWR$QGUHD0RURSFIRUGUDZLQJP\DWWHQWLRQWR
+LQ]HQ¶V remark).
‡ determine if there are objects or events, moods, emotions, or anything in
our mental universe corresponding to them.
This yields a new mode of imagination: the language lab, producing both
science and poetry.
1st vs. 2nd generation imagination
Applying fixed interpretation rules to combinations of expressions enables human
imagination to transcend the initial boundaries of the imaginable. Thus language
enabled the transition from a 1st generation imagination to a 2nd generation (and
perhaps even higher orders of) imagination.
1st generation imagination:
Manipulates representations of observable objects on visuo-spatial sketch pad in terms of
primary properties (lines, surfaces, colours)
2nd generation imagination:
- Manipulates derivative representations: combine forms freely, interpret the result by fixed
rules, interpret with disregard for plausibility and then explore its possible uses ± from black
to white holes, from natural numbers to imaginary numbers, from Euclidean to non-Euclidean
spaces
- Leads to the development of the formal languages of mathematics and theoretical physics,
etc.
- Leads to much of our societal superstructure, including Euro-crises, credit crunches, and
management-induced stress.
78
Design features and real languages
One of the recurrent themes in the study of languages is that of
Unity versus variation.
‡ There is unity in the basic design features sketched (arbitrariness, recursion,
word formation, meanings from concepts to meta-instructions, fixed blind
interpretation rules - modification, predication, relation to discourse).
‡ In the superficial manifestations of language we see substantial variability:
± Sound systems: great variability (but within limits)
± Lexicon: no way to predict the meaning of a basic lexical element from its form
or vice versa (except in a few cases of onomatopeia)
± Grammar:
‡ Variation in word order (compare English to Dutch, Russian, or Latin)
‡ Variation in richness of inflection (case, tense - English, Dutch, German, Russian, Latin),.
‡ Covert, versus overt scope marking, etc.
± Semantics, discourse: little variability (perhaps some)
‡ But: Whereever we see variation in language, there are also clear limits to
the variation.
79
The design gap
All this reflects two crucial issues:
The division of labor between nature and culture in language.
The gap between the functional design features and the actual
design features of language as we know it.
These lead to a more fundamental question:
How flexible is the human mind in the face of the products of our
own imagination? Can we cope with everything our
imagination can produce?
‡ What can we say about our limitations in this respect?
Model: Our ability to handle invented languages.
‡ Aims: - understanding the design gap
- language as a model for the general issue
Design features and word order
There is a tension between unity and variation. Are there
restrictions on grammars that GRQ¶W follow from the design
features? We can take two facts as a starting point:
Fact 1: Languages use differences in word order to mark how an
expression is to be used (for instance as an assertion or a
question, and in the latter case to identify what type of
information a proper answer should provide).
Fact 2: The design features we identified GRQ¶W restrict the
relation between word order and interpretation.
These two facts give us two options:
1. All possible operations on word order are in fact permissible
2. The possible operations on word order are restricted.
81
The question is, then, what is the case?
Restrictions on word order
This leads to the debates on language universals. Newmeyer, who adopts a
neutral position in the debate, finds at least the following universals in
operations on word order (rephrased from Newmeyer 2005:4)
Universal 1: Languages cannot µcount past two¶no syntactic process refers to
µthird ZRUG¶µfourth ZRUG¶etc
Universal 2: In no language can a syntactic process be sensitive to the sound
properties of an element (e.g. passive formation restricted to verbs that end
in a consonant cluster)
Both universals can be expressed somewhat differently, as in Chomsky (1986):
‡ Gra mm atical operations are structure dependent
Informally: Operations DUHQ¶W defined on µlinear strings¶but obey natural
groupings of words, as in:
(1) The scientist explained the problem versus (2) The problem was explained
But (1) is never mapped onto: (3) *Problem was scientist explained the
82
‡ 7KHUHVHDUFKHU¶VWDVNLVWRGHWHUPLQHLQGHWDLOZKDWQDWXUDOJURXSLQJVDUH
An example: questions
Many languages form yes-no questions by a putting a verbal element in initial
position, some do so just by intonation, etc. (Friedman 2002)
‡ English: Miri draws a portrait
Q: Does Miri draw a portrait?
‡ Hebrew: Miri mecaryeret portret.
Q: Miri mecaryet portret? (only intonation)
No language forms yes-no questions by systematically putting the verb after
the third word as in:
‡ *Language X (rendered by glossing):
The man from Siberia drew a portrait Æ Q: The man from drew Siberia a
portrait?
The old man from Siberia drew a portrait Æ Q: The old man drew from
Siberia a portrait?
The rule is quite simple. Yet,it seems crazy. But why?
83
Informally: it is a crazy rule since it disregards natural groupings
A Question
What happens if imagination enters the picture?
We can imagine such a crazy µOLQHDU¶ODQJXDJH. I just did!
What is, then, the status of the fact that we find no language
that does it that way? &RXOGQ¶W that be accidental? The
ancestor of current surviving human languages GLGQ¶W have this
property, hence no living language has it? If so, we GRQ¶W have
to look for further language specific design features. How can
we choose between accident and design property?
There is a way to test this: See what happens if we teach human
subjects a language with rules that violate this structure
dependence.
This leads again to the general issue: How free is the human mind
in coping with systems that come from its (= our) own
84
drawing board?
Invented languages
The claim that structure dependence is a fundamental design property of human
language, coupled with the fact that language acquisition is only possible given a
UHVWULFWLRQRQSRVVLEOHJUDPPDUVDUHVWULFWLRQRQWKHµVSDFHRIK\SRWKHVHV¶WKDWFDQEH
considered by the language learner) and that this hypothesis space better reflect the
fundamental design properties of language, together make a clear prediction:
‡
Languages violating structure dependence should not be learnable in the same way
as natural languages.
More specifically, while language acquisition is by and large independent of general
intelligence, to the extent in which rules of language violating structure dependence can
be acquired, this should be sensitive to general intelligence.
Exploring this prediction started a line of research comparing the acquisition and
processing of artificial languages with those of natural language.
It is such investigations that may help us close the design gap by relating identifiable87
general properties of language to the working of the structures subserving language.
Processing invented languages
Currently available brain imaging techniques (measuring on-line changes in activation
and/or location of brain processes) reveal much abut the resources used in language
processing. In view of this our current knowledge allows us to make predictions about
the way non-structure based dependencies are processed.
A brain area that lesion studies have shown to be involved in the processing of
OLQJXLVWLFGHSHQGHQFLHVVXFKDVGLVORFDWLRQVµPRYHPHQWV¶DVLQWKHIRUPDWLRQRI
questions or passives) is %URFD¶V area (see Grodzinsky 2000, Avrutin 2001 for
overviews). Its role is not limited to language; for instance it is also involved in the
processing of music. However, what is common to the dependencies which it has been
shown to be involved in so far, is that they are structure based.
The question is then what happens if subjects are taught a language with some nonstructure based dependencies.
The prediction is that the processing of such non-structure based dependencies will not
involve %URFD¶V area. A further prediction is that if %URFD¶V DUHDSURYLGHVWKHµEHVW¶
routines for the processing of dependencies, such non-structure based dependencies,
once acquired, will require more effort, and be processed less efficiently.
89
An experiment
An experiment to this effect was carried out by Musso et al. (2003). In this experiment
native speakers of German were taught Italian (in fact the experiment was also
FRQGXFWHGIRU-DSDQHVHDVDWDUJHWODQJXDJHEXWIRUEUHYLW\¶VVDNH,ZLOOOLPLW
discussion to Italian).
One group was taught standard Italian, the other group was taught a variety of Italian
with some non-VWUXFWXUHEDVHGGHSHQGHQFLHV+HUH,ZLOOUHIHUWRWKLVYDULHW\DVµOLQHDU
,WDOLDQ¶
Both varieties of Italian had the standard property of being a null-subject language
(unlike German). Also passives were formed in the standard way: with the auxiliary
essere as compared to German werden, and with da translating von, as in Die Birne
wird von mir gegessen versus La pera è mangiata da mè. Both varieties had the finite
verb in post subject position as opposed to the final position of the German verb in
subordinate clauses.
Musso et al. used fMRI (functional Magnetic Resonance Imaging) to determine the
brain structures involved. The basis of this technique is that there is a relation between
brain activity and the oxygen demand in an area. This is reflected in the so-called
BOLD signal (Blood Oxygen Level Dependent signal) which can be measured. This
90
LQGLUHFWO\DOORZVRQHWRGHWHUPLQHµZKHUHWKHDFWLRQLV¶
Real versus linear Italian
The experiment is summarized below:
Two groups of German speaker were tested on the following tasks:
- acquisition and processing of real Italian
- acquisition and processing of linear Italian
German vs Real Italian : Three standard characteristics
Null-subjects
Ich esse die Birne vs. Mangio la pera
Passive
wird «von «vs. è «GD«
Subordinate word order
dass Paul die Birne isst vs. que Paolo mangia la pera
Linear Italian: Three non-structure dependent rules
Negation:
negative no after 3rd word of the sentence
Yes-no questions:
invert the order of the words of the sentence
Indefinite article:
gender form of the article determined by the last
noun of the sentence, not by the noun it µbelongs to¶
91
Examples are given on the next slide.
Processing linear Italian
Linear Italian (Musso et al. 2003) :
‡ Negation: negative no after 3rd word of the sentence
Paolo mangia la no pera| Paola ha mangiato no la pera
‡ Question: invert the order of the words
Paolo mangia la pera Æ pera la mangia Paolo
Paolo ha mangiato la pera Æ pera la mangiato ha Paolo
‡ Use of indefinites: grammatical form determined by last noun of the sentence:
Un ragazza ama un ragazzo | una ragazzo ama una ragazza
Results:
Subjects acquired linear Italian quite well, but processing was significantly slower.
Processing experiment (fMRI )
The results show a significant correlation between the increase in BOLD signal in the left
inferior frontal gyrus (%URFD¶V area) and the online performance for the real but not for the
unreal language learning tasks.
This indicates that the acquisition of new linguistic competence in adults involves a brain
system that is different from that involved in grammar rules that violate [principles of]
Universal Grammar, and that this system is less efficient in carrying out its task.
This result is important since, prima facie, non-structure dependent operations are very simple.
Their workings depend on immediately observable properties of the strings involved.
Thus, what is simple on the drawing board need not be simple for the brain.
92
Language and beyond
Summarizing:
We can imagine linear - superficially economical ± languages, and investigate how our
cognitive system copes with them.
The comparison of the savant Christopher with controls (Smith and Tsimpli 1995)
shows that:
Our ability to acquire linear dependencies involves general intelligence, rather than
the dedicated language faculty.
The comparison between Real Italian and Linear Italian (Musso et al. 2003) shows:
Our ability to process natural versus linear languages involves different brain
structures. Furthermore, linear languages are not processed as efficiently as natural
languages.
Crucial consequence: Drawing board simplicity need not correspond to a simplicity that
is effective for the human mind.
In turn this shows that there are limits to the freedom of our minds to deal with the
results of our own imagination.
Does this apply to all drawing board languages? Not necessarily. An invented language
such as Esperanto can work since it is organically designed. It is based on principles
93
found in natural language, and therefore reflects ³natural economy´
Overview
‡ Discussion of imagination: down-to-earth view ± recursive
operations on mental representations;
‡ 2nd generation imagination mediated by language
‡ Different types of tool making put increasing demands on
memory resources (complex hafting)
‡ Pieces of art place demands on memory resources ± necessary,
but not sufficient for language
‡ Language and its prerequisites - abitrariness & coupled
handling of form and meaning: crucial innovation
‡ Non-linguistic diagnostics for language (?)
‡ The gap between functional design features and what we know
about the structure of language: investigating the role of the 94
neural structures subserving language
Thank you!
Discussion
References 1
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
Pictures Blombos Cave: Archeologie-Nieuws.nl, and various further sources for other pictures (only the waterfall is my own).
Baddeley, Alan. 2007. Working memory, thought and action. Oxford: Oxford University Press
Beaman, C. Philip. 2010. Working Memory and Working Attention. What Could Possibly Evolve? Current Anthroplogy 51.
Supplement 1, S27-S38
Botha, Rudolph. 2010. On the Soundness of Inferring Modern Language from Symbolic Behaviour. Cambridge
Archaeological Journal 20.3, 345-356
Berndt, R. M & C. H. 1964. The World of the F irst Australians. Ure Smith Pty Ltd.
Bronowski.Jacob. 1967. The reach of imagination. Essay.
Coolidge, Fred, and Tom Wynn. 2005. Working Memory, its Executive Functions, and the Emergence of Modern Thinking.
Cambridge Archaeological ]ournal 15:1, 5-26
Coolidge, Fred, Karenleigh Overmann, and Tom Wynn 2011. Recursion: what is it, who has it, and how did it evolve?.
WIREs Cogn Sci, 2: 547±554. doi: 10.1002/wcs.131
Croft, William A. 2001. Radical Construction Grammar: Syntactic Theory in Typological Perspective . Oxford: Oxford
University Press.
Evans, Jonathan St. B. T. 2010. Thinking twice: Two minds in one brain. Oxford, UK: Oxford University Press
Evans, Jonathan St. B. T. 2012. Questions and challenges for the new psychology of reasoning, Thinking & Reasoning, 18:1,
5-3
Fiset S, Beaulieu C, Landry F. 2003. Duration of dogs' (Canis familiaris) working memory in search for disappearing
objects. Anim Cogn. 2003 Mar;6(1):1-10
Fiset S, Doré FY. 2006. Duration of cats' (Felis catus) working memory for disappearing objects. Anim Cogn. 2006 Jan;
9(1):62-70.
Fodor, Jerry, and Piatelli-Palmarini. 2011. What Darwin got wrong. London: Profile books
Friedman, Naama. 2002. Question production in agrammatism: The Tree pruning hypothesis. Brain and Language 80, 160187
Hauser, Marc, Noam ChomMasssimo sky, and Tecumseh Fitch. 2002. The faculty of language: what is it, who has it, and
how did it evolve? Science 298: 1569±1579.
References 2
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
‡
Henshilwood, C.S., F. G¶(UULFR, R. Yates et al.. 2002. Emergence of modern human behaviour: Middle Stone Age engravings
from South Africa. Science 295, 1278±80.
Kahneman, Daniel. 2011. Thinking, F ast and Slow. New York: Farrar, Straus and Giroux
Moro, Andrea. 2008. The Boundaries of Babel . Cambridge, MA: MIT Press.
Musso, M., Moro, A., Glauche, V., Rijntjes, M., Reichenbach, J., Büchel, C., and Weiller, C. 2003. %URFD¶V area and the
language instinct. Nature Neuroscience 6., 774-781
Roebroeks, Wil and Paola Villa. 2011. On the earliest evidence for habitual use of fire in Europe. PNAS 108.13, 5209±5214
Rossem, Maarten van. 2010. Waarom is de burger boos? Nieuw Amsterdam.
Sanz, C., & Morgan, D. 2007. Chimpanzee tool technology in the Goualougo Triangle, Republic of Congo. Journal of
Human Evolution, 52, 420±433
Smith, Neil, and Ianthi Tsimpli. 1995. The mind of a savant. Oxford: Blackwell
Stroop, John Ridley. 1935. Studies of interference in serial verbal reactions. Journ.Exp. Psychology 18 (6): 643±662.
Tettamanti, Marco, Hatem Alkadhi, Andrea Moro, Daniela Perani, Spyros Kollias, and Dorothea Weniger. 2002. Neural
Correlates for the Acquisition of Natural Language Syntax. NeuroImage 17, 700±709
Tettamanti, Marco, Irene Rotondi, Daniela Perani, Giuseppe Scotti, Ferruccio Fazio, Stefano F. Cappa, and Andrea Moroa.
2008. Syntax without language: Neurobiological evidence for cross-domain syntactic computations. Cortex 45, 825-838.
Tulving, Endel. 2002. Episodic Memory: From Mind to Brain. Annu. Rev. Psychol. 2002. 53:1±25
Ullman, Michael. 2004. Contributions of neural memory circuits to language: the declarative/procedural model. Cognition
92:231±270.
Van Essen, David, William T. Newsome, John H.R. Maunsell. 1984. The visual field representation in striate cortex of the
macaque monkey: asymmetries, anisotropies, and individual variability. Vision Research 24.5, 429-448
Wadley, Lyn. 2010. Compound-Adhesive Manufacture as a Behavioral Proxy for Complex Cognition in the Middle Stone
Age. Current Anthropology, Vol. 51, No. S1, Working Memory: Beyond Language and Symbolism, S111-S119
109
Wynn, Thomas, and Fred Coolidge. 2007. A Stone-Age Meeting of Minds. American Scientist 96, 44-51