Functional decomposition of spoken
word production and its control
Ardi Roelofs
Max Planck Institute
for Psycholinguistics
F.C. Donders Centre for
Cognitive Neuroimaging
Nijmegen Institute for
Cognition and Information
Collège de France, Paris, Jeudi 11 Mai 2006
A few numbers
We produce about 2 to 5 words per second.
Vocabulary of more than 30,000 words.
About 1 error per 1,000 words.
With 1 hour talking per day, one produces
about 4.000.000 words per year,
about 200.000.000 words in half a century,
i.e., each word, on average, about 6500 times.
POIRE
1
“I can resist everything
except temptation.”
Oscar Wilde
“Une distraction, à l’instant où l’irritation
agit, est toujours punie par une
prolongation de durée de l’acte.”
F. C. Donders (1868)
Subtraction method
“[One] discovers in comparing the incoming
and outflowing blood that oxygen has been
consumed [by the brain].”
F. C. Donders (1868)
(Archives néerlandaises des sciences exactes et naturelles, 1868)
1868
2006
Left-lateralized perisylvian network
(2004)
2
J. Ridley Stroop
(1935)
BLEU
VERT
ROUGE
CAFÉ
CLÉ
TANGO
Stroop asymmetry
Task: Name ink color!
Task: Read word aloud!
BLEU ROUGE XXX
BLEU ROUGE VERT
J. Ridley Stroop
(1935)
J. Ridley Stroop
(1935)
3
Two accounts of Stroop asymmetry
1
2
Pathway strength
Prefrontal-parietal control network
(GRAIN)
Functional architecture
(WEAVER++)
Two accounts of ACC function
Anterior Cingulate Cortex
1
2
Outline of the remainder
1
2
3
Structure of lexical
memory.
Stroop asymmetry and
its time course.
Control processes.
Conflict detection
(GRAIN)
Top-down regulation
(WEAVER++)
Outline of the remainder
1
2
3
Structure of lexical
memory.
Stroop asymmetry and
its time course.
Control processes.
4
Slips of the tongue
Slips of the tongue
Rossi & Defare (Int. J. Psycholing., 1995)
Rudolf Meringer
& Karl Mayer
Versprechen und Verlesen (1895)
Meaning
“Il faut faire grossir les gros obèses”
maigrir
Syntax
“sur la femme de leurs genoux”
les genoux de leur femme
“une contusion à la chambre”
jambe
“Franche Dimance”
c
ch
Form
Meaning
PLANT(X)
Conceptualizing
IS-A
WEAVER++
Levelt, Roelofs, & Meyer (BBS, 1999)
STEM(X)
HAS-A
FORM
FORM
<pommier>
1
/@/
/p/
2
[pom]
Form encoding
<-z>
3
/o/
ONSET NUCLEUS
Form
singular
NUMBER
FORM
2
/l/
plural
pommier
GENDER
<le>
1
Lemma retrieval
MEANING
Syntax
masculine
APPLE
TREE(X)
CODA
4
/m/
5
/j/
ONSET
/e/
NUCLEUS
[je]
Form encoding
Articulation
Conceptualizing
Lemma retrieval
POMMIER
Form encoding
POMMIER
“pommier”
Articulation
“pommier”
Conceptualizing
Lemma retrieval
Indefrey & Levelt (Cognition, 2004)
5
Outline of the remainder
1
2
3
Structure of lexical
memory.
1
Stroop asymmetry and
its time course.
Control processes.
“Hi, I am
Colin MacLeod”
Outline of the remainder
2
3
“Hi, I am
Colin MacLeod”
J. M. Cattell (1886)
Structure of lexical
memory.
Stroop asymmetry and
its time course.
Control processes.
Stroop asymmetry
Word
Color
“Reading is faster than color naming.”
6
VERT
COLOR FIRST
-300
-200
VERT
COLOR SECOND
-100
0
100
200
WORD FIRST
300
-300
-200
SOA (in milliseconds)
Melara & Algom (Psych. Rev., 2003)
COLOR SECOND
200
COLOR FIRST
300
300
400
0
WORD SECOND
200
-100
-400
-200
-300
WORD FIRST
300
Melara & Mounts (Mem. & Cogn., 1993)
400
-50
-100
0
-50
-100
200
VERT
100
0
-200
50
0
-100
50
100
VERT
Word reading
-400
100
-300
150
100
RT EFFECT (msec)
Color naming
100
0
SOA (in milliseconds)
incongruent
congruent
150
-100
WORD SECOND
SOA (msec)
Glaser & Glaser (JEP:HPP, 1982)
Repeating is as fast as color naming
COLOR NAMING LATENCY
(in milliseconds)
550
incongruent
congruent
525
(hear: bleu) incongruent
500
475
(hear: rouge) congruent
450
425
control task: keypress
400
-300 -200 -100 0
Roelofs (Mem. & Cogn., 2005)
100 200 300
SOA
(in milliseconds)
7
Two accounts of Stroop asymmetry
WORD REPETITION LATENCY
(in milliseconds)
incongruent
congruent
600
1
550
Pathway strength
(GRAIN)
500
450
2
400
Functional architecture
(WEAVER++)
350
300
-300 -200 -100 0
100 200 300
SOA
(in milliseconds)
GRAIN
incongruent
congruent
TASK
COLOR INPUT
color
word
naming reading
WORD INPUT
Color naming
RT (msec)
ROUGE VERT
Word reading
550
700
500
650
450
600
400
550
350
500
300
WORD FIRST
“rouge" “vert"
VOCAL RESPONSE
GRAIN
750
WORD SECOND
-300 -200 -100 0 100 200
COLOR FIRST
300
COLOR SECOND
-300 -200 -100 0 100 200
SOA (msec)
300
SOA (msec)
Cohen & Huston (1994, Att. & Perf.)
Cohen et al. (1990, Psych. Rev.), Botvinick et al. (2001, Psych. Rev.)
WEAVER++ model of spoken word production
COLOR
color
perception
WORD
word-form
perception
conceptual
identification
(a)
(b)
lemma
retrieval
word-form
encoding
articulation
OVERT SPEECH
Levelt et al. (BBS, 1999),
Roelofs (Cognition, 1992, 1997; Psych. Rev., 2003, 2004)
8
TASK
TASK
COLOR NAMING(X)
GRAY
(X)
WHITE
(X)
BLACK
(X)
PINK
(X)
IF task is color naming
THEN select concept for color
and enhance its activation
PURPLE
(X)
ORANGE
(X)
OLIVE
(X)
RED
(X)
IF concept is selected
THEN select lemma
CONCEPTUAL
STRATUM
YELLOW
(X)
BROWN
(X)
COLOR INPUT
VIOLET
(X)
BLUE
(X)
COLOR
(X)
GREEN
(X)
gris
pourpre
blanc
noir
rose
orange
olive
violet
bleu
couleur
SYNTACTIC
STRATUM
Colin MacLeod’s (1991)
top-16
incongruency, congruency, reverse Stroop,
response set, semantic gradient, time course, stimulus,
spatial, multiple task, manual, bilingual, training, age,
and pathological effects.
jaune
brun
rouge
WORD INPUT
adj.
vert
ROUGE
<rouge>
/r/
/u/
WORD -FORM
STRATUM
/G/
[ruG]
Roelofs (Psych. Rev., 2003)
OBSERVED
WEAVER++
Color naming
150
150
100
100
0
100
200
300
400
100
200
300
400
-200
-100
-400
-300
100
200
300
400
-200
-100
-100
0
-100
-400
-300
0
-50
0
50
0
-50
-100
50
Word reading
150
150
100
100
50
50
0
100
200
300
400
-100
0
-200
-50
-100
-400
-300
-50
-100
-200
0
-400
-300
RT EFFECT (msec)
250 data points from 16 classic studies
3 free parameters taking 2 values each.
r(model-data) = .98
WEAVER++ explains 96 percent of the variance.
SOA (msec)
distractor
SOA = -100 msec
ACTIVATION
10.0
distractor
SOA = 0 msec
COLOR
color
perception
WORD
word-form
perception
target
8.0
6.0
conceptual
identification
(a)
(b)
4.0
2.0
lemma
retrieval
word-form
encoding
0
0
50
100
150
200
250
articulation
TIME (msec)
OVERT SPEECH
9
COLOR
color
perception
WORD
word-form
perception
conceptual
identification
lemma
retrieval
(a)
(b)
COLOR
color
perception
WORD
word-form
perception
conceptual
identification
lemma
retrieval
(a)
(b)
word-form
encoding
word-form
encoding
articulation
articulation
OVERT SPEECH
OVERT SPEECH
color naming
word reading (after task switch)
mediated
direct
ROUTE
LATENCY EFFECT (msec)
VESTE
40
OBJECT
object-form
perception
WORD
word-form
perception
conceptual
identification
mediated (e.g., VESTE)
direct (e.g., ROUTE)
20
0
lemma
retrieval
(a)
-20
(b)
word-form
encoding
-40
-60
articulation
-80
0
100
200
SOA (msec)
OVERT SPEECH
Roelofs (Psych. Rev., 2003)
Picture naming
150
150
100
100
0
100
200
300
400
200
300
50
0
-50
-50
-100
-100
400
400
-200
-100
300
100
SOA (msec)
200
100
100
200
300
400
0
-100
0
-100
-100
0
100
0
-200
-200
-400
-300
150
100
-400
-300
-300
-200
Word reading
150
50
-400
-400
-300
0
100
-100
200
300
400
-100
-200
0
-50
-100
CHIEN
50
0
-50
-400
-300
LATENCY EFFECT (msec)
50
OBSERVED
WEAVER++
SOA (msec)
(Glaser & Düngelhoff, JEP:HPP, 1984)
10
Picture distractor
TASK
CHIEN
OBSERVED
WEAVER++
word reading
25
Say “chien"
Say “animal"
LATENCY EFFECT (msec)
-25
word categorizing
CHIEN
Outline of the remainder
0
1
100
Structure of lexical
memory.
50
0
2
-50
Stroop asymmetry and
its time course.
-100
word reading with determiner
50
CHIEN
3
25
Control processes.
0
Say “le chien"
-25
semantic
Roelofs (Psych. Rev., 2003)
unrelated
DISTRACTOR
Outline of the remainder
1
2
3
Structure of lexical
memory.
Stroop asymmetry and
its time course.
Prefrontal-parietal
control network
Control processes.
anterior cingulate cortex
ACC
Posner & Raichle (Images of Mind, 1994)
11
BOLD RESPONSE (% CHANGE)
Two accounts of ACC function
incongruent
congruent
0.4
0.3
1
Conflict detection
(GRAIN)
0.2
2
0.1
Top-down regulation
(WEAVER++)
0.0
1
2
3
4
5
SCAN (2.5 sec)
MacDonald et al. (Science, 2000)
GRAIN
TASK
(DLPF)
color
word
naming reading
COLOR INPUT
WORD INPUT
ROUGE VERT
ENERGY
GRAIN
1.5
incongruent
congruent
1.0
0.5
conflict
detection (ACC)
0.0
0
500
250
“rouge" “vert"
TIME (msec)
VOCAL RESPONSE
Botvinick et al. (Psych. Rev., 2001)
Cohen et al. (Psych. Rev., 1990), Botvinick et al. (Psych. Rev., 2001)
WEAVER++
IF task is color naming
THEN select concept for color
and enhance its activation
central
sulcus
frontal
lobe
art
wfe
wfp
lateral
fissure
VERBAL
RESPONSE
ci/lr
goal
control
cp
WORD
COLOR
frontal
lobe
BOLD RESPONSE
(% CHANGE)
0.4
input
control
anterior
cingulate
occipital
lobe
word
planning
executing
control
temporal
lobe
Roelofs (Psych. Rev., 2003)
incongruent
congruent
fMRI observed
(ACC)
WEAVER++
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0.0
0.0
1
2
3
4
5
1
2
3
4
5
SCAN (2.5 sec)
MacDonald et al.
(Science, 2000)
Roelofs & Hagoort
(Cogn. Brain Res., 2002)
12
Stroop subtractions in neuroimaging
VERT
ROUGE
XXXX
neutral
observed
index
finger
middle
finger
LEFT
(X,(Y))
RIGHT
(X,(Y))
IS-NOT
SENSE
WORD
CLASS
adj.
WORD
droit
CLASS
FORM
nu
40
20
0
Arrow Word
Target
/sh/
/d/
2
forms
3
/r/
4
/w/
manual
/A/
on1 on2 on3 nu
co
[gosh]
[drwA]
vocal
response
60
40
20
0
Arrow Word
Target
RT effect (msec)
1
RT effect (msec)
on
60
lemmas
<droit>
3
/o/
/g/
vocal
WEAVER++
60
40
20
0
Arrow Word
Target
FORM
<gauche>
2
concepts
SENSE
gauche
1
?
WEAVER++
manual
response
GAUCHE
GAUCHE
RT effect (msec)
congruent
RT effect (msec)
incongruent
60
40
20
0
Arrow Word
Target
Turken & Swick (Nat. Neurosc., 1999)
Incongruent vs. congruent in word task
Manually
Respond to the arrow
Respond to the word
congruent
neutral
incongruent
RIGHT
DROIT
XXXX
GAUCHE
LEFT
RIGHT
DROIT
RIGHT
DROIT
GAUCHE
LEFT
Roelofs, Van Turennout, & Coles (submitted)
13
GRAIN
congruent
neutral
incongruent
incongruent
WEAVER++
625
575
575
525
525
475
%E
0.9
1.2
4.6
arrow task
1.5
1
Mean
0.5
regression
0
coefficient -0.5
-1
-1.5
arrow task
1.2
2.8
475
6.3
word task
word task
selection
1.5
1
0.5
0
-0.5
-1
-1.5
arrow task
word task
arrow task
word task
ENERGY
Mean RT
(ms)
625
congruent
neutral
selection
CYCLE
Botvinick et al. (Psych. Rev., 2001)
Summary
1
2
3
Structure of lexical memory:
Concepts, lemmas, forms
Stroop asymmetry and its
time course:
Strength vs. architecture
http://www.nici.ru.nl/~ardiroel/weaver++.htm
Control processes:
ACC: conflict detection
vs. regulation
Merci beaucoup!
http://www.nici.ru.nl/~ardiroel/stroop.htm
14