Language
Overview of Models
Distributed lexicon (ortho, phono, sem): reading & dyslexia.
1
Overview.
2
Biology and Basic Representations.
3
Distributed Lexicon & Dyslexia.
4
Reading & Past Tense.
Semantics to phonology and past tense
overregularizations.
5
Semantics & Sentences.
Semantic representations from word co-occurrences.
Orthography to phonology in reading: regularities and
exceptions.
Sentence-level processing and the sentence gestalt.
1 / 33
Biological Substrates of Language
2 / 33
Phonology Features: Vowels
nasal cavity
velum
(soft palate)
teeth
uvula
epiglottis
l
eta
l
Wernicke’s
Broca’s
l
pora
glottis
(vocal cords)
ˆ
u
p
Pa
ri
pharanx
tongue
nta
Fro
Tem
Occipital
hard palate
alveolar
ridge
lips
d
o
w
n
v
<- front | back ->
1
2 3
4 5
6
7
+---------------------+
1 E--A_
U
2
\
i --\
u |__hi_ˆ__
3
e
\
--Y-/ O
\
\
\-/
|
4
\
\
ˆ / \--o
5
@
-IW
|__lo_v__
\
\ /
|
6
----- a ------+
round/flat, long/short
3 / 33
4 / 33
Phonology Features: Consonants
Distributed Lexicon Model & Dyslexias
Semantics
Hidden
Orthography
Hidden
Hidden
Phonology
place: labial, labio-dental, dental, alveolar, palatal, velar, glottal
manner: plosive, fricative, semi-vowel, liquid, nasal
vocalized, not vocalized
Phonological: nonwords (“nust”) impaired.
Deep: phono + semantic errors (“dog” as “cat”) +
visual errors (“dog” as “dot”).
Surface: nonwords OK + semantic access impaired +
difficulty reading exception words (“yacht”) + visual
errors.
5 / 33
The Model
6 / 33
Corpus and Semantics
Concrete/Abstract Semantics
Semantics
Y
40.00
star
35.00
30.00
25.00
20.00
OS_Hid
15.00
SP_Hid
flan
tart
w
s t t w t w
t w y
p r p r p r i
a d e g k n r s
i
l
l
l
n o p r s v
n l
z
t v t v t v
o r s r s r s
k l
5.00
g j
k l
0.00
OP_Hid
Phonology
0.00
Cleanup self-connections.
lass
flea
tent
g j
− d − d − d @A − d − d − d
n
wage
cost
rent
loan
hire
stay
role
fact
loss
gain
past
ease
tact
deed
need
lack
flaw
plan
hint
plea
10.00
z
n a e n p n p n p
f g f g f g E I g j
p r s t w a e i
c d e f g h l
n l
h k h k h k O U k l
o r t a c e g
Orthography
z
10.00
flag
coat
case
lock
post
rope
20.00
hind
deer
hare
loon
reed
wave
grin
face
30.00
X
40.00
20 concrete (more features), 20 abstract.
7 / 33
8 / 33
Simulating Dyslexia
Dyslexia Type
Surface
Deep
Surface
Surface
Phono
Deep
Deep
Surface
Errors
Visual
Vis + Sem
Semantic
Blend
Other
Errors
Layer(s) lesioned
Comp Sem
Comp Dir
OS Hid
SP Hid
OP Hid
OS Hid + Comp Dir
SP Hid + Comp Dir
OP Hid + Comp Sem
Semantic Pathway Lesions, Intact Direct
Semantics
Hidden
Orthography
Hidden
20
15
10
5
0
20
15
10
5
0
0.0
0.2
Abstract
SP_Hid
SP_Hid
OS_Hid
OS_Hid
0.4
0.6
0.8
0.0
Lesion Proportion
Phonology
Hidden
Concrete
0.2
0.4
0.6
0.8
Lesion Proportion
9 / 33
Semantic Pathway Lesions, Lesioned Direct
10 / 33
Direct Pathway Lesion
0.0
0.2
Concrete
Abstract
SP_Hid
SP_Hid
OS_Hid
OS_Hid
0.4
0.6
0.8
Lesion Proportion
0.0
0.2
0.4
0.6
Errors
20
15
10
5
0
20
15
10
5
0
Visual
Vis + Sem
Semantic
Blend
Other
Errors
Errors
Errors
Visual
Vis + Sem
Semantic
Blend
Other
0.8
20
15
10
5
0
20
15
10
5
0
0.0
0.2
Concrete
Abstract
No Sem
No Sem
Full Sem
Full Sem
0.4
0.6
0.8
Lesion Proportion
Lesion Proportion
11 / 33
0.0
0.2
0.4
0.6
0.8
Lesion Proportion
12 / 33
Language II: Reading and Past Tense
Reading: Regularities & Exceptions
Regularities in pronunciation are often partial, context
dependent:
i in mint, hint.. vs mind, find.. except: pint
Exceptions are extreme of context dependent.
1
Reading.
2
Past Tense.
Seidenberg & McClelland (1989) used extremely context
dependent wickelfeatures: _think_ = _th, thi, hin, ink,
and nk_
PMSP used hand-coded context independent inputs (+ some
conjunctions).
Need a range of context dependency for regulars and
exceptions.
13 / 33
Reading as Object Recognition
14 / 33
Reading Model
Phon
Tradeoff between dependence & independence similar to object
recognition: need invariance but also need to bind features.
phonemes
Hidden
hidden
o
wor rds r d
w
Ortho_Code
words...words
Ortho
15 / 33
16 / 33
Nonword Performance
Past Tense
Regularity tests (Glushko): bint → /bint/
Simple production model: semantics to phonology, with
inflections:
Pseudo-homophones (McCann & Besner):
phoyce → /fYs/, choyce → /CYs/
Matched regularity/exception cases (Taraban):
High freq: poes → /pOz/, goes → /gOz/, does → /dˆz/
Low freq: mose → /pOs/, poes → /pOz/, lose → /lUz/
Nonword Set
Glushko regulars
Glushko exceptions raw
Glushko exceptions alt OK
McCann & Besner ctrls
McCann & Besner homoph
Taraban & McClelland
Model
95.3
79.0
97.6
85.9
92.3
97.9
PMSP
97.7
72.1
100.0
85.0
n/a
n/a
Phonology
ed
(strong
correlation)
People
93.8
78.3
95.9
88.6
94.3
100.01
Semantics
past
tense
17 / 33
Past Tense: U-Shaped Curve
18 / 33
U-Shaped History
This is the interesting target developmental phenomenon:
Overregularization in Adam
Initially: overzealous rule usage.
1−Overreg Rate
1.00
Eventual
correct
performance
assumed
0.98
Then: Rumelhart & McClelland, U-shaped curve based on
network processing of regularities. BUT: trained irregulars first,
then regs.
0.96
(much controversy ensues)
0.94
Later: Plunkett et al., etc, manipulate enviro in graded way.
0.92
Problem: backprop is gradient descent!
0.90
2
3
4
5
6
7
Years of Age
8
9
10
19 / 33
20 / 33
U-Shaped Model in Leabra
The Past Tense Model
Phonology
Competition & Hebbian learning produce network that is in
dynamic balance between reg & irreg mappings.
Small tweaks can shift it one way or the other! (priming model).
Hebbian sensitivity to correlational regularity.
Hidden
Semantics
21 / 33
The Past Tense Model
22 / 33
Past Tense Results
a)
Overregularization in Leabra
Reg sfx
–
Past
-ed
3rd pers sing
Progressive
Past participle
-s
-ing
-en
Regular/Irregular examples
I walk to the store daily.
I go to the store daily.
I walked to the store yesterday.
I went to the store yesterday.
She walks to the store daily.
She goes to the store daily.
I am walking to the store now.
I am going to the store now.
I have walked to the store before.
I have gone to the store now.
0.8
0.6
0.4
OR
Responses
0.2
0.0
0
b)
25000
50000
Number of Words
75000
Overregularization in Bp
1.0
1−OR, Responses
Inflection
Base
1−OR, Responses
1.0
0.8
0.6
0.4
0.0
23 / 33
OR
Responses
0.2
0
25000
50000
Number of Words
75000
24 / 33
Past Tense Results
Language III: Semantics and Sentences
Early Correct Responding
100
Total Overregularizations
200
50
25
0
Bp
L H0
L H001
L H005
Overregularizations
Responses
75
150
100
1
Semantics.
2
Sentences.
50
To 1st OR
To 2nd OR
0
Bp
L H0
L H001
L H005
25 / 33
Distributed Semantics
Model: Correlational Semantics
kinesthetic
ke
bra
foot
phone
form
D
3−
Use Hebbian learning to encode structure of word
co-occurrence.
visual
Same idea as:
tele
color c
tactile
V1 receptive field learning: learn the strong correlations.
lou
ds
th
u
Latent Semantic Analysis (LSA) (but avoids “blobs” of
PCA).
nd
er
ve
lve
t
kettle
action
oriented
26 / 33
auditory
Hidden
orthography
phonology
Input
Semantics is distributed across specialized processing areas.
27 / 33
28 / 33
Sentences
Toy World
Traditional approach:
S
NP
(subject)
People: busdriver (adult male), teacher (adult female),
schoolgirl, and pitcher (boy).
VP
Actions: eat, drink, stir, spread, kiss, give, hit, throw, drive, rise.
Art
N
The
boy
V
NP
(direct object)
chases
the cats
Objects: spot (the dog), steak, soup, ice cream, crackers, jelly,
iced tea, kool aid, spoon, knife, finger, rose, bat (animal), bat
(baseball), ball, ball (party), bus, pitcher, and fur
Locations: kitchen, living room, shed, and park.
Syntax: Active & Passive, phrases.
Alternative approach:
Distributed reps of sentence meaning: The sentence Gestalt!
Parallel to object recognition issues: 3D structural model vs.
distributed reps that distinguish different objects.
29 / 33
Network
30 / 33
Tests
Gestalt
Gestalt Context
Task
Role assignment
Active semantic
Active syntactic
Passive semantic
Passive syntactic
(control)
Word ambiguity
Encode
Concept instantiation
Role elaboration
Decode
Online update
(control)
Conflict
Input
Sentence
The schoolgirl stirred the kool-aid with a spoon.
The busdriver gave the rose to the teacher.
The jelly was spread by the busdriver with the knife.
The teacher was kissed by the busdriver.
The busdriver kissed the teacher.
The busdriver threw the ball in the park.
The teacher threw the ball in the living room.
The teacher kissed someone (male).
The schoolgirl ate crackers (with finger).
The schoolgirl ate (soup).
The child ate soup with daintiness.
The pitcher ate soup with daintiness.
The adult drank iced-tea in the kitchen (living-room).
Role Filler
31 / 33
32 / 33
Gestalt Representations
SG Gestalt Patterns
Y
16.00
sc_st_ko.
15.00
te_st_ko.
14.00
bu_st_ko.
13.00
pi_st_ko.
12.00
te_dr_ic.
11.00
bu_dr_ic.
10.00
sc_dr_ic.
9.00
pi_dr_ic.
8.00
pi_at_st.
7.00
te_at_st.
6.00
bu_at_st.
5.00
sc_at_st.
4.00
pi_at_so.
3.00
te_at_so.
2.00
bu_at_so.
1.00
sc_at_so.
0.00
X
0.00
2.00
4.00
6.00
8.00
33 / 33