1
Language
2
Just another set of input/output paths.
Models
Distributed lexicon (orthography, phonology, semantics):
reading & dyslexia.
Levels: phonemes/letters, words, phrases, sentences,
paragraphs, and beyond..
Orthography to phonology: regularities and exceptions.
Semantics to phonology: past tense overregularizations.
Semantic representations from word co-occurrences.
Sentence-level processing and the sentence gestalt.
3
Phonology Features: Vowels
ˆ
u
p
d
o
w
n
v
<- front | back ->
1
2 3
4 5
6
7
+---------------------+
1 E--A_
U
2
\
i --\
u |__hi_ˆ__
3
e
\
--Y-/ O
\
\
\-/
|
4
\
\
ˆ / \--o
5
@
-IW
|__lo_v__
\
\ /
|
6
----- a ------+
round/flat, long/short
4
Phonology Features: Consonants
Restricting airflow.
place: labial, labio-dental, dental, alveolar, palatal, velar, glottal
manner: plosive, fricative, semi-vowel, liquid, nasal
vocalized, not vocalized
5
Distributed Lexicon Model
6
Distributed Lexicon Model & Dyslexias
Semantics
Hidden
Semantics
Hidden
Hidden
Orthography
Orthography
Hidden
Phonology
Hidden
Phonology
Hidden
Phonological: nonwords (“nust”) impaired. Ortho-phono
damage.
Distributed reps (not localized to one region).
Interactive (not modules), leads to interesting divisions of
labor.
7
Distributed Lexicon Model & Dyslexias
8
Distributed Lexicon Model & Dyslexias
Semantics
Hidden
Orthography
Semantics
Hidden
Hidden
Hidden
Phonology
Deep: phono + semantic errors (“dog” as “cat”) +
visual errors (“dog” as “dot”) + combined visual-semantic
(“sympathy” as “orchestra”). Ortho-phono + semantic
damage. Or just more severe damage to ortho-phono.
Orthography
Hidden
Hidden
Phonology
Surface: nonwords OK + problems getting meanings from written
words + difficulty reading exception words (“yacht”) + visual
errors. Semantic damage.
9
10
The Model
Corpus and Semantics
Concrete/Abstract Semantics
Semantics
Y
40.00
star
35.00
wage
cost
rent
loan
hire
stay
role
fact
loss
gain
past
ease
tact
deed
need
lack
flaw
plan
hint
plea
30.00
25.00
20.00
15.00
OS_Hid
flan
tart
SP_Hid
10.00
w
t w y
s t t w t w
a d e g k n r s
i
l
l
p r p r p r i
n o p r s v
l
o r t a c e g
n l
z
z
t v t v t v
5.00
o r s r s r s
k l
g j
k l
0.00
− d − d − d @A − d − d − d
OP_Hid
Phonology
10.00
12
Simulating Dyslexia
X
40.00
Visual
Vis + Sem
Semantic
Blend
Other
Hidden
Hidden
Phonology
Errors
Orthography
Hidden
Errors
Dyslexia Type
Surface
Deep
Surface
Surface
Phono
Deep
Deep
Surface
30.00
Semantic Lesions, Intact Direct: Surface Dyslexia
Semantics
Layer(s) lesioned
Comp Sem
Comp Dir
OS Hid
SP Hid
OP Hid
OS Hid + Comp Dir
SP Hid + Comp Dir
OP Hid + Comp Sem
20.00
hind
deer
hare
loon
reed
wave
grin
face
20 concrete (more features), 20 abstract. (more errors on abstract in
deep dyslexia)
Cleanup self-connections.
11
tent
0.00
g j
lass
flea
n a e n p n p n p
f g f g f g E I g j
n
Orthography
n l
h k h k h k O U k l
p r s t w a e i
c d e f g h l
z
flag
coat
case
lock
post
rope
20
15
10
5
0
20
15
10
5
0
0.0
0.2
Concrete
Abstract
SP_Hid
SP_Hid
OS_Hid
OS_Hid
0.4
0.6
0.8
Lesion Proportion
0.0
0.2
0.4
0.6
0.8
Lesion Proportion
13
Semantic Lesions, Lesioned Direct: Deep Dyslexia
14
Direct Pathway Lesion
Visual
Vis + Sem
Semantic
Blend
Other
Abstract
SP_Hid
SP_Hid
OS_Hid
OS_Hid
20
15
10
5
0
20
15
10
5
0
0.0
0.2
Concrete
Abstract
No Sem
No Sem
Full Sem
Full Sem
0.4
0.6
0.8
Lesion Proportion
0.0
0.2
0.4
0.6
0.8
0.0
Lesion Proportion
15
Errors
20
15
10
5
0
20
15
10
5
0
Concrete
Errors
Errors
Errors
Visual
Vis + Sem
Semantic
Blend
Other
0.2
0.4
0.6
0.8
Lesion Proportion
Distributed Lexicon Model
0.2
0.4
0.6
0.8
Lesion Proportion
Full Sem. Less damage like phonological dyslexia (visual errors),
then like deep (visual + semantic, worse at abstract).
16
Regularities & Exceptions: A Continuum
Regularities in pronunciation are often partial, context dependent:
i in mint, hint.. vs mind, find.. except: pint
Semantics
Hidden
0.0
Hidden
Exceptions are extreme of context dependent.
Orthography
Hidden
Phonology
Seidenberg & McClelland (1989) used extremely context dependent
wickelfeatures: _think_ = _th, thi, hin, ink, and nk_
PMSP used hand-coded context independent inputs (+ some
conjunctions).
Distributed reps (not localized to one region).
Interactive (not modules), leads to interesting divisions of
labor.
Orthography to phonology: regularities and exceptions.
Need a range of context dependency for regulars and exceptions.
17
Reading as Object Recognition
18
Reading Model
Phon
Tradeoff between dependence & independence similar to object
recognition: need invariance but also need to bind features.
phonemes
Hidden
hidden
Ortho_Code
o
wor rds r d
w
Ortho
words...words
19
Reading Model
Reading as object recognition.
20
Models
Distributed lexicon (orthography, phonology, semantics):
reading & dyslexia.
Regulars and exceptions in single system: Differing degrees of
context dependence, rather than qualitatively different.
Orthography to phonology: regularities and exceptions.
Model generalizes to non-words.
Semantics to phonology: past tense overregularizations.
Semantic representations from word co-occurrences.
Sentence-level processing and the sentence gestalt.
21
Distributed Semantics
kinesthetic
ke
bra
fo o t
phone
form
D
3−
Model: Correlational Semantics
Use Hebbian learning to encode structure of word co-occurrence.
Same idea as:
tele
color c
ve
lve
t
kettle
action
oriented
visual
22
tactile
lou
ds
th
un
de
r
V1 receptive field learning: learn the strong correlations.
auditory
Latent Semantic Analysis (LSA).
Hidden
phonology
orthography
Semantics is distributed across specialized processing areas.
Input
23
0.
A
B
C
1.
A
B
C
2.
A
B
C
3.
A
B
C
4.
A
B
C
Multiple-Choice Quiz
neural activation function
spiking rate code membrane potential pt
interactive bidirectional feedforward
language generalization nonwords
transformation
emphasizing distinctions collapsing diffs
error driven hebbian task model based
spiking rate code membrane potential pt
bidirectional connectivity
amplification pattern completion
competition inhibition selection binding
language generalization nonwords
cortex learning
error driven task based hebbian model
error driven task based
gradual feature conjunction spatial invar
object recognition
gradual feature conjunction spatial invar
error driven task based hebbian model
amplification pattern completion
5.
A
B
C
6.
A
B
C
7.
A
B
C
8.
A
B
C
9.
A
B
C
attention
competition inhibition selection binding
gradual feature conjunction spatial invariance
spiking rate code membrane potential point
weight based priming
long term changes learning
active maintenance short term residual
fast arbitrary details conjunctive
hippocampus learning
fast arbitrary details conjunctive
slow integration general structure
error driven hebbian task model based
dyslexia
surface deep phonological reading problem
speech output hearing language nonwords
competition inhibition selection binding
past tense
overregularization shaped curve
speech output hearing language nonwords
fast arbitrary details conjunctive
24
Models
Distributed lexicon (orthography, phonology, semantics):
reading & dyslexia.
Orthography to phonology: regularities and exceptions.
Semantics to phonology: past tense overregularizations.
Semantic representations from word co-occurrences.
Sentence-level processing and the sentence gestalt.
25
26
Sentences
Traditional approach:
People: busdriver (adult male), teacher (adult female), schoolgirl,
and pitcher (boy).
S
NP
(subject)
Actions: eat, drink, stir, spread, kiss, give, hit, throw, drive, rise.
VP
Art
N
The
boy
Toy World
V
NP
(direct object)
chases
the cats
Objects: spot (the dog), steak, soup, ice cream, crackers, jelly, iced
tea, kool aid, spoon, knife, finger, rose, bat (animal or baseball), ball
(sphere or party), bus, pitcher (boy or for drinks), and fur
Locations: kitchen, living room, shed, and park.
Alternative approach:
Distributed reps of sentence meaning: The sentence Gestalt!
Syntax: Active & Passive, phrases.
Parallel to object recognition issues: 3D structural model vs.
distributed reps that distinguish different objects.
27
28
Network
Gestalt
Gestalt Context
Task
Role assignment
Active semantic
Active syntactic
Passive semantic
Passive syntactic
(control)
Word ambiguity
Concept instantiation
Role elaboration
Encode
Decode
Online update
(control)
Conflict
Input
Role Filler
Tests
Sentence
The schoolgirl stirred the kool-aid with a spoon.
The busdriver gave the rose to the teacher.
The jelly was spread by the busdriver with the knife.
The teacher was kissed by the busdriver.
The busdriver kissed the teacher.
The busdriver threw the ball in the park.
The teacher threw the ball in the living room.
The teacher kissed someone (male).
The schoolgirl ate crackers (with finger).
The schoolgirl ate (soup).
The child ate soup with daintiness.
The pitcher ate soup with daintiness.
The adult drank iced-tea in the kitchen (living-room).
29
30
Gestalt Representations
SG Gestalt Patterns
Distributed lexicon (orthography, phonology, semantics):
reading & dyslexia.
Y
16.00
sc_st_ko.
15.00
te_st_ko.
14.00
bu_st_ko.
13.00
Orthography to phonology: regularities and exceptions.
pi_st_ko.
12.00
te_dr_ic.
11.00
bu_dr_ic.
10.00
Semantics to phonology: past tense overregularizations.
sc_dr_ic.
9.00
pi_dr_ic.
8.00
pi_at_st.
7.00
Semantic representations from word co-occurrences.
te_at_st.
6.00
bu_at_st.
5.00
sc_at_st.
4.00
pi_at_so.
3.00
te_at_so.
2.00
Sentence-level processing and the sentence gestalt.
bu_at_so.
1.00
sc_at_so.
0.00
X
0.00
31
Models
2.00
4.00
6.00
8.00
Past Tense
32
Past Tense: U-Shaped Curve
Simple production model: semantics to phonology, with inflections:
Overregularization in Adam
Phonology
ed
(strong
correlation)
Semantics
past
tense
1−Overreg Rate
1.00
Eventual
correct
performance
assumed
0.98
0.96
0.94
0.92
0.90
2
3
4
5
6
7
Years of Age
8
9
10
33
U-Shaped History
Initially: overzealous rule usage.
Then: Rumelhart & McClelland, U-shaped curve based on network
processing of regularities. BUT: trained irregulars first, then regs.
(much controversy ensues)
34
U-Shaped Model in Leabra
Competition & Hebbian learning produce network that is in
dynamic balance between reg & irreg mappings.
Small tweaks can shift it one way or the other! (priming model).
Hebbian sensitivity to correlational regularity.
Later: Plunkett et al., etc, manipulate enviro in graded way.
Problem: backprop is gradient descent!
35
The Past Tense Model
Phonology
Hidden
Semantics
36
The Past Tense Model
Inflection
Base
Reg sfx
–
Past
-ed
3rd pers sing
-s
Progressive
-ing
Past participle
-en
Regular/Irregular examples
I walk to the store daily.
I go to the store daily.
I walked to the store yesterday.
I went to the store yesterday.
She walks to the store daily.
She goes to the store daily.
I am walking to the store now.
I am going to the store now.
I have walked to the store before.
I have gone to the store now.
37
Past Tense Results
a)
38
Early Correct Responding
Overregularization in Leabra
Overregularizations
Responses
1−OR, Responses
75
0.6
0.4
OR
Responses
0.2
25000
50000
Number of Words
75000
0
Overregularization in Bp
1.0
Bp
L H0
L H001
To 1st OR
To 2nd OR
0.8
0.6
0.4
OR
Responses
0.2
0.0
50
25
0
b)
1−OR, Responses
200
0.8
0.0
Total Overregularizations
100
1.0
39
Past Tense Results
0
25000
50000
Number of Words
75000
Models
Distributed lexicon (orthography, phonology, semantics):
reading & dyslexia.
Orthography to phonology: regularities and exceptions.
Semantics to phonology: past tense overregularizations.
Semantic representations from word co-occurrences.
Sentence-level processing and the sentence gestalt.
L H005
150
100
50
0
Bp
L H0
L H001
L H005