Where is the Meaning in
Standard “Semantic” Tasks?
Dave Balota
Taking a Walk through
Doug’s Associative
N i hb h d
Neighborhood
The Lure of “Semantics”
•
•
•
•
•
•
•
•
Semantic Features
Semantic Relatedness
Semantic Clustering
Semantic Networks
Semantic vs Episodic Memory
Semantic Nodes
Latent Semantic Analyses
Etc
Etc….
Is this a Red Herring?
• Many researchers have attempted to draw a distinction
between the effects of association and meaning by creating
materials that are semantically but, ostensibly not
associatively related. However, given the small world
nature of associative links, such attempts will be difficult if
not futile. If only a few associative links separate most
words, how can it be said that two words are not
associated? The problem is not in showing that pairs of
words are meaningfully related in some semantic
classification, but in showing that meaningfully related
pairs are not associated (Nelson, McEvoy, & Schreiber,
2004 page 402-403)
2004,
402 403).
Outline
• Explore “Semantics” vs Associate Effects in
Standard tasks
• Pit associative level information against deeper
LSA estimates in predicting semantic priming
• Is “associative” information really implicitly
activated, as PIER would predict?
• Importance of Large-Scale Databases for progress
in the field.
Where’s
Wh
’ the
h “S
“Semantics”
i ”
in Semantic Priming?
• “Semantic” Priming in Naming and LDT
Reaction time
• Dog-Cat
D C
• Chair-Cat
500
550
• Does this effect involve Meaning???
A
Associations
i i
and/or
d/ Meaning
M i
• Associative Co-occurrence
• DOG-CAT
• Meaning/featural Overlap
• DOG-CAT
Thank God, this Problem has been solved!
Semantic Priming really reflects Semantics
•
•
•
•
•
For example:
Thompson-Schill
Thompson
Schill et al. (1998)
Hines, et al. (1986)
D M
De
Mornay-Davies
D i (1998)
Lucas (2000) review paper concludes that
– priming is indeed semantic and not associatively
mediated
Wh t’ the
What’s
th evidence?
id
?
• When associative strength is presumably
equated
q
one finds more ppriming
g for words that
are also semantically related (Hines, et al.,
1986; de Mornay-Davies,
y
1998; Thompsonp
Schill, et al, 1998)
But Remember what Nelson et al.
But,
al
said….
• “The problem is not in showing that pairs of
words are meaningfully
g y related in some
semantic classification, but in showing that
meaningfully
g y related ppairs are not
associated” (Nelson, McEvoy, & Schreiber,
2004,, ppage
g 402-403).
)
OOPs!
• Hutchison (2003) noted that each of these
studies used items that were reliablyy more
associated in the “semantic” condition
based on the Nelson et al. norms.
Evidence for associative activation in
“semantic” priming
• Mediated Priming Effects:
(Balota & Lorch, 1986; McNamara & Altarriba, 1988)
– “lion”
“li ” Æ “stripes”
“ i ” via
i “tiger”
“i ”
– What are the semantic features overlapping
between LION and STRIPES?
Balota and Paul (1996)
Semantic vs Lexical Level Priming
• Ambiguous Targets
• Unambiguous Targets
•
•
•
•
•
•
•
•
RR
UR
RU
UU
kidney-piano-ORGAN
wagon-piano-ORGAN
kid
kidney-soda-ORGAN
d ORGAN
wagon-soda-ORGAN
RR
UR
RU
UU
lion-stripes-TIGER
fuel-stripes-TIGER
lion shutter TIGER
lion-shutter-TIGER
fuel-shutter-TIGER
Lexical-Level Association
Unambiguous
Ambiguous
P1
Target
lion
TIGER
P1
Target
P2
kidney
ORGAN
piano
P2
stripes
Semantic Level Representations
Unambiguous
P1
Target
lion
TIGER
P1
Ambiguous
kidney
Target
ORGAN 1
ORGAN 2
P2
stripes
P2
piano
Prediction:
• If Semantic Priming reflects “Semantics”
then one should find different p
patterns when
primes converge on the same meaning (e.g.,
LION-TIGER-STRIPES)) compared
p
to
when primes diverging onto different
meanings
g ((e.g.,
g , KIDNEY-PIANOORGAN).
Short SOA (133 ms) Naming
Ambiguous
Prime Type
Mean
Priming
Unambiguous
Mean
Priming
UU
525
520
RU
519
6
512
8
UR
514
11
509
11
RR
510
15
507
13
P di t d
Predicted
17
19
Difference
-2
-6
Across 4 experiments (N = 208)
varying SOA, Task, and Stimulus
Degradation
Ambiguous
A
bi
Pred Obs Diff
22 23 +1
Unambiguous
U
bi
Pred Obs Diff
30 29 -1
Conclusion
• Unnecessary to assume that Semantic
Priming
g Effects engage
g g semantics,, and
hence, Doug’s Pretty Cool!
What happens if now we force
Meaning Selection?
• Relatedness Decisions
I th
Is
the third
thi d wordd related
l t d in
i any way to
t the
th first
fi t
two primes?
Short Duration Relatedness Decisions
Ambiguous
Prime Type
Mean
Priming
Unambiguous
Mean
Priming
UU
1028
980
RU
844
184
839
141
UR
865
163
858
122
RR
828
200
718
262
P di t d
Predicted
347
Difference
-147***
263
-1
C l i
Conclusions
• “Semantic” priming effects in word naming
and lexical decision can be accommodated by
y
simple lexical co-activation.
• Additional effects of “meaning”
meaning require the
direction of attention to semantic-based
representations, such as in relatedness
decisions.
What about Memory Performance?
• Memory researchers have historically
argued
g
that semantics is critical in gguiding
g
retrieval during free recall tests
• This has become even more central in
recent False Memory Studies
Semantics in the DRM Paradigm
Study List
BED
REST
AWAKE
TIRED
DREAM
WAKE
SNOOZE
BLANKET
.
.
.
Recall
BED
TIRED
WAKE
.
.
SLEEP—non-presented
SHEET
RELAX
DOZE
REST
BED
PILLOW
TIRED
BLANKET
SNOOZE
AWAKE
SLEEP
NAP
DREAM
NOSE
SLUMBER
SNORE
WAKE
ASSOCIATIVE STRENGTH IS CLEARLY A STRONG
PREDICTOR OF FALSE MEMORY
(Deese, 1959; Roediger et al., 2001).
BUT, SAME OLE PROBLEM
CO-OCCURRENCE
SEMANTIC OVERLAP
Hutchison & Balota (2005):
A bi
Ambiguous
andd Unambiguous
U
bi
Critical Items in DRM
• Replicate Summation Studies with False Memoryy
Paradigm
• List items related to either one meaning or two
meanings
i
off an ambiguous
bi
wordd
6 Item List construction
DRM
Homograph
•
•
•
•
•
•
•
•
•
•
•
•
•
Snooze
Wake
Bedroom
Unconscious
Deep
Blanket
---OR---Slumber
Lay
Motel
Trance
Lazy
Nightmare
•
•
•
•
•
•
•
•
•
•
•
•
•
Wrong
Correct
Accurate
Proper
Exact
A
Answer
----OR----Left
Starboard
Clockwise
Turn
Direction
Handed
12 Item List construction
DRM
Homograph
•
•
•
•
•
•
•
•
•
•
•
•
Snooze
Wake
Bedroom
Unconscious
Deep
Blanket
Slumber
Lay
Motell
Trance
Lazyy
Nightmare
•
•
•
•
•
•
•
•
•
•
•
•
Wrong
Correct
Accurate
Proper
Exact
A
Answer
Left
Starboard
Clockwise
Turn
Direction
Handed
E
Experiment
i
t1
50
45
6-rel
12-rel
Percen
nt Recall
40
35
30
25
20
15
10
5
0
drm
hom
List Items
drm
hom
Critical Items
E
Experiment
i
t1
50
45
6-rel
12-rel
Percen
nt Recall
40
35
30
25
20
15
10
5
0
drm
hom
List Items
drm
hom
Critical Items
12 Item List construction
DRM
Homograph
•
•
•
•
•
•
•
•
•
•
•
•
Snooze
Wake
Bedroom
Unconscious
Deep
Blanket
Slumber
Lay
Motell
Trance
Lazyy
Nightmare
•
•
•
•
•
•
•
•
•
•
•
•
Wrong
Correct
Accurate
Proper
Exact
A
Answer
Left
Starboard
Clockwise
Turn
Direction
Handed
•
•
•
•
•
•
•
•
•
•
•
•
Experiment 2:
Mi d List
Mixed
Li
DRM
Homograph
g p
Snooze
Slumber
Wake
Lay
Bedroom
Motel
Unconscious
Trance
Deep
Lazy
Blanket
Nightmare
‹
‹
‹
‹
‹
‹
‹
‹
‹
‹
‹
‹
Wrong
Left
Correct
Starboard
Accurate
Cl k i
Clockwise
Proper
Turn
Exact
Direction
Answer
Handed
Experiment
pe e 2
Mixed List
60
6-rel
12-rel
Percen
nt Recall
50
40
30
20
10
0
drm
hom
List Items
drm
hom
Critical Items
Percen
nt Recall
Experiment
pe e 3
Mixed, 200 ms presentation
40
6-rel
35
12-rel
30
25
20
15
10
5
0
drm
hom
List Items
drm
hom
Critical Items
Experiment
pe e 4
Mixed, 80 ms presentation
30
6-rel
12-rel
Percen
nt Recall
25
20
15
10
5
0
drm
hom
List Items
drm
hom
Critical Items
What if one again forces selection?
• Subjects were asked to make “gist”
based responses,
responses i.e.,
i e “rate
rate how closely
the test word is in meaning to the
studied words”
R l t d
Relatedness
Decision
D ii
Mean Relattedness R
M
Rating
8.5
6-rel
12-rel
8.1
7.7
73
7.3
6.9
6.5
drm
hom
List Items
drm
hom
Critical Items
Conclusions
1. False Recall in the DRM paradigm occurs equally for lists
that converge
g on the same meaningg and diverge
g on different
meanings.
2. Attention to semantics is necessary to find an influence of
meaning.
3. Co-occurrence associative information can take one
relatively
l i l far
f in
i accounting
i for
f both
b h “semantic”
“
i ” priming
i i effects
ff
and “semantic” influences in DRM.
Associative vs LSA
Accounts for “Semantic” Priming
Is there a lot more in semantic priming than
simple
i l associative
i ti effects.
ff t If so, possibly
ibl
LSA would pick up that extra something.
Hutchison, Balota
Hutchison
Balota, Cortese,
Cortese &
Watson (in press)
• Directly pitted estimates from LSA against
Doug’s
g associative estimates in ppredicting
g
“semantic” priming in a large database of
200 subjects
j
and 300 prime-target
p
g pairs.
p
• Used regression techniques to partial out
item covarying variables
variables.
Results from Regression Analyses
FAS
LDT
.13
13*
Pronunciation
.17
17*
Results from Regression Analyses
FAS
BAS
LDT
.13
13*
.14*
Pronunciation
.17
17*
.05
Results from Regression Analyses
FAS
BAS
LSA
LDT
.13
13*
.14*
.02
02
Pronunciation
.17
17*
.05
.03
03
Conclusions
• After Controlling for a host of other
variables,, via regression
g
techniques,
q ,
Forward and Backward Associative
Strength
g surpasses
p
LSA in ppredicting
g
semantic priming effects.
Do Associates Really Get Implicitly
Activated?
Do Associates Really Get Implicitly
Activated?
• Current models of ISOLATED word
recognition
g
emphasize
p
the processes
p
leadingg
up to some threshold, and then the goodies
((i.e.,, the semantics/associates are activated).
)
Do Associates Really Get Implicitly
Activated?
• Current models of ISOLATED word
recognition
g
emphasize
p
the processes
p
leadingg
up to some threshold, and then the goodies
((i.e.,, the semantics/associates are activated).
)
• However, PIER emphasizes the implicit
activation of associates at both encoding
and retrieval, which is based primarily on
episodic memory performance
performance.
Question
• Is there implicit activation of associates in
route to recognizing
g
g a target
g word as
reflected by Lexical Decision and
Pronunciation Performance?
To Answer the Question
• Used the English Lexicon Project
(
(elexicon.wustl.edu)
)
• A web-based repository of over 40,000
words and nonwords that were collected
across 6 institutions (including Doug and
USF) including over 1600 subjects
USF),
subjects.
The Proxy for Implicit Activation is
Connectedness
• Connectedness was defined as the number
of associates produced from a word and the
number of times a word was produced in
response to other associates, based on
Nelson et al.’s norms.
Collins & Q
Quillian
Collins & Loftus
Smallworld structure
Results
(Balota et al., 2004, JEP:General)
• Connectivity predicts both naming and LDT
(p < .05) via regression analyses on the
English Lexicon Project data,
data replicating
and extending an earlier observation by
Steyvers & Tenenbaum (2005)
(2005).
Conclusion
• Yup, it looks like there is implicit activation
of associates and these can drive even
isolated word recognition, strong support
for a basic tenant of PIER.
General Conclusions
• The role of “Semantics” in standard semantic
tasks appears to be accommodated by associative
level information
• Associative strength as measured by Nelson et al.
d
does
a muchh better
b
job
j b off predicting
di i “semantic”
i
priming effects than LSA
• There
Th is
i clear
l evidence
id
off iimplicit
li it activation
ti ti off
associations even in isolated word recognition, ala
PIER.
PIER
Conclusion Cont.
• Large scale databases are particularly
critical in an cumulative science when there
are so many item level differences.