A parametric study of hierarchical structure building in fMRI and MEG
William
1
Matchin ,
Christopher
2
Hammerly ,
Ellen
1Dept.
of Linguistics, University of Maryland
2Dept. Of Linguistics, University of Massachusetts
1
Lau
Contact: [email protected]
Introduction
Whole-brain contrasts
ROI analyses
Coordinates from Pallier et al. (2011)
Structure-based contrasts
Natural sentence >
natural list
10 Natural sentence >
natural phrase
Natural phrase >
natural list
IFGtri
0.25
Left
0 Left
Jabberwocky sentence >
jabberwocky list
Left
Left
Jabberwocky phrase >
jabberwocky list
Jabberwocky sentence >
jabberwocky phrase
0
0.1
-0.02
0.05
List
Phrase
Sentence
pSTS
aSTS
Math
0.1
IFGorb
Phrase
Sentence
Math
List
Temp. pole
-0.2
Left
-0.1
0.1
0.15
0.1
Temp. pole
0.05
0.05
Content-based contrasts
Natural list >
jabberwocky list
Natural phrase >
jabberwocky phrase
aSTS
0
List
0
List
Natural sentence >
jabberwocky sentence
Phrase
-0.05
-0.25
Left
Math
0.05
0
-0.15
Left
Sentence
pSTS
0.1
TPJ
0
List
Math
0.15
0.05
-0.05
Sentence
-0.08
0.2
IFGorb
Phrase
-0.06
0.2
0.15
List
-0.04
-0.1
Methods
Phrase
Sentence
Math
Phrase
Sentence
-0.05
-0.05
-0.1
Math
–– Natural –– Jabberwocky -0.1
-0.15
Conclusions
Left
Left
Left
All contrasts: uncorrected voxel-wise p < 0.001 (one-tailed)
Preliminary MEG Results
–– natural sent –– natural phrase –– natural list -­‐-­‐-­‐ jabberwocky sent -­‐-­‐-­‐ jabberwocky phrase -­‐-­‐-­‐ jabberwocky list All
words
Magnetic field strength
•  Jabberwocky generated by replacing open-class items with
phonologically-plausible nonwords
•  Stimuli presented in blocks of six sequences of six words
•  List condition created by scrambling words within blocks
•  Phrase condition created by segregating Determiner-Noun
sequences and Auxiliary-Verb sequences to their own six-word
sequences
•  Three separate stimulus lists (no repetition of items)
•  Math (baseline) condition: sequences of 0 or #
•  Task: determine whether a probe word appeared in the preceding
sequence (left/right hand for yes/no, counterbalanced within
each modality)
•  500 ms/word rapid serial visual presentation
•  N=16, within-subjects, counterbalanced order of fMRI and MEG
•  fMRI analysis: block design
•  MEG analysis: event-related design; no baseline correction used
because of overlapping responses; open class and closed class
words coded separately for further sub-analyses
0.02
0.15
0
TPJ
0.04
IFGtri
-0.15
•  Three levels of
structure: list, phrase,
sentence
•  Two levels of content:
natural, jabberwocky
•  No overt morphology
on open-class items to
limit structure-building
within words (Halle &
Marantz, 1993)
0.06
Left
0.2
t-value
•  Does this activation reflect syntactic structurebuilding, or other mechanisms?
•  What is the timecourse of structural effects? Is
the structural effect associated particularly with
closed- or open-class items?
•  We performed a modified replication in fMRI
and MEG, using a different task and more
tightly controlled stimulus materials
0.3
Percent Signal Change
•  The neural correlates of
syntactic structure building
remain poorly understood
•  Pallier et al., 2011 showed
increasing activation for
increasing constituent size
in several language-related
regions of the left hemisphere
fMRI Results
Closedclass
Openclass
IFG
•  Sentence > list and natural > jabberwocky in posterior and anterior
temporal lobe, replicating previous work (Fedorenko et al., 2012).
•  Potential linear effect of structure for jabberwocky and natural stimuli in
pars orbitalis.
•  Little to no indication of structural effects for jabberwocky in temporal
lobe and pars triangularis, and no effect of natural phrase condition,
speaking against a syntactic function of incremental structure-building.
- Alternative function of these areas: lexical cue-based working
memory? (Glaser et al., 2013)
•  Anterior temporal areas show intermediate effects of structure for
natural condition, consistent with combinatory semantics (Bemis &
Pylkkanen, 2011)
•  TPJ does not show effect of intermediate structure for natural condition
- sentence-level semantic function? (Dronkers et al., 2004)
•  Differential effects of structure and content for open- and closed-class
items
References
ATL
PTL
Bemis, D. K., & Pylkkänen, L. (2011). Simple composition: A magnetoencephalography
investigation into the comprehension of minimal linguistic phrases. The Journal of
Neuroscience, 31(8), 2801-2814.
Dronkers, N. F., Wilkins, D. P., Van Valin, R. D., Redfern, B. B., & Jaeger, J. J. (2004). Lesion
analysis of the brain areas involved in language comprehension.Cognition, 92(1), 145-177.
Glaser, Y. G., Martin, R. C., Van Dyke, J. A., Hamilton, A. C., & Tan, Y. (2013). Neural basis of
semantic and syntactic interference in sentence comprehension. Brain and
language, 126(3), 314-326.
Halle, M., & Marantz, A. (1993). Distributed morphology and the pieces of inflection.
Pallier, C., Devauchelle, A. D., & Dehaene, S. (2011). Cortical representation of the constituent
structure of sentences. Proceedings of the National Academy of Sciences, 108(6),
2522-2527.