Précis for
Structure, Meaning, and Constituency in Visual Narrative Comprehension
By Neil Cohn
Ph.D. in Psychology
Tufts University
2012
Committee:
Ray Jackendoff, Gina Kuperberg, Phillip Holcomb, and Marianna Eddy
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
2
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
Visual narratives told through sequential images extend throughout history,
whether drawn on cave walls, painted on pottery, or printed in contemporary comic
books and strips. How are we able to make sense of a sequence of images like those
found in comics or films? I argue that this question is directly analogous to the linguistic
question of how we make sense of a sequence of words in a sentence. However,
compared with the study of narrative and sentence processing in language, the study of
sequential image comprehension has been relatively impoverished. Overall, I argue that
sequences of images at the narrative level are structured and processed analogously to
sequences of words at the sentence level. The main idea is that a narrative “grammar”
organizes the structure of sequential images similar to the way that syntax organizes
words into coherent sentences. To investigate this theory, I have drawn upon the research
and methods of the study of sentence processing, combining theoretical linguistics with
empirical experimentation from cognitive psychology and cognitive neuroscience.
The dissertation itself examines two key parts of the analogy between sequential
images and sentences. First, Chapter 1 explores the idea that visual narrative
comprehension involves an interaction between two systems: narrative structure and
semantic coherence. This correspondence is akin to the interaction between syntax and
semantics at the sentence level. In Chapter 2, I explore the idea that narrative structure
hierarchically organizes images into groupings of constituents, analogous to the phrase
structures of syntax in sentences. My Closing Remarks then briefly discuss the overall
implications for this analogy with regard to broader cognition.
3
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
Theoretical model
Like the “pre-Chomskyan” models of syntax, early approaches to sequential
image comprehension looked mainly at the one-to-one relationships between “panels”—
the image units of a visual sequence (e.g., McCloud, 1993). However, like sentences,
visual sequences have several traits that limit a linear approach. For example, like words
in sentences, some panels may connect with others across distances, panels can be
grouped into constituents, constituents can embed inside of other constituents, and
sequences may be structurally ambiguous in their parsing (Cohn, 2010, 2013). Thus, one
facet of my graduate work has been to design a theoretical model that formalizes the
grammar of visual narrative (Cohn, 2013). This model builds on the limitations of
previous grammatical approaches to narrative, like “story grammars” (e.g., Mandler &
Johnson, 1977), but organizes individual panels in a visual narrative into hierarchic
constituents. Importantly, just as syntax is separate from semantics (Jackendoff, 2002),
this model keeps the narrative grammar separate from meaning, though prototypical
correspondences are maintained between these structures.
In this model, each panel is classified as a particular narrative category and
prototypically sequenced into a canonical order. Each category can also serve as the
higher-level node to a whole constituent in a tree structure. Figure 1 illustrates the
narrative structure for a 6-panel Peanuts comic strip. This sequence depicts a baseball
game that starts with Lucy hitting a ball, which allows Charlie Brown to run home and
score, escaping a tag by Schroeder. The first panel shows Lucy tossing a ball. This panel
functions as an Initial, initiating the interactions in the sequence. In the second panel,
Lucy hits the ball, a narrative “Peak” as the culmination of the initiated action. Together,
4
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
these two panels act as an Initial constituent that propels the rest of the sequence. A
second constituent begins with Schroeder waiting for the ball—nothing happens here
except a set-up of the characters (Establisher). The next Initial begins the new set of
events, which climax in the penultimate Peak panel—Charlie interrupts the catch by
sliding into the base. The last panel then resolves this interaction (Release). Together,
these panels act as a Peak that is set up by the constituent-level Initial of the first panels.
Thus, at a higher level of structure, the first constituent (Lucy hitting the ball) acts as an
Initial, which facilitates the second constituent (Charlie scoring), a Peak. As a result, the
narrative structure operates on both the panel level and the level of whole constituencies.
Figure 1. Constituent structure in a visual narrative
Importantly, the comparison between syntax and narrative structure is an analogy
only. Words in sentences and images in visual narratives differ prominently. Foremost,
images usually convey far more information than words, perhaps even at the level of a
whole sentence or more. Because of this, the narrative structure guiding sequential
5
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
images may be the same structure guiding sequential sentences in verbal discourse.
Nevertheless, the analogy between visual narrative and sentences serves as a useful
framework to inform and interpret the study of sequential image comprehension. For
example, this linguistic approach to theoretical modeling establishes a structure of visual
narrative that can be tested by psychological experimentation. Thus, in my dissertation, I
directly used this model to design two sets of experiments testing the psychological
validity of this theory of narrative grammar. My experimental methodologies were
similarly interdisciplinary, seeking converging evidence using behavioral measures from
cognitive psychology (self-paced comic viewing) and neurophysiological measures from
cognitive neuroscience (event-related potentials).
Chapter 1: Structure and Meaning in Visual Narrative Comprehension
In my first experiment, I tested the claim that this narrative grammar is separate
from the system of meaning. Earlier research from our laboratory used behavioral and
ERP techniques to directly examine the contributions of structure and semantic
relationships in sequential images in my model of narrative grammar (Cohn, Paczynski,
Jackendoff, Holcomb, & Kuperberg, 2012). We first replicated the results of a classic
study of sentence processing (Marslen-Wilson & Tyler, 1980), by comparing the reaction
times to target panels in sequences that globally manipulated the presence or absence of
structure or semantics. We compared normal sequences that had both structure and
semantics with “structural only” sequences that maintained a felicitous narrative structure
but no semantic relations between panels (i.e., analogues of the sentence Colorless green
ideas sleep furiously, that had a narrative arc, but no semantic associations between
images), and “semantic only” sequences with semantically related panels but no narrative
6
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
(i.e., panels all related to a common theme—like “baseball”—but with no unfolding
narrative). Finally, scrambled sequences had neither narrative structure nor semantic
relationships using random strings of panels. We found the fastest reaction times to
panels in normal sequences, intermediate times to panels in structural only and semantic
only sequences, and the slowest times to panels in fully scrambled sequences. These
results indicated that structure and semantics both played important roles in the
comprehension of sequential images.
A second experiment in this study examined the event-related potentials (ERPs)
elicited by panels in these sequences, based on another classic study of sentence
processing (Van Petten & Kutas, 1991). ERPs allow us to analyze the
electrophysiological activity of the brain during online comprehension. Collapsing across
all panels, we found an “N400 effect”—a waveform typically associated with semantic
processing (Kutas & Federmeier, 2011)—that was largest to panels in scrambled
sequences and structural only sequences, intermediate to panels from semantic only
sequences, and smallest to panels in normal sequences. These results indicated that
narrative structure, in the absence of semantics, had little effect on reducing the
amplitude of the N400 (i.e., because amplitudes to panels in scrambled and structural
only sequences did not differ). This combination of behavioral and ERP research
provided converging evidence suggesting that both narrative structure and semantics are
involved with the processing of sequential images.
This previous study provided initial support for a separation between narrative
structure and semantic coherence. This evidence came from the absence of an attenuated
N400 effect to panels in sequences with only structure. Nevertheless, this study did not
7
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
provide positive evidence in the form of two different ERP components. Indeed, this
study only manipulated the global presence or absence of narrative structure in the full
sequence. Thus, the first experiment of my dissertation focused at a local level, by
introducing panels that violated the narrative structure and/or semantic context of an
otherwise normal sequence. Like prior studies looking at the interaction of syntactic
structure and semantics at the sentence level (e.g., Osterhout & Nicol, 1999), such a
manipulation at the narrative level might also be expected to elicit two different
neurocognitive responses. While the N400 effect has primarily been associated with
semantic processing, I expected that incongruities in this narrative structure would elicit
an ERP component more associated with the violation of grammar in language, such as
the P600, which has been thought to reflect continued attempts to integrate structure and
meaning following violations of structural expectancies (Kuperberg, 2007).
I designed four types of sequences that introduced incongruous panels that fully
crossed narrative structure and semantic coherence. Normal sequences featured climactic
panels that were congruous both structurally (they are Peaks) and semantically (they fit
the theme of the sequence), as in Figure 2a. Semantic Anomalies introduced panels that
maintained the structural context in the sequence as climactic Peaks, but had no semantic
coherence to the sequence (they did not fit the theme of the strip), as in Figure 2b.
Structural Anomalies were the opposite: these panels maintained semantic coherence
with the sequence, but violated the narrative structure by replacing the Peak with an
Initial (Figure 2c). Finally, Dual Anomalies introduced panels that violated both structure
(Initials instead of Peaks) and semantics (no meaningful relationship to the sequence), as
8
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
Figure 2d. Participants controlled the pace at which they viewed each panel of a strip,
while I measured ERPs and viewing times to each critical panel and the subsequent panel.
Figure 2. Image sequences that manipulate the contributions of semantic coherence and narrative structure
in a critical panel.
The ERP results showed the most convincing evidence for a separation of
structure and meaning in processing (for complete results, see dissertation). At the critical
panel, I found a smaller N400 effect when the content was semantically congruous
(Normal, Structural Anomalies) than incongruous (Semantic and Dual Anomalies), as
depicted in Figure 3. Normal Peaks had the smallest N400 effect with a slightly larger
effect evoked by Structural Anomalies. A larger N400 effect was shown to Semantic
9
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
Anomalies, with the largest shown to Dual Anomalies. Thus, at the critical panel,
semantic incongruities are more difficult to process than semantically congruous panels.
Figure 3. Waveforms from Frontal electrode sites for Critical Panels. Voltage maps illustrate the
differences across the scalp surface of ERPs evoked by panels between each ascending sequence type at the
400-600 ms (N400) time window.
At the subsequent panel, a distinctly different ERP component appeared as a
positivity with a posterior distribution, consistent with the P600. This effect was larger
following structural incongruities (Structural and Dual Anomalies) than following a
coherent structure (Normal, Semantic Anomalies), as depicted in Figure 4. These results
suggest that a reanalysis of structure is necessary following violations to the narrative
grammar (when an unexpected narrative category appears), yet such reanalysis is not
10
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
necessary when the narrative grammar remains intact, despite semantic incongruity (as in
the Semantic Anomalies).
Figure 4. Waveforms from Parietal electrode sites for Critical Panels +1. Voltage maps illustrate the
differences across the scalp surface of ERPs evoked by panels between structurally congruous (Normal and
Semantic Anomalies) and structurally incongruous (Structural and Dual Anomalies) sequence types at the
600-900 ms (P600) time window.
These results showed that, as in sentence processing, the brain response to
violations of narrative grammar (P600) differed from the brain response to violations of
semantic coherence (N400). Altogether, these results provide evidence for an interaction
between a narrative grammar and semantic coherence in the processing of sequential
images. Such a manipulation would not be possible without the predictions made by my
11
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
theoretical model of narrative structure. Thus, despite important differences between the
“grammars” in the verbal and visual modalities, this interaction between narrative
structure and semantics in sequential images is analogous to the interaction between
syntax and semantics in sentences.
Chapter 2: Constituent Structure in Visual Narrative Comprehension
The second set of studies in my dissertation examined the claim that a constituent
structure guides narrative structure. In a classic experiment on constituent structure in
syntax, Fodor and Bever (1965) pioneered a “click technique” where they played a verbal
sentence in one ear of a participant, and then introduced short bursts of white noise
(“clicks”) in the other ear. They reasoned that, if clauses constitute the perceptual
processing units of sentences, clicks disrupting those units would be harder to discern
than clicks occurring between clauses. They found that participants were better able to
recall clicks placed at the clause boundaries than those before or after it. Subsequent
studies found similar results using a variety of measures, with the overall interpretation
that such responses to disruptions provide evidence for the psychological validity of a
syntactic constituent structure (for review see Garrett & Bever, 1974).
Based on these precedents, I used an analogous “disruption” paradigm to
determine whether the comprehension of visual sequences also draws upon a constituent
structure in narrative sequences. I measured viewing times in graphic sequences where
blank white “disruption” panels were inserted Before, At, or After the boundary between
narrative constituents. I reasoned that, if participants use a constituent structure in the
comprehension of sequential images, processing should be easier for sequences when the
disruption cleanly divides constituents than when the disruption interrupts an individual
12
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
constituent. Using this paradigm, Experiment 1 measured viewing times as participants
read comic strips one panel at a time at their own pace. Experiment 2 then examined
ERPs to the disruptions specifically.
Using my theoretical model, I designed novel sequences that had two narrative
constituents. These constituents were confirmed using a behavioral task in which 20
participants drew lines to divide strips into two parts. Final strips had a 71% agreement
for the location of the constituent boundaries. Each sequence had constituent boundaries
appearing after panel 2, 3, or 4 (40 of each type). Using these strips, I then inserted white
“disruption” panels Before, At, or After the constituency boundaries (as notated at the
bottom of Figure 5), along with No-Disruption control sequences.
Figure 5: Narrative structure in a comic strip with markings Before, At, and After the narrative constituent
boundary.
13
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
Experiment 1: Self-Paced Viewing
In Experiment 1, I analyzed “self-paced viewing times” for how long participants
spent viewing each panel of a sequence. I began the analysis by looking directly at
disruption panels themselves. Disruption panels After the boundary were viewed
significantly slower than those Before or At the boundary, as in Figure 6. This suggested
that the greatest disruptions occurred after participants had crossed the narrative
boundary into the second constituent.
Figure 6. Mean viewing times to disruption panels placed Before, At, or After the narrative constituent
boundary. Error bars represent standard error.
The placement of disruption panels in a sequence also impacted the processing of
subsequent panels (See Figure 7). Viewing times to panels immediately after the
disruption differed from corresponding panels in sequences that had No-Disruption (i.e.,
Disruption +1, see Figure 3). Panels following disruptions Before the narrative
constituent boundary were viewed significantly slower than corresponding No-Disruption
14
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
panels, as were panels following disruptions After the boundary, a difference
approaching significance. However, panels following disruptions At the boundary were
not significantly slower than corresponding panels in No-Disruption sequences. These
results suggested that disruptions had more impact on the comprehension of subsequent
panels when they appeared within constituents (Before or After the boundary) than
between constituents (At the boundary).
Viewing Times (ms)
1100
*
n.s.
^
1080
1060
1040
1020
1000
980
960
940
920
Before
At
After
Figure 7: Viewing times to panels immediately after the disruption panel (Disruption +1, colored)
compared with corresponding panels in sequences with No-Disruptions (black). * = p<.05, ^= p.074.
Delayed effects also appeared three panel positions after the narrative constituent
boundary (Boundary+3, see Figure 8). Panels following disruptions Before or After the
boundary were viewed slower than panels following disruptions At the boundary and
panels in sequences with No-Disruptions. However, viewing times did not differ between
panels following disruptions Before and After the boundary, or between panels following
a disruptions At the boundary and in sequences with No-Disruption.
15
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
Figure 8: Viewing times to panels three positions after the narrative constituent boundary (Boundary +3) in
all sequence types.
Altogether, these results are consistent with the presence of constituent structure
to visual narrative: disruption panels had greater impact within (Before/After) as opposed
to between (At) constituents. Despite these results supporting constituent structure, I did
not find evidence that comprehenders actively make predictions about this structure as
they progress through a sequence panel-by-panel. One limitation of this experiment could
have been the behavioral measure of viewing times, which only indexes the sum result of
all cognitive processes involved in comprehension. In order to provide evidence of
prediction, Experiment 2 used event-related potentials (ERPs), which directly index the
neurocognitive processing of comprehension at the disruption panel itself.
Experiment 2: Event-related potentials
ERPs provide an additional way to gain insight on the structural processing of
sequential images, again drawing upon research from sentence processing. Violations of
16
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
constituent structure evoke a distinct brain response as an anterior negativity, often with
left lateralized distribution: the Left Anterior Negativity (LAN) ERP component (Neville,
Nicol, Barss, Forster, & Garrett, 1991). Though it often precedes a P600 effect, unlike the
P600, the LAN also appears in pseudoword sentences that lack meaning (Münte, Matzke,
& Johannes, 1997), suggesting that it is quite specific to structural aspects of language
processing. Anterior negativities outside of language have also been associated with
violations of “grammatical” structure in music. Patel and colleagues (1998) found a P600
to structural violations in musical sequences (e.g., a nearby key chord or a distant-key
chord appearing after an otherwise in-key musical sequence). Another negative-going
effect, distributed over right anterior and temporal sites, also appeared between 300 and
400 milliseconds. This “(early) right anterior negativity” has led researchers to argue for
overlap in the neural resources used to process structure in both music and language
(Koelsch, Gunter, Wittfoth, & Sammler, 2005; Patel, et al., 1998).
In Experiment 2, I predicted that violations of constituency in visual narratives
would elicit similar ERP effects as violations of structure during sentence processing:
P600 and LAN effects. I used the same set of stimuli from Experiment 1 and measured
ERPs directly at disruption panels.
ERPs recorded to the disruption panels are depicted in Figure 9. A larger
negativity was found to disruption panels that appeared within constituents (Before/After)
than to disruptions that appeared between constituents (At). This negativity localized to
prefrontal and left lateralized locations. This is consistent with the anterior negativities
shown to violations of syntax in sentences (Neville, et al., 1991) and music (Koelsch, et
al., 2005; Patel, et al., 1998). It is important to note that it is only possible to confirm or
17
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
verify that a disruption panel has actually violated a narrative constituent once the
subsequent panel is reached. However, the amplitude to disruption panels Before the
boundary were larger than those At the boundary, prior to confirmation on the
subsequent panel. This suggests that the brain may make online predictions about the
building of constituent structure as a narrative progresses.
Figure 9. Waveforms at left anteriorly distributed (top and left) and posteriorly distributed (bottom)
electrode sites comparing amplitudes of disruption panel placed Before, At, or After the narrative
constituent boundary. Voltage maps illustrate the scalp distribution of these effects at the 500-700 ms time
window.
Additionally, a P600 effect appeared to disruption panels placed After (versus
Before or At) the boundary. Because this positivity appeared to disruptions only after a
18
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
new constituent had been reached, it may reflect the failure of integrating all of the prior
panels into a single constituent. This disruption followed a panel after the constituent
boundary had been reached, meaning that the preceding panel would be unable to be
integrated into a single constituent with the other prior panels, thereby evoking a
reanalysis of starting a new constituent (Kuperberg, 2007). Such an effect would be
analogous to putting a comma one word too far in a sentence, thereby leading to
problems parsing the constituents of the sentence.
General Discussion
The results of the self-paced viewing experiment suggested that the introduction
of a disruption panel into a sequence had a greater impact on viewing times of subsequent
panels if disruptions fell within a narrative constituent (Before/After a narrative
boundary) than between narrative constituents (At a narrative boundary). This provided
evidence that comprehenders use narrative boundaries during the processing of visual
sequences. The results of the ERP experiment support this interpretation, which found a
larger LAN to disruption panels within constituents than between them. Moreover,
because effects were seen on the disruption panel itself, this further suggests that
comprehenders of visual sequences make active predictions about narrative constituent
structures.
Taken together, these results show that disruptions within a narrative constituent
have a greater impact on processing than disruptions between narrative constituents. This
suggests that the comprehension of sequential images draws upon a narrative structure
that is organized into constituents, analogous to grammatical structure in language.
19
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
Concluding Remarks
Overall, I have hypothesized that the structure and processing of visual narratives
is analogous to the structure and processing of sentences. First, this involves an
interaction between two independent components of narrative structure and semantic
coherence, analogous to the interaction between syntactic structure and semantics at the
sentence level. Second, like syntactic structure, this narrative structure hierarchically
combines panels into recursively embedded constituents. The results of the studies in this
dissertation support both of these hypotheses. Such insights are only available given the
rigorous development of a linguistic theoretical model of narrative structure for the
grammar of sequential images. These experiments were designed explicitly using the
constraints and predictions of this theoretical model, thereby supporting the
interdisciplinary importance of combining theory and experimentation to inform and
constrain each other.
Cognition across domains
It is important to stress that, despite this broader analogy, narrative and syntax
differ in several ways. As the saying goes, “a picture is worth a thousand words,” and
indeed images alone contain as much if not more information than a whole sentence.
Thus, it is not likely that the same system of hierarchic embedding applies to both images
and words. Rather, it is more likely that the narrative structure used to organize sequential
images is the same structure that organizes sequences of clauses and sentences into a
narrative in verbal discourse (and film). Nevertheless, three of the primary waveforms
associated with the processing of structure and meaning in sentences have appeared in the
processing of structure and meaning of the visual language used in sequential images (i.e.,
20
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
N400, P600, LAN). While this implies some form of overlap between the processing of
these domains, it does not mean that narrative structure and syntax are the same
structures. Why then do the same neurocognitive responses appear to the comprehension
of sequential images at the narrative level and sequential words at the sentence level?
Rather than postulate that narrative and sentences are built with the same structure,
it seems more feasible that these overlapping ERP components index domain-general
processing that equally applies to sentence-level and narrative-level processing. Such a
conclusion is not novel, especially since P600 effects and a right lateralized anterior
negativity have been found to structural violations in music (Koelsch, 2005; Patel, et al.,
1998). This similarity implies that language, visual narrative, and music all use a
common, domain-general system in processing. Indeed, hierarchic organization not only
appears in syntax, sequential images, and music (Lerdahl & Jackendoff, 1982), but also
in vision (Marr, 1982), individual drawings (Willats, 2005), event structure (Zacks &
Tversky, 2001), and a variety of other domains (Jackendoff, 2011).
The overall picture then is that the human brain uses a processor for organizing
various domains into hierarchic constituents (Corballis, 1991; Jackendoff, 2011). In the
case of certain behaviors, these hierarchic systems (syntax, narrative) organize
expressions in producible modalities (sound, graphics), which can be connected to a
system of semantic meanings (as in sentences and sequential images) or not (as in music).
Thus, not only does the study of comics reveal a “grammar” in visual narratives and the
connection of that system to language, but it can contribute insights to the broader study
of human cognition.
21
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
References
Cohn, N. (2010). The limits of time and transitions: Challenges to theories of sequential
image comprehension. Studies in Comics, 1(1), 127-147.
Cohn, N. (2013). Visual narrative structure. Cognitive Science, 37(3), 413-452. doi:
10.1111/cogs.12016
Cohn, N., Paczynski, M., Jackendoff, R., Holcomb, P. J., & Kuperberg, G. R. (2012).
(Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image
comprehension.
Cognitive
Psychology,
65(1),
1-38.
doi:
10.1016/j.cogpsych.2012.01.003
Corballis, M. (1991). The Lopsided Ape: Evolution of the Generative Mind. Oxford:
Oxford University Press.
Fodor, J., & Bever, T. G. (1965). The psychological reality of linguistic segments.
Journal of Verbal Learning and Verbal Behavior, 4(5), 414-420.
Garrett, M. F., & Bever, T. G. (1974). The Perceptual Segmentation of Sentences. In T. G.
Bever & W. Weksel (Eds.), The Structure and Psychology of Language. The
Hague: Mouton and Co.
Jackendoff, R. (2002). Foundations of Language: Brain, Meaning, Grammar, Evolution.
Oxford: Oxford University Press.
Jackendoff, R. (2011). What is the human language faculty?: Two views. Language,
87(3), 586-624.
Koelsch, S. (2005). Neural substrates of processing syntax and semantics in music.
Current
Opinion
in
Neurobiology,
15(2),
207-212.
doi:
10.1016/j.conb.2005.03.005
Koelsch, S., Gunter, T. C., Wittfoth, M., & Sammler, D. (2005). Interaction between
syntax processing in language and in music: An ERP study. Journal of Cognitive
Neuroscience, 17(10), 1565-1577.
Kuperberg, G. (2007). Neural mechanisms of language comprehension: Challenges to
syntax. Brain Research, 1146, 23-49.
Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in
the N400 component of the Event-Related Brain Potential (ERP). Annual Review
of Psychology, 62(1), 621-647.
Lerdahl, F., & Jackendoff, R. (1982). A Generative Theory of Tonal Music. Cambridge,
MA: MIT Press.
Mandler, J. M., & Johnson, N. S. (1977). Remembrance of things parsed: Story structure
and recall. Cognitive Psychology, 9, 111-151.
Marr, D. (1982). Vision. San Francisco, CA: Freeman.
Marslen-Wilson, W. D., & Tyler, L. K. (1980). The temporal structure of spoken
language understanding. Cognition, 8, 1-71.
McCloud, S. (1993). Understanding Comics: The Invisible Art. New York, NY: Harper
Collins.
Münte, T. F., Matzke, M., & Johannes, S. (1997). Brain activity associated with syntactic
incongruencies in words and psuedo-words. Journal of Cognitive Neuroscience, 9,
318-329.
22
Précis for Structure, Meaning, and Constituency in Visual Narrative Comprehension by Neil Cohn
Neville, H. J., Nicol, J. L., Barss, A., Forster, K. I., & Garrett, M. F. (1991). Syntactically
based sentence processing classes: Evidence from event-related brain potentials.
Journal of Cognitive Neuroscience, 3(2), 151-165.
Osterhout, L., & Nicol, J. L. (1999). On the Distinctiveness, Independence, and Time
Course of the Brain Responses to Syntactic and Semantic Anomalies. Language
and Cognitive Processes, 14(3), 283-317.
Patel, A. D., Gibson, E., Ratner, J., Besson, M., & Holcomb, P. J. (1998). Processing
syntactic relations in language and music: An event-related potential study.
Journal of Cognitive Neuroscience, 10(6), 717-733.
Van Petten, C., & Kutas, M. (1991). Influences of semantic and syntactic context on
open- and closed-class words. Memory and Cognition, 19, 95-112.
Willats, J. (2005). Making Sense of Children's Drawings. Mahwah, NJ: Lawrence
Erlbaum.
Zacks, J. M., & Tversky, B. (2001). Event structure in perception and conception.
Psychological Bulletin, 127(1), 3-21.
23