Contemporary Educational Psychology 35 (2010) 59–74
Contents lists available at ScienceDirect
Contemporary Educational Psychology
journal homepage: www.elsevier.com/locate/cedpsych
Cognitive activities in complex science text and diagrams
Jennifer G. Cromley a,*, Lindsey E. Snyder-Hogan a, Ulana A. Luciw-Dubas b
a
b
Temple University, 1301 Cecil B. Moore Ave., RA201, Philadelphia, PA 19122, United States
National Board of Medical Examiners, 3750 Market St., Philadelphia, PA 19104, United States
a r t i c l e
i n f o
Article history:
Available online 31 October 2009
Keywords:
Comprehension
Diagrams
Strategy use
Knowledge level
Inference
a b s t r a c t
Ainsworth’s (2006) DeFT framework posits that different representations may lead learners to use different strategies. We wanted to investigate whether students use different strategies, and more broadly, different cognitive activities in diagrams vs. in running text. In order to do so, we collected think-aloud
protocol and other measures from 91 beginning biology majors reading an 8-page passage from their
own textbook which included seven complex diagrams. We coded the protocols for a wide range of cognitive activities, including strategy use, inference, background knowledge, vocabulary, and word reading.
Comparisons of verbalizations while reading running text vs. reading diagrams showed that high-level
cognitive activities—inferences and high-level strategy use—were used a higher proportion of the time
when comprehending diagrams compared to when reading text. However, in running text vs. diagrams
participants used a wider range of different individual cognitive activities (e.g., more different types of
inferences). Our results suggest that instructors might consider teaching students how to draw inferences
in both text and diagrams. They also show an interesting paradox that warrants further research—students often skipped over or superficially skimmed diagrams, but when they did read the diagrams they
engaged in more high-level cognitive activity.
! 2010 Elsevier Inc. All rights reserved.
1. Introduction
Science textbooks include a wide range of images, including line
diagrams, naturalistic drawings, flow charts, chemical diagrams,
and hybrid diagrams (e.g., a photograph with a schematic diagram;
Pozzer & Roth, 2003). Comprehending these visual representations
is particularly important for student learning from science texts
(Otero, León, & Graesser, 2002), but what are the cognitive processes involved in comprehending visual vs. textual representations? Ainsworth’s (2006) DeFT (designs, functions, tasks)
framework describes effective student learning from multiple representations such as running text and diagrams. The design aspect
of her framework concerns the number and form of representations in a text or learning environment. The pedagogical functions
aspect of her framework concerns the way that similarities and differences among representations can foster different types of cognitive processes. The tasks aspect concerns the demands that
multiple representations make on learners to abstract from representation, transfer learning from one representation to others, and
to relate between representations. In the current research, we specifically investigate the functions aspect of Ainsworth’s framework
* Corresponding author. Address: Department of Psychological Studies in Education, Temple University, Ritter Annex 201, 1301 Cecil B. Moore Avenue,
Philadelphia, PA 19122-6091, United States. Fax: +1 (215) 204 60134.
E-mail address: [email protected] (J.G. Cromley).
0361-476X/$ - see front matter ! 2010 Elsevier Inc. All rights reserved.
doi:10.1016/j.cedpsych.2009.10.002
by analyzing think-aloud protocols collected from 91 beginning
biology majors reading from a chapter in their course textbook
and coding for cognitive processes verbalized while reading running text vs. cognitive processes verbalized while reading
diagrams.
2. Visual representations and learning from illustrated text
Despite the saying that ‘‘a picture is worth a thousand words,”
research on whether text or diagrams are better for learning has
shown very mixed results—sometimes there is an advantage for
diagrams (Chi, Feltovich, & Glaser, 1981; Kriz & Hegarty, 2007),
while other studies show that students have a great deal of difficulty comprehending diagrams (Bodemer, Ploetzner, Bruchmüller,
& Häcker, 2005; Graesser, Lu, Olde, Cooper-Pye, & Whitten, 2005;
Hannus & Hyönä, 1999; Hegarty & Just, 1993; Mayer, 2005; Paas,
Renkl, & Sweller, 2003). Several recent studies suggest that better
comprehension of the visuals in illustrated text is associated with
better overall comprehension of the text. More proficient learners
spend more time studying visual representations compared to lessproficient learners (Schwonke, Berthold, & Renkl, 2009). Eye tracking studies suggest that better comprehension of diagrams is associated with more time spent in relevant regions of the diagram and
less time spent in irrelevant regions, including seductive details
(Canham & Hegarty, in press; Sanchez & Wiley, 2006). In general,
however, most learners spend the majority of their time in running
60
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
text and little time inspecting diagrams (Schmidt-Weigand,
Kohnert, & Glowalla, in press; Schwonke et al., 2009). The current
literature does not offer evidence about why this is the case,
although we can speculate about a few reasons—learners might
be appropriately adjusting their reading times based on their level
of prior knowledge or diagram comprehension skills, learners
might be focusing on the representation with which they feel most
comfortable, or learners may skim or skip over diagrams because
they do not appreciate—or perhaps do not know about—the role
of and importance of diagrams for comprehending science texts
(Schwonke et al., 2009).
In the present study, we wished to not only test Ainsworth’s
(2006) claim about strategies, but also to expand it to cognitive
activities generally. The range of cognitive activities seen in
think-aloud studies include inference, cognitive and metacognitive
strategies, verbalization of knowledge, vocabulary, and word-reading difficulties (Fox, 2009). We hypothesized that, based on the
studies cited by Ainsworth, diagrams vs. running text might lead
to the use of qualitatively different sets of cognitive activities
broadly (e.g., different types of inference), or might lead differentially to the use of cognitive activities that are more effective for
learning (e.g., more use of elaborative inferences).
3. Ainsworth’s DeFT framework: functions of multiple
representations
4. Diagrams in science text
Ainsworth’s (2006) DeFT framework is a comprehensive synthesis of the literature on learning with multiple representations, with
much of the research conducted in computer-based environments.
This complex framework includes the effects of learning environment design (e.g., written captions vs. narrated captions), functions
of representations (e.g., to complement each other, to constrain
interpretations, or to foster deeper understanding), and the task
demands of multiple representations (building abstractions,
extending knowledge to new representations, and understanding
relations between representations). Multiple representations can
have many functions, and Ainsworth specifically hypothesizes that
‘‘Different forms of representation can encourage learners to use
more or less effective strategies” (p. 188). For example, in her own
research, Ainsworth (Ainsworth & Loizou, 2003) found more monitoring statements expressing understanding in text than in diagrams
and more principle-based explanations in diagrams than in text. We
interpret strategies to be consistent with Samuelstuen and Braten’s
(2007) definition as ‘‘forms of procedural knowledge that students
voluntarily use for acquiring, organizing or transforming information, as well as for reflecting upon and guiding their own learning,
in order to reduce a perceived discrepancy between a desired
outcome and their current state of understanding” (p. 352).
Although some authors have used the term strategies to encompass
inferences, we separate out these two different cognitive activities in
our analyses (for other definitions, see our coding scheme on pp. 17–
18). In the context of the present study, Ainsworth’s (2006) framework would predict that the strategies used in running text would
differ from the strategies used in a diagram such as the one shown
in Fig. 1. Based on the two studies cited by Ainsworth, running text
vs. diagrams might lead to the use of qualitatively different sets of
strategies, or might lead differentially to the use of strategies that
are more effective for learning.
The diagram shown in Fig. 1 is typical of undergraduate biology
text: it includes a stylized representation of microscopic structures—cells and cell parts—and microscopic processes. It features
a lengthy caption, labels that name different parts in the diagram,
arrows, and explanations of various processes. Line diagrams in the
textbook used in the present study had a mean of 4.3 such features
per image, with the most frequent features being captions, naming
labels, explanatory labels, arrows, and color coding. Comprehending a diagram such as the one shown in Fig. 1 requires numerous
cognitive activities, such as activating prior knowledge about
how pathogens are handled by the body; inferential processes such
as noticing similarities between the two halves of the diagram
(both show a T cell reacting to a pathogen and an MHC molecule
bringing a foreign antigen to the cell surface) and noticing differences between the two halves of the diagram (infected cell, cytotoxic T cells, and Class I MHC molecules on the left and
macrophage, helper T cells, and Class II MHC molecules on the
right); knowledge of diagrammatic conventions such as the
caption, use of color, arrows, abbreviations, and lettering and numbering systems; knowledge of specialized scientific vocabulary;
and integration of all of this knowledge and cognitive processes.
Previous research on students’ reasoning with representations
has used simplified images, typically with only one (arrows; Butcher, 2006; naming labels; Berthold & Renkl, 2009; Grosse & Renkl,
2006; Schnotz & Bannert, 2003; numbering; Bodemer & Faust,
2006) or two (arrows and words; Löhner, van Joolingen, Savelsbergh, & van Hout-Wolters, 2005; caption and abbreviations; Carlson, Chandler, & Sweller, 2003; caption and labels; Lewalter 2003;
symbols and labels; Lowe, 2003) of these features. Studies by
Mayer and colleagues on multimedia representations have used
the most complex representations, which include captions, arrows,
and shading (Mayer, 2003; Mayer, Hegarty, Mayer, & Campbell,
2005). In summary, studies have not been conducted on the kinds
Fig. 1. Sample figure (Campbell & Reece, 2001, p. 910 !2001).
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
of highly complex diagrams that students actually encounter as
they try to master science content at the undergraduate level.
In addition to the use of simplified representations in prior research, most studies have been focused on the process of integrating or coordinating information presented in running text with
visual representations (Bartholomé & Bromme, 2009; Florax & Ploetzner, in press; Hegarty, Kriz, & Cate, 2003). However, in order to
understand how readers coordinate these different types of representations, it is critical to understand what cognitive processes are
similar and different across those representations. As Ainsworth
(2006) states ‘‘Learning to use [Multiple External Representations]
requires learners to understand each individual representation.
This is a complex process in its own right” (p. 187). To summarize,
in the present study we are interested in the differences between
cognitive processes used in purely textual representations vs. cognitive processes used in these highly complex visual representations which almost always include verbal information in the
form of captions and/or labels.1
5. Cognitive processes in text and diagrams
A wide range of theory and research suggests that some cognitive processes contribute more to understanding text than others.
There is much evidence that cognitive activities that go beyond a
literal sentence-by-sentence understanding and involve the reader
actively grappling with the text are associated with better comprehension. Bridging and elaborative inferences as well as strategies
such as summarizing, generating and answering questions, and
drawing or constructing concept maps have variously been termed
high-level strategies, deep strategies, deep-processing strategies,
or knowledge-transforming activities (Fox, 2009). By contrast,
other strategies that do not transform knowledge are less strongly
associated with good comprehension. These have been variously
termed low-level, surface, shallow, or text-based strategies and include highlighting, re-reading, and single-sentence paraphrases
(Alexander, Sperl, Buehl, Fives, & Chiu, 2004; Cerdan & Vidal-Abarca, 2008; Meece & Miller, 2001; Weinstein, Husman, & Dierking,
2000).
In addition to strategies, Kintsch’s (1998) construction-integration model posits a key role for inference in text comprehension. In
the CI model, the reader must actively combine prior knowledge
with information from the text in order to move from a textbase
model to build a more sophisticated situation model of the text.
In addition to the critical role of inference, other theories such as
Graesser’s (2007) constructionist theory posit a key role for both
inference and strategic activity—including strategies such as selfquestioning and summarizing. In the constructionist theory, strategies vary depending on reader goals, and some goals may, for
example, foster a great deal of summarizing or ‘‘what-questions.”
High-level strategies and inference may also be important for
students’ understanding of diagrams, which are a prominent
feature of scientific text (Otero et al., 2002). A handful of studies
comparing students’ cognitive processes in text vs. diagrams have
found that students make more inferences in diagrams than in running text (Ainsworth & Loizou, 2003; Butcher, 2006; Moore & Scevak, 1997). These studies suggest that inference, and perhaps
summarizing, occurs more often when students are reading diagrams than when they read running text, even when all students
are trained to self-explain. These studies of cognitive processes
while reading diagrams have all, however, been conducted with
non-science majors reading researcher-developed text. Are high1
Moreno and Valdez (2005) found dramatically lower retention and transfer scores
for undergraduates who were presented with wordless diagrams, on the order of a
33–68% decrement over diagrams with text.
61
level strategies and inference important for comprehension of both
text and diagrams for science majors reading authentic texts?
6. Studies comparing cognitive processes in text vs. diagrams
The superiority of text or diagrams for learning could be due to
a number of factors, one of which is the cognitive processes that
the representation (running text vs. diagram) encourages the learner to use. In order to better understand the role of these cognitive
processes, it is important to gather data about cognitions during
the learning process itself. For example, to the extent that diagrams
lead students to draw more inferences, students may learn better
from diagrams than from text. However, we were only able to locate three studies that directly compared cognitions while learning
from text vs. learning from diagrams (Ainsworth & Loizou, 2003;
Butcher, 2006; Moore & Scevak, 1997).2
Ainsworth and Loizou (2003) trained undergraduate students to
self-explain, and then assigned one-half of their participants to
text-only and one-half to diagram-only conditions. Students in
the diagrams condition verbalized more self-explanations, and also
learned more, compared to students in the text-only condition.
Specifically, these students verbalized significantly more goal-driven or causal self-explanations than text-only students. In addition, whereas students in the diagrams condition verbalized
principle-based explanations, students in the text-only condition
did not While the self-explanation effect is robust, two different
types of reasons have been put forth to explain its effectiveness:
monitoring/correcting knowledge (e.g., Ainsworth & Burcham,
2007; Griffin, Wiley, & Thiede, 2008; Kastens & Liben, 2007), and
inference (e.g., Aleven & Koedinger, 2002; Chi, Bassok, Lewis,
Reimann, & Glaser, 1989). Because Ainsworth and Loizou did not
code for monitoring, they could not address this question of why
self-explanation is effective.
Butcher (2006) trained undergraduate psychology students to
self-explain while learning in one of three conditions: text-only,
text-with-complex-diagram, or text-with-simplified-diagram condition. Learning was measured using change from pretest to posttest in scores on a mental models rubric based on Chi, de Leeuw,
Chiu, and LaVancher (1994). Self-explanation verbalizations were
coded as inferences or other types of self-explanations (paraphrase, elaboration, or monitoring). Participants in the two diagram conditions verbalized significantly more inferences (36%
and 39% of self-explanations, respectively) compared to those in
the text-only condition (22% of self-explanations).
Moore and Scevak (1997) asked middle school and high school
students to think-aloud from illustrated history and science texts.
Students verbalized a higher proportion of main ideas in diagrams
than in text for both the history (69% vs. 49%) and science (88% vs.
54%) passages, and verbalized a smaller proportion of details in
text for the history passage. Moore and Scevak did not code separately for inferences, therefore these codes may include inferences
about details or inferences about main ideas.
In summary, across the three studies reviewed above, untrained
students verbalize more main ideas when reading diagrams compared to text, and trained students verbalize more inferences when
reading diagrams compared to text. All of these studies were conducted with non-science majors reading researcher-developed
texts.
The present study is designed to build our understanding of the
similarities and differences in the cognitive processes used while
making sense of text vs. complex diagrams. In addition to collect2
Note that there is a multitude of studies that collect post-reading measures of
learning from text vs. diagrams or compare the amount of learning from information
presented in different media. In this review, we focus only on studies that collected
data about cognitive processes used while learning.
62
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
ing data from a relatively large sample, we have also coded the
think-aloud transcripts for a wider range of variables than did
researchers in the three studies reviewed above. A secondary goal
is to gather more ecologically valid data about the processes involved in comprehension of both text and complex diagrams by
studying science majors reading from their own course textbook.
We assumed that readers’ cognitive processes would include the
inferences which are prominent in theories of reasoning with
diagrams (Ainsworth, 2006; Larkin & Simon, 1987; Narayanan &
Hegarty, 1998), interactions with prior knowledge that are prominent in several theories (Kintsch, 1998; Sweller, 2005), and both
high-level and low-level strategies frequently recorded in the
think-aloud literature (Fox, 2009). In order to compare the cognitive processes involved in comprehension of complex science text
and diagrams, we collected and analyzed think-aloud protocols
from 91 undergraduate students in a course for biology majors,
who were asked to say everything they were thinking out loud
while reading from a chapter in their own textbook and then asked
to provide a verbal free recall of the content. Our primary research
questions were, for beginning biology students thinking aloud
from their own course textbook:
1. Are there differences in the number and type of cognitive
processes verbalized when reading text vs. diagrams?
2. Are there differences in the mean proportion of use of various
cognitive processes verbalized when reading text vs. diagrams?
3. If so, are differences in the use of various cognitive processes
associated with level of understanding of the text as a whole?
7. Method
7.1. Participants
Participants were 97 undergraduate students enrolled in an
introductory biology course for life sciences majors at a large, moderately-selective urban university in the mid-Atlantic who participated in exchange for extra course credit. Because of equipment
failure, we did not collect audiotape data for six participants; we
therefore report data for 91 participants below. Their mean age
was 19.8 (SD = 2.3); there were 62 women (68%), 28 men (31%),
and one student who did not identify sex (1%). Most of the students
were freshmen (37%) or sophomores (41%), with some juniors
(18%), and two post-baccalaureate students (2%). Participants were
racially diverse (40% White, 23% Black, 31% Asian, and 6% mixed
race or other races). A substantial minority of participants were
first-generation college students; 40% had neither parent with a
bachelor’s degree or higher. Twenty-three percent of participants
had taken AP Biology in high school but had not obtained a high
enough score to waive the Bio 101 course requirement.3 Overall,
this was a relatively high-achieving group of undergraduate students, but they were by no means homogeneous in their knowledge
about the biology topic in the text we presented, or in their SAT
scores or undergraduate GPAs (see below).
7.2. Materials and measures
7.2.1. Student demographics
Students completed a demographics form, reporting their sex,
age, race, parental education, their current GPA, SAT scores, educational and vocational aspirations, number of course credits
3
Participants who had taken AP Biology in high school had virtually identical
background knowledge scores compared to participants who had not taken AP
Biology (M = 3.57, SD = 1.88 and M = 3.53, SD = 1.82, respectively, F [1, 89] = .007,
MSE = 3.377, p = .936).
currently taken, time spent studying for this biology course, hours
of paid work per week, and other major time commitments (e.g.,
parenting, involvement in sports, fraternities, sororities or other
organizations).
7.2.2. Background knowledge
Students completed an untimed, open-ended measure of background knowledge about the immune system. The instructions
were: ‘‘Please write down below everything you know and can
remember about the immune system. Be sure to explain the parts
of the immune system, what purpose each part has, and how the
parts work both separately and together.” We developed a coding
system that captures the complexity of students’ mental models
(see coding below).
7.2.3. Think-aloud protocols
Students produced a 40-min think-aloud protocol while reading
from a passage about the vertebrate immune system from a notyet-read chapter in their own Biology textbook (Campbell & Reece,
2001; see Appendix A). The topic was covered in class 10 days after
data were collected, during the 11th week of the spring semester.
The passage was 3463 words long and had six figures, five of which
were schematic diagrams, and one of which was a flow chart. The
focus of the text was on the two branches of the immune system
which involve white blood cells. We shortened the introductory
material for the chapter, but ensured that definitions of terms used
in the white blood cell sections were retained. With permission
from the publisher, we re-typed the text and scanned the illustrations. In the resulting 8-page text, we positioned the illustrations at
the same point they had been in the original text (i.e., if the illustration preceded the relevant text in the original, we formatted the
passage so that the illustration still preceded the relevant text). The
passage was formatted and printed to resemble the original as
much as possible, including font and text size, page layout, bolding,
italics, and color illustrations. Participants had access to pen, pencil, highlighter, and paper with which to take notes, and were permitted to write on the text if they chose to do so.
The diagrams in the text included five schematic line diagrams,
one flow chart, and one photomicrograph (see Fig. 1 for a sample
diagram; see Table 1 for detailed coding of the diagram features).
With regard to features of the diagrams, every diagram had a caption, and each figure had a median of 10 labels naming specific
parts (range: 2–17), a median of four explanatory labels (range:
0–6), and a median of three arrows (range: 0–22). Taken together,
the caption, naming labels, and explanatory labels included a mean
of 165 words in each figure. These features are typical of diagrams
in the Campbell and Reece (2001) textbook.
7.2.4. Verbal free recall
After students produced the think-aloud protocol, we removed
the text and any notes they had taken, and asked them to say
everything they remembered from what they had just read.
7.3. Equipment
The 40-min think-aloud session and untimed verbal free recall
session were audiotaped on a cassette recorder using a clip-on
microphone. The think-aloud session was also videotaped. In an
effort to ensure anonymity, the video camera was set up to capture
only the participants’ text and note-taking activities.
7.4. Procedure
Data were collected in individual sessions conducted by the first
two authors in spring, 2006. After we obtained participant informed consent, students first completed the demographics form
63
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
Table 1
Features of the diagrams in the think-aloud text.
Topic
Figure number
Type
Caption
Naming Labels
Explan. Labels
Arrows
Words
Clonal selection
The interaction of T cells with MHC
An overview of the immune responses
The central role of helper T cells
The functioning of cytotoxic T cells
The functioning of cytotoxic T cells
Humoral response to a T-dependent antigen
43.6
43.9
43.10
43.11
43.12 (a)
43.12 (b)
43.13
Line diagram
Line diagram
Flow chart
Line diagram
Line diagram
Photo-micrograph
Line diagram
1
1
1
1
1
1
1
6
10
12
11
9
2
17
4
2
4
6
3
0
5
3
0
22
4
3
0
7
178
122
149
167
162
209
178
and written background knowledge measure (untimed). Each
student then completed the think-aloud (40 min) and verbal free
recall (untimed) measures, and was debriefed.
For the think-aloud session, the following instructions to participants were displayed and read out loud (adapted from Azevedo &
Cromley, 2004): ‘‘You are being presented with an abridged passage from your own Biology 101 textbook. We are interested in
learning about how students learn from what they read. I want
you to read this passage as if you were learning the material for
Biology 101. You will have 40 min to learn as much of this material
as you can while studying at your usual pace. You have paper, a
pen and a highlighter for taking notes, if that is what you usually
do when you are studying by yourself from this Biology 101 textbook. However, I will collect them when you are done reading. In
order to understand how you learn from a textbook, I need you
to think out loud while you are reading. Please say everything
you are thinking out loud while you read the text. I will be here
in case anything goes wrong with the tape recorder or video camera, but I cannot answer any questions about the reading or help
you with it. Please remember that it is very important to say everything you are thinking while you are working on this task.” Participants were free to ask questions, which we answered, for example,
if a participant asked ‘‘Should I read aloud?”, we answered ‘‘Yes.”
While reading, participants were prompted to think out loud with
one of three reminders: ‘‘Please say what you are thinking,” ‘‘Do
not forget to read out loud,” or ‘‘Say what you are doing.”4 Using
the criteria from Crain-Thoreson, Lippman, and McClendon-Magnuson (1997), we gave these prompts until participants were verbalizing thoughts at the rate of approximately three utterances per 100
words read. In practice, this meant that we gave reminders every
4–8 sentences if participants were not verbalizing their thoughts.
For the few participants who finished reading the entire passage before the 40 min time limit, we repeated the instructions: ‘‘You will
have 40 min to learn as much of this material as you can.” We
emphasized the instruction ‘‘while studying at your usual pace” in
order to de-emphasize ‘‘getting through” the passage. Other research
with students from this same course suggests that they read at a rate
of approximately 140 wpm, but these students read a mean of 1950
words in 40 min. We conclude that our instructions did indeed
encourage them to emphasize learning over ‘‘getting through” the
passage.
At the end of the 40 min, we removed the instructions, passage,
and any notes taken. We then asked participants to verbally recall
information from the text, with the instruction, ‘‘Please say back
everything you can remember about what you just read” (untimed). When participants finished, they were prompted with the
question, ‘‘Anything else?” If they added any more statements,
they were prompted once more with the same question. After
any further responses, the session was concluded. All of the verbal
recalls took less than 10 min.
4
In the unusual case that students did not read aloud as soon as the session began,
we asked them to ‘‘please read aloud.”
7.5. Data analysis and scoring
Below, we describe the transcription and video coding procedures we applied before coding.
7.5.1. Transcription
Participants’ statements on the open-ended written background
knowledge measure were typed verbatim. Inspection of these written measures suggested that participants began the study knowing
little about the immune system, which is not surprising given that
many had probably last studied biology in 9th or 10th grade of high
school, three or four years previous to our data collection. The mean
number of words per participant was 85.4 (SD = 51.2, with a range
from 8 to 219 words, and 68% in the 33–134 word range). Each
think-aloud session was transcribed verbatim from the audiotape,
segmented, and later coded (see below). This resulted in a total of
1215 typed pages (M = 13.4 pages per participant, range 7–20); each
transcript was then segmented into clauses including a subject and a
verb (as in Chi, 1997). We also counted the last word that the participant read out of the possible 3463 words that the participant could
have read within the 40 min time limit; on average, participants read
about two-thirds of the text (M = 2189 words, range = 822–3463).
Verbal recall protocols were transcribed according to the same
conventions; there was a total of 28,924 words (M = 318 words per
participant, range = 42–1169).
7.5.2. Video coding
The videotapes were then viewed along with the transcripts in
order to code for note-taking, reading notes, and Coordinating Informational Sources (e.g., pointing from text to diagram and then back
to text). We also viewed the time-stamped videotapes in order to
identify the time and the point in the transcript when the participant switched from verbalizing about text alone to verbalizing
about a diagram or verbalizing from his/her notes. Following the
coding for time and representation, the first author then coded verbalizations on all of the transcripts using the coding scheme described below. The third author further coded the transcripts to
note which features of which diagrams were verbally noted by participants. Some examples of coded behavior are: any verbalization
indicating viewing of any part of the diagram at all (e.g., ‘‘OK, looking at the chart”), reading the caption, reading three out of five labels, or verbalizing ‘‘goes to” for an arrow in a diagram.
7.5.3. Coding scheme for the think-aloud protocols
The coding scheme for the think-aloud protocols was adapted
from the Self-Regulated Learning coding scheme developed by
Azevedo and colleagues (Azevedo & Cromley, 2004; Cromley &
Azevedo, 2006), and modified based on previous think-aloud reading studies (Fehrenbach, 1991; Laing & Kamhi, 2002; McNamara,
2001; Neuman, 1990; Robertson, 1990; Zwaan & Brown, 1996).
Major classes of codes in the coding scheme are strategy use, inference, background knowledge, vocabulary, and word reading (see
Appendix B for definitions and examples from reading running text
and diagrams).
64
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
Strategy use was divided into productive strategies and low-level
strategies based on the distinction in the literature between knowledge-transforming and surface-level processes (Pintrich, 2000;
Weinstein et al., 2000). That is, we use productive strategies to refer to moves such as summarizing across several sentences, which
require the reader to restructure the information given in the text.
Productive strategies were further subdivided into high-level, metacognitive, and judging quality of text classes.
High-level strategies are strategies that transform the information in the text and organize the reading session. These include
summarizing,
self-questioning,
coordinating
informational
sources, help seeking, imagery, drawing, taking notes, organizing
notes, reading notes, setting learning goals, and time and effort
planning.
We use the term metacognitive strategies to describe activities
related to monitoring the reader’s own level of understanding,
consistent with what Muis (2008) terms regulation of cognition:
‘‘processes of planning activities prior to engaging in a task, monitoring activities during learning, and checking outcomes against
set goals” (p. 181). These strategies include feeling of knowing,
judgment of learning, monitoring use of strategies, planning, task
difficulty and task ease. Participants frequently combined these
metacognitive strategies with re-reading and other ‘‘fix-up”
strategies.
Judging quality of text was defined as strategies that focus on
the quality of the text itself, such as, ‘‘This is not written well.”
These include codes for adequacy of text, inadequacy of text, adequacy of diagrams, inadequacy of diagrams, and order of text and
diagrams.
Low-level strategies are defined as those that use little restructuring of information from text; they focus on the surface level of
the text, and do not make connections between different sentences
of text—even adjacent sentences—or between prior knowledge and
text. These codes include paraphrasing, re-reading, highlighting,
memorizing, mnemonics, course demands, find location, not thinking, omission of figure, importance of information, and text structure. Importance of information and text structure are included as
low-level strategies because these codes consisted of students
commenting that bolded vocabulary words were important. This
was frequently followed by copying or highlighting the definitions,
indicative of surface-level reading strategies.
We adopt a definition of inference from Edmonds and Pring
(2006): ‘‘to fill in missing information by applying general knowledge, and to make links between different sections of text” (p. 338).
Therefore, we coded as inference all deductions that the reader
makes, but we did not code for anaphoric reference (e.g., stating
what the word ‘‘it” in the current sentence refers to, when that
information was given in a previous sentence). Inferences are
highly effortful, and inference has strongly differentiated students
with higher levels of comprehension from those with lower
comprehension (Cain, Oakhill, Barnes, & Bryant, 2001). These codes
include deductions drawn from information entirely within text
and codes that include both background knowledge and information from text. Vocabulary includes verbalizations related to either
knowing or not knowing or remembering the meaning of words in
the text (as coded by Wade et al., 1990). Word-reading difficulties
include mis-pronouncing a word and immediately self-correcting
the pronunciation or not self-correcting (as coded by Schellings,
Aarnoutse, & van Leeuwe, 2006).
7.5.4. Coding scheme for background knowledge and verbal free recall
protocols
We created a single mental model coding scheme for coding
both the open-ended written background knowledge measure
and the verbal free recall protocols. We coded both measures
using a 9-point mental model coding scheme that captured
increasingly-integrated knowledge. This coding scheme was
developed with guidance from the course instructors, who are
both practicing biologists (see Appendix C for the full coding
scheme). We coded a mean of 4.1 clauses per participant at pretest and a mean of 7.5 clauses at post-test. We did not deduct any
points for incorrect or irrelevant information, misspellings, or
mis-pronunciations in these measures. With regard to misconceptions, 41 participants verbalized at least one misconception at
pretest (a mean of 1.2 misconceptions per person), and 30 participants verbalized at least one misconception at post-test (a mean
of 1.4 misconceptions per person), however these misconceptions
were not the same at pre- and post-test except for one
participant.5
7.5.5. Coding participant notes
We typed up participants’ notes, whether written on separate
notepaper or on the text itself. The mean number of words per participant was 183 (SD = 121, range 0–470). Twenty-six participants
took either no notes at all, or wrote less than 10 words of notes. For
the 65 participants who did take notes, we counted strings of 4 or
more words in a row that were taken directly from the text as verbatim note-taking. The mean proportion of verbatim notes per participant was 29% (SD = 22%, range 0–91%).
7.6. Inter-rater agreement
For all measures, the first author coded all of the data and the
second author—a graduate student in school psychology—served
as the second coder. She was trained on data other than those used
to calculate inter-rater reliability and then re-coded 35% of each
corpus. In all cases, both raters were blind to scores on all other
measures; after re-coding all differences were resolved by discussion. For the prior knowledge measure, the two coders agreed on
30 out of 32 mental models, yielding an inter-rater agreement of
94%. For the think-aloud protocols, we agreed on 3628 out of the
3891 final codes, yielding an inter-rater agreement of 93%. For
the final mental model (verbal free recall protocol) coding, we
agreed on the scores for 29 out of the 32 protocols, yielding an inter-rater agreement of 91%.
7.6.1. Data analysis
First, we converted the raw number of verbalizations within
each representation to a proportion for each participant, in order
to control for differences in amounts verbalized (these ranged from
26 to 235 coded verbalizations per participant in running text and
from 0 to 60 coded verbalizations per participant in diagrams). For
mis-readings of words, we used the last word read in the text as
the denominator for this proportion (i.e., a participant who misread seven words and read up to the 1473rd word mis-read .005
of words read). We note that participants almost never skipped
more than one paragraph of the text, even though they frequently
skipped diagrams; only two of the 91 participants used a skimming
approach in text.
Proportion data for verbalizations are not typically normally
distributed, and include students who never verbalized the
variable of interest (e.g., 0 inferences). We therefore used the
Wilcoxon matched pairs signed ranks test for comparisons between representations; this test is the non-parametric equivalent
of a repeated-measures t test. We interpreted the results of
these tests to be statistically significant at an alpha level of
p < .05.
5
Analyses using our original rubric score minus a correction for number of
misconceptions yielded exactly the same pattern of results.
65
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
8. Results
8.1. Descriptive statistics
Below, we describe scores on the pre- and post-test measures,
time spent in text vs. diagrams and diagram features verbalized,
and raw counts and mean proportions of use for each think-aloud
code.
8.1.1. Scores on pre- and post-test measures
Scores on the written background knowledge measure had a
mean of 3.54 (out of nine) and a standard deviation of 1.82; scores
on the final mental model had a mean of 5.64 (out of nine) and a
standard deviation of 1.94. In order to rule out the possibility that
higher mental models might simply be due to reading faster or
farther in the text, we computed a correlation between how far
students read in text and their final mental model scores; this
was non-significant (r [89] = !.07, p = .26). A dependent-samples
t-test showed a significant increase from the written background
knowledge measure score (M = 3.54, SD = 1.83) to the final mental
model score (M = 5.64, SD = 1.94), t (90) = 8.60, p < .001, d = 1.15,
suggesting that participants did comprehend the text well enough
to learn from it.
8.1.2. Time spent in text vs. diagrams
On average, participants spent 32 min 32 s in text and 7 min 28
s in diagrams, but amount of time spent in diagrams was not
significantly correlated with post-test mental model scores. With
regard to viewing diagrams, participants on average verbalized
something about 78% of the possible images (that is, images they
could have viewed given how far they read in the excerpt). In the
images that were viewed, participants made verbalizations about
52% of the captions, 36% of the naming labels, 61% of the explanatory labels, 13% of the process arrows, and 7% of the symbols.
8.1.3. Think-aloud codes
The raw counts for each think-aloud code, as well as mean percentage of use for each code (the frequency of use for each code for
each participant divided by the total number of verbalizations for
that participant within each representation) are shown in Table 2.
8.1.4. Data to support the validity of the background knowledge and
free recall scores
We present two types of evidence to support the validity of the
background knowledge and free recall scores: correlations of these
variables with background knowledge verbalized during the thinkalouds, and differences among students with and without prior
biology coursework. The non-parametric Spearman correlation between scores on the background knowledge measure and proportion of verbalizations of background knowledge was rs [89] = .28,
the correlation between scores on the background knowledge
measure and final mental model scores was rs [89] = .21. Juniors
and post-baccalaureate students had significantly higher background knowledge scores (F [3, 85] = 2.964, MSE = 3.040, p = .037)
Table 2
Frequency and percentage of coded student verbalizations for each variable.
Coded variable
In text
Total number of
occurrences
In diagrams
Mean percentage of
use within rep. (%)
Total number of
occurrences
Mean percentage of
use within rep. (%)
Background knowledge
Prior knowledge activation+
Prior knowledge activation-
192
62
2.0
0.6
62
33
2.7
1.4
Inference
Inference+
Knowledge Elaboration+
Other (see text for codes)
90
187
92
0.5
1.8
1.0
39
62
29
1.7
3.8
1.3
Productive strategies
High-level strategies
Coordinating info. sources
Organizing notes
Reading notes
Summarizing+
Taking notes
Other (see text for codes)
73
182
122
563
1361
231
0.7
1.5
1.1
5.7
14.3
2.4
119
11
42
328
97
72
6.6
0.6
2.0
16.8
5.6
4.3
Metacognitive strategies
Feeling of knowing
Judgment of Learning
Monitoring use of strategy
Other (see text for codes)
397
395
123
113
4.1
4.0
1.2
1.1
95
124
16
11
0.1
6.1
0.9
0.5
Judging text
Inadequacy of text
Other (see text for codes)
127
114
1.2
1.2
15
121
1.7
4.0%
Low-level strategies
Highlights text
Importance of information
Omission of figure
Rereads text
Other (see text for codes)
872
189
84
1737
572
8.8
1.8
1.0
18.2
6.1
74
13
8
192
115
3.6%
0.6%
1.5%
10.2%
5.5%
658
6.8
145
6.1%
1073
12.4
90
5.6%
Vocabulary
Vocabulary
Word reading
Self-corrects word
Note: codes are defined in Appendix B.
66
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
than freshman and sophomore students, and juniors and post-baccalaureate students were more likely to have taken a prior collegelevel biology course (89% vs. 33%).
8.1.5. Research question 1: are there differences in the number and
type of cognitive processes verbalized when reading text vs. diagrams?
While reading text, participants tended to verbalize more overall (9670 vs. 1913 coded utterances), consistent with the greater
amount of time spent in text vs. diagrams. Across all classes of
codes, a significantly larger proportion of participants verbalized
at least one coded variable from that class when reading text than
when reading diagrams (see Table 3 for details and results of statistical tests). For example, in running text 67 students (74% of participants) verbalized at least one inference, but in diagrams 47
students (53% of participants) verbalized at least one inference.
All of these proportions are significantly different by a z test for difference between two proportions.
In addition, the types of cognitive processes within a class were
significantly more varied in text than in diagrams (see Table 3 for
details and results of statistical tests). For example, in running text
participants verbalized a mean of 4.20 different high-level strategies (out of a total of 10 coded types), but in diagrams they verbalized a mean of 2.67 different types. All of these means are
significantly different by non-parametric repeated-measures tests
(Wilcoxon matched pairs signed ranks test). Diagrams provoked
significantly less frequent use of various cognitive processes and
a significantly smaller number of cognitive processes per participant compared to running text.
8.1.6. Research question 2: are there differences in the mean
proportion of use of various cognitive processes verbalized when
reading text vs. diagrams?
We conducted a series of non-parametric repeated-measures
tests (Wilcoxon matched pairs signed ranks test) on proportion
of verbalization of each think-aloud variable with representation
as the within-subjects variable. We show the mean percentage of
use of each variable by representation in Table 4.
The Wilcoxon test showed a significant difference between
representations (text vs. diagrams) in proportion of inferences (T
[N = 91] = 2.594, p < .001). Participants verbalized a significantly
higher proportion of inferences in diagrams (7%) than they did in
text (4%). There was a significant difference between representations in proportion of high-level strategies (T [N = 91] = 2.977,
p = .003). Participants verbalized a higher proportion of high-level
strategies in diagrams (34%) compared to running text (26%; see
Fig. 2). There was a significant difference between representations
in proportion of judging text strategies (T [N = 91] = 5.419, p < .001;
8% in diagrams vs. 3% in running text), but no difference in metacognitive strategies (T [N = 91] = 1.476, p = .140; 12% in diagrams
vs. 10% in running text).
The Wilcoxon test showed a significant difference between representations in proportion of low-level strategies (T
[N = 91] = 5.448, p < .001). Participants verbalized a significantly
higher proportion of low-level strategies in text (36%) than they
did in diagrams (21%). There was no significant difference between
representations in proportion of verbalizations of background
knowledge (T [N = 91] = 1.668, p = .095). Participants verbalized
roughly the same proportion of verbalizations of background
knowledge in text (3%) as they did in diagrams (5%). There was a
significant difference between representations in proportion of
vocabulary difficulty (T [N = 91] = 2.197, p = .028). Participants verbalized a significantly higher proportion of vocabulary difficulty in
text (7%) than they did in diagrams (6%). There was also a significant difference between representations in proportion of wordreading difficulties (T [N = 91] = 5.681, p < .001; 5% in diagrams
vs. 12% in running text).
8.1.7. Research question 3: are differences in the use of various
cognitive processes associated with level of understanding of the text
as a whole?
The results above suggest that—compared to purely textual representations—diagrams seem to provoke the use of significantly
fewer and less varied cognitive activities, but a significantly higher
mean proportion of these activities are inferences and high-level
strategies, while a significantly lower mean proportion of these
activities are low-level strategies and vocabulary difficulty.
How are these differences in diagram vs. text processing related
to passage comprehension, as reflected in mental model scores? A
series of non-parametric correlations between mental model
scores and proportion of verbalization for each of the various classes of cognitive processes in text showed a significant relationship
for inferences (rs [89] = .28, p = .007) and for verbalizations of background knowledge (rs [89] = .36, p = .002). Participants who verbalized a higher proportion of inferences and background knowledge
when reading text tended to have higher mental model scores. The
same analyses for diagrams showed a significant relationship for
inferences (rs [87] = .22, p = .037). Participants who verbalized a
Table 3
Descriptives on percentages by representation and results of non-parametric repeated-measures t tests on cognitive activities.
Representation
Percentage of inferences
Percentage of high-level strategies
Percentage of judging text strategies
Percentage of metacognitive strategies
Percentage of low-level strategies
Percentage of verbalization of background knowledge
Percentage of verbalizations of vocabulary difficulty
Percentage of verbalizations of word reading difficulty
Note: df for all analyses is 90.
Text
Diagrams
3.6%
(3.6%)
26.3%
(16.0%)
2.5%
(2.7)
10.1%
(6.6)
35.9%
(16.4%)
2.6%
(3.6%)
6.8%
(5.0%)
12.4%
(10.7%)
6.5%
(12.0%)
33.5%
(18.9%)
7.7%
(12.6)
12.4%
(12.0)
21.2%
(20.1%)
4.5%
(8.6%)
6.4%
(8.4%)
5.5%
(8.9%)
T
p
2.594
<.001
2.977
.003
5.419
<.001
1.476
.140
5.448
<.001
1.668
.095
2.197
.028
5.681
<.001
67
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
Table 4
Number of students verbalizing at least one cognitive process and number of different types of processes, by representation.
Representation
z
Text
Inference
Number (%) of students verbalizing at least one
Mean (SD) number of different types per student
High-level strategy
Number (%) of students verbalizing at least one
Mean (SD) number of different types
Judging text strategy
Number (%) of students verbalizing at least one
Mean (SD) number of different types
Metacognitive strategy
Number (%) of students verbalizing at least one
Mean (SD) number of different types
Low-level strategy
Number (%) of students verbalizing at least one
Mean (SD) number of different types
Fact from background knowledge
Number (%) of students verbalizing at least one
Mean (SD) number of different types
Vocabulary difficulty
Number (%) of students verbalizing at least once
Word reading difficulty
Number (%) of students verbalizing at least once
p
Diagrams
67
(74%)
1.72
(1.34)
47
(53%)
2.92
.002
.90
(1.03)
5.49
<.001
89
(99%)
4.20
(1.89)
82
(93%)
2.67
(1.34)
2.06
.022
5.93
<.001
65
(72%)
1.32
(1.12)
59
(67%)
0.73
.233
.97
(.90)
2.81
<.001
86
(96%)
2.70
(1.12)
69
(78%)
1.38
(1.00)
3.60
<.001
6.26
<.001
91
(100%)
4.56
1.64)
73
(83%)
1.53
(1.17)
4.11
<.001
7.67
<.001
3.25
<.001
3.53
.001
62
(69%)
.97
(.77)
40
(45%)
72
(80%)
51
(58%)
3.19
<.001
88
(98%)
46
(52%)
7.15
<.001
.59
(.74)
Note: df for all analyses is 89.
scores (r [89] = .32, p < .001). The same analyses for diagrams
showed that the wider the variety of inference types verbalized
in diagrams, the higher were mental model scores (r [89] = .22,
p < .001). Students who verbalized a larger proportion and greater
variety of inference types in both text and diagrams showed better
comprehension.
9. Discussion
Fig. 2. Mean proportion of verbalizations by representation.
higher proportion of inferences when reading diagrams tended to
have higher mental model scores.
With regard to variability in the number of cognitive activities,
Pearson correlations between the number of cognitive processes
used within each class of cognitive processes in text and post-test
mental model scores showed that the wider the variety of inference types verbalized in running text, the higher were mental
model scores (r [89] = .35, p < .001) and the wider the variety in
activation of background knowledge (i.e., activation of both accurate and inaccurate knowledge), the higher were mental model
It appears from the results of our think-aloud protocol analysis
that when reading diagrams, students engage in significantly more
inference and high-level strategies and show significantly less use
of low-level strategies and vocabulary difficulty, compared to
when reading running text. Results of the think-aloud protocols
and other measures also suggest that both variability in inference
types and greater use of inference is as prevalent in the process
of comprehending diagrams as in comprehending scientific text.
Participants did learn from the passage—their mental model
scores increased significantly from pretest to post-test—but overall
low levels of inference in running text and diagrams were associated with increases that were only of modest size, consistent with
results from much prior research with non-science undergraduates
learning from scientific text (Otero et al., 2002). Overall, the most
frequently used cognitive processes included some that have been
found to be effective in prior research—taking notes, summarizing,
metacognitive monitoring—and some that have been found to be
68
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
relatively ineffective—highlighting, re-reading, expressions of
vocabulary difficulty, and mis-readings of words (Pintrich, 2000).
Overall, there was a large negative correlation between proportion of low-level strategies and proportion of productive strategies
(r [89] = !.85, p < .05). That is, there seemed to be a trade-off between the low-level strategies and more knowledge-transforming
strategies such as summarizing, self-questioning, and coordinating
text and diagrams. Together with our results for inference, these
results are strongly consistent with a ‘‘knowledge transforming,”
‘‘knowledge building,” or ‘‘generative learning” perspective (Bereiter & Scardamalia, 2006; Linn, 2006; Wittrock, 1990). Readers need
to internalize the material, connect it with what they know, summarize in their own words across sentences, and actively draw
conclusions between information in order to effectively learn from
text about complex topics such as this one. Being a ‘‘good strategy
user” (Pressley & Harris, 2006) is not enough for these undergraduate majors to learn from this challenging scientific text with
diagrams. We note, however, that researchers who categorize
inferences as a strategy—rather than coding these separately—
might consider our results to show that a subset of strategies is
important for learning from this text.
Contrary to prior research, time spent in text vs. diagrams was
not related to final mental model scores (Schwonke et al., 2009).
One difference between our study and other research that has related time in representations to learning outcomes, is that we used
a relatively long passage with multiple diagrams, a large amount of
running text, and a 40 min learning period. Participants might
make different decisions about relative time in text and diagrams
in this situation than in a short learning session with one diagram
and a small amount of running text. Consistent with prior research,
participants verbalized about relatively few features of the diagrams (34% of possible features), and entirely skipped a mean of
22% of the figures (Schmidt-Weigand et al., in press), even though
only two of the 91 participants skipped more than one paragraph
of text. It is possible that students perceived there to be a tradeoff between reading text and reading diagrams because of the
40 min time limit imposed (despite our repeated instructions to
‘‘study at your usual pace”). However, if students did perceive
there to be a tradeoff, they almost unanimously chose to skip diagrams, and that this in and of itself is useful information about students’ approaches to reading scientific text. Because reading
diagrams is associated with significantly more inference and
high-level strategy use, together with significantly less low-level
strategy use and vocabulary difficulty, students may be depriving
themselves of opportunities to learn science content when they
skip or merely skim diagrams in biology text.
With regard to the number and type of cognitive activities in text
vs. diagrams, text may foster use of a wider variety of types of cognitive activities (e.g., more students who verbalized at least one inference). Text may also foster a wider variety of specific activities
within types (e.g., verbalizing a larger number of different inference
codes, such as accurate and inaccurate knowledge elaborations and
bridging inferences). In our data, we found this association across all
types of cognitive processes—inference, high-level strategies, judging text strategies, metacognitive strategies, activating facts from
background knowledge, vocabulary difficulties, and word-reading
difficulties. It seems that on average there is a reduced repertoire
when students look at diagrams, even though all codes were used
by at least one student in diagrams as shown in Table 2 (that is, these
are not ‘‘nonexistent” codes in diagrams). With regard to Ainsworth’s (2006) framework, this finding suggests that diagrams do
not lead to the use of qualitatively different sets of strategies. We
speculate that students may not be as proficient at comprehending
diagrams as they are at comprehending running text, so in diagrams
they do not use the cognitive activities that we know they possess
because they used them in running text.
With regard to the relative use of various cognitive activities,
diagram reading was associated with a significantly higher proportion of inferences (7% vs. 4% in running text), high-level strategies
(34% vs. 26%), and judging text strategies (8% vs. 3%), while running
text was associated with a higher proportion of low-level strategies
(36% vs. 21% in diagrams), vocabulary difficulty (7% vs. 6%), and
word reading difficulty (12% vs. 6%). There were no significant differences for metacognitive strategies or verbalization of background knowledge.
These finding are consistent with prior findings regarding inference from Ainsworth and Loizou (2003) and Butcher (2006) and
with prior findings for high-level strategies from Moore and Scevak
(1997). Our findings regarding judging text strategies, low-level
strategies, metacognitive strategies, verbalization of background
knowledge, and vocabulary and word reading difficulty have not
been researched previously. Specifically, we believe that the superior learning from self-explanation in diagrams found by Ainsworth and Loizou (2003) might be explained by the larger
amount of inference associated with looking at diagrams rather
than by a metacognitive explanation, since we found non-significant differences for metacognitive strategies and significant differences for inference.
How are these cognitive activities related to learning from the
text as a whole? Our correlational results suggest that when reading text, verbalizations of a higher proportion of use of inference
and background knowledge is associated with better free recall
(as is verbalization of a wider variety of inferences). When reading
diagrams, verbalizations of a higher proportion of use of inference
and a wider variety of inferences is associated with better free recall. These results are consistent with a large number of thinkaloud (Fox, 2009) and experimental (National Reading Panel,
2000) studies. Our study shows how diagram use is associated
with a higher proportion of and greater variability in (when compared to running text) those very cognitive activities that are associated with increased comprehension. However, the finding that
proportions of use of productive and low-level strategies are unrelated to free recall may come as a surprise. We speculate that there
may be distinct sub-types of readers who use different combinations of inference, low-, and high-level strategies within our sample (Alexander et al., 2004), leading to overall non-significant
results for strategy use.
9.1. Limitations
This study had certain limitations that limit the generalizability
of the findings. First, these results are correlational; better learning
from diagrams could be due to either a bi-directional relationship
or an unmeasured variable that is associated with more reading of
diagrams, rather than to the cognitive processes verbalized when
participants try to learn from diagrams.
Second, we took a highly domain-specific approach in this
study—we analyzed the relationship of biology background knowledge to comprehension of biology text rather than the relationship
of general background knowledge to general reading comprehension. We thereby sacrificed some generalizability in order to tap
domain-specific processes. While think-aloud reports have been
extremely useful in reading research, participants may not verbalize everything they are thinking (e.g., verbalizing an understanding
of vocabulary), even when prompted. Furthermore, students do not
‘‘usually study” by reading aloud, so the requirement to read
aloud—in order to understand what specific part of the passage
students were verbalizing about and to track re-reading—makes
this study less ecologically valid.
Using a lengthy naturalistic text means that we were not able to
analyze the relationship between text segments and verbalizations
at a fine-grained level. Using open-ended written background
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
knowledge and verbal free recall measures probably resulted in
weaker relationships than would have been shown by using the
same response format (oral or written) on both measures and also
using closed-ended measures. In addition, we only used one, nonstandardized measure of immediate comprehension, and we did
not include a filler task. We should confirm the results of our correlational analyses of think-aloud data with experimental studies.
We only considered one university and one course (albeit one with
a very diverse student population) and only one text. Furthermore,
undergraduate biology majors are a self-selected group—they succeeded in entering college, have ambitious career goals, had already passed college-level chemistry and math courses, and were
concurrently enrolled in second-semester chemistry and math
courses as well as this biology course. Our results may therefore
be university-, course-, and/or text-specific.
10. Conclusion
With regard to representations, our findings for inference and
productive strategies with untrained students were consistent
with the handful of studies that have compared verbalizations
for students trained to self-explain in text and in diagrams (Ainsworth & Loizou, 2003; Butcher, 2006). Overall, students used a significantly higher proportion of inferences and high-level strategies
and a significantly lower proportion of low-level strategies in diagrams than in text. This results in an interesting paradox—on the
one hand, students often skipped diagrams (as found by
Schmidt-Weigand et al., in press), skimmed only a few elements
in the diagrams, or complained about how difficult or useless they
were, yet diagrams seemed to promote more high-level, integrative activity and seemed to discourage low-level superficial strategies. One possible explanation is that diagrams do encourage more
integrative activity and better learning, but students simply do not
enjoy being made to work hard and therefore skip or skim the diagrams, even though they are very useful. Another possible explanation which we cannot test in the present study is that students skip
or skim diagrams because they have a low self-efficacy for understanding diagrams—they feel they cannot understand diagrams, so
they skip over them. A third possibility is either a bi-directional
relationship between diagram reading and inference, or a third
variable such as print exposure that affects both diagram reading
and inference.
Diagrams have not always been found to promote the most
effective learning for all students (Mayer & Sims, 1994). Researchers have found effects of learner characteristics (e.g., spatial ability), diagram features (e.g., cross-sections vs. other types of
diagrams), and learning tasks (e.g., effects on factual knowledge
vs. conceptual knowledge)—as well as interactions among these
learner characteristics—on using diagrams (Sanchez & Wiley,
2006). In our sample, with these diagrams, and the task to read
‘‘as if you were learning the material for Biology 101,” it seems that
students were able to engage in some of these integrative activities
and this was associated with better understanding of the topic.
Perhaps previous studies that have found worse learning from diagrams have studied participants with very little knowledge of the
topic and/or poor strategic knowledge. This might have led to
low levels of inference and productive strategy use in diagrams,
and therefore worse learning from diagrams compared to text. Another possibility that we cannot test in this study is that diagrams
use certain conventions (e.g., an arrow can symbolize motion,
change, or enlargement; Heiser & Tversky, 2006; Sweller, 2005),
and students need a certain minimum amount of knowledge of
these conventions before diagrams will lead to higher levels of
inference and productive strategy use. A third possibility is that
there is either a bi-directional relationship or an unmeasured var-
69
iable that is associated with both more inspection of diagrams and
better learning, but that better learning is not due to cognitive
activities used while learning from diagrams.
Our findings in the present study suggest that the processes
found with non-science majors reading both non-scientific and scientific text (Otero et al., 2002) hold for science majors reading scientific text (Ozuru, Dempsey, & McNamara, 2009), suggesting
some generality of reading comprehension processes. All readers
need to engage in a variety of inferential processes in order to comprehend both text and diagrams. Even though inferences do not
make up a very large proportion of total verbalizations (7% of verbalizations in diagrams), they are strongly associated with comprehension of both text and diagrams.
10.1. Implications for theories of diagrammatic reasoning
Overall, our findings are consistent with the theoretical importance of background knowledge for comprehension of diagrams.
They are also consistent with Ainsworth’s (2006) proposal that
different representations lead to differential use of cognitive activities, but we have evidence to broaden her claim beyond high-level
strategies to include inference and low-level strategies. In addition,
our findings shed light on self-explanation as it relates to diagrams.
Our findings support the inferential account of self-explanation
(Aleven & Koedinger, 2002) over the metacognitive account (Griffin
et al., 2008). That is, we believe that self-explanation is beneficial
for diagram comprehension because it encourages students to
draw inferences.
10.2. Implications for instruction with illustrated texts
Our results suggest that instructors can and should encourage
students to thoroughly read and process diagrams in scientific text,
as a way of encouraging inference, high-level strategies, and activation of background knowledge and thereby increase learning.
Instructors may not be aware that many students simply skip over
many of the diagrams, and only superficially skim those diagrams
that they do inspect. Since so few studies have tested methods of
teaching students to understand diagrams, there is no basis for recommending a specific type of intervention. Those designing
instruction might consider building on existing instructional interventions to foster inference, high-level strategies, and activation of
background knowledge as part of diagram instruction. We are currently conducting research on classroom interventions at the high
school level, specifically comparing instruction in conventions of
diagrams to instruction in coordinating text and diagrams.
10.3. Implications for future research
In addition to further research on interventions to improve reasoning with diagrams, we see the need for more basic research on
the processes underlying diagram comprehension. Future research
should triangulate results by using more than one measure of
learning processes—such as eye tracking, log files or other trace
data, reaction time data, think-aloud protocols, and/or psychophysiological data. Researchers could also compare reasoning with
diagrams in different domains—what are the differences between
typical graphical representations in chemistry and biology, for
example? How do visual representations in different domains
make different demands on learners—do representations in some
domains demand more or less spatial ability or working memory
skills? Are there subskills such as comprehending three-dimensional visualizations that are more necessary for some domains
(e.g., cutaway diagrams in geoscience) than others (e.g., free body
diagrams in physics)?
70
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
The process of coordinating textual information and diagrammatic information is also under-specified in research. Studies,
including ours, tend to track the number of times learners move
from textual to visual information and occasionally relate this to
individual differences, but rarely analyze the cognitive processes
involved in such coordination. Which cognitive processes precede
a switch from text to diagrams and vice versa? Which processes
are used to connect textual and visual information—is this primarily a process of matching? of analogy? of inference? Are there different patterns of activity between text and diagrams that
characterize participants with different scores on individual difference variables such as knowledge, spatial ability, or working
memory?
Acknowledgments
This study was partially funded by a Return on Indirect Research Incentive Grant from Temple University to the first author.
A previous version of this manuscript was presented at the 2007
annual meeting of the American Educational Research Association.
The authors wish to thank Todd Mendelssohn for assistance with
transcribing and video coding, and James P. Byrnes for comments
on an earlier version of the article.
Class code
Appendix A
A.1. Excerpt from the think-aloud text (adapted from Campbell &
Reece, 2001, p. 906)
The selective proliferation and differentiation of lymphocytes
that occurs the first time the body is exposed to an antigen is the
primary immune response. In the primary response, about 10–
17 days are required from the initial exposure to antigen for selected lymphocytes to generate the maximum effector cell response. During this period, selected B cells and T cells generate
antibody-producing effector B cells, called plasma cells, and effector T cells, respectively. Eventually, symptoms of illness diminish
and disappear as antibodies and effector T cells clear the antigen
from the body. If that individual is exposed to the same antigen
at some later time, the response is faster (only 2–7 days), of greater
magnitude, and more prolonged. This is the secondary immune response. The immune system’s capacity to generate secondary immune responses is called immunological memory.
Appendix B
B.1. Classes, descriptions, and examples of the variables used to code
think-aloud protocols
Definition
Background knowledge
Prior knowledge Recalls specific correct information from memory or from previous
activation
text read during the TA session
accurate
Prior knowledge Recalls specific incorrect information from memory or from
activation
previous text read during the TA session
inaccurate
Inference
Hypothesis
Inference
accurate
Inference
inaccurate
Makes a prediction or hypothesis about how something works or
what will come up next in the text.
Accurately draws a conclusion from current text + current text
within one paragraph
Inaccurately draws a conclusion from current text + current text
within one paragraph
Example (T = text, D = diagram)
‘‘So meaning that a single antibody will bind to a single
antiagent and not to many others.” (D)
‘‘Secrete cytokines which are going to go and attempt to kill
bacteria or the virus or whatever the harmful this is in
there.” (T)
‘‘I bet that’s why you never get the same cold twice” (T)
‘‘So it’s going to stimulate antibodies and it’s going to
stimulate cytotoxic t cells.” (D)
‘‘Do these MHC molecules bind to antigens? I’m guessing
they’re probably and they combat and produce antibodies.”
(T)
‘‘It shows the infected cell and the antigen fragments that
are left over.” (D)
Knowledge
elaboration
accurate
Accurately draws a conclusion from prior knowledge + current text
Knowledge
elaboration
inaccurate
Inaccurately draws a conclusion from prior knowledge + current
text
‘‘And AIDS I think would be a cell mediated because it has to
do with the t cells.” (D)
Puts text together with diagram (e.g., reads text, then finds part on
diagram) or puts diagram together with text (e.g., flips back and
forth between diagram and text)
Makes a drawing or diagram/flow chart. Can be coded from video
States a help-seeking strategy that he/she would use, e.g., looking
up a word in a dictionary, asking a peer or instructor
Participant forms an internal mental image
Reads about antigen-presenting cells, then looks at diagram
‘‘there it is” (T)
Strategies
High level
Coordinating
Informational
Sources
Drawing
Help-seeking
strategy
Imagery
Organizing
notes
Talks about organizing notes, using outline, adding notes to an
earlier section, using highlighting, asterisks/stars, drawing arrow
from one section to another, making a table that contrasts two
concepts
‘‘I’m going to copy this diagram” (D)
‘‘I would look that up in a dictionary” (T)
‘‘You have my cell, you have my fragment, and how it goeshow it presents it really, it says it carries it to the cell
surface, so I am trying to picture it. . .” (D)
‘‘I’m writing this in sort of an outline form” (T)
71
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
Appendix B (continued)
Class code
Definition
Example (T = text, D = diagram)
Reads notes
Reads own notes or shows evidence of having read (e.g., ‘‘Yes, I do
have that in my notes”)
States a curiosity question that might or might not be answerable
by later text
Accurately re-states what was read/diagram in own words across
two or more sentences (not simply paraphrasing one sentence or
re-reading). Note that a sentence can be split into accurate (SUM+)
and inaccurate (SUM!) portions
Writes notes on paper or on the text directly
‘‘What I usually do now is read over my notes” (T)
Self-questioning
Summarizing
accurately
Takes notes
Time and effort
planning
Metacognitive
Feeling of
knowing
Judgment of
learning
Monitor use of
strategy
Planning
Task difficulty
Task ease
Judging text
Adequacy of
diagram
Adequacy of
text
Inadequacy of
diagram
Inadequacy of
text
Order of text
and diagrams
Notices how much time is remaining and/or has passed
States that he/she understands, or that information is familiar (i.e.,
matching new information to information in memory)
States that he/she does not understand
Mnemonic
Not thinking
Rereads text
Paraphrasing
accurately
Paraphrasing
inaccurately
Summarizing
inaccurately
‘‘And antigen first exposure, and its engulfed by the macrophagz, and it stimulates helpers T cells, and then it stimulates
the B and cytotoxic cell. . .” (D)
‘‘I have to go back and take notes. . . . large number of effector
cells” (D)
‘‘I’ve been working for 14 min; I have 26 min left” (T)
‘‘OK, that makes sense” (T)
Comments on the usefulness (or uselessness) or effect on easiness
of a strategy such as taking notes, COIS, etc
Plans out how to approach learning, including skimming/flipping
through text to plan the comprehension session. States any two
strategies to be enacted in an order
States that the task is difficult
‘‘. . .trying to figure out what’s going on here in this
diagram. . .don’t understand,. . .” (D)
‘‘don’t need to write a definition because if I already know
what primary immune [sic] is” (D)
‘‘I’m just writing down, making the topic into questions
actually, then as I read I’m going to try to answer these
questions” (T)
‘‘uh, this figure is kind of hard to memorize” (D)
States that the task is easy, not difficult
‘‘It’s not as complicated as I thought it would be” (T)
States that a diagram is helpful, useful for comprehension, or
‘‘good” or states a preference for pictures
States that text is informative or upcoming text will be informative
‘‘I like learning from pictures a lot better than reading” (T)
States that a diagram is not helpful, not useful for comprehension,
‘‘bad” or ‘‘disliked,” ‘‘confusing,” or participant ‘‘doesn’t learn from
diagrams.”
States that text is confusing, unclear, or could have been stated
better
Talks about the order in which a) text and diagrams appear in the
text or b) the order in which participant should read them, but
without stating that one representation is ‘‘better” or ‘‘more
helpful”
Low-level strategies
Course demands Mentions what is required for course tasks/assignments such as
quizzes, labs, or exams
Find location
Highlights text
Importance of
information
Memorize
‘‘How do these antigens interact with the antibodies?” (T)
Finds the place where he/she was last reading
Uses highlighter on text or underlines
States that certain information is important or key
States that he/she is (trying to) memorize what was read.
Creates a verbal or visual memory aid to help remember
something from the text
States that he/she is not thinking
Re-reads five or more words in a row (in text or diagram)
Accurately re-states what was read within one sentence in own
words (not simply re-reading). Note that a sentence can be split
into accurate (PARA+) and inaccurate (PARA!) portions
Inaccurately re-states what was read within one sentence in own
words (not simply re-reading). A sentence can be split into
accurate (PARA+) and inaccurate (PARA!) portions
Inaccurately re-states what was read/diagram in own words across
two or more sentences (not simply paraphrasing one sentence or
re-reading). A sentence can be split into accurate (SUM+) and
inaccurate (SUM!) portions
‘‘I do not know what an MHC molecule is, and I haven’t read
the passage yet so I’m sure it will be explain what it is” (D)
‘‘I have a difficult time sometimes interpreting these
pictures” (T)
‘‘the previous passage, . . . was complicated for me.” (D)
‘‘I just wish that the diagram came after the explanation that
they just gave me” (T)
‘‘Um, I’m just looking at the pictures, and looking at what’s
labeled where, just to get a visual, just in case this would be
on a test.” (D)
‘‘Um, trying to find my place.” (T)
‘‘I’m going to highlight where the B cell is” (D)
‘‘Ok, ah, antigen-presentation is highlighted so it must be
important” (T)
‘‘the diagrams is what I need to memorize and understand it
better” (D)
‘‘meditation [mediation] takes a long time and this cellmeditated response if long-term” (T)
‘‘Right now I’m not thinking anything” (T)
A foreign molecule that elicits a specific response by
lymphocytes is called an antigen. . . . A foreign molecule that
elicits a specific response by lymphocytes (T)
‘‘So, there’s different kinds of B cells and antigens, they hook
onto one of the B cells onto its antigen receptors,” (D)
‘‘So T-dependent antigens can only take place with the help
of T cells” (T)
‘‘by. . .a white blood cells. . . no, just a regular cell” (D)
(continued on next page)
72
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
Appendix B (continued)
Text signaling
Uses bolding, italics, paragraph headings to guide comprehension
(e.g., to guide note-taking, re-reading)
‘‘I am also writing down the main concepts that are basically
bolded.” (D)
Class code
Definition
Example (T = text, D = diagram)
Accurately states or recalls a definition of a term OR states that he/
she does not remember what a previously-defined word means
‘‘I don’t remember what an antibody is” (T)
Immediately self-corrects a full mis-pronunciation or partial mispronunciation of a word
Mis-reads/mis-pronounces a word in a way that affects meaning
while reading/re-reading, and without self-correcting
‘‘In this sem – S.E.M.” (D)
Vocabulary
Vocabulary
Word reading
Self-Corrects
Word
Word reading
error
‘‘pereforin” for perforin (D)
Appendix C
C.1. Mental models coding scheme
Level
%Pre
%Post
Level 1 basic
16%
0%
9%
1%
33%
9%
7%
34%
21%
6%
13%
10%
4%
1%
1%
12%
12%
11%
A. Protects from/defends/fights against diseases/infection/keeps body healthy
B. Diseases caused by any of the following pathogens: viruses, bacteria, fungi, etc. (see T for HIV)
Level 2 nonspecific immune
C. Pathogens have foreign (proteins (called antigens))
D. Mucus as a defense
E. Temperature up/fever as a defense
F. Skin as a defense
G. Histamine/inflammation as a defense
H. Mentions lymph nodes/lymphatic system
DD. A person can be immune to a disease = never get it again
EE. Vaccination/immunization can make a person immune
FF. Pathogens can adapt/mutate/change
Level 3 = 1 part (no functions)
Level 4 = 2 parts (no functions)
I. White blood cells/lymphocytes as part (not B cells [U] or T cells [M] specifically; code for NK)
J. WBCs are in blood/lymph
K. Macrophages as part
M. T cells as part (not specific Helper T [N] or Cytotoxic T [Q]
N. Helper T (T4) cells as part
T. T cells (and/or macrophages and/or immune system) affected by HIV
Q. Cytotoxic T cells/killer T cells as part
U. B cells as part
Y. Antibodies as part
Mentions CD4 (without Class II MHC OR helper T)
JJ. Mentions Class I MHC (without CD8 OR cytotoxic T)
KK. Mentions Class II MHC (without CD4 OR helper T)
MM. Mentions CD8 (without Class I MHC OR cytotoxic T)
Level 5 = level 3 + at least 1 function OR Level 4 + only 1 function
Level 6 = level 4 + at least 2 functions
L. Macrophages phagocytize/engulf/attack/absorb pathogens
O. Helper T cells recognize/‘‘locate” antigen/pathogens
R. Cytotoxic T cells secrete enzyme (perforin)
S. T cells/Tc lyses/destroys pathogens/infected cells
V. B cells recognize/mark antigen/pathogens
W. B cells recognize/are activated by ‘‘markers” (i.e., antibodies) on infected cells
X. B cells make antibodies
GG. Immune system (or effector/plasma) cells proliferate in response to antigen
QQ. Mentions IL-1 (without PP)
RR. Mentions IL2 (without PP)
UU. Immune system exhibits self-tolerance; if self-tolerance fails = autoimmune disease
VV. APCs (or B cell/T cell, etc.) ‘‘present” antigen
Level 7 = Level 6 + at least 1 from any one of the following categories
Level 8 = Level 6 + at least 1 from any two of the following categories
Level 9 = Level 6 + at least 1 from all three of the following categories
AA. Antibodies are specific to particular pathogens
HH. Antibodies attach to invaders
NN. Any pair or triplet from among: CD4, Class II MHC, helper T
OO. Any pair or triplet from among: CD8, Class I MHC, cytotoxic T
SS. Tc—cell-mediated—infected cells
TT. B—humoral—free antigen
73
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
Appendix C (continued)
Level
%Pre
%Post
Interactions among lymphocytes
P. Helper T cells organize/‘‘mobilize” other immune cells; mention t-dependent antigens
Z. T cells signal B cells
CC. Antibodies enable immune system/other cells to kill pathogens [NOT antibodies kill pathogens]
PP. cytokines specifically signal Th ? B or Th ? Tc or Macro ? Th
Secondary
BB. Antibodies/memory cells are long-lasting/have memory/body is prepared for later infection
LL. Secondary response faster and stronger
References
Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with
multiple representations. Learning and Instruction, 16(3), 183–198.
Ainsworth, S., & Burcham, S. (2007). The impact of text coherence on learning by
self-explanation. Learning and Instruction, 17(3), 286–303.
Ainsworth, S., & Loizou, A. T. (2003). The effects of self-explaining when learning
with text or diagrams. Cognitive Science. A Multidisciplinary Journal, 27(4),
669–681.
Aleven, V. W. M. M., & Koedinger, K. (2002). An effective metacognitive strategy:
Learning by doing and explaining with a computer-based cognitive tutor.
Cognitive Science, 26(2), 147–179.
Alexander, P. A., Sperl, C. T., Buehl, M. M., Fives, H., & Chiu, S. (2004). Modeling
domain learning: Profiles from the field of special education. Journal of
Educational Psychology, 96(3), 545–557.
Azevedo, R., & Cromley, J. (2004). Does training on self-regulated learning facilitate
students’ learning with hypermedia? Journal of Educational Psychology, 96,
523–535.
Bartholomé, T., & Bromme, R. (2009). Coherence formation when learning from text
and pictures: What kind of support for whom? Journal of Educational Psychology,
101(2), 282–293. doi:10.1037/a0014312.
Bereiter, C., & Scardamalia, M. (2006). Education for the knowledge age: Designcentered models of teaching and instruction. In P. A. Alexander & P. H. Winne
(Eds.), Handbook of educational psychology (pp. 695–714). Mahwah, NJ:
Lawrence Erlbaum.
Berthold, K., & Renkl, A. (2009). Instructional aids to support a conceptual
understanding of multiple representations. Journal of Educational Psychology,
101(1), 70–87.
Bodemer, D., & Faust, U. (2006). External and mental referencing of multiple
representations. Computers in Human Behavior, 22(1), 27–42.
Bodemer, D., Ploetzner, R., Bruchmüller, K., & Häcker, S. (2005). Supporting learning
with interactive multimedia through active integration of representations.
Instructional Science, 33(1), 73–95.
Butcher, K. (2006). Learning from text with diagrams: Promoting mental model
development and inference generation. Journal of Educational Psychology, 98(1),
182–197.
Cain, K., Oakhill, J. V., Barnes, M. A., & Bryant, P. E. (2001). Comprehension skill,
inference making ability and their relation to knowledge. Memory and Cognition,
29, 850–859.
Campbell, N. A., & Reece, J. B. (2001). Biology (6th ed.). San Francisco, CA: Benjamin
Cummings.
Canham, M., & Hegarty, M. (in press). Effects of knowledge and display design on
comprehension of complex graphics. Learning and Instruction, 10.1016/
j.learninstruc.2009.02.014.
Carlson, R., Chandler, P., & Sweller, J. (2003). Learning and understanding science
instructional material. Journal of Educational Psychology, 95(3), 629–640.
Cerdan, R., & Vidal-Abarca, E. (2008). The effects of tasks on integrating information
from multiple documents. Journal of Educational Psychology, 100(1), 209.
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical
guide. The Journal of the Learning Sciences, 6(3), 271–315.
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Selfexplanations: How students study and use examples in learning to solve
problems. Cognitive Science, 13, 145–182.
Chi, M., Feltovich, P. J., & Glaser, R. (1981). Categorization and representations of
physics problems by experts and novices. Cognitive Science, 5(2), 121–152.
Chi, M. T. H., deLeeuw, N., Chiu, M., & LaVancher, C. (1994). Eliciting selfexplanations improves understanding. Cognitive Science, 18(3), 439–477.
Crain-Thoreson, C., Lippman, M., & McClendon-Magnuson, D. (1997). Windows on
comprehension: Reading comprehension processes as revealed by two thinkaloud procedures. Journal of Educational Psychology, 89(4), 579–591.
Cromley, J., & Azevedo, R. (2006). Self-report of reading comprehension strategies:
What are we measuring? Metacognition and Learning, 1(3), 229–247.
Edmonds, C. J., & Pring, L. (2006). Generating inferences from written and spoken
language: A comparison of children with visual impairment and children with
sight. British Journal of Developmental Psychology, 24, 337–351.
Fehrenbach, C. R. (1991). Gifted/average readers: Do they use the same reading
strategies? Gifted Child Quarterly, 35(3), 125–127.
Florax, M., & Ploetzner, R. (in press). What contributes to the split-attention effect?
The role of text segmentation, picture labeling, and spatial proximity. Learning
and Instruction, 10.1016/j.learninstruc.2009.02.021.
Fox, E. (2009). The role of reader characteristics in processing and learning from
informational text. Review of Educational Research, 79(1), 197–261.
Graesser, A. C. (2007). An introduction to strategic reading comprehension. In D. S.
McNamara (Ed.), Reading comprehension strategies: Theories, interventions, and
technologies (pp. 3–26). Mahwah, NJ, US: Lawrence Erlbaum Associates
Publishers.
Graesser, A., Lu, S., Olde, B., Cooper-Pye, E., & Whitten, S. (2005). Question asking
and eye tracking during cognitive disequilibrium: Comprehending illustrated
texts on devices when the devices break down. Memory & Cognition, 33(7),
1235–1247.
Griffin, T. D., Wiley, J., & Thiede, K. W. (2008). Individual differences, rereading, and
self-explanation: Concurrent processing and cue validity as constraints on
metacomprehension accuracy. Memory & Cognition, 36(1), 93–103.
Grosse, C. S., & Renkl, A. (2006). Effects of multiple solution methods in
mathematics learning. Learning and Instruction, 16(2), 122–138.
Hannus, M., & Hyönä, J. (1999). Utilization of illustrations during learning of science
textbook passages among low- and high-ability children. Contemporary
Educational Psychology, 24(2), 95–123.
Hegarty, M., & Just, M. (1993). Constructing mental models of machines from text
and diagrams. Journal of Memory & Language, 32(6), 717–742.
Hegarty, M., Kriz, S., & Cate, C. (2003). The roles of mental animations and external
animations in understanding mechanical systems. Cognition and Instruction,
21(4), 325–360.
Heiser, J., & Tversky, B. (2006). Arrows in comprehending and producing mechanical
diagrams. Cognitive Science, 30(3), 581–592.
Kastens, K., & Liben, L. (2007). Eliciting self-explanations improves children’s
performance on a field-based map skills task. Cognition and Instruction, 25(1),
45–74.
Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY:
Cambridge University Press.
Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learning
from animations. International Journal of Human–Computer Studies, 65(11),
911–930.
Laing, S., & Kamhi, A. (2002). The use of think-aloud protocols to compare
inferencing abilities in average and below-average readers. Journal of Learning
Disabilities, 35(5), 436–447.
Larkin, J., & Simon, H. (1987). Why a diagram is (sometimes) worth ten thousand
words. Cognitive Science, 11(1), 65–100.
Lewalter, D. (2003). Cognitive strategies for learning from static and dynamic
visuals. Learning and Instruction, 13(2), 177–189.
Linn, M. C. (2006). The knowledge integration perspective on learning and
instruction. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning
sciences (pp. 243–264). NY: Cambridge University Press.
Löhner, S., van Joolingen, W. R., Savelsbergh, E. R., & van Hout-Wolters, B. (2005).
Students’ reasoning during modeling in an inquiry learning environment.
Computers in Human Behavior, 21(3), 441–461.
Lowe, R. K. (2003). Animation and learning: Selective processing of information in
dynamic graphics. Learning and Instruction, 13(2), 157–176.
Mayer, R. E. (2003). The promise of multimedia learning: Using the same
instructional design methods across different media. Learning and Instruction,
13(2), 125–139.
Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.),
The Cambridge handbook of multimedia learning (pp. 31–48). Cambridge:
Cambridge University Press.
Mayer, R., Hegarty, M., Mayer, S., & Campbell, J. (2005). When static media promote
active learning: Annotated illustrations versus narrated animations in
multimedia instruction. Journal of Experimental Psychology: Applied, 11(4), 256.
Mayer, R., & Sims, V. (1994). For whom is a picture worth a thousand words?
Extensions of a dual-coding theory of multimedia learning. Journal of
Educational Psychology, 86(3), 389–401.
McNamara, D. S. (2001). Reading both high-coherence and low-coherence texts:
Effects of text sequence and prior knowledge. Canadian Journal of Experimental
Psychology, 55(1), 51–62.
Meece, J. L., & Miller, S. D. (2001). A longitudinal analysis of elementary school
students’ achievement goals in literacy activities. Contemporary Educational
Psychology, 26(4), 454–480.
Moore, P. J., & Scevak, J. J. (1997). Learning from texts and visual aids: A
developmental perspective. Journal of Research in Reading, 20(3), 205–223.
Moreno, R., & Valdez, A. (2005). Cognitive load and learning effects of having
students organize pictures and words in multimedia environments: The role of
74
J.G. Cromley et al. / Contemporary Educational Psychology 35 (2010) 59–74
student interactivity and feedback. Educational Technology Research &
Development, 53(3), 1042–1629.
Muis, K. (2008). Epistemic profiles and self-regulated learning: Examining relations
in the context of mathematics problem solving. Contemporary Educational
Psychology, 33(2), 177–208.
Narayanan, N. H., & Hegarty, M. (1998). On designing comprehensible interactive
hypermedia manuals. International Journal of Human–Computer Studies, 48(2),
267–301.
Panel, National. Reading. (2000). Teaching children to read: An evidence-based
assessment of the scientific research literature on reading and its implications for
reading instruction. Washington, DC: National Institute for Child Health and
Human Development.
Neuman, S. (1990). Assessing children’s inferencing strategies. In J. Zutell & J.
McCormick (Eds.), Literacy theory and research: Analyses from multiple paradigms
thirty-ninth yearbook of the national reading conference (pp. 267–274). Chicago,
IL: National Reading Conference.
Otero, J., León, J., & Graesser, A. (2002). The psychology of science text comprehension.
Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.
Ozuru, Y., Dempsey, K., & McNamara, D. (2009). Prior knowledge, reading skill, and
text cohesion in the comprehension of science texts. Learning and Instruction,
19(3), 228–242.
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional
design: Recent developments. Educational Psychologist, 38(1), 1–4.
Pintrich, P. (2000). The role of goal orientation and self-regulated learning. In M.
Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation
(pp. 452–494). San Diego: Academic Press.
Pozzer, L., & Roth, W. (2003). Prevalence, function, and structure of photographs in
high school biology textbooks. Journal of Research in Science Teaching, 40(10),
1089–1114.
Pressley, M., & Harris, K. R. (2006). Cognitive strategies instruction: From basic research to
classroom instruction. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational
psychology (2nd ed., pp. 265–286). Mahwah, NJ: Lawrence Erlbaum.
Robertson, W. C. (1990). Detection of cognitive structure with protocol data:
Predicting performance on physics transfer problems. Cognitive Science. A
Multidisciplinary Journal, 14(2), 253–280.
Samuelstuen, M., & Braten, I. (2007). Examining the validity of self-reports on scales
measuring students’ strategic processing. British Journal of Educational
Psychology, 77, 351–378.
Sanchez, C., & Wiley, J. (2006). An examination of the seductive details effect in
terms of working memory capacity. Memory & Cognition, 34(2), 344–355.
Schellings, G., Aarnoutse, C., & van Leeuwe, J. (2006). Third-grader’s think-aloud
protocols: Types of reading activities in reading an expository text. Learning and
Instruction, 16(6), 549–568.
Schmidt-Weigand, F., Kohnert, A., & Glowalla, U. (in press). A closer look at split
visual attention in multimedia learning. Learning and Instruction, 10.1016/
j.learninstruc.2009.02.011.
Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from
multiple representation. Learning & Instruction, 13(2), 141–156.
Schwonke, R., Berthold, K., & Renkl, A. (2009). Applied Cognitive Psychology, 23(9),
1227–1243. doi:10.1002/acp.1526.
Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In
R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 19–30).
Cambridge: Cambridge University Press.
Wade, S. E., Trathen, W., & Schraw, G. (1990). An analysis of spontaneous study
strategies. Reading Research Quarterly, 25(2), 147–166.
Weinstein, C. E., Husman, J., & Dierking, D. R. (2000). Self-regulation interventions
with a focus on learning strategies. In M. Boekaerts, P. R. Pintrich, & M. Zeidner
(Eds.), Handbook of self-regulation (pp. 727–747). San Diego, CA, US: Academic
Press.
Wittrock, M. C. (1990). Generative processes of comprehension. Educational
Psychologist, 24, 345–376.
Zwaan, R., & Brown, C. (1996). The influence of language proficiency and
comprehension skill on situation-model construction. Discourse Processes,
21(3), 289–327.