Leuphana Universität Lüneburg
The Process
and Product
of Content
Reading
Assessment
Dr. William G. Brozo
Professor of Literacy
George Mason University
November 2009
Brozo09
1
Overview
Rationale for Content Reading Assessment
 The Development and Features of the
Adolescent Literacy Inventory (ALI)









Formative Assessment
Academic Vocabulary
Relevant Prior Knowledge
Reading Engagement
Content Reading
Interactive Assessment
Reading Fluency
Oral Reading Miscues
Brozo09
2
Rationale for Content Reading Assessment
In order to be prepared for the new
economy of the 21st century our students
must be able to function skillfully and
flexibly in an information-rich society.
 This means they must possess skillful and
flexible reading skills with information
text.
 In school settings, we refer to this as
content text.

Brozo09
3
Rationale for Content Reading Assessment

To be a successful reader of content text, students must bring
specialized cognitive skills and strategies to the act of
constructing meaning.

They must also bring their prior knowledge for content areas,
related to history, biology, economics, earth science or other
content areas.

Lastly, students must bring domain-specific strategies to help
construct meaning. For example, students in science use
schemas for scientific procedure to help them conceptualize,
understand and remember information and ideas in science
textbooks. Students of history may examine texts for evidence
that an author uses to make claims about people, places and
events.
Brozo09
4
Rationale for Content Reading Assessment



More than 30 years of theory development and
research has characterized reading as an
interactive, context-bound, purposeful process of
meaning construction (Bissex, 1980; Clay, 1975;
Gee, 2000).
During the same time, we have progressed in our
understanding of what reading processes should
be assessed.
We know that it is critical to consider the
contexts in which students are asked to construct
meaning from texts and to use what they learn
from reading (Afflerbach, 2007).
Brozo09
5
Rationale for Content Reading Assessment

In the disciplines this should translate into
opportunities to assess students as they interact
with content text (Brozo & Simpson, 2007).

These very texts can then be used to gather
diagnostic information about students’ reading in
the content areas.

With this knowledge we can customize instruction
in relation to specific demands of content text.
Brozo09
6
The Adolescent Literacy Inventory (ALI)

The Adolescent Literacy Inventory was developed in response to
the growing and widespread concern among teachers in the
United States about how best to assess students’ ability to handle
the authentic challenges of academic reading.

Academic reading skills and abilities are needed for success in
school settings.

To be a successful reader in secondary school, adolescents must
be able to read fluently, comprehend meaningfully, and
understand content vocabulary found in textbook prose.

To assess these abilities the ALI uses actual textbook passages
from the four core content domains: English/Language Arts,
Science, Social Studies, and Math.
Brozo09
7
The Adolescent Literacy Inventory (ALI)

The ALI provides the important innovation of
situating reading and our diagnostic approach to
reading within particular school content areas.

The ALI uses texts that are representative of real
reading in secondary school, thus they bring the
added feature of ecological validity to the
assessment and diagnostic process.

In this way, assessment results can inform our
teaching of reading in school content areas.
Brozo09
8
The ALI Materials at a Glance





Cloze-Maze Placement Passages for Grades 6-12
(includes Grade 13 in Math)
1 High-Interest Content Passage and 2 Textbook
Passages per grade level per content area
domain (English, Social Studies, Math, Science)
Text Impression Activity for every textbook
passage
Vocabulary Self-Awareness Activity for every
textbook passage
Embedded Comprehension Questions and Post
Comprehension Questions for every passage
Brozo09
9
ALI Reading Passages
6th Grade Mathematics Passages
High Interest Passage
Comprehension Questions
Mathematics – #1
Text Impression
Vocabulary Self-Awareness Activity
Mathematics Textbook Passage
Comprehension Questions
Mathematics # 2
Text Impression
Vocabulary Self-Awareness Activity
Mathematics Textbook Passage
Comprehension Questions
Brozo09
10
The Adolescent Literacy Inventory (ALI)

The ALI is designed to be administered
individually and can yield critical diagnostic
information about a student’s reading needs and
abilities with the kind of text most commonly
required in secondary school.

ALI users can discover how well students (1)
decode words, (2) understand vocabulary in
context, (3) read with appropriate speed and
accuracy, and (4) comprehend what they read.
Brozo09
11
The Adolescent Literacy Inventory (ALI)




The ALI offers a range of administration options depending
upon the goals of the user and the needs of the student.
For example, it can assist a content teacher in deciding
whether a particular student has the reading ability
commensurate with the reading level of the required
textbook or whether alternative texts and other support
materials are needed.
It can reveal to a skilled teacher what textbook reading
strategies a student is using and whether new ones will
improve performance for the student.
The ALI can also be used as one important indicator to help
determine the most effective instructional placement or
setting for a student and the most responsive reading
interventions.
Brozo09
12
Decision Tree
for the ALI
Why are
you using
the ALI?
What do you
already know
about the student
to be assessed ?
How
detailed of
a diagnosis
is needed?
New student
may need full
assessment
Decoding;
miscues;
high
interest
texts
Consult
literacy
coach or
diagnosti
cian
Comprehension;
vocabulary; gather
text impression;
embedded and
post reading
comprehension
questions
What is
your
reading
assessment
skill
level?
Brozo09
In a
specific
subject
area
domain
Adequate
to conduct
on my
own
13
Development of
The Adolescent Literacy Inventory (ALI)

The Adolescent Literacy Inventory is an informal
assessment, not a standardized, norm-referenced
instrument.

Although the assessment features and
procedures of the ALI were developed based on
sound and rigorous research evidence and it was
piloted with middle and high school students, it
does not possess the kind of normative data of a
standardized reading test, such as stanines and
percentiles, so cannot be used to make direct
comparisons of students.
Brozo09
14
Development of
The Adolescent Literacy Inventory (ALI)




The ALI is not standardized in the sense that
every user and every student must adhere to
exactly the same administration procedures.
Instead, the ALI is designed to offer users
maximum flexibility.
Flexible assessment options are necessary
because reading diagnosis is as much an art as
an exact science.
This means users can decide to employ certain
aspects of the assessment or not depending upon
the desired information about a student’s reading
practices.
Brozo09
15
Development of
The Adolescent Literacy Inventory (ALI)

The ALI also allows users to learn about a
student’s reading behavior in ways that are
impossible with norm-referenced, standardized
instruments.

For example, the one-on-one nature of the
assessment process means users can probe more
deeply after asking a question; can explore a
student’s reasoning for giving a response; and
can interact with a student while preparing to
read a textbook passage, during reading and
meaning making, and after the passage has been
Brozo09
16
read.
The ALI and Formative Assessment

Formative assessment helps us identify, understand and describe
students’ current needs and abilities.

Thus, formative assessment should inform our ongoing
instruction.

Student learning and teacher accountability are measured on a
single day by single test, in actuality this accountability is
established across the school year by careful formative
assessment.

That is, test scores tend to get better when we have an ongoing
and developmental assessment focus, for this helps us in the here
and now of our daily teaching.
Brozo09
17
The ALI and Formative Assessment

The Adolescent Literacy Inventory provides teachers with
critical formative assessment information.

As we observe students reading content area texts, we
have a window in which to observe their cognitive reading
skill and strategy development.

The prior knowledge that students bring to these reading
endeavors also helps us identify reading strength and
challenge in relation to both skill and strategy, and content
area learning.
Brozo09
18
Academic Vocabulary and the ALI



Academic vocabulary refers to word knowledge that makes
it possible for students to engage with, produce, and talk
about texts that are valued in school (Brozo & Simpson,
2007).
Since textbooks are the most common print source in
secondary school, students must be knowledgeable of the
key terminology of the disciplines in order to learn from
their reading (Marzano & Pickering, 2005).
Concern about how secondary students expand and use
word knowledge has been growing, say Pearson, Hiebert,
and Kamil (2007), who note ―after a nearly 15-year
absence from center stage, vocabulary has returned to a
prominent place in discussions of reading, and it is alive
and well in reading instruction and reading research‖ (p,
282).
Brozo09
19
Academic Vocabulary and the ALI

The RAND Reading Study Group (2002) reaffirmed the
fundamental relationship vocabulary knowledge has to
overall reading comprehension.

This relationship is even more critical for content texts due
to the new and numerous technical words youth must
understand for successful meaning making to occur
(Harmon, Hedrick, & Wood, 2005).

The ALI offers users at least three assessment options,
each of which can yield insights into students’ knowledge of
academic vocabulary.
Brozo09
20
Academic Vocabulary and the ALI
 The
text impression activities before
each textbook reading passage will
provide insights into how sensibly
students can use content vocabulary
within the context of their own
anticipatory compositions.
Brozo09
21
Academic Vocabulary and the ALI

The vocabulary self-awareness activity adds another
assessment dimension to a student’s knowledge of critical
content vocabulary.

In advance of reading, key terms from the passage are
presented to students.

With these terms, students are to rate their knowledge,
provide definitional information, and examples.

After reading, students return to the vocabulary selfawareness charts and reconsider their understandings of
key vocabulary terms from the passage.
Brozo09
22
Vocabulary Self-Awareness
Word
+ √
-
Example Definition
preserved
hominids
nomads
originated
geographical
23
Vocabulary Self-Awareness





Procedures:
Examine the list of words you have written in the
first column.
Put a ―+‖ next to each word you know well, and
give an accurate example and definition of the
word. Your definition and example must relate to
the unit of study.
Place a ―‖ next to any words for which you can
write only a definition or an example, but not
both.
Put a ―-― next to words that are new to you.
Brozo09
24
Academic Vocabulary and the ALI

A third way users can assess students’ word
knowledge is through the maze placement
passages.

This part of the ALI requires students to select
the most appropriate words to fill in the blanks of
short textbook excerpts.

These words are taken from The Living Word
Vocabulary (Dale & O’Roarke, 1981), the most
comprehensive study in the United States of
grade-level appropriate words ever conducted.
Brozo09
25
Academic Vocabulary and the ALI
Cloze tasks require students to read a
passage in which every nth word or a
particular type of word has been deleted.
 The maze aspect is that students must
select answers from among three choices,
one of which is the correct word.
 Cloze/maze has been well-documented in
the research literature as a viable
approach to reading assessment (DuBay,
2004; Madelaine & Wheldall, 2004).

Brozo09
26
Example of Cloze Maze Passage from the ALI
6th Grade Student Cloze with Maze
The Beginning of Human Society
Instructions: Read the story to yourself. When you come to a part where
there are three underlined words in very dark print, choose the one word
that makes sense in the sentence. Circle that word.
Oral traditions are still an important part of many societies today. Not all
oral stories are historically (1) inaccurate, truthful, accurate. Stories
often change as they are told and retold. Like myths and (2) policy,
legends, democracy, they often contain facts mixed with personal
beliefs and (3) exaggerations, understatement, containment about
heroes. Still, (4) oral, written, collected traditions tell how a (5)
people, society, citizens lived and what the people considered
important.
Brozo09
27
Scoring Criteria
for Cloze Maze Placement Passages
Only the correct words from the maze are
counted. Each correct word is worth one
point.
Percent
Correct
70 -100
14 – 20
Independent
35 – 69
7 – 13
Instructional
0 – 34
0–6
Frustrational
Brozo09
level
28
Scoring Criteria
for Cloze Maze Placement Passages

Reading of Textbook Passages begins one grade level
below the highest Independent Level on the Placement
Passages

Independent. A score of 70 percent or higher indicates student
will read the passage with competence. Reading individually with
text at this level will not be difficult for these students.

Instructional. A score between 35 and 70 percent indicates the
passage can be read with some competence by the student;
however, reading text at this level may require some guidance
would be beneficial.

Frustrational. A score below 35 percent will probably be too
difficult for these students. A great deal of guidance will be
needed to read text at this level,
or other material should be
Brozo09
substituted.
29
Relevant Prior Knowledge and the ALI

We have known for some time that what learners
take from what they read depends on how much
they bring to it (Bransford & Johnson, 1972;
Pressley, 2000; Wilson & Anderson, 1986).

Skillful readers use their prior knowledge as they
interact with text to enhance comprehension
(Afflerbach, 1986; Chiesi, Spilich, & Voss, 1979;
Snow & Sweet, 2003; Spires & Donley, 1998).
Brozo09
30
Relevant Prior Knowledge and the ALI

Students who have been the beneficiaries of rich
and varied print and life experiences are likely to
possess the kind of prior knowledge necessary to
comprehend content area texts at meaningful
levels (Best, Rowe, Ozuru, & McNamara, 2005;
Kintsch, 1998; Nassaji, 2002).

On the other hand, students with limited prior
knowledge for text topics, even if they possess
word attack skills, are likely to find it difficult to
learn much from their reading.
Brozo09
31
Relevant Prior Knowledge and the ALI

With sophisticated content area material, found in the
textbooks secondary school students must read every day,
being a good decoder of words is not nearly enough for
thoughtful comprehension to occur.

On our National Assessment of Educational Progress (NAEP)
(Donahue, Daane, & Grigg, 2003), many 8th and 12th
graders demonstrate competency with Basic reading tasks;
tasks that require reliance on students’ decoding and
superficial comprehension skills.

Far fewer of these students reach Proficient or Advanced
levels, which place a greater demand on students’ prior
knowledge and skillful reading strategies.
Brozo09
32
Relevant Prior Knowledge and the ALI

Thus, in addition to decoding and
vocabulary skills, students’ levels of
background knowledge of text topics will
determine whether they have successful
reading experiences (Kintsch, 2005).
Brozo09
33
Relevant Prior Knowledge and the ALI

Prior knowledge for textbook topics can be
assessed with the ALI by noting the accuracy of
responses students give for the clozed words in
the maze placement passage.

Since these short passages are taken directly
from the same sections of the textbook from
which the longer reading passages are taken,
students’ ability to read them accurately indicates
a degree of relevant prior knowledge for the
topic.
Brozo09
34
Relevant Prior Knowledge and the ALI



When ALI users ask students to compose or
dictate a text based on the impression words,
another important indicator of the extent of
relevant prior knowledge is obtained.
Students who create sketchy or irrelevant
impression texts are likely to possess limited
prior knowledge for the topic.
Conversely, those who can generate an elaborate
and highly relevant impression text are certain to
have the knowledge bases in place for successful
comprehension of the textbook passage.
Brozo09
35
Text Impression Activity from the ALI
The Beginning of Human Society
huge explosion
the surrounding
land preserved in
early hominids
of prehistoric culture
were nomads
originated more
of geographical conditions
Directions: The above list of words and the italicized words in
phrases are taken from the passage you are about to read. In the
space below, use all of the listed words to write a short
description of what you believe the passage will be about.
Brozo09
36
Reading Engagement and the ALI




Although cognitive skills and strategies are
central to reading success, they do not, by
themselves, guarantee it.
Students must also be engaged readers (Guthrie
& Wigfield, 1997).
When student readers are engaged, they
approach reading tasks with a clear sense of
purpose, and a set of cognitive tools that they
use to construct meaning.
Engaged readers have appropriate prior
knowledge for the text being read, which helps
connect what they know with what they will
Brozo09
37
learn.
Reading Engagement and the ALI
Engaged readers view reading as a
positive force in their lives.
 Reading for these students is a tool to
learn, a means to reflect and gain insight
and a way to be entertained.
 Engaged readers, having experienced
success, view themselves in a positive
light, contributing to enhanced selfconcept as readers and self-esteem as
students.

Brozo09
38
Reading Engagement and the ALI




Without engagement, students will demonstrate
a lack of motivation, unwillingness to persevere
when confronted with a reading challenge or
problem, and disinterest in reading.
The lack of engagement can lead a student to
avoid reading at all costs.
This prevents opportunities for students to
practice the reading skills and strategies they
learn, and limits their reading development.
The benefits one can gain from continued
reading, including increased fluency and
vocabulary, cannot be realized.
Brozo09
39
Reading Engagement and the ALI




The interactive assessment option of the ALI offers users
many insights into students’ level of engagement for the
texts and related activities.
The interactive process makes it possible for user and
student to experience something like an authentic reading
lesson around a textbook passage.
This form of assessment involves interactions before,
during, and after reading in order to determine which
strategies help students activate and build prior knowledge,
learn new vocabulary, decode words in context, read
fluently, and answer comprehension questions accurately
and fully.
Each interactive assessment activity allows teachers and
specialists to note the degree of enthusiasm students
exhibit for them.
Brozo09
40
Content Literacy and the ALI

Content literacy is a unique form of literacy requiring both critical
foundational reading skills as well as specialized abilities and
strategies (Brozo & Simpson, 2007).

Secondary students need highly developed content literacy skills
and abilities in order to cover the volume and negotiate the
complexity of required textbook reading.

Thus, it can’t be assumed that once students are taught how to
read in primary school they have the necessary skill set for the
reading demands of secondary textbook prose.

This is because reading is a complex, developmental process, and
the ability to understand text grows with each new print
experience.
Brozo09
41
Content Literacy and the ALI

There is plenty of evidence that reading skills and
abilities developed in primary school are not
adequate for the challenges of increasingly
complex text students and adults encounter in
secondary school and beyond (Duke, Pressley, &
Hilden, 2004; Underwood & Pearson, 2004).

Every new text and reading situation requires a
refined application of literacy skills and abilities.
This is especially true of content-area literacy.
Brozo09
42
Content Literacy and the ALI



The ALI’s focus is entirely on content literacy. Students
must demonstrate reading ability with passages taken
directly from content textbooks in science, math, social
studies, and English: the four core disciplines of a U.S.
secondary school curriculum.
Comprehension questions that are interspersed throughout
and at the conclusion of each reading passage sample
students’ understanding of content text at the explicit and
implicit levels.
Further indicators of students’ understanding of content
text can be obtained by completing the Venn diagram of
the text impression activity. This requires students to
reconsider their pre-reading impression text after
completing a passage and add shared and newly learning
information and ideas to the Venn diagram.
Brozo09
43
Comprehension Questions for ALI Passages

Embedded Questions
Directions for students: Read the following passage. When you
come to the word STOP in the text, respond to the questions in
the box
A million years after the footprints were made, early hominids
began making stone tools. By studying these tools, we
learn about the development of prehistoric culture. STOP
1. What role did finding footprints preserved in the
mud play in understanding history?______________
__________________________________________
__________________________________________
Brozo09
44
Comprehension Questions for ALI Passages

Embedded Questions
Archaeologists divide the Stone Age into three periods: the
Old Stone Age, the Middle Stone Age, and the New Stone
Age. During the Old Stone Age, modern humans and other
hominids did not yet know how to farm. They were huntergatherers who survived by hunting animals and gathering
wild plants. Almost all of human prehistory took place
during the Old Stone Age. STOP
2. Describe the Stone Age?______________________
_________________________________________
_________________________________________
Brozo09
45
Comprehension Questions for ALI Passages
Post-Reading Questions
Directions: After you have read the passage, The Beginning of Human Society, answer
the following questions.
1. What was left in the mud 3.5 million years ago that was of interest to scientist in
1976? (explicit)
2. Why is it important to study the tools of early man? (implicit)
3. What three periods do Archeologist divide the Stone Age into? (explicit)
4. How do Archeologist believe early hominids learned how to use fire? (implicit)
Brozo09
46
Comprehension Questions for ALI Passages
Post-Reading Questions
5.
A.
B.
C.
Hominids are:
Modern humans and their earlier relatives.
Early humans who began making stone tools.
Early humans who eventually spread out over much of Earth.
6.
A.
B.
C.
____________was a period during which hominids made tools from stone.
Stone Age
Prehistory
Late Stone Age
7.
A.
B.
C.
People who have not settled into a home are known as
Early hominids
Modern Humans
Nomads
Brozo09
___.
47
Venn Diagram for Text Impression
My Ideas
Author’s Ideas
Shared Ideas
Brozo09
48
Interactive Assessment and the ALI



Some have characterized interactive assessment
as teaching while testing (Brozo, 1990).
This approach to diagnostic testing has its roots
in dynamic assessment (Feuerstein, 2000) which
focuses on the ability of the learner to respond to
interventions.
Applied to reading assessment it involves the
teacher or specialist modeling reading processes
and eliciting these same processes from the
students to determine whether they read more
fluently, decoded and learned key vocabulary
more effectively, and comprehended at a more
Brozo09
49
thoughtful level.
Interactive Assessment and the ALI

Interactive assessment for literacy makes sense because reading itself is
an interactive process.

According to Rand Reading Study Group (2002) reading comprehension is
―the process of simultaneously extracting and constructing meaning
through interaction and involvement with written language‖ (2002, p. 11).

Active readers and learners interact with text to enhance comprehension
(Afflerbach, 1986; Chiesi, Spilich, & Voss, 1979; Pressley, 2000; Snow &
Sweet, 2003; Spires & Donley, 1998).

If reading is an interactive, constructivist process, then our assessments
should account for ways new strategies are learned and new
understandings emerge for students as a result of interactions with the
teacher during text processing (Bean, 2000).
Brozo09
50
Interactive Assessment and the ALI

Interactive assessment allows for teachers and specialists
to explore ways of apprenticing and scaffolding youths’ acts
of meaning making with print.

In this way reading assessment becomes a process of
discovering what students can do as a result of interactions.
What is learned can then be translated directly into support
strategies and interventions to increase students’ academic
literacy achievement in school.

The ALI is a highly flexible reading diagnostic tool that can
be administered using an interactive process, depending
upon the user’s desired goals for the assessment and the
literacy needs of the student.
Brozo09
51
Steps in Conducting
an Interactive Assessment with the ALI






Examiner decides in advance the kinds of interactions s/he will
employ with Instructional Level passages.
The results of interactive assessment should reveal the extent to
which the student responded favorably to modeling and teaching
of reading processes.
Examiner reviews the interactive assessment process with the
student by talking about the various strategies employed.
Examiner and student reflect together on which of the strategies
helped increase engagement and thoughtful reading.
Student is encouraged to employ the strategies that were helpful
while reading textbook prose independently.
Useful strategies should also be shared with student’s other
teachers so they can continue to use them when working with the
student.
Brozo09
52
Reading Fluency and the ALI



Fluent reading is a goal of most primary reading
programs.
With appropriate instruction most students learn
to identify words with accuracy and speed and
apply these skills with proper phrasing and
expression to continuous text (Kuhn & Stahl,
2003).
Thus, attention to reading fluency diminishes as
students progress through primary school years
and into secondary school years (Rasinski, et. al.,
2005).
Brozo09
53
Reading Fluency and the ALI

How accurately, rapidly, and with expression--that is how
fluently--a student reads a text will depend on the student’s
word recognition, print knowledge, and print experience
skills (Hasbrouck & Tindal, 2006).

The more automatic these fundamental reading skills are,
the more cognitive energy is reserved for thinking about
and comprehending text (Tractenburg, 2002).

The contrary may be true, as well. If a student has limited
automaticity with fluency-related skills, then less cognitive
capacity is available for the most important work of
reading—comprehension (Strong, Wehby, Falk, & Lane,
2004).
Brozo09
54
Reading Fluency and the ALI

Although curricular attention to fluency in
language arts programs wanes as students
advance through the grades, many struggling
adolescent readers may still experience problems
with accurate and quick decoding.

This may be especially true for youth from urban
areas (National Center for Educational Statistics,
n.d.), English language learners (Vaughn,
Mathes, Linan-Thompson, & Francis, 2005), and
learning disabled students (Archer, Gleason, &
Vachon, 2003).
Brozo09
55
Reading Fluency and the ALI




The ALI offers users a measure of fluency using
an oral reading fluency (ORF) approach.
This approach focuses on two dimensions of
fluency, word identification accuracy and reading
rate.
An ORF assessment for adolescents is considered
a useful and valid measure of fluency (Hasbrouck
& Tindal, 2006; Rasinski, 2004).
Specifically, ORF is determined by words correct
per minute (WCPM) , which tells users how many
words a student can say correctly within one
minute’s time.
Brozo09
56
Steps in Conducting a WCPM
Assessment with the ALI







Using the high interest passages only, a determination is made of the
passage level that approximates the student’s Instructional reading level
using the miscue and comprehension guides.
A new high interest content passage at the student’s Instructional reading
level is selected.
Before the student begins reading aloud from an Instructional level high
interest passage, a timer is set for one minute. A wristwatch with a
second hand may be used for this purpose.
The student is asked to read aloud while examiner marks her/his copy
which words are pronounced incorrectly.
After one minute the student is asked to stop.
The total number of words read by the student is tallied and subtracted
from that total are the number of words pronounced incorrectly. The
product derived will be the total words correct per minute (WCPM).
A student’s WCPM score can be compared to the expected average
performance of a student at her/his grade level using tabular information.
Brozo09
57
Oral Reading Fluency Norms
for Grades 6-12 in WCPM
%tile
50th
6
7
8
9
10
11
12
139
149
159
164*
169*
172*
175*
*Ex t r ap o l at ed ORF n o r ms bas ed o n
Has br o u ck an d Ti n dal (2006)
Brozo09
58
Reading Fluency and the ALI




Some students may exhibit reading rates that far exceed
these averages.
Those students should be watched for signs of ―disfluency‖
(Rasinski, 2009).
Disfluency occurs when a student’s reading rate is in excess
of what would be appropriate to comprehend a particular
text.
In other words, students who are skillful at monitoring their
own comprehension will adjust their rate to match the
complexity of the reading material; whereas, students with
limited self-monitoring skills may read at one rapid rate for
all types of text.
Brozo09
59
Oral Reading Miscues and the ALI

An oral reading fluency measure allows users a quick check
of students’ oral reading abilities.

Because fluency has been shown to be related to overall
reading ability, an ORF score may be an initial indicator of
potential problems students are experiencing with word
identification, word meanings, and even text
comprehension.

A common follow-up step to ORF screening is to undertake
a closer analysis of potential problems by examining the
nature of the oral reading deviations readers make. This
diagnostic process is called miscue analysis.
Brozo09
60
Oral Reading Miscues and the ALI




A miscue is meant to denote any deviation a student makes
from the cued or printed text.
Goodman (1969) is credited with devising a diagnostic
method for documenting and analyzing oral reading
miscues.
His research demonstrated that by using a systematic
approach to recording and examining a reader’s oral
rendition of a text, valuable insights into word-level and
meaning-level skills and abilities could be gleaned.
When oral reading miscues are compared with
comprehension tasks, users can further analyze the
relationship between word level skills and meaning making
abilities of students.
Brozo09
61
Oral Reading Miscues and the ALI




In the years since Goodman’s ground breaking work, numerous
studies have been conducted that point to the benefits of
analyzing miscues for determining how well students decode
words in context (Brown, Goodman, & Marek, 1996; Fawson, et.
al., 2006).
When reading any text orally three cueing systems are involved:
(a) the ability to sound words according to knowledge of the
alphabetic principle or grapho-phonics; (b) the ability to
understand words based on knowledge of word order or
syntactics; and (c) the ability to determine word meanings based
on surrounding text or semantics.
These cueing systems are highly developed in good readers and
poorly developed or non-existent in struggling readers (Clay,
2005; McKenna & Picard, 2007).
Thus, the more oral reading miscues a reader makes, the more
difficult it is to make meaning with a text.
Brozo09
62
Oral Reading Miscues and the ALI

Research into miscue analysis has confirmed the logic and
validity of established reading levels based on the number
and seriousness of oral reading miscues readers make.

Hargis (2005), for example, has determined that to be an
Independent reader, students must make two percent or
fewer significant miscues; an Instructional reader no more
than five percent; and a Frustrational reader, six percent
and more.

Although these criteria seem strict, they are consistent with
and supported by long-standing recommendations by Betts
(1946), Harris and Sipay (1985) and Leslie and Caldwell
(2006).
Brozo09
63
Oral Reading Miscues and the ALI




The ALI’s versatility allows for miscue analysis. Users may
choose to conduct this level of assessment with high
interest passages taken from ancillary texts that support
the content textbooks used in the inventory.
Because oral reading of actual textbook prose is an
uncommon practice, users can acquire samples of students’
oral reading abilities and subsequent comprehension with
these graded high-interest, content-based passages.
Results will yield rich data about students’ patterns of
miscues and the influence these miscues may be having on
the ability to comprehend key vocabulary and make overall
meaning of the text.
In addition, approximate reading levels and ranges by
grade can be established based on the results.
Brozo09
64
Oral Reading Miscues and the ALI

A miscue is a value-neutral term which should
not to be used synonymously for mistake.

It is meant to denote any deviation a student
makes from the cued or printed text. Thus, if the
word ―trial‖ appears in the text and the student
says ―trail‖ that’s a miscue and should be
recorded.
Brozo09
65
Oral Reading Miscues and the ALI

Deviations from the printed text can be analyzed
for their significance and for patterns which may
provide insight into decoding, word recognition,
and/or contextual reading abilities.

Miscue analysis is an appropriate compliment to a
WCPM score (ORF), since it allows the ALI user to
make a closer examination of a student’s
complete oral performance across multiple
passages.
Brozo09
66
Oral Reading Miscues and the ALI



Miscues should be documented using a consistent
system so patterns of oral reading can be
efficiently analyzed.
As the student reads aloud from a high interest
passage, miscues are recorded on the examiner’s
copy.
There are several suggested miscue marking
systems. Experienced users will develop their
own system for recording miscues. The key is
that the various miscue designations are
consistent.
Brozo09
67
Suggested Marking System for the Three Most
Common Oral Reading Miscues
Type of Miscue
Description
Substitution
The cued word is
substituted by
another word. Write
the substituted word
above the cued word
Insertion
Omission
An extra word is
added to the text.
Use an insert mark
and write the added
word(s) above the
line.
A word, group of
words, or line is not
read. Put a strike
through the omitted
Brozo09
word(s)
Marking System
felt
History is filled with
such cases…
often
Exactly how life
began is still a
mystery to scientists.
Geometric patterns
can be found
everywhere in the
natural world.
68
Steps in Conducting an Assessment
of Oral Reading Miscues Using the ALI



Examiner administers successive maze placement
passages until a highest Independent level is
established
Examiner begins work with the reading passages.
Working with the high interest passages only,
examiner starts one grade level below the
student’s highest Independent level achieved on
the placement passages
Examiner has student read the high interest
passage out loud while documenting miscues
and other oral reading patterns.
Brozo09
69
Steps in Conducting an Assessment
of Oral Reading Miscues Using the ALI



As the student reads aloud, examiner stops
her/him and asks the interspersed
comprehension questions and record responses.
When student finishes reading the passage aloud,
examiner asks her/him the post-reading
comprehension questions and record responses.
Examiner continues working with the high
interest passages using the same procedures
until an adequate sample of student’s reading
ability is acquired or until student becomes
frustrated.
Brozo09
70
Steps in Conducting an Assessment
of Oral Reading Miscues Using the ALI




Once a student has completed reading aloud a high interest
content passage, the total number of words the student
was unable to decode or skipped are tallied. The same
word is not counted twice. Proper nouns are not counted;
Nor are repetitions or self-corrections.
Next, the recorded miscues are reviewed and a
determination is made about which ones are significant.
A significant miscue alters the meaning in such a way that
it interferes with the student’s ability to comprehend the
passage.
To know whether comprehension was depressed because of
oral reading miscues, it is essential to look closely at the
student’s responses to the comprehension questions.
Brozo09
71
Steps in Conducting an Assessment
of Oral Reading Miscues Using the ALI




Once the total number of significant miscues is derived, it is
divided by the total number of words of the passage.
This oral reading miscue score is combined with the reading
comprehension score to establish an overall reading level
for a given high interest passage.
Using the scoring guide at the bottom of the examiner’s
copy, users can determine the reading level for a particular
passage.
In cases in which the oral reading and comprehension
scores yield different reading levels, we recommend
following special guidelines for establishing an overall
reading level for a given passage.
Brozo09
72
Establishing Overall Passage Reading Level on the ALI
When Oral Reading and Comprehension Scores Differ
Reading Level
based on Oral
Reading Miscues
Reading Level
based on
Comprehension
Questions
Overall Reading
Level
Independent
Instructional
Instructional
Independent
Frustration
Frustration
Instructional
Independent
Independent
Instructional
Frustration
Frustration
Frustration
Independent
Instructional
Frustration
Frustration
Frustration
Brozo09
73
The ALI Allows Diagnosis of
Several Patterns of Reading Challenges
1. The Reader Who Lacks Prior Knowledge for The Text
2. The Reader Who Needs to Better Understand That Reading
Is Done to Construct Meaning from Text And Not to
Demonstrate Perfect Oral Reading
3. The Student Who Focuses on Word by Word Reading But
Does Not Understand
4. The Student Who Has Individual Skills and Strategies But
Isn’t Able To Coordinate Them to Achieve Success in
Reading
5. The Reader Who is Experiencing Word-Level Processing
Bottlenecks
Brozo09
74
Pattern 1
Text Impression Activity
8th Grade Science Textbook Passage #1
What is Science? Pre-Reading Activity

with subdividing science

solve practical problems

world dramatically by

the application of

antibiotics

interdependent

technology

a relatively small amount

use of transistors
“Dre” Wrote:
Some people like to subdivide
science. Science can help solve
practical problems. Everything in
the world is dramatic with science.
The application of science is hard
for some people. Even a relatively
small amount of science is a good
thing. The use of transistors means
science is interdependent.
Antibiotics and technology are
science.
Brozo09
75
Pattern 1
“Dre’s Vocabulary Self-Awareness Chart
Word
illustrates
application
+ √
antibiotics
Example Definition
+
a comic book
To draw something
+
like at
McDonalds
To apply to a job
shooting free
throws
To practice something
aunt, uncle
your relatives
√
practical
relatively
-
+
--
76
Pattern 5
10th Grade Student Cloze with Maze
Occupation: Conductorette from I Know Why the Caged Bird Sings by
Maya Angelou
Portion of ―Maria’s‖
performance on
the cloze-maze
passage
Not only are her
word selections
often incorrect,
but she also makes
significant miscues
The young Angelou has just returned to San
advance it
Francisco after an adventure-fitted trip. Things are
altering changing at home, she (2)
(1) varying, altering,
unearths discovers, and her
discerns, unearths,
brother moves out
Rested and distant
soon after her return. Restless, discontent, Angelou
thinks
(3) ponders, broods, thinks her next step.
Brozo09
77
Pattern 5
8th Grade Language Arts High Interest Passage
After Twenty Year
by O. Henry
―Maria’s‖ oral
reading miscues
For an 8th grade
High interest
Passage
She made
frequent meaning
changing thus
significant
miscues
a new
The policeman on the beat moved up the avenue
impress
his habit
impressively. The impressiveness was habitual
specials
and not for show, for spectators were few. The time was
guests
barely 10 o-clock at night, but chilly gusts of wind
night the people
with a taste of rain in them had well nigh depeopled the
streets.
Brozo09
78
Kinder sind unsere Hoffnung für
die Zukunft,
Sie sind die Hoffnung der Kinder
fuer heute.
Herzlichen Dank fuer Ihre
Aufmerksamkeit!
Bill Brozo
Brozo09
79

Kinder sind unsere Hoffnung für die
Zukunft,
Sie sind die Hoffnung der Kinder fuer
heute.

"Herzlichen Dank fuer Ihre
Aufmerksamkeit"
Brozo09
80
Principle 4

Use assessment as a tool for learning and
future growth
Brozo09
81
Vocabulary Self-Assessment

Provide a list of key words to students at the beginning of the unit
and have them complete a self-assessment of their knowledge of
the words using a chart (see next slide).

Do not give students definitions or examples at this stage. Ask
students to rate their understanding of each word with either a
―+‖ (understand well), a ―√‖ (limited understanding or unsure), or
a ―—― (don’t know).

Over the course of the readings and exposure to other information
sources throughout the unit, students should be told to return
often to the chart and add new information to it.

The goal is to replace all the check marks and minus signs with a
plus sign. Because students continually revisit their vocabulary
charts to revise their entries, they have multiple opportunities to
practice and extend their growing understanding of key terms
related to the topic.
82
Content Literacy Inventory

Assess students’ reading, thinking, and
study processes with content texts teacher
plans to use for instruction

Results can be turned into instruction to
address specific reading and study needs
83
Content Literacy Inventory
Using Book Parts
1. On what pages can you find information on smoking and driving?
2. In what part of the book can you find the meaning of kinetic energy?
Understanding Graphs and Charts
1. According to the chart on page 61, what is the second-largest cause of
rural fatal accidents?
2. What does the graph on page 334 imply about the relationship
between speed and fuel consumption?
Vocabulary in Context
1. What does the word converse mean in the following sentence: Do not
take your eyes off the road to converse with a passenger.
2. What does the word enables mean in this sentence:
It enables you to carry out your decisions promptly and in just
the way you planned.
84
Content Literacy Inventory
Summarizing and Sensing Key Ideas
1. Write a one-page summary for the section entitled ―A Defensive
Driver’s Decision Steps‖ on page 101. Be sure to include in
your summary the key ideas an any other pertinent
information. Use your own words as you write your
summary.
2. In your own words, state the key idea of the paragraph on
page 262, second paragraph from the top.
Creating Study Reading Aids
1. Imagine that you will have a multiple-choice and short-answer
test on Chapter 18. Organize the material in that chapter by
taking notes over it or by creating some form of study aid.
85
Overview
AS A RESULT OF THE CLASS, STUDENTS WILL:
•
Understand a constructivist rationale for
content literacy strategies
•
Be familiar with a variety of literacy strategies
through descriptions, examples, and
demonstration activities
•
Be able to produce through guided practice
useful applications of the strategies
Brozo09
86
Guiding Questions for Viewing
the Split-Page Notetaking Strategy

To what extent does the teacher describe
and model the notetaking strategy for her
students?

How are students provided guided practice
in split-page notetaking?

What else would the teacher need to do to
ensure all students were effective note
takers using the split-page method?
Brozo09
87
Guidelines for Content Reading Assessment
Content Reading Assessment is a
Continuous Process of Becoming Informed
about Students’ Text Processing and
Knowledge Acquisition
 Assessment of Content Reading and
Learning Should Use Multiple Data
Sources Across Contexts and Over Time
 Content Reading Assessment Should
Maximize Student Involvement in the
Assessment Process

Brozo09
88
Why All Students Need To Be Good Readers

Better readers are better students in
all subject areas (C.F., NAEP, 2003;
2007)

Better readers are more successful
in their personal lives (C.F.,
Hofstetter, Sticht, & Hoffstetter,
1999)

Better readers are more
successful in their professional
lives (Sum, et. al., 2007)
Standardized achievement tests and
college entrance exams require high
levels of traditional print literacy 89
Brozo09
(C.F., Johnston & Costello, 2005)

Steps in Conducting
an Interactive Assessment with the ALI






Examiner begins by talking with the student about the
―different‖ way this assessment will occur.
The student is told that the examiner will be there for
support as they go through the passage together.
The student should be prepared for interruptions when the
examiner feels it is necessary to model a strategy or assist
with a problem.
Work with the passages is begun at the student’s
Instructional level reading.
This can be determined by using the maze placement
passages
Either type of passage (i.e., high interest content, or
textbook) is optional for interactive assessment.
Brozo09
90
Steps in Conducting
an Interactive Assessment with the ALI



Text impression words are discussed. The
examiner models how to write a sentence or two
using a few of the words. Student and examiner
can trade off writing related sentences that
incorporate the impression words.
As this occurs, examiner takes note of words that
present problems for the student and how the
student constructs contexts for the words.
This provides the opportunity to teach the
student how to use the impression words to
access relevant prior knowledge before reading
and how to write coherent and meaningful
Brozo09
91
statements.
Steps in Conducting
an Interactive Assessment with the ALI



Next, the examiner demonstrates for the student
how to complete the vocabulary self-awareness
chart. The examiner models making entries for a
couple of the terms and guides the student in
filling out the others.
The examiner talks about the importance of
making connections between prior knowledge
and key vocabulary.
The examiner discusses the value of tracking
one’s evolving understanding of important terms
as material is read and words are contextualized.
Brozo09
92
Steps in Conducting
an Interactive Assessment with the ALI




Examiner and student move on to the Venn diagram as they refer
back to the text impression the student (and examiner) wrote.
Examiner demonstrates how to organize ideas from the
impression text in the Venn format. In the left circle, examiner
paraphrases one or two points from the impression text, then
guides the student in doing the same with the remaining points.
The examiner might label the left side of the Venn diagram with
―Ana’s Key Points‖ or ―What Devon Says‖.
The student is told that as s/he reads the passage with the
examiner, they will return to the Venn diagram to add key points
in the right circle, which might be labeled ―What the Author Says.‖
Ideas that are shared by the student and the author should be
written in the middle, overlapping section of the Venn diagram.
Brozo09
93
Steps in Conducting
an Interactive Assessment with the ALI




At this point, the examiner and the student should begin
reading the selected passage.
Since interactive assessment is usually conducted with
adolescent readers who are struggling and because it
requires modeling and eliciting of important reading
processes, passages should be read aloud.
Reading aloud allows the examiner to observe and identify
problems with decoding the student might exhibit while
affording the examiner the opportunity to demonstrate
fluent reading.
The examiner is urged to trade off with the student each
taking a turn to reading a paragraph.
Brozo09
94
Steps in Conducting
an Interactive Assessment with the ALI








There are a range of interactions appropriate during oral reading.
This part of the interactive assessment allows the examiner to model critical reading
comprehension processes.
For example, the examiner can read certain excerpts with expression and ask the
student to imitate examiner’s fluent rendition.
Examiner should intervene to demonstrate a decoding strategy when the student
struggles with a word.
Examiner and student can trade off asking and answering each other’s questions
after reading a paragraph.
Examiner can take advantage of the interspersed questions built into the passage to
reflect on the important ideas in the text.
Examiner can also supply an oral summary of a paragraph, explaining it was
constructed it based on the information and ideas in the text. Then, after the
student reads the next paragraph, he/she can give a summary with explanation.
This process of demonstrating a strategy then eliciting the same from the student will
offers vital information about whether the student is learning the thinking and
reading strategies being modeled. If not, additional modeling will be necessary.
Brozo09
95
Steps in Conducting
an Interactive Assessment with the ALI








As the examiner and the student read the passage, they can refer frequently to the
Venn diagram and the Vocabulary Self-Awareness chart.
Examiner demonstrates for the student how and what to add to the diagram and
chart, explaining how to do so with evidence from the passage.
As key words are encountered in the passage, they can be added to the chart.
Examiner can show the student how to use context clues to determine or narrow the
meaning of the terms.
Examiner continues to look for evidence that the approaches being used to facilitate
word learning are effective by noting the student’s use of the strategies modeled.
Likewise, examiner looks for reading and thinking behaviors similar to the ones
modeled as the student attempts to fill in the right circle and overlapping portion of
the Venn.
Examiner discusses how closely the student’s pre-reading ideas written in the form of
an impression text and recorded in the Venn diagram match the author’s.
Student is remind of the importance of activating prior knowledge for the passage
content.
Brozo09
96
Steps in Conducting
an Interactive Assessment with the ALI




Examiner goes over the post-reading
comprehension questions with student.
Examiner models how to return to the text and
skim and scan for answers, then requests
demonstrations of these same reading processes
from the student.
Examiner models how answers to higher-level
questions may not be found directly in the text
but will require inferential thinking.
Examiner and student trade off reading and
answering questions, supporting one another in
the process of finding answers.
Brozo09
97