Reading Renaissance
Testing the Reading Renaissance Program Theory:
A Multilevel Analysis of Student and Classroom Effects on Reading Achievement
Geoffrey D. Borman
N. Maritza Dowling
University of Wisconsin-Madison
1
Reading Renaissance
2
In the recent National Reading Panel report, the panel concluded that guided oral
reading is an important component for developing reading fluency (NICHD, 2000). In
the panel’s review, it found that guided oral reading helped students across a wide range
of grade levels to learn to recognize new words, read accurately and easily, and
comprehend what they read. In contrast to these benefits associated with guided oral
reading, the panel found considerably less research to support the practice of silent
independent reading. Though the Panel’s conclusions did not refute that encouraging
children to read more improves reading achievement, the report did state that guided oral
reading procedures are more effective in this regard than is free, voluntary silent reading.
One of the major benefits of the oral reading approach is that it can serve as an
important diagnostic tool for teachers to assess a student’s word attack skills, fluency,
and comprehension by listening to the child’s emphasis and phrasing. By
overemphasizing oral reading, though, the teacher may divert a student’s attention from
comprehension, especially if the teacher stresses reading the text absolutely correctly
over understanding its meaning (Graves, Juel, & Graves, 1998). On the other hand, there
is evidence suggesting that encouraging a greater amount of silent reading time in which
students read extensively and have ownership over the materials they choose to read can
contribute to higher intrinsic motivation to read (Ivey & Broaddus, 2001), and increased
vocabulary and comprehension skills (Cunningham & Stanovich, 1998).
The Reading Renaissance (RR) program developed by Renaissance Learning,
offers the structure and diagnostic feedback that is a key strength of the oral reading
approach along with the flexible and student-centered emphasis of silent reading.
Diagnostic feedback is provided by the computerized Accelerated Reader (AR)
Reading Renaissance
3
information system, which was first made available to schools in 1986 and, according to
the developers, is currently used in more than half of the schools in the U.S. (Paul, 2003).
The AR provides both students and instructors with immediate feedback on student
reading practice through short, computer-based quizzes that assess students’
comprehension of the books they have read. The teacher then uses the information to
guide further reading by helping students choose books that are appropriate to their
ability levels and interests and diagnosing reading difficulties for individual students and
the classroom as a whole. With more than 65,000 AR Reading Practice Quizzes, there
are assessments available for a wide range of books. The independent reading program
coupled with the systematic feedback provided by AR form the core of the RR guided
independent reading program.
Using a large-scale database of students and classrooms, this analysis tested the
central premises that underlie the RR guided independent reading program. These
premises are that higher-quality implementations of the program are marked by a greater
amount of reading, a high reading success rate, use of reading material that is
appropriately matched to a student’s ability and, ultimately, that these indicators of
implementation are related to improved reading achievement. Specifically, applying a
two-level hierarchical linear model, with students nested within RR classrooms, we
assess the relationships between students’ individual reading behaviors and achievement
and the relationship between classroom-level implementation of RR and its overall and
compensatory effects on achievement outcomes.
Reading Renaissance
4
Background
RR uses the computer-provided information to track and manage the development
of students’ reading skills and to motivate students of all ages to read more and better
books. Although the program is available for K-12, most of the sites using RR are
elementary and middle schools. The RR program has 4 main components: reading
practice; the computerized AR system; a motivational system; and the teacher’s role to
motivate, instruct, monitor, and intervene. Teachers adopting the RR program design
independent reading as part of classroom instruction along with the diagnostic feedback
provided by the AR system. Using information from AR, teachers set goals for each
student on an individual basis. Teachers monitor students’ progress and confirm that
students are reading appropriate books that match their reading level and that they are
making progress. Teachers also provide instruction on reading skills needed to improve
reading comprehension and they intervene when students are not successful, helping
them choose appropriate books and use more appropriate reading strategies.
RR recommends three goals for independent reading practice: quality, quantity,
and challenge. Quality is measured by the average percent correct on the AR quizzes.
The program stresses high levels of successful reading, with an average percent correct of
85% or more on the AR quizzes of students’ comprehension of the books they have read.
The quantity goal is determined by the amount of in-school time that students have spent
engaged in guided independent literature-based reading of self-selected books. The last
factor or goal, challenge is related to students being guided to select reading material that
is at the appropriate difficulty level for their reading abilities. These goals of quality,
quantity, and challenge that drive high-quality implementations of the program have been
Reading Renaissance
5
informed by research and by the developers’ practical experiences implementing AR and
RR across the country.
Quality: Percent Correct
The average percent correct on AR quizzes is the key indicator that students are
enjoying success as independent readers and is an important indicator of high-quality
implementation of the RR program. There is extensive research showing how high levels
of success result in large reading achievement gains (Allington, 1984; Rosenshine &
Stevens, 1984; Betts, 1946). It has been argued that success rate on tasks, as measured by
percent correct, is a powerful indicator not only assessing comprehension but also
engagement and, to some extent, motivation and interest (Cunninghan & Cunninghan,
2002). High success also indicates that students are receiving sufficient help to meet
their needs, context clues necessary to learn new material, and appropriate scaffolding
(Beck & MacKeown, 1991). Low-achievers appear to benefit most from high rates of
success compared to high-achievers. For example, Marliave & Filby (1985) found that
with low-achieving students, high rates of success have larger effects on learning than
engaged time on task.
The most important achievement goal for all students in RR programs is to
maintain a minimum average of 85% correct on the AR reading practice quizzes. In fact,
in the grades 4-12, the developers suggest maintaining an average in the 90% range to
optimize reading growth (Paul, 2003). Initial minimum percent correct goals are often
set at 85% to prevent students from getting discouraged at the outset of the program.
Since the RR model stresses formative evaluation and continuous student monitoring,
diagnosis, feedback, and reading adjustment, if students have difficulties maintaining the
Reading Renaissance
6
minimum average percent correct, instructors may decide on an appropriate strategy to
help them succeed. Guidance is often provided on selecting appropriate books and
students also are taught strategies, such as re-reading or taking and reviewing notes
before taking the quizzes.
Quantity: Number of Words Read
The most fundamental indicator of student progress in a guided independent
reading program designed to motivate students to read more is, of course, the quantity of
text read. As noted in the National Reading Panel report, there are literally hundreds of
correlational studies that have shown that the best readers read the most and the worst
readers read the least (NICHD, 2000). This suggests that the more children read, the
better their literacy outcomes. As the panel noted, though, it could also be the case that
better readers simply choose to read more. In a comprehensive meta-analysis and
literature review on independent reading practices, reading exposure, and other correlates
of reading achievement, Lewis & Samuels (2003) also found strong empirical evidence
that students who read more are better readers and obtain higher overall achievement
scores than students who read less and spend less time reading. When this independent
reading is guided or monitored, as it is in RR, it leads to even greater gains in
achievement (Yu, 1993). The RR developers suggest that 60 minute of in-school time
per day be allocated to guided independent reading across all grades, 1-12 (Paul, 2003).
Though 60 minutes per day is more that some reading experts suggest, Allington (2001)
has recommended that students may benefit from up to 90 minutes per day.
Challenge: Zone of Proximal Development
Reading Renaissance
7
The idea that the potential for reading development depends upon the zone of
proximal development (ZPD) is based on principles introduced by Vygotsky (1978) in his
social development theory. Cognitive development, states Vygotsky, is limited to a
certain range at any given age. Translated into the RR philosophical context, optimal
learning takes place when students read books with readability levels that match their
ZPD. This idea has been supported in a number of research studies conducted since the
late 1970s applying Vygotsky’s theoretical principles (Din, 1998; Slavin, Karweit, &
Madden, 1989; Gickling & Armstron, 1978).
The ZPD is then conceptualized as a recommended book-level range for the
student geared to maximize the effectiveness of the reading program. It is a level of
difficulty that is not too hard or too easy for the student. In setting student learning goals,
RR provides the flexibility for teachers to choose a new range or create a wider one that
better matches the student’s abilities if books in the recommended range seem too hard or
easy for the student. After identifying an initial ZPD, teachers monitor students’ reading
practice and adjust their ZPD if necessary. The intention is to adequately match the ZPD
with the actual book reading ability for the student. Usually the starting point is near the
low end of the student’s ZPD. This practice allows students to select book levels within a
wider overall ZPD range.
Method
Sample and Data
The sample and data for this study were obtained from the Reading Practice
Database (RPD) assembled and maintained by the Research and Evaluation Department
of Renaissance Learning. The RPD is a large database that contains information on
Reading Renaissance
8
student guided independent literature-based reading. It was developed primarily for the
purpose of analyzing student reading practice patterns in schools implementing RR
schoolwide. The RPD contains the reading records of 50, 807 students in 139 U.S.
schools in grades 1-12 using the Accelerated Reader (AR) computerized information
system in the 2001-2002 school year. The data contained in RPD are not a nationally
representative sample of U. S. schools but rather a typical sample of schools participating
in the RR program.
To conduct the multilevel analyses, two files were created from the RPD: a
student-level file and a classroom-level file. Records in the student-level file were
included only if a student (a) belonged to a classroom with five or more student records
in the RPD, (b) had both pre- and posttest STAR Reading scores, (c) had some activity in
the AR between the pre- and posttest dates, and (d) had no missing information on the
grade level variable. Only six records in the student-level file had no information on
grade level. The sample was further restricted to students in grades 2 to 12, as students in
grade 1 do little independent reading. Therefore, 5,150 first-grade students were not
included the HLM analyses. The final sample consisted of 45,108 student records. This
sample spanned 2,434 classrooms with an average class size of 19 students. The
demographic characteristics of the students in the analytic sample are listed in Table 1.
___________________________
Insert Table 1 about here
___________________________
The classroom-level file was created from the student-level file by aggregating
the data by classroom and school identification code. Additional data regarding teacher
Reading Renaissance
9
certification status was added to the aggregated file based on data from Renaissance
Learning customer records. The explanatory variables were created similarly by taking
classroom-level averages (aggregates) of the student data. Separate HLM analyses were
conducted for the following three clusters of grades: 2nd to 5th graders, 6th to 8th graders,
and 9th to 12th graders.
Measures
Dependent variable. The dependent variable in the multilevel analysis was the
normal curve equivalent (NCE) posttest score obtained from the STAR Reading test.
NCEs are normalized percentile scores with a population mean of 50 and standard
deviation of 21.06. The STAR Reading test is a nationally normed standardized
computer-adaptive test developed by Renaissance Learning with test-retest reliabilities,
by grade, ranging from 0.70 to 0.91, and an overall reliability of 0.94. The scores on the
STAR reading achievement test in the present study ranged from an NCE of 1 to a
maximum of 99.
Student-level variables. Five student-level variables, summarized in Tables 2 to 4
for each cluster of grade-levels analyzed, were of interest in this study. Two explanatory
variables were continuous: (a) actual number of words read (calculated from the actual
number of words in books that students read for which they also passed the quiz), and (b)
NCE pretest scores from the fall STAR Reading achievement test. The natural logarithm
of the variable “actual number of words read” was taken to normalize the distribution of
the variable.
___________________________
Insert Tables 2-4 about here
Reading Renaissance
10
___________________________
The remaining three explanatory variables, grade-level, percent correct, and zone
of proximal development range (ZPD), were categorical. After the student-level file was
split into the three grade-level clusters, the variable grade was dummy coded using the
lowest grade in the cluster as the reference category. The percent correct variable was
developed through a two-step process. First, we calculated each student’s average score
based on all quizzes attempted by the student between the pre- and posttest. If the percent
correct was greater than or equal to 85%, then the variable was coded as “1” and if the
percent correct was less than 85% it was coded as “0.”
The ZPD variable was created from two variables: each student’s average book
level read and STAR pretest grade equivalent score. These two scores were used to
create book level ranges called ZPD ranges. Each ZPD included a range of book levels,
suggested by the developer, at which students should be able to read independently while
being challenged enough but without reaching a state of frustration that will block a
healthy motivation level. See Appendix for the specific ZPD ranges corresponding to
each pretest grade equivalent score. According to the ZPD range, students were coded as
(1) low ZPD or low challenge, (2) optimum ZPD or optimum challenge, or (3) high ZPD
or high challenge.
For instance, if a student obtained a pretest grade-equivalent score from 0.0 to 1.4
then s/he had an “optimum” ZPD or challenge range of 1.0 to 2.0. If the student’s actual
average book level assigned was between this range, then a code of 2, optimum
challenge, was assigned. If the actual book level assigned was below 1.0, then a code of
1, or low challenge, was assigned, and if it was above 2.0 it was coded as 3, meaning
Reading Renaissance
11
high ZPD or high challenge. A series of three dummy codes represented each category,
low, optimum, and high. In our analyses, the optimum category was treated as the
omitted and referent category.
Classroom-level variables. Most of the classroom-level predictor variables were
obtained by aggregating (averaging) student-level variables by classroom. Six variables
were of interest: (a) classroom mean STAR Reading NCE pretest score; (b) percentage of
students in the classroom with average quiz scores greater than 85%; (c) classroom mean
of log of number of words read; (d) number of students in the classroom; (e) percentage
of students in the classroom receiving assignments at the optimum reading challenge or
ZPD; and (f) the RR teacher’s master certification status. A summary of these variables
is also presented in Tables 2 to 4. Two variables, the number of students in the classroom
and the RR teacher’s certification status, were obtained from the classroom-level data
created by Renaissance Learning. The teacher certification variable was dummy coded
so that master certification equaled 1 and all other non-master certified teachers were
coded as 0.
Procedure
Using a hierarchical linear modeling (HLM) approach (Raudenbush & Bryk,
2002), the analyses distinguished between student-level and classroom-level effects of a
greater amount of reading, a high reading success rate, and use of reading material that is
appropriately matched to students’ abilities. The analysis helped us understand how
much variation in achievement is at the classroom versus student level and how much
variation can be explained using both student and classroom-level predictors of the
outcome. This information allowed us to identify more clearly the effects that quality
Reading Renaissance
12
classroom-level implementations of the RR tenets had on students’ test score gains that
were above and beyond the student-level effects of their own reading behaviors.
In addition, the analyses examined aptitude-by-treatment (ATI) interaction effects
by assessing whether higher-quality guided independent reading implementations closed
the within-classroom achievement gaps between students who scored higher and lower
on the baseline pretest. In these ways, our analyses assessed the extent to which the
model practices of RR were associated with both excellence and equity in students’
achievement outcomes.
We developed three separate series of two-level HLM models for: the elementary
school sample (grades 2-5), middle school sample (grades 6-8), and high school sample
(grades 9-12). In each case, student performance on the spring reading test was the
outcome predicted by students’ fall pretest score, grade level, number of words read, the
dummy code indicating that the child’s average percent correct was above 85 on the
STAR quizzes, and two dummy codes indicating that the assigned reading material was,
on average, below or above the ability indicated by the student’s baseline pretest level.
At the second level of the model, we used the classroom-level aggregates of four student
variables to form the classroom average pretest score, number of words read, the
classroom-mean percent correct above 85 percent correct, and the percentage of students
in the class assigned reading material at the optimum ability level. In addition, the class
size was used as a covariate and the teacher’s status as a master RR teacher was used as
an additional measure of high-quality implementation.
For each of the three groups we studied, elementary, middle, and high school, we
began by specifying an unconditional multilevel model, with no student or classroom
Reading Renaissance
13
predictors of the reading achievement outcome. This model decomposed the variance in
the outcome into its within- and between-classroom components and provided an upperbound estimate of the proportion of variability in reading achievement that can be
explained by differences across classrooms. The second set of multilevel models that we
estimated introduced the fall pretest and the students’ specific grade levels as covariates
to predict achievement. These models estimated inequalities in students’ outcomes and
accounted for within-school variability that was associated with students’ initial
achievement and grade level.
After assessing the variability in achievement associated with these student-level
characteristics, the third model turned to measures of students’ individual reading
behaviors as predictors of achievement. These models estimated the relationships
between achievement and the number of words a student read, whether the child attained
the 85% correct standard on STAR quizzes, and whether each student was assigned
reading material below or above his/her optimum ability level, after controlling for
pretest and grade level.
In the fourth model, along with the student-level predictors included in the third
model, we modeled the classroom-level mean pretest score and the class size as
predictors of between-classroom variability in the outcome. Fifth, after controlling for
the class size and the classroom-level average pretest score, we included the classroomlevel aggregates of student reading behaviors: classroom mean words read; classroom
percentage of students achieving at or above the 85% correct standard; and percentage of
students in the classroom assigned reading material that is at their optimum reading level.
These variables, along with an indicator of whether or not the classroom teacher was a
Reading Renaissance
14
master RR instructor that was added in the sixth and final model, provided classroomlevel measures of the quality of the RR implementation. In models 5 and 6, we also used
this collection of implementation measures to model classroom-to-classroom differences
in the within-school pretest-posttest slope. That is, did better implementations of the
program help attenuate the relationship between pretest and posttest and, thus, improve
equality of educational outcomes within classrooms?
Results
Outcomes for Students and Classrooms from Grades 2-5
Tables 5 and 6 display the maximum likelihood (ML) results for grades 2-5
starting with Model 1, the empty model, to Model 6. The first analytical model, the
empty or unconditional multilevel model with no student- or classroom-level predictors,
shows the overall average value on the outcome measure. This model partitions the
variance in the outcome into its between and within classroom components and tests
whether there is a statistically significant amount of between-classroom variance to
model with independent variables. For the reading achievement outcome, the
unconditional model yielded an average score of 51.78 NCEs. In other words, students
were, on average, scoring just above the national average NCE, or the percentile score of
50. The intraclass correlation coefficient, which represents the proportion of variance in
the outcome that is between classrooms was 0.26. This result indicated that 26% of the
variance in reading achievement was between classrooms and that there was a
statistically significant, χ2 (1337, N = 1338) = 9,269.85, amount of variability in the
posttest outcome potentially explainable by classroom-level characteristics.
___________________________
Reading Renaissance
15
Insert Tables 5-6 about here
___________________________
Our next steps involved attempting to control pretest reading achievement and the
grade level of the child. With the exception of the pretest predictor, our HLM models
treated all other student-level indicators and measures as fixed slopes. That is, it was
assumed that the effect of most student-level predictors was homogeneous across
classrooms. We chose this model due to both practical and theoretical considerations.
From a practical standpoint, classrooms typically served only a single grade level, 2, 3, 4,
or 5. Having no variability on these student-level grade indicators for the vast majority of
classrooms, it was not possible to model these grade level indicators as sources of
random variation within classrooms. In addition, from an analytical and theoretical
perspective, the primary focus of the current study was on the sources of between-school
differences in mean verbal achievement rather than on processes of within-school
achievement differentiation. The one exception was, of course, the within-classroom
inequalities associated with students starting the academic year at a higher or lower
pretest level.
In the initial prediction model, Model 2, the pretest and grade level explained
61.70% of the between-classroom variance. Therefore, the student-level predictors did
account for considerable between-classroom variability, but a statistically significant
amount of between-classroom variability remained even after controlling for these
variables.
All of the variables were statistically significant predictors of reading
achievement. On average, students beginning the academic year during the fall with
Reading Renaissance
16
higher test scores obtained posttest scores that were also higher. For every one point
increase on the pretest, students showed a 0.74 point increase on the posttest. After
controlling for pretest, students from the second grade tended to outperform students
from the higher elementary grades.
The HLM results also showed that there were statistically significant, χ2
(df=1337, N = 1337) = 254.08, differences across schools in the relationship between
pretest and posttest. These results provided evidence that the distribution of achievement
varied across schools. That is, some schools were more equitable and some were less
equitable with respect to the relationship between students’ pretest scores and their
subsequent posttest outcomes.
Model 3 introduced the student-level measures of reading behaviors. All 3
variables were statistically significant predictors of posttest reading achievement.
Achieving the 85% correct standard was related to higher posttest scores. The difference
between a student scoring at or above 85% correct and a student performing below the
standard was associated with a nearly 4 NCE point difference. Also, the difference
between a student who read a considerable amount of text and a student who read only an
average amount of text (i.e., a 2 standard deviation difference on the actual number of
words read variable) was equivalent to an advantage of over 10 NCE points for the
student who read more text. Finally, after controlling for the other variables in the model,
students who were assigned readings that were, on average, below their optimum
challenge level tended to perform 3.19 NCEs lower on the posttest than those students
who were assigned readings at their optimum level. When students were assigned books
that were above their optimum reading level, students tended to score 3.65 NCEs higher
Reading Renaissance
17
on the reading posttest than students who were reading at their optimum level. These
reading behaviors explained nearly 5% of additional within-classroom variability on the
posttest.
In Model 4 in Table 6, we added the classroom-level mean pretest score and class
size as predictors of differences in the true classroom means. Both variables were
statistically significant predictors of posttest. For each one point increase on the mean
classroom-level pretest, classrooms were predicted to gain an additional 0.88 points on
the posttest. Students attending larger classes actually tended to achieve better posttest
outcomes than students attending smaller classes, though the result was not of
considerable practical significance. For each additional 10 students in the class, the
classrooms attained mean posttest scores that were 0.80 points higher. Together, the
classroom-level pretest and the number of students within the classroom explained over
86% of the variance that lied between classrooms.
Next, in Model 5, we included the mean words read, percentage of students
reading at the optimum level, and the percentage of students achieving 85% correct or
greater as predictors of the classroom mean achievement outcome and the NCE pretest
slope. For classroom mean achievement, the number of words read and the percent
correct greater than 85% were the two classroom-level indicators to attain statistical
significance. The outcome for the number of words read variable suggested that
classrooms that engaged in a considerable amount of reading (i.e., 2 standard deviations
above the mean for the classroom-mean log number of words read) outperformed
classrooms that engaged in an average amount of reading by approximately 4.5 NCE
points. Also, the HLM predicted a 1.8 NCE point difference between a classroom with
Reading Renaissance
18
100% of its students attaining the 85% correct standard and a classroom in which no
students met the standard. After controlling for these variables, the number of students in
the classroom was no longer a statistically significant predictor of reading achievement.
For the other primary outcome of Model 5, the NCE pretest slope, this model
indicated a statistically significant relationship between classrooms that produced a
greater amount of reading and the widening of the achievement gap between initially
lower and higher achieving students. However, this relationship was of little practical
significance. Classrooms that were two standard deviations above the mean on the
classroom-mean log number of words read variable widened the achievement gap by only
a fraction of an NCE point, 0.04 NCEs.
Finally, Model 6 added the indicator variable that distinguished master RR
teachers from those who had not achieved this status. Relative to the non-master teacher,
master teachers’ classrooms scored nearly 1 NCE higher, even after taking into account
the other classroom-level indicators of high-quality RR implementation.
Outcomes for Students and Classrooms from Grades 6-8
The estimated coefficients from each of the six two-level models for the middle
school grades, 6 to 8, are shown in Tables 7 and 8. The unconditional model, Model 1 in
Table 7, yielded an expected mean reading test score for all classrooms of 44.58 NCEs.
Therefore, the group of 6th to 8th graders in this sample, on average, scored somewhat
below the national average percentile score of 50. The intraclass correlation, or measure
of dependence between level-2 (classroom) units, was 0.38. That is, 38% of the total
variance in reading test scores for this sample was associated with classrooms as opposed
to individuals. This amount of variability was statistically significant, χ2 (df= 415, N =
Reading Renaissance
19
416) = 4490.77. In this case, we rejected the null hypothesis that mean test scores of
students from all classrooms under study were equal, and concluded that there was
significant variability in means across classrooms to justify a multilevel analysis.
___________________________
Insert Tables 7-8 about here
___________________________
Controlling for pretest reading achievement and the grade level of the child in
Model 2, helped explain 59.72% of the total within-classroom variance. However, the χ2
test of the level-2 residual variance revealed that even after controlling for these
variables, a significant amount of between-classroom variance remained unexplained.
The grade level dummy variables were not statistically significant predictors of reading
achievement. NCE pretest score, however, was statistically significant. Students
beginning the academic year during the fall with higher pretest scores, on average, also
performed better on posttest scores. For the middle school group, a one-point increase on
the pretest scores was associated with a 0.77 point increase in the posttest reading score.
In addition, the test statistic for the variance component estimate for the pretest slope was
statistically significant, χ2 (df = 413, N = 416) = 496.79, indicating variation in pretest
achievement across all classrooms.
In Model 3, we introduced the student-level indicators of reading achievement
progress: the 85% correct indicator; the actual number of words read; and the two dummy
indicators for ZPD, or challenge: low challenge and high challenge. Among the added
predictors, several statistically significant effects emerged. Having an average percent
correct greater than 85% had a positive association with posttest scores. After
Reading Renaissance
20
controlling for the other student-level characteristics, students who met the standard held
a 3.34 NCE point advantage over students who did not meet the 85% standard. The
coefficient estimate for the number of words read indicated that the difference between a
student who read a considerable amount of text and a student who read only an average
amount of text (i.e., a 2 standard deviation difference on the actual number of words read
variable) was equivalent to an advantage of 4.33 NCE points favoring the student who
read more text. Students who were assigned average book levels that were below their
optimum challenge or ZPD level tended to score approximately 2.59 NCE units lower on
the posttest compared to students who were reading books at an optimum challenge level.
On the other hand, students reading books above their optimum challenge or ZPD level,
on average, tended to score 1.74 NCE units higher on the posttest than students reading
books at their optimum challenge level. Adding these individual-level measures of
reading achievement progress accounted for an additional 1.73% of the initial withinclassroom variation in posttest scores.
Having established that there was statistically significant across-classroom
variation, both in term of average posttest scores and the pretest slopes, Models 4 to 6 in
Table 8 were developed to account for this variation. In Model 4, we first added two
level-two covariates: the classroom mean pretest score and the number of students in the
classroom. Classroom mean pretest score was a statistically significant predictor of
reading achievement for the middle-school sample. On average, for each unit increase in
classroom mean pretest score, middle school classrooms were expected to gain 0.93 NCE
points on the posttest. Model 4 accounted for approximately 97% of the initial betweenclassroom variance in posttest scores.
Reading Renaissance
21
In the next model, Model 5, we included three more level-2 variables, the mean
number of words read, percentage of students reading at the optimum level, and the 85%
correct indicator for the classrooms, as predictors of both the classroom mean
achievement outcome and the NCE pretest slope. For classroom-level mean
achievement, the classroom-level 85% correct variable was the one statistically
significant predictor. The model predicted a 3.08 NCE point difference between a
classroom with 100% of its students attaining the 85% correct standard and a classroom
in which no students met the standard. None of these variables, though, were statistically
significant predictors of the NCE pretest slope. This model helped explain only an
additional 0.65% of the total between-classroom variation.
Finally, Model 6 introduced a dummy variable that distinguished teachers who
had received RR master status from those who had not achieved this status. After
controlling for the other student- and classroom-level indicators of high-quality RR
implementation, master teachers’ classrooms for the middle school sample scored over 2
NCE units higher on reading achievement than the non-master teachers’ classrooms.
Adding this variable to the model accounted for an additional 0.17% of the total betweenclassroom variance in posttest scores.
Outcomes for Students and Classrooms from Grade 9-12
The ML estimates from Model 1 to Model 6 for the high-school sample (grades 9
through 12) are displayed in Tables 9 and 10. The unconditional model for the high
school sample produced an average reading achievement score of 47.50 NCEs, which is
slightly below the national average NCE of 50. The estimate of the intraclass correlation,
revealed that 28% of the total variance in reading achievement resided between
Reading Renaissance
22
classrooms. This variance was statistically significant, χ2 (df= 349, N = 350) = 2302.22,
indicating the need to model meaningful classroom-level properties that would help
explain sources of variability.
___________________________
Insert Tables 9-10 about here
___________________________
With the exception of the dummy variable “Grade 12,” all predictors in Model 2
were statistically significant. On average, students with higher NCE pretest scores
obtained NCE posttest scores that were also higher. For every one point increase on the
pretest, students showed a 0.75 point increase on the posttest. After controlling for the
pretest, students from the 10th and 11th grade tended to outperform students from the
referent 9th grade level. This model accounted for approximately 56% of the initial
within-classroom variance in NCE scores on the STAR Reading test. Additionally,
statistically significant variation across classrooms both in terms of the intercept, or
average test scores, (χ2 = 4895.01, df= 347), and the pretest slope, (χ2 = 464.68, df= 347),
lead us to reject the hypothesis of constant distribution of achievement across classrooms.
The next model, Model 3, introduced the three student-level measures of reading
behaviors. All of these variables proved to be statistically significant predictors of
posttest outcomes. First, the difference between a student scoring at or above 85%
correct and a student performing below 85% correct was associated with a 3.17 NCE
point difference. Second, the difference between a student who read a large amount of
text and a student who read an average amount of text (i.e., a 2 standard deviation
difference on the actual number of words read variable) was equivalent to an advantage
Reading Renaissance
23
of 2.66 NCE points. Third, after controlling for the other variables in the model, students
who were assigned readings that were, on average, below their optimum challenge level
tended to perform over 3 NCEs lower on the posttest than those students who were
assigned readings at their optimum level. Finally, when students were assigned books
that were above their optimum reading level, students tended to score 1.83 NCEs higher
on the posttest than students who were reading at their optimum level. These reading
behaviors explained 1% of additional within-classroom variability on the posttest.
The level-2 predictor variables introduced in Models 4 to 6 in Table 10 attempted
to account for the significant variation in variance components across classrooms in
average test scores and the pretest slope. First, in Model 4, we added the classroom-level
mean pretest score and class size as predictors of differences in expected classroom
means. Only classroom mean pretest score was a statistically significant predictor of the
reading achievement posttest. The coefficient estimate indicated that, on average, a oneunit increase in a classroom’s mean pretest score was associated with a 0.93-point
increase in its posttest reading achievement score. Model 4 accounted for approximately
94% of the initial variance in posttest scores that lied between classrooms.
In Model 5, we included the mean number of words read, percentage of students
reading at the optimum level, and the percentage of students achieving 85% correct or
greater as predictors of both the classroom mean achievement outcome and the NCE
pretest slope. For classroom mean achievement, the average optimum reading challenge
and the percentage of students achieving 85% correct or greater both reached statistical
significance. After controlling for all the other variables in the model, students in
classrooms with a higher average number of students reading books at their optimum
Reading Renaissance
24
challenge or ZPD level, showed greater improvement in their reading skills compared to
students in classrooms in which this was not the case. If one compared a class in which
all students were challenged at the optimum level to a class in which none of the students
were challenged at the optimum level, the model predicted a difference of 6.69 NCE
points. In addition, classrooms in which 100% of the students were achieving 85% or
greater on the AR quizzes outperformed classrooms in which none of the students were
meeting the 85% standard by 4.21 NCEs.
Examining the estimates for the pretest slope, or the average within-classroom
relationship between students’ pretest and posttest outcomes, the results indicated that
classrooms in which a higher percentage of students were reading at the optimum level
tended to close the achievement gap between initially lower and higher achieving
students. Also, classrooms with a higher percentage of students attaining the 85% correct
standard tended to widen the achievement gap between initially lower and higher
achieving students. However, as the results in Model 6 revealed, this relationship was no
longer statistically significant in the final model, which controlled teachers’ master
certification status.
In the final model, Model 6, the indicator variable that distinguished master RR
teachers from those who had not achieved this status was introduced. Interestingly,
master certification status was not a significant predictor of reading achievement for the
high school sample. However, the sample of master teachers at the high-school level was
so small that this coefficient was estimated with considerable error. The final model for
the pretest slope continued to show that classrooms with a higher percentage of students
reading at the optimum challenge closed the achievement gap. However, as stated above,
Reading Renaissance
25
after controlling for master teacher status the percentage of students attaining 85% correct
on AR quizzes was no longer a statistically significant predictor of the relationship
between pretest and posttest across the classrooms in the sample.
Discussion
The results of our analyses show that student performance on a spring reading
achievement test is predicted by both students’ individual reading behaviors, which are
shaped by the guided independent reading program, and by classroom-level features of
the RR program implementation. First, we find that even after using rigorous statistical
controls for the students’ initial reading ability levels, their reading success rate, and their
reading challenge level, the amount of text that an elementary or middle-school child
reads is a key predictor of his or her literacy development over the school year. This
result provides further support for the intuitive, yet often disputed, idea that regardless of
a child’s initial ability level, more reading is associated with greater learning.
Also consistent with the RR program theory, the student-level results suggest that
a high success rate over the course of the school year predicts better outcomes at the end
of the year. This finding is consistent across all samples, the elementary, middle-school,
and high-school groups. In contrast to the general theory of the model, though, after
controlling for students’ baseline scores, number of words read, and reading success rate,
students who were assigned reading material that was, on average, beyond their baseline
ability performed better on the posttest than did students who were assigned material
within their optimum reading range. Consistent with the theory, though, students who
were assigned material below their optimum reading range performed worse on the
outcome than did students who read material that tended to be within the optimum range.
Reading Renaissance
26
This result suggests that if students’ success rates are not suffering, teachers should
modify their plans and assign material to students that is above their apparent baseline
ability. In this respect, the finding supports suggestions provided to teachers by RR to
adjust book levels if the suggested optimum range appears to be too easy or difficult for
the student. This result was relatively uniform across all three groups—elementary,
middle school, and high school—we studied.
As much as 38% of the variance in the reading achievement outcome was
attributable to the classroom level. That is, beyond the individual behaviors of students,
as much as 38% of the variability in achievement is associated with differences across
RR classrooms. In the elementary grades, RR classrooms that effectively encouraged a
greater overall amount of guided independent reading and that maintained a high success
rate showed statistically significant improvements in the overall achievement level of the
classroom.
Beyond these results, there were statistically significant positive effects for
classrooms taught by master RR teachers. This finding held for both the elementary and
middle-school samples. Even after taking into account students’ initial abilities and their
individual reading behaviors along with the range of classroom-level measures of highquality implementation, these teachers brought to bear additional skills that promoted
student learning beyond the levels achieved by non-master teachers. These abilities are
likely ones that go beyond assuring that students are reading quality assignments and are
experiencing a high success rates while reading an array of literature. These skills may
include adept diagnosis of students’ reading problems, high-quality feedback, skilled use
Reading Renaissance
27
of incentives, and effectively teaching students appropriate strategies to help them
become better readers.
In the middle and high-school grades, improved posttest outcomes were
encouraged by those teachers who helped ensure that a higher percentage of students in
their classrooms were achieving the 85% correct standard. Among high school
classrooms, it was also the case that teachers who ensured that more of their students
were reading at the optimum reading level achieved a statistically significant and positive
difference on the posttest relative to teachers who maintained an optimum level for fewer
of their students. In contrast to the elementary school and middle school outcomes,
master teachers did not achieve outcomes that were different from those of non-master
RR teachers. The reason for this latter finding, though, was simply due to the fact that
only 1% of the high school teachers held a master certification and the outcomes for this
small group were estimated with considerable error.
Teachers must be sure that all students are moving ahead productively and that
lower-performing students are not falling behind in otherwise successful classrooms.
This point was raised by the result found for high school classrooms. Specifically, those
classrooms in which a higher percentage of students were receiving assignments targeted
toward their optimum reading challenge level closed the achievement gaps separating
students who entered the RR classroom with lower and higher initial reading abilities.
Overall, these results provide support for the RR guided independent reading
program theory. High-quality implementations of the program do appear to make a
difference, especially when the program is offered by a skilled master RR teacher.
Further, in the early grades, greater exposure to literature supports improved achievement
Reading Renaissance
28
and, in the later grades, a greater reading success rate makes a positive difference. These
findings have important implications for the RR guided independent program and for
future efforts to develop teachers’ skills in implementing the program and in promoting
improved student learning.
Reading Renaissance
29
References
Allington, R. L. (1984). Content coverage and contextual reading in reading groups,
Journal of Reading Behavior, 16, 85-85.
Allington, R.L. (2001). What really matters for struggling readers: Designing researchbased programs. New York: Longman.
Beck, I. L. & MacKeown, M. G. (1991). Social studies texts are hard to understand:
Mediating some of the difficulties. Language Arts, 68, 482-490.
Betts, E. A. (1946). Foundations of reading instruction with emphasis on differentiated
guidance. New York: American Book Company.
Cunninghan, P. M. & Cunninghan, J. W. (2002). What we know about how to teach
phonics. In Farstrup, A. E. & Samuels, S. J. (Eds), What research has to say
about reading instruction. Newark, DE: International Reading Association.
Cunningham, A. E. & Stanovich K. E. (1998). What reading does for the mind. American
Educator, 22(1&2), 8-15.
Din, F.S. (1998, March). Use Direct Instruction to quickly improve reading skills. Paper
presented at the Annual National Conference on Creating the Quality School.
Arlington, VA.
Gickling, E. E. & Armstron, D. L. (1978). Levels of instructional difficulty as related ontask behavior, task completion, and comprehension. Journal of Leaning
Disabilities, 1, 559-566.
Graves, M.J., Juel, C., & Graves, B. (1998). Teaching reading in the 21st century.
Boston: Allyn and Bacon.
Reading Renaissance
30
Ivey G. & Broaddus, K. (2001). Just plain reading: A survey of what students want to
read in middle school classrooms. Reading Research Quarterly, 36, 350-377.
Lewis, M. & Samuels, S. J. (2003). Read more--read better? A meta-analysis of the
literature on the relationship between exposure to reading and reading
achievement. Minneapolis, MN: University of Minnesota. Available online at
http://www.tc.umn.edu/~samue001/publications.htm.
Marliave, R. S. & Filby, N. (1985). Success rate: A measure of task appropriateness. In
Fisher, C. W. & Berliner, D. C. (Eds.), Perspectives on instructional time. White
Plains, NY: Longman.
National Institute of Child Health and Human Development. (2000). Report of the
National Reading Panel. Teaching children to read: An evidence-based
assessment of the scientific research literature on reading and its implications for
reading instruction (NIH Publication No. 00-4769). Washington, DC: U.S.
Government Printing Office.
Paul, T. (2003). Guided independent reading: An examination of the Reading Practice
Database and the scientific research supporting guided independent reading as
implemented in Reading Renaissance. Wisconsin Rapids, WI: Renaissance
Learning, Inc.
Rosenshine, B. & Stevens, R. (1984). Classroom instruction in reading. In Pearson, P. D.
(Ed.), Handbook of reading research (pp. 745-798). New York: Longman.
Slavin, R.E., , Karweit, N. L. & Madden, N. A. (1989). Effective programs for students at
risk. Boston: Allyn & Bacon.
Vygotsky, L.S. (1978). Mind in Society. Cambridge, MA: Harvard University Press.
Reading Renaissance
Yu, V. (1993). Extensive reading programs: How can the best benefit the teaching and
learning of English? TESL Reporter, 26, 1-9.
31
Reading Renaissance
32
Table 1
Student Race/Ethnicity and Gender for the Analytic Sample by Grade Level.
Grade
Race
Gender
Asian
Male
Female
Total
African American
Male
Female
Unknown
Total
Hispanic
2
3
4
5
6
7
8
9
10
11
Total
12
6
6
12
13
8
21
8
6
14
8
9
17
5
6
11
4
3
7
6
9
15
12
13
25
9
6
15
5
4
9
0
5
5
76
75
151
231
202
0
433
283
265
4
552
191
208
0
399
153
158
1
312
102
83
5
190
80
81
1
162
54
73
0
127
67
55
1
123
41
35
0
76
26
27
0
53
15
23
0
38
1,243
1,210
12
2,465
Male
87
84
73
82
154
65
106
186
46
32
30
945
Female
85
69
58
66
146
65
98
176
42
48
21
874
Unknown
Total
0
2
0
1
0
0
0
0
0
0
0
3
172
155
131
149
300
130
204
362
88
80
51
1,822
2
5
7
2
1
3
3
9
12
1
4
5
3
1
4
38
40
78
4
3
7
4
7
11
4
3
7
2
1
3
2
1
3
65
75
140
Native American
Male
Female
Total
White
Male
Female
Unknown
Total
676
626
2
1,304
732
628
4
1,364
576
545
1
1,122
441
379
0
820
225
266
0
491
194
175
1
370
297
301
0
598
330
295
0
625
314
296
0
610
237
228
1
466
163
159
0
322
4,185
3,898
9
8,092
Unknown
Male
2,108
2,023
1,999
1,784
1,488
990
711
518
366
291
279
12,557
Female
1,850
1,716
1,716
1,558
1,362
848
694
520
339
300
292
11,195
Unknown
1,746
1,475
1,408
1,140
980
968
262
256
166
142
143
8,686
Total
5,704
5,214
5,123
4,482
3,830
2,806
1,667
1,294
871
733
714
32,438
7,632
7,309
6,801
5,785
4,826
3,553
2,618
2,440
1,667
1,344
1,133
45,108
Grand Total
Reading Renaissance
33
Table 2
Descriptive Statistics for Student- and Classroom-Level Variables for Grades 2 to 5
(Analytical Sample)
Student-Level Descriptive Statistics
Variable Name
n
M
SD
Min
Max
NCE Pretest
24,925
48.98
21.46
1
99
NCE Posttest
24,925
52.27
20.78
1
99
Average Percent Correct Above 85
24,925
0.69
0.46
0
1
Actual Number of Words Read
24,925 358,038.75 409,091.90
0 5,750,622.00
Log of Actual Number of Words
24,925
12.16
1.30
0
15.56
Read
Low Challenge
24,925
0.04
0.19
0
1
High Challenge
24,925
0.26
0.44
0
1
Grade 3
24,925
0.27
0.44
0
1
Grade 4
24,925
0.25
0.43
0
1
Grade 5
24,925
0.21
0.41
0
1
Classroom-Level Descriptive Statistics
Classroom-Mean Number of Words
1,338 335,044.66 266,566.89 1,019.31 1,648,440.63
Read
Classroom-Mean of Log of Number
1,338
12.07
1.07
5.48
14.27
of Words Read
Percent Optimum Reading
1,338
0.70
0.19
0
1
Challenge
Classroom-Mean Pretest Score
1,338
48.16
11.83
3.42
79.88
Classroom-Mean Percent Correct
Above 85
1,338
0.69
0.22
0
1
Master Certification
1,338
0.23
0.42
0
1
Number of Students in Classroom
1,338
18.71
4.58
5
44
Reading Renaissance
Table 3
Descriptive Statistics for Student- and Classroom-Level Variables for Grades 6 to 8 Analytical
Sample
Student-Level Descriptive Statistics
n
M
SD
Variable Name
NCE Pretest
8,402
46.88
21.63
NCE Posttest
8,402
46.24
21.26
Average Percent Correct
Above 85
8,402
0.57
0.50
Actual Number of Words Read
8,402 614,000.13 542,076.04
Log of Actual Number of
Words Read
8,402
12.78
1.57
Low Challenge
8,402
0.04
0.21
High Challenge
8,402
0.14
0.34
Grade 7
8,402
0.32
0.47
Grade 8
8,402
0.22
0.41
Classroom-Level Descriptive Statistics
Classroom-Mean Number of
416
Words Read
573,595 340,260.89
Classroom-Mean of Log of
Number of Words Read
416
12.59
1.39
Percent Optimum Reading
Challenge
416
0.79
0.20
Classroom-Mean Pretest Score
416
44.80
15.22
Classroom-Mean Percent
Correct Above 85
416
0.55
0.26
Master Certification
416
0.09
0.29
Number of Students in
Classroom
416
20.38
11.23
Min
Max
1
1
99
99
0
0
1
5,641,805
0
0
0
0
0
15.55
1
1
1
1
1,096.50
2,092,026.75
1.69
14.45
0
1
1
84.17
0
0
1
1
5
135
34
Reading Renaissance
Table 4
Descriptive Statistics for Student- and Classroom-Level Variables for Grades 9 to 12 Analytical
Sample
Student-Level Descriptive Statistics
n
M
SD
Variable Name
Min
Max
NCE Pretest
5,817
48.38
21.82
1
99
NCE Posttest
5,817
48.18
21.99
1
99
Average Percent Correct Above
85
5,817
0.28
0.45
0
1
Actual Number of Words Read
5,817 312,696.97 337,908.23
0 4,022,247
Log of Actual Number of Words
Read
5,817
11.49
2.96
0
15.21
Low Challenge
5,817
0.06
0.24
0
1
High Challenge
5,817
0.11
0.31
0
1
Grade 3
5,817
0.24
0.43
0
1
Grade 4
5,817
0.20
0.4
0
1
Grade 5
5,817
0.18
0.39
0
1
Classroom-Level Descriptive Statistics
Classroom-Mean Number of
Words Read
350 304,267.78 183,183.90 8,832.14 925,142.69
Classroom-Mean of Log of
Number of Words Read
350
11.41
1.44
5.71
13.66
Percent Optimum Reading
Challenge
350
0.82
0.17
0
1
Classroom-Mean Pretest Score
350
47.47
13.44
1
78.23
Classroom-Mean Percent
Correct Above 85
350
0.28
0.17
0
1
Master Certification
350
0.01
0.08
0
1
Number of Students in
Classroom
350
16.7
5.64
5
34
35
Reading Renaissance
36
Table 5
HLM Models for Predicting Posttest NCE Reading Achievement: Grades 2 to 5 Sample
Effect
Effects of Within Classroom
Variables
NCE Posttest
NCE Pretest
Grade 3
Grade 4
Grade 5
Average Percent Correct Above 85
Actual Number of Words Read
Low Challenge
High Challenge
Effects of Between Classroom
Variables
Classroom-Mean Pretest Score
Number of Students in Classroom
Classroom Average Daily Engaged
Reading Time
Classroom Mean Percent Correct
Above 85
Average Optimum Reading
Challenge
Master Certification
NCE Pretest Slope
Model 1
se
***51.78
0.32
t
164.13
Effect
***54.83
***0.74
***-3.36
***-4.04
***-6.57
Model 2
se
0.56
0.005
0.67
0.81
0.85
t
Effect
97.24
152.39
-5.01
-5.02
-7.70
***55.22
***0.65
***-3.90
***-4.32
***-7.51
***3.95
***4.07
***-3.19
***3.65
df
Estimate
Model 3
se
0.54
0.01
0.64
0.79
0.81
0.17
0.14
0.47
0.21
t
101.98
93.94
-6.09
-5.47
-9.23
22.60
28.22
-6.86
17.36
Classroom Average Daily
Engaged Reading Time
Classroom Average Percent
Correct
Average Optimum Reading
Challenge
Professional Development
Sessions
Master Certification
Variation Between Classrooms
Intercept
NCE Pretest Slope
Variation Within Classrooms
Model Statistics
Percentage of Parameter Variance
Explained
Intra-class Correlation
2
Within-Classrooms: R
Between-Classrooms: R2
Note: * p < .05; ** p < .01; *** p < .001
Estimate
χ2
114.45 ***9269.85
328.23
df
1337
Estimate
χ2
123.64 ***22960.07
0.01 ***2042.23
125.71
1337
1337
124.69 ***26007.69
0.01 ***2044.41
110.94
25.85%
61.70%
-
χ2
66.20%
-
df
1337
1337
Reading Renaissance
37
Table 6
HLM Models for Predicting Posttest NCE Reading Achievement: Grades 2 to 5 Sample (Cont.)
Model 4
Effect
Effects of Within Classroom Variables
NCE Posttest
NCE Pretest
Grade 3
Grade 4
Grade 5
Average Percent Correct Above 85
Actual Number of Words Read
Low Challenge
High Challenge
Effects of Between Classroom Variables
Classroom-Mean Pretest Score
Number of Students in Classroom
Mean Actual Number of Words Read
Average Optimum Reading Challenge
Classroom Mean Percent Correct
Above 85
Master Certification
NCE Pretest Slope
Mean Actual Number of Words
Read
Average Optimum Reading
Challenge
Classroom Average Percent
Correct Above 85
Master Certification
Variation Between Classrooms
Intercept
NCE Pretest Slope
Variation Within Classrooms
Model Statistics
Percentage of Parameter Variance
Explained
Within-Classrooms: R2
2
Between-Classrooms: R
Note: * p < .05; ** p < .01; *** p < .001
se
Model 5
t
Effect
se
Model 6
t
Effect
se
t
***54.56
***0.65
***-3.06
***-3.84
***-6.28
***3.94
***4.06
***-3.23
***3.65
0.31
0.01
0.38
0.38
0.39
0.17
0.14
0.47
0.21
176.19
93.87
-8.03
-10.22
-16.28
22.54
28.08
-6.94
17.33
***55.80
***0.65
***-4.67
***-5.51
***-8.44
***3.93
***2.09
***-3.06
***3.69
0.33
0.01
0.39
0.45
0.50
0.18
0.20
0.47
0.21
170.00
92.17
-11.88
-12.37
-16.81
22.34
10.58
-6.49
17.29
***55.74
***0.65
***-4.61
***-5.40
***-8.29
***3.93
***4.10
***-3.06
***3.69
0.33
0.01
0.39
0.45
0.51
0.18
0.14
0.47
0.21
169.32
92.19
-11.69
-12.09
-16.41
22.34
24.47
-6.49
17.29
***0.88
**0.08
0.01
0.03
67.85
2.87
***0.79
0.03
***2.09
-1.40
0.02
0.03
0.20
0.82
46.08
1.18
10.58
-1.71
***0.79
0.03
***2.03
-1.41
0.02
0.03
0.20
0.82
46.26
1.24
10.14
-1.73
**1.83
0.69
2.64
*1.39
**0.87
0.68
0.31
2.04
2.81
**0.02
0.01
3.01
**0.02
0.01
3.04
0.002
0.03
0.07
0.002
0.03
0.06
-0.003
0.02
-0.13
0.001
-0.01
0.02
0.01
0.03
-0.50
Estimate
12.14
0.01
110.92
χ2
***3980.48
***2028.67
df
1332
1334
Estimate
12.04
0.01
110.92
χ2
***3956.66
***2028.11
df
1331
1332
Estimate
15.77
0.01
110.91
χ2
***4760.50
***2044.84
df
1335
1337
66.21%
66.21%
66.21%
86.22%
89.39%
89.48%
Reading Renaissance
38
Table 7
HLM Models for Predicting Posttest NCE Reading Achievement: Grades 6 to 8 Sample
Effect
Effects of Within Classroom
Variables
NCE Posttest
NCE Pretest
Grade 7
Grade 8
Average Percent Correct Above 85
Actual Number of Words Read
Low Challenge
High Challenge
Effects of Between Classroom
Variables
Classroom-Mean Pretest Score
Number of Students in Classroom
Classroom Average Daily Engaged
Reading Time
Classroom Mean Percent Correct
Above 85
Average Optimum Reading
Challenge
Master Certification
NCE Pretest Slope
Model 1
se
***44.58
0.70
t
Effect
63.62
***44.71
***0.77
-0.71
-0.39
df
Estimate
Model 2
se
0.88
0.01
1.17
1.49
t
Effect
50.69
93.01
-0.61
-0.26
***44.68
***0.73
-0.71
-0.28
***3.34
***1.38
***-2.59
***1.74
df
Estimate
Model 3
se
0.88
0.01
1.17
1.46
0.30
0.17
0.69
0.40
t
50.83
75.03
-0.61
-0.19
11.07
8.31
-3.78
4.39
Classroom Average Daily
Engaged Reading Time
Classroom Average Percent
Correct
Average Optimum Reading
Challenge
Professional Development
Sessions
Master Certification
Variation Between Classrooms
Intercept
NCE Pretest Slope
Variation Within Classrooms
Model Statistics
Percentage of Parameter Variance
Explained
Intra-class Correlation
2
Within-Classrooms: R
Between-Classrooms: R2
Note: * p < .05; ** p < .01; *** p < .001
Estimate
χ2
185.86 ***4490.77
306.46
415
χ2
205.14 ***11004.66
0.01 ***496.79
123.44
413
413
205.59 ***11495.19
0.004
**483.38
118.13
37.75%
59.72%
-
χ2
61.45%
-
df
413
413
Reading Renaissance
39
Table 8
HLM Models for Predicting Posttest NCE Reading Achievement: Grades 6 to 8 Sample (Cont.)
Model 4
Effect
Effects of Within Classroom Variables
NCE Posttest
NCE Pretest
Grade 7
Grade 8
Average Percent Correct Above 85
Actual Number of Words Read
Low Challenge
High Challenge
Effects of Between Classroom Variables
Classroom-Mean Pretest Score
Number of Students in Classroom
Mean Actual Number of Words Read
Average Optimum Reading Challenge
Classroom Mean Percent Correct
Above 85
Master Certification
NCE Pretest Slope
Mean Actual Number of Words
Read
Average Optimum Reading
Challenge
Classroom Average Percent
Correct Above 85
Master Certification
Variation Between Classrooms
Intercept
NCE Pretest Slope
Variation Within Classrooms
Model Statistics
Percentage of Parameter Variance
Explained
Within-Classrooms: R2
2
Between-Classrooms: R
Note: * p < .05; ** p < .01; *** p < .001
se
Model 5
t
Effect
se
Model 6
t
Effect
se
t
***43.70
***0.72
**1.33
*0.89
***3.36
***1.39
***-2.57
***1.68
0.24
0.01
0.41
0.42
0.30
0.17
0.68
0.40
184.31
74.75
3.21
2.11
11.08
8.31
-3.79
4.24
***43.66
***0.73
**1.42
*0.91
***3.35
***1.38
***-2.60
***1.72
0.23
0.01
0.42
0.40
0.30
0.17
0.69
0.41
190.01
68.56
3.45
2.25
11.09
8.22
-3.76
4.16
***43.64
***0.73
***1.51
*0.89
***3.36
***1.38
***-2.60
***1.73
0.23
0.01
0.41
0.40
0.30
0.17
0.69
0.41
191.77
68.08
3.67
2.22
11.11
8.22
-3.79
4.18
***0.93
-0.01
0.01
0.01
70.55
-0.75
***0.90
-0.01
0.03
0.70
0.02
0.01
0.16
1.44
45.30
-1.01
0.20
0.49
***0.90
-0.01
-0.01
0.73
0.02
0.01
0.16
1.42
43.13
-0.75
-0.06
0.52
***3.08
0.74
4.17
**2.36
***2.11
0.75
0.58
3.13
3.67
-0.004
0.01
-0.33
-0.005
0.01
-0.45
0.005
0.06
0.08
0.01
0.06
0.18
-0.01
0.04
-0.20
-0.02
0.05
0.04
0.03
-0.59
1.91
Estimate
5.53
0.004
118.00
χ2
***777.39
**483.69
df
408
410
Estimate
5.22
0.004
118.10
χ2
***757.80
***480.87
df
407
409
Estimate
6.19
0.004
117.97
χ2
***827.81
**484.62
df
411
413
61.51%
61.50%
61.46%
96.67%
97.02%
97.19%
Reading Renaissance
40
Table 9
HLM Models for Predicting Posttest NCE Reading Achievement: Grades 9 to 12 Sample
Effect
Effects of Within Classroom
Variables
NCE Posttest
NCE Pretest
Grade 10
Grade 11
Grade 12
Average Percent Correct Above 85
Actual Number of Words Read
Low Challenge
High Challenge
Effects of Between Classroom
Variables
Classroom-Mean Pretest Score
Number of Students in Classroom
Classroom Average Daily Engaged
Reading Time
Classroom Mean Percent Correct
Above 85
Average Optimum Reading
Challenge
Master Certification
NCE Pretest Slope
***47.50
Model 1
se
0.68
t
Effect
69.76
***46.00
***0.75
***3.29
**2.96
-0.39
df
Estimate
Model 2
se
0.89
0.01
0.86
0.99
1.15
t
Effect
51.67
66.48
3.82
3.01
-0.34
***45.92
***0.73
***3.22
**3.09
0.01
***3.17
***0.45
***-3.08
**1.83
df
Estimate
Model 3
se
0.88
0.01
0.82
0.97
1.14
0.39
0.07
0.77
0.67
t
52.23
59.52
3.90
3.20
0.01
8.11
6.53
-3.99
2.74
Classroom Average Daily
Engaged Reading Time
Classroom Average Percent
Correct
Average Optimum Reading
Challenge
Professional Development
Sessions
Master Certification
Variation Between Classrooms
Intercept
NCE Pretest Slope
Variation Within Classrooms
Model Statistics
Percentage of Parameter Variance
Explained
Intra-class Correlation
2
Within-Classrooms: R
Between-Classrooms: R2
Note: * p < .05; ** p < .01; *** p < .001
Estimate
χ2
138.05 ***2302.22
362.92
349
159.38
0.01
160.16
χ2
***4895.01
***464.68
347
347
159.11 ***4991.06
0.01
**453.16
156.46
27.65%
55.87%
-
χ2
56.89%
-
df
347
347
Reading Renaissance
41
Table 10
HLM Models for Predicting Posttest NCE Reading Achievement: Grades 9 to 12 Sample (Cont.)
Model 4
Effect
Effects of Within Classroom Variables
NCE Posttest
NCE Pretest
Grade 10
Grade 11
Grade 12
Average Percent Correct Above 85
Actual Number of Words Read
Low Challenge
High Challenge
Effects of Between Classroom Variables
Classroom-Mean Pretest Score
Number of Students in Classroom
Mean Actual Number of Words Read
Average Optimum Reading Challenge
Classroom Mean Percent Correct
Above 85
Master Certification
NCE Pretest Slope
Mean Actual Number of Words
Read
Average Optimum Reading
Challenge
Classroom Average Percent
Correct Above 85
Master Certification
Variation Between Classrooms
Intercept
NCE Pretest Slope
Variation Within Classrooms
Model Statistics
Percentage of Parameter Variance
Explained
Within-Classrooms: R2
2
Between-Classrooms: R
Note: * p < .05; ** p < .01; *** p < .001
se
Model 5
t
Effect
se
Model 6
t
Effect
se
t
***47.11
***0.73
*1.19
0.88
*-1.86
***3.06
***0.46
***-2.85
1.31
0.35
0.01
0.52
0.54
0.77
0.39
0.07
0.78
0.67
132.83
59.70
2.30
1.62
-2.41
7.79
6.59
-3.67
1.94
***46.87
***0.73
**1.34
**1.41
-1.31
***3.08
***0.46
***-3.05
**1.69
0.34
0.01
0.51
0.54
0.74
0.39
0.07
0.77
0.69
138.54
61.96
2.65
2.61
-1.78
7.82
6.56
-3.99
2.42
***46.88
***0.73
*1.34
*1.40
*-1.32
***3.08
***0.46
***-3.05
**1.69
0.36
0.01
0.53
0.59
0.63
0.41
0.07
0.79
0.71
129.23
59.27
2.54
2.38
-2.10
7.49
6.79
-3.88
2.40
***0.93
0.05
0.02
0.05
49.31
0.98
***0.84
0.02
0.37
***6.69
0.03
0.05
0.20
1.66
31.63
0.40
1.87
4.02
***0.84
0.02
0.37
**6.74
0.03
0.04
0.20
2.00
32.13
0.44
1.84
3.39
**4.21
1.41
2.97
*4.47
-0.83
1.51
3.02
2.83
-0.27
-0.004
0.01
-0.49
-0.004
0.01
-0.42
***-0.32
0.07
-4.66
***-0.32
0.08
-3.90
*0.14
0.07
1.97
0.14
0.05
0.08
0.15
1.88
0.32
Estimate
7.45
0.01
156.16
χ2
***606.24
**428.72
df
342
344
Estimate
7.49
0.01
156.16
χ2
***605.94
***428.54
df
341
343
Estimate
8.63
0.01
156.29
χ2
***661.70
**456.83
df
345
347
56.94%
56.97%
56.97%
93.75%
94.60%
94.57%
Reading Renaissance
Appendix
42
Reading Renaissance
ZPD Ranges by Grade Equivalent Scores
Pretest Grade Equivalent Scores
.00
.10
.20
.30
.40
.50
.60
.70
.80
.90
1.00
1.10
1.20
1.30
1.40
1.50
1.60
1.70
1.80
1.90
2.00
2.10
2.20
2.30
2.40
2.50
2.60
2.70
2.80
2.90
3.00
3.10
3.20
3.30
3.40
3.50
3.60
3.70
3.80
3.90
Suggested ZPD
1.0 to 2.0
1.5 to 2.5
2.0 to 3.0
2.3 to 3.3
2.6 to 3.6
2.8 to 4.0
43
Reading Renaissance
4.00
4.10
4.20
4.30
4.40
4.50
4.60
4.70
4.80
4.90
5.00
5.10
5.20
5.30
5.40
5.50
5.60
5.70
5.80
5.90
6.00
6.10
6.20
6.30
6.40
6.50
6.60
6.70
6.80
6.90
7.00
7.10
7.20
7.30
7.40
7.50
7.60
7.70
7.80
7.90
3.0 to 4.5
3.2 to 5.0
3.4 to 5.4
3.7 to 5.7
4.0 to 6.1
4.2 to 6.5
4.3 to 7.0
4.4 to 7.5
44
Reading Renaissance
8.00
8.10
8.20
8.30
8.40
8.50
8.60
8.70
8.80
8.90
9.00
9.10
9.20
9.30
9.40
9.50
9.60
9.70
9.80
9.90
10.00
10.10
10.20
10.30
10.40
10.50
10.60
10.70
10.80
10.90
11.00
11.10
11.20
11.30
11.40
11.50
11.60
11.70
11.80
11.90
4.5 to 8.0
4.6 to 9.0
4.7 to 10.0
4.8 to 11.0
45
Reading Renaissance
12.00
12.10
12.20
12.30
12.40
12.50
12.60
12.70
12.80
12.90
13.00
4.9 to 12.0
46