Technical Appendix Navigating the Middle Grades: Evidence from New York City Michael J. Kieffer April 2012 Navigating the Middle Grades: A Descriptive Analysis of the Middle Grades in New York City Technical Appendix Michael J. Kieffer Teachers College, Columbia University April 2012 © 2012 Research Alliance for New York City Schools. All rights reserved. You may make copies of and distribute this work for noncommercial educational and scholarly purposes. For any other uses, including the making of derivative works, permission must be obtained from the Research Alliance for New York City Schools, unless fair use exceptions to copyright law apply. CONTENTS I. Methods ....................................................................................................... 1 Sample......................................................................................................... 1 Measures ...................................................................................................... 1 II. Data Analysis and Results .................................................................... 3 Descriptives ............................................................................................................. 3 Who’s on track to graduate and why? Predicting High School Graduation based on Grade Nine Predictors ............................................................................................................... 4 What do students’ grade four-eight achievement and attendance trajectories look like? ........... 4 Does students’ grade four-eight achievement predict who’s on track in grade nine? ...............11 Does students’ grade four-eight attendance predict who’s on track in grade nine? .................14 Do particular demographic groups of students demonstrate middle-grades trajectories that are associated with being off-track in grade nine? ................................................................19 Is middle grades performance equally predictive of later on-track status across ethnic and language groups? ....................................................................................................29 Do these patterns hold across schools? III. .......................................................................32 Exploratory Analysis of School Characteristics .........................35 I. METHODS Sample As noted in the main text, the analytic sample for the high school graduation analyses was the cohort of New York City students who were first-time ninth graders in the 2005-2006 school year. The analytic sample for the middle grades analyses included four cohorts of students who were first-time fourth graders between the 2000-2001 and 2003-2004 school years. Our data cover the former cohort’s progress through high school graduation and the latter cohort’s progress through grade nine. We identified first-time fourth graders by selected students who were in grade four in the appropriate school year for their cohort, but were not in grade four during the previous school year. We conduct the middle grades analyses primarily with the entire population of students who ever appear in these four cohorts (N = 303,845), using fullinformation maximum likelihood to account for data missing due to attrition or other causes. This sample thus included all students, including students classified as English language learners and students with disabilities. Descriptive statistics on the sample are displayed in Table 1 below. We also checked results against an analyses using the subset of students with complete data (n = 169, 953), i.e., those who do not enter or exit the district at any point between grades four and nine, who progress through each grade annually, and who have complete data on the variables of interest; results were largely similar when analyses were conducted with this subsample, so the results for the complete sample are reported here. Measures High School Graduation. Students’ on-time graduation in the fourth year after they enrolled as first-time ninth graders was drawn from the DOE’s Student Trackng System Dataset. Thus, graduation was defined as graduating within four years, so this variable equaled 0 for students who graduated later or completed a General Equivalency Diploma. Performance in Grade Nine. Measures of grade nine performance include credits earned over the course year, courses failed over the course of the year, and grade point average across the year, each drawn from the Course Detail Records file. Measures of grade nine performance also included the total attendance rate, as a percent of days enrolled, drawn from the DOE’s Student Tracking System Dataset. Measures also included a dummy variable for whether a Regents exam was attempted and one for whether a Regents test was passed in grade nine. Achievement in Grades four-eight. Achievement in the areas of mathematics and reading/English-language arts was assessed using the New York State tests, with scores drawn from the NY State ELA and Math Test Score 1 file. Because the scale for these tests changed over the years of administration and were not vertically linked to be comparable across grade levels, scores were rescaled to be within-grade zscores, based on the district means and standard deviations. This approach is not ideal for growth modeling because it subtracts out the normative trajectory for growth and assumes homogeneity of variance across time. However, given the limitations of the scaling of the test scores, it is more appropriate than using the original scaled scores. It also has benefits over using proficiency levels, in that it preserves the continuous nature of achievement, as opposed to arbitrarily dividing the distribution into discrete categories. Tests were taken annually, yielding one score per year. Attendance in Grades four-eight. Attendance was measured as a percent of the days enrolled for each semester, yielding two attendance rate values for each year. Attendance data were drawn from the DOE’s Attendance System. 2 II. DATA ANALYSES AND RESULTS Descriptives Table 1: Means and standard deviations for achievement test scores, attendance along with demographics for the analytic sample (N = 303,845) Mean Mathematics Achievement (within-grade zscores) Reading Achievement (within-grade z-scores) Attendance (Percent of Days Enrolled) Race/ethnicity Language Background Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Fall, Grade 4 Spring, Grade 4 Fall, Grade 5 Spring, Grade 5 Fall, Grade 6 Spring, Grade 6 Fall, Grade 7 Spring, Grade 7 Fall, Grade 8 Spring, Grade 8 African-American Asian Hispanic Native American White ELL in Grade 4 Language Minority 3 Standard Deviation -0.02 1.01 -0.06 1.03 -0.08 1.04 -0.10 1.06 -0.15 1.09 -0.50 1.02 -0.07 1.03 -0.09 1.04 -0.09 1.05 -0.12 1.06 94.00 7.53 92.91 8.25 93.06 11.35 92.14 9.06 92.03 12.58 91.05 10.91 91.39 12.47 89.64 12.64 90.75 12.95 87.08 13.88 Percentage of Sample 33.3% 12.3% 39.1% 0.4% 14.8% 9.4% 41.4% Who’s on track to graduate and why? Predicting High School Graduation based on Grade Nine Predictors Logistic regression was used to determine whether the grade nine measures predicted ontime high school graduation. As shown in Table 1, credits earned, courses failed, GPA, attendance rate, whether a Regents test was attempted, and whether a Regents test was passed all predicted high school graduation. As mentioned in the main text, these predictors remain strong when controlling for students’ grade-eight test scores and for fixed effects of high school. These parameter estimates were used as relative weights for each measure in order to estimate a single “on-track” indicator that summarizes these predictive relationships for each student. These scores were then used in subsequent analyses. Table 2: Results of Logistic Regression Predicting On-time High School Graduation based on Grade (G) 9 Predictors Unstandardized Estimate Wald χ2 Intercept -7.12 1605.62*** G9 Credits Earned 0.31 3893.69*** G9 Courses Failed -0.09 235.93*** G9 Grade Point Average 0.03 211.09*** G9 Annual Attendance Rate 0.03 675.88*** G9 Regents’ Test Attempted 0.33 88.77*** G9 Regents’ Test Passed 0.94 543.00*** *** p < .001 What do students’ grade four-eight achievement and attendance trajectories look like? To address the question concerning the nature of students’ growth trajectories in achievement and attendance across grades four through eight, we fitted a series of piecewise unconditional growth models using latent growth modeling in a structural equation modeling (SEM) framework (Bollen & Curran, 2006). Piecewise models allow for nonlinear trajectories in which students demonstrate different rates of growth during different specified periods. They have the advantage of directly modeling true rates of growth (i.e., growth rates that are freed of occasion-specific measurement error) for each theoretically important period for which sufficient data points are available. Figure 1 displays a path diagram for the hypothesized unconditional piecewise growth model for attendance. As shown, students’ individual growth trajectories in attendance were specified to have an initial (Fall, grade four) status and four slopes representing growth in four distinct periods: Fall, grade four to Fall, grade five; Fall, grade five to Spring, grade six; Spring, grade six to Spring, grade seven; and Spring, grade seven to Spring, grade eight. Each slope was allowed to vary across children and the slopes were allowed to covary with one another and with 4 students’ initial (Fall, grade four) status. Inspection of empirical growth plots (Singer & Willett, 2003) suggested that this piecewise model was appropriate to compare the population average growth trajectory as well as individual differences in the shape and elevation of students’ growth trajectories. Fitting of various unconditional models also indicated that this model was superior to other theoretically viable specifications. It is worth noting that the second period is longer due to the number of measurement occasions; with ten occasions, a four-slope piecewise model is only possible if one slope covers a longer period than the other three. Comparisons of alternate models indicated that this particular piecewise model, with a longer second period, fitted the data better than potential alternatives. As shown in Table 3, fitting the unconditional piecewise growth model for attendance provides insight into the average trajectory for attendance as well as individual variation around that trajectory. As shown in the second row of Table 3, students’ initial status, on average, was relatively high (approximately 94 percent of days enrolled) in the fall of grade four. As shown in the third through sixth rows of Table 2, each slope was negative, indicating declines in attendance on average, with the largest decline (a decline of 3 percent of days enrolled) occurring between spring of grade seven and spring of grade eight. The variance components displayed in the seventh through eleventh rows of Table 2 indicate that there was substantial variation in students’ initial status and each rate of growth, with the largest variance occurring again between spring of grade seven and spring of grade eight. Together, these two findings suggest that this period involves not only the largest declines in attendance for all students but also the widest variation in declines, with some students declining relative to other students to a much greater extent than in previous periods. In addition, this unconditional piecewise growth model provides insight into the relationship between early levels and later rates of growth in attendance. Correlations between students’ initial status and their rates of growth are displayed in the twelfth through fifteenth rows of the right column titled “Selected Standardized Estimates.” As shown, initial status had a moderately sized negative relationship with students’ rates of growth between fall, grade four and fall, grade five, but only trivially sized relationships with students’ rates of growth during later periods. This suggests that students’ levels of attendance prior to the middle grades provides little information for predicting the extent to which they will maintain or decline in attendance during the middle grades. Figure 2 displays a path diagram for the hypothesized unconditional piecewise growth model for mathematics achievement. As shown, students’ individual growth trajectories in mathematics achievement were specified to have an initial (grade four) status and two slopes representing growth in two distinct periods: grade four to grade six and grade six to grade eight. Each slope was allowed to vary across children and the slopes were allowed to covary with one another and with students’ initial (grade four) status. A parallel model with the same piecewise specification was fitted to reading/English-language arts achievement. Inspection of empirical growth plots suggested that this model was appropriate for both mathematics achievement and reading/English-language arts achievement. Fitting of various unconditional models also indicated that this model was superior to other theoretically viable specifications for both achievement outcomes. As shown in Table 4, fitting the unconditional piecewise growth model for mathematics and reading/English-language arts achievement provides insight into the levels and relative change in achievement for the average student in NYC schools. In interpreting these results, it is 5 important to recall that within-grade z-scores were used, in the absence of a more absolute developmental scaled score, so change represents students’ relative movements within the rank order rather than growth in a traditional sense. As shown in the second through fourth rows of Table 4, estimates of initial status were close to the city average in grade four and average change was minimal, as we would expect given that the within-grade z-score scale eliminates average growth with increasing grade level. More interestingly, the variance components displayed in the fifth through seventh rows indicate much wider variation in initial (grade four) status (.86 within-grade SD) than in either rate of growth (.03 within-grade SD per year for both Slope 1 and Slope 2 in mathematics; .02 for both Slope 1 and Slope 2 in reading/Englishlanguage arts). This suggests that there is substantial stability in the rank-order of students’ achievement levels. For instance, a student with a high rate of growth in mathematics relative to the sample (i.e., 1 SD above the mean in Slope 1) would only change in the rank-order by 0.17 within-grade SD each year; a student with an analogously high rate of growth in reading/ELA would only change by 0.14 within-grade SD. These unconditional growth models of achievement also provide insight into the extent to which early levels of achievement predict later rates of growth, as shown in the eighth through eleventh rows and the fourth and sixth columns of Table 4. For both mathematics achievement, students’ initial (grade four) status had a trivially sized relationship with students’ later rates of growth (rs between -.01 and -.11). This should not be interpreted to mean that early levels do not strongly predict later levels of achievement; in fact, the previous findings above concerning stability of the rank-order suggests that they do. Rather, they suggest that growth trajectories are largely parallel, with students who start substantially higher in grade four demonstrating growth trajectories that neither increase nor decrease substantially than those of their peers. 6 Figure 1: Path diagram for hypothesized piecewise linear growth model for attendance 7 Figure 2: Path diagram for hypothesized piecewise linear growth model for mathematics achievement 8 Table 3: Selected Results for Unconditional Piecewise Growth Model for Attendance (N = 303,845) Unstandardized Estimates Fixed Effects Variance Components Initial (Fall, Grade 4) Status 93.95*** Slope 1 (Fall, Grade 4 – Fall, Grade 5) Slope 2 (Fall, Grade 5 – Spring, Grade 6) Slope 3 (Spring, Grade 6 – Spring, Grade 7) Slope 4 (Spring, Grade 7 – Spring, Grade 8) Initial Status -1.32*** -0.67*** -1.28*** -3.01*** 49.45*** Selected Standardized Estimates Slope 1 40.44*** Slope 2 17.60*** Slope 3 45.84*** Slope 4 88.49*** Covariances Initial Status with Slope 1 -16.86*** -.38 Initial Status with Slope 2 0.36** .01 Initial Status with Slope 3 2.34*** .05 Initial Status with Slope 4 0.02 .00 Slope 1 with Slope 2 -9.20*** -.35 Slope 1 with Slope 3 -1.55*** -.04 Slope 1 with Slope 4 1.20*** .02 Slope 2 with Slope 3 1.76*** .06 Slope 2 with Slope 4 -2.12*** -.05 Slope 3 with Slope 4 -13.80*** -.22 Note: For the purposes of FIML, this model also included factors and indicates for mathematics and reading/ELA achievement and Grade 9 ontrack indicator score, which were allowed to covary with the latent growth factors for attendance. ** p < .01; *** p < .001 9 Table 4: Selected Results for Unconditional Piecewise Growth Models for Mathematics and Reading Achievement (N = 303,845) Fixed Effects Mathematics Achievement Unstandardized Selected Estimates Standardized Estimates -0.02*** Reading/ELA Achievement Unstandardized Selected Estimates Standardized Estimates -0.05*** Initial (Grade 4) Status Slope 1 (Grade 4 – -0.03*** -0.02*** Grade 6) Slope 2 (Grade 6 – -0.03*** -0.01*** Grade 8) Variance Initial Status 0.86*** 0.81*** Components Slope 1 0.03*** 0.02*** Slope 2 0.03*** 0.02*** Covariances Initial Status with -0.02*** -0.11 -0.01*** -.06 Slope 1 Initial Status with -0.01*** -0.07 -0.001 -.01 Slope 2 Slope 1 with Slope 2 0.003*** 0.12 -0.01*** -.29 Note: For the purposes of FIML, this model also included factors and indicates for attendance and Grade 9 ontrack indicator score, which were allowed to covary with the latent growth factors for mathematics and reading/ELA achievement. *** p < .001 10 Does students’ grade four-eight achievement predict who’s on track in grade nine? Two SEM models were fitted to investigate whether variation in students’ levels and rates of relative change in achievement predict their grade nine indicator score. As shown in the path diagram in Figure 3, the grade nine ontrack indicator for students’ probability of on-time high school graduation was regressed on the growth terms for the piecewise growth model for mathematics achievement. A parallel model was fitted for reading/English-langauge arts achievement predicting the grade nine ontrack indicator. Table 5 displays the selected results of fitting this SEM model for mathematics achievement. As shown, initial status in mathematics has a strong relationship with the grade nine ontrack indicator. The two slopes in mathematics achievement also had moderate to large relationships with the grade nine ontrack indicator, with the stronger relationship demonstrated by the later growth term, representing growth between grade six and grade eight. Together, these findings suggest that initial (grade four) status in achievement provides substantial information for later probability of high school graduation, but also that the extent to which students change during the middle grades also provides valuable information. In particular, growth during the middle grades is substantially more predictive of later probability of high school graduation than growth during the upper-elementary grades. Table 6 displays the selected results of fitting a second, analogous SEM model for reading/English-language arts achievement. As shown, initial status in reading/ELA has a strong relationship with the grade nine on-track indicator. The two slopes in reading/ELA achievement also had moderate relationships with the grade nine on-track indicator that were approximately the same as each other. As with mathematics achievement, these findings suggest that grade four levels of reading/ELA achievement provide substantial information to predict probability of later high school graduation, but also that changes during the middle grades provide valuable information. However, unlike mathematics achievement, changes during the middle grades were similarly predictive of later graduation as change during the upper-elementary grades. 11 Figure 4: Path diagram for hypothesized structural equation model in which latent growth in mathematics achievement predicts Grade 9 indicator for the probability of ontime high school graduation 12 Table 5: Selected Results from Structural Equation Model with Initial Status and Rates of Growth in Mathematics Achievement Predicting Grade (G) 9 Ontrack Indicator Score (N = 303,845) Paths Unstandardized Estimates 1.63*** Standardized Estimates Math Initial (Grade 4) Status G9 Ontrack .51 Indicator Math Slope 1 (Grade 4 –Grade 6) G9 Ontrack 3.70*** .21 Indicator Math Slope 2 (Grade 6 –Grade 8) G9 Ontrack 6.62*** .39 Indicator Note: Model also included variances and intercepts for mathematics achievement initial status, slope 1, and slope 2 as well as residual variances for Grade 9 ontrack indicator score. For FIML purposes, model also included latent growth factors for reading/ELA and attendance which were allowed to covary with mathematics growth terms and Grade 9 ontrack indicator. *** p < .001 Table 6: Selected Results from Structural Equation Model with Initial Status and Rates of Growth in Reading/English-Language Arts (ELA) Achievement Predicting Grade (G) 9 Ontrack Indicator Score (N = 303,845) Paths Unstandardized Estimates 1.51*** Standardized Estimates .47 Reading/ELA Initial (Grade 4) Status G9 Ontrack Indicator Reading/ELA Slope 1 (Grade 4 –Grade 6) G9 Ontrack 5.79*** .28 Indicator Reading/ELA Slope 2 (Grade 6 –Grade 8) G9 Ontrack 6.79*** .33 Indicator Note: Model also included variances and intercepts for reading/ELA achievement initial status, slope 1, and slope 2 as well as residual variances for Grade 9 ontrack indicator score. For FIML purposes, model also included latent growth factors for mathematics and attendance which were allowed to covary with reading/ELA growth terms and Grade 9 ontrack indicator. *** p < .001 13 Does students’ grade four-eight attendance predict who’s on track in grade nine? A SEM model analogous to that fitted for the question above was fitted to investigate the extent to which levels and rates of growth in attendance predict students’ grade nine on-track probability of later high school graduation. As shown in Figure 4, grade nine on-track indicator score was regression on the intercept and four slope terms for the attendance growth model. Table 6 displays selected results for fitting this SEM model. As shown, students’ initial status in attendance had a strong relationship with their grade nine on-track indicator score and each of the four slope terms also had a moderate relationship with the grade nine on-track indicator. As with achievement, this finding indicates that students’ level of attendance in grade four provides information about whether they will be on track in grade nine for ultimately graduating from high school, but also that students’ growth or declines in attendance during the upper-elementary and middle grades provide additional information about whether they will be on track in grade nine. The magnitudes of the relationships between rates of growth and grade nine on-track indicator are largely similar across the different periods studied. To investigate the relative contributions of attendance and achievement during the middle grades to grade nine on-track indicator score, an additional SEM model that included regression paths between grade nine on-track indicator and the growth parameters for both attendance and achievement was fitted. This hypothesized model is displayed in Figure 5. Due to the high covariances among growth parameters for mathematics achievement and reading/ELA achievement, these were not modeled separately. Instead, a simple composite for achievement for each time point was estimated by averaging the z-scores for mathematics achievement and reading/ELA achievement; these then served as the indicators for a piecewise latent growth model for achievement as shown in Figure 5. Table 8 presents the results from fitting this model. As shown in the rightmost column, the standardized regression paths indicated that effects of both attendance and achievement initial levels and rates of growth remained robust when accounting for both simultaneously. These estimates are somewhat smaller than those presented in the attendance-only model in Table 7 and the achievement-only models in Tables 5 and 6, but remain non-trivial in magnitude. Moreover, the finding that growth in attendance and achievement during the middle grades adds information beyond that provide by students’ initial status in these predictors continues to hold when both predictors are accounted for simultaneously. 14 Figure 4: Path diagram for hypothesized structural equation model for latent growth in attendance predicting Grade (G) 9 ontrack indicator for probability of high school graduation. 15 Table 7: Selected Results from Structural Equation Model with Initial Status and Rates of Growth in Attendance Predicting Grade (G) 9 Ontrack Indicator Score (N = 303,845) Paths Unstandardized Estimates 0.195*** Standardized Estimates .47 Attendance Initial (Fall, G4) Status G9 Ontrack Indicator Attendance Slope 1 (Fall, G4 –Fall, G5) G9 0.175*** .38 Ontrack Indicator Attendance Slope 2 (Fall, G5 –Spring, G6) G9 0.241*** .34 Ontrack Indicator Attendance Slope 3 (Spring, G6-Spring, G7) 0.137*** .31 G9 Ontrack Indicator Attendance Slope 4 (Spring, G7-Spring, G8) 0.085*** .27 G9 Ontrack Indicator Note: Model also included variances and intercepts for reading/ELA achievement initial status, slope 1, and slope 2 as well as residual variances for Grade 9 ontrack indicator score. For FIML purposes, model also included latent growth factors for mathematics and attendance which were allowed to covary with reading/ELA growth terms and Grade 9 ontrack indicator. *** p < .001 16 Figure 5: Path diagram for hypothesized structural equation model in which latent growth in achievement (simple composite of mathematics and reading/ELA achievement) and attendance predicts Grade 9 on-track indicator Note: Model also included measurement models for attendance as shown in Figure 1 and for achievement analogous to the model shown in Figure 2. Att = Attendance; Ach = Achievement Composite 17 Table 8: Selected Results from Structural Equation Model with Initial Status and Rates of Growth in Both Attendance and Achievement Predicting Grade (G) 9 Ontrack Indicator Score (N = 303,805) Paths Unstandardized Standardized Estimates Estimates Attendance Initial (Fall, G4) Status G9 0.13*** 0.30 Ontrack Indicator Attendance Slope 1 (Fall, G4 –Fall, G5) G9 0.12*** 0.26 Ontrack Indicator Attendance Slope 2 (Fall, G5 –Spring, G6) G9 0.16*** 0.22 Ontrack Indicator Attendance Slope 3 (Spring, G6-Spring, G7) 0.09*** 0.21 G9 Ontrack Indicator Attendance Slope 4 (Spring, G7-Spring, G8) 0.07*** 0.21 G9 Ontrack Indicator Achievement Composite Initial (G4) Status G9 1.23*** 0.37 Ontrack Indicator Achievement Composite Slope 1 (G4-G6) G9 3.42*** 0.17 Ontrack Indicator Achievement Composite Slope 2 (G6-G8) G9 4.51*** 0.22 Ontrack Indicator Note: As shown in Figure 5, mnodel also included variances and intercepts for and covariances among all latent growth terms as well as a residual variance for Grade 9 ontrack indicator score. *** p < .001 18 Do particular demographic groups of students demonstrate middle-grades trajectories that are associated with being off-track in grade nine? Given the relationships between middle-grades trajectories (level and growth) in attendance and achievement with the later grade nine on-track indicator, we next investigated whether particular demographic characteristics including race/ethnicity and language background predict students’ middle-grades trajectories. Table 9 presents descriptive statistics on achievement and attendance by race/ethnicity group while Table 10 presents descriptive statistics on achievement and attendance by language backgrounds. Specifically, we fitted a series of SEM models in which demographic characteristics predicted initial status and slopes for attendance and achievement. Figure 5 presents the SEM model for ethnicity (represented as a series of dummy variables with White specified as the reference category) predicting initial status and piecewise rates of growth in attendance, while Figure 6 presents the analogous SEM model for each achievement outcome. Analogous models were fitted for language background (represented as a series of two dummy variables for English language learner designated in grade four and Language Minority, non-ELL, with native English speakers specified as the reference category). 19 Table 9: Means and standard deviations for achievement and attendance by race/ethnicity AfricanAsian (n Latino (n Native White American = 37,329) = 118,911) American (n (n = (n = = 1155) 44,871) 101,237) Mathematics Grade 4 -0.28 0.62 -0.21 -0.32 (0.98) 0.53 Achievement (0.90) (1.08) (0.91) (1.01) (within-grade zGrade 5 -0.34 0.66 -0.24 -0.40 (1.08) 0.50 scores) (0.94) (1.04) (0.93) (0.96) Grade 6 -0.37 0.70 -0.27 -0.40 (1.05) 0.42 (0.94) (1.06) (0.94) (0.97) Grade 7 -0.41 0.73 -0.29 -0.47 (1.15) 0.44 (0.96) (1.04) (0.94) (0.98) Grade 8 -0.46 0.88 -0.33 -0.49 (1.08) 0.34 (0.96) (1.10) (0.94) (1.01) Reading Grade 4 -0.20 0.41 -0.30 -0.30 (0.96) 0.53 Achievement (0.92) (1.08) (0.92) (1.08) (within-grade zGrade 5 -0.27 0.42 -0.29 -0.36 (0.99) 0.60 scores) (0.94) (1.01) (0.95) (1.06) Grade 6 -0.28 0.47 -0.30 -0.36 (1.02) 0.48 (0.94) (1.05) (0.96) (1.05) Grade 7 -0.29 0.48 -0.20 -0.38 (1.02) 0.49 (0.95) (1.03) (0.98) (1.05) Grade 8 -0.33 0.51 -0.32 -0.41 (0.98) 0.44 (0.93) (1.14) (0.93) (1.13) Attendance (Percent Fall, 93.29 96.76 93.51 92.75 (8.95) 94.62 of Days Enrolled) Grade 4 (8.55) (5.21) (7.49) (6.01) Spring, 91.90 96.40 92.43 91.61 (9.30) 93.63 Grade 4 (9.37) (5.57) (8.18) (6.55) Fall, 92.36 96.17 92.46 91.52 93.71 Grade 5 (12.03) (9.19) (11.58) (13.18) (10.04) Spring, 91.12 95.84 91.68 90.14 92.76 Grade 5 (10.14) (6.04) (8.97) (12.24) (7.49) Fall, 91.09 95.62 91.47 90.30 92.75 Grade 6 (13.39) (9.96) (12.48) (13.23) (12.14) Spring, 89.91 95.58 90.38 88.43 91.93 Grade 6 (12.08) (7.24) (10.72) (13.42) (9.36) Fall, 90.32 95.47 90.66 89.41 92.50 Grade 7 (13.38) (9.55) (12.51) (13.56) (11.22) Spring, 88.31 94.95 88.67 86.88 91.04 Grade 7 (13.81) (9.79) (12.58) (15.25) (10.77) Fall, 89.66 95.32 89.87 88.26 91.87 Grade 8 (14.00) (9.05) (13.14) (15.20) (11.40) Spring, 86.00 92.38 85.99 84.38 88.16 Grade 8 (15.01) (9.47) (14.13) (15.63) (12.01) 20 Table 10: Means and standard deviations for achievement and attendance by language background Mathematics Achievement (withingrade z-scores) Reading Achievement (within-grade z-scores) Attendance (Percent of Days Enrolled) Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Fall, Grade 4 Spring, Grade 4 Fall, Grade 5 Spring, Grade 5 Fall, Grade 6 Spring, Grade 6 Fall, Grade 7 Spring, Grade 7 Fall, Grade 8 Spring, Grade 8 Native English Speakers (n = 177,868) -0.04 (0.98) -0.11 (1.00) -0.15 (1.00) -0.18 (1.03) -0.25 (1.03) 0.00 (1.00) -0.04 (1.03) -0.07 (1.02) -0.08 (1.03) -0.12 (1.04) 93.41 (7.89) Language Minority, Non-ELL (n = 98,083) ELLs in Grade 4 (n = 28,572) 0.20 (0.98) 0.19 (1.00) 0.18 (1.03) 0.17 (1.05) 0.15 (1.11) 0.07 (0.97) 0.08 (0.96) 0.08 (0.99) 0.09 (1.00) 0.08 (1.04) 95.07 (6.60) -0.73 (0.99) -0.65 (1.07) -0.60 (1.08) -0.58 (1.08) -0.49 (1.06) -1.01 (0.89) -0.89 (0.97) -0.86 (0.98) -0.84 (1.04) -0.77 (0.96) 93.98 (7.77) 92.03 (8.73) 94.33 (7.04) 93.50 (8.21) 92.50 (11.38) 94.13 (10.89) 92.78 (12.39) 91.27 (9.56) 93.59 (7.78) 92.74 (8.90) 91.37 (12.85) 93.29 (11.90) 91.89 (12.71) 90.12 (11.53) 92.65 (9.52) 91.56 (10.26) 90.66 (12.93) 92.78 (11.36) 91.22 (12.61) 88.63 (13.32) 91.45 (11.12) 89.87 (12.16) 89.99 (13.50 92.23 (11.67) 90.40 (13.09) 86.16 (14.52) 88.75 (12.48) 87.14 (13.68) 21 Figure 5: Path diagram for hypothesized latent growth in attendance predicted by racial/ethnic group Note: Model also included measurement model for attendance as shown in Figure 1. F= Fall, S = Spring, G = Grade 22 Figure 6: Path diagram for latent growth in mathematics predicted by racial/ethnic group Note: Model also included measurement model for achievement as shown in Figure 2. G = Grade 23 Table 11 presents the results for attendance predicted by ethnic group while Table 12 presents the results for achievement. As shown in Table 11, Native American, Latino, and African-American students all have notably lower initial (grade four ) status in attendance, compared to White students, while Asian students have notably higher grade four status. African-American students demonstrate steeper declines in each of the first three periods, but a less steep decline in the last period, compared to White students. Latino and Native American students demonstrate steeper declines during the middle two periods, spanning grade five to grade seven. Asian students have less steep declines in each of the four periods. It is worth noting that the residual variance for the final measurement occasion was set to 0 to avoid convergence problems. As shown in Table 12, Native America, Latino, and African-American students had substantially lower mathematics and reading/ELA achievement in grade four than their White peers (nearly 1 SD in each case). These gaps persist through grade eight as shown by the relatively trivial differences in rates of growth demonstrated by these three ethnic groups. These results are also illustrated in Figures 5-7 in the main text. Table 13 presents the results for attendance predicted by language background and Table 14 presents the results for achievement. As shown in Table 9, ELLs had slightly higher attendance in grade four than their native English-speaking counterparts, and relatively similar rates of growth. In contrast, language minority learners who were not designated as ELLs had notably higher attendance rates in grade four and slightly less steep rates of decline, compared to native English speakers. As with the models for attendance by ethnicity, the residual variance for the final measurement occasion was set to 0 to avoid convergence problems in this model. In contrast, ELLs’ achievement was much slower than their counterparts, as shown in Table 14. As shown in the column marked Y-standardized estimates, the standardized difference between ELLs and their native English-speaking peers was ¾ of a SD for mathematics achievement and nearly 9/10ths of a SD for reading/ELA achievement in grade four. ELLs made notably improvements over time, as indicated by their more positive rates of growth in both the grade four-six and grade six-eight periods, but remain far below their peers as shown in Figures 9 and 10 in the main text. Language minority learners who were not designated as ELLs had somewhat higher achievement in grade four in both mathematics and reading/ELA as well as slightly higher rates of growth in both periods. 24 Table 11: Selected Results for Piecewise Growth Model for Attendance Predicted by Race/Ethnicity (N = 303,699) Fixed Effects Initial (Fall, Grade 4) Status Slope 1 (Fall, Grade 4 – Fall, Grade 5) Slope 2 (Fall, Grade 5 – Spring, Grade 6) Slope 3 (Spring, Grade 6 – Spring, Grade 7) Slope 4 (Spring, Grade 7 – Spring, Grade 8) Variance Components Intercept (for White) Native American Asian Latino African-American Multiracial Unknown Intercept (for White) Native American Asian Latino African-American Multiracial Unknown Intercept (for White) Native American Asian Latino African-American Multiracial Unknown Intercept (for White) Native American Asian Latino African-American Multiracial Unknown Intercept (for White) Native American Asian Latino African-American Multiracial Unknown Initial Status Unstandardized Estimates 94.59*** Selected Y-Standardized Estimates -1.89*** 2.18*** -1.12*** -1.36*** 0.13 -2.40*** -1.38*** -0.27 0.31 -0.16 -0.19 0.02 -0.34 -0.10 0.64*** 0.07 -0.14** 0.14 -1.45 -0.51*** -0.02 0.10 0.01 -0.02 0.02 -0.23 -0.79*** 0.31*** -0.30*** -0.26*** -1.32* 0.81 -0.75*** -0.19 0.07 -0.07 -0.06 -0.32 0.20 -0.78* 0.31*** -0.89*** -0.82*** -0.83 -3.63* -3.27*** -0.12 0.05 -0.13 -0.12 -0.12 -0.53 0.16 0.40*** 0.10 0.38*** 2.23* 2.50 48.74*** 0.02 0.04 0.01 0.04 0.24 0.27 Slope 1 41.54*** Slope 2 17.38*** Slope 3 46.08*** Slope 4 89.01*** Note: Model also included residual covariances among the latent growth terms as in the unconditional growth model. * p < .05; ** p < .01; *** p < .001 25 Table 12: Selected Results for Piecewise Growth Model for Mathematics and Reading/English-language Arts Achievement Predicted by Race/Ethnicity (N = 303,699) Fixed Effects Initial (G4) Status Slope 1 (G4 – G6) Slope 2 (G6 – G8) Intercept (for White) Reading/ELA (N = 294,863) Unstandardized Selected YEstimates Standardized Estimates 0.56*** Native American Asian Latino AfricanAmerican Multiracial Unknown Intercept (for White) -0.86*** -0.94 -0.86*** -0.96 0.10*** -0.75*** -0.81*** 0.10 -0.81 -0.88 -0.13*** -0.83*** -0.76*** -0.14 -0.93 -0.85 -0.34*** -0.47*** -0.04*** -0.37 -0.52 -0.25*** -0.44*** -0.02*** -0.28 -0.49 Native American Asian Latino AfricanAmerican Multiracial Unknown Intercept (for White) 0.01 0.07 0.004 0.03 0.08*** 0.02*** 0.003 0.51 0.15 0.02 0.05*** 0.02*** -0.01*** 0.33 0.17 -0.08 0.05 -0.001 -0.03*** 0.32 -0.01 0.008 0.003 -0.02*** 0.06 0.02 0.00 -0.001 0.01 0.04 0.09*** 0.005* -0.002 0.52 0.03 -0.01 0.05*** 0.01*** 0.01** 0.35 0.09 0.04 0.00 0.006 0.71*** 0.04 -0.002 0.84*** 0.07* -0.06 0.68*** 0.48 -0.39 Native American Asian Latino AfricanAmerican Multiracial Unknown Variance Components Mathematics (N = 296,323) Unstandardized Selected YEstimates Standardized Estimates 0.54*** Initial Status Slope 1 0.03*** 0.98*** 0.02*** Slope 2 0.03*** 0.97*** 0.02*** Note: Model also included residual covariances among the latent growth terms as in the unconditional growth model. * p < .05; ** p < .01; *** p < .001 26 Table 13: Selected Results for Piecewise Growth Model for Attendance Predicted by Language Background (N = 303,622) Fixed Effects Initial (Fall, G4) Status Slope 1 (Fall, G4 – Fall, G5) Slope 2 (Fall, G5 – Spring, G6) Slope 3 (Spring, G6 – Spring, G7) Slope 4 (Spring, G7 – Spring, G8) Variance Components Intercept (for English-only) Language Minority non-ELL English language learner in G4 Intercept (for English-only) Language Minority non-ELL English language learner in G4 Intercept (for English-only) Language Minority non-ELL English language learner in G4 Intercept (for English-only) Language Minority non-ELL English language learner in G4 Intercept (for English-only) Language Minority non-ELL English language learner in G4 Initial Status Unstandardized Estimates 93.35*** Selected YStandardized Estimates 1.70*** 0.24 0.63*** 0.09 -1.52*** 0.44*** 0.07 0.60*** 0.09 -0.72*** 0.11*** 0.03 -0.03 -0.01 -1.44*** 0.32*** 0.05 -0.20*** -0.03 -3.03*** -0.07 -0.01 -0.03 -0.00 49.52*** Slope 1 41.60*** Slope 2 17.34*** Slope 3 46.19*** Slope 4 89.01*** Note: Model also included residual covariances among the latent growth terms as in the unconditional growth model. * p < .05; ** p < .01; *** p < .001 27 Table 14: Selected Results for Piecewise Growth Model for Achievement Predicted by Language Background Fixed Effects Initial (G4) Status Slope 1 (G4 – G6) Slope 2 (G6 – G8) Intercept (for English-only) Language Minority nonELL English language learner in G4 Intercept (for English-only) Language Minority nonELL English language learner in G4 Intercept (for English-only) Language Minority nonELL English language learner in G4 Mathematics (N = 292,250) Unstandardized Selected YEstimates Standardized Estimates -0.03*** Reading/ELA (N = 294,791) Unstandardized Selected YEstimates Standardized Estimates 0.01*** 0.24*** 0.26 0.08*** 0.27 -0.67*** -0.73 -0.95*** -0.87 -0.04*** -0.03*** 0.04*** 0.26 0.04*** 0.14 0.12*** 0.73 0.10*** 0.45 -0.04*** -0.02*** 0.03*** 0.19 0.02*** 0.14 0.09*** 0.57 0.06*** 0.45 Variance Components Initial 0.78*** 0.93 0.74*** 0.91 Status Slope 1 0.03*** 0.95 0.02*** 0.96 Slope 2 0.03*** 0.97 0.02*** 0.98 Note: Model also included residual covariances among the latent growth terms as in the unconditional growth model. *** p < .001 28 Is middle grades performance equally predictive of later on-track status across ethnic and language groups? To investigate whether middle-grades performance is equally predictive of being on-track in grade nine for ultimate high school graduation, we fitted the model displayed in Figure 5 separately to sub-samples comprises of each ethnic group and of each language group. To investigate whether middle-grades performance is equally predictive of being on-track in grade nine across the different ethnic and language groups, we fitted the model displayed in Figure 5 separately to sub-samples comprises of each ethnic group and of each language group. As shown by the standardized estimates in Table 15, the relative predictive power of attendance and achievement levels and rates of growth appeared to be largely similar across ethnic groups, with some exceptions. For African-Americans, both initial status and Slope 1 in attendance had notably larger effects on grade nine “on-track” indicator score than for other groups. In addition, Slope 3 for White students was notably less predictive of grade nine “on-track” indicator score than for other groups. As shown in Table 16, predictive relationships were also largely similar across language backgrounds, with perhaps only one exception. Attendance Slope 3 was less predictive for language minority students not designated as ELLs compared to other groups. 29 Table 15: Selected Results from Structural Equation Models with Initial Status and Rates of Growth in Both Attendance and Achievement Predicting Grade (G) 9 Ontrack Indicator Score, Fitted Separately for Each Ethnic Group Paths White (n = 44, 871) .29*** Standardized Estimates Asian (n = African-American (n 37, 327) = 101,237) .26*** .37*** Latino (n = 118, 898) .31*** Attendance Initial (Fall, G4) Status G9 Ontrack Indicator Attendance Slope 1 (Fall, G4 –Fall, G5) .27*** .23*** .32*** .27*** G9 Ontrack Indicator Attendance Slope 2 (Fall, G5 –Spring, .30*** .23*** .31*** .26*** G6) G9 Ontrack Indicator Attendance Slope 3 (Spring, G6-Spring, .03 .16*** .23*** .16*** G7) G9 Ontrack Indicator Attendance Slope 4 (Spring, G7-Spring, .39*** .35*** .28*** .31*** G8) G9 Ontrack Indicator Achievement Composite Initial (G4) .31*** .30*** .35*** .33*** Status G9 Ontrack Indicator Achievement Composite Slope 1 (G4-G6) .11*** .12*** .17*** .18*** G9 Ontrack Indicator Achievement Composite Slope 2 (G6-G8) .17*** .19*** .19*** .21*** G9 Ontrack Indicator Note: As shown in Figure 5, mnodel also included variances and intercepts for and covariances among all latent growth terms as well as a residual variance for Grade 9 ontrack indicator score. *** p < .001 30 Table 16: Selected Results from Structural Equation Models with Initial Status and Rates of Growth in Both Attendance and Achievement Predicting Grade (G) 9 Ontrack Indicator Score, Fitted Separately for Each Ethnic Group Native English Speakers (n = 177,842) .33*** Standardized Estimates English Language Learners in G4 (n = 28, 567) Language Minority, Non-ELL (n = 98, 074) Attendance Initial (Fall, G4) Status G9 Ontrack .30*** .29*** Indicator Attendance Slope 1 (Fall, G4 –Fall, G5) G9 Ontrack .29*** .29*** .24*** Indicator Attendance Slope 2 (Fall, G5 –Spring, G6) G9 .28*** .25*** .30*** Ontrack Indicator Attendance Slope 3 (Spring, G6-Spring, G7) G9 .21*** .20*** .03 Ontrack Indicator Attendance Slope 4 (Spring, G7-Spring, G8) G9 .28*** .27*** .38*** Ontrack Indicator Achievement Composite Initial (G4) Status G9 .37*** .32*** .31*** Ontrack Indicator Achievement Composite Slope 1 (G4-G6) G9 .17*** .18*** .14*** Ontrack Indicator Achievement Composite Slope 2 (G6-G8) G9 .18*** .21*** .19*** Ontrack Indicator Note: As shown in Figure 5, mnodel also included variances and intercepts for and covariances among all latent growth terms as well as a residual variance for Grade 9 ontrack indicator score. *** p < .001 31 Do these patterns hold across schools? To investigate the extent to which these patterns hold across different school or are specific to certain schools, we used multilevel SEM modeling to partition the variance in the growth terms for attendance and achievement into within-school and between-school components. This approach provides insight into the extent to which middle schools influence these outcomes. In particular, we fitted models for individual growth nested within grade six school, including only those indicators for attendance and achievement between grade six and grade eight. Figure 7 presents the path diagram for the hypothesized two-level model, with both within-school and between-school components. Table 13 presents selected results from fitting this model. As shown by the interclass correlations (which represent the proportion of total variance that is between schools) for attendance in Table 17, only very small amounts of the variance in attendance initial (grade six) status (five percent) and slopes (two-three percent) are associated with differences between schools. As shown by the interclass correlations for achievement, larger but still relatively small percentages of the variance in achievement initial (grade six) status (27 percent) and slopes (10 percent) are associated with differences between schools. This is evidence to suggest that individual student differences within schools are much larger than differences between schools and that this is particularly the case for attendance. 32 Figure 7: Path diagram for hypothesized multilevel latent growth model for attendance and achievement as nested within grade six school Note: Model also included measurement models for attendance as shown in Figure 1 and for achievement analogous to the model shown in Figure 2. Att = Attendance; Ach = Achievement Composite; G = Grade 33 Table 17: Selected Results for 3-Level Piecewise Growth Model for Unconditional Attendance Growth Nested within Grade 6 School (N = 266,450) Fixed Effects Variance Components Attendance Initial (Fall, G6) Status Attendance Slope 3 (Fall, G6 – Spring, G7) Attendance Slope 4 (Spring, G7 – Spring, G8) Achievement Initial (G6) Status Achievement Slope 2 (G6 –G8) Attendance Initial Status Attendance Slope 3 Attendance Slope 4 Achievement Initial Status Achievement Slope 2 Note: Model also included covariances among the latent growth terms. 34 Intercept Intercept Intercept Intercept Intercept Within-school Between-school Interclass Correlation Within-school Between-school Interclass Correlation Within-school Between-school Interclass Correlation Within-school Between-school Interclass Correlation Within-school Between-school Interclass Correlation Estimates 91.98*** -1.23*** -2.39*** -0.09*** -0.02*** 115.35*** 6.06*** .05 34.51*** 0.66*** .02 35.93*** 1.22*** .03 0.62*** 0.22*** .27 0.03*** 0.003*** .10 III. EXPLORATORY ANALYSIS OF SCHOOL CHARACTERISTICS To explore whether these patterns of achievement and attendance growth differ by observable school characteristics, we conducted some additional descriptive analyses. Specifically, we identified which school students were enrolled in for grade six. We then divided the schools into four quartiles based on three dimensions: 1) their students’ average growth rate in achievement between grade six and grade eight, 2) their students’ average growth rate in attendance between Fall, grade six and Spring, grade seven (i.e., attendance slope three in the original attendance growth models), and 3) their students’ average growth rate in attendance between Spring, grade seven and Spring grade eight (i.e., attendance slope four). We then estimated descriptive statistics (means and standard deviations) for several publically-available variables for school characteristics by achievement and attendance growth quartile. School-level variables were measured in the year in which the sampled students were in grade six; in most cases, this means that we had four values for the school-level variables corresponding to the four cohorts, which were then averaged. It is worth noting that this approach characterizes schools based not on their students’ levels of achievement and attendance (as might typically be done for accountability purposes), but rather on how their students change in achievement and attendance during the middle grades. Table 18 and 19 present the results of these exploratory analyses, with Table 18 presenting results for teacher characterized and Table 19 presenting results for student characteristics aggregated to the school level. When one compares the four quartiles in Table 18, it appears that the measured teacher characteristics, including experience and “highly qualified” status, were roughly the same across schools with substantially different achievement and attendance growth. In contrast, demographic variables do appear to be somewhat different across the schools, consistent with our previous student-level analyses of demographic predictors, as shown in Table 19. For instance, consistent with our previous results, schools which demonstrated higher average achievement growth in the middle grades had lower proportions of African-American students, compared with schools which demonstrated lower average achievement growth. Similarly, schools which demonstrated higher average attendance growth during the Fall, grade six to Spring, grade seven period had lower proportions of AfricanAmerican and Latino students, compared to schools which demonstrated lower average attendance growth. As described in the main text, schools with higher rates of achievement and attendance growth also tended to have much lower concentrations of students receiving free lunch. These exploratory analyses are not meant to support causal inferences about the school characteristics that matter most, but are rather intended to spur additional research into what schools can do to support more positive achievement and attendance trajectories during the middle grades. 35 Table 18: Means for Teacher Characteristics for Schools Classified in the First through Fourth Quartile, based on Achievement and Attendance Growth Percent of teachers with more than 2 years in this school Percent of teachers with more than 5 years experience anywhere School Average G6 - G8 Achievement Growth Quartile 1 Quartile 2 Quartile 3 Quartile 4 67.31 63.96 66.63 64.19 56.26 53.79 55.16 53.09 Percent of core classes taught by highly qualified teachers 88.38 85.61 88.64 87.59 School Average F,G6 - S,G7 Attendance Growth Quartile 1 Quartile 2 Quartile 3 Quartile 4 65.48 64.58 66.39 65.65 54.93 54.23 55.33 53.76 85.28 86.02 87.99 91.19 School Average S,G7 - S, G8 Attendance Growth Quartile 1 Quartile 2 Quartile 3 Quartile 4 66.61 65.83 64.69 65.95 54.40 55.54 53.49 55.63 88.28 87.48 87.11 87.57 Table 19: Means for Student Characteristics for Schools Classified in the First through Fourth Quartile, based on Achievement and Attendance Growth Enrollment Total School Average G6 - G8 Achievement Growth Quartile 1 Quartile 2 Quartile 3 Quartile 4 695 689 738 804 Percent of students receiving free lunch 72.30 75.01 67.70 64.21 School Average F,G6 - S,G7 Attendance Growth Quartile 1 Quartile 2 Quartile 3 Quartile 4 654 676 782 790 84.38 78.91 67.37 46.73 46.49 42.39 31.80 19.91 46.50 43.48 37.96 25.75 15.45 14.80 14.77 12.91 24.72 19.00 13.32 10.20 School Average S,G7 - S, G8 Attendance Growth Quartile 1 Quartile 2 Quartile 3 Quartile 4 898 703 671 715 71.21 67.74 70.15 70.78 29.21 34.56 35.82 40.81 38.57 37.74 39.75 37.58 15.70 14.40 14.77 13.56 13.89 19.80 18.18 12.30 36 Percent of students who are AfricanAmerican 40.97 42.45 31.89 23.22 Percent of students who are Latino 37.78 37.81 39.76 38.80 Percent of students who are ELLs 11.90 14.49 15.13 16.95 Percent of students in special education 18.09 20.86 14.96 12.28 285 Mercer Street, 3rd Floor | New York, New York 10003-9502 212 992 7697 | 212 995 4910 fax [email protected] | www.steinhardt.nyu.edu/research_alliance The Research Alliance for New York City Schools conducts rigorous studies on topics that matter to the city’s public schools. We strive to advance equity and excellence in education by providing non-partisan evidence about policies and practices that promote students’ development and academic success.
© Copyright 2025 Paperzz