Year One Report on Philadelphia’s Accelerated High Schools August 2012 Prepared by Research for Action Michael Norton Eva Gold Ph. D. Renata Peralta Year One Report on Philadelphia’s Accelerated Schools Prepared by Research for Action for the Philadelphia Youth Network August 2012 Authors Michael Norton Eva Gold, Ph.D. Renata Peralta Copyright © 2012 Research for Action About Research for Action Research for Action (RFA) is a Philadelphia-based nonprofit organization. We seek to use research as the basis for the improvement of educational opportunities and outcomes for traditionally underserved students. Our work is designed to strengthen public schools and postsecondary institutions; provide research-based recommendations to policymakers, practitioners, and the public at the local, state, and national levels; and enrich the civic and community dialogue about public education. For more information, please visit our website at www.researchforaction.org. Acknowledgments This research is being sponsored by the Philadelphia Youth Network with the support of the William Penn Foundation. We express our appreciation to Jenny Bogoni and Harvey Chism at the Philadelphia Youth Network, to Majeedah Scott and Dr. Tonya Wolford from the District, and to the entire Project U-Turn Steering Commitee for their guidance and support in year one this project. We would also like to thank the accelerated high school providers: Big Picture Philadelphia, Camelot, Communities in Schools, Delaware Valley High School, Philadelphia OIC, Ombudsman Educational Services, One Bright Ray, and Phase 4 Learning Center for their participation in year one of this research. This report was greatly enhanced by the contributions of many RFA staff. We especially appreciate the oversight provided by Dr. Kate Shaw, Executive Director, Dr. Stephanie Levin, Associate Director, Alison Murawski, Communications Director, and Allison Petrosky, Communications Assistant. Table of Contents Executive Summary .......................................................................................................................................................................................... i Introduction ...................................................................................................................................................................................................... 1 Research Design, Questions and Methods ................................................................................................................................................ 3 Students Who Attend Accelerated High Schools ................................................................................................................................... 6 Intermediate Outcomes: Attendance and Retention ............................................................................................................................ 17 Comparative Analyses: Credit Accumulation and Graduation ........................................................................................................... 23 Intermediate Outcome: Rapid Credit Accumulation ............................................................................................................................ 24 Long-Term Outcome: Graduation ............................................................................................................................................................ 28 Conclusion....................................................................................................................................................................................................... 36 Technical Appendix ....................................................................................................................................................................................... 39 Year One Report on Philadelphia’s Accelerated High Schools Executive Summary August 2012 Introduction This report presents findings from the first year of a two year evaluation of the School District of Philadelphia’s accelerated high schools. In the 2010-11 school year, eight private organizations, both for-profit and non-profit, operated accelerated high schools in the School District of Philadelphia (SDP). Designed to support students at high risk of dropping out of school, or who had previously dropped out, accelerated high schools provide students who are “over-age” and “under-credited” with an opportunity to accelerate their pace towards graduation. The Year One report presents findings from the following descriptive analyses: 1. An historical overview of the expansion of accelerated high schools in the SDP; 2. An analysis of the student populations served by the eight different providers; and 3. An analysis of accelerated high schools’ enrollment, attendance and retention patterns. The main outcomes analyses in the Year One report compared a sample of accelerated high school students to a matched comparison group of students from neighborhood high schools along the following measures: 1. 2. 3. 4. Four-year graduation rate Five-year graduation rate Six-year graduation rate Credit accumulation: 2009-10 and 2010-11 Accelerated high school students were matched to neighborhood high school students at the time they transferred into accelerated high schools to create comparison groups that were equivalent across the following measures: year of 9th grade entry, age, gender, race/ethnicity, special education status, limited English proficiency status, free/reduced lunch status, school days enrolled, average daily attendance, 8th grade PSSA performance, credits earned, and GPA. In addition, the Year One report also provides an overview of the strategies accelerated high school providers employ to support student enrollment, attendance, retention, and rapid credit accumulation. Key Findings As a group, accelerated high schools are out-performing neighborhood high schools in graduating over-age, under-credited students, and spurring credit accumulation among students who otherwise might be student dropouts. The findings in this report also reveal considerable variation across providers in the degree to which they are able to retain students, support accelerated credit accumulation, and graduate their students. i While accelerated high school students significantly outperformed the matched comparison group of students, overall, the percentages of accelerated high school students who graduate remain well below overall SDP averages. Graduation The Year One study found that overall, a significantly greater percentage of accelerated high school students graduated in four, five and six years than a set of comparison students. • Four-year graduates: 33% v. 16% (p<.001) • Five-year graduates: 46% v. 20% (p<.001) • Six-year graduates: 52% v. 21% (p<.001) Factors that Effect Five-Year Graduation for Accelerated High School Students The key academic and demographic factors that significantly influenced accelerated high school students’ likelihood of graduation are many of the same factors that are crucial for any student to succeed in their academic careers. The Year One study found the following factors significantly influenced accelerated high school students’ chances of graduation: • • • • • Provider: There was significant variation in graduation rates across providers; Gender: Girls were more likely to graduate than boys; Age: After age 19, students became increasingly less likely to graduate; Retention: The longer students were enrolled in accelerated high schools, up to three years, the more likely they were to graduate; Attendance: The more often students attended accelerated high schools the more likely they were to graduate. Credit Accumulation The goal of accelerated high schools is to help students earn significantly more credits so they can ‘catch up’ and move forward towards graduation. Every provider adopted some combination of strategies designed to promote rapid credit accumulation for their students. The Year One study found: Accelerated Students v. Comparison Students • • Overall, students at accelerated high schools accumulated more credits at a faster rate than comparable students in neighborhood high schools in 2009-10 and 2010-11; • 2009-10: 8.6 v. 3.4 (p<.001) • 2010-11: 7.3 v. 4.2 (p<.001) Across providers there was considerable variation in the amount of credits students earned in 2009-10 and 2010-11. Strategies to Spur Credit Accumulation • • • • Lengthen the school day and year. Keep focused on progress with regular check-ins. Track credits incrementally and by assignment. Fast track students close to “aging out.” Enrollment, Re-Engagement and Retention Accelerated high schools are designed to re-engage students with their academic careers who have previously dropped out of school, or who are at high risk of drop out. The enrollment and “matching” process for students entering accelerated high schools is the first step in creating the conditions for these students to be successful ii moving forward towards graduation. While the District and providers employ a number of strategies to create the best ‘matches’ between students and schools, attendance and retention remain challenges for accelerated high schools overall. Re-Engaging Student Drop-Outs • • From year to year, more than 20% of students who newly enroll in accelerated high schools had previously dropped out of school. Individual providers vary considerably in the percentage of students they enroll each year who had previously dropped out of school. o In 2010-11 providers ranged from a low 0f 3% to a high of 31% of newly enrolled students who had previously dropped out of school. Attendance and Retention • Attendance and retention continue to be challenges for all accelerated high schools, which are below the overall District averages for average daily attendance and student dropouts. Strategies to Re-Engage and Retain Students • • • • Assign students to an accelerated high school whose academic approach and social services are wellmatched to their particular needs. Prepare students with robust orientation and intake. Build relationships with students from day one. Provide social support systems and/or referrals for students. Accelerated High School Student Populations The students who attend accelerated high schools are “overage and under-credited,” and these students have historically struggled with their academics and attendance. These schools serve a student population that is predominately Black and Latino, and the percentage of special education population they serve is roughly equivalent to the District overall. Demographics • • • • • • Predominately Black and Latino; Slightly more boys than girls; An equivalent percentage of special education students as the District overall; A very limited English language learner population that has been historically well below the District population; Roughly half of students receive free/reduced priced lunch; Enroll in accelerated high schools at age 17.5. Academic Performance Prior to Entry into Accelerated High Schools • • • • • 8th grade PSSA math and reading scores considerably below the District average; Enrolled in school less than a full academic year; Attended less than 70% of the time; Accumulated fewer than 7 credits; A cumulative GPA below 1.00. Programmatic Implications The findings presented in the Year One report point to the following programmatic implications for accelerated high schools and providers: iii 1. Maintain focus on retention and attendance. The longer students remain enrolled in accelerated high schools, and the more frequently they attend, the more likely they will be to graduate from these schools. 2. ‘Fast tracking’ for accelerated high school students should begin well before students reach age 19. After age 19 students become significantly less likely to graduate in five years; finding ways to ‘fast track’ students well before they reach this critical age may go a long way to getting more students to graduation. 3. Increase focus on male students. Additional strategies and/or curriculum adjustments designed to meet the unique needs of male students may help to address the gap between male and female graduation observed in our analyses. Future Research in Year Two In the second year of this evaluation, a set of case study analyses will provide insight into the provider and district conditions and contexts that contribute to the success of high achieving accelerated high schools. In addition, the outcomes analyses will include an additional year of graduation and credit accumulation findings, along with expanded analyses of: 1) school-level factors that influence credit accumulation and graduation, and 2) accelerated high school students’ postsecondary enrollment, retention, and completion. Understanding how the most effective accelerated high schools are producing positive outcomes for their students will provide key insights for other providers, District officials, and the public to better understand the most effective means of supporting students at high risk of dropping out through their high school careers and beyond. iv Year One Report on Philadelphia’s Accelerated High Schools August 2012 Introduction Since 2004, the Office of Multiple Pathways to Graduation of the School District of Philadelphia (SDP), in alliance with Project U-Turn, a city-wide dropout and recovery program, has overseen the creation and operation of 17 accelerated high schools, managed by four for-profit and five not-for-profit providers. These outside school management providers have contracts with the SDP for their services. The accelerated high schools offer over-age, academically-struggling youth an opportunity to accelerate their accumulation of credits and graduate from high school in a timely manner. The Multiple Pathways Office, which oversees all SDP alternative education programs, has developed a portfolio of school options in order to meet the various learning needs of students who are at risk of dropping out or who are already out of school youth. The providers run schools with distinct instructional approaches intended to meet the range of learning needs of this student population. The Technical Appendix contains a table that describes the eight providers operating in 2010-11 and their instructional approaches. In March 2011, the Philadelphia Youth Network (PYN), which convenes Project U-Turn, commissioned Research for Action (RFA) to conduct a two-year mixed method study to assess outcomes for accelerated high school students, and to identify practices at the accelerated high schools that contribute to those outcomes. This Year One report extends a 2010 study of the performance of Philadelphia’s accelerated high schools, which found that students of Philadelphia’s accelerated high schools achieve slightly better rates of credit accumulation and graduation than comparable students in traditional Philadelphia neighborhood high schools (Chaing and Gill, 2010). The research design for this study differs from the 2010 study in a number of key ways. First, the present study expands the pool of accelerated high school students who could be eligible for the comparative analyses. Second, the present study’s treatment of missing data does not rely on estimations for key student outcomes. Third, the present study includes analyses of the impact of key programmatic elements on accelerated high school students’ chances of graduation. Finally, this study includes qualitative and case study research to examine the strategies that the accelerated high schools use to support accelerated graduation, and to identify school, district and provider contextual factors that support or inhibit the success of these schools. Key Finding The findings presented in this Year One report confirm that: • Greater percentages of accelerated high school students continue to graduate in four, five, and six years than comparable neighborhood high school students. • Students at accelerated high schools accumulate more credits at a faster rate than comparable students in neighborhood high schools, although there is considerable variation across providers. 1 RFA’s research reveals that all providers are working to maintain and improve retention by employing a wide array of strategies intended to encourage attendance and high school completion. Nonetheless, moving the needle on student retention remained a challenge over the past five years for accelerated high schools. Developing an understanding of how some providers are effectively meeting the challenges of this unique student population will guide Year Two of this ongoing evaluation. Multiple Pathways to Graduation Philadelphia’s accelerated high schools offer over-age, under-credited students the opportunity to graduate from high school in an accelerated timeframe: the goal is to accumulate approximately 10 credits a year and graduate within 2.5 years. Figure 1. Accelerated high schools: Multiple Pathways to Graduation Students have enrolled in accelerated high schools through a variety of pathways throughout the history of these schools. In recent years, increased collaboration and communication between the District and providers has resulted in a more consistent set of entry points for students entering accelerated high schools. Figure 1 shows that most students now enroll in accelerated high schools through three main entry points: 1. Walk-ins. Youth that go directly to the school to register for an accelerated high school are “walk-ins.” Big Picture, Camelot, and One Bright Ray, have a significant portion of walks-ins because of close ties with community social service providers and/or strong reputations in their local communities; 2. The Re-Engagement Center. The Re-Engagement Center is an SDP office that provides information and placement services for youth interested in earning a high school diploma or GED. The Center now refers almost 100% of the students to all but three accelerated high schools providers: Big Picture, Camelot, and One Bright Ray; and, 3. The Re-Entry Transition Initiative-Welcome Return Assessment Process (RETI-WRAP). RETI-WRAP is an SDP office which helps youth returning from the juvenile justice system to return to school. Only a small portion of accelerated high school students come from RETI-WRAP, and often they go through the Re-Engagement Center. Students who attend accelerated high schools work towards two intermediate outcomes – staying in school (retention) and rapid credit accumulation. Achieving these outcomes is necessary for students to stay on-track to graduation before they “age out” of the District at 22 years of age. 2 About this Report We begin the report by presenting the research questions that guided Year One of this study, and our methods for data collection and analysis. We then follow the Multiple Pathways to Graduation graphic above to sequence our findings. First, we provide an overview of the students who enroll in the Accelerated High Schools and describe: 1) the providers and schools that are part of this network, and 2) the processes used to identify and place students. Next, we present our findings for the two Intermediate Outcomes: attendance and retention, and credit accumulation. Finally, we provide detailed findings on graduation that look at the ways in which accelerated high school providers, student demographics, and students’ attendance, retention and credit accumulation all influence their chances of graduation from accelerated high schools. For attendance and retention analyses we compare outcomes for accelerated high school students to all district students and then examine these outcomes by provider. For credit accumulation and graduation outcomes, we compare a sample of accelerated high school students to a matched set of district students and to all district students. We also examine outcomes by provider. The report concludes with implications for the next round of research. A Technical Appendix provides a detailed review of the methodologies employed. Research Design, Questions and Methods This study is a two-year mixed method study. The first year, which focuses on the 2011-12 school year, provides a descriptive analysis of accelerated high school student populations, providers, and outcomes analyses of student retention, credit accumulation, and graduation. Qualitative data collection and analyses inform the interpretation of the quantitative findings and provide criteria for case study sites and further research. Year Two of this study will extend the quantitative analyses, and include a cross-case study analysis to deepen our understanding of the school, as well as institutional and contextual factors that support accelerated high schools’ success in retaining and graduating students. The first year of this study was guided by the following set of research questions: 1. What are the key characteristics of accelerated high schools, and who do they serve? 2. How do the attendance and retention patterns at accelerated high schools compare to the District overall? 3. Do students at accelerated high schools accumulate credits at a higher rate than similar students at neighborhood high schools? 4. Do students at accelerated high schools graduate at a higher rate than similar students in neighborhood high schools? 5. What are the district, provider and school strategies for retaining and engaging students, and for spurring credit accumulation? Data Collection and Analysis Quantitative Data and Analyses The analyses presented in this report are divided into two sections: • • An historical analysis describing the accelerated high schools, their student populations, and trends in enrollment, attendance, and retention. Findings from a series of comparative analyses that compare the credit accumulation and graduation of a sample of accelerated high school students to a comparison group of SDP neighborhood high school students. 3 The SDP provided student level records for all students in grades 8-12 for the school years 2002-03 through 2010-11. School year 2002-03 served as a baseline year to sort students into eight separate cohorts, defined by ninth grade matriculation. Table 1 below provides an overview of the quantitative data and how it was analyzed. Table 1: Quantitative Data and Analyses Data Analyses Student-level records for all SDP students in grades 8-12 Years: 2002-03 through 2010-11 • Historical Descriptive Analyses • Comparative Analyses • Accelerated high school population • Accelerated Students v. Comparison Group • Demographics • Graduation • Enrollment & Retention • Credit Accumulation Sample Selection The sample of accelerated students was selected using the following basic criteria: Students attended another SDP school for at least one full year prior to entry into the Accelerated high school The last school attended prior to transfer to an accelerated high school could provide information needed to match students with a comparison group (this excludes Charter and Alternative Schools) Attended an accelerated high school for at least 10 months Attending another District school for at least one year prior to entry into an accelerated high school provides students with academic and behavioral indicators to match with other students in a comparison group. Charters and Alternative Schools do not report a number of the key matching indicators (i.e. credit accumulation, GPA, suspensions), making it impossible to match students who attended these schools prior to entering an accelerated high school. A threshold of 10 months, equivalent to a full academic year, was established on the assumption that students would need at least that much time in an accelerated high school setting to substantively benefit from the accelerated curriculum. After identifying accelerated students for the “study population,” a propensity score matching technique was used to select the most equivalent comparison group possible for each analysis; accelerated students were matched to comparison students along each of the following indicators: 1 • • Cohort Baseline Period 2 • • • • Student age at 9th grade entry Gender Ethnicity Limited English Proficiency Status For a detailed review of the matching process, and tables showing the equivalence of the comparison groups to the accelerated high school students, please see Appendix 2. 2 The ‘Baseline Period’ refers to the number of years between the time an Accelerated high school student started 9th grade and when they enrolled in an Accelerated high school. For comparison group students this indicator is the number of years enrolled since 9th grade entry. 1 4 • • • • • • • • • Free/Reduced Lunch Status Special Education Status Total Days Enrolled Average Daily Attendance Percentage of days enrolled in Neighborhood Magnet, or Alternative Schools 3 Percentage of days suspended 8th Grade PSSA Math and Reading Scores Cumulative Credits Earned Cumulative GPA The comparative analyses assess the extent to which a sample of accelerated students and comparison students differ on the following outcomes: • • • • • Average credits accumulated in 2009-10 Average credits accumulated in 2010-11 Percentage of students graduating in 4 years Percentage of students graduating in 5 years Percentage of students graduating in 6 years The credit accumulation analysis relied on basic means comparisons between the average numbers of credits earned in 2009-10 and 2010-11 (accelerated students v. comparison students). Graduation analyses were conducted at two levels: 1) a simple chi-square test of significance between the percentage of students graduating in 4, 5 or 6 years (accelerated students v. comparison groups); and 2) a series of logistic regression analyses estimating the likelihood that students would graduate in 4, 5, or 6 years, (accelerated students vs. comparison group). Following the comparative analyses, provider by provider analyses within the accelerated student group are presented, revealing considerable variation in students’ graduation and credit accumulation by provider. For a detailed account of the regression models used to generate the results presented in this report, please see the Technical Appendix. Qualitative Data and Analysis Accelerated high school providers supplied a range of materials outlining the instructional approaches used in their schools; these materials included school brochures, instructional materials, guides and lesson plans. Interviews were conducted in spring 2012 with eight providers, along with SDP administrators who refer students to accelerated high schools through the Re-Engagement Center and the RETI-WRAP Program. Provider interviews were conducted with the provider CEOs who often invited other relevant staff (e.g., enrollment specialists, lead administrators) to join them in the interview. Table 2 below summarizes the qualitative data collection. 3 ‘Alternative Schools’ refer to any District school that falls under Division 4 and is not an accelerated high school. 5 Table 2. Qualitative Data Collection Qualitative Data Number Interviews Providers SDP Staff 8 2 Documents Reviewed Curricular Documents School Overview Documents Orientation/Intake Documents Re-Engagement Center Documents 12 14 4 2 Websites Reviewed Provider Websites 8 Interviews were designed to better understand provider policies and practices related to student recruitment and enrollment, along with school-level practices that support enrollment and engagement, retention, credit accumulation and graduation. School materials and provider interviews also were used to develop the description of the educational approaches of the providers. Interviews were semi-structured, recorded and transcribed. Transcriptions and documents were reviewed and coded to identify patterns in school based practices as well as variation across providers. This narrative analysis allowed us to expand upon and contextualize the quantitative findings presented in this report. Students Who Attend Accelerated High Schools This section presents descriptive data on students in accelerated high schools. Below, we review trends in enrollment between 2004-05 and 2010-11, students’ prior experiences prior to accelerated high school enrollment, and socio-demographic characteristics of accelerated high school students. Student Enrollment Patterns Overall Accelerated High School Enrollment Eight accelerated high school providers 4 served a total of 3,820 students in the SDP during the 2010-11 academic year. Student enrollment in accelerated high schools has steadily increased as the SDP has opened more accelerated high schools. Figure 2 below shows total accelerated high school student enrollment and the total number of accelerated high schools between the years 2004-05 and 2010-11. 4 Nine providers are listed below because Community Education Partners' schools closed in the 2009-2010 school year due to poor performance. 6 Figure 2. Accelerated High Schools (number of schools in box) and Yearly Enrollment 14 14 8 7 5 4 2 Enrollment by Provider and School Table 3 shows the providers, their schools, years of operation, and the number of students enrolled in each school at the end of each year. The final row shows the overall percentage of 9-12 graders in the SDP who attended accelerated high schools in each year. Table 3. Accelerated High Schools, Providers, and Years of Operation Provider School Big Picture El Centro de Estudiantes 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10 2010-11 300 202 Excel Central Camelot Excel North 148 415 371 330 Excel South Total Communities In Schools (CIS) 379 Performance Learning Center 148 415 371 330 346 374 424 70 494 510 416 868 1313 224 149 Delaware Valley High Accelerated Academy School (DVHS) SW Ombudsman 457 West 90 80 Northeast 69 85 Northwest 88 91 Total 247 256 7 Fairhill One Bright Ray Phase 4* 188 422 391 422 382 449 402 383 63 182 376 315 445 631 778 698 CLA 380 CADI 123 240 223 333 295 282 255 Total 123 240 223 333 295 282 635 14 110 Phase 4 Learning Center Accelerated Learning Academy Community Education Partners (CEP) 391 North Philadelphia Community Total Opportunities Industrial Center (OIC) 188 25 239 294 376 379 212 304 350 182 268 337 339 25 421 774 1017 1068 Accelerated Learning Academy South Southwest Accelerated Total Total Accelerated Students 459 1,071 1,437 1,882 2,359 3,781 3,820 Percentage of District 9-12 Grade Enrollment 1% 2% 2% 3% 4% 7% 7% *Phase 4 Learning Center operates on a different model that the rest of the accelerated high schools in the District. With different points of entry for students and a comparatively smaller population, the student population at Phase 4 is somewhat unique among accelerated high schools. Overall, the accelerated high schools are serving an increasing percentage of the total district population of 912th graders, starting with 1% in 2004-05 and rising to 7% by 2010-11. This increase in percentage is largely due to more providers operating an increasing number of schools with the largest provider, Camelot, operating three schools and serving 1,313 students in 2010-11. The smallest provider, Phase 4, operates one school serving 110 students. As seen in Table 3, enrollment fluctuations from year to year can be partially attributed to school openings and closings. However, as this report will show, student retention is an ongoing challenge for all these schools. Year-to-year declines in enrollment may also reflect these student enrollment challenges. After the 2009-10 academic year, three schools operated by Community Education Partners (CEP), a for- profit operator, were closed. District officials confirmed that the closings were due to lack of data on the number of students they enrolled, little tracking of students, no sufficient evidence that students were progressing in reading and math, and no records of graduation. Despite these closures, enrollments in 2010-11 did not decline overall since CEP schools were taken over by other accelerated high school providers including Delaware Valley, OIC, and Camelot. Overall Feeder Patterns of Enrollment for District Students As described in the introduction to this report, students currently access accelerated high schools in three main ways. However, prior to 2010-11 individual schools and providers enrolled students through a range of formalized and ad hoc relationships with neighborhood high schools, other District contacts, and people in their local communities. For example, prior to 2010-11 a number of neighborhood high schools paid for “seats” in accelerated high schools, to which they could send students believed to be at risk of dropping out. Table 4 below shows the overall distribution of 9-12th grade students across the SDP in four broad school types: 8 Neighborhood, Citywide/Magnet, and Alternative schools. In Table 4, the “Alternative” schools include all schools that fall under the District’s Division 4, which includes the accelerated high schools along with a number of other schools for non-traditional students. 5 Table 4. Percentages of 9-12 t h Grade Students Attending School Types of SDP Schools School Type 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10 2010-11 Alternative 1% 2% 3% 4% 6% 8% 8% Citywide/Magnet 25% 25% 25% 28% 29% 31% 32% Neighborhood 74% 73% 71% 67% 65% 61% 59% Overall Feeder Patterns of Enrollment for Accelerated Students Figure 3 shows the type of school first-time accelerated students attended prior to their enrollment in an accelerated high school. The category of “Alternative Schools” includes all schools that fall under the District’s Division 4, but exclude accelerated high schools. Overall, the patterns seen in Figure 3 mirror the overall trends in the District, with neighborhood schools serving a declining majority of students while alternative schools slowly increase their percentage of students District-wide from 2004-05 to 2010-11. Figure 3. Accelerated High School Students: Overall Feeder Schools In the 2009-10 school year, a growing percentage of students enrolled in accelerated high schools came from outside the District, suggesting that these schools are increasingly accepting students from charters, other districts or private schools. It is also notable that accelerated high schools draw very few students from citywide/magnet schools. 5 http://webgui.phila.k12.pa.us/offices/r/alternative/ 9 Provider-by-Provider Feeder Pattern of Enrollment Figure 4 below, shows the overall feeder patterns across accelerated high school providers for the percentages of their students transferring from neighborhood schools, magnet schools, alternative schools, directly from 8th grade, or from out of District. Figure 4. Accelerated High School Students: Feeder Patterns by Provider Across providers, Figure 4 shows relatively consistent feeder patterns with several notable exceptions. Only Big Picture has not enrolled a majority of its students directly from neighborhood high schools. A sizable minority of Big Picture’s students transferred into El Centro from outside the district, which greatly exceeds the “Out of District” share of students for any other. 6 Also, DVHS, Ombudsman, and OIC schools have historically accepted the greatest percentage of students from alternative schools, compared to the other providers. That accelerated high schools have historically drawn heavily from neighborhood schools is a function of several factors: • • • Neighborhood schools serve a large majority of the District’s overall student body Neighborhood schools are non-selective and serve a disproportionate percentage of academically at risk students, and Neighborhood schools have an historical relationship with accelerated high school providers. According to a number of providers, from 2004-05 until 2009-10, neighborhood high schools ’bought’ seats at their local accelerated high schools to move students directly from their school who were at-risk of dropping out. Some providers expressed concern with this practice because, they believe, neighborhood high schools were “dumping” their students with behavioral problems on accelerated high schools, and that accelerated high schools should be serving students who had already dropped out. As one provider explained, “the kids should be out of school. I don't think this program is for kids who are already attending a school but may not be served well." Reengagement Center staff noted that the disproportionate percentage of students transferring to El Centro from ‘out of the District’ is likely the result of an administrative practice at the school. When El Centro opened, students were assigned new student IDs by the school, and these students have not all be updated in the Districts student database. 6 10 On the other hand, some providers felt that having relationships with the neighborhood schools was ideal. For example, one provider commented that "before things changed, I used to interact with the parents and the staff of the sending schools, which I don't do as much now that the re-engagement center picks who will attend. The main three sending schools were relatively close to [us], so that was good too." Overall Dropout Re-Enrollment While accelerated high school students primarily come from other schools in the District, a primary goal of accelerated high schools in Philadelphia is to re-engage students who have dropped out of school. Figure 5 shows that the percentage of students who drop out school prior to re-enrolling in an accelerated high school has fluctuated between roughly 20 and 40 percent of between 2005- 2011 7. Figure 5. Re-Enrolling Student Dropouts: Overall Accelerated High School Provider by Provider Dropout Re-Enrollment Across accelerated high school providers, Figure 6 reveals considerable variation over time in the percentage of students enrolled each year who had previously dropped out of school. Student dropouts were identified in the school district data set in consultation with District staff and based on the dropout criteria used by Neild and Balfanz (2006). 7 11 Figure 6. Re-Enrolling Student Dropouts: Provider by Provider Among the longest-running providers still in operation in 2010-11, OIC has historically enrolled the greatest percentage of students who had previously dropped out of school, while Camelot has enrolled the smallest percentage of such students. Among those providers that began operation in 2009-10, only Big Picture appreciably expanded the percentage of newly enrolled students who had previously dropped out of school in 2010-11. Among the other ‘new providers’ there was not much change in the percentage of newly enrolled students who had previously dropped out of school from 2009-10 to 2010-11. By 2010-11, OIC and One Bright Ray admitted the greatest percentage of students who had previously dropped out of schools (roughly 31%), while CIS (roughly 6%)Phase 4 (roughly 3%), admitted the least. The enrollment of previously dropped-out students at Ombudsman, Big Picture, DVHS, and Camelot clustered 20%. Schools enrolling greater percentages of these students are likely to face particular challenges in meeting intermediate (attendance, grades) and long-term (graduation) goals. 8 Student Socio-Demographics Overall Accelerated High School Student Demographics Although the number of students enrolled in accelerated high schools has risen significantly since 2004-2005, their demographic profile has remained relatively unchanged. Figures 7 and 8, below, show overall stability in the racial profile of accelerated students since 2006, and only modest fluctuations in gender, limited English proficiency status, receipt of free and reduced-price lunch, 9 and special education status. 8 In Year II of the current study, this school level factor will be incorporated into broader impact analyses of student retention, credit accumulation and graduation. 9 Free and reduced priced lunch percentages reflect the actual percentage of students receiving a free/reduced priced lunch. This is not a measure of the percentage of students who are eligible to receive free/reduced priced lunch. 12 Figures 7-8: Overall Accelerated High School Student Demographics Accelerated high schools serve a student population that is largely Black and Latino. These schools serve slightly more boys than girls, a relatively constant minority of special education students, and a very limited number of English language learners. Roughly half their students receive free/reduced priced lunch. Accelerated Student Demographics and District Student Demographics In most instances, accelerated high schools serve a student population that is demographically similar to 9-12 graders across the District over time. For example, as can be seen in Figures 9 and 10, accelerated high schools serve equivalent populations of Black (around 65%) and White (around 14%) students as District high schools (excluding high school students in charter schools). They also have comparable populations of special education students (between 14 and 16%), while serving considerabley fewer students with Limited English Proficiency (LEP) students (around 5%) than the District overall (around 10%). Figures 9-10. Overall District Demographics However, there are a number of key socio-demographic differences between accelerated high schools and students in other District high schools: • • • Accelerated high schools serve a disproportionate percentage of Latino students. Accelerated high schools serve slightly more boys than the District overall. Accelerated high schools have historically served a greater percentage of students receiving free/reduced priced lunch. However, by 2010-11 this measure was roughly equivalent between accelerated high schools and the District, due to overall stability in accelerated high schools and steady 13 growth in the District from 37% of students receiving free/reduced lunch in 2004-05 up to roughly 50% in 2010-11. Student Demographics by Provider The demographic characteristics of accelerated students vary in some important ways across providers. Below, we present provider-by-provider socio-demographics. Ethnicity There is considerable variation across providers in terms of the ethnicity of their student populations, although their populations have remained relatively stable over time. As Figure 11 below shows, both Big Picture and One Bright Ray schools serve a majority of Latino students, while Camelot, Communities in Schools Delaware Valley High School, Ombudsman and OIC serve populations that are majority Black. Phase 4 is the only provider serving a majority White student population. Figure 9. Accelerated Student Ethnicity: By Provider Gender, LEP and Free/Reduced Price Lunch Table 6 illustrates relatively consistent patterns in the types of students enrolled over time for each provider. The key ways providers vary across these measures are listed below: • • • Gender. Camelot, Ombudsman, and Phase 4 serve substantially larger male populations than the other providers. LEP. There is little variation with regard to serving the small percentage of LEP students, with the exception of Big Picture, which has greater capacity to serve Spanish speakers because they have hired bilingual staff. Free and Reduced-Priced Lunch. The considerable fluctuation in the percentage of students receiving free/reduced priced lunch seen in Figure 5, above, is mirrored across providers. While the proportion of students in this category has been steadily moving up in the District overall, there have 14 • been significant reductions in the percentage of students receiving free/reduced priced lunch at Camelot and OIC schools. 10 Special Education. Table 6 illustrates a consistent reduction in the percentage of special education students being served by providers. This phenomenon is particularly true of Camelot and Communities in Schools, whose special education populations have reduced by roughly half during their years of operation. Table 6. Accelerated High School Providers: Student Demographics Provider Big Picture Camelot Communities In Schools (CIS) Delaware Valley High School (DVHS) Ombudsman One Bright Ray Opportunities Industrial Center (OIC) Phase 4 Indicator 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10 2010-11 Men - - - - - 53% 46% LEP - - - - - 7% 11% Free Lunch - - - - - 31% 53% Special Ed. - - - - - 14% 16% Men 68% 64% 64% 60% 66% 62% 56% LEP 6% 3% 3% 1% 1% 2% 2% Free Lunch 86% 77% 81% 32% 35% 47% 37% Special Ed. 32% 26% 23% 22% 21% 20% 16% Men - - - - - 63% 60% LEP - - - - - 1% 1% Free Lunch - - - - - 56% 62% Special Ed. - - - - - 18% 7% Men - - - - - - 52% LEP - - - - - - 1% Free Lunch - - - - - - 35% Special Ed. - - - - - - 14% Men - - - - - 57% 58% LEP - - - - - 1% 2% Free Lunch - - - - - 43% 34% Special Ed. - - - - - 13% 13% Men 45% 40% 37% 37% 40% 42% 39% LEP S 7% 5% 3% 2% 2% 2% 3% Free Lunch 33% 42% 42% 39% 53% 42% 40% Special Ed. 9% 15% 17% 12% 10% 11% 9% Men 54% 56% 48% 45% 39% 43% 49% LEP 1% 0% 0% 0% 0% 1% 1% Free Lunch 52% 50% 57% 56% 55% 42% 40% Special Ed. 5% 9% 11% 13% 15% 12% 12% Men - - - - - 64% 66% LEP - - - - - 7% 2% Free Lunch - - - - - 43% 32% Special Ed. - - - - - 7% 8% 10 Interpreting fluctuations in students’ receipt of free/reduced priced lunch is difficult given the diversity of factors that may influence whether students enroll in the program, which could range from social stigma associated with receipt of free lunch, attending half-day programs in which students don’t have ‘lunch, to simply not liking the food, among others. 15 Community Education Partners (CEP) Men - 72% 52% 49% 47% 48% - LEP - 0% 0% 2% 1% 1% - Free Lunch - 32% 35% 35% 51% 42% - Special Ed. - 28% 10% 9% 7% 10% - In the face of the challenges of working with their student populations, many providers expressed concern over their ability to also serve the special education population. For example, one provider commented, “sometimes ESL and special education kids can get more help at their neighborhood school." Providers appeared especially concerned about their having the resources for students with more serious learning disabilities. One provider cited a special education funding disparity as a possible explanation for providers’ concern about serving special education students. He claimed that accelerated high schools receive slightly more money per student than the district average, but considerably less than district schools for each of their special education students. Regardless, a number of providers did point to key features of their learning environments that they believe provide good support to special education students. For example, one provider said, With an unusual IEP, we can't really accommodate, although special education kids can work well here because classes are small, teachers have some special education backgrounds, we have pull out classes, and we work with community groups and social service agencies. In particular, providers using A Plus, the computer-based instructional program, pointed out that they could adjust the program to accommodate a diverse range of Individual Education Plans (IEPs). Challenges Faced by Students in Accelerated High Schools Providers and District staff identified a number of challenges accelerated high school students face that must be taken into account to understand the student population accelerated high schools serve. These challenges fall into three broad categories: school issues, home issues, and personal issues. They are summarized in Table 7. Table 7. Challenges Faced by Students in Accelerated High Schools Category School Issues Home Issues Personal Issues Characteristics • • • • • • • • • • • • • • • • • Boredom Not challenged in school Didn't like assigned school Truancy Behavioral Issues Learning difficulties Safety Overage differences Family issues Homelessness Death of a loved one Financial problems Housing instability Pregnancy Childcare Drug and alcohol abuse Arrest/Incarceration Providers discussed these characteristics as both barriers to graduation and as areas in which schools need to provide services and supports. For example, pregnant or parenting youth might need time off from school for a sick child or to arrange childcare. Some providers said they can accommodate this need either through use a half-day program or a more tailored environment. One provider said, “surprisingly a large portion of our student body, unfortunately, is…are pregnant, or are parenting. So I mean we get a lot of students who just need a smaller environment and more hands on.” The same is true for the other issues listed in Table 7. Many 16 providers point out that their schools offer an environment tailored to fit these students’ characteristics and needs. Summary Accelerated high schools serve a student population that in many ways mirrors the overall District population of 9-12th graders, with a few notable exceptions. Accelerated high schools serve more male students than District schools overall, more Latino students, and receive a disproportionate percentage of their students from alternative/disciplinary schools in the District; each of these groups of students have historically underperformed in the District. In addition, while accelerated high schools serve roughly equivalent populations of special education and students with Limited English Proficiency, a number of providers believed that for the most part they do not have the resources to meet the unique needs of these student populations. Only two providers were serving English language learners, and while all the providers serve special education students, there was an overall concern about being able to serve those students with more serious needs. Intermediate Outcome: Attendance and Retention Attendance and retention in accelerated high schools are crucial factors that influence students’ likelihood of graduation from accelerated high schools. Providers, school leaders and the Reengagement Center staff all employ one or more strategies to keep students in school and persisting to graduation. These strategies, described in detail below, include involving students in selecting the school they will attend, implementing orientation and intake procedures which familiarize students and parents with the expectations and norms of the school, building strong student-adult relationships to reengage previously disconnected students, and providing robust academic and social supports. Figures 10 and 11 below present the average daily attendance, and retention patterns, first for accelerated high schools as a group, and then by provider. Attendance As Figure 12 shows, after an initial surge in attendance in their second year of operation, average daily attendance in accelerated high schools has steadily declined since 2006, with a slight uptick in attendance in 2010-11. Figure12. Accelerated High School Average Daily Attendance In 2009-10 and 2010-11 attendance at accelerated high schools fell below the District wide average for 9-12th graders, which has consistently remained just below 80%. One possible explanation for this dip is providers’ strict adherence to dropping students who are absent for 10 days. If students are dismissed from accelerated 17 high schools after having missed 10 days of class, their overall attendance for the year will likely be rather low, with fewer days enrolled that ‘typical’ accelerated high school students along with 10 absences. Without the opportunity to stay enrolled and improve their overall attendance as the academic year proceeds, these students’ attendance may bring down the rest of the schools’ average daily attendance as well. This policy also has potential effects for accelerated high schools ability to retain students from year to year. Attendance by Provider An analysis of attendance patterns by provider gives some insight into why the average overall attendance rates at accelerated high schools dropped in 2009-10. As can be seen in Figure 13, Ombudsman, Phase 4, and Communities in Schools became new providers, and their schools struggled with attendance in their first year of operation. In addition, there were sharp declines in attendance at both CEP 11 and One Bright Ray schools. In 2010-11, attendance across providers remained inconsistent with Big Picture, OIC, and CIS experiencing declines in attendance, while Phase 4 and Ombudsman improved their overall attendance, and Camelot and One Bright Ray held steady. Figure 13. Accelerated High School Average Daily Attendance by Provider While the early accelerated high school providers experienced increasing attendance rates going into their second year of operation, new providers seem to be struggling with attendance in their first years of operation. Retention Retention is another important intermediate outcome that contributes to whether students will graduate from high school before they age out. We provide our analysis of retention for the accelerated high schools as a group, and by provider, in the following sections. Overall Accelerated High School Retention Figure 14 below indicates accelerated high school students’ status at the end of each school year. Students fall into one of four categories: graduates, retentions, transfers or dropouts. 12 After a strong start in 2004-05, student retention dropped in 2005-06 and has remained relatively stable since then, with roughly half of CEP schools were closed after the 2009-10 school year. The School District typically measures ‘retention’ within individual student cohorts, whereas the retention findings here are presented as ‘end of year status’ for individual students. Assessing retention within cohorts in accelerated high schools does not make sense conceptually since these students do not enter accelerated high schools at the same point in their academic careers, or in their lives more generally. For this reason, we look at retention as a ‘year to year’ measure across accelerated high schools and providers, rather than as a cohort measure. 11 12 18 students returning to their schools from year to year, with the notable expansion of transfer students after the 2009-10 school year. Figure 14. Overall Accelerated Students’ End of Year Status The expansion of “transfers” in 2009-10 is directly due to the closure of the CEP schools, seen in Figure 15. When the CEP schools closed after the 2009-10 school year the vast majority of these students transferred and enrolled in other accelerated high schools. If we consider these students as “retained” in the accelerated high schools portfolio overall, then retention remained roughly stable in 2009-10. 19 Accelerated High School Retention v. District Retention Figure 15. Overall District End of Year Status Overall retention, measured as the percentage of students returning to the same school from year to year, has remained steady throughout the District at just under 60% from 2004-05 through 2009-10. That accelerated high schools, which serve students most at risk of dropping out, have retained around 50% of their students from year to year during this period is remarkable. However, the percentage of students dropping out of accelerated high schools from year to year has been considerably higher than the District overall. The percentage of students dropping out of accelerated high schools has been creeping up since 2007-08 towards 30%, whereas District-wide this measure has been receding from a peak of 22% in 2004-05 to roughly 8% in 2009-10. Retention by Provider Figure 16 below shows the percentage of students who were either retained or graduated from each accelerated high school provider from 2004-05 to 2009-10. 13 As Figure 16 shows, overall there have been considerable fluctuations in retention and graduation across the longest running providers: Camelot, One Bright Ray, OIC and CEP. The large jump of transfers in 2009-10 noted above in Figure 14 is reflected in the closure of the CEP schools seen in Figure 16. Another pattern seen in Figure 14 that is reflected here in Figure 16, is that the longest running providers were most effective in retaining students in their first year of operation. However, after their first year retention and graduation has been inconsistent across these providers. 13 Only these two ‘end of year status’ indicators are pictured in Figure 14 since they represent positive year to year outcomes. 20 Figure 16. Accelerated High School Retention, by Provider In addition, there was considerable variation across those providers that began operation in 2009-10, with Phase 4 retaining/graduating nearly 80% of their students and Big Picture retaining/graduating only about 40% of their students. Emerging Best Practices: Student Engagement Keeping Accelerated Students Engaged and Enrolled Accelerated high school providers and District staff are well aware of the challenges associated with working with a student population who are a high risk for dropping out, and as the figures in this section suggest, attendance and retention are a challenge at accelerated high schools. However, providers have adopted a number of promising strategies to support these young people on the way to graduation. Below are four key strategies providers have adopted to keep students in school, attending, and engaged in their education. Strategy 1: Assigning Students to the Right Accelerated High School SDP staff consider retention a key goal as they assign students to an accelerated high school. To increase its likelihood, the assignment process is designed to invest youth in the school selection process. For example, when youth come to the Re-Engagement Center, they can view a video introducing them to the accelerated high schools, which provides them with the opportunity to be part of the decision about which school in the portfolio of accelerated high schools best meets their interests and needs. In addition, Reengagement staff guide students in their decisions: they believe that students who work independently are better suited to computer based programs, and those who need more guidance are more apt to succeed in teacher led programs. Generally, providers believe that the Re-Engagement Center’s knowledge of the accelerated high schools has improved to a degree that has positively influenced the student assignment process. One provider’s comments reflect this increased confidence in the Reengagement Center. “I really do believe that the referral process, and the type of student that has been referred to us, has gotten a lot better…” In addition, several providers reported that the majority of their students seek out their schools. Youth or their families had heard about the accelerated high school through “word of mouth,” which the providers believed 21 reflected the strong reputations their schools have with local community. They commented that they were enrolling siblings, cousins and friends of prior students. Providers and Re-engagement staff agreed that the following two factors also should inform decisions about school assignment. • • Reengagement Defined Reconnecting students who have become disconnected from school Geography—schools should be close to where students live to avoid long commutes; and Flexibility—the portfolio of accelerated high schools should include schools that can accommodate a diverse range of needs: specifically the needs of students who are pregnant, parenting and/or working. The half day programs seem particularly well-suited for these types of students. Strategy 2: Orientation and Intake Orientation and intake are means through which providers and accelerated high school leaders make sure students understand what is required of them in their schools. These processes are also an opportunity for students to decide if they are willing to commit to what it will take to succeed in an accelerated high school. Orientation and intake processes involve one of more of the following: • • • • Identify readiness for an accelerated high school; Involve the entire family in ensuring success; Allow students to experience some the challenges they will encounter in an accelerated high school before enrolling; and Establish academic and behavioral baselines for reference later in the program. A number of providers said that during orientation and intake they are looking for whether a student is ready for the challenges of an accelerated high school. Although their philosophy was often that it is the schools responsibility to adapt to the student, when they believed a young person was not ready, they said they might counsel him/her to come back in a few weeks or a semester, when they might be more ready. Often, they found, that older students were more ready for the demands of their schools. To make young people aware of what will be required, in their orientation or intake process they emphasized that enrollment in their school requires the need to conform to school behavioral norms and keep up with the accelerated pace. Often parents or guardians are invited to a portion of orientation, or to the intake interview, to make sure the family is aware of the effort the student will have to make, and will assist the school in keeping the young person on track to graduation. One provider described instituting a long and intense orientation, a strategy he learned from a nearby charter school, saying that only students that could persist through orientation would then enroll. The orientation in this case, was a “taste” of what the student would experience in the school, and offered the young person a chance to decide if s/he was ready to commit to the effort it would take to be successful in this accelerated high school. Another provider talked about setting a base line in the intake interview, helping students to identify habits they need to change. Then, the provider described being able to use this interview later, if a student begins to “slip.” Most kids come in and they start pretty strong, and then they tend to maybe, possibly go into some of their old ways….You can go back to their [initial] interview, and say,….Hey, you’re slipping back into what you did at [high school]….How can we help you maintain your goal, go back to what is that you want to do [here]? Strategy 3: Relationship Building 22 All the providers believe that reengagement is the key to retaining students, and that creating strong adultstudent relationships in their schools is a core reengagement strategy. Through advisories, mentoring, and other means the providers said the schools were ensuring that students felt that adults in the school building “cared” about them. Many providers initiated the student-adult relationship building at orientation or in-take, to be sure the students saw the difference between the accelerated high schools and their students’ prior schools, where many young people told them they felt that no one really knew them. Some providers also believed that reengagement was enhanced by clear structures and predictable environments. They believed that clarity and predictability engendered trust, and that trusting relationships increased the likelihood that a young person would stay engaged and persist to graduation. Strategy 4: Social Support Systems “Our biggest issue is that, if they’re not mentally healthy, or physically healthy, we can’t educate them… We provide—I wouldn’t say that we provide every single service here, but at least if the conversation starts here, we can try to internally support or externally connect students to some sort of support … It helps with retention.” (Accelerated High School Provider) As indicated earlier in this report, youth attending accelerated high schools face numerous social, behavioral and familial challenges. A number of the providers, such as the one quoted above, recognize the importance of helping students with their many needs, in order to keep them in school. Some providers have relationships with social service agencies and refer students to them, others incorporate services into their programs, and some do a combination of both. Regardless, many providers believe that unless students get help with their emotional and/or physical problems, they will not succeed in staying focused on their academics, and meeting the requirements to accumulate credits at an accelerated pace. Summary The District and providers have an array of strategies designed to keep students engaged. These strategies aim to make sure that the match between student and school will meet the learning and social needs of diverse youth. They also are intended to ensure that youth and their families are aware of the demands of the accelerated schools programs and feel ready to meet these demands. Despite these strategies—all intended to address issues that affect retention—overall retention in 2009-10 in accelerated high schools was only 32%, compared to 59% in the District overall. There was significant variation in retention among providers, with five providers retaining less than 50% of their students. Comparative Analyses: Credit Accumulation and Graduation The next two sections present findings from a set of comparative analyses designed to assess the significance of differences between accelerated high school students and a matched comparison group of neighborhood high school students’ credit accumulation and graduation. The overall samples of accelerated students in these analyses are restricted to students who started as first time 9th graders between 2003-04 and 2009-10. In addition, these students all attended a District neighborhood high school for at least one year before entering an accelerated high school and they attended an accelerated high school for at least 10 months. Overall, the samples of both the accelerated students and the matched comparison students represent students who were struggling with attendance and academics. At the time when students were selected for the matched comparisons these students had: 1. 2. 3. 4. 5. 8th grade PSSA math and reading scores considerably below the District average; Enrolled in 9th grade near age 15; Enrolled in school less than a full academic year; Attended less than 70% of the time; Accumulated fewer than 7 credits; 23 6. A cumulative GPA below 1.00 Intermediate Outcome: Rapid Credit Accumulation Increasing the rate at which students can accumulate high school credit is the primary tool accelerated high schools employ to support students toward graduation. As such, credit accumulation should be thought of as an intermediate outcome along the path to graduation. In the SDP, students need a total of 23.5 credits to graduate and these credits have to be earned across a combination of core content areas and elective courses. 14 Literature on early warning signs for dropping out of high school often cites the failure to accumulate enough credits in ninth grade as a predictor for dropping out. 15 Because of this, providers discuss strategies for accelerating and maintaining credit accumulation for their students as an important measure of success. This section provides evidence that the accelerated high schools have been successful in spurring credit accumulation; and that they employ a number of key strategies to achieve this goal. While the findings presented in this section are encouraging, at least one provider expressed concern about the quality of their credit accumulation data may under represent their students’ performance. This provider suggested that misalignment between the Districts’ timing for data collection and accelerated high schools’ extended academic year, has the potential to misrepresent the actual progress of accelerated high school students who continue to accumulate credits after other District schools close at the end of each school year. Accelerated Students v. Comparison Students The findings presented in Figures 15 and 16 were generated through a comparative analysis of a sample of accelerated students’ credit accumulation compared to a matched comparison group of neighborhood high school students in 2009-10 and 2010-11. 16 The findings were generated through a simple means comparison between the average numbers of credits earned by accelerated high school students compared to the average number of credits earned by the comparison group of students. 17 Credit Accumulation, Matched Comparison: Accelerated Students vs. Comparison Students As seen in Figure 17 below, in 2009-10, the sample of accelerated students earned more than twice as many credits, on average, than the comparison group of students – 8.6 v. 3.4 (p<.001). In 2010-11, while the average number of credits earned by the accelerated students declined, their credit accumulation was still significantly higher than the comparison group: 7.3 v. 4.2 (p<.001). 14 The data requested for this analysis did not include the types of student credits students earned, so we were unable to analyze which credits accelerated students were coming in to their schools with, and which credits they were accumulating through their progress in these schools. 15 Neild, R; Balfanz,R; Herzog, L. An Early Warning Sign. Educational Leadership. 2007. 65: 2. 16 Prior to 2009-10, record keeping associated with student credit accumulation at accelerated high schools has been acknowledged by providers and the District as unreliable (Chaing & Gill 2010). For this reason, the credit accumulation analyses are restricted to the 2009-10 and 2010-11 academic years. 17 For these analyses, all students with ‘missing credits’ were dropped. While the majority of students with ‘missing credit’ data were either ‘drop outs’ or ‘transfers’, there was a sizable minority of ‘retained’ students and ‘graduates’ who also did not have records for credits earned in each year. 24 Figure 17. Credit Accumulation: Accelerated Students v. Comparison Students In Figure 17, the District average is represented by the vertical blue and green lines. It is noteworthy that accelerated students also earned, on average, significantly more credits than all other 9-12th grade students in the District: 8.6 v. 7.1 (p<.001) in 2009-10; and were also above the District average in 2010-11 7.3 v. 7.0 in 2010-11 18. Credit Accumulation by Provider Figure 18 below compares students’ average credit accumulation by provider in 2009-10 and 2010-11. 19 18 19 While this difference was not statistically significant, it was very closely approaching significance. Figure 16 only represents those students who were part of the matched comparison groups in the previous figure. 25 Figure 18. Matched Accelerated Students’ Credit Accumulation: Provider Comparisons The considerable variation in credit accumulation across providers, along with the year to year changes at individual providers is difficult to explain within this basic comparison. Figure 18 does reveal several outliers, with CIS and Ombudsman students experiencing substantial year to year gains, while One Bright Ray students recorded substantial year to year reductions in their average credit earnings. With only two years of reliable credit data it remains unclear whether a discernible pattern will emerge with respect to credit accumulation across accelerated high school and providers. What is clear, however, is that there is considerable variation in the student populations served by these providers. The amount and type of credits accelerated these students will need to graduate from their accelerated high schools largely depends on where these students are in their high school careers when they enter accelerated schools; this variability across students may be one possible explanation for the variation observed in Figure 16. Emerging Best Practices: Spurring Credit Accumulation While a number of providers reported that their contracts with the SDP do not specify how many credits students should be earning each year, half of the providers interviewed said they had set for themselves yearly credit targets, which ranged from 9.5 to 10.5 credits annually, in contrast to the District overall, where student accumulate roughly 7 credits each school year to stay on track to graduation. Each of the providers identified strategies for accelerating credit accumulation, which were intended to support the goal of accumulating credits without sacrificing the acquisition of content knowledge along the way. Below, we discuss four of them: • • • • Lengthening the school year and day Keeping a focus on progress Tracking credits Fast tracking students about to age out Strategy 1: Lengthening the School Year and School Day Many providers extend students’ learning hours to support their credit accumulation. Some programs extend their school year into July; one provider had summer school where students could earn extra credit; and some offer tutoring outside of school hours. As one provider explained, “We will run summer school, we will extend 26 the day, we will give them an opportunity to earn an additional credit, and… when we had the money, the funds, we would do evening school.” Strategy 2: Keeping a Focus on Progress To keep students focused on credit accumulation, many providers create ways for students to chart their progress toward earning credits. Providers believe that making credit accumulation concrete for students makes them substantially different from neighborhood high schools where students only find out at the end of the year how many credits they actually earned. As one provider explained, “We set goals with each student, and we go over credits once or twice a month together. Being able to check on your credit accumulation throughout the year is a big difference from the sending schools where you can’t get your credits until June. At [our school], you can get your credits in one or two months. This allows for short or achievable goals.” Along similar lines, another provider reported that they drill a mantra of 3.5 credits per semester, arguing that “more and more information, and simplifying and clarifying [credit accumulation]” helps with keeping students on track. Strategy 3: Tracking Credits One provider argued that to keep students motivated it was important to break course credits down into subparts. By doing so, students could easily see that they were making progress toward the goal of earning a full course credit. This provider explained: "In order to earn a credit, you need to earn a certain number of apples. There might be a good 4 or 5 apples that are taught to students, and the students are taking tests and that way they get the apples from the teacher. Or, in English class they have to do a resume or a research paper in order to earn maybe two or three apples in that class." Strategy4: Fast tracking Students Close to Aging Out Another strategy to ensure credit accumulation is to provide extra supports for students who are approaching the age threshold for being enrolled in an accelerated high school – 22 years old. For example, one provider reported that, “if there are those that are about to reach the age threshold, we will fast-track them into the computer assistance program and we have on occasion provided students with internet service and laptops to take home. We let them do a lot of the coursework on line at night….” Summary The findings presented in this section confirm that accelerated high school providers are spurring credit accumulation for a sample of their overall student population. On average, the match sample of accelerated students outperformed similar students not enrolled in such schools, and also accumulated credits at a rate that exceeds the district’s average. Although no provider used every one of the credit accumulation strategies listed above, the rate of credit accumulation is something that providers are constantly striving to improve, and all used one or more strategies to help students achieve the goal of accumulating credits at an accelerated pace. The findings presented here point to the following questions for further consideration: 1. What types of credits (core vs. elective) do students accumulate prior to entering accelerated high schools, and how does this impact their credit accumulation patterns at accelerated high schools? 2. How does the timing of data collection related to credit accumulation impact the accuracy of the patterns seen here? 3. How do providers vary in their ability to support accelerated credit accumulation across the range of core subject areas? 27 Long-Term Outcome: Graduation Graduation is the most important measure of success for accelerated high schools. The 2010 Mathematica study of Philadelphia’s accelerated high schools found that a sample of accelerated students were graduating more often than a matched comparison group of neighborhood high school students, both in five and six years. This study extends the findings of the 2010 study by assessing differences in the percentage of students graduating in four, five, or six years. In this section, we first provide a comparative analysis of graduation rates between the matched accelerated students and the comparison students. We then look more closely at how significant factors influenced accelerated students’ likelihood of five year graduation across providers. Overall, accelerated high schools continue to be generating positive results. Students at high risk for dropping out in Philadelphia continue to graduate more often in accelerated high schools than in District-run neighborhood schools. In addition, there are a number of key factors that significantly influence accelerated students’ likelihood of graduation: their provider, gender, credits earned prior to entering an accelerated high school, attendance at accelerated high schools, age they exit the District, and overall time they spend in their accelerated high school (retention). Comparative Graduation Analyses: Accelerated Students v. Comparison Students Overall Graduation: Accelerated Students v. Comparison Students The overall sample of accelerated and comparison students in this analysis is restricted to those who started as first time 9th graders between 2003-04 and 2007-08 so that students could plausibly be counted as four, five, or six year graduates. For this analysis, graduation status was considered cumulative; that is, a student who graduated in four years is also considered a five and a six year graduate as well. For this reason, Figure 19, below, shows a gradual increase in the percentage of students graduating in 4, 5, and 6 years, for both the Accelerated and Comparison students. Figure 19. Student Graduation: Accelerated Students v. Comparison Students 28 Overall, a significantly greater percentage of Accelerated Students graduated in four, five and six years than comparison students. 20 • • • Four Years: Five Years: Six Years: 33% v. 16% (p<.001) 46% v. 20% (p<.001) 52% v. 21% (p<.001) Overall Graduation: Accelerated Students v. Comparison Students v. District Students While these findings are encouraging, the percentage of accelerated students graduating in four, five, and six years remains below the District’s overall graduation rates. However, with each successive year, accelerated students narrow the distance between themselves and the District overall. That is, substantial numbers of accelerated high school students continue to graduate even after four years, whereas both the comparison students and students in the District overall do not continue towards graduation in a similar way. Five Year Graduation Analyses: Matched Accelerated Students, by Provider Having established that our matched accelerated students were significantly more likely to graduate in five years than the comparison group, a logistic regression model was developed using only the matched sample of accelerated students from the five year graduation analysis. There are two key reasons this analysis was limited to five year graduation. First, since accelerated students are, by definition, behind in school it did not make conceptual sense to assess their likelihood of four year graduation. Second, an analysis of five year graduation provides a larger sample of students to include since only three cohorts could justifiably be included in a six-year graduation analysis. The findings presented below show how the following significant factors independently influenced these students likelihood of graduation, controlling for the influence of every other factor: • • • • • • Their provider; Gender; Credits earned prior to entering an accelerated high school; Attendance at accelerated high schools, The age they exit the District; Retention. Notably, restricting our analysis to five year graduation rates allows us to examine only three providers: Camelot, OIC, and One Bright Ray. Due to the fact that these are the longest operating providers in the District, these are the only providers with enough students eligible for the five-year graduation analysis. Overall, the combined effect of these significant factors in our model accounted for roughly 29% of the variation in five year graduation in this population, indicating that our predictive model is very strong by educational research standards. 21 Table 8 below provides the average levels of each of the indicators that we examine for Camelot, OIC, and One Bright Ray. Each of these indicators was a statistically significant predictor of accelerated high school students’ likelihood of five year graduation. Statistical significance was assessed using both a simple chi-square test, in addition to logistic regression models, which controlled for students: cohort, gender, age, and average daily attendance. For full details of model results please see the Technical Appendix. 21 For full model results see Appendix. 20 29 Table 8. Significant Predictors of Accelerated Students’ Five Year Graduation Camelot OIC One Bright Ray Overall 65% 38% 38% 52% 6.5 6.3 5.6 6.3 0.82 0.79 0.92 0.84 Average Age at Exit from the District 18.9 19.5 19.0 19.1 Average Months Enrolled in Accelerated High School 21.9 19.7 19.8 20.5 Total Students 445 157 356 1030 Indicator Percent Boys Average Credits Earned Prior to Accelerated High School Entry Average Daily Attendance in Accelerated high school * Totals do not add up to the overall. Seventy-two students who attended schools operated by “other providers” are not represented in this table, but these students are included in the logistic regression model. See Appendix for a full description of the modeling procedures. As Table 8 illustrates, on average, Camelot students in this sample are disproportionately male, and they enter with more credits, stay in their accelerated high schools longer, and are slightly younger when they exit from the District. At the other end of the spectrum, students at One Bright Ray on average, enter with the fewest credits, but have the highest attendance rate. One Bright Ray and the OIC had equivalent male populations, while OIC students had, on average, the lowest attendance and were the oldest students to exit the District. Each of the figures to follow presents how the predicted probability that a student will graduate within five years changes as each one of the indicators in Table 8 change, controlling for the effect of each of the other factors. 22 A “predicted probability” is best understood as the estimated chance that a student will graduate in five years. Provider Effects on Five-Year Graduation Figure 20 shows the predicted probability that students will graduate in five years from schools operated by each provider in this analysis – Camelot, OIC, and One Bright Ray – controlling for the effects of cohort, gender, entry credits, age, and overall time in an accelerated high school. The horizontal blue line in Figure 20 shows the overall predicted probability that a student in the total sample would graduate in five years. On average, these students had 41% chance of graduating in five years, while Camelot, OIC, and One Bright Ray students had a 62%, 38%, and 23% chance of five year graduation respectively. Predicted probabilities for five year graduation were calculated based on the logistic regression model results. See Appendix for full review of the logistic regression modeling and full results. 22 30 Figure 20. Provider Effects on Five-Year Graduation As Figure 20 shows, Camelot students were significantly more likely to graduate in five years than students at both OIC and One Bright Ray. OIC students were also significantly more likely to graduate in five years than the One Bright Ray students. Each of the figures that follow explore how variations in student characteristics across providers influenced their likelihood of five year graduation. These analyses provide important context for understanding the stark disparity observed in Figure 20, while also raising questions for future inquiry. Gender Effects on Five-Year Graduation The only demographic factor that significantly influenced students’ likelihood of five year graduation was their gender. Mirroring a common trend across the District, women at accelerated high schools were significantly more likely to graduate in five years than men, with a predicted probability of five year graduation of 48%, compared to 33% for men (Figure 21). 31 Figure 21. Gender Effects on Five-Year Graduation The disparity between men and women seen in Figure 21 makes the performance of Camelot students all the more remarkable, considering that Camelot schools served a population that was roughly two thirds male, while the OIC and One Bright Ray schools each served student populations with fewer than 40% men (see Table 8). Prior Credit Accumulation Effects on Five-Year Graduation The amount of credits students earned prior to entry into an accelerated high school was also a significant predictor of the likelihood of graduation. Figure 22 illustrates how the likelihood of five year graduation steadily increases with the number of credits accumulated prior to arrival. 23 Each of the three vertical lines in Figure 22, and each of the figures to follow in this section, represents the average for each provider: Camelot, OIC, and One Bright Ray, along each measure. In Figure 22, the vertical lines refer to the average number of credits students earned prior to their entry into Camelot, OIC, or One Bright Ray schools. 23 It should be noted that students are supposed to enter accelerated high schools with no more than 8 credits. While, on average, accelerated students did arrive with fewer than 8 credits, roughly 30% of students in this sample had accumulated 8 or more credits prior to their transfer into an accelerated high school. In addition, students also come with varying types of credits. The data requested for this analysis did not include the types of student credits students earned, but on average, this sample of students, and our overall population of accelerated students, arrived in accelerated high schools with less than a year of academic progress; i.e. fewer than 8 credits. 32 Figure 22. Prior Credit Accumulation Effects on Five Year Graduation Overall, students in the sample entered accelerated high schools with an average of 6.3 credits, but a plurality of students, just over 17% of all students in this sample, entered their accelerated high school with zero credits. “Average students” entering with 6.3 credits had an estimated 41% chance of five year graduation—more than twice that of students entering with no credits, who had only an estimated 18% chance of graduation. Across providers, Camelot students entered with the most credits, on average 6.5 credits, roughly one credit more than One Bright Ray students who entered with 5.6 credits. OIC students fell between three two other providers with 6.3 credits, on average. Attendance Effects on Five-Year Graduation Attendance is consistently one of the most reliable predictors of student graduation. Unsurprisingly, students’ average daily attendance in their accelerated high schools exerted a significantly positive effect on students’ likelihood of graduation in five years. Figure 23 shows that as students’ average daily attendance increases, their likelihood of five year graduation improves dramatically, particularly beyond 80%. Again, the three vertical lines refer to the average daily attendance at each of the three providers’ schools: Camelot, OIC, and One Bright Ray. 33 Figure 23. Average Daily Attendance Effects on Five Year Graduation One curious finding on Figure 23 is the fact that One Bright Ray students had, on average, the highest average daily attendance, while at the same time having the overall, lowest likelihood of five-year graduation. This apparent contradiction is an instructive reminder that the predictive modeling results presented here do not account for all the factors that contribute to students’ likelihood of graduation in accelerated high schools. Rather, these results point to trends across accelerated high schools and are useful to highlight these types of apparent contradictions as key areas for future inquiry. Age Effects on Five-Year Graduation As students get older their estimated chance of graduation in 5 years steadily increases to a peak near 19 years of age. Age 19 emerged as a “tipping point” after which graduation in five years becomes significantly less likely for all students (Figure 24). The three vertical lines in Figure 24 show the average age when students exit from each provider’s schools: Camelot, OIC, and One Bright Ray. 34 Figure 24. Age Effects on Five Year Graduation Across providers, Camelot students were, on average, the youngest when they exited the District. OIC students were, on average, the oldest when they exited District, while One Bright Ray students fell between these two. This finding is particularly striking since, on average, accelerated high school students enrolled in 9th grade at roughly age 15. For these students, we would expect ‘five year graduation’ to ‘tip’ closer to 20, yet by age 19 these students become less likely to graduate in five years. Retention Effects on Five-Year Graduation Accelerated high schools seek to move students towards graduation within three years of their enrollment. As Figure 25 shows, as students approach three full years in an accelerated high school their likelihood of five year graduation steadily increases. However, after three years, their predicted probability of five year graduation begins to level off and then slightly decline. The three vertical lines in Figure 10 represent the average number of months students remain in schools operated by each provider: Camelot, OIC, and One Bright Ray. Figure 25. Retention in Accelerated high schools Effects on Five Year Graduation 35 As in Figure 25 shows, the average number of months students stay in accelerated high schools ranges only between 19.7 for OIC to a high of 21.8 for Camelot students, suggesting there is still considerable room, among all providers, to improve students’ prospects of graduation if they can keep students enrolled. Summary A greater percentage of the sampled accelerated students graduated in four, five and six years than comparable neighborhood high school students. Given the challenges facing accelerated students, the fact that any accelerated students graduate in four years is remarkable, and it is also encouraging that these students are more likely than the comparison group students to graduate in five and six years after they have fallen behind the traditional four-year trajectory. In addition, our analyses indicate that there are a number of factors that influence students likelihood of graduating from these schools, both factors they “bring with them” to the accelerated high school (gender, age, and prior credits earned), and factors related to their time in accelerated high schools (retention and attendance). Conclusion The research presented in this Year One report provides promising evidence of strong performance among the accelerated high schools. As a group, these schools are out-performing neighborhood high schools in graduating over-age, under-credited students, and spurring credit accumulation among student who otherwise might be student dropouts. However, while 52% of accelerated high school students graduated in six years, this is still well below the overall District average for the cohorts included in our analyses. In addition, this report has also revealed considerable variation among providers in the degree to which they are able to retain students, support accelerated credit accumulation, and graduate their students. The findings presented in this report point to a number of possible implications for program improvement and for future research. Programmatic Implications The findings presented in this Year one report point to the following programmatic implications for accelerated high schools and providers: 36 4. Maintain focus on retention and attendance. The longer students remain enrolled in accelerated high schools, and the more frequently they attend, the more likely they will be to graduate from these schools. 5. ‘Fast tracking’ for accelerated high school students should begin well before students reach age 19. After age 19 students become significantly less likely to graduate in five years; finding ways to ‘fast track’ students well before they reach this critical age may go a long way to getting more students to graduation. 6. Increase focus on male students. Additional strategies and/or curriculum designed to meet the unique needs of male students may help to address the gap between male and female graduation observed in our analyses. Future Research in Year Two The overarching goals of the research were intended to assist the SDP, accelerated high school providers, school leaders, and Project U-Turn understand the contexts and conditions that contribute to accelerated schools’ success in graduating students ready for post-secondary opportunities. The first year of this two-year research project has focused on the overall performance of the accelerated high schools and the variation across providers, in terms of their enrollment, retention, credit accumulation and graduation. This report has also begun to identify the range instructional approaches used in accelerated high schools, and the strategies they employ to retain students and drive credit accumulation. Moving forward, additional insight is needed into the school, provider, and District contexts that create the conditions for an accelerated high school to be successful. In Year two, we will conduct school case studies of accelerated high schools having success in order to understand the school, provider and district conditions and contexts that contribute to their success. Based on year one findings, and in collaboration with the SDP Multiple Pathways staff and PYN staff, RFA will select three accelerated high schools for case study; these schools would have been successful in two or more key areas: retention, attendance, credit accumulation, and graduation. Intensive examination of these schools will yield crucial insights into the particular circumstances for success in individual schools; a cross case study analysis will then identify the core areas across all accelerated high schools that are necessary for success. In Year Two the quantitative analyses will be replicated and expanded to include an additional year of student and school level factors that may influence student enrollment, retention, credit accumulation and graduation. Relevant school level factors could include the instructional model employed by providers, the student – teacher ratio in the school, average class sizes, and teacher experience in the District. Additional student-level factors that may be included in the Year Two analyses include measures of student mobility, both across schools, and living conditions, along with a selection of neighborhood indicators that may provide insight into the way students’ out-of-school living environments influence their in-school performance. The Year Two analyses will also include: 1. A more detailed analysis of credit accumulation will look more closely at the type of credits students enter with and accumulate while in accelerated high school, along with an analysis of the student and school level factors that influence credit accumulation; this analysis will provide additional insight into the variation in credit accumulation observed in this report. 2. An analysis of accelerated high school students’ post-secondary enrollment, retention, and completion, drawing on records from the National Student Clearinghouse, With the benefit of findings from the case study analysis and an additional year of performance outcomes, the Year Two report will provide a robust picture of accelerated high schools’ enrollment, retention, credit accumulation and graduation both in the aggregate and by provider. It will also identify provider, school and district policies and practices that contribute to high achieving accelerated high schools. Understanding how 37 the most effective accelerated high schools are producing positive outcomes for their students will provide key insights for other providers, District officials, and the public to better understand the most effective means of supporting students at high risk of dropping out through their high school careers and beyond. 38 Technical Appendix One of the primary objectives of this Year One report was to reproduce and extend the findings of a 2010 study of Philadelphia’s accelerated high schools that found accelerated students to accumulate more credits, and to graduate more often, than comparable neighborhood high school students (Chaing and Gill, 2010). Given that students attending accelerated high schools are, by program design, over-age and under-credited, an overall comparison between accelerated high school students and District high school students is unwarranted. A key element of both studies was the selection of the sample of accelerated school students and a viable comparison group for the analyses of credit accumulation and graduation. It should be noted that the sample of accelerated high school students used for the comparative analyses presented in this report represent those students who remain enrolled at accelerated high schools for at least a full academic year. Provider Instructional Approaches The rest of this technical appendix provides an overview of the quantitative methods employed to select the study and comparison populations, along with a review of the analyses performed to generate the graduation findings presented in the full report. The technical appendix is divided into two main sections: 1. Student matching methodology; 2. Predictive modeling results for graduation analyses; 3. An overview of the curricular models used by providers. Student Matching Methodology Student Population For year one of the study, the School District of Philadelphia provided records for 259,067 individual students in grades 8 – 12 from 2002-03 to 2010-11. The 2002-03 school year served as a baseline from which to sort students into 8 separate cohorts, according to the year of students’ first-time ninth grade attendance in the School District of Philadelphia. For instance, Cohort 1 students were first time 9th graders in 2003-04, Cohort 2 students were first time 9th graders in 2004-05, and so on. This year one report assessed differences between accelerated high school students and comparison students along two key outcomes: credit accumulation and graduation. Graduation was assessed for four, five and six year graduation, and graduation was considered cumulative for this study. That is, a student who graduates in four years, also graduated in five year, and six years. Since students could only logically graduate in four, five or six years on the basis of when they entered the District as first time 9th graders, cohort membership was the initial selection criteria for inclusion in both the credit accumulation and graduation analyses. Table A1 displays which cohorts were eligible for each analysis. Table A1. Cohort Eligibility for Outcomes Analyses 6 Year Graduation 5 Year Graduation 4 Year Graduation Credit Accumulation (09-10 & 10-11) Cohort 1 - 3 Cohort 1 - 4 Cohort 1 - 5 Cohort 1 – 7 Selection Criteria for Inclusion in the Accelerated Student Group To be included in the matched sample of accelerated students for both the graduation and credit accumulation analyses, students had to meet the following four criteria: 39 1. Students attended another SDP school for at least one full year prior to entry into the accelerated high school. Cohort 8 students, who were first time 9th graders in 2010-11, do not meet the first criteria and are therefore not included in any of the matched comparison analyses. 2. The last school a student attended prior to transfer to an accelerated high school was either a Neighborhood or Magnet School (not a Charter or other Alternative School). Attending a District school for at least one year prior to entry into an accelerated high school provides the necessary student enrollment, academic and behavioral data to match with comparison group students. Charters and Alternative Schools do not report a number of the key matching variables (i.e., credit accumulation, GPA, suspensions), making it very challenging to match students who attended these schools prior to entering an accelerated high school. 3. Students attended an accelerated high school for at least 10 months. A threshold of 10 months, equivalent to a full academic year, was established based on the assumption that students would need at least that much time in an accelerated school setting to benefit from the accelerated curriculum. a. The 10 month threshold marks a significant deviation from the selection criteria imposed by the 2010 study. In the 2010 study, the accelerated high school student population included in the analyses was limited to students who could have been enrolled in an accelerated high school for up to three years prior to their assessed graduation status, based on the assumption that accelerated high schools need up to three years to get their students caught up and graduated. By relaxing this restriction, the present study expands the population of students to include students with fewer than three years in an accelerated high school. 4. Students did not attend an accelerated high school managed by Community Education Partners (CEP). The accelerated high schools managed by CEP, which were closed after the 2009-10 school year, consistently underperformed vis-a-vis other accelerated high schools. The 2010 study found students at these schools to earn, on average, significantly fewer credits, and to be less likely to graduate, than students attending accelerated high schools operated by other providers. Since CEP is no longer operating accelerated high schools in the District, students attending CEP schools were excluded from the credit accumulation and graduation analyses for year one. On the basis of these selection criteria, the number of accelerated high school students included in each analysis are listed in Table A2. Table A2. Accelerated Student Populations for Matched Comparison Analyses Total Accelerated Students 6 Year Graduation 5 Year Graduation 4 Year Graduation Credit Accumulation 2009-10 Credit Accumulation 2010-11 677 1,030 1,434 887 747 Comparison Group Selection For each analysis, a separate comparison group of students was selected from the overall population of neighborhood high school students using a propensity score matching procedure. The matching criteria employed for each analysis matched accelerated high school students to comparison students based on cumulative records from the end of the final year prior to an accelerated high school students’ entry into an accelerated high school. 40 The propensity score matching relied on a logistic regression model to estimate the likelihood that neighborhood high school students could be accelerated high school students. Accelerated high school students were matched using a ‘nearest neighbor’ matching procedure that identified non-treatment group students with the nearest predicted probability of being an accelerated high school student. Matching was performed, without replacement, within truncated samples corresponding to each ‘baseline period’ for accelerated high school students and neighborhood high school students. The resulting matched comparison groups within each baseline period were then appended together to create the full matched study population for each analysis. Even though neighborhood students were matched without replacement, it was possible for a single neighborhood student to be matched to more than one accelerated high school student since matching took place at the completion of each student-year. That is, a neighborhood high school student could have been selected as a comparison student at the end of their first year, the end of their second year, or the end of their third year, and so on without violating the no-replacement restriction imposed in the propensity score matching procedure. For this reason, the accelerated student population is slightly larger than the comparison group of students for each analysis, as seen in Tables A5 through A9. Accelerated high school students were matched to comparison students on the basis of the following indicators: Table A3. Comparison Group Matching Indicators • • • • • • • • Cohort Baseline Period 24 Student age at 9th grade entry Gender Ethnicity Limited English Proficiency Status Free/Reduced Lunch Status Special Education Status • • • • • • • Total Days Enrolled Cumulative Average Daily Attendance Cumulative Percentage of days enrolled in Neighborhood, Magnet, or Alternative schools Cumulative Percentage of days suspended Standardized 8th Grade PSSA Math and Reading Scores Cumulative Credits Earned Cumulative GPA Dealing with Missing Data In the current study, decisions related to the treatment of missing student data for key matching and outcome indicators were made to match as many accelerated high school students with as many neighborhood high school students as possible. Unlike the 2010 study, this study did not employ any multiple imputation techniques to estimate either missing outcome or matching indicators. Table A4 provides an overview of the techniques employed to account for significant amounts of missing data in key outcome and matching indicators. Table A4. Adjustments for Key Missing Data Missing Indicator Graduation Status Adjustment Strategy Students who transfer out of the School District of Philadelphia prior to graduation do not have a graduation status in the District records. These students were dropped from the graduation analyses. The ‘Baseline Period’ refers to the number of years between the time an accelerated school student started 9th grade and when they enrolled in an accelerated school. For comparison group students this indicator is simply a count of their years enrolled since 9th grade entry. 24 41 8th Grade PSSA Scores All students 8th grade PSSA math and reading scores were converted to Zscores for the matching process. Students with missing PSSA math and reading scores were assigned the average PSSA math and reading score and were flagged as having missing PSSA math and reading scores. Average Daily Attendance (ADA) Students with missing ADA were assigned the average cumulative ADA for all 9-12th grade students and were flagged as having missing ADA. GPA Students with missing GPA were assigned the average GPA for all 9-12th grade students for a particular year and were flagged as having missing GPAs. Credits Earned Students with missing credits were assigned the average credits earned by 9-12th graders for a particular year and were flagged as having missing credits. Students with missing credits were excluded from both the 2009-10 and 2010-11 credit accumulation analyses. Equivalence of Matched Comparison Groups For each outcomes analysis, credit accumulation in 2009-10 and 2010-11 and four, five, and six year graduation, a separate matching procedure generated the relevant comparison group for the accelerated high school student population. Tables A5 through A9 present the equivalence of the matched accelerated high school students and the comparison group of students for each analysis. None of the differences between the matching indicators for these groups were significantly, or substantively, different from zero. However, the differences between the accelerated high school students and the matched comparison students along the key outcome for each analysis were significant. By virtue of the matching techniques employed, the simple comparisons between accelerated high school students and the comparison students effectively control for all the matching indicators listed in Tables A5 through A9. As Tables A5 through A9, while accelerated high school students and the matched comparison students were effectively equivalent student populations at the point when matching took place, the accelerated high school students performed significantly better in terms of their four, five and six year graduation, in addition to earning significantly more credits in 2009-10 and 201011. Tests of statistical significance between matching indicators and outcomes were performed using simple ttests and chi-square tests of significance. 42 Table A5. Matched Accelerated & Comparison Groups: Four Year Graduation Analysis Accelerated Students Comparison Students Difference (Accelerated – Comparison) % Women 48 48 0 % Black 59 60 -1 % Asian 2 1 1 % Latino 28 29 -1 % White 10 10 0 % Other Race 0 0 0 % Limited English Proficiency 3 3 0 % Free/Reduced Lunch Status 53 53 0 % Special Education 16 16 0 % Cohort 1 15 17 -2 % Cohort 2 13 13 0 % Cohort 3 19 20 -1 % Cohort 4 25 23 2 % Cohort 5 28 27 1 Avg Baseline Period 3.38 3.17 0.21 Avg Total Credits Earned 6.87 6.55 0.32 Avg Cumulative GPA 0.80 0.78 0.02 Avg 8th Grade PSSA Math Z Score -.35 -.37 0.02 Avg 8th Grade PSSA Reading Z Score -.33 -.31 -0.02 Avg Total Days Enrolled 147 142 5 0.69 0.67 .02 90 91 -1 % of days at Alternative High Schools 1 1 0 % of days at Magnet High Schools 6 5 1 % of days Suspended 1 1 0 14.91 14.91 0 33 16 17 1,434 1,362 Matching Indicators Avg Cumulative Average Daily Attendance Rate % of days at Neighborhood High Schools Avg Age at 9th Grade Enrollment % of 4 Year Graduates*** Total Students ***p<.001 43 Table A6. Matched Accelerated & Comparison Groups: Five Year Graduate Analysis Accelerated Students Comparison Students Difference (Accelerated – Comparison) % Women 48 47 1 % Black 62 60 2 % Asian 1 1 0 % Latino 27 29 -2 % White 9 9 0 % Other Race 0 0 0 % Limited English Proficiency 2 3 -1 % Free/Reduced Lunch Status 50 51 -1 % Special Education 15 15 0 % Cohort 1 20 23 -3 % Cohort 2 19 18 1 % Cohort 3 27 28 -1 % Cohort 4 34 32 2 Avg Baseline Period 3.45 3.37 0.08 Avg Total Credits Earned 6.63 6.51 0.12 Avg Cumulative GPA 0.76 0.73 0.03 Avg 8th Grade PSSA Math Z Score -.34 -.35 0.01 Avg 8th Grade PSSA Reading Z Score -.31 -.34 0.03 Avg Total Days Enrolled 145 142 3 .68 .66 .02 91 92 -1 % of days at Alternative High Schools 1 0 1 % of days at Magnet High Schools 5 4 1 % of days Suspended 1 1 0 14.90 14.93 -.03 46 20 26 1,030 974 Matching Indicators Avg Cumulative Average Daily Attendance Rate % of days at Neighborhood High Schools Avg Age at 9th Grade Enrollment % of 5 Year Graduates*** Total Students ***p<.001 44 Table A7. Matched Accelerated & Comparison Groups: Six Year Graduate Analysis Accelerated Students Comparison Students Difference (Accelerated – Comparison) % Women 48 49 -1 % Black 59 59 0 % Asian 1 1 0 % Latino 30 29 1 % White 9 11 -2 % Other Race 0 0 0 % Limited English Proficiency 3 3 0 % Free/Reduced Lunch Status 46 49 -3 % Special Education 16 16 0 % Cohort 1 31 31 0 % Cohort 2 28 29 -1 % Cohort 3 41 41 0 Avg Baseline Period 3.37 3.14 0.23 Avg Total Credits Earned 6.16 5.90 0.26 Avg Cumulative GPA 0.70 0.71 -0.01 Avg 8th Grade PSSA Math Z Score -.33 -.35 0.02 Avg 8th Grade PSSA Reading Z Score -.32 -.31 -0.01 Avg Total Days Enrolled 147 143 4 0.58 0.66 -0.08 91 90 1 % of days at Alternative High Schools 0 1 -1 % of days at Magnet High Schools 5 6 -1 % Days Suspended 1 1 0 14.90 14.95 -.05 % of 6 Year Graduates*** 52 21 31 Total Students 677 650 Matching Indicators Avg Cumulative Average Daily Attendance Rate % of days at Neighborhood High Schools Avg Age at 9th Grade Enrollment ***p<.001 45 Table A8. Matched Accelerated & Comparison Groups: 2009-10 Credit Accumulation Analysis Accelerated Students Comparison Students Difference (Accelerated – Comparison) % Women 49 48 1 % Black 61 61 0 % Asian 1 1 0 % Latino 27 25 2 % White 11 13 -2 % Other Race 0 0 0 % Limited English Proficiency 3 3 0 % Free/Reduced Lunch Status 57 56 1 % Special Education 18 22 -4 % Cohort 1 1 1 0 % Cohort 2 2 2 0 % Cohort 3 13 13 0 % Cohort 4 30 30 0 % Cohort 5 35 33 2 % Cohort 6 19 21 -2 Avg Baseline Period 3.21 2.91 0.30 Avg Total Credits Earned 6.46 6.45 0.01 Avg Cumulative GPA 0.84 0.88 -0.04 Avg 8th Grade PSSA Math Z Score -.36 -.40 0.04 Avg 8th Grade PSSA Reading Z Score -.35 -.42 0.07 Avg Total Days Enrolled 149 147 2 .70 .69 .01 90 92 -2 % of days at Alternative High Schools 1 1 0 % of days at Magnet High Schools 6 4 2 % of days Suspended 1 1 0 Avg Age at 9th Grade Enrollment 14.91 14.94 -.03 Total Credits Earned: 2009-10*** 8.59 3.48 5.11 Total Students 887 813 Matching Indicators Avg Cumulative Average Daily Attendance Rate % of days at Neighborhood High Schools ***p<.001 Table A9. Matched Accelerated & Comparison Groups: 2010-11 Credit Accumulation Analysis 46 Accelerated Students Comparison Students Difference (Accelerated – Comparison) % Women 49 50 -1 % Black 61 61 0 % Asian 2 3 -1 % Latino 27 26 1 % White 10 10 0 % Other Race 0 0 0 % Limited English Proficiency 3 3 0 % Free/Reduced Lunch Status 59 62 -3 % Special Education 17 17 0 % Cohort 1 0 0 0 % Cohort 2 1 0 1 % Cohort 3 5 4 1 % Cohort 4 22 23 -1 % Cohort 5 40 39 1 % Cohort 6 28 31 -3 % Cohort 7 4 3 1 Avg Baseline Period 3.26 3.02 0.24 Avg Total Credits Earned 6.85 6.93 -0.08 Avg Cumulative GPA 0.86 0.92 -0.06 Avg 8th Grade PSSA Math Z Score -.40 -.41 0.01 Avg 8th Grade PSSA Reading Z Score -.35 -.36 0.01 Avg Total Days Enrolled 147 147 0 Avg Cumulative Average Daily Attendance Rate .69 .69 0 %of days at Neighborhood High Schools 89 91 -2 % of days at Alternative High Schools 2 1 1 % of days at Magnet High Schools 7 5 2 % of days Suspended 1 1 0 Avg Age at 9th Grade Enrollment 14.91 14.92 -.01 Total Credits Earned: 2010-11*** 7.34 3.67 3.67 Total Students 747 698 Matching Indicators ***p<.001 47 Comparative Graduation Analysis: Accelerated Students v. Comparison Students To estimate the likelihood that students will graduate in 4, 5, or 6 years, three separate logistic regression models were used to assess the impact of being an accelerated school student on the likelihood of graduation. Descriptive measures of the indicators included in each model are presented in Table A10. Table A10. Descriptive Measures of Indicators in Graduation Models Four Year Graduation Categorical Indicators Five Year Graduation Six Year Graduation N Percentage N Percentage N Percentage Accelerated Students 1434 51 1030 51 677 51 Women 1351 48 954 48 645 49 Cohort 1 440 16 431 22 410 31 Cohort 2 369 13 364 18 378 28 Cohort 3 549 2 547 27 539 41 Cohort 4 669 24 662 33 - - Cohort 5 769 28 - - - - Interval Indicators Cumulative Average Daily Attendance Mean SD Mean SD Mean SD 0.73 0.26 0.73 0.29 0.75 0.16 Age at 9th Grade Entry 14.79 0.79 14.81 0.74 14.8 0.94 Age at SDP Exit 18.64 1.08 18.84 1.13 18.72 1.3 The results of each graduation model are presented in Table A10. In each model, the key independent variable, membership in the accelerated school group, is a highly significant, positive predictor of a student’s likelihood of graduation in four, five and six years; and these models accounted for roughly 29%, 23%, and 20% of the variation observed in student graduation in the study population for each analysis: four, five and six year graduation respectively. The coefficients in each model in Table A11 show that: • • • • • • Membership in the accelerated student group was a significant, positive, predictor of graduation in four, five and six years; Students in cohorts 2-5 were all more likely to graduate than students in cohort 1 (the excluded comparison group); Students average daily attendance was significant, positive predictor of student graduation; Women were significantly more likely to graduate than men; Students age at 9th grade entry was a significant positive predictor of four year graduation, but it was a significant negative predictor of five and six year graduation; Students’ age at the time they exit the District had a curvilinear relationship to students’ likelihood of graduation in four or five years. Up to a point, increasing age is a positive predictor of students’ likelihood of graduation, and then increasing age becomes a negative predictor of four and five year graduation. This relationship is captured in the interaction term in the four and five year graduation models. Students’ age at the time they exit the District was still a significant and positive predictor of six year graduation, but the strength of this relationship receded in the six-year model, and the interaction term was no longer significant in the six year graduation model. 48 Table A11. Graduation Models: Accelerated Students v. Comparison Group Students Four Year Graduation Model Five Year Graduation Model Six Year Graduation Model B SE B SE B SE Accelerated Student 0.636*** 0.114 0.669*** 0.116 0.946*** 0.136 Cohort 2^ 0.998*** 0.227 0.578** 0.184 0.706*** 0.174 Cohort 3^ 1.107*** 0.215 0.347* 0.167 0.795*** 0.159 Cohort 4^ 0.887*** 0.207 0.460** 0.161 - - Cohort 5^ Cumulative Average Daily Attendance 1.646*** 0.198 - - - - 8.714*** 0.499 6.481*** 0.493 5.711*** 0.547 Gender# -0.402*** 0.106 -0.461*** 0.112 -0.507*** 0.132 Age at 9th Grade Entry 0.541*** 0.082 -0.196* 0.083 -0.259** 0.081 Age at SDP Exit 29.948*** 3.104 22.665*** 2.337 0.362*** 0.060 Exit Age Interaction Term Constant Log Likelihood -0.813*** 0.083 -0.594*** 0.061 - - -292.133*** 29.014 -218.907*** 22.316 -8.592*** 1.330 -1106.303 -985.446 -700.630 Chi Square 909.81 584.4 346.55 R Square 0.2914 0.2287 0.1983 N 2,796 2,004 1,327 ** p<.01; *** p<.001 ^Cohort 1 is comparison group #Female is comparison group To develop a more refined sense of the independent effect of accelerated school attendance, predicted probabilities were calculated for accelerated and comparison students’ chances of four, five or six graduation, controlling for the effects of each of the indicators in the models. Figure A12 presents the differences between accelerated students and the comparison students’ predicted probability of graduation in four, five and six years. As Figure A12 shows, accelerated high school students have a consistently greater likelihood of graduation than the comparison group of students. However, overall the predicted probabilities for both sets of students, accelerated and comparison, only improves slightly from four to six years, and remains below a ‘fifty-fifty’ chance of six year graduation for accelerated high school students. 49 Figure A12. Predicted Probabilities of Graduation: Accelerated Students v. Comparison Students Five Year Graduation Analysis: Matched Accelerated Students Only A second set of analyses was restricted to the five year graduation population of accelerated high school students. By restricting this analysis to only students attending accelerated high schools, a logistic regression model could control for factors specific to accelerated high school students. Descriptive measures of the indicators included in the accelerated student model are presented in Table A13. Table A13. Descriptive Measures of Indicators in Accelerated High School Student Five-Year Graduation Model Categorical Variables N Percentage Women 496 48 Cohort 1 211 21 Cohort 2 191 19 Cohort 3 275 27 Cohort 4 353 34 Camelot 445 43 OIC 157 15 One Bright Ray 356 35 Other Provider 72 7 Mean SD Credits Earned Prior to Accelerated School Entry Cumulative Average Daily Attendance in Accelerated Schools 6.27 5.19 0.84 0.15 Total Months in Accelerated Schools 20.46 7.82 Age at SDP Exit 19.05 0.93 Interval Variables Table A14 presents the results of two models to show the significance of each predictor in the models. In each model the provider comparison group is changed to show comparisons across providers. The coefficients in Table A14 show that: 50 • • • • • • • • The number of credits students earn prior to entering an accelerated high school was a significant, positive, predictor of five year graduation; Student attendance at accelerated high schools was a significant, positive, predictor of five year graduation; that is, the more a student attended their accelerated high school, the more likely they would be to graduate in five years. The amount of time students stay enrolled in accelerated high schools had a curvelinear relationship to five year graduation. Up to about three years, students are more likely to graduate in five years, after this time their likelihood of five year graduation levels off and declines slightly. Students in cohorts 2-4 were significantly more likely to graduate in five years than students in cohort 1; Students’ age at the time they exit the District had a curvilinear relationship to students’ likelihood of graduation in five years. Up to about 19 years of age, increasing age is a positive predictor of students’ likelihood of graduation, and then increasing age becomes a negative predictor of five year graduation. This relationship is captured in the interaction term. Camelot students were more significantly more likely to graduate in five years than OIC and One Bright Ray students; OIC students were significantly more likely to graduate in five years than One Bright Ray students; Women were significantly more likely to graduate than men. A detailed depiction of the independent effects of each of predictors in Table A14 is included in the full report. Table A14. Model Predicting Relative Results for Five Year Graduation of Accelerated High School Students Model I (Results Relative to OIC) Model IA (Results Relative to One Bright Way) B SE B SE Credits Earned Prior to Accelerated School Entry 0.176*** 0.019 0.176*** 0.019 Cumulative Avg. Daily Attendance in Accelerated High School 6.116*** 0.783 6.116*** 0.783 Total Months in Accelerated Schools 0.182*** 0.050 0.182*** 0.050 Interaction Term: Total Months in Accelerated High School -0.002* 0.001 -0.002* 0.001 1.031*** 0.250 1.031*** 0.250 1.3190*** 0.241 1.3190*** 0.241 1.411*** 0.245 1.411*** 0.245 Age at SDP Exit 23.445*** 3.510 23.445*** 3.510 Interaction Term: Exit Age -0.617*** 0.092 -0.617*** 0.092 Camelot 0.975*** 0.246 1.685*** 0.209 - - 0.711** 0.258 One Bright Ray -0.711** 0.258 - - Other Provider -0.862* 0.423 -0.151*** 0.412 # -0.598*** 0.167 -0.598*** 0.167 -233.402*** 33.586 -233.402*** 33.586 Cohort 2 ^ Cohort 3^ Cohort 4 ^ OIC Gender Constant Log Likelihood -507.72443 -507.72444 Chi Square 406.82 406.82 R Square N 0.286 1,030 0.286 1,030 * p<.05; ** p<.01; *** p<.001 ^Cohort 1 is comparison group #Female is comparison group 51 Provider Overviews and Curricular Models Accelerated high school providers have the freedom to select the instructional model they believe will best support their student population toward rapid credit accumulation and graduation. Table A15, below, provides an overview of the instructional models operating across the eight providers who operated accelerated high schools in the School District of Philadelphia in the 2010-11 school year. For each provider, Table A15 presents the following information about providers’ curriculums: 1. Instructional Approach a. ‘Teacher-led’ v. a ‘Blended’ model of teacher-led and computer based instruction; b. ‘Full day’ v. ‘Half day’ program; 2. Standards Alignment a. Degree of alignment to Pennsylvania state standards; b. Degree of alignment to SDP standards; 3. Key Features of the Curriculum 4. The Role/Expectations of the Teacher 5. Postsecondary Focus Table A15. Accelerated High School Providers: Curricular Models Provider Big Picture Instructional Approach Standards Alignment Teacher-led; PA state standards Full day Key Features of the Curriculum Curriculum focuses on real-world learning through connections with the community via internships, and meaningful project work. Role/Expectations of the Teacher Teachers create direct assignments for students. They attend professional development on Wednesdays to support lesson planning. Internships are a centerpiece of the El Centro’s curriculum and help students "identify passions and interests and reach out to individuals in the community who share their interests and passions". Camelot Teacher-led; PA state standards; Full day SDP District standards Students intern two days a week and projects are based on student interest. Curriculum uses SDP core curriculum, including English, math, science, social studies, electives, languages, arts and humanities courses. Teachers are expected to attend professional development opportunities to support their lesson planning, offered by Jobs For the Future. PostSecondary Focus A Post-Secondary plan incorporates real-world experiences into the curriculum and provides support with financial aid forms. College readiness and postsecondary success is focus of the curriculum. Enhances SDP's core 52 curriculum with the Jobs for the Future Common Instructional Framework, which includes strategies for collaborative group work, questioning, writing to learn, classroom talk, literacy groups, and concept scaffolding. Communities in Schools (CIS) Teacher-led; PA state standards; Full Day SDP District standards Delaware Valley High School (DVHS) Blended; Opportunities Industrialization Center (OIC) Blended; SDP District standards Half day Full day SDP District standards Provides a vocational track for students that includes: entrepreneurship, culinary arts, and graphic design through a Dept. of Labor grant. Curriculum adopts the "most important pieces" of the SDP core curriculum, and creates lesson plans from those pieces. Curriculum uses SDP core curriculum with the inclusion of lessons from Standard Aligned System (SAS), a computer based program. The SAS program includes instructional videos and assessments designed to remediate, build skills, and prepare students for the PSSAs. Curriculum emphasizes computer based instruction, EXTRA Learning, for both remediation and advanced work. Teachers create differentiated instruction plans for students based on credit needs, rather than "grade." One of the "five pillars" for CIS is a ‘future focus’ which includes college and career transition planning. Teachers remediate students who are behind and support credit accumulation through targeted instruction before students start with the SAS program. N/A Teachers teach multi-levels in a single setting (classroom, computer lab). Teacher instruction is differentiated to the skill level of each student. The Career Intern Program focuses on preparing students for post-graduation opportunities. Two other programs focus on non-academic skills: 1) the Quantum Opportunities Program focuses on community engagement, and 2) the Advisory Program, cultivates character 53 Ombudsman Blended; SDP District standards Half day One Bright Ray Teacher-led; PA State standards Full day Phase 4 Blended; Half-day PA state standards education, academic and social skill building, and college and career planning. Curriculum is approximately 60% computer-based, and 40% teacher-led. Computer-based curriculums include: A Plus Curriculum, Reading Plus, and Lexia. Curriculum is focused on project-based learning. Projects are split up into eight week sections which include two four-week "storyboards," that visually represent the curriculum. Students collaborate to produce portfolios and presentations of their work. Curriculum uses the A Plus computer-based curriculum, and teachers lead small group instruction based on student needs. Teachers provide classroom instruction in addition to one-on-one sessions with students to assess how students should be grouped in classroom settings. N/A Teachers ‘plan backwards’ from the final project to determine which skills are needed, which assessments are appropriate, how to differentiate instruction, and what the lessons will look like over the course of student projects. Careers classes are offered for all students. Teachers are mostly supplemental to computerbased instruction. N/A The Peace4Kids program incorporates character education and life skills into the curriculum focusing on empathy, anger management, respect, perseverance, and social skills. 54
© Copyright 2026 Paperzz