TECHNICAL APPENDICES
Preparing Black and Latino Young Men
for College and Careers:
A Description of the Schools and Strategies in
NYC’s Expanded Success Initiative
Sarah Klevan
Adriana Villavicencio
Suzanne Wulach
November 2013
APPENDICES
Appendix A: ESI Documents
ESI Theory of Action….. ................................................................................................................................ 2
ESI Design Challenge Application and Rubric…. .......................................................................................... 3
Year 1 Planning Template (Work Plan) ......................................................................................................... 6
Appendix B: Schools Participating in ESI
List of ESI Schools......................................................................................................................................... 7
Appendix C: Coding
Detailed Coding Process ............................................................................................................................... 8
Full Codebook ............................................................................................................................................. 10
2
PREPARING BLACK AND LATINO YOUNG MEN FOR COLLEGE AND CAREERS
APPENDIX A: ESI DOCUMENTS
This appendix consists of documents from the Expanded Success Initiative that we used to develop our coding. The
ESI Theory of Action shows the three domains that structured our coding system. It also defines the initiative’s
intended implementation, which serve as the framework for our implementation evaluation.
The Design Challenge application and rubric, as well as the Year 1 planning template, are some the documents we
used to identify each school’s existing and proposed ESI strategies. See Chapter 4 of the full report or Appendix C for
more details.
ESI Theory of Action
Source: NYC Department of Education (nd). “ESI In Context.” Retrieved 10/14/13 from
http://schools.nyc.gov/Offices/ESI/About/ESI+in+Context.htm
3
ESI Design Challenge Application
4
PREPARING BLACK AND LATINO YOUNG MEN FOR COLLEGE AND CAREERS
5
Source: NYC Department of Education (2012). “ESI Design Challenge Application.” (Not published).
6
PREPARING BLACK AND LATINO YOUNG MEN FOR COLLEGE AND CAREERS
Year 1 Planning Template (Work Plan)
Source: NYC Department of Education (2012). “ESI Year 1 Planning Template.” (Not published).
7
APPENDIX B: SCHOOLS PARTICIPATING IN ESI
• Academy for Young Writers
• High School for Law Enforcement
• ACORN Community High School
• High School for Civil Rights & Law
• Bedford Academy High School
• High School for Enterprise, Business, &
Technology
• Bronx Academy of Letters
• High School for Law and Public Service
• Bronx Leadership Academy II
• High School for Sports Management
• Brooklyn Academy of Science & the
Environment
• Brooklyn High School for Law &
Technology
• High School of Service and Learning
• Manhattan Bridges High School
• Brooklyn Preparatory High School
• Mott Hall Bronx High School
• Central Park East High School
• New Design High School
• Channel View School for Research
• Performing Arts and Technology High
School
• Collegiate Institute for Math & Science
• Queens Preparatory Academy
• Eagle Academy for Young Men
• Queens Vocational & Technical High
School
• East Bronx Academy for the Future
• Renaissance school
• East Side Community School
• School for Human Rights
• El Puente Academy for Peace & Justice
• Science, Technology & Research Early
College High School at Erasmus
• Essex Street Academy
• Teachers Preparatory High School
• Explorations Academy
• Frederick Douglass Academy VII
• George Washington Carver High School
for the Sciences
• Gregorio Luperon High School for Science
& Mathematics
• Thurgood Marshall Academy for Learning
& Social Change
• Transit Tech Career & Technical Education
High School
• Urban Assembly School for Careers in
Sports
• Urban Assembly School for Design &
Construction
8
PREPARING BLACK AND LATINO YOUNG MEN FOR COLLEGE AND CAREERS
APPENDIX C: CODING PROCESS
Detailed Coding Process
Of the 40 ESI schools, two joined late and so did not have materials ready as we wrote this report. Two others did
not start ESI in fall 2012 due to delays caused by Hurricane Sandy. Three schools did not fully describe existing
initiatives in their applications. We excluded these seven schools, creating a final sample of 33 schools (See Appendix
B for full list of all 40 ESI schools).
We analyzed two types of documents that schools created during the design and startup phases of ESI: the application
and the work plan. The application for ESI funding contains a section in which schools describe existing academic,
youth development, and school culture strategies aimed at supporting Black and Latino male students. We used this
document to gain a sense of the breadth of strategies already present in ESI schools across the three domains. Our
analysis of intended ESI strategies is based on the final work plans that ESI schools developed in consultation with the
DOE’s ESI team during the summer and early fall of 2012.
We developed 25 codes to describe the various strategies for supporting Black and Latino male students. Each code in
our codebook represents a different strategy by which ESI schools support, or plan to support, their Black and Latino
male students. Our choice to define each code as a “strategy” is in part driven by the language of the ESI Design
Challenge Description (Appendix A), which indicates that “ESI will support 40 high schools that have shown promise
in graduating Black and Latino young men and develop strategies that work to raise the bar,” and specifies that design
teams should select “strategies that build on and connect” ESI’s three domains (emphasis added).
The codes we created were based on the Design Challenge description (see Appendix A), as well as the applications
and work plans submitted by individual ESI schools (see Appendix A). A team of five researchers, trained in the
coding process, coded both the applications and the work plans. Each researcher was assigned between 5 and 12
schools, so each of the 33 schools included in our analysis was coded by at least one researcher. A subset of school
materials was designated for re-coding by a second team member to assess inter-rater reliability. The recoding
process is described in detail below.
See page 10 for our complete codebook. As the table illustrates, each strategy falls into one of the three ESI domains.
A variety of supports and services comes under the umbrella of each strategy. For example, both culturally relevant
curricula and new course materials are classified as “curricular enhancement” strategies, while professional
development in culturally relevant education and subject-specific professional development are both classified as
“academic professional development” strategies.
Many programs schools proposed in their work plans or described in their applications incorporate more than one
strategy. For example, consider a program wherein teachers receive professional development in culturally relevant
education, develop new curriculum, and utilize new instructional materials. Though the school has described only
one program, we identified two different strategies per our codebook: “academic professional development” and
“curricular enhancements.” Our coding system thus highlights the breadth of strategies that the school offers or plans
to offer to support Black and Latino male students, rather than the number of discrete programs. Also note that the
number of codes we generated does not necessarily reflect the intensity with which a particular strategy is used. For
example, the program described above includes two components related to curriculum (new curriculum and new
course materials), but this only generated one code (curricular enhancements). Thus, our coding system captures the
breadth of strategies employed within ESI’s three domains better than the intensity with which schools pursue any
single strategy.
In order to enhance the reliability of our coding system overall and as applied to individual schools, we designated 17
applications and 20 work plans to be independently coded by two separate researchers. Different pairs of researchers
were assigned to code these materials, so that by the end of the reliability checking process, every researcher had at
some point been paired with every other researcher to code at least one school.
9
Once all the schools designated for reliability checking had been coded twice, we used binary output from our online
coding tool (Qualtrix) to determine the extent to which researchers had recorded the same codes for the same
schools. Our reliability measure was a simple percent match calculation. Across all programming approaches, we
calculated the number of times two researchers’ codes matched for a given school divided by the total number of
programming approaches available (25). For our purposes, a match was defined as any instance of two researchers
recording the same code for the same school (a positive match), as well as any instance of neither researcher
recording a given code for the same school (an empty match). A non-match, by contrast, was any instance of one
researcher recording a code that the other did not record.
Any application or work plan that did not have a 75 percent match rate was automatically flagged for recoding, and
several applications or work plans were flagged for recoding for other reasons (e.g., to ameliorate disparities between
the amount of training different researchers received in the coding system). Re-coding responsibilities were shared
between researchers, so that two to three researchers reached consensus on the applicable codes for each application
or work plan flagged for recoding. At the completion of the reliability checking process, we had reached an 88
percent overall match for existing programming approaches and an 81 percent match for proposed programs.
10
PREPARING BLACK AND LATINO YOUNG MEN FOR COLLEGE AND CAREERS
Full Codebook
Strategy
Academics
Programming
Examples
Programming for four-year sequence of college-ready courses in core subject, expanded learning time, etc.
Digital literacy
Additional textbooks, instructional materials, licenses for online materials, additional course requirements, common core
alignment, AP/honors courses, International Baccalaureate program, culturally relevant curriculum, new course offerings,
new curricula, alternative assessments (e.g., portfolios), special research projects, cross-curricular or interdisciplinary
courses, etc.
Additional use of technology in curriculum, additional technological resources, online courses, etc.
Learning habits & skills
Organization, time management, study skills, note-taking, etc.
Attendance supports
Attendance incentives, attendance calls, attendance tracking, systems for following-up with absent/truant students, etc.
Academic supports
Tutoring, summer/transitional programs (non-college focused), Regents prep, test prep, credit recovery, smaller classes,
Saturday school, remediation and support classes, inquiry teams to support struggling students, recognition of academic
accomplishments, Collaborative Team Teaching, etc.
Parent/family outreach
Student progress conferences, parent classroom visits, parent workshops (pertaining to: homework help, subject-specific
content, curriculum, test prep, etc), “Back to school night”, phone contact with parents, home visits, etc.
Professional development
Trainings on: subject-specific content, instructional methods, technology, culturally relevant pedagogy/curriculum. Inquiry
into practice; etc.
Personnel
ELA/Math coaches, resource room staff, tutors, volunteers, curriculum developers, new teachers, etc.
Curricular enhancements
11
Strategy
Youth Development
Mentoring programs
Examples
Rites of passage programs, peer mentoring, adult-student mentoring, etc.
Wraparound services
Mental health services, physical health services (eye exams, dental services, etc.), counseling, health clinic, nutrition, family
legal or immigration support, etc.
Extracurricular programs
Clubs, sports teams, visual and performing arts activities, etc.
Non-academic supports
Service learning, life-skills training (persistence, grit, resilience, etc.), community- and team-building, leadership training,
advisory, etc.
Parent/family outreach
Informational materials for parents around youth development, workshops for parents/family members, home visits, etc.
Enrichment activities
Field trips, travel, non-academic summer programs, etc.
Student behavior supports
Professional development
Incentive systems, recognition events, school-wide discipline code, alternatives to suspension (in-house suspensions,
reflective assignments, community service), restorative justice programs, etc.
Trainings on: youth development, cultural competence, etc.
Personnel
Social worker, school psychologist, school psychiatrist, guidance counselor, advisor, etc.
DRAFT: Not for Publication/Distribution
School Culture
Career supports
Internships, co-op programs, workplace experiences, aptitude tests, career workshops, vocational certification programs,
career days, etc.
College supports
One-on-one college advising, peer college advising, push-in college counseling, programs to address undermatching,
college workshops, visits from college representatives, alumni panels, higher-education partnerships, college-focused
summer bridge programs, SAT/PSAT prep, college tours, college courses, etc.
Parent/family outreach
Workshops for parents or family members on financial aid, course planning, application process, SAT prep, etc. Meetings
with guidance counselor, information materials and resource packets, etc.
School environment
Uniforms, award assemblies, school spirit events, school orientation programs, school leadership opportunities, etc.
College-going environment
Recognition of seniors with college acceptances, college displays, posters, slogans, college pendants, etc.
Professional development
Trainings on: CUNY/SUNY application process, financial aid, SAT prep, college advising, college matching, etc.
Personnel
Guidance counselor, college advisor, etc.
285 Mercer Street, 3rd Floor | New York, New York 10003-9502
212 992 7697 | 212 995 4910 fax
[email protected] | www.steinhardt.nyu.edu/research_alliance
The Research Alliance for
New York City Schools conducts
rigorous studies on topics that
matter to the city’s public schools.
We strive to advance equity and
excellence in education by
providing non-partisan evidence
about policies and practices that
promote students’ development
and academic success.