Final Results of the Summative
Evaluation of the New Educator
Support Team (NEST) Project
CENTER FOR RESEARCH ON TEACHING AND LEARNING
Department of Teaching and Learning
82 Washington Square East, Suite 700
New York, NY 10003 | 212 998 5872 | 212 995 3636 fax
www.steinhardt.nyu.edu/teachlearn/crtl
Final Results of the Summative Evaluation of the New
Educator Support Team (NEST) Project
Robert Tobias, Director
Beth McDonald, Research Scientist
Rosa Rivera-McCutchen, Research Assistant
CRTL External Evaluation Series
EE-0304-02
March 2004
Center for Research on Teaching and Learning
Department of Teaching and Learning
The Steinhardt School of Culture, Education, and Human Development
New York University
© Copyright 2004 by the Center for Research on Teaching and Learning
CRTL External Evaluation Series EE-0304-02
i
11/21/2011
Abstract
This report summarizes the findings, conclusions and recommendations of the evaluation
of the New Educator Support Team (NEST) conducted by the Center for Research on
Teaching and Learning. They find that:
- NEST has contributed to the professional growth of new teachers in schools that
fully implemented the program.
- New teachers in NEST schools reported that they felt high levels of teacher
efficacy, a strong sense of confidence in their teaching skills, and low levels of
isolation.
- NEST did not have a systematic effect upon the retention rates of new teachers
after one and two years of program support.
- NEST has strengthened the capacity of some of its schools to support the
induction process for new teachers.
- Five essential program characteristics emerged from a review of the qualitative
data: alignment of the program with school priorities, direct support from the
NEST site facilitator, the “teacher-to-teacher” nature of the support, leadership
opportunities, and connections to people and organizations outside the school.
Recommendations include:
- Help the school and coaches identify and prioritize the skills that are most
important for the particular context or individual teacher.
- Increase the amount or clarity of communication regarding NEST activities in the
school as a means of building awareness and continued support for the program.
- Increase opportunities for coaches and administrators to learn from each other
about program design and implementation.
- Work with schools to articulate greater alignment between program activities and
school priorities for curriculum and achievement.
- Increase the support and focus on the issue of sustainability.
- Enhance the supports and incentives for the coaches.
CRTL External Evaluation Series EE-0304-02
ii
11/21/2011
Contents
Abstract ............................................................................................................................ ii
List of Tables ................................................................................................................... iv
Introduction....................................................................................................................... 1
Introduction....................................................................................................................... 1
Findings and Conclusions ................................................................................................. 1
Recommendations............................................................................................................ 9
References........................................................................................................................ 16
CRTL External Evaluation Series EE-0304-02
iii
11/21/2011
List of Tables
Table 1. Summary of Statistical Tests Comparing Mean Cluster Scores
Of Cohorts 1 and 2 on the Elements of Effective Classrooms and
Instruction Survey............................................................................................................ 12
Table 2. Summary of Statistical Comparison of Meat DRTOS Cluster
Ratings for NEST Teachers in Cohort 1 and Cohort 2 .................................................... 12
Table 3. Quasi-Longitudinal of Gains in ELA Achievement Test Scores
For NEST Schools and Their Respective Districts.......................................................... 13
Table 4. Quasi-Longitudinal Comparison of Gains in Math Achievement
Test Scores for NEST Schools and Their Respective Districts ....................................... 14
Table 5. Mean Responses of New Teachers to the Affective Characteristics
Of Teaching Survey ......................................................................................................... 15
Table 6. Percentages of New Teachers in NEST Elementary and Middle Schools
Remaining in the Same School, Remaining in the NYCDOE After One and
Two Years Compared to All Other Elementary or Middle Schools in their
Respective Districts ......................................................................................................... 16
Table 7. Percentages of New Teachers in NEST High Schools Remaining in
the Same School, Remaining in the NYCDOE After One and Two Years Compared
to All Other Elementary or Middle Schools in Their Respective Districts ..................... 17
CRTL External Evaluation Series EE-0304-02
iv
11/21/2011
Introduction
This report summarizes the findings, conclusions and recommendations of the evaluation
of NEST conducted by the Center for Research on Teaching and Learning. The
conclusions and recommendations are based on findings from analyses of extensive
quantitative and qualitative data collected during the 2002-2003 school year, the fourth
year of project implementation. The evaluation addressed six questions concerning the
summative effects of the project upon the teachers and schools that it served from its
inception in 1999-2000. This report addresses the first five evaluation questions; the
sixth question, which involves a cost-benefit analysis of the program, will be the subject
of a subsequent report. For convenience, all tables referred to in the text below are
displayed at the end of the report.
Findings and Conclusions
1.
Has NEST enhanced the capacity of new teachers to educate their students
effectively?
Data from self-report surveys and classroom observations of a sample of new teachers
suggest that NEST has contributed to the professional growth of new teachers in schools
that fully implemented the program. Analysis of trends in NEST schools’ standardized
test scores indicate that the program’s impact on teaching and learning varies
considerably among the schools. This variablility is probably related to differences
among the sites in the level of program implementation and the site-specific
customization of goals and services that is central to the program’s philosophical and
structural model.
The evaluation assessed the direct effects of NEST on the professional acumen of new
teachers in two ways: first, through a self-report survey and second, through classroom
observations. The survey, Elements of Effective Classrooms and Instruction Survey
(EECIS), measured teachers’ self-perceived level of development in 29 characteristics of
effective teaching, clustered in four areas—Planning and Preparation, Classroom
Environment, Instruction, and Professional Responsibilities. Two cohorts of teachers in
the NEST schools were asked to respond to the EECIS. Cohort 11 was comprised of
more experienced teachers who were part of NEST’s target population of new teachers
during the first two years of program implementation, i.e. 1999 – 2000 and 2000 - 2001.
Cohort 2 included teachers who had no more than two years experience in the NEST
schools prior to the year of data collection, i.e. 2002 – 2003, and, consequently, had less
opportunity to participate in program services. Table 1 summarizes the results of
statistical comparisons of the mean cluster ratings for the two cohorts. T-tests for
independent samples showed that the mean ratings for Cohort 1 were significantly higher
1
NEST program documents and NEST evaluation documents use the term “cohort” in different ways. In
program documents, Cohort I is comprised of the nine continuing schools and Cohort II refers to the two
schools that began NEST in the 2002 – 2003 school year. In this and other evaluation reports, Cohort 1
refers to new teachers who were served by the program for three or four years, while Cohort 2 refers to new
teachers served by the program for one or two years.
CRTL External Evaluation Series EE-0304-02
1
11/21/2011
statistically than those for Cohort 2 in all four clusters. Thus, on average, Cohort 1
teachers, who had more opportunities for program participation, reported higher levels of
professional-skills development than the teachers in Cohort 2, who had fewer program
experiences. Further, the effect sizes for these comparisons indicated that the differences
were educationally meaningful; the effect sizes for Planning and Preparation and
Instruction were large and those for Classroom Environment and Professional
Responsibilities were moderate. Providing additional evidence of the effectiveness of the
project, correlational analysis showed revealed moderately strong relationships between
the teachers’ self-reports of professional competence and the level of mentoring and
professional support they reported having received. For instance, teachers who reported a
high degree of professional competence in Instruction also reported receiving high levels
of program support in that area. Although the influence of other potential causal
factors—including experience and other sources of professional support—cannot be
completely ruled out, the convergence of the findings from the two sets of analyses lends
tenability to the conclusion that NEST was effective in promoting the professional
growth of new teachers.
Analysis of a third source of data—classroom observations of 41 NEST teachers using
the Domain Referenced Teacher Observation Scale (DRTOS)— provides additional
evidence in support of the positive impact of NEST on the professional skills of new
teachers. Table 2 presents the results of a comparison of the observation ratings for
Cohort 1 teachers and Cohort 2 teachers. The DRTOS ratings were on a five-point scale
using the following ordinal levels: 1= Not Proficient, 2=Partially Proficient,
3=Approaching Proficient, 4=Proficient, 5=Distinguished. The results summarized in
Table 2 show that for all four clusters, the mean ratings for Cohort 1 were higher than for
Cohort 2. Overall, the mean ratings for Cohort 1 were firmly in Level 3 while those for
Cohort 2 ranged from the high end of Level 2 to the low end of Level 3. The mean
differences between the cohorts were not statistically significant at the p < .05 level,
primarily due to small sample size. However, using the criterion measure of Effect Size
(J. Cohen, 1977), the differences were educationally meaningful at the small to moderate
level. The mean differences for Planning and Preparation and Instruction were
moderately meaningful, while those for the Classroom Environment and Professional
Responsibilities were small.
The evaluation also assessed the indirect effects of NEST upon the effectiveness of
teaching and learning by analyzing trends in standardized tests of English language arts
(ELA) and mathematics in NEST elementary and middle schools. The analysis was
conducted in four continuing schools—three elementary and one middle school. A quasilongitudinal analysis was conducted by comparing the change in performance of cohorts
of NEST school students across three test administrations to the overall test-score change
in each school’s respective district. The three years spanned by the analysis included
spring 1999—the pre-project year—and spring 200 and spring 2001—the first two years
of the project. The analysis employed two metrics—mean scale scores and the
percentage of students meeting or exceeding standards, i.e. scoring in performance levels
3 or 4. The data were extracted from tables of city and state test scores reported on the
CRTL External Evaluation Series EE-0304-02
2
11/21/2011
NYC Department of Education’s website (www.nycenet.edu).2 Tables 3 and 4
summarize the analyses of test score trends in ELA and math, respectively, for each
NEST school and district. The test score gains for NEST schools B and C far exceeded
those for their respective districts in both ELA and mathematics; Nest school F, a middle
school, showed gains that were similar to those for their District; and NEST school C
showed gains that were far below those for their district. The variability in the findings
pointed to the need for a case study approach in the evaluation’s second year aimed at
illuminating the interactive effects of school-specific contextual factors and
programmatic variables in determining the summative effects of the program.
2.
Has NEST reduced new teachers’ sense of isolation and contributed to their
sense of inclusion in the school community, feelings of teacher efficacy and job
satisfaction?
New teachers in NEST schools reported that they felt high levels of teacher efficacy, a
strong sense of confidence in their teaching skills, and low levels of isolation. However,
the evaluation was not able to attribute improvements in these attributes directly to the
project.
Table 5 shows the mean responses on the Affective Components of Teaching Survey
(ACTS) for 82 new teachers in NEST schools. The data indicated that the respondents
had a high degree of confidence in their teaching ability and a strong sense of personal
efficacy as teachers. Further, they reported low levels of feelings of isolation and
moderate job satisfaction. However, the differences in the mean responses of Cohort 1
and Cohort 2 were not statistically significant and the effect sizes were not educationally
meaningful, thereby suggesting that greater levels of program support did not lead to
gains in mean ACTS scores. Accordingly, the hypothesis that the positive affective
findings were directly attributable to the program was not supported by these data.
On the other hand, qualitative data support the overall ACTS findings of high levels of
teacher efficacy and low levels of isolation among new teachers in NEST schools.
Comments from surveys and focus groups included language such as “moral support,”
“more friendly environment,” “teachers are talking to each other,” and “decreased
isolation” in response to questions concerning the benefits resulting from NEST support.
The convergent qualitative and quantitative findings lend support to NEST as a positive
force for affective support of new teachers. The second year evaluation will explore this
issue further through case study.
3.
To what extent has NEST improved the retention rates of new teachers?
A longitudinal analysis of teacher-retention data suggests that NEST did not have a
systematic effect upon the retention rates of new teachers after one and two years of
2
Although the students in each cohort are not exactly the same from one year to the next, the stability of
the cohorts is substantial, with more than 85 percent overlap across the three years of data. Accordingly,
the evaluators assert that this quasi-longitudinal analysis is internally valid.
CRTL External Evaluation Series EE-0304-02
3
11/21/2011
program support. There were large school-to-school differences in the results of
comparisons of retention rates for NEST schools and their respective districts.
The evaluators tracked a cohort of new teachers with no more than five years of total
experience in each of eight NEST schools and new teachers meeting the same criteria in
all other schools in the same districts (NEST District schools) for two school years
beginning in September 2000, the second year of the NEST program. The schematic
diagram at the end of the report depicts the tracking process. The status of the September
2000 cohort teachers in each NEST school and the respective NEST District schools was
determined at the beginning of the following year, September 2001. Cohort teachers
were divided into three status categories: Remained in the same school, transferred to
another school in the New York City Department of Education (NYCDOE), or left the
NYCDOE. Those who remained in the same schools and the NYCDOE were tracked for
a second year with their status similarly determined in September 2002, after the
program’s third year.
Table 6 displays the results of the longitudinal teacher retention analysis for NEST
elementary and middle schools. After one and two years, the retention rate for the NEST
middle school, School D, exceeded that for its district’s other middle schools and the
percentage of teachers leaving the NYCDOE was lower for NEST School D. For the
elementary schools, after two years the retention rates for School B and School C were
slightly higher than their respective districts, while the retention rate for School A was far
below that for its district. For the high schools, Table 7 shows that School F had a far
higher retention rate than its district after one and two years, as well as a far lower
percentage leaving the NYCDOE. On the other hand, School E and School H had lower
retention rates and higher percentages leaving the NYCDOE than their respective
districts after one and two years. Finally, after exceeding its district in retention rate after
one year, School G reversed the trend, falling behind the district in retention rate after
two years. The instability in the findings for the high schools is partly attributable to the
small sample sizes for the NEST schools.
4.
Has NEST strengthened the capacity of participating schools to establish
and sustain support systems for new teachers?
The findings indicate that NEST has strengthened the capacity of some of its schools to
support the induction process for new teachers. However there was considerable
variability among the schools in the continuity of program implementation when funding
and direct support from NEST staff were reduced in the nine original schools.
During the evaluation year, 7 of the 9 continuing schools implemented a program, several
of them after multiple revisions of their NEST school plans over the course of the year. In
schools where the program was sustained, several factors were cited in interviews and
focus groups as contributing to the school’s capacity to continue providing services.
Those factors included the presence of “key people” committed to doing the necessary
coordination, administrative support, the empowerment of outside funding, and
deliberate, focused planning regarding the sustainability of program components.
CRTL External Evaluation Series EE-0304-02
4
11/21/2011
Challenges leading to reduced implementation included staff turnover and/or loss of “key
people”, reductions in funding for stipends, workshops, and materials, and the loss of
release time for coaches/coordinators due to competing demands on staff time. The
project director cited the uniqueness of each school with regard to implementation and
had not discerned any particular pattern with regard to sustainability.
5.
What are the elements of NEST that are essential to its success?
Five essential program characteristics emerged from a review of the qualitative data:
alignment of the program with school priorities, direct support from the NEST site
facilitator, the “teacher-to-teacher” nature of the support, leadership opportunities, and
connections to people and organizations outside the school.
Examples of evidence supporting the identification of the five essential elements are as
follows:
•
•
•
•
•
NEST steering committees in each school designed and submitted annual
proposals for their NEST programs. Inspection of the proposals and interviews
with steering committee members indicated the importance of the alignment of
the plans with school priorities.
Many stakeholders indicated that the direct support of the NEST site facilitator in
the first years of program participation was highly valuable, provided the
individual working in the school was seen as competent and helpful.
Once the “outside” support of the NEST facilitator was reduced, coach teachers
continued to support the new teachers in various ways. The newer teachers were
therefore being supported by more experienced colleagues in a non-evaluative
context, which allowed for more trust and openness about needs.
The coach and site coordinator roles were described as offering leadership
opportunities for teachers, and as an acknowledgement of the limitations of
“expert-driven” professional development. On the other hand, the demands
placed on coaches, most of whom continued to teach full-time, have led to some
of the sustainability challenges mentioned previously.
Affiliations with AOL Time Warner and NYU were seen as valuable.
Participants have also reported benefits from opportunities to converse with their
counterparts in other participating schools.
Recommendations
The recommendations below are based on the preliminary findings of the evaluation and
are aimed at enhancing the effectiveness of future replications of the project. These
recommendations can also be considered as important lessons learned from NEST
implementation and evaluation for supporting and retaining new teachers.
CRTL External Evaluation Series EE-0304-02
5
11/21/2011
1. Since one of NEST’s articulated goals is “improving new teacher skills,” it might
be useful to help the school and coaches identify and prioritize the skills that are
most important for the particular context or individual teacher. Sets of teaching
standards, such as INTASC (Interstate New Teacher Support and Assessment
Consortium), or the elements of effective teaching described by Charlotte
Danielson (which provided the foundation for DRTOS) can provide a “common
language” for self-assessment and coaching, and promote teacher growth within a
continuum of teacher development that is accepted by the larger community of
professionals.
2. Increase the amount or clarity of communication regarding NEST activities in the
school as a means of building awareness and continued support for the program.
The program is intended to be embedded within the school community in an
integrated manner, which is supported by the professional research regarding
effective professional development. Nevertheless, if teachers are not aware that
the program exists in the school, or that a particular activity is supported by
NEST, coaches and especially new teachers will not be able to articulate their
support for maintaining the program.
3. Increase opportunities for coaches and administrators to learn from each other
about program design and implementation. Some sharing of successes and
challenges occurred during spring 2003 coach meetings, yet not all schools were
represented in these sharing opportunities. Based on our observations and the
feedback offered by attendees, these opportunities to network were seen as
valuable and might be strengthened as a means of maintaining interest and
promoting sustainability. In interviews, several respondents, including principals,
suggested that they would like to know more about how other schools implement
and sustain the program, perhaps through school inter-visitations. Electronic
networking might be enhanced to allow communication and problem-solving
across schools. The frequency of the NEST newsletter might be increased and
could include a “Teacher to Teacher” column of ideas and challenges.
4. Work with schools to articulate greater alignment between program activities and
school priorities for curriculum and achievement. This issue was raised in the
context of gaining and keeping administrative support for the program. The
proposal application form for the 2002-2003 included the question: “How does
this program meet the particular needs of your school community?” Perhaps this
question, which elicited broad responses in most cases, could be refined to elicit
more concrete information about the alignment between needs of new teachers
and the specific improvement goals outlined annually by the school
administration and school leadership team. (The 2003-2004 form eliminated this
question.)
5. Increase the support and focus on the issue of sustainability. The 2002-2003
proposal asked: “How will this program build capacity in your school to support
and develop new teachers after the funding terminates?” The 2003-2004
CRTL External Evaluation Series EE-0304-02
6
11/21/2011
eliminated this question. Based on the sustainability issues surfaced by the
evaluation, it seems apparent that greater attention to this question might enhance
the school’s ability to sustain services as funding and outside support are reduced.
Besides having the NEST committee at each school formulate a written response
to this question, real sustainability will also require ongoing discussion and
deliberate planning with school personnel to design program components that can
realistically be supported. This is especially true in light of the turnover of
administrative staff which characterizes many NYC schools. In the case of NEST,
only two of the original nine principals are still in their schools, and a third took
office just as program implementation began, meaning this administrator had not
been part of the proposal writing and initial planning discussions.
6. Enhance the supports and incentives for the coaches. The turnover in the role of
coaches has been identified as a problem in sustaining the program. Release time
for coaches to support new teachers during the day, and opportunities for their
own professional development, such as the Critical Friends Group for coaches
planned for 2003-2004, seem to be key elements necessary to increase the
capacity and willingness of the identified coaches to carry through with the plans
outlined in the school proposals.
CRTL External Evaluation Series EE-0304-02
7
11/21/2011
Table 1
Summary of Statistical Tests Comparing Mean Cluster Scores of Cohorts 1 and
2 on the Elements of Effective Classrooms and Instruction Survey
Cluster
Cohort
N
Mean
Planning and Preparation
Cohort 1
Cohort 2
25
40
3.92
3.37
The Class. Environment
Cohort 1
Cohort 2
25
40
Instruction
Cohort 1
Cohort 2
Prof. Responsibilities
Cohort 1
Cohort 2
S.D.
t
df
P
Effect
Size
0.61 3.3
0.69
63
0.002
.90
4.15
3.77
0.51 2.09
0.80
63
0.041
.75
24
40
4.07
3.51
0.50 3.35
0.72
62
0.001
1.12
24
40
4.03
3.65
0.66 2.12
0.70
62
0.038
.58
Note: Cohort 1 refers to new teachers who were served by the program for three or four years,
while Cohort 2 refers to new teachers served by the program for one or two years.
Table 2
Summary of Statistical Comparison of Mean DRTOS Cluster Ratings for NEST Teachers
in Cohort 1 and Cohort 2
Observation
Group
N
Mean
SD
Mean
t-test
P
Effect
Cluster
Diff.
Size
Planning &
Preparation
Classroom
Environment
Instruction
Professional
Responsibilities
Cohort 1
14
3.23
.75
Cohort 2
Cohort 1
27
14
2.87
3.25
.66
.80
Cohort 2
Cohort 1
27
14
3.00
3.28
.68
.82
Cohort 2
Cohort 1
27
14
2.90
3.38
.62
.86
Cohort 2
27
3.15
.70
.36
1.61
.12
.56
.25
1.13
.27
.40
.38
1.68
.10
.61
.23
.90
.38
.31
Table 3
Quasi-Longitudinal Comparison of Gains in ELA Achievement Test Scores
CRTL External Evaluation Series EE-0304-02
8
11/21/2011
for NEST Schools and Their Respective Districts
Elementary Schools
Year Tested
School/District
School/District
Change from
Base
Year
School A
Cohort
Cohort
G3 Cohort
Statistic
Statistic
Mean SS
% Levels 3&4
N
1999
2000
2001
Grade During Year Tested
3
4
5
617.9
625.1
646.2
41.9%
24.5%
43.3%
129
139
141
District A
G3 Cohort
Mean SS
% Levels 3&4
N
617.7
38.9%
3339
633.3
37.0%
3352
650.8
46.3%
3311
33.1
7.4%
-0.8%
School B
G3 Cohort
Mean SS
% Levels 3&4
N
616.7
32.4%
241
642.0
45.1%
226
658.5
55.6%
216
41.8
23.2%
-10.4%
District B
G3 Cohort
Mean SS
% Levels 3&4
N
630.3
52.1%
2634
651.5
55.4%
2670
663.4
63.1%
2572
33.1
11.0%
-2.4%
School C
G3 Cohort
Mean SS
% Levels 3&4
N
615.0
31.7%
230
630.3
37.3%
209
655.9
54.0%
202
40.9
22.3%
-12.2%
District C
G3 Cohort
Mean SS
% Levels 3&4
N
622.3
43.6%
3644
641.3
44.6%
3612
655.5
54.2%
3558
33.2
10.6%
-2.4%
28.3
1.4%
9.3%
Middle Schools
Year Tested
School/District
School/District
School F
Cohort
Cohort
G6 Cohort
Statistic
Statistic
Mean SS
% Levels 3&4
N
1999
2000
2001
Grade During Year Tested
6
7
8
645.0
NA
678.6
13.8%
NA
17.1%
305
NA
292
District F
G6 Cohort
Mean SS
% Levels 3&4
N
659.6
32.8%
2688
CRTL External Evaluation Series EE-0304-02
9
NA
NA
NA
693.8
39.0%
2520
Change from
Base
Year
33.6
3.3%
-4.3%
34.2
6.2%
-6.3%
11/21/2011
Table 4
Quasi-Longitudinal Comparison of Gains in Math Achievement Test Scores
for NEST Schools and Their Respective Districts
Elementary Schools
Year Tested
School/
School/District
Change from
Base
Year
School A
Cohort
Cohort
G3 Cohort
Statistic
Statistic
Mean SS
% Levels 3&4
N
1999
2000
2001
Grade During Year Tested
3
4
5
596.7
625.0
635.0
42.3%
31.4%
20.0%
142
140
150
District A
G3 Cohort
Mean SS
% Levels 3&4
N
597.3
41.0%
3425
631.4
44.6%
3366
643.7
29.9%
3343
46.4
-11.1%
-2.4%
School B
G3 Cohort
Mean SS
% Levels 3&4
N
597.6
40.5%
259
643.5
60.3%
229
651.7
31.7%
227
54.1
-8.8%
-12.4%
District B
G3 Cohort
Mean SS
% Levels 3&4
N
616.9
61.4%
2693
650.3
68.1%
2695
660.7
48.3%
2613
43.8
-13.1%
-3.0%
School C
G3 Cohort
Mean SS
% Levels 3&4
N
587.6
36.6%
262
627.2
45.1%
215
642.0
22.9%
201
54.4
-13.7%
-23.3%
District C
G3 Cohort
Mean SS
% Levels 3&4
N
601.2
45.4%
3891
635.6
48.8%
3757
646.5
31.1%
3723
45.3
-14.3%
-4.3%
38.3
-22.3%
5.6%
Middle Schools
Year Tested
School/
School/ District
School F
Cohort
Cohort
G6 Cohort
Statistic
Statistic
Mean SS
% Levels 3&4
N
1999
2000
2001
Grade During Year Tested
6
7
8
643.2
665.3
680.7
12.7%
14.5%
13.3%
322
304
331
District F
G6 Cohort
Mean SS
% Levels 3&4
N
664.6
36.3%
2862
CRTL External Evaluation Series EE-0304-02
10
682.2
38.0%
2781
698.6
33.2%
2683
Change from
Base
Year
37.5
0.6%
2.8%
34.0
-3.1%
-6.3%
11/21/2011
Table 5
Mean Responses of New Teachers to the Affective Characteristics of Teaching
Survey
Factor
N
Mean
S.D.
Teacher Efficacy
82
3.83
0.60
Job Satisfaction
82
3.23
0.46
Isolation
82
2.38
0.55
Self Confidence
82
4.13
0.66
Note: Scores are reported on a 5-point (1 – 5) Likert scale. High values indicate positive
responses except for "Isolation" where high values indicate a high degree of perceived
isolation.
CRTL External Evaluation Series EE-0304-02
11
11/21/2011
Table 6
Percentages of New Teachers in NEST Elementary and Middle Schools Remaining in the Same
School,
Remaining in the NYCDOE, and Leaving the NYCDOE After One and Two Years Compared to
All Other Elementary or Middle Schools in Their Respective Districts
Sept.
2000
Total
New
Teachers
End of 2000 - 2001
Remain Remain
Leave
in
in
School
DOE
DOE
End of 2001 – 2002
Remain Remain
Leave
in
in
School
DOE
DOE
Elementary Schools
School A
N
% Total
New
District A
N
% Total
New
% Point
Diff.
64
49
2
13
34
1
29
709
76.6%
593
3.1%
37
20.3%
79
53.1%
477
1.6%
18
45.3%
214
83.6%
5.2%
11.1%
67.3%
2.5%
30.2%
-7.1%
-2.1%
9.2%
-14.2%
-1.0%
15.1%
School B
61
54
4
3
48
3
10
464
88.5%
411
6.6%
22
4.9%
31
78.7%
358
4.9%
12
16.4%
94
88.6%
4.7%
6.7%
77.2%
2.6%
20.3%
-0.1%
1.8%
-1.8%
1.5%
2.3%
-3.9%
53
45
1
7
40
0
13
874
84.9%
770
1.9%
37
13.2%
67
75.5%
645
0.0%
19
24.5%
210
88.1%
4.2%
7.7%
73.8%
2.2%
24.0%
-3.2%
-2.3%
5.5%
1.7%
-2.2%
0.5%
62
53
8
1
47
2
13
249
85.5%
207
12.9%
21
1.6%
21
75.8%
181
3.2%
13
21.0%
55
83.1%
8.4%
8.4%
72.7%
5.2%
22.1%
2.4%
4.5%
-6.8%
3.1%
-2.0%
-1.1%
District B
School C
N
% Total
New
N
% Total
New
% Point
Diff.
N
% Total
New
District C
N
% Total
New
% Point
Diff.
Middle Schools
School D
N
% Total
New
District D
N
% Total
New
% Point
Diff.
CRTL External Evaluation Series EE-0304-02
12
11/21/2011
Table 7
Percentages of New Teachers in NEST High Schools Remaining in the Same School,
Remaining in the NYCDOE, and Leaving the NYCDOE After One and Two Years Compared to
All Other Elementary or Middle Schools in Their Respective Districts
School E
District E
School F
District F
School G
District G
School H
District H
N
% Total
New
N
% Total
New
% Point
Diff.
N
% Total
New
N
% Total
New
% Point
Diff.
N
% Total
New
N
% Total
New
% Point
Diff.
N
% Total
New
N
% Total
New
% Point
Diff.
Sept.
2000
Total
New
Teachers
17
End of 2000 - 2001
Remain Remain
in
in
Leave
School
DOE
DOE
12
1
4
End of 2001 – 2002
Remain Remain
in
in
Leave
School
DOE
DOE
10
1
6
70.6%
1106
5.9%
148
23.5%
172
58.8%
935
5.9%
90
35.3%
401
77.6%
10.4%
12.1%
65.6%
6.3%
28.1%
-7.0%
-4.5%
11.5%
-6.7%
-0.4%
7.2%
23
21
1
1
19
2
2
676
91.3%
498
4.3%
73
4.3%
105
82.6%
402
8.7%
34
8.7%
240
73.7%
10.8%
59.5%
5.0%
35.5%
17.6%
-6.5%
15.5%
11.2%
23.1%
3.7%
-26.8%
30
26
0
4
18
1
11
1008
86.7%
799
0.0%
114
13.3%
95
60.0%
718
3.3%
55
36.7%
235
79.3%
11.3%
9.4%
71.2%
5.5%
23.3%
7.4%
-11.3%
3.9%
-11.2%
-2.1%
13.4%
19
13
3
3
12
1
6
1852
68.4%
1575
15.8%
111
15.8%
166
63.2%
1354
5.3%
54
31.6%
444
85.0%
6.0%
9.0%
73.1%
2.9%
24.0%
-16.6%
9.8%
6.8%
-10.0%
2.3%
7.6%
1426
CRTL External Evaluation Series EE-0304-02
13
11/21/2011
References
Cohen, Jacob. Statistical Power Analysis for the Behavioral Sciences. New York:
Academic Press, 1977.
Danielson, C. Enhancing Professional Practice: A Framework for Teaching. Alexandria,
VA: Association for Supervision and Curriculum Development, 1996.
CRTL External Evaluation Series EE-0304-02
14
11/21/2011
Summative Evaluation of NEST
NEST Schools
Sept 2000
Data File
NEST Districts
New Teachers
New Teachers
NEST
2000-2001
Year 2
Remained in
same school
Sept 2001
Data File
Transferred
to other
DOE
schools
Left
DOE
Remained in
Same
School
Transferred
to other
DOE
schools
Left
DOE
NEST
2001-2002
Year 3
Sept 2002
Data File
Remained in
same school
Transferred
to other
DOE
schools
Left
DOE
Remained in
same school
Transferred
to other
DOE
schools
Left
DOE
Schematic Model for Two-Year Longitudinal Retention Analysis
CRTL External Evaluation Series, EE-0304-01
19