Are We Developing High Quality Teachers
and Can We Prove It? A Study of the
Reliability, Validity and Utility of a Student
Teacher Assessment System
CENTER FOR RESEARCH ON TEACHING AND LEARNING
Department of Teaching and Learning
82 Washington Square East, Suite 700
New York, NY 10003 | 212 998 5872 | 212 995 3636 fax
www.steinhardt.nyu.edu/teachlearn/crtl
Are We Developing High Quality Teachers and Can We Prove It?
Are We Developing High Quality Teachers
and Can We Prove It?
A Study of the Reliability, Validity and Utility of a Student
Teacher Assessment System
Paper presented at the Annual Meeting of the American Educational
Research Association, San Diego, CA, April 2009
Robert Tobias, Director,
Center for Research on Teaching & Learning
Kimberly Woo, Research Assistant,
Center for Research on Teaching & Learning
Frank Pignatosi, Director,
Office of Clinical Studies in Teaching and Learning
New York University
CRTL Research Paper Series
RP-1109-01
Updated November 12, 2009
Center for Research on Teaching and Learning
Department of Teaching and Learning
The Steinhardt School of Culture, Education, and Human Development
New York University
© Copyright 2009 by the Center for Research on Teaching and Learning
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
CONTENTS
Objectives of the Study………………………………………………………………………… ...1
Perspective of the Study ………………………………………………………………..…………2
Connections to Literature……………………………………………………………..…………...3
Mode of Inquiry………………………….………………………………………………..….. … 5
Findings: Research Question 1 - Inter-rater Agreement and Internal Consistency.……..……… 7
Findings: Research Question 2 - Content and Construct Validity …………..................................8
Findings: Research Question 3 - Patterns of Strengths and Weaknesses..………………………10
Findings: Research Question 4 - Perceptions of Usefulness and Accuracy………………….... 11
Conclusions………………………………………………………………………………..……. 13
Significance of the Findings………………………………………………………………...….. 13
Directions for the Future…………………………………………………………………………14
TABLES AND FIGURES
Table 1: Results of the Assessment of the Inter-Rater Agreement of the DRSTOS-R ……...……15
Table 2: Internal Consistency Reliability of the DRSTOS-R …………………………………....15
Table 3: Summary of the Comparisons Between the DRSTOS-R Mean Domain and Total Scores
for the Early and Late Student Teacher Placement Groups Fall 2004 and Spring 2005……......16
Table 4: Results of T-Tests for Paired Samples Applied to Mean Differences Between Early and
Late Placement DRSTOS-R Scores ……………………………………………………………...17
Figure 1: Scree Plot form the Principal Components Factor Analysis of DRSTOS-R Item Scores
for Phase 1 Study Sample………………………………………………………………………..18
Table 5: Loadings of DRSTOS-R Items on the Single Latent Factor…………………………....19
Table 6: Perceived Usefulness of the DRSTOS-R………………………………………………..19
Table 7: Perceived Accuracy of the DRSTOS-R………………………………………………....20
References………………………………………………………………………………………..21
Appendix: The DRSTOS-R Protocol…………………………………………………………….22
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
2
Are We Developing High Quality Teachers and Can We Prove It?
ABSTRACT
The Domain Referenced Student Teacher Observation Scale, Revised (DRSTOS-R) was
developed as part of an ongoing research and development effort aimed at creating a reliable and
valid system for assessing the developing pedagogical proficiency of teacher education students
in a private university with a large and complex teacher education program. Using data collected
over the first four years of DRSTOS-R administration, supervisor surveys, and pupil
achievement data, this three-phase study examines the psychometric properties of the instrument,
the implications of its daya for the assessment of program strengths and weakness, the perceived
usefulness and accuracy of its scores, and its predictive validity. This report presents findings
from Phases 1 and 2 of the study; a separate report will be released in spring 2010 with the
results from Phase 3. The results from Phase 1 showed evidence for DRSTOS-R’s content
validity, construct validity, inter-rater reliability, and internal-consistency reliability. In addition,
this phase identified strengths and weaknesses among final-placement student teachers as well as
minor differences in mean performance between and among program areas. Phase 2 found that
the majority of supervisors administering the DRSTOS-R perceived the information it provided
about the pedagogical development of student teachers to be useful and accurate.. Further, it was
found that half of the responding supervisors made use of the DRSTOS-R to inform the teachertraining process, which is a substantive extension beyond its intended program evaluation
function.
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
OBJECTIVES OF THE STUDY
Demand for accountability of traditional teacher education programs has never been
higher (Darling-Hammond and Bransford, 2005). The No Child Left Behind Act of 2001 and
national accreditation agencies require evidence that the graduates of these programs are highly
qualified educators. As researchers in a private university with a large and complex teacher
education program, we have been engaged over the last four years in a research and development
effort aimed at creating a reliable and valid system for assessing the developing pedagogical
proficiency of our teacher education students. This work has involved the development of an
observation protocol system known as the Domain Referenced Student Teacher Observation
Scale, Revised (DRSTOS-R).
The objectives of this study are to use the data from this evidence base to assess the
psychometric properties of the protocol system. In addition, the protocol database will be
augmented with survey data provided by university supervisors and faculty to determine the
usefulness of the DRSTOS-R for providing information about the developing skills of teacher
education students and the efficacy of teacher education programs. Finally, follow-up surveys
and pupil achievement data will be obtained for a sample of the teacher education graduates to
assess the predictive validity of the protocol.
The specific research questions of this multiple-methods empirical study are as follows:
1. What are the levels of inter-rater agreement and internal-consistency reliability for
the ratings of supervisors trained to use the DRSTOS-R?
2. What are the content and construct validity of DRSTOS-R ratings?
3. What information does the evidence base provide concerning the strengths and
weaknesses of our teacher education programs and how can this information be
used to inform program-improvement efforts?
4. What are the perceptions of student teacher supervisors concerning the accuracy
and usefulness of DRSTOS-R ratings?
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
1
Are We Developing High Quality Teachers and Can We Prove It?
5. What is the predictive validity of DRSTOS-R for estimating the future success of
pre-service teachers in obtaining a teaching position and in having a positive
impact on the achievement of the pupils they teach?
PERSPECTIVE OF THE STUDY
The perspective that guides the design and implementation of this study is three pronged.
First, teacher education institutions must be self-reflective in conducting inquiries into the
efficacy of their programs for meeting the goal of developing highly qualified educators. This
attitude of self reflection is evident in the processes the institution develops for collecting
evidence on the core goals and values of the program and the ways in which these data are used
to inform program decision making.
Second, the evidence used to assess pre-service teacher development should be connected
to quality indicators for practicing teachers in terms of both the content of the constructs that are
used to assess their developing proficiencies and the performance standards to which teachers are
held. Despite its roots in research on teaching, research in teacher education has developed in
isolation from mainstream research on teaching (Grossman and McDonald, 2008). The
researchers used Charlotte Danielson’s taxonomy of the essential elements of effective teaching
and learning (Danielson, 2007) as the framework for the DRSTOS-R, thereby situating the preservice assessment along a developmental continuum connecting aspiring teachers to practicing
teachers.
Finally, the data provided by assessments of teacher education students must show
empirical evidence of validity for making inferences about their developing expertise in teaching
and their future success in providing teaching and learning experiences that will lead to higher
levels of pupil achievement.
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
2
Are We Developing High Quality Teachers and Can We Prove It?
CONNECTIONS TO THE LITERATURE
In response to Federal legislation’s demands for high-quality teachers, increased attention
has turned to defining the skills required of such teachers and the methods by which these skills
might be measured (Goldrick, 2002). In turn, national accreditation agencies such as the Teacher
Education Accreditation Council (TEAC) have placed pressure on traditional teacher education
programs to provide evidence that graduates have mastered the prerequisite skills over their
course of study.
Research on teacher evaluation has identified several common methods by which
teachers’ professional development can be assessed, including portfolio assessments, peer
reviews, and performance-based assessments (Goldrick, 2002). Organizations such as the
Interstate New Teacher Assessment and Support Consortium (INTASC), who have worked on
developing standards for teacher quality and who recognize the complexity of defining good
teaching practice have pushed for the increased use of performance-based assessments, such as
observation protocols, with teachers seeking initial licensure as well as with pre-service teachers
in teacher education programs (Limback & Mansfield, 2002).
Although research on measures of teacher quality have varied in their definitions of
effective teaching, the notion of effectiveness as the influence of the teachers’ behaviors on
students’ achievement has been a consistently recurring element (Blanton, Sindelar, & Correa,
2006). In a review of the major conceptual paradigms for research on teacher effectiveness,
Doyle (1977) identified three primary frameworks, of which only the process-product paradigm
placed the teacher and the teacher’s behaviors at the forefront. Under this paradigm, it is
assumed that teacher’s behaviors are of primary importance in shaping student outcomes and, in
turn, teachers’ effectiveness is defined as the extent to which their students are found to achieve,
typically measured via standardized achievement tests. Mujis (2006), citing Mujis & Reynolds
(2002), note that up to 75% of the variance in pupil outcomes are attributable to teacher
behaviors, further emphasizing the importance of considering teachers’ behaviors in measuring
the effectiveness or quality of a teacher. Process-product measures are also said to be particularly
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
3
Are We Developing High Quality Teachers and Can We Prove It?
useful for certain types of research, including “comparison studies of licensed and unlicensed
teachers and longitudinal studies of change” (Blanton, Sindelar, & Correa, 2006, p.120).
In a review of the common methods of measuring teacher quality, Blanton, Sindelar, &
Correa (2006) found that process-product studies typically took the form of classroom
observations, with teachers’ behaviors codified into categories. Identifying the advantages and
disadvantages to various methodological approaches used in process-product studies of teacher
quality (e.g. surveys, classroom observations, interviews, questionnaires, etc.) Mujis (2006)
found that when teacher behaviors were of interest, the use of classroom observation was
advantageous when compared to other methods of data collection. For instance, when compared
to survey research, classroom observations were said to allow for more objectivity, a broader and
more informed perspective coming from the experience of observing multiple classrooms, and
“fine-grained explorations of behaviors which would be hard to achieve in survey-style studies”
(p.58). Although classroom observations were found to suffer from costliness and potential
observer effects, the authors note that these disadvantages could be minimized through design.
Alternatives to observations (surveys, questionnaires, and interviews), however, were found to
suffer from disadvantages more difficult to circumvent, such as social desirability responses and
a lack of correspondence between responses and what was observed to actually occur in the
classroom.
An observation protocol frequently identified in the literature is the Praxis III, a
commercial observation system that was found suitable for process-product research and which
has begun to be adopted at the state-level for demonstrating teacher performance (Blanton,
Sindelar, & Correa, 2006; Goldrick, 2002). The Praxis III is an indirect ancestor of the
DRSTOS-R, having been the basis for Charlotte Danielson’s work on the “Framework for
Teaching” (Danielson, 2007, vii-vii), which is the in-service model that provided the inspiration
for DRSTOS-R.
Although the Praxis III has demonstrated utility as part of assessment systems for the
licensing of beginning teachers, administration is costly and labor intensive (Blanton, Sindelar &
Correa, 2006). As a consequence, it is impractical for use in the context of a student-teacher
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
4
Are We Developing High Quality Teachers and Can We Prove It?
placement. The DRSTOS-R was developed to fill the pre-service teacher assessment niche in
the developmental continuum from pre-service, to beginning, through practicing teacher. As an
observation protocol, it must be subjected to research on issues including the influence of context
(subject area, instructional level, etc.) on observer ratings, reliability (stability over time and
inter-rater agreement), and the role of professional judgment in its development and use
(Limback, & Mansfield, 2002; Dwyer, 1998; Stodolsky, 1984). This study investigates the
tenability of DRSTOS-R for student-teacher assessment by analyzing evidence concerning its
technical properties and utility.
MODE OF INQUIRY
This empirical study used a three-phase, mixed methods design. The first phase
addressed research questions 1, 2, and 3; the second phase focused on research question 4, and
the third phase investigated research question 5. In Phase 1, quantitative data were collected
using the DRSTOS-R protocol (see appendix), a 20-item summative observation scale based on
Charlotte Danielson’s text Enhancing Professional Practice (2007). The protocol measures the
development of pre-service teachers’ professional practice across four domains: (1) Planning and
Preparation; (2) Classroom Environment; (3) Instruction; and (4) Professional Responsibility.
Each item is rated on a four-point scale, ranging from Not Yet Proficient (1) to Proficient (4),
with Level 3 representing Entry-Level Proficiency.
In order to be qualified for administration of the DRSTOS-R, student-teacher field
supervisors must participate in an intensive full-day training session. In the four years of work
since the pilot, 34 field supervisors underwent training and used the protocol to assess 609
student teacher placements, 91 with observations over multiple semesters and placements, from a
variety of specialization areas including math, science, English, social studies, childhood
education, and second- and foreign-language instruction. This database provided the evidence
for investigation of the research questions 1, 2, and 3.
First, to address research question 1, inter-rater agreement was assessed using data
collected from supervisors during the training process. During training, supervisors were asked
to view a videotaped lesson and independently rate the teacher’s performance. Ratings were
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
5
Are We Developing High Quality Teachers and Can We Prove It?
collected and entered into an Excel database in which the differences between each individual
pairs of raters and the overall mean differences in ratings were calculated.
In addition, to assess
the internal-consistency reliability of the domain scores and the total scale, coefficients alpha
were computed using a subset of the database, which contained data from 447 observations of
student-teacher placements.
The DRSTOS-R was piloted in 2004 by 12 trained supervisors on a sample of student
teachers proportionately stratified by degree and area of study. To address research question 2,
content validity was examined qualitatively through focus groups conducted with the supervisors
who had participated in the pilot and 30 faculty members from the university’s teacher-education
faculty. Focus-group participants provided feedback regarding the use of the protocol, offered
suggestions for its refinement, and discussed the extent to which the protocol aligned with the
goals of the university’s teacher education program. Construct validity was assessed in several
ways. First, the ratings for students in their first (early) student teaching placements were
compared to students in the last or penultimate placement for their programs in a cross-sectional
design. Second, a repeated measures design was used to assess the sensitivity of the protocol for
measuring the gains in pedagogical proficiency of the Second, factor analyses were performed to
confirm the construct and domains measured by the protocol. Finally, the influences of content
area, program degree, and raters upon score variance were investigated through regression
analysis.
Phase 2 of the study will address research question 3 and 4, the information that
DRSTOS-R results provide about the strengths and weaknesses of our teacher education
programs and the usefulness of DRSTOS-R data for assessing students and informing program
improvement. Analyses of the extant database will be used to prepare reports and presentations
for faculty and students concerning the profile of student development in the four domains and
the 20 items of the DRSTOS-R, including disaggregation by program areas and degrees. Focus
groups will be conducted with samples of faculty and students to obtain their perceptions of the
usefulness and accuracy of the data and the reports.
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
6
Are We Developing High Quality Teachers and Can We Prove It?
Phase 3 will address research question 5, the predictive validity of DRSTOS-R, by using
standardized test data for the pupils of a sample of 60 graduates who are currently teaching in a
large urban district in which the university is located. Value-added modeling techniques will be
used to determine the estimated effects of the graduates on the test-score gains of their pupils,
after which the mean test-score gains of the graduates will be regressed on their pre-service
DRSTOS-R scores to determine the magnitude and significance of the relationship.
FINDINGS
This section summarizes the evidence from Phases 1 - 2 of the study. The data for Phase
3 on predictive validity are currently being collected and will be presented in an addendum to
this report.
Research Question 1: Inter-rater Agreement and Internal Consistency
Inter-rater agreement is the extent to which there is consistency in the protocol ratings
assigned by independent judges observing the same subjects at the same time. High inter-rater
agreement is crucial to the accuracy of scores obtained from observation scales. Before being
permitted to use the DRSTOS-R protocol as part of their field supervision, student teacher
supervisors are required to attend a daylong DRSTOS-R training session using videotapes of
classroom teaching to simulate the observation process. Two to three training sessions have been
held each academic year since the protocol was piloted in 2004, with five to ten supervisors
participating per session. The purpose of these trainings is to familiarize supervisors with the
instrument and to establish sufficient inter-rater agreement.
During these training sessions, participants watch two to three videotapes of a teacher
instructing her class. After each video (i.e. trial), participants independently rate the instructor
using the DRSTOS-R protocol. These ratings are then compared for inter-rater differences;
differences between pairs of raters for each item were calculated and then averaged to arrive at
the mean difference for the item. Mean differences across all items for each trial were then
calculated. If a rater did not observe behavior in the videotape that pertained to an item, he or she
did not rate it, and that rater’s score was not included in the calculation. In an attempt to increase
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
7
Are We Developing High Quality Teachers and Can We Prove It?
inter-rater agreement over subsequent trials, discussions of the ratings are held among the
participants following each trial, focusing on discrepant items and the reasons for the
discrepancies.
During Phase 1, we have consistently obtained high levels of inter-rater agreement across
more than ten training sessions with student-teacher supervisors. Table 1 (all tables appear at the
end of the paper) displays the levels of inter-rater agreement that typify data collected during
training and calibration sessions. On a four-point scale, mean inter-rater differences for 11
training items is 0.39 points, with exact agreement obtained for about 50% of all pairs of raters
and around 90% agreement within one score point.
Internal-consistency reliability for the full 20-item protocol was high. Alpha coefficients
for a sample of 430 protocols ranged from .83 - .91 for each of the four domains, with a
coefficient of .96 for the total scale (see Table 2).
Research Question 2: Content and Construct Validity
Content Validity. The first version of the DRSTOS was piloted by a group of 12 student
teaching supervisors who participated in training in April and May of 2004. The purposes of the
pilot were to refine the instrument, validate its use for assessing mastery of the entry-level
standards of practice expected of teacher education students, and to confirm the procedure’s
inter-rater agreement. This initial form contained 21 items that were selected and designed to
align with the university’s claims for accreditation. These items were adapted with faculty and
supervisor input from the 76-item framework described in Danielson’s (1996) book, Enhancing
Professional Practice: A Framework for Teaching.
A focus group was held on May 12, 2004, to gather feedback from the pilot-study
supervisors to assess the content validity of the protocol. Focus group members agreed that, for
the most part, items on the DRSTOS protocol aligned well with the goals of the university’s
teacher education programs and facilitated their assessment of the performance of student
teachers with respect to these goals. Focus group members recommended certain changes to the
items and rubric descriptions that would further increase the content validity of DRSTOS.
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
8
Are We Developing High Quality Teachers and Can We Prove It?
Many of these recommendations were incorporated into a revised version of the protocol
(DRSTOS-R), the form used in the four years of work on student teacher assessment.
Throughout the years, in effort to maintain content validity, supervisors’ suggestions and
feedback collected during training sessions and via direct discussion and correspondence have
resulted in minor modifications to the protocol with respect to wording and the inclusion and
exclusion of certain items.
Construct validity. was demonstrated by the ability of the protocol to discriminate
between the performance of student teachers in their early versus late placements, as one would
hypothesize, if the scale was measuring the construct of pedagogical proficiency (see Table 3).
Late-placement students showed mean DRSTOS-R scores that were significantly higher
statistically than the early placement students with effects sizes for these differences in the
moderate to large range. These differences were observed for all four domains and the total
scores. Moreover, the differences were obtained for both undergraduate (BS) and graduate (MA)
teacher education students.
In addition to the cross-sectional analysis above, the mean gains in DRSTOS-R scores
were analyzed for a sample of 91 student teachers with repeated-measures data for two
successive semesters. The mean gains were statistically significant in all four domains and the
total scores for both BS and MA students, with the exception of professional responsibilities
domain for MA students, which showed a very high baseline score (see Table 4).
Construct validity was also assessed through exploratory factor analysis. A principal
components factor analysis revealed that a single factor explained 60% of the variance in the
scores of the 20 items for a sample of 432 teacher education students. The scree plot (see Figure
1) for this factor analysis shows the strong uni-dimensionality of the factor structure for the
scale. This finding is consistent with the hypothesis that a single construct of pedagogical
proficiency underlies the items in the four domains of the scale. The potency of this latent
construct is indicated by the high loadings of all items on this single factor (see Table 5). This
___________________________________________________________________________
CRTL Paper Presentation Series RP-1109-01
9
Are We Developing High Quality Teachers and Can We Prove It?
finding does not negate the utility of the domain scores for instructional purposes. We have
observed patterns of differences and similarities in the domain and item scores that suggest they
are systematically measuring other unique but important aspects of teaching and learning.
Research Question 3: Patterns of Strengths and Weaknesses
In Phase 1, we also conducted a preliminary analysis of the database to assess the relative
strengths and weaknesses of the developing teaching proficiencies of student teachers at varying
points in our teacher education programs. The analyses revealed statistically significant and
educationally meaningful differences in student development among and within the four
domains, as well as between certification areas and degree programs. The key findings are
summarized below.
•
The percent of student teachers meeting the beginning teacher proficiency standard on
the DRSTOS-R total score was significantly higher statistically for students in their last
placements than those in their first. Nevertheless, despite a 9.6 percentage point
difference, only 64.4% of the last placements met the standard and 35.6% did not; the
entry-level proficiency rates were 60.9% for undergraduate students and 66% for
graduate students.
•
Among the four domains, Classroom Environment and Instruction showed the largest
first versus last placement differences, approximately 15 percentage points. However,
Professional Responsibilities was the domain with the highest beginning proficiency rate
(84.2%) for last placements.
•
Post hoc analyses of differences among certification areas showed a significantly higher
mean total score for science program students than for childhood/special education and
early childhood/special education students. They also showed significantly higher mean
total scores for multilingual/multicultural program students than for early
childhood/special education students. Caution should be exercised in drawing
conclusions from these analyses, because of possible interaction with the effects of raters
and ordinal (i.e. first versus last) placements.
___________________________________________________________________________ 10
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
•
An analysis of the mean ratings on the 20 DRSTOS-R items for the 146 students who
were in their last student teaching placement, having completed our teacher education
program, showed the following profile:
o Nearly 90% demonstrated entry-level pedagogical content knowledge, but only
78% displayed sufficient understanding of the content standards for New York
State and New York City.
o Nearly all (98.6%) regularly created a classroom environment in which students
were free to express themselves.
o More than 90% showed knowledge of their pupils’ skills, special needs, and
learning styles and 95% used and encouraged pupils to use clear and appropriate
spoken and written language.
o More than 90% were becoming reflective practitioners, engaging in selfassessment of their student teaching and making specific suggestions for selfimprovement.
o Two areas that require attention are discussion patterns in instruction and
awareness of pupil behavior. Only 68% of the graduating student teachers were
able to take a facilitating instructional role in which students were mostly engaged
in discussion and only 63% were alert to the behavior of all pupils at all times.
Research Question 4: Perceptions of Usefulness and Accuracy
In phase 2 of the study, conducted during the 2008-2009 academic year, a survey was
distributed to the 41 DRSTOS-R trained supervisors asking their perceptions of the usefulness
and accuracy of the observation protocol. These 41 supervisors included individuals who were
currently supervising, as well as those who had supervised in the past but were no longer doing
so due to other obligations.
The survey asked supervisors to identify on a checklist the ways in which they used the
DRSTOS-R, as well as complete eight three-point Likert items and three open-ended questions
assessing the usefulness and accuracy of the DRSTOS-R, based on their experiences in using the
instrument. Six of the Likert items dealt specifically with perceptions of the protocol’s usefulness
for different purposes and asked supervisors to provide ratings ranging from “1 – Very Useful”
___________________________________________________________________________ 11
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
to “3 – Not Useful At All”. For these items, with the exception of the item rating overall
usefulness, supervisors had the option of answering “Don’t Know,” as their individual
experiences in using the DRSTOS-R may not have allowed them to accurately rate the protocol
for a given purpose. Two Likert items addressed the accuracy of the DRSTOS-R, asking
supervisors to rate on a scale of “1 – Very Accurate” to “3 – Not Accurate at All” whether they
perceived the DRSTOS-R to be accurate as a measure of good teaching and learning practice
across content areas and whether the DRSTOS-R provided an accurate reflection of student
teachers’ professional development. The three open-ended questions provided supervisors with
the opportunity to elaborate upon their responses or comment on aspects of the protocol not
addressed in the Likert items. These questions asked supervisors to state what they believed to be
the strengths and weaknesses of the DRSTOS-R, as well as to provide any additional comments
they had regarding the protocol.
A total of 21 surveys were returned, attaining a response rate of 51.2%. Responding
supervisors represented nearly every cohort trained to use the DRSTOS-R since the pilot in 2004
and included supervisors whose supervisory experience ranged from this being their first
semester to nine years of experience. Responding supervisors reported typically supervising
between 2 to 10 student teachers per semester across eight major areas of teacher education:
childhood education, early childhood education, special education, math, science, English, social
studies, and Multilingual Multicultural Studies (the university’s combined program for teachers
of English as a Second Language and/or foreign languages).
On the checklist, all but one supervisor indicated that they used the DRSTOS-R to grade
or assess their student teachers (95.2%), consistent with the intended use of the instrument as a
summative rating. However, more than half of the responses indicated that they used the
instrument for other purposes as well, including setting expectations for student teachers
(66.7%), as a tool in post-lesson reflection with student teachers (61.9%), establishing goals for
observations (61.9%), and recordkeeping during observation (52.4%). A third or less of
respondents indicated that they also used the DRSTOS during their three-way conference with
the student teacher and cooperating teacher (33.3%) or planning with the student teacher
(23.8%).
___________________________________________________________________________ 12
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
The responses to the Likert items relating to usefulness are reported in Table 6. As can be
seen, across all items, the majority of respondents rated the DRSTOS-R as at least “Somewhat
useful” for the purposes indicated. For the item requesting supervisors’ ratings of the protocol’s
overall usefulness, it was found that all responses indicated that the DRSTOS useful in some
capacity; the majority of responses (71.4%) reported the DRSTOS-R as “somewhat useful,”
while the other 28.6% provided an overall rating of “very useful.” The purposes for which the
DRSTOS-R received the highest ratings of perceived use (combining ratings of “somewhat
useful” and “very useful”) were “providing a comprehensive overview of student teachers’
performance” (89.5%) and “assessing student teachers’ strengths and weaknesses” (90.5%).
Table 7 shows the results of the supervisors’ ratings of the accuracy of the DRSTOS-R.
As a description of good teaching and a reflection of student teachers’ professional development,
respondents most frequently rated the instrument as “somewhat accurate” (65% and 66.7%
respectively). Overall, responses overwhelmingly indicate that the instrument is perceived to be
at least somewhat accurate across both dimensions, with only one respondent rating the
DRSTOS as “not useful at all” with regard to its accuracy as a description of good teaching and
learning.
CONCLUSIONS
The findings from Phase 1 of the study support the technical quality and utility of the
DRSTOS-R as a reliable and valid assessment of the developing proficiency of pre-service
teachers. More evidence about the technical properties and utility of the DRSTOS-R, including
its predictive validity, is being collected in Phase 2 of the study. Phase 2 will also address a
concern raised in the Phase 1 analysis regarding a statistically significant and meaningful rater
effect by extending the training period for raters and conducting recalibration training sessions.
SIGNIFICANCE OF THE FINDINGS
Valid and reliable assessment of student teachers is crucial to ensure that their programs
are designed to train them well in the skills and knowledge that are essential to high quality
teaching and learning and that they are qualified for induction in the profession. These data are
___________________________________________________________________________ 13
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
needed for accountability but more importantly for institutional self-study aimed at informing
continuous program improvement. While research has focused on performance assessment for
teacher licensing and the assessment of in-service teaching, the methods are often impractical for
use with pre-service teachers and the research results do not necessarily generalize to pre-service
assessment. The data from Phase 1 of this study are promising with respect to the technical
quality and utility of the DRSTOS-R as a tool for our teacher education programs and a potential
model for other institutions.
DIRECTIONS FOR THE FUTURE
We continue to expand the use of DRSTOS-R as a student teacher assessment tool for
research and accountability. Although these findings are useful for stimulating faculty
discussion about the effectiveness of our teacher education programs, I urge caution in their
interpretation. Our programs are complex and the research on the psychometric properties of the
DRSTOS-R is ongoing. We found evidence of a systematic rater effect in the data we have
collected to date. This evidence suggests a tendency for certain raters to give low scores to their
students, even after controlling for the effects of degree program and ordinal placement. We
need to explore this phenomenon further. We are continuing to devise ways to improve the
reliability and validity of the rating process through updates to the protocol, improved training,
and mixed-methods studies of the use of DRSTOS-R in situ.
___________________________________________________________________________ 14
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
Table 1
Results of the Assessment of the Inter-Rater Agreement of the DRSTOS-R
(Spring 2005)
Reliability
Measure
Trial (Video) 1
Trial (Video) 2
Item with Item with Mean for Item with Item with Mean for
highest
lowest
All 11
highest
lowest
All 11
agreement agreement
Items
agreement agreement
Items
Mean InterRater Difference
0.00
1.21
0.39
0.25
0.93
(Scale of 1 – 4)
Percent of Pairs
of Raters with
100%
32.1%
51.0%
75%
32.1%
Exact
Agreement
Percent of Pairs
of Raters within
100%
71.4%
86.4%
100%
75%
One Score Point
of Each Other
Note: Eight raters participated in the training and rated Items 7 – 17
0.39
45.1%
92.9%
Table 2
Internal Consistency Reliability of the DRSTOS-R
Domain
Number of
Number of
Coefficient
Items
Participants
Alpha
Planning & Preparation
6
424
.911
7
427
.905
4
432
.831
3
437
.834
20
397
.964
Classroom Environment
Instruction
Professional
Responsibilities
Total Protocol
(Based on data for student teachers assessed from fall 2004 through fall 2007)
___________________________________________________________________________ 15
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
Table 3
Summary of the Comparisons Between the DRSTOS-R Mean Domain
and Total Scores for the Early and Late Student Teacher Placement Groups
Fall 2004 and Spring 2005
DRSTOS-R Domains
Classroom Instruction
Professional
Total
Environment
Responsibilities
Score
(4 Items)
(7 Items)
(3 Items)
(20 Items)
BS Students
Early
Mean/
2.56
2.64
2.63
2.83
2.64
(N)
(14)
(14)
(14)
(14)
(14)
SD
0.46
0.46
0.56
0.65
0.49
Late
Mean/
3.09
3.18
3.14
3.41
3.18
(N)/
(32)
(32)
(32)
(32)
(32)
SD
0.48
0.46
0.49
0.57
0.45
t-Test/
t
3.40
3.65
3.14
3.02
3.62
Effect
(df)
(44)
(44)
(44)
(44)
(44)
Size(ES)
p
.001
.001
.003
.004
.001
ES
1.07
1.16
0.92
0.88
1.09
MA Students
Early
Mean/
2.61
2.86
2.68
2.96
2.76
(N)/
(17)
(17)
(17)
(17)
(17)
SD
0.60
0.63
0.58
0.62
0.59
Late
Mean/
3.16
3.24
3.08
3.37
3.16
(N)/
(42)
(42)
(42)
(42)
(42)
SD
0.44
0.55
0.48
0.58
0.44
t-Test/
t
3.93
2.34
2.78
2.42
3.04
Effect
(df)
57
57
57
57
57
Size(ES)
p
.000
.023
.007
.019
.004
ES
0.93
0.61
0.70
0.66
0.76
Notes. Scale is (1) Not Yet Proficient (2) Partially Proficient (3) Entry Level Proficient
(4) Proficient. The standard for proficiency is 3.
Placement
Group/
t-Test
Mean/
(N)/
SD
Planning &
Preparation
(6 Items)
___________________________________________________________________________ 16
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
Table 4
Results of T-Tests for Paired Samples Applied to Mean Differences Between
Early and Late Placement DRSTOS-R Scores
Mean
Domain
Mean SD N
Diff.
t
df
p
ES
Undergraduate Students (BS)
Late
3.08 0.59
Planning &
27
0.32
2.45 26 0.02 0.52
Preparation
Early 2.76 0.62
Late
3.25 0.54
Classroom
27
0.40
2.89 26 0.01 0.66
Environment
Early 2.85 0.61
Late
3.09 0.65
Instruction
27
0.36
2.58 26 0.02 0.61
Early 2.73 0.59
Late
3.35 0.62
Professional
27
0.31
1.98 26 0.06 0.45
Responsibilities Early 3.04 0.69
Late
3.18 0.57
Total
27
0.35
2.70 26 0.01 0.61
Early 2.83 0.58
Graduate Students (MA)
Late
3.36 0.44
Planning &
63
0.24
3.06 62 0.00 0.39
Preparation
Early 3.12 0.61
Late
3.45 0.46
Classroom
63
0.26
3.30 62 0.00 0.46
Environment
Early 3.19 0.57
Late
3.39 0.50
Instruction
63
0.28
3.36 62 0.00 0.47
Early 3.11 0.60
Late
3.59 0.48
Professional
62
0.14
1.77 61 0.08 0.23
Responsibilities Early 3.45 0.60
Late
3.42 0.42
Total
63
0.24
3.31 62 0.00 0.43
Early 3.19 0.56
Total Teacher Education Students
Late
3.27 0.50
Planning &
91
0.26
3.92 90 0.00 0.42
Preparation
Early 3.01 0.63
Late
3.39 0.49
Classroom
91
0.31
4.50 90 0.00 0.52
Environment
Early 3.08 0.60
Late
3.30 0.56
Instruction
91
0.30
4.14 90 0.00 0.48
Early 3.00 0.62
Late
3.52 0.53
Professional
90
0.19
2.71 89 0.01 0.30
Responsibilities Early 3.32 0.65
Late
3.35 0.48
Total
91
0.27
4.36 90 0.00 0.47
Early 3.08 0.58
___________________________________________________________________________ 17
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
Figure 1
Scree Plot form the Principal Components Factor Analysis of DRSTOS-R Item Scores for Phase
1 Study Sample
___________________________________________________________________________ 18
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
Table 5
Loadings of DRSTOS-R Items on the Single Latent Factor
Component Matrix
Pedagogical Content
Knowledge of Content
Long/Short Term Planning
Constraints on Teaching & Learning
Criteria and Standards
Feedback, Reflection and use for Planning
Teacher Interaction w/ Pupils
Classroom Interaction
Functioning of Learning Groups
Transitions
Materials and Supplies
Mutual Expectations
Awareness of Pupil Behavior
Clarity of Goals
Knowledge of Students
Teacher/ Pupil Communications
Discussion Patterns
Relationships w/ Adults
Cultural Context of School & Community
Use in Future Teaching
a
Compone
nt
1
.837
.793
.819
.798
.750
.754
.809
.685
.768
.765
.702
.785
.794
.806
.813
.712
.746
.746
.759
.796
Extraction Method: Principal Component Analysis.
a. 1 components extracted.
___________________________________________________________________________ 19
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
Table 6
Perceived Usefulness of the DRSTOS-R
N
Very
Useful
Somewhat
Useful
Not Useful
at All
Don’t
Know
Facilitating Discussion with Student Teachers
21
23.8%
57.1%
0%
19%
Establishing Goals for Student Teachers
21
33.3%
42.9%
0%
23.8%
Creating Common Language
21
19.0%
66.7%
0%
14.3%
Providing a Comprehensive Overview of Student
Teachers’ Performance
Assessing Student Teachers’ Strengths &
Weaknesses
19
42.1%
47.4%
0%
10.5%
21
47.6%
42.9%
0%
9.5%
Overall Usefulness
21
28.6%
71.4%
0%
0%
Table7
Perceived Accuracy of the DRSTOS-R
N
Very
Accurate
Somewhat
Accurate
Not Accurate
at All
Description of Good Teaching & Learning
20
30.0%
65.0%
5.0%
Reflection of Student Teachers’ Professional Development
21
33.3%
66.7%
0%
___________________________________________________________________________ 20
CRTL Paper Presentation Series RP-1109-01
Are We Developing High Quality Teachers and Can We Prove It?
REFERENCES
Blanton, L.P., Sindelar, P.T., & Correa, V.I. (2006). Models and measures of beginning teacher
quality. Journal of Special Education, 40(2), 115-127.
Danielson, C. (2007). Enhancing Professional Practice: A Framework for Teaching 2nd Edition.
Association for Supervision and Curriculum Development. Alexandria, VA. pp. vii-viii.
Darling-Hammond, L. and Bransford, J. (2005). Preparing Teachers for a Changing
World: What Teachers Should Learn and Be Able to Do. San Francisco, CA:
Jossey- Bass.
Doyle, W. (1977). Paradigms for Research on Teacher Effectiveness. Review of Research in
Education Vol.5., pp. 163-198.
Dwyer, C.A. (1998). Psychometrics of Praxis III: Classroom performance assessments. Journal
of Personnel Evaluation in Education, 12(2), 163-187.
Goldrick, L. (2002). Issue Brief: Improving teacher evaluation to improve teaching quality.
National Governors' Association, Washington, DC. Center for Best Practices. Retrieved
June 29, 2008 from National Governor’s Association
http://www.nga.org/cda/files/1202IMPROVINGTEACHEVAL.pdf
Grossman, P. & McDonald, M. (2008). Back to the future: Directions for research in teaching
and teacher education. American Educational Research Journal, 45(1), 184-205.
Limback, R.E. & Mansfield, J.B. (2002). Student teacher assessment: A multidimensional
process. The Delta Pi Epsilon Journal, 44(I), 50-58.
Mujis, D. (2006). Measuring teacher effectiveness: Some methodological reflections.
Educational Research and Evaluation 12(1), pp. 53-74.
Stodolsky, S.S. (1984). Teacher evaluation: The limits of looking. Educational Researcher,
13(9), 11-18.
___________________________________________________________________________ 21
CRTL Paper Presentation Series RP-1109-01
Appendix: DRSTOS-R Protocol
Student teacher Name: ___________________________________________
Supervised by: ______________________________
Semester: ____________________________
Student Teacher & Placement Information
Please check one:
€ Junior
Major/Program(s):
€ Senior
Undergraduate
€ Regular Track
Graduate
_____________________________________
Certification track?
€ Yes
€ No
Native English Speaker?
€ Yes
€ No
Placement
(check one)
Placement
Information
€ Fast Track
€
€
€
€
1
2
3
4
out
out
out
out
of
of
of
of
4
4
4
4
€
€
1 out of 2
2 out of 2
* Early Childhood Majors Only
€ 1 out of 3
€ 2 out of 3
€ 3 out of 3
€
General Education
€
0 - 25% English Language Learners
Grade(s) ______
€
Self-Contained Special Education
€
26 - 50% English Language Learners
Content/Specialty Area (if applicable)
€
CTT
€
51%+ English Language Learners
School Name/PS #
Additional Notes on Placement
(ex: push-in, pull-out, SETTS/Resource Room)
________________________________
____________________________________________________
Cooperating Teacher
____________________________________________________
Last __________________________
____________________________________________________
First __________________________
4/1/2011
22
Inspired by Danielson, C. (1996). Enhancing Professional Practice: A Framework for Teaching. Alexandria, Virginia: Association for Supervision & Curriculum Development.
NOT FOR REPRODUCTION.
DRSTOS-R
Appendix: DRSTOS-R Protocol
Student teacher Name: ___________________________________________
Supervised by: ______________________________
Semester: ____________________________
LEVEL OF PERFORMANCE
ELEMENT
NOT YET PROFICIENT
PARTIALLY PROFICIENT
ENTRY LEVEL
PROFICIENT
PROFICIENT
EVIDENCE
PLANNING AND PREPARATION
1. PEDAGOGICAL
CONTENT
KNOWLEDGE
Student teacher displays
inadequate understanding of
pedagogical issues involved in
pupil learning of the content.
1
2. KNOWLEDGE OF
CONTENT
STANDARDS
Student teacher displays
inadequate evidence of
familiarity with content
standards.
1
3. LONG/SHORT
TERM PLANNING
Planning for instruction is not
connected to longer-term
goals or to the pedagogical
content knowledge of the
subject, the pupils, or the
standards, and are unclear to
most pupils in the class.
1
Student teacher displays
basic content knowledge
but does not articulate
connections among
content, pedagogy, and
pupil development.
2
Student teacher displays
basic knowledge of
content standards, without
evidence of connecting to
standards beyond the
current lesson.
2
Planning for instruction is
partially connected to
longer-term goals and
there is limited use of
pedagogical content
knowledge of the subject,
the pupils, or the
standards.
2
Student teacher displays
sufficient content
knowledge but does not
sufficiently articulate
connections among
content, pedagogy, and
pupil development.
3
Student teacher displays
a sufficient understanding
of the city/state content
standards and makes
connections to other
standards within and/or
beyond content area.
Pedagogical practices
reflect current research on
best pedagogical practice
within the discipline and
the anticipation of potential
pupil misconceptions.
Student teacher makes
connections with or to
other disciplines.
4
Student teacher displays a
strong understanding of
the city/state content
standards and makes
connections to other
standards within and/or
beyond content area.
3
4
Planning for instruction
connects to longer-term
goals and sufficiently uses
pedagogical content
knowledge of the subject,
the pupils, or the
standards.
Planning for instruction
connects to longer-term
goals and effectively uses
pedagogical content
knowledge of the subject,
the pupils, or the
standards.
3
4
4/1/2011
23
Inspired by Danielson, C. (1996). Enhancing Professional Practice: A Framework for Teaching. Alexandria, Virginia: Association for Supervision & Curriculum Development.
NOT FOR REPRODUCTION.
DRSTOS-R
Appendix: DRSTOS-R Protocol
Student teacher Name: ___________________________________________
Supervised by: ______________________________
Semester: ____________________________
LEVEL OF PERFORMANCE
ELEMENT
4. CONSTRAINTS ON
TEACHING AND
LEARNING
5. CRITERIA AND
STANDARDS
PARTIALLY PROFICIENT
Student teacher plans and
teaches without regard to the
particular possibilities and
limits of his/her classroom
context.
Student teacher
understands some of the
curricular and resource
possibilities and
constraints of the context
but does not effectively
use them in planning or
teaching.
Student teacher
sufficiently understands
the curricular and
resource possibilities and
constraints of the context
and begins to use them in
planning or teaching.
1
2
3
The proposed approach
contains no clear criteria or
standards.
1
6. FEEDBACK,
REFLECTION AND
USE FOR
PLANNING
ENTRY LEVEL
PROFICIENT
NOT YET PROFICIENT
Information from assessments
(tests, observations,
conferences, etc.) affects
planning for these pupils only
minimally.
1
Assessment criteria and
standards are unclear.
2
Student teacher uses
assessment results to
plan for the class as a
whole.
2
Assessment criteria and
standards are generally
appropriate and
sufficiently clear.
3
Student teacher uses
assessment results to
plan for individuals and
groups of pupils as well as
the class as a whole.
3
PROFICIENT
EVIDENCE
Student teacher
thoroughly understands
the curricular and resource
possibilities and
constraints of the context
and uses them effectively
in planning or teaching.
4
Assessment criteria and
standards are well
developed and explicit.
4
Student teacher uses
assessment results to plan
for individuals and groups
of pupils as well as the
class as a whole and uses
pupil input in assessment
planning.
4
4/1/2011
24
Inspired by Danielson, C. (1996). Enhancing Professional Practice: A Framework for Teaching. Alexandria, Virginia: Association for Supervision & Curriculum Development.
NOT FOR REPRODUCTION.
DRSTOS-R
Appendix: DRSTOS-R Protocol
Student teacher Name: ___________________________________________
Supervised by: ______________________________
Semester: ____________________________
CLASSROOM ENVIRONMENT
7. STUDENT
TEACHER
INTERACTION
WITH PUPILS
Student teacher’s voice
controls the classroom
environment. Student voice
needs to be nurtured and
validated.
1
8. CLASSROOM
INTERACTION
Classroom interactions are
frequently characterized by
conflict, sarcasm, or putdowns.
1
9. FUNCTIONING OF
LEARNING
GROUPS
Pupils not working with the
student teacher are not
productively engaged in the
task(s).
1
Student teacher is
beginning to elicit student
voices in the classroom
environment.
2
Classroom interactions
are occasionally
characterized by conflict,
sarcasm, or put-downs.
Student teacher regularly
provides students with a
venue to make their
voices heard.
3
Classroom interactions
are generally polite and
mutually respectful.
2
3
Tasks for group work are
partially organized,
resulting in some off-task
behavior when student
teacher is involved with
one group.
Tasks for group work are
organized, and groups are
managed so most pupils
are engaged most of the
time.
2
3
The classroom
environment reflects a
blend of student teacher
voice and student voice.
Student voice is nurtured
and encouraged,
4
Classroom functions as a
genuinely polite, caring
and mutually respectful
community.
4
Tasks for group work are
well organized, and
groups are managed so
most pupils are engaged
at all times.
4
4/1/2011
25
Inspired by Danielson, C. (1996). Enhancing Professional Practice: A Framework for Teaching. Alexandria, Virginia: Association for Supervision & Curriculum Development.
NOT FOR REPRODUCTION.
DRSTOS-R
Appendix: DRSTOS-R Protocol
Student teacher Name: ___________________________________________
Supervised by: ______________________________
10. TRANSITIONS
Much time is lost during
transitions.
1
11. MATERIALS AND
SUPPLIES
Materials are handled
inefficiently, resulting in
significant loss of instructional
time.
1
12.
MUTUAL
EXPECTATIONS
No standards of conduct
appear to have been
established, or pupils are
confused as to what the
standards are.
1
13.
AWARENESS OF
PUPIL BEHAVIOR
Pupil behavior is not
monitored, and student
teacher is unaware of what
pupils are doing.
1
Transitions are
sporadically efficient,
resulting in some loss of
instructional time.
2
Transitions mostly occur
smoothly, with minimal
loss of instructional time.
Semester: ____________________________
Transitions occur
smoothly, with almost no
loss of instructional time.
3
4
Routines for handling
materials and supplies are
mostly efficient, with
minimal disruption of
instruction.
Routines for handling
materials and supplies are
consistently efficient.
2
3
4
Standards of conduct
appear to have been
established for most
situations, and most
pupils seem to understand
them.
Standards of conduct are
clear to all pupils.
2
3
4
Student teacher is
generally aware of pupil
behavior but may miss the
activities of some pupils.
Student teacher is alert to
pupil behavior at all times.
Student teacher is alert to
pupil behavior at all times
and pupils participate in
the monitoring process.
2
3
4
Routines for handling
materials and supplies are
sporadically efficient,
resulting in some
disruption of instruction.
Standards of conduct are
clear to all pupils, and
there is evidence of some
student participation in
their formulation.
4/1/2011
26
Inspired by Danielson, C. (1996). Enhancing Professional Practice: A Framework for Teaching. Alexandria, Virginia: Association for Supervision & Curriculum Development.
NOT FOR REPRODUCTION.
DRSTOS-R
Appendix: DRSTOS-R Protocol
Student teacher Name: ___________________________________________
Supervised by: ______________________________
Semester: ____________________________
INSTRUCTION
Goals are inappropriately
selected and are not suitable
for most pupils.
Goals are appropriately
selected and partially
suitable for most pupils.
14. CLARITY OF GOALS
1
15. KNOWLEDGE OF
STUDENTS:
PUPILS’ SKILLS ,
CULTURAL
HERITAGE,
KNOWLEDGE,
INTERESTS,
LEARNING STYLES
INSTRUCTIONAL
NEEDS
16. STUDENT TEACHER/
PUPIL
COMMUNICATIONS
Goals are sufficiently
selected in their content
and level of expectations
and are suitable for most
pupils in the class.
Goals are highly sufficient
in their selection of
content and level of
expectations and are
suitable for most pupils in
the class.
2
3
4
Student teacher
recognizes the value of
understanding pupils’
skills, knowledge and
learning styles, but
displays this knowledge
for the class only as a
whole and rarely for those
with special needs.
Student teacher
demonstrates a sufficient
knowledge of pupils’ skills,
knowledge and learning
styles for groups of pupils
including those with
special needs and
recognizes the value of
this knowledge.
Student teacher
demonstrates a strong
knowledge of pupils’ skills,
knowledge and learning
styles for groups of pupils
and recognizes the value
of this knowledge
including those with
special needs.
Student teacher’s or pupils’
spoken language is inaudible,
or written language is illegible.
Spoken or written language
may contain many grammar
and syntax errors. Vocabulary
may be inappropriate, vague,
or used incorrectly, leaving
pupils confused.
2
Student teacher’s or
pupils’ spoken language is
audible, and written
language is legible. Both
are used correctly.
Student teacher
vocabulary is correct but
limited or is not
appropriate to pupils’ ages
or backgrounds.
3
Student teacher’s and
pupils’ spoken and written
language are sufficiently
clear and appropriate to
pupils’ age and interests.
4
Student teacher’s spoken
and written language is
clear, correct, and
enhances the learning of
the subject. Pupils are
mastering the standard
written language as
writers and readers.
1
2
3
4
Student teacher demonstrates
an inadequate knowledge of
pupils’ skills, knowledge and
learning styles, and does not
indicate that such knowledge is
valuable.
1
4/1/2011
27
Inspired by Danielson, C. (1996). Enhancing Professional Practice: A Framework for Teaching. Alexandria, Virginia: Association for Supervision & Curriculum Development.
NOT FOR REPRODUCTION.
DRSTOS-R
Appendix: DRSTOS-R Protocol
Student teacher Name: ___________________________________________
Supervised by: ______________________________
17.
DISCUSSION
PATTERNS
Interaction between student
teacher and pupils is
predominantly recitation style,
with student teacher mediating
all questions and answers.
Student teacher attempts
to engage pupils in
discussion, with uneven
results.
1
2
Most classroom
interaction represents
discussion, with student
teacher taking a
facilitating role.
Semester: ____________________________
Classroom interaction
represents discussion,
with student teacher
stepping, when
appropriate, to the side so
pupil-pupil talk dominates.
3
4
Support and cooperation
characterize relationships
with others.
Student teacher is able to
maintain positive
relationships with adults
and functions effectively
as part of a team.
2
3
4
Student teacher
demonstrates knowledge
of the cultural context of
the school and the
community.
Student teacher
demonstrates sufficient
knowledge of the cultural
context of the school and
the community.
1
2
3
Student teacher has no
suggestions for how a lesson
may be improved another time.
Student teacher makes
general suggestions about
how a lesson may be
improved.
Student teacher
demonstrates an
expanding knowledge of
the cultural context of the
school and the
community.
4
Student teacher is a
reflective practitioner, is
able to learn from
mistakes and successes
and adjusts accordingly.
1
2
PROFESSIONAL RESPONSIBILITIES
18. RELATIONSHIPS
WITH ADULTS:
SUPERVISOR,
COOPERATING
TEACHER,
TEACHERS,
SCHOOL STAFF, &
PARENTS/
GUARDIANS.
19. CULTURAL
CONTEXT OF
SCHOOL AND
COMMUNITY
20.
Student teacher’s relationships
with adults are negative or selfserving.
1
Student teacher appears to be
unaware of the cultural context
of the school and community.
ABILITY TO
Student teacher maintains
cordial relationships with
adults.
REFLECT
Student teacher is
becoming a reflective
practitioner and makes a
few specific suggestions
of what might be tried if
the lesson was taught
again.
3
4
4/1/2011
28
Inspired by Danielson, C. (1996). Enhancing Professional Practice: A Framework for Teaching. Alexandria, Virginia: Association for Supervision & Curriculum Development.
NOT FOR REPRODUCTION.
DRSTOS-R
Appendix: DRSTOS-R Protocol
Student teacher Name: ___________________________________________
Supervised by: ______________________________
Semester: ____________________________
OTHER COMMENTS
PLANNING AND PREPARATION
CLASSROOM ENVIRONMENT
INSTRUCTION
PROFESSIONAL RESPONSIBILITIES
4/1/2011
29
Inspired by Danielson, C. (1996). Enhancing Professional Practice: A Framework for Teaching. Alexandria, Virginia: Association for Supervision & Curriculum Development.
NOT FOR REPRODUCTION.
DRSTOS-R