Roundtable Session
Title: Assessment of teachers’ differentiation skills: the construction of an observation scale
General abstract of the presentation (max. 150 words)
presenting the outline of your (planned) research project or study and including the problem(s),
question(s), dilemma’s, and/or challenge(s) that you wish to discuss during your roundtable session
Recently, the call for differentiated instruction has increased. But how can assessment of differentiated
teaching take place? In this project, an observation scale for differentiated teaching is used in three small
case studies. The contribution will highlight both the interesting findings from and obscurities of this
instrument. For example, the instrument led to ambiguous results in assessing the effects of a course in
differentiation, yet the participants of the course experienced the scale as an eye-opener. Furthermore, the
frequency of differentiation activities was not related to the quality of the activities. In all, observing and
coding differentiated teaching may lead to new insights in what differentiated instruction is really about.
Detailed abstract
1.
What is the background of your (planned) research project, study or general
problem that you wish to discuss? More specifically, how is your study or problem
founded by theory and/or how does it originate from practice? (max. 200 words)
Every school class has a diverse student population. There is no ‘one-size fits all’ formula available for
any class. At its most basic level, differentiated teaching consists of the efforts of teachers to respond
to differences among learners in a classroom (Tomlinson, 2000; 2005a). Recently, the call for the
modification of teaching methods, curricula, and resources to match students’ learning
characteristics more closely has increased . Many (practical) books or websites have appeared that
offer teachers advice on how to differentiate instruction (Strickland, 2009; Tomlinson, 2005a; Van
Tassel-Baska et al., 2008). Yet differentiated instruction is not done as a matter of course. Van de
Grift perceives differentiated instruction as a higher level instructional strategy; and this idea is
confirmed in the observational data in representative samples of primary schools in six different
countries (Van de Grift, et al., 2011). In our study, a course was constructed to fulfil the
differentiation needs of secondary school teachers and train their differentiation skills. But what
happens inside the classroom: is assessment of differentiated instruction possible? Can the effects of
the course be made visible? This contribution describes the construction of an observation scale,
which will be examined within three small case studies.
2.
Describe the general outline of your (planned) research project or study, and/or
problem that you wish to discuss as much as possible, i.e. the research questions,
research design, methods, timeline,... (max. 300 words)
The observation scale is based on interviews, two observation instruments (Van de Grift, 2007; Van TasselBaska et al., 2008), and differentiation rubrics of Strickland (2009). Observers rate both frequency and
quality of activities (high-inference event sampling instrument). The instrument includes five subscales.
Each scale is specified by “good practice examples”. In all, there are 20 “good practice examples” that are
coded on a three-point frequency scale. The activity is qualitatively judged using rubrics (1=Novice;
2=Apprentice; 3=Practitioner; 4=Expert). The scale will be used in three studies.
The first study examined whether a training in differentiation skills led to an increase in differentiated
teaching. Four teachers were observed before and after taking the course. The Assessment subscale was
hardly observed. Activities were mostly labelled at “Apprentice” level. By comparing the first and second
measurements, seven increases and three decreases in level were counted. The frequency part of the
instrument showed more increases than decreases in differentiation activities. The frequency of the
activities appears not to be related to the quality of the activities.
The second study replicated the first. Eight teachers participated. Again, the comparison of the two
measurements yielded ambiguous results. There were 13 increases and 8 decreases in Quality. The
increase from “Practitioner” level to “Expert” level was never scored. In the Frequency part, 35 increases
and 17 decreases were counted. Both in studies 1 and 2, the observation scale was experienced as a
powerful instructional tool.
In the third study, 4 teachers are observed twice. These teachers work at different schools and do not
participate in an experimental course. New raters will score the observations. Our aim is to test whether
the instrument that was constructed at one specific school is suited for other schools; and whether
observation scores are more stable when teachers and observers are not trained in differentiation skills.
3.
How is practice involved in your (planned) research project or study, and which kind
of dissemination, implementation or other ways of sharing your research output are
you thinking of in order to contribute to the improvement of educational practice
and learning? (max. 150 words)
Examining the observation scale is just a small part of a larger research project. Two professional
teachers design and teach a course in differentiated teaching. Additionally, they examine the effect
of this course on teachers’ professional development in differentiated teaching (reference removed
for review) . The school intends to present this course to (beginning and experienced) teachers in
order to support them in their differentiation needs. Obviously, the course must be evaluated with
appropriate instruments. This design research project is mainly school-bound, but the school joins a
research consortium to become an academically oriented instruction school. Teachers from four
secondary schools form a community of teacher researchers and discuss and disseminate their
research outcomes. Outside the community, there is also a interest in schools in how to adapt
instruction to individual learners’ needs. So, the teacher researchers are asked to give workshops at
various secondary schools.
4.
Which question(s), problem(s) or challenge(s) would you like to present for
discussion to the conference delegates? Limit yourself to 2 or 3
questions/problems/challenges, in order to be able to discuss them in-depth.
1) The results found with the observation scale were somewhat ambiguous (for example both increases
and decreases in differentiation activities). Can the observation scale be used as a reliable evaluation
instrument or should it solely be used as an instructional tool ?
2) The frequency of the differentiation activities turned out not to relate to the quality of these activities.
Should the instrument be adjusted to match frequencies and quality in more congruent ways?
3) The instrument is quite complex to score. Can the instrument also be used by relative layman in the area
of differentiated teaching?
5.
List of references
Strickland, C.A. (2009), Professional Development for Differentiating Instruction, ASCD
Publication, Alexandria, U.S.A.
Tomlinson, C.A. (2000). Differentiation of Instruction in the Elementary Grades. ERIC Digest;
EDO-PS-00-7.
Tomlinson , C.A. (2005a). Grading and differentiation: Paradox or good practice? Theory
into Practice, 44, 262-269.
Van de Grift, W.J.C.M. (2007). Quality of teaching in four European countries: a review of the
literature and application of an assessment instrument. Educational Research 49, 127152.
Van de Grift, W.J.C.M., Van der Wal, M. & Torenbeek, M. (2011). Ontwikkeling in de
pedagogisch didactische vaardigheid van leraren in het basisonderwijs. [Development in
the pedagogical and didactic skills of teachers in primary education]. Pedagogische
Studiën, 88, 416-432.
Van Tassel-Baska, J., Xuemei Feng, J., Brown, E., Bracken, B., Stambaugh, T., French, H.,
McGowan, S., Worley, B, Quek, C. & Bai, W. (2008). A study of differentiated
instructional change over 3 years. Gifted Child Quarterly, 52, 297-312.
Please upload this document, which will serve as your official proposal,
as an attachment to your submission in the EAPRIL Conference System.