4
Evidence-based learning for students and teachers
Jessica Thompson, Melissa Braaten, Mark Windschitl, Bethany Sjoberg, Margaret Jones, and Kristi Martinez
cross the nation, teachers gather regularly to examine student work and uncover how their students
are thinking about science. These teacher groups
aim to improve both their instructional methods and
the effect these methods have on student learning. Achieving positive outcomes, however, requires teachers to use
thoughtful, intentional methods to design instruction, select
student work, analyze these artifacts, and make evidencebased changes to instruction (Bray et al. 2000; NRC 1996;
Wood 2007).
48
The Science Teacher
This article presents a model of collaborative inquiry
for groups of science teachers who want to systematically
improve their practice through analyses of student work.
The five-phase APEX ST (Advancing High-Leverage
Practices by Examining Student Thinking) model
(Figure 1) is appropriate for students of all achievement
levels. It focuses on high-leverage practices (e.g., pressing
for evidence-based explanations) and longitudinal
learning for both students and teachers over the course
of a year.
C r i t i c a l F r i e n d s G ro u p s
We have designed and tested the APEXST model of collaborative inquiry for Critical Friends
Groups (CFGs). A CFG is a learning community of 8–12 educators who gather for about two
hours a month to discuss improving their practice through collaborative learning. In collegial
CFG meetings, teachers engage in a cycle of
inquiry, reflection, and action to promote adult
growth that is directly linked to student learning
(Curry 2008; NSRF 2009). The APEXST model
provides a structure and focus for these groups
to engage in meaningful examination of student
work. There are five phases of the APEXST model that support the improvement of teaching and
student learning. These are described in the following sections.
Figure 1
APEXST model of collaborative inquiry.
1. Collaboratively
defining a vision of
worthwhile learning.
5. Enacting change and
reporting back at next
meeting.
2. Teaching lessons and
collecting samples of
students’ written work.
4. Linking trends in
student thinking with
opportunities to learn.
3. Analyzing student work
and uncovering trends
and patterns of thinking.
Phase 1: Defining a vision of
wo r t h w h i l e l e a r n i n g
Perhaps the most important decision CFGs can make is to
identify some aspect of student learning that is important
enough to focus on for a full academic year (Ball et al. 2009;
Curry 2008; Windschitl, Thompson and Braaten Forthcoming). We recommend teacher groups choose a core
scientific practice they can develop across different topics
and courses. In our most recent project, we chose students’
“construction of evidence-based explanations.” This type
of scientific thinking is critical to understanding the more
conceptual ideas in science and is a valued scientific practice
(NRC 2000; Windschitl 2008). It is important that CFGs do
not choose topics that can change from meeting to meeting—this will make it difficult to analyze any one scientific
practice’s development over time.
Creating lessons that press students for evidence-based
explanations can be difficult work. The challenge is that
lessons from standard curriculum materials are often
organized around narrow topics or processes, not big ideas
with underlying explanations. Big ideas have complex causal
stories, composed of a web of events and concepts that help
explain why observable phenomena occur. An example of one
observable phenomenon is the diffusion of materials across a
membrane. The causal explanation for why this occurs has
to do with core concepts such as equilibrium, concentrations
of solutions, and permeability of membranes.
Once a big idea is selected, teachers can work together
to identify a lesson or series of lessons that aim to explain
observable phenomena. Lessons that ask students to gather
information through investigation or already existing
evidence help them understand both the development of an
explanation and the use of evidence. One such example would
be a pulley investigation in which students use ideas about
forces, work, and energy to explain why a single person can
lift a very heavy load using simple machines.
“What-how-why” explanation
After the teachers have selected a lesson, the next step is to
construct a full explanation for the phenomena (i.e., a causal
story or stories describing why a phenomenon occurs). To
connect this explanation to evidence from an investigation,
it is helpful to first draw a diagram of the phenomenon
under study and then draw in features and processes that
are not observable—thus creating a scientific model for the
phenomenon (Windschitl 2008; Windschitl, Thompson,
and Braaten 2008).
Once the “why” is outlined, the team of teachers can then
write a rubric that details a “how” and “what” explanation that
students might provide when asked for a “why” explanation
(see “On the web”). Figure 2 (p. 50) is an example of the “whathow-why” explanation framework that a group of teachers
developed for a cellular respiration investigation in which
students were asked to explain why respiration increased
after exercise. Students breathed into a Bromothymol Blue
(BTB) solution before and after exercising. (Safety note:
Students were strictly supervised and cautioned not to
accidentally inhale or swallow the solution.)
In the example provided in Figure 2, students
were asked to use multiple forms of evidence and to
coordinate the evidence with a scientific explanation. They
collected three types of data before and after exercising:
heart rate, number of breaths per minute, and amount of
time (in seconds) it takes for BTB to change from blue to
yellow. After collecting the data, students used the “whathow-why” framework to analyze their results. The teachers
November 2009
49
Figure 2
“What-how-why” explanation.
Level 1: What
Depth of
explanation
u
u
Example
explanation
Student describes
what happened.
Student describes,
summarizes, or
restates a pattern or
trend in data without
making a connection
to any unobservable
or theoretical
components.
The BTB changed from
blue to yellow after
the exercise because
the body exhaled more
carbon dioxide than
when it was stationary.
Level 2: How
u
u
Student describes
how or partially why
something happened.
Student addresses
unobservable or
theoretical components tangentially.
When exercising, the
body requires more
oxygen. As oxygen
intake increases, so
does carbon dioxide
output.
used the rubric to design questions for students and evaluate
their written work.
P h a s e 2 : C o l l e c t i n g s t u d e n t wo r k
samples
A key feature of the APEXST model is that attention is
given to students of all achievement levels. We recommend
selecting nine students to track throughout the school
year—three high-performing students, three average students, and three underperforming students. The amount of
student work you analyze can be small (e.g., a 3–5 sentence
response to an explanation-type question asked on a quiz)
or large (e.g., an evidence-based claim students made following an investigation).
Many of the teachers we worked with found it helpful to
collect video samples as a way to share images of rich classroom
conversations. Recording and editing video samples can be
time consuming, but if available, this can be done by a districtlevel science coach, an administrator, or a research partner
from a university. For video segments, try to capture studentto-student conversation as a way to provide insight into how
students wrestle with evidence or scientific explanations.
50
The Science Teacher
Level 3: Why
u
u
u
Student explains why something
happened.
Student can trace a full causal story
for why a phenomenon occurred.
Student uses powerful science ideas
that have observable or theoretical
components (e.g., kinetic molecular
theory) to explain observable event.
When exercising, the body requires more
oxygen, which is taken from the lungs to
muscle cells (via the circulatory system
and diffusion). The cells use the oxygen
to break down glucose into energy and
carbon dioxide. Muscles use the energy
to do work. The carbon dioxide diffuses
into the blood and then the lungs and is
exhaled. Cellular respiration happens at
a faster rate when a person is exercising,
so more carbon dioxide is produced.
The exhaled carbon dioxide reacts with
water to produce carbonic acid, causing
the BTB indicator to change color.
P h a s e 3 : A n a l y z i n g s t u d e n t wo r k
Most assessments will tell you if students have learned, but
close examinations of student work can reveal why some
students have learned something and others have not. The
analysis phase of the APEXST model is usually completed
individually by teachers. The first step is to use the rubric
(see “On the web”) you developed in Phase 1 (p. 49) as a
reference to detect what “partial understandings” students
might have compared to the intended understanding.
This provides clues about different paths to understanding, rather than simply making judgments about whether
students’ responses were correct or incorrect.
Partial understandings by students may take several
forms:
uu
uu
uu
uu
Student is familiar with a scientific idea, but uses it in the
wrong context.
Student understands part, but not all, of an idea.
Student recognizes when a vocabulary term should be
used, but does not provide evidence that he or she understands what it means.
Student can go through the motions of a scientific prac-
Examining Student Work
tice (e.g., interpreting data or creating a model), but does
not know when or how to apply this practice in a new
situation.
Another important aspect of the analysis phase is to look
carefully at the full range of high-performing, average, and
underperforming students in your classroom. This will
provide a foundation for addressing how different groups
of students are learning. One of the teachers working with
us found that her underperforming students were having
difficulty writing scientific conclusions—not because they
comprehended the task differently from their peers, but
because they did not know how to respond to feedback
the teacher had given them a few weeks earlier on this
type of writing. The teacher realized she had to help her
underperforming students not only understand scientific
writing practices, but also how to use her feedback more
productively.
When analyzing student work, sticky notes and
highlighters can be used to mark places where students
show elements of understanding the target idea. Students
who have similar responses to one another can be grouped
together, and teachers can then look for one of two types of
patterns in student thinking:
uu
uu
An unexpected trend is when many students responded to
instruction in an unpredicted way. It signals a learning situation that did not present students with an opportunity
to process ideas in the way a teacher thought it would.
A relationship is when groups of students, who share similar characteristics, perform similarly on a feature of the
task or question. In one of our cases, the English Language Learners (ELLs) in a classroom were able to draw
sophisticated models of air pressure during small-group
work, but during whole-class discussions, they were unable to communicate their thinking to others.
P h a s e 4 : L i n k i n g t re n d s w i t h
opportunities to learn
When CFG meetings occur, the following materials are
needed:
uu
uu
Student work: At each meeting, one teacher serves as the
presenter. The presenter should bring copies of three to
four students’ work that are particularly illustrative of
the patterns he or she noted. The assignment questions
that were analyzed should be circled and any necessary
notes included. There should be one set of papers for
each group member.
Rubric (see “On the web”): The presenter should bring
copies of the rubric that the group constructed in the first
meeting and notes about all nine students’ thinking.
uu
Patterns and questions: The presenter should bring written reflections on his or her analysis,
summarizing both the patterns
seen in the data and the questions
peers should focus on during the
session (see “On the web”).
Keywords: Research on
student achievement
at www.scilinks.org
Enter code: TST110902
In the APEXST model of collaborative inquiry, the
CFG meeting should be characterized by three kinds of
accountability: accountability to peers, to the science itself,
and to understanding student work. Being accountable to
peers means each presenter collects and analyzes student
work, makes copies for everyone, and is ready for an in-depth
discussion (typically at least 50 minutes). For other participants,
being accountable means reserving the time to engage in this
process with others and using constructive criticism.
The second type of accountability is to the science itself.
At each meeting, everyone is asked to present a scientific
explanation of the phenomenon present in the student work
sample. This helps participants to see certain indications
of students’ scientific understanding in their work. It also
forms the basis of a common language that can be used
during the session.
The third type of accountability is to understanding the
student work. During the meeting, a segment of time is
devoted to a generous review of student work. Teachers
look for important clues in every paper—no matter how
sparse—that provide insight into the workings of a student’s
mind. Participants need to resist glossing over a sample of
work, only to dismiss it as “wrong,” and should look not only
within the work of each student, but also across the samples
of student work.
During these meetings, it is helpful to avoid “repair
talk” (i.e., talk of how one would fix the instructional
activity). This talk can be contagious; a skilled facilitator
should instead steer participants toward an understanding
of student thinking.
The rubric and the CFG protocol (see “On the web”)
are tools that are especially important for maintaining
accountability. The facilitator should bring copies of the
protocol for all members so it can be used to structure the
peer conversations that take place. The three most important
parts of this science-specific CFG protocol, based on our
research, are:
1. the invitation to come to a group understanding of
the best possible scientific explanation of the focal
phenomenon;
2. the prompt to have participants seek out evidence of partial understandings; and
3. the practice of hypothesizing how, based on evidence,
changes in instruction could positively impact learning.
November 2009
51
Examining Student Work
Phase 5: Enacting change and
re p o r t i n g b a c k
For the APEXST model of collaborative inquiry, groups
will want to leave about 30–60 minutes at the end of
each CFG meeting to return to Phase 1 (p. 49). After the
presentations, the strategies that were discussed are labeled and recorded. Each teacher can choose one idea to
try prior to the next meeting. Teachers can then work in
pairs to discuss exactly which practices might be applied
to an upcoming lesson and work through the activities
listed in Phase 1.
Conclusion
We have worked with CFGs who meet once a month and
others who have negotiated three full professional development days per year for these meetings. Regardless of
structure, most teachers made significant gains in how
they conceptualized which ideas were worth teaching,
how they posed “why”-level questions in the classroom,
and how they identified and responded to students’ partial
scientific understandings. Through collaborative inquiry,
teachers were able to greatly improve students’ opportunities to engage in rich forms of scientific reasoning. Over
time, teachers formed a community of “critical colleagues”
who were willing to hold each other accountable, take intellectual risks, and open up windows into each other’s
classrooms.
The principled and collaborative analysis of one’s practice
for the purposes of improvement is the work of professionals.
This inquiry itself is scientific—it involves the generation
of questions, the production and collection of evidence, and
the coconstruction of theories about how and why students
respond to instruction in particular ways. Through the
APEXST model of collaborative inquiry, it is entirely possible
for committed groups of science educators to understand
their students’ thinking in new and deeper ways, and to
eventually make evidence-based changes that bring science
achievement within the grasp of all students. n
Jessica Thompson ([email protected]) is a research
associate, Melissa Braaten ([email protected]) is a
research assistant, and Mark Windschitl ([email protected].
edu) is an associate professor, all at the University of Washington
in Seattle; Bethany Sjoberg ([email protected]) is a
science teacher at Evergreen High School in Seattle, Washington;
Margaret Jones ([email protected]) is a science teacher
at Rainier Beach High School in Seattle, Washington; and Kristi
Martinez ([email protected]) is a science teacher at Eastlake
High School in Sammamish, Washington.
Acknowledgments
This material is based upon work supported by the National Ed-
52
The Science Teacher
ucational Association Foundation, the Teachers for a New Era
Project sponsored by the Carnegie Corporation, the Annenberg
Foundation, the Rockefeller Foundation, and the National Science Foundation under grant no. DRL-0822016. Any opinions,
findings, and conclusions or recommendations expressed in this
material are those of the authors and do not necessarily reflect the
views of these organizations.
On the web
Analysis of student work record sheet: www.nsta.org/highschool/
connections.aspx
Critical Friends Group protocol: www.nsta.org/highschool/
connections.aspx
Rubric for analysis of student understanding of evidence-based
explanations: www.nsta.org/highschool/connections.aspx
Tips for identifying patterns in the data and devising a question
for a follow-up CFG meeting: www.nsta.org/highschool/
connections.aspx
References
Ball, D.L., L. Sleep, T.A. Boerst, and H. Bass. 2009. Combining
the development of practice and the practice of development
in teacher education. The Elementary School Journal 109 (5):
458–474.
Bray, J.N., J. Lee, L.L. Smith, and L. Yorks. 2000. Collaborative
inquiry into practice: Action, reflection, and meaning making.
Thousand Oaks, CA: Sage Publications.
Curry, M. 2008. Critical Friends Groups: The possibilities and
limitations embedded in teacher professional communities aimed at instructional improvement and school reform.
Teachers College Record 110 (4): 733–774.
National Research Council (NRC). 1996. National science education standards. Washington, DC: National Academy Press.
NRC. 2000. Inquiry and the national science education standards:
A guide for teaching and learning. Washington, DC: National
Academy Press.
National School Reform Faculty (NSRF). 2009. Adult learning
in the face of student achievement. www.nsrfharmony.org
NSRF. 2008. National school reform faculty resource book. Bloomington, IN: Harmony Education Center.
Windschitl, M. 2008. Science as inquiry in the secondary setting.
Arlington, VA: NSTA Press.
Windschitl, M., J. Thompson, and M. Braaten. 2008. Beyond the
scientific method: Model-based inquiry as a new paradigm of
preference for school science investigations. Science Education
92 (3): 1–27.
Windschitl, M., J. Thompson, and M. Braaten. Forthcoming.
Fostering ambitious pedagogy in novice teachers: The new
role of tool-supported analyses of student work. Teachers College Record.
Wood, D. 2007. Teachers’ learning communities: Catalyst for
change or a new infrastructure for the status quo? Teachers
College Record 109 (3): 699–739.