How to Improve Diagnostic
Assessment Reporting Practices: What
Information Do Teachers Need and
How Can We Best Communicate It?
Communicating Results of Learning
Progression-Based Assessments
Emily Lai
Jennifer Kobrin
Insight Learning System
Game
Classroom Activity
Learning
Progression
Professional Development
Performance Assessment
Student Profile
Insight Learning System
Game
Classroom Activity
Learning
Progression
Professional Development
Performance Assessment
Student Profile
Professional Development: Goals
✓ Familiarize teachers with the learning progression
and increase their content knowledge
✓ Help teachers elicit and identify evidence of
students’ mastery of the stages of the learning
progression
✓ Help teachers use the learning progression to
make relevant instructional decisions to support
student progress
Student Profile Development
➢ Designed to help teachers interpret information to
support instructional decisions, by responding to
key questions:
– Who is struggling? With what?
– How could I group my students to improve
learning?
– What should I reteach or focus on in class?
– How can I personalize learning for my students?
➢ Quick, iterative design process
3 rounds of testing with 8-12 teachers in each
round
–Think alouds with follow-up on designs
1st Round Feedback: Interpreting Probability
Prompt: probability of mastery is .70
Response: So they got 70% of the questions right?
This will be a major challenge for diagnostic
models and communication
Decided to set cut points for probability and display as
mastery, non-mastery, or undetermined.
For class aggregation, what percent of class should
have mastered for the picture to indicate the class has
mastered? Response: Consensus right around 80%
Third
Round
Design
Solutions
Third Round Design Solutions
Third Round Solutions
“What now”
Individual Observation
Solution:
- Emphasize behavioral
observations
- Provide suggestions
for next steps
Sort by Column
(group)
“Columnize”
SOWOs
Pilot Testing Results with
Six Third Grade Teachers
Professional Development
Six third grade teachers from an elementary school in the
Midwest participated in approximately 20 hours of professional
development over 5 months:
❖
Introductory Webinar: What are Learning Progressions?
How can they be used for Formative Assessment?
❖
IV.
Face-to-Face Sessions:
The Learning Progression
Performance Assessments
Classroom Activities
Student Profile
❖
Virtual Forum: Questions of the Week
I.
II.
III.
Professional Development
Six third grade teachers from an elementary school in the
Midwest participated in approximately 20 hours of professional
development:
❖
Introductory Webinar: What are Learning Progressions?
How can they be used for Formative Assessment?
❖
IV.
Face-to-Face Sessions:
The Learning Progression
Performance Assessments
Classroom Activities
Student Profile
❖
Virtual Forum: Questions of the Week
I.
II.
III.
Professional Development Session on the
Student Profile
• Overview of the Student Profile
• Instructional Decisions Based on Mock Report
• Think Aloud with Their Own Class Results
• Feedback on the Usability of the Student Profile
Individual
Student
Mastery
Report
What can you
infer about
Harry?
“The area where
you have a halfcircle is where
you would be
working.”
So it looks like you need to start in
the area progression, and make
sure that he can properly calculate
area, and iterate an area, and then
challenge him with maybe some
different shapes to see if he can
figure out about area is additive
before you even go into the length
progression for perimeter because if
he can't distinguish the difference
then you want to start on that area
progression and make sure that
he's mastered all of that before you
continue on.
Usability Feedback
• Teachers felt there were too many stages and several
times mentioned that it would be more usable if “similar
stages were lumped together” and it was more like a
rubric.
• “The part I struggle with is that I don't know if it's
necessary for them to go through every level of the
progression, and they're gonna bounce back and forth
based on what their previous knowledge is, how they
view it, how they perceive [the task].”
• “I think while this is a nice tool, if I think about doing this,
though, for every math topic, it’s kind of, um, seems
overwhelming…”
Following Indiana Tryouts
Following Indiana Tryouts
Questions for Future Research & Development
• Can teachers interpret the Student Profile without
participating in professional development to familiarize
them with the learning progression?
• What is the right “grain size” for reporting?
• If we build it, will they use it? What prevents teachers
from personalizing instruction?