Interpreting data for program
evaluation and planning
Literacy Coach’s Focus
In Data Analysis
Regrouping
Form needs-based
groups for classroom
instruction
Program
Evaluation
To what extent is my program keeping
Benchmark children at benchmark?
To what extent is small-group work
moving strategic children to benchmark?
Choose instructional
emphasis
Assign children
to interventions
To what extent is my program moving
Intensive children to benchmark?
To what extent are classroom effects
apparent?
Literacy Coach’s Focus
In Data Analysis
Regrouping
Form needs-based
groups for classroom
instruction
Which DIBELS reports
should I use?
Choose instructional
emphasis
Assign children
to interventions
Do you have curriculum
materials to accomplish this?
Literacy Coach’s Focus
In Data Analysis
Program
Evaluation
To what extent is my program keeping
Benchmark children at benchmark?
To what extent is small-group work
moving strategic children to benchmark?
To what extent is my program moving
Intensive children to benchmark?
To what extent are classroom effects
apparent?
100%
90%
80%
70%
60%
At Risk
Some Risk
Benchmark
50%
40%
30%
20%
10%
0%
K
1
2
3
State-Level Year-End Data, 2004-05
General Impressions
1. We are increasingly successful in
prevention-based instruction in
Kindergarten
2. We need to continue to experiment in
intervention, particularly for second and
third grade
Consider time, focus, and explicitness for
needs-based work?
Consider additional intervention programs?
Cross-Sectional Analysis
How well are the kindergarten children this
year doing compared to last year?
– Did they start out stronger or weaker?
– Did they make more or less progress between
fall and winter?
And yes, these are different children, but the
teachers are the same and the program is
the same
For Kindergarten
Beginning of kindergarten status includes
weighted combinations of measures
Middle kindergarten directs attention to initial
sound fluency
End of kindergarten directs attention to
phoneme segmentation fluency
*You have to look at your own data,
considering all measures, to really
evaluate your program
State K Cross-Section
Fall 2003
Winter 2004
Spring 2004
I
S
B
I
S
B
I
S
B
31% 43% 26% 30% 40% 30% 24% 34% 42%
Fall 2004
Winter 2005
(ISF)
Spring 2005
(PSF)
I
S
B
I
S
B
I
S
B
30% 42% 27% 15% 46% 38% 8% 18% 74%
New Directions
What did you decide to do differently next
year when you saw these data for your
school?
For first grade
Beginning of first grade status includes
weighted combinations of measures
Middle first grade directs attention to
nonsense word fluency
End of first grade directs attention to oral
reading fluency
*You have to look at your own data,
considering all measures, to really
evaluate your program
State 1 Cross-Section
Fall 2003
Winter 2004
Spring 2004
I
S
B
I
S
B
I
S
B
33% 32% 36% 32% 30% 39% 25% 30% 45%
Fall 2004
Winter 2005
(NWF)
Spring 2005
(ORF)
I
S
B
I
S
B
I
S
B
19% 29% 53% 13% 40% 48% 16% 26% 58%
New Directions
What did you decide to do differently next
year when you saw these data for your
school?
For Second Grade
Beginning of second grade status includes
weighted combinations of measures
Middle second grade directs attention to oral
reading fluency
End of second grade directs attention to oral
reading fluency
*You have to use the cognitive model of
assessment to interpret these data
State 2 Cross-Section
Fall 2003
Winter 2004
Spring 2004
I
S
B
I
S
B
I
S
B
27% 32% 42% 32% 18% 50% 38% 22% 40%
Fall 2004
Winter 2005
Spring 2005
I
S
B
I
S
B
I
S
B
21% 32% 47% 22% 19% 59% 26% 20% 54%
New Directions
What did you decide to do differently next
year when you saw these data for your
school?
For Third Grade
Third grade data include only oral reading
fluency
*You have to use the cognitive model of
assessment to interpret these data
State 3 Cross-Section
Fall 2003
Winter 2004
Spring 2004
I
S
B
I
S
B
I
S
B
25% 31% 44% 33% 34% 33% 28% 40% 33%
Fall 2004
Winter 2005
Spring 2005
I
S
B
I
S
B
I
S
B
26% 35% 40% 27% 31% 42% 20% 38% 41%
New Directions
What did you decide to do differently next
year when you saw these data for your
school?
Cohort Analysis
Given children’s experience at your school
over time, to what extent is your
instructional program actually accelerating
literacy growth over time?
(and you are right when you say it’s not
EXACTLY the same children if your
population is highly transient)
State Cohort K-1
Fall 2003
Winter 2004
Spring 2004
I
S
B
I
S
B
I
S
B
31% 43% 26% 30% 40% 30% 24% 34% 42%
Fall 2004
Winter 2005
(NWF)
Spring 2005
I
S
B
I
S
B
I
S
B
19% 29% 53% 13% 40% 48% 16% 26% 58%
60
50
40
30
K-1 Benchmark %
20
10
0
F03
W04
S04
F04
W05
S05
State Cohort 1-2
Fall 2003
Winter 2004
Spring 2004
I
S
B
I
S
B
I
S
B
33% 32% 36% 32% 30% 39% 25% 30% 45%
Fall 2004
Winter 2005
(ORF)
Spring 2005
I
S
B
I
S
B
I
S
B
21% 32% 47% 22% 19% 59% 26% 20% 54%
60
50
40
30
1-2 Benchmark
20
10
0
F03
W04
S04
F04
W05
S05
State Cohort 2-3
Fall 2003
Winter 2004
Spring 2004
I
S
B
I
S
B
I
S
B
27% 32% 42% 32% 18% 50% 38% 22% 40%
Fall 2004
Winter 2005
Spring 2005
I
S
B
I
S
B
I
S
B
26% 35% 39% 27% 31% 42% 20% 38% 41%
50
45
40
35
30
25
2-3 Benchmark
20
15
10
5
0
F03
W04
S04
F04
W05
S05
Interpretation
• To what extent have you set and communicated
the plan?
• To what extent are teachers understanding and
implementing the curriculum?
• How are they using time?
• How are they monitoring progress and adjusting
their instruction and groupings?
• How well are they using intervention options?
If you’re not getting the results you want,
you have to do something different.
Start with yourself
Work more closely with administration.
Spend more time in classrooms.
Focus your pd time on differentiation.
Next Steps
• Reflect on your own data; check on individual
indicators in K and 1 to see if there are particular
areas that are troublesome
• Find your most and least successful classrooms
and observe so that you can learn about the
curriculum and you can evaluate the
effectiveness of your own professional support
system
• Try something different