Michiana Math Track
Program Evaluation
Year 5 Results
2010-2011 School Year
Kindergarten Assessment 1 (Orange Assessment)
Only students who completed both pre and post tests were included in the analysis. There
were 228 KG students in the control group and 374 in Michiana Math Tracks (MMT) schools
who completed both assessments. Initial group means at the beginning of the year were
statistically different based on treatment and control groupings, F (1, 600) = .54, p =.463. The
control group and MMT students had similar scores on the KG1 pretest. At yearend MMT
students demonstrate a statistically significant increase in achievement compared to the control
group, F(1,600) = 133.9, p < .000. The MMT group scored 4 point higher on average than the
control group. The average gain was approximately 4 points more for MMT students. This
assessment has a total score of 26 point. The results of an Analysis of Covariance (ANCOVA)
using September scores as a covariate, yearend test scores as the dependent variable, and the
Michiana Math Tracks intervention as the independent variable suggest that this intervention did
have an impact on student success on the year end assessment. The effect size (ES) calculations
indicate that approximately 20% of the variance in final assessment could be attributed to the
MMT program. However, the ability of the student was also a significant factor in students’
achievement. Based on the effect size calculation 8% of the variance in math achievement could
be attributed to students’ previous or natural ability and effort as measured by the pretest.
30
25
24.2
20
20.5
Michiana Math Tracks
Control Group
15
10
5
5.9
5.6
0
Fall Pretest
Spring PostTest
Figure 1. Kindergarten Achievement by Group for Kindergarten Assessment 1
Kindergarten Assessment 2 (Purple assessment)
Group means for assessment 2 were statistically different based on comparison grouping
in September, F (1, 600) = 2.87, p =.091. Control group students achieved similar results
compared to MMT students at the beginning of the year. By yearend MMT students demonstrate
a statistically significant increase in achievement when compared to students in the control
group, F (1, 600) = 240.7, p < .000. The MMT students mean on the final was 24 points higher.
The averages gain for MMT students was approximately 25 point more that students in the
control group. The results of an ANCOVA suggest that the MMT intervention did have an
impact on student success on the year end assessment. The effect size (ES) calculations indicate
that 39% of the variance in final assessment could be attributed to the MMT program. However,
the ability of the student was also a significant contributing factor in students’ achievement.
Based on the effect size calculation about 29% of the variance in math achievement could be
attributed to students’ previous or natural ability and effort.
90
80
79.3
70
60
55.7
50
Michiana Math Tracks
Control Group
40
30
20
10
14.5
12.6
0
Fall Pretest
Spring PostTest
Figure 2. Kindergarten Achievement by Group for Kindergarten Assessment 2
First Grade Assessment
Only student how completed both assessments were included in this analysis. This
consisted of 132 MMT students and 203 control group students. Overall the group means for
first grade students were statistically different at the beginning of the year. In September, MMT
students performed better than control group students, F (1, 333) = 9.5, p = .002. Average scores
on the pretest were about 5 point higher for students in the MMT group. At yearend MMT
students also did significantly better in terms of gain compared to control group students,
F(1,333) = 59.7, p < .000. Average scores on the final were 18 point higher on average for
MMT students. The averages gain for MMT students was approximately 13 point more that
students in the control group. The results of an ANCOVA suggest that this intervention had a
significant impact on student success as measured by this assessment. The effect size (ES)
calculations indicate that approximately 14% of the variance in final assessment could be
attributed to the MMT program. The ability of the students again was a significant factor in
students’ achievement. Based on the effect size calculation 37% of the variance in math
achievement could be attributed to students’ previous or natural ability and effort.
70
59.6
60
50
41.4
40
Michiana Math Tracks
30
20
Control Group
15.6
10
10.7
0
Fall Pretest
Spring PostTest
Figure 3. First Grade Achievement by Group
Second Grade Assessment
Only student how completed both assessments were included in this analysis. There were
67 MMT students and 45 control group students used in this analysis. Overall the group means
for second grade students were statistically similar at the beginning of the year, F (1, 110) = 2.7,
p = .104. Average scores on the pretest were about 3 point higher for students in the MMT
group. However, by yearend MMT students did significantly better compared to control group
students, F(1,110) = 28.0, p < .000. Average scores on the final were 23 point higher on average
for MMT students. The averages gain for MMT students was approximately 20 point more that
students in the control group. The results of an ANCOVA suggest that this intervention had a
significant impact on student success as measured by this assessment. The effect size (ES)
calculations indicate that approximately 21% of the variance in final assessment could be
attributed to the MMT program. The ability of the students again was a significant factor in
students’ achievement. Based on the effect size calculation 36% of the variance in math
achievement could be attributed to students’ previous or natural ability and effort.
80
75.5
70
60
52.2
50
Michiana Math Tracks
40
30
20
28.6
25.2
Control Group
10
0
Fall Pretest
Spring PostTest
Figure 4. Second Grade Achievement by Group
Average Gain Comparison by Assessment and School
This analysis included all students who completed the pre and post assessments for each
test. Looking at the average gains by treatment group, MMT students consistently made greater
gains compared to control group students. Gains for Kindergarten Assessment 1 were
approximately 4 points greater than control group students indicating a significant difference,
F(1, 600) = 65.1, p < .000. Gains in the Kindergarten Assessment 2 was also statistically
significant and represents an average difference in gain of 26 points, F(1, 600) = 375.0, p < .000.
The gains on the first grade assessment were also statistically different, F(1, 333) = 49.7, p <
.000. MMT students showed an average gain of about 13 points more than students in the
control group. Gains in the second grade assessment was likewise statistically significant and
represents an average difference in gain of 20 points, F(1, 110) = 31.1, p < .000.
Average Gain in Points
80
67
70
60
47
50
41
40
44
31
30
27
19
20
Control
15
10
0
KG 1
MMT
KG 2
1st Grade
2ndGrade
Figure 5. Average Gain in Achievement by Group and Assessment
Average Gain in Points
80
70
66
63
58
60
48
50
44
42
40
34
30
20
73
70
68
22
14
14
14
16
KG1
20
20
20
15
12
KG2
10
0
C1
C2
C3
C4
MMT1 MMT2 MMT3 MMT5 MMT6 MMT7
Schools (Contol and MMT)
Figure 6. Average KG Gains in Achievement by School and Assessment
Average Gain in Points
80
69
70
58
60
50
37
40
37
1st Grade
30
20
10
19
15
8
2nd Grade
19
13
5
0
C1
C3
C4
MMT1 MMT4
Schools (Contol and MMT)
MMT5
Figure 7. Average 1st and 2nd Grade Gains in Achievement by School and Assessment