Title:
Formalizing Mathematics Knowledge: How are Early Number Skills Related to Mathematics
Achievement and Gains for Students At-Risk for Mathematics Difficulties?
Authors & Affiliations:
Lina Shanley, University of Oregon
Ben Clarke, University of Oregon
Christian Doabler, University of Oregon
Hank Fien, University of Oregon
Background
Findings from numerous studies suggest that early mathematics skills measured as early
as kindergarten entry are predictive of later mathematics achievement (Duncan & Murnane,
2011; Morgan, Farkas, Hillemeier, & Maczuga, 2014). Likewise, research has found that whole
number knowledge and basic number skills are reliable indicators of concurrent and future
mathematics achievement (Gersten et al., 2009). As such, a good deal of intervention
development research has focused on supporting the development of whole number knowledge
as a critical foundation for future mathematics attainment (Dyson et al., 2011; Fuchs et al, 2005;
Gersten et al., 2015). These interventions provide essential support for many students at risk for
mathematics difficulties and shore up early mathematics deficits before they become more
serious. However, there are a number of students who do not respond to these targeted
interventions and more still who do not maintain their gains in future years (Starkey & Klein,
2008). This is of particular concern because these students are likely to continue to struggle for
years to come. In fact, Morgan et al. (2009) found that students who were in the lowest 10th
percentile at entrance and exit from kindergarten had a 70% chance of remaining in the lowest
10th percentile five years later. Thus, identifying skills that are most indicative of success or
difficulties for at-risk students is of great import.
Evaluations of general mathematics achievement and research on mathematics
development conducted with typically achieving samples have established the importance of
foundational whole number skills. For example, understanding symbolic numerical magnitude
representations has a strong relationship with mathematics achievement (Fazio et al. 2014;
Schneider et al., 2015), and achievement in later number domains is strongly predicted by early
achievement in similar number-based domain (Mok et al., 2014). Similarly, numeral knowledge
(i.e., number identification and number sequencing) is critical (Göbel, et al., 2014) and mediates
the relationship between informal and formal mathematics knowledge for preschool and
primary-aged children (Purpura et al., 2013). However, it is unclear if these findings hold with
at-risk student samples, and the extent to which other prerequisite skills (e.g., non-symbolic
number knowledge or knowledge of quantity) may be more predictive of mathematics
achievement for struggling students is unknown. Knowledge of which specific early number
skills are related to mathematics achievement for students at-risk for mathematics difficulties
(i.e., those who are likely to be non-responders) is particularly important to support intervention
in the primary grades.
Purpose and Research Questions
Given that little is known about the categories and types of early mathematics skills that
are related to general mathematics achievement for kindergarten students at-risk for mathematics
difficulties, this study sought to explore these relationships in the context of a larger kindergarten
intervention study. Student performance on a researcher-developed assessment of key number
concepts was analyzed to determine (a) the presence of distinct factors and mathematics skills
assessed in the measure, and (b) the relationships between those key mathematics skills, and
concurrent summative achievement scores, achievement gains, and future achievement. These
research aims are represented by the following research questions:
1.
To what extent are there unique factors defined by mathematics skill category
(symbolic or not) and type (knowledge of quantity, number knowledge,
magnitude comparison, etc…) in an assessment of key mathematics skills
(RAENS)?
2 2.
What are the relationships between key mathematics skills defined by category
and type and a summative measure of mathematics achievement (SESAT) at
the end of kindergarten?
3.
Which mathematics skills (defined by category and type) are related to
summative kindergarten mathematics achievement gains (TEMA-3 pre/post)?
4.
To what extent do symbolic and non-symbolic mathematics skills measured at
the end of kindergarten predict mathematics achievement in first grade (SAT10)?
Whereas research findings suggest that symbolic skills are highly related to early
mathematics achievement (Kolkman et al., 2013; Sasanguie et al., 2014), the relationship
between non-symbolic number skills and general mathematics achievement remains largely
unknown particularly with regard to struggling learners (DeSmedt et al., 2013). However, a
recent study found that explicit practice of non-symbolic numerical processes was associated
with enhanced performance in symbolic mathematics tasks (Hyde, Khanum, & Spelke, 2014),
and previous research demonstrated that students with mathematics learning disabilities have
fewer difficulties with non-symbolic number magnitude than symbolic number tasks when
compared to typical peers (Rousselle & Noel, 2007). For these reasons, it was hypothesized that
non-symbolic number knowledge may be the lynchpin of early mathematics understanding for
students with underdeveloped skills, and thus would be an important predictor of mathematics
achievement for the at-risk sample in this study.
Method
Data analyzed here were collected during Year 1 of a four-year Efficacy Trial of the
ROOTS intervention funded by the Institute of Education Science (IES; Grant R324A120304).
Blocking on classrooms, the 10-12 lowest performing students from each participating
kindergarten classroom were selected for participation in the study. These intervention eligible
students comprised the analytic sample for the current study.
Setting
This study took place in 37 kindergarten classrooms from four school districts in the
Pacific Northwest. One school district was located in a metropolitan area, while the remaining
three districts were located in suburban and rural areas. Across the four districts, student
enrollment ranged from 2,736 to 38,557 students. Within the participating schools, between 8%23% received special education services, 5%-68% were English learners and 17%-86% were
eligible for free or reduced lunch.
Participants
Across the 37 classrooms, 850 kindergarten students were screened in late fall of 2012 to
determine eligibility for the intervention study. In all, 290 met the inclusion criteria and were
assigned to an intervention condition. Over the course of the school year, 17 students moved or
left the study, so the analytic sample for the current study included 273 students of whom were
43% were male, 58% were white/non-Hispanic, 32% were Hispanic, 31% were English language
learners, and 11% were eligible for special education services.
Measures
Student-level mathematics achievement data were collected during the kindergarten year
at the intervention’s pretest (T1) and posttest (T2) times, and at a follow-up (T3) approximately
six months into the students’ first-grade year.
ROOTS Assessment of Early Numeracy Skills (RAENS; Doabler, Clarke & Fien,
2012). The RAENS was administered at pretest and posttest time periods to assess key features
3 of early numeracy. Posttest data were analyzed in this study. RAENS is an individually
administered assessment consisting of 32 items. Items assess aspects of counting and cardinality,
number operations, and the base-10 system. In an untimed setting, students are asked to count
and compare groups of objects, write, order, and compare numbers, label visual models (e.g. tenframes), and write and solve single digit addition expressions and equations. RAENS’ predictive
validity ranges from .68 to .83 with widely used measures of mathematics achievement including
the TEMA and the NSB. Inter-rater scoring agreement was reported at 100% (Clarke, Doabler,
Smolkowski, Fien, & Baker, 2014) and internal consistency was high, Cronbach’s alpha = .91
(Clarke et al., 2015).
Test of Early Mathematics Ability - Third Edition. The Test of Early Mathematics
Ability-Third Edition (TEMA-3; Pro-Ed, 2007) is a standardized, norm-referenced, individually
administered measure of beginning mathematical ability. The TEMA-3 assesses mathematical
understanding for children ranging in age from 3 to 8 years 11 months. The TEMA-3 addresses
children’s conceptual and procedural understanding of math, including counting and basic
calculations. The TEMA-3 manual reports alternate-form and test-retest reliabilities of .97 and
.82 to .93, respectively. For concurrent validity with other math outcome measures, the TEMA-3
manual reports coefficients ranging from .54 to .91.
The Stanford Achievement Test-Tenth Edition (SAT-10; Harcourt Educational
Measurement, 2002). The SAT-10 measure is a group administered, standardized, norm
referenced test with two mathematics subtests, Problem Solving and Procedures. The
kindergarten version of the SAT-10 is the Stanford Early Achievement Test (SESAT). The SAT10 is a standardized achievement test with adequate and well-reported validity (r = .67) and
reliability (r = .93). All participating students, were administered the SESAT at posttest (T2) and
the SAT-10 midway through their first grade year (T3). In this study, SAT-10 data was collected
for 194 of the participating students.
Criteria for Participation
A three-step process was utilized to identify students who were at risk for mathematics
difficulties. First, within the 37 participating classrooms, all kindergarten students with parental
consent were screened in late fall of their kindergarten school year. Screening measures included
two standardized assessments of early mathematics: Assessing Student Proficiency in Early
Number Sense (ASPENS; Clarke, Gersten, Dimino, & Rolfhus, 2011) and the Number Sense
Brief (NSB; Jordan, Glutting, & Ramineni, 2008). Students were selected for participation if
they scored 20 or less on the NSB (Jordan et al., 2008) and had a composite score on the
ASPENS that placed in the strategic or intensive range (Clarke et al., 2011). These thresholds
were selected because prior research suggests that students who score in these ranges at the start
of kindergarten are at risk for developing long-term mathematics difficulties (Clarke et al., 2011;
Jordan et al., 2008). Second, prior to random assignment, students’ ASPENS and NSB scores
were separately converted into standard scores and then combined to form an overall composite
standard score. Third, students’ composite standard scores were rank ordered and the lowest ten
students that met our established a priori criteria were considered eligible for random assignment
and were included in our analytic sample.
Data Analysis
Preliminary descriptive analyses were conducted using SPSS 20.0 for Mac OS (IBM
Corp., 2011) and all subsequent models were investigated using the Weighted Least Squares
Means and Variance (WLSMV) estimator for categorical variables in Mplus 7.1 (Muthén &
Muthén, 2013).
4 Confirmatory factor analytic (CFA) methods were employed to determine the best fitting
factor structure for the RAENS. A model of mathematics skill type with five factors (i.e.,
quantity knowledge, magnitude comparison, number combination, number sequence, and
number knowledge) was compared to a bi-factor model representing mathematics skill category
(i.e., non-symbolic and symbolic) and the two models were combined into a final model much
like a Multi-Trait Multi-Method (MTMM, Campbell & Fiske, 1959) CFA model (see Figure 1).
All models were compared using chi-square difference tests via the DIFFTEST option in Mplus
and overall model fit was evaluated using standard model fit statistics and rules of thumb (Kline,
2010). Lastly, structural equation models were developed to evaluate the relationships between
the mathematics skill type and category factors in the RAENS and the summative measures of
general mathematics achievement (see research questions 2–4).
Results
All measures met the assumptions for the analyses conducted and there were moderate to
strong correlations amongst all study variables (see Table 1). The MTMM CFA model
demonstrated excellent fit (χ2 (316) = 323.42, p = .37; RMSEA = .01; CFI = .99) with both
mathematics skill type and category constructs uniquely represented in the RAENS. To address
research question 2, all of the key number concept factors were entered as predictors of the
SESAT standard scores. The key number concept factors explained almost 69% of the variance
in SESAT scores with magnitude comparison and non-symbolic number knowledge emerging as
the only statistically significant predictors β = .48, p < .01 and β = .37, p < .001, respectively.
Next, the key number concept factors were entered as predictors of end of kindergarten TEMA-3
scores, controlling for TEMA-3 scores at kindergarten entry to investigate the extent to which
key numeracy constructs were associated with gains in mathematics achievement in kindergarten
as described in research question 3. Together the predictors explained almost 78% of the
variance in TEMA-3 scores at posttest, and non-symbolic number knowledge was the only
unique predictor of TEMA-3 gains, β = .14, p = .01. Finally, SAT-10 scores were regressed on
the symbolic and non-symbolic number knowledge factors. The number knowledge factors
explained nearly 75% of the variance in SAT-10 scores, and both factors demonstrated a
statistically significant association with SAT-10 scores (symbolic β = .65, p < .001 and nonsymbolic β = .37, p < .05).
Conclusions
Developing interventions that are maximally effective for all students and that
demonstrate sustained long-term impacts on achievement is critical to reduce achievement gaps.
Numerous studies have confirmed the presence of a relationship between key, early whole
number skills and later mathematics achievement, however the extent to which those skills are
important predictors of achievement for struggling students is largely unknown. Using a sample
of students who were at-risk for mathematics difficulty at kindergarten entry, the results of this
study suggest that robust non-symbolic number knowledge may be a critical indicator of
kindergarten gains, kindergarten achievement, and mathematics achievement in first grade.
Whereas symbolic skills may be most predictive of mathematics learning for typically achieving
students, these findings suggest that targeted instruction and scaffolded practice of non-symbolic
number skills may be necessary to support the mathematics development of at-risk learners in the
early grades.
5 Appendices
Appendix A. References
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the
multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105.
Clarke, B., Doabler, C. T., Smolkowski, K., Kosty, D. B., Baker, S. K., Fien, H., & Strand Cary,
M. (2014). Examining the efficacy of a tier 2 kindergarten mathematics intervention.
Journal of Learning Disabilities, Advanced online publication. doi:
10.1177/0022219414538514
Clarke, B., Doabler, C.T., Smolkowski, K., Kurtz-Nelson, E., Fien, H., & Baker, S. (2015).
Testing the immediate and long-term efficacy of a tier 2 kindergarten mathematics
intervention (Technical Report 1502). Eugene, OR: University of Oregon.
Clarke, B., Gersten, R., Dimino, J., & Rolfhus, E. (2011). Assessing student proficiency in early
number sense (ASPENS) [Measurement instrument]. Longmont, CO: Cambium Learning
Sopris.
De Smedt, B., Noël, M. P., Gilmore, C., & Ansari, D. (2013). How do symbolic and nonsymbolic numerical magnitude processing skills relate to individual differences in
children's mathematical skills? A review of evidence from brain and behavior. Trends in
Neuroscience and Education, 2(2), 48-55.
Doabler, C.T., Clarke, B. & Fien, H. (2012). ROOTS Assessment of Early Numeracy Skills
(RAENS). Unpublished student mathematics assessment. Center on Teaching and
Learning, University of Oregon.
Duncan, G. J., & Murnane, R. J. (Eds.). (2011). Whither opportunity? Rising inequality, schools,
and children's life chances. New York: Russell Sage Foundation.
Dyson, N. I., Jordan, N. C., & Glutting, J. (2011). A number sense intervention for low-income
kindergartners at risk for mathematics difficulties. Journal of Learning Disabilities. 46,
166–181. doi: 10.1177/0022219411410233
Fazio, L. K., Bailey, D. H., Thompson, C. A., & Siegler, R. S. (2014). Relations of different
types of numerical magnitude representations to each other and to mathematics
achievement. Journal of Experimental Child Psychology, 123, 53-72.
Fuchs, L. S., Compton, D. L., Fuchs, D., Paulsen, K., Bryant, J. D., & Hamlett, C. L. (2005). The
prevention, identification, and cognitive determinants of math difficulty. Journal of
Educational Psychology, 97, 493–513. doi: 10.1037/0022-0663.97.3.493
Gersten, R. M., Beckmann, S., Clarke, B., Foegen, A., March, L., Star, J. R., & Witzel, B.
(2009). Assisting students struggling with mathematics: Response to Intervention (RtI)
for elementary and middle schools. (Practice Guide Report No. NCEE 2009-4060).
6 Gersten, R., Rolfhus, E., Clarke, B., Decker, L. Wilkins, C. & Dimino, J. (2015). Intervention for
first graders with limited number knowledge: A replication of a randomized controlled
trial. American Educational Research Journal. Advanced online publication. doi:
10.3102/0002831214565787
Göbel, S. M., Watson, S. E., Lervåg, A., & Hulme, C. (2014). Children’s arithmetic development
it is number knowledge, not the approximate number sense, that counts. Psychological
Science, 25(3), 789-798. doi: 10.1177/0956797613516471
Harcourt Brace Educational Measurement. (2002). Stanford Achievement Test [SAT-10]. San
Antonio, TX.
Hyde, D. C., Khanum, S., & Spelke, E. S. (2014). Brief non-symbolic, approximate number
practice enhances subsequent exact symbolic arithmetic in children. Cognition, 131(1),
92-107.
IBM Corp. (2011). IBM SPSS Statistics for MacIntosh, Version 20.0. Armonk, NY: IBM Corp.
Jordan, N., Glutting, J., & Ramineni, C. (2008). A Number Sense assessment tool for identifying
children at risk for mathematical difficulties. In A. Dowker (Ed.), Mathematical
difficulties: Psychology and intervention (pp. 45–57). San Diego, CA: Academic Press.
Kline, R. B. (2010). Principles and practice of structural equation modeling (3rd Ed.). New
York: Guilford Press.
Kolkman, M. E., Kroesbergen, E. H., & Leseman, P. P. (2013). Early numerical development
and the role of non-symbolic and symbolic skills. Learning and Instruction, 25, 95-103.
Mok, M. M. C., Zhu, J., McInerney, D. M., & Or, A. (2014). Growth in mathematics cognitive
and content domains: A 6-year longitudinal study. Assessment & Learning, 3, 129-159.
Morgan, P. L., Farkas, G., Hillemeier, M. M., & Maczuga, S. (2014). Who is at risk for
persistent mathematics difficulties in the United States? Journal of Learning Disabilities,
47(4), 1-15.
Morgan, P. L., Farkas, G., & Wu, Q. (2009). Five-year growth trajectories of kindergarten
children with learning difficulties in mathematics. Journal of Learning Disabilities,
42(4), 306–321.
Muthén, L. K., & Muthén, B. O. (2013). Mplus User’s Guide, v 7.1. Los Angeles, CA.: Muthén
and Muthén, UCLA.
Pro-Ed. (2007). Test of early mathematics ability, third edition. Austin, TX: Author.
Purpura, D. J., Baroody, A. J., & Lonigan, C. J. (2013). The transition from informal to formal
mathematical knowledge: Mediation by numeral knowledge. Journal of Educational
Psychology, 105(2), 453-464.
7 Rousselle, L., & Noël, M. P. (2007). Basic numerical skills in children with mathematics
learning disabilities: A comparison of symbolic vs non-symbolic number magnitude
processing. Cognition, 102(3), 361-395.
Sasanguie, D., Defever, E., Maertens, B., & Reynvoet, B. (2014). The approximate number
system is not predictive for symbolic number processing in kindergarteners. Quarterly
Journal of Experimental Psychology, 67(2), 271-280.
Schneider, M., Beeres, K., Coban, L., Merz, S., Schmidt, S. S., … & De Smedt, B. (2015).
Associations of non-symbolic and symbolic numerical magnitude processing with
mathematical competence: A meta-analysis. Developmental Science, Advanced online
publication.
Starkey, P., & Klein, A. (2008). Sociocultural influences on young children’s mathematical
knowledge. In Saracho, O. N. and Spodek, B (Eds.) Contemporary perspectives on
mathematics in early childhood education. (pp.253–276). Charlotte, NC: Information
Age Publishing.
8 Appendix B. Tables and Figures
Table 1
Descriptive Statistics and Correlations for All Study Variables (n = 273)
Variable
1
2
3
4
1. TEMA-3 pretest
M (SD)
17.85 (6.82)
2. TEMA-3 posttest
.74**
25.86 (7.96)
3. RAENS total score
.65**
.78**
4. SAT-10 raw score
.46**
.48**
.42**
5. SESAT scaled score
.64**
.64**
.63**
21.78 (6.69)
31.68 (8.93)
.52**
453.23 (35.60)
Note. Correlations calculated using pairwise deletion. M = mean, SD = standard deviation.
* p < .01, ** p < .001.
Non$
Symbolic,
1,&,2,
3–5,
Quant,
6,&,7,
8,&,9,,
Mag,
Comp,
10–12,
Symbolic,
13–15,
16–18,
19,&,20,
Num,
Comb,
21,&,22,
Num,
Seq,
23–27,
28–30,
31,&,32,
Num,
Know,
Figure 1. Confirmatory Factor Analytic model for RAENS
9