Digital Game for Undergraduate Calculus Education: The affordances of game design and
its effects on immersion, calculation, and conceptual understanding
Abstract
Digital games have the potential to help develop higher-education mathematical skills
and promote deep conceptual understanding. Using a digital game for undergraduate calculus
that we developed, this study has two goals: The first goal is to investigate the effectiveness of
using a digital game to teach undergraduate-level calculus in terms of improving task immersion,
sense of control, calculation skills, and conceptual understanding. The second goal is to examine
student behavior during gameplay to investigate how digital game affordances can facilitate
conceptual understanding of calculus. 132 undergraduate students were recruited to participate in
a controlled lab experiment. Students were randomly assigned to either a game-playing condition,
a practice quiz condition, or a no-treatment control condition. We collected two types of data,
self-reported survey data and behavioral-tracking data recorded by the server during gameplay.
The results showed that students who played the game reported highest task immersion but not
sense of control. Students in the game condition also performed significantly better in conceptual
understanding and equally well in calculation skills as students who solved a practice quiz and
the control group. Gameplay behavioral-tracking data showed that active manipulation of visual
representations was a significant predictor of conceptual understanding. The study has
implications for undergraduate mathematics education and digital game based learning design.
Keywords: Calculus; Digital Game; Mathematics; Technology; Undergraduate Education
1. Introduction
Calculus is the foundation for higher-level mathematics, which is essential for disciplines
such as physics, engineering, and economics. Calculus is not only important for understanding
more advance courses in school, it is also a significant predictor of one’s earnings at work
beyond school (Rose & Betts, 2004). However, several studies have reported a disconnect
between the calculus that students learned in classrooms and students’ ability to apply calculus
concepts to other disciplines and to utilize calculus outside of schools (Lesh & Zawojewski,
2007). Students often fail to transfer their calculus knowledge because they lack hands-on
experiences of applying their understanding to solving authentic problems; in fact, around 70%
of problems in one calculus textbook are solved by mimicking the examples shown in the
textbook (Lithner, 2004). This might cause students to be less motivated to learn because they do
not understand the value of calculus in application. Studies have shown that students who
experienced problem-solving scenarios in pre-calculus classes have better conceptual
understanding of calculus applications, can identify and use appropriate resources, and are more
motivated to take an active role in learning calculus (Stanley, 2002). Learning across multiple
contexts (e.g., different media or different problem context) can also promote transfer because
students can compare their experiences to abstract general concepts and construct a flexible
understanding that can be applied to different contexts (Bransford, Brown, & Cocking, 1999).
Digital games have been proposed as an effective way to promote students’ conceptual
understanding of abstract knowledge and problem-solving transfer (Boyle, Connolly, & Hainey,
2011; Garris, Ahlers, & Driskell, 2002; Gee, 2007). Modern digital games can facilitate
meaningful problem-solving experiences for students, allowing them to visualize abstract
concepts and situate the concepts in different contexts to gain a better understanding (Squire,
2003). They can provide immediate, or just-in-time feedback for students to assess and adjust
their process (C.-Y. Lee & Chen, 2009). Games encourage players to form initial hypotheses,
test them, observe the outcome, and revise their hypotheses. This process is similar to the process
of experiential learning (Kolb & Kolb, 2005). In other words, digital game can simulate
authentic problems for students to apply their calculus knowledge. They also allow students to
visualize and actively manipulating factors to construct a flexible mental model which improves
transfer across contexts.
While many studies have examined the use of digital games to enhance mathematics
education, most of them focus on primary to secondary school mathematics or drill-and-practice
for mathematical calculations (e.g., Ke, 2008a, 2008b; Mayo, 2009). Few studies have
investigated using digital games to facilitate undergraduate-level mathematics, especially
calculus, which is a complex foundational concept that affects student performance in more
advanced courses. A major challenge of designing a calculus game is balancing the complex
concepts and skills while keeping students immersed in the game. In this study we developed a
game to teach undergraduate-level calculus called Mission Prime which is based on
mathematical education principles.
The main goal of this study is to compare the effects of a digital game to teach universitylevel calculus to a traditional method of solving practice questions and a no-treatment control
group. A secondary goal is to investigate (if any) what affordances of the game promote students’
conceptual understanding. We used behavioral-tracking data of player actions during gameplay
to investigate whether the affordances of digital games to provide feedback and visual
manipulation improved students’ conceptual understanding of calculus. The study design is a
controlled lab experiment with random assignment that employs both pre- and post-test
questionnaires paired with server-based player behavioral data to examine the following general
research questions:
1. Is playing a calculus video game more effective in promoting conceptual understanding
than traditional practice questions or no-treatment?
2. Is playing a calculus video game more effective in promoting calculation skills than
traditional practice questions or no-treatment?
3. Is the experience of playing a calculus game more immersive than traditional practice
questions or no-treatment?
4. Do the number of feedback provided by the video game and the ability to manipulate
visual representations improve students’ conceptual understanding of the content
mathematics?
2. Theoretical Background
2.1. Promoting calculus transfer
Traditional mathematics courses are designed to guide students through a sequence of
modules that makes up complex mathematical concepts. Students are introduced to one small
module at a time and are expected to be able to piece together the modules and understand how
the modules fit together to form a bigger picture. However, student may fail to realize the bigger
picture and are left with isolated, incomplete understanding of the concepts (Tall, 1991). Even
students who do well in mathematics classes may understand equations as symbol manipulation
and fail to realize its relation to real-life problems and applications (Siegler, 2009). This may be
a reason that can explain why many STEM educators feel that students are under-prepared in
their calculus training and fail to understand its application in other disciplines (Lesh &
Zawojewski, 2007).
The major goal in the calculus education reform since the 80s is to shift from the
traditional focus on memorization and calculation techniques to promote conceptual
understanding and focus on calculus applications. Many mathematics educators argue that in
order to reach this goal, mathematics course should be designed the opposite way around (Kaput,
1994; Tall, 1991), meaning that students are first exposed to authentic application problems so
that they can develop a need for learning the concepts. Then the complex systems are broken
down into smaller modules facilitated by visual aides for students to manipulate and observe
(Disessa & Sherin, 2000; Kaput, 1994). This type of design situates the mathematic concepts
within authentic problems, allowing students to take on a more active role in knowledge
construction (Oehrtman, 2009). This may facilitate better appreciation of concepts and promote
transfer between contexts.
Several studies have tested the effects of teaching mathematics through authentic
problems and visual manipulations. For example, Stanley (2002) provided students with real-life
problems such as designing drug dosage or developing a saving plan. She then asked them to
work in groups to solve these problems using calculus. The study found that although the
students did not perform better in quizzes than students who were not exposed to real-life
problems, but the students who solved real-life problems were more motivated, were better at
identifying appropriate resources, and had a better understanding of calculus applications.
Another study by Kidron and Zehavi (2002)found that using software to display dynamic
graphics helped students visualize the mathematical processes and gave meaning to the abstract
concepts.
2.2. Using digital games to promote calculus education
Digital games have been proposed as effective means for teaching mathematics because
of several unique affordances. Digital games can (a) provide meaningful problems in situated
contexts for players to solve (Gee, 2007; Steinkuehler & Duncan, 2008), (b) allow players to
visualize complex systems and manipulate dynamic factors (Shaffer, 2006), (c) present multiple
representations to demonstrate the underlying concepts (Betz, 1995), (d) give immediate
performance or formative feedback for players to track the progress or adjust their hypotheses
(Delacruz, 2012), and (e) promote sense of immersion and motivations (Squire et al., 2003).
Educators have been experimenting with using digital games for education for quite some
time. “Edutainment,” or the combination of education with entertainment was a concept that was
thought as the future of education in the 80s. However, poor design that simply masks practiceand-drill with animations has earned edutainment a sour reputation for being neither effective
education nor entertaining (Van Eck, 2006). As digital games progressed from simple games to
games that can simulate complex systems and engage players in dynamic relations, so has
research on the affordances of digital games and how to utilize them for education (Charsky,
2010). A recent study found that when used appropriately, even a practice-and-drill game can be
designed to motivate students to learn mathematics (Ke, 2008a).
Many studies have empirically examined the use of digital games to enhance
mathematics education, but most of those studies focused on primary or secondary school
education with mixed results. Meta-analyses and systematic reviews of games for education
generally attribute the mixed results to small sample sizes and the lack of control groups in many
studies (Connolly, Boyle, MacArthur, Hainey, & Boyle, 2012). Cordova and Lepper (1996)
found that students who learned through a game in mathematics classes outperformed students in
classes that did not incorporate games. They also found that the sense of control, challenge,
curiosity and the ability to situate knowledge into context increased students’ intrinsic motivation.
Another study compared classrooms that used a game to teach ninth-grade algebra and those that
did not, the findings showed that students who used the games outscored students that did not in
the ETS algebra assessments (Morgan & Ritter, 2002). Spotnitz (2001) examined the effect of a
mathematics game on forth to sixth-grade students’ intrinsic motivation, self-efficacy, task
involvement, and performance. After four weeks of 30-minute sessions, students answered a
questionnaire along with a mathematics quiz. The study found that students who learned through
the game reported higher intrinsic motivation, self-efficacy, and task involvement. Although
there was significant improvement between the pre and post- test, students who played the game
did not perform significantly better than students who received traditional education. Similarly,
Ke (2008a) also found that student were more motivated and engaged when using digital games,
yet they did not perform significantly better when assessed with traditional paper-and-pencil tests.
We hypothesize that students in the game condition would report higher immersion and sense of
control, but we believe the effects on improving calculation skills can go either way, and thus
pose it as a research question.
H1. Participants in the game condition report higher immersion than (a) the practice quiz
condition and (b) the no-treatment control condition.
H2. Participants in the game condition report higher perceived control than (a) the
practice quiz condition and (b) the no-treatment control condition.
RQ1. Will participants in the game condition perform better in calculation skills than (a)
the practice quiz condition and (b) the no-treatment control condition?
While there are mixed evidence on digital games’ ability to improve mathematics
calculation skills, studies have shown that it may be effective in promoting higher-order
metacognition and deep conceptual understanding. For example, Liang and Zhou (2009)
conducted a qualitative assessment of students’ experience with a mathematics game and their
academic performance. They found that students who used the game reported more positive
attitudes towards mathematics. The students also felt that the feedback provided by the games
helped them develop a sense of responsibility for their own learning and correct their mistakes
without fear of embarrassment. More importantly, the students felt that they have a better
understanding of how mathematics is connected to their everyday life when learning through the
games. Another study by Lopez-Morteo and López (2007) used a game-like computer supported
system to engage students in mathematics. They found that students who used the game-like
system gained a positive attitude towards mathematics, which is a significant predictor of
mathematics performance. Students in the experiment also appreciated the system for presenting
theorems and problems from multiple perspectives, allowing them to understand that
“mathematics are more than counting numbers” (p.636). Because problem-solving experiences
and multiple representations were argued to promote deep conceptual understanding of
mathematics (Kaput, 1994; Tall, 1991), we hypothesize that students who played the calculus
game would have better conceptual understanding than students who received a traditional
practice quiz or received no treatment.
H3. Participants in the game condition perform better in conceptual understanding than (a)
the practice quiz condition and (b) the no-treatment control condition.
In addition to comparing the effectiveness of a calculus game to traditional methods of
education, we are also interested in examining which features of the game would facilitate
students’ conceptual understanding. As suggested by Anderson, Greeno, Reder, and Simon
(2000), in order to promote effective learning, it is crucial to examine the students’ learning
activities and cognitive procedures to gain a deeper understanding of how students are learning
from our educational design. We use in-game behavioral tracking data to observe student
activities in the game and test the hypotheses that games promote conceptual understanding
through providing feedback and allowing students to manipulate visual representations.
H4. Visual manipulation positively predicts conceptual understanding among participants
in the game condition.
H5. Number of feedback positively predicts conceptual understanding among participants
in the game condition.
3. Method
3.1. Experiment design
A between-subject experiment design was used to test the hypotheses and research
questions. 132 students were recruited from Calculus II classes at a large Midwestern university
using extra credits as incentives. When participants arrived at the lab, after giving informed
consent, they were administered a short online survey measuring their attitude towards
mathematics, the number of calculus classes that they have taken, and basic demographics
including age, gender, and department majors. Next, the participants were randomly assigned to
one of the three conditions (game, practice quiz, control). Of the three conditions, the digital
game group was asked to play a calculus game until they finished all the scenarios or until one
hour had elapsed; the practice quiz group received a calculus practice quiz for them to solve in
one hour; the third group control group did not receive any treatment before the measurements
and played the game after they completed all the questionnaires. After the stimulus, the
participants’ calculation skills and conceptual understanding were measured using a paper-andpencil calculus test designed by members of the research team from the Mathematics department.
The participants’ perceived immersion and sense of control were measured with a subset of the
cognitive absorption scale developed by Agarwal and Karahanna (2000). While the game group
played the game, we tracked their behaviors in the game including the frequency of different
actions, the number of feedback they received, the duration of time on each action and scenario,
whether they used the visual manipulation function, and how long they spent manipulating the
visual representations.
After removing seven participants who did not complete the study, a total of 125
participants were included in the analyses with 50 in the game condition, 38 in the practice quiz
condition, and 37 in the control condition. The average age of the participants was 19.39 years
old (SD=2.57). There were more male participants (n=77, 65.8%) than female (n=40, 34.2%),
and the majority were university freshman (n=97, 82.9%), perhaps due to recruitment from
calculus courses which are often required foundational courses in many STEM departments.
3.2. The game: Mission Prime
The game used in this study is a game developed by the research team to enhance the
learning experience of undergraduate calculus to freshmen and sophomores. The project is part
of a university-wide initiative to incorporate technology into foundational STEM courses to
enhance student learning. For calculus, we designed a game called Mission Prime. In the game,
the players’ goal is to help set up a space colony using their knowledge of optimization, one of
the key concepts in calculus. In each scenario, players are given a problem such as maximizing
the interior of a fence with limited resources. The player will be able to select objects, adjust
viable parameters, view the problem space from multiple perspectives, and ultimately select the
mathematical ‘tools’ necessary to solve the given problem.
The goal of the game is to train students to: (a) Identify the type of problem, (b) model
the problem, (c) select the appropriate resources to solve the problem, (d) set up a function to
solve the problem, and (e) find a correct answer to the problem. The game instruction focuses on
the identifying, modeling, selecting tools, and setting up a function to solve the problems.
Calculations were not emphasized in favor of deeper, perhaps more conceptual understanding of
the problem.
The players were asked to assemble a function to solve the problem out of a set of
formulae provided. For example, in a scenario the players were asked to construct a hydraulic
generator and the player must figure out the dimensions (see Figure 1). Once the function is built
out of these various components the player will be able to perform various operations on it and
observe the final output. If the function yields the correct answer, the player will complete the
scenario. If the function yields an incorrect answer, the player will be given feedback and will be
asked to modify the function until the right answer is found. This design allowed players to focus
on learning the conceptual framework of the problem without having to memorizing formulae
and perform calculations. During the problem-solving process, players can manipulate the visual
representation of the objects, rotate their view of the problem space, and switch between twodimensional and three-dimensional perspectives to help them come up with a solution to the
problem (see Figure 2).
Each scenario in the game represents a new problem that is built on previous learning.
This design allows players to learn the concepts through multiple representations and allow the
players to incorporate new mathematical concepts with their existing mental representations as
they progress.
The game Mission Prime was designed with mathematical educational theories in mind.
Students were given an application problem to facilitate purposeful engagement, and each
scenario is a different representation of the basic concept of optimization. Visual aides were
given that allow students to move around and observe. Both performance and formative feedback
was given to help students keep track of their progress and adjust their actions.
[Figure 1 here]
[Figure 2 here]
3.3. Measurements
3.3.1. Conceptual understanding. Calculus conceptual understanding was measured by two
open-ended questions developed by members of the research team from the Mathematics
department. The first question asked participants to describe “What do you think are the
important concepts in optimization?” and the second question asked participants to explain “how
and why we use the derivative to solve optimization problems?” The open-ended questions were
scored by the researchers according to a scoring rubric while blind to the experiment conditions.
The scores ranged from 0 to 4 on each question and were averaged to gain a general calculus
concept score. The average score across the conditions was 1.66 (SD=1.15).
3.3.2. Calculation skills. Calculus calculation skills were measured using two quiz questions to
test the students’ ability to solve an optimization problem. Each of the two questions requires
two answers for width and height, which result in four answers. Each of the answers were scored
0 for incorrect answers and 1 for correct answers, the scores were aggregated to create a
calculation score that ranged from 0 to 4. The average score across the conditions was 1.30
(SD=1.37).
3.3.3. Immersion. Perceived immersion was measured by a subset of the cognitive absorption
scale developed by Agarwal and Karahanna (2000). The subscale consisted of five Likert-type
items that asked participants to rate how much they agree with the statements (e.g., “While I was
engaged with the training tool, I was able to block out most other distractions.”). The items were
reliable with Cronbach’s α=.91.
3.3.4. Sense of control. Sense of control was measured by another subset of the cognitive
absorption scale which consisted of three Likert-types items (e.g., “When I was engaged with the
training tool, I felt in control.”). The items were reliable with Cronbach’s α=.85.”
3.3.5. Game behavioral-tracking data: Feedback and visual manipulation. We measured number
of feedback and number of visual manipulation using unobtrusive behavioral-tracking from the
server that hosted the game. A major advantages of using behavioral data is that the
measurement is unobtrusive, thus the player will not feel threated when making mistakes and
spending large amount of time on manipulating the visual representations (Y.-H. Lee, Heeter,
Magerko, & Medler, 2012). Behavioral-tracking also allowed us to gain exact frequency and
duration of actions without the potential problem of having human observer errors (Heeter, Lee,
Medler, & Magerko, 2013). These behavioral data also gave the game development team insights
into parts of the game that players were having trouble in and improve the game in future
iterations.
4. Results
4.1. Immersion
Hypothesis 1 posited that participants in the game condition report higher perceived
immersion in the task than the practice quiz condition and the control condition. We conducted
one-way Analysis of Variance (ANOVA) to test the hypothesis. Experiment conditions were
used as the independent variable, and perceived immersion was the dependent variable.
The result showed that there was significant difference between the three conditions, F(2,
122)=4.77, p=.010, eta squared=.07 in perceived immersion. Post-hoc comparison with Tukey
HSD showed that the game condition (M=5.67, SD=1.05) reported significantly more immersed
in the task than the practice quiz condition (M=4.90, SD=1.26) and the control condition
(M=5.04, SD=1.50). The practice quiz condition was not significantly different from the control
condition (p=.873). The result was consistent with hypothesis 1. The findings indicate that
participants felt more immersed in playing the game than doing practice quiz.
4.2. Sense of control
Since digital games can facilitate self-paced learning that may increase students’ sense of
control in their own learning progress and results, hypothesis 2 posited that participants in the
game condition would report higher sense of control than participants in the other two
experiment conditions. We conducted a similar ANOVA to test the hypothesis, this time with
sense of control as the dependent variable. The results showed that unlike what we hypothesized,
there was no significant difference between the conditions in their sense of control, F(2,
122)=1.22, p=.300. Participants in the game condition (M=4.60, SD=1.47) did not report higher
sense of control than the practice quiz condition (M=4.85, SD=1.16), nor the control condition
(M=4.33, SD=1.55). The result was not consistent with hypothesis 2.
4.3. Calculus calculation skills
Research question 1 asked whether the game will improve students’ calculation skills
over the traditional practice quiz or no treatment. In order to test this research question, we
conducted an Analysis of Covariance (ANCOVA). Experiment condition was entered as the
independent variable, and the calculation skill score was used as the dependent variable. Since
number of calculus classes taken would potentially affect one’s calculation skills, we controlled
for the number of calculus classes taken as a covariate.
The result showed that there were no significant difference between the three conditions, F(2,
113)=.20, p=.818. This suggests that the game condition (M=1.30, SD=1.57) did not perform
significantly better than the homework condition (M=1.39, SD=1.28) or the control condition
(M=1.18, SD=1.17) in terms of calculation skills. The findings indicate that playing an hour of
the game was not more effective in improving the students’ calculation skills than doing practice
quiz or no treatment at all. However, the findings can also be interpreted as that the game
performed similar to the traditional method of doing practice quizzes. A surprising finding was
that the practice quiz did not outperform the control group in terms of calculation skills, perhaps
because the practice quiz did not help students retrieve relevant knowledge and improve their
calculation skills.
4.4. Conceptual understanding
One of the key arguments for using digital games for mathematics education is that it has
the potential to facilitate conceptual understanding. Hypothesis 3 posited that participants in the
game condition would perform better than the homework and control condition in their
understanding of calculus concepts. We conducted another ANCOVA similar to the one for
calculation skills, this time with conceptual understanding as the dependent variable. The result
showed that there was a significant difference between the three conditions, F(2, 113)=5.22,
p=.007, eta squared=.08. Post-hoc comparison showed that the game condition (M=2.10,
SD=1.26) was significantly higher than the practice quiz condition (M=1.57, SD=1.04) and the
control condition (M=1.32, SD=1.01). Again, the practice quiz condition was not significantly
different from the control condition (p=.747). The data was consistent with hypotheses H3a and
H3b. The findings indicate that the game was more effective in improving students’
understanding of calculus concepts than doing one hour of practice quiz or no treatment at all.
4.5. Feedback, visual manipulation and conceptual understanding
Literature suggests that games promote conceptual understanding through giving players
feedback and the affordance of visual manipulations (e.g., Delacruz, 2012; Tall, 1991).
Hypotheses 4 and 5 focused on the game condition to investigate whether number of feedback
and visual manipulations predicts conceptual understanding. After removing seven participants
who were either idling during the gameplay or did not play for the required duration, a total of 43
participants were included in the analyses. Because not all the participants were able to complete
all the scenarios within the one hour experiment time, we focused on analyzing player behavioral
data in the first scenario, which is where players learn the mechanics of the game and the basic
concepts. On average, players received 37.72 feedback (SD=45.43, ranging from zero to 195)
about their performance and how to correct their mistakes. On average, the students in the game
condition actively used the visual manipulation 6.12 times (SD=5.67, ranging from zero to 27
times) in scenario 1. We define “actively used” as usage beyond the required usage in the tutorial
phase of scenario 1. Because the behavioral tracking system tracked each visual manipulation as
a separate action and did not distinguish between one long move and a short move, this makes it
difficult to interpret as some players may use one long move while other use several small moves
to obtain the same distance. Therefore, we recoded the data into a dichotomous variable of
whether the player actively used the visual manipulation function or not in the analysis.
In order to test hypotheses 4 and 5, we conducted an OLS regression. The independent
variables were number of feedback and whether the player actively used the function of visual
manipulations. Conceptual understanding was entered as the dependent variable. The overall
model was significant, F(2, 31)= 4.00, p=.028, 𝑎𝑑𝑗 𝑅! =.15.
Visual manipulation was a significant predictor of conceptual understanding, β=.36,
t=2.19, p=.036. Participants who actively used the visual manipulation function (M=1.83,
SD=1.21) performed better in conceptual understanding than participants who did not use visual
manipulations (M=.88, SD=1.11). The result was consistent with hypothesis 4; the function of
visual manipulation seems to improve students’ conceptual understanding.
Number of feedback was also a significant predictor of conceptual understanding, but in
the opposite direction, β=-.39, t=-2.33, p=.026. The result was not consistent with hypothesis 5.
The finding suggests that students who were exposed to more feedback actually performed worse
in the conceptual understanding. This maybe because players only received feedback when they
make a mistake, therefore the number of feedback may also be an indication of the players’
performance. When we eliminated students with extremely high number of feedback (i.e. outliers
with more than 50 feedback) in the first scenario (n=8) from the analysis, number of feedback
did not negatively predict conceptual understanding, suggesting that the students with extremely
high number of feedback may have skewed the analyses.
5. General Discussion
This study was designed with two main goals in mind. The first goal was to compare the
effectiveness of a digital game approach to a traditional approach in promoting calculus
conceptual understanding, calculation skills, and sense of immersion and control among
undergraduate students. The second goal was to examine actual behavioral data to test whether
the affordances digital games to support visual manipulation and feedback improved students’
conceptual understanding of calculus.
In line with previous literature (e.g., Ke, 2008a, 2008b; Kebritchi, Hirumi, & Bai, 2010;
Kim & Chang, 2010; Liang & Zhou, 2009; Ota & DuPaul, 2002), we found that students felt
more immersed in the task when learning through the game in comparison to doing practice
quizzes or no treatment. Digital games communicate through designed problems that invite
players to solve with limited resources (Gee, 2007). In other words, digital games can facilitate
what Siegler (2009) calls purposeful engagement. When students understand the purpose for
learning, they are more likely to allocate attention to the task and cognitively process the
information because they are motivated to solve the problems. Previous literature also suggest
that students felt more ownership and responsibility for their own learning (Liang & Zhou, 2009).
However, in our study, the students who played the game did not report higher sense of control
than the students who solved the practice quiz.
Previous studies on using digital games for mathematics education have shown mixed
results in its effectiveness to promote calculation skills (see Connolly et al., 2012 for review).
Our result showed that students who played the game did not perform better than students who
solved a practice quiz or received no treatment at all. This may be because the game Mission
Prime was designed to promote conceptual understanding and intentionally deemphasized
calculation and formula memorization in the game. However, had the game required more
calculation, the calculation could potentially decrease players’ sense of immersion because the
gameplay flow would be constantly interrupted by calculations. What was surprising was that the
paper-and-pencil practice quiz did not improve calculation skills than no treatment. The most
common instructional goal of practice quizzes in education is to let students practice and refresh
their memory of materials that they learned in class (Cooper, Robinson, & Patall, 2006). A
practice quiz can also act as a self-assessment for student to understand how well they
understand. However, students may not benefit from practice quiz alone. Without guidance or
feedback to help students understand how well they are doing, and how to correct their mistakes
and improve, students cannot learn from doing quizzes. Instead, students may feel demotivated
because they do not see the purpose of the practice quiz.
In terms of promoting conceptual understanding, our results found that undergraduate
students who played the game had deeper conceptual understanding than students who solved a
practice quiz or no treatment. Mathematics educators have argued that in order to promote
conceptual understanding of calculus, it is suggested that calculus could be taught by providing
students with a general understanding of its application, and then supporting self-paced learning
through feedback and visual representations (e.g., Disessa & Sherin, 2000; Kaput, 1994; Tall,
1991). Digital games can afford this type of educational design by providing meaningful problem
for students to solve, constant feedback on the students’ process, and allow students to
manipulate multiple visual representations that explain the underlying concepts.
Next, we focused on the game condition to investigate whether the number of feedback
messages received and the visual manipulations predicted better conceptual understanding. Our
analyses using behavioral-tracking data of the students’ behavior in during gameplay showed
that both number of feedback and visual manipulation predicted conceptual understanding, but in
the opposite direction. Students who actively manipulated the visual representation could explain
the underlying concepts better, indicating that they have a deeper or more complete
understanding of the concepts involved in solving the problems. However, students who received
large numbers of feedback performed worse in conceptual understanding. One possible
explanation of this finding is that in the game, feedback appears after the player has made an
incorrect attempt. Therefore, a large number of feedback may also indicate that the player is
performing poorly and that the feedback were not effective in improving the player’s
understanding. Another possible explanation is that some students were simply trying random
combinations without putting much thought into reading the feedback. For example, one student
received as much as 195 feedback in first scenario alone. Because these students were not
cognitively processing the feedback message, their understanding may not have improved.
Overall, we found that students reported the gameplay experience as more immersive
than traditional practice quiz, and students who played the game also gained a deeper
understanding of the underlying concepts that were used to solve the problems in the game.
There was no significant difference in terms of improving calculation skills. When examining
what caused students in the game condition to have better conceptual understanding, behavioraltracking data revealed that the students who actively manipulated the visual representation
performed better when asked to explicate their understanding of the concepts. This finding
supports theories that argued the affordances of manipulating visual representations and
presenting abstract mathematical concepts through multiple representations can promote deeper
understanding of the concepts and its applications.
5.1. Limitations
This study was conducted in a controlled laboratory setting with random assignment.
Therefore, external factors such as teachers’ ability, classroom environment, duration of class,
etc. were controlled in the experiment. As previous researcher have argued, the effectiveness of
digital games in classroom education largely depend on the dynamics between the learners, the
instructors, curriculum design, and the game design (Ke, 2008a). The findings from this study
are the results of comparing the digital game approach to practice quiz and no treatment
approach in isolation and should be generalized with caution. In a classroom setting, students
may be able to seek additional support from their instructors or fellow students, which may
supplement the game or practice quiz to improve learning effect. Other students in the classroom
may distract players of the game or students may feel fatigued because of the long duration of
classes, which may decrease the effect of the game or the practice quiz. By eliminating these
external factors that can influence the students’ performance, we are able to focus on comparing
the two different approaches and its affordances.
Due to resource and time constraints, students in this study only played the game once for
an hour in the laboratory. Longer gameplay time and repeated play may increase the
effectiveness of digital game based learning.
The participants in this study were recruited from calculus classes; they have already
learned the materials covered in the game and the practice quiz. Therefore the results should be
interpreted as the effect of using digital game or practice quiz to enhance calculus classroom
education, not the effect of using digital games or practice quiz as an initial learning approach.
6. Conclusion
Few studies have examined the application of digital game based learning in
undergraduate-level mathematics education; fewer studies have examined the actual gameplay
processes to identify affordances of digital games that promote deep cognitive understanding of
mathematics. Findings from this study suggest that a well-designed digital game can be used to
promote student motivation and conceptual understanding in undergraduate-level calculus
education. Some mathematics educators have argued that a problem-based learning approach and
visual manipulations can motivate learners to take on an active role in learning, and through
manipulating and experimenting with visual representations, construct a deep understanding of
the underlying concepts (C.-Y. Lee & Chen, 2009; Stanley, 2002; Tall, 1991). This study shows
that digital games may provide meaningful problems that immerse students in tasks and support
manipulation of visual representations, which are key factors that predict better conceptual
understanding of the mathematics and its application.
Does this imply that digital games will always improve conceptual understanding over
traditional approaches? Not necessarily. The effect of any educational design is ultimately
determined by the interaction between the instructor, learner, content, and context. While some
instructors can effectively incorporate digital game based learning into their course design,
others may be able to motivate authentic learning and conceptual understanding without using
digital games. Future studies should test the effects of digital games in conjunction with
traditional approaches, and replicate the study in classroom setting to observe how students and
instructors interact with the new medium, especially if the digital games are designed to target
persistent problem areas in undergraduate calculus education.
Future instructional game designers also need to put more efforts into making sure
students understand the affordances and feature of the game that support learning. For example,
while the game Mission Prime provided the function of visual manipulation, not all the students
actively used the function. Findings suggest that students who actively used the function
performed better in conceptual understanding. A challenge for instructional games at more
advanced mathematical levels is the balance between maintaining engagement in the game and
attaining complex learning goals. Mission Prime intentionally emphasized conceptual
understanding of optimization over computations, in part to maintain game engagement. If
learning goals are more computational in nature, a game would necessarily require creative
design solutions so that performing computations does not disrupt gameplay.
References
Agarwal, R., & Karahanna, E. (2000). Time Flies When You're Having Fun: Cognitive
Absorption and Beliefs about Information Technology Usage. MIS Quarterly, 24(4), 665694.
Anderson, J. R., Greeno, J. G., Reder, L. M., & Simon, H. A. (2000). Perspectives on learning,
thinking, and activity. Educational Researcher, 29(4), 11-13.
Betz, J. A. (1995). Computer games: Increase learning in an interactive multidisciplinary
environment. Journal of Educational Technology Systems, 24(2), 195-205. doi:
10.2190/119M-BRMU-J8HC-XM6F
Boyle, E. A., Connolly, T. M., & Hainey, T. (2011). The role of psychology in understanding the
impact of computer games. Entertainment Computing, 2, 69-74. doi:
http://dx.doi.org/10.1016/j.entcom.2010.12.002
Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind,
experience, and school: National Academy Press.
Charsky, D. (2010). From edutainment to serious games: A change in the use of game
characteristics. Games and Culture, 5(2), 177-198. doi: 10.1177/1555412009354727
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic
literature review of empirical evidence on computer games and serious games.
Computers & Education, 59(2), 661-686. doi:
http://dx.doi.org/10.1016/j.compedu.2012.03.004
Cooper, H., Robinson, J. C., & Patall, E. A. (2006). Does homework improve academic
achievement? A synthesis of research, 1987–2003. Review of educational research, 76(1),
1-62. doi: 10.3102/00346543076001001
Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of learning:
Beneficial effects of contextualization, personalization, and choice. Journal of
educational psychology, 88(4), 715. doi: 10.1037/0022-0663.88.4.715
Delacruz, G. C. (2012). Impact of incentives on the use of feedback in educational videogames
CRESST report (Vol. 813, pp. 1 - 18).
Disessa, A. A., & Sherin, B. L. (2000). Meta-representation: An introduction. The Journal of
Mathematical Behavior, 19(4), 385-398.
Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and
practice model. Simulation & gaming, 33(4), 441-467. doi: 10.1177/1046878102238607
Gee, J. P. (2007). Good Video Games + Good Learning: Collected essays on video games,
learning and literacy (Vol. 27): Peter Lang.
Heeter, C., Lee, Y.-H., Medler, B., & Magerko, B. (2013). Conceptually meaningful metrics:
Inferring optimal challenge and mindset from gameplay Game Analytics (pp. 731-762):
Springer.
Kaput, J. J. (1994). The representational roles of technology in connecting mathematics with
authentic experience. Didactics of mathematics as a scientific discipline, 379-397.
Ke, F. (2008a). A case study of computer gaming for math: Engaged learning from gameplay?
Computers & Education, 51(4), 1609-1620. doi: 10.1016/j.compedu.2008.03.003
Ke, F. (2008b). Computer games application within alternative classroom goal structures:
cognitive, metacognitive, and affective evaluation. Educational Technology Research
and Development, 56(5-6), 539-556. doi: 10.1007/s11423-008-9086-5
Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modern mathematics computer
games on mathematics achievement and class motivation. Computers & Education, 55(2),
427-443. doi: 10.1016/j.compedu.2010.02.007
Kidron, I., & Zehavi, N. (2002). The Role of Animation in Teaching the Limit Concept.
International Journal of Computer Algebra in Mathematics Education, 9(3), 205-227.
Kim, S., & Chang, M. (2010). Computer Games for the Math Achievement of Diverse Students.
Educational Technology & Society, 13(3), 224-232.
Kolb, A. Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing experiential
learning in higher education. Academy of management learning & education, 4(2), 193212.
Lee, C.-Y., & Chen, M.-P. (2009). A computer game as a context for non-routine mathematical
problem solving: The effects of type of question prompt and level of prior knowledge.
Computers & Education, 52(3), 530-542. doi: 10.1016/j.compedu.2008.10.008
Lee, Y.-H., Heeter, C., Magerko, B., & Medler, B. (2012). Gaming mindsets: Implicit theories in
serious game learning. Cyberpsychology, Behavior, and Social Networking, 15(4), 190194. doi: 10.1089/cyber.2011.0328.
Lesh, R., & Zawojewski, J. (2007). Problem solving and modeling. Second handbook of research
on mathematics teaching and learning, 2, 763-804.
Liang, X., & Zhou, Q. (2009). Students' experiences of mathematics learning in technology
integrated classrooms. Teaching and Learning, 5(1), 62-74.
Lithner, J. (2004). Mathematical reasoning in calculus textbook exercises. The Journal of
Mathematical Behavior, 23(4), 405-427. doi: 10.1016/j.jmathb.2004.09.003
Lopez-Morteo, G., & López, G. (2007). Computer support for learning mathematics: A learning
environment based on recreational learning objects. Computers & Education, 48(4), 618641. doi: 10.1016/j.compedu.2005.04.014
Mayo, M. J. (2009). Video games: A route to large-scale STEM education? Science, 323(5910),
79-82. doi: 10.1126/science.1166900
Morgan, P., & Ritter, S. (2002). An experimental study of the effects of Cognitive Tutor®
Algebra I on student knowledge and attitude. Pittsburgh, PA: Carnegie Learning, Inc.
Oehrtman, M. (2009). Collapsing dimensions, physical limitation, and other student metaphors
for limit concepts. Journal for Research in Mathematics Education, 396-426.
Ota, K. R., & DuPaul, G. J. (2002). Task engagement and mathematics performance in children
with attention-deficit hyperactivity disorder: Effects of supplemental computer
instruction. School Psychology Quarterly, 17(3), 242. doi: 10.1521/scpq.17.3.242.20881
Rose, H., & Betts, J. R. (2004). The effect of high school courses on earnings. Review of
Economics and Statistics, 86(2), 497-513. doi: 10.1162/003465304323031076
Shaffer, D. W. (2006). How computer games help children learn: Macmillan.
Siegler, R. (2009). Implications of cognitive science research for mathematics education.
Spotnitz, S. (2001). Intrinsic motivation in students with learning disabilities as examined
through computer based instruction in mathematics. Columbia University, unpublished
thesis.
Squire, K. (2003). Video games in education. Int. J. Intell. Games & Simulation, 2(1), 49-62.
Squire, K., Jenkins, H., Holland, W., Miller, H., O'Driscoll, A., Tan, K. P., & Todd, K. (2003).
Design Principles of Next-Generation Digital Gaming for Education. Educational
Technology, 43(5), 17-23.
Stanley, S. S. (2002). Revitalizing precalculus with problem-based learning. The Journal of
General Education, 51(4), 306-315.
Steinkuehler, C., & Duncan, S. (2008). Scientific habits of mind in virtual worlds. Journal of
Science Education and Technology, 17(6), 530-543. doi: 10.1007/s10956-008-9120-8
Tall, D. (1991). Advanced mathematical thinking (Vol. 11): Springer.
Van Eck, R. (2006). Digital game-based learning: It's not just the digital natives who are restless.
EDUCAUSE review, 41(2), 16.
Figure 1. Example of a problem scenario in Mission Prime
Figure 2. Example of visual manipulation in Mission Prime