Association for Information Systems
AIS Electronic Library (AISeL)
PACIS 2014 Proceedings
Pacific Asia Conference on Information Systems
(PACIS)
2014
THE ROLES OF INTRINSIC AND
EXTRINSIC MOTIVATION AND
PERCEIVED COMPETENCE IN
ENHANCING SYSTEM USE AND
PERFORMANCE
Siew H. Chan
Nova Southeastern University, [email protected]
Qian Song
Rochester Institute of Technology, [email protected]
Laurie E. Hays
Western Michigan University, [email protected]
Pailin Trongmateeru
Thammasat University, [email protected]
Follow this and additional works at: http://aisel.aisnet.org/pacis2014
Recommended Citation
Chan, Siew H.; Song, Qian; Hays, Laurie E.; and Trongmateeru, Pailin, "THE ROLES OF INTRINSIC AND EXTRINSIC
MOTIVATION AND PERCEIVED COMPETENCE IN ENHANCING SYSTEM USE AND PERFORMANCE" (2014). PACIS
2014 Proceedings. 382.
http://aisel.aisnet.org/pacis2014/382
This material is brought to you by the Pacific Asia Conference on Information Systems (PACIS) at AIS Electronic Library (AISeL). It has been
accepted for inclusion in PACIS 2014 Proceedings by an authorized administrator of AIS Electronic Library (AISeL). For more information, please
contact [email protected].
THE ROLES OF INTRINSIC AND EXTRINSIC MOTIVATION AND
PERCEIVED COMPETENCE IN ENHANCING SYSTEM USE AND
PERFORMANCE
Siew H. Chan, H. Wayne Huizenga School of Business & Entrepreneurship, Nova Southeastern
University, Fort Lauderdale-Davie, FL, USA, [email protected]
Qian Song, Saunders College of Business, Rochester Institute of Technology, Rochester, NY,
USA, [email protected]
Laurie E. Hays, Haworth College of Business, Western Michigan University, Kalamazoo, MI,
USA, [email protected]
Pailin Trongmateerut, Thammasat Business School, Thammasat University, Bangkok, Thailand,
[email protected]
Abstract
This study builds on the extant literature on motivation and information systems by examining the
mediating role of intrinsic motivation in the relationship between system type and system use, the
moderating role of perceived usefulness in the effect of intrinsic motivation on system use, and the
moderating role of perceived competence in the impact of system use on performance. This study
manipulates three system types; that is, PATH (Principles Aren’t That Hard), Blackboard, and the
traditional paper medium, and measures the participant’s intrinsic motivation, perceived usefulness,
perceived competence, system use, and performance. PATH incorporates interest-enhancing features,
Blackboard has limited interest-enhancing features, and the traditional paper medium does not have
these attributes. A total of 173 undergraduate students enrolled in the introductory financial accounting
course participated in this study. The structural equation model results provide support for the
hypotheses in the research model. An important contribution of this study is development of an
educational computer game, PATH, and inclusion of Blackboard and the traditional paper medium to
facilitate comparison of the level of intrinsic motivation associated with each system type. Another
contribution is administration of the treatment variable (i.e., system type), measurements of the key
constructs, and direct assessment of the participants’ performance in the same experimental setting.
Keywords: Intrinsic motivation, Perceived competence, Perceived usefulness, Educational game
features, System type, System use, Performance.
1. INTRODUCTION
A National Summit on Educational Games investigates the features of computer games to be
incorporated into the U.S. education system and concludes that powerful features of computer games can
improve learning and enhance the quality of education. Although previous studies have indicated the
beneficial effect of educational computer games on learning performance in various domains including
the business field such as strategic management (e.g., Vandercruysse et al. 2012; Wolfe 1997; Wouters et
al. 2009; Pasin & Giroux 2011), the findings have not been consistent (Leemkuil & De Jong 2012). An
extensive analysis indicates the absence of instructional design in most game-based learning studies and
only about 10 percent of the studies can be considered to be based on learning theory (Wu et al. 2012).
Instructional design is a necessary but not sufficient condition for enhanced learning; specifically, learner
support in a game-based learning environment is essential for obtaining the beneficial effects of learning
(Garris et al. 2002; Knotts & Keys 1997; O'Neil et al. 2005). Computer simulation can act as a platform
for enhancing game-based learning (Leemkuil and De Jong 2012). The game-based strategy is more
effective at enhancing learning compared to the case-based approach (Wolfe 1997) and the traditional
approach where mastery of decision-making ability is essential for solving complex and dynamic
problems (Pasin & Giroux 2011). However, the evidence on assessment of business games is primarily
anecdotal, descriptive or judgmental (Bell et al. 2008; Salas et al. 2009). Thus, empirical studies are
needed to assist the formation of meaningful conclusions on the positive influence of educational
computer games on performance.
Empirical evidence is needed to assist the design of effective games to obtain positive educational
objectives and outcomes (De Freitas 2006). Increased efforts have been directed at exploring
advantageous game features such as fantasy, interactivity, and sensory stimuli of value to education (Law
& Sun 2012). The present study builds on previous research by designing a computer game named
Principles Aren’t That Hard (PATH) to facilitate learning of complex abstract accounting concepts which
requires higher order cognition, an area where empirical work is scarce. The current study includes
Blackboard and the traditional paper medium as additional system types for comparison purposes.
Specifically, system type is a treatment condition. This experimental treatment is necessary for measuring
the level of intrinsic motivation associated with each system type. We propose and test a research model
that investigates (1) the mediating effect of intrinsic motivation in the relationship between system type
(i.e., PATH, Blackboard or the traditional paper medium) and system use, (2) the moderating role of
perceived usefulness in the effect of intrinsic motivation on system use, and (3) the moderating role of
perceived competence in the impact of system use on performance.
Intrinsically motivated individuals perform a task or activity out of interest and enjoyment (Deci &
Ryan 1985), and this engagement is likely to be sustainable (Deci & Ryan 2000). A unique characteristic
of educational computer games is sustainability of user attention and acquisition of essential information
for fulfilling the objectives of learning. Interesting-enhancing features facilitate intrinsic motivation which
increase user engagement in and sustainability of system use. PATH, an educational computer game
approach, incorporates interest-enhancing features such as interesting computer simulation, enhanced
user-computer interaction, appealing interface design, and immediate system feedback (available in
certain PATH modules). Hence, intrinsic motivation is predicted to be highest for PATH users.
Blackboard, a widely used tool, has limited interest-enhancing features compared to PATH. In particular,
Blackboard does not have an educational game context or simulation for facilitating user-system
interaction. Thus, the intrinsic motivation of Blackboard users is expected to be lower than that of PATH
users. Further, since the linear presentation format of the traditional paper method lacks interestenhancing features, intrinsic motivation is predicted to be lowest for this group of users.
Despite the importance of interesting educational game features in promoting learning of technical
materials such as accounting, it is imperative that researchers and practitioners recognize that the purpose
of educational computer games is to focus on instruction rather than entertainment. Therefore, educational
computer games should be designed to obtain desirable learning outcomes and specific learning goals. An
educational computer game is not a transformation of an entertainment game or the traditional paper
medium. Instead, educational computer games should entail educational value with an appropriate level
of fun to motivate increased engagement in the games (De Freitas 2006). The value perceived in the
educational computer games (i.e., perceived usefulness) is another determining factor of system use, an
important construct that has received extensive attention in previous research (Saadé & Bahli 2005;
Johnson et al. 2008). In addition, consistent with Davis et al. (1992), this study examines perceived
usefulness as a type of extrinsic motivation which affects the strength of intrinsic motivation on system
use. Further, the inconsistent findings of previous research (e.g., Hou 2012; Leemkuil & DeJong 2012;
Szajna 1993) suggest that system use may not necessarily lead to positive learning outcomes. The current
study provides insight into the contradictory results by identifying perceived competence as a moderator
which influences the strength of system use on performance.
A total of 173 undergraduate students enrolled in the introductory financial accounting course
participated in this study. They completed their homework using PATH, Blackboard, or the traditional
paper medium. The experiment spanned an entire semester. Since this study proposes and tests a research
model, structural equation modeling (SEM) is used to test the hypotheses. The results provide support for
the hypotheses in the research model.
This study contributes to the extant literature on intrinsic motivation, system use, and performance.
Specifically, the present study identifies intrinsic motivation as a significant factor where engagement in
and increased usage of an educational computer system such as PATH is necessary for obtaining the
benefits of this game-based learning strategy. PATH includes an instructional design which focuses on
the educational game context for facilitating learning of technical materials. In an experimental setting,
we measure the participants’ intrinsic motivation pertaining to each system type; namely, PATH,
Blackboard and the traditional paper medium. This approach facilitates the formation of meaningful
inferences on the level of intrinsic motivation associated with each system type. Further, the extant
literature has investigated the positive impact of perceived usefulness on system use (e.g. Saadé& Bahli
2005; Johnson et al. 2008). This study extends previous research by examining perceived usefulness as a
form of extrinsic motivation which interacts with intrinsic motivation to exert a positive effect on system
use. The present study illustrates a situation where intrinsic motivation is not undermined by extrinsic
motivation due to the positive information content inherent in the perceived usefulness construct which
increases system use. In addition, perceived competence has been proposed as a process variable or
mediator in prior motivation studies (e.g., Elliot & Harackiewicz 1994; Harackiewicz & Elliot 1993,
1998; Harackiewicz & Larsen 1986). The current study builds on previous motivation research by
examining perceived competence as a moderator which affects the strength of system use on
performance. Specifically, the moderating effect of perceived competence may shed light on the
inconsistent results reported in prior research on the impact of system use on performance.
The next section explains the theory that leads to the hypotheses. The following two sections
address the experimental approach for testing the hypotheses and the results respectively. Finally, the
implications of this study and suggestions for future research are discussed.
2. THEORY AND HYPOTHESES
2.1 Educational Computer Games-PATH
Computer games and simulations assist users to develop abilities, master a new skill, accomplish a
challenging activity, and understand learning materials (Meece et al. 2006). Computer games also
improve decision-making and problem-solving skills (Wolf 1972) and provide training for reasoning
algorithms (von Ahn & Dabbish 2008). Researchers and practitioners are exploring advantageous features
of computer games and simulations for education purpose (Law & Sun 2012). For example, Civilization,
a computer game on how world civilizations develop and grow, has been used to teach history,
geography, science, arts, etc. in classrooms around the world. Teachers have also used SimCity, a city
planning computer game, to reinforce concepts relating to physics (Kirriemuir & McFarlane 2003). In
addition, computer simulation can be introduced in introductory courses to enhance learning of basic
concepts and to reinforce skills in upper-level courses (Butler et al. 1979).
A review of the literature identifies the following game features: fantasy, goals, sensory stimuli,
challenge, and control (Garris et al. 2002). These features are of educational interest because they are
more engaging and immersive than the traditional paper format. Instructional content embedded in
fantasy contexts promotes student interest and learning (Cordova & Lepper 1996). Fantasies facilitate
focused attention and self-absorption when users are immersed in the games (Garris et al. 2002). Sound
effects, dynamic graphics, and other sensory stimuli enhance fantasy and attention (Malone & Lepper
1987; Garris et al. 2002). For example, students choose to return to practice activities that include
dynamic graphics (Rieber 1991). User-friendly interface and simulation also increase user interaction and
cognitive processing, resulting in enhanced learning (Bryant & Hunton 2000). Sensory stimuli can be
used effectively as feedback for performance (Malone 1980; Wilson et al. 2009). Performance feedback
and score keeping enable one to track progress toward desired goals (Garris et al. 2002). Challenging
features such as scores and reward (e.g., bonus questions and the concept of “lives”) reinforce positive
behavior. Empirical studies reveal that challenge is associated with improved performance (Keller &
Bless 2008). Hence, goals are meaningful when activities are linked to valued personal competencies,
embedded in absorbing fantasy scenarios, or facilitated via competitive motivations. Specifically,
embedding activities in fantasy contexts allows one to encounter imaginary situations that differ from our
knowledge of how things work in the real world which stimulates curiosity (Garris et al. 2002).
This study designs an educational computer game, PATH, which incorporates game and simulation
features to promote motivation for learning, cognitive and affective learning, and interactivity (O’Neil et
al. 2005). Users can also progress through the PATH modules at their own pace, leading to increased
perception of autonomy. The intangible reward systems (e.g., bonus questions and the concept of lives) as
well as challenging features embedded in PATH are postulated to enhance learning and performance.
Challenging features that facilitates objective outcomes and performance feedback help students to
interpret their success in achievement situations (Meece et al. 2006). Further, the explanation and/or
performance feedback incorporated into PATH assists users to monitor their progress and promote
positive attitude toward learning. Immediate scoring also communicates specific goals and current
performance which facilitate commitment and self-improvement (Locke & Latham 1990). When
performance is below desired goals, individuals respond to the feedback-standard discrepancy by
increasing their effort to attain the standard (Garris et al. 2002).
2.2 The Mediating Role of Intrinsic Motivation
Intrinsic motivation pertains to performance of an activity for the sake of pleasure and satisfaction
inherent in the activity itself (Deci 1971; Deci & Ryan 1985). When individuals are intrinsically
motivated, they are willing to devote effort because of the interest and enjoyment derived from
engagement in an activity (Ryan & Deci 2000). Thus, intrinsic motivation exerts a more powerful and
sustainable influence on individual behavior than external rewards (Deci & Ryan 2000). This study
predicts that use of a particular system type; that is, PATH, Blackboard or the traditional paper medium
creates different levels of intrinsic motivation. Specifically, PATH users are expected to exhibit the
highest intrinsic motivation followed by Blackboard and the traditional medium users. PATH comprises
interest-enhancing features such as computer simulation, interactivity, and appealing interface design. In
contrast, Blackboard has limited interest-enhancing features while the traditional paper medium lacks
interest-enhancing features. The interactivity1 embedded in PATH has a positive influence on individuals’
1
Interactivity is defined as a characteristic that allows individuals to adjust the instruction to conform to their needs
and capabilities (Weller 1988). Interactive systems provide a variety of features of user control; the most common is
attitude toward the learning content because this favorable format for imparting knowledge increases user
control and motivation (Kettanurak et al. 2001). An interactive learning environment enhances
satisfaction (Liaw & Huang 2013).
Information systems research has investigated the pivotal role of intrinsic motivation in system
usage (e.g., Davis et al. 1992; Malone 1981a, 1981b, Webster & Martocchio 1992; Venkatesh & Speier
1999, 2000; Venkatesh 2000). Motivation theory has also been adapted extensively to fit specific contexts
to promote understanding of adoption and use of technology (e.g., Davis et al. 1992; Venkatesh & Speier
1999). Interfaces that reduce cognitive effort or incorporate a certain degree of challenge promote the
level of interest in a task which enhances system usage (Pilke 2004). Previous research has also explored
levers such as autonomous job design and socialization tactics to enhance intrinsic motivation to increase
system usage (Ke et al. 2013). Consistent with prior research (e.g., Deci 1975; Venkatesh 2000), this
study postulates that intrinsic motivation has a positive impact on system usage. The expected positive
relationships between system type (i.e., PATH, Blackboard, or the traditional paper medium) and intrinsic
motivation, and between intrinsic motivation and system usage suggest the presence of a mediating effect
of intrinsic motivation in the relationship between system type and system use. This leads to the first
hypothesis as follows:
H1: Intrinsic motivation mediates the effect of system type on system use.
2.3 The Moderating Role of Perceived Usefulness
Perceived usefulness refers to the belief that use of a system or technology will enhance one’s
performance (Davis 1989). Previous research has reported a positive relationship between perceived
usefulness and system use (Saadé& Bahli 2005; Johnson et al. 2008). Technology can add value to users
by expanding the quality or quantity of information, and allowing users to manage and control their
learning process by taking advantage of the technology’s increased flexibility and convenience in learning
(i.e., technology removes the constraints associated with time and place) (Johnson et al. 2008). Enhanced
learning ensues when users have greater control over the pace, flow, and interaction with a system
(Wydra 1980; Johnson et al. 2008). A meta-analysis concludes that training transfer occurs when one
perceives value in the training activity (Alliger et al. 1997). Users are also likely to use technology when
they see the value of such usage; that is, enhanced learning and increased control over the learning
process (Hornik et al. 2007).
Perceived usefulness is considered as a type of extrinsic motivation (Davis et al. 1992). Motivation
theory suggests that human behaviors are driven by both intrinsic motivation and extrinsic motivation,
and extrinsic motivation can either facilitate or undermine the effect of intrinsic motivation on individual
behavior (Ryan & Deci 2000). When people are rewarded for the performance of an interesting activity,
they exhibit less interest in and willingness to work on the same activity upon termination of the reward
compared to those who had participated in the activity without any reward (Deci & Ryan 1987). Deci and
Ryan (1980) define this phenomenon as the undermining effect, and contend that this occurs when
rewards are expected (Lepper et al. 1973), salient (Ross 1975), and contingent on task engagement (Ryan
et al. 1983). Prior motivation studies reveal that participants consider material rewards (Deci, 1971,
1972), threats of punishment (Deci & Cascio 1972), evaluations (Smith 1974), deadlines (Amabile et al.
1976), imposed goals (Mossholder 1980), and good player awards (Lepper et al. 1973) as factors with a
controlling function that undermine intrinsic motivation. Although one’s intrinsic task interest may be
debilitated by the prospect of reward during task performance, this effect may be offset by enhanced
performance motivated by the expectation of reward (Deci & Ryan 1985b). While extrinsic rewards may
debilitate intrinsic motivation when the controlling aspect of the evaluative contingencies is elicited,
control over the instructional pace and sequencing/branching (Kettanurak et al. 2001). Additionally, interactive
systems can accommodate different learning styles and abilities in terms of the nature of learning and interaction,
speed, or information content (Cohen 1984).
extrinsic incentives can promote intrinsic motivation when the evaluative outcome provides competency
information to the user (Harackiewicz et al. 1984).
The extant literature has examined the impact of perceived usefulness on system use (e.g., Adams
et al. 1992; Davis 1989; Hendrickson et al. 1993; Segars & Grover 1993; Subramanian 1994; Szajna
1994). The present study is different from prior research in that we examine the moderating role of
perceived usefulness in the relationship between intrinsic motivation and system use. Specifically, this
study provides insight into how higher (lower) perceived usefulness will strengthen (weaken) the positive
impact of intrinsic motivation on system use. Thus,
H2: The effect of intrinsic motivation on system use is stronger when perceived usefulness
is higher than lower.
2.4 The Moderating Role of Perceived Competence
Despite the advantages of educational computer games in enhancing learning, the literature is
equivocal on the relationship between system use and learning outcome (Leemkuil & DeJong 2012). In
addition, while some studies report a positive effect of system use on individual performance (Doll &
Torkzadeh 1998; Goodhue & Thompson 1995; Igbaria & Tan 1997; Liedner & Elam 1993; Doll &
Torkzadeh 1990; Hou 2012), other studies find a negative impact (Pentland 1989; Szajna 1993) or an
insignificant effect (Gelderman 1998; Lucas & Spitler 1999) of system use on performance. The current
study attempts to provide insight into the contradictory results of system use on performance by
identifying perceived competence as a moderator that strengthens this relationship.
Perceived competence refers to feelings or perceptions of competence with respect to an activity or
domain (Deci & Ryan 1980; 1985). The extant intrinsic motivation research has demonstrated perceived
competence as an important process variable (Bandura & Schunk 1981; Elliot & Harackiewicz 1994;
Harackiewicz & Elliot 1993, 1998; Harackiewicz & Larsen 1986; Harackiewicz et al. 1984; Harackiewicz
et al. 1985; Manderlink & Harackiewicz 1984; Reeve & Deci 1996; Sansone 1986, 1989; Sansone et al.
1989). Prior research also suggests that perceived competence is a mediator of intrinsic motivation
(Bandura 1982; Deci & Ryan 1980; Harackiewicz & Sansone 1991) which exerts an enhancing and
sustaining influence on interest/enjoyment (Elliott et al. 2000).
In school-setting educational research, perceived competence is a significant predictor of domainspecific academic performance, after controlling for achievement test scores (Miserandino 1996). When
competence or autonomy is perceived as unfulfilled, participants report negative affect and withdrawal
behavior; consequently, their performance is impaired (Miserandino 1996). Previous studies also report a
positive relationship between the level of task performance and perceived competence regardless of the
presence of external rewards (Arnold 1985). Since individuals are intrinsically motivated when they
believe they are able to perform well in a task (Harackiewicz et al. 1985), perceived competence plays a
significant role in individual performance, especially when the learning materials are challenging.
In the context of course materials-mediated computer technology, it is essential that individuals
acquire knowledge, comfort, and confidence so that technology can be leveraged for maximum benefit
(Bandura 1997; Johnson et al. 2008). Perceived competence enhances goal attainments and provides
individuals with a sense of need satisfaction from engagement in an activity which they feel competent at
(Deci & Ryan 2000).
While prior motivation research has examined perceived competence as a process or mediator of
intrinsic motivation, this study promotes understanding of the moderating role of perceived competence in
the effect of system use on performance. Specifically, the present study contends that participants will
increase usage of the system when their perceived competence at using the system is enhanced, leading to
improved performance. The moderating impact of perceived competence may provide insight into the
inconsistent findings on the relationship between system use and performance. The final hypothesis is
formally stated as follows:
H3: The effect of system use on performance is stronger when perceived competence is
higher than lower.
3. EXPERIMENTAL METHOD
3.1 Participants
A total of 173 participants enrolled in four sections of the financial accounting course at a
university participated in the study2. Students in two sections (87) of the course used PATH while the
remaining two sections used Blackboard (46) and the traditional paper medium (40) respectively to
complete their homework3. About 72 percent of the participants were females and their average age was
204.
3.2 Procedures
The experiment entailed a longitudinal study that spanned an entire semester. The experiment
involved homework materials provided to participants via the following systems: PATH, Blackboard, or
the traditional medium (i.e., paper). The PATH group received a CD containing 10 modules of PATH
(i.e., the 10 homework assignments) during the first day of class. The Blackboard group accessed the
homework assignments via Blackboard. The homework assignments were in the format of multiplechoice questions. All the questions were programmed in a database on Blackboard. At the end of each
homework assignment, Blackboard showed the correct answers, incorrect answers and explanations. The
paper group received the homework assignments (in the form of multiple-choice questions) from the
instructor in class.
One of the authors administered the experiment during the semester. The experiment started with
several tasks on the first day of class. Except for the homework section, participants received the same
syllabus. The instructor informed the participants that the purpose of the study was to examine how
students learned the course materials. The instructor also assured the participants that the study would not
affect their work during the semester. Participants were told that all the consent forms would be kept in a
sealed envelope and analysis of the data would not commence until after all the final course grades had
been submitted. Participants then completed a 10-item questionnaire that assessed their prior knowledge
in accounting. Finally, they completed a questionnaire that collected data on their computer proficiency,
demographic information, and questions pertaining to their interest in accounting and prior exposure to
PATH. Participants who were absent during the first day of class were given the materials at the start of
the next class and requested to complete the questionnaires outside the classroom. This procedure was
necessary to ensure that the participants understand and complete the required materials before they
received their first instruction. During the last day of class, participants completed the intrinsic
motivation, perceived usefulness, and perceived competence scales. They also responded to questions on
the course, instructor, interest in accounting, and intention to major in accounting. The questionnaire was
similar for all participants, except for the wordings of the type of delivery systems (i.e., PATH,
Blackboard or paper) that they used to complete their homework.
2
The ANOVA results show that prior accounting knowledge, interest in accounting, and intention to major in accounting are
comparable across the treatment groups prior to the experiment.
3
As discussed later, this study employs SEM to test the hypotheses. SEM does not make distributional assumptions about
exogenous (i.e., independent) variables (Kline 2005). Thus, the unequal sample sizes in the three treatment conditions do not
affect the SEM results.
4
Gender does not have any effect on the results.
4. DATA ANALYSIS
4.1 Constructs
The Intrinsic Motivation Inventory (IMI) has been used in several studies (e.g., Ryan 1982; Ryan et
al. 1983; Plant & Ryan 1985; Ryan et al. 1990; Ryan et al. 1991; Deci et al. 1994). We modified the
interest/enjoyment, perceived usefulness, and perceived competence subscales (developed by Deci, Ryan
and colleagues) to fit the context of our study. Deci, Ryan, and colleagues consider the interest/enjoyment
scale as a self-report measure of intrinsic motivation and the only scale that assesses intrinsic motivation
(http://www.selfdeterminationtheory.org/questionnaires/10-questionnaires/50). Consistent with this
reasoning process, this study uses the interest/enjoyment as a self-report measure of intrinsic motivation.
The treatment variable, system type, refers to PATH, Blackboard, or the traditional paper medium.
System use pertains to the participants’ reported usage of their respective system to complete the
homework. Performance, an observed variable, is measured by each participant’s score (ranging from
zero to 10) on questions that assessed their competency in the course materials after usage of their
respective systems.
4.2 Results
Structural equation modeling (SEM) is employed to test the hypotheses proposed in the research
model. SEM is particularly useful when the theoretical model involves relationships among the latent
constructs and relationships between the latent constructs and the indictors of these constructs (Edwards
& Bagozzi 2000). SEM does not make distributional assumptions about exogenous (i.e., independent)
variables (Kline 2005). Thus, the unequal sample sizes in the three treatment conditions (i.e., PATH,
Blackboard and paper) do not affect the SEM results. The research model comprises three latent
constructs5; namely, intrinsic motivation, perceived usefulness, and perceived competence. System type,
system use, and performance are manifest variables6. The descriptive statistics as well as the Cronbach’s
alpha reliability results for the three latent constructs are presented in Table 1.
Measures
Intrinsic Motivation (Cronbach’s alpha=0.924)
IM1 While I was using (PATH, Blackboard or paper) to do
homework, I was thinking about how much I enjoyed it
IM2 I found using (PATH, Blackboard or paper) to do homework
very interesting.
IM3 Using (PATH, Blackboard or paper) to do homework was fun.
IM4 I enjoyed very much using (PATH, Blackboard or paper) to do
homework.
IM5 I thought it was very boring to use (PATH, Blackboard or
paper) to do homework.
IM6 I thought using (PATH, Blackboard or paper) to do homework
was very interesting.
IM7 I would describe using (PATH, Blackboard or paper) to do
5
Means
Factor
Scale End-points (Std Dev) Loadings
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
2.96
(1.57)
4.12
(1.73)
3.35
(1.82)
3.86
(1.82)
4.40
(1.62)
3.99
(1.52)
3.49
0.724
0.858
0.815
0.783
0.542
0.812
0.922
A latent construct is a theoretical construct measured by multiple indicators. A latent construct cannot be measured
directly.
6
A manifest (observed) variable can be observed directly; therefore, it does not behave like an indicator of a latent
construct.
homework as very enjoyable.
Perceived Usefulness (Cronbach’s alpha=0.850)
PU1 I believe that using (PATH, Blackboard or paper) to do
homework was of some value to me.
PU2 I think that using (PATH, Blackboard or paper) to do
homework is useful.
PU3 I think using (PATH, Blackboard or paper) to do homework is
important.
PU4 I would be willing to use (PATH, Blackboard or paper) to do
homework again because it has some value to me.
PU5 I think that using (PATH, Blackboard or paper) to do
homework is helpful.
PU6 I believe that using (PATH, Blackboard or paper) to do
homework could be beneficial to me.
Perceived Competence (Cronbach’s alpha=0.839)
PC1 I think I am pretty good at using (PATH, Blackboard or paper)
to do homework.
PC2 I think I did pretty well at using (PATH, Blackboard or paper)
to do homework, compared to other students.
PC3 I am satisfied with my performance at using (PATH,
Blackboard or paper) to do homework.
PC4 I felt pretty skilled at using (PATH, Blackboard or paper) to do
homework.
PC5 After using (PATH, Blackboard or paper) to do homework for a
while, I felt pretty competent.
- very true
(1.76)
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
5.78
(1.25)
5.65
(1.32)
5.20
(1.46)
5.17
(1.64)
5.54
(1.38)
5.53
(1.31)
0.683
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
- not at all true
- very true
5.90
(1.28)
4.55
(1.44)
5.18
(1.69)
4.91
(1.42)
5.26
(1.43)
0.626
0.801
0.645
0.547
0.717
0.723
0.664
0.712
0.897
0.681
Table 1: Scales and Descriptive Statistics
The SEM test involves two steps; namely, the measurement model and structural model. The
measurement model examines the relationships among the latent constructs and the indicators of these
constructs. The structural model tests the hypothesized relationships among the latent constructs as well
as the relationships among the latent constructs and other manifest variables. A valid and reliable
measurement model provides assurance of the validity of the relationships indicated in the structural
model.
The Mplus Version 7.0 software recommended by Muthen and Muthen (2007) is used to test the
SEM models in this study. The measurement model can be evaluated by testing the measures of each
construct individually or testing the measures of all the constructs simultaneously (Cheng 2001). The
second approach is preferred by researchers because discriminant validity (correlations among the
indicators of different constructs) as well as convergent validity can be statistically tested in the model.
Thus, confirmatory factor analysis is performed with the three latent constructs and their measures (i.e.,
indicators). Further, inter-correlations among the latent constructs are allowed (Cheng 2001). Mplus
provides model fit indices to measure the validity and reliability of the model. The model fit indices
(CFI=0.960, RMSEA=0.058, SRMR=0.069) reveal a good model fit for the measurement model (with the
three latent constructs and their measures). As illustrated in table 1 above, the factor loadings of the three
latent constructs (i.e., intrinsic motivation, perceived usefulness, and perceived competence) are
sufficiently high and statistically significant (p=0.000). Prior research has suggested various cut-offs for
factor loadings starting from 0.4 regardless of sample size (Stevens 1992) to more stringent cut-offs
treating 0.32 as poor, 0.45 as fair, 0.55 as good, 0.63 as very good, and 0.71 as excellent (Comrey & Lee
1992; Tabachnick & Fidell 2007). Previous studies also indicate that a reliable factor has four or more
factor loadings of 0.6 or above, regardless of sample size (Field 2005, Guadagnoli & Velicer 1988). Rules
of thumb are present to evaluate the practical significance of standardized loadings with lower cut-off
values for a large sample size; that is, 0.5 for a sample size of 120, and 0.45 for a sample size of 150 (Hair
et al. 1998). Nevertheless, each of the three latent construct in Figure 2 has four or more factor loadings
above 0.6 and all the factor loadings are above 0.542 for the given sample size of 173. Hence, the results
demonstrate a highly reliable measurement model and assure the quality of the subsequent structural
model.
Next, the structural model is tested. The model fit indices (i.e., CFI, RMSEA and SRMR) are not
available because the model involves an interaction between a latent variable and a manifest variable.
Hence, the model fit for the structural model is assessed by comparing the Akaike information criterion
(AIC) and Bayesian information criterion (BIC) values of the model with and the model without the latent
interaction term. Specifically, smaller AIC and BIC values indicate a better model fit (Burnham &
Anderson 2004). The fit indices for the model without the interaction term (CFI=0.957; RMSEA=0.050;
SRMR=0.074) suggest a good model fit. In addition, the model with the interaction term (i.e., the
structural model) has smaller AIC and BIC values7, indicating a better model fit. Thus, the structural
model meets the requirement of a good model fit.
Intrinsic
Motivation
Perceived
Usefulness
0.290 (p= 0.012)
0.645 (p= 0.000)
0.244 (p= 0.003)
Indirect effect:
0.187 (p=0.015)
P=0.149
P=0.149
System
Type
Performance
P=0.149
System Use
P=0.149
Direct effect:
0.224 (p=0.027)
P=0.149
Figure 1: Model Results
0.184 (p= 0.000)
Perceived
P=0.149
Competence
The hypothesized relationships in the structural model are evaluated and the results are shown in
Figure 1. Hypothesis 1 proposes that intrinsic motivation mediates the effect of system type on system
use. This mediation effect is tested using the SEM indirect effect function. Specifically, an indirect effect
tests the mediating effect of one or more variables on the relationship between two variables (Weston &
Gore 2006). A full mediating effect is obtained when the relationship between two variables (i.e., direct
effect) is not significant in the presence of the significant indirect effect. A partial mediator is present
when both the direct and indirect effects remain significant. This indirect effect method is consistent with
the theoretical logic of Baron and Kenney’s (1986) three-step process. As depicted in Figure 1, the path
from system type to intrinsic motivation is significant (coefficient=0.645, p=0.000). The link from
intrinsic motivation to system use is also significant (coefficient=0.290, p=0.012). Further, the indirect
effect of system type on system use via intrinsic motivation is significant (coefficient=0.187, p=0.015),
indicating the mediating role of intrinsic motivation, while the direct effect still remains significant
7
The AIC and BIC values for the model with (without) the interaction term is 9350.397 (9744.405) and 9595.053
(9999.321).
(coefficient=0.224, p=0.027). These results suggest the partial mediating effect of intrinsic motivation on
the relationship between system type and system use; hence, hypothesis 1 is supported.
Hypothesis 2 posits that perceived usefulness moderates the effect of intrinsic motivation on system
use. This hypothesis is supported by the significant positive interaction effect of intrinsic motivation and
perceived usefulness on system use (coefficient=0.244, p=0.003). Specifically, the impact of intrinsic
motivation on system use is stronger when perceived usefulness is higher than lower. Hypothesis 3 states
that perceived competence moderates the effect of system use on performance. This hypothesis is also
supported by the significant positive interaction effect of system use and perceived competence on
performance (coefficient=0.184, p=0.000). The results suggest that the impact of system use on
performance is stronger when perceived competence is higher than lower.
4.3 Additional Analysis
A supplementary analysis is performed by including the participants’ demographic information
such as age, gender, school year, GPA, and their pretest performance as covariates in the research model.
These variables do not have a significant effect (untabulated) on the results of the research model.
The participants’ performance 8 prior to (i.e., pretest) and after (i.e., posttest, the measure of
performance in the current study) usage of the systems are compared and the t-test results reveal that
posttest performance (mean of 5.85) is significantly (p=0.000) higher than pretest (mean of 1.07),
indicating an overall positive learning effect from pretest to posttest. In addition, pretest performance does
not vary significantly among the three treatment groups of PATH, Blackboard and the traditional medium
(F=0.087; p=0.971; means of 1.06, 1.13, and 1.03 respectively). This result further suggests that pretest
performance does not have a significant influence on the overall model results.
The planned contrast comparison results show that the PATH group exhibits the highest intrinsic
motivation (mean of 4.54). The average intrinsic motivation score of PATH users is 1.13 (p=0.000)
higher than the average intrinsic motivation score of Blackboard users, and 1.99 (p=0.000) higher than the
average intrinsic motivation score of the traditional paper users. The average intrinsic motivation score
for the Blackboard users is 0.86 (p=0.001) higher than the average intrinsic motivation score of the
traditional paper users.
Further analysis reveals a positive effect of intrinsic motivation on the participants’ evaluation of
the instructor (p=0.000), perceived competence for learning (p=0.000), and understanding of the materials
(p=0.000). Additional analysis indicates a significant positive effect of intrinsic motivation on interest in
accounting (p=0.000) and intention to major in accounting (p=0.013). The findings also suggest that
interest in accounting has a significant effect on intention to major in accounting (p=0.000).
5. DISCUSSION
The present study manipulates the system types; measures intrinsic motivation, perceived
usefulness, and perceived competence; and assesses the participants’ performance directly in a
longitudinal experimental study spanning an entire semester. A significant contribution of this study is
development of an educational computer game, PATH, which comprises an instructional design that
enhances intrinsic motivation. The features of PATH are designed to promote positive learning
experience by assisting students to master technical (accounting) materials. Motivational features of
computer games are incorporated into PATH to deliver materials in an interactive, fun, engaging, and
challenging manner to stimulate interest in learning (Garris et al. 2002). Students engage in active
interaction with the computerized tutorial, exercise control of receipt of feedback from the program, and
8
The participants’ scores on questions that assessed their competency in the course materials were measured before
the study (i.e., pretest performance) and after the study (i.e., posttest performance).
learn at their own pace. PATH incorporates game attributes for enhancing intrinsic motivation with
positive effects on attitudinal outcomes such as evaluation of the instructor, perceived competence for
learning, understanding of the materials, interest in accounting, and intention to major in accounting. For
comparison purposes, we include two additional system types; that is, Blackboard and the traditional
paper medium, to provide additional insight into the findings. The results support our contention that the
interest-enhancing features in PATH lead to the highest level of intrinsic motivation relative to
Blackboard and traditional paper users. The limited interest-enhancing features in Blackboard enable
these users to experience a higher level of intrinsic motivation than the traditional paper users. The
traditional medium users exhibit the lowest level of intrinsic motivation because of the uninteresting,
inflexible, and linear format of the paper format.
The results reveal that intrinsic motivation partially mediates the effect of system type on system
use, demonstrating the important role of intrinsic motivation in explaining how system type influences
system use. The findings also indicate that the effect of intrinsic motivation on system use is stronger
when perceived usefulness is higher than lower. When users perceive a system to be useful for attaining
their goals, this form of extrinsic motivation facilitates rather than debilitates intrinsic motivation which
further enhances system use. Additionally, the results suggest that the impact of system use on
performance is stronger when perceived competence is higher than lower. This study’s identification of
perceived competence as a moderator provides some insight into the inconsistent findings reported in
previous research on the effect of system use on performance. Finally, an important contribution of this
study is use of an objective outcome measure. Specifically, performance is measured by each participant’s
total number of correct answers to questions on the content materials.
Implications, Limitations, and Suggestions for Future Research
This study helps educators understand the benefits of delivering materials in a creative and
motivation-enhancing manner to promote learning and performance. Researchers can develop effective
strategies to help students learn materials that they frequently perceive as boring and difficult. Designers
can also incorporate motivational and creativity-enhancing features into systems to satisfy the users’
needs for these activities (Selker 2005). Further, human-computer-interaction research recognizes the
importance of enjoyment and fun in user interfaces (Shneiderman 2004). Interface designs that facilitate
high quality human-computer interaction are crucial for sustaining the individuals’ interest in the
materials and facilitating their learning process. Indeed, enhancing flow (i.e., optimal user experience) is
an important objective of user interface design.
Games evoke a sense of personal control when users can select strategies, manage the direction of
an activity, and make decisions that directly affect outcome, even if the actions are not instructionally
relevant (Garris, et al. 2002). Future work can examine whether enhanced motivation and learning occur
when students are given control over elements of the instructional program (Cordova & Lepper 1996).
Future work can also investigate other game attributes that have positive effects on performance. Finally,
researchers can help designers understand whether individual attributes influence features to be
incorporated into educational computer games which affect learning outcomes.
As for any study, there are limitations in this study. One limitation is use of subjective usability
measure; that is, the participants’ reported usage of the media. Subjective usability measures pertain to the
user’s perception of or an attitude toward an interface, interaction, or outcome. Objective measures are
gathered, discussed, validated, and independent of user perception (Hornbaek 2006). Both subjective and
objective usability measures must be examined to ensure that the same conclusions are reached about
usability. One prior study conducted a meta-analysis of employee performance and indicated that
subjective and objective measures may capture different aspects of performance (Bommer et al. 1995).
Future research can gather objective usability data such as actual system usage to determine whether
similar findings are obtained.
References
Adams, D. A., Nelson, R. R. and Todd, P. A. (1992). Perceived usefulness, ease of use, and usage of
information technology: A replication. MIS Quarterly, 16, 227–247.
Alliger, G. M., Tannenbaum, S. I., Bennett Jr., W., Traver, H. and Shotland, A., (1997). A meta analysis of
the relations among training criteria. Personnel Psychology, 50, 341-358.
Amabile, T. M., DeJong, W. and Lepper, M. R. (1976). Effects of externally imposed deadlines on
subsequent intrinsic motivation. Journal of Personality and Social Psychology, 34, 92-98.
Arnold, H. (1985). Task performance, perceived competence, and attributed causes of performance as
determinants of intrinsic motivation. Academy of Management Journal, 28(4), 876-888.
Bandura, A. (1982). Self-efficacy mechanism in human agency. American psychologist, 37(2), 122.
Bandura, A. (1997). Self-efficacy: The Exercise of Control. Prentice-Hall, Eaglewood Cliffs, NJ.
Bandura, A. and Schunk, D. H. (1981). Cultivating competence, self-efficacy, and intrinsic interest
through proximal self-motivation. Journal of personality and social psychology, 41(3), 586.
Baron, R. M. and Kenny, D. A. (1986). The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic and statistical considerations. Journal of Personality and
Social Psychology, 51, 1173-1182.
Bell, B. S., Kanar, A. M. and Kozlowski, S. W. J. (2008). Current issues and future directions in
simulation-based training, international. Journal of Human Resource Management, 19, 1416-1434.
Bommer, W.H., Johnson, J.L., Rich, G.A., Podsakoff, P.M. and Mackenzie, S.B. (1995). On the
interchangeability of objective and subjective measures of employee performance: A meta-analysis.
Personnel Psychology, 48, 587-605.
Bryant, S. M. and Hunton, J. E., (2000). The use of technology in the delivery of instruction: implications
for accounting educators and education researchers. Issues in accounting education. 15(1), 129-162.
Burnham, K.P. and Anderson, D.R. (2004). Multimodel inference: Understanding AIC and BIC in Model
Selection. Sociological Methods and Research, 33, 261-304.
Butler, R.J., Pray, T.F. and Strang, D.R. (1979). An extension of Wolfe’s study of simulation game
complexity. Decision Sciences, 10, 480-485.
Cheng, E. W. L. (2001). SEM being more effective than multiple regression in parsimonious model testing
for management development research. Journal of Management Development, 20(7/8), 650-667.
Cohen, V. (1984). Interactive features in the design of videodisc material. Educational Technology, 24,
16-20.
Comrey, A. L. and Lee, H. B. (1992) A first course in factor analysis (2nd edition). Hillsdale,NJ:
Lawrence Erlbaum Associates.
Cordova, D. I. and Lepper, M. R. (1996). Intrinsic motivation and the process of learning: Beneficial
effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88(4),
715-730.
Davis, D. F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information
technology. MIS Quarterly 13(3), 319-339.
Davis, F.D., Bagozzi, R.P. and Warshaw, P.R. (1992). Extrinsic and intrinsic motivation to use computers
in workplace. Journal of Applied Social Psychology, 22(14), 1111-1132.
De Freitas, S. (2006). Learning in immersive worlds. London: Joint Information Systems Committee.
Deci, E. L. (1971). Effects of externally mediated rewards on intrinsic motivation. Journal of Personality
and Social Psychology, 18, 105-115.
Deci, E. L. (1972). Intrinsic motivation, extrinsic reinforcement, and inequity. Journal of Personality and
Social Psychology, 22, 113-120.
Deci, E.L. (1975). Intrinsic motivation. New York: Plenum.
Deci, E. L. and Cascio, W. F. (1972). Changes in intrinsic motivation as a function of negative feedback
and threats. Paper presented at the meeting of the Eastern Psychological Association (April), Boston.
Deci, E. L. and Ryan, R. M. (1980). The empirical exploration of intrinsic motivational processes, in:
Berkowitz, L., (Ed.), Advances in experimental social psychology. New York: Academic Press. 13, 3980.
Deci, E. L. and Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New
York: Plenum Press.
Deci, E. L. and Ryan, R. M. (1985b). The General Causality Orientations Scale: Self-determination in
personality. Journal of Research in Personality, 19, 109-134.
Deci, E. L. and Ryan, R. M. (1987). The support of autonomy and the control of behavior. Journal of
Personality and Social Psychology, 53(6), 1024-1037.
Deci, E. L. and Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the selfdetermination of behavior. Psychological Inquiry, 11 (4), 227-268.
Deci, E.L., Eghrari, H., Patrick, B.D. and Leone, D.R. (1994). Facilitating internalization: The selfdetermination theory perspective. Journal of Personality, 62, 119-142.
Doll, W. J. and Torkzadeh, G. (1998). Developing a multidimensional measure of system-use in an
organizational context. Information & Management, 33, 171-185.
Doll, W. J. and Torkzadeh, G. (1990). The measurement of software end-user involvement. Omega, 18
(4), 399-406.
Edwards, J. R. and Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs
and measures. Psychological Methods, 5(2), 155-174.
Elliot, A. J., Faler, J., McGregor, H. A., Campbell, W. K., Sedikides, C. and Harackiewicz, J. M. (2000).
Competence valuation as a strategic intrinsic motivation process. Personality and Social Psychology
Bulletin, 26(7), 780-794.
Elliot, A. J. and Harackiewicz, J. M. (1994). Goal setting, achievement orientation, and intrinsic
motivation: A mediational analysis. Journal of personality and social psychology, 66(5), 968.Field, A.
(2005). Discovering statistics using SPSS (Second edition). Sage.
Garris, R., Ahlers, R. and Driskell, J. E. (2002). Games, motivation, and learning: A research and practice
model. Simulation & Gaming, 33(4), 441-467.
Gelderman, M. (1998). The relation between user satisfaction, usage of information systems and
performance. Information & Management, 34(1), 11-18.
Goodhue, D. L. and Thompson, R. L. (1995). Task-technology fit and individual performance. MIS
Quarterly, 19(2), 213-226.
Guadagnoli, E. and Velicer, W. (1988). Relation of sample size to the stability of component patterns.
Psychological Bulletin, 103 265-275.
Hair J.F., Tatham R.L., Anderson R.E. and Black W. (1998). Multivariate data analysis. (Fifth Ed.)
Prentice-Hall:London.
Harackiewicz, J. M. and Elliot, A. J. (1993). Achievement goals and intrinsic motivation. Journal of
personality and social psychology, 65(5), 904.
Haradkiewicz, J. M. and Elliot, A. J. (1998). The joint effects of target and purpose goals on intrinsic
motivation: A mediational analysis. Personality and Social Psychology Bulletin, 24(7), 675-689.
Harackiewicz, J. M. and Larsen, J. R. (1986). Managing motivation: The impact of supervisor feedback on
subordinate task interest. Journal of Personality and Social Psychology, 51(3), 547.
Harackiewicz, J. M., Manderlink, G. and Sansone, C. (1984). Rewarding pinball wizardry: Effects of
evaluation and cue value on intrinsic interest. Journal of Personality and Social Psychology, 47(2), 287.
Harackiewicz, J. M. and Sansone, C. (1991). Goals and intrinsic motivation: You can get there from
here. Advances in motivation and achievement, 7, 21-49.
Harackiewicz, J. M., Sansone, C. and Manderlink, G. (1985). Competence, achievement orientation, and
intrinsic motivation: A process analysis. Journal of Personality and Social Psychology, 48(2), 493.
Hendrickson, A. R., Massey, P. D. and Cronan, T. P. (1993). On the test-retest reliability of perceived
usefulness and perceived ease of use scales. MIS Quarterly, 17, 227–230.
Hornbæk, K. (2006). Current practice in measuring usability: Challenging to usability studies and
research. International Journal of Human-Computer Studies, 64, 79-102.
Hornik, S., Johnson, R. D. and Wu, Y. (2007). When technology doesn’t support learning: Conflicts
between epistemological beliefs and technology support in virtual learning environments. Journal of
Organizational and End User Computing, 19(2), 23-46.
Hou, C. K. (2012). Examining the effect of user satisfaction on system usage and individual performance
with business intelligence systems: An empirical study of Taiwan’s electronics industry. International
Journal of Information Management, 32, 560-573.
http://www.selfdeterminationtheory.org/questionnaires/10-questionnaires/50.
Igbaria, M. and Tan, M. (1997). The consequences of information technology acceptance on subsequent
individual performance. Information & Management, 32(3), 113-121.
Johnson, R. D., Hornik, S. and Salas, E. (2008). An empirical examination of factors contributing to the
creation of successful e-learning environments. International Journal of Human-Computer Studies, 66,
356-369.
Ke, W., Tan, C-H, Sia, C-L and Wei, K-K. (2012). Inducing Intrinsic Motivation to Explore the Enterprise
System: The Supremacy of Organizational Levers. Journal of Management Information Systems ,Vol.
29, No. 3, pp. 257–289.
Keller, J. and Bless, H. (2008). Flow and regulatory compatibility: An experimental approach to the flow
model of intrinsic motivation. Journal of Personality and Social Psychology Bulletin, 34, 196–209.
Kettanurak, V., Ramamurthy, K. and Haseman, W. D. (2001). User attitude as a mediator of learning
performance improvement in an interactive multimedia environment: An empirical investigation of
the degree of interactivity and learning styles. International Journal of Human-Computer
Studies, 54, 541-583.
Kirriemuir, J. and McFarlane, A. (2003). Use of computer and video games in the classroom. Proceedings
of the level up digital games research conference. Universiteit Utrecht, Netherlands.
Kline, R. B. (2005). Principles and practice of structural equation modeling. Guilford Press, New York,
NY.
Knotts, U. S. and Keys, J. B. (1997). Teaching strategic management with a business game. Simulation
and Gaming, 28, 377-394.
Law, E. L. C. and Sun, X. (2012). Evaluating user experience of adaptive digital educational games with
Activity Theory. International Journal of Human-Computer Studies, 70(7), 478-497.
Leemkuil, H. and de Jong, T. (2012). Adaptive advice in learning with a computer-based knowledge
management simulation game. Academy of Management Learning & Education, 11(4), 653-665.
Lepper, M. R., Greene, D. and Nisbett, R. E. (1973). Undermining children’s intrinsic interest with
extrinsic reward: A test of the “overjustification” hypothesis. Journal of Personality and Social
Psychology, 28(1), 129-137.
Leidner, D. E. and Elam, J. J. (1993). Executive information systems: Their impact on executive decision
making. Journal of Management Information Systems, 29(1), 89-98.
Liaw, S. S. and Huang, H. M. (2013). Perceived satisfaction, perceived usefulness and interactive learning
environments as predictors to self-regulation in e-learning environments. Computers & Education, 60,
14-24.
Locke, E. A. and Latham, G. P. (1990). A theory of goal setting and task performance. Hillsdale, NJ:
Prentice Hall.
Lucas, H. C. and Spitler, V. K. (1999). Technology use and performance: A field study of broker
workstations. Decision Sciences, 30(2), 291-311.
Malone, T. W. (1980). What makes things fun to learn? A study of intrinsically motivating computer
games. Ph.D. Dissertation, Standford University.
Malone, T.W. (1981a).What makes computer games fun? Byte, 6(12), 258-277.
Malone, T.W. (1981b). Toward a theory of intrinsically motivating instruction. Cognitive Sciences, 4,
333-369.
Malone, T. W. and Lepper, M. R., (1987). Making learning fun: A taxonomy of intrinsic motivations for
learning, in: Snow, R. E., Farr, M.J. (Eds.), Aptitude, learning, and instruction: Vol. 3. Cognative and
affective process analysis, pp. 223-253. Hillsdale, NJ: Erlbaum.
Manderlink, G. and Harackiewicz, J. M. (1984). Proximal vs. distal goal setting and intrinsic motivation.
Journal of Personality and Social Psychology, 47, 918-928.
Meece, J. L., Anderman, E. M., and Anderman, L. H. (2006). Classroom goal structure, student
motivation, and academic achievement. Annual Review of Psychology, 57, 487-503.
Miserandino, M. (1996). Children who do well in school: Individual differences in perceived competence
and autonomy in above-average children, Journal of Educational Psychology, 88(2), 203-214.
Mossholder, K. W. (1980). Effects of externally mediated goal setting on intrinsic motivation: A
laboratory experiment. Journal of Applied Psychology, 65, 202-210.
Muthen, L. K. and Muthen, B. Q. (2007). Mplus (Version 6.0). [Computer Software]. Muthen and
Muthen, Los Angeles, CA.
O'Neil, H. F., Jr., Wainess, R., and Baker, E. L. (2005). Classification of learning outcomes: Evidence
from computer games literature. Curriculum Journal, 16(4), 455-474.
Pasin, F., and Giroux, H. (2011). The impact of a simulation game on operations management education.
Computers and Education, 57, 1240-1254.
Pentland, B. T. (1989). Use and productivity in personal computing: An empirical test. In Tenth
international conference on information systems, December, Boston, MA, 211-222.
Pilke, E.M. (2004). Flow experiences in information technology use. International Journal of HumanComputer Studies, 61, 347-357.
Plant, R.W. and Ryan, R.M. (1985). Intrinsic motivation and the effects of self-consciousness, selfawareness, and ego-involvement: An investigation of internally-controlling styles. Journal of
Personality, 53, 435-449.
Reeve, J., and Deci, E. (1996). Elements of the competitive situation that effect intrinsic motivation.
Personality and Social Psychology Bulletin, 22, 24-33.
Rieber, L. P. (1991). Animation, incidental learning, and continuing motivation. Journal of Educational
Psychology, 83, 318-328.
Ross, M. (1975). Salience of reward and intrinsic motivation. Journal of Personality and Social
Psychology, 32, 245-254.
Ryan, R.M. (1982). Control and information in the intrapersonal sphere: An extension of cognitive
evaluation theory. Journal of Personality and Social Psychology, 43, 450-461.
Ryan, R.M., Connell, J.P. and Plant, R.W. (1990). Emotions in non-directed text learning. Learning and
Individual Differences, 2, 1-17.
Ryan, R. M. and Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation,
social development, and well-being. American Psychologist, 55, 68-78.
Ryan, R.M., Koestner, R. and Deci, E.L. (1991). Ego-involved persistence: When free-choice behavior is
not intrinsically motivated. Motivation and Emotion, 15, 185-205.
Ryan, R.M., Mims, V. and Koestner, R. (1983). Relation of reward contingency and interpersonal context
to intrinsic motivation: A review and test using cognitive evaluation theory. Journal of Personality and
Social Psychology, 45, 736-750.
Saadé, R. and Bahli, B. (2005). The impact of cognitive absorption on perceived usefulness and perceived
ease of use in on-line learning: An extension of the technology acceptance model. Information and
Management, 42, 317-327.
Salas, E., Wildman, J. L., and Piccolo, D. F. (2009). Using simulation based training to enhance
management education. Academy of Management Learning and Education, 8, 559-573.
Sansone, C. (1986). A question of competence: The effects of competence and task feedback on intrinsic
interest. Journal of Personality and Social Psychology, 51(5), 918.
Sansone, C. (1989). Competence feedback, task feedback, and intrinsic interest: An examination of
process and context. Journal of Experimental Social Psychology, 25(4), 343-361.
Sansone, C., Sachau, D. A. and Weir, C. (1989). The effects of instruction on intrinsic interest: The
importance of context. Journal of Personality and Social Psychology, 57, 819-829.
Segars, A. H. and Grover, V. (1993). Re-examining perceived ease of use and usefulness: A confirmatory
factor analysis. MIS Quarterly, 17, 517–525.
Selker, T. (2005). Fostering motivation and creativity for computer users. International Journal of HumanComputer Studies, 63, 410-421.
Shneiderman, B. (2004). Designing for fun: How can we design user interfaces to be more fun?
Interactions, 11(5), 48-50.
Smith, W. E. (1974). The effects of social and monetary rewards on intrinsic motivation. Unpublished
doctoral dissertation, Cornell University, Ithaca, NY.
Stevens, J. P. (1992). Applied multivariate statistics for the social sciences (2nd edition). Hillsdale,
NJ:Erlbaum.
Subramanian, G. H. (1994). A replication of perceived usefulness and perceived ease of use
measurement. Decision Sciences, 25(5/6), 863–873.
Szajna, B. (1993). Determining information systems usage: Some issues and examples. Information &
Management, 25(3), 147-154.
Szajna, B. (1994). Software evaluation and choice: predictive evaluation of the Technology Acceptance
Instrument. MIS Quarterly, 18(3), 319–324.
Tabachnick, B.G. and Fidell, L.S. (2007). Using multivariate statistics (Fifth Edition). Pearson Education
Inc.
Vandercruysse, S., Vandewaetere, M. and Clarebout, G. (2012). Game-based learning: A review on the
effectiveness of educational games. Handbook of Research on Serious Games as Educational,
Business, and Research Tools. Hershey, PA: IGI Global, 628-647.
Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating perceived behavioral control,
computer anxiety and enjoyment into the technology acceptance model. Information Systems Research,
11, 342-365.
Venkatesh, V. and Speier, C. (1999). Computer technology training in the workplace: A longitudinal
investigation of the effect of mood. Organizational Behavior and Human Decision Processes, 79, 1-28.
Venkatesh, V. and Speier, C. (2000). Creating an effective training environment for enhancing telework.
International Journal of Human-Computer Studies, 52, 991-1005.
von Ahn, L. and Dabbish, L. (2008). Designing games with a purpose. Communications of the ACM,
51(8), 58-67.
Webster, J. and Martocchio, J.J. (1992). Microcomputer playfulness: Development of a measure with
workplace implication. MIS Quarterly, 16(2), 201-226.
Weller, H. (1988). Interactivity in microcomputer-based instruction: Its essential components and how it
can be enhanced. Educational Technology, 28, 23-27.
Wilson, K. A., Bedwell, W. L., Lazzara, E. H., Salas, E., Burke, C. S., Estock, J. L., Orvis, K. L. and
Conkey, C. (2009). Relationship between game attributes and learning outcomes: Review and research
proposals, Simulation & Gaming, 40(2), 217-266.
Wolf, G. (1972). Some research and teaching with an on-line oligopoly game using an artificial player.
Decision Sciences, 3, 101-114.
Wolfe, J. (1997). The effectiveness of business games in strategic management course work. Simulation &
Gaming, 28(4), 360-376.
Wouters, P., van der Spek, E. D. and Van Oostendorp, H. (2009). Current practices in serious game
research: A review from a learning outcomes perspective. Games-based learning advancements for
multisensory human computer interfaces: techniques and effective practices, 232-255.
Wu, W. H., Hsiao, H. C, Wu, P. L., Lin, C. H., and Huang, S. H. (2012). Investigating the learning theory
foundations of game-based learning; A meta-analysis. Journal of Computer Assisted Learning, 28; 265279.
Wydra, F. T. (1980). Learner Controlled Instruction. Educational Technology Publications, Eaglewood
Cliffs, NJ.