Learning and Instruction 21 (2011) 587e599
www.elsevier.com/locate/learninstruc
Differential effects of problem-solving demands on individual
and collaborative learning outcomes
Femke Kirschner a,b,*, Fred Paas a,b, Paul A. Kirschner a, Jeroen Janssen c
a
Centre for Learning Sciences and Technologies, Open University of The Netherlands, P.O. Box 2960, 6401 DL Heerlen, The Netherlands
b
Institute of Psychology, Erasmus University Rotterdam, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands
c
Research Centre Learning in Interaction, Utrecht University, P.O. Box 80140, 3508 TC Utrecht, The Netherlands
Received 19 January 2010; revised 9 January 2011; accepted 12 January 2011
Abstract
The effectiveness and efficiency of individual versus collaborative learning was investigated as a function of instructional format among
140 high school students in the domain of biology. The instructional format either emphasized worked examples, which needed to be studied or
the equivalent problems, which needed to be solved. Because problem solving imposes a higher cognitive load for novices than does studying
worked examples it was hypothesized that learning by solving problems would lead to better learning outcomes (effectiveness) and be more
efficient for collaborative learners, whereas learning by studying worked examples would lead to better learning outcomes and be more efficient
for individual learners. The results supported these crossover interaction hypothesis. Consequences of the findings for the design of individual
and collaborative learning environments are discussed.
Ó 2011 Elsevier Ltd. All rights reserved.
Keywords: Cognitive load theory; Worked-example study; Problem-solving; Collaborative learning; Learning efficiency; Learning outcomes
1. Introduction
Cognitive load theory (CLT: Kirschner, 2002; Paas, Renkl, &
Sweller, 2003; Sweller, 1988; Sweller, Van Merriënboer, & Paas,
1998) focuses on learning from complex cognitive tasks based on
what is known about human cognitive architecture (Sweller,
1988, 2004). This architecture consists of an unlimited longterm memory (LTM), which interacts with a working memory
(WM) that is limited in both capacity (Miller, 1956) and duration
(Peterson & Peterson, 1959). For new information, the processing capacity is limited to 4 1 interacting information
* Corresponding author. Institute of Psychology, Erasmus University Rotterdam, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands. Tel.: þ31 10
4088845; fax: þ31 10 408 9009.
E-mail addresses: [email protected] (F. Kirschner), [email protected]
(F. Paas), [email protected] (P.A. Kirschner), [email protected]
(J. Janssen).
0959-4752/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved.
doi:10.1016/j.learninstruc.2011.01.001
elements which is lost if not rehearsed within 30 s (Cowan,
2001). LTM stores and organises knowledge in cognitive
schemas that incorporate multiple elements of information into
a single element (also referred to as chunking; Chase & Simon,
1973; Miller, 1956; Simon, 1974) with a specific function (i.e.,
learning). If learning has occurred over a long period of time,
one’s schemas may consist of huge amounts of information.
Because a schema can be treated by WM as a single element or
even bypass WM if it has become sufficiently automated through
long and consistent practice, WM limitations will disappear for
more knowledgeable learners.
According to CLT, learning task complexity is determined
by the number of new (i.e., to be learned) interacting information elements; the more new interacting elements, the more
complex the task. Although highly interactive information
elements can be processed in isolation, they can only be
understood when all of them and their interactions are processed simultaneously. Given that WM capacity is limited
588
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
with respect to new information, tasks that contain many new
and interacting elements place high cognitive demands on
WM. Within CLT, the load imposed on WM by the element
interactivity or task complexity is called intrinsic cognitive
load.
As basis for instructional design, CLT focuses on effectively dealing with individual WM limitations by creating
instruction that is compatible with human cognitive architecture. Research, therefore, primarily has been on developing
techniques for managing individual WM load and optimising
information-processing in individual learning settings (Ayres
& Paas, 2009). Kirschner, Paas, and Kirschner (2009b, 2010)
have recently emphasised an alternative way of effectively
dealing with individual WM limitations, namely making use
of the multiple WMs of individuals in a collaborative learning
setting. From their perspective, groups of collaborating
learners are considered to be information-processing systems
(Hinsz, Tindale, & Vollrath, 1997; Ickes & Gonzalez, 1994;
Tindale & Kameda, 2000), consisting of multiple limited
WMs which can create a collective working space. Within
these systems, valuable task-relevant information and knowledge held by each group member can be consciously and
actively shared (i.e., retrieving and explicating information),
discussed (i.e., encoding and elaborating information), and
remembered (i.e., personalising and storing information)
(Hinsz et al., 1997; Tindale & Kameda, 2000; Tindale &
Sheffey, 2002). As long as the information is communicated
between the group members and they coordinate their actions,
not all group members need possess all necessary knowledge,
or process all available information alone and at the same time
(Johnson, Johnson, & Stanne, 2001; Langfred, 2000; Wegner,
1978, 1995). This, however, requires positive interdependence
as described in Johnson and Johnson’s (1981) social interdependence theory. Positive interdependence reflects the extent
to which group members must depend upon each other for
effective group performance; each individual group member is
responsible for the work of the group and the group as a whole
is responsible for the learning of each individual group
member. Group members are linked to each other such that
each member cannot succeed unless the others succeed; each
member’s work benefits the others and each member benefits
from the others. Essential here is social cohesion and
a heightened sense of ‘belonging’ to a group.
The perspective on shared memory systems in which
communication plays a key role has also been investigated in
the context of Wegner’s (1978) transactive memory theory.
This theory is based on the idea that group members can serve
as external memory aids for each other, and that they can
benefit from each other’s knowledge and expertise if they
develop a good, shared understanding and an awareness of
‘who knows what’ in the group. This could, for example, be
achieved by training individuals on how to share their
knowledge effectively (Deiglmayr & Spada, 2010; Prichard,
Stratford, & Bizo, 2006). Findings indicate that transactive
memory can facilitate group performance in groups whose
members are aware of other group members’ knowledge and
expertise as opposed to groups where the members are not
aware of fellow group members’ knowledge and expertise
(Michinov & Michinov, 2009). A transactive memory system
enables groups to better utilise the knowledge that their
members possess, and to reach higher levels of performance
than they would have reached without such a system (for
a review, see Moreland & Argote, 2003).
The CLT perspective, in which groups are considered to
possess a shared memory system, has two conflicting consequences for individual group members. On the one hand,
collaborating individuals can invest less cognitive effort
compared to learners working alone, because the task’s interactive information elements with its associated cognitive load
(i.e., the intrinsic cognitive load) can be divided across a larger
reservoir of cognitive capacity (Kirschner, Paas, & Kirschner,
2009a; Ohtsubo, 2005; Stasser, Stewart, & Wittenbaum, 1995).
This is what is called the distribution advantage. On the other
hand, collaborating individuals need to invest cognitive effort in
communicating information with each other and coordinating
their actions, which individuals working alone do not have to
exert. These, so called, transactional activities (Ciborra & Olson,
1988; Kirschner et al, 2009b; Yamane, 1996) can be beneficial
for or deleterious to learning. CLT argues that, while a cognitive
investment in beneficial transactional activities such as negotiating common ground should be stimulated, an investment in
deleterious activities such as discussing ways to share information should be minimised.
The trade-off between the advantage of dividing information-processing among group members and the disadvantage
in terms of having to cognitively invest in the associated
transactional activities can be an indicator for the efficiency of
group learning. This so called collective working memory
effect was demonstrated in a study by Kirschner et al. (2010)
on the effects of low-complexity (i.e., low intrinsic load) and
high-complexity (i.e., high intrinsic load) tasks on individual
and group learning efficiency. Other than learning effectiveness, which is related primarily to learning outcomes (i.e.,
posttest performance), learning efficiency is related to the
relationship between learning outcomes and the amount of
mental effort learners invest to attain those outcomes; the
higher the learning outcomes and the lower the effort, the
higher the efficiency (Paas & Van Merriënboer, 1993; Van Gog
& Paas, 2008; Tuovinen & Paas, 2004). Mental effort in
combination with learning outcome can indicate the quality of
learning in terms of the efficiency of cognitive schema
acquisition. By giving groups and individuals low and highcomplexity learning tasks and then assessing their learning
efficiency on an individual posttest, Kirschner et al (2010)
showed that group learning was superior to individual
learning for high-complexity tasks, but inferior for lowcomplexity tasks.
For high-complexity tasks, Kirschner et al. (2010) argued
that by sharing the task’s high intrinsic cognitive load among
learners, the risk of exceeding the limits of the individual
group members’ WMs was reduced (i.e., distribution advantage). Although the additional cognitive load imposed by
communicating information and coordinating actions (i.e.,
transactional activities) has to be taken into account, this load
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
could be considered to be relatively low compared to the
distribution advantage for complex tasks. Consequently,
learning was more efficient for group members, allowing them
to construct higher quality schemas in LTM than for individual
learners who had to process all of the information individually
(i.e., the collective working-memory effect: Kirschner et al.,
2010).
For low-complexity tasks, they argued that individual
learners had sufficient capacity to carry out the tasks alone
and no advantage of learning together was expected. Sharing
the information-processing among the group members
required information communication and action coordination
which for low-complexity tasks imposed a relatively high
load (in relation to the benefits that will be accrued) thereby
negating the distribution advantage. Consequently, learning
was more efficient for individuals, allowing them to
construct higher quality schemas in LTM than for group
learners who had to engage in relatively high transaction
activities.
In addition to the intrinsic load imposed by the complexity
of the learning task, the cognitive load that learners experience
is affected by a task’s instructional format. This can take two
forms. When the instructional format imposes load that is not
effective for learning, for example, when it provides redundant
information (e.g., Chandler & Sweller, 1991), or requires
students to mentally integrate spatially or temporally separated
materials (e.g., Sweller, Chandler, Tierney, & Cooper, 1990), it
is called extraneous cognitive load. When it is beneficial to
learning, for example when it presents students with highvariability materials (Paas & Van Merriënboer, 1994), or
encourages students to self-explain their actions (e.g., Renkl,
1997), it is referred to as germane cognitive load (Paas,
Renkl, & Sweller, 2004; Sweller, Van Merriënboer, & Paas,
1998). Although extraneous load does not hamper learning
for low in intrinsic load tasks (i.e., low-complexity), it does
hamper learning for high intrinsic load tasks (i.e., highcomplexity); hence, reducing extraneous load is imperative
for high-complexity tasks (Van Merriënboer & Sweller,
2005). A consistent finding of CLT research is that for tasks
high in intrinsic load, instruction with high problem-solving
demands imposes extraneous load on novice learners (Sweller
et al., 1998). In addition to Kirschner et al.’s (2009a, 2010)
recent studies into the effects of different levels of intrinsic
load on individual and collaborative learning efficiency, the
present study kept intrinsic load constant and investigated
the differential effects of extraneous load induced by the
instructional format in terms of problem-solving demands.
Specifically, individual and collaborative learning e by
novices e from either tasks that heavily rely on problemsolving or tasks that heavily rely on example study were
compared regarding their effects on learning efficiency and
effectiveness.
Sweller (1988) has shown that problem-solving search
when carrying out complex conventional tasks, places not
only heavy intrinsic demands, but also heavy extraneous
demands on WM. The strategy most commonly used by
learners faced with novel problems for which they do not have
589
previously constructed schemas e a means-ends analysis e
requires them to consider the current problem state, consider
the desired goal state, extract differences between the two
states, and find or choose a problem-solving operator that can
be used to reduce or eliminate differences between the current
problem state and the desired goal state. In addition, any subgoals that have been established need to simultaneously be
kept in mind. This problem-solving search strategy imposes
high extraneous cognitive load on learners and, consequently,
does not leave sufficient processing capacity for them to
induce the generalised solutions or schemas that are prerequisite to learning.
An effective alternative to such instruction, and one that has
been supported by multiple overlapping experiments using
different instructional materials in a variety of populations, is
instruction with low problem-solving demands relying on
example study, either by using exampleeproblem pairs, problemeexample pairs, or example study only (e.g., Carroll, 1994;
Hübner, Nückles, & Renkl, 2010; Paas, 1992; Paas & Van Gog,
2006; Paas & Van Merriënboer, 1994; Stark, Kopp, & Fischer,
2010; Sweller, 1988; Sweller, 2006; Sweller & Cooper, 1985;
Trafton & Reiser, 1993; Van Gog, Kester, & Paas, 2010; for an
overview see Atkinson, Derry, Renkl, & Wortham, 2000).
Instruction that relies heavily on studying worked examples as
a substitute for instruction involving problem-solving is argued
to be beneficial because it decreases extraneous load by eliminating means-ends search. In contrast to instruction with a heavy
reliance on problem-solving, worked examples focus attention
on problem states and associated operators (i.e., solution steps),
enabling learners to induce generalised solutions or schemas.
The cognitive capacity that becomes available by reducing
extraneous load can be devoted to activities beneficial to
learning (i.e., germane load). This reasoning has led to the
counterintuitive instructional guideline that for novices learning
to solve problems, studying worked examples is a better strategy
than solving the equivalent problems (Rourke & Sweller, 2009;
Sweller, 1988; Sweller et al., 1998).
2. Research Question and Hypotheses
In the context of CLT, instruction with low problem-solving
demands (i.e., studying worked examples) should impose
lower cognitive load on novices than instruction with high
problem-solving demands (i.e., equivalent problems that need
to be solved). Combining this view on instructional formats
with the results of the collective working memory effect
(Kirschner et al., 2010), the question arises as to whether for
students learning individually, instruction emphasising worked
example study would be more effective and efficient than
instruction emphasising solving problems. Analogous to this is
the question of whether for students learning collaboratively,
instruction emphasising problem-solving would be more
effective and efficient than instruction emphasising worked
example study. To this end, this study tested the following
hypotheses:
For learning from instruction emphasising problemsolving, the load on the limited cognitive capacity of an
590
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
individual learner should be too high for effective learning.
For learners in a group, the benefits of dividing informationprocessing (i.e., the cognitive load) among group members
would be greater than the additional cognitive investment of
inter-individual communication of information and coordination of actions in the learning phase. Group members would
consequently be able to devote the freed-up cognitive capacity
to activities that foster schema construction and automation
(i.e., germane load), resulting in higher performance (i.e.,
learning outcome; Hypothesis 1a) and a more favourable
relationship between performance and mental effort (i.e.,
higher learning efficiency; Hypothesis 1b) for learners who
carried out the learning tasks in groups than for those who
carried out the tasks individually.
With regard to learning from instruction emphasising
worked examples, learners working individually or as
members of a group should have sufficient cognitive capacity
to process all information themselves. Hence, the transactional
cognitive activities for learners working as members of
a group would be high relative to the benefits of dividing the
information-processing across group members in the learning
phase. Consequently, qualitative differences in constructed
schemas were expected between learners learning in a group
and learners learning individually, resulting in higher learning
outcomes (Hypothesis 2a) as well as higher learning efficiency
(Hypothesis 2b) on an individual posttest for those who
learned individually than for those who learned as members of
a group.
3. Method
3.1. Participants and design
The effect of instructional format on the learning outcome
and efficiency of individual versus collaborative learning was
investigated using learning tasks emphasizing either problemsolving (further referred to as problem-solving tasks) or
worked-example study (further referred to as worked-example
study tasks). In the learning phase, 140 Dutch high school
sophomore students (71 boys, 69 girls) with an average age of
14.98 years (SD ¼ 0.96), were assigned to four conditions in
such a way that 34 participants had to learn individually from
problem-solving tasks, 34 had to learn individually from
worked-example study tasks, 36 had to learn in 3-person
groups (i.e., triads) from problem-solving tasks, and 36 had to
learn in 3-person groups from worked-example study tasks.
They participated in the study as part of their regular biology
curriculum without any academic or financial compensation.
They were told that the topic of heredity was part of their
annual exam later that year. No differences in prior knowledge
were expected since all participants had followed the same
biology curriculum in the previous three years and the study
topic was new. To further assure equivalence of conditions in
all ways except for the treatment, participants were randomly
assigned to the different experimental conditions. To determine how much was learned, participants were required to
individually take a posttest.
3.2. Materials
The materials were in the biology domain and were concerned with heredity; specifically genotypic and phenotypic
transmission of biological traits from parents to offspring (e.g.,
eye colour in humans, fur length in dogs, leaf shape in plants).
To this end, a general introduction and instruction on solving
inheritance problems, worked example study tasks, problemsolving tasks, and transfer-test tasks were designed. All
materials were approved by two biology teachers as being
suitable for the learners. All materials were paper based.
3.2.1. Introduction
Relevant terminology, rules, and theory underlying
heredity, as well as a worked example on solving heredity
problems was discussed in the introduction. This introduction
gave the definition of genes, genotype and phenotype, homozygosity or heterozygosity of dominant or recessive genes, the
pedigree chart, and rules concerning Punnett squares (i.e.,
diagrams used to predict the outcome of a particular cross or
breeding experiment). The worked example demonstrated how
to solve a heredity problem by combining terminology, rules,
and theory. Learners were required to use the definitions, rules,
and theory underlying heredity problems when carrying out
the learning tasks.
3.2.2. Learning tasks
Three learning tasks that were deemed complex for novices
were presented as tasks emphasising either problem-solving
(i.e., problem-solving task) or worked-example study (i.e.,
worked-example study task). These tasks consisted of nine
information elements on a biological trait in a human family
(e.g., ear shape, eye colour, hair colour) and a question about
the proportion of possible genotypes and phenotypes of the
offspring. Each individual information element was relevant
but insufficient for successfully carrying out the task. A
problem could only be solved by combining all nine information elements (JIGSAW; Aronson, Blaney, Stephan, Silkes,
& Snapp, 1978; Moreno, 2009; Slavin, 1990). A simplified
example of such a task would be that: information element 1 is
the mother’s eye colour: blue; element 2 is the father’s eye
colour: brown; and element 3 is the dominance of brown over
blue for eye colour. Each element gives a certain amount of
information, but to determine the eye colour of the offspring,
the learner must combine all three information elements. The
main difference between the two instructional formats (i.e.,
problem-solving or worked-example study) is the degree of
example elaboration. In the problem-solving condition,
learners received the solution to the problem (i.e., the correct
answers to the questions), and were asked to use the information elements that they had received to determine how the
solution was reached. Working from the givens to the solution,
learners had to rely on problem-solving techniques. The
problem solution presented to the learners could be used to
verify the result of their problem-solving efforts. In the
worked-example study condition, learners were in addition
presented with worked-out solution steps, and asked to study
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
the way the solution was reached (see Fig. 1 for a task
emphasising problem-solving and Fig. 2 for a task emphasising worked-example study (b)). Working from the givens to
the final solution learners could study 11 worked-out solution
steps and sub steps.
To prevent learners from incorrectly believing that they had
arrived at the correct answer, the study did not use conventional problem-solving tasks (i.e., tasks with only a description
of givens and a question without the final solution; Sweller,
1988). In previous studies using the same genetic problems
as conventional problem-solving tasks, learners quickly
decided that they had arrived at the correct problem solution
without this actually being the case (Kirschner et al., 2009b,
2010). Providing the correct solution is one way of preventing this and stimulating them to invest effort in the problemsolving steps towards the problem solution. It also provides
learners with equal opportunities for learning how to correctly
solve a genetics problem in both conditions (i.e., problemsolving and example study). In this way, participants become
aware that they had arrived at a correct or incorrect solution,
and in the latter case are stimulated to try again.
The tasks that had to be carried out in collaboration were
structured such that there was high positive task interdependence (Johnson, 1981; Saavedra, Early, &Van Dyne, 1993).
Group members had to rely on each other and interact with
each other to obtain resources and to effectively carry out the
task. Positive interdependence reflects the degree to which
group members are dependent upon each other for effective
group performance (i.e., enhanced intra-group interaction). It
holds that team members are linked to each other in such a way
that each team member cannot succeed unless the others
succeed; each member’s work benefits the others and vice versa
(Kirschner, Strijbos, Kreijns, & Beers, 2004). This was achieved by giving each group member a booklet containing only
one third of the total number of information elements needed to
591
TASK 1: PIET AND LUCY’S FAMILY
GIVEN
For humans, the gene for green eye color (G) is dominant over the gene for blue eyes
(g).
Sandra has green eyes.
Sandra’s mother has green eyes.
Sandra’s mother is homozygote for eye color.
Sandra’s father has green eyes.
Sandra’s father is homozygote for eye color.
Wim has blue eyes.
Lucy, a child of Sandra and Wim, marries Piet who has green eyes.
Piet’s father has blue eyes.
QUESTIONS
What could be the genotypes and phenotypes of Piet and Lucy’s children? And what are their
proportions?
SOLUTION STEPS
Below, the solution steps for answering the questions are given. You need the given information to
study the solution steps and find out how the steps and final answers were reached.
STEP 1. Determine Piet’s genotype for eye color.
Piet’s genotype is Gg.
STEP 2. Determine Lucy’s genotype for eye color.
It is not possible to know Lucy’s genotype at once; first it has to be determined:
STEP 2.1. What is Wim’s genotype for eye color?
Wim’s genotype for eye color is gg.
STEP 2.2. What is Sandra’s genotype for eye color?
It is not possible to know Sandra’s genotype at once; first it has to be determined:
STEP 2.2.1. What is Sandra’s father’s genotype for eye color?
Sandra’s father’s genotype is GG.
STEP 2.2.2. What is Sandra’s mother’s genotype for eye color?
Sandra’s mother’s genotype is GG.
STEP 2.2.3. Make a Punnett square between the genotypes of Sandra’s mother and
father.
G
G
G
GG
GG
G
GG
GG
STEP 2.2.4. Determine Sandra’s genotype.
Sandra’s genotype is GG.
STEP 2.3. Make a Punnett square between the genotypes of Sandra and Wim.
g
g
G
Gg
Gg
G
Gg
Gg
STEP 2.4. Determine Lucy’s genotype for eye color.
Lucy’s genotype is Gg.
STEP 3. Make a Punnett square between the genotypes of Piet and Lucy.
G
g
G
GG
Gg
g
Gg
gg
STEP 4. Determine the genotypes and their proportions of Piet and Lucy’s children.
25% GG – 25% gg – 50% Gg.
STEP 5. Determine the phenotype and their proportions of Piet and Lucy’s children.
75% green eyes – 25% blue eyes.
Fig. 2. A learning task emphasizing worked-example study.
Fig. 1. A learning task emphasizing problem-solving.
identify or understand the solution steps. To stimulate collaboration, cognitive load distribution (i.e., the distribution
advantage; Ciborra & Olson, 1988; Hinsz et al., 1997; Kirschner
et al., 2009b; Yamane, 1996), and information exchange
amongst the learners, there were no redundant information
elements and the number of information elements was equal for
all group members (i.e., three information elements per group
member). In addition to the individual booklets, the whole
592
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
group received a booklet containing either a question and its
solution (i.e., problem-solving condition) or a question, the
solution steps, and the solution (i.e., worked-example study
condition). This booklet was available to all group members
at the same time. Learners working individually received a
booklet containing all nine information elements plus the
question, solution, and additional solution steps depending on
the experimental condition.
3.2.3. Posttest
To determine how much was learned and to see if learners
could apply the knowledge and skills that they were assumed
to have acquired in the learning phase to a different kind of
problem, four transfer-test tasks were used. These tasks
required learners to use the same basic terminology, rules, and
underlying theory, but they differed from the learning tasks
with respect to families and traits used, kind of information
elements given, task structure, and questions asked (e.g.,
genealogical tree, X-chromosome linked inheritance, dihybrid
crossings; see Fig. 3 for an example). The internal consistency
of the posttest (Cronbach’s alpha) was 0.73.
The posttest consisted of two questions per task related to
heredity characteristics of a certain trait in a family. With
regard to performance, the questions could be scored on
multiple elements, with 1 point for a correctly mentioned
element and 0 points for an incorrectly or not mentioned
element. A maximum score of 28 points could be earned for the
four tasks. The minimum score for all tasks was 0. For the
statistical analysis, the performance scores were transformed
into proportions. In other words, a participant’s score on the
four tasks was divided by the maximum score of tasks (i.e., 28).
3.2.4. Instruction
A non-content related instruction on the procedure, rules,
and regulations concerning solving problems was presented to
the participants twice; once preceding the learning tasks and
once preceding the posttest. The instruction preceding the
learning tasks differed slightly between conditions. The individual learner had to read all information elements before
solving the problem, while each group learner only had to read
those information elements specifically allotted to her/him
(i.e., one third of the total number of elements), but were also
required to share the information elements with each other. In
the problem-solving condition, participants had to find out
how the solution to the problem was established, while in the
worked-example study condition participants had to study the
given solution steps. Because this study focuses on how
instructional format influences collaborative learning, learners
were not allowed to write things down during the learning
phase. Using an external memory aid (i.e., writing things
down) provides an external representation or visualisation of
a problem and can, thus, seen as an ‘augmenting’ cognitive
activity (Jonassen, Peck, & Wilson, 1999; Pea, 1993). As
revealed by Duffy and Cunningham (1996) as well as Van
Bruggen, Kirschner, and Jochems (2002), one impact of this
is that it leads to ‘cognitive offloading’ allowing learners to
devote more memory to other aspects of problem-solution. In
this respect, allowing learners to use pencil and paper would
skew the research since the basic premise of the research, as
stated, was to determine the differential effects of instructional
format (and the accompanying cognitive load) on individual
and group learning when carrying out a learning task. Introducing memory aids would make it impossible to determine
whether the learning formats and contexts increased or
decreased cognitive load or whether the results were due to the
cognitive offloading.
The instruction preceding the posttest was the same for all
participants: First they had to read all information elements
thoroughly, then read the questions, and finally try to answer
the questions as correctly and quickly as possible using all
information elements using pen or pencil and paper to write
down the solution steps.
3.2.5. Cognitive load measurement
To measure learner cognitive load after each learning task
and test task, the subjective 9-point mental-effort rating scale
developed by Paas (1992) was used. Participants were asked to
rate the level of effort required to solve a problem on a scale
ranging from very, very low effort (1) to very, very high effort
(9). This measure provides an overall indication of the
cognitive load (i.e., the total of intrinsic, extraneous, and
germane cognitive load), has been used in numerous studies
dealing with cognitive load, and has proven to be non intrusive, valid, and reliable (Paas, Van Merriënboer, & Adam,
1994).
Fig. 3. A transfer-test task used in the posttest.
3.2.6. Efficiency measurement
The combination of performance and cognitive load
measures can provide a reliable estimate of the relative efficiency of instructional methods, both in terms of the learning
process and learning outcomes. Paas and Van Merriënboer’s
(1993) computational approach (see Van Gog & Paas, 2008)
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
was used to calculate learning efficiency as a function of
instructional format. The basic idea underlying this approach
is that the combination of performance and mental-effort
data collected during a test phase is indicative of the quality
of the cognitive schemas constructed during learning.
Instructional conditions with higher performance on the
posttest in combination with lower or equal invested mental
effort (i.e., a lower level of reported cognitive load) are more
efficient than lower performance on the posttest in combination with higher or equal invested mental effort; cognitive
schemas have been more efficiently acquired. Learning efficiency was calculated by standardising each learner’s scores
on the posttest and the cognitive load invested working on
these tests. For this purpose, the grand mean was subtracted
from each score and the result was divided by the overall
standard deviation, yielding z-scores for effort (R) and
performance (P). Finally, each participant’s performance
efficiency score, E, was computed using the formula:
E ¼ [(P R)/21/2]. High learning efficiency was indicated by
relatively high posttest performance combined with relatively
low mental effort. In contrast, low learning efficiency was
indicated by relatively low posttest performance combined
with relatively high mental effort.
3.3. Procedure
All participants first individually studied a paper-based
general introduction to heredity-related concepts along with
a worked example and returned the introduction after 15 min.
During the whole experiment, the time (i.e., study and testing)
was fixed and was managed by a proctor. They were then
randomly assigned to one of the four experimental conditions
(i.e., individual problem-solving, individual worked-example
study, group problem-solving, and group worked-example
study) to carry out the first series of tasks in 7 min. After each
task, independent of whether in a group or in an individual
condition, all participants rated the amount of invested
cognitive load on the 9-point mental effort rating scale. The
second and third series of learning tasks followed the same
procedure. The instruction for all participants preceding the
series of learning tasks consisted of advising them to read all
information elements thoroughly, read the questions carefully,
and finally try to identify the solution steps to the problem in
the case of the problem-solving condition or understand the
solution steps in the case of the worked-example study
condition, using all information elements. For learners in the
group conditions, it was also stressed that working together
was necessary. However, to keep the load that is ineffective for
learning to a minimum, learners were only permitted to
communicate about task-related topics. To prevent them from
offloading their WM, they were not allowed to write anything
down. After the learning phase, the test phase required
participants to individually work on four transfer tasks for
5 min each. Invested mental effort was measured after each
transfer-test task with the rating scale. Use of pen or pencil and
paper was allowed and stimulated in this phase.
593
3.4. Data analyses
When analysing the effects of social context (individual vs.
group) and instructional format (problem-solving vs. workedexample study), the data-analytical problem of nonindependence had to be taken into account (Cress, 2008; Kenny,
Mannetti, Pierro, Livi, & Kashy, 2002). Because students in
the group condition worked in triads, they influenced each
other through their shared experiences and collaborative
discussions. This might cause group members to experience
similar amounts of mental effort invested in the learning tasks.
Furthermore, because they discussed how to solve the learning
task, their performance on the posttest may be correlated. For
example, when group members collaborated effectively and
found efficient solutions to the problem, they may all perform
well on the posttest. On the other hand, when collaboration
was not effective and they failed to find solutions to the
problem, all group members may perform poorly on the
posttest. This violates the assumption of nonindependence of
observations of individuals, making the results of traditional
analytical techniques, such as ANOVA or MANOVA unreliable (Kenny, 1995). Multilevel analysis (MLA) can cope with
this nonindependence and is therefore a more appropriate
technique (Bonito, 2002; Snijders & Bosker, 1999). When
investigating the effects of social context and instructional
format, MLA will therefore be used.
4. Results
The results for the learning and test phases are described
separately. A significance level of 0.05 was used for all
analyses. Due to registration problems there were incomplete
data from 4 participants in the learning phase and an additional 5 in the test phase. Case-wise deletion of those
participants was carried out in the analyses. Table 1 shows
the resulting number of participants as well as the means
and standard deviations for mental effort as a function of
social context and instructional format in the learning phase,
and performance, mental effort, and learning efficiency as
a function of social context and instructional format in the test
phase.
4.1. Learning phase
The data from the learning phase were analysed using
a random intercept multilevel model (Snijders & Bosker,
1999) that included three predictor variables: social context
(dummy coded with 0 ¼ individual and 1 ¼ group), instructional format (dummy coded with 0 ¼ problem-solving and
1 ¼ worked-example study), and the interaction between
social context and instructional format. The dependent variable was the perceived amount of mental effort invested in
studying. The random intercept regression equation is given in
Eq. (1), where g00 is the intercept, g10 is the regression
coefficient of the group level variable social context, g20 is the
regression coefficient for the group level variable instructional
format, g30 is the regression coefficient for the interaction
594
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
Table 1
Means and standard deviations of the dependent variables in the learning and test phase as a function of social context and instructional format.
Instructional format
Social context
Individual
a
Learning Phase
Mental Effort (1e9)
Test Phaseb
Performance (0e1)c
Mental effort (1e9)
Learning efficiencyd
a
b
c
d
Problem-solving
Example study
Problem-solving
Example study
Problem-solving
Example study
Problem-solving
Example study
SD
M
SD
4.14
4.24
0.57
0.69
4.90
4.59
0.37
0.16
1.29
1.20
0.22
0.19
1.45
1.30
1.24
1.16
4.91
4.75
0.76
0.66
4.88
4.75
0.26
0.02
0.94
1.19
0.13
0.23
1.29
1.56
0.80
1.45
n ¼ 34 for each condition.
n ¼ 32 for all individual conditions, n ¼ 34 for the e group-problem-solving conditions, n ¼ 33 for the group e worked-example study conditions.
Performance is the proportion of correct answers on the posttest.
Based on z-scores of mental effort and performance in the test phase.
between social context and instructional format, U0j is group
level variance, and Rij is individual level variance.1
Mental effortij ¼ g00 þ g01 Contextj þ g02 Formatj
þ g03 Contextj Formatj þ U 0j þ Rij
ð1Þ
This model was estimated with MLwiN version 2.18. Table 2
shows the estimated model parameters. The MLA shows
a significant effect of social context on perceived mental effort,
p < .01. The positive sign of g01 shows that group members rated
the mean mental effort higher than individuals, independent of
instructional format. No effect of instructional format was found,
nor was there an interaction effect between social context and
instructional format in the learning phase. The deviance reported
in Table 2 can be used as a test for the goodness-of-fit of the
multilevel model. By comparing the deviance value shown in
Table 2 to the deviance of a multilevel model without predictor
variables (i.e., empty model, labelled Model 1 in Table 2), one
can test whether the estimated model fits the data better than the
1
Group
M
Note: In our study we have a multilevel structure with students nested in
groups only as a subset of our sample. For students who worked individually
(I-condition), there is no hierarchical clustering in groups, whereas there was
clustering for students who worked in triads (T-condition). The research
design, thus, can be considered to be a partially nested design (cf., Bauer,
Sterba & Hallfors, 2008). The analyses deal with this by considering
students in the I-condition to be members of different 1-person groups. This
means that in the ML analyses a nested structure was forced on students in
both the I- and the T-condition. We are aware that is not fully consistent with
our partially nested design. This for example means that for students in both
the I- and T-conditions variance in the dependent variable is decomposed into
individual level variance as well as group level variance, while there is actually
no group level variance in the I-condition. Bauer et al. proposed a method that
better matches such a partially nested dataset where students in the I- condition
are still considered to be sole members of a group. However, by using a fixed
intercept random slopes multilevel model, variance is decomposed into groupand individual level variance only in the T-condition, which is consistent with
the partially nested design. When using Bauer et al.’s approach, highly similar
results were obtained. This leads to the conclusion that the initial MLA
approach was appropriate. Accordingly, the results presented hereafter refer to
ML analyses that do not take the partial nesting into account.
empty model. In this case, the decrease in deviance was significant, c2(3) ¼ 7.444, p < .05. As seen in Table 2, the model
including social context and instructional format was able to
explain 12% of the group level variance and 8% of the individual
level variance.
4.2. Test phase
The data from the test phase were analysed using a similar
multilevel model as the one in Eq. (1). Three different
multilevel models were estimated with posttest performance,
mental effort, and learning efficiency as dependent variables.
For posttest performance, the MLA results in Table 3 show
a significant effect of social context, p < .01 and instructional
format, p < .01. These main effects were qualified by
a crossover interaction effect of social context and instructional format, p < .01. Inspection of regression coefficient g03
shows that participants who had learned individually performed better on the posttest when they learned from solving
problems than when they learned from studying worked
examples. The opposite effect was found for participants who
Table 2
Estimates for random intercept model for perceived amount of mental effort in
the learning phase.
g00 ¼ Intercept
b1 ¼ Social context
b 2 ¼ Instructional format
b 3 ¼ Social context * Instructional
format
Variance
Group level
Individual level
Deviance
Decrease in deviance
*p < .05 **p < .01.
b
SE
4.09
0.821**
0.11
0.31
0.20
0.34
0.29
0.49
0.71
0.60
385.91
7.45*
0.20
0.13
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
Table 3
Estimates for random intercept model for performance in the test phase.
g00 ¼ Intercept
b1 ¼ Social context
b 2 ¼ Instructional format
b 3 ¼ Social context Instructional format
Variance
Group level
Individual level
Deviance
Decrease in deviance
b
SE
0.58
0.180**
0.12**
0.22**
0.03
0.05
0.05
0.07
0.01
0.03
63.17
12.27**
**p < .01.
had learned in groups. They performed better on the posttest
when they learned from studying worked examples than when
they learned from solving problems (see Fig. 4). These results
confirm Hypotheses 1a and 2a. Because of this interaction
effect, it is important that the effects of social context and
instructional format on posttest performance are not considered separately, but rather that they are interpreted together.
The goodness-of-fit of the model was adequate as indicated by
a significant decrease in deviance compared to the empty
model, c2(3) ¼ 12.272, p < .01. Furthermore, the model
explained 22% of the variance of the group level variance and
12% of the individual level variance.
The results of the MLA of the effect of social context and
instructional format on the perceived mental effort invested in
solving test-problems are shown in Table 4. As can be seen,
the effects of social context and instructional format, as well as
the interaction between social context and instructional format
were not significant. The goodness-of-fit of the model was
poor, c2(3) ¼ 0.480, ns. Compared to the empty model (Model
1 in Table 4), the model including social context and
instructional format explained only a small part of the variance
at the group- and individual level (2% and 1% respectively).
Finally, Table 5 shows the results of MLA on the effect of
social context and instructional format on learning efficiency. A
significant effect was found for social context, p < .05, and
595
Table 4
Estimates for random intercept model for perceived amount of mental effort in
the test phase.
b
SE
4.91
0.03
0.35
0.24
0.24
0.36
0.35
0.51
g00 ¼ Intercept
b1 ¼ Social context
b 2 ¼ Instructional format
b 3 ¼ Social Context Instructional format
Variance
Group level
Individual level
Deviance
Decrease in deviance
0.27
1.64
451.72
0.48
instructional format, p < .05. These main effects were qualified
by the crossover interaction effect of social context and
instructional format, p < .05. The negative sign of g03 should be
interpreted as follows: participants who had learned individually carried out the posttest more efficiently e as indicated by
a more favourable relationship between posttest effort and
posttest performance e when they learned from solving problems than when they learned from studying worked examples.
The opposite effect was found for participants who had learned
in groups; they carried out the posttest more efficiently when
they learned from studying worked examples than when they
learned from solving problems. This resulted in an interaction
pattern which was similar to the pattern for the posttest
performance. Therefore, Hypotheses 1b and 2b were both
confirmed. Inspection of the decrease in deviance compared to
the empty model revealed that the goodness-of-fit of the model
was only marginally significant, c2(3) ¼ 5.572, p ¼ .05, and that
the model explained 5% of the group level and 8% of the individual level variance.
5. Discussion
This study examined the effects of instructional format and
social context along with their interaction on learning
outcomes, cognitive load, and learning efficiency. From the
performance interaction graph (Fig. 4) it is apparent that
Table 5
Estimates for random intercept model for learning efficiency in the test phase.
g00 ¼ Intercept
b1 ¼ Social context
b 2 ¼ Instructional format
b 3 ¼ Social context * Instructional format
Variance
Group level
Individual level
Fig. 4. Performance scores in the test phase as a function of social context and
instructional format.
Deviance
Decrease in deviance
*p < .05.
b
SE
0.37
0.64*
0.59*
0.88*
0.20
0.31
0.29
0.44
0.26
1.08
404.87
6.57*
596
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
although learning from solving problems in a group led to
higher learning outcomes than learning from studying worked
examples, the reverse relationship between learning from
solving problems and studying worked examples was found
for individual learners. As there were no differences in the
amount of mental effort invested during the posttest, the
similar interaction between context and format that was found
for learning efficiency seems to be caused by learning
outcomes only. These test phase results provide an affirmative
and clear answer to the research question, namely that for
students learning individually, instruction emphasising worked
example study is more effective and efficient than instruction
emphasising solving problems while for students learning
collaboratively, instruction emphasising problem-solving is
more effective and efficient than instruction emphasising
worked example study.
Learning outcome and learning efficiency were determined
by the crossover interaction between learning individually or
in a group from tasks imposing high or low total cognitive
load. For problem-solving tasks, though the total cognitive
load was high for the individual learner, within groups it could
be distributed across group members. Whereas individual
learners can be expected to have difficulties processing all
information and constructing schemas, as indicated by the
lower learning outcome and learning efficiency scores, group
members could devote freed-up capacity to activities that
fostered schema construction and learning, as indicated by the
higher learning outcome and (Hypothesis 1a) learning efficiency scores (Hypothesis 1b). Those results make clear that
the cognitive investment of inter-individual integration and
coordination of information were lower within groups than the
benefits of dividing the processing of information across
individuals. With regard to studying equivalent worked
examples, it can be argued that the decrease in extraneous load
left individual learners with enough processing capacity to
successfully deal with the information, negating the distribution advantage of groups. Although both individual and group
learners could successfully process the information, only
group learners had to invest in transactional activities. This
disadvantage for group learners resulted in group learners
constructing lower quality schemas, as indicated by a lower
learning outcome (Hypothesis 2a) and efficiency (Hypothesis
2b) scores than individual learners.
These results were supported by the differential effect of
social context on mental effort investment in the learning phase
where learners in the group condition invested more mental
effort than learners in the individual condition, regardless of
instructional format. For learning by studying worked examples,
this result supports our explanation that group members, as
opposed to individual learners, need to invest additional mental
effort for inter-individual integration and coordination of information and that this additional effort is not directly effective for
learning. While group members invested more mental effort than
individual learners, this did not result in the construction of
cognitive schemas; they learned less efficiently than individual
learners. For learning by solving problems, the results support
our explanation that the freed-up WM capacity of group
members which resulted from the distribution advantage was
devoted to activities that foster learning. The greater invested
mental effort of group members in the learning phase resulted in
higher performance as well as a more favourable relationship
between mental effort and performance on the posttest. This
explanation should be further investigated by, for example,
observing and analysing group communication and coordination
processes to determine what the precise nature of the invested
mental effort was.
A closer look at the absolute scores of invested mental
effort in the learning phase reveals fairly low average scores
(i.e., group M ¼ 4.83; individual M ¼ 4.19), and subtle but
significant differences between the experimental conditions.
These results could be explained by the consistently found
effect that, although the scale ranges from 1 to 9, participants
tend to stay in the middle range of the scale (Paas, Tuovinen,
Tabbers, & Van Gerven, 2003), and that the subtle differences
are reliably indicative for substantial differences in experienced cognitive load (Van Gog & Paas, 2008).
Although the cognitive load experienced by individual
learners learning from tasks emphasising problem-solving was
not high enough for it to be considered cognitive overload
(i.e., group M ¼ 4.83; individual M ¼ 4.19), their posttest
performance was relatively low. An alternative explanation for
this might be that these learners were not able to profit from
the discussion, argumentation, and reflection in the group,
rather than from not having the advantage of an expanded
cognitive capacity. In line with this, the finding that individuals had relatively high posttest performance when learning
from tasks emphasising worked-example study could be
explained by the absence of having to engage in these
collaboration processes. The provided solution steps in these
learning tasks may have made elaborate problem-solving
activities unnecessary. Consequently, individual learners may
have performed better than group learners because the latter
could not profit from positive interdependence, but more likely
suffered from working together with other group members
(i.e., collaborative inhibition; Weldon & Bellinger, 1997). To
investigate this, combined qualitative process-oriented
research and quantitative cognitive load theory research,
would lead to better understanding of the coordinative and
communicative processes that contribute to the cognitive
investment made by learners in collaborative learning (see
Janssen, Kirschner, Erkens, Kirschner, & Paas, 2010).
The crossover interaction found for learning individually or
collaboratively by solving problems or studying worked
examples is similar to the collective working memory effect
found by Kirschner et al. (2010) for learning individually or
collaboratively on low and high-complexity problems.
Combining the findings of both studies, one can argue that the
efficiency of individual versus collaborative learning is
determined by the complexity of the learning task (i.e.,
intrinsic load) and the instructional format of the learning
tasks (i.e., extraneous and germane load). At a more general
level, one can argue that the total cognitive load imposed by
the learning tasks, that is, the sum of intrinsic, extraneous, and
germane cognitive load (Paas, Tuovinen et al., 2003),
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
determines the efficiency of individual versus collaborative
learning. Learning from tasks that impose high cognitive load
can be expected to be more efficient for groups than for
individuals, while the opposite can be expected for learning
from tasks that impose low cognitive load. An interesting topic
for future research would be to investigate the level of
cognitive load at which it becomes more effective and/or
efficient to learn in a group as opposed to learning
individually.
The positive interdependence that was stimulated in this
experiment by dividing the information elements over the
group members might have influenced the effects of social
context. Each of the group members received three information elements which had to be exchanged to solve the problem
or to understand the problem solution. Individual learners, in
contrast, were confronted with all nine information elements
which might have initiated qualitatively different mental
representations for group and individual learners. To determine the magnitude of this effect, future studies should
include an experimental condition where, similar to the individual learning condition, each learner in a group is confronted
with all information elements from the outset.
It should be noted that the learning conditions in this study
were designed to optimise collaboration. This creates tension
between ecological validity and experimental validity. The
learning environment differed from settings encountered in
‘real’ education in that all collaborating participants received
only part of the unique information elements and, consequently, were required to exchange information to solve the
problems or study the worked examples. Also, participants
were not allowed to offload their WMs by using pencil and/or
pen and paper while learning, which also might have stimulated them to collaborate. Finally, the learning setting was
highly structured and scripted causing a minimal cognitive
investment with respect to transactional activities. In this
sense, it is not clear to what extent the results obtained in this
study can be generalised to real classroom settings. It can be
assumed that when all individuals have access to all information, where offloading WM is possible, and where there are
normal transactional activities, different results might be
obtained. Future research should investigate the contribution
of the ‘artificial’ aspects of the study to the effects of the load
imposed by learning tasks on the effectiveness and efficiency
of learning in a group.
Finally, a theoretical implication of the results is that the
limited processing capacity of an individual learner can be
expanded by learning in collaboration with other learners.
From this perspective, collaborative learning can be considered an instructional technique for managing individual
working memory load. When groups of collaborating learners
are considered as information-processing systems (see e.g.,
Hinsz et al., 1997), the information necessary for carrying out
a learning task and its associated cognitive load can be
distributed across multiple collaborating working memories.
The freed-up cognitive capacity of group members can
consequently be devoted to activities that foster learning. In
other words, through good instructional design, a collective
597
working memory effect can be achieved. This view has
implications for CLT, one of which is that it seems that the
functional properties of the learner’s cognitive architecture
change when collaborating with others. The expanded limited
processing capacity can only be used effectively by interindividual information communication and coordination
processes. It is clear that these processes not only lead to
a cognitive investment, but also to affective investments. Until
now, CLT has focused on the alignment of instruction with
cognitive processes, without recognising the role of affective
ones. This research on group learning might stimulate cognitive load theorists to address affective issues in their research.
Practically, the results suggest that wholesale adoption of
collaborative learning is not a sensible educational practice.
The challenges that a learning task poses to the learner’s
cognitive capacity and/or the amount of cognitive load a task
imposes, should be determining factors when deciding
whether to employ a learning model or environment based
upon an individual or a collaborative learning paradigm. The
higher the load imposed by the learning tasks, the more likely
that collaborative learning will lead to better learning
outcomes, in terms of effectiveness, efficiency, or both. This
means that when choosing collaborative learning as an
educational approach, educational designers e most often the
teachers themselves e must assure themselves that the
learning tasks given to the groups (e.g., problems, projects, et
cetera) are complex enough that they cannot be carried out
easily by an individual. This also suggests that it would be
better for practitioners not to make an exclusive choice for
individual or collaborative learning, but rather vary their
approach depending on the complexity of the learning tasks to
be carried out and the goals of the instruction.
References
Aronson, E., Blaney, N., Stephan, G., Silkes, J., & Snapp, M. (1978). The
jigsaw classroom. Beverly Hills, CA: Sage Publications.
Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. W. (2000). Learning
from examples: instructional principles from the worked-examples
research. Review of Educational Research, 70, 181e214. doi:10.3102/
00346543070002181.
Ayres, P., & Paas, F. (2009). Interdisciplinary perspectives inspiring a new
generation of cognitive load research. Educational Psychology Review, 21,
1e9. doi:10.1007/s10648-008-9090-7.
Bauer, D. J., Sterba, S. K., & Hallfors, D. D. (2008). Evaluating group-based
interventions when control participants are ungrouped. Multivariate
Behavioral Research, 43, 210e236. doi:10.1080/00273170802034810.
Bonito, J. A. (2002). The analysis of participation in small groups: methodological and conceptual issues related to interdependence. Small Group
Research, 33, 412e438. doi:10.1177/104649640203300402.
Carroll, J. (1994). Relating complexity to practical performance in parsing
with wide-coverage unification grammars. In Proceedings of the 32nd
annual meeting of the association for computational linguistics, NMSU,
Las Cruces, NM, USA, 287e294. doi:10.3115/981732.981772.
Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of
instruction. Cognition and Instruction, 8, 293e332.
Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive
Psychology, 4, 55e81. doi:10.1016/0010-0285(73)90004-2.
Ciborra, C., & Olson, M. H. (1988). Encountering electronic work groups:
a transaction costs perspective. In Proceedings of the 1988 ACM
598
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
conference on computer-supported cooperative work. doi:10.1145/62266.
62274. Portland, OR, USA.
Cowan, N. (2001). The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24,
87e114. doi:10.1017/S0140525X01003922.
Cress, U. (2008). The need for considering multilevel analysis in CSCL
research: an appeal for the use of more advanced statistical methods.
International Journal of Computer-Supported Collaborative Learning, 3,
69e84. doi:10.1007/s11412-007-9032-2.
Deiglmayr, A., & Spada, H. (2010). Training for fostering knowledge coconstruction from collaborative inference-drawing. Learning and Instruction. Advance Online Publication. doi:10.1016/j.learninstruc.2010.06.004.
Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: implications for
the design and delivery of instruction. In D. H. Jonassen (Ed.), Handbook
of research for educational communications and technology (pp.
170e198). New York: Macmillan Library Reference.
Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging
conceptualization of groups as information processors. Psychological
Bulletin, 121, 43e64.
Hübner, S., Nückles, M., & Renkl, A. (2010). Writing learning journals:
instructional support to overcome learning-strategy deficits. Learning and
Instruction, 20, 18e29. doi:10.1016/j.learninstruc.2008.12.001.
Ickes, W., & Gonzalez, R. (1994). “Social” cognition and social cognition:
from the subjective to the intersubjective. Small Group Research, 25,
294e315. doi:10.1177/1046496494252008.
Janssen, J., Kirschner, F., Erkens, G., Kirschner, P. A., & Paas, F. (2010).
Making the black box of collaborative learning transparant: combining
process-oriented and cognitive-load approaches. Educational Psychology
Review, 22, 139e154. doi:10.1007/s10648-010-9131-x.
Johnson, D. W. (1981). Student-student interaction: the neglected variable in
education. Educational Research, 10, 5e10, Retrieved from. http://www.
jstor.org/stable/1175627.
Johnson, D. W., & Johnson, R. T. (1981). Effects of cooperative and individualistic learning experiences on interethnic interaction. Journal
of Educational Psychology, 73, 444e449. doi:10.1037/0022-0663.73.3.
444.
Johnson, D. W., Johnson, R. T., & Stanne, M. B. (2001). Impact of goal and
resource interdependence on problem-solving success. The Journal of
Social Psychology, 129, 621e629.
Jonassen, D. H., Peck, K. L., & Wilson, B. G. (1999). Learning with technology: A constructivist perspective. Upper Saddle River, NJ: Prentice
Hall.
Kenny, D. A. (1995). The effect of nonindependence on significance testing in
dyadic research. Personal Relationships, 2, 67e75. doi:10.1111/j.14756811.1995.tb00078.x.
Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The
statistical analysis of data from small groups. Journal of Personality and
Social Psychology, 83, 126e137. doi:10.1037//0022-3514.83.1.126.
Kirschner, F., Paas, F., & Kirschner, P. A. (2009a). A cognitive load approach
to collaborative learning: united brains for complex tasks. Educational
Psychology Review, 21, 31e42. doi:10.1007/s10648-008-9095-2.
Kirschner, F., Paas, F., & Kirschner, P. A. (2009b). Individual and group-based
learning from complex cognitive tasks: effects on retention and transfer
efficiency. Computers in Human Behavior, 25, 306e314. doi:10.1016/j.
chb.2008.12.008.
Kirschner, F., Paas, F., & Kirschner, P. A. (2010). Task complexity as a driver
for collaborative learning efficiency: the collective working-memory
effect. Applied Cognitive Psychology. Advance Online Publication. doi:10.
1002/acp.1730.
Kirschner, P. A. (2002). Cognitive load theory: implications of cognitive load
theory on the design of learning. Learning and Instruction, 12, 1e10. doi:
10.1016/S0959-4752(01)00014-7.
Kirschner, P. A., Strijbos, J. W., Kreijns, K., & Beers, P. J. (2004). Designing
electronic collaborative learning environments. Educational Technology
Research and Development, 52, 47e66. doi:10.1007/BF02504675.
Langfred, C. (2000). Work-group design and autonomy: a field study of the
interaction between task interdependence and group autonomy. Small
Group Research, 31, 54e70. doi:10.1177/104649640003100103.
Michinov, N., & Michinov, E. (2009). Investigating the relationship between
transactive memory and performance in collaborative learning. Learning
and Instruction, 19, 43e54. doi:10.1016/j.learninstruc.2008.01.003.
Miller, G. (1956). The magical number seven, plus or minus two: some limits
on our capacity for processing information. Psychological Review, 63,
81e97. doi:10.1037/h0043158.
Moreland, R. L., & Argote, L. (2003). Transactive memory in dynamic
organizations. In R. Peterson, & E. Mannix (Eds.), Understanding the
dynamic organization (pp. 135e162). Mahwah, NJ: Erlbaum.
Moreno, R. (2009). Constructing knowledge with an agent-based instructional
program: a comparison of cooperative and individual meaning making.
Learning and Instruction, 19, 433e444. doi:10.1016/j.learninstruc.2009.
02.018.
Ohtsubo, Y. (2005). Should information be redundantly distributed among group
members? Effective use of group memory in collaborative problem solving.
Applied Cognitive Psychology, 19, 1219e1233. doi:10.1002/acp.1162.
Paas, F. (1992). Training strategies for attaining transfer of problem-solving
skill in statistics: a cognitive load approach. Journal of Educational
Psychology, 84, 429e434.
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: recent developments. Educational Psychologist, 38, 1e4.
doi:10.1007/s10648-005-3951-0.
Paas, F., Renkl, A., & Sweller, J. (2004). Cognitive load theory: instructional
implications of the interaction between information structures and cognitive architecture. Instructional Science, 32, 1e8. doi:10.1023/B:TRUC.
0000021806.17516.d0.
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003).
Cognitive load measurement as a means to advance cognitive load theory.
Educational Psychologist, 38, 63e71.
Paas, F., & Van Gog, T. (2006). Optimising worked example instruction:
different ways to increase germane cognitive load. Learning and Instruction, 16, 87e91. doi:10.1016/j.learninstruc.2006.02.004.
Paas, F., & Van Merriënboer, J. J. G. (1993). The efficiency of instructional
conditions: an approach to combine mental-effort and performance
measures. Human Factors, 35, 737e743.
Paas, F., & Van Merriënboer, J. J. G. (1994). Variability of worked-examples
and transfer of geometrical problem-solving skills: a cognitive-load
approach. Journal of Educational Psychology, 86, 122e133, Retrieved
from. http://doc.utwente.nl/26427/1/Paas94variability.pdf.
Paas, F., Van Merriënboer, J. J. G., & Adam, J. J. (1994). Measurement of
cognitive load in instructional research. Perceptual and Motor Skills, 79,
419e430.
Pea, R. D. (1993). Practices of distributed intelligence and designs for education.
In G. Salomon (Ed.), Distributed cognition: Psychological and educational
considerations (pp. 47e87). Cambridge: Cambridge University Press.
Peterson, L., & Peterson, M. (1959). Short-term retention of individual verbal
items. Journal of Experimental Psychology, 58, 193e198. doi:10.1037/
h0049234.
Prichard, J. S., Stratford, R. J., & Bizo, L. A. (2006). Team-skills training
enhances collaborative learning. Learning and Instruction, 16, 256e265.
doi:10.1016/j.learninstruc.2006.03.005.
Renkl, A. (1997). Learning from worked-out examples: a study on individual
differences. Cognitive Science, 21, 1e29. doi:10.1207/s15516709cog2101_1.
Rourke, A., & Sweller, J. (2009). The worked-example effect using ill-defined
problems: learning to recognise designers’ styles. Learning and Instruction, 19, 185e199. doi:10.1016/j.learninstruc.2008.03.006.
Saavedra, R., Early, P. C., & Van Dyne, L. (1993). Complex interdependence
in task-performing groups. Journal of Applied Psychology, 78, 61e72.
Simon, H. A. (1974). How big is a chunk? Science, 183, 482e488. doi:10.
1126/science.183.4124.482.
Slavin, R. E. (1990). Cooperative learning. Review of Educational Research,
50, 315e342. doi:10.3102/00346543050002315.
Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction
to basic and advanced multilevel modeling. London: Sage Publications.
Stark, R., Kopp, V., & Fischer, M. R. (2010). Case-based learning with worked
examples in complex domains: two experimental studies in undergraduate
medical education. Learning and Instruction. Advance Online Publication.
doi:10.1016/j.learninstruc.2009.10.001.
F. Kirschner et al. / Learning and Instruction 21 (2011) 587e599
Stasser, G., Stewart, D. D., & Wittenbaum, G. M. (1995). Experts roles and
information exchange during discussion: the importance of knowing who
knows what. Journal of Experimental Social Psychology, 31, 244e265.
doi:10.1006/jesp.1995.1012.
Sweller, J. (1988). Cognitive load during problem solving: effects on
learning. Cognitive Science, 12, 257e285. doi:10.1016/0364-0213(88)
90023-7.
Sweller, J. (2004). Instructional design consequences of an analogy between
evolution by natural selection and human cognitive architecture. Instructional Science, 32, 9e31. doi:10.1023/B:TRUC.0000021808.72598.4d.
Sweller, J. (2006). The worked example effect and human cognition. Learning
and Instruction, 16, 165e169. doi:10.1016/j.learninstruc.2006.02.005.
Sweller, J., Chandler, P., Tierney, P., & Cooper, M. (1990). Cognitive load as
a factor in the structuring of technical material. Journal of Experimental
Psychology: General, 119, 176e192.
Sweller, J., & Cooper, G. A. (1985). The use of worked examples as
a substitute for problem solving in learning algebra. Cognition and
Instruction, 2, 58e89.
Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture
and instructional design. Educational Psychology Review, 10, 251e295.
doi:10.1023/A:1022193728205.
Tindale, R. S., & Kameda, T. (2000). Social sharedness as a unifying theme for
information processing in groups. Group Processes and Intergroup Relations, 3, 123e140. doi:10.1177/1368430200003002002.
Tindale, R. S., & Sheffey, S. (2002). Shared information, cognitive load and
group memory. Group Processes and Intergroup Relations, 5, 5e18. doi:
10.1177/1368430202005001535.
Trafton, J. G., & Reiser, B. J. (1993). The contributions of studying examples
and solving problems to skill acquisition. In M. Polson (Ed.), Proceedings
599
of the fifteenth annual conference of the cognitive science society (pp.
1017e1022). Hillsdale, NJ: Erlbaum.
Tuovinen, J. E., & Paas, F. (2004). Exploring multidimensional approaches to
the efficiency of instructional conditions. Instructional Science, 32,
133e152. doi:10.1023/B:TRUC.0000021813.24669.62.
Van Bruggen, J., Kirschner, P. A., & Jochems, W. (2002). External representation
of argumentation in CSCL and the management of cognitive load. Learning
and Instruction, 12, 121e138. doi:10.1016/S0959-4752(01)00019-6.
Van Gog, T., Kester, L., & Paas, F. (2010). Effects of worked examples,
example-problem, and problem-example pairs on novices’ learning.
Contemporary Educational Psychology. Advanced Online Publication. doi:
10.1016/j.cedpsych.2010.10.004.
Van Gog, T., & Paas, F. (2008). Instructional efficiency: revisiting the original
construct in educational research. Educational Psychologist, 43, 1e11.
doi:10.1080/00461520701756248.
Van Merriënboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and
complex learning: recent developments and future directions. Educational
Psychology Review, 17, 147e177. doi:10.1007/s10648-005-3951-0.
Wegner, D. M. (1978). Transactive memory: a contemporary analysis of the
group mind. In B. Mullen, & G. R. Goethals (Eds.), Theories of group
behavior (pp. 185e208). New York: Springer-Verlag.
Wegner, D. M. (1995). A computer network model of human transactive
memory. Social Cognition, 13, 319e339.
Weldon, M. S., & Bellinger, K. D. (1997). Collective memory: collaborative
and individual processes in remembering. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 23, 1160e1175.
Yamane, D. (1996). Collaboration and its discontents: steps toward overcoming barriers to successful group projects. Teaching Sociology, 24,
378e383, Retrieved from. http://www.jstor.org/stable/1318875.