Online Problem-based Learning: Panacea or Problematic?
Richard F. Kenny
Athabasca University
Introduction
Problem-Based Learning (PBL) is an educational approach in which complex, illstructured problems serve as the context and the stimulus for learning. It contrasts with
more traditional subject based approaches where students are first taught a body of
knowledge and then may have an opportunity to apply what they have learned to sample
problems. Students work collaboratively in groups to identify what they need to learn in
order to understand the problem and to learn about the broader concepts and principles
related to the problem. PBL, therefore, is designed to encourage active participation by
the students by plunging them into a situation, requiring them to define their own
learning needs within broad goals set by faculty, and identify, and search for, the
knowledge that they need to obtain in order to approach the problem.
PBL, as a pedagogical approach, has been used since the early 1970's and is most
widely used in Medical Education, but is also employed in a range of other fields,
including Nursing, Dentistry and Agriculture (Barrows, 1996, 1998; Boud & Faletti,
1991; Savery & Duffy, 2001). Research on PBL, especially as used in medical schools,
has focused primarily on comparing the outcomes of PBL methods to more traditional
instruction (Albanese, 2000; Albanese and Mitchell, 1993; Colliver, 2000; Smits,
Verbeek & Buisonjé, 2002; Vernon and Blake, 1993). Little work has been done on the
specific learning processes occurring in students engaged in PBL (Norman & Schmidt,
1992) or on the applicability of this approach in an online, Distance Education context
(Barrows, 2002; Lehtinen, 2002; Orrill, 2002).
How do you know if you are really doing PBL?
What exactly is PBL and how do you know if you are really doing it? Perhaps the
most well know theorist on PBL is Howard Barrows. Barrows, now Chair of the
Department of Medical Education at Southern Illinois University, pioneered the use of
PBL at McMaster University in the 1960’s in response to “the impoverished knowledge
base that medical students accrued during their neurology clinical clerkships
(residencies)” (Maudsley, 1999, p. 178). In 1986, Barrows published a taxonomy of the
variety of teaching methods referred to as PBL and suggested how each variety
addressed different educational goals (Barrows, 1986):
1.
2.
3.
4.
Structuring of knowledge for use in clinical contexts.
The development of an effective clinical reasoning process.
The development of effective self-directed learning skills.
Increased motivation for learning.
1
Barrows then compared three common instructional strategies used in these
contexts: a) the presentation of complete cases or case vignettes, b) partial problem
simulations such as structured management problems, and c) full problem simulations
involving free inquiry. Each could be used in a teacher-directed learning context, a
student-directed learning context or a mixture of both. For example, a lecture-based case
approach, involving the presentation of information with case vignettes as examples,
would be the most distant from PBL because it would at best address only the first
objective. The case method, used in business and law, in which students are given a
complete case for research and study prior to discussion in class, would be closer to PBL
because it addresses the first and last objectives fairly well. However, it limits the
amount of clinical reasoning and self-directed learning because it is already organized
for the learners. Only methods using problems with limited information and which allow
for free inquiry on the part of the students could be said to be PBL.
More recently, Barrows (1998) noted an increase in the variations on PBL and
chose to describe what he calls “authentic PBL”, which is meant to address an expanded
set of educational objectives:
1. The acquisition of a deeply understood knowledge integrated from a variety of
disciplines.
2. The development of effective clinical problem-solving.
3. The development of self-directed learning.
4. The development of team and interpersonal skills.
5. The development of a desire to continually learn.
To accomplish these goals, authentic PBL should have the several important
characteristics as discussed below.
Problem-based
A PBL approach begins with the presentation of a real life (authentic) problem
stated as it would be presented to the practitioner and providing only limited information.
These problems usually consist of descriptions of sets of events that need explanation
(Norman & Schmidt, 1992). In medical education, they typically involve descriptions of
patients presenting complaints supplemented with some critical symptoms. However,
PBL is not restricted to Medicine and has been used in a variety of other areas, such as
Nursing and Agriculture. There, problems introduce the learner to problematic situations
relevant to practitioners in those fields. In all cases, learners are then required to generate
hypotheses about the cause of the problem to determine the important facts in the case
and develop a solution (diagnosis and treatment in medical education). Norman and
Schmidt (1992) provide the following example of a medical PBL problem:
A 55-year-old woman lies crawling on the floor in obvious pain. The pain
emerges in waves and extends from the right lumbar region to the right side of the
groin and to the right leg.
2
In this case, students need to find an explanation of the source of the pain described,
what physiological processes are occurring and how it is extending to other areas of the
body.
Problem-solving
Authentic PBL problems support the application of problem-solving skills
required in “clinical practice”. The role of the tutor is to facilitate the application and
development of effective problem-solving process.
Student – centred
Students assume responsibility for their own learning and faculty act as
facilitators. Barrows (1998) notes that teachers must avoid making students dependent
on them for what they should learn and know.
Self-directed learning
Authentic PBL develops research skills. Students need to learn how to get
information when it is needed and will be current, as this is an essential skill for
professional performance.
Reflection
This should take place following the completion of problem work and is meant to
enhance transfer of learning to new problems. Barrows (1998) claims that this is best
accomplished through group discussions about what was learned with the problem, what
are its essential elements, and how it relates to previously encountered problems.
How is PBL Instruction Structured?
Face-to-face PBL instruction can be structured in a variety of ways. The
following process is used in nursing classes in a Western Canadian college program (Van
Neste-Kenny, personal communication) and can be viewed as prototypical:
1. Read the problem and underline key data:

group selects a facilitator, a scribe, and a recorder

one student reads the problem

each group member underlines what are key data to him/her

no discussion at this time
2. Define the problem & list phenomena requiring explanation:

assist students pm to outline the essential elements of the problem

assist students pm to identify points to explain (tutor may assist by asking
questions)

ensure that all phenomena identified are written down on board
3

ensure that discussion does not begin at this point (goal is to classify
information)
3. Propose working hypotheses to explain the problem:

encourage student participation

encourage depth of thinking

monitor time

if thinking is blocked around a phenomenon, the ideas can be included in
learning to be done below
4. Organize elements of the problem(s)

guide prn in the construction of a diagram illustrating relationships among
hypotheses

uncertain relationships can be indicated with dotted lines

stimulate, encourage students (the schema will guide studying and facilitate
encoding)
5. Formulate learning to be done
 Based on the schema and the discussion, assist students to specify the learning
to be done (not giving them resources but rather, aspects to research)
 If they are missing key learning to be done, enable them to discover what has
to be learned by asking open questions
 Ask how / which resources students are planning to use
6. Synthesis






Which resources have been used? (not content but quality)
Starting from the schema, validate and bring corrections
Don't deviate from the problem
Invite students to ask questions, discuss controversies
Ensure that concepts are mastered
Give feed-back on learning at the end of this step
7. Debriefing





Tutor facilitates
Guide discussion on the process
Have integration objectives been met?
PBL methodology
Group process
4
Is PBL is an effective instructional model?
A review of the literature on “effectiveness” leads to mixed conclusions
concerning this question. In particular, several meta-analyses have been conducted in the
last 12 years, mainly examining the use of PBL in Medical Education. While comparison
research on media effectiveness has led to decades of no significance difference results
(Russell, 1999; Clark, 1983, 1994), these reviews may hold more promise because they
compare entire curricula that use PBL or “traditional methods” over a period of several
years, rather than in one single course.
Meta-analyses and other studies
Meta-analysis is a technique which permits quantitative reviews and syntheses of
the research issues (Wolf, 1986) and makes use of an effect size statistic (E.S.) (e.g.,
Glass, McGaw, and Smith, 1981). The simplest method for calculating an E.S. is to use
the following formula:
Treatment 1 Mean – Treatment 2 Mean
Standard Deviation of the Control Group
A pooled SD can be used if studies contain no control group. This results in a
standardized score. The E.S. allows the comparison of studies which vary in design,
sample selection, and setting, in order to form conclusions and, because it is based on
standard deviations, also permits an assessment of degree of effect. Thus, for t-tests of
independent means, an E.S. of 0.20 could be considered of mild strength (i.e. the mean of
the population with the higher mean score exceeds the scores of 58% of the group with
the lower mean score). An E.S. near 0.50 would be viewed as moderate (mean of upper
group > 69% of lower group scores) and those 0.80 and above judged as strong (mean of
higher group > 79% of lower group) (Cohen, 1988, pp. 25-26).
Two meta-analyses conducted in the early 1990’s are frequently cited as
providing evidence that PBL is more effective than “traditional” methods of medical
education (specifically lecture courses). Vernon and Blake (1993) compared 35 studies
conducted between 1970 and 1992. While the results were complex, the authors found
that PBL was superior with respect to students’ program evaluations (E.S. of +0.55) and
on measures of students’ clinical performance (E.S. of + 0.28). However, PBL and
traditional methods did not differ on tests of factual knowledge (E.S. of -0.09) and
students taught using traditional methods outperformed their PBL counterparts on the US
National Board of Medical Examiners (NMBE) Part 1 licence exam (E.S. of – 0.18).
The second meta-analysis, by Albanese and Blake (1993) produced similar
findings. Students of conventional curricula outperformed PBL students on measures of
basic science (such as the NMBE Pt. 1) in 6 of 10 studies, 3 of which were statistically
significant. PBL students, on the other hand, scored higher on clinical examinations (e.g.,
5
NMBE Pt. 2) in five of seven studies, with only one study producing statistically
significant results. Further, in seven studies that used clinical ratings by faculty
supervisors as a measure, the ratings for PBL students were either more positive or the
difference was not significant.
More recently, Distlehorst and Robbs (1998) carried out a 3 year study to
compare the academic performance of students in a reiterative problem-based learning
(PBL) curriculum with that of their counterparts in the standard curriculum to ensure that
the PBL students were not disadvantaged. They examined the performance of three
graduating classes’ on a number of outcome measures, including the United States
Medical Licensing Examination (USMLE), clinical clerkship, and a clinical practice
examination performance. The E.S. for the USMLE Step 1 was 0.18 and 0.39 for the Step
2, but neither was statistically significant. Analysis of the clerkship performance,
however, showed an E.S. of 0.50, which was significant. They concluded that students in
the PBL curriculum performed at least as well as, and in some instances better than, their
counterparts in the standard curriculum.
Not all researchers agree with the conclusions of these studies, however. Colliver
(2000), however, provided a critical overview of problem-based learning (PBL), focusing
on the credibility of both the empirical and theoretical claims about the ties between PBL
and educational outcomes and on the magnitude of the effects. The author reviewed the
medical education literature, including the reviews published in 1993, published from
1992 through 1998. Colliver claims his review of the literature revealed no convincing
evidence that PBL improves knowledge base and clinical performance, at least not of the
magnitude that would be expected given the resources required for a PBL curriculum. In
this respect, he contended that a major educational intervention should minimally
produce a large effect size (0.80 – 1.00), an outcome demonstrated by none of the
reviews and studies considered.
Albanese (2000) countered these claims with an analysis of what constitutes
reasonable ES in terms of the impacts on individuals and published reports. He concludes
that large effect sizes of 0.8 - 1.0 are an unreasonable expectation from PBL for three
reasons: a) this would require some students to move from the bottom quartile to the top
half of the class or more, b) leading up to medical school, students are groomed and
selected for success in a traditional curriculum and expecting them to do better in a PBL
curriculum than a traditional curriculum is an unreasonable expectation, and c) the
average ES reported in the literature was 0.50 and that many commonly used and
accepted medical procedures and therapies are based upon studies with ESs below 0.50.
Problem-based Learning and Learning Theory
Problem-based Learning and Cognitive Theory
Barrows and other proponents of PBL have argued strongly that this instructional
approach sets the conditions for effective and deep learning of both disciplinary
knowledge and problem-solving (e.g., Albanese, 2000; Barrows, 1998, Norman &
6
Schmidt, 1992, 2000). Recall that Barrows (1998) claimed that only “authentic” PBL
could foster both the acquisition of a deeply understood knowledge integrated from a
variety of disciplines and the development of effective clinical problem-solving. Does
theory and research on human learning provide support for these claims?
Norman and Schmidt (1992) reviewed the evidence concerning the psychological
basis for PBL and came to several conclusions about these claims. First, they found no
support for the claim that PBL curricula result in any improvement in general, contentfree problem-solving skills. In fact, they claim that the concept is not especially useful
and cite substantive research pointing to the importance of context in problem-solving.
Regardless, based on research in cognitive psychology, the authors envisage three roles
for problem-based learning: a) the acquisition of knowledge in the context in which it
will be used, b) the mastery of general principles such that they can be transferred to new,
similar problems, and c) the acquisition of prior examples that can be used for problem
solutions via pattern recognition. Second, Norman and Schmidt conclude that learning
from PBL may initially reduce levels of learning but lead to increased retention over a
period of years. Third, the authors found some evidence that PBL curricula may enhance
both the transfer of concepts to new problems and integration of basic science concepts
into clinical problems. Finally, they claim conclusive evidence that PBL enhances selfdirected learning skills and this is maintained.
However, in addition to his analysis of the meta-analyses, Colliver (2000)
disputed such claims, concluding there was insufficient evidence from educational
theory, concluding that the cognitive psychological theory on which PBL is based is
vague and imprecise. He keyed in on two concepts: the role of context in learning and
the activation of prior knowledge (knowledge networks in memory), claiming that both
are based on metaphor and unable to permit prediction and control.
Albanese (2000) countered that other educational theory, including information
processing theory, cooperative learning, self-determination theory and control theory,
provide better theoretical support for PBL than contextual learning theory. Of these,
perhaps information – processing theory is the most well known and accepted. In broad
strokes, this theory has three main elements, all of which are commonly stressed in PBL:
a) activation of prior knowledge, b) encoding specificity, and c) elaboration of
knowledge.
Activating prior knowledge. Learners are prompted to recall and use knowledge
they already possess to understand and structure new material-to-be-learned.
Brainstorming, advance organizers and other instructional strategies can be used to
trigger recall and prepare learners’ cognitive structure for encoding the new material.
Encoding specificity. Information-processing theory holds that the closer the
resemblance the situation in which something is learner resembles the situation in which
it will be applied, the more likely it is that transfer of learning will occur. Well written
PBL problems focus on real-life situations and, in medical education at least, present
situations most commonly seen in practice.
7
Elaboration of knowledge. Information will be better understood and remembered
if learners actively work with the material-to-be-learned. Elaboration includes such
instructional strategies as discussion, spatial mapping, teaching peers and critiquing.
In a later response to Colliver’s paper, while agreeing with his claim that PBL
does not result in dramatic differences in cognitive outcomes, Norman and Schmidt
(2000) also challenged Colliver’s claims about the support for PBL in cognitive theory.
They note that Colliver’s conclusions are based on specific studies and argue that these
do not adequately represent the diversity of findings from cognitive psychology. By way
of a counter example, they discuss a more recent study (De Grave, Schmidt & Boshuizen,
as cited in Norman & Schmidt, 2000) in which prior knowledge activation in a PBL
context produced average effect sizes of 1.30, well above the limit Colliver described as
moderate.
Problem-Based Learning and Constructivist Learning Theory
While the medical education researchers discussed above have turned to cognitive
learning theory for support for PBL, theorists in Educational Technology have found
connections between PBL and Constructivist theory. Savory and Duffy (2001) provided
an excellent discussion of the link between the theoretical principles of constructivism
and the practice of instructional design, pointing to PBL as to be one of the best
exemplars of a constructivist learning environment. In their view, Constructivism is a
philosophical view that can be captured in terms of three primary propositions:
1. Understanding is constructed individually through our interactions with the
environment and is a function of the content, the context, the activity of the
learner, and the goals of the learner. We cannot share understandings directly, but
we can test the degree to which our individual understandings are compatible.
2. Cognitive conflict or puzzlement is the stimulus for learning and determines the
organization and nature of what is learned. It is a primary factor in determining
what the learner attends to, what prior experience the learner brings to bear in
constructing an understanding, and what understanding is eventually constructed.
3. Knowledge evolves through social negotiation and through the evaluation of the
viability of individual understandings. The social environment is critical to the
development of our individual understanding as well as to the development of the
body of propositions we call knowledge. Collaborative groups are important
because we can test our own understanding and examine the understanding of
others as a mechanism for expanding our understanding of particular issues or
phenomena.
Savory and Duffy (2001) then identified eight instructional principles for the design of a
constructivist learning environment and argue that PBL exemplifies all eight principles.
Table 1 compares Duffy and Savory’s principles to Barrow’s characteristics of authentic
PBL.
8
Table 1.
A comparison of Constructivist instructional principles to the characteristics of authentic
PBL.
Constructivist Instructional Principles
Anchor all learning activities to a larger
task or problem.
Characteristics of Authentic PBL
Problem - based
Support the learner in developing
ownership for the overall problem or task
Student-centred; Self-directed learning
Design the task and the learning
environment to reflect the complexity of
the practice environment.
Problem - based
Design an authentic task.
Problem - based
Give the learner ownership of the process
used to develop a solution.
Problem-solving; Student-centred;
Self-directed learning
Encourage testing ideas against alternative
views and alternative contexts.
Problem-solving
Design the learning environment to support
and challenge the learner's thinking.
Problem-solving
Provide opportunity for and support
reflection on both the content learned and
the learning process.
Reflection
9
Designing PBL for online learning
What would an Online PBL course look like?
What are the critical factors for the design of online PBL? The following section
provides an overview of the structure for an online course, Agriculture course, Agro 260,
AgroEcology, one of two online PBL courses taught at the University of British
Columbia. These courses were delivered using WebCTTM Campus Edition 3.8.
Figure 1. Agro 260 Splash Page
10
The course material in Agro 260 is introduced through the study of four Cases impacting
on the practice of Agroecology: a) grazing ecosystems, b) organic vegetable production,
c) tree fruit agroecosystems, and d) genetically modified organisms and rural
communities. Students are assigned to PBL groups of no more than 8 – 10 students and
each group is supported in the process by a dedicated tutor.
Figure 2. Agro 260 Case listing.
11
PBL process & evaluation forum. Each case has its own PBL Process Forum,
found in the Discussion section of the course website. This forum is used to
review/discuss the group’s ground rules and the overall process for conducting work
within each Working Round. It provides an opportunity to define and critique the group
process, and to give individual feedback, separate from the content discussions in the
Working Rounds Discussion Forum (see below).
Figure 3. Agro 260 discussion groups.
Each group member is expected to make at least one contribution to this forum in
the first two (2) days of the case, at which point the Ground Rules are expected to have
been established. The forum remains open for the length of the case to provide a place
where group members and the tutor can raise questions or concerns about how the group
12
is working and how the case is proceeding (equivalent to ‘time outs’ in face-to-face PBL
groups).
Working Rounds
Each case is comprised of multiple ‘rounds’, each of which may include
disclosures and discussion forums.
Disclosures. Here students are presented with the scenario that introduces the
problem that they are being asked to address or else provides more information about it
(supplementary disclosures).
Figure 4. Agro 260 Case 1 problem statement.
13
In most cases, these supplementary disclosures are made available as learners discuss the
scenario and identify further information that they require. In reference to Barrows’
criteria of authentic PBL, it is interesting to note here that most of the resources students
need find a solution to the scenarios are provided on the course website or via links to
other web sites, especially governmental sites. In addition, students are required to
purchase a course textbook and readings are stipulated for each case.
14
Discussion Forums. Each scenario is accompanied with some general guidance
and discussion questions to help the group identify what the problem is, what the learners
already know that will help solve the problem, and what further information they will
need to address the problem. These discussion questions are meant to help the group to
identify learning issues, i.e., specific questions that one or more group members will
research and report back to the group with their findings. The discussion of questions and
identification of and reporting on learning issues takes place in a Discussion Forum.
Figure 5. Agro 260 Case 1 discussion questions.
15
Assignments. While the learning process in PBL is designed as a cooperative
effort, student assessments in Agro 260 are based primarily on individual assignments
and examinations. There is one group assignment (Case 1) where the submission
requires the collaborative effort of the group to develop a single submission. Otherwise,
group members complete an individual written assignment designed to address the
problem(s) raised in the case. Details of these assignments are provided through
disclosures as the case progresses. Assignments are submitted to the tutor by email. There
is also an evaluation component of the Process & Evaluation Forum that provides for
assessment of both group process and individual participation. This process involves selfevaluation, peer- evaluation, and facilitator- evaluation, as well as an assessment of how
well each student thinks his / her group is working. Participation is rated on a pass-fail
basis. This means that no marks are assigned for participation per se, but if a student’s
involvement is not rated as satisfactory, he or she fails the course regardless of the other
marks assigned.
Figure 6. Agro 260 Assignment 1 (Case 1) instructions.
16
Learning objectives. On the final day of each case, the learning objectives for the
case are made available via a case Icon. The intention is to inform the students what they
were expected to learn from the case. The final and mid-term exams are based on the
learning objectives from all four cases.
Figure 7. Agro 260 Case 1 learning outcomes.
17
References
Albanese, M.A. (2000). Problem-based learning: Why curricula are likely to show little
effect on knowledge and clinical skills. Medical Education, 34, 729-738.
Albanese, M.A. & Mitchell, S. (1993). Problem-based learning: A review of the literature
on its outcomes and implementation issues. Academic Medicine, 68(1), 68-81.
Barrows, H. S. (1986). A taxonomy of problem-based learning methods. Medical
Education, 20, 481-186.
Barrows, H. S. (1996). Problem-based learning in Medicine and beyond: A brief
overview. New Directions for Teaching and Learning, 68, 3 – 12.
Barrows, H. S. (1998). The essentials of problem-based learning. Journal of Dental
Education, 62(9), 630-633.
Barrows, H. S. (2002.). Is it truly possible to have such a thing as dPBL? Distance
Education, 23(1), 119-122
Boud D. & Faletti, G. (1991). The challenge of problem-based learning. London: Kogan
Page.
Clark, R.E. (1983). Reconsidering research on learning from media. Review of
Educational Research, 53, 445 – 459.
Clark, R.E. (1994). Media will never influence learning. Educational Technology,
Research and Development, 42(2), 21-29.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ:
Lawrence Erlbaum Associates.
Colliver, J.A. (2000). Effectiveness of problem-based learning curricula: Research and
theory. Academic Medicine, 75(3), 259-266.
Distlehorst, L. H., & & Robbs, R. S. (1998). A Comparison of Problem-Based Learning
and Standard Curriculum Students: Three Years of Retrospective Data. Teaching
and Learning in Medicine, 10(3), 131-137.
Glass, G.V., McGaw, B., & Smith, M.L. (1981). Meta-analysis in social research.
Beverly Hills, CA: Sage Publications.
Lehtinen, E. (2002.). Developing models for distributed problem-based learning:
Theoretical and methodological reflection. Distance Education, 23(1), 109-117.
Maudsley, G. (1999). Do we all mean the same thing by "problem-based learning"? A
review of the concepts and a formulation of the ground rules. Academic Medicine,
18
74, 178-185.
Murphy, E. (2004). Identifying and measuring ill-structured problem formulation and
resolution in online asynchronous discussions. Canadian Journal of Learning and
Technology, 30(1), 5-20. Retrieved January 26, 2005, from http://www.cjlt.ca.
Norman, G. R. & Schmidt, H.G. (1992). The psychological basis of problem-based
learning: A review of the evidence. Academic Medicine, 67, 557-565.
Norman, G. R. & Schmidt, H.G. (2000). The effectiveness of problem-based learning
curricula: theory, practice and paper darts. Medical Education, 34, 721-728.
Orrill, C.H. (2002.). Supporting online PBL: Design Considerations for supporting
distributed problem-solving. Distance Education, 23(1), 43-57.
Russell, T. (1999). The no significant difference phenomenon: a comparative research
annotated bibliography on technology for distance education. Montgomery, AL:
The International Distance Education Certification Center.
Savery, J. R. & Duffy, T.M. (2001). Problem based learning: An instructional model and
its constructivist framework (Tech. Rep. No. 16-01). Indiana University, Center
for Research on Learning and Technology.
Smits, P.B.A., Verbeek, J.H.A.M. & de Buisonjé, C.D. (2002). Problem-based learning in
continuing medical education:A review of controlled evaluation studies. British
Meidcal Journal, 324, 153-156.
Vernon, D.A. & Blake, R.L.(1993). Does Problem-based Learning work? A metaanlaysis of evaluative research. Academic Medicine, 68(7), 550-563.
19