Multimedia design and performance in a higher-order
thinking task
Lee Campbell
Flinders University
E-mail: [email protected]
The infusion of Information and Communications Technology (ICT) into the
curriculum is gaining momentum in many educational systems, however, the
debate about ICT effectiveness continues.
This paper is written by an early career researcher and examines how the
development of higher-order thinking skills was incorporated in the design of a
learning object on water conservation. This study examines the effect of the
learning object on higher-order thinking skills.
Year 4/5 students were given a knowledge test before and after they completed
the learning object. Their answers were subsequently scored according to
Biggs’ Structure of the Observed Learning Outcomes (SOLO) Taxonomy.
The paper presents the results of the study, gives implications for the use of
technology in the classroom and directions for developing multimedia/learning
objects for students in Years 4 and 5.
ICT in the Classroom
According to the International Society for Technology in Education (1998), the practices in traditional education
are no longer sufficient to give students the skills they need to participate in the workforce today. “The most
effective learning environments meld traditional approaches and new approaches to facilitate learning of relevant
content while addressing individual needs” (The International Society for Technology in Education, 1998, p 2).
Advantages of Using Learning Objects
According to the Australian Education Systems Officials Committee (AESOC, 2001) the design and flexibility
of learning objects allows them to be used in a variety of curriculum contexts. They can be reusable, provide a
representation of authentic settings, include activities which encourage interactivity and self-directed learning
and provide resources that have visual appeal to students (Australian National Training Authority, 2003).
A significant benefit of using learning objects in the classroom is that they are versatile. AESOC (2001) states
that they can be used for students to learn about topics, as separate components in a unit of work, or as part of a
sequence of learning. Teachers can decide how best to use learning objects with a whole class, small groups or
with individual learners, depending on what suits their particular situation.
ICT Teaching Model
When discussing how ICT can be infused with the curriculum, it is important to examine the process of
education and the use of ICT. Figure 1 is a model, which depicts the dynamics of ICT in education as a two way
process between teaching and learning. Both teachers and students initiate instructional activities and various
teaching media and methods are used. Learning is student-directed and self-paced. In addition to teacher
assessment, students carry out self-assessments and peer assessments. Evaluation of teaching methods and media
used is done by students and teachers and can be used for assessment, reporting and for teachers to evaluate their
own needs in professional development (Campbell, 2003c).
Figure 1: ICT Teaching Model (Campbell, 2003a)
This model has been applied in the design and development of The Warriparinga Wetlands learning object
(Campbell, 2003b) used in this study. The learning object is interactive, allowing student learning to be selfpaced and giving teachers the freedom to facilitate student learning, observing students and providing
individualised assistance.
The PowerPoint project included in this study gives students the opportunity to present their research in a fun
format and can be used by teachers to assess student learning. It can also be used for students to assess and
provide feedback to their peers and for presentations to parents on the work students have completed. Teachers
can be provided with professional development training on the use of learning objects within their curriculum,
how to create movies, how to use PowerPoint, extension activities which can be incorporated with the use of ICT
in the curriculum, etc.
Design of the Learning Object
Campbell (2003c) discusses several considerations incorporated in the design of the learning object. They are,
higher-order thinking skills (HOTS), interactivity, visual demonstrations and animations, interest, pedagogical
appropriateness and adaptability. This paper will concentrate on the inclusion of HOTS.
Instructional Design
Jonassen, Tessmer and Hannum (1999) point out that the first step in instructional design is task analysis. This is
arguably the most important component of the process of instructional design and is used as a blueprint for
instruction (Jonassen et al., 1999). Task analysis consists of five functions, as described in Table 1.
Table 1: Functions of task analysis (for more detail see Jonassen et al., 1999)
Function
Description
Inventorying tasks
Process of identifying or generating a list of the tasks, which are relevant for
instructional development.
Selecting tasks
Tasks are selected from the inventory, according to educational objectives and
avoiding repetition of material students already know.
Decomposing tasks
Components of tasks, goals or objectives selected from the inventory are
identified and outlined.
Sequencing tasks
Refers to the sequence in which the instruction should occur.
Classifying learning
Refers to the analysis of tasks and their components to determine the type of
outcomes
learning involved.
From prior teaching experience and study, the researcher was aware that a truly useful learning object would not
simply give students information, but would include tasks that would direct student learning and require them to
think about the information presented. In choosing these tasks, the researcher wanted to develop HOTS in
students and therefore included problem-solving activities, using open-ended questions that required students to
research answers. When designing the learning object, a storyboard was used to plan the visual content and
sequence it logically, including the open-ended questions.
When selecting the method for presenting learning object content, the researcher wanted to use material that
would generate interest for the students and give powerful visual demonstrations. For this reason, Macromedia
Flash animations were included. Video footage was also included to provide authentic examples. Campbell
(2003a) details several learning outcomes associated with the learning object.
HOTS
Higher order thinking was a foremost concern in the design of the learning object. Porter (May 2003, p 15) states
that, “Meeting the demand for 21st-century skills will require shifting student work into higher gear from
activities that use knowledge to activities that help students become information seekers, analysers, evaluators,
innovative thinkers, problem-solvers, decision-makers and producers of knowledge.” Higher-order thinking is
then discussed in relation to Bloom’s Taxonomy (Bloom, 1956) and the top three categories in the Cognitive
Domain. According to Blooms’ revised Taxonomy, these categories are analysing, evaluating and creating (Pohl,
2003).
For the purposes of this study, Biggs’ Structure of the Observed Learning Outcomes (SOLO) Taxonomy was
used. This was a useful instrument for the current study because it measured the depth of learning and “provides
a powerful tool to assess the outcomes of learning, and by implication the effectiveness of the learning activity”
(Jackson, 1998, p 22). The SOLO Taxonomy provided a “systematic way of describing a hierarchy of
complexity which learners show in mastery of academic work” (Jackson, 1998, p 23). Biggs’ Structure of the
Observed Learning Outcome Taxonomy (2003) states that the sequence of levels in the SOLO Taxonomy can be
used as a guide for formulating specific tasks or assessing specific learning outcomes. In this study, it was used
for both. Table 2 describes each level of the SOLO Taxonomy and shows the corresponding level in Bloom’s
revised Taxonomy.
Table 2: Description of SOLO levels with matching levels in revised version of Bloom’s Taxonomy (Black,
1997; Jackson, 1998).
SOLO Level
Bloom’s Taxonomy
(revised)
General description
Remembering
Understanding
Applying
Analysing
The student does not understand the topic and therefore does
not attack the task appropriately.
One aspect of the topic is understood.
Students have learned several aspects of the topic but treat
each part separately.
Students demonstrate that they can amalgamate several
related aspects into a coherent whole. They are able to
display cause and affect relationships.
Students are able to generalise their knowledge into a new
topic or area.
1. Pre-structural
2. Unistructural
3. Multistructural
4. Relational
5. Extended Abstract
Evaluating
Creating
This Taxonomy was useful in the current study because it provided a valuable tool for creating the research
instrument used to assess HOTS gained by the sample. It also gave detailed outcomes, which could be seen in
student work and was used to score the data collected from the study sample (see “Research Design and Data
Collection”).
Incorporation of HOTS in the Design of the Learning Object
Stoney and Oliver (1999, p3) found that, “…a well-designed program will drive learners towards greater levels
of higher-order thinking…” This is followed by a discussion of the relevant aspects of such a program. They are
that materials within the program are understandable, interesting and relevant, linked to real-life contexts and
require the application of the content and problem-solving skills. The animations within the learning object
create interest for students. The issues raised by the learning object are relevant to students’ every day lives
because they are encouraged to think about how they can personally assist in conserving water. The learning
object is linked to the real-life problem of water conservation and the Warriparinga Wetlands project provides an
authentic example of community action. Students are required to apply problem-solving skills and use the
information in the learning object to answer open-ended questions about water conservation and the
Warriparinga Wetlands.
Cradler, McNabb, Freeman and Burchett (2002, p 48) state that, “The role of the teacher is paramount in guiding
the development of students’ higher order thinking skills during learning activities involving technology tools.”
The PowerPoint project has been included within the study to examine the importance of teachers in providing
additional activities to further enhance HOTS development.
The paper written by The Department of Education in Hawaii (date unknown) highlights that students become
independent critical thinkers when they use the Internet to research topics, share information and complete a
final project. Depth of learning is also increased when students use technology to publish, present and share
results of projects (Department of Education, Hawaii, date unknown). The learning object was designed with the
intent of students using the Internet for research and sharing information on water conservation. In order to
promote higher-order thinking and create a resource in which students needed to do more than merely read
information, four open-ended questions were included within the learning object. These questions were designed
to encourage students to problem-solve by researching solutions and considering the impact humans have on the
environment. The PowerPoint project further consolidates student learning from the learning object and provides
an opportunity to publish, present and share the information they learned.
The Department of Education in Hawaii (date unknown) found that HOTS could be developed when students are
able to problem-solve and develop solutions, utilising technology. HOTS are particularly enhanced when
problem solving is done in collaborative groups. The learning object was designed for students to collaborate in
pairs, facilitating communication of ideas on conserving water.
Description of the Learning Object
The Warriparinga Wetlands learning object (Campbell, 2003b) was designed and developed by the researcher. It
is a resource for teachers to use to help students understand how waterways become polluted and what they can
do to help solve this problem. It highlights the Warriparinga Wetlands on Sturt Road, Bedford Park as an
example of community action against the problem of water pollution. Campbell (2003c) gives a detailed
description of the learning object and its design and development. Below is a brief explanation of how it works.
The Warriparinga Wetlands (Campbell, 2003b) is an interactive program, which was created using a
combination of video footage, PowerPoint slides and Macromedia Flash animations. It has been interfaced with
Microsoft PowerPoint, which provides instructions to students so they can work through the exercises at their
own pace. These instructions can be adapted by the teacher as needed (see Figure 2).
Students click here
to view movie
Instructions for students
(can be adapted by the
teacher as necessary)
Links to
research pages
Figure 2: Microsoft PowerPoint interface (Campbell, 2003b)
The first two research questions are, “How do our rivers and lakes become polluted?” and, “How can we help to
protect our rivers and lakes?” The movie stops after each of these questions and students research the answers
using the links contained within the learning object (see Figure 2). The movie then continues, giving students
both written and visual examples of the answers to these questions (see Figures 3 and 4).
Figure 3: Written information giving possible
answers to the question
Figure 4: Video footage demonstrating how to
dispose of cooking oil appropriately
Following this, the Warriparinga Wetlands are shown. Macromedia Flash animations have been incorporated in
order to create interest for students and to give a powerful demonstration of the processes involved at the
Warriparinga Wetlands (see Figure 5). After each animation is a transition into real-life video footage of the
Warriparinga Wetlands (see Figures 5 and 6).
Figure 5: Animation demonstrating the gross
pollutant traps catching litter and debris
Figure 6: Video footage of the gross pollutant traps
shown in the animation
The final two questions were included to encourage students to think about the processes at the Warriparinga
Wetlands. Again, they are open-ended, asking students to relate what they have learned about the Warriparinga
Wetlands to water conservation. Students need to use the animations and video footage to answer these
questions, which are, “What do the Warriparinga Wetlands do?” and, “How do they do this?”
Research Design and Data Collection
For the purposes of this paper, the research question being examined is as follows:
 How does the selected learning object impact on HOTS?
Study Sample
The participants in this study were from two Year 4/5 classes, in a co-educational private metropolitan school. A
total of 28 students, 13 male and 15 female participated in the study. The average age of the participants at the
time of the study was 9.6 years. A total of 24 participants have a computer at home, which they use for
homework related activities as well as for playing games and exploring the Internet. The school was chosen
randomly and the two Year 4 /5 classes were the only classes of that combined year level within the school.
ICT is taught as a separate subject on a weekly basis and the students involved in the study have two to three
lessons per week with their class teacher. The ICT co-coordinator’s role is as a support person and she works
with each class and their teacher for one lesson per fortnight. The school made available its computer lab, which
accommodated two students per computer.
Research Instruments
During the course of data collection, students were given a questionnaire asking them for general information.
This information was used for data analysis and to enable the researcher to gather information on students’ prior
experience with computers. This questionnaire (adapted from Harris, 1999) collected data about the subjects’
name, age, whether they have a computer at home, what sort of programs they use, whether they play games and
how many hours a week they spend on the computer. It was distributed and collected by the teacher on behalf of
the researcher.
To assess subjects’ HOTS gained from each activity, they were given a worksheet (titled ‘Knowledge
Questionnaire’), which included all of the questions listed in Table 3. Subjects wrote their answers on the sheet,
which was then collected by the teacher on behalf of the researcher. It was important that this questionnaire test
for HOTS, therefore, the items were constructed using the SOLO Taxonomy. The comparative nature of this
study required that students be given the same questionnaire in Phases 1, 2 and 3 (see Methodology). This
enabled non-biased comparison of student achievement in each of these Phases. In order to minimise tedium as a
result of students repeating the test, the items were kept to a minimum. Table 3 lists each item (question) and the
correlating SOLO level. The items were presented in a logical sequence according to content and therefore, do
not appear in order of difficulty.
Table 3: Items within the knowledge questionnaire with matching SOLO levels.
Item
1. What does the word pollution mean?
2. Explain what you think water conservation is.
3. Draw a picture of someone doing something, which would pollute the water.
Describe what they should be doing instead.
4. Under the following categories, list the main things that pollute water.
5. List three actions you can take to help protect/save our water. Explain how each
action would help.
6. Do you think we need to have clean water? Why/Why not?
SOLO Level
4. Relational
5. Extended Abstract
2. Unistructural
3. Multistructural
4. Relational
5. Extended Abstract
As mentioned in Table 2, the first level in the SOLO Taxonomy is prestructural. No questions were created for
this level. The second level is unistructural. Question 3 was aligned with this level because it only asked for one
aspect of water pollution and the answer could be obtained directly from the learning object. Question 4 was
aligned with multistructural because it asked students to categorise several aspects of water pollution, without
requiring them to be linked together and these aspects were listed in the learning object. The relational level was
aligned with questions 1 and 5. To answer question 1, students needed to be able to link together the information
they had learned about what pollutes the water and come up with a definition of water pollution. In question 5,
students needed to discuss what they could do to prevent water pollution and link this to how it would help the
problem (they needed to display cause and effect judgements). Question 2 was aligned with extended abstract
because students needed to think about everything they learned from the learning object about water pollution
and create a definition of water conservation. Item 6 was eventually removed from the results because it did not
promote the anticipated extended abstract response from students. All students gave the obvious answer of ‘yes’
giving reasons, which were not relevant to the learning object. For example, most stated that we need clean water
for our own health or the health of animals.
Scoring and interpretation of Knowledge Questionnaire
Student answers in the Knowledge Questionnaire were scored on a scale of 1 to 5 using Biggs’ SOLO
Taxonomy. Table 4 shows the criteria used for each score given.
Table 4: Criteria used to score student responses to the questions within the knowledge test. Adapted from
Biggs’ Structure of the Observed Learning Outcome (SOLO) taxonomy (2003).
Level
Description
Example
Score
(from item number 5*)
Prestructural
Response does not address the
“More bags, not plastic bags. Less trees
1
question or is not present.
planted on the banks. No rocks like dirt.”
Unistructural
Response includes one aspect
“Clean up. Rubbish in bin.”
2
relevant to the question.
Multistructural
More than one relevant aspect is
“Make up a clean up a "save water" club.
3
mentioned but they are not linked in
Use less water when you don't need it.
a meaningful way.
Use paper bags instead of plastic ones.”
Relational
Students demonstrate an adequate
“Have short showers, which would save
4
understanding of the topic. Data is
water by not using as much. Wash your
linked together coherently and
car on the grass, which would prevent
students display cause and effect
acids going into the drain. Wash your car
judgments.
with a bucket so you don't leave the tap
on.”
Extended Abstract Students make connections beyond
None of the students displayed this level
5
the subject area.
of cognition.
* See Table 3 (Item 5 is, “List three actions you can take to help protect/save our water. Explain how each action
would help.”)
Validity of Knowledge Questionnaire
The principal of the participating school and four other teachers were asked to review the questionnaire. They
found that it was appropriate for the year levels participating in the study.
Methodology
To test the research questions, this quantitative study took place in four phases.
Phase 0: Planning, Design and Development of the learning object
The design of the learning object commenced early in 2003. Whilst in the process of development, the learning
object was shown to the ICT technician and a senior lecturer at Flinders University. Feedback from these two
experts was invaluable in completing the learning object. Before commencing data collection with the students,
the learning object was evaluated by four teachers at the participating school. The teachers made several valuable
suggestions for modification to the learning object and it was adjusted accordingly.
Phase 1: Pre-study Questionnaires
Prior to the commencement of the learning object, students were given the research instruments mentioned
above. They collected general information about their age, gender and computer usage, as well as assessing
subjects’ current knowledge of water conservation.
To alleviate any difficulties with class timetables, the classroom teachers administered the questionnaires. The
teachers were given written instructions on how to introduce the questionnaires to the students. For example,
teachers were instructed to inform students that the Knowledge Questionnaire was purely an exercise to find out
their current knowledge.
Phase 2: Learning Object
Students from each class completed the learning object during their timetabled class times, allocated for working
in the computer laboratory. Each class was allocated one single lesson per fortnight and one double lesson per
week to complete Phases 2 and 3 of the study.
Students were given two to three lessons to complete the learning object (this time frame was based on the
advice given by teachers in Phase 0). At the commencement of the first lesson, the researcher explained the
learning object to students, giving a visual demonstration of how to use it. Students were then allowed to work in
pairs. In a separates lesson, students were given the Knowledge Questionnaire to complete individually. This
assessed subjects’ knowledge gained from the learning object
Phase 3: Microsoft PowerPoint Project
Subjects were asked to work individually, writing down what they had learned under each of the following three
headings:
1. Water Pollution
2. Saving our Water
3. Warriparinga Wetlands
Subjects worked in pairs to create a short project using Microsoft PowerPoint. They were instructed to create
four slides, including a title page and one slide for each of the three headings above. Students combined the
information they had written down under each heading, into the PowerPoint slides. They were given freedom in
the creativity they put into the presentation of their slides. Each class was allowed four lessons to complete this
activity.
In a separate lesson, subjects were again tested on their knowledge gained from the activity, using the same
research instrument as in Phase 2. Students worked individually on this questionnaire.
Data Analysis
Given the small sample size (N<30), non-parametric statistics was considered appropriate when analysing data.
The analysis is self-contained, without any reference being made to the wider population. The measurement used
was the Wilcoxon signed-ranks test. This test compares the performance of the same subjects on two separate
occasions, to determine whether their scores were significantly different in the two performances (Burns, 2000).
As previously mentioned, the size of the student sample was small, therefore all the results have been interpreted
with caution. In order to enable generalisations of findings, it would be beneficial to duplicate this study with a
larger sample.
Higher Order Thinking Skills
The results of the Knowledge Questionnaire were used to answer the question, “How does the selected learning
object impact on HOTS?” A graphical representation of the results is shown in Figure 7. The x-axis represents
each of the five HOTS questions from the least difficult to the most difficult. The y-axis shows the mean score
for the study sample. In a comparison of the pre-study, post learning object and post PowerPoint project results,
it was evident that for questions 1, 2, 3 and 5, each time the test was given, students’ mean HOTS scores
increased. This indicates that, during each phase of the study, students, depth of learning was enhanced.
easy
Figure 7:
more difficult
Mean scores achieved for each question after each time the Knowledge Questionnaire was given
(N=28, maximum score = 4)
To ascertain if this increase was significant, the results were then transferred into SPSS to execute a Wilcoxon
signed-ranks test. Table 5 details these results. Each column indicates the significance of the difference between
HOTS scores from the pre-study test (Phase 1) to the post-study test (Phase 3). For example, column 2 (3C-3A)
tests for the significant difference between pre- and post-study scores in question 3.
Table 5: The difference in HOTS scores for each question during Phases 1 and 3 of the study (N = 28)
Z score
Significance
(2-tailed)
3C-3A
4C-4A
1C-1A
5C-5A
2C-2A
-1.63
-.71
-1.21
-2.39
-3.04
0.10
0.48
0.22
*0.02
**0.00
* Denotes significance at the 0.05 level
** Denotes significance at the 0.01 level
A = Question given at Phase 1 (pre-study)
C = Question given at Phase 3 (post-study)
In the two highest SOLO Taxonomy levels of relational and extended abstract (questions 5 and 2 respectively),
students scored significantly higher at the completion of the study (after the PowerPoint project) than prior to
commencing the study (question 5: Wilcoxon, Z = -2.39, p<0.05 and question 2: Wilcoxon, Z = -3.04, p<0.01).
As seen in Table 6 there are no other significant results.
Discussion
These results indicate that students’ HOTS were enhanced as a result of completing the learning object and
associated PowerPoint project. This was evidenced by the statistically significant increase in HOTS scores in the
questions designed to test for knowledge gained at the two highest levels of the SOLO Taxonomy. These
findings are in line with previous research papers written on the impact of technology on HOTS (Cradler et al.,
2002; Department of Education, Hawaii, date unknown; Stoney and Oliver, 1999).
As mentioned previously when designing the learning object, consideration was given to the content of the
program being understandable, interesting, relevant, linked to real-life contexts and including the application of
problem-solving skills (Stoney and Oliver, 1999). Stoney and Oliver (1999) found that programs containing all
of these aspects enhance HOTS. The results of the current study concur with this finding, indicating that HOTS
were enhanced. When using the learning object, students became excited about the real-life example of the
Warriparinga Wetlands, generating discussion between partners. When problem-solving to answer the openended questions about the Warriparinga Wetlands, students would replay the movies and pause in places to
ensure they had their information correct and as a result, produced detailed responses.
Cradler et al. (2002) discuss the importance of the teacher in designing programs using ICT activities. This
concurs with the results of the current study because the PowerPoint project, as an additional activity, further
enhanced the HOTS gained from the learning object. This demonstrated the importance of the teacher’s role in
designing holistic educational programs around the use of learning objects in the classroom.
The design of the learning object considered several aspects mentioned by the Department of Education, Hawaii
(date unknown), including the use of the Internet for research, sharing of information and completion of projects
to enhance HOTS. Depth of learning is also enhanced by using technology to publish, present and share results
of projects; and by employing technology to problem-solve in collaborative groups (Department of Education,
Hawaii, date unknown). This is in line with the current study, which included all of these aspects and resulted in
a significant increase in student scores in the questions aimed at the higher order thinking levels of relational and
extended abstract from the SOLO Taxonomy.
Despite the strong increase in HOTS scores of students in this study, due to the small sample size, it would be
unwarranted to suggest that the use of strategies for increasing HOTS detailed above will typically have such
positive effects on learning (Cordova and Lepper, 1996). Although, background reading on the subject concurs
with the findings indicated above, it would be useful to conduct further studies on a larger scale in order to
broaden findings to the wider population. The final section will detail implications of this study.
Implications for Teachers and Educational Software Designers
Teachers play a substantial role in the infusion of ICT in the curriculum. The following points are important to
note when considering HOTS in the implementation of ICT. To enhance HOTS, teachers need to look for
programs which provide opportunities for problem-solving, ask open-ended questions and allow for research
using the Internet and other sources. Teachers need to consider the activities they include with ICT programs to
increase depth of learning. For example, provide opportunities for collaboration on problem-solving activities,
utilise technology for presentation of work and allow students a sense of control over their own learning. All of
these aspects should also be considered by teachers when selecting software for use in their classrooms.
There are several factors to be considered when designing software, simulations and learning objects. Programs
need to be linked to authentic situations, provide interest for the intended audience, be easy to understand and
require the application of content to solve problems. Giving consideration to student-centred learning and
including open-ended research questions to direct student learning also assists in enhancing depth of knowledge.
References
Australian Education Systems Officials Committee (AESOC). (2001). The Le@rning Federation homepage.
Retrieved April 28, 2004 from http://www.thelearningfederation.edu.au.
Australian National Training Authority (ANTA). (2003, July 1). VET learning object repository green paper.
Retrieved March 15, 2004 from
http://www.learnscope.anta.gov.au/LearnScope/golearn.asp?Category=15&DocumentId=3801.
Biggs’ Structure of the Observed Learning Outcome (SOLO) taxonomy. (2003). Retrieved April 22, 2004 from
http://www.tedi.uq.edu.au/downloads/Biggs_Solo.pdf.
Black, Paul. (1997, June 5). A tyranny of numbers? Distinguishing subject year levels. Part 3: aspects of
assessment. Retrieved April 22, 2004 from
http://www.ntu.edu.au/education/csle/research/DYL/unit_levels3.html.
Bloom, Benjamin S. (1956). Taxonomy of Educational Objectives. Book 1: Cognitive Domain. United States of
America: Longman Group Ltd.
Burns, Robert B. (2000). Introduction to Research Methods (4th ed). New South Wales: Pearson Education
Australia.
Campbell, Lee A. (2003a). ICT In Education. Retrieved October 25, 2003 from
http://www.geocities.com/leeb19982001.
Campbell, Lee A. (2003b). Warriparinga Wetlands. [CD-ROM]. Unpublished.
Campbell, Lee A. (2003c). The design and development of a simulation to teach water conservation to primary
school students. Under review.
Cordova, D and Lepper, MR. 1996. Intrinsic motivation and the process of learning: Beneficial effects of
contextualisation, personalisation and choice. Journal of educational Psychology 88(4) 715 – 730.
Cradler, John, McNabb, Mary, Freeman, Molly and Burchett, Richard. 2002, May. How does technology
influence student learning? Learning and Leading with Technology 29(8) 46-49.
Department of Education, Hawaii. Date unknown. Research on technology’s impact on higher order thinking,
Hawaii Networked Learning Communities. [Online]. Available:
http://www.hnlc.org/documents/public/4%20research-thinking.doc. [2004, May 11].
Harris, S. 1999. Secondary school students’ use of computers at home. British Journal of Educational
Technology 30(4) 331-339.
International Society for Technology in Education (June 1998) National Educational Technology Standards for
Students. Eugene, United States of America.
Jackson, Barry. (1998). Evaluation of learning technology implementation. Retrieved April 30, 2004 from
www.icbl.hw.ac.uk/ltdi/evalstudies/esevalimp.htm.
Jonassen, David H., Tessmer, Martin., Hannum, Wallace H. (1999). Task Analysis Methods for Instructional
Design. United States of America: Lawrence Erbaum Assiciates, Inc.
Pohl, Michael. (2003). Revised Bloom’s Taxonomy. Excerpt retrieved April 30, 2004 from
http://rite.ed.qut.edu.au/oz-teachernet/index.php?module=ContentExpress&func=display&ceid=29.
Porter, Bernajean. (2003). Raising the bar for student performance and assessment. Learning and Leading with
Technology, 30, 14-18.
Stoney, Sue and Oliver, Ron. 1999, October. Can higher order thinking and cognitive engagement be enhanced
with multimedia? Interactive Multimedia Electronic Journal of Computer-Enhanced Learning 1(2).
[Online]. Available: http://imej.wfu.edu/articles/1999/2/07/printver.asp. [2004, May 12].