Maaike Prangsma - Multimodal representations in collaborative history learning
5.
General discussion
We cannot – in the 21st century – imagine a school textbook not containing many
different types of illustrations. Modern schoolbooks are packed with pictures, tables,
graphs, and diagrams – in addition to texts. Visual or multimodal representations are
seen as being instrumental in encouraging deep learning. When explaining my research
to non-educationalists the most common response is: “Of course pictures make learning
easier and better!” However, this opinion was not always predominant, and pictures
were not always taken for granted as an ingredient of learning materials. The question,
which has been researched since the 1960s, is: Do these pictures, tables, graphs, et
cetera really do what we think they do and why? The main topic of this thesis is
multimodal representation used in learning. The aim was to determine whether using
multimodal representations in history learning tasks makes a difference for the learning
outcomes achieved and/or the learning process carried out. Using representations is
defined in this thesis as co-constructing representations in small groups.
This chapter is outlined as follows. First, the four research questions mentioned
in the introduction to this thesis are answered, followed by an answer to the main
research question. Next, methodological issues of the studies described in this thesis are
considered. Next, theoretical implications of the findings are discussed. Finally,
suggestions for future research are made.
5.1 Answering the research questions
What are the effects of the general type of co-constructed representation – textual,
multimodal, or integrated multimodal – on the acquisition of a chronologicalconceptual frame of reference?
Based on the literature on learning with multimodal representations (e.g., Ainsworth,
1999; Paivio, 1991; Mayer, 2001) the collaborative construction and completion of
multimodal representations should contribute more to history learning than the
collaborative completion and construction of textual representations. In addition,
multimodal representations may also facilitate the acquisition of a chronologicalconceptual frame of reference, which is important in history learning (e.g., Leinhardt,
Stainton, & Virji, 1994; Carretero, Jacott, Limón, López-Manjón, & León, 1994).
Consequently, integration of multimodal representations in a timeline should strengthen
such an effect. The individual learning results presented in Chapter 2 showed that the
collaborative completion and construction of multimodal representations did not
contribute more to learning than the collaborative completion and construction of
textual representations: Students who constructed and completed multimodal
representations did not outperform students who constructed textual representations,
although the means did point in the direction of the hypotheses. However, the results
also showed that working on multimodal representations integrated in a timeline did
lead to higher learning short-term outcomes than co-constructing textual
representations, although this significant difference disappeared in the long run. These
outcomes raised the next question, investigated in Chapter 3:
GENERAL DISCUSSION
63
Maaike Prangsma - Multimodal representations in collaborative history learning
What are the effects of the collaborative completion and construction of integrated
multimodal representations versus textual representations on the content of the student
dialogue?
Chapter 2 showed that there were some differences in learning outcomes between the
textual and timeline conditions. This raised the question – answered in Chapter 3 –
whether the content quality of the learning process might mediate this in the timeline
condition. Student dialogues were recorded during data collection for Study 1. The
utterances for Task 2 in the Text and Timeline conditions were coded as procedural,
social or content utterances. Content utterances were analysed further for level (core vs.
auxiliary) and concept use.
The results showed that there were some differences between the two conditions.
The multimodal timeline representation task elicited more utterances in general. This
was reflected in all main topics (procedural utterances, social utterances and content
utterances), although only the former two showed a significant difference. A closer
analysis of the content utterances showed that the multimodal timeline elicited more
auxiliary content utterances, i.e., it elicited more content utterances, but did this in less
direct terms; by referring to content without using full sentences with historical
concepts.
What is the effect of the type of multimodal representation – process diagram, network
chart, structure diagram, or cartogram – on the content of the student dialogue?
The preliminary exploratory analyses of pupil discourse discussed in Chapter 2
suggested substantial differences in dialogue quality between tasks within the timeline
condition, even though all four tasks were designed to elicit productive interaction. This
outcome was further explored in Chapter 3. Student dialogues were recorded during
data collection for Study 1. The utterances for all four tasks in the Timeline condition
were coded as procedural, social or content utterances. Content utterances were
analysed further for level (core vs. auxiliary), reference to visual elements, integration
of modes, and concept use.
The results showed that there were indeed some interesting differences between
the four multimodal representation task types. Task 1 – involving determining the
chronological order of pictures and writing the story they tell underneath the pictures –
elicited more utterances in general, more content utterances in particular, and more
references to visual elements on the task sheet as well as integration of visual elements
of the task sheet with information from the task text. In addition, this task elicited more
historical concepts and a wider range of the concepts available from the task was used.
Possible explanations might be found in differences between Task 1 and the other three
tasks. Task 2 – in the shape of a causal network – involved only selecting pictures, not
ordering them, and students were not asked to distinguish causes from effects. Task 3 –
a structure diagram – contained a single picture, so that discussion of selecting or
ordering pictures did not apply to this task, and questions focused mainly on the task
text. This task was perhaps too easy and the picture not really necessary to complete the
task successfully. Finally, the two cartograms in Task 4 did not contain very concrete
visual elements, and pupils seemed to have problems understanding the maps.
The references to visual elements on the task sheet as well as the number of
historical concepts used correlated positively with learning outcomes on the post-test,
indicating that it is worth encouraging learners to do both of these things.
64
CHAPTER 5
Maaike Prangsma - Multimodal representations in collaborative history learning
Further reflection on the studies described in Chapters 2 and 3 led to the idea that
perhaps one of the distinguishing factors between different types of multimodal
representations is the inclusion, exclusion or combination of abstract (schematic) and
concrete (depictive) representations. This led to the following question, investigated in
Chapter 4:
What are the effects of combining text and different types of visualisations – abstract
and concrete – in collaborative history tasks on learning historical phenomena and
concepts?
In a follow-up experiment that dealt with differences in learning results between
different types of multimodal representations – abstract, concrete or a combination of
abstract and concrete – the short term and long term learning outcomes for these three
multimodal conditions were compared to those for a textual condition. As learning
about a historical phenomenon requires understanding both abstract and concrete
aspects of this phenomenon, concepts, and relations, a combination of abstract and
concrete visualisations should have the greatest potential for enhancing learning.
Although working on abstract, concrete or a combination of abstract and concrete
representations did not result in significant differences in learning outcomes, students
who worked with concrete visualisations rated their task as easier and thought that they
had learnt more. It seems that combining text and different types of visualisations in
learning tasks does not necessarily enhance history learning. Possible explanations are a
lack of active use of the visualisations, elements of the real life setting (e.g., ability of
pupils to concentrate, absence of whole-class teaching), the way representations are
presented and relationships defined (i.e., the semiotics of the representations used) in
the domain of history, and the difficulty of multi-interpretable representation of
historical phenomena and concepts – that calls for a clear contextual frame of reference.
The central issue in this research is how multimodal representations can support
verbalisation and co-construction of meanings and relations, and thus contribute to the
attainment of domain specific conceptual knowledge. The studies in this thesis look at
learning tasks with multimodal representations, and at integrating different
representations in a timeline, and contrast these with learning tasks with textual
representations. In other words: How can learning activities with different multimodal
representations in relation to each other contribute to meaningful learning? The central
question addressed in this thesis was specified as follows:
How does making and connecting different types of multimodal representations
affect the collaborative learning process and the acquisition of a chronologicalconceptual frame of reference in 12 to 14-year-olds enrolled in pre-vocational
education?
Several conclusions can be drawn from these results. First, learning with multimodal
representations without integration in a timeline does not result in higher learning
outcomes than learning with textual representations – neither in the short term, nor in
the long run. Second, learning with multimodal representations integrated in a timeline
resulted in higher learning outcomes than learning with textual representations –
although the difference disappears in the long run. Third, compared to learning with
GENERAL DISCUSSION
65
Maaike Prangsma - Multimodal representations in collaborative history learning
textual representations, learning with multimodal tasks integrated in a timeline does
affect the collaborative learning process by eliciting more interaction in general and
more auxiliary content utterances in particular. Also, discussing content by referring to
visual elements and using more historical concepts are positively related to short-term
learning outcomes. Finally, learning with either concrete or abstract multimodal
representations, or with a combination of concrete and abstract multimodal
representations does not result in significantly higher learning outcomes than learning
with textual representations.
5.2 Methodological issues
5.2.1 Participants
All studies in this thesis involved 12 to 14-year-old participants in the first year of
pre-vocational secondary education (VMBO). A majority of Dutch pupils in secondary
school (some 60%) attend this level. In general, the language proficiency of pupils
attending such schools is lower than that of pupils in general secondary education, in
part due to the fact that a considerable proportion of these pupils (about 24%) are from
migrant families and do not have Dutch as a mother tongue.
However, because of their relatively low language proficiency, visualisations
might be particularly suitable for these learners. This also means that there may be a
differential effect for the use of visualisations. Previous research in the field of
representations for educational purposes has focused primarily on the undergraduate and
college preparatory levels. The existing lack of diversity in the choice of participants
was, of course, also an important reason to choose a different target population for the
research reported in this thesis. Rather than saying the findings reported here do not
necessarily generalise to other learner types, we can now say that perhaps the findings
of previous research do – at least to some extent – generalise to lower levels.
In addition, the practical nature of the tasks used in this research seems suitable
for this learner level, as pre-vocational secondary education is practice-oriented to suit
its pupils‟ needs. The evaluation questionnaire described in Chapter 4 showed that
pupils enjoyed working on the multimodal tasks, and that they felt they had learnt from
the tasks. At the same time, the pupils seemed to have some specific problems whilst
working on the multimodal representations that might be related to their skills and
experience. First, some of the participants in the studies described in Chapters 2 and 3
made some unexpected mistakes in colouring the maps in the post-test, for example by
colouring the North Sea as Islamic. This suggests that some of the participants lacked
basic map reading skills. It was beyond the scope of this research to investigate this
further. However, several studies have detected difficulties in working with and
interpreting maps (Postigo & Pozo, 1998; Scevak & Moore, 1998). Further research on
cartogram tasks may shed light on the question whether our target group – 12 to 14year-olds in pre-vocational secondary education – can recognise and interpret maps of
Europe. Second, the tasks might have been more effective if they had also been
reviewed in a whole-class discussion with the teacher, because these pupils are not used
to working independently for three consecutive lessons. Of course, this would probably
also have decreased differences between conditions.
There was considerable attrition in the experiment described in Chapter 2 and
Chapter 3. Although the experiment started out with 143 pupils, the final sample
includes only 85, an attrition rate of 40%. The high proportion of discarded dyads was
66
CHAPTER 5
Maaike Prangsma - Multimodal representations in collaborative history learning
mainly due to loss of concentration among pupils who were participating in the Muslim
religious festival Ramadan during the period the experiment took place. Many of these
pupils did not manage to finish all tasks, and several different pupils were absent from
each lesson. Due to the random assignment of students to the conditions, no effect of
this attrition on the results is expected.
5.2.2 Research design
The experiments described in this thesis were all carried out in ecologically valid
settings. Choosing for ecological validity has its pros and cons. The experiments were
closer to school learning situations than much previous research on multimodal
representations has been, but choosing for an ecologically valid setting also sets some
limitations on control and ethics. First, working with pupils and schools in an authentic
school setting implies that some things will go wrong, for example because people are
absent, rooms are not available, or pupils start walking around. Ethically, it is not
possible in a setting like this one to have a proper control group that only does the pretests, post-tests and retention tests or one that does not include essential learning
materials. The pupils in that condition would have to skip part of the curriculum or deal
with the topic much later, at an illogical point in the curriculum.
In addition, the experimenters encountered some technical and practical
problems with recording the pupil dialogues and organising sufficient space. First, out
of the 85 pupils left in the final sample of the first experiment, only 26 dyads (52 pupils)
delivered complete and usable recordings of all four tasks. Pupil discourse was recorded
using small portable tape recorders placed on the pupils‟ desks. Problems encountered
with these recorders concerned mainly technical malfunctioning and improper use of the
machines by pupils.
The second practical problem was organising sufficient space. The original idea
for the experiment described in Chapter 2 and Chapter 3 was to take dyads out of the
classroom one by one (two pupils at a time, or four at a time if a research assistant was
available) and give them three consecutive hours to finish the tasks. This would have
allowed for video recordings instead of only audio tape recordings and thus for the
capturing and analysis of full interaction. It would also have been possible to administer
the post-test immediately after the tasks for each dyad, instead of one to several days
after the task lessons. However, with three consecutive lessons for each dyad (out of
eight for a normal school day), about 24 pupils in each class, two classes per school, one
research assistant and two video cameras it would have taken six full days per school.
Unfortunately, it was not logistically possible for the schools to set aside space for such
an extended period of time.
No time was taken to prepare the participants for learning with multimodal
representations, whilst all participants were experienced users of textual representations.
The pilot study used for the studies described in Chapters 2 and 3 did not give reason to
include a training session. This may have given the pupils in the textual conditions an
advantage over those in the visual, timeline, concrete, abstract, and combined
conditions. In hindsight, some pupils did seem to have problems with the cartogram task
(see Chapter 3), and it might have been a good idea to build up the multimodal
representations instead of introducing all elements at once, in particular in the
experiment described in Chapter 4, where the abstract and concrete elements of the
representations were presented rather than assembled by the pupils. Building up the
representations might be especially important in the light of Nadolski, Kirschner, and
GENERAL DISCUSSION
67
Maaike Prangsma - Multimodal representations in collaborative history learning
Van Merriënboer (2005) who suggest that there might be different levels of construction
involved in complex learning tasks. This idea might be particularly helpful in improving
the cartogram task by introducing steps to help pupils understand the maps.
The task texts used for the studies reported on in Chapters 2 and 3 had different
numerical and topic characteristics, and these differences were not evened out between
the four tasks (Chapter 3), in part because the different topics required different texts.
Thus, it is possible to ascribe differences found between tasks to differences between
the accompanying texts. Recently research has been done on suitable text characteristics
focusing specifically on the same type of learners as in this thesis (pre-vocational
secondary education, VMBO). Research by Land, Sanders and Van den Bergh (2005)
shows that formal texts and narrative texts can have very different effects: Whilst the
former improves comprehension, the latter increases enjoyment, and thus motivation.
Although the texts used for the four tasks were carefully written to ensure their internal
coherence, the texts were not aligned to each other regarding a number of text
characteristics, such as the number of words. Possibly, the textual condition performed
as well as the other two conditions because the texts by themselves were powerful
enough to result in sufficient understanding of the topic. Still, the design of the first
study might have been strengthened by removing the pictures from the task texts,
although these texts and accompanying pictures were the same for all conditions. This
might have enlarged the differences between the textual and multimodal conditions.
In addition, it should also be noted that one condition was missing from the study
in Chapter 2: textual representations integrated in a timeline. The conclusion that the
timeline combined with multimodal representations is the catalyst for integration of
textual and visual information must be drawn cautiously: Possibly, differences between
the conditions are attributable to the timeline per se, and not to the combination of
timeline and multimodal representations.
In the experimental study with abstract and concrete visualisations (Chapter 4)
no differences between conditions were found. However, the test in that experiment
only measured factual recognition and recall. Van Meter and Garner (2005) showed that
benefits of tasks in which students construct visual representations (in their case
student-generated drawings) are revealed especially in higher-order assessments, such
as the test items used in the first experiment (Chapter 2).
5.2.3 Domain and topics
The tasks in the studies in Chapters 2 and 3 were presented in a fixed order for all dyads
in all conditions. It may seem a little odd to start a section on domain and topics by
saying that the tasks were not ordered randomly, as in a solid research design they
should be. However, this thesis deals with history learning, and chronology is an
inevitable ingredient of any topic in the domain of history – probably even its main
characteristic. As was noted on several occasions in this thesis, pupils have problems
developing a consistent chronological frame of reference. Consequently, the
chronological order of the topics used for the different tasks described in this thesis was
kept intact.
The next question is: Would the results have been different if different topics had
been chosen for the task content? This question can be looked at from two perspectives,
one comparing different conditions, and another comparing different tasks. For the first
point of view, it is important to realise that a topic can have a certain level of
„imaginability‟ or concreteness, but also that different texts about the same topic can
68
CHAPTER 5
Maaike Prangsma - Multimodal representations in collaborative history learning
have different levels of „imaginability‟ or concreteness. For example, an explanation of
the fall of the Roman Empire can be quite abstract, whereas an explanation of the retreat
of the Roman administrators from Western Europe which is a subtopic of the former can
be more easily visualised as actual people leaving a part of the world that still exists on
today‟s maps. Also, Vikings, for example, seem to tickle many children‟s imagination,
and once they know what they look like (the Vikings are part of the primary school
history curriculum in The Netherlands) they will likely not need extra pictures to make
their imagination run wild. Following this line of thought, a study comparing textual
and multimodal tasks about abstract topics should lead to larger differences than a study
comparing textual and multimodal tasks about concrete topics. However, the topics in
the studies discussed in this thesis all incorporated both abstract and concrete aspects:
not the abstractness or concreteness of the topics was varied, but the abstractness or
concreteness of the representation tasks (see Chapter 4).
The second point of view takes us back to Chapter 3, where different types of
multimodal representations were compared. There, the problem was that the four tasks
also had different topics, and different types of topics (e.g., developments, structure, and
causal relations). It might have been interesting to vary only one of these characteristics
at a time across four tasks. The four tasks would then have had a) the same
representation type and topic type, but a different topic (e.g., four developments, each
shown in a process diagram), b) the same representation type and topic, but a different
topic type, or c) the same topic and topic type, but a different representation type.
However, although it is possible to represent a topic from the viewpoint of different
topic types (situation b), for example by showing the development of manorialism, and
the structure of manorialism, the contents covered would not be the same, and no single
representation type would be suitable for representing both topic types. Similarly, using
different representations to represent the same topic types can easily result in illmatched topic-representation combinations. In other words, it is not always possible to
vary the factors of representation type and topic type, as they are often inherently linked
to each other.
5.3 Theoretical implications
Although Dual Coding Theory (Paivio, 1991) and the Cognitive Theory of Multimedia
Learning (Mayer, 2001) and other previous research on learning with multimodal
representations offer valuable insights, they do have their limitations, most notably in
(1) their generalisability from the technical or natural science domains to other domains,
(2) their tendency to focus on presented representations rather than construction of
representations by learners, and (3) their focus on individual settings and learning
outcomes rather than on collaborative settings and learning processes. The results of the
studies described in this thesis are discussed in the light of these aspects below.
5.3.1 Multimodal representations in different domains
Because of the specificity of the domain of history, generalisability of the results of this
thesis to other domains or topics may be limited. However, this is also one of the main
reasons why the domain of history was chosen: Most studies on learning with
multimodal representations have been limited to the natural sciences. Consequently, the
research presented in this thesis shows that perhaps learning with multimodal
representations per se is not as beneficial as some of the research on science learning
GENERAL DISCUSSION
69
Maaike Prangsma - Multimodal representations in collaborative history learning
suggests, although there does seem to be a bright future for learning by integrating
multiple multimodal representations.
The introduction to this thesis described several differences between the domains
of history and science. It was noted that representations in the natural sciences mainly
show „How things work‟, suiting the needs of learning in that domain. At the same time,
these representations also often show „What things look like‟. History learning on the
other hand requires building a chronological frame of reference. Knowing „What things
look like‟ certainly plays a role in forming such a framework, but what holds it together
is knowledge of „What things mean‟ and „Why things happened when they happened‟.
For example, after reading this thesis one may be able to interpret Figure 5.1b, but
Figure 5.1a can be interpreted more easily.
Figure 5.1a. Operation of a bicycle
pump (from Tro, 2003).
Figure 5.1b. Timeline of the transition between
Antiquity and the Early Middle Ages.
On the other hand, the task in the first study that most closely resembled a „How things
work‟-approach (i.e., Task 1 with its process diagram) also seemed to elicit the most
productive dialogue. Perhaps multimodal representations are most effective for topics
dealing with step-by-step processes such as the fall of the Roman Empire in Task 1.
The scale of the represented phenomena might also play a role here, both in
terms of length of time, and in terms of the size of objects or locations. Perhaps it is
more difficult to understand a timeline covering several ages with multiple events and
phenomena, than understanding how a bicycle pump fills a bicycle tire with air in a few
seconds, because the latter can be felt, relived in its entirety, making it easier also to
visualise. In addition, the scale of the overall objects is very different, as Figure 5.1a
and Figure 5.1b show: A bicycle pump can be held in one hand, whilst a village or
Empire cannot. Within the history domain this aspect of scale could be investigated
further by comparing, for example, learning about European versus local issues.
As was noted earlier in Chapters 2 and 4, the grammar – or semiotics – of
visualisations in the domain of history has no unambiguous principles or rules as there
are in science. A domain like science, even though it deals with complex phenomena
like waves, offers a structured visual grammar linked to basic governing principles –
such as frequency and wavelength –that remain the same, whether the wave runs
through water, air or solid matter. In history, however, even seemingly simple symbols
like arrows can be interpreted in different ways, for example as indicating dynamic
relations such as causality, or merely as temporal relations, or as static relations
describing the structure of phenomena (O‟Donnell et al., 2002). For example, the arrow
70
CHAPTER 5
Maaike Prangsma - Multimodal representations in collaborative history learning
shown in Figure 5.2, with a picture of a woman giving a chicken to a man, could be
interpreted in several ways: Why does she give him a chicken? Did it escape? Is she
paying him? Is he her husband, her boss, or a tax collector? Does she get anything in
return? The truth in the context of the Early Middle Ages is that the woman is a serf,
who gives part of her crop to the lord in exchange for protection. However, even when
the context is known pictures and schemas are often still open to multiple
interpretations. This makes it hard for students to learn to understand the form of
representation – the importance of which is underlined by Ainsworth (2006): There are
no fixed formats in history, for example for visualising time, war or manorialism.
Figure 5.2. What does the arrow mean?
5.3.2 Activity continuum: presented vs. construction
As was noted in Chapter 1, some studies on collaborative learning with multimodal
representations in small groups have shown that new domain knowledge can be built by
articulating or representing ideas and by making connections in the shape of concrete,
visual structures in concept maps (Van Boxtel, Van der Linden & Kanselaar, 2000;
Roth & Roychoudhury, 1994). By comparing interaction processes and learning
outcomes of learning activities with different types of representations we can gain
insight into the possibilities, and even more so the limitations of each of the
representational forms in (collaborative) history learning. The different types used in the
studies in Chapters 2 and 3 were: making a timeline, completing a storyboard (process
diagram), completing a causal map (network chart), describing a historical image
(structure diagram), and completing a cartogram. A comparison of these different
multimodal representation tasks in Chapter 3 showed that they do elicit different
dialogue, and some of the characteristics of the four different tasks might well be linked
to their place on the activity continuum. The task that seemed to encourage the most
content utterances involved the activity of ordering or sequencing pictorial information,
whilst less physical activity was involved in the tasks that elicited significantly fewer
content utterances. Although no dialogue analysis was done for the study in Chapter 4,
perhaps the absence of significant differences between conditions in that study is
attributable to the lack of physical activities – thus minimizing differences between the
conditions (i.e., textual, concrete, abstract, and combined). At the other end of the
continuum, research by Van Boxtel and Van Rijn (2006) suggests that letting pupils
GENERAL DISCUSSION
71
Maaike Prangsma - Multimodal representations in collaborative history learning
draw their own pictures – in their study drawings of the fall of the Roman Empire –
leads to better learning outcomes than presenting pictures to the pupils.
5.3.3 Facilitating collaborative learning with multimodal
representations
At the start of this research project, it was thought that multimodal representations
would have a facilitating role in collaborative learning, by providing a common referent
that makes communication easier, and by providing a shared frame of reference.
Dialogue analyses of the Timeline condition in Chapter 3 showed that some of the
content utterances did indeed include references to visual elements of the multimodal
representations. Thus, multimodal representations do seem to facilitate discussion of
content. One of the differences found between the textual and integrated multimodal
conditions that might explain the difference found in learning outcomes was, however,
not an increase in explicit talking about the frame of reference, but rather an increase in
implicit reference to content.
As was noted in Chapter 3, verbal deixis (e.g., through use of demonstrative
pronouns such as “this”, “that one”) in the dialogue suggested that physical deixis (i.e.,
extra-linguistic behaviour such as pointing) may have played an important role in the
communication between dyad partners. Pupils working individually would not point or
refer to anything, simply because there would be no one to point things out for or to talk
to. Still, the deictic properties of multimodal versus textual representations might be
clarified to some extent by videotaping dialogues for collaborative groups, versus eye
movement tracking for individuals. This would be an interesting topic for future
research.
5.3.4 Integrating multiple multimodal representations
As was noted in the introductory chapter, acquiring a chronological frame of reference
calls for visualisation and conceptualisation of specific periods in history and of
relationships between aspects of these periods. Previous research has shown that
students have trouble developing a consistent „chain of events and developments‟, and
as a result they cannot properly remember the chronological overview or reuse it,
causing confusion of phenomena and concepts (Beck & McKeown, 1994; Leinhardt,
Stainton & Virji, 1994). The results of the analysis of learning outcomes reported in
Chapter 2 show that integrating multiple multimodal representations in a timeline does
encourage the development of a chronological frame of reference. Dialogue analysis
confirmed the idea that visualisation and conceptualisation are important, as there was a
positive correlation between these aspects of the dialogue and learning outcomes for
learners working with multimodal timeline tasks. However, integration did not lead to
higher learning outcomes in the long run. Possibly, the chronological order of events is
harder to remember in the long term than other types of relations, and a timeline
difficult to reconstruct after a while so that it no longer functions as an anchor for
remembering.
5.3.5 Artificial laboratory studies vs. ecologically valid field studies
The research reported on in this thesis was done in a classroom setting in pre-vocational
secondary education, where the tasks were incorporated in the normal history
curriculum. Pupils were graded for their work and worked for an entire lesson during
72
CHAPTER 5
Maaike Prangsma - Multimodal representations in collaborative history learning
each session. Learning with multimodal representations in the context of the Cognitive
Theory of Multimedia Learning has been primarily tested in artificial, experimental,
laboratory-like settings, where participants were – sometimes paid – volunteers, who
did not receive grades for their learning outcomes, and where distraction by other
participants or surroundings was eliminated as much as possible. In addition, the
learning assignments were usually quite short, lasting from 30 seconds to 20 minutes.
For determining the value of theories on learning with multimodal representations for
use in education and in learning materials could benefit from an increase in the amount
of ecologically valid research, as there is apparently a gap between traditional
laboratory research and ecologically valid classroom-based research. As a result, it is
not clear whether learning with multimodal representations has equivalent positive
effects on learning outcomes in the context of the classroom where there is more
interference from factors such as motivation, or class atmosphere. The studies in this
thesis are an attempt to bridge this gap, and Table 5.1 gives an overview of some of the
major differences between Mayer‟s research on which the Cognitive Theory of
Multimedia Learning is based (Mayer, 2001), and the studies described in this thesis.
This gap may be bridged further by replicating better controlled laboratory research in
more ecologically valid settings, varying specific factors very carefully (e.g., learner
characteristics, domain characteristics, use of collaboration, types of representations,
integration of representations) one at a time. The research reported in this thesis also
suggests that in realistic educational settings, additional requirements may play a role in
the effectiveness of learning with multimodal representations. For example, multimodal
tasks might be more effective when they are embedded in a series of lessons that
include whole-class instruction and take into account the optimum attention span of the
pupils.
Table 5.1. Comparison between Mayer‟s work and the studies in this thesis.
Aspects
Mayer
This thesis
setting
lab
school
domain
physics
history
age
18-20
12-14
education level higher education psychology students
pre-vocational secondary education
location
US
NL
time
5 minutes
30 minutes
embedding
no embedding
in curriculum
group size
individual
collaborating dyads
activities
study the item
complete
no integration of multiple
integration
integration of multiple representations
representations
5.4 Future research
It would be interesting to investigate how text can support correct interpretation of
representations in different domains, as Ainsworth (2006) claims that text guides the
interpretative process in picture comprehension. In history a picture without text will
often result in a guessing game. Text is required there. Again, this might be very
different for domains or topics that deal with „How things work‟. Future research could
investigate what text characteristics are suitable for a task in which visual and textual
elements need to be integrated.
GENERAL DISCUSSION
73
Maaike Prangsma - Multimodal representations in collaborative history learning
Although no explicit attempt was made to falsify the Cognitive Theory of
Multimedia Learning in this thesis, the results suggest that prudence is in order in
generalising the principles that lie at the foundation of CTML to their use and effects in
actual learning situations in home, study and school settings. Many other studies
referring to Mayer‟s work – whether they are replication studies or indirect validations
such as the studies in this thesis – have raised the same doubts. The modality effect, for
example, could not be confirmed by a number of studies (De Westelinck, Valcke,
De Craene, & Kirschner, 2005; Jeung & Chandler, 1997; Kalyuga, Chandler, &
Sweller, 2000; Tabbers, 2002; Tabbers, 2006). It would be interesting to see how well
CTML holds when one element is changed at a time, for example, the context, the
domain, or learner type.
The research was done in pre-vocational schools with students who do not have
much prior knowledge of the topics chosen for the tasks. The effects of using
multimodal representations were often weaker than expected. In future research
variation in student characteristics – such as prior knowledge, level of experience in the
construction of multimodal representations in the domain of history, and collaboration
skills – is necessary to investigate the prerequisites for using this type of material in
pre-vocational education.
The results in this thesis suggest that it is worth taking a closer look at the role of
deictic properties of multimodal representations in collaborative learning, both with
respect to verbal deixis and physical deixis (Suthers, Girardeau, & Hundhausen, 2003).
A similar approach is also suggested by Jewitt, Kress, Ogborn, and Tsatsarelis (2001),
who make a plea for a multimodal approach to interaction research where not just the
linguistic interaction is analysed, but also the visual and physical interaction.
Finally, there is a dearth of research on how different types of representations
(such as the eleven categories defined in Lohse, Biolsi, Walker, and Rueter, 1994) work
differentially by eliciting differences in learning processes. In addition, more research is
needed on how these different types of representations work together. From a domainspecific point of view, this could lead to further research on understanding and
remembering temporal relations through construction of timelines, including through
integration of multiple (multimodal) representations.
74
CHAPTER 5