Usability Issues in ICT Assisted Learning
Mikovec, Z., Cmolik, L., Slavik, P.
Czech Technical University in Prague,
Department of Computer Graphics and Interaction
Karlovo nam.13,
121 35 Praha 2
Czech Republic
E-mail: xmikovec|cmolikl|[email protected]
Abstract
One of key problems that users meet when interacting with any software product is easiness
of communication with such a product – we speak about usability issues. This problem has
not been addressed properly in case of software products used in ICT assisted learning. In this
paper we will deal with usability issues in this field. There are various systems that support
ICT supported learning like e-learning, m-learning, t-learning etc. We will present experience
with usability testing in t-learning (iDTV – Interactive Digital Television) environment. The
topic that was taught in the iDTV learning module was cultural heritage. This topic offers a
possibility to use various media (text, pictures, videos etc.) together with different ways of
interaction. In such a way the requirements for usability in this environment are rather
complex. The paper describes the usability issues handled in individual stages of the learning
module development. Description of experiments and the results obtained will be presented in
the paper. Last but not least also set of recommendations (derived from findings we
discovered during usability testing) that led to the improvements of this particular user
interface will be presented.
Introduction
Various forms of learning supported by information technologies of all kinds exist for
decades. Many approaches have been developed that define methodologies how to create
content of courses, how to evaluate learner’s performance and how to obtain feedbacks that
could be used for course improvement. These approaches are heavily influenced by
appearance of new technologies that allow to develop new methods of the course delivery to
the learners.
The individual technologies offer large variety of ways by means of which the information are
delivered to the learners in more or less attractive ways (video, 3D animation, virtual reality
etc.). The common denominator of these ways (through which the information is delivered) is
interaction between the (computer) system and the learner. Up to now only little attention has
been paid to usability issues concerning educational programs. In this paper we will deal with
usability in standard way – roughly speaking the usability can be measured as the easiness the
user can communicate with program [2].
Usability in the learning environment has several specific features like specific characteristics
of individual groups of learners (kids, teenagers, seniors etc.), specific characteristics of
devices that are used for education (notebook, PDA, TV set etc.) and also environment where
the educational process takes place (home, public transport, school etc.). All these features
(and some others too) influence the comfort of the user during communication with the
system. In case when this comfort is low the user must concentrate himself/herself on
communication with the system and not on the learning stuff that is delivered (what can
substantially decrease the efficiency of the learning process). In other words the main goal of
usability testing is evaluation of user interface by means of which the user communicates with
the program.
Figure 1 – TV remote control.
Our research was done in the field of iDTV (Interactive Digital TV) [1]. Here we had to
develop special user interface where the user communicates with the course by means of
remote control – see Figure 1. It is obvious that the possibility for interaction is much more
limited than in the case of traditional interaction devices like keyboard, etc. [3]. There are
several ways of interaction like cursor movement, direct interaction, etc. In such a way we can
interact with objects on the screen (e.g. menus) – see Figure 2. This limited way of interaction
puts high requirements on the quality of the design of the user interface. This means that
usability testing should be involved in all phases of the user interface design and
implementation.
Figure 2 – The user interface of iDTV course
Usability testing
There exist standardized methods for usability testing that are used for testing general
software products [5]. In this paper we will describe how these general methods were used for
usability testing of educational programs. The guidelines developed for this type of programs
were applied at CTU Prague for usability testing of educational programs in iDTV
(Interactive Digital TV) environment. Special attention was dedicated to the testing of
specially developed user interface for iDTV environment. Such an interface was based on
standard remote control for TV (the users were familiar with such a device) where some
specific interaction features were added [4].
The testing consisted of two phases. In the first phase the qualitative test on a group of 7
participants was performed. Based on the test results of this first phase the user interface was
improved with the intention to minimize influence of the usability issues of user interface on
the qualitative test (second phase).
In the second phase the quantitative test on 37 participants was performed with the aim to
perform the statistical analysis of the results obtained (see Table 1 and Table 2). In this case
we had to concentrate ourselves to two main aspects: low familiarity with modern
technologies in general, and health problems related to their age (visual and hearing
impairments, etc.). Participants of the testing were divided into two groups. Participants in
both groups filled two identical questionnaires; the first was filled before the participant saw
the course and the second after the course. Then the difference in number of right answers
can be used as a measure of gained knowledge. The experiment was organized in such a way
that the short term memory of the participants was used.
The participants in the first group (see Table 1) were not allowed to interact with the course
(e.g. answer the questions – the traditional passive TV watching was used in this case) in
contrast to the participants in the second group (see Table 2) who were allowed to interact
with the course. Thus the difference, if any, of gained knowledge between the groups may
indicate the influence of interactivity on the learning process.
participant #
correct answers
before
correct answers
after
improvement
1
2
3
4
5
6
7
8
9
10
11
12
5
4
2
4
2
4
5
3
4
6
3
3
7
2
9
5
9
7
7
3
8
6
8
4
9
4
8
5
9
5
8
2
6
3
6
3
Table 1. Results of participants who did not interact with the educational course
participant #
correct
answers
before
correct
answers
after
improvement
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19
2
2
4
2
5
3
3
1
4
2
4
4
1
2
5
3
7
6
5
7
5
7
5
4
0
5
3
7
2
5
2
9
6
2
1
4
0
7
5
5
1
9
5
9
8
9
7
9
4
6
3
9
2
8
2
9
4
Table 2. Results of participants who did interact with the educational course
Results
The collected data were statistically processed with the following results:
The number of right answers after the course was significantly bigger than the number of right
answers before the course in both groups. That is that participants in both groups have gained
knowledge during the course. The difference of gained knowledge in the groups is not
significant. Therefore we cannot say what influence has the interactivity on the learning
process. Only an assumption can be made.
The mean value of samples in the first group (4.083) is bigger than mean value of samples in
the second group (3.421). The variance of the samples in the first group (2.447) is lesser that
variance of samples in the second group (5.257). From these facts we can make assumption
that the interactivity was disturbing for the participants and they did not pay enough attention
to the content, because they had to focus also on the interaction itself.
The main finding from the experiment is that the quality of user interface plays a significant
role in the quality of the learning process. This means that the poor user interface occupies
user’s mind and the remaining mental capacity is not in general sufficient for problem
solving. This is especially obvious in situations when the problem (e.g. answering question)
should be solved within limited amount of time.
In another set of experiments the users worked with two kinds of user interfaces. The first
user interface was designed in an intuitive way. The users performed tests with this user
interface and several flaws were detected in the user interface design. In the design of the
second user interface these findings were taken into account.
The improved user interface was much easier to use for the users than the first one (both from
the point of easiness of use and from the point of time necessary to perform the task). Our
hypothesis (derived from these experiments) is that the quality of the user interface in tlearning courses can influence the quality of the learning process. To prove the correctness of
this hypothesis will require extensive experiments. These experiments should include various
learning topics (history, physics, etc.) with conceptually different user interfaces.
Conclusion
In this paper the set of experiments was described. These experiments have shown that the
quality of user interface influence the efficiency of the learning process. The user attention is
divided between the learning process and the interaction with the user interface. The poor
quality of the user interface leads in poor performance of the user in the learning process. The
role of the user interface in the learning process was shown in the first experiment. The role of
the quality of the user interface in the user mental load was shown in the second experiment.
The future work should be concentrated on experiments that interconnect findings from both
types of mentioned experiments (influence of user interface on the learning process and
mental load when using various user interfaces).
Acknowledgements
The research is conducted within the framework of the ELU project (Enhanced Learning
Unlimited) and is funded by European Commission within FP6 (IST-4-027866). See
http://www.elu-project.com/.
This research has been partially supported by MSMT under research program MSM
6840770014.
This work has been partly supported by the MSMT under the research program LC-06008
(Center for Computer Graphics).
References
1. Bates, P. J., 2003. A Study into TV-based Interactive Learning to the Home. Available at:
http://www.pjb.co.uk/t-learning/contents.htm
2. Nielsen, J. 1993. Usability Engineering. Academic Press, London.
3. ELU project. Enhanced Learning Unlimited. Project funded by European Commission
within FP6 (IST-4-027866). See http://www.elu-project.com/.
4. Cmolik, L., Mikovec, Z., Slavik, P. 2008: Prototyping Tool for T-learning Courses. To
appear in proceedings of the Didmattech 2008 conference.
5. Kuniavsky, M. Observing the User Experience 2003. Morgan Kaufmann Publishers, San
Francisco.