Ambient Intelligence in the
Classroom: an Augmented
School Desk
Margherita Antona1, George Margetis1, Stavroula Ntoa1, Asterios Leonidis1,
Maria Korozi1, George Paparoulis1 and Constantine Stephanidis1,2
1
Foundation for Research and Technology – Hellas (FORTH),
Institute of Computer Science, Heraklion
Crete, Greece
E-mail: [email protected]
2
University of Crete
Department of Computer Science
Greece
ABSTRACT
This paper discusses the opportunities and challenges of Ambient Intelligence
(AmI) technologies in the context of classroom education, and presents the
methodology and preliminary results of the development of an augmented school
desk which integrates various AmI educational applications. The overall objective is
to assess how AmI technologies can contribute to support common learning
activities and enhance the learner’s experience in the classroom. Young learners
were involved from the first phases of the design of the desk and its applications
using scenario-based techniques.
Keywords: Ambient Intelligence, educational applications, eLearning, English as a
Foreign Language, learner-centered design
INTRODUCTION
Ambient Intelligence (AmI) is an emerging field of research and development that
is rapidly gaining wide attention by an increasing number of researchers and
practitioners worldwide [13]. As a consequence, the notion of AmI is becoming a de
facto key dimension of the emerging Information Society, since many of the new
generation industrial digital products and services are clearly shifted towards an
overall intelligent computing environment.
AmI will have profound consequences on the type, content and functionality of
the emerging products and services, as well as on the way people will interact with
them, bringing about multiple new requirements for the development of the
Information Society (e.g., [9]). While a wide variety of different technologies is
involved, the goal of AmI is to either hide the presence of technology from users, or
to smoothly integrate it within the surrounding context as enhanced environment
artifacts. This way, the computing-oriented connotation of technology essentially
fades-out or disappears in the environment, providing seamless and unobtrusive
interaction paradigms. Therefore, people and their social situation, ranging from
individuals to groups, and their corresponding environments (office buildings,
homes, public spaces, etc), are at the centre of the design considerations.
AmI is often claimed to bring a significant potential in the domain of education
[1]. Information and Communication Technologies already permeate the classroom
environment in many ways. They can play an important role in education by
increasing students’ access to information, enriching the learning environment,
allowing students’ active learning and collaboration and enhancing their motivation
to learn [5]. This paper presents a preliminary attempt to develop an augmented
school desk and the related applications, which aim at integrating AmI technology
in the learning process. Following a discussion of current issues in technology
integration in the classroom, the paper describes the overall concept and the
hardware and software characteristics of the prototype desk. Then, a scenario of use,
and the related software applications currently under development are introduced.
Finally, a formative evaluation experiment with a small group of young learners of
English as a Foreign Language is reported and the results are discussed.
BACKGROUND
In the recent past, learning with the use of ICT was strongly related to concepts such
as distance learning [2], educational games [11], intelligent tutoring systems and elearning applications [3]. Overall, two major trends characterize recent efforts:
• Learner-centered design: learner-centered design approaches adopt and
enhance traditional user-centered design practices in the educational
context. Learner-centered design may be intended as design with the needs
of the learner at the forefront, and possibly involving learners at various
stages of the design process. Related work has addressed on the one hand
the specific characteristics of the very young population as users of
interactive technologies [8, 15], and on the other hand pedagogical and
learning-related requirements [16].
• Adaptive and personalized eLearning: these approaches aim at exploiting
intelligent tutoring and adaptive hypermedia techniques in eLearning in
order to provide educational content matching the individual learning
requirements of different learners, and to support content reusability [4]. In
particular, in the domain of foreign language learning, which is of
relevance to the work reported here, intelligent techniques are targeted to
personalized support for learning strategies and error correction [12].
Furthermore, the notion of smart classrooms has become prevalent in the past
decade [19]. Smart classroom is used as an umbrella term, implicating that
classroom activities are enhanced with the use of pervasive and mobile computing,
sensor networks, artificial intelligence, robotics, multimedia computing, middleware
and agent-based software [6]. Following the rationale of augmented technology in
the educational environments, new means of interaction - such as interactive
whiteboards, touch screens and tablet PCs - have gained popularity and have
become a major tool in the educational process, allowing more natural interaction.
Smart classrooms, for example, may support one or more of the following
capabilities: video and audio capturing in classroom [18], automatic environment
adaptation according to the context of use, such as lowering the lights for a
presentation [7], lecture capturing enhanced with the instructor’s annotations, or
information sharing between class members.
The work reported in this paper, which is conducted in the context of the AmI
Programme of ICS-FORTH [10], constitutes an initial step towards investigating the
role of AmI technologies in the educational context and in the classroom
environment. The underlying approach is learner-centered, involving a small group
of young learners from the very first steps of the design, and targeting to provide
intuitive and seamless tools to improve the learning and classroom experience. On
the other hand, adaptation and personalization techniques, as well as techniques in
the domain of intelligent computer-supported language learning [12] are revisited
here in an AmI perspective, aiming at exploiting the interaction possibilities offered
by AmI technologies towards facilitating learning of English as a foreign language.
An augmented school desk has been selected as a first AmI artifact to be developed
in the context of the project, along with a series of educational applications which
integrate ambient interaction as well as digital augmentation of physical paper (e.g.,
[17]).
DESIGNING AN AUGMENTED SCHOOL DESK
PHYSICAL ARTIFACT DESIGN
In the context of AmI, the classroom is a challenging environment. In practice,
there are severe space and layout limits to the introduction of AmI equipment,
which should be unobtrusive, hidden or embedded in traditional classroom
equipment and furniture. It is very important that such equipment can be installed
smoothly and easily moved around in the environment, and that space requirements
are as limited as possible. This implies several constraints on how the AmI
classroom environment can be developed. To address this issue, the AmI Classroom
project has adopted an artifact-oriented approach, by stepwise introducing
independent AmI augmented artifacts in the environment.
As already mentioned, the first such artifact is an augmented school desk
(Figure 1), where an additional piece of furniture has been designed to fit typical
school desks of standard dimensions according to EU normative1. Such an ‘add-on’
provides a custom plexiglass 27 inches diagonal wide screen whose inclination can
range from 30o (with respect to desk surface) to completely horizontal. It embeds
almost invisibly all the devices required for the operation of the AmI applications,
and has a width of 40 cm, thus requiring relatively limited additional space with
respect to the standard desk. A wooden prototype of the desk has been built. The
green color has been chosen to facilitate vision processing.
Figure 1. The design of the augmented school desk
HARDWARE AND SOFTWARE SET UP
An important consideration in the design of the augmented school desk was that
AmI in the classroom should be compatible with the school of today, as the
anticipated transition to the paperless classroom does not appear so imminent.
Therefore, as a first step, the augmented school desk should smoothly support
paper-based learning materials and the use of handwriting. This requires the
adoption of vision techniques on the front part of the desk, as well as smart pen
1 EN 1729-2:2006 Furniture. Chairs and tables for educational institutions. Safety requirements and test
methods.
integration. On the other hand, it was considered important to embed in the
enhanced desk vision-based back projection multi-touch interaction, in order to
avoid ceiling mounted or hanging projectors and cameras and ensure gesture
interaction quality under variable lighting conditions. This option had several
important implications, as it was not an easy task to realize a back-projection multitouch set-up within the available space. The resulting hardware set up includes
(Figure 2):
• An Intel Core 2 Quad Core PC
• 2 DLP mini projectors located behind the screen
• 1 mirror for reducing the projection distance
• 2 cameras located behind the screen
• 4 infrared projectors located behind the screen
• 1 camera located on top of the screen and capturing images of the
conventional desks
• 1 smart pen and its transmitting device2
Figure 2. Hardware set up
Figure 3. Multi-touch screen
The PC runs MS Windows 7. To better exploit the custom screen dimensions, a
horizontal window manager was developed which includes two application and two
menu areas. In order to support multi-touch interaction, the software reported in
[14] is used. The smart pen input is captured through the Pegasus SDK3. The
necessary front vision software is currently under development. The multi-touch
display embedded in the desk (Figure 3) allows augmenting physical learning
materials with additional context-dependent information.
USAGE SCENARIO AND EDUCATIONAL APPLICATIONS
An important objective of the AmI Classroom project is to ensure that the
developed technological artifacts go at the heart of the learning process. To this
purpose, learning of English as a foreign language was selected as a testing domain,
and current practices were examined to identify useful support which can be offered
2
3
http://www.gandc.gr
http://www.pegatech.com/_Uploads/Downloads/DevelopersWebSite/index.html
through AmI technologies, focusing on preparation for the first and advanced
certificates (thus addressing an adolescent student population). Based on such an
analysis, extensive usage scenarios were compiled. The scenarios mainly address
activities which take place in the classroom; however, an important consideration is
that the software to be developed should be general enough to support also learning
at home or elsewhere, independently from the classroom infrastructure and the
augmented desk itself. In particular, the scenarios address the seamless contextdependent provision of useful additional information during language-learning
activities. Interaction with the provided facilities is based on gestures, either through
the desk screen or directly on paper resources. For example, the learner can indicate
a word on the page to view additional information about it, or the answer to a fillthe-gap exercise to receive feedback or hints. Text entry during learning activities is
based mainly on handwriting using a smart pen. However, an on-screen keyboard is
also currently under design to support small text entry tasks on the touch screen.
The desk should also be able to identify each user through a personal object (e.g., a
school diary, pen, etc).
The developed scenarios led to the identification of a set of software
applications for enhancing English learning experience through the augmented desk.
These include the login screen, the welcome screen, an individual personal area
summarizing the current delivery status of all assignments, a dashboard where
students can temporarily save material, the assigned exercises in electronic form
with the supported hints and help, a dictionary-thesaurus application, a personal
dictionary application, a note-taking applications, an application for viewing course
related multimedia, and language-learning games (e.g., hangman). The dictionarythesaurus presents a short definition of the word, its pronunciation, a button
allowing the student to hear the pronunciation, some synonyms and a few examples
of use in a sentence. In addition, the thesaurus offers options for extended
descriptions, grammar information, complete list of synonyms, antonyms and
several examples. Figures from 4 to 7 show some mock-ups of the designed
applications.
Although the enhanced desk is designed for individual use, collaboration is
foreseen for some tasks. For example, the students can exchange materials and the
teacher can send materials (e.g., assignments) to the personal areas of all students
through simple gestures. The smart environment automatically restricts actions that
can be carried out by the students according to the current context. For example,
when a reading task is active, the students cannot use some functionality in their
system (e.g., multimedia, saving, printing, etc.), and when an exercise task or a test
is being carried out, students are not allowed to send material to other student’s
personal areas. The system will also produce statistics regarding the hints that have
been requested, per student and for the whole class, so the teacher will be able to
monitor the student’s progress. The system will also monitor the success rate of
each exercise and provide statistics. Teacher can then combine this with the hint
statistics to review results and adjust the difficulty (remove/alter exercise) in order
to improve the learning curve. The system can also measure the time needed by
each person to complete a specific task.
FORMATIVE EVALUATION
A formative evaluation experiment was conducted involving 5 young learners of
English as a foreign language (1 male and 4 females, within the age range from 11
to 16, all studying for a first or advanced certificate in English, and all familiar with
PCs and mobile phones, but not with AmI environments). The experiment was
targeted to collect users’ opinion regarding:
• The desk itself.
• The overall idea of interactive student desk in the AmI class and how it can
assist learning.
• The usefulness of each application regarding the English course and in
particular the thesaurus, the multimedia application, the personal area for
assignments and homework delivery, the myVocabulary area, and the hints
and confirmations during exercises.
• The User Interface (UI) layout and aspects of the supported gesture
interaction.
The experiment took place individually for each learner. After a very brief
introduction, the learners were driven through a simplified scenario illustrating the
main aspects of desk and of the related applications.
During the execution of the experiment, the children were asked questions about
various aspects of the scenario, and notes were taken with all their comments.
Figure 4. The Personal area with
assignments and their delivery status
Figure 5. Gradual hints for solving fillthe-gap exercises
Figure 7. The Thesaurus application
Figure 6. An exercise in electronic form
After the completion of the scenario, they were asked to answer a questionnaire
composed of 17 questions. Of the questions, formulated in an informal style to
appeal the young learners, 15 used a Likert scale from 1 (sure!) to 5 (no way!). The
remaining two questions concerned listing aspects of the scenario that the children
particularly liked or disliked. The results of the questionnaire are depicted in figure
8, where lower scores correspond to positive and higher scores to negative answers.
Overall, the results are very positive, as all the children involved in the
experiment were very interested about the desk and its applications, and some
where enthusiastic about having such an artifact available in their classroom.
The preferred features of the desk were the personal area and the dictionary,
followed by the educational games, the dashboard, the hints and confirmations,
touch interaction, pointing at things and viewing info, and the electronic submission
of assignments. The features they disliked most were mainly the desk size and
color, and, with respect to the UI mockups, again the colors and the fonts.
The young learners appreciated the educational support which the desk aims to
provide, as well as the potential for better organization of work between the
classroom and the home environment and collaboration with teachers and other
learners.
User Questionnaire Results
5
4
3
2
2,40
2,40
1,80
1,73
2,00
2,60
2,40
2,00
1,60
1,60
1,80
t
La
yo
u
To
uc
h
Fe
el
&
Lo
ok
hi
nt
s
tin
g
at
io
ns
Co
nf
irm
G
ra
du
al
s'
po
in
ca
bu
la
ry
Im
ag
e
Vo
g
Di
ct
io
na
ry
oi
nt
in
W
or
ds
'P
Pr
ot
ot
yp
e
tu
de
At
ti
Da
sh
bo
ar
d
1,20
1
Figure 8. Questionnaire results
Some of the children also proposed to include a grammar application similar to
the thesaurus application, displaying grammar rules, verb tables, etc, related to the
task at hand. On the other hand, the young learners appeared to view the desk as a
‘trendy gadget’ and demonstrated to be very sensitive to aesthetic issues, asking for
the possibility of personalizing the desk color, selecting fonts, colors, background
images and avatar images. Regarding interaction, they appreciated the gesture-based
applications, but they also asked for more traditional interaction means such as the
keyboard.
CONCLUSIONS
The results of the conducted study confirmed that AmI technologies have the
potential to enhance the classroom learning experience and to be positively viewed
by young learners, provided that they are carefully designed and tested. Adolescents
seem to be aware of the opportunities offered by novel technologies and willing to
embrace them, but also face classroom technologies as fancy personal gadgets
which should be colorful, aesthetically pleasant, and fun. They also appear aware of
potential technological difficulties, as well as space and practical limitations in the
classroom environment. On the other hand, young learners easily grasp the potential
of novel interaction techniques, such as gestures, but do not ignore more traditional
means with which they are familiar already.
Currently, the prototype desk has been assembled and all the necessary software
building blocks have been installed. Fully functional applications are being
developed, along with the vision software required for supporting page and word
recognition on paper material. Following full implementation, a larger evaluation
experiment is being planned, involving also, besides children, teachers and parents.
On the other hand, new collaborative games scenarios are being elaborated in order
to fully exploit multi-touch interaction.
The designed enhanced desk is intended to constitute a significant part of a
complete AmI classroom environment which will be realized in an AmI Research
Facility currently under construction at ICS-FORTH.
Acknowledgements. This work is supported by the ICS-FORTH internal RTD
Programme 'Ambient Intelligence and Smart Environments'.
REFERENCES
1. Abrami P. C., Bernard R. M., Wade C. A., Schmid R. F., Borokhovski E.,
Tamim R., Surkes M., Lowerison G., Zhang D., Nicolaidou I., Newman S.,
Wozney L. & Peretiatkowicz, A. (2006), A Review of E-learning in Canada:
A Rough Sketch of the Evidence, Gaps and Promising Directions. http://cclcca.ca/NR/rdonlyres/FE77E704-D207-4511-8F74E3FFE9A75E7E/0/SFRElearningConcordiaApr06.pdf
2. Bates A. W. (2005), Emerging trends: convergence and specialization in
distance education, In: Technology, e-learning and distance education (pp. 69), Routledge Taylor & Francis Group (ISBN: 0-415-28437-6)
3. Brooks C., Greer J., Melis E., and Ullrich C. (2006), Combining ITS and
eLearning Technologies: Opportunities and Challenges, In: Proceedings of
the 8th International Conference on Intelligent Tutoring Systems (ITS '06),
pp. 278-287
4. Brusilovsky, P. 2004. KnowledgeTree: a distributed architecture for adaptive elearning. In Proceedings of the 13th international World Wide Web
Conference on Alternate Track Papers &Amp; Posters (New York, NY,
USA, May 19 - 21, 2004). WWW Alt. '04. ACM, New York, NY, 104-113.
5. Cook, D. J., Augusto, J. C., Jakkula, V. R. (2009). Ambient intelligence:
Technologies, applications, and opportunities. Pervasive and Mobile
Computing, Volume 5, Issue 4, August 2009, Pages 277-298.
6. Cook, D. J., Das, S.K. (2007). How smart are our environments? An updated
look at the state of the art, Journal of Pervasive and Mobile Computing 3 (2)
(2007), pp. 53---73.
7. Cooperstock, J. (2001): Classroom of the Future: Enhancing Education through
Augmented Reality. In: Proc. Conf. Human-Computer Interaction (HCI Int’l
2001), pp. 688---692. Lawrence Erlbaum Assoc, Mahwah
8. Druin, A. (2002). The Role of Children in the Design of New Technology.
Behaviour and Information Technology, 21(1) 1-25.
9. Emiliani, P.L., Stephanidis, C. (2005). Universal Access to Ambient
Intelligence Environments: Opportunities and Challenges for People with
Disabilities. IBM Systems Journal, Special Issue on Accessibility, 44 (3),
605-619.
10. Grammenos, D., Zabulis, X., Argyros, A.A., Stephanidis, C. (2009). FORTHICS internal RTD Programme 'Ambient Intelligence and Smart
Environments, In the Proceedings of the 3rd European Conference on
Ambient Intelligence, Salzburg, Austria, November 18-21, 2009.
11. Gros B. (2007) Digital games in education: the design of game-based learning
environments. Journal of Research on Technology in Education 40, 23---38.
12. Heift, T., Schultze, M. (2007). Errors and intelligence in computer-assisted
language learning: parsers and pedagogues. Routledge, Taylor&Francis, NY.
13. IST Advisory Group (2003). Ambient Intelligence: from vision to reality.
Electronically
available
at:
tp://ftp.cordis.lu/pub/ist/docs/istagist2003_consolidated_report.pdf
14. Michel, D., Argyros, A. A., Grammenos, D., Zabulis, X., Sarmis, T. (2009).
Building a multi-touch display based on computer vision techniques. In
Proceedings of the IAPR Conference on Machine Vision and Applications
(MVA’09), pp. 74-77, Keio University, Japan, May 20-22, 2009.
15. Nousiainen, T., Kankaanranta, M. (2008). Exploring Children’s Requirements
for Game-Based Learning Environments. Advances in Human-Computer
Interaction. Volume 2008, Article ID 284056, 7 pages.
16. Quintana, C., Soloway, E., Norris, C. (2001). Learner-Centered Design:
Developing Software That Scaffolds Learning,
Advanced Learning
Technologies, Second IEEE International Conference on Advanced Learning
Technologies (ICALT'01), 2001, pp. 0499.
17. Robinson, J., A. Robertson, C. (2001). The LivePaper system: augmenting
paper on an enhanced tabletop, Computers & Graphics, Volume 25, Issue 5,
October 2001, Pages 731-743.
18. Shi, Y., Xie, E.A. (2003): The smart classroom: Merging technologies for
seamless tele-education. IEEE Pervasive Computing Magazine.
19. Xu P., Han G., Li W., Wu Z., Zhou M. (2009). Towards Intelligent Interaction
in Classroom, In: Universal Access in Human-Computer Interaction.
Applications and Services (pp 150-156), Berlin: Springer.