u-Annotate: An Application for User-Driven Freeform Digital Ink Annotation
of E-Learning Content
Mohamed Amine Chatti1, Tim Sodhi1, Marcus Specht2, Ralf Klamma1, Roland Klemke1
1
RWTH Aachen University, Informatik V (Information Systems),
{chatti,sodhi,klamma,klemke}@i5.informatik.rwth-aachen.de
2
Open University Netherlands, [email protected]
Abstract
User-driven annotation of learning content allows
the user to interact with it in a flexible and an intuitive
manner and hence personalize the content. This
interaction, in most cases, follows the “paper and
pen” note-taking paradigm. Though not without its
advantages, this paradigm does not scale when the
courses are delivered online as web pages. These
paper notes are not synchronous with the content,
hence may seem out of context, they may be misplaced,
and don’t make allowances for collaborative learning.
In this paper we describe u-Annotate, a user-driven
freeform digital ink annotation application for web eLearning content which aims at facilitating the learner
to annotate the online content with the aid of pen
computing devices such as graphic tablets, etc.
Learners can freely mark up the content, save the
annotations for recall at a later date, as well as share
these with other learners.
1. Introduction
Most e-Learning content today is delivered via
learning management systems that rarely allow the
learner to integrate her interaction with content into a
personal portfolio and store personal annotations with
the content for later reference or recall. On the other
hand current research approaches stress the importance
of personal annotations and structuring of information
according to personal episodes in the sense of episodic
memory [1]. Freeform annotation of learning materials
forms a crucial part of every learner’s life. When it
comes to annotations, it is found that there is a great
variety in form and application – learners use
highlighters, pens, pencils to write on the margins,
make symbolic notations, draw on and over text, write
between the lines, underline, circle, box, and highlight
all kinds of elements of documents [2]. Despite the
growing popularity of e-Learning in the recent years,
studies have shown that when it comes to reading and
annotating the content people still turn to pen and
paper [3]. This is reasonable, as pen and paper offer
many advantages. One key advantage is the relative
ease with which the reader may sketch unstructured
notes and drawings with respect to document content.
However, this “pen-and-paper” approach is not
without its limitations. The notes are not synchronized
with the content, and when consulted exclusively,
appear out of context and disconnected. Sharing such
notes and annotations with other learners is possible
when the learners are present in the same geographical
location, which is seldom the case in e-Learning.
Owing to the recent advances in pen-computing device
technology, one can envision the use of the same in the
e-Learning scenario, to completely replace the
dependence of the users on paper and pen for their
annotating needs.
The paper presents the concepts and ideas behind
the freeform digital ink annotation of e-Learning
content delivered as web pages. The authors focus on
the need and importance of annotations and how this
ties in with the broader picture of personalizing the
learning content, thus delivering effective learning
pedagogies. These concepts have been applied in the
design of u-Annotate (user-driven Annotations) – an
application which has been developed at Aachen
University of Technology, bureau42 GmbH, Germany,
with active support from the Open University at
Heerlen, Netherlands. u-Annotate incorporates various
technologies in order to create an application for
annotation of e-Learning content, allowing the learner
to actively mark-up the content, uniquely store, recall
the annotations as well as export the same for sharing
with other learners. The emphasis is on replacing the
paper-and-pen paradigm of note taking with the pencomputing approach, allowing the learner to treat the
digital content like a paper document and interact with
it much in the same way. The application may be used
in various scenarios, ranging from: annotation by the
learner, to, adding additional information to the
generated content by the content developers. The final
deliverable once embedded in the HTML source,
allows the annotation functionality to be used across
browsers and platforms.
2. Annotation Approaches
It is possible to make a clear demarcation between
the different approaches to the annotation of content.
These are the metadata-driven and the mark-up
annotation
approaches.
By
metadata-driven
annotations one refers to the annotation of learning
objects using metadata (e.g. LOM). These endeavor to
provide a description of semantics, vocabulary and
extensions of the e-Learning content. The end learner
has little, to no control over the annotations made.
Decidedly different from this approach are the markup annotations. These rely on the end user(s) to make
her own customizations to the static learning content.
These range from, and are not restricted to: drawing
freehand in the marginalia, embellishing the text and
figures with special mark-ups like circles, underlines,
text highlights and the like. u-Annotate broadly falls
under the latter category.
Annotations can be used to personalize the learning
content on the learner’s as well as on the pedagogical
level. In our context, by personalization we mean the
ability on the part of the learner to interact with the
content, the way she deems fit. Annotations may be
employed by the learners and content developers for
three tasks: 1) the author of a document is able to
enrich the document with additional information.
Analogous to footnotes, such annotations are called
Extensive Annotations [4]. 2) Annotations can be used
by the learner to extend teaching material with her own
thoughts. This promotes knowledge construction and
assists in the cognitive domain of educational activities
as identified by the Bloom’s Taxonomy [5]. Called
Intensive annotations [4], they do not bring new
information to the document, but enable the annotator
to extract information to synthesize, classify, organize,
structure etc. 3) Annotations can also be used as a way
of communication embedded in documents. This can
be used to promote collaboration with other learners or
members in social networks.
3. Related work
So far, there have been attempts to address the
issue of freeform annotations of web content. The
same have met with limited success owing to a number
of reasons. We briefly touch upon the various solutions
aimed at learner or end-user driven annotation of
online content.
YAWAS was a prototype annotation system written
in Java and JavaScript developed at Syscom,
University of Savoy, France [6]. YAWAS was
designed so that the users can highlight and annotate
specific texts in a document. It however, fell short in a
number of areas. The types of annotations are limited.
It is only possible to underline and highlight the web
content and that too, through browser context menus.
Freeform digital ink annotations are not supported. The
Avaya Prototype for Ink annotations on web
documents aimed at developing a preliminary
prototype to capture, render and understand ink data on
web documents [7]. Though the Avaya Annotations
prototype suffers from a number of limitations, its
support for freeform digital ink annotations as well as
for storage of annotation data in a native XML schema
is noteworthy. These limitations include the Reliance
on ActiveX which severely restricts it’s usability in an
e-Learning scenario as different learners may have
different choices of browsers. Probably one of the best
examples of web annotations is the commercially
available tool, iMarkup [8]. iMarkup offers a robust
annotation engine along with a choice of annotation
tools like dynamic text boxes, paint brushes and the
like. This approach, though revolutionary is not
without its limitations. 1) Reliance on the ActiveX
plug-in severely restricts the choice of browsers in the
learning scenario. 2) Short of unloading the plug-in
itself, there is no way to unload the annotations on the
webpage.
Existing solutions for web page annotations and
mark-ups suffer from a number of drawbacks which
limit their application to e-Learning. To our
knowledge, with u-Annotate, we have for the first time
proposed a solution exclusively for the e-Learning
arena with an eye on allowing the learners to
personalize the content as well as share the annotations
with other learners.
4. u-Annotate: Requirements,
implementation and integration
design,
u-Annotate endeavors to completely replace the
paper-and-pen paradigm for the annotating needs.
With the aid of alternate input devices, the web page
can be thought of as a digital canvas where the user
can enter her notes in the marginalia, and draw in a
freeform way with a myriad of tools, and enter text via
a keyboard into special text boxes.
4.1. u-Annotate: Requirements
Having analyzed the requirements specified by
learners; the existing format and structure of web
based courses; the limitations of the client machine, as
well as the current systems and solutions in the market,
the requirements from a mark-up annotations
application can be determined. For the front-end, the
main requirement is that the drawing be on a
transparent overlay canvas, such that the source
doesn’t get altered. The user should also have a myriad
of tools available to her for the purposes of annotation
along with the choice to alter the size and the color of
the tool. Since the application presents an overlay
layer, the original content might get hidden if the user
makes many annotations etc. It should thus be possible
for the user to hide the annotations from the page she
is currently viewing. Cross-browser compatibility is
also an important requirement. The application should
run equally well on all major browsers.
Moreover, the application should rely on a robust
backend support for powering the storage and
subsequent retrieval of the annotations. For the
purpose of storage two mechanisms for the saving of
annotation information have been envisioned. The
first, meant for quick retrieval and rendering of the
annotation data, is specific to the current page being
viewed and annotated, as well as the current user
logged on to that machine and course. This method
involves storing the data as a file on the local disk. The
second, meant for export and exchange of annotation
information, should be in an exchangeable format.
Furthermore, since the annotations made on the page
make sense only in the learning context of the content
they appear with [3], a very important requirement
from the application is that it stores and retrieves the
annotations that the user has made back into the same
context of the learning elements.
4.2. Development Tools
Flash 8 with ActionScript 2.0 was chosen to be the
development tool of choice for the provision of the
front-end and rendering/retrieval based on criteria
which were a) Support for overlays: SWF files can be
transparently overlaid on top of the HTML content.
This is the ability to be transparently overlaid in a
separate DIV layer with Z Index greater than that of
the underlying content. b) Browser Independence: The
Flash program once embedded in a web page, can be
run on any browser that supports the Macromedia
Flash 8 player plug-in. c) Performance: The suitability
of the application depends in great part on the basis of
how well it performs i.e. speed of rendering, the
amount of CPU stress it entails etc. The Flash
application performs quite well on these tests.
Key considerations for the choice of backend
solutions are based on our supposition that the user’s
computer is not part of a local network and that there
are no existing database solutions (MySQL, Oracle
etc) preinstalled on the user’s machine. For uAnnotate, we have used Flash local Shared Objects
and XML file storage. Flash local Shared Objects
allow the local storage of potentially large amounts of
data (if the user allows) quickly and very efficiently,
allowing for quick retrieval and rendering of the
annotation data. The intent of this project is to use
XML as a repository for the annotation information
that is generated when the user chooses to export her
mark-up annotations made on the web page to an XML
file which can then be sent to, and imported by another
learner or agent in a pedagogical role, thus supporting
collaborative learning in social networks.
4.3. u-Annotate: Design and Implementation
In the ensuing subsections, we will describe uAnnotate with an eye on the architectural and
implementation details. The system design will be
followed by description of the user interface and the
underlying functionalities. We conclude this section
with suggestions on how to integrate u-Annotate with
the e-Learning content.
4.3.1. Architecture. Figure 1 illustrates the interaction
between the various modules of u-Annotate along with
the flow of information/data. The user interacts with
the Web browser, which displays the e-Learning
content. The e-Learning content presents a transparent
overlay layer (implemented in Macromedia Flash) to
the user on which she can input her annotations with
the aid of different input devices. The annotation
information is detected on the overlay layer and with
the elements in the HTML DOM tree, these mark-ups
are in the scope of, is passed to the ActionScript
Annotation API by JavaScript functions.
The smoothened curve is drawn on the overlaid
layer by the Rendering/ Smoothening module, which
computes the control points and renders the
smoothened
curve
instantaneously.
The
Capture/Retrieve Point Data module is passed relevant
annotation information from the API and the
Rendering component, which consists of: a) the URL
of the page being annotated, b) the current logged in
user, c) the inked points, d) the range of the
annotations with respect to the HTML DOM structure,
e) time stamps i.e. the time of annotation, f) the color
and size of the tool used and g) the size, location and
the text information entered in the text boxes.
This module interacts with the Serialize Point Data
or the serializing component which formats the
annotation information and makes it conducive to be
saved on the local storage or for export via XML files.
Flash local Shared Objects are used for the day-to-day
storage meant for instant retrieval and saving, which
does not require any input from the user. On
navigation back to the same page, the Shared Objects
are read and the annotation information is retrieved.
Shared Object file. The serializing component parses
through the file, recognizing the annotation data which
is indexed by the URL and time of the annotations. In
case the annotations for the specified URL for the
current user exist, they are read back by the serializing
component, and the annotation information is sent
back
to
the
upper
modules.
The
Rendering/Smoothening module redraws these
annotations on the screen. If, however, the HTML
document structure has changed since the last time the
user annotated the content, the Annotations API
prevents the redrawing of the annotations. This
information is subsequently removed from the local
Shared Object. The retrieval complete, the user is now
free to add to those annotations, or erase the same from
the overlay.
4.3.2. u-Annotate: UI. This section illustrates uAnnotate in action. The course being annotated has
been created with bureau42’s e-Learning authoring
tool: author42™. The toolbox for the selection of the
various annotation tools is shown in figure 2. The user
can make her selection from the Freeform Drawing
and the Text Box tools as well as choose to customize
the stroke color from amongst 216 web safe colors
from the palette. The size of the stroke can also be
altered as shown.
Figure 2. u-Annotate toolbox
Figure 1. u-Annotate architecture
On the other hand, the second storage scenario deals
with the export of the annotation information for
exchange, or for non-volatile storage. This is done via
the generation of XML formatted annotation data.
On retrieval, when the user chooses to display the
annotations she had made in a previous session. The
Annotations API passes the request to the
Capture/Retrieve Point Data module, which in turn
passes the request along with the URL of the page and
the current logged in user ID to the serializing
component. This module in turn reads the Flash local
Figure 3 illustrates the tools in action. The
Freeform Drawing tool is selected and the drawing
mode is activated. The user in this mode can mark-up
the e-Learning content much in the way shown in the
figure, where the user has added handwritten notes in
the space below the content, and has drawn arrows for
emphasis. Other than the Freeform drawing tool, uAnnotate features the dynamic addition of a text box,
where the user can add notes of her own, with the aid
of a keyboard. Selecting the tool, and double clicking
on the digital canvas opens up a text box as shown.
Annotations drawn can be selected for deletion or
dragged to a new position on the overlay drawing
canvas. Pressing the Select button the user can drag a
selection marquee and can either choose to delete the
same by pressing the Erase button, or move the
annotation to another location on the canvas by
dragging and dropping the selected annotations. A
prompt is displayed when the user presses the Clear
button, whereby all annotations (Freeform and
otherwise) added to the page are deleted. The Flash
local Shared Object storage may be invoked by
pressing the appropriate Save and Load buttons as
shown in figure 2. The Export and Import buttons
allow for the export/import of annotation information
as XML files. u-Annotate allows the user to explicitly
hide/unhide the annotations from view, by clicking on
a simple link (implemented in JavaScript).
her to store and recall the annotations, as well as
export the same for sharing with other users.
A natural extension of this implementation is the
concept
of
real-time
shareable
annotations.
Annotations can be used as a way of communication
embedded in documents and can be used to collaborate
on a document, which provides a natural way to
communicate ideas and feedback even if the learner(s)
are geographically distant. Students in a class can use
digital annotations as a way to discuss a paper, so that
all comments and mark-ups are visible and shared. The
application would then start to resemble a digital
whiteboard with local storage. The next step for uAnnotate could also be in the area of audio-visual
annotations. In addition to marking-up the content with
Freeform Digital Ink and text boxes, content could also
be annotated by the learner with her thoughts and
interpretations with the aid of a microphone and/or a
webcam. The A/V annotations could then be linked to
the content with anchors. A possible application of this
prototype could then be, in the domain of supporting
learners with special needs.
5. References
Figure 3. u-Annotate in action
4.3.3. u-Annotate: Integration. Since it has been our
aim to keep the size and number of the files, small, as
well as the mode of integration as easy as possible, uAnnotate consists of just two files: the main SWF
compiled Flash file and a JavaScript file. The SWF file
is dynamically linked to the HTML page by the
JavaScript functions which also interact with the DOM
API as well as implement the hiding and restoring of
the Flash overlay layer on user request. This dynamic
integration works across browsers and enables the
functionality immediately. The process can be
extrapolated for authoring tools, where the two files
can be integrated as the course is being generated by
the authoring tool. The inclusion of the JS file in the
course page ensures that the functionality extends to all
pages in the course.
5. Conclusion and outlook
In this paper we presented our application for userdriven freeform digital ink annotations of e-Learning
content – u-Annotate. The focus has been on
completely replacing the dependence of the user on
paper and pen for her note-taking needs. u-Annotate
allows the user, to actively annotate content, allowing
[1] M. Ringel, E. Cutrell, S. Dumais, E. Horvitz: Milestones
in Time: The Value of Landmarks in Retrieving Information
from Personal Stores, Interact 2003, Ninth IFIP Interact
2003, p. 184-191.
[2] C. Marshall: Toward an ecology of hypertext annotation,
Proceedings of the ninth ACM conference on Hypertext and
hypermedia: links, objects, time and space---structure in
hypermedia systems, May 1998
[3] D. Bargeron and T. Moscovich: Techniques for on-screen
shapes, text and handwriting: Reflowing digital ink
annotations, Proceedings of the SIGCHI conference on
Human factors in computing systems, April 2003
[4] S. Bringay, C. Barry and J. Charlet: Annotations: A new
type of document in the Electronic Health Record, Paper
presented DOCAM'04, University of California, Berkeley,
October 2004
[5] B. S. Bloom: Taxonomy of Educational Objectives,
Handbook I: The Cognitive Domain. New York: David
McKay Co Inc, 1956
[6] L. Denoue and L. Vignollet: An annotation tool for Web
browsers and its applications to information retrieval,
Recherche d’Information Assistée par Ordinateur, Paris,
France, 2000
[7] R. Kashi and S. Ramachandran: An Architecture for Ink
Annotations on Web Documents, Seventh International
Conference on Document Analysis and
(ICDAR'03) - Volume 1 p. 256, January 2003
[8]
iMarkup
Annotation
http://www.imarkup.com
Tool
Recognition
(Commercial):