Educational Technology & Society 3(2) 2000
ISSN 1436-4522
7HDFKLQJ)RUPDO/DQJXDJHVE\DQ,QWHOOLJHQW7XWRULQJ6\VWHP
Vladan Devedzic
Department of Information Systems
FON - School of Business Administration
University of Belgrade
POB 770, Jove Ilica 154, 11000 Belgrade, Yugoslavia
[email protected]
John Debenham
Key Centre for Advanced Computing Sciences
University of Technology, Sydney, PO Box 123
Broadway, NSW 2007, Australia
[email protected]
Dušan Popovic
"Mihailo Pupin" Institute
Automation Lab
Volgina 15, 11000 Belgrade, Yugoslavia
[email protected]
ABSTRACT
The paper describes design of the FLUTE system, an intelligent tutoring system in the domain of formal
languages and automata. The basic idea of the FLUTE system is a systematic introduction of students into
the system's domain, in accordance with both the logical structure of the domain and individual background
knowledge and learning capabilities of each student. Other intelligent tutoring systems in that domain are
not described in the open literature. The knowledge in the FLUTE system is represented starting from ITS
ontology that has been defined within a recently developed framework and a model of intelligent tutoring
systems, called GET-BITS. A brief overview of the model/framework is also included. The contents that
should be presented to the student during tutoring sessions are discussed and logical organization of such
contents within the system is described. The system is analyzed in the paper from the pedagogical point of
view. Every concept that a student has to learn during a session with FLUTE, the system illustrates by a
number of examples. This makes the tutoring process more dynamic and facilitates learning.
Keywords
Intelligent tutoring systems, Ontology, Formal language learning, Automata, GET-BITS
1.
Introduction
A major problem for many students of computer science in adopting and mastering the material taught in the
courses of formal languages and automata is a large number of abstract, complex, specific, and seemingly very
similar concepts they have to learn. This leads to potential monotony and difficulties in capturing the essence
and diversity of the concepts when the material is presented in the traditional way.
In order to make learning formal languages and automata more attractive for students and make the abstract
material more concrete, development of an intelligent tutoring system (ITS) called FLUTE (Formal Languages
and aUTomata Environment) has been started. The system design is based on the GET-BITS framework and
model of ITS (GEneric Tools for Building ITS) (Devedzic et al., 1999).
This paper presents the design of the FLUTE system and analyzes the system's functionality from the
pedagogical point of view. Section 2 defines the problem more precisely, and Section 3 briefly overviews the
GET-BITS model. Some discussion on why have we developed FLUTE is provided in Section 4. In Section 5
the architecture of the FLUTE system is described and it is shown how different parts of knowledge are
effectively represented and built in the knowledge base. Sections 6 through 8 present the analysis of the FLUTE
system, showing its advantages and limitations.
36
2.
Problem Statement
It is an ultimate goal of any ITS to make the learning process in the corresponding domain most effective.
Careful selection of the contents, kinds and structure of knowledge, information and data to be built into the ITS
is of an extreme importance in achieving that goal. In that sense, this paper describes:
1. the overall architecture of the FLUTE system from the knowledge and data organization perspective;
2. the structure of domain knowledge built in the system; all the important contents the student has to learn are
specified, as well as possible paths that she/he can take in learning;
3. how the domain knowledge is related to the necessary pedagogical knowledge and the student models;
4. some details from FLUTE's cognitive support for the learner, as well as how the system tries to increase the
learner's interest for further study of related topics;
5. how the GET-BITS framework is used in designing the system from the ontology engineering perspective;
and
6. the role of numerous examples in illustrating each topic.
3.
Previous Work
Other similar ITSs in the domain of formal languages and automata are not reported in the open literature.
However, design of several other successful ITSs in different domains of computer science (Anderson et al.,
1990; Johnson, 1990; Matsuda et al., 1997), as well as of some other ITSs from related areas (Chee and Xu,
1997; Herzog, 1997; Katz and Lesgold, 1993; Kong, 1994; Wong et al., 1996) has helped in designing the
FLUTE system. In formulating the domain knowledge that should be built into the FLUTE system several
textbooks on formal languages and automata, such as (Hopcroft and Ullman, 1979) or (Revesz, 1983), were
consulted. The structures of some relevant university courses and the experience of several human instructors
teaching formal languages at the university level were also considered. Since one of the project ideas was that
the FLUTE system should be practically oriented, i.e. to represent knowledge that can be practically applied,
literature on compilers and compiler writing (Appel, 1998; Fischer, 1988; and Muchnick, 1997) has also been
used.
In developing FLUTE, we have used the GET-BITS framework (Devedzic et al., 1999). The heart of the
framework is our view of the ITS ontology as a layered, hierarchical scheme of composition of constituent parts
- components and agents - of any ITS, shown in Figure 1. All agents and components in an ITS are defined at
one of five levels of abstraction, and along several dimensions (such as knowledge representation, methods, and
inference techniques). Primitives are components like plain text, logical expressions, attributes and numerical
values. They are used to compose units like rules, frames, and different utility functions. These are then used as
parts of certain building blocks that exist in every ITS, e.g. topics, lessons and teaching strategies. At the system
level, we have self-contained systems or agents like explanation planners, student-modeling agents, and learning
actors, all composed using different building blocks. Finally, at the integration level there are collaborative
learning systems, distributed learning environments, and Web-based tutoring systems.
Level of
abstraction
Level of
abstraction
Objective
Level 1
Integration
Multiple agents or systems
Level 1
Level 2
System
Single agent or system
Level 2
Level 3
Blocks
System building blocks
Level 3
Level 4
Units
Units of blocks
Level 4
Level 5
Primitives
Parts of units
Level 5
Semantics
(a)
Dimensions
D1 D2 . . . Dn
(b)
Figure 1. Layers and dimensions of the ITS ontology in the GET-BITS approach
37
The integrated concept of the ITS ontology is split in GET-BITS into a number of separate but interrelated
ontologies for all constituent parts of an ITS, such as System ontology, Domain knowledge ontology,
Pedagogical knowledge ontology, Student model ontology, and Interface ontology. These ontologies are defined
in small composable modules at each level of abstraction and using the top-down approach, so that the
knowledge they need can be assembled. All ontologies have the same starting point (the root concept) defined as
an abstract knowledge element (Devedzic and Radovic, 1999). It reflects the possibility of representing all data,
information and knowledge in an intelligent system in a unified way (Debenham, 1998). In this way, each
ontology assigned to a higher level of abstraction includes the taxonomies of the lower levels, and forms a set of
inclusion lattices of ontologies.
4.
Motivation - Why Teaching Formal Languages by an ITS?
While studying formal languages is essential in theoretical computer science, experience of several human
instructors shows that teaching such courses to students of computer engineering often causes some practical
problems. Such students typically want "concrete things", frequently get bored when they "cannot examine the
material by the keyboard and the mouse", and usually feel hard to master the subject because "everything looks
alike". Of course, these phrases may also reveal deficiencies in the instructor's capability to transfer the relevant
knowledge in an appropriate way. However, since students of computer engineering are generally practically
oriented, it is usually desirable for them to have some software to "play with" and simultaneously learn about
formal languages and their importance in practical software engineering.
We have also found that the students' motivation for learning about formal languages increases if they know at
any moment where exactly they can apply the knowledge they acquire. Their interest increases if somebody
always tells them in what other disciplines grammars, parsers and automata are important. They get a better
feeling for such concepts if they are illustrated in compiler design, or even in more advanced domains, such as
designing adaptive screen forms (Valente et al., 1999) or development of ontologies (Lopez et al., 1999). On the
other hand, it is not always easy to cover all such practical aspects in teaching a classical course on formal
languages.
These observations have led us to the idea of developing FLUTE. The starting points in our conceptualization of
the FLUTE system have been to develop an ITS that will help the students of computer engineering study the
field of formal languages individually, help them learn efficiently and dynamically, and also help them in
starting to explore related fields themselves.
5.
Architecture of the FLUTE System
The architecture of the FLUTE system is shown in Figure 2. Note that the figure only stresses the details relevant
for this paper, i.e. the nature and kinds of knowledge and data in the system. It doesn't show explicitly the active
components of the modules (things like planners, inference mechanisms, example generators, exercise
generators, etc.) and submodules of the individual modules. Student administration module, also omitted from
Figure 2, maintains a database of students (users of the FLUTE system), records the current status of their
knowledge and progress, handles student registration, and can be accessed directly through the user interface
module.
5.1.
Expert Module
The Expert module contains domain knowledge and is organized according to the GET-BITS' Domain
knowledge ontology. Figure 2 shows three most relevant parts of domain knowledge used in the FLUTE system.
The first part contains all the concepts, topics, facts, procedural knowledge and domain heuristics the student has
to learn. A meaningful subset of such items can be logically organized as a lesson. Items in a lesson are closely
coupled but they can refer to items in other lessons. Some important attributes of each lesson are sets of
objectives and goals, sets of topics, concepts, facts, theorems, etc. taught in that lesson, a set of the
corresponding teaching rules, and a set of associated problems (tests, questions and exercises). An example is
shown in Table 1 and Figure 3. Note that rules, problems and examples can be associated with both level 3 and
level 4 items in Figure 1.
38
Expert module
Concepts, facts,
and heuristics
Pedagogical
structure
Examples
Explanation
module
Student
model
Pedagogical
module
User
interface
Student
Figure 2. Architecture of the FLUTE system
A major difficulty in mastering the basics of formal languages and automata is the domain's inherent
abstractness. The best way to overcome it is to use a number of examples to illustrate all the concepts, principles
and theory of the domain. Hence an essential component of domain knowledge is a database of examples (see
Figure 2), which are systematically associated with every topic, concept, etc. that a student has to learn during a
session. Presenting examples to the student is under the control of the system's pedagogical module. This makes
the tutoring process more dynamic and facilitates learning.
It should be noted in Figure 2 that the pedagogical structure of the domain is considered a part of the domain
knowledge rather than a part of the pedagogical module. This idea is not new, and has been simply transferred to
the GET-BITS' Domain knowledge ontology from other successful ITSs (e.g., see Vassileva, 1990). It makes
possible to separate all of the domain-dependent knowledge, stored within the expert module, from the other
kinds of knowledge (i.e., from domain-independent pedagogical and explanation strategies and student model).
Pedagogical structure of the domain is defined in FLUTE as a set of directed graphs showing explicitly
precedence relationships of knowledge units within each lesson and among the topics of different lessons.
Level
1
2
3
4
Item
Lesson
Objective;
Goal
Topic
Concept;
Fact;
Theorem
Example
Derivation trees
Redundancy of grammars;
Obtaining reduced grammar
Redundant grammar
Inactive (dead) symbol;
Redundant grammar contains unnecessary nonterminal symbols
Pumping lemma
Table 1. Hierarchy of items in a lesson
39
Rule D98:
Problem P112:
Example E123:
If a context-free grammar is reduced,
Then it contains no inactive symbols.
Find a reduced grammar that is
equivalent to the context-free grammar ~E123
G = ({S,A,B,C}, {a,b,c}, S, F), F = {...}
Figure 3. A rule, a problem, and an example grammar
Such a dependency graph, defining the entire domain of the FLUTE system, is shown in Figure 4. In fact, that
graph represents the top-level structure of the domain-specific ontology of formal languages and automata. It
defines subjects of learning and possible learning paths in the domain. Wherever there is a fork in the path, the
learning process can go either way. Wherever there is a join of two paths, the learning process cannot be
continued unless the student has proved to master the knowledge along both paths (this is checked by the
pedagogical module and is recorded in the student model). There are four learning levels: preliminaries, basic
level, senior level, and advanced level. They must be passed in strict sequence. One of the parameters of the
student model is the learning level, reflecting the student's background knowledge. Pedagogical module
periodically and automatically tests the student to check her/his mastery of domain knowledge at the associated
learning level.
A set of lessons is associated with every subject in Figure 4. For example, the subject of context-free grammars
(CFG) is taught in the following set of lessons: derivation trees, normal forms, linear CFG, regular expressions
and special types of CFG. Lessons of such a set are also treated as nodes of a dependency graph, hence
precedence relationships for teaching lessons related to a subject are specified explicitly within the ontology of
formal languages and automata.
5.2.
Student Model
Student model in FLUTE is derived from components that make up the GET-BITS' Student model ontology. The
components reflect:
• the state of the student's current knowledge (a combination of one of the four learning levels and one or more
"points" on a path (paths) at that level);
• the student's current progress in learning a particular lesson ("point(s)" in the lesson's dependency graph);
• timing and frequency of trying various exercises and asking for hints, explanations, and help;
• the student's performance in answering the system's questions and solving problems (grades and timing, as
well as the ratio of correct and incorrect answers); an important part of it is a degree of remembering "old"
lessons (answers to questions from the lessons learned during previous sessions);
• the student's learning behavior (frequencies of following correct/incorrect paths in learning a topic or a
concept);
• measures of how continuously and how frequently the student learns from the system;
• measures of the student's initiative in working with the system (e.g., how many times the student has tried the
restricted option of including a new example in the knowledge base, or how often she/he consults the links to
related topics).
The values of the attributes are calculated from applying a set of rules and simple functions from the pedagogical
module to a set of parameters that the system acquires and updates automatically during each session. Some of
the parameters are indicated in parentheses in the above descriptions of the attributes. All of these activities are
performed by the student-modeling agent, which is a part of the Student model ontology. The student model is
saved after each session and is used in the beginning of the next session. If a student is having a session with
FLUTE for the first time, the student-modeling agent gives her/him a set of exercises and tests before the
tutoring process starts. In this way the student-modeling agent estimates the new student's background
knowledge and determines initial values of the attributes of the student model. The knowledge that drives the
operation of the student-modeling agent in updating the student model is encoded in the student model rules.
Although it is the task of the student-modeling agent to update and maintain the student model, users with
special privileges (human instructors) can also modify the student model themselves.
40
Mathematical
preliminaries
The concept of
language
Finite
automata
Regular
sets
Pushdown
automata
Grammars
CFG
CSG
Preliminaries
Relation between
grammars and
automata
Basic level
Closure properties of
families of languages
Turing
machines
DCFL
Decidability
Decision properties
of DCFL
Computational
complexity theory
Syntax analysis
Gramatical
transformations
Senior level
Advanced level
Other important
language classes
Figure 4. Pedagogical structure of the domain
(CFG - Context-Free Grammars; CSG - Context-Sensitive Grammars; DCFL - Deterministic Context-Free
Lang.)
5.3.
Pedagogical Module
FLUTE always operates in one of the following three modes of operation: teaching, examination, and consulting.
It is actually the pedagogical module that runs one of the corresponding pedagogical agents at any time, thus
causing FLUTE to operate in one of these three modes.
When the teaching mode agent runs, FLUTE presents a lesson to the student. In the beginning, the agent
executes a dedicated rule-based planner (called the lesson presentation planner) and develops an initial plan of
teaching actions. The plan is based on the contents of the lesson being presented, the relevant dependency graphs
and the student model. The plan actually represents a detailed outline of the lesson presentation. The strategy of
Raymond and Pilon (Raymond and Pilon, 1994) is used for guiding the plan generation: each objective of the
lesson being presented is decomposed into a number of transparencies, each one of them roughly corresponding
to one concept, fact, procedure or theorem. Each transparency is set up of several pedagogical points, covering
the item contained in that transparency. Starting from the initial plan, the teaching agent runs the lesson
presentation planner again and generates sub-plans for presenting the lesson's objectives, topics, concepts, etc.
Interaction with the student in teaching mode is provided through a set of buttons and dialog boxes. Depending
on the student's reactions, the student-modeling agent regularly updates the student model (attributes like current
progress, learning behavior, average speed of mastering lessons, etc.). The teaching agent communicates with the
student-modeling agent and can change the sub-plans and possibly the entire lesson presentation plan if
necessary. The two agents communicate easily by sharing their knowledge through the GET-BITS' Pedagogical
knowledge ontology and Student model ontology.
In the examination mode, FLUTE's examination agent can generate problems and exercises for the student, offer
hints and suggestions for problem solving, and conduct a more complex assessment of her/his knowledge
through a systematic series of tests and examinations. After each examination session, no matter how complex it
was, the examination agent sends request to the student-modeling agent to update the relevant attributes of the
student model (grades, answering speed, current progress, etc.). Then it generates a plan of remedial actions and
presents it to the student in the form of suggestions for further sessions and learning. All of the questions,
problems, and exercises are designed as parts of examples in the corresponding database of examples, which is a
part of the expert module. The pedagogical module differs between illustrative examples (used mostly by
teaching and consulting agents) and test examples, and can store and retrieve past exams and tests on student's
request.
The consulting mode agent is the one that answers the student's questions, explains domain concepts and
heuristics, repeats required parts of any lesson corresponding to the student's current knowledge (as recorded in
the student model), refines problem solutions, provides hints for further learning, etc. In this mode, the
pedagogical module frequently transfers its control to the explanation module. The consulting mode is entered
41
strictly on request. The consulting agent can send the student-modeling agent a request to alter the values of
some attributes of the student model in this mode as well, but the timing functions are turned off. The student's
initiative is considered most relevant in this mode and is calculated by dedicated functions.
Under the control of a human instructor, the student can extend the example database by a new example.
Although there is no automatic recognition of the type, quality and relevance of the newly included example, this
is a very useful way of extending the system's knowledge during the sessions themselves. Also, all three
pedagogical agents often refer the student to related topics to explore them individually.
The core of knowledge that pedagogical agents rely on is the Pedagogical knowledge ontology, which defines
the following sets of rules:
•
•
•
•
•
lesson presentation rules (rules of the lesson presentation planner)
remedial rules (rules of the remedial actions planner)
diagnostic rules (for evaluating the student's knowledge and understanding of topics, concepts, etc.)
didactic rules (defining the presentation's "look-and-feel" such as pointing, highlighting, underlining,...)
control rules (a kind of meta rules for the overall control of the pedagogical module operation)
Figure 5 shows an example of a diagnostic rule (PD8) and of a control rule (PC6):
Rule PD8:
Rule PC6:
If test-score > 55 then set passed-test.
If passed-test then calculate grade.
Figure 5. Examples of pedagogical rules
5.4.
Explanation Module
FLUTE's explanation module is a user-oriented, rule-based agent, using all the available information from the
domain knowledge base (contents of lessons, objectives, topics, dependency graphs and examples), as well as
from the student model, in order to answer the student's questions and provide desired explanations. Specifically,
it is designed to accept a request for explanation from the student and to do the following in response:
• determine the contents of the answer/explanation;
• decide upon the explanation presentation style (level of details, hints, augmentation material to be included,
e.g., illustrations, examples, pointers to related concepts and references, etc., depending on the question itself
and on the values of relevant parameters of the student model such as current knowledge, current progress,
performance);
• select the knowledge model that provides the easiest explanation in cases when multiple knowledge models
can be used (e.g., select what formalism to use in the explanation if something can be explained both by CFG
and automata);
• compose the explanation and order its statements in a coherent, reasonable and comprehensible way.
The questions the student can ask are selected from a context-sensitive menu (Why, How, What If, Related
Topic, Illustration, and More Explanation).
The explanation module is described in more details in (Devedzic et al., 1999).
5.5.
User Interface
The user interface of the FLUTE system has been developed after the GET-BITS Interface ontology. It provides
knowledge browsers and navigation tools, showing dependency graphs, lesson and course outlines,
previous/next/arbitrary objective, transparency or pedagogical point, etc., and parameters of the student model
(current knowledge and current progress such as "positions" within the corresponding dependency graphs,
performance, learning behavior,...).
Students are allowed to use only a subset of the system's options. The user interface shows the other, restricted
subset of options only to privileged users (human instructors). Restricted options allow the user to modify and
update the contents of the knowledge base and (re)structure lessons through a set of specialized knowledge
42
editors and browsers, thus enabling a rudimentary knowledge acquisition. They also allow for manual
modifications of the student models.
We also plan to provide several other general-purpose tools of the user interface (notepad for student's notes,
sound and light signals, color, markers, intensity, zooming, and other visual indicators, timer, context-sensitive
on-line help, etc.).
6.
Implementation and Examples
The FLUTE system has been implemented as a Windows 98 application, using MS Visual C++. It is designed as
a single-user system, i.e. as an individual learning tool. At the moment we do not plan to extend it to operate as a
collaborative learning tool or a Web-based ITS.
A session with FLUTE starts with the student's identification, which is followed by showing the student a brief
summary of the part of the curriculum that she/he has supposedly mastered so far. The student can also see the
values of relevant parameters of the student model that has been saved after the previous session has been
completed. Then she/he is supposed to select one of the three modes of operation (teaching, examination, or
consulting).
Let's assume that the student has selected the teaching mode. FLUTE allows the student to go again through the
lessons that she/he has already learned, or to select some new material to learn about. In the later case, the
material that the student can select is restricted according to the domain-specific ontology (see the dependency
graph in Figure 4). An example of such a selection is shown in Figure 6. The student is supposed to select a field
supported by the contents of the curriculum and a set of possible topics corresponding to that field. Every such a
selection reflects a fork in the dependency graph at a certain level of details. In this example, the student has
selected to learn a new lesson about grammars of formal languages, and has expressed the wish to learn more
about the classes of grammars. In response, FLUTE consults the curriculum and the student model again and
suggests a lesson to learn next. The student can review the lesson presentation plan if she/he wants to see its
details prior to starting to learn. In some cases, the student may ask FLUTE to suggest an alternative lesson if
she/he concludes that the current one doesn't suit their current learning goals.
Figure 6. Selecting the field (contents) and the corresponding topics to learn about
When the student starts learning a lesson, its presentation is done according to the lesson presentation plan. At
fixed points in the plan, the student can switch to the examination or consulting modes. If, for example, the
student learns about the elements of CFGs, FLUTE may generate an exercise as in Figure 7. In the exercise,
43
FLUTE shows an example of a CFG and allows the student to perform some limited editing of the grammar's
rules. The student can then try to answer some questions about the properties of the resulting grammar.
Figure 7. An exercise about grammar rules
Figure 8 shows an example of how the student might answer a FLUTE's question in the examination mode of
operation. The question is about the elements of the definition of automata. The student is supposed to assemble
the correct list of basic elements of the definition of automata, starting from a larger list that contains redundant
elements as well.
Figure 8. Answering a question about the definition of automata
44
Finally, Figure 9 shows an example of using FLUTE in consulting mode. In the example, the student is supposed
to have come to a point (within a lesson) at which it is possible to demand explanations from the system. The
explanation she/he requires is of type "Illustration", and it has to clarify the process of translating a grammar's
derivation rules. Note that the process of explanation is highly interactive: the student can "discuss" the
explanation with FLUTE by editing some parts of the illustration, asking the system to solve the problem again,
and the like.
Figure 9. Translating derivation rules (consulting mode)
7.
Assumptions and Constraints
Once a student reaches a certain knowledge/learning level and the system assigns her/him certain appropriate
knowledge and progress indicators, it is assumed that the students knows everything (i.e., she/he can apply every
domain-dependent rule) before the corresponding point on the relevant dependency graphs. It depends on the
pedagogical agents to decide when and how to check the student's current knowledge and whether to return
her/him to a previous point or learning level if her/his performance is not satisfactory.
FLUTE is neither multimedia system nor a hypertext system, although in the analysis below it is compared to
some multimedia and hypertext supported ITS.
Only the first two learning levels from Figure 4 are currently implemented (see Devedzic et al., 1999 for details
of the implementation). The rules (knowledge) of the pedagogical and explanation agents/modules are still
subject to change. To develop the whole system is an extremely complex task, and it should be noted that several
potentially useful add-ons were omitted from the current design for the sake of reduced complexity. They
include parsing and analysis of newly included examples, case-based reasoning, analogical reasoning, concept
generalization, and many more.
45
8.
Analysis and Evaluation
In this Section, FLUTE is analyzed from the pedagogical point of view and is informally compared to some
other ITSs that have been used as referent systems in designing it. FLUTE's advantages are clearly shown.
8.1.
Cognitive support and motivation
FLUTE's users are supposed to be adults - students of computer engineering who have their specific learning
goals. In designing FLUTE we have tried to pay maximum attention to that specific group of potential users.
Also, FLUTE does not provide support for collaborative learning. These two facts have guided our decisions on
what kind of cognitive support and motivation FLUTE should provide.
We have treated the motivation issue in a rather traditional and restrictive way, as a combination of factors like
the learner's curiosity, competition, challenge, control and confidence. See (Issroff and Del Soldato, 1996;
Keller, 1987; and Lepper et al., 1993) for more elaborated discussions on the effect these factors have on
learning. In brief, FLUTE always tries to create the learner's feelings of self-improvement and self-esteem in
learning. We also want the FLUTE's users to have a feeling of getting rewarded for "winning in competing with
the system". Whenever it is possible and appropriate, FLUTE lets the learners know there are more interesting
issues and possibilities of applying the knowledge they acquire from the system. FLUTE's pedagogical agents
combine the "Learner in control" and "System in control (Guidance)" approaches to instruction. Most users in
the target group prefer the "Learner in control" philosophy, but novice students always need some guidance.
During a learning session, FLUTE's pedagogical agents care about increasing/maintaining the learner's
confidence by occasionally showing her/his progress factor from the student model. For example, after a
successful completion of an exercise such as the one shown in Figure 7, FLUTE will occasionally (but not
always!) show an indicator of improvement, taken from the student model. This action is related to the
recognition that confidence is related to the perception of self-efficacy (Keller, 1987).
FLUTE is not a replacement for human instructor. It is a tool that helps students in learning individually about
formal languages and automata. It doesn't simply transfer all of the built-in knowledge to the student - the idea is
that it helps the student gradually and individually develop her/his knowledge and cognitive skills in that
domain.
The built-in knowledge is, of course, limited. Therefore selected knowledge items in the knowledge base
(lessons, topics, concepts, and examples) contain pointers to some references and related topics. These are shown
to the student either on request, or when the pedagogical agents decide it could be helpful, or when they exhaust
the other teaching possibilities. We have found such pointers to be extremely important for maintaining the
students' interest at an acceptable level. For example, after completing the lessons and a set of examples treating
CFGs, the student gets congratulations from FLUTE and can also see some additional material and references
showing how CFGs are used in compiler design, design of adaptive screen forms, and ontology design. This
way, practically oriented students get an immediate feeling of the purpose and benefits of learning such abstract
things as formal languages and automata. It is a state in their minds that is difficult to measure, but is easily
recognized and is often described by the students' words like: "Although much of the teaching material looks
alike, I am curious about what will be the next pointer." In fact, some students are waiting for such pointers as if
they were a reward.
In all three modes of operation, FLUTE's pedagogical agents occasionally perform some actions in order to
increase the learner's curiosity, attention and interest. These actions include encouraging the learner to try more
examples, showing useful pointers, explaining why something is good/useful to know, expressing disagreement
if the learner's actions show that she/he is heading in wrong direction, complimenting the learner's success in
doing exercises, and the like. Such actions have nothing to do with the teaching material itself. However, they
are very important for getting the learners motivated and increasing their receptiveness or alertness, both of
which are important parts of the system's functionality. They also increase the learners' curiosity and urge to ask
for additional explanations. Other authors have performed more extensive studies of such issues (Issroff and Del
Soldato, 1996; Mizoguchi et al., 1996; Murray, 1997). We have used their results in designing the FLUTE's
pedagogical agents. However, there is a shortcoming in our current design in this regard: we don't use emotional
pedagogical agents in the current implementation. There is evidence that such agents can greatly increase the
learner's interest, motivation and attention (Rist et al., 1997).
46
We have also used some other ideas from (Mizoguchi et al., 1996), such as the idea of developing the learner's
domain-dependent capabilities through gradually building her/his understanding of domain concepts, improving
her/his problem-solving capabilities in the domain, and enhancing her/his skills. For example, FLUTE's
examination mode agent is instructed to exercise the student by first asking her/him to solve easy problems,
including a typical problem-solving schema of the related topic (i.e., an "expected" problem). More difficult
problems include using a known schema, i.e. a schema that the student is supposed to have already used before.
FLUTE doesn't support modifying an existing problem-solving schema, combining several schemata, and
creating a new schema. In helping the student solve a problem, FLUTE can make analogies between known and
unknown topics in the domain, in order to increase curiosity and stimulate individual motivation.
In order to illustrate the cognitive support that FLUTE offers to the student by a practical example, consider
solving the problem represented in Figure 8. Let's assume that the question about the elements of the definition
of automata is a typical question to ask the student after she/he has completed a corresponding lesson in the
teaching mode. Before asking the student the question, the examination agent checks with the student-modeling
agent to ensure that the student's current knowledge is sufficient for answering such a question. In the beginning,
the student is let alone to try to solve the problem. She/He is allowed to ask for a hint, and the examination agent
will offer a hint if it notices that the student is making a mistake when selecting the elements of the definition of
automata from the window on the right-hand part of the screen.
Accepting the hint or asking for one enables many additional options which were disabled before the hint. We
have built these options into the FLUTE's examination agent starting from the suggestions from (Murray, 1997).
First, the student can suspend solving the problem and request to learn/review prior or prerequisite knowledge.
Also, the examination agent can decide out of its own initiative to relate the problem to the student's prior
knowledge, to remind the student what she/he already knows, to summarize/review prior knowledge, to test
prerequisite knowledge and remedy it if necessary. The student is allowed to take initiative, switch to the
consulting mode, ask questions, and ask for another similar problem in order to make analogies if possible.
FLUTE might decide to give an analogy, or show an example, or tell what is relevant or what to ignore. In the
example from Figure 8, suppose the student has already learned about grammars. FLUTE can relate the problem
of automata definition to the problem of definition of grammars. It may also show analogies between
terminal/nonterminal symbols of a grammar and input alphabet/states of an automaton. The student can switch
back to the problem at any time and try to solve it. In the case of success, FLUTE will congratulate the student
(but not too affectively - the students are skilled adults), encourage her/him to try more exercises (e.g. definitions
of some specific kind of automata), and possibly show some useful pointers and the student's progress indicator.
Otherwise, FLUTE will reveal the correct answer, explain what it believes the student needs to know in order to
assemble the definition of automata correctly, and suggest to remediate the erroneous knowledge or missing
knowledge. The suggestion will often require switching to the teaching mode. In both cases FLUTE shows some
short summary of the problem/exercise in the end. The summary contains the success indicator, the statement
about the problem type, the statement about the problem-solving method used, and suggestions for further
learning process.
8.2.
Evaluation Criteria
Pedagogical aspects and design suitability were the most important criteria used for the analysis and evaluation
of FLUTE in comparison with other ITSs. The other criteria used were design flexibility (modularity, reusability
and portability), ease of use, and flexibility of the student model. The results are briefly shown below.
8.3.
Comparing FLUTE to some other systems
We have used several other ITSs as referent systems in developing FLUTE. Some of them are described in Katz
and Lesgold (1993), Kong (1994), Vassileva (1990), and Wong et al. (1996). The main conceptual and design
difference between FLUTE and such systems is that FLUTE has been developed after a thorough ontological
analysis and with the help of previously developed GET-BITS framework. When FLUTE is compared to these
other ITSs, its advantages come from:
• the systematic design, based on the GET-BITS model and framework;
• its explicit, highly modular, and extremely flexible model of domain knowledge; it is open and easy for
extensions, since several new inputs can be added to any dependency graph; occasionally a new output from a
graph can be generated (for example, another advanced topic can be added at the advanced level in Figure 4);
47
• its strong, knowledge-based lesson presentation and didactic components (the way and time it shows lesson
outlines, objectives, transparencies, etc., always depends on the student's recent reactions and the student
model);
• its knowledge-based and student-model-based deduction on the moments when the student must undergo
exercises regarding both current lesson and past lessons;
• several different ways and levels of assessment of the student's knowledge (questions, exercises, tests, etc.);
• supporting timing functions in the examination and teaching modes, which is reflected in the student model;
• the way it switches between three different modes of operation, both on the user's request and automatically,
when the pedagogical module decides it is necessary;
• its user-oriented, separate explanation module;
• the way it treats examples (the role of examples is stressed in all three modes of operation, which makes the
learning process much easier and more attractive); and
• its more refined student model, allowing for more appropriate representation of student's knowledge and
learning capabilities.
FLUTE's pedagogical module has been designed starting from the variants of the coach model used in the
systems described in Vassileva (1990) and Wong et al. (1996), but with additional features described in Section
4.
Most apparent limitations and deficiencies of FLUTE's design with respect to the design of the other ITSs
mentioned above are the lack of hypertext support, the lack of histories of past lessons, learning, and assessments
in the student model, and weak example classification.
9.
Conclusions
Design of FLUTE, an intelligent tutoring system in the domain of formal languages and automata, is featured by
an explicit, flexible and easily extensible model of domain knowledge, three modes of operation (teaching,
examination, and consultation), highly refined student model, and an extensible database of examples that
facilitate both learning and tutoring processes. Domain-dependent pedagogical knowledge is included in the
expert module, rather than in the pedagogical module, and the explanation facility is separated from the
pedagogical module and constitutes a separate module in the system's architecture. The main advantages of such
design are improved lesson presentation and knowledge maintenance, as well as more explicit distinction
between different pedagogical features. The design is based on GET-BITS, an ontology-based framework for
building intelligent tutoring systems, and can be used for developing other intelligent tutoring systems as well.
There are several directions in future research and further development of the FLUTE system:
• development of appropriate techniques for knowledge simplification and maintenance, since the knowledge
base in the expert module is upgraded time after time;
• development of an automatic classifier of the examples in the database; such a classifier should be based on
both the domain knowledge and the pedagogical knowledge;
• development of a case-based generator of new problems for students, starting from the database of examples
and incomplete input data.
10. References
¾
¾
¾
¾
¾
Anderson, J. R., Boyle, C. F., Corbett, A. T. and Lewis, M. W. (1990). Cognitive Modelling and Intelligent
Tutoring. Artificial Intelligence, 42 (1), 7-49.
Appel, A. W. (1998). Modern Compiler Implementation in Java. Cambridge, MA: Cambridge University
Press.
Chee, Y. S. & Xu, S. (1997). SIPLeS: Supporting Intermediate Smalltalk Programming through Goal-based
Learning Scenarios. In du Boulay, B., Mizoguchi, R. (Eds.) Artificial Intelligence in Education, Amsterdam:
IOS Press, 95-102.
Debenham, J. K. (1998). Knowledge Engineering - Unifying Knowledge Base and Database Design. Berlin:
Springer-Verlag.
Devedzic, V. & Radovic, D. (1999). A Framework for Building Intelligent Manufacturing Systems. IEEE
Transactions on Systems, Man, and Cybernetics, Part C - Applications and Reviews. 29 (3), 422-439.
48
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
¾
Devedzic, V., Radovic, D. & Jerinic, L. (in press). The GET-BITS Model of Intelligent Tutoring Systems.
International Journal of Continuing Engineering Education and Life-Long Learning, Special Issue on
Intelligent Systems/Tools in Training and Life-Long Learning.
Fischer, C. N. & LeBlanc, R. J. (1988). Crafting a Compiler. New York: Benjamin/Cummings Publishing
Company.
Herzog, C. (1997). Cooperation and Competition of Sysntax-Oriented and Execution-Oriented Problem
Solvers in Intelligent Programming Tutors. In du Boulay, B., Mizoguchi, R. (Eds.): Artificial Intelligence in
Education, Amsterdam: IOS Press, 309-316.
Hopcroft, J. E. & Ullman, J. D. (1979). Introduction to Automata Theory, Languages and Computation.
Reading, Massachusetts: Addison-Wesley.
Johnson, W. L. (1990). Understanding and Debugging Novice Programs. Artificial Intelligence, 42 (1), 5197.
Issroff, K. & del Soldato, T. (1996). Incorporating Motivation into Computer-supported Collaborative
Learning. In Proceedings of The European Conference on Artificial Intelligence in Education, Lisbon:
Edicoes Colibri, 284-290.
Katz, S. & Lesgold, A. (1993). The Role of the Tutor in Computer-Based Collaborative Learning Situations.
In: S. Lajoie and S. Derry (Eds.) Computers as Cognitive Tools, Hillsdale, NJ: Lawrence Erlbaum
Associates, 289-317.
Keller, J. M. (1987). Strategies for Stimulating the Motivation to Learn. Performance and Instruction, 26
(8), 1-7.
Kong, H. P. (1994). An Intelligent, Multimedia-Supported Instructional System. Expert Systems with
Applications, 7 (3), 451-465.
Lepper, M. R., Woolverton, M., Mumme, D. L. & Gurtner, J. (1993). Motivational Techniques of Expert
Tutors: Lessons for the Design of Computer-Based Tutors. In S. Lajoie and S. Derry (Eds.) Computers as
Cognitive Tools, Hillsdale, NJ: Lawrence Erlbaum Associates, 75-105.
Lopez, M. F., Gomez-Perez, A., Sierra, J. P. & Sierra, A. P. (1999). Building a Chemical Ontology Using
Methontlogy and the Ontlogy Design Environment. IEEE Intelligent Systems, 14 (1), 27-36.
Matsuda, N., Kashihara, A., Hirashima, T. & Toyoda, J. (1997). An Instructional System for BehaviorBased Recursive Programming. In du Boulay, B., Mizoguchi, R. (Eds.) Artificial Intelligence in Education,
Amsterdam: IOS Press, 325-330.
Mizoguchi, R., Sinitsa, K. & Ikeda, M. (1996). Task Ontology Design for Intelligent Educational/Training
Systems. Paper presented at the Workshop "Architectures and Methods for Designing Cost-Effective and
Reusable ITSs". Montreal, Canada, June 12-14, 1996.
Muchnick, S. (1997). Advanced Compiler Design and Implementation. Morgan-Kaufman Publishers.
Murray, T. (1997). Toward a Conceptual Vocabulary for Intelligent Tutoring Systems. Working paper
available at:
http://www.cs.umass.edu/~tmurray/papers.html.
Raymond, J. & Pilon, D. (1994). Software Tools for Computer-Aided Lecturing. IEEE Transactions on
Education, 37 (1), 23-30.
Revesz, G. E. (1983). Introduction to Formal Languages. New York: McGraw-Hill.
Rist, T., Andre, E. & Muller, J. (1997). Adding Animated Presentation Agents to the Interface. In
Proceedings of the IUI'97 Conference, Orlando, Florida: ACM Press, 79-86.
Szyperski, C. (1998). Component Software: Beyond Object-Oriented Programming, NY/Reading, MA:
ACM Press/Addison-Wesley.
Vassileva, J. (1990). An Architecture and Methodology for Creating a Domain-Independent, Plan-Based
Intelligent Tutoring System. Educational and Training Technology International, 27 (4), 386-397.
Valente, A., Russ, T., MacGregor, R. & Swartout, W. (1999). Building and (Re)Using an Ontology of Air
Campaign Planning. IEEE Intelligent Systems, 14 (1), 27-36.
Wong, L.-H., Looi, C.-K. & Quek, H.-C. (1996). Design of an ITS for Inquiry Teaching. In Proceedings of
The Third World Congress on Expert Systems, Seoul, Korea: Cognizant Communication Corporation, 12631270.
49