Intelligent web-based tools to
support e-learning
Eva Millán
(IA)2 Group
University of Málaga
(ia)2
A little bit about me...
(ia)2
I am associate professor at Malaga University, where I lecture on
 Approximate Reasoning (fuzzy logic, Bayesian networks)
 Operations Research
Most of my research work has been about “deep thinking” in
student modeling with Bayesian networks (the subject of my PhD
thesis)
In parallel, I have developed several tools for teaching Linear
Programming
I have also worked in other projects developed in my research
team, like MEDEA and SIETTE
Lately, I have also been working in the evaluation of LeActiveMath.
Since 2003, I am also vice-dean (of innovation in education) of the
Computer Science School, in which I lead the Bologna process
Eva Millán
What am I working (in research) at the moment…
Writing a paper with Tomek Loboda (preliminary title “Bayesian student
modeling without tears”)
Modeling Felder’s learning styles with Bayesian Networks and using
learning algorithms (with Gladys Castillo and Cristina Carmona, for
Cristina’s PhD)
Using Bayesian Networks to model collaboration in virtual communities
(with Beatriz Barros and Javier Burón, for Javier’s master thesis project)
Developing ITS for education in the moral
(ethical) values, dedicated to children in
Venezuela prisons (with Arlenys Varela from
Venezuela, for her master thesis project)
Keep collaboration with other members in my
research team in projects like MEDEA and
SIETTE
Eva Millán
(ia)2
(ia)2
So what I am going to present in this talk?
Something “old” TAPLI, a web-based tutor for linear
programming (2003)
Something “new” A tool based on Bayesian Networks
to analyze the collaboration from the logs of a virtual
community (submitted but not published yet)
Eva Millán
Something “old”
TAPLI: a web-based tool for Linear
Programming
Eduardo Guzman (connection with SIETTE)
Emilio Garcia (implementation)
Eva Millán (domain expert, design, development of
contents)
Introduction
a)
b)
c)
TAPLI is an adaptive web-based learning environment
for Linear Programming, that consists in the
integration of three tools in the same environment:
An adaptive hypermedia component, that is
responsible of presenting the learning contents;
An adaptive testing component, that allows selfevaluation using tests
An adaptive drill-and-practice component, which:
dynamically generates exercises
 coaches students, offering guidance, support, help and
feedback.

Eva Millán
(ia)2
Motivation
(ia)2
Tapli is a web-based learning environment about Linear
Programming
TAPLI has been designed and implemented to be used
by students of Operations Research in the Computer
Science School of the University of Málaga as an extra
help for learning.
TAPLI was based in previous work of our group,
namely EPLAR and ILESA, which were former versions
and also SIETTE, and adaptive web-based tool for
testing
Eva Millán
(ia)2
The domain
What is a Linear Programming Problem?
A linear programming problem is a problem of the
type:
Optimize
Subject to
c1x1+...+cnxn
a11x1+…+a1nxn  b1
am1x1+…+amnxn  bm
It has all sorts of applications in any situation in which
resources are scarce
Eva Millán
(ia)2
The domain
To solve a linear programming problem, there is a
systematic procedure called the Simplex Algorithm
(Dantzig, 1940)
The simplex algorithm has been named among the 10
more important algorithms developed in the 20th century.
In a finite number of steps, it always conducts to the
solution. An example:
First step: Introduce slack variables to convert the
inequalities into equalities
Maximize 3x + 2y
Subject to 2x + y  3
-5x + 2y  6
Eva Millán
Maximize 3x + 2y
Subject to 2x + y + s1
=3
-5x + 2y
+ s2 = 6
To our purposes, the important thing is that….
(ia)2
The domain is strongly based in problem solving
The steps are always performed in the same order
Types of errors are easily identified
Problems can be generated at the right level of
difficulty
Which, in our context, allows:
Coached problem solving
Dynamic generation of problems
Eva Millán
Components in TAPLI
(ia)2
TAPLI is in fact a set of three tools, running in the
same environment:
 An adaptive hypermedia component, responsible of
presenting content to students
 A testing tool, that allows the evaluation of students
 A drill-and-practice environment, in which students
are posed a problem adapted to their knowledge
level and they can solve it while being coached by
the system
Eva Millán
The tools in TAPLI: Adaptive hypermedia
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Curriculum
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Learning
contents
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Recommendations
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Student model
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Student model
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Student model
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Student model
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Pages visited by
student
Eva Millán
(ia)2
The tools in TAPLI: Adaptive hypermedia
Pages visited by
student
Eva Millán
(ia)2
Adaptation in the hypermedia component
(ia)2
Adaptable features:

Content presentation is adapted to student’s goals
and level, by means of stretch text and link hiding
(tests)
Adaptive features:

The list of visited pages is used to suggest the
next piece of content,

Student’s knowledge level is used to suggest the
next activity (take a test, solve an exercise, read
some content),

This information is combined in a recommendation
to the student.
But TAPLI only suggests (free navigation is supported)
Eva Millán
The testing component in TAPLI
(ia)2
While learning contents, students can test their
knowledge using tests or exercises
Both activities are supported by the SIETTE system
SIETTE is a web-based environment for adaptive
testing, that can be used by
 Instructors to develop web-based tests
 Students to take such tests
Though SIETTE supports adaptive testing (e.g.
different test lengths for different users) based on IRT
theory, tests in TAPLI are not adaptive (due to the lack
of a database of properly callibrated items for linear
programming).
Eva Millán
(ia)2
A few words about SIETTE
For students to
take tests
Eva Millán
For teachers
to define tests
Some words about SIETTE
(ia)2
It is the result of TEN years of intensive work in adaptive testing
(by Ricardo Conejo and Eduardo Guzman)
It has been used in around twenty different real courses to
evaluate more of 2000 real students, in different locations, in all
kinds of domains (from Java programming, to Artificial
Intelligence, Botany, etc.)
It has solid theoretical foundations, grounded in Probability
theory, and in particular in Item Response Theory which allows for
adaptive testing (reduced test length while increasing accuracy)
It can be used as a tool for testing or, even more interesting for
ITS developers, as a diagnosis tool to perform the role of the
student modeling component in web-based learning environments
(just as we did in TAPLI), thus saving lots of work to AWES
developers.
Much more information in related publications (just type SIETTE in
google)
Eva Millán
Linear Programming tests in SIETTE
Eva Millán
(ia)2
(ia)2
Communication between TAPLI and SIETTE:
Architecture
STUDENT MODEL
STUDENT
MODEL
REPOSITORY
Eva Millán
TESTING
COMPONENT
DRILL-AND-PRACTICE
COMPONENT
SIETTE
Students
INTERFACE
INSTRUCTIONAL
PLANNER
ADAPTIVE HYPERMEDIA
COMPONENT
Interactivity with the hypermedia component
Eva Millán
(ia)2
Students
INTERFACE
INSTRUCTIONAL
PLANNER
STUDENT MODEL
STUDENT
MODEL
REPOSITORY
Eva Millán
Presents theoretical
concepts and examples
ADAPTIVE HYPERMEDIA
COMPONENT
TESTING
COMPONENT
DRILL-AND-PRACTICE
COMPONENT
SIETTE
The architecture of TAPLI
(ia)2
(ia)2
The architecture of TAPLI
Students
INTERFACE
INSTRUCTIONAL
PLANNER
STUDENT MODEL
STUDENT
MODEL
REPOSITORY
Eva Millán
TESTING
COMPONENT
DRILL-AND-PRACTICE
COMPONENT
Evaluates
student’s
knowledge
SIETTE
ADAPTIVE HYPERMEDIA
COMPONENT
(ia)2
The architecture of TAPLI
STUDENT MODEL
STUDENT
MODEL
REPOSITORY
Eva Millán
TESTING
COMPONENT
DRILL-AND-PRACTICE
COMPONENT
Coaches problem
solving
SIETTE
Students
INTERFACE
INSTRUCTIONAL
PLANNER
ADAPTIVE HYPERMEDIA
COMPONENT
(ia)2
The architecture of TAPLI
Selects the next
component
STUDENT MODEL
STUDENT
MODEL
REPOSITORY
Eva Millán
TESTING
COMPONENT
DRILL-AND-PRACTICE
COMPONENT
SIETTE
Students
INTERFACE
INSTRUCTIONAL
PLANNER
ADAPTIVE HYPERMEDIA
COMPONENT
(ia)2
The architecture of TAPLI
Students
INTERFACE
INSTRUCTIONAL
PLANNER
STUDENT MODEL
STUDENT
MODEL
REPOSITORY
Eva Millán
TESTING
COMPONENT
DRILL-AND-PRACTICE
COMPONENT
Stores the
student
model
SIETTE
ADAPTIVE HYPERMEDIA
COMPONENT
Communication between TAPLI and SIETTE:
Procedure
(ia)2
The communication of this component with SIETTE is based on
URL calls with the proper parameters.
Initially, the testing component sends to SIETTE:
a) the set of topics to be assessed;
c) the number of knowledge levels in which the student can be
classified;
b) the current estimation of student’s knowledge about these
topics;
d) the URL to which the results will be returned, and
e) additional parameters to configure the test (item selection
mechanism, finalization criteria, ...).
Once the evaluation has finished, SIETTE invokes the given url and
passes the new estimated knowledge level of the student.
With these data, TAPLI updates the student model.
Eva Millán
The drill and practice component in TAPLI
This component in TAPLI
 is able to generate problems at the adequate level
of difficulty.
 supports coached problem solving
How is this achieved?
Eva Millán
(ia)2
Generation of problems in TAPLI
Relationship among skills
and types
of problems
is incremental
Eva Millán
(ia)2
1. Introduce slack variables.
2. Fill in the simplex tableau with data.
3. Identify solution and objective value in the
tableau.
4. Select entering variable for maximization
LP0´s.
5. Select leaving variable.
6. Perform calculations.
7. Identify optimal solutions.
8. Level 1. Solve max. problems with unique
solutions.
9. Recognize alternative optimal solutions.
10. Level 2. Solve problems with alternative
solutions.
11. Recognize unbounded solution.
12. Level 3. Solve problems with unbounded
solutions.
13. Level 4. Solve any maximization problem.
14. Select entering variable for minimization
LP0´s.
15. Level 5. Solve any minimization problem.
16. Introduce artificial variables.
17. Construct problem for Phase 1.
18. Identify unfeasibility in Phase 1.
19. Level 6. Solve problems with unfeasible
solutions.
Generation of problems in TAPLI
There are several approaches for automatic generation of
problems (see Belmonte et al, 2002).
In the TAPLI case, we need to generate:
 A criterion (maximize, minimize)
 A set of numbers for objective function and constraints
 A direction for constraints (>=, <=)
Some basic rules control the random generation process, for
example
 For infinite solutions, one of the constraints should be parallel
to the objective function.
Mechanisms are also used to control the difficulty of the
computations
In this way problems for each of the levels can be generated 
UNLIMITED SET OF PROBLEMS
Eva Millán
(ia)2
Coached problem solving in TAPLI
(ia)2
The level of the student is the highest level of
problems that he/she can correctly solve.
Therefore there are seven levels for SIETTE to classify
students in.
The integration with SIETTE is transparent to the
student.
Once the problem has been generated by the system
or introduced by the student, he/she can solve it within
the same environment.
The sequence of steps will be guided by a set of
applets integrated in SIETTE
Eva Millán
Coached problem solving in TAPLI
Eva Millán
(ia)2
Coached problem solving in TAPLI
Students can ask for help
If students make mistakes, the system will provide
feedback
Both help and feedback are penalized by the system
Problems are evaluated by the applet, which classifies
them as correct or incorrect
Then SIETTE returns this information to TAPLI, so it
can update the student model.
Eva Millán
(ia)2
Conclussions
(ia)2
TAPLI is an adaptive web-based learning environment for
Linear Programming
It is composed of three educational components:
 An adaptive and adaptable hypermedia component
 A drill-and-practice component
 A testing component
It allows for several types of adaptation:
 Adaptive navigation support (recommendations)
 Adaptive content presentation (stretch text, link hiding)
 Adaptive problem generation
The system is being used by students at UMA as an extra aid
for learning, but has not been formally evaluated
Eva Millán
Something “new”
A Bayesian model to analyze collaboration
in virtual communities
Beatriz Barros (expert on analyzing collaboration)
Javier Burón (implementation)
Eva Millán (expert on Bayesian Networks)
Motivation
Virtual learning communities allow now e-learning in
groups, with a new perspective that allows active
learning in collaboration with other people
These new trends in e-learning offer new challenges
for researchers in social and collaborative learning, as
the environments provide a set of rich data to use for
the study of social activity:





Eva Millán
How does the group organize the work?
How does collaboration arise?
When do conflicts arise?
Which cases demand help?
What was the procedure to get the solution?
(ia)2
(ia)2
Our proposal:
Virtual learning
Community (logs)
Quantitative indicators
(Martinez, 2003)
About the
interaction
About
actions
Of social
type
About
performance
Analysis algorithm
Bayesian model
Quantitative indicators
Indicators of
social states
Eva Millán
Mechanism to control
the virtual community
Indicators of
collaboration
Mechanisms for adapting
resources and content
Indicators
Quantitative from interaction
Quantitative from action
Quantitative of social type
Qualitative of social type
Qualitative from collaboration
Eva Millán
(ia)2
Indicators
(ia)2
Quantitative indicators: Interaction




Average intervention size, representing average size of the
contributions, measured in terms of the number of characters
and weighted according to the activity type, divided by the
total number of interventions in the community
Average number of interventions (in forums, chats,
workshops), weighted by community size and by the duration
of the activity
Average level of activity, which counts the number of
interventions that were answered by a user different to the
one that initiated it.
The average intervention size and number of interventions is
weighted according to the type of activity, as shown in the
following table
Eva Millán
Weights for the different types of activities
Activity
Action
Value
Forums
Add a post
1
Add a discussion
1
Evaluate a post
0.7
Update a post
0.5
Delete a post
-1
Add a new term
1
Update a term
0.5
Add a comment
0.7
Evaluate a term
0.7
Chat
Talk
0.7
Workshop
Workshop delivery
1
Workshop update
0.8
Wiki
Edit a wiki
1
Message
Write a message
0.7
Glosary
Eva Millán
(ia)2
More indicators
(ia)2
Quantitative indicators: actions
 Number of actions, which counts for the number of
actions like access to web pages, clicks on links,
resources or activities, divided by the number of
participants.
 Division of work, which measures if students divided
the work (instead of collaborating to do it)
Eva Millán
More indicators
(ia)2
Quantitative indicators for performance:
 Grade, that measures the quality of the
contributions of the group
Quantitative indicators of social type:
 Density, which measures the degree of
interconnection in the network
 Centralization, which is an structural measure that
indicates to what extent the network depends on
some of its actors. A high value will indicate that the
network depends on few actors, and vice-versa.
Eva Millán
More indicators
(ia)2
Qualitative indicators of social type
 Sociability, measuring if all individuals interact in
order to solve the required tasks.
 Quality of the participation, which relates the
activity of each individual with the cognitive result
as individually evaluated by a teacher.
 Impasse, which accounts for situations of nonactivity
 Passivity, which accounts for situations in which
individuals do not interact with each other.
 Leadership, which measures if there is an individual
which is leading the coordination of the group.
Eva Millán
(ia)2
More indicators
Qualitative indicators of different types of group work
 Cooperation (individuals divide work)
 Coordination (there is a person that organizes the
work: he/she divides it, assigns it to the other
members of the community, and generates results)
 Collaboration (a group of individuals gets organized
so that all members work together (all members in
all tasks) and participate in a balanced way in all
parts of the solution.
Eva Millán
(ia)2
Methodology
Moodle
database
Logs of
users
XML: logs of the community
XML: values of indicators
CLIENT-MOODLE
Web services
Eva Millán
Web services
compute values
of indicators
The Bayesian network for GeNIe
Eva Millán
(ia)2
(ia)2
An example
Logs of a MOODLE community studying Biology were
analyzed.
From the logs, it was automatically inferred that:








Eva Millán
Average number of interventions = low
Interactivity level = high,
Average intervention size = high,
Average number of actions = low,
Division of work = yes,
Density = high,
Centralization = low,
Average grade = passed
Set of evidence for
the BN
Results in this example:
(ia)2
Computed values for indicators
Average number of interventions = low
Interactivity level = high,
Average intervention size = high,
Average number of actions = low,
Division of work = yes,
Density = high,
Centralization = low,
Average grade = passed
From the evidence available in this community,
it can be deduced that:
• The community is not very sociable
• The level of activity has been low
• Is quite probable they did not reach an impass
• There was not a clear leader
• Probably there were coalitions
• There was more cooperation than coordination
and collaboration
• In general, the quality of the participation was low
Eva Millán
Conclussions
(ia)2
A log analyzer based on Bayesian networks has been
implemented and integrated into MOODLE.
This log analyzer allows to infer the type and quality of
the collaboration activity in a virtual community
This is only a first step in the construction of a module
that allows to analyze actions and interactions in virtual
communities, independently of the platform being
used.
Our next steps are:
 Testing this algorithm in other platforms
 Evaluating these algorithms compared to other
methods and algorithms to analyze collaboration
Eva Millán
(ia)2
Thanks for your attention…
Contact at [email protected]
Eva Millán
(ia)2
Some more reasons to visit us:
Malaga
Ronda
Zahara
Sweet grapes,
Sweet wine
Paella
Eva Millán