SimStudent:
Building a Cognitive Tutor by
Teaching a Simulated Student
Noboru Matsuda
Human-Computer Interaction Institute
Carnegie Mellon University
CTAT: Cognitive Tutor Authoring Tools
• Example-Tracing Tutor with zero programming
– A cognitive model specific to a particular problem
– Some generalization by modifying a behavior graph
SimStudent Project
• Model-Tracing Tutor requires a cognitive model
– Cognitive task analysis is challenging
– Writing production rules is even more challenging
– Performing the task is much easier…
2
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Next Generation Authoring
Build a tutor GUI
Teaching a solution
SimStudent Project
SimSt. learning
Production Rules
Rule simplify-LHS:
Rule simplify-LHS:
IF
is-equation( Eq ),
IFRule simplify-LHS:
is-equation(
Eq ),
is-lhs(
Eq, Lhs ),
is-lhs(
Eq,
IF polynomial(
is-equation(
Lhs Lhs
), ),Eq ),
polynomial(
Lhs
all-var-terms(
is-lhs( Lhs
Eq, )),
Lhs ),
all-var-terms(
Lhs
Then
simplify(
polynomial(
Lhs, S-lhs
Lhs
),) ),
Then
simplify(
), )
enter(
S-lhs
all-var-terms(
)Lhs, S-lhsLhs
enter(
S-lhs ) Lhs, S-lhs ),
simplify(
enter( S-lhs )
Then
3
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
SimStudent
• Machine learning agent
– Learns problem-solving steps by …
– Observes model solutions / solving problems, and …
SimStudent Project
– Outputs a set of production rules
• Fundamental technology
– Programming by Demonstration
– Inductive Logic Programming
Lau, T. A., & Weld, D. S. (1998). Programming by demonstration
Blessing, S. B. (1997). A programming by demonstration authoring tool for model-tracing tutors
4
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Authoring Strategies
• Authoring by demonstration
SimStudent Project
– Learning from worked-out examples
– Demonstrate whole solutions
– Learning by generalizing examples (when it can’t “selfexplain”)
• Authoring by tutoring
– Learning by doing (with tutor feedback)
– Interactively tutor with immediate feedback and hint
– Learning by generalizing hint with taking feedback into
account
5
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Demo!
SimStudent Project
6
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Learning Production Rules in 3 parts:
What-When-How
If
such and such constraints hold
When
SimStudent Project
among this and that GUI elements
What
Then
do actions with the GUI elements
How
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Production Rule in JESS
(defrule trans-lr-lhs
GUI elements
Working Memory Element (WME)
?problem <- (problem (interface-elements ?table1 ? ? ?))
?table1 <- (table (columns ?column1))
?column1 <- (column (cells $?m1 ?cell0 $?))
?cell0 <- (cell (value ?val0&~nil))
WME path
LHS
?problem <- (problem (interface-elements ? ?table2 ? ?))
?table2 <- (table (columns ?column2))
?column2 <- (column (cells $?m2 ?cell1 $?))
?cell1 <- (cell (value ?val1&~nil))
SimStudent Project
?column1 <- (column (cells $?m3 ?cell2 $?))
?cell2 <- (cell (name ?selection) (value ?input))
(test
(test
(test
(test
(test
(test
(consecutive-row ?cell0 ?cell2))
(same-column ?cell0 ?cell2))
(distinctive ?cell0 ?cell2))
(consecutive-row ?cell1 ?cell2))
(same-column ?cell1 ?cell2))
(distinctive ?cell1 ?cell2))
(test (polynomial ?val0))
(test (not (has-var-term ?val1)))
Topological
constraints
Feature
constraints
WME
conditions
Constraints
=>
(bind ?input (first-var-term ?val0))
(modify ?cell2 (value ?input)) )
Actions
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
RHS
Background Knowledge
• Domain concepts to “explain” demonstrations
– Operators
– Feature predicates
• External Jess function written in Java
SimStudent Project
(defrule multi-lhs
…
?var22140 <- (column (cells ? ? ?var22143 ? ? ? ? ?))
?var22143 <- (cell (value ?val0&~nil))
=>
)
9
(test (fraction-term ?val0 ))
(bind ?val2 (denominator ?val0))
(bind ?input (mul-term-by ?val0 ?val2))
…
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Example： Algebra domain
• 16 Feature predicates & 28 operators
SimStudent Project
10
Feature Predicates for LHS conditions
Operators for RHS actions
HasCoefficient
HasConstTerm
HasVarTerm
Homogeneous
IsFractionTerm
IsConstant
IsDenominatorOf
IsNumeratorOf
IsPolynomial
Monomial
NotNull
VarTerm
IsSkillAdd
IsSkillSubtract
IsSkillDivide
IsSkillMultiply
AddTerm
AddTermBy
Coefficient
CopyTerm
Denominator
DivTerm
DivTermBy
EvalArithmetic
FirstTerm
FirstVarTerm
GetOperand
InverseTerm
LastConstTerm
LastTerm
LastVarTerm
MulTerm
MulTermBy
Numerator
ReverseSign
RipCoefficient
SkillAdd
SkillClt
SkillDivide
SkillMultiply
SkillRf
SkillMt
SkillSubtract
VarName
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Learning Results
SimStudent Project
% Correct rule firings (10 test tasks)

0, if C  0
C
StepScore  

 C  I
Authoring by
tutoring

Authoring by
demonstration
# of training tasks
11
Better
than
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Authoring Time
Authoring by Tutoring
• Authoring by tutoring
took 86 minutes
• Authoring by demonstration
took 238 minutes
SimStudent Project
•
A 2.8x speed-up!
Authoring by Demonstration
12
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Example: Stoichiometry Tutor
SimStudent Project
Approximate Time Distribution (hours)
30
40
Operator generation
Feature predicate generation
Debugging
Testing
55
13
15
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Learn more about SimStudents
• Project Web
– www.cs.cmu.edu/~mazda/SimStudent
• Download & Tutorial
SimStudent Project
– http://ctat.pact.cs.cmu.edu (linked from project web)
• Contact Noboru
– [email protected]
14
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
SimStudent Projects
• Intelligent Authoring
– Building a Cognitive Tutor as a CTAT Plug-in
• Student Modeling and Simulation
SimStudent Project
– Controlled educational studies
– Error formation study
– Prerequisite conceptual knowledge study
• Teachable Peer Learner
– Learning by teaching
15
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)
Learning by
Teaching
SimStudent
Prepare Lucky for Quiz Level 3 !
Prepare Lucky for Quiz Level 3 !
Curriculum Browser
Level 1:
Curriculum
Browser
[+] One-Step
Linear Equation
Level
Level 1:
2:
[+]
[+] One-Step
Two-Step Linear
Linear Equation
Equation
Level
Level 2:
3:
[+]
Two-Stepwith
Linear
Equation
[-] Equation
Similar
Terms
SimStudent Project
Overview
Level 3: In this unit, you will solve
[-] Equation
withwith
Similar
Terms
equations
integer
or decimal
Overview
coefficients, as well as equations
In
this unit,
you than
will solve
involving
more
one variable.
equations with integer or decimal
More…
coefficients, as well as equations
involving more than one variable.
More…
Lucky
LuckyTutor Lucky
Next Problem
Quiz Lucky
Tutor Lucky
Next Problem
Quiz Lucky
PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)