Simulation-based tutoring
systems
CPI 494/598, April 16
Kurt VanLehn
Classification of step-based tutors
User interface
• Forms e.g., CTAT
• Canvas e.g., Andes
• Text e.g., Criterion
• Dialogue e.g., AutoTutor
• Simulation e.g., ???
Tutoring on
• Process
• During e.g., Cog Tutors
• After e.g., ???
• Both e.g., Andes
• Product e.g., Criterion
• Both e.g., AutotTutor
Prototypes (ill-defined classes) of
simulation-based tutoring systems
• Equipment operation & maintenance
– E.g., nuclear reactor operation
• Vehicle driving
– E.g., flight simulator
• Tactical decision making
– E.g., defend ship from possible aerial attack
• Negotiation, leadership, other soft skills
– E.g., convince an NGO’s clinic to move
• Lab/field experimentation
– E.g., discover cause of an epidemic
What’s different between simulationbased interface vs. others?
•
•
•
•
•
•
•
Time matters in the sim
Heavy reliance on delayed feedback
Game-based expectations of sim quality
Game-based technology e.g., game engines
Just making the sim work – huge!
Game-like expense e.g., $1M each!
Most research done outside universities
– Mostly military
Steamer – Very early example
(Hollan, Hutchins, Weitzman 1984)
• Steam plant occupies 1/3 of space on Navy
ships; miles of pipe;
• Engineer of the watch supervises 16 to 25
people who run it
• Years of experience required
Operator’s screen:
conceptual, not physical, fidelity of controls
One actual operator’s console
physical & conceptual fidelity
First, build the simulation
• Underlying mathematical model – huge!
– Built before Steamer
• Objects for each gauge & control authored
– Visual presentation
– Hooked up to underlying math model
• Augment math model: models of faults/failures
• Math may not be perfect
– Especially in abnormal operating conditions or failures
• Validate the sim
– check that standard operating procedures work
– Compare real to sim behaviors; many conditions
Sim is not an ITS but…
• Can be used for training anyway
– Drill to memorize standard op procedures
– Exercise in handling standard breakdowns
• Instructor(s) supervises trainees
–
–
–
–
–
Decides on exercise
Sets sim into initial state
Watches trainees
Gives immediate feedback occasionally
Gives delayed feedback: After Action Review (AAR)
Problem: After training, still can’t
handle non-routine situations
• Need a “mental model” of the whole steam
plan
– Helps to diagnose what is causing trouble
– Helps to hypothesize & test (mentally) repairs
• This sim is a black box; need a “glass box”
– Shows multiple levels of detail
– Should it to be active; can manipulate & watch
– Animation; real time?
Top level view of the whole steam plant
One of about 100 subsystem diagrams
Controlling sim via the schematics
• Can click on schematics to e.g., open a valve
• Can do “abstract” actions that are hard or
impossible to do in real world
– Replace boiler
– Raise chill-water temp to 100 degrees
Problem: Can’t understand time
courses
• Neg feedback loops in steam plant control
e.g., fuel flow to boiler
• Need to see first and second derivatives as the
sim is running
– Add new “gauges” to the sim
– Some were later added to the real steam plant
• Need to see graphs
New gauges at bottom; graph at top
Was Steamer ever an ITS?
• Only for safety issues
– Wired to give student immediate negative feedback if
certain safety conditions violated
– E.g., starting a pump without first opening the input
and output values
• Doesn’t know what student is attempting to do
–
–
–
–
Can’t offer immediate or delayed feedback
Can’t offer hints
Even if it knew goals, can’t connect goals to actions
Has no representation of knowledge-to-be-learned
• Except mental model of the plant itself, which is huge
RIDES
(Munro Johnson Pizzini Surmon Towne Wogulis, 1997)
• 15 years later
• Like Steamer, has authoring system for
widgets
• Has underlying language for implementing
Sim
• Integrates widgets & sim
Has panels with widgets
some are controls, some are gauges
Has better tools for authoring
instruction
Instructional design procedure
• Specify objectives; specify scenes
• Develop sim
– Graphical objects
– Behavior rules for each
– Debug sim
• Develop lessons
– Author procedures using the sim
– Add texts
– Debug lessons
Object palette: Note how mouse
controls rotation
Authoring a procedure
• Put sim in an initial state; name the procedure
• Do it
• End it
Put sim in an initial state
Half done with the procedure
Three modes for instruction
Provided automatically
• Demo mode
– Tutor executes procedure
– Student controls pace
• Practice model
– Student tries to execute
– Gets immediate feedback before sim reacts
– If authored hints exists, gets them
• Test mode
– No feedback & no hints
– Sim does whatever
Practice mode
Curriculum editor
Is Rides an ITS Authoring tool?
• Procedural training = CTAT = ITS
• Drill on identifying parts = tutor, but not so
intelligent?