An Experiment
Using CTAT to Explore the Role of
Self-Regulation in the Robust Learning of
Middle School Math
•
•
•
•
•
•
•
•
Research Questions & Hypotheses
Theoretical Assumptions: Good, Bad & Ugly
Using CTAT to test hypotheses
The Interface
Beneath the Interface: Models & Behavior Graphs
Lessons Learned
Extensions to the CTAT Interface Tools
Future work
Quincy Brown
Kallen Tsikalas
Research Questions & Hypotheses
1.
Effect of providing a self-regulatory goal. What
is the effect of giving students an explicit selfregulatory goal [to be “error detectives”] on their
robust learning and the accuracy of their self-efficacy
ratings?
2.
Effect of providing self-regulatory feedback and
practice opportunities. What is the effect of
providing students with feedback on and practice with
a self-regulatory skill [error detection and correction]
on their robust learning and the accuracy of their selfefficacy ratings?
3.
Predictive power of accurate self-efficacy
ratings. To what extent does the accuracy of
students’ self-efficacy ratings effect their learning
curve and help-seeking behavior?
Outcome Variables
- Accuracy of self-efficacy ratings
- Learning curves from CTAT data
- Pre-, post-, and delayed post-test scores
How sure are you
that you can solve
this problem?
Likert scale (1-10)
Theoretical Assumptions

Interventions that target students’ selfregulatory processes can lead to improved
cycles of learning and improved academic and
non-academic outcomes.


Providing feedback on self-regulatory skills
effects students’





Examples of self-regulatory interventions are training and/or
feedback on motivational beliefs, goal-setting, monitoring,
self-judgments, etc.
Ability to create internal feedback and self-assess
Attributions about success or failure
Proficiency at help-seeking
Willingness to invest effort in dealing with feedback
information
Cognitive load theory may suggest that attending to
errors introduces extraneous load which may diminish
robust learning.
Using CTAT to Test Hypotheses


2x2 factorial design
Control condition = Cognitive Tutor
with no self-regulation
enhancements’

Multiple versions of Cognitive Tutor
Error ID
Feedback

Opportunities for assisted practice of cognitive
skills
Self-Regulatory Goal
+
-
-
Control:
CogTutor w/ no SR
enhancements
The Interface
Two Versions

Example-Tracing Tutor




Executed in Flash
Steps on separate screens
Dynamic feedback: Students have opportunity to
interact with feedback screens
Full Cognitive Tutor


Executive in Flash
Interface represents deep mathematical structure
The CTAT Example-Tracing Interface



Executed in Flash
Steps on separate screens (Flash frames)
Dynamic feedback: Students have opportunity to
interact with error feedback on screens (through
Flash movies)
The CTAT Cognitive Tutor Interface


Executed in Flash
Streamlined format representing deep structure of
mathematics
The CTAT Full Cognitive Tutor
Behavior
Graph
Conflict
Tree
Working
Memory
Cognitive
Model
The CTAT Full Cognitive Tutor
Production Rules

All production rules functioning
The CTAT Example-Tracing Behavior
Graph for the CogTutor Interface
Lessons Learned


How to use the CTAT tools
Importance of think-alouds for building
example-tracing and production rules
-


To create correct branching structure
To optimize the number of rules – not more
than needed
Potential threats to the efficacy of our
intervention: Ken’s talk on design
principles
Ideas about new types of learning
outcomes (learning curves, help
requests that lead to greater learning)
Extensions to CTAT Interface Tools


Multiple screens for one tutor
Navigation between screens that
communicates with CTAT
-




Via ActionScript
Intratutor communication
Separate functions (e.g., visible and
invisible Flash movies) for displaying
feedback
Adjustments to Flash Widgets
Widgets just to log student actions/ideas
rather than to tutor
Debugging of Flash tutorials
Future Work

Extension to mobile devices

Use of student characteristics (e.g., selfefficacy ratings) to guide specific
tutoring actions

Use of student characteristics (e.g.,
accuracy of self-efficacy ratings) to
predict learning curves
Special Thanks to…
Everyone who helped us figure out what’s going on!

John and Brett for assistance with Flash
widgets and communication between ExampleTracing functions and Flash interface

Jonathan and Vincent for assistance with full
cognitive tutor development and production

Noboru for assistance with SimStudent

The PLSC Summer School students and staff
for their good humor and great ideas!