Exploiting Cognitive Constraints
To Improve Machine-Learning Memory Models
Michael C. Mozer
Department of Computer Science
University of Colorado, Boulder
Why Care About Human Memory?


The neural architecture of human vision has inspired computer vision.
Perhaps the cognitive architecture of memory can inspire the design of
RAM systems.
Understanding human memory essential for ML systems that predict what
information will be accessible or interesting to people at any moment.
 E.g., selecting material for students to review to maximize long-term
retention (Lindsey et al., 2014)
The World’s Most Boring Task
Stimulus X -> Response a
Stimulus Y -> Response b
response latency
frequency

response latency
Sequential Dependencies

Dual Priming Model
(Wilder, Jones, & Mozer, 2009; Jones,
Curran, Mozer, & Wilder, 2013)
 Recent trial history leads to
expectation of next stimulus
 Responses latencies are fast
when reality matches
expectation
 Expectation is based on
exponentially decaying traces
of two different stimulus
properties
Examining Longer-Term Dependencies
(Wilder, Jones, Ahmed, Curran, & Mozer, 2013)
Declarative Memory
study
test
Cepeda, Vul,
Rohrer, Wixted,
& Pashler (2008)
Forgetting Is Influenced By The Temporal Distribution Of Study
Spaced study
produces more
robust & durable
learning than
Massed study
Experimental Paradigm To Study Spacing Effect
% Recall
Cepeda, Vul, Rohrer, Wixted, & Pashler (2008)
Intersession Interval (Days)
Optimal Spacing Between Study Sessions
as a Function of Retention Interval
Predicting The Spacing Curve
50
25
70
75
50
25
0
105
100
Percent Recall
Percent Recall
100
RI = 70
days)
17 21 35
70
50
25
0
17 21 35
70
ISI (days)
105
predicted
recall
100
25
7 day retention
80
17 21 35
70
100
75
105
ISI (days)
RI = 350
ISI (days)
Percent Recall
Percent Recall
days)
105
70
intersession
interval
50
ISI (days)
100
70
17 21 35
Multiscale
Context
Model
75
0
105
characterization
forgetting
after
of student
one
and session
domain
75
0
= 35
Forgetting Curve
75
recall
%%Recall
Percent Recall
100
105
35 day retention
60
70 day retention
40
50
20
350 day retention
25
0
17 21 35
70
ISI (days)
0
105 1
7
14 21
35
70
spacing
(days)
Intersession
Interval
(Days)
105

Multiscale Context Model
(Mozer et al., 2009)
 Neural network
 Explains spacing effects

Multiple Time Scale Model
(Staddon, Chelaru, & Higa, 2002)
 Cascade of leaky integrators
 Explains rate-sensitive habituation

Kording, Tenenbaum, Shadmehr (2007)
 Kalman filter
 Explains motor adaptation
Key Features Of Models





Each time an event occurs
in the environment…
A memory of this event
is stored via multiple traces
Traces decay exponentially
at different rates
fast
trace
strength

medium
+
slow
+
Memory strength is
weighted sum of traces
Slower scales are downweighted relative to faster scales
Slower scales store memory (learn) only when faster scales fail to predict event
event
occurrence
event
occurrence
time
time
Exponential Mixtures ➜ Scale Invariance

Infinite mixture of exponentials gives exactly power function

Finite mixture of exponentials gives good approximation to power function
+
+
=

With
, can fit arbitrary power functions
Relationship To Memory Models In Ancient NN Literature

Focused back prop (Mozer, 1989), LSTM (Hochreiter & Schmidhuber, 1997)
 Little/no decay

Multiscale backprop (Mozer, 1992), Tau net (Nguyen & Cottrell, 1997)
 Learned decay constants
 No enforced dominance of fast scales over slow scales

Hierarchical recurrent net (El Hihi & Bengio, 1995)
 Fixed decay constants

History compression (Schmidhuber, 1992;
Schmidhuber, Mozer, & Prelinger, 1993)
 Event based, not time based
Sketch of Multiscale Memory Module

xt: activation of ‘event’ in input to be remembered, in [0,1]

mt: memory trace strength at time t

Activation rule (memory update) based on error, et = max(0, mt - xt )
 Activation rule consistent with the 3 models
(for Koerding model, ignore KF uncertainty)
 This update is differentiable ➜
can back prop through memory module
+
mt
 Redistributes activation across time scales in a manner that is
dependent on temporal distribution of input events

Could add output gate as well to make it even more LSTM-like
-1
fixed
∆
+1
xt
learned
Sketch of Multiscale Memory Module


Pool of self-recurrent neurons with fixed time constants
Input is the response of a feature-detection neuron
+1
 This memory module stores the particular feature that is detected
 When the feature is present, the memory updates
Update depends on error between
+
is a feature detected at time t

When feature detected, memory state compared to input, and a correction
-1
∆
is made to memory to represent input strongly
fixed
+1
learned
Why Care About Human Memory?

Understanding human memory essential for ML systems that predict what
information will be accessible or interesting to people at any moment.
 E.g., shopping patterns
 E.g., pronominal reference
 E.g., music preferences