Dynamic Decision Making in Complex Task
Environments:
Principles and Neural Mechanisms
Progress and Future Directions
November 17, 2009
A High-Stakes, Time-Critical
Decision
• A diffuse form is coming toward you rapidly: What
should you do?
– You could shoot at it, but it may be your friend
– You can hold your fire, but it might shoot you!
– You could wait to decide, but that might be risky too
• How can we optimize our choices, and the timing of our
choices, in time-critical, uncertain situations?
FY07 MURI BAA06-028 Topic 15
Building Bridges between Neuroscience, Cognition,
and Human Decision Making
Objective: The general goal is to form a complete and thorough
understanding of basic human decision processes … by building a
lattice of theoretical models with bridges that span across fields ….
The main effort of this work is intended to be in the direction of new
integrative theoretical developments … using mathematical and/or
computation modeling … accompanied and supported by rigorous
empirical model tests and empirical model comparisons …. .
From BAA 06-028, Topic 15
Our MURI Grant
• Builds on past neurophysiological, behavioral and theoretical
investigations of the dynamics of decision making in humans
and non-human primates toward the development of an
integrated theory.
• Extends the empirical effort by employing fMRI, EEG, and MEG
convergently to understand the distributed brain systems
involved in decision making.
• Bridges to investigations concerned with decision making
processes in real-life situations (e.g. those faced by air-traffic
controllers and pilots).
Who We Are
• PIs:
–
–
–
–
McClelland, Newsome (Stanford)
Holmes, Cohen (Princeton)
Urban (Carnegie Mellon)
Johnston (NASA Ames)
• And many other professional scientists, postdocs, and graduate students in applied
mathematics, neuroscience, cognitive science,
and engineering
A Classical Model of Decision Making:
The Drift Diffusion Model of Choice
Between Two Alternative Decisions
•
At each time step a small sample
of noisy information is obtained;
each sample adds to a cumulative
relative evidence variable.
•
Mean of the noisy samples is +m
for one alternative, –m for the
other, with standard deviation s.
•
When a bound is reached, the
corresponding choice is made.
•
Alternatively, in ‘time controlled’
or ‘interrogation’ tasks, respond
when signal is given, based on
value of the relative evidence
variable.
The DDM is an optimal model, and it
is consistent with neurophysiology
• It achieves the fastest possible decision on average for
a given level of accuracy
• It can be tuned to optimize performance under different
kinds of task conditions
– Different prior probabilities
– Different costs and payoffs
– Variation in the time between trials…
• The activity of neurons in a brain area associated with
decision making seems to reflect the DD process
Neural Basis of Decision Making in
Monkeys (Shadlen & Newsome;
Roitman & Shadlen, 2002)
RT task paradigm of R&T.
Motion coherence and
direction is varied from
trial to trial.
Neural Basis of Decision Making in
Monkeys: Results
Data are averaged over many different neurons that are
associated with intended eye movements to the location
of target.
Two Problems with the DDM
• The model predicts correct
and incorrect RT’s will
have the same
distribution, but incorrect
RT’s are generally slower
than correct RT’s.
Hard
Errors
RT
• Accuracy should gradually
improve toward ceiling
levels as more time is
allowed, even for very
hard discriminations, but
this is not what is
observed in human data.
Prob. Correct
Easy
Correct
Responses
Hard -> Easy
Usher and McClelland (2001)
Leaky Competing Accumulator Model
• Inspired by known neural
mechanisms
• Addresses the process of deciding
between two alternatives based
on external input (r1 + r2 = 1)
with leakage, mutual inhibition,
and noise:
dx1/dt = r1-k(x1)–bf(x2)+x1
dx2/dt = r2-k(x2)–bf(x1)+x2
f(x) = [x]+
Wong & Wang (2006)
~Usher & McClelland (2001)
One-dimensional reduced version of the
LCAM (U&M, 2001; Bogacz et al, 2006)
•
Neglect the non-linearity in the LCAM:
dx1/dt = r1-k(x1)–b(x2)+x1
dx2/dt = r2-k(x2)–b(x1)+x2
Then subtract and let x = x1-x2,
I = r1 - r2 ; x = x1 - x2
To obtain:
dx/dt = I -k(x)+b(x)+x
•
This allows precise mathematical analysis, while
approximating the outcomes found in the full non-linear
version of the LCA…
•
And it reduces to the DDM if k-b = 0
•
So can see a direct link between neural mechanisms
and optimal decision making under uncertainty
Roles of k and b
Bifurcation
Produces
A DecisionLike Outcome
x1 – x2 represents the difference in activation of the two
accumulators for the same value of r1 – r2. Time proceeds
from stimulus onset. Distribution of values of x1-x2 is shown
at three different time points for three combinations of k and b
Time-accuracy curves for different
|k-b|
|k-b| = 0
|k-b| = .2
|k-b| = .4
Can we Distinguish Alternative
Explanations?
• LCAM fits time-accuracy data well, but there are other
possible reasons for bounded accuracy
– Trial-to-trial variation in the direction of drift
– Bounded integration prior to the termination of the stimulus
• Distinguishing these alternatives as best we can is one
of the goals of our project
Specific Aims
• Aim 1: Extend the theory of the dynamics of decision
making to address integration of stimulus and payoff
information in real-time decision making task situations,
integrating behavioral, neuroscientific, and theoretical
investigations.
• Aims 2 and 3: Extend this multi-pronged approach to
more complex tasks and task environments
– Continuous time and space
– Uncertain timing of stimulus onset
– Real-life situations including distraction, multitasking, and payoff uncertainty
Some of the Questions we Will be
Addressing in Our Work
• To what extent can humans and other participants achieve
optimality in decision making, in a range of different decision
contexts? What can we learn from deviations from optimality?
• Within the space of dynamic decision making models, can we
find evidence that distinguishes among alternatives consistent
with existing data?
• Can we identify the brain mechanisms that underlie the
decision process and determine how these mechanisms
achieve optimal performance?
• To what extent are the dynamics of decision making fixed
characteristics of the decision mechanism, and to what extent
are the tunable to task demands? If so, how is this tuning
achieved?
The Schedule
0930 Introduction and scientific
background
Jay McClelland
1330 EEG and MEG studies of decision
dynamics
Patrick Simen
1000 Integrating Payoff and stimulus
information
Bill Newsome
1400 Effects of stimulus perturbations
and switches on decision dynamics
Juan Gao
1030 Modeling the dynamics of choice
behavior
Phil Holmes
1430 Developing continuous measures of
decision state
Joel Lachter & James Johnston
1100 Break
1430 Break
1115 Integration of payoff and stimulus
information in humans
Jay McClelland
1515 Multi-single unit recording
studies in primates
Bill Newsome
1145 Using fMRI to identify decision
structures in humans
Jon Cohen
1545 Other future directions and
general discussion
Jay McClelland
1215 Discussion
1230 Lunch