Modeling the Effects of Scaffolding in Human Decision Making
Jessica A.
Darrell A.
2
Worthy ,
Neda
1
Abdul-Razzak ,
University of Texas at Austin Psychology Department;
Rewards based on ten most recent responses
using a sliding window:
0,1,1,[0,1,0,1,0,1,0,1,0,1]1,1,0
1: increasing option (optimal)
0: decreasing option (suboptimal)
We propose that increased difficulty and social pressure (performance bonus and
fictitious partner) will increase task demand, causing older adults to reach their
crunch point. However, younger adults may perform better with increased task
demand due to scaffolding and a shift toward heuristic-based decision making.
With increased task demand, pressure should also
cause a decline in younger adult performance.
0.8
0.6
0.5
0.4
0.3
0.2
Younger adults age 18-25 from the University of Texas and Texas A&M
University and older adults age 60-80 from the Austin and College Station
communities. Older adults are less than 2 SD from the norm on tests of cognitive
impairment.
Older Adults
Young Adults
No Pressure
21
21
Pressure
21
26
0.1
Modeling Continued
•  Basic WSLS has two free parameters, P(stay | win) and P(shift | loss), and
compares the reward on the current trial to reward received on the previous trial.
The Extended Exploration WSLS model distinguishes between the two ways to
“win” or “lose”: initial and extended exploration
•  Initial exploration: choosing different option
than the previous trial
•  Extended exploration: choosing the same
option as the previous trial
0
Younger Adults
Older Adults
•  Use of these heuristic-based strategies is thought to be frontally mediated.7,9
Experiment 2: Age and Task Difficulty
1
0.9
Younger Adults
0.8
Older Adults
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Easy (1:1)
Medium (2:2)
Task Difficulty
Hard (3:1)
Sample Size
Older Adults
Young Adults
2 suboptimal
25
22
3 suboptimal
34
33
•  Significant Age x Task difficulty interaction
(p<.01)
•  Older adults perform better with one
decreasing option than two (p<.05) and
better with two than three (p<.05)
•  Younger adults perform better with two
decreasing options than one (p<.01) and
better with two than three (p<.001)
Experiment 3: Task Difficulty and Pressure
1
Sample Size
No Pressure
Pressure
Easy (1 sub-opt)
From Exp 1
From Exp 1
Hard (3 sub-opt)
31
30
0.9
0.8
No Pressure
0.7
Pressure
0.6
0.5
0.4
0.3
0.2
0.1
•  Younger adults perform better under
pressure than under no pressure in the
easy task (p<.05).
•  Younger adults perform worse under
pressure than under no pressure in the
difficult task (p<.05)
0
Difficult
Modeling
•  Data were fit with a baseline model, Softmax reinforcement learning model with
eligibility trace (ET), and Extended Exploration Win-Stay Lose-Shift model.
•  The baseline model captures random responding5.
•  The ET model updates Expected Values of each option on each trial to develop
probabilities for selecting each option6 and have been correlated with striatal activity.7,8
(1)
Participants
Sample Size
•  Significant Age x Pressure interaction
(p<.001).
•  OA performed worse under pressure than
no pressure (p<.05); YA performed better
under pressure than no pressure (p<.05)
0.7
Easy
Experiment 1: Age and pressure
Easy task (1 optimal, 1 sub-optimal)
Experiment 2: Age and task difficulty
Medium: 4 options (2 optimal, 2 sub-optimal)
Hard: 4 options (1 optimal, 3 sub-optimal)
Experiment 3: Task difficulty and pressure
Hard: 4 options (1 optimal, 3 sub-optimal)
A&M University Psychology Department
No Pressure
Pressure
0.9
1
Maddox
(2)
•  Learning is modulated by a recency, or learning rate parameter, α
•  The ET (λ) credits the reward on each trial to options chosen on previous trials
•  The ET for each option decays based on the decay parameter ζ,
, the ET for an
option is increased each time it is chosen:
•  In eq. 2, θ2 is an additional exploitation parameter and d is updated in the same manner as
the ET on each selection.
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Baseline
No Pressure
Pressure
Younger Adults
• 
• 
• 
• 
• 
ET
WSLS
AkaikeWeight
The goal of this research is to test predictions about the effects of task demand on
decision making generated under STAC using a history dependent decisionmaking task, the Mars Farming task.
1
& W. Todd
Akaike Weight
Application to Decision Making
Experiment 1: Age and Pressure
Proportion Optimal Selections
•  Previous work from our lab found:
•  An older adult advantage in a history-dependent decision making task that
we attributed to compensatory scaffolding and increased monitoring of the
reward environment1
•  An increase in younger adults performance under pressure relative to no
pressure, also attributed to increased monitoring of the reward
environment2
•  The scaffolding theory of aging and cognition (STAC) suggests that the
recruitment of additional frontal regions occurs:
•  Across the lifespan in response to challenge
•  In older adults to compensate for age-related neural declines3
•  The presence of an age-based “crunch” point when additional resources
cannot be recruited4 suggests that age advantage may be fragile, and thus
may be affected by increased task demand.
2Texas
Results
Proportion Optimal Selections
Introduction
Proportion Optimal Selections
1The
1
Cooper ,
No Pressure
Pressure
Older Adults
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Baseline
ET
WSLS
No Pressure Pressure No Pressure Pressure
Easy
Difficult
ET model best fit for YA in the easy condition, while OA best fit by WSLS
No pressure: ET best fit for YA and WSLS model best fit for OA, p’s<.05
Fit of ET model higher (better) for YA no pressure than pressure, p<.05.
Fit of the WSLS model were better for OA under no pressure than pressure, p<.05
WSLS fit in the difficult condition also decreased for YA under pressure, p<.05
Conclusions
•  Older adults outperform younger adults in a history-dependent task, but the
advantage depends on situational factors and task difficulty.
•  Younger adults under pressure and with increased task difficulty perform better
on this task than older adults.
•  In harder tasks, pressure may also cause younger adult performance to decline.
•  Consistent with the crunch hypothesis, older adults show decreased use of
heuristic-based strategies under pressure; younger adults show increased use
of these strategies in easy tasks under pressure, but decreased use under
pressure in more difficult tasks.
References
1.  Worthy, D.A., Gorlick, M.A., Pacheco, J.L., Schnyer, D.M., & Maddox, W.T. (2011). With Age Comes Wisdom: Decision Making in Younger and Older Adults. Psychological Science,
22(11), 1375-1380.
2.  Worthy, D.A., Cooper, J.A., & Maddox, W.T., Decision-Making Under Time and Social Pressure. (Under Review)
3.  Park, D. & Reuter-Lorenz, P.A. (2009). The Adaptive brain: Aging and neurocognitive scaffolding. Annual Review of Psychology, 60, 173-96.
4.  Reuter-Lorenz, P.A. & Cappell, K. (2008). Neurocognitive aging and the compensation hypothesis. Current Directions in Psychological Science. 17(3),177-182.
5.  Worthy, D.A., Otto, A.R., & Maddox, W.T. (in press). Working-Memory Load and Temporal Myopia in Dynamic Decision-Making. Journal of Experimental Psychology: Learning, Memory,
& Cognition.
6.  Sutton, R. S. and Barto A. G. (1998). Reinforcement Learning: An Introduction. The MIT Press, Cambridge, MA.
7.  Worthy, D.A., & Maddox, W.T. (2012). Age-based differences in strategy-use in choice tasks. Frontiers in Neuroscience, 5, 1-10.
8.  Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behavior in humans. Nature, 442, 1042–
1045.
9.  Ashby, F.G., Alfonso-Reese,L.A., Turken, A.U., & Waldron,E.M.(1998). A neuropsychological theory of multiple systems in category learning. Psychol.Rev. 105, 442–481.
Special thanks to Kirsten Smayda and Taylor Denny for their help with data collection
For more information, please visit: http://homepage.psy.utexas.edu/homepage/Group/MaddoxLAB/index.htm
This research was supported by the Department of Defense (DoD) through the National Defense Science & Engineering Graduate Fellowship (NDSEG) Program
to JAC and NIDA grant DA032457 to W. Todd Maddox