How rational are your decisions? — Neuroeconomics —
How rational are your decisions?
— Neuroeconomics —
Hecke
CNS Seminar WS 2006/07
How rational are your decisions? — Neuroeconomics —
Motivation
How rational are your decisions? — Neuroeconomics —
Motivation
Ferdinand Porsche
"Wir wollen Autos bauen, die keiner braucht
aber jeder haben will."
How rational are your decisions? — Neuroeconomics —
Outline
1
Introduction
2
Neural Coding of Utilities and Value Functions
3
The Role of Emotion in Decision Making
4
Social Decision Making
5
Learning and Decision Making
6
Mentalizing and Theory of Mind
How rational are your decisions? — Neuroeconomics —
Introduction
Introduction
homo economicus
is widely assumed to be a rational and self-interested decision
maker.
in reality
routines to approximate optimal decision making:
emotionally driven
experience based
taking into account decisions of others
estimate outcome of alternative actions
Review: D. Lee: Neural basis of quasi-rational decision making. Current Opinion in
Neurobiology 2006, 16: 191–198
How rational are your decisions? — Neuroeconomics —
Neural Coding of Utilities and Value Functions
Neural Coding of Utilities and Value Functions
utility
experienced utility:
subjective pleasure from the effect of an action
decision utility:
estimate of experienced utility for decision making
brain regions that code for utility
show neural activity that is affected by the
amount of reward
striatum, dorsolateral PFC, orbitofrontal cortex,
anterior and posterior CC and posterior parietal
cortex
are also affected by anticipated outcomes while
decision making
How rational are your decisions? — Neuroeconomics —
Neural Coding of Utilities and Value Functions
Neural Coding of Utilities and Value Functions
How rational are your decisions? — Neuroeconomics —
Neural Coding of Utilities and Value Functions
Matching Behavior in Monkeys
Matching behavior in monkeys.
) The sequence of
events of an oculomotor matching task: (i)
Fixate. To begin a run
of trials, the animal
must fixate the central
cross. (ii) Delay. Saccade targets appear
(randomized spatially
by color) in opposite
hemifields while the
animal maintains fixation. (iii) Go. Dimming
of the fixation cross
cues a saccadic response and hold. (iv)
Return. Brightening of
the fixation cross cues
return, target colors are
then
rerandomized,
and the delay period of
the next trial begins.
Reward is delivered at
the time of the response, if at all. Overall
maximum reward rate
is set at 0.15 rewards
per second. Relative reward rates changed in
100 to 200
trials) without warning;
Experiment
poissonian reward
different rates
change without
warning
How rational are your decisions? — Neuroeconomics —
Neural Coding of Utilities and Value Functions
A local Version of Matching
A model of dynamic matching be) Equation
(top) shows a restatement of the classical
global matching law,
relating fractional income to fractional
choice (stated here in
terms of the red target). Schematic (bottom) shows that in
global matching, cu, is
computed by perfect
integration of the
stream of rewards up
.
) Equation (top)
shows a local implementation of the
matching law, relating
local fractional income to instantaneous probability of
. Schematic
(bottom) shows that
, is
computed with the
use of a leaky integrator with time constant
. In practice, the
monkey’s history of
choices and rewards
on each color was represented as a vector of
1’s and 0’s, indicating
rewarded and unre-
global matching
needs perfect integration
local matching
Local fractional income
relates to instantaneous
probability of choice.
leaky integrator: oblivion
How rational are your decisions? — Neuroeconomics —
Neural Coding of Utilities and Value Functions
Utility coding
neuron shown in Fig.
delayed saccade task
matching task
Lateral Interparietal representation of fractional
income of saccadic eye movements
How rational are your decisions? — Neuroeconomics —
The Role of Emotion in Decision Making
The Role of Emotion in Decision Making
expected utility theory
maximizing the utilities
weighted by their
probabilities
choice
A
B
C
D
Allai’s paradox
0$
1000 $
0
1
0.01
0.89
0.89
0.11
0.9
0
5000 $
0
0.1
0
0.1
Too simple, does not take into account emotions.
subjective bias towards certainty via nonlinear transformation
regret theory
minimization of regret
regret: cognitive or emotional reaction associated
with the realization that an unchosen action would
have yielded better outcome
arises from too high imagined outcome
How rational are your decisions? — Neuroeconomics —
The Role of Emotion in Decision Making
Regret in Orbitofrontal Cortex
Experiment
200
50
-50
Game
value
-50
choice of one gamble
Choice
200
-50
50
-50
200
-50
Partial
feedback
Spinning arrow holds at the
outcome.
Wait
50
-50
Outcome
200
-50
50
-50
200
-50
Complete
feedback
Wait
50
-50
Outcome
-50
Extremely
sad
-25
0
Neither sad
nor happy
25
50
Extremely
happy
Affective
rating
The outcome of the
alternative gamble is once
shown, once not.
Subjects rate their emotions.
How rational are your decisions? — Neuroeconomics —
The Role of Emotion in Decision Making
Regret in Orbitofrontal Cortex
Normal subjects
(N=18)
A
Emotional rating
Orbitofrontal patients
(n=5)
B
50
50
40
40
30
30
20
20
0
0
-10
-10
-20
-20
-30
-30
-50
-50
-50
50
D
50
lesioned people don’t
50
obtained outcome
obtained outcome
Emotional rating
Both subjective rating and skin
conductance reveal:
normal subjects show
huge regret
-40
-50
50
40
40
30
30
20
20
10
10
0
0
-10
-10
-20
-20
Complete
feedback
(regret)
-30
-30
-40
-40
-50
-50
-50
50
-50
obtained outcome
E
Partial
feedback
(disappointment
10
10
-40
C
-200 unobtained
200 unobtained
50
obtained outcome
F
1.2
1.2
Skin conductance
partial
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
-1mm
complete
-4mm
-16mm
0.2
0
0
[-50,200]
[50,200]
[Obtained, non-obtained]
[-50,200]
[50,200]
[Obtained, non-obtained]
Lesion Overlap
How rational are your decisions? — Neuroeconomics —
The Role of Emotion in Decision Making
Emotion in Decision Making
elation and disappointment
arise if the outcome is irrespective of the chosen action
not good for learning (if the goal is rational behavior)
prospect theory
choice outcomes are measured relative to the
status quo
this is supported by several monkey studies
and weighted by a nonlinear function of the
probabilities
How rational are your decisions? — Neuroeconomics —
Social Decision Making
Social Decision Making
game theory
characterizes the problem of social interaction
by a payoff matrix for multiple players and individual actions
Classical game theory (rational decisions!) predicts the
reaching of a Nash equilibrium.
Nash equilibrium
a set of strategies for all players from which no
player can increase their payoffs by altering their
strategies individually.
How rational are your decisions? — Neuroeconomics —
Social Decision Making
Social Decision Making
Ultimatum Game
Trust Game
How rational are your decisions? — Neuroeconomics —
Social Decision Making
Social Decision Making beyond Nash eq.
ethical valuation of other’s decisions
not related to payoffs
enhanced activation of certain brain regions
only with human partners
no effect with computer pertners
How rational are your decisions? — Neuroeconomics —
Social Decision Making
Social Decision Making beyond Nash eq.
Desire to punish uncooperative others
activation of the caudate nucleus
IC: decision to punish, punishment costly
IF: decision to punish, punishment free
IS: decision to send a (free) symbolic message
NC: random action, punishment costly
How rational are your decisions? — Neuroeconomics —
Learning and Decision Making
Learning and Decision Making
reinforcement learning
reward prediction error: discrepancy between predicted
and actual reward
used to update decision utilities or prediction functions
dopamine
dopamine neurons in the
ventral tegmental area
and substantia nigra
encode reward prediction
errors
How rational are your decisions? — Neuroeconomics —
Learning and Decision Making
Learning and Decision Making
belief theory
enhancement of reinforcement learning
belief of the strategy of the other players
it is also updated regularly
than maximizing payoff given the beliefs
approaching reality
belief theoretic algorithms need to alter every expected
payoff if a belief is changed (stone/paper/scissors)
mixed strategies seem to match behavior
neural mechanisms poorly understood
but: fMRI evidence for increased activity in ventral striatum
if reasoning about the other’s strategy
How rational are your decisions? — Neuroeconomics —
Learning and Decision Making
Self-Referential Thinking
OR
R
EC
TE
D
PR
OO
F
What do you think, he thinks,
you wan’t to do?
difference between
1st and 2nd order belief
in equilibirium, (certainty)
no change in neural activity
posterior and anterior
cingulate regions, frontal
insula, dorsolateral PFC
for ’theory of mind’
2nd order beliefs activate
the anterior insula
mixture of forming beliefs
and making choices
How rational are your decisions? — Neuroeconomics —
Mentalizing and Theory of Mind
Mentalizing and Theory of Mind
Mentalizing
repeated playing with the same other players
accumulation of information about the behaviors
theory of mind is
the ability to
represent mental
states of others:
beliefs
desires
knowledge
theory of mind
might play a key role in optimizing
decision making strategies during
social interactions
mostly unique to humans
How rational are your decisions? — Neuroeconomics —
Mentalizing and Theory of Mind
Prisonners Dilemma Game
Two players choose to
Cooperate or to Defect.
They learn about the other’s
choice afterwards.
They get reward according to
the matrix on the left.
How rational are your decisions? — Neuroeconomics —
Mentalizing and Theory of Mind
Prisonners Dilemma Game
Neural correlates for a theory of mind
right mid posterior superior
temporal sulcus
yellow means more activation
with human partners compared
to computers in CC and CD
outcomes
20 consecutive rounds with the
same partner
How rational are your decisions? — Neuroeconomics —
Mentalizing and Theory of Mind
Conclusions
the assumption of rationality is false
but useful to analyze complex decision making problems
to generate hypotheses about corresponding cognitive and
neural processes
studies of emotion based choices and learning in decision
making help to link formal theories to reality
additionally, neurobiological studies reveal neural bases