RevealedIndifference:UsingResponseTimestoInferPreferences*
ArkadyKonovalov1andIanKrajbich2
April23,2017
Abstract
Revealedpreferenceisthedominantapproachforinferringpreferences,butitrelieson
discrete,stochasticchoices.Thechoiceprocessalsoproducesresponsetimes(RTs)which
arecontinuousandcanoftenbeobservedintheabsenceofinformativechoiceoutcomes.
Moreover, there is a consistent relationship between RTs and strength-of-preference,
namelythatpeoplemakeslowerdecisionsastheyapproachindifference.Thisrelationship
arises from optimal solutions to sequential information sampling problems. Here, we
investigateseveralwaysinwhichthisrelationshipcanbeusedtoinferpreferenceswhen
choiceoutcomesareuninformativeorunavailable.WeshowthatRTsfromasingletwoalternativechoiceproblemcanbeenoughtousefullyrankpeopleaccordingtotheirdegree
oflossaversion.Usingalargenumberofchoiceproblems,wearefurtherabletorecover
individual utility-function parameters from RTs alone (no choice outcomes) in three
differentchoicedomains.Finally,weareabletouselongRTstopredictwhichchoicesare
inconsistentwithasubject’sutilityfunctionandlikelytolaterbereversed.Theseresults
provideaproofofconceptforanovel“methodofrevealedindifference”.
JELCodes:C91;D01;D03;D87;D81;D90.
*
TheauthorsthankPaulHealy,LucasCoffman,RyanWebb,DanLevin,JohnKagel,KirbyNielsen,
JeevantRampal,andPujaBhattacharyafortheirhelpfulcommentsandconversationsandYosuke
Morishima,ErnstFehr,ToddHare,ShabnamHakimi,andAntonioRangelforsharingtheirdata.
1
DepartmentofEconomics,TheOhioStateUniversity;[email protected].
2DepartmentofPsychology,DepartmentofEconomics,TheOhioStateUniversity
[email protected].
1
1Introduction
When inferring a person’s preferences, businesses, managers, economists, and policy
makerstypicallyrelyontheirchoices.Thisisthestandardrevealedpreferenceapproach
(Samuelson, 1938). In this paper we challenge this fundamental notion, by arguing that
individualpreferencescanbeinferredwithoutobservingchoiceoutcomes.
Choice itself is not the only output of the choice process. We are also often able to
observeotherfeaturessuchasresponsetimes(RT).Moreover,RTsarecontinuousandso
may carry more information than discrete stochastic choice outcomes (Loomes, 2005;
Webb,2013)3.Thetrickliesindiscoveringhowtheinteractionbetweenpreferencesand
choiceoptionsproducesRTs.Ifwecouldcharacterizeandtheninvertthisfunction,then
wecouldinferpreferencesfromRTs.
Considerthefollowingexample.Supposeweareattemptingtodeterminewhichoftwo
agentsismoreimpatient.Weaskeachofthemthesamequestion:“wouldyouratherhave
$25 today or $40 in two weeks?” Suppose that both agents take the $40. With just this
informationthereisnowaytodistinguishbetweentheagentswithoutfurtherquestioning.
Now suppose Agent A made his choice in 5 seconds, while Agent B made hers in 10
seconds.Whoislikelymorepatient?
Weargue,usingdatafromseveralchoiceexperimentsindifferentpreferencedomains,
thattheanswerisAgentA.Whilestandardeconomictheoryissilentinthissituation,there
arerelevanttheoreticalframeworksforansweringthisquestion.Onerelevantframework
of particular interest, due to its success in cognitive psychology and neuroscience, is the
3
ForareviewontheuseofRTsineconomics,seeClithero(2016a)andSpiliopoulosandOrtmann
(2014).
2
class of sequential sampling models (SSM) (also known as drift-diffusion or evidenceaccumulationmodels).Thatframeworkviewsdecisionslikethisoneasamentaltug-ofwar between the options. For options that are similar in strength (utility) it takes more
timetodeterminethewinner,andinmanycasestheweakersidemayprevail.
To put it another way, rather than basing their decisions on deterministic utilities for
eachoption,agentsinsteadbasetheirdecisionsonstochasticutilities,likeinrandomutility
theory(RUT).AswithRUT,SSMsareagnosticaboutwheretheaverageutilitiesornoise
originate,theyaresimplytheresultofsomeevaluationprocess4.WhereSSMsdepartfrom
standardRUTisbyassumingthatagentsrepeatedlyestimateor“sample”thesestochastic
utilities, averaging out the noise across samples until one option emerges as the clear
winner. This evaluation process takes an amount of time that depends on the difference
betweentheaverageutilities,i.e.thestrength-of-preference.
As a result of this process, a relationship emerges between strength-of-preference,
choice probability, and RT. Returning to our example, we know that since Agent A was
relativelyfasterthanAgentB,itismorelikelythatAgentAfacedaneasierdecision.Since
it was easier for Agent A to choose the later option, we can infer that he is likely more
patient.
Of course, other independent factors may influence RT, but if there is a statistically
reliable relationship between decision speed and proximity to indifference, this could be
used to economists’ and policy makers’ advantage in a number of ways. First, one could
design simpler and shorter preference-elicitation mechanisms such as the single-trial
4
Theassumptionisthatthisevaluationprocess,followingthenotionofutilitytheoryasan“asif”
model,doesnottypicallyinvolveexplicitutilitycalculation.Otherwisetheutilitieswouldbecome
deterministicandRTswouldnotdependonutilitydifference,astheyoftendoempirically.
3
binary-choice example above. Second, one could recover utility functions in cases where
agents’ preferences fall outside of the range of a given elicitation procedure, where it is
currently only possible to put a lower or upper bound on preferences. Third, one could
estimateutility-functionparametersinthecompleteabsenceofchoiceoutcomesorwhen
these outcomes are not informative (i.e. the same). By asking a variety of questions, one
couldseewhichoneselicittheslowestresponsesandinferthatthosequestionsmadethe
person roughly indifferent. This last point is to highlight that it is still possible to do
preference analysis in the absence of informative choice outcomes. This suggests, for
example, that in order to protect their private information, agents must consider
observabilityofboththeirchoicesandtheirRTs.
Here, we demonstrate that inferring preferences from RTs is indeed a promising
approach, using experimental data from three prominent choice domains: risk, time, and
social preferences. In Section 5.3 we show that single-trial RTs can be used to rank
subjects according to their degree of loss aversion. In Section 5.4 we show that RTs on
“extreme” trials (where most subjects choose the same option) can also be used to rank
subjectsaccordingtotheirloss,time,andsocialpreferences.FinallyinSections5.5and5.6
we examine RTs from the full datasets and use each subject’s RT data to estimate their
utility-function parameters. In every case these rankings/parameters significantly align
with those estimated from subjects’ choices over the full datasets. Taken together, these
results serve as a proof of concept that individual preferences can be inferred from RTs
alone.
2Model
4
2.1.Background
Aseconomists,weoftendefendexpectedutilitytheorybyarguingthatpeoplemerely
behave “as if” they are calculating and maximizing utility. But if people do not explicitly
calculateutilities,howdotheymakedecisions?Thishasbeenatopicofintensestudyin
thefieldsofjudgmentanddecisionmaking,andmorerecently,neuroeconomics.Fromthis
research there is emerging consensus that people make decisions by accumulating and
comparing evidence for each of the alternatives, until the evidence for one option
sufficiently outweighs the other. In some cases this process may involve explicitly
collecting information from the environment, e.g. reading reviews before deciding which
movietowatch.Inothercases,itmayinvolveintrospection:collectinginternalevidence
basedonpastexperiencesandone’scurrentstate,e.g.decidingwhichofyourtwofavorite
movies to re-watch. Economists have traditionally treated these two types of decisions
separately,referringtotheformeras“perceptibility”andthelatteras“desirability”(Block
&Marschak,1960).
Whilethesemayseemlikeverydifferenttypesofdecisions,thereisnowoverwhelming
empiricalevidencethattheycanbeexplainedbyacommonmathematicalframework.The
precisemechanismbywhichthisevidenceaccumulationandcomparisonoccursisstillan
activetopicofstudyinneuroscience5,butfornowwetreatSSMsinthesamewaythatwe
treat expected utility: as “as if” models. That is, we don’t yet know exactly what or how
evidence is being accumulated and compared in the brain, but this modeling framework
5
Theexactmechanismbehindnoisysampling(e.g.,samplingfrommemory,shiftsinattention,or
noiseinneuralnetworks)remainsasubjectofongoingdebateintheliterature(Shadlen&
Shohamy,2016)andisoutsideofthescopeofourstudy.
5
allowsustoaccountforchoiceandRTdata(aswellaseye-trackingandneuraldata)that
othermodelscan’t.
TheideaofapplyingSSMstodecisionsthatarebasedsolelyoninternalevidenceiswell
establishedincognitivepsychology6.RogerRatcliff’s(1978)seminalworkonthediffusion
modelwasasatheoryofmemoryretrieval.Subjectswouldinitiallystudyalistofwords
andthenlaterbepresentedwitha“probe”word.Theirtaskwastoindicatewhetherthe
probewasawordthathadappearedonthestudylist.Evidenceinthismodelwassimply
aninternalcomparisonbetweentheprobewordandthesubject’simperfectmemoryofthe
study words. Similarly, economic choice relies on a comparison between imperfect
memoriesand/orrepresentationsofthechoicealternatives.
TheideaofapplyingSSMstoeconomicchoicewasfirstintroducedbyBusemeyerand
colleagues in the 1980s (Busemeyer, 1985; Busemeyer & Diederich, 2002; Busemeyer &
Rapoport, 1988; Busemeyer & Townsend, 1993; Johnson & Busemeyer, 2005; Rieskamp,
2008;Roe,Busemeyer,&Townsend,2001).Recentyearshaveseenrenewedinterestin
thisworkduetotheabilityofthesemodelstosimultaneouslyaccountforchoices,RTs,eye
movements, and brain activity in many individual preference domains such as risk and
uncertainty (Busemeyer, 1985; Busemeyer & Townsend, 1993; Fiedler & Glöckner, 2012;
Hunt et al., 2012; Stewart, Hermens, & Matthews, 2015), intertemporal choice (Dai &
Busemeyer,2014;Rodriguez,Turner,&McClure,2014a),socialpreferences(Hutchersonet
al.2015;Krajbichetal.2015a;Krajbichetal.2015b),aswellasfoodandconsumerchoice
(DeMartino,Fleming,Garret,&Dolan,2013;Krajbich,Armel,&Rangel,2010;Krajbich,Lu,
6Forover50years,cognitivepsychologistshavebeenusingSSMstoexplainindividualbehaviorin
simpleperceptionandmemorytasks(BrownandHeathcote2008;Laming1979;Link1975;
Ratcliff1978;RatcliffandMcKoon2008;Stone1960;UsherandMcClelland2001).
6
Camerer,&Rangel,2012;Krajbich&Rangel,2011;Milosavljevic,Malmaud,Huth,Koch,&
Rangel, 2010; Philiastides & Ratcliff, 2013; Towal, Mormann, & Koch, 2013), and more
complex decision problems (Caplin & Martin, 2016). Our approach builds on this
literature,withoneimportantchange:insteadoffittingthemodeltochoicesandRTs,wefit
themodeltoonlytheRTs.Thisletsustesthowusefulthemodelcanbewithoutobserving
choiceoutcomes.
Beyondtheirdescriptivepower,SSMscanalsobenormative.Wheneverydecisionis
equallydifficultandequallyvaluable,theconstant-thresholddriftdiffusionmodel(DDM)is
optimal (Wald, 1945), in the sense that for a desired accuracy, the DDM minimizes mean
RT. When decisions vary in difficulty or value, the optimal SSMs are more complex and
involve collapsing thresholds (Busemeyer & Rapoport, 1988; Frazier & Yu, 2007;
Fudenberg,Strack,&Strzalecki,2015;Tajima,Drugowitsch,&Pouget,2016;Webb,2013b;
Woodford,2014).Itisworthnotingthatrecentworkindecisiontheoryhasfocusedonthe
problem of optimal sampling and stochastic choice (Che & Mierendorff, 2016; Matejka &
McKay,2014;Sims,2003;Steiner,Stewart,&Matějka,2017;Woodford,2016).
In this paper we deliberately avoid the issue of whether agents are using the optimal
SSM. Instead we rely on the simplest, most robust SSM variant, which is the DDM with
constantthresholds.Therearemanypapersthathaveempiricallyinvestigatedoptimality,
withlittleevidenceforcollapsingthresholds(Hawkins,Forstmann,Wagenmakers,Ratcliff,
&Brown,2015;Oudetal.,2016),andourstudyisnotdesignedtoaddressit(suchstudies
typicallyinvolvehundredsoreventhousandsofchoicetrials).Admittedlythismayhinder
our ability to infer preferences, but the simple DDM provides a proof-of-concept and a
7
lowerboundontheusefulnessofRTsineconomicanalysis.Laterinthetextweexamine
othermodelsandstatisticaltechniquesforinferringpreferencesfromRTs.
Onekeythingtonoticewiththesemodelsisthattheagentchoosesherstoppingrule
(thresholds)priortothesamplingprocessandsowhatdeterminesheractualchoiceand
RT, conditional on that stopping rule, is the sequence of evidence that she accumulates7.
That sequence is stochastic, but does depend on the agent’s subjective evaluation of the
alternatives. More specifically, following previous literature, we assume that the net
evidence is the difference between the average utilities of the two options (from a RUT
perspective)8. Therefore the speed and accuracy of the decision are monotonically
increasinginthisdifference.Thisquantityisoftenreferredtoasthe“driftrate”,andinour
economicsettingitrepresentsstrength-of-preference,i.e.thedifferenceincardinalutility.
Thus,whenapersonisclosetoindifference,thedriftrateapproacheszero,causingadelay
inthechoice(Chabris,Morris,Taubinsky,Laibson,&Schuldt,2009;Dickhaut,Rustichini,&
Smith,2009;Moffatt,2005;Mosteller&Nogee,1951).
2.2.Model
Hereweuseastandarddrift-diffusionmodel(DDM)(Ratcliff1978)toestablishthelink
betweenRTsandtheunderlyingutilitiesinsimplebinarychoicetasks.Asnotedabove,this
model is a standard tool in cognitive psychology for simultaneously capturing choice
probabilitiesandRTdistributions.Ithasseenmostofitsuseinperceptionandmemory,
7
Thisisastandardassumption,withveryfewexceptions.Thisrelatesbacktothenormativeissues
discussedaboveandisdiscussedatlengthinthosearticles.
8
Again,weassumethatsubjectsarenotexplicitlycalculatingexpectedvaluesorotherutilities.If
thatwasthecase,themaindeterminantofdifficultywouldbethecomplexityofthecalculations.
Instead,weassumethatsubjectsestimatehowgoodeachalternativeis,anditistherelative
attractivenessofthetwooptionsthatdeterminesthedifficulty.
8
but it has also more recently been applied to economic choice, e.g. intertemporal choice,
performingbetterthancompetingmodelssuchaslogisticchoice(Dai&Busemeyer,2014).
Letusassumethatanagentobservesasetofalternatives𝑗 ∈ {1,2}.Thechoiceprocess
involves two components: a constant boundary threshold 𝑏 and a decision variable 𝑦(𝑡)
thatevolvesovertimeaccordingtothefollowingdifferentialequation:
𝑑𝑦 𝑡 = 𝑣 ∙ 𝑑𝑡 + 𝜎 ∙ 𝑑𝑊
(1)
where𝑦(𝑡)isaccumulatedevidencetowardsoption1(with𝑦(0) = 0),𝑣isthedriftrate,
whichisassumedtobealinearfunctionoftheutilitydifference:
𝑣 ≡ 𝑧 ⋅ (𝑢9 𝜃 − 𝑢< 𝜃 ),
(2)
where𝑧 ∈ 𝑅> isascalingparameter,𝑢? (⋅)istheutilityofthegivenalternative,and𝜃isthe
agent-specificutilityfunctionparameter.Finally,𝜎 ∙ 𝑑𝑊isaWienerprocess(i.e.Brownian
motion)thatrepresentsGaussianwhitenoisewithvariance𝜎 < .Withoutlossofgenerality,
wenormalize𝜎 = 1asitcanonlybeidentifieduptoscale(duetothearbitraryunitsony).
We define the response time 𝑅𝑇 as the first time that the absolute value of the
decision variable reaches a boundary 𝑏 ∈ 𝑅> , plus a non-stochastic component known as
non-decisiontime(𝜏 ∈ 𝑅> ,typicallyinterpretedasthetimethatasubjectneedstoprocess
theinformationonthescreen):
𝑅𝑇 = min 𝑡: 𝑦 𝑡
≥ 𝑏 + 𝜏.
(3)
(4)
Thechoiceoutcome𝑎 ∈ {1,2}isdefinedasfollows:
𝑎=
1𝑖𝑓𝑦 𝑅𝑇 = 𝑏
2𝑖𝑓𝑦 𝑅𝑇 = −𝑏
9
Now, assuming without loss of generality 𝑣 ≥ 0, we can calculate the choice
probabilities𝑝(𝑎 = 𝑗)9,theexpectedRT,andanapproximatebivariateprobabilitydensity
function (PDF) for the RTs (minus 𝜏) as follows (Blurton, Kesselmeier, & Gondan, 2012;
Navarro&Fuss,2009;Ratcliff,1978;Srivastava,Feng,Cohen,Leonard,&Shenhav,2015;
Wabersich&Vandekerckhove,2014):
N OPQ R9
𝑝(𝑎 = 1) =
9
<
V
𝑓(𝑡) =
X
YV
𝑒R
O
𝐸 𝑅𝑇 =
PO [
O
a
\b9
−1
W
<
N OPQ RN SOPQ
𝑖𝑓𝑣 > 0
𝑖𝑓𝑣 = 0
1−
<
N OPQ RN SOPQ
+ 𝜏𝑖𝑓𝑣 > 0
𝑏 + 𝜏𝑖𝑓𝑣 = 0
\R9
𝑘⋅𝑒
R
^O _ O [
`QO
sin
\X
<
(𝑒 WV + 𝑒 RWV )
(5)
(6)
(7)
Typically, if choice data is available, one can use equation (5) to estimate
parameterspurelyfromchoices,maximizinglog-likelihood:
𝐿𝐿 =
e(1
𝑎e = 1 ⋅ log 𝑝 𝑎e = 1 𝑏, 𝑣e
+ (1 𝑎e = 2 ⋅ log 1 − 𝑝 𝑎e = 1 𝑏, 𝑣e ,(8)
where𝑛denotestrialnumber,and1(⋅)istheindicatorfunction.
Here,unlikepreviousstudies,wewillassumethatonlyRTdataisavailableandwilluse
theRTprobabilitydensitiestoestimatetheutilityparameterforeachsubject𝑖(𝜃j )given
theempiricaldistributionofRTsbymaximizingthefollowinglikelihoodfunction:
𝐿𝐿 =
e(log(𝑓
𝑅𝑇e , 𝑎e = 1 𝑏, 𝜏, 𝑣e ) + log(𝑓 𝑅𝑇e , 𝑎e = 2 𝑏, 𝜏, 𝑣e )). (9)
9
Previousworkhasshownthatthestandardrandomutilitymodelthatimpliesalogitchoicerule
canbederivedfromthedrift-diffusionmodel(Webb,2013b).
10
5
●
●
● ●
●●
●
●
●
4
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●●
●
●
3
●
●
●
●
●
●
● ● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
● ● ● ●●
●
●
●
●
●●
●
●
●
●
● ●
● ● ●
●
●
●
●
●
●
● ●● ● ●
●
●● ● ●● ●
●
●
●
●●
●● ●
●●
●
●
● ● ● ●
●
●
●
●
●
●
●● ●●
● ●
●
●
●
●●●●
●● ●
●
●
●
●●
●● ● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●● ●
● ● ●● ●
●
● ●● ● ●
● ●
●●●
●
● ● ●● ●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●●●●
●
● ●
●
●
●●
● ● ●
●●●●●● ● ● ● ● ● ●
●
● ●
●
●●● ● ●●●
● ●● ●
●
● ●
● ● ●
●●
● ●
●● ●● ●●
●
● ●
● ●● ●
●● ●
●
●
●●●● ● ●
●●
●● ●● ●●
●
●● ● ● ● ●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●●
● ●
●●
●
●●
●● ●●●
●
●
●● ●
●● ● ●●
● ●●
●
●
● ●● ●
●●●
● ● ●●
●
●● ● ●● ●
● ●● ● ●●●
●●●● ●●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●●
●●●
● ● ●●● ● ● ●
●
●
●●●●
●
●● ● ●
●
●
●
●●● ●
● ● ● ●
● ●● ●
●●
●
● ●●●● ●● ●●
●●
●● ●●
●●
● ●
●
●●●
●
●
●
●●● ●
● ● ●
●
●
●●●● ●
●●
● ●●●●
●●●
●
●●
●
●● ●
●●
●●
●●●● ●
●●●
●
●●● ●
●● ●●
●●● ●
●●●
●
●
●
●●●
●● ●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●● ● ●●
●● ●
●
●
● ●● ●
●●
● ●
●●●
●●●
● ●●
●
●
●●●●
●● ● ● ●●
●●●
●
●● ●
● ●
●●
●●
●●
●
●●●●●●●● ● ● ●● ●
● ●●●
●
●
●
●●●●● ● ● ●● ●
● ●
●
●
●● ●
●
●● ●
●
● ● ●●● ●●●● ● ●
●
●●●●
●●
●
●● ● ● ●
●●
● ●●●
●●●
●
● ●●● ●●●
●
● ●●
● ●
● ●
●●
●
● ● ●●
●● ●
● ● ●● ●
●
●
●●
●● ● ● ●
●●●
●
●●● ●
●● ● ●●
●●
●
●
●●●●
●●
●
●
● ●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●● ●●
●● ●
●
●● ●
●
●
●●
●
●●
●
2
RT [s]
●
●
●
●
●
1
●
●
●
●
0
●
●
−20
−10
0
utility difference
10
20
Figure 1. Example simulation of the drift-diffusion model (DDM). Response times
(RTs)asafunctionofthedifferenceinutilitiesbetweentwooptionsin900simulatedtrials.
Thegraydotsshowindividualtrials,theblackcirclesdenoteaverageswithbinsofwidth
10.Theparametersusedforthesimulationcorrespondtotheparametersestimatedatthe
group level in the time discounting experiment (𝑏 = 1.33, 𝑧 = 0.09, 𝜏 = 0.11). Utility
differencesaresampledfromauniformdistributionbetween-20and20.
2.3.Modelpredictions
There are several important qualitative predictions of the model that allow us make
inferencesbeyondstructuralestimation.
First, the expected RTs decrease as the utility difference between the two options
lm(no)
becomes larger (
lW
< 0, see Figure 1 for a simulation example). If the drift scaling
parameter and non-decision time are the same (or similar) across subjects, we should
observe a group-level correlation between the RT and the utility function parameter of
interest, conditional on the subjects making the same choice. Thus we should be able to
11
rank subjects according to their individual preferences using RTs from a single decision
problem.WeexplorethispredictioninSection5.3.
More generally, this relationship can be used to rank subjects using sets of choice
problemswheretheyallmadethesamechoice,andsochoicesareuninformative.Consider
anintertemporalchoiceexamplewithtwoindividuals:apatientpersonwhoisindifferent
between$10todayand$12tomorrow,andanimpatientpersonwhoisindifferentbetween
$10todayand$19tomorrow.Thepatientindividualwilllikelytakesometimetodecide
between$10todayand$11tomorrow,butchooseveryquicklybetween$10todayand$20
tomorrow.Ontheotherhand,theimpatientindividualwillstruggletochoosebetween$10
todayand$20tomorrow,butchooseveryquicklybetween$10todayand$11tomorrow.
Notice that in this example both individuals would most likely choose $10 today over
$11tomorrowand$20tomorrowover$10today,basedontheirindifferencepoints.Thus
arevealedpreferenceapproachwouldbeunlikelytomakeadistinctionbetweenthesetwo
individuals based only on these choices. In contrast, the revealed indifference approach
would tell us that a slow choice between $10 today and $11 tomorrow is indicative of
patience, while a slow choice between $10 today and $20 tomorrow is indicative of
impatience.WeconsiderthecaseofuninformativechoiceoutcomesinSection5.4.
Second, long RTs are considerably more informative than short RTs. Sequential
samplingmodelscorrectlypredictthatshortRTscanoccuratanylevelofchoicedifficulty
(i.e.strength-of-preference),butlongRTsalmostexclusivelyoccurfordifficultchoices(i.e.
nearindifference;again,seeFigure1foranexamplesimulation).Theintuitionhereisthat
there is a lot of noise in the decision process and so the quickest choices are those with
noise pointed in the same direction (Ratcliff, Philiastides, & Sajda, 2009; Ratcliff &
12
Tuerlinckx, 2002). Because these sequences of aligned noise occur independently of the
driftrate,thereistypicallyno(oraveryweak)correlationbetweentheshortestRTsand
difficulty (Ratcliff & McKoon 2008). However, the slowest choices are ones where the
combinationofdriftplusnoiseleadstoroughlyzeronetevidenceaccumulation.Giventhat
thesamplednoiseismean-zero,net-zeroevidenceaccumulationisfarmorelikelytooccur
withsmalldrift(difficultproblems)thanwithlargedrift(easyproblems).Thisproducesa
correlationbetweenthelongestRTsanddifficulty.
By taking advantage of this knowledge, we can improve the RT-based estimation of
utility-function parameters (relative to structural estimation, Section 5.5), either by
focusingonthelongestRTsorbyestimatingthepeakinRTsusingthefullsubjectdataset.
ThesemethodsareexploredinSection5.6.
Finally, the model predicts that most “errors”, or choices inconsistent with the utility
function,shouldhappenwhenthesubjectisclosetoindifferenceandthustendstomake
slowchoices.WetestthishypothesisinSection5.7.
2.4.Modellimitations
WithRTs,“slow”isarelativeconceptwhereaschoiceisabsolute.Inordertousethe
revealed indifference approach, we must therefore develop methods to identify slow
decisions for a given individual. Another potential issue is that even though RTs are
continuous,theycanbeinfluencedbyotherfactorsasidefromstrength-of-preferenceand
thusmayappeartobequitenoisy(Krajbichetal.2015a).Forinstance,othershaveargued
thatlongRTsreflectmoreeffortordeliberativethinkingonthepartofthesubject(Gabaix,
13
Laibson, Moloche, & Weinberg, 2006; Hey, 1995; Rubinstein, 2007; Wilcox, 1993). This
couldinterferewithourabilitytouseRTstoinferpreferences.
There are certain characteristics of the choice environment that we believe may
facilitateestimatingpreferencesfromRTs.First,RTsshouldberecordedwithmillisecond
precisionsincemanysimplebinarychoicestakeonlyafewseconds.Second,choicesshould
bemadeusingakeyboardbuttonpressratherthanusingamousetoselectanoption.This
is simply to minimize noise in RTs due to hand movements. Third, we limit ourselves to
binary decisions since indifference is not straightforward with multiple alternatives.
Fourth,forthelastexercise,whereweusetherevealedindifferenceapproachandallthe
trials to estimate utility functions, the range of indifference points implied by the choice
problemsintheexperimentshouldcovertherangeofparametervaluesinthepopulation.
It should be mentioned that SSMs have been successfully applied to datasets that violate
someoftheserequirements(e.g.Krajbichetal.2015b;KrajbichandRangel2011),butsuch
datasetsarenotidealforourcurrentgoals.
Inadditiontotheserequirements,thereareotherfactorsthatmayaffectourabilityto
successfullyinferindifferencepointsfromRTs.Onequestioniswhetherthereshouldbe
constraints on RTs. RT restrictions are common in binary choice tasks in order to keep
subjects focused, but overly restrictive cutoffs may attenuate the effect of strength-ofpreference on RTs (Fudenberg et al., 2015). Here we examine datasets with varying time
constraints(3s,10s,andunlimited)inordertoexploretherangeofsituationsinwhichwe
canemploythemethodofrevealedindifference.Giventheveryfewmissedtrials,andthe
robustness of our results across the datasets, it is unlikely that the constraints were
binding. Another question is whether the method of revealed indifference can be used in
14
situations where the utility function contains multiple parameters. This will obviously
complicatematters,butifoneparameterisrelativelymoreimportantthantheothers,the
methodmaystillwork,aswedemonstrateintheriskychoicetask.
3Relevantliterature
It is important to acknowledge that we are not the first economists to use RT data to
predictbehavior.Inadditiontotheworkmentionedabove,SchotterandTrevino(2014)
and Chabris et al. (2009) have used RTs to predict strategic behavior and aggregate
intertemporalpreferences,respectively.
Schotter and Trevino (2014) use the longest (and second longest) RTs to estimate
threshold strategies in a global game; the RT-based estimates are able to explain out-ofsamplechoicebetterthantheequilibriumprediction.Thispaperissimilarinspirittoours,
but studies strategic situations while we focus on individual preference elicitation. The
authorsarguethatlongRTsintheirsettingreflecteitherdiscoveringtheoptimalthreshold
strategyorexplicitcalculation.Theseconsiderationsandexplanationsarequiteseparate
fromtheideasandmodelsexploredinourwork.
In the preference domain, Chabris et al. (2009) and an earlier unpublished working
paperbythesameauthors(Chabris,Laibson,Morris,Schuldt,&Taubinsky,2008)useda
methodinfluencedbythedirectedcognitionmodel(Gabaix&Laibson,2005;Gabaixetal.,
2006)andfullRT-distributiondataatthegroupleveltoestimateaggregatedintertemporal
preferences using a structural estimation approach. This method aims to identify utility
parameters using the fact that RTs increase with choice difficulty. Their study finds a
correlation between the RT-based estimates and choice-based predictions for three
15
separatelargegroupsofsubjects,butdoesnotattemptanyanalysisofindividualsubjects’
preferences.Intheappendix,weprovideacomparisonbetweenthismethodandourown.
Finally, Clithero (2016b) used a DDM to augment choice data with RT information,
producingbetterout-of-samplepredictionsinfoodchoice.Whilethisworkisveryrelated
toourown,weshowthatRTinformationalonecanbeusedtopredictdecisions.
Our work also relates to a growing literature on incomplete preferences (Agranov &
Ortoleva, 2015; Cettolin & Riedl, 2015; Danan, Ziegelmeyer, & others, 2006; Eliaz & Ok,
2006; Mandler, 2005; Ok, Ortoleva, & Riella, 2012). This literature has shown that when
given the choice between two alternatives, individuals may prefer a lottery over the two
alternatives.Thisproducesprobabilisticchoiceoutcomes,justlikewithSSMs.Moreover,
Danan and Ziegelmeyer (2006) report that a majority of the subjects in their experiment
postponesimplechoicesbetweenlotteries,eventhoughpostponementiscostly.Whilenot
exactlyameasureofRT,thispostponementbehaviorisinlinewiththeslow-indifference
phenomenonthatwereport.
Our work contrasts with the work of Rubinstein, Wilcox, and others (Chen &
Fischbacher,2015;Hey,1995;Recalde,Riedl,&Vesterlund,2014;Rubinstein,2007,2013,
2014; Wilcox, 1993), where long RTs are associated with deliberative thought and short
RTsareassociatedwithintuition.Thesepapershaveprimarilystudiedstrategicsettings
where effort is likely to vary more substantially across subjects and so dominate the
strength-of-preferenceeffectsthatdriveourresults.Whilethiscanbeaccountedforwith
varyinglevelsof𝑏intheDDMthatwehavediscussed,itmayalsobethatotherdecision
processesareinvolvedinstrategicdecisions.
16
4ExperimentalDesign
We analyze four separate datasets: the last two were collected with other research
goalsinmind,butincludedprecisemeasurementsofRTs,whilethefirsttwowerecollected
specificallyforthisstudy.
4.1.Loss-aversionexperiments
These experiments were conducted at The Ohio State University. In each round,
subjects chose between a sure amount of money and a 50/50 lottery that included a
positive amount and a loss (which in some rounds was equal to $0). The set of decision
problemswasadaptedfromSokol-Hessneretal.(2009).Subjects’RTswerenotrestricted.
Intheadaptiveexperiment,eachsubject’schoicedefinedthenexttrial’soptionsusinga
Bayesian procedure (DOSE, see Wang et al. 2010) to ensure an accurate estimate of the
subject’sriskandlossaversionwithinalimitednumberofrounds.Eachsubjectcompleted
the same three unpaid practice trials followed by 30 paid trials. Importantly, every
subject’sfirstpaidtrialwasidentical.Eachsubjectreceivedtheoutcomeofonerandomly
selectedtrial.61subjectsparticipatedinthisversionoftheexperiment,earning$17-20on
average10.
In the non-adaptive experiment, each subject first completed a three-trial practice
followed by 276 paid trials. These trials were presented in two blocks of the same 138
choice problems, each presented in random order without any pause between the two
blocks. Subjects were endowed with $17 and additionally earned the outcome of one
10
Inordertocoveranypotentiallossesincurredduringthistask,subjectsfirstcompletedan
unrelatedtaskthatendowedthemwithenoughmoneytocoverforanypotentiallosses.
17
randomly selected trial (in case of a loss it was subtracted from the endowment). 39
subjectsparticipatedinthisexperiment,earning$18onaverage.Allsubjectsgavewritten
consent,andthestudieswereapprovedbytheOSUInstitutionalReviewBoard.
For both experiments we assumed a standard prospect theory value function
(Kahneman&Tversky,1979):
𝑈 𝑊, 𝐿, 𝑆 =
0.5𝑊t − 0.5𝜆 −𝐿 t
,
𝑆t
where 𝑊 is the gain in the lottery, L is the loss, S is the sure amount, 𝜌 reflects risk
aversion,and𝜆captureslossaversion.Inthenon-adaptiveexperiment,theutilityfunctions
wereestimatedusingastandardMLEapproachwithalogitchoicefunction.
Similartopriorworkusingthistask,wefoundthatriskaversionplaysaminimalrolein
this task relative to loss aversion, with 𝜌estimates typically close to 1. Therefore,
acknowledging that varying levels of risk aversion could add noise to the RTs, for the
analyses below (both choice- and RT-based) we assumed risk neutrality (𝜌 = 1). For the
non-adaptiveexperiment,weusedonlytrialswithnon-zerolosses(specifically,112outof
138 decision problems) for both estimation and out-of-sample prediction. Two subjects
withoutlyingestimatesof𝜆(beyondthreestandarddeviationsofthemean)wereremoved
fromtheanalysis.Thesameexclusioncriterionwasusedfortheotherdatasets.
4.2.Temporaldiscountingexperiment
This experiment was conducted while subjects underwent functional magnetic
resonanceimaging(fMRI)attheCaliforniaInstituteofTechnology(Hare,Hakimi,&Rangel,
2014).In each round, subjectschosebetweengetting$25rightaftertheexperimentora
largeramount(upto$54)atalaterdate(7to200days).Therewere108uniquedecision
18
problemsandsubjectsencounteredeachproblemtwice.All216trialswerepresentedin
random order. Each trial, the amount was first presented on the screen, followed by the
delay,andsubjectswereaskedtopressoneoftwobuttonstoacceptorrejecttheoffer.The
decision was followed by a feedback screen showing “Yes” (if the offer was accepted) or
“No”(otherwise).Thedecisiontimewaslimitedto3seconds,andifasubjectfailedtogive
a response, the feedback screen contained the text “No decision received”. These trials
(2.6%acrosssubjects)wereexcludedfromtheanalysis.Trialswereseparatedbyrandom
intervals(2-6seconds).
41subjectsparticipatedinthisexperiment,earninga$50show-upfeeandtheamount
from one randomly selected choice. The payments were made using prepaid debit cards
thatwereactivatedatthechosendelayeddate.Allsubjectsgavewrittenconsent,andthe
experimentwasapprovedbyCaltech’sInternalReviewBoard.
For this experiment, we used a hyperbolic discounting utility function (Herrnstein,
1981;Laibson,1997):
𝑈 𝑥, 𝑡 =
𝑥
,
1 + 𝑘𝑡
where𝑥isthedelayedamount,𝑘isthediscountfactor(higherismoreimpatient),and𝑡
is the delay period in days. One subject that chose $25 now on every trial was removed
from the analysis. The utility functions were estimated using a standard MLE approach
withalogitchoicefunction.Twosubjectswithoutlyingestimatesof𝑘wereremovedfrom
theanalysis.
4.3.Binarydictatorgameexperiment
19
This dataset was collected while subjects underwent fMRI at the Social and Neural
Systems laboratory, University of Zurich (Krajbich et al. 2015a). Subjects made choices
between two allocations, X and Y, which specified their own payoff and an anonymous
receiver’s payoff. The payoffs were displayed in experimental currency units, and 120
predeterminedallocationswerepresentedinrandomorder.Eachallocationhadatradeoff
betweenafairoption(moreequaldivision)andaselfishoption(withhigherpayofftothe
dictator).72outof120decisionproblemspersubjecthadhigherpayofftothedictatorin
bothoptionsXandY(toidentifyadvantageousinequalityaversion),whiletherestofthe
problems (48/120) had higher payoffs to the receiver in both options (to identify
disadvantageous inequality aversion). In each round, subjects observed a decision screen
that included the two options, and had to make a choice with a two-button box. Subjects
were required to make their decisions within 10 seconds; if a subject failed to respond
under this time limit, that trial was excluded from the analysis (4 trials were excluded).
Intertrialintervalswererandomizeduniformlyfrom3to7seconds.
Subjects read written instructions before the experiment, and were tested for
comprehension with a control questionnaire. All subjects passed the questionnaire and
understoodtheanonymousnatureofthegame.Intotal,30subjectswererecruitedforthe
experiment. They received a show-up fee of 25 CHF and a payment from 6 randomly
chosenrounds,averagingatabout65CHF.Allsubjectsprovidedwrittenconsent,andthe
ethicscommitteeoftheCantonofZurichapprovedthestudy.
To fit choices in this experiment, we used a standard Fehr-Schmidt other-regarding
preferencemodel(Fehr&Schmidt,1999):
𝑈 𝑥j , 𝑥? = 𝑥j − 𝛼 ⋅ max 𝑥? − 𝑥j , 0 − 𝛽 ⋅ max 𝑥j − 𝑥? , 0 ,
20
where𝑥j isthedictator’spayoff,𝑥? isthereceiver’spayoff,𝛼reflectsdisadvantageous
inequality aversion, and 𝛽 reflects advantageous inequality aversion. As described above,
wewereabletoseparatetrialsthatidentified𝛼and𝛽,sothesechoicesweretreatedastwo
separate datasets. The utility functions were estimated using a standard MLE approach
withalogitchoicefunction.Onesubjectwithanoutlyingestimateof𝛼wasremovedfrom
theanalysis.
5Results
5.1.Choice-basedestimations
Thethreeutilityfunctionsweselectedtomodelsubjects’choicesperformedwellabove
chance. To examine the number of choices that were consistent with the estimated
parameter values, we used standard MLE estimates of logit choice functions (for the
estimation procedure details see Appendix A) to identify the “preferred” alternatives in
every trial and compared those to the actual choice outcomes. More specifically, we
calculated utilities using parameters estimated purely from choices, and in every trial
predicted that the alternative with the higher utility would be chosen with certainty. All
subjectswereveryconsistentintheirchoiceseveninthedatasetswithalargenumberof
trials:socialchoice𝛼:94%,socialchoice𝛽:93%,intertemporalchoice:83%,non-adaptive
riskychoice:89%(seeAppendixBforthesubject-levelchoicepredictions).
5.2.Responsetimesreflectchoicedifficulty
ThemethodofrevealedindifferenceispredicatedontheDDM-basedideathatasubject
willtakemoretimetodecideasthedecisionbecomesmoredifficult.Forthefollowing
21
Social choice (β)
3.5
3.5
Social choice (α)
●
3.0
3.0
●
●
●
●
●
●
●
2.5
RT [s]
●
●
●
●
●
●
2.0
2.5
●
●
●
●
1.0
1.5
1.5
2.0
RT [s]
●
−0.4
−0.2
0.0
0.2
0.4
0.6
−1.0
−0.5
0.0
0.5
1.0
subject α − indifference α
subject β − indifference β
Intertemporal choice
Risk choice
1.4
2.5
−0.6
●
●
2.0
●
●
1.0
●
●
●
1.5
●
RT [s]
●
0.8
RT [s]
1.2
●
●
●
●
●
●
1.0
0.6
●
−0.03
−0.02
−0.01
0.00
0.01
0.02
0.03
−4
−2
0
2
subject λ − indifference λ
subject k − indifference k
4
Figure2.RTspeakatindifference.MeanRTinsecondsasafunctionofthedistance
betweentheindividualsubject’sutilityfunctionparameterandtheindifferencepointona
particular trial; data are aggregated into bins of width 0.02 (top row), 0.01 (bottom left
panel), and 1 (bottom right panel), which are truncated and centered for illustration
purposes. Bins with fewer than 10 subjects are not shown. Bars denote standard errors,
clusteredatthesubjectlevel.
analysis, our measure of difficulty was the difference between the subject’s utility
functionparameter(estimatedpurelyfromthesubject’schoices)andtheparametervalue
thatwouldmakeapersonindifferentbetweenthetwoalternativesinthattrial(wereferto
this as the “indifference point”). When a subject’s parameter value is equal to the
indifference point of a trial, we say that the subject is indifferent on that trial and so the
choiceismaximallydifficult.
22
Letusillustratethisconceptwithasimpleexample.Supposethatintheintertemporalchoicetaskasubjecthastochoosebetween$25todayand$40in30days.Ifthesubjecthas
ahyperbolicdiscountingutilityfunction(asweassume),theywouldbeindifferentwithan
individual discount rate𝑘 that is the solution to the equation 25 = 40/(1 − 30𝑘), or k =
0.0125.Thiswouldbetheindifferencepointofthisparticulartrial.Asubjectwiththisk
value would be indifferent on this trial, a subject with a lowerk would favor the delayed
option,andasubjectwithahigherkwouldfavortheimmediateoption.
The bigger the absolute difference between the subject’s parameter and the trial’s
indifference point, the easier the decision, and the shorter the average RT. Indeed, this
effectisobservedinallofourdatasets,withRTspeakingatindifference(Figure2).Mixedeffects regression models (treating subjects as random effects) show strong, statistically
significant effects of the absolute distance between the indifference point and subjects’
individualutilityparameterson log(RTs)11forallthedatasets(fixedeffectofdistanceon
RT:dictatorgame𝛼:t=-7.5,p<0.001;dictatorgame𝛽:t=-9.1,p<0.001;intertemporal
choice𝑘:t=-9.9,p<0.001;non-adaptiveriskychoice𝜆:t=-9.6,p<0.001,adaptiverisky
choice𝜆:t=-4.4,p<0.001).
5.3.One-trialpreferenceranking
Intheadaptiveriskychoiceexperiment(Section4.1),allsubjectsfacedthesamechoice
problem in the first trial. They had to choose between a 50/50 lottery with a positive
amount($12)andaloss($7.5),andasureamount($0).Assumingriskneutrality,asubject
11
A key feature of RTs generated by sequential sampling models is that they are roughly lognormally distributed. Thus it is typical to transform RTs with a natural logarithm before
performingstatisticalanalyses.
23
Risky option chosen
4
●
● ●
●
1.5
2.0
●
8
●
●
●
●●
●
●
●
2.5
●
●
●
3.0
●
●
●
●
●
●
●
●
●
●
●
●
6
●
RT [s]
●
●
●
p < 0.1
●
●
●
● ●
●
●
●
3.5
4.0
●
4
●
●
●
r = 0.41
p = 0.004
●
●
●
6
RT [s]
8
●
10
r = −0.43
●
●
●
●
●
2
10
●
●
2
12
12
Safe option chosen
4.5
0.0
λ estimated from choices
0.5
●
●●
●
●
1.0
●
1.5
2.0
2.5
λ estimated from choices
3.0
Figure3.Preferencerankcanbeinferredfromasingledecisionproblem.RTsinthe
first round of the adaptive risk experiment as a function of the individual subject’s lossaversion coefficient from the whole experiment; Spearman correlations displayed. In this
round,eachsubjectwaspresentedwithabinarychoicebetweenalotterythatincludeda
50%chanceofwinning$12andlosing$7.5,andasureoptionof$0.Theleftpaneldisplays
subjects who chose the safe option, and the right panel shows those who chose the risky
option.ThesolidblacklinesareOLSfits.
with a loss aversion coefficient of 𝜆 = 1.6 should be indifferent between these two
options, with more loss-averse subjects picking the safe option, and the rest picking the
riskyoption.
Because the mean loss aversion in our sample was 2.5 (median = 2.46), most of the
subjects(44outof61)pickedthesafeoptioninthisfirsttrial.Now,ifwehadtorestrictour
experimenttojustthisonetrial,theonlywaywecouldclassifysubjects’preferenceswould
betodividethemintotwogroups:thosewith𝜆 ≥ 1.6andthosewith𝜆 ≤ 1.6.Withineach
groupwewouldnotbeabletosayanythingabouteachindividual’slossaversion.
ByobservingRTswecanestablisharankingofthesubjectsineachgroup.Specifically,
the DDM predicts that subjects with longer RTs would exhibit loss aversion closer to 1.6
(assuming that the threshold, drift, and non-decision time parameters are similar across
24
subjects). We ranked subjects in each group according to their RTs and then compared
thoserankingstothe“true”lossaversionparametersestimatedfromthe30choicesinthe
fulldataset(seeFigure3).
In line with the results of the previous section, RTs peaked around the indifference
point(𝜆 = 1.6).Therewasasignificantrank-based(Spearman)correlationbetweenRTs
andlossseekinginthe“safeoption”group(r=0.43,p=0.004)andbetweenRTsandloss
aversion in the “risky option” group (r = 0.41, p = 0.1). Thus the single-trial RT-based
rankingsalignedquitewellwiththe30-trialchoice-basedrankings.
As mentioned in the introduction, this approach could have great appeal since
economistsandpractitionersareoftenconstrainedwhencollectingdata,hencetheappeal
of adaptive experiments (Blais & Weber, 2006; Cavagnaro, Myung, Pitt, & Kujala, 2010;
Kim,Pitt,Lu,Steyvers,&Myung,2014;Rodriguez,Turner,&McClure,2014b;Wangetal.,
2010,andothers).Ourresultssuggestthatinfactveryfewtrialsmaybeneededtogeta
reliableestimateofsubjects’preferences12.
5.4.Uninformativechoices
Another possible use of RT-based inference is the case where an experiment (or
questionnaire) is flawed in a such way that most subjects give the same answer to the
choice problems (or one might consider a situation where people feel social pressure to
give a certain answer, even if it contradicts their true preference). This is similar to the
situationfromthelastsection,exceptthatsubjects’choicesmaybeevenlessinformative,
andwithmultipletrials,theremaybeevenmoretogainfromexaminingtheRTs.
12
PredictivepowermaybefurtherimprovedbyaccountingforotherpotentialcontributorstoRTs,
suchasdemographics,baselinemotorresponsetime,etc.
25
To model this situation, for each dataset (non-adaptive risk choice, intertemporal
choice, and social choice) we isolated trials with the most extreme indifference point,
wheremostsubjectschosethesameoption(e.g.,thelotteries),andlimitedouranalysisto
those subjects who picked this most popular option. In some instances, this involved
several trials since some of the choice problems were repeated or shared the same
indifferencepoint.
We found that the RTs on these trials were strongly correlated with subjects’
preferenceparametersinallfourdomains(riskychoice:r=0.55,p<0.001;intertemporal
choice:r=0.43,p=0.007;socialchoice𝛼:r=0.4,p=0.03;socialchoice𝛽:r=0.68,p<
0.001,Spearmancorrelations).Againweseethatitispossibletosomewhatreliablyrank
subjectsaccordingtotheirpreferencesintheabsenceofdistinguishingchoicedata.Similar
to the previous section, this method could be used to bolster datasets that are limited in
scopeandsounabletorecoverallsubjects’preferences.
5.5.DDM-basedutilityestimationfromRTs
TheresultsdescribedintheprevioussectionsdemonstratethatwecanuseRTstorank
subjectsaccordingtotheirpreferenceswithoutinformativechoicedata.Inthissection,we
explore ways to estimate individual subjects’ utility-function parameters from their RTs
acrossmultiplechoiceproblems.
The DDM predicts more than just a simple linear relationship between strength-ofpreferenceandmeanRT;itpredictsentireRTdistributions.AsdescribedinSection2,the
DDMapproachassumesthatineachtrial,subjectsmakedecisionsusingaWienerrandom
processthatproducesadistributionofRTsgiventhreefreeparameters,whichcanbe
26
Social choice (β)
0.0
●
−0.5
●
● ●●
● ●
● ●●
●
●● ●●●
1.0
●
●
●
●
−0.5
−1.0
●
●
●
0.0
0.5
1.0
−1.0
−0.5
0.0
0.5
1.0
α estimated from choices
β estimated from choices
Intertemporal choice
Risk choice
0.05
−1.0
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●● ●●
●
●●
r = 0.52
p = 0.003
0.5
●
0.0
●
p = 0.04
●●
−0.5
0.5
r = 0.39
β estimated from RTs
1.0
●
●
● ●
●
●
−1.0
α estimated from RTs
1.5
Social choice (α)
8
● ●
1.5
●
0.00
0.01
r = 0.57
p < 0.000
●
●
0.02
0.03
6
●
●
●
●
● ●
●
●
● ●●
●● ●
●
● ●●●
● ●
●●● ●
●●
●
●
●
●
●
●
●
●
●
●
0
0.02
0.01
●
●●
●
● ●● ●
●
●
●
●●●
●
● ●●
●
●●
● ●
●
●
●
●
p = 0.03
4
●
2
0.03
●
λ estimated from RTs
0.04
●
●
0.00
k estimated from RTs
r = 0.36
●
0.04
0.05
0
2
4
6
λ estimated from choices
k estimated from choices
8
Figure4.TheDDMestimatesofsubjects’utilityfunctionparameters,estimatingDDM
parameters at the group level. Subject-level correlation (Pearson) between parameters
estimated from choice data and RT data using the drift-diffusion model (DDM). The solid
linesare45degreelines.
estimated for each individual subject (Wabersich and Vandekerckhove 2014, see
AppendixAfortheestimationdetails).Inparticular,thedriftrateinthemodelisalinear
functionoftheutilitydifferenceandsobyestimatingdriftrateswecanidentifythelatent
utility-functionparameters.
27
5.5.1.Group-levelDDMestimation
First, we estimated the DDM, assuming that the boundary parameter 𝑏, non-decision
time 𝜏, and drift rate 𝑧 are fixed across subjects (see Section 2.2 and Appendix A for the
modelandestimationdetails).Thisisacommonlyusedtechniquetohelpwithparameter
identificationwithlimiteddatasets,thoughinSection5.5.2.werelaxthisassumption.The
identificationproblemtypicallyoccurswhenRTspeakattheboundaryofanexperiment’s
indifference points. In these cases, it is impossible to say whether the RTs would have
continuedtorisebeyondtheboundaryandsowecannotknowthetruelocationoftheRT
peak. By fixing the other parameters in the model we constrain the distribution of RTs,
allowing us to estimate the expected RT at indifference. Take for example the risk task,
wheremeanRTspeakatapproximately2.1sandthemaximumindifferencelis5.33.Ifwe
observedasubject’sRTsincreasingwiththeindifferencelbutonlyaveraging1.5satl=
5.33,wemightinferthatthesubject’slisnot5.33,butrather7.33(giventherelationship
inFig.2d).
Ineachdataset13,wefoundthatindividualutility-functionparametersestimatedfrom
RTsalone(usingthestructuralDDMmodel,withoutchoicedata)werecorrelatedwiththe
sameparametersestimatedfromthechoicedata(socialchoice𝛼:r=0.39,p=0.04;social
choice𝛽:r=0.52,p=0.003;intertemporalchoice𝑘:r=0.57,p<0.001;riskychoice𝜆:r=
0.36, p = 0.03; Pearson correlations; Figure 4,). These results showed an improvement
overthedirectedcognitionmodelsuggestedbyChabrisetal.(2009)(seeAppendixAfor
estimationdetailsandAppendixEforthemodelcomparisonresults).
13
Wedonotusetheadaptiveriskdatasetinthissectionsincetheadaptivenatureofthetask
createsautocorrelationsinthedatathatwilllikelyinterferewithourestimationprocedure.
28
Another way to assess the goodness-of-fit is to examine the number of choices that are
consistent with the estimated parameter values, as in Section 5.1. To do so, we used the
RT-estimated parameters to identify the “preferred” alternatives in every trial and
compared those to the actual choice outcomes. RT-estimated parameters were able to
explainahighproportionofchoicesinthedatasets(socialchoice𝛼:79%respectively(p<
0.001); social choice 𝛽: 80% (p < 0.001); intertemporal choice: 76% (p < 0.001); risky
choice:77%(p<0.001);p-valuesdenotetwo-sidedWilcoxonsignedranktestsignificance
at the subject level, comparing these proportions to chance). For a stricter test, we
calculated an average of all indifference points for each experiment, which roughly
corresponds to the mean of the experimenter’s prior parameter distribution, and made
choice predictions for each subject using this single value. The DDM accuracy rates beat
thisbaselineintwooutoffourcases(fortheintertemporalchoiceandsocialchoice𝛽,p<
0.05,two-sidedWilcoxonsignedranktest).
5.5.2.Subject-levelDDMestimation
In addition, we estimated the DDM for each subject separately, assuming individual
variabilityintheboundaryparameter𝑏,non-decisiontime𝜏,anddriftrate𝑧.Asdescribed
intheprevioussection,thiscausesidentificationproblemsforcertainsubjects.Therefore,
inthissectionweexcludesubjectswhoseutility-functionparameterestimatesfelloutside
the range of indifference points employed in each experiment. After excluding these
subjects (2/30 and 2/30 in the social choice dataset, 16/39 in the intertemporal choice
dataset,and7/37intheriskchoicedataset)weobtainedcorrelationsthatshowed
29
Social choice (β)
1.5
●
−0.5
1.0
●
0.0
●
●
●●
●●●● ●
●●
●
●
●
●
●
●
● ●●
●
●
●
0.5
r = 0.74
p < 0.001
●
●
●●
●
−1.0
●
●● ●
● ●
●
−1.0
−1.0
●
0.0
0.5
0.0
●
●
●
●
0.5
1.0
1.5
−1.0
−0.5
0.0
0.5
1.0
α estimated from choices
β estimated from choices
Intertemporal choice
Risk choice
●● ● ● ●
● ●●●●
●●
●
●●
●●
1.5
● ●
●
●
●
●
●
●●●
r = 0.63
p < 0.000
●
0.00
6
● ●
4
●
●
●
●●
●●
●●●●
●
●
●●
●
●●●● ● ●
●
● ●●
●
●
0
●
●
●
●
●
●
●
●●
●
●
●
●
●
p = 0.002
2
0.04
λ estimated from RTs
r = 0.53
0.03
0.02
0.00
0.01
k estimated from RTs
0.05
8
0.06
β estimated from RTs
1.0
● ●
●
●
●
p = 0.65
●
●
●
● ●
●
●
● ●
●●
r = 0.09
−0.5
α estimated from RTs
●
−0.5
1.5
Social choice (α)
0.01
0.02
0.03
0.04
0.05
0.06
0
2
4
6
λ estimated from choices
k estimated from choices
8
Figure 5. The DDM estimates of subjects’ utility function parameters, fitting the
model to each subject individually. Subject-level correlation (Pearson) between
parametersestimatedfromchoicedataandRTdatausingthedrift-diffusionmodel(DDM),
including only the subjects with parameters estimated within the range of indifference
pointsintheexperiment(reddottedlines).Thesolidlinesare45degreelines.
improvementoverthepreviousDDMestimationinthreeoutof4cases,withcorrelations
ashighas0.74(Figure5,p<0.01).
Forthe𝛼parameterinthesocialchoicetask,thecorrelationwasinsignificant(Figure5,
p = 0.65), most likely due to the small number of trials (only 48 trials to estimate 4
parameters)andthetightdistributionofsubjects’indifferencepointsinthattask.
30
Theseresultshighlighttheimportanceoftaskdesign.Ifthetask’sindifferencepoints
span the range of indifference points in the population then the DDM method can yield
quiteaccurateutility-functionparameters.
5.6.AlternativeapproachestoutilityestimationfromRTs
The DDM may seem optimal for parameter recovery if that is indeed the data
generatingprocess.However,severalfactorslikelylimititsusefulnessinthissetting.The
DDM has several free parameters that are identified using features of choice-conditioned
RT distributions. Identification thus typically relies on many trials and observing choice
outcomes. Without meeting these two requirements, the DDM approach may struggle to
identifyparametersaccurately.Belowweexplorealternativeapproachestoanalyzingthe
RTs.
5.6.1.TopRTdecilemethod
One alternative approach is to focus on the longest RTs: for instance, Schotter and
Trevino (2014) successfully use the longest RT over a number of trials to identify a
decisionthresholdinasimpleglobalgame.AslongRTsindicateindifference,theycouldbe
usedtoidentifytrialswherethesubjectiveutilitiesofthetwooptionswereequal,andthus
obtain an estimate of the utility function parameter from the indifference points in those
trials.Hereweexplorewhetherthismethodworksinourdatasets.
31
Social choice (β)
10
10
Social choice (α)
●
●
●
●
●
●
● ●●
●● ● ●
●
●
●● ●
●
● ● ●
●
●● ●
●
● ●●
●
●
●
●
● ●●
●
● ●
●
●
● ● ● ●
●
●●●
●
●
●
● ● ●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●●
● ●
● ●●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
● ●
●
●●
●
0.5
1.0
● ●
●
● ●
● ●
●
● ●●
●● ●●
●●●
● ●
●● ● ● ● ●
●
●
●● ● ● ●
●
●
●●
●
●
●
●
●
−0.5
0.0
0.5
1.0
6
●
●
●
●
●
●
●
●
●
1.5
−2
●
●
● ●●
●●
●
●●
● ●
●
●●●
●
●
●●
● ●● ●
● ●●● ●
●● ●●
●
●
●
●
●
● ●●
●●● ● ●
●●
●●●
●
●
●
● ●
●●
● ●●
● ●●
● ●●●
●
●
●
●
●
●
●
●● ●
●●
●
● ●● ●
●
●
●●● ●
●● ●
● ●
●
● ●●●● ●● ●
● ●●●●
●●
●●● ● ●
● ● ● ● ●● ●
●
● ●●●●●●●
● ●●
●●
● ●●●●
●
●
●●
●
●●
● ●● ●●
●●
●●●
●
●●
●●●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
● ●●●●● ●
●
●
●
●
●
●
●
●
●
●●●●● ●
●
●
●
● ●● ● ● ●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
● ● ●
●
●●●●
●●
●●●
● ●●
●●
●●
●
●
●
●
●●
●●●●
● ●●●●
●
●●
● ●●
●
●●
●
●●●
●●
●
●
●●
● ●
●
●●
●
●●●
●
●
●
●●
●
●●
●
●
●
● ●●
●● ● ●
● ● ●●
●●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
● ●
●●
●
●
●● ●
● ●●
●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
●
●
●●●
●●
●●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
● ●●●
●
●
●
●
●
● ●
●
●
● ●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
● ●
●
●
●●
●●
● ●●
● ●
●
●
●●●●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●●
●●
●
●
●
●
●
●●●● ● ● ●
●
●●
●●
●●
●
●
●●
●●●
●
●● ●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
● ●● ●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●● ● ●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
● ●
●
●●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●● ●
●
● ●
●
●
●●
●●●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●●● ● ●●
●
●
●●●●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●● ●●●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●●
● ●●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●● ●●●
●
●●
●
●●
●
●●
●●● ●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●● ●●
●●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●●
●●●●
●
●
●●●
●
●●●
●
●
●●●
● ●
●●
●●●
●●
●●
●
●
●
● ●
●●●
●●
●●●●
●
●●
● ●
●
●
●
●
●● ● ●●●●
●
●
●●
●
●
●
●●
●●
●
●●●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●●●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
● ● ●●
●●● ● ● ●
●
● ●●●
●●
●●
●●
● ●●
●●
●●
● ●●
●●●
●
●
●
● ●
● ●● ●
●●
●●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●●●●
●
●●
●●●
●
●
●
●●
●
●
●●●●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
● ●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●●
● ●●
●
●
●
●●
●●●● ●
●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●●
●●
●
● ●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●● ●
●● ●● ●
●●●●●●
●●●● ●
●
●● ● ● ●
●●●
●
● ●●
●●
●●● ●●●●●
●
●●
●
● ●
●● ●
● ● ●
●
●
● ●
●
−1
0
1
subject β − indifference β
Intertemporal choice
Risk choice
0.00
0.02
●
●
●
●
●
●
30
●
●
●
0.04
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●● ● ● ●●
● ● ● ● ●● ●
●
● ●
●
● ●
●
●
● ● ●
●● ●
●●●
●
●● ●●
●
● ●
●
●
●
●
● ●
●
● ● ●●●●●●
●● ● ●
●
●● ● ●
●
●●
●
●
●●●
●
●●
●
●● ●
●
●
●
●
●●●
●●
● ●●
●
●
● ●●●
●
● ●
●
●
●●●●
●
●
●
●
● ●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●●
●●●●● ●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●● ● ● ●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
● ● ●●● ●●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●● ●● ●●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●● ●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●● ●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●●
●
●
●●
●
●
● ● ●●● ●● ●●
●●●
●
●
●●
●●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●●●
●●
●
●
●●
●●
●
●
●
●
●● ●
●
●
●
●
●●
●●
●
●●
●●
●
●
●●●●
●
●
●●●●
●●●
●
●
●●
●
●●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●●
●●●●●
●
●
0
● ●
−0.02
●
2
subject α − indifference α
●●● ●●
●
●●
●
●
●
●●
● ●
● ●●
●●●
●
●
● ●●
● ●●
●●
● ●●
●●
● ●●● ●●●● ●●
●
●
●●●
●
● ● ● ●● ●
●
●● ●
● ●●● ● ● ●
●●●
●●
●●● ●
● ●●
●
●●
●
●
●
●●
●● ●
●● ● ●
●
●
●●●
●●●
●
●●●
●
●
● ●
● ●
●
●
●●●
● ●
●
●●●
●●
● ●●●●●●
●● ●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
● ●
●●●
●● ●●
●●
●
●
● ●●
●
●●
●
●
●
●
●●
●
●
●● ●
●
●●●
●
●●● ● ●●
●
●
●●
●
● ●
●
● ●
●
● ●
●
● ●●
●
●
●
●
●
●●
●
●●●
●● ●
●
●●●
●●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●● ●●●
●
● ●●
●
●
●● ●●●●
●
●
●
●
●
● ●
●
● ●●
●●
●
●
●
●●
●
● ● ●●
●●
●
●●
●●
●
●
●
●●●●●
●●
●● ●●●
●●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
● ● ●●
●●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●●
●●●
●
●
●
●
●● ●● ●●
●
●●
●●●
●●
●●● ●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●● ●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●● ● ●
●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●●●
●
●
●
●
●
● ●●●
●
●
●
●● ●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●●
●
●●●●
●
●●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●●● ●
●● ●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●●●●
●
●
●●
● ●●●
● ●
●
●
●
●
●
●
●●●
● ●
●●
●● ●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●●
●
●
● ●●
●
●
●
● ●● ● ●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●●
●●
●
●●
●
●● ●●●
●
●
●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●● ●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●●● ●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ● ●●●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●● ●●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●●
● ●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●● ● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●● ● ●
●
●
●●
●
●
●
●
●●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●● ●
● ●
●● ●
●
●●
●
●●●
●●
●●
●
●
●● ●
●
●● ● ●●
●
●●
●
●●
●
●
●●
●●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●● ●
●
●
●
●
●
●
●●
●
●●●
●●● ●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●● ●●● ●
●
●
●●
●
● ●●●
●
●●
●
●
●●●
●●
●●
●
●● ● ●
●
●
●
●●
●
●
●
●● ● ●●●●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●● ●●●
●
●●
●
●●●●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
● ●● ●●
●●●
●
●
●
●
●
●●
●● ● ● ●
●
●
●●●
●
−0.04
●
●
●
●
RT [s]
●●
●
1.5
RT [s]
●●
●
●
●
2.0
●
●
−1.0
●
●
●
●
10
2
2.5
3.0
−1.5
RT [s]
●
●
4
●
●
20
6
●
4
RT [s]
●
●
8
●●
●
●
●
●
●
●
●
●
● ●●●●● ●
●●
●
●
●●
● ●
●
●
●
●
●
● ●
●
● ●
● ●
●● ● ●
●
●
●
●
●
●
●
●
● ●●
●
●
● ●
●● ● ● ● ●● ● ● ●
● ● ●●
●
●
●● ●●
●
●
● ●●● ● ● ● ●
● ●
● ●●
●
● ●●
●
●
● ●● ●
●
●
●
● ●●
●● ● ●
●●
●
●
●
●●
●
●●●
●
●●
●●
● ●●
●
●
●
●● ●●
●
●
●
●
●
●
●● ●
●
●
●
●
●● ● ● ● ● ● ●
● ● ●● ●●
●
●
●●●
●
●
●● ● ●
● ●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●●
● ●●●
● ●
●
●●●●●
●● ●●
●●
●
● ●
●
●
●
●●●●●
●
●●
●●●
●●●●
● ●●
●●
● ●●●●
●●
●
●●
● ●
●
● ●
● ●●
●●
●●
●● ● ●
● ●●
●●●
●
●●
●●●
●
●● ●
●●
●●
● ●
●●●
●● ●
●
●
●●●
●
●● ●
●●
●
●
●●
●
● ●● ●●●
●● ● ●
●
●
●●
●
● ●
●
●●●
● ● ●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●●●
●●●
●●
●
●●● ●
●●
●●
●
●
●●
●
●
●
●● ● ●●
●
●
●●
●
●●
●
●●
● ●
●
●
●●●
●
●●●● ●●
●
●●
● ●●
●
●●
●
●
●●
●
●●
● ●● ●●●●●
●●●
● ●●
●●
●●●
●
●
●●
●●●
●●●● ●
●
●
●
●●
●●
●
●●● ●
● ●
●
●●
●●
●●
●●●
●
●●● ● ●●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●●
●●
●●
●
●●
●
●●●
●
●
●
●●●
●
●
●●
●
●●
●
●
●●
●●
●●●●
●
●●
●●
●● ● ●
●●
●
●
●●
●●
●
●
●
●
●●
●●●
●
●
●
●
●●●
●
●●
●
●●●●
●
●
●●
●●
●
●●●
●●
● ●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
● ●
● ●●●
●
●●
●●●
●●
●
●
●
●●
●
●
●●●
●●
●●
●●
●●●
●●
●●
●●
●●
●●●
●●
●
●●
●●
●●
●
●●●
●
●
●●
●
●●
●
●●●●
●●
●
●●
●
●
●
●
●● ●
●
●●●
●
●
●●
●●
●
●
●●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●●
●● ●●
●
●●
●●● ●
●
●●●
●
●
●
●
●
●
●●●
●●
●
● ●●
●
●●●
●
●
●
● ●
●
●●●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●● ●●
● ●
●● ●
●●●
● ●●
●
● ●●●
● ●●
●● ●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ● ●
●
●●
●●
●
● ●●
●●
●●
●●
●● ●
●
●
●●
●
●● ●
●●
●
● ●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
● ●●
●
●
2
8
●
−6
−4
−2
0
2
4
6
subject λ − indifference λ
subject k − indifference k
Figure6.SinglelongestRTsareanoisypredictorofindifference.RTinsecondsasa
functionofthedistancebetweentheindividualsubjectutilityfunctionparameterandthe
indifference point on a particular trial; gray dots denote individual trials. Red triangles
denotetrialswiththehighestRTforeachindividualsubject.
Figure 6 plots RTs as a function of strength-of-preference for every trial in the
experiments. It is easy to see that even though for many subjects the single slowest trial
providesagoodsignalof“true”preference(asdefinedbythechoice-basedestimation),for
others the longest RT is far from indifference. This is especially true for the risky choice
data (possibly due to the two-parameter utility function or the unrestricted RTs). This
suggestedtousthatitmaybebettertousemorethanthesingleslowesttrial.
32
●
●
●
●
●
2.0
●
●
●
●
●
●
●
1.5
RT [s]
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1.0
●
●
●
●
●
−0.5
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−1.0
●
●
●
●
●
●
●
●
●
●
●
0.0
indifference β
0.5
1.0
Figure7.Exampleofanindividualsubject’sRT-basedparameterestimation.Theplot
shows RTs in all trials as a function of the indifference parameter value on that trial.
ObservationsinthetopRTdecileareshowninred.TheredtriangleshowsthelongestRT
for the subject. The solid vertical red line shows the subject’s choice-based parameter
estimate.ThedottedverticalredlineshowstheaverageindifferencevalueforthetopRT
decile approach. The dotted grey line shows the local regression fit (LOWESS, smoothing
parameter=0.5).
With these factsin mind, we setaboutconstructing a methodforusingRTstoinfera
subject’sindifferencepoint.Clearly,focusingontheslowesttrialswouldyieldlessbiased
estimates of subjects’ indifference points. However, using too few slow trials would
increase the variance of those estimates. We settled on a simple method that uses the
slowest10%ofasubject’schoices,thoughwealsoexploredothercutoffs(AppendixD).
Inshort,ourestimationalgorithmforanindividualsubjectincludesthefollowingsteps:
(1) identify trials with RTs in the upper 10% (the slowest decile); (2) for each of these
trials, calculate the value of the utility-function parameter that would make the subject
indifferentbetweenthetwoalternatives;(3)averagethesevaluestogettheestimateofthe
subject’s utility-function parameter (see Figure 7 for an example and Appendix A for
estimation details). It is important to note that this method puts bounds on possible
33
parameterestimates:theaverageofthehighest10%ofallpossibleindifferencevaluesis
the upper bound, while the average of the lowest 10% of the indifference values is the
lower bound. Thus it is crucial to choose choice problems with enough range in their
indifferencevaluestomorethanspantherangeofvaluesinthepopulation.
Again, the parameters estimated using this method were correlated with the same
parameters estimated purely from the choice data, providing a better fit than the DDM
approach (social choice 𝛼: r = 0.44, p = 0.02; social choice 𝛽: r = 0.56, p = 0.001;
intertemporal choice 𝑘: r = 0.71, p < 0.001; risky choice 𝜆: r = 0.64, p < 0.001; Pearson
correlations; Figure 8, see Appendix A for estimation details for both methods).
Furthermore, these parameters provided prediction accuracy that was better than an
informedbaseline(seeSection5.5)inthreeoutoffourcases(excludingsocialchoice𝛼).In
all cases, a random 10% sample of trials produced estimates that were not a meaningful
predictor of the choice-based parameter values (since these estimates are just a mean of
10% random indifference points).14 As noted previously, the RT-based estimations have
upperandlowerboundsduetoaveragingovera10-percentsampleoftrialsandthusare
notabletocapturesomeoutliers(e.g.seeFigure8,bottomleftpanel).Furthermore,the
numberof“extreme”indifferencepointsinthedatasetsthatweconsideredislow,biasing
theRT-basedestimatestowardsthemiddle.
14
We drew a random 10% sample 1000 times for each individual in each parameter dataset and
estimated the correlation between the average indifference point and the true parameter value
(socialchoice𝛼:meanPearsonr=-0.01;socialchoice𝛽:r=0.01;intertemporalchoice𝑘:r=-0.02;
riskychoice𝜆:r=-0.003).
34
Social choice (β)
1.0
1.0
Social choice (α)
0.5
●
●
−1.0
−1.0
●
0.0
0.0
●
● ●
●
●● ●
●
●
●
●●
● ●
● ●● ●
●●●●
●●
●
● ●
●
●●
● ●● ● ●
●
●●● ●
●●
●● ●
●
●●
●
●
●
p = 0.001
−0.5
β estimated from RTs
0.5
r = 0.56
●
−0.5
α estimated from RTs
r = 0.44
p = 0.02
−0.5
0.0
0.5
1.0
−1.0
0.5
Intertemporal choice
Risk choice
6
1.0
5
r = 0.64
●
● ●
●
● ● ●●●●●●
●● ●●●
● ●
●●●
●
● ●
●
●
●
●
●
● ●
●●
●
4
3
●●
● ●●● ●
●
●●
●●●● ●●● ●
●
●
●
●●
●
●
●● ●●
1
0.02
●
p < 0.000
2
λ estimated from RTs
0.03
p < 0.000
● ●
●
●
● ●
0
0.00
0.01
0.0
β estimated from choices
r = 0.71
k estimated from RTs
−0.5
α estimated from choices
0.04
−1.0
0.00
0.01
0.02
0.03
0.04
0
1
2
3
4
λ estimated from choices
k estimated from choices
5
6
Figure 8. The top RT decile estimates of subjects’ utility function parameters..
Subject-level correlation (Pearson) between parameters estimated from choice data and
response time (RT) data using trials with RTs in the top decile. The solid lines are 45
degrees.ThedottedredlinesshowtheboundsonRTparameterestimations.
5.6.2.Localregressionmethod
Asthetop10%approachonlyutilizespartofthedata,wedevelopedanotherapproach
based on the peaks in RTs, this time using all the available RT data. The local regression
(LOWESS) method also uses the “revealed indifference” approach: for each individual
subject,werunalocalpolynomialregressionofRTsontheindifferenceparametervalues
and use that regression to identify the indifference value that produces the highest
predictedRT(seeFigure7foranexampleandAppendixAfordetails).Asthepeakofthis
35
line is typically close to the choice-based estimate and the observations with the highest
RTs, this method is quite similar in its predictions to the top-RT-decile method. This
methodrequirestheresearchertochoosethelocalregressionsmoothingparameter.Here
weuseavalueof0.5asitproducesthebestresultsacrossallofthedatasets(thoughother
valuescanworkbetterforspecificdatasets;seeAppendixC).
Althoughthisapproachgenerallyproducesresultssimilartothetopdecileapproach,it
can be affected by outliers (e.g. sparse data and unusually high RTs around extreme
indifference points) and thus sometimes misestimates individual subject parameters,
producing correlations that are in some cases worse than those produced by the top RT
decile approach: social choice 𝛼: r = 0.12, p = 0.52; social choice 𝛽: r = 0.6, p = 0.001;
intertemporal choice 𝑘: r = 0.44, p = 0.01; risky choice 𝜆: r = 0.88, p < 0.001; Pearson
correlations.Thiscanbemitigatedbyusingaspecificsmoothingparameterforeachdata
set,butourgoalwastoidentifyamethodthatworkswellacrossallthedatasets.
5.7.Choicereversals
Finally, we wish to explore one additional set of predictions from the revealed
indifference approach. We know that when subjects are closer to indifference, their
choicesbecomelesspredictable.Wealsoknowthatsubjectsgenerallychoosemoreslowly
theclosertheyaretoindifference.Therefore,wehypothesizedthatslowerchoicesareless
predictableandthereforealsolesslikelytoberepeated.
Inallthreedatasets,thechoice-estimatedutilitymodelwassignificantlylessconsistent
withlong-RTchoicesthanwithshort-RTchoices(basedonamediansplitwithinsubject):
80% vs 89% (p <0.001) in the risky choice experiment, 71% vs 79% (p < 0.001) in the
36
intertemporalchoiceexperiment,88%vs94%(p=0.008)and90%vs96%(p<0.001)in
the dictator game experiment; p-values denote Wilcoxon signed rank test significance on
thesubjectlevel.
Asecond,morenuancedfeatureofSSMsisthatthey(mostly)predictslowerrors,even
conditioningondifficulty.Whiletherearesomeexceptions(seeRatcliff&McKoon2008),
the“slowerror”phenomenoniscommonlyobservedinexperimentsandaccountingforit
wasabreakthroughintheliterature(Ratcliff1978).Totestforslowerrorsinourdata,we
ran mixed-effects regressions of choice consistency on the RTs and the absolute utility
difference between the two options. In all cases we found a strong negative relationship
betweentheRTsandthechoiceconsistency(slowerchoice=lessconsistent)(fixedeffects
of RTs: social choice 𝛼: z = -2.62, p = 0.009; social choice 𝛽: z = -3.3, p < 0.001,
intertemporalchoice:z=-5.28,p<0.001,riskchoice:z=-5.35,p<0.001).Thusweindeed
observeevidenceforslowerrorsinallofourtasks.
In two of the datasets (intertemporal choice and non-adaptive risk choice) subjects
faced the same set of decision problems twice. This allowed us to perform a more direct
test of the slow error hypothesis by seeingwhether slow decisions in the first encounter
weremorelikelytobereversedonthesecondencounter.
In the intertemporal choice experiment, the median RT for a later-reversed decision
was 1.36 s, compared to 1.17 s for a later-repeated decision. A mixed-effects regression
effect of first-choice RT on choice reversal, controlling for the absolute utility difference,
was highly significant (z = 4.04, p < 0.001). The difference was even stronger in the risk
choice experiment: subsequently reversed choices took 2.36 s versus only 1.4 s for
subsequently repeated choices. Again, a mixed-effects regression revealed that RT was a
37
significant predictor of subsequent choice reversals (controlling for absolute utility
difference,z=5.2,p<0.001).
6Discussion
Herewehavedemonstratedaproof-of-conceptforthemethodofrevealedindifference.
The method of revealed indifference contrasts with the standard method of revealed
preference, by using response times (RTs) rather than choices to infer preferences. This
newmethodreliesonthefactthatpeoplegenerallytakelongertodecideastheyapproach
indifference. Using datasets from three different choice domains (risk, temporal, and
social)weestablishedthatpreferencesarehighlypredictablefromRTsalone.Finally,we
also found that long RTs are predictive of choice mistakes, as captured by inconsistency
withtheestimatedutilityfunctionandlaterpreferencereversals.
Throughout the paper we have highlighted ways in which we think RTs may be
important for economists. First, using RTs may allow us to estimate agents’ preferences
using very short and simple decision tasks, even a single binary-choice problem (Section
5.3). For example, if you want to know whether people will buy your product for $50,
knowingthattheywouldbuyitfor$30isnotveryusefulinformation.However,ifyoualso
know that they were very quick to say yes at $30, you might reasonably infer that many
wouldstillpurchasetheproductfor$50.
Second,usingRTscanhelpustorecoverpreferenceswhen,evenwithmultiplechoice
problems, some agents always give the same response, meaning that we could otherwise
onlyputboundsontheirpreferences(Section5.4).
38
Third,thefactthatRTscanbeusedtoinferpreferenceswhenchoicesareunobservable
or uninformative (Sections 5.5 & 5.6) is an important point for those who are concerned
about private information, mechanism design, etc. For instance, while voters are very
concernedabouttheconfidentialityoftheirchoices,theymaynotbethinkingaboutwhat
their time in the voting booth might convey about them. In an election where most of a
community’s voters strongly favor one candidate, a long stop in the voting booth may
signal dissent. Another famous example from outside of economics is the implicit
associationtest(IAT),wheresubjects’RTsareusedtoinferpersonalitytraits(e.g.racism)
thatthesubjectswouldotherwiseneveradmittoorevenbeawareof(Greenwald,McGhee,
&Schwartz,1998).Thusprotectingprivacymayinvolvemorethansimplymaskingchoice
outcomes.
Fourth,ourworkhighlightsamethodfordetectingchoiceerrors.Whilethestandard
revealed preference approach must equate preferences and choices, the revealed
indifferenceapproachallowsustoidentifychoicesthataremorelikelytohavebeenmade
bymistake,orattheveryleast,withverylowconfidence.ThusRTsmayplayanimportant
normativeroleinestablishinghowconfidentlywecansaythatagivenchoicetrulyreveals
thatagent’sunderlyingpreference.
InthispaperwehavedescribedaframeworkthatmathematicallylinksRTsandchoices
tounderlyingpreferences.Thisframeworkisnotaheuristic;infact,itarisesastheoptimal
solutiontoasequentialsamplingproblem,wherewhatissampledarestochasticsignalsof
the underlying true preference. In addition to their normative appeal, these SSMs have
enjoyed much empirical success in capturing choice probabilities and RT distributions in
many domains, including economic choice (Fudenberg et al. 2015; Krajbich et al. 2010,
39
2014; Ratcliff and McKoon 2008; Webb 2013; Woodford 2014). Moreover, they are
appealing from a neuroscience perspective, as SSMs are biologically plausible and align
wellwithneuralrecordingsinbothhumansandotheranimals(Bogaczetal.2009;Polania
etal.2014).However,itisimportanttonotethatSSMsarenotuniqueintheirprediction
ofslowdecisionscorrespondingwithindifference.Forexample,thisisalsoafeatureofthe
directedcognitionmodel(Gabaix&Laibson,2005),thoughthatmodelperformedslightly
worseinourmodelcomparison.
OnequestionthatarisesfromourresultsiswhatistheoptimalwaytomakeuseofRTs
inordertoinfersubjects’preferences?Ofthemethodswetested,thetop10%ruleseems
toworkverywellacrossourdatasetsandisaneasymethodtouseinpractice.However,
this cutoff will generally depend on the number of trials in a particular experiment: the
fewerthetrials,thelargerthetoppercentileneedstobe.Foreachofthethreedatasets,we
calculatedtheoptimalpercentilecutoffandfoundthatthetop10-20%RTsgeneratedthe
bestpredictions.Astheindividual-trialRTinformationisnoisy(fasttrialscanbebotheasy
anddifficult,butslowtrialsarealmostalwaysdifficult),usingfewertrialsproduceshigher
variancepredictions,whilemoretrialsmayintroducebias.
ThereareofcourselimitationstousingRTstoinferindifference.Itimportanttokeep
inmindthatotherfactorsmayinfluenceRTsinadditiontostrength-of-preference,suchas
complexity,stakesize,andtrialnumber(Moffatt2005;Krajbichetal.2015b).Aswithany
analysis, it is important to control or account for these factors in order to maximize the
chanceofsuccess.
A second potential criticism of these findings is that we have focused on repeated
decisions which are made quite quickly (1-3 seconds on average) and so may not be
40
representativeof“realworld”decisionsormaybebeingmadeusingsimpleheuristics.We
haveseveralresponsestothiscriticism.First,theseareallmulti-attributechoiceproblems
and so it is unclear what simple heuristics subjects could be using. Second, the use of
simpleheuristicswouldonlyimpairourabilitytoestimatepreferencesfromRTs,sincein
thosecasesthereshouldbenorelationshipbetweenstrength-of-preferenceandRT.The
less people can rely on heuristics and instead have to evaluate the alternatives to
determinewhichisthebest,themoreeffectivethemethodofrevealedindifferenceshould
be.Third,wewouldarguethatmany,ifnotmost,realworlddecisionsareminorvariants
of other decisions that we make repeatedly over the course of our lives. So while these
tasksmaynot,forexample,fullycapturetheprocessofbuyingahouse,theymaybevery
representativeofroutineeconomicdecisions.Finally,whatresearchhasbeendoneonRTs
inone-shot,slowdecisions,issofarconsistentwiththeSSMpredictions.
For example, Krajbich et al. 2015b study a voluntary contribution public goods game
experimentwheresubjectsmadeonlythreedecisionsandtookonaverage43.5stodecide
eachtime.Inthatexperiment,slowdecisionstendedtofavorthelessattractivealternative,
consistentwithbeingclosertoindifference.Inparticular,withalow-benefitpublicgood,
slow contributions tended to be higher, while with a high-benefit public good, slow
contributionstendedtobelower.
More careful analysis is required to distinguish between the SSM and alternative
interpretationsbyRubinsteinandothers(Chen&Fischbacher,2015;Hey,1995;Recaldeet
al., 2014; Rubinstein, 2007, 2013, 2014, 2016),wherelong RTsareassociatedwith more
carefulordeliberativethoughtandshortRTsareassociatedwithintuition.Itmayinfact
bethecasethatinsomeinstancespeopledousealogic-basedapproach,inwhichcasea
41
longRTmaybemoreindicativeofcarefulthought,whileinotherinstancestheyrelyona
SSM approach, in which case a long RT likely indicates indifference. This could lead to
contradictory conclusions from the same RT data; for example one researcher may see a
long RT and assume the subject is very well informed, while another researcher may see
thatsameRTandassumethesubjecthasnoevidenceonewayortheother.Moreresearch
isrequiredtotestwhetherSSMs,whicharedesignedtoteaseapartsuchexplanations,can
besuccessfullyappliedincomplexeconomicdecisions.
42
References
1. Agranov, M., & Ortoleva, P. (2015). Stochastic choice and preferences for
randomization.AvailableatSSRN2644784.
2. Blais, A.-R., & Weber, E. U. (2006). A domain-specific risk-taking (DOSPERT) scale
foradultpopulations.JudgmentandDecisionMaking,1(1).
3. Blurton,S.P.,Kesselmeier,M.,&Gondan,M.(2012).Fastandaccuratecalculations
for cumulative first-passage time distributions in Wiener diffusion models. Journal
ofMathematicalPsychology,56(6),470–475.
4. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice
responsetime:Linearballisticaccumulation.CognitivePsychology,57(3),153–178.
https://doi.org/10.1016/j.cogpsych.2007.12.002
5. Busemeyer,J.R.(1985).Decisionmakingunderuncertainty:Acomparisonofsimple
scalability, fixed-sample, and sequential-sampling models. Journal of Experimental
Psychology:
Learning,
Memory,
and
Cognition,
11(3),
538–564.
https://doi.org/10.1037/0278-7393.11.3.538
6. Busemeyer, J. R., & Diederich, A. (2002). Survey of decision field theory.
MathematicalSocialSciences,43,345–370.
7. Busemeyer,J.R.,&Rapoport,A.(1988).Psychologicalmodelsofdeferreddecision
making.JournalofMathematicalPsychology,32,91–134.
8. Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamiccognitive approach to decision making in an uncertain environment. Psychological
Review,100(3),432–459.
43
9. Caplin, A., & Martin, D. (2016). THE DUAL-PROCESS DRIFT DIFFUSION MODEL:
EVIDENCE FROM RESPONSE TIMES. Economic Inquiry, 54(2), 1274–1282.
https://doi.org/10.1111/ecin.12294
10. Cavagnaro, D. R., Myung, J. I., Pitt, M. A., & Kujala, J. V. (2010). Adaptive design
optimization: A mutual information-based approach to model discrimination in
cognitivescience.NeuralComputation,22(4),887–905.
11. Cettolin,E.,&Riedl,A.(2015).Revealedincompletepreferencesunderuncertainty.
12. Chabris, C. F., Laibson, D., Morris, C. L., Schuldt, J. P., & Taubinsky, D. (2008).
Measuring intertemporal preferences using response times. National Bureau of
EconomicResearch.
13. Chabris, C. F., Morris, C. L., Taubinsky, D., Laibson, D., & Schuldt, J. P. (2009). The
Allocation of Time in Decision-Making. Journal of the European Economic
Association,7(2–3),628–637.
14. Che, Y.-K., & Mierendorff, K. (2016). Optimal Sequential Decision with Limited
Attention.Unpublished,ColumbiaUniversity.
15. Chen, F., & Fischbacher, U. (2015). Response Time and Click Position: Cheap
IndicatorsofPreferences.ThurgauerWirtschaftsinstitut,Universit�tKonstanz.
16. Cleveland, W. S. (1979). Robust locally weighted regression and smoothing
scatterplots.JournaloftheAmericanStatisticalAssociation,74(368),829–836.
17. Clithero,J.A.(2016a).ImprovingOut-of-SamplePredictionsUsingResponseTimes
andaModeloftheDecisionProcess.AvailableatSSRN2798459.
18. Clithero,J.A.(2016b).ResponseTimesinEconomics:LookingThroughtheLensof
SequentialSamplingModels.AvailableatSSRN.
44
19. Dai,J.,&Busemeyer,J.R.(2014).Aprobabilistic,dynamic,andattribute-wisemodel
ofintertemporalchoice.JournalofExperimentalPsychology:General,143(4),1489–
1514.https://doi.org/10.1037/a0035976
20. Danan, E., Ziegelmeyer, A., & others. (2006). Are Preferences Complete?: An
Experimental Measurement of Indecisiveness Under Risk. Max Planck Inst. of
Economics,
Strategic
Interaction
Group.
Retrieved
from
https://www.researchgate.net/profile/Anthony_Ziegelmeyer/publication/5018150
_Are_preferences_complete_An_experimental_measurement_of_indecisiveness_unde
r_risk/links/0c960516d8217911fe000000.pdf
21. DeMartino,B.,Fleming,S.M.,Garret,N.,&Dolan,R.J.(2013).Confidenceinvaluebasedchoice.NatureNeuroscience,16(1),105–110.
22. Dickhaut, J., Rustichini, A., & Smith, V. (2009). A neuroeconomic theory of the
decisionprocess.ProceedingsoftheNationalAcademyofSciences,106(52),22145–
22150.
23. Eliaz, K., & Ok, E. A. (2006). Indifference or indecisiveness? Choice-theoretic
foundationsofincompletepreferences.GamesandEconomicBehavior,56(1),61–86.
https://doi.org/10.1016/j.geb.2005.06.007
24. Fehr,E.,&Schmidt,K.M.(1999).Atheoryoffairness,competition,andcooperation.
TheQuarterlyJournalofEconomics,114(3),817–868.
25. Fiedler,S.,&Glöckner,A.(2012).TheDynamicsofDecisionMakinginRiskyChoice:
An
Eye-Tracking
Analysis.
Frontiers
https://doi.org/10.3389/fpsyg.2012.00335
45
in
Psychology,
3.
26. Frazier,P.,&Yu,A.(2007).Sequentialhypothesistestingunderstochasticdeadlines
(pp.465–472).PresentedattheAdvancesinneuralinformationprocessingsystems.
27. Fudenberg, D., Strack, P., & Strzalecki, T. (2015). Stochastic Choice and Optimal
SequentialSampling.arXivPreprintarXiv:1505.03342.
28. Gabaix, X., & Laibson, D. (2005). Bounded rationality and directed cognition.
Harvard
University.
Retrieved
from
http://pages.stern.nyu.edu/~xgabaix/papers/boundedRationality.pdf
29. Gabaix, X., Laibson, D., Moloche, G., & Weinberg, S. (2006). Costly Information
Acquisition: Experimental Analysis of a Boundedly Rational Model. American
EconomicReview,96(4),1043–1068.https://doi.org/10.1257/aer.96.4.1043
30. Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual
differencesinimplicitcognition:theimplicitassociationtest.JournalofPersonality
andSocialPsychology,74(6),1464.
31. Hare, T. A., Hakimi, S., & Rangel, A. (2014). Activity in dlPFC and its effective
connectivity to vmPFC are associated with temporal discounting. Frontiers in
Neuroscience,8.https://doi.org/10.3389/fnins.2014.00050
32. Hawkins, G. E., Forstmann, B. U., Wagenmakers, E.-J., Ratcliff, R., & Brown, S. D.
(2015). Revisiting the Evidence for Collapsing Boundaries and Urgency Signals in
Perceptual Decision-Making. The Journal of Neuroscience, 35(6), 2476–2484.
https://doi.org/10.1523/JNEUROSCI.2410-14.2015
33. Herrnstein,R.J.(1981).Self-controlasresponsestrength.QuantificationofSteadyStateOperantBehavior,3–20.
46
34. Hey, J. D. (1995). Experimental investigations of errors in decision making under
risk.EuropeanEconomicReview,39(3),633–640.
35. Hunt,L.T.,Kolling,N.,Soltani,A.,Woolrich,M.W.,Rushworth,M.F.S.,&Behrens,T.
E. (2012). Mechanisms underlying cortical activity during value-guided choice.
NatureNeuroscience,15,470–476.
36. Hutcherson,C.A.,Bushong,B.,&Rangel,A.(2015).ANeurocomputationalModelof
AltruisticChoiceandItsImplications.Neuron,87(2),451–462.
37. Johnson,J.G.,&Busemeyer,J.R.(2005).Adynamic,stochastic,computationalmodel
ofpreferencereversalphenomena.PsychologicalReview,112(4),841–861.
38. Kahneman,D.,&Tversky,A.(1979).Prospecttheory:ananalysisofdecisionunder
risk.Econometrica,4,263–291.
39. Kim, W., Pitt, M. A., Lu, Z.-L., Steyvers, M., & Myung, J. I. (2014). A hierarchical
adaptiveapproachtooptimalexperimentaldesign.NeuralComputation.
40. Krajbich, I., Armel, K. C., & Rangel, A. (2010). Visual fixations and the computation
andcomparisonofvalueinsimplechoice.NatureNeuroscience,13(10),1292–1298.
41. Krajbich,I.,Bartling,B.,Hare,T.,&Fehr,E.(2015).Rethinkingfastandslowbased
on a critique of reaction-time reverse inference. Nature Communications, 6, 7455.
https://doi.org/10.1038/ncomms8455
42. Krajbich, I., Hare, T., Bartling, B., Morishima, Y., & Fehr, E. (2015). A Common
MechanismUnderlyingFoodChoiceandSocialDecisions.PLoSComputBiol,11(10),
e1004371.
47
43. Krajbich,I.,Lu,D.,Camerer,C.,&Rangel,A.(2012).TheAttentionalDrift-Diffusion
Model Extends to Simple Purchasing Decisions. Frontiers in Psychology, 3.
https://doi.org/10.3389/fpsyg.2012.00193
44. Krajbich, I., Oud, B., & Fehr, E. (2014). Benefits of Neuroeconomic Modeling: New
Policy Interventions and Predictors of Preference †. American Economic Review,
104(5),501–506.https://doi.org/10.1257/aer.104.5.501
45. Krajbich,I.,&Rangel,A.(2011).Multialternativedrift-diffusionmodelpredictsthe
relationship between visual fixations and choice in value-based decisions.
ProceedingsoftheNationalAcademyofSciences,108(33),13852–13857.
46. Laibson,D.(1997).Goldeneggsandhyperbolicdiscounting.TheQuarterlyJournalof
Economics,443–477.
47. Laming, D. (1979). A critical comparison of two random-walk models for choice
reactiontime.ActaPsychologica,43,431–453.
48. Link, S. W. (1975). The relative judgment theory of two choice response time.
JournalofMathematicalPsychology,12(1),114–135.
49. Loomes, G. (2005). Modelling the stochastic component of behaviour in
experiments:Someissuesfortheinterpretationofdata.ExperimentalEconomics,8,
301–323.
50. Mandler, M. (2005). Incomplete preferences and rational intransitivity of choice.
Games
and
Economic
Behavior,
https://doi.org/10.1016/j.geb.2004.02.007
48
50(2),
255–277.
51. Matejka, F., & McKay, A. (2014). Rational inattention to discrete choices: A new
foundationforthemultinomiallogitmodel.TheAmericanEconomicReview,105(1),
272–298.
52. Milosavljevic, M., Malmaud, J., Huth, A., Koch, C., & Rangel, A. (2010). The drift
diffusion model can account for the accuracy and reaction time of value-based
choicesunderhighandlowtimepressure.JudgmentandDecisionMaking,5(6),437–
449.
53. Moffatt, P. G. (2005). Stochastic choice and the allocation of cognitive effort.
ExperimentalEconomics,8(4),369–388.
54. Mosteller,F.,&Nogee,P.(1951).AnExperimentalMeasurementofUtility.Journalof
PoliticalEconomy,59(5),371–404.
55. Navarro, D. J., & Fuss, I. G. (2009). Fast and accurate calculations for first-passage
times in Wiener diffusion models. Journal of Mathematical Psychology, 53(4), 222–
230.
56. Ok,E.A.,Ortoleva,P.,&Riella,G.(2012).Incompletepreferencesunderuncertainty:
Indecisivenessinbeliefsversustastes.Econometrica,80(4),1791–1808.
57. Oud, B., Krajbich, I., Miller, K., Cheong, J. H., Botvinick, M., & Fehr, E. (2016).
Irrational time allocation in decision-making. Proceedings of the Royal Society B:
BiologicalSciences,283(1822),20151439.https://doi.org/10.1098/rspb.2015.1439
58. Philiastides, M. G., & Ratcliff, R. (2013). Influence of branding on preference-based
decisionmaking.PsychologicalScience,24(7),1208–1215.
59. Ratcliff,R.(1978).Atheoryofmemoryretrieval.PsychologicalReview,85,59–108.
49
60. Ratcliff,R.,&McKoon,G.(2008).Thediffusiondecisionmodel:theoryanddatafor
two-choicedecisiontasks.NeuralComputation,20(4),873–922.
61. Ratcliff,R.,Philiastides,M.G.,&Sajda,P.(2009).Qualityofevidenceforperceptual
decisionmakingisindexedbytrial-to-trialvariabilityoftheEEG.Proceedingsofthe
NationalAcademyofSciences,106(16),6539–6544.
62. Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model:
Approaches to dealing with contaminant reaction times and parameter variability.
PsychonomicBulletin&Review,9(3),438–481.
63. Recalde, M. P., Riedl, A., & Vesterlund, L. (2014). Error prone inference from
response
time:
The
case
of
intuitive
generosity.
Retrieved
from
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2507723
64. Rieskamp, J. (2008). The probabilistic nature of preferential choice. Journal of
ExperimentalPsychology:Learning,Memory,andCognition,34(6),1446.
65. Rodriguez, C. A., Turner, B. M., & McClure, S. M. (2014a). Intertemporal Choice as
DiscountedValueAccumulation.PloSOne,9(2),e90138.
66. Rodriguez, C. A., Turner, B. M., & McClure, S. M. (2014b). Intertemporal choice as
discountedvalueaccumulation.PLoS.ONE,9,e90138.
67. Roe,R.M.,Busemeyer,J.R.,&Townsend,J.T.(2001).Multialternativedecisionfield
theory: A dynamic connectionist model of decision making. Psychological Review,
108(2),370–392.
68. Rubinstein, A. (2007). Instinctive and Cognitive Reasoning: A Study of Response
Times*.TheEconomicJournal,117(523),1243–1259.
50
69. Rubinstein,A.(2013).Responsetimeanddecisionmaking:Anexperimentalstudy.
JudgmentandDecisionMaking,8(5),540–551.
70. Rubinstein, A. (2014). A Typology of Players: between Instinctive and
Contemplative.Retrievedfromhttp://arielrubinstein.tau.ac.il/papers/Typology.pdf
71. Samuelson, P. A. (1938). A note on the pure theory of consumer’s behaviour.
Economica,5(17),61–71.
72. Schotter, A., & Trevino, I. (2014). Is response time predictive of choice? An
experimental study of threshold strategies. WZB Discussion Paper. Retrieved from
http://www.econstor.eu/handle/10419/98843
73. Shadlen, M. N., & Shohamy, D. (2016). Decision Making and Sequential Sampling
from
Memory.
Neuron,
90(5),
927–939.
https://doi.org/10.1016/j.neuron.2016.04.036
74. Sims, C. A. (2003). Implications of rational inattention. Journal of Monetary
Economics,50(3),665–690.
75. Sokol-Hessner,P.,Hsu,M.,Curley,N.G.,Delgado,M.R.,Camerer,C.F.,&Phelps,E.A.
(2009). Thinking like a trader selectively reduces individuals’ loss aversion.
ProceedingsoftheNationalAcademyofSciences,106(13),5035–5040.
76. Spiliopoulos, L., & Ortmann, A. (2014). The BCD of response time analysis in
experimental economics. Available at SSRN 2401325. Retrieved from
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2401325
77. Srivastava, V., Feng, S. F., Cohen, J. D., Leonard, N. E., & Shenhav, A. (2015). First
PassageTimePropertiesforTime-varyingDiffusionModels:AMartingaleApproach.
arXivPreprintarXiv:1508.03373.
51
78. Steiner, J., Stewart, C., & Matějka, F. (2017). Rational Inattention Dynamics: Inertia
andDelayinDecision-Making.Econometrica,85(2),521–553.
79. Stewart,N.,Hermens,F.,&Matthews,W.J.(2015).EyeMovementsinRiskyChoice:
Eye Movements in Risky Choice. Journal of Behavioral Decision Making, n/a-n/a.
https://doi.org/10.1002/bdm.1854
80. Stone,M.(1960).Modelsforchoice-reactiontime.Psychometrika,25(3),251–260.
81. Tajima, S., Drugowitsch, J., & Pouget, A. (2016). Optimal policy for value-based
decision-making.
Nature
Communications,
7,
12400.
https://doi.org/10.1038/ncomms12400
82. Towal, R. B., Mormann, M., & Koch, C. (2013). Simultaneous modeling of visual
saliency and value computation improves predictions of economic choice.
ProceedingsoftheNationalAcademyofSciences,110(40),E3858–E3867.
83. Usher,M.,&McClelland,J.(2001).Thetimecourseofperceptualchoice:theleaky,
competingaccumulatormodel.PsychologicalReview,108(3),550–592.
84. Wabersich, D., & Vandekerckhove, J. (2014). The RWiener package: An R package
providingdistributionfunctionsfortheWienerdiffusionmodel.RJournal,6(1).
85. Wald, A. (1945). Sequential tests of statistical hypotheses. The Annals of
MathematicalStatistics,16(2),117–186.
86. Wang, S. W., Filiba, M., & Camerer, C. F. (2010). Dynamically optimized sequential
experimentation (DOSE) for estimating economic preference parameters.
ManuscriptSubmittedforPublication.
87. Webb,R.(2013a).DynamicConstraintsontheDistributionofStochasticChoice:Drift
DiffusionImpliesRandomUtility(SSRNScholarlyPaperNo.ID2226018).Rochester,
52
NY:
Social
Science
Research
Network.
Retrieved
from
http://papers.ssrn.com/abstract=2226018
88. Webb,R.(2013b).Dynamicconstraintsonthedistributionofstochasticchoice:Drift
Diffusion implies Random Utility. Available at SSRN 2226018. Retrieved from
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2226018
89. Wilcox, N. T. (1993). Lottery choice: Incentives, complexity and decision time. The
EconomicJournal,1397–1417.
90. Woodford, M. (2014). Stochastic Choice: An Optimizing Neuroeconomic Model.
American
Economic
Review,
104(5),
495–500.
https://doi.org/10.1257/aer.104.5.495
91. Woodford,M.(2016).OptimalEvidenceAccumulationandStochasticChoice.
53
ONLINEAPPENDIX
A. Parameterestimationmethodology
Choice-basedmethod
Weestimateeachindividualutilityfunction𝑢(⋅ |𝜃),where𝜃isasubject-specificparameter
described in Section 3, in the standard way as follows. We assume that for each pair of
options and choice 𝑎 = 1,2 the error terms in utilities follow the type I extreme value
distribution,sotheprobabilityofchoosingoption1isalogisticfunction
𝑝(𝑎j = 1) =
1
1 + 𝑒 R•
‚ƒ (⋅|„)R‚O (⋅|„ )
,
where 𝜇 and 𝜃 are free parameters that can be estimated for each subject individually
maximizingalikelihoodfunction
𝐿𝐿 =
(log 𝑝(𝑎e = 1) ⋅ 1 𝑎† = 1 + log 1 − 𝑝 𝑎e = 2
⋅ 1(𝑎† = 2)),
e
where𝑛isthetrialnumber,𝑎e isthechoicemadebythesubjectonthattrial,and1(⋅)is
theindicatorfunction.
Appendix B shows the subject level correlations between the predicted and the actual
choices.
Top10%responsetimemethod
For each decision problem on each trial 𝑛, we calculate the indifference parameter value
𝜃eje‡ asasolutiontotheequation
𝑢9 ⋅ 𝜃e = 𝑢< ⋅ 𝜃e .
54
ThenweaveragetheindifferencevaluesonthetrialsinthetopRTdeciletoobtainthefinal
parameterestimate:
𝜃=
je‡
e(𝜃e
⋅ 1(𝐹(𝑅𝑇e ) ≥ 0.9))
,
e 1(𝐹(𝑅𝑇e ) ≥ 0.9)
where 𝑅𝑇e is the response time on trial 𝑛, 𝐹(⋅) is the empirical RT distribution for the
specificsubject,and1 ⋅ istheindicatorfunction.
Localregression(LOWESS)method
As in the previous method, for each decision problem on each trial 𝑛, we estimate the
indifferenceparametervalue𝜃eje‡ solvingtheequation
𝑢9 ⋅ 𝜃e = 𝑢< ⋅ 𝜃e .
For each individual subject, we regress response time log(𝑅𝑇) in every trial 𝑛 on the
corresponding indifference parameter value 𝜃eje‡ using a local polynomial regression
(LOWESS,Cleveland1979)intheRpackagestats:
𝑅𝑇 = 𝑓 𝜃eje‡ + 𝜀e .
We set the smoothing parameter to 0.5 as it has provided the best prediction accuracy
acrossallfourdatasets(seeAppendixC).
Then we obtain the parameter estimate 𝜃 by inverting the fitted regression line at the
maximumpredictedresponsetime𝑅𝑇:
𝜃 = 𝑓 R9 max 𝑅𝑇 .
55
Drift-diffusionmodel(DDM)method
In the DDM (see Section 2) a latent decision variable evolves over time with an average
drift rate plus Gaussian noise (the Wiener diffusion) until it reaches one of two predeterminedboundaries,whichcorrespondtothetwochoiceoptions..Giventheboundary
separation parameter, the drift rate, the non-decision time (the component of RT not
attributable to the decision process itself, e.g. moving one’s hand to indicate the choice),
and the variance of the Gaussian noise, it is possible to calculate choice probabilities and
choice-contingentRTdistributions.
Inourparticularcaseweassumethatchoicesareunknown,andsowecanonlyuse
thecombined(summed)RTdistributiontoestimatethefreeparametersofthemodel.We
assumethatallsubjectssharethesameconstantboundaryparameter𝑏,non-decisiontime
𝜏,anddriftrateparameter𝑧,whichmultipliestheutilitydifferenceoneverysingletrial:
𝑣 ≡ 𝑧 ⋅ 𝑢9 ⋅ 𝜃 − 𝑢< ⋅ 𝜃 .
We use a density function of the Wiener distribution from the RWiener R package
(WabersichandVandekerckhove2014)toestimatethelikelihood(9)fortheobservedRT
on every given trial assuming a set of parameters (𝑏, 𝜏, 𝑧, 𝜽), where 𝜽 is a vector of
individualsubjects’parameters.Essentially,theidentificationoftheindividualparameters
ispossibleduetothefactthatRTsarepredictedtovaryastheutilitydifference𝑣varies
acrosstrialsandsubjects.
Chabrisetal.(2009)method
Here we follow the method suggested by Chabris et al. (2009), which uses the full RT
distributiontoestimatetheutilityfunctionparameters.
56
Letthedifferenceinthetwoutilitiesbe
𝛥† ≡ 𝑢9† ⋅ 𝜃 − 𝑢<† ⋅ 𝜃 .
Assumethatthedecisiondifficultyisaconvexanddecreasingfunctionofthisdifference:
𝛤 𝛥† ≡
2
,
1 + 𝑒 •Ž[
where𝜔isafreeparameter.
The response times are then modeled as a function of the trial number and the choice
difficulty:
𝑅𝑇† = 𝛽• + 𝛽9 𝑡 + 𝛽< 𝛤 𝛥† + 𝜀† .
To estimate the set of parameters (𝜃, 𝜔, 𝛽• , 𝛽9 , 𝛽< ) we follow the original paper and
minimizetheerrorfunction
(𝑅𝑇† − 𝛽• − 𝛽9 𝑡 − 𝛽< 𝛤 𝛥† ).
†
UnlikeChabrisetal.(2009),weestimatethismodelforeveryindividualsubject.
57
B. Choice-basedfits
Social choice (β)
0.1
●
●●●●
●●●● ●
r = 0.91
●
0.6
●●●●
0.2
0.3
0.4
0.5
r = 0.99
●
p < 0.000
●●●● ●
0.6
●
0.0
0.2
0.4
0.6
0.8
P(fair) in data
P(fair) in data
Intertemporal choice
Risk choice
1.0
0.1
0.4
●
0.0
0.2
0.4
0.6
0.8
●
●●●
●●
●●
●
●●
●●
●
● ●
●
●
r = 0.99
p < 0.000
0.0
0.0
0.2
● ●
● ● ●●
●
●
●
●
r = 0.97
●
●
● ●●
p < 0.000
●●
●
●●
●●●
0.6
0.6
●●
●
● ● ●●
●
● ●
●
●
1.0
●●●
●●
●
0.4
0.8
P(safe) estimated from choices
●
●●
●
● ●●
●
0.2
1.0
0.0
P(later) estimated from choices
●●●●
●
0.0
p < 0.001
0.8
1.0
0.2
●●
●
●●
●●
0.4
0.3
●
●
●●
0.2
P(fair) estimated from choices
0.5
0.4
●
0.0
P(fair) estimated from choices
0.6
Social choice (α)
0.8
1.0
0.0
P(later) in data
0.2
0.4
0.6
P(safe) in data
0.8
1.0
FigureB1.Subject-levelcorrelation(Pearson)betweenchoiceproportionsinthedataand
as predicted by choice-estimated utility functions (see Appendix A for details). The solid
linesare45degrees.
58
C. Local regression (LOWESS) method choice prediction accuracy for various
levelsofthesmoothingparameter
Social choice (β)
0.85
0.80
prediction accuracy
0.90
0.85
0.80
prediction accuracy
0.90
0.95
Social choice (α)
0.4
0.6
0.8
1.0
0.0
0.4
0.6
smoothing parameter
Intertemporal choice
Risk choice
0.8
1.0
0.8
1.0
0.85
0.75
0.80
prediction accuracy
0.80
0.75
0.70
prediction accuracy
0.2
smoothing parameter
0.90
0.2
0.85
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
smoothing parameter
0.2
0.4
0.6
smoothing parameter
Figure C1. Choice prediction accuracy as a function of the smoothing parameter of the
LOWESS regression model. The solid black lines denote mean prediction accuracy across
subjects,theshadedareasshowstandarderrorsatthesubjectlevel.
59
D. ExploringtheoptimalcutoffforthetopRTestimationmethod
Social choice (β)
0.90
0.85
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0.75
0.80
prediction accuracy
0.90
0.85
0.80
prediction accuracy
0.95
Social choice (α)
0.80
0
20
40
60
80
100
0
20
40
60
top percentile
top percentile
Intertemporal choice
Risk choice
100
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
0.82
0.80
0.65
0.76
0.78
prediction accuracy
0.75
0.70
prediction accuracy
0.84
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
80
0
20
40
60
80
100
0
top percentile
20
40
60
top percentile
80
100
Figure D1. Averaging the longest 10-20% RT trials provides the best choice
predictionaccuracy.Choicepredictionaccuracyasafunctionofthepercentageofslowest
trialsusedintheparameterestimationfrom1to100%.Thesolidblacklinesdenotemean
predictionaccuracyacrosssubjects,theshadedareasshowstandarderrorsatthesubject
level,theredlinesabovethegraphsindicatesignificantdifferencefromthebaselineatthe
p = 0.05 level (Wilcoxon signed rank test). The baseline is the average of all indifference
60
pointsacrosstrials.Itisimportanttoemphasizethatthebaselinetowhichwecomparethe
predictive power is not chance (50%) as almost any experimenter uses some prior
knowledgeoftheparameterdistributioninthepopulationtoselecttheirchoiceproblems.
For example, an experimenter studying intertemporal choice might select a set of choice
problemssothattheaveragesubjectwouldchoosetheimmediateoptionhalfofthetime
andthedelayedoptiontheotherhalfofthetime.Soifyouweretoaveragetheindifference
pointsfromthetrialsinsuchanexperiment,youwouldbeabletopredictbehaviorquite
accurately,onaverage.Insuchanexperiment,behaviorintrialswithextremeindifference
pointswillbeverypredictable.Thatis,onatrialdesignedtomakeaverypatientsubject
indifferent, most subjects will have a strong preference for the immediate option.
Similarly, on a trial designed to make an impatient person indifferent, most subjects will
haveastrongpreferenceforthedelayedoption.Thusbehaviorinmanyofanexperiment’s
trialsisquiteeasytopredictbecausethosetrialsareonlyincludedtoidentifyparameter
values for extreme subjects. For instance, a single loss-aversion coefficient of 𝜆 = 2 can
predictabout75%ofchoicesinourrisky-choicedataset.
61
E. Modelcomparisonresults
DDM
Chabrisetal(2009)
Top10%RT
LOWESS(0.5)
Socialchoice(𝛼)
0.39
0.22
0.44
0.12
Socialchoice(𝛽)
0.52
0.53
0.56
0.6
Intertemporalchoice
0.57
0.59
0.71
0.44
Riskchoice
0.36
0.16
0.64
0.88
PearsoncorrelationbetweenparametersestimatedfromchoicesandRTs,atthesubject
level.Thebestperformingmethodforeachdatasetisshowninbold.Forestimation
methodsdetailsseeAppendixA.
62
F. Instructionsfortheriskexperiment
Instructions
Thankyouforparticipatingintoday’sstudy.
Pleasecarefullyreadthematerialonthefollowingpagestounderstand
• Therules
• Thedecisionsyouwillbemakingtoday
Ifyouhaveanyquestionsafterreadingtheseinstructionsorduringtheexperiment,please
askthembeforetheexperimentorduringthedesignatedbreaks.
Therules
• Pleasechecknowtoensurethatyourmobilephoneisonsilentmodeandputitin
yourbagorpocket.
• Pleasedonottalkduringtheexperiment.
Thestudy
Todayyouwillbemakingaseriesofchoices,andyourfinalpaymentwilldependonlyon
yourownchoicesandchance.
Payment
Yourpaymentwillconsistoftwoamounts:
• Afixedendowmentof34experimentalcurrencyunits(ECUs)thatyouaregivenat
thebeginningofthestudy.
• Your earnings from one randomly selected choice round. You may earn additional
moneybeyondthe34ECUs,oryoumaylosesomeofthat34ECUs,dependingon
yourchoicesandonchance.Theminimumamountofmoneyyoucanearntodayis
10ECUs=$5.
• All the amounts in today’s study will be shown in ECUs and will be converted to
dollarsattheendofthestudyatarateof2ECUs=$1.
63
Yourchoices
In each round of the experiment you will be asked to make a choice between one of two
options. Option one consists of two possible amounts, each one with a probability of 50%.
Optiontwoconsistsofoneamount,withaprobabilityof100%.
Belowisanexampledecisionscreen.InthisroundOptionone(ontheleft)consistsofagain(in
green)of30ECUsandaloss(inred)of10ECUs.Ifyoupickthisoption,thecomputerflipsafair
digital coin (chances are 50-50). In case of heads, you would earn 30 ECUs on top of your
endowment. In case of tails, you would lose 10 ECUs, which would then be subtracted from
yourendowment.Tochoosethisoption,youwouldpress1.
Optiontwo(ontheright)isasuregainof15ECUs.Ifyoupickthisoption,youwouldearn15
ECUsontopofyourendowment.Tochoosethisoption,youwouldpress2.
30
vs
15
-10
1
2
Yourfinalearningswillonlydependononeofyourchoices:attheendofthestudy,onlyoneof
theroundswillberandomlyselectedforpayment.Theoutcomefromthisroundwillbeadded
orsubtractedfromyourinitialendowment.
Ifyouhaveanyquestions,pleaseraiseyourhandnow.
64