One Sided Matching: Choice Selection With Rival Uncertain
Outcomes∗
David B. Johnson
[email protected]
Matthew D. Webb
[email protected]
July 8, 2015
Preliminary, comments welcome.
Abstract
We examine decision making in the context of one sided matching: where individuals
simultaneously submit several applications to vacancies, each match has an exogenous
probability of forming, but each applicant can only fill one vacancy. In these environments individuals choose among interdependent, rival, uncertain outcomes. We design
an experiment that has individuals choose a varying number of interdependent lotteries
from a fixed set. We find that: 1) with few choices, subjects make safer and riskier
choices, 2) subjects behave in a manner inconsistent with expected utility maximizing
behavior. We discuss these findings in the context of college application decisions.
JEL classification: C9, D8, D81
Keywords: Decision Making, Uncertain Outcomes, One Sided Matching, Online Experiment, College Application
∗
We are thankful to Steven Kivinen, Steve Lehrer, Rob Oxoby, John Ryan, Tim Salmon, Derek Stacey,
Ryan Webb, and Lanny Zrill for very helpful discussions and suggestions. All mistakes are our own. The
majority of this research was done at the University of Calgary and we are grateful for the time spent there.
We are also thankful for helpful comments from audience members at Gettysburg College, University of
Central Missouri, and the 49th Annual Meeting of the Canadian Economics Association. Webb’s research
was supported, in part, by a grant from the Social Sciences and Humanities Research Council.
1
1
Introduction
We explore decision making in an environment in which there are rival, uncertain, outcomes using an online experiment. The college application process is an example of such an
environment. From a decision making standpoint, these decisions are distinct from many
other environments, as individuals select a portfolio of applications (i.e., choosing which
schools are sent applications) but may only attend one. These decision are also rival, while
many applications can be sent only one match can be formed. Additionally, the outcomes
are uncertain, as a successful match occurs only with an exogenous probability. Although
very little is known about decision making in this environment, past research about college
applications has assumed that agents maximize expected utility.1 We find evidence of behavior inconsistent with expected utility maximization.
We discuss how decisions would be made if individuals follow different strategies such as expected utility maximization and maxi-min. We then propose an alternative strategy which
depends on a focal outcome. In the context of our experimental set up we discuss what
decisions would be made under each of these strategies. We then discuss the experiment,
posit some hypothesis, and discuss the results of the experiment.
In our experiment, subjects select a given number of lotteries from a fixed set of lotteries.
In our primary experiments subjects are only paid based on the lottery with the most favorable ex-post outcome, which makes outcomes rival. We vary the number lotteries subjects
can select while holding the set of lotteries constant. Results of the experiments show that
when subjects are given few choices, they pick safer and riskier lotteries. This is inconsistent
with the theoretical prediction of expected utility maximization, where individuals would
pick only riskier lotteries. These results cannot be rationalized using standard risk aversion
or MAXMIN preferences. Rather than diversifying the portfolio to include riskier lotteries
(as predicted by expected utility maximizing), a plurality of subjects select a portfolio (referred to as ‘bundles’ hereafter) of lotteries minimizing a relative variance in expected utility.
As an application we discuss these results in the context of college application decisions.
We propose that choice restriction (due to costly nature of applications) may partly explain
the application behavior of low income students. These students apply to fewer schools,
and under apply in terms of selectivity (Hoxby and Avery, 2013). Our results suggest that
recent policy reforms designed to decrease the cost of applications for low income students
should be welfare improving.
1
See for instance Chade et al. (2013) and Fu (2014).
2
2
Theoretical Discussion, Experimental Design, and Hypotheses
We design an experiment to study decision making with rival uncertain outcomes. We do so
using lotteries in which people earn only the top winning prize from all lotteries they select.
Here, the rival outcomes require the subject to take the maximum payout. This is realistic
as one normally takes the best date to the dance, accepts the best job offer, or enrolls in the
best school. Although different in many regards, each of these are cases of rival outcomes.
We first investigate how an expected utility maximizer would make selections, then explore
two rival strategies. We then discuss the experiment in detail, and propose hypotheses.
2.1
Theory
To develop hypotheses, we begin with a theoretical discussion. Decision maker j’s utility is
determined by the outcome of a subset of lotteries (l) chosen from the set L. The lotteries
in L vary by their unconditional probability (pi ) of paying a prize (wi ).2 j is not allowed
to pick the same lottery twice.
2.1.1
Expected Utility Maximization
The expected utility maximizing (EU) strategy assumes j chooses a bundle to maximize her
expected utility. The rival nature of the lotteries complicates the calculation of j’s expected
utility. As an example, assume j has two rival choices (k = 2). In this case (Equation 1),
her expected utility is the probability of the first lottery having a favorable outcome times
the utility of the prize, plus the probability of the first prize being unfavorable for her times
the probability of the second lottery being favorable times the utility of the second prize.3
E[Ui (k|k = 2)] = p1 v(w1 ) + (1 − p1 )p2 v(w2 )
where
v(wi ) = wiα , and α > 0.
More generally, with k choices, the expected utility of a bundle can be expressed as:
!
j
k
X
Y
E[Ui ] =
(1 − pj−1 )pj v(wj ) ,
j=1
(1)
(2)
i=1
where k is the number of choices and p0 = 0 .
With a few assumptions we can show that if j maximizes expected utility, the initial lottery
selection with k = 1 will the lottery with the max E[l]. With k ≥ 1 the bundles will include
2
We say “unconditional” because, in the primary treatments, subject j is only paid for the outcome of,
at most, one of the lotteries - the one most favorable to her.
3
All examples impose exponential utility. However general results are also robust to CRRA specifications.
The only differences being the spread of riskiest to safest lotteries and the utility maximizing lottery.
3
the choice with k = 1 and riskier lotteries.
Proposition 1 Assuming a symmetric distribution of expected values, increasing the number of choices will lead to j choosing increasingly risky lotteries.
A sketch of the intuition for proposition 1 is provided in Appendix A. Although the probabilistic reasoning of EU can be misleading, the intuition behind it is quite simple. To
illustrate, consider adding a lottery (lr ) to an existing bundle of k lotteries, Bk , and denote
the new lottery bundle as Br . Lottery lr is riskier and has a higher payoff, if successful,
than any other lottery in Bk . Alternatively, consider adding a lottery ls to the existing bundle, Bk , and denote this new bundle Bs . Lottery ls has the same expected utility as lr but
is the safest lottery in Bs . Specifically, it has a lower prize but a higher probability of success.
Because the lotteries are interdependent, we can think of the payoff function as having a set
of stopping rules, which begins by sorting all of the selected lotteries in bundles Br and Bs
by their payoffs, if favorable, in descending order. For the highest payoff lottery, j realizes
the outcome of the lottery with certainty (or probability 1). If the outcome is favorable,
the player receives this payoff, the game stops, and no other outcomes are realized. If the
outcome is unfavorable, the process repeats for the next highest payoff. Because lr is the
lottery in Br with the maximum payoff, its outcome is realized with certainty. Additionally,
it does not alter the independent probabilities of the other lotteries in Br , just the probability
that the outcomes of these lotteries are realized. On the other hand, ls is only played if
every other lottery in Bs fails, which occurs with probability,
ω=
k
Y
(1 − pk−1 ).
(3)
i=1
As such, lr in expectation yields pr v(wr ) with probability 1 and ls in expectation yields
ps v(ws ) with probability ω. Therefore it follows if pr v(wr ) = ps v(ws ), then pr v(wr ) >
ωps v(ws ) since ω < 1. Accordingly, adding a riskier lottery increases the expected payoff
by more than adding a safe lottery with the same expected value.
Solving for the optimal bundle as k increases is burdensome as there are N “choose” k permutations of lottery selections.4 Instead, we solve for the optimal bundle in our experiment
numerically, by calculating the subsets of 2, 4, and 6 lotteries that maximize j’s expected
utility.
Figure 1 demonstrates how risk preferences affect the optimal bundle. Following EU, when
α increases, the selected portfolios j shift in the direction of riskier lotteries and the choices
are less disperse. Conversely, as α decreases the selected portfolios shift towards safer lotteries and the dispersion increases. That is, even with rather risk averse preferences, j will
still select a bundle of risky and safe lotteries.
4
The set of lotteries, L, in the experiment can be found in the Experiment Instructions (Appendix G).
4
Figure 4 illustrates other predictable results: (1) Increasing the number of choices increases
expected utility. Adding a lottery gives positive probability of winning a specific amount.
If the new lottery is greater than the current highest paying lottery in the existing lottery,
the additional, higher paying lottery simply gives the decision maker a chance at a higher
prize. In the event of unfavorable outcome of the new lottery, the conditional expected
utility is unchanged in comparison to the previous bundle. (2) Increases in the number of
choices increases the average riskiness in the lotteries j selects; and (3) as j becomes more
risk loving j selects a relatively greater number of risky lotteries.
Figure 1: Expected Utility Maximizing Choices under Varying Risk Preferences
Notes: RL and RA indicate a risk loving (α = 2) and risk averse (α = .5) decision maker.
2.1.2
Variance Minimization (VMIN)
Alternatively, j may select lotteries in reference to her favorite lottery (l1? ). If j is more risk
loving, l1? is riskier. If she is more risk averse, l1? is safer. Subsequently selected lotteries
are then based upon the difference in the expected utility between l1? and the next selected
lottery.
In other words, j is selecting her k favorite lotteries from the set L where L is defined as
L = {l1 , l2 , . . . , ln }
with payoffs
li = pi v(wi ).
Her selected lotteries are a subset of L made up of k elements such that l? ∈ L, with
l? = {l1? , l2? , . . . , lk? }.
5
Therefore, the first step in j’s decision process identifies the lottery which maximizes her
utility and satisfies
l1? = max{l1 , l2 , ..., ln }.
(4)
Once she selects l1? , j then selects the lottery most similar to her first choice in expected
utility, selecting the lottery offering the second highest expected utility. This minimizes the
absolute difference between the expected utility of her first choice and her second choice,
such that
l2? = min{|l1 − l? |, |l2 − l? |..., |ln − l? | : l2? 6= l? }.
She goes through this process k times (or the given number of choices) until she arrives at a
bundle that minimizes the sum of the differences in payoffs between l1? and her subsequent
choices,
lk? = min{|l1 − l? |, |l2 − l? |..., |ln − l? | : lk? 6= lk?−1 }.
(5)
We again simulate choice bundles that minimize VMIN as a function of the given number
of choices, while also varying risk preferences. The results are in Figure 2. As with EU,
risk preferences shift the initial lottery that j picks. That is, more risk averse subjects pick
a safer initial lottery and risk loving agents select a riskier initial lottery. However, there
are substantive differences between the two decision rules. Most notably, under VMIN,
j selects roughly equal numbers of lotteries that are riskier and that are safer than their
initial pick. Under EU this is not the case, since j only selects lotteries that are riskier.
Interestingly, VMIN does not take into account the rival nature of the lotteries, but instead
treats lotteries as “next best” outcomes.
2.1.3
Maxmin Utility
Finally, j may decide to choose her lotteries such that she maximizes the worst possible
outcome. This is the familiar MAXIMIN utility function as in Viappiani and Kroer (n.d.)
Figure 2: VMIN Maximizing Choices under Varying Risk Preferences
Notes: RL and RA indicate a risk loving (α = 2) and risk averse (α = .5) decision maker.
6
(MAXMIN). If this is the case, we expect j to select the lottery which offers the safest
outcome. In our experimental design, one lottery has a certain payoff. This lottery would
always be selected under a MAXMIN strategy. More formally, j selects the lottery which
maximizes her minimum utility,
l∗ = arg max min u(l; U ).
l∈L u∈U
(6)
However, this only gives us information about one of j’s selection while not offering us much
in the way of j’s other selections. We assume that if j uses MAXMIN, after she makes her
she makes her first choice, she will select her next choice using the same strategy she used
for her first choice. This means j will select the lottery, of the lotteries remaining, offering
the safest outcome. j will go through this k times and end up selecting the k safest lottery.
Figure 3: MAXMIN Maximizing Choices under Varying Risk Preferences
2.1.4
A Comparison of Strategies
Figure 4: Optimal Choices by Subset Size and Strategy
In Figure 4, we compare optimal bundles given the choices faced by subjects in our experi7
ment, according to the assumed strategy and the number of choices they are given (k). We
assume subjects are risk neutral. This implies that if the individual only had one choice,
their optimal choice (ignoring MAXMIN) would be to select lottery 10 or 11, which both
have the highest expected value.5
2.2
Experiment Design
In the experiment, we vary the number of choices subjects have as well as the payment rule.
Payments will either be RIVAL and based on the maximum successful outcome, or RANDOM and based on one randomly selected lottery. Regardless of how subjects are paid,
subjects participate in sessions where they choose 1, 2, 4, or 6 lotteries from the set of 20 lotteries shown in the Experiment Instructions (Appendix G). Subjects observe each lottery’s
prize and the probability of winning. Moreover, to prevent a conflation between lottery selection and the ability to calculate expected values, we present each lottery’s expected value.
For ease of exposition we refer to treatments using both payment rule (RANDOM or RIVAL) and the given number of choices (ONE, TWO, FOUR, and SIX). As such, we have 7
treatments, ONE|RIVAL and ONE|RANDOM are identical treatments. We also index the
lotteries from 1 to 20 to simplify the analysis, while the experiment labeled the lotteries by
letters.
We conduct experiments on Amazon Mechanical Turk (AMT). AMT is an online workplace
where workers can complete tasks for wages.6 Prior to starting the main task, subjects consent to participate then complete a short survey that includes two expected value questions
and English comprehension questions.7 For participating, we pay subjects a base wage of
25 cents.
We use AMT for multiple reasons. Fundamentally, AMT provides a demographically richer
subject pool than a typical university’s subject pool. This is important because problems
like the college application problem are not problems exclusively faced by Western, Educated, Industrialized, Rich, and Developed (or WEIRD) populations (Henrich et al., 2010).8
In addition to the base wage, subjects earn a bonus contingent on the outcome of their lottery choices. In RIVAL sessions, for each lottery a subject chooses, we generate a random
number between 0 and 1. A number less than or equal to the probability of the lottery
winning results in the lottery having a favorable outcome for the subject. If the randomly
5
We impose risk neutrality only to follow convention.
For more details about AMT please see Cooper and Johnson (2013).
7
Ideally, we would have preferred to conduct the survey after the main experiment however the requirement that subjects be able to comprehend English required us to begin with the survey. This should not
bias results as it occurs in all treatments.
8
Given the diversity of our subject pool we have confidence that our results are generalizable to the
population as a whole. We find that demographics have little influence on subject behavior.
6
8
generated number is greater than the probability of the lottery winning, the lottery results
in an unfavorable outcome for the subject. The subject then receives payment equal to the
maximum favourable outcome across her selected lotteries. In RANDOM sessions, a chosen
lottery is selected randomly, and a random number generated. If the number exceeds the
probability of winning the lottery, the lottery result is unfavorable for the subject. The
other lotteries the subject selected (regardless of their outcome) do not affect the subject’s
final earnings.
After selecting their lotteries, but before outcomes are revealed, subjects are then asked to
indicate what they consider to be the best and worst lotteries from the 20 possible lotteries.
While identifying these lotteries, subjects see which lotteries they selected but are unconstrained by their past selections. Finally, subjects complete a short questionnaire consisting
of a demographic survey, un-incentivized risk preference questions, the Barrat impulsivity
scale, and a simple Ellsberg paradox task, explained in Appendix B.
We ask subjects multiple risk preference questions across various domains but we are primarily interested in responses to the most general question. This question comes from the
German Socio-Economic Panel (Burkhauser and Wagner, 1993; Schupp and Wagner, 2002),
and is worded as follows:
How do you see yourself: Are you generally a person who is fully prepared to take
risks or do you try to avoid taking risks?
Subjects answer the question above using an 11 point scale ranging from 0 to 10 - with 0
being “I avoid risk” and 10 being “Fully prepared to take risks”. Although some readers
may question the use of an unincentivized risk elicitation question, Dohmen et al. (2011)
presents convincing evidence that this specific risk question provides “a reliable predictor
of actual risky behavior”. Moreover, we repeatedly find that this question significantly
predicts subjects’ decisions and that the estimated coefficients are always consistent with
theoretical predictions.
2.3
Hypotheses
The theoretical discussion leads to testable hypotheses. We have two broad sets of hypotheses, one set tests whether subjects follow an EU strategy, the other set tests the implications
of expanding/restricting choice. We state all hypothesis in terms of the null which can imply different strategies depending on the context.
Regardless of the number of choices subjects have, l∗ should remain the same for a given
strategy whether or not subjects follow EU, VMIN, or MAXMIN. However, if subjects
use the MAXMIN strategy, their baseline lottery will be much safer than if they follow
either of the other strategies as MAXMIN has subjects first select the safest lottery. At the
same time, subjects should regard the worst lottery as the one furthest from l∗ . As l∗ is
independent of the treatments, it also implies that the worst lottery should be as well. This
leads to Hypothesis 1.
9
Hypothesis 1 The number of choices will have no affect on what subjects regard as the
best lottery or the worst lottery.
Recall that under EU, j will only select k > 1 lotteries riskier than l∗ . If EU types dominate
the population, we expect to observe an increase in the number of risky lottery selections
as j receives more choices - which would drive up the average lottery selected. Likewise,
because picking riskier choices dominates picking safer choices, we expect to see zero lottery
selections safer than l∗ .
Alternatively, a failure to reject the null, suggest subjects follow VMIN. This logic leads us
to Hypothesis 2.
Hypothesis 2 The riskiness of the average lottery selected will not be affected by the number of choices.
Note that under both VMIN and EU, we expect the number of choices to increase the
riskiness of the subject’s riskiest selection. However, individuals following EU will have far
riskier picks than those following VMIN. This leads us to Hypothesis 3.
Hypothesis 3 Increasing the number of choices a subject has will not affect the riskiness
of the riskiest lottery selected.
If subjects follow EU, the safest lottery should always be the choice subjects made when
they are given only one choice, l∗1 . With MAXMIN, it will always be the safest lottery
available. With VMIN, the safest choices will get safer as the number of choices increases.
As such, in the case of Hypothesis 4, rejecting the null suggests subjects behave according
to VMIN.
Hypothesis 4 The number of choices a subject receives has no affect on their safest lottery
selection.
Hypothesis 5 Stated risk aversion preferences will have no affect on subjects’ lottery selections.
We regard Hypothesis 5 as a test of whether the unincentivized risk aversion question has
any relevance to subjects’ behavior.
Hypothesis 6 Chosen bundles will be no different when lotteries are independent.
Regarding Hypothesis 6, when lotteries are no longer interdependent as in RANDOM treatments, the payoff function changes to,
E[U i] =
k
1X
(pj v(wj )) .
k
(7)
j=1
In this case, EU behavior is indistinguishable from VMIN behavior. If subjects are EU
types, then once the interdependent nature of the lotteries is removed, we expect a significant change in the bundle subjects select. If the distribution of lottery selection remains
10
unchanged, it suggests subjects have a baseline lottery and seek a level of expected utility
that is similar to that baseline or are VMIN. 9 If subjects utilize VMIN, predictions under
both payment mechanisms would be identical. Moreover, this implies that the expected
value of bundles for random and rival (both calculated as rival or rival) should be the same,
for a given number of choices.
3
Experimental Results
3.1
Subject Characteristics
Appendix D (Table 7) presents summary statistics. We gather information regarding the
gender, age, education, and income of subjects.As expected, subjects in our experiment
are more heterogeneous than subjects in laboratory studies, especially in terms of age and
education. Appendix C contains the variable descriptions. The gender of workers is slightly
unbalanced: 63% of workers indicate they are male. The average age of workers is 35 years.
The modal income of workers is between $25,000 - $37,499 per year and the modal education is a bachelor’s degree.
Table 1: Treatments
ONE
TWO
FOUR
SIX
TOTAL
RIVAL
RANDOM
47
-
44
39
43
38
43
40
177
117
TOTAL
47
83
81
83
294
The data are generally high quality as the software ensures fields are not left empty.10 Although this strategy is not perfect (e.g., there are a few accidental submissions - which we
must omit) the program successfully prevented empty fields. Further evidence of the data
quality can be seen in subjects’ responses to expected value questions. Overall, 61% of
subjects correctly calculate the more precise expected value question (question 6 in G.2).
Demographic variables and individual characteristics are roughly evenly distributed across
treatments. In Table 8 (Appendix E), we regress AGE, MALE, INCOME, EDUCTION,
RISK, and AMB against the treatments to check for differences in subject characteristics
across treatments. As expected, there are few substantive differences and demographic
variables are roughly evenly distributed across treatment groups. However, FOUR|RIVAL
has more male subjects; SIX|RIVAL has more educated subjects; SIX|RANDOM has a
higher percentage of ambiguity averse subjects; ONE has older subjects; TWO|RIVAL has
9
A second, but somewhat unattractive proposition, is that subjects select their bundles using subjective
expected utility. While possible, we believe this is unlikely as all the probabilities are provided.
10
There is some measurement error, as can be seen in Table 7, one subject broke the world record of
human lifespan (Weon and Je, 2009).
11
subjects more prepared to take risks.
With the observable characteristics split relatively evenly across the treatments, we take
differences in outcomes to be a result of the different treatments. In regards to outcomes,
subjects generally regard the best lottery as the safer lottery of the two that maximize the
expected payoff – lottery 11. Subjects generally regard Lottery 7 as the worst lottery.11
This subjective opinion of the worst lottery changes across treatments.
Based on choices, subjects are risk averse. The average riskiest lottery subjects pick is lottery
8 while the safest lottery picked is around lottery 13 or 14. Unconditional on treatment,
subjects select their identified best lottery only 72% of the time, much less frequently than
our prior of 100%.
3.2
Treatment Effects
We now analyze treatment effects. Recall that lottery 1 is the riskiest and lottery 20 is the
safest. With only one choice, we hypothesize that subjects will choose their favorite. As
such, if the best lottery is in the middle of all the possible choices (e.g., 10 or 11), it follows
the worst lottery is either the safest or riskiest lottery.
3.2.1
The Effect of Choice Restriction on Lottery Perception
Result 1 The number of choices has little affect on what subjects perceive as the best or
worst lottery.
Recall that hypothesis 1 suggests the treatment will have no affect on what subjects regard
as the best or worst lottery. Figure 5 presents what subjects identify as the worst lottery.
Subjects generally fall into one of two categories: they either consider the worst lottery
to be the riskiest lottery, or the safest lottery. However, Figure 5 shows that, subjects in
all treatments are generally in agreement as to what are the best lotteries. Predictably,
subjects regard the lotteries with the highest expected values as best.
In Table 2, we estimate subjects’ ex-post classification of the best and worst lotteries as a
function of the treatments and demographic variables using a Tobit specification. Several
demographic variables affect subjects’ classifications. First, individual risk preferences have
a significant affect on what subjects perceive as the best lottery. That is, subjects who are
relatively prepared to take risk, select a best lottery that is riskier, than subjects who are
less prepared to take risk. Additionally, after taking into account risk preferences, males
have higher best lottery selections than do females. Males are generally more willing to
take risk (p=0.010). Finally, more educated subjects generally classify riskier lotteries as
the best. In terms of rankings, there are few treatment effects to report. Only subjects in
11
As we discuss in great detail later this is a bit misleading. Subjects generally regard the riskiest (lottery
1) or the safest (lottery 20) lotteries as the worst.
12
SIX|RANDOM are more likely to classify the riskier lotteries as the best.
Figure 5: Best and Worst Lotteries by Treatment (RIVAL)
Note: Left Panel is Best Lottery, Right Panel is Worst Lottery.
Figure 6: Best and Worst Lotteries by Treatment (RANDOM)
Note: Left Panel is Best Lottery, Right Panel is Worst Lottery.
Treatments have little affect on what subjects perceive as the best and worst lotteries.
After controlling for individual characteristics, subjects in SIX|RANDOM have a riskier
best lottery selection. Models 4, 5, and 6 all show that subjects in SIX treatments have a
significantly greater “worst” lottery. The other treatments are not different in the lottery
they perceive as the “worst”. This is somewhat surprising as subjects in SIX|RIVAL,
SIX|RANDOM, FOUR|RIVAL, and FOUR|RANDOM are all significantly more likely to
select lottery 20 than subjects in TWO|RIVAL, TWO|RANDOM, and ONE|RIVAL.12
12
Why do subjects in the treatments with six choices often select what they regard as the worst lottery?
We believe in this case the treatment is affecting subjects’ responses. While subjects in treatments with 4
and 6 choices are equally likely to select lottery 20 as the worst, realizing that outcome requires all other
chosen lotteries to have unfavourable outcomes, so subjects may regard it as very unfavourable. In FOUR
13
Table 2: Best and Worst Lottery by Treatment
Best Lottery
TWO|RIVAL
TWO|RANDOM
FOUR|RIVAL
FOUR|RANDOM
SIX|RIVAL
SIX|RANDOM
AMB
RISK
CONSTANT
Obs.
R2
LL
UL
Worst Lottery
-0.707
(1.145)
-0.285
(1.188)
0.053
(1.159)
0.296
(1.196)
0.255
(1.157)
-1.797
(1.176)
11.73***
(0.799)
-0.521
(1.143)
-0.37
(1.18)
0.348
(1.162)
0.287
(1.189)
0.427
(1.151)
-1.864
(1.187)
0.624
(0.658)
-0.241**
(0.117)
12.612***
(1.012)
3.234
(13.602)
9.635
(13.86)
13.858
(13.427)
2.72
(14.135)
26.691*
(13.694)
22.051
(13.896)
-20.072*
(10.299)
3.061
(13.425)
11.038
(13.692)
9.125
(13.316)
2.27
(13.972)
24.983*
(13.457)
25.83*
(14.052)
-13.596*
(7.779)
1.793
(1.363)
-22.864*
(12.719)
294
0.0027
20
27
293
0.0058
20
27
294
0.0088
168
87
293
0.0153
168
87
Standard Errors in parentheses. ***: p < .01, **: p < .05, *: and p < .10.
Upper Limit (UL) and Lower Limit (LL) are lotteries 1 and 20.
14
3.2.2
Lottery Selections
Recall from Section 2, we expect subjects to generally follow one of three strategies. Subjects
following Expected Utility (EU) will select lotteries that are riskier than their favorite.
Subjects following variance minimization (VMIN) will select lotteries that are close (in
expected utility) to their favorite. Subjects following (MAXMIN) will select the certain
outcome lottery, followed by the next safest lotteries. If the population is predominantly
made of subjects using an EU strategy then one expects subjects to pick 5, 3, and 1
lotteries/lottery that are/is riskier than their favorite. Alternatively, VMIN types will
evenly distribute their additional picks between lotteries that are riskier and safer than
their favorites.
Result 2 The number of choices has no affect on the average lottery selected within a
portfolio.
Hypothesis 2 suggests choice restriction will have no impact on the average lottery selected.
We find no evidence against this null. Table 3, presents the average indicator number of
the selected lotteries, by treatment. First, note the average of the lotteries selected is in
the range of 10 to 12 for all treatments. Moreover, not one average is statistically significantly different from any other. Recall from Figures 1 and 4 that as the number of choices
increases, subjects who are EU types will have a lower average choice. This occurs because
they choose increasingly risky lotteries as the number of choices increases and this occurs
regardless of their risk preferences. Conversely, VMIN subjects will have picks that are
centered around a particular lottery and expand out in both directions - leaving the mean
unchanged across the treatments.
Figure 7: Distribution of Choices and Choice Counts by Treatment (RIVAL)
We find evidence of subjects following a VMIN strategy. Subjects in SIX|RIVAL on average
pick 2.74 (±.586) lotteries safer than their favorite and 2.42 (±.567 lotteries that are riskier
to get paid the 25 cent bonus, subjects need 3 bad outcomes to happen while in 6, subjects require 5 bad
draws. In a sense, the worst thing that can happen in SIX takes a lot more bad luck than in FOUR. We
think this is what is driving subjects to select lottery 20 as the “worst” lottery.
15
Figure 8: Distribution of Choices and Choice Counts by Treatment (RANDOM)
Table 3: Average Choice by Treatment
RIVAL
RANDOM
ONE
TWO
FOUR
SIX
ALL
10.936
(0.637)
-
11.239
(0.449)
10.615
(0.609)
11.401
(0.583)
11.875
(0.619)
12.004
(0.515)
10.492
(0.519)
11.384
(0.276)
10.982
(0.338)
Standard Deviations in parentheses.
than their favorite). Second, their counterparts in FOUR|RIVAL behave similarly and on
average pick 1.53 (±.421) lotteries safer than their favorite and 1.84 (±.435) lotteries that
are riskier than their favorite. Subjects in TWO|RIVAL on average pick .88 (±.22) lotteries
that are safer than their favorite and .55 (±.22) lotteries riskier than their favorite. This
shows that subjects are roughly distributing their choices evenly across lotteries riskier and
safer than their favorite - evidence against EU.
We find further evidence of subjects using a VMIN strategy by analyzing the frequency
and distribution of the selected lotteries across treatments, illustrated in Figures 7 and 8.
The left side of both Figures 7 (RIVAL) and 8 (RANDOM) are the probability distribution
function of choices by the choice treatments. The right side of these figures are histograms
(counts) of subjects’ actual choices. As expected, the mass is mostly centered on the best
choices.
For ease of exposition, we now analyze subjects’ riskiest and safest choices. We define
the safest choice as maxpi li ∈ lk? . We define the riskiest choice as minpi li ∈ lk? . This
delineation leads to two additional results:
Result 3 Subjects with more choices select riskier “risky” lotteries.
Result 4 Subjects with more choices select safer “safe” lotteries.
16
17
294
0.0203
34
1
NO
-1.449
(1.002)
-2.146**
(1.038)
-3.368***
(1.016)
-1.359
(1.042)
-3.342***
(1.009)
-5.546***
(1.04)
10.864***
(0.697)
294
0.0291
34
1
NO
-1.103
(0.977)
-2.205**
(1.006)
-2.793***
(0.993)
-1.459
(1.011)
-3.115***
(0.98)
-5.444***
(1.024)
0.345
(0.563)
-0.384***
12.556***
(0.864)
293
0.0338
33
1
NO
-1.136
(0.972)
-2.116**
(1.004)
-2.995***
(0.993)
-1.57
(1.011)
-2.952***
(0.981)
-5.496***
(1.022)
0.14
(0.566)
-0.412***
(0.1)
1.419**
(0.573)
0.145
(0.099)
-0.119
(0.161)
0.007
(0.014)
11.653***
(1.32)
293
0.0402
33
1
YES
-0.977
(0.977)
-1.836*
(1.004)
-3.017***
(0.984)
-1.452
(1.01)
-2.914***
(0.977)
-5.552***
(1.013)
-0.046
(0.564)
-0.451***
(0.101)
1.497***
(0.575)
0.006
(0.013)
6.074
(4.653)
(4)
294
0.0302
3
36
NO
10.875***
(0.632)
2.139**
(0.908)
1.426
(0.937)
4.339***
(0.92)
3.634***
(0.952)
5.707***
(0.928)
3.917***
(0.938)
-
(5)
293
0.0337
3
36
NO
2.283**
(0.905)
1.373
(0.929)
4.658***
(0.921)
3.615***
(0.944)
5.84***
(0.921)
3.886***
(0.945)
0.404
(0.524)
-0.191**
(0.093)
11.617***
(0.802)
(6)
293
0.0341
3
36
NO
2.317**
(0.912)
1.324
(0.938)
4.659***
(0.931)
3.56***
(0.955)
5.935***
(0.935)
3.877***
(0.953)
0.427
(0.532)
-0.189**
(0.094)
0.019
(0.539)
-0.004
(0.093)
-0.112
(0.152)
0.001
(0.013)
12.107***
(1.239)
Safest Choice
(7)
293
0.0408
3
36
YES
2.241**
(0.915)
1.474
(0.938)
4.736***
(0.922)
3.597***
(0.955)
5.952***
(0.93)
3.725***
(0.944)
0.311
(0.531)
-0.218**
(0.095)
0.116
(0.54)
-0.002
(0.013)
4.416
(4.365)
(8)
Notes: Tobit estimates. Standard Errors in parentheses. ***:p <.01, **:p <.05, and *:p <.10. Upper Limit (UL) and Lower Limit (LL) are lotteries
1 and 20. Cat. Dum. are categorical dummies for income and education variables.
Obs.
R2
LL
UL
Cat. Dum.
CONSTANT
AGE
EDUC
INCOME
MALE
RISK
AMB
SIX|RANDOM
SIX|RIVAL
FOUR|RANDOM
FOUR|RIVAL
TWO|RANDOM
TWO|RIVAL
(1)
Riskiest Choice
(2)
(3)
Table 4: Riskiest and Safest Choice by Treatment
We find evidence to reject the null hypotheses 3 and 4. In Table 4 we estimate subjects’
riskiest and safest lottery selected, conditional on individual characteristics and treatments.
Regardless of the specification, increasing the number of choices, induces subjects to select riskier riskiest picks and safer safe picks. Result 3 is consistent with EU maximizing
behavior, however, result 4 is inconsistent with EU maximization. This suggests (again)
that subjects are generally following a VMIN strategy.13 This observation coupled with the
lack of significant differences in the average lottery selected across treatments is particularly
strong evidence against the expected utility hypothesis.
Result 4 can also be taken as evidence against MAXMIN, as someone following a strict
MAXMIN strategy would always select the safest lottery. That we see the safest picks
becoming safer suggests that individuals, on average, are not following a strict MAXMIN
strategy either.
3.2.3
The Effect of Risk Preferences on Lottery Selection
Result 5 Subjects who indicate that they are more prepared to take risks have relatively
riskier “safe” picks and riskier “risky” picks.
Table 5: Percentage Picking a Series of Lotteries
RIVAL
RANDOM
ALL
TWO
FOUR
SIX
ALL
0.5
(0.076)
0.513
(0.081)
0.488
(0.077)
0.737
(0.072)
0.535
(0.077)
0.5
(0.08)
0.508
(0.044)
0.585
(0.046)
0.506
(0.055)
0.605
(0.055)
0.518
(0.055)
0.544
(0.032)
Standard Deviations in parentheses.
Hypothesis 5 suggests that stated (and unincentivized) individual risk preferences will have
no affect on lottery selection. We find strong evidence against this null. Recall that after
subjects select their lotteries, they are then asked about their risk preferences. As one
would expect, individual risk preferences have a close relationship with subjects’ lottery
selection. Visually, this can be seen in Figures, 9, 10, 11, and 12. In Figures 9, 10, 11, and
12 we present subjects’ choices in each treatment, sorted as stated from most risk averse
to most risk loving. Each column represents a subject and each row represents a lottery.
A large square within a lottery square is a subject’s choice and smaller square indicates
the subject’s favorite lottery. “X” indicates the subject selected what they indicated as the
13
As a robustness check, in a separate treatment we allow subjects to select the same lottery more than
once while giving them six choices. 14% of subjects use all their choices on same lottery and 38% of subjects
pick the same lottery at least twice.
18
best lottery. Additionally, these figures provide strong evidence in support of VMIN. These
figures combined with Table 5 (which shows the percentage of subjects who selected only
adjacent lotteries - e.g., 5,6,7, and 8) are strong evidence in support of VMIN.
Figure 9: Subjects’ Decisions Sorted by Risk Preferences
Treatment: ONE|RIVAL
Notes: A large square within a lottery square is a subject’s choice and smaller square
indicates the subject’s favorite lottery. “X” indicates the subject selected what they
indicated as the best lottery.
Figure 10: Subjects’ Decisions Sorted by Risk Preferences
Treatment: TWO|RIVAL
Notes: See notes from Figure 9.
In Table 4 it is easy to see that risk aversion also is correlated to what subjects pick as their
riskiest lottery. Comparatively risk-loving subjects pick considerably riskier risky lotteries.
Additionally, risk aversion is also negatively related to subject’s average lottery selection.14
After controlling for treatment, a 10% decrease in subjective risk averse preferences is
associated with a .28 decrease in the average lottery selected (p = 0.000).
14
Full estimates available upon request.
19
Figure 11: Subjects’ Decisions Sorted by Risk Preferences
Treatment: FOUR|RIVAL
Notes: See notes from Figure 9.
Figure 12: Subjects’ Decisions Sorted by Risk Preferences
Treatment: SIX|RIVAL
Notes: See notes from Figure 9.
20
4
Expected Value
Result 6 Holding the number of choices fixed, there are few differences in the chosen bundles across random and rival sessions.
Recall Hypothesis 6 suggests that after controlling for the number of choices a subject is
given, there should be no significant differences between random and rival sessions. A failure
to reject this hypothesis would suggest subjects are not maximizing expected utility in rival
sessions but are rather using VMIN or picking lotteries as if they are independent. Clearly,
while expected utility increases with the number of choices subjects are given, there are few
significant differences across rival and random sessions. This is most clearly seen in Table
6 which estimates the expected value subject i’s bundle, conditional on the treatment and
demographic characteristics.
Below each regression in Table 6 we present p-values from a set of f-tests testing if coefficient estimates of X|RIVAL = X|RANDOM for X = 2, 4, 6. We find little in the way
of differences across RANDOM and RIVAL sessions. The only significant differences are
found when testing if SIX|RIVAL is equal to SIX|RANDOM. The test results suggest that
subjects’ selected bundles in SIX|RANDOM have a higher expected value than subjects in
SIX|RIVAL, which is somewhat surprising. This is particularly strong evidence that subjects are not maximizing their expected utility since their expected value in rival sessions
would be greater than expected value in random sessions if they did so.15
5
College Applications
Having discussed how subjects behave in a manner inconsistent with expected utility maximization, and how choice restriction results in fewer risks being taken, we now discuss the
relevance of these findings specifically for the college application decision. Attending and
completing college generally leads to increased lifetime earnings while also providing an
opportunity for upward social mobility. However, low income individuals are less likely to
apply for college and if they do, they generally apply to less selective schools. Additionally, low income students apply to fewer schools than their high income counterparts. This
results in low income students attending less selective schools and reduces the aggregate
returns to higher education for low income students.
Our experiment is consistent with findings in Pallais (2013). When our subjects receive
more choices, they are more likely to choose both riskier and safer lotteries.This behavior is
inconsistent with expected utility theory. In a setting we refer to as “rival interdependent
lotteries” many subjects appear to make decisions based on a relative variance heuristic
rather than expected utility theory. This is an important distinction as previous theoretical and empirical work assumes application portfolio selection is done by maximizing
15
Although we only present regression results with the dependent variable calculated using rival expected
value, the results are robust (and even stronger) when the dependent variable is calculated using random
lottery selection expected value.
21
Table 6: Expected Value of Selected Portfolios
TWO|RIVAL
TWO|RANDOM
FOUR|RIVAL
FOUR|RANDOM
SIX|RIVAL
SIX|RANDOM
AMB
RISK
MALE
INCOME
EDUC
AGE
CONSTANT
Obs.
R2
2ri = 2ra
4ri = 4ra
6ri = 6ra
Cat. Dum.
(1)
(2)
(3)
(4)
0.654***
(0.104)
0.603***
(0.107)
1.132***
(0.104)
1.11***
(0.108)
1.443***
(0.104)
1.638***
(0.107)
1.143***
(0.072)
0.619***
(0.103)
0.604***
(0.106)
1.094***
(0.104)
1.122***
(0.107)
1.426***
(0.103)
1.612***
(0.107)
0.027
(0.059)
0.033***
(0.01)
0.974***
(0.091)
0.618***
(0.104)
0.627***
(0.107)
1.092***
(0.105)
1.137***
(0.108)
1.424***
(0.105)
1.623***
(0.108)
0.01
(0.06)
0.031***
(0.011)
0.084
(0.061)
0.008
(0.01)
0.016
(0.017)
0.002
(0.001)
0.768***
(0.14)
0.645***
(0.106)
0.627***
(0.109)
1.103***
(0.106)
1.164***
(0.11)
1.452***
(0.106)
1.649***
(0.109)
0.03
(0.061)
0.032***
(0.011)
0.061
(0.062)
0.002
(0.001)
1.408***
(0.509)
293
0.5396
0.6412
0.8418
0.0738
NO
293
0.5553
0.8898
0.7980
0.0897
NO
293
0.5623
0.9308
0.6836
0.0732
NO
293
0.5788
0.9308
0.6836
0.0732
YES
Notes: Standard Errors in parentheses. ***: p < .01, **: p < .05, and *: p < .10.
Upper Limit (UL) and Lower Limit (LL) are lotteries 1 and 20. Cat. Dum. are
categorical dummies for income and education.
22
expected utility. For instance, Fu (2014) models the portfolio decision in a large structural
model assuming students select their application portfolio by solving an expected utility
maximization problem.16 Our experiments show that the assumption of expected utility
maximization may result in too few predicted applications to safer schools.
One policy implication would be to provide low income individuals subsides for college applications. Encouragingly, in the 2014-2015 academic year the College Board enacted a
policy in which certain low income students would be able to both write the SAT for free,
and receive vouchers for up to four free college applications, at participating colleges.17 The
analysis from our experiment suggests that this policy may greatly improve the welfare of
low income students - particularly those with high ability. However, we also advice policy
makers to be careful when considering who qualifies for the application subsidization. If
qualification for the subsidies are too generous, it is possible that much of the gains experienced by low income high ability types could be lost due to crowding out.
Finally, one of the larger contributions of this work is that it suggests why the interventionist
policies, like the one discussed by Carrell and Sacerdote (2013) are so effective. Obviously,
giving more choice fosters a greater number of riskier lottery choices which we argue is
similar to applying to more selective universities. However, at a deeper level, we also show
that potential students may not take into account the rival nature of the lotteries and
instead pick lotteries as if they are independent. This is unfortunate because it translates
into students applying to schools that are less selective. In scenarios, with rival uncertain
outcomes, like the college application process, people do not appear to be expected utility
maximizers. Essentially, expert advice corrects this behavioral tendency. The expert advice
also advises students to choose safety schools, which are not predicted under EU. Hoxby
and Avery (2013) shows that expert advice leads to students applying to more selective
universities which increases their likelihood of attending more selective schools. Policy
makers should consider this and encourage students to perhaps apply to a riskier stretch
school...after they are given one more chance.
16
The application decision is part of a larger model determining tuition, applications, admission, and
enrollment. The application decision is essentially one of whether to apply to certain categories of schools:
public vs. private, elite vs. non-elite, in-state vs. out-of-state, and the interactions of those.
17
See https://bigfuture.collegeboard.org/get-in/applying-101/college-application-fee-waivers
and https://sat.collegeboard.org/register/sat-fee-waivers for details.
23
References
Burkhauser, Richard V, and Gert G Wagner (1993) ‘The english language public use file of
the german socio-economic panel.’ Journal of Human resources 28(2), 429–433
Carrell, Scott E., and Bruce Sacerdote (2013) ‘Late Interventions Matter Too: The Case
of College Coaching New Hampshire.’ NBER Working Papers 19031, National Bureau of
Economic Research, Inc, May
Chade, Hector, Gregory Lewis, and Lones Smith (2013) ‘Student portfolios and the college
admissions problem.’ The Review of Economic Studies
Cooper, David J, and David B Johnson (2013) ‘Ambiguity in performance pay: An online
experiment.’ Available at SSRN 2268633
Dohmen, Thomas, Armin Falk, David Huffman, Uwe Sunde, Jürgen Schupp, and Gert G
Wagner (2011) ‘Individual risk attitudes: Measurement, determinants, and behavioral
consequences.’ Journal of the European Economic Association 9(3), 522–550
Fu, Chao (2014) ‘Equilibrium tuition, applications, admissions, and enrollment in the college
market.’ Journal of Political Economy 122(2), 225 – 281
Henrich, Joseph, Steven J Heine, and Ara Norenzayan (2010) ‘The weirdest people in the
world?’ Behavioral and brain sciences 33(2-3), 61–83
Hoxby, Caroline, and Christopher Avery (2013) ‘The Missing ’One-Offs’ The Hidden Supply
of High-Achieving, Low-Income Students.’ Brookings Papers on Economic Activity 46(1
(Spring), 1–65
Pallais, Amanda (2013) ‘Small differences that matter: Mistakes in applying to college.’
Working Paper 19480, National Bureau of Economic Research, September
Schupp, Jürgen, and Gert G Wagner (2002) ‘Maintenance of and innovation in long-term
panel studies: The case of the german socio-economic panel (gsoep)’
Viappiani, Paolo, and Christian Kroer ‘Optimization and elicitation with the maximin
utility criterion’
Weon, Byung Mook, and Jung Ho Je (2009) ‘Theoretical estimation of maximum human
lifespan.’ Biogerontology 10(1), 65–71
24
A
Sketch of Proof of Proposition 1
With some loss of generality, we discuss strategies when choosing among a set of three
lotteries, l0 , l1 , and l10 . We define these lotteries by:
l0 = p0 w0 , l1 = p1 w1 , and l10 = p01 w10
where
θ<1
p1 = θp0 , w1 = (2 − θ)w0
p01 = (2 − θ)p0 , w10 = θw0
E[l0 ] > E[l1 ] = E[l10 ].
(8)
This implies that the probability that l1 rewards a favorable outcome for j is (1 − θ) percent
less than the probability that l0 rewards the favorable outcome. However, l1 pays (1 − θ)
percent more if the favorable outcome occurs. The other alternative lottery, l10 , is similar
but pays (1 − θ) percent less upon the occurrence of the favorable outcome. This loss in
monetary winnings is fully compensated with a (1 − θ) percent increase in the probability
of the favorable outcome occurring. Thus E[l1 ] = E[l10 ].
With three lotteries in the choice set, we consider what the individual would pick when
choosing 1, 2, or 3 lotteries if using expected utility. The preferred bundle when allowed
to choose 3 lotteries is trivial, as she would choose all available lotteries. The preferred
bundle with one choice is also straightforward; lottery l0 has the highest expected payout
and would thus be chosen.
The choice with a bundle of two is more complicated. The individual has the option of
choosing (l0 , l1 ), (l0 , l10 ), or (l1 , l10 ). We now demonstrate that the individual would choose
(l0 , l1 ). We do this by demonstrating that (l0 , l1 ) (l10 , l1 ) and that (l0 , l1 ) (l0 , l10 ).
In comparing (l0 , l1 ) and (l10 , l1 ), consider what happens when the outcomes of the lotteries are realized. Recall that l1 is the lottery with the highest payout. If either bundle is
selected, and l1 is successful, then she would receive p1 and the outcome of the other lottery is irrelevant. If l1 is unsuccessful, then the payout received will solely depend on either
l0 or l10 . Since E[l0 ] > E[l10 ], she would choose the bundle containing l0 , thus (l0 , l1 ) (l10 , l1 ).
In comparing (l0 , l1 ) and (l0 , l10 ) lottery l0 is constant, so we compare the addition of either
l1 or l10 . Assume j selects l10 instead of l1 . This implies the payoff of lottery bundle (l0 , l1 ) is
less than the bundle consisting of (l0 , l10 ). When we compare the expected utilities of these
bundles, we get:
p1 w1 + (1 − p1 )p0 w0 < p0 w0 + (1 − p0 )p01 w10
which simplifies to
(9)
0
0
0
0
p1 w1 − p1 p0 w0 < p1 w1 − p0 p1 w1 .
25
Which can be rewritten in terms of θ, p0 , and w0 :
θp0 (2 − θ)w0 − θp0 p0 w0 < (2 − θ)p0 θw0 − p0 (2 − θ)p0 θw0 .
(10)
However, this implies that 1 < θ which contradicts the definition of 1 > θ. So we have a
proof by contradiction, and (l0 , l1 ) (l0 , l10 ). Thus when faced with two choices she will
choose bundle (l0 , l1 ). That bundle will include a riskier lottery when compared with the
bundle with only one choice.
26
B
Ellesberg Details
We use the Ellesberg paradox to classify subjects as ambiguity averse, or not. The questions
we use for ambiguity aversion classification are shown below. A subject is classified as
ambiguity averse if they say they prefer choice(a) in question 1 and choice (d)in question 2.
1) Suppose there is a bag containing 90 balls. You know that 30 are red and the other 60
are a mix of black and yellow in unknown proportion. One ball is to be drawn from
the bag at random. You are offered a choice to (a) win $ 100 if the ball is red and
nothing if otherwise, or (b) win $100 if it’s black and nothing if otherwise. Which do
you prefer?
2) The bag is refilled as before, and a second ball is to drawn from the bag at random.
You are offered a choice to (c) win $ 100 if the ball is red or yellow, or (d) win $ 100
if the ball is black or yellow. Which do you prefer?
C
Variable Descriptions
• AMB is a dummy variable equal to 1 if the subject was classified as ambiguity averse.
• RISK self reported risk preference. 0 if risk averse; 10 if full prepared to take risks.
• MALE is a dummy variable that is equal to 1 if the subject indicates they are male;
0 otherwise.
• AGE is the subject’s reported age.
• INCOME is the subject’s reported income. This variable is categorical, increasing in
intervals of $12,500 and can take a value from 1 to 10, with 1 indicating an income
less than $12,500 and 10 being greater than $ 100,000. All figures in US dollars.
• EDUC is the subject’s level of education. This variable is categorical, increasing in
educational achievement.
• BEST is the lottery that the subject specifies as the best.
• WORST is the lottery that the subject specifies as the worst.
• RISKIEST is the subject’s riskiest choice.
• SAFEST is the subject’s safest choice.
• PICKED FAV. is a dummy variable equal to one if the subject picked what they
thought of as the best lottery.
27
D
Summary Statistics
Table 7: Summary Statistics
Variable
Obs
Mean
Std. Dev.
Min
Max
AMB
RISK
MALE
AGE
INCOME
EDUC
BEST
WORST
RISKIEST
SAFEST
PICKED FAV.
294
293
294
294
293
294
294
294
294
294
294
0.473
5.179
0.631
34.771
4.275
4.756
11.372
7.318
8.411
13.661
0.723
0.5
2.78
0.483
19.06
2.813
1.728
4.713
8.514
4.599
4.233
0.448
0
0
0
18
1
1
1
1
1
1
0
1
10
1
323
10
7
20
20
20
20
1
28
E
Characteristics across Treatments
Table 8: Distribution of Characteristics across Treatments
TWO|RIVAL
TWO|RANDOM
FOUR|RIVAL
FOUR|RANDOM
SIX|RIVAL
SIX|RANDOM
CONSTANT
Obs.
R2
F-Test
AGE
MALE
INCOME
EDUC
RISK
AMB
-6.727
(4.211)
-7.315*
(4.349)
-7.946*
(4.237)
-8.762**
(4.38)
-6.294
(4.237)
-8.741**
(4.319)
41.341***
(2.929)
0.149
(0.103)
0.028
(0.106)
0.234**
(0.103)
0.121
(0.107)
0.048
(0.103)
0.14
(0.105)
0.511***
(0.072)
-0.258
(0.591)
-0.708
(0.611)
-0.339
(0.599)
-0.554
(0.615)
-0.53
(0.595)
-0.129
(0.607)
4.554***
(0.411)
0.344
(0.358)
-0.421
(0.37)
0.043
(0.36)
-0.492
(0.372)
0.787**
(0.36)
-0.003
(0.367)
4.703***
(0.249)
0.986*
(0.58)
-0.095
(0.599)
0.904
(0.587)
-0.314
(0.603)
0.562
(0.583)
0.513
(0.594)
4.788***
(0.403)
0.098
(0.103)
0.088
(0.106)
-0.077
(0.103)
-0.084
(0.107)
-0.054
(0.103)
0.325***
(0.105)
0.426***
(0.072)
294
0.0211
0.4042
294
0.0045
0.2944
293
0.007
0.9171
294
0.0535
0.0144
293
0.0283
0.2182
294
0.0695
0.0020
Standard Errors in parentheses. ***: p < .01, **: p < .05, and *: p < .10.
29
F
Probability of Selecting Favorite by Demographic Characteristics and Treatment
Table 9: Probability of Picking Best Lottery
TWO|RIVAL
TWO|RAND
FOUR|RIVAL
FOUR|RAND
SIX|RIVAL
SIX|RAND
ABM
RISK
MALE
INCOME
EDUC
AGE
CONSTANT
Obs.
R2
(1)
(2)
(3)
-0.006
(0.269)
-0.387
(0.275)
0.63**
(0.296)
0.28
(0.288)
0.63**
(0.296)
0.928***
(0.329)
0.354*
(0.187)
0.001
(0.272)
-0.378
(0.276)
0.602**
(0.299)
0.266
(0.289)
0.618**
(0.297)
0.97***
(0.336)
-0.126
(0.168)
0.004
(0.03)
0.393
(0.249)
0.023
(0.277)
-0.419
(0.282)
0.651**
(0.307)
0.252
(0.294)
0.657**
(0.306)
0.977***
(0.341)
-0.099
(0.172)
0.015
(0.031)
-0.367**
(0.179)
0.012
(0.031)
-0.035
(0.05)
-0.005
(0.005)
0.822**
(0.397)
294
0.0027
293
0.0781
293
0.095
Standard Errors in parentheses. ***:p <.01, **:p <.05,
and *:p <.10.
30
G
Experiment Instructions
G.1
Introduction
Welcome to the HIT! The instructions for this HIT are straightforward. If you follow them
carefully, you can earn a considerable amount of money in addition to your participation fee
of 25 cents. The additional amount you earn will be paid through the Amazon Mechanical
Turk Bonus. Your confidentiality is assured.
In this HIT, there are 20 lotteries that vary both by their jackpots (payouts) and their odds
(probability of winning). As such the expected value of the lotteries(probability of wining
times the payout) also vary. You will be asked to select your X favorite lotteries, from the
20 available lotteries.
For each chosen lottery, a computer will randomly draw number between zero and one to
determine whether you have won that lottery. If the number drawn is less than or equal to
the probability of the lottery winning, then you won that lottery.
For example, let us assume that lottery C has a probability of winning of 15%, then any
number drawn by the computer between 0.00 and 0.15 would win the lottery and any number between 0.16 and 1.00 would not win the lottery. Your payment for this HIT will be the
maximum payment from any successful lottery. If you were to win only one lottery, than
the payout from that lottery would be your payment. If you were to win two lotteries, your
payment would be the highest value of the two payouts. If you do not win any lotteries
than you will only receive your participation fee.
Before you begin, we would like you to complete a brief survey to make sure that you comprehend written English.
Do not click the Submit button until specifically instructed to do so.
G.2
Survey
Before we begin please take a few minutes to complete this short survey. When you are
finished, please click the NEXT button. Note there are three English comprehension questions. If you fail to answer any of them correctly, you will be asked to return the HIT and
will not be able to continue. After you finish selecting your favorite lotteries and are told
of your earnings, you will be instructed to complete another short survey.
1. What is your gender?
2. What is your age?
3. What country do you currently live in?
31
4. Paul bought a baseball for X dollars. Jim bought a candybar for one dollar. How
much did Paul’s baseball cost?
5. You and your friend are playing game with a coin. If the coin is flipped and ends up
heads you win dollar. If it ends up tails you win nothing. What is the expected value
of this game (in CENTS)?
6. What is the expected value of game that pays 200 dollars with probability 25 percent?
Please just enter as an integer number.
7. William is not not going to the store. Is William going to the store?
Do not click the Submit button until specifically instructed to do so.
G.3
Practice
For example, you are asked to select your favourite 3 lotteries from lotteries A, B, C, D and
E. The lotteries’ payoffs are as follows:
Prob
Prize
EV
Lottery A
Lottery B
Lottery C
Lottery D
5%
$5.00
$0.25
10%
$4.75
$0.48
15%
$4.50
$0.68
20%
$4.25
$0.85
From the table above you can see that Lottery A pays 4.50 with a probability of 10%, Lottery B pays 4.00 with a probability of 20%, Lottery C pays 3.00 with a probability of 50%,
Lottery D pays 2.00 with a probability of 60% and Lottery E pays 1.00 with a probability
of 80%.
You select lotteries A, C, and E. The outcomes of each of these lotteries are as follows:
1. Lottery A draws the number .2, which is greater than .1 and therefore unfavorable to
you.
2. Lottery C draws the number .1, which is less than .5 and therefore favorable to you.
3. Lottery E draws the number .8, which is greater than .8 and therefore favorable to
you.
Your earnings for the lottery part of the HIT would therefore be 3.00 dollars. This is because the outcome of lottery A was unfavorable to you. While both lotteries C and E where
favorable to you, the favorable outcome of Lottery C ($3.00) is greater than the favorable
outcome of Lottery E ($0.80).
32
Lottery A
Lottery B
Lottery C
Lottery D
5%
$5.00
$0.25
10%
$4.75
$0.48
15%
$4.50
$0.68
20%
$4.25
$0.85
Prob
Prize
EV
When you are asked to make your actual decisions, you will see a chart like the one shown
on the practice stage but there will be more Lotteries and there will be box underneath
each lottery. In each of these boxes, there will be two radio buttons. One radio button
will correspond to “No” while the other will correspond to “Yes”. The ”No” button will be
indicated with a “N” while the yes button will be indicated with a ”Y” You will use these
radio buttons to indicate your preferred lotteries.
If a given lottery is among your X preferred lotteries, click the corresponding “Yes” radio
button. If not, click the “No” radio button.
After you have indicated your X preferred lotteries, you will be asked to click the ”next”
button to complete the final parts of the HIT.
Do not click the Submit button until specifically instructed to do so.
G.4
Game
Please indicate your X favorite lotteries. After you have indicated your X preferred lotteries,
click the ”next” button to finish up the final parts of the HIT.
A
Prob 5%
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
10%
15%
20%
25%
30%
35%
40%
45%
50%
55%
60%
65%
70%
75%
80%
85%
90%
95%
100%
Prize $5.00 $4.75 $4.50 $4.25 $4.00 $3.75 $3.50 $3.25 $3.00 $2.75 $2.50 $2.25 $2.00 $1.75 $1.50 $1.25 $1.00 $0.75 $0.50 $0.25
EV
$0.25 $0.48 $0.68 $0.85 $1.00 $1.13 $1.23 $1.30 $1.35 $1.38 $1.38 $1.35 $1.30 $1.23 $1.13 $1.00 $0.85 $0.68 $0.48 $0.25
Pick
Do not click the Submit button until specifically instructed to do so.
G.5
Game Continued
You have now indicated your most preferred lotteries. We now would like to find out how
you order to these lotteries from most desirable to least desirable. Below you should see
the each of the lotteries you selected and next to each lottery an input box. Use the input
boxes to rank your preference of lotteries. To indicate that a lottery is your favorite, click
the input box corresponding to that lottery and type 1. Do this for each of the lotteries
on this page. Remember: 1 indicates your favorite lottery, 2 indicates your second favorite
lottery, 3 indicates your third favorite lottery and so on and so on. DO NOT GIVE TWO
33
LOTTERIES THE SAME RANK.
Once you are finished ranking your lotteries, click the ”Next” button to go to the final
part of the HIT. THIS PORTION OF THE HIT WILL HAVE NO BEARING ON YOUR
PAYMENT.
Reminder! Do not click the Submit button until specifically instructed to do so.
G.6
END
Thank you for you your participation! Please answer the questions below. If you do so, you
will earn an additional 10 cent bonus! If you do not wish to complete the survey. Feel free
to submit the HIT!
1. What is your Nationality?
2. Which of the following best describes your highest achieved EDUC level?
3. What is the total income of your household?
4. Why do you complete tasks in Mechanical Turk? Please check any of the following
that applies:
5. How do you see yourself: Are you generally a person who is fully prepared to take
risks or do you try to avoid taking risks?
34