The effect of financial goals and incentives on
labor.
An experimental test in an online market∗†
David B. Johnson and Justin Weinhardt
PRELIMINARY DRAFT
October 2, 2014
Abstract
Empirical studies investigating work motivation over time find people with fluctuating wages
work more on days when their wage rate is lower compared to when wages are higher. The
authors of these studies theorize individuals use daily income goals and stop working once they
reach their goal. This study involves assignment and manipulation of financial goals in an online
labor market that is nearly frictionless. Workers can quit at any time and can start jobs posted
by competing employers almost instantly. Results, with pooled data, indicate financial goals
do not lead to workers stopping work once they reached their goals and there is no significant
wage related crowding out. However, when we separate the sample by western and non-western
workers, we find production by western workers is greatest in treatments with goals and low
incentives. This effect is absent in non-western workers.
JEL classification:
Keywords: Experiment; Online; Goal
∗
David Blake Johnson: University of Calgary, Department of Economics, 2500 University Dr NW, Suite 554 , Social
Sciences, Calgary, Alberta, Canada T2N 1N4. [email protected]. Justin Weinhardt: University of Calgary,
Department of Human Resources and Organizational Dynamics, Scurfield Hall 442, Calgary, Alberta, Canada T2N
1N4. [email protected]
† We are grateful to valuable comments provided by Tim Salmon, Alexi Thompson, Gary Fournier, and Rob Oxoby.
All mistakes are our own.
1
Introduction
Several field studies investigate the influence of fluctuating wages on labor. Many of these studies
find productivity does not increase with wages (e.g., Laisney et al., 1996). Instead, individuals work
longer when the wage rate is low (Camerer et al., 1997; Chou, 2002; Lynn, 2002; Fehr and Goette,
2007).1 This set of findings is inconsistent with traditional theories of labor supply suggesting individuals work longer when their wage rate is high (e.g., Lucas Jr and Rapping, 1969) or, in other
words, when there is a greater “extrinsic” work motivation. On the other hand, goals introduced by
employers, experimenters, and individuals (Austin and Vancouver, 1996; Wu et al., 2008; GómezMiñambres et al., 2012; Sun and Weinhardt, 2014) enhance production and even in cases where the
goals have no impact on the workers earnings.2 Such responses are thought to relate to “intrinsic”
motivation and, when taken together, these findings suggest a “crowding out” effect of high wages.
In economics and psychology, the crowding out theory posits tangible benefits and costs (e.g., monetary rewards and punishments or any motivation that is from outside the individual) can reduce an
individual’s intrinsic motivation (e.g., task enjoyment or any motivation from within the individual).
Several models explaining the observed crowding out phenomenon have been introduced with many
piggybacking off of Kahneman and Tversky (1979).3 For instance Falk and Knell (2004) present a
model where individuals set goals based upon the performance of their peers. Doing so enhances
individual utility through self-enhancement and self-improvement.4 Falk and Knell (2004) go further
and present survey evidence confirming the validity of their model. Wu et al. (2008) extend this
line of research by presenting an elegant theory explaining the underlying mechanisms behind the
performance gains. Put simply, gains relative to the goal (i.e., the reference point) enhance utility
while losses (i.e., deviations from the goal) are costly.
Our work most directly extends that of Gómez-Miñambres et al. (2012) and Goerg and Kube (2013).
Gómez-Miñambres et al. (2012) builds upon Wu et al. (2008) by introducing exogenously assigned
goals (e.g., goals introduced by an employer). Such a model has similar properties to that of Wu
et al. (2008) which lead to goals increasing productivity. Gómez-Miñambres et al. (2012) provide
experimental evidence in support of their model. In treatments with goals that have no direct impact
on subjects’ monetary earnings, productivity significantly increases. Moreover, and quite surprising,
Gómez-Miñambres et al. (2012) find no evidence of extrinsic crowding out, as the wage rate increases
production further. This is not a trivial result and is in stark contrast to the “crowding out” hypothesis. Goerg and Kube (2013) furthers this line of research with a real effort experiment finding
the use of personal goals increases worker production. This occurs if the goals are self-assigned or
if selected by a principal.5 Moreover, much like previous work, these goals are effective even in
instances where they are not tied to monetary earnings.
Although similar to both Gómez-Miñambres et al. (2012) and Goerg and Kube (2013) our work
furthers current understanding of goals. Instead of an environment where subjects know they are
in an experiment, we take advantage of a venue in which it is left ambiguous as to whether or not
subjects are in an experiment. Moreover, we use a task purposefully selected to be natural for the
venue. We find little evidence of a wage effect seen in Gómez-Miñambres et al. (2012). Subjects
1 And
to some extent in Lynn (2002). In this case, the author observes a negative correlation between tipping and
turnover in low volume restaurants but not in high volume restaurants.
2 For a review see Locke and Latham (2002).
3 A general discussion of models is outside the scope of the current work but interested parties should see Kőszegi and
Rabin (2006), Koch and Nafziger (2011), Gómez-Miñambres (2012) and Hsiaw (2013).
4 Self-enhancement increases the individual’s utility by making them feel better about themselves. On the other hand,
self-improvement increases utility by giving the individual something to strive for which increases their utility by
way of increasing their performance.
5 There is a caveat here in that the size of the goal matters quite a bit as easily obtained goals lead to the worker
being less productive.
1
in treatments with a high wage do not significantly increase their productivity. Further, we find
only weak evidence supporting a goal hypothesis and this effect is only present with workers from
developed western nations.
Our work does not contradict previous work but rather illustrates a need for a more nuanced view
of goals. Specifically, workers in our experiment have the opportunity to switch employers or quit to
attend to a personal task. Moreover, workers may do so at any time. In other words, there is little
cost to quit. This is quite novel and most relevant in workplaces where employees act more like their
own bosses. In such cases, because of the lack of a concrete work schedule and/or a “boss”, it is
therefore less surprising that goals have a relatively weak effect. Moreover, a larger take away of the
present work is how the importance of behavioral aspects change as the labor market approaches
a neoclassical frictionless labor market with homogenous workers. Ex-post this might seem quite
obvious; more complicated markets contain greater frictions which can alter behavior. Remove these
frictions and behavior neatly follows many implications of neoclassical models.
2
Present Study
Ideally, to test the effects of varying wage rates on work hours, one needs a job where wages vary
across some temporal period (e.g., days) but relatively constant within a period. Further, individuals have a choice regarding how long they work and these variables (i.e., rate of pay and amount of
time worked) must be observable. One job where the above conditions are met is New York City cab
drivers (Camerer et al., 1997). Specifically, cab drivers wages stay relatively constant throughout
the day, but fluctuate between days, presumably because of changes in the weather. Moreover, most
NYC cab drivers are able to determine how long they work and the cab companies track this information. Camerer et al. (1997) find across three samples that wages and time worked are negatively
correlated. Follow-up studies by Chou (2002) and Fehr and Goette (2007) discover similar effects
for taxi drivers in Singapore and bike messengers in Zurich Switzerland.
To account for their empirical results, as well as those seen in later studies, Camerer et al. (1997)
posits individuals work in reference to an income goal and that, once reached, causes individuals to
stop working for the day. Essentially, because individuals reach their goal relatively quickly on high
wage days they work less. Yet, before accepting the goal-based account as a universal, a rigorous
test with a more independent work environment and across cultures seems prudent.
To investigate the effect of time-varying wage rates on labor supply in a market with fewer frictions
and across cultures, we designed an online experiment allowing us to manipulate goals and wages
while monitoring worker productivity. Specifically, to test the goal mechanism directly we assign
a monetary goal to a random set of participants (Locke and Latham, 1990). If wages correlate
negatively with productivity for individuals in the monetary goal conditions, it suggests the influential nature of monetary goals (Helson, 1964; Kahneman and Tversky, 1979). To contrast the goal
condition, we include a control condition where individuals are told to do-your-best. We select this
control because research demonstrates specific goals lead to better performance than vague do-yourbest goals (Locke and Latham, 1990). An additional advantage of the present study is that we are
able to run the same experiment across dramatically different populations. Previous work is not
designed to investigate how identical incentive schemes can have different effects across populations.
In world where the outsourcing of interchangeable labor is becoming increasingly the norm, such an
investigation is a worthy extension.
2
3
Methods
The experiment takes place on Amazon Mechanical Turk (AMT). AMT is an online labor market
where requesters pay workers to complete human intelligence tasks (HITs). Total compensation
HITs depend on a flat fee (generally between 5 and 50 cents), and a bonus individually assigned by
the requester.6 Following the critiques presented by Levitt and List (2007) workers are not told they
are in an experiment until after they complete it.7 Workers are required to correctly answer English
comprehension questions prior to the start of the experiment. Workers failing one or more of the
English comprehension questions are not eligible to continue.8 The answers to these questions are
randomly generated and serve as the treatment assignment mechanism. This random assignment
allows all treatments to be posted at the same time, preventing autocorrelation in the treatments.
The English comprehension questions are interspersed within the initial survey.9 After completing
the survey, workers complete 5 practice tasks and begin the experiment.
3.1
Task and Treatments
The task we use is particularly well suited for the market as it is a task that is difficult for a computer to do and would reasonably be needed for a firm trying to keep digital copies of “strange”
documents. Workers transcribe pieces of an instruction manual from a 1996 Oldsmobile Cutlass
(essentially a series of CAPTCHAs). Each of these pieces is a jpeg file so workers cannot copy and
paste in the answer. After five transcriptions, paying 2 cents per transcription, workers are paid a
piece rate of two or five American cents for each transcription they complete. There is no ambiguity
in the currency; workers know they are being paid in American currency because they are paid
through Paypal. Additionally, competing HITs advertise wages in American currency (always to
our knowledge in cents). Connection speed should not matter. The HIT was coded in HTML which
is essentially static.10 Workers complete as many transcriptions as they want (with the maximum
being 100) and can quit anytime. Workers are only required to complete the first five transcriptions.
The maximum number of transcriptions is capped at 100 mostly for ease. Subjects have computers
of varying speed and we wanted to make sure load times would be reasonable even on mediocre
machines with weak connectivity.11
Figure 1: Example Task
As discussed above, before seeing the treatment screen, workers complete five practice transcriptions
paying 2 cents each. The amount paid for practice questions is the same for all treatments. In the
treatment screen, workers view 1 of 2 texts assigning them an unenforced goal. In“Do Your Best”
treatments (DYB), which serve as a control, workers view the text “Working for us your goal is to
6 For
an in-depth discussion of the bonus mechanism see Cooper and Johnson (2013).
course workers have the option of requesting their data to not be used; this is truthfully respected.
8 If this happens, workers are shown a screen that requests them to “return” the HIT. A returned HIT can be completed
by a different worker and any worker who dropped an HIT is no longer eligible to participate. Other than being no
longer eligible to participate, returning a HIT does not harm the worker.
9 A common complaint of AMT is that workers randomly enter text in hopes of bilking the requesters. Any worker
doing so, in our study, would likely answer the questions incorrectly and be kicked out of the experiment.
10 2 workers did lose connection but no others reported problems. These workers are paid based upon the number of
transcriptions they said they completed but are omitted from the analysis.
11 We tested this with an older laptop and wifi connection which we gradually moved away from the wireless internet
source.
7 Of
3
just do your best and do as many sentences as you can” while in GOAL treatments workers are
told “Working for us your goal is to make $2”. Because we vary the piece rate we end up with 4
treatments (DYB - 2 Cents, DYB - 5 Cents, GOAL - 2 Cents, and GOAL - 5 Cents).
Because our treatment relies on English comprehension we split the sample by national language
but also present pooled results. Summary statistics for each of the treatments and split by Western
English Speaking (WES) can be seen in Table 1. These treatments at a cursory glance maybe
problematic as they require turkers to be aware of the value of a dollar is. Normally this is a valid
criticism but in the case of AMT it is unwarranted. AMT is well established and has a long history
of operating with the US dollar. Most, if not all, workers will be aware of this as well as the buying
power of the dollar in their home country.
Table 1: Treatments
ALL
WES
Non - WES
DYB - 2 Cents
55
24
31
GOAL - 2 Cents
53
26
27
DYB - 5 Cents
64
25
39
GOAL - 5 Cents
62
32
30
ALL
234
107
127
The stakes of the experiment are not trivial - especially by AMT standards. With the bonus, workers
in the five cents treatments could make up to 4 dollars and 95 cents (or around a half day’s work in
India) in under an hour.12 Workers in the 2 cent treatments could top out at lower amount ($2.20)
but this amount is still generous by AMT standards. All currency is in United States denominations.
This is not probably does not create any confusion as AMT is an established labor market and the
working currency has been US dollars (or cents) since its inception.
4
Results
Over ninety percent of workers are in the USA (45 %) or India (46 %). The remainder hail from
a variety of nations but primarily developed western ones. As such, it comes as no surprise that
our workers are much more heterogeneous than usual laboratory subjects. The gender of workers is
evenly split; 49 % of workers indicated they are female. We gather information regarding the age
of workers, education and income. The average age of workers is 34 years. The modal income and
education of workers is between $12,500 and $25,000 per year and a bachelors degree.13
Average production (the number of transcriptions completed) is in Table 2. The distribution of
production by treatment and by Western English Speaking nations14 (WES) can be found in the
appendix (Figures 2, 3, and 4). The differentiation between WES and non-WES workers is essentially differentiating between high and low ability workers. Workers from WES nations spend
about 15 seconds less per transcription on HIT (t = −1.91) .15 Moreover, workers from non-WES
12 National
Sample Survey Organisation. (2011). Employment and Unemployment in India, 2009-10, NSS Sixty Sixth
Round. Report No. 537. New Delhi: Department of Statistics, Government of India. Retrieved February 1, 2013,
from http://mospi.nic.in/Mospi New/upload/NSS Report No 537.pdf (Login Required).
13 We present modes instead of averages here because workers are asked to select an appropriate category rather than
enter a number. Of course averages are available upon request.
14 Australia, Canada and the United States of America.
15 This is rough approximation as it includes time spent on the survey.
4
Table 2: Production by Treatment and English
ALL
WES
Non-WES
DYB - 2 Cents
60.60
(4.34)
58.92
(6.69)
61.90
(5.78)
GOAL - 2 Cents
62.81
(4.91)
70.00
(6.79)
55.89
(6.94)
DYB - 5 Cents
58.89
(4.18)
64.16
(6.68)
55.51
(5.36)
GOAL - 5 Cents
57.70
(3.97)
55.88
(5.72)
59.58
(5.57)
ALL
59.86
(2.15)
61.93
(3.21)
58.13
(2.9)
Standard deviations in parentheses.
nations spend over five and a half minutes more on the HIT than workers from WES nations (−2.81).
Initially, with pooled data we find no significant treatment effects. However when we separate by
WES, we find Goal - 2 Cents production of workers in WES countries to be greater than similar
workers in DYB - 2 Cents (p = 0.126) and Goal - 5 Cents (p = 0.057). Identical methods with
non-WES nations’ residents result in no significant differences.
In Table 3 we estimate worker production and whether or not they completed all the possible transcriptions as a function of the treatment, income, and whether or not they live in a WES nation.
Models 1 through 3 are tobits with the dependent variable being the number of transcriptions completed; models 4 through 6 are probits with the dependent variable being equal to 1 if the worker
completed all 100 transcriptions.16 We find no significant treatment effects in the regressions with
pooled data (models 1 and 4) but workers from a WES nation are more productive and are more
likely to complete the all 100 transcriptions. Workers reporting higher incomes complete fewer transcriptions but this primarily being driven by workers from WES nations; when we remove them from
the sample the coefficient estimate on income switches sign and is not significantly different from zero.
Returning now to productivity we find a single modest treatment effect occurring in Goal - 2 cents
and only when we separate the sample by WES. Workers in non-WES nations exhibit no change in
productivity across all treatments. This result is echoed in the probit models (4, 5, and 6) where
only the workers in WES nations have higher likelihood of completing all 100 transcriptions.17
16 To
save space, we do not present marginal effects here. However, they can be found in the appendix. Additionally,
alternative models (e.g., simple regression and ordered probits with a bin size of ten) and “kitchen sink” models are
also tested. Results did not change significantly.
17 Increasing the sample size may be worthwhile endeavor here and may result in significant differences. However we
argue these differences, except in the case of the Goal -2 cents in WES workers, would not be economically significant
- especially after controlling for income and nation of origin. Since only one treatment is large in magnitude and
significant, increasing the sample size would almost certainly only shrink standard errors down to the point where
we are reporting significant “zeros.”
5
Table 3: Empirical Results
Model 1
Model 2
Model 3
Model 4
Model 5
Model 6
Tobit
Tobit
Tobit
Probit
Probit
Probit
DYB - 2 Cents
4.12
(8.45)
5.17
(12.74)
2.99
(11.15)
0.10
(0.25)
0.13
(0.37)
0.06
(0.35)
DYB - 5 Cents
2.71
(8.00)
13.66
(12.65)
-5.02
(10.46)
0.01
(0.25)
0.23
(0.36)
-0.19
(0.34)
GOAL - 2 Cents
8.90
(9.42)
22.56†
(14.55)
-3.46
(12.08)
0.40†
(0.25)
0.7**
(0.35)
0.08
(0.36)
Income
-2.73*
(1.59)
-3.58*
(1.86)
0.88
(3.14)
-0.06
(0.05)
-0.06
(0.05)
-0.03
(0.11)
WES
10.97†
(6.99)
Constant
66.84***
(7.55)
75.83***
(11.78)
64.53***
(11.08)
-0.64***
(0.23)
-0.43
(0.32)
-0.56*
(0.34)
Obs
Log L
Subject Pool
Lower Limit
Upper Limit
233
-910.071
107
-393.520
126
-513.942
233
-138.524
107
-65.429
126
-71.995
ALL
WES
Non-WES
ALL
WES
Non-WES
2
70
2
37
0
33
NA
NA
NA
NA
NA
NA
0.34*
(0.20)
Standard Errors in parentheses. ***: p < .01, **: p < .05, *: p < .1 and †: p < .15. Lower
and Upper Limit are the number of workers who completed the 5 and 100 transcriptions.
6
5
Discussion and Conclusions
We find no evidence of an increase in productivity in any of the 5 cent piece rate treatments. This
result, while somewhat surprising, is consistent with much of the previous literature (Gneezy and
List, 2006; Kube et al., 2006; Hennig-Schmidt et al., 2010) and in contrast to Gómez-Miñambres
et al. (2012). Regarding our treatment, we find limited evidence in support of a negative correlation between time spent working and wage/piece rate. The treatment effect is only significant if
the sample is separated and each sub-sample estimated independently. This results in an odd but
non-trivial outcome where there are significant treatment and income effects in one sample that are
absent in the other.
Because income is insignificant in non-WES nations, income differences across the populations do
not explain the results (poor people in non-WES nations are as productive as comparatively richer
people in the same block of nations). Within the realm of WES nations, our results present additional evidence in support of the papers mentioned in the introduction. Positive wage shocks in
labor markets where workers set their own working hours, reduces worker productivity. How this
extends to labor markets where workers have set hours and/or wages based upon their time at work,
we are somewhat agnostic to. However, we posit the following: within the lab, there exists a norm
to not allow subjects to quit the experiment. While natural to most work environments it is far
from universal.
For instance, Gómez-Miñambres et al. (2012) allow subjects to goof off on the internet, and this
option has much relevance in the world outside the lab, but it does not replicate environments where
workers have almost literally any outside option they wish. What this means, is that goals can be
quite effective in workplaces where workers have set hours and high job search costs. Relax both
of these traits and ex-post it becomes less surprising that goals have little impact on productivity.
This is especially interesting because it demonstrates that as labor markets move in the direction of
a neoclassical firm (with homogeneous labor and no search costs ), the behavioral aspects become
less important. It follows that as the environments become more complicated and move away from
a neoclassical firm, the behavioral aspects becoming increasingly important.
The lack of significance in the treatment effect in pooled and non-WES sample also presents a limitation researchers should consider before running an experiment on AMT. Read and Loewenstein
(1995) suggest and present evidence of a diversification bias. An interesting but problematic twist of
this bias is also present in AMT: workers have multiple HITs they are able to complete at any given
time. Consequently, workers may select to both maximize their monetary earnings and also vary
the portfolio of the HITs they complete. Such a bias would be more pronounced in populations less
skilled in English. Presumably, workers in these populations glean less from instructions and learn
by doing rather than the instructions and if they are not good at the task, they quit. In these cases,
their stopping point would depend on a time per HIT threshold which would be independent of our
treatment. This explains why workers from non-WES nations are not impacted by the treatments;
they simply are not good enough at the task for the goals to matter.
However, such a bias requires the AMT labor market to also be more in the direction of “real” labor
market as it has a search feature which absent in the lab. This introduces another interpretation
of our results: workers in WES nations are treating AMT as a real labor market where they can
quit their job at anytime. This would be consistent with the observation that workers from WES
nations are significantly richer (t= -7.35) and are more likely to be working on AMT for primary
or secondary source of income (t=-2.37). This presents evidence to the contrary that turkers from
WES nations are on AMT for more recreational purposes.
7
References
Austin, James T, and Jeffrey B Vancouver (1996) ‘Goal constructs in psychology: Structure, process,
and content.’ Psychological bulletin 120(3), 338
Camerer, Colin, Linda Babcock, George Loewenstein, and Richard Thaler (1997) ‘Labor supply of
new york city cabdrivers: One day at a time.’ The Quarterly Journal of Economics 112(2), 407–441
Chou, Yuan K (2002) ‘Testing alternative models of labour supply: Evidence from taxi drivers in
singapore.’ The Singapore Economic Review 47(01), 17–47
Cooper, David J, and David B Johnson (2013) ‘Ambiguity in performance pay: An online experiment.’ Available at SSRN 2268633
Falk, Armin, and Markus Knell (2004) ‘Choosing the joneses: Endogenous goals and reference
standards.’ The Scandinavian Journal of Economics 106(3), 417–435
Fehr, Ernst, and Lorenz Goette (2007) ‘Do workers work more if wages are high? evidence from a
randomized field experiment.’ The American Economic Review pp. 298–317
Gneezy, Uri, and John A List (2006) ‘Putting behavioral economics to work: Testing for gift exchange
in labor markets using field experiments.’ Econometrica 74(5), 1365–1384
Goerg, Sebastian J, and Sebastian Kube (2013) ‘Goals (th) at work.’ Max Planck Institute for
Research on Collective Goods
Gómez-Miñambres, Joaquı́n (2012) ‘Motivation through goal setting.’ Journal of Economic Psychology 33(6), 1223–1239
Gómez-Miñambres, Joaquı́n, Brice Corgnet, and Roberto Hernán González (2012) ‘Goal setting and
monetary incentives: When large stakes are not enough.’ Technical Report
Helson, Harry (1964) Adaptation-level theory. (Harper & Row)
Hennig-Schmidt, Heike, Abdolkarim Sadrieh, and Bettina Rockenbach (2010) ‘In search of workers’
real effort reciprocitya field and a laboratory experiment.’ Journal of the European Economic
Association 8(4), 817–837
Hsiaw, Alice (2013) ‘Goal-setting and self-control.’ Journal of Economic Theory 148(2), 601–626
Kahneman, Daniel, and Amos Tversky (1979) ‘Prospect theory: An analysis of decision under risk.’
Econometrica: Journal of the Econometric Society pp. 263–291
Koch, Alexander K, and Julia Nafziger (2011) ‘Self-regulation through goal setting*.’ The Scandinavian Journal of Economics 113(1), 212–227
Kőszegi, Botond, and Matthew Rabin (2006) ‘A model of reference-dependent preferences.’ The
Quarterly Journal of Economics pp. 1133–1165
Kube, Sebastian, Michel André Maréchal, and Clemens Puppe (2006) ‘Putting reciprocity to workpositive versus negative responses in the field.’ University of St. Gallen Economics Discussion
Paper
Laisney, Francois, Winfried Pohlmeier, and Matthias Staat (1996) Estimation of labour supply functions using panel data: A survey (Springer)
Levitt, Steven D, and John A List (2007) ‘What do laboratory experiments measuring social preferences reveal about the real world?’ The journal of economic perspectives pp. 153–174
8
Locke, Edwin A, and Gary P Latham (1990) A theory of goal setting & task performance. (PrenticeHall, Inc)
(2002) ‘Building a practically useful theory of goal setting and task motivation: A 35-year odyssey.’
American psychologist 57(9), 705
Lucas Jr, Robert E, and Leonard A Rapping (1969) ‘Real wages, employment, and inflation.’ The
Journal of Political Economy 77(5), 721
Lynn, Michael (2002) ‘Turnover’s relationships with sales, tips and service across restaurants in a
chain.’ International Journal of Hospitality Management 21(4), 443–447
Read, Daniel, and George Loewenstein (1995) ‘Diversification bias: Explaining the discrepancy in
variety seeking between combined and separated choices.’ Journal of Experimental Psychology:
Applied 1(1), 34
Sun, S., Vancouver J.B., and J.M. Weinhardt (2014) ‘Goal choices and planning: Distinct expectancy
and value effects in two goal processes.’ Organizational Behavior and Human Decision Processes
Wu, George, Chip Heath, and Richard Larrick (2008) ‘A prospect theory model of goal behavior.’
Technical Report, mimeo
9
A
Figures
Figure 2: Productivity of Workers
Figure 3: Productivity of Non - WES Workers
10
Figure 4: Productivity of WES Workers
B
Marginal Effects
Table 4: Marginal Effects from Table 3
Model 1
Tobit
Model 2
Tobit
Model 3
Tobit
Model 4
Probit
Model 5
Probit
Model 6
Probit
DYB - 2 Cents
4.12
(8.63)
5.17
(13.19)
2.99
(11.3)
0.04
(0.09)
0.05
(0.14)
0.02
(0.11)
DYB - 5 Cents
2.71
(8.32)
13.66
(13.11)
-5.02
(10.7)
0.00
(0.08)
0.09
(0.14)
-0.06
(0.1)
Goal - 2 Cents
8.9
(8.88)
22.56*
(13.62)
-3.46
(11.64)
0.14
(0.09)
0.27**
(0.13)
0.03
(0.12)
Income
-2.73*
(1.6)
-3.58*
(1.93)
0.88
(3.28)
-0.02
(0.02)
-0.02
(0.02)
-0.01
(0.03)
WES
10.97
(6.89)
OBS
Subject Pool
Lower Limit
Upper Limit
0.12*
(0.07)
233
107
126
233
107
126
ALL
WES
Non-WES
ALL
WES
Non-WES
2
70
2
37
0
33
NA
NA
NA
NA
NA
NA
Standard Errors in parentheses. ***: p < .01, **: p < .05, *: p < .1 and †: p < .15. Lower
and Upper Limit are the number of workers who completed the 5 and 100 transcriptions.
11
C
C.1
Online Appendix
Instructions
Phase 1: Before we begin we would like to gather some information regarding our workers (you).
Please complete this short survey. When you are finished, please click the NEXT button.
What is your gender?
What is your age?
What country do you currently live in?
Which of the following best describes your highest achieved education level?
Over the weekend, Bob watched two football games. In the box below below, type the number of
football games that Bob watched over the weekend. Be sure to use a numeric character!
What is the total income of your household?
Why do you complete tasks in Mechanical Turk? Please check any of the following that applies:
Next year, Jack and Jill are planning on visiting Disneyland. Jill has been to Disneyland many times
while Jack has never been. In the box below below, type how often Jack has been to Disneyland.
Be sure to use a numeric character!
We are crowdsourcing the transcription of an instruction manual of the 1996 Oldsmobile cutlass.
You will be given a series of photocopies depicting short texts from the manual. Your job is to type
these short bits of text into the provided text box. We will spot check your work to make sure it is
of sufficient quality. You will be paid 2 cents for each of these practice transcriptions. After which,
you will start the actual task.
Please click the NEXT button to begin the practice messages. YOU MUST COMPLETE THESE
PRACTICE MESSAGES TO BE PAID.
You have now completed the practice questions.
Thank you for completing the practice assignments. You will again be given sentences that were
scanned from a book. We need you to type out each sentence exactly. You will be paid {2 cents, 5
cents} for every sentence that you type.
$2 - Working for us your goal is to make $ 2 dollars.
DYB - Working for us your goal is to just do your best and do as many sentences as you can.
In order to be sure you are reading the instructions, please input the number X in the input box
below.
Please note that you can quit at any time by clicking the quit button. Once/if you quit, you will be
given some brief instructions and then be told to submit the HIT. Your performance will be recorded
12
to pay you and with your consent may be used for research in the future.
The following study investigated the influence financial goals and financial incentives had on motivation to work and the decision to stop working. The reason we did not inform you in the beginning
that this was a study was because we wanted you to treat the work task as a real Amazon Mechanical
Turk job. Therefore, we are able to understand how people decide to continue working and stop
working in a real economic marketplace. Previous research shows that when people have a smaller
financial incentive but a difficult goal, they will work longer. We are trying to understand why this
effect occurs.
Survey Description and Consent Form: This study is conducted by a team of researchers at the
Haskayne School of Business, the University of Calgary. The principal investigator is Dr. Justin
Weinhardt, Assistant Professor at the Haskayne School of Business. This research protocol is approved by the Conjoint Faculties Research Ethics Board (CHREB). The purpose of this research is
to investigate and better understand the factors affecting decision making under different conditions.
Your participation in this study is voluntary and you may refuse to participate altogether or may
choose to withdraw from the study at any time.
In the experiment, you were asked type out various sentences. You were paid based on your performance. You will now be asked additional questions, consisting of general background, personality
and demographic data. The collected data will be kept on a password protected computer drive,
stored in a secured location and accessible only by the researchers for research purposes, including
the publication of scientific papers. Participation is completely voluntary, anonymous and confidential. No one except the principal investigator and the research team will be allowed to see any of the
answers to the questions. No personal identifying data will be collected in this study. There are no
names collected and attached to the responses. Only group information will be summarized for any
presentation or publication of results. In case you withdraw from this study, the data collected to
the point of withdrawal will be deleted. There were no foreseeable risks, harms, or inconveniences
associated with your participation in this study. The only cost on your part is the time you will
spend for participating in this survey.
Your responses may be used for research purposes, but only with your consent. Regardless of whether
or not you want your responses used for research purposes, you will be paid. If you agree with the
terms of this study, please press USE MY DATA FOR RESEARCH. If you do not agree for your
data to be used in a research project, please click DO NOT USE DATA FOR RESEARCH.
In the following survey, you will be presented with a number of different decision problems and fill
out a scale regarding personality and mental focus. If you have questions about this study, you can
contact Dr. Justin Weinhardt, the principal investigator, at [email protected].
If you have concerns about your rights as a research participant, you may contact the University of
Calgary Ethics Resource Officer through email: [email protected] or telephone: 403-210-9863.
USE MY DATA FOR RESEARCH
DO NOT USE MY DATA FOR RESEARCH
13