APPENDIX
Probabilistic Decision Function
In the PROBABILISTIC case, the probability 𝑝𝐴 that the principal chooses market A is
given as
10𝑎
𝑝𝐴 = 𝑒 ⁄ 10𝑎
𝑒
+ 𝑒 10𝑏
with a and b being the prices in markets A and B, respectively. The exact function is not
revealed to the participants either ex-ante or ex-post, so the exact definition is unlikely
to influence the participant too much.
Starting Price Scenarios
Table A1: Scenarios for starting prices of markets
Probability A
67.9
67.9
67.9
67.9
67.9
67.9
67.9
67.9
Probability B
32.1
32.1
32.1
32.1
32.1
32.1
32.1
32.1
Price A
81.8
73.1
50
50
22.3
73.1
22.3
62.2
Price B
37.8
77.7
26.9
77.7
50
50
26.9
18.2
Mechanical Turk Users and Low Stakes
To appreciate the value of Mechnical Turk as experimental platform, a couple of points
should be noted: (1) Mechnical Turk provides a more representable population than a lab
(Paolacci et al. 2010). (2) External validity has been addressed in methodological papers
in, e.g., economics (Horton et al., 2011; Amir et al. 2012), political science (Berinsky et al.
2012), and psychology (Buhrmester et al. 2011): These papers report a close match of
experimental results from Mechnical Turk and the lab for a wide variety of decision
tasks and games. Fehr et al. (2013) present a labor market experiment where stakes
vary by factor 10 without substantial behavioral effect. Similarly, Slonim and Roth
(1998) use a factor of 25 without substantial behavioral effect for inexperienced subjects.
(3) Our payout is above average for Mechnical Turk. In this crowd labor market, it is
considered fair and induces conscious behavior. (4) Stakes are about the same for all our
treatments and both experiments. While low stakes might be speculated to induce
arbitrary behavior, this cannot explain treatment /experiment differences. (5) Low
average payouts are standard in real live implementations of decision markets where
users are motivated by reputation, which is also a driver in Mechnical Turk.
Amir, O., Rand, D.G., Gal, Y.K., 2012. Economic Games on the Internet: The Effect of $1 Stakes. PLoS ONE 7(2), e31461.
Berinsky, A.J., Huber, G.A., Lenz, G.S., 2012. Evaluating Online Labor Markets for Experimental Research:
Amazon.com’s Mechanical Turk. Political Analysis 20(3), 351-368.
Buhrmester, M., Kwang, T., Gosling, S.D., 2011. Amazon's Mechanical Turk: A New Source of Inexpensive, Yet HighQuality Data? Perspectives on Psychological Science 6(1), 3-5.
Fehr, E., Tougareva, E., Fischbacher, U. 2014. Do high stakes and competition undermine fair behaviour? Evidence from
1
Russia. Journal of Economic Behavior and Organization, 108, 354-363.
Horton, J.J., Rand, D.G., Zeckhauser, R.J., 2011. The online laboratory: conducting experiments in a real labor market.
Experimental Economics 14(3), 399-425.
Paolacci, G., J. Chandler and P. Ipeirotis, (2010) “Running Experiments on Amazon Mechanical Turk,” Judgment and
Decision Making (5:5), pp. 411–419.
Slonim, R., Roth, A.E., 1998. Learning in High Stakes Ultimatum Games: An Experiment in the Slovak Republic.
Econometrica 66(3), 569-596.
Experiment 1: ANOVA Results, Post-hoc Test
Running an ANOVA, we see that there is a significant effect of the treatment and on the
trading error [F(3, 1544)=10.94, p < 0.1%] for the states with manipulation incentives.
The pair-wise comparison shows the same results as the regression; participants in the
treatments DETERMINISTIC and PROBABILISITC behave similar to each other but
different to the treatments RANDOM and UNCONDITIONAL in states with
manipulation incentives.
Behavioral Metric, Measuring the Extent of Manipulation in Experiment 1
As additional metric for manipulation, we analyze individual trading behavior by looking
at the ratio of trades that participants do to drive the price towards the correct value. In
order to detail the treatment differences we use linear regressions on the ratio of correct
trades per market. As baseline we use the RANDOM treatment.
Table A2 gives the regression results. As expected we see that participants learn to
improve their trading as with an increasing number of rounds played, their share of
correct trades increases. As expected, in the RANDOM treatment there is no difference
between the Manipulative Situations and the Non-Manipulative Situations as the
dummy “Non-Manipulative Situations” is small and insignificant. If we look at the
interaction effects, we see that in treatments DETERMINISITC and PROBABILISITC,
users have a higher ratio of trades driving the price in the right direct in Nonmanipulative Situations. As expected this is not the case in the UNCONDITIONAL
treatment. We conclude that participants improved their trading skill and
understanding of the task and that there are clear behavioral differences between
treatments.
2
Table A2: OLS regression on the ratio of trades driving the price towards the correct value by
treatment and situation for Experiment 1
Estimate
Std. Error
(Intercept)
0.58
***
0.02
DETERMINISTIC
PROBABILISITC
-0.08
***
0.02
-0.11
***
0.02
UNCONDITIONAL
-0.05
*
0.02
Non-Manipulative Situations
0.02
DETERMINISTIC*Non-manipulative Situations
0.08
*
0.03
PROBABILISITC*Non-manipulative Situations
0.08
**
0.03
UNCONDITIONAL*Non-manipulative Situations
0.04
0.02
0.04
Round
0.01
*
0.00
Note: * significance at 5%, ** significance at 1%, and *** significance at 0.1%. 6,184 observations and an AdjR2 of 0.013. A two-sided censored Tobit regression [0-1] yields the qualitatively same results.
Analyzing the timing of trades in Experiment 1 and Experiment 2
Experiment 1 did not impose a time constraint on participants. To illustrate the
frequency of trades over time, we binned the number of trades according to their
execution time relative to the start of the trading round. Figures A1 and A2 have the
results. In experiment 1, we see that most participants were done trading after a short
period. 94% of all trades happened in the first 60 seconds (98% in the first 120 seconds).
In order to keep participants synchronized, we had to decide on fixed time frame for the
trading periods in Experiment 2. In the test rounds participants had 120 seconds to get
used to interface and experiment. Thereafter a trading round lasted 60 seconds. One
clearly sees that trading drops off towards the end. Thus, time constraints seem not to
play a major role in explaining participant behavior.
Figure A1: Timing of trades in Experiment 1.
3
Figure A2: Timing of trades in Experiment 2.
Participant Compensation
The return from manipulation exceeded the return from honest revelation on average.
As we paid relative to performance per treatment there was a significant incentive to
manipulate. However average payment per treatment remained the same.
Table A3: Average sum of all earning per treatment in Experiment 1.
Treatment
Virtual Currency
UNCONDITIONAL
-9.44
DETERMINSTIC
1.97
PROBABILISTIC
0.53
RANDOM
-6.13
Instructions for Experiment 1
Welcome
You will participate in an experiment about prediction and decision making. The experiment
takes about 20 minutes.
Your payment will be $1.00 U.S. in base pay and a bonus. Your bonus depends on your
performance. It ranges from $0 to $3.00, where the average bonus is $1.50. We encourage
you to follow the instructions carefully, make good decisions, and earn as much money as
possible.
You will only be paid if you complete the experiment in one pass (about 20 minutes) without
interruption. If you can’t do so right now, please don’t start now but return later.
You can participate only once.
When you use the “backward” function of your browser or close your browser window, you
will be excluded from the experiment without payment.
4
Read the instructions carefully, we will test your understanding. If you fail the test we
cannot accept your HIT.
Instructions page 1 of 4 - Overview
Please follow these instructions carefully – you will have to demonstrate your understanding
by answering some questions correctly.
You will participate in 8 rounds. Each round proceeds along the same 3 phases:
1. Information
2. Trading
3. Outcome
The scenario is the following: There are two bingo cages (labeled A and B), each holding
black and white balls. There is a principal who in phase 3 decides to draw the ball either
from bingo cage A or B, and a computer will draw a ball from that bingo cage.
In phase 1, you receive private information on the number of black and white balls. In phase
2, you can trade in both markets. In phase 3 the principal makes a decision for either bingo
cage and a ball is drawn. Also the outcome of the draw will be revealed.
During the experiment you can earn e-dollars. In the end, your earnings are converted into
U.S. dollars. The final bonuses will be between $0 and $3, with the average bonus set to
$1.50. The exact conversation factor will be determined based on the performance of today's
players. The bonus will be paid out by tomorrow.
The better trading success, the higher your earnings.
The rounds are independent of each other: Each round, the number of black and white balls
per bingo cage and your information is independent of the previous rounds.
Instructions page 2 of 4 – Private information
The bingo cages hold 100 balls each,
some are black, and the others are
white.
Once a round begins, you will
truthfully be told how many black
and white balls are in each bingo cage.
5
At the end of each round in phase 3, a ball (black or white) is drawn by the computer.
Instructions page 3 of 4 – Trading
In the trading phase, you can trade tickets
which are essentially bets on the
probability of drawing a black ball from
the bingo cage.
• You can choose whether to trade, when
to trade, and how much to trade.
• You may change your mind about your
predictions too.
Typically, a way to assess the value of a
ticket is to evaluate the probability of the
outcome associated with it. If one thinks
there is an 81% chance of the ball being
black, then a ticket is worth 1 e-dollar to
him 81% of the time, and zero e-dollars
19% of the time. On average, it is worth
81% × 1 + 19% × 0 = 0.81 e-dollars to
him. This is the expected payout, to
maximize this, the best strategy is to buy
tickets when the market price is below
0.81 e-dollars and sell tickets when the
price is above 0.81 e-dollars.
The screenshot on the left shows the trading interface. At the top, you see your private
information as on the previous screen. Below you see a price graph that is updated each time
you make a trade.
At any time, there is a current price at which you can buy or sell tickets. The volume of a
trade is always 5 tickets. You trade by clicking the “Buy (+5)” button or the “Sell (-5)”
button. When one buys, the price rises; when one sells, the price falls.
The trading screen provides additional information:
• Your holdings: Each round you start with 0 units. You may not hold less than -50 units or
more than 50 units.
• Cash: When you buy, you pay in cash; when you sell, you get cash. Your cash balance
might be positive or negative.
• Expected Value: This is your expected payout which equals your expected return minus the
initial cash endowment of 50 e-dollars.
TREATMENT DIFFERENCES: DETERMINISTIC, PROBABILISTIC, RANDOM
6
Instructions page 4 of 4 – Outcome
The screenshot on the left
illustrates how you are informed
on the outcome of a round. You
receive a payment based on your
performance in trading.
After trading closes, the
principal makes a decision for
either A or B.
Sum: 6.13 e-dollars
PROBABILISTIC
The principal bases his decisions on the final price in each market: The higher a price
is compared to the other, the more likely the respective bingo cage will be chosen.
If the final price for ticket A is way higher than for ticket B, the principal will very
likely (but not with certainty) decide to draw from bingo cage A. If the price for ticket
B is higher than the price for A, the principal will likely decide for B. If final prices in
both markets are equal, the decision is entirely random with equal probability for A
and B.
7
DETERMINISTIC
The principal bases his decisions on the final price in each market: He chooses the
bingo cage where the final market price is higher. If final prices in both markets are
equal, the decision is entirely random with equal probability for A and B.
RANDOM
After trading closes, the principal makes a decision for either A or B. The decision is
entirely random with equal probability for A and B.
In the example the decision is for market B. Thus, the value of your holdings of ticket B is
determined. As the decision was for market B in this example, the trading in market A is not
relevant for your bonus. Each ticket is worth 1 e-dollar if the ball is black and zero e-dollars
otherwise. In the example, a ticket is worth 1 e-dollar and, thus, your total holdings of 15
tickets is worth 15 e-dollars. If your holdings were -10 tickets, and the ball was black that
would be worth -10 e-dollars.
You received an initial endowment of 50 e-dollars per market. This endowment is now
subtracted. The cash position is 41.13 and is added to the holdings (worth 15 e-dollars) which
sums up to 6.13 e-dollars you earned in this round.
TREATMENT DIFFERENCES: UNCONDITIONAL
Instructions page 4 of 4 – Outcome
The screenshot on the left
illustrates how you are informed
on the outcome of a round. You
receive a payment based on your
performance in trading.
The computer draws twice. Once
from bingo cage A once from
bingo cage B. The draws are
independent. Each ticket is worth
1 e-dollar if the ball for that bingo
cage is black and zero e-dollars
otherwise.
Sum: 2.86 e-dollars
In the example, a ticket of market
A is worth 1 e-dollar as the draw
was black. Your holdings were -5
tickets, hence they were worth -5
e-dollars. Also in this example
tickets in market B were worth 1
e-dollar, thus your holdings are
8
worth 15 e-dollars. You received an initial endowment of 50 e-dollars per market. This
endowment is now subtracted. The cash position is 51.73 + 41.13 - 2*50 = -7.14. It is added
to the holdings (-5+15 = 10) which sums up to 2.86 e-dollars you earned in this round.
Instructions for Experiment 2
Welcome
You will participate in an experiment about prediction and decision making. The experiment
takes about 25 minutes.
Your payment will be $1.00 U.S. in base pay and a bonus. Your bonus depends on your
performance. It ranges from $0 to $3.00, where the average bonus is $1.50. We encourage
you to follow the instructions carefully, make good decisions, and earn as much money as
possible.
You will only be paid if you complete the experiment in one pass (about 25 minutes) without
interruption. If you can’t do so right now, please don’t start now but return later.
You can participate only once.
When you use the “backward” function of your browser or close your browser window, you
will be excluded from the experiment without payment.
Read the instructions carefully, we will test your understanding. If you fail the test we
cannot accept your HIT.
Instructions page 1 of 4 - Overview
Please follow these instructions carefully – you will have to demonstrate your understanding
by answering some questions correctly.
You will be matched with another participant. The two of you form a group. Your identity
will remain unknown to the other participant; you both get the exact same instructions.
9
You will participate in 8 rounds. Each round proceeds along the same 3 phases:
1. Information
2. Trading
3. Outcome
The scenario is the following: There are two bingo cages (labeled A and B), each holding
black and white balls. There is a principal who in phase 3 decides to draw the ball either
from bingo cage A or B, and a computer will draw a ball from that bingo cage.
In phase 1, you receive private information on the number of black and white balls. In phase
2, you can trade in both markets. In phase 3 the principal makes a decision for either bingo
cage and a ball is drawn. Also the outcome of the draw will be revealed.
During the experiment you can earn e-dollars. In the end, your earnings are converted into
U.S. dollars. The final bonuses will be between $0 and $3, with the average bonus set to
$1.50. The exact conversation factor will be determined based on the performance of today's
players. The bonus will be paid out by tomorrow.
The better trading success, the higher your earnings.
The rounds are independent of each other: Each round, the number of black and white balls
per bingo cage and your information is independent of the previous rounds.
Each phase has a time limit. The first test-round is a little longer (3 minutes) so you can get
used to the interface, the following six rounds are quicker (2 minutes each).
Instructions page 2 of 4 – Private information
The bingo cages hold 100 balls each,
some are black, and the others are
white.
Once a round begins, you will
truthfully be told how many black
and white balls are in each bingo cage.
At the end of each round in phase 3, a ball (black or white) is drawn by the computer.
10
Instructions page 3 of 4 – trading
In the trading phase, you can trade tickets which are essentially bets on the probability of
drawing a black ball from the bingo cage.
• You can choose whether to trade, when to trade, and how much to trade.
• You may change your mind about your predictions too.
Typically, a way to assess the value of a ticket is to evaluate the probability of the outcome
associated with it. If one thinks there is an 81% chance of the ball being black, then a ticket is
worth 1 e-dollar to him 81% of the time, and zero e-dollars 19% of the time. On average, it is
worth 81% × 1 + 19% × 0 = 0.81 e-dollars to him. This is the expected payout, to maximize
this, the best strategy is to buy tickets
when the market price is below 0.81 edollars and sell tickets when the price is
above 0.81 e-dollars.
The screenshot on the left shows the
trading interface. At the top, you see
your private information as on the
previous screen. Below you see a price
graph that is updated each time you make
a trade.
At any time, there is a current price at
which you can buy or sell tickets. The
volume of a trade is always 5 tickets. You
trade by clicking the “Buy (+5)” button
or the “Sell (-5)” button. When one buys,
the price rises; when one sells, the price
falls.
The trading screen provides additional
information:
• Your holdings: Each round you start
with 0 units. You may not hold less than 50 units or more than 50 units.
• Cash: When you buy, you pay in cash; when you sell, you get cash. Your cash balance
might be positive or negative.
• Expected Value: This is your expected payout which equals your expected return minus the
initial cash endowment of 50 e-dollars.
11
Instructions page 4 of 4 – outcome
The screenshot
on the left
illustrates how
you are informed
on the outcome
of a round. You
receive a
payment based
on your
performance in
trading.
After trading closes, the principal makes a decision to draw from either market A or
market B.
In the example the decision is for market B. Thus, the value of your holdings of ticket B is
determined. As the decision was for market B in this example, the trading in market A is not
relevant for your bonus. Each ticket is worth 1 e-dollar if the ball is black and zero e-dollars
otherwise. In the example, a ticket is worth 1 e-dollar and, thus, your total holdings of 5
tickets is worth 5 e-dollars. If your holdings were -10 tickets, and the ball was black that
would be worth -10 e-dollars.
You received an initial endowment of 50 e-dollars per market. This endowment is now
subtracted. The cash position is 48.53 and is added to the holdings (worth 5 e-dollars) which
sums up to 3.53 e-dollars you earned in this round.
12
Instructions after partner matching
The experiment starts in 60 seconds. In the screenshot below you see one typical trading
situation.
The expected value of Market A is higher than the expected value of market B. That means, if
market A is selected you earn more in expectation.
DETERMINISTIC
However, if the trading time is up, market B has a higher price than market A. Hence, market
B will be selected and market A will be voided.
RANDOM
There is an equal chance of either market A or market B being selected and the other voided.
REMEMBER your bonus depends on your trading success.
13