Linking multi-agent simulation to
experiments in economy
Re-implementing John Duffy’s model of
speculative learning agents
Experimental economics
[Hayek, "The use of knowledge in Society", 1945 ]
Production of a setting where individuals face an economic
situation that is similar to real ones – controlled and easy to
reproduce.
Limited possibilities of action
Control of motivation of agents / interaction mode / initial
information
Observation of behaviours of real humans
Interpretation of results
Comparison of behaviours with the theoretical setting
Drawing hypothesis explaining the differences
Simulations to design experiments
"
"
"
"
Usual protocol: humans interact only through computers to get an
absolute control of communication.
Choice of parameters to design experiments by running tests with
only computers
Test of behavioural hypothesis with learning IA in place of humans
following the experiments to reproduce the results
Mixing humans and AI in the same environment
• Duffy , J, 2001, Learning to Speculate: Experiments with
Articial and Real Agents, JEDC, 25, pp 295-319.
The Kiyotaki and Wright model: how to induce
speculative behaviours?
The aim of KW is to find an economy in which
the production and consumption rules would
Force the agents to exchange
Have them use a good as an exchange
value (enable them to store a good that has no
direct value to them)
Discretisation of action – best response
Knowing possession of the whole population
Kiyotaki N., Wright R., 1989, On money as a medium of exchange,
Journal of Political Economy 97, 924-954.
Situation of the agents
3 types of agents : 1, 2, 3
3 goods : 1, 2, 3
Agent i consumes good i
Agent i produces good i +1
Consumption > gain in utility = u.
Agent produces iff it has just consumed
Production > exchange > consumption
A time-step
storage
At the end of a time-step: an agent 1 possesses only
good 2 or 3
Only stores ONE unit of non-divisible good
Storage costs
0< c1 <c2 < c3 < u
“discount factor”  : 0< < 1 – end of the exchange
rounds
A time-step
exchange
A
1
1
B
3
3
3
1
1
3
3
1
1
2
C
1
2
2
1, 3
3
3
2
2
2
Bilateral decentralised negotiation - choose to exchange
or not.
No « simple exchange » (A is forbidden) > at least three
agents have been involved when everyone is satisfied
Fundamental and speculative behaviours
Expected gain for agent i (imagine that it will get good i at time t + 1
by exchanging against the good it possesses now):
Possesses good (i+1): γi +1 = - c i +1 + u.
Possesses good (i+2): γi +2 = - c i +2 + u.
Fundamental = when facing an exchange accepts to store the good that
gives the best expected gain.
Speculative = when facing an exchange accepts to store the good that
gives the worst expected gain if there is a higher chance to perform the
exchange to get i at the next time-step.
Fundamental for agents 1 and 3 is to refuse to exchange i+1 for i+2
Fundamental for agents 2 is to accept to exchange i+1 for i+2
Speculative: depends on actual probabilities of exchange
Fundamental and speculative behaviours
Notation:
Proportion of agents i possessing the good they produce: pi, and
possessing the other good: (1-pi)
(s1,s2,s3) the set of strategies by agents 1, 2 and 3,
Si = 0 if agents refuses to get i+2 when facing the opportunity
Si = 1 if agents accepts to get i+2 when facing the opportunity
Solution by KW: the agent decides if it will speculate or not by anticipating its
ability to exchange at the next time-step.
>> Strategic equilibrium depending on u, c1, c2, c3,  and
(p1,p2,p3) - either (0,1,0) ou (1,1,0)
Issue for Duffy:
agents have no complete knowledge and learn anticipated gain through
experience
experience is individual – not all agents of the same type choose the same
Reproduction of the speculative setting
Several production of this model put in a distributed setting (multiagent type models)
•Marimon, R., E.R. McGrattan and T.J. Sargent, 1990, Money as a medium of
exchange in an economy with artificially intelligent agents, Journal of Economic
Dynamics and Control 14, 329-373.
• Basci, E., 1999, Learning by imitation, Journal of Economic Dynamics and
Control 23, 1569-1585.
Several production of this model put in an experimental setting
• Duffy, and J. Ochs, 1999a, Emergence of money as a medium of exchange: An
experimental study, American Economic Review 89, 847-877.
• Duffy, J. and J. Ochs, 1999b, Fiat money as a medium of exchange: Experimental
evidence, working paper, University of Pittsburgh.
Duffy: getting close to experimental setting for
comparison and extension
Limited number of agents 16 or 24
Short simulation: only repetition of 10 games in a row
Chooses setting where (1,1,0) is the solution profile
agents of type 1 “should” play speculative
agents of type 2 and 3 “should” play fundamental
Same setting is used for artificial agents and humans,
Manipulation of the setting to influence learning
change proportion for different probabilities of meeting
automate some of the behaviours
>> once the simulation shows how interesting the setting is, reproduce
it with real humans / mix artificial and real agents
Learning algorithm for the agents
A simulation is a set of 10 games with probability β to stop at each
time-step, agents cannot exchange when it stops and they start with
their production good again
when an agent A meets another agent (B)
– if B proposes the good A owns: no exchange
– B has good i: A proposes the exchange
– otherwise depends on memory
ν i+1 = Σ (I i+1) * γi +1 - Σ (I i+2) * γi +2
ν i+2 = Σ (I i+2) * γi +2 - Σ (I i+1) * γi +1
Where “I i+1 = 1 for a time-step where i succeeded in getting i
+1 with i at start and I i+1 = –1 in the opposite case”
x = ν i+1 - ν i+2
And probability to refuse: exp x / (1 + exp x)
Reproducing the learning algorithm in simulations
Homogeneous Simulations
Ambiguity to interpret the algorithm:
“I i+1 = 1 for a time-step where i succeeded in getting i with i+1 at
start and I i+1 = 0 in the opposite case”
Algo1: any time the agents has possessed the good
Algo2: any time he could have exchanged and it was refused (=if it
had had the other good, the exchange would have been accepted)
Algo3: Algo1 but doesn’t’ exchange back to get its production good
Constrained simulations
Only agents of type 1 have to learn and the others are automated
Experimental data (8-8-8)
Agents type 1 offers
2 for 3
Agents type 2 offers 3
for 1
Agents type 3 offers 1
for 2
first half of
the
experiment
second half
of the
experiment
first half of the
experiment
second half
of the
experiment
first half of
the
experiment
second half of
the
experiment
R1
0.13
0.18
0.98
0.97
0.29
0.29
R2
0.38
0.65
0.95
0.95
0.17
0.14
R3
0.48
0.57
0.96
1.00
0.13
0.14
R4
0.08
0.24
0.92
0.98
0.12
0.02
R5
0.06
0.32
0.93
0.97
0.25
0.18
Average
on R1R5
0.23
0.37
0.95
0.96
0.20
0.16
Agents type 1
Agents type 2
Agents type 3
Time-step
for the first
half
Time-step
for the
second half
Time-step
for the first
half
Time-step
for the
second half
Time-step
for the first
half
Time-step
for the
second half
Average on A1-A5
0,19
0,32
0,77
0,99
0,22
0,04
SIM 1
Rational
agents
Average
speculation
rate
0.74
0.68
0.80
0.93
0.73
0.81
MSD
0.03
0.10
0.08
0.09
0.01
0.11
SIM 2
varrational
agents
Average
speculation
rate
0.45
0.42
0.53
0.47
0.42
0.52
MSD
0.19
0.27
0.14
0.27
0.3
0.24
SIM 3
Stable
agents
Average
speculation
rate
0.68
0.77
0.76
0.79
0.66
0.66
MSD
0.07
0.12
0.01
0.09
0.04
0.12
Agents type 1
Agents type 2
Agents type 3
Time-step
for the first
half
Time-step
for the
second half
Time-step
for the first
half
Time-step
for the
second half
Time-step
for the first
half
Time-step
for the
second half
Duffy:
Average
on 5
sessions
Average
speculation
rate
0.62
0.73
1.00
1.00
0.00
0.00
SIM 1’
Rational
agents
Average
speculation
rate
0.91
1.00
1.00
1.00
0.00
0.00
MSD
0.04
0.01
0.00
0.00
0.00
0.00
SIM 2 ‘
varrational
agents
Average
speculation
rate
0.80
1.00
1.00
1.00
0.00
0.00
MSD
0.15
0.05
0.00
0.00
0.00
0.00
SIM 3 ‘
Stable
agents
Average
speculation
rate
0.80
0.88
1.00
1.00
0.00
0.00
MSD
0.00
0.05
0.00
0.00
0.00
0.00
Agents of type 1 – algo1
Agents of type 1 – algo2
Agents of type 1 – algo3
Reasons why comparison hasn’t work (but will)
"
My mistake in reproducing the setting
"
Dependence to random generator
"
Not enough meetings or agents to establish comparison
with experiments
"
"
"
My simulations: same results
Not enough meetings, possibility to exchange with such a
simple algorithm
Comparison at the macro level is not enough > use the
actions one-by-one (need of the whole set of
experimental data).