Outline
Social Dilemmas
1 Social dilemmas
Game-theoretic models
Simulating social dilemmas
Gennaro Di Tosto
BBL-515
[email protected]
2 Cooperation among strangers
Image-score
Tag-based systems
3 Communication
SimNorm
September 13, 2012
4 References
INFOMSOCS (Lesson 3)
13-09-12
1 / 33
INFOMSOCS (Lesson 3)
Social dilemmas
13-09-12
Social dilemmas
2 / 33
Game-theoretic models
Altruistic behaviour and the problem of cooperation
Outline
• Performing actions that are
beneficial to others, at a cost to
oneself (c < b).
1 Social dilemmas
Game-theoretic models
Simulating social dilemmas
• Problem: without the possibility to
assess the trustworthiness of a
partner, cooperation is prone to
exploitation.
2 Cooperation among strangers
Image-score
Tag-based systems
• Behavioural studies prove
cooperation’s rate is higher than
theory prediction.
3 Communication
SimNorm
Prisoner’s dilemma
Strategy
C
D
D
R R
S T
T
S
P
P
• T >R>P>S
• 2R > (T + S)
Reciprocity
• Accepted solutions: reciprocal
altruism and reputation.
4 References
direct
INFOMSOCS (Lesson 3)
C
13-09-12
3 / 33
INFOMSOCS (Lesson 3)
13-09-12
indirect
4 / 33
Social dilemmas
Game-theoretic models
Social dilemmas
Contributing to society
Human potential for cooperation
• Public goods in society: clean air/pollution, clean cities/littering,
information on the web, joining the army, etc.
• The dilemma is to have individuals investing effort in creating a public
good, while everybody, including those who did not contribute
(free-riders), benefit.
πi = ω − xi + r
xj
interactions, there is the
potential for cooperation.
• People don’t leave in a vacuum
• The cost associated to one’s
j=1
(in-)actions are not always
monetary fines. Loss of status
or credibility are consequences
people want to avoid.
ω initial endowment;
xi individual contribution;
r marginal per capita
return.
INFOMSOCS (Lesson 3)
• Even in anonymous, one-shot
and they are concerned about
their social image (Bateson,
Nettle, and Roberts 2006; Haley
and Fessler 2005).
Linear public good
N
X
Game-theoretic models
From (Bateson, Nettle, and Roberts
2006).
Results from Isaac and Walker (1988).
13-09-12
Social dilemmas
5 / 33
INFOMSOCS (Lesson 3)
Game-theoretic models
13-09-12
Social dilemmas
Altruistic punishment
6 / 33
Simulating social dilemmas
The role of agent-based social simulation
• Considerations about fairness and moral
values trigger behaviour that bring
people to act as enforcers of norms,
promoting cooperation in a social
group.
• The mismatch between theoretical predictions and behavioural data is
neither news nor dramatical.
• Experimental data show that
• However some of the models behind people’s strategic behaviour are
participants in a public good game are
willing to sacrifice part of their
endowment to punish free-riders.
still missing.
• The mechanism approach of ABSS and generative social sciences can
be useful in generating testable hypothesis.
• Punishment of free-riders is a
second-order problem. But people to
act as altruistic punisher (punish even if
costly without the prospect of material
benefit).
INFOMSOCS (Lesson 3)
13-09-12
From Fehr and Gachter
(2002).
7 / 33
INFOMSOCS (Lesson 3)
13-09-12
8 / 33
Social dilemmas
Simulating social dilemmas
Social dilemmas
Axelrod’s original tournament
TIT-FOR-TAT
• Axelrod “outsourced” the research of a viable solution to the problem
of cooperation (Axelrod 1984).
• Tournament: algorithms playing a finite repeated prisoners dilemma
game.
• Instructions: each submitted algorithm would play 200 rounds against
each other algorithm, itself and an algorithm that randomly plays
cooperate and defect.
• Payoffs:
Strategy
C
D
C
3,3
5,0
13-09-12
Social dilemmas
Tournament’s entry
Cumulative payoff
Mutual Cooperation
TIT-FOR-TAT
Worst entry
Strategy random
Mutual Defection
600
504
282
276
200
• Results: Tit-For-Tat, winner of the tournament, was also the simplest
D
0,5
1,1
strategy – start cooperating, then imitate the opponent previous
move.
• TFT did not do better against each other strategy, but did the best
Thus, Mutual defection: 200 / Mutual cooperation: 600.
INFOMSOCS (Lesson 3)
Simulating social dilemmas
on average.
9 / 33
INFOMSOCS (Lesson 3)
Simulating social dilemmas
13-09-12
Social dilemmas
TIT-FOR-TAT (II)
10 / 33
Simulating social dilemmas
Implementing Axelrod’s tournament
1
always defects
2
tit-for-tat
3
cooperate 30% of the time
4
cooperate 60% of the time
1
The strategy space: from which agents’ strategies are selected.
5
always cooperate
2
The interaction process: types of environment.
3
The adaptive process: govern the changes in agents’ strategies over
time.
Details about this line of research can be found in Cohen, Riolo, and
Axelrod (1999).
In there the authors describe:
100 agents playing 50 games against each other. At the end of the
iterations 10% of the population is discarded and replaced. Strategies for
the replaced agents are drawn from the existing population of agents.
Agents who have higher average payoff have a higher probability to be
drawn. In scenario without noise TFT is the best performing strategy.
INFOMSOCS (Lesson 3)
13-09-12
11 / 33
INFOMSOCS (Lesson 3)
13-09-12
12 / 33
Social dilemmas
Simulating social dilemmas
Social dilemmas
Strategy space
Interaction process
Every agent strategy is defined by three variables:
Selecting agents’ neighbours:
• 2DK (2-Dimensional topology, Keep neighbors.) a Von Neumann
i the probability of cooperation on the initial move.
neighbourhood.
p the probability of cooperation after the other player
cooperated.
q the probability of cooperation after the other player defected.
Hence, the classical strategies are specified as:
• i = p = 1; q = 1: Always cooperate (all-C).
• i = p = 0; q = 1: Anti-Tit-For-Tat (aTFT).
• RWR. (Random-With-Replacement.) at each period each agent
• i = p = 0; q = 0: Always defect (all-D).
Social dilemmas
2DK, in that each agent has exactly 4 other agents as neighbors
(symmetric neighbors), but each agent’s neighbors are chosen at
random.
neighbours for an entire run (as in 2DK and FRNE), but unlike 2DK
and FRNE the neighbourhood relation is not symmetrical.
(TFT).
13-09-12
• FRNE. (Fixed Random Network, Equal number of neighbors.) like
• FRN. (Fixed Random Network.) each agent has a fixed set of
• i = p = 1; q = 0: Mirror opponent’s last action, i.e., Tit-for-Tat
INFOMSOCS (Lesson 3)
Simulating social dilemmas
selects 4 other agents and then replaces them in the next period.
13 / 33
INFOMSOCS (Lesson 3)
Simulating social dilemmas
13-09-12
Social dilemmas
Adaptive process
14 / 33
Simulating social dilemmas
Tournament’s procedure
• Imitation: copies perfectly the strategy of some other
better-performing agent it has played in the current period.
• BMGA. (Best-Met Genetic Algorithm hybrid.) introduces a
comparison error and a copy error.
• 1FGA. (1-Fixed Agent Genetic Algorithm.) The 1FGA adaptive
process is designed to be a global learning version of BMGA. an agent
can learn from any agent in the population.
INFOMSOCS (Lesson 3)
13-09-12
15 / 33
INFOMSOCS (Lesson 3)
13-09-12
16 / 33
Social dilemmas
Simulating social dilemmas
Cooperation among strangers
Results
Outline
1 Social dilemmas
Game-theoretic models
Simulating social dilemmas
2 Cooperation among strangers
Image-score
Tag-based systems
3 Communication
SimNorm
4 References
INFOMSOCS (Lesson 3)
13-09-12
Cooperation among strangers
17 / 33
INFOMSOCS (Lesson 3)
Image-score
13-09-12
Cooperation among strangers
Evolution of indirect reciprocity
18 / 33
Tag-based systems
Groups
• Groups membership can be defined on minimal and irrelevant
attributes (Tajfel 1970). This can nevertheless trigger discrimination.
• In ABSS this minimal group attributes are TAGS (Riolo, Cohen, and
Axelrod 2001).
Implementing tags
• Each agent has two traits, a tag t ∈ [0, 1], and a tolerance threshold
T ≥ 0.
• At set up t and T are uniformly sampled from [0, 1].
• In each generation, each agent acts as a potential donor with P (= 3)
others chosen at random, with replacement.
• A donates only when |tA − tB | ≤ TA .
• If A does donate to B, A pays a cost, c (0.1), and B receives a
benefit, b (1.0).
From Nowak and Sigmund (1998).
INFOMSOCS (Lesson 3)
13-09-12
19 / 33
INFOMSOCS (Lesson 3)
13-09-12
20 / 33
Cooperation among strangers
Tag-based systems
Cooperation among strangers
Selecting tags
Tag-based systems
Evolution of cooperation with tags
After all agents have participated in all pairings in a generation, agents are
reproduced on the basis of their score relative to others:
• This is accomplished by comparing each agent with another randomly
chosen agent, and giving an offspring to the one with the higher score.
• With probability 0.1, the offspring receives a new tag with a value
drawn at random in [0, 1].
• Also with probability 0.1, the tolerance is mutated by adding mean 0,
standard deviation 0.01 gaussian noise to the old tolerance. If the
new T < 0, it is set to 0.
• One run of the model consists of 100 agents and 30,000 generations.
INFOMSOCS (Lesson 3)
13-09-12
21 / 33
INFOMSOCS (Lesson 3)
Communication
Communication
22 / 33
SimNorm
Reputation and norms
Outline
1 Social dilemmas
• SimNorm is the codename of a model (Castelfranchi, Conte, and
Game-theoretic models
Simulating social dilemmas
Paolucci 1998) designed to simulate an artificial population living
under conditions of resource scarcity.
• Designed to answer the questions:
2 Cooperation among strangers
What is the effect of norms on global (i.e. system level) and local (i.e.
individual level) efficiency?
2 What is the role of normative reputation in reducing the costs of
complying with norms?
1
Image-score
Tag-based systems
3 Communication
• Results are compared along an efficiency measure (average strength
SimNorm
of agents after n periods of simulation), and a fairness measure
(agents’ deviation from the average strength).
4 References
INFOMSOCS (Lesson 3)
13-09-12
13-09-12
23 / 33
INFOMSOCS (Lesson 3)
13-09-12
24 / 33
Communication
SimNorm
Communication
SimNorm: the model
SimNorm
SimNorm: the model
• 10 X 10 toroid space.
• Agent density = 50%;
= food.
•
• Initial strength = 40 units;
• Food density = 25%.
Actions:
• Time needed for eating food = 2
• Eat = +20 units;
turns.
• Move = -1 unit;
• When a food item is consumed, it is
restored at a random location.
• Aggression = -4 units (for attacks
given and received); the strongest
agents keeps the food (in case of
ties, the defender wins).
• Ownership is ascribed on the
grounds of spatial proximity
(neighbourhood).
• Food owned is flagged.
INFOMSOCS (Lesson 3)
13-09-12
Communication
25 / 33
INFOMSOCS (Lesson 3)
SimNorm
13-09-12
Communication
SimNorm: the model
26 / 33
SimNorm
SimNorms: Results
The cost of compliance: number of attacks (Agg), the average strength
(Str), and the variance of individual strengths (Var) recorded after 100
matches (2000 time steps each) for each conditions.
• norm = “moral rights” over food
(finder-keeper).
•
= normative agents; they
conform to the finder-keeper law.
•
= utilitarian agents; they attack
agents with lower strength.
Str
stdev
Var
stdev
Agg
stdev
Homogeneous
Util.
Norm.
4727
5585
135
27
1775
604
59
41
4634
3018
248
76
Mixed
Util. Norm.
5897 3634
85
134
1219
651
72
108
3168 2034
122
71
Homogeneous populations: 50 agents sharing the same strategy; Mixed
populations: 50% / 50%
INFOMSOCS (Lesson 3)
13-09-12
27 / 33
INFOMSOCS (Lesson 3)
13-09-12
28 / 33
Communication
SimNorm
Communication
SimNorm: Experimental conditions
SimNorm
SimNorm: effects of communication
Redistributing the cost of compliance:
• Add reputational information: each normative agent can have access
to a vector of information about the behaviour of other agents.
• This information is binary and discriminates between “friends”, that
will abide with the norm (Respectful) and “enemies”, that will not
respect the principle of finders-keepers (Cheaters).
• The reputation vector is initialised to “all Respectful” (presumption of
innocence), but every time a normative agent is attacked while eating
its own food the attacker is recorded as a Cheater.
• The normative algorithm is modified so that agents respect the norm
only with agents known as Respectful.
• One last mechanism, which allows neighbours to exchange their list of
cheaters, is introduced to allow two experimental conditions: with
and without communication (Reputation and Image condition).
INFOMSOCS (Lesson 3)
13-09-12
29 / 33
Without communication (left) social information is not sufficient to protect
respectful agents against aggressions. Reputation (right) balance the
performance of the two subpopulations.
INFOMSOCS (Lesson 3)
References
30 / 33
References
Outline
Bibliography I
Axelrod, R. (1984). The evolution of cooperation. New York: Basic Books.
Bateson, M., D. Nettle, and G. Roberts (Sept. 2006). “Cues of being
watched enhance cooperation in a real-world setting”. In: Biology
Letters 2.3, pp. 412–414. issn: 1744-9561. doi:
10.1098/rsbl.2006.0509. url:
http://dx.doi.org/10.1098/rsbl.2006.0509.
Castelfranchi, C., R. Conte, and M. Paolucci (1998). “Normative
reputation and the costs of compliance”. In: Journal of Artificial
Societies and Social Simulation 1.3. url:
http://jasss.soc.surrey.ac.uk/1/3/3.html.
Cohen, M., R. Riolo, and R. Axelrod (1999). “The Emergence of Social
Organization in the Prisoner’s Dilemma: How Context-Preservation and
Other Factors Promote Cooperation”. In: SFI Working Paper
99-01-002. url:
http://www.santafe.edu/media/workingpapers/99-01-002.pdf.
1 Social dilemmas
Game-theoretic models
Simulating social dilemmas
2 Cooperation among strangers
Image-score
Tag-based systems
3 Communication
SimNorm
4 References
INFOMSOCS (Lesson 3)
13-09-12
13-09-12
31 / 33
INFOMSOCS (Lesson 3)
13-09-12
32 / 33
References
Bibliography II
Fehr, E. and S. Gachter (2002). “Altruistic punishment in humans”. In:
Nature 415.6868, pp. 137–140.
Haley, K. and D. Fessler (2005). “Nobody’s watching?: Subtle cues affect
generosity in an anonymous economic game”. In: Evolution and
Human behavior 26.3, pp. 245–256.
Nowak, M. A. and K. Sigmund (June 1998). “Evolution of indirect
reciprocity by image scoring”. In: Nature 393.6685, pp. 573–577. issn:
0028-0836. doi: 10.1038/31225. url:
http://dx.doi.org/10.1038/31225.
Riolo, R. L., M. D. Cohen, and R. Axelrod (2001). “Evolution of
cooperation without reciprocity”. In: Nature 414, pp. 441–443. doi:
10.1038/35106555. url: http://dx.doi.org/10.1038/35106555.
Tajfel, H. (Nov. 1970). “Experiments in Intergroup Discrimination”. In:
Scientific American 223.5, pp. 96–102. issn: 0036-8733. doi:
10.1038/scientificamerican1170-96. url:
http://dx.doi.org/10.1038/scientificamerican1170-96.
INFOMSOCS (Lesson 3)
13-09-12
33 / 33