Automata-based adaptive
behavior for economic
modeling using game theory
Rawan Ghnemat, Khalaf Khatatneh, Saleh Oqeili
Al-Balqa’ Applied University, Al-Salt, Jordan
Cyrille Bertelle,
LIH, University of Le Havre, France
Gérard H.E. Duchamp
LIPN, University of Paris XIII
Outline
1. From an application in game theory to
fundamental concepts for adaptive
strategies
2. Multi-scale complex system modeling
3. Genetic automata and emergent
systems computation
4. Conclusion
17/11/2005
EPNADS'05
2
1. From an application in game
theory to fundamental concepts
for adaptive strategies
 Example of modelization for game theory
using an adaptive strategy
 How can one strategy be modelized ?
 Multi-strategies in one model
 A model for an adaptive strategy
17/11/2005
EPNADS'05
3
Prisoner dilemma
 Two players game involving a
cooperation/competition model
 Basic model for economic purpose
 Rules:
 2 options for each player at each play:
cooperate (C) or betray (D)
 Associated payoff for each situation in
following table
17/11/2005
EPNADS'05
4
Prisoner dilemma payoff
table
Player 2
C
D
(cooperate)
(betray)
C
Player 1
(cooperate)
(+3, +3)
(0, +5)
(+5, 0)
(+1, +1)
D
(betray)
17/11/2005
EPNADS'05
5
Prisoner dilemma
application
 Great number of applications, especially
in military or economic domains
 Example - Competition between two
companies:
 Aggressive competition behavior
 Cooperative behavior
 … see payoff table in the following slide
17/11/2005
EPNADS'05
6
Prisoner dilemma
application (2)
Company C1
Cooperative
politic
Company
C2
Cooperative
politic
Aggressive
politic
17/11/2005
Aggressive
politic
Medium
profit for
each
Poor profit for
C2 and huge
profit for C1
Poor profit for
C1 and huge
profit for C2
Poor profit
EPNADS'05
7
Iterative version for
prisoner dilemma
 Successive steps
 Each player do not know adversary’s
action …
 … but he knows the previous action of
his adversary
 So different strategies can be defined for
player behavior (goal: having maximal
payoff for himself)
17/11/2005
EPNADS'05
8
Some strategies
 Vindictive strategy:
 If the adversary cooperates at previous play,
I cooperate
 If the adversary betrays one time, I will
always defect, whatever the adversary does
later!
 Tit-for-Tat strategy:
 I always do what my adversary has done at
the previous play
17/11/2005
EPNADS'05
9
Model for “Tit for tat”
strategy
 One state to define my
behavior
 Transition for each
play:
 Input: what the
adversary does at the
previous play
 Output: what I do in
consequence
→ Automata with outputs
17/11/2005
EPNADS'05
10
Model adaptation for
“vindictive” strategy
17/11/2005
EPNADS'05
11
Multi-strategies automaton
with evolution
 Probabilistic
automaton for multistrategies in one
model
 Evolution on
probabilistic values
to adapt the strategy
(explained in section
3)
17/11/2005
EPNADS'05
12
2. Multi-scale complex
system modeling
 Previously, we presented how
cooperative/competitive behaviors can be
modelized with automata with outputs
 Now, we will explain how such models
can be used for complex systems
modeling
17/11/2005
EPNADS'05
13
Multi-scale complex
system modeling - a
17/11/2005
EPNADS'05
14
Multi-scale complex
system modeling - b
17/11/2005
EPNADS'05
15
Multi-scale complex
system modeling - c
17/11/2005
EPNADS'05
16
Multi-scale complex
system modeling - d
17/11/2005
EPNADS'05
17
Multi-scale complex
system modeling - e
 Problem :
How to compute the emergent
systems formation?
And the retro-action of the system
on their contitutive components?
17/11/2005
EPNADS'05
18
Multi-scale complex
system modeling - a
17/11/2005
EPNADS'05
19
3. Genetic automata for game
theory and emergent systems
computation
The solution is to use genetic
algorithms on automata:
•For evolutive strategies (Prisoner
Dilemma)
•For emergent systems computation
17/11/2005
EPNADS'05
20
3. Genetic automata for game
theory and emergent systems
computation
Return to the probabilistic automaton
Linear representation:
  P1 ;1  P1    P6 ;1  P6 
t
 p2 1  p2 
 (C )  

p
1

p
3
 3
1  p 4 p 4 
C 

1

p
p
5
5


17/11/2005
EPNADS'05
21
Genetic algorithm on
automata
 Chromosomes: sequence of all the
matrices associated to each letter
 One allele (primitive data) of
chromosomes corresponds to one line of
the matrix representation
→ 3 classical operators: Duplication, crossingover and mutation
17/11/2005
EPNADS'05
22
General GA process for genetic
automata for game theory
 A population of automata is initially generated
 Each automaton makes a set of plays against
a predefined strategy, named SO
 At each play, we execute the probabilistic
automaton which gives output: C or D
 With this output and SO’s output, we compute
the payoff of the automaton with the payoff
table
17/11/2005
EPNADS'05
23
General GA process for genetic
automata for game theory(2)
 At the end of the set of play, the automaton
payoff is the sum of all payoff of each play →
fitness of the automaton
 At the end of the set of play, each automaton has
a fitness → selection process selects the best
automaton → new automata population
 New computation of the 3 genetic operators over
this new population.
17/11/2005
EPNADS'05
24
Genetic automata for
emergent self-organizations
 Computation of a new fitness allowing
emergent systems of similar behaviour
 Fitness computation based on
 e(x): Evaluation on agent behaviour
automaton x as the matrix M of outputs M(i,j)
from all possible successive perceptions
from an initial state i to a final state j
 d(x,y) = || e(x)-e(y) || : semi-distance
between two agent behaviors x and y
17/11/2005
EPNADS'05
25
Genetic automata for
emergent self-organizations
 Fitness computation of an agent x where
Vx corresponds to a neighbourhood of x:
 card  x 
 d ( x, y ) 2
i
f ( x)   
yi  x

,

17/11/2005
if
EPNADS'05
2
d
(
x
,
y
)
0

i
yi  x
otherwise
26
Genetic automata for
emergent self-organizations
 So selected agents of the population will
become more near to each others, in
respect of the semi-distance defined
 So GA is a way to modelize the feedback of emergent systems which leads to
gather agents of similar behaviour, in a
dynamical way.
17/11/2005
EPNADS'05
27
4. Conclusion
 Genetic automata are appropriate
models for adaptive behaviour
 Adaptive behavior are the basis for game
theory implementation
 Adaptive behavior can be the expression
of feed-back of emergent systems over
their constitutive agents
17/11/2005
EPNADS'05
28