A Dynamic Level-k Model in Games
Teck Ho and Xuanming Su
UC Berkeley
April 2011
Teck Hua Ho
1
Dual Pillars of Economic Analysis
 Utility Specification
 Only final allocation matters
 Self-interests
 Exponential discounting
 Solution Method
 Nash and subgame perfect equilibrium (instant
equilibration)
April 2011
Teck H. Ho
2
Challenges: Utility Specification

Reference point matters: People care both about the final
allocation as well as the changes with respect to a target level

Fairness: People care about others’ payoffs. We are nice to
others who have been kind to us. We also get upset when
others treat our peers better than us.

Hyperbolic discounting: People are impatient and prefer
instant gratification
April 2011
Teck H. Ho
3
Challenges: Solution Method





Nash and subgame perfect equilibrium: standard theories in
marketing for predicting behaviors in competitive settings.
Subjects do not play Nash or subgame perfect equilibrium
in experimental games.
Behaviors often converge to equilibrium with repeated
interactions (especially when subjects are motivated by
substantial financial incentives).
Multiplicity problem (e.g., coordination and infinitely
repeated games).
Modeling subject heterogeneity really matters in games.
April 2011
Teck H. Ho
4
Bounded Rationality in Markets:
Revised Utility Function
Behavioral Regularities
Standard Assumption
New Model Specification
Reference Example
Marketing Application Example
1. Revised Utility Function
- Reference point and
loss aversion
- Expected Utility Theory
- Prospect Theory
- Ho and Zhang (2008)
Kahneman and Tversky (1979)
- Fairness
- Self-interested
- Inequality aversion
Fehr and Schmidt (1999)
Ho and Su (2009)
- Cui, Raju, and Zhang (2007)
- Impatience
- Exponential discounting
- Hyperbolic Discounting
Ainslie (1975)
- Della Vigna and Malmendier (2004)
Ho, Lim, and Camerer (JMR, 2006)
April 2011
Teck H. Ho
5
Bounded Rationality in Markets:
Alternative Solution Methods
Behavioral Regularities
Standard Assumption
New Model Specification
Example
Marketing Application Example
2. Bounded Computation Ability
- Nosiy Best Response
- Best Response
- Quantal Best Response
McKelvey and Palfrey (1995)
- Lim and Ho (2008)
- Limited Thinking Steps
- Rational expectation
- Cognitive hierarchy
Camerer, Ho, Chong (2004)
- Goldfrad and Yang (2007)
- Myopic and learn
- Instant equilibration
- Experience weighted attraction
Camerer and Ho (1999)
- Amaldoss and Jain (2005)
April 2011
Teck H. Ho
6
Outline
 Motivation
 Backward induction and its systematic violations
 Dynamic Level-k model and the main theoretical results
 Empirical estimation

Alternative explanations: Reputation-based model and social
preferences
 Conclusions
April 2011
Teck Hua Ho
7
A 4-stage Centipede Game
P
A
T
4
1
April 2011
P
B
T
2
8
P
A
T
16
4
Teck Hua Ho
P
B
64
16
T
8
32
8
A 4-stage Centipede Game
A
B
A
B
4
1
2
8
16
4
8
32
4
3
2
1
Round
1-5
6-10
4
6.2%
8.1%
Backward Induction 100%
April 2011
64
16
0
Outcome
3
2
1
30.3% 35.9% 20.0%
41.2% 38.2% 10.3%
0%
Teck Hua Ho
0%
0%
0
7.6%
2.2%
0%
9
A 6-Stage Centipede Game
A
B
4
1
6
Round
1-5
6-10
A
B
2
8
16
4
8
32
64
16
32
128
5
4
3
2
1
6
0.0%
1.5%
Backward Induction 100%
April 2011
A
256
64
B
0
Outcome
5
4
3
2
1
0
5.5% 17.2% 33.1% 33.1% 9.00% 2.10%
7.4% 22.8% 44.1% 16.9% 6.60% 0.70%
0%
0%
Teck Hua Ho
0%
0%
0%
0%
10
Backward Induction Principle
Nobel Prize, 1994

Backward induction is the most widely accepted principle to generate
prediction in dynamic games of complete information
 Extensive-form games (e.g., Centipede)
 Finitely repeated games (e.g., Repeated PD and chain-store paradox)
 Dynamics in competitive interactions (e.g., repeated price competition)

Multi-person dynamic programming

For the principle to work, every player must be willingness to bet on
others’ rationality
April 2011
Teck Hua Ho
11
Violations of Backward Induction

Well-known violations in economic experiments include:
(http://en.wikipedia.org/wiki/Backward_induction ):
 Passing in the centipede game
 Cooperation in the finitely repeated PD
 Chain-store paradox
 Market settings?

April 2011
Likely to be a failure of mutual consistency condition (different people
make initial different bets on others’ rationality)
Teck Hua Ho
12
Standard Assumptions in
Equilibrium Analysis
Assumptions
Backward
Induction
DLk
Model
Strategic Thinking
X
X
Best Response
X
X
Mutual Consistency
X
?
X
?
Solution Method
Instant Equilibration
April 2011
Teck Hua Ho
13
Notations
S
: Total number of subgames (indexed by s)
I
: Total number of players (indexed by i)
Ns
: Total number of players who are active at subgame s
April 2011
A
B
A
B
4
1
2
8
16
4
8
32
S  4,
I  2,
64
16
N1  N 2  N3  N 4  1
Teck Hua Ho
14
Deviation from Backward Induction
1
 ( L1 ,..., LI , G) 
S
 1


s 1  N s
S

Ds ( L , L )

i 1

Ns
i
i

1
,
a
(
L
)  a( L )
Ds(Li ,L )  

0, otherwise

0   (.)  1
April 2011
Teck Hua Ho
15
Examples
A
B
A
B
4
1
2
8
16
4
8
32
64
16
A
B
Ex1: L  {P,, T ,}; L  {, P,, T }, L  {T , T , T , T }
 ( LA , LB , G4 )  14 [1  1  0  0] 
1
2
Ex2: LA  {P,, T ,}; LB  {, T ,, T }, L  {T , T , T , T }
 ( L , L , G4 ) 
A
April 2011
B
1
4
1
[1  0  0  0] 
4
Teck Hua Ho
16
Systematic Violation 1: Limited Induction
64
16
A
B
A
B
4
1
2
8
16
4
8
32
A
B
A
B
A
B
4
1
2
8
16
4
8
32
64
16
32
128
256
64
 ( LA , LB , G4 )   ( LA , LB , G6 )
April 2011
Teck Hua Ho
17
Limited Induction in Centipede Game
Figure 1: Deviation in 4-stage versus 6-stage game
April 2011
Teck Hua Ho
18
Systematic Violation 2: Time Unraveling
A
B
A
B
4
1
2
8
16
4
8
32
64
16
 ( LA (t ), LB (t ), G)  0 as t  
April 2011
Teck Hua Ho
19
Time Unraveling in Centipede Game
Figure 2: Deviation in 1st vs. 10th round of the 4-stage game
April 2011
Teck Hua Ho
20
Outline
 Motivation
 Backward induction and its systematic violations
 Dynamic Level-k model and the main theoretical results
 Empirical estimation

Alternative explanations: Reputation-based model and social
preferences
 Conclusions
April 2011
Teck Hua Ho
21
Research question
To develop a good descriptive model to predict the probability of
player i (i=1,…,I) choosing strategy j at subgame s (s=1,.., S) in
any dynamic game of complete information
Pij (s )
April 2011
Teck Hua Ho
22
Criteria of a “Good” Model
 Nests backward induction as a special case
 Behavioral plausible

Heterogeneous in their bets on others’ rationality

Captures limited induction and time unraveling
 Fits data well
 Simple (with as few parameters as the data would allow)
April 2011
Teck Hua Ho
23
Standard Assumptions in
Equilibrium Analysis
Assumptions
Backward
Induction
Hierarchical
Strategizing
Strategic Thinking
X
X
Best Response
X
X
Mutual Consistency
X
Heterogenous
Bets
Learning
Solution Method
Instant Equilibration
April 2011
X
Teck Hua Ho
24
Dynamic Level-k Model: Summary
 Players choose rule from a rule hierarchy
 Players make differential initial bets on others’ chosen rules
 After each game play, players observe others’ rules
 Players update their beliefs on rules chosen by others
 Players always choose a rule to maximize their subjective
expected utility in each round
April 2011
Teck Hua Ho
25
Dynamic Level-k Model: Rule Hierarchy
 Players choose rule from a rule hierarchy generated by bestresponses
 Rule hierarchy: L0 , L1 , L2 ,....
 Lk  BR ( Lk 1 )
 Restrict L0 to follow behavior proposed in the existing
literature (i.e., pass in every stage)
 L  BI
April 2011
Teck Hua Ho
26
Dynamic Level-k Model: Poisson Initial Belief
 Different people make different initial bets on others’ chosen rules
 Poisson distributed initial beliefs:
f (K )  e


K
K!
 : average belief of rules used by opponents
 f(k) fraction of players think that their opponents use Lk rule.
April 2011
Teck Hua Ho
27
Dynamic Level-k model:
Belief Updating at the End of Round t
 Level k’s initial belief strength b entirely on k-1
 Update after observing which rule opponent chose
N ki (t )  Ν ki  t  1  I(k,t)1
B (t ) 
i
k
N ki (t )
S
N
k ' 0
i
k
(t )
 I(k, t) = 1 if opponent chose Lk and 0 otherwise
 Bayesian updating involving a multi-nomial distribution with a
Dirichlet prior (Fudenberg and Levine, 1998; Camerer and Ho,
1999)
April 2011
Teck Hua Ho
28
Dynamic Level-k model: :
Optimal Rule in Round t+1
 Optimal rule k*:
S i

k  arg max k 1,.., S   Bk ' (t )   (aks , ak 's )
s 1  k '1

S
*
 Let the specified action of rule Lk at subgame s be aks
April 2011
Teck Hua Ho
29
The Centipede Game (Rule Hierarchy)
Player A
Player B
0
(P, -, P, -)
(-, P, -, P)
1
(P, -, P, -)
(-, P, -, T)
2
(P, -, T, -)
(-, P, -, T)
3
(P, -, T, -)
(-, T, -, T)
4
(T, -, T, -)
(-, T, -, T)
April 2011
Teck Hua Ho
30
A 4-stage Centipede Game
A
B
A
B
4
1
2
8
16
4
8
32
4
3
2
1
Round
1-5
6-10
4
6.2%
8.1%
Backward Induction 100%
April 2011
64
16
0
Outcome
3
2
1
30.3% 35.9% 20.0%
41.2% 38.2% 10.3%
0%
Teck Hua Ho
0%
0%
0
7.6%
2.2%
0%
31
Player A in 4-Stage Centipede Game
b0.5
i
N k(t)
Round (t) L 0
L1
L2
L3
L4
Rule Used by Opponent Optimal Rule (Player A)
0
b
1
b
1
L3
L2
2
b
2
L3
L2
3
b
3
L3
L4
April 2011
L2
Teck Hua Ho
32
Dynamic Level-k Model: Summary
 Players choose rule from a rule hierarchy
 Players make differential initial bets on others’ chosen rules
 After each game play, players observe others’ rules
 Players update their beliefs on rules chosen by others
 Players always choose a rule to maximize their subjective
expected utility in each round
 A 2-paramter extension of backward induction ( and b)
April 2011
Teck Hua Ho
33
Main Theoretical Results: Limited Induction
Theorem 1: The dynamic level-k model implies that the
limited induction property is satisfied. Specifically, we
have:
 ( LA , LB , Gs )   ( LA , LB , GS ); s1  s2
1
April 2011
2
Teck Hua Ho
34
Main Theoretical Results: Time Unraveling
Theorem 2: The dynamic level-k model implies that the time
unraveling property is satisfied. Specifically, we have:
 ( LA (t ), LB (t ), G)  0 as t  
April 2011
Teck Hua Ho
35
Outline
 Motivation
 Backward induction and its systematic violations
 Dynamic Level-k model and the main theoretical results
 Empirical estimation

Alternative explanations: Reputation-based model and social
preferences
 Conclusions
April 2011
Teck Hua Ho
36
4-Stage versus 6-Stage Centipede Games
April 2011
64
16
A
B
A
B
4
1
2
8
16
4
8
32
A
B
A
B
A
B
4
1
2
8
16
4
8
32
64
16
32
128
Teck Hua Ho
256
64
37
Caltech versus PCC Subjects
April 2011
Teck Hua Ho
38
Caltech Subjects
April 2011
Teck Hua Ho
39
Caltech Subjects: 6-Stage Centipede Game
April 2011
Teck Hua Ho
40
Model Predictions; Caltech Subjects
April 2011
Teck Hua Ho
41
Model Predictions: PCC subjects
April 2011
Teck Hua Ho
42
Alternative 1:
Reputation-based Model (Kreps, et al, 1982)
large
q = proportion of altruistic players (level 0 players)
April 2011
Teck Hua Ho
43
Alternative 1: Reputation-based Model
Subject Pool q
e
LL
Caltech
0.050 0.620 -329.8
PCC
0.075 0.310 -518.8
April 2011
Teck Hua Ho
LL(DLk)
-305.8
-514.8
44
Alternative 2: Social Preferences
April 2011
Teck Hua Ho
45
Alternative 2: Empirical Estimation
Subject Pool
Caltech
PCC
April 2011
e
1.000
1.000
LL
-357.1
-646.5
Teck Hua Ho
LL(DLk)
-305.8
-514.8
46
Conclusions
 Dynamic level-k model is an empirical alternative to BI
 Captures limited induction and time unraveling
 Explains violations of BI in centipede game
 Dynamic level-k model can be considered a tracing procedure
for BI (since the former converges to the latter as time goes to
infinity)
April 2011
Teck Hua Ho
47
p-Beauty Contests
 n=7 players (randomly chosen)
 Every player simultaneously chooses a number from 0 to 100
 Compute the group average
 Define Target Number to be p=0.7 times the group average
 The winner is the player whose number is the closet to the
Target Number
 The prize to the winner is US$20 (Ho & H0)
April 2011
Empirical Regularity 1:
Groups with Smaller p Converge Faster
April 2011
Teck Hua Ho
49
Empirical Regularity 2:
Larger Groups Converge Faster
April 2011
Teck Hua Ho
50
Dynamic Level-k Model Predictions
April 2011
Teck Hua Ho
51
April 2011
Teck H. Ho
52
Modeling Philosophy
Simple
General
Precise
Empirically disciplined
(Economics)
(Economics)
(Economics)
(Psychology)
“the empirical background of economic science is definitely inadequate...it
would have been absurd in physics to expect Kepler and Newton without Tycho
Brahe” (von Neumann & Morgenstern ‘44)
“Without having a broad set of facts on which to theorize, there is a certain
danger of spending too much time on models that are mathematically elegant,
yet have little connection to actual behavior. At present our empirical
knowledge is inadequate...” (Eric Van Damme ‘95)
April 2011
Teck H. Ho
53