Dynamic Stochastic Choice under Evolving Private
Information∗
Preliminary and Incomplete—Comments welcome
Mira Frick
Ryota Iijima
Tomasz Strzalecki
Abstract
We study dynamic stochastic choice behavior resulting from an agent (or population
of agents) solving a dynamic decision problem subject to evolving private information. A
key feature of our model is that observed choices are typically history dependent: different
sequences of past choices reflect different private information of the agent, which may lead
to different distributions of her current choices. Our main axioms impose discipline on the
amount of history dependence that choices can display: We identify simple equivalence
classes of past choices that reflect the same private information and thus have the same
implications for current choices. Additional axioms allow us to distinguish between two
sources of history dependence: randomly evolving utility vs. gradual learning about fixed
but unknown tastes. An agent’s observed choices allow for essentially unique identification
of her underlying utility process/private information. We characterize the relationship
between persistence in choices and in tastes. Finally, we relate our model to dynamic
discrete choice models widely used in empirical work, highlighting new modeling tradeoffs
in this literature.
∗
This version: 5 March 2017. Frick: Yale University ([email protected]); Iijima: Yale University (ry-
[email protected]); Strzalecki: Harvard University (tomasz [email protected]). This research was supported by the National Science Foundation grant SES-1255062. For helpful comments, we thank David Ahn,
Dirk Bergemann, Drew Fudenberg, Larry Samuelson, as well as seminar audiences at ASU Economic Theory
Conference, Berkeley, Bocconi, Caltech Choice Conference, Harvard–MIT, LMU, Rochester, Stanford, and Yale.
1
1
Introduction
Random utility models are widely used throughout economics. In the static model, the agent
chooses from her choice set after observing the realization of a random utility function U . In the
dynamic model, the agent solves a dynamic decision problem, subject to a stochastic process
(Ut ) of utilities. The key feature of the model is an informational asymmetry between the
agent (who knows her realized utility) and the analyst (who does not). In both the static and
dynamic setting, this asymmetry gives rise to choice behavior that appears stochastic to the
analyst but is deterministic from the point of view of the agent.1
In the dynamic setting, the informational asymmetry has an additional key implication:
If (Ut ) displays serial correlation, then choices will appear history dependent to the analyst.
For example, we expect the agent’s probability of voting Republican in 2020 to be different
conditional on voting Republican in 2016 than conditional on voting Democrat in 2016. This
is because her past voting behavior reveals relevant information about her past political preferences, which we expect to be at least somewhat persistent. This form of history dependence is
distinct from habit formation, memory utility, recency bias, and other behavioral effects, where
past consumption directly affects the agent’s future preferences. Here, it is past choices that
change predicted future choices, and this effect occurs for completely standard preferences as a
consequence of the informational asymmetry between the analyst and the agent.2
History dependence due to serially correlated private information is an important issue in
the dynamic discrete choice literature, which has recognized that “ignoring serial correlation in
unobservables [...] can lead to serious misspecification errors” (Norets, 2009). This literature
has developed and estimated a number of models that generate history dependent choices, but
rely on specific parametric forms of serial correlation. Pakes (1986) highlights the limitations of
the parametric approach: “In all these models both the estimation technique and the empirical
results depend on the details of the stochastic specification and, because of the complexity of
1
An equivalent interpretation of the model is that the analyst observes a fixed population of heterogeneous
individuals. Throughout the paper, we use “the agent” to refer to both interpretations.
2
There are two ways to distinguish past choices from past consumption. The first one is to vary the choice
sets from which the agent is choosing. The second one is to look at choices between lotteries and use the fact
that each lottery can result in multiple consumption outcomes. This paper uses both approaches.
2
the estimation problem, it is difficult to determine the robustness of the conclusions to the
stochastic assumptions chosen.”
Our paper provides the first analysis of the fully general model of dynamic random utility.
Our contribution is threefold: First, we axiomatically characterize the model. Our main new
axioms pin down the relationship between history dependent choices and serially correlated
private information in a non-parametric setting; additional axioms distinguish between two
central sources of serial correlation: randomly evolving tastes vs. gradual learning about fixed
but unknown tastes. Our axiomatization answers in the dynamic setting a question that has
given rise to an extensive literature in the static setting.3 Second, we provide identification
results, showing that the agent’s choice behavior uniquely reveals her underlying private information. Finally, our analysis sheds new light on modeling tradeoffs in the dynamic discrete
choice literature.
Our model generalizes the static random expected utility framework of Gul and Pesendorfer
(2006) to decision trees as defined by Kreps and Porteus (1978). Each period t, the agent
chooses from a menu of lotteries over current outcomes and continuation menus by maximizing
a random vNM utility Ut . A history ht−1 = (A0 , p0 , . . . , At−1 , pt−1 ) summarizes that the agent
chose lottery p0 from menu A0 , then was faced with A1 and chose p1 , and so on. Observed
behavior at t is given by a history dependent choice distribution ρt (·|ht−1 ), specifying the choice
frequency ρt (pt ; At |ht−1 ) of pt from any menu At that can arise after ht−1 .
Turning to the axiomatic characterization, our first main insight is the following: The
fact that history dependence arises purely as a result of serial correlation in (Ut ) entails two
history independence conditions. Each condition identifies simple equivalence classes of histories
that reveal the same private information to the analyst; if ht−1 and ĥt−1 are equivalent, then
ρt (·|ht−1 ) and ρt (·|ĥt−1 ) must coincide. Theorem 1 shows that our most general model, dynamic
random expected utility (DREU), is fully characterized by these two independence conditions
along with a continuity condition and Gul and Pesendorfer’s (2006) axioms that ensure static
random utility maximization at each history.
In DREU, the stochastic process (Ut ) is unrestricted. We next study the special case in
3
See the first paragraph under Related Literature.
3
which (Ut ) is given by a Bellman equation: The agent has separable preferences over current
consumption and continuation problems and is forward-looking with a correct assessment of
option value. This allows us to distinguish evolving utility, where the agent faces taste shocks
that evolve randomly over time, from its special case, gradual learning, in which the agent has
fixed but unknown tastes about which she learns over time through signals about the state
of the world. Evolving utility is characterized by adding separability, flexibility, and dynamic
sophistication axioms to DREU (Theorem 2). To obtain these axioms, we identify a history
dependent revealed preference relation from stochastic choice data; this allows us to translate
known conditions from the menu preference literature to our setting. The additional behavioral
content of gradual learning is encapsulated by a consumption stationarity axiom, reflecting the
martingale property of beliefs, along with an intertemporal calibration axiom (Theorem 3).
Proposition 1 establishes identification results for the three representations. In DREU,
the agent’s ordinal private information is uniquely pinned down; evolving utility, and more
so gradual learning, impose discipline on the cardinal private information (Ut ); additionally,
gradual learning allows for unique identification of the discount factor.
A key challenge throughout our analysis is the following “limited observability” problem:
In contrast with the static setting, where the analyst observes choices from all possible menus,
in the dynamic setting each history of past choices restricts the set of current and future choice
problems. Over time, this severely limits the history dependent choice data ρt (·|ht−1 ) on which
axioms can be imposed and from which (Ut ) can be inferred. We overcome this problem by
means of the following extrapolation procedure (Definition 4): For any menu At and history
ht−1 that does not lead to At , we define the agent’s counterfactual choice distribution from At
following ht−1 by extrapolating from the situation where the agent plans to make the sequence of
choices captured by ht−1 , but is sometimes forced, with exogenous probability, to instead make
a sequence of choices that does lead to menu At . By specifying this sequence appropriately, we
can ensure that it reveals the same private information as the original choice sequence ht−1 , thus
justifying the extrapolation. This extrapolation procedure relies crucially on the inclusion of
lotteries as choice objects. We discuss the connection with similar uses of plausibly exogenous
randomization to perform counterfactual analyses in empirical and experimental work.
4
Finally, Section 7 discusses the relationship with the dynamic discrete choice literature, in
particular Rust (1994), which has become the workhorse model for structural estimation.
We find that DREU nests the Rust model, which in turn nests a version of evolving utility
in which the agent’s private information is fully summarized by her past realizations of consumption utility. However, the widely used i.i.d. specification of the Rust model is incompatible
with evolving utility, as the two make opposite predictions about the option value. In particular, in the evolving utility model the agent likes bigger menus, as they provide her with more
flexibility; she also likes to make her decisions as late as possible because more information
will arrive with time and having more information is always better. On the other hand, the
i.i.d. Rust agent sometimes prefers to commit to smaller menus and to make her decisions as
early as possible, thus displaying a negative value of information. This is a consequence of
the i.i.d. specification of utility shocks, which fails to capture Bayesian learning and is instead
motivated by the need for a statistically non-degenerate likelihood function. Our findings point
to a possible modeling tradeoff between having well-behaved likelihoods and positive value of
information.
In Section 9 (in preparation), we extend our model to allow preferences to be influenced
directly by past consumption. As mentioned above, such dependence occurs under habit formation, but it is also present in models with controlled subjective state variables, such as models
of active learning and experimentation, because in those models current consumption provides
valuable information about the state, and thus changes the future expected utility.
1.1
Related Literature
An extensive literature studies axiomatic characterizations of random utility models in the
static setting (Barberá and Pattanaik, 1986; Block and Marschak, 1960; Falmagne, 1978; Luce,
1959; McFadden and Richter, 1990).4 Our approach incorporates as its static building block the
elegant axiomatization of Gul and Pesendorfer (2006) and Ahn and Sarver (2013). A technical
4
Lu (2016) studies a related model with an objective state space and choice between acts which traces all
randomness of choice to random arrival of signals. This is similar in spirit to our gradual learning representation; however, our states are subjective and utility can be state-dependent. Another recent contribution by
Apesteguia, Ballester, and Lu (2017) considers a setting in which choice options are linearly ordered.
5
contribution of our paper is the extension of their result to an infinite outcome space, which
is needed since the space of continuation problems in the dynamic model is infinite. A recent
paper by Lu and Saito (2016) studies period 0 random choice between consumption lottery
streams and attributes the source of randomness of choice to the stochastic discount factor.
In the dynamic setting, the literature on axiomatic characterizations of random utility models (and stochastic choice more generally) is very sparse. Our choice domain is exactly the one
of Kreps and Porteus (1978); however, while they study deterministic choice in each period, we
focus on random choice in each period. To the best of our knowledge, Fudenberg and Strzalecki
(2015) is the only other axiomatic study of stochastic choice in a fully dynamic domain, but
they focus on logit-type utility shocks that are i.i.d over time.5 Thus, their representation is
not suited to studying the relationship between serially correlated utilities and history dependent stochastic choice, which is the main focus of our paper. Ahn and Sarver (2013) study a
model where the analyst observes the deterministic period 0 choices between menus and period
1 random choices from menus. They show how to connect the analysis of Gul and Pesendorfer
(2006) and of Dekel, Lipman, and Rustichini (2001) to obtain better identification properties.
Their sophistication axiom plays a key role in our characterization of evolving utility. Our work
generalizes theirs by allowing for information to arrive in period 0, which makes it necessary
to study random choices also in period 0. As discussed, this allows us to study the behavioral
implications of serially correlated private information and to make a connection to the dynamic
discrete choice literature.
The literature on static and deterministic preferences over menus (Dekel, Lipman, and
Rustichini, 2001; Dekel, Lipman, Rustichini, and Sarver, 2007; Dillenberger, Lleras, Sadowski,
and Takeoka, 2014; Kreps, 1979) assumes that in period 0 the agent does not receive any
information, but anticipates receiving information in period 1. This leads to a preference for
bigger menus over smaller ones to capitalize on option value. The papers closest to ours are
Krishna and Sadowski (2014) and Krishna and Sadowski (2016) who study preferences over
5
Gul, Natenzon, and Pesendorfer (2014) and Ke (2016) study settings in which the agent receives an outcome
only once at the end of a decision tree, and characterize stochastic choice models of bounded rationality. There
is also non-axiomatic work studying various special cases of our representation on limited domains, where the
agent chooses from the same set in each period but has the option of delaying the choice and gaining more
information: Fudenberg, Strack, and Strzalecki (2016) and Natenzon (2016).
6
infinite horizon dynamic choice problems that are stationary versions of our evolving utility
representation (making them unsuited to study gradual learning). The main difference is that
they assume the existence of a special period 0 that involves no randomness, thanks to which
they can focus on period 0 deterministic preferences. By contrast, we allow information to
arrive in each period (and therefore focus on random choice) and study the predictions of the
model for behavior in each period (instead of just in period 0). This allows us to make more
precise comparisons with models of dynamic discrete choice, which by design look at behavior
in each period and are meant to be estimated on data sets that involve multiperiod decisions.
Within the dynamic discrete choice literature, seminal models include Miller (1984), Wolpin
(1984), Pakes (1986), and Rust (1987); see Rust (1994), Aguirregabiria and Mira (2010) and
Arcidiacono and Ellickson (2011) for surveys. Our results provide an axiomatic foundation
for these models; we also relate our uniqueness results to the identification results in that
literature. A prominent special case of the general model is the so-called dynamic logit model
used by Rust (1989), Hendel and Nevo (2006), Kennan and Walker (2011), Sweeting (2011), and
Gowrisankaran and Rysman (2012). As mentioned, Fudenberg and Strzalecki (2015) provide
an axiomatization of this model; they are also the first paper to notice that this model induces
negative value of information; our results in Section 7 generalize theirs.
2
Domain and Primitive
We analyze choice behavior in decision trees, as defined by Kreps and Porteus (1978). For any
set Y we denote by ∆(Y ) the set of all simple (i.e., finite support) lotteries on Y and by K(Y )
the set of all nonempty finite subsets of Y . There are finitely many periods t = 0, 1, . . . , T . In
each period the agent receives a payoff from a finite set Z. A period-T menu is a finite set of
lotteries over XT := Z. The set of all period-T menus is denoted by AT := K(∆(XT )). For all
t ≤ T − 1, a period-t menu is a finite set of lotteries over Xt := Z × At+1 ; the set of all period-t
menus is denoted by At := K(∆(Xt )). We denote a typical lottery by pt ∈ ∆(Xt ) and a typical
menu by At ∈ At . For each t ≤ T − 1 and pt ∈ ∆(Xt ) let pZt ∈ ∆(Z) and pA
t ∈ ∆(At+1 ) denote
the respective marginal distributions.
7
Including lotteries in the domain of choice simplifies our analysis, as it allows us to rely
on the static framework of Gul and Pesendorfer (2006) instead of the more complicated one of
Falmagne (1978). Lotteries play a similar role in the original work of Kreps and Porteus (1978),
by letting them rely on the vNM framework, as well as in the ambiguity aversion literature
that extensively relies on the Anscombe and Aumann (1963) framework rather than the more
complicated one of Savage (1972).6 In Section 4.1.2, lotteries play a key role in overcoming the
limited observability problem mentioned in the Introduction, while in Section 9 lotteries help us
disentangle the two sources of history dependence mentioned in the Introduction: dependence
on past choices vs. dependence on past consumption. While some applications may be limited
to choices between deterministic outcomes, in Section 4.1.2 we provide examples of experimental
and empirical work that explicitly incorporates choice data over lotteries.
The dynamic stochastic choice rule ρ is defined recursively. The observed choice distribution
P
at period 0 is summarized by a map ρ0 : A0 → ∆(∆(X0 )) such that p0 ∈A0 ρ0 (p0 ; A0 ) = 1 for
all A0 ; ρ0 (p0 ; A0 ) denotes the frequency with which the agent chooses lottery p0 when faced
with menu A0 . Let H0 := {(A0 , p0 ) : ρ0 (p0 , A0 ) > 0} be the set of all period 0 histories,
i.e., choices that occur with positive probability, and for any history h0 = (A0 , p0 ) ∈ H0 , let
0
A1 (h0 ) := supp pA
0 denote the set of period 1 menus that follow h with positive probability.
For each t = 1, . . . , T and history ht−1 ∈ Ht−1 , observed period t choices following ht−1 are
P
summarized by a map ρt (·|ht−1 ) : At (ht−1 ) → ∆(∆(Xt )) such that pt ∈At ρt (pt ; At | ht−1 ) = 1
for all At ∈ At (ht−1 ); ρt (pt ; At | ht−1 ) denotes the frequency with which the agent chooses pt
when faced with menu At after history ht−1 . The set of period-t histories is denoted by
Ht := {(ht−1 , At , pt ) : ht−1 ∈ Ht−1 and At ∈ At (ht−1 ) and ρt (pt ; At |ht−1 ) > 0}.
For each t ≤ T − 1, the set of period t + 1 menus that follow history ht = (ht−1 , At , pt ) with
positive probability is At+1 (ht ) := supp pA
t and the set of period-t histories that lead to At+1
with positive probability is Ht (At+1 ) := {ht ∈ Ht : At+1 ∈ At+1 (ht )}.
As is usual in work on dynamics, the observability requirements are larger than in the
6
The notable exceptions include Gilboa (1987) and Epstein (1999).
8
static setting because the analyst needs to observe choices in each period and after any history.
Moreover, as customary in the random choice literature, the analyst observes the limiting
choice frequencies from each menu, which implies that the data set is infinite. Assuming the
statistical inference problem away in this manner is also typical in the econometric analysis of
identification.
Of course in any application, the data set will be finite. However, considering the implications of a model on the full domain is crucial for uncovering all the assumptions that are
behind the model. In this spirit, our axioms are not tools for testing theories on data, but
rather conceptual tools that are helpful in better understanding the properties of the models
and in making comparisons between different models. For instance, the evolving utility model
and the i.i.d. Rust model have representations that look very similar. While the analysis of
these representations may reveal some differences, it may not be apparent that for example
in one model the value of information is positive, while in the other negative; it is only the
axiomatic characterization that uncovers all possible differences. Understanding the properties
of models on an idealized data set can be helpful in guiding choices of analysts who consider
estimating multiple models on a given finite data set, as it may help them expect which model
will lead to estimation biases and in which direction.
3
Representations
For any set X, RX denotes the set of vNM utility indices over X. Endow RX with the product
topology and with the induced Borel sigma-algebra. Given any U ∈ RX , the expected utility
P
of each simple lottery p ∈ ∆(X) is U (p) := x∈supp(p) U (x)p(x), which we sometimes denote
by U · p. For any U, U 0 ∈ RX , we write U ≈ U 0 if U and U 0 represent the same preference on
∆(X). For any menu A ∈ K(∆(X)), let M (A, U ) := argmaxp∈A U (p) denote the set of lotteries
in A that maximize U .
9
3.1
Static Random Expected Utility
We first briefly review the static model of random expected utility that will serve as the building
block of our dynamic representation at each history. This model is based on Gul and Pesendorfer
(2006) and Ahn and Sarver (2013),7 but allows for an infinite outcome space; this extension is
necessary for our purposes, because in the dynamic setting the period-t outcome space Xt =
Z × At is infinite for each t ≤ T − 1. Let X be an arbitrary metric space of outcomes and
A := K(∆(X)) the corresponding set of menus. The primitive is a stochastic choice rule
P
ρ : A → ∆(∆(X)) such that p∈A ρ(p, A) = 1 for all A ∈ A.
Let (Ω, F ∗ , µ) be a finitely additive probability space. In each state of the world, the
agent chooses a lottery p from menu A to maximize her expected utility subject to her private
information. Her payoff-relevant private information is captured by a sigma-algebra F ⊆ F ∗
and an F-measurable random vNM utility index U : Ω → RX . In case of indifference, ties are
broken by a random vNM index W : Ω → RX , which is measurable with respect to F ∗ .
For tractability, we follow Ahn and Sarver (2013) in assuming that the agent’s payoffrelevant private informaton (F, U ) is simple, i.e., (i) F is generated by a finite partition such
that µ(F(ω)) > 0 for every ω ∈ Ω, where F(ω) denotes the cell of the partition that contains
ω; and (ii) each U (ω) is nonconstant and U (ω) 6≈ U (ω 0 ) whenever F(ω) 6= F(ω 0 ). Moreover,
the tie-breaker W is regular, ensuring that in each menu ties occur with probability 0; that is,
µ({ω ∈ Ω : |M (A, W (ω))| = 1}) = 1 for all A ∈ A.
The analyst does not observe the agent’s private information and thus cannot condition
on events in F. From his perspective, the event that the agent chooses p from A is given by
C(p, A) := {ω ∈ Ω : p ∈ M (M (A, U (ω)), W (ω))}.
Definition 1. A random expected utility (REU) representation of ρ is a tuple (Ω, F ∗ , µ, F, U, W )
such that (Ω, F ∗ , µ) is a finitely additive probability space, the sigma-algebra F ⊆ F ∗ and the
F-measurable utility U : Ω → RX are simple, the F ∗ -measurable tiebreaker W : Ω → RX is
regular, and ρ(p, A) = µ(C(p, A)) for all p and A.
7
The original formulation is due to Gul and Pesendorfer (2006). For tractability, we follow Ahn and Sarver
(2013) in assuming that the agent’s payoff relevant private information is finitely generated.
10
3.2
Dynamic Random Expected Utility
Let (Ω, F ∗ , µ) be a finitely additive probability space. Each period t, the agent chooses a lottery
pt from menu At to maximize her expected utility subject to her dynamically evolving private
information. Her payoff-relevant private information is captured by a filtration (Ft )0≤t≤T ⊆ F ∗
and an Ft -adapted process of random vNM utility indices Ut : Ω → RXt over Xt . In case of
indifference, ties at each t are broken by a random F ∗ -measurable vNM utility index Wt : Ω →
Xt .
We impose the following dynamic analogs of simplicity and regularity. The pair (Ft , Ut )0≤t≤T
is simple, i.e., (i) each Ft is generated by a finite partition such that µ(Ft (ω)) > 0 for every
ω ∈ Ω; and (ii) each Ut (ω) is nonconstant and Ut (ω) 6≈ Ut (ω 0 ) whenever Ft (ω) 6= Ft (ω 0 )
and Ft−1 (ω) = Ft−1 (ω 0 ).8
The tiebreakers (Wt )0≤t≤T are regular, i.e., (i) µ({ω ∈ Ω :
|M (At , Wt (ω))| = 1}) = 1 for all At ∈ At ; (ii) conditional on FT (ω), W0 , . . . , WT are independent; and (iii) µ(Wt ∈ Bt |FT (ω)) = µ(Wt ∈ Bt |Ft (ω)) for all t.9
The analyst does not observe the evolution of the agent’s private information and thus
cannot condition on events in Ft . The only events that the analyst can condition on are the
past choices of the agent.10 He interprets each history ht−1 = (A0 , p0 , . . . , At−1 , pt−1 ) as the event
Tt−1
C(pk , Ak ), where C(pk , Ak ) := {ω ∈ Ω : pk ∈ M (M (Ak , Uk (ω)), Wk (ω))}.
C(ht−1 ) := k=0
Definition 2. A dynamic random expected utility (DREU) representation of ρ is a tuple
(Ω, F ∗ , µ, (Ft , Ut , Wt )0≤t≤T ) such that (Ω, F ∗ , µ) is a finitely additive probability space, the
filtration (Ft ) ⊆ F ∗ and the Ft -adapted utility process Ut : Ω → RXt are simple, the
F ∗ -measurable tiebreaking process Wt : Ω → RXt is regular, and for all pt ∈ At and
ht−1 ∈ Ht−1 (At ),
ρt (pt ; At |ht−1 ) = µ C(pt , At )|C(ht−1 ) ,
(1)
where for t = 0, we abuse notation by letting C(ht−1 ) := Ω and ρ0 (p0 ; A0 |h−1 ) := ρ0 (p0 ; A0 ).
8
For t = 0, we let Ft−1 (ω) := Ω for all ω.
(ii) rules out additional serial correlation of tiebreakers, over and above the serial correlation inherent in
the agent’s payoff-relevant private information FT (ω). (iii) ensures that to the extent that period-t tie breaking
relies on payoff-relevant private information, it can rely only on the information Ft (ω) available at t.
10
Here we assume that the analyst doe not keep track of particular realization of consumption outcomes zt ,
which is without loss in absence of history dependent utilities. We consider a more general case in Section 9.
9
11
DREU is a very general model because it imposes no particular structure on the family
(Ut ); in particular, it does not have to satisfy the Bellman equation. Our characterization of
this model can thus be viewed as isolating the behavioral implications of history dependence
due to serially correlated private information, without imposing any additional restrictions. Of
course, of great interest are cases of DREU where the agent is forward-looking and correctly
takes her future behavior into account, so that (Ut ) satisfies the Bellman equation.11
This is the case in the evolving utility representation, where the agent has a time-separable
preference and there is an Ft -adapted process of state-dependent felicity functions ut : Ω → RZ
over instantaneous payoffs such that Ut is given by the Bellman equation
Ut (ω)(zt , At+1 ) = ut (ω)(zt ) + E
max Ut+1 (pt+1 )|Ft (ω)
pt+1 ∈At+1
(2)
for all t and (zt , At+1 ) ∈ Xt , where we abuse notation and set UT +1 := 0. Notice that we did
not include the discount factor in (2); this is because it cannot be identified with a general
time-dependent felicity ut .
The discount factor is identified in the special case of gradual learning, where felicity is
time-invariant but unknown to the agent, who is learning about it over time through exogenous
news.12 In this case, there exists an F ∗ -measurable state-dependent felicity ũ : Ω → RZ
and a discount factor δ ∈ (0, 1) such that UT (ω) = uT (ω) and Ut (ω)(zt , At+1 ) = ut (ω)(zt ) +
δE maxpt+1 ∈At+1 Ut+1 (pt+1 )|Ft (ω) for all t ≤ T − 1 and (zt , At+1 ), where ut (ω) := E[ũ|Ft (ω)]
for all t. It is without loss to normalize ũ to UT , so that gradual learning is described by
Ut (ω)(zt , At+1 ) = ut (ω)(zt ) + δE
max Ut+1 (pt+1 )|Ft (ω) ∀t ≤ T − 1, (zt , At+1 );
pt+1 ∈At+1
(3)
ut (ω) = E[UT |Ft (ω)] ∀t ≤ T − 1.
Definition
3. An evolving utility representation of ρ is a DREU representation
(Ω, F ∗ , µ, (Ft , Ut , Wt )0≤t≤T ) along with an Ft -adapted process of felicities ut : Ω → RZ
11
DREU could also accommodate various “behavioral” effects, such as temptation, regret, imperfect foresight,
etc. but we do not pursue this direction in the current paper.
12
Thus, gradual learning is a model of passive learning. The more general model in Section 9 can accommodate active learning, or experimentation, where each period the agent obtains additional information from her
consumption zt .
12
such that (2) holds.
A gradual learning representation of ρ is a DREU representation
(Ω, F ∗ , µ, (Ft , Ut , Wt )0≤t≤T ) along with a discount factor δ ∈ (0, 1) such that (3) holds.
3.3
Examples
Example 1 (Ahn–Sarver). Ahn and Sarver (2013) study a two period (T = 1) special case
of evolving utility with no randomness in period 0. There is no intermediate consumption:
in period 0 the agent chooses between menus of lotteries and in period 1 she chooses from
those menus. Let n ∈ Z denote no consumption; formally, the subdomain they are looking at
involves choices from menus in K({n} × K(∆(Z))) ⊆ A1 . The evolving utility representation
has a trivial period 0 sigma-algebra F0 = {∅, Ω} and an arbitrary finitely generated F1 . Thus,
the observed choices ρ0 involve no randomization (except for tie breaking) and can therefore be
summarized by a preference relation %0 , which by equation (2) is represented by U0 (n, A1 ) =
E[maxp1 ∈A1 u1 (p1 )]. By (1), in period 1 the probability that the agent chooses p1 from A1
equals µ[C(p1 , A1 )] (the conditioning event is trivial since no information is revealed by period
0 choices).
Example 2 (Patent Renewal). Pakes (1986) uses an evolving utility model to study optimal
decisions about patent renewal. In his model T is the initial lifetime of a patent and in each
period t < T the agent can decide whether to renew the patent (r) or not (n); in our notation
this corresponds to setting Z = {r, n} and all choices being between degenerate lotteries.
Choosing n leads to zero payoff in all periods greater or equal to t without any possibility of
future choices. Choosing r leads to an instantaneous net payoff ut (equal to the current return
to patent protection minus the cost of renewal) and an option to renew in the next period. Thus,
for any t < T there is only one possible history that the agent is facing (having renewed in all
previous periods) and only one possible menu (equal to the set Z). Equation (2) then becomes
Ut (n) = 0 and Ut (r) = ut + E[max{Ut+1 (n), Ut+1 (r)}|Ft ], which is equivalent to equation (1) of
Pakes (1986), the only difference being that he includes a discount factor which is identified in
his model due to the Markov assumption on the returns.
Example 3 (Advertising). Erdem and Keane (1996) use a version of the gradual learning
13
representation to model the effects of advertising on dynamic brand choice.13 In each period
the agent can choose one of the possible brands in the set Z. The prior utility of each brand
is distributed normally. It can be partially learned from advertising signals, which are also
assumed to be normal, thus resulting in a normal posterior; in our notation the dynamics of the
posterior is captured by the filtration (Ft ). The Bellman equation in this model is exactly our
equation (3), where ut = E[ũ|Ft ] + zt and zt is an additional utility shock (with an extreme
value distribution) added by Erdem and Keane (1996) to obtain simple expressions for choice
probabilities.
4
Dynamic Random Expected Utility
4.1
Axioms
To characterize DREU, we will impose two history independence axioms, the usual static random expected utility axioms at each history, and a history continuity axiom.
4.1.1
History Independence Axioms
Our first two axioms identify two cases in which histories ht−1 and ĥt−1 reveal the same information to the analyst. Capturing the fact that history dependence arises in DREU only
through the private information revealed by past choices, the axioms require that period-t
choice behavior be the same after two such histories.
The first axiom compares choices after two histories ht−1 and ĥt−1 that differ only in period k: Under ht−1 , the agent chooses lottery pk from menu Ak , whereas under ĥt−1 , she
chooses the same lottery pk from the larger menu Bk ⊇ Ak . Suppose additionally that conditional on hk−1 the choice of pk from Ak and the choice of pk from Bk in fact occur with the
same probability. Then we require that choice behavior be the same after ht−1 and ĥt−1 . For
0
0
any history ht−1 = (A0 , p0 , ..., At−1 , pt−1 ), let (ht−1
−k , (Ak , pk )) denote the sequence of the form
13
Their model also includes an element of experimentation: their consumers are learning both from exogenous
news (advertising) and from their own experience with different brands. Here we focus only on the exogenous
learning part; our general model in Section 9 captures both effects.
14
(A0 , p0 , ..., A0k , p0k , ..., At−1 , pt−1 ).14
t−1
Axiom 1 (Contraction History Independence). For any ht−1 = (ht−1
=
−k , (Ak , pk )), ĥ
k−1
(ht−1
) = ρk (pk , Bk |hk−1 ), we have
−k , (Bk , pk )) ∈ Ht−1 (At ) such that Bk ⊇ Ak and ρk (pk , Ak |h
ρt (·, At |ht−1 ) = ρt (·, At |ĥt−1 ).
To see the idea, suppose for simplicity that T = 1, in which case the axiom requires that
for any p0 ∈ A0 ⊆ B0 if ρ0 (p0 , A0 ) = ρ0 (p0 , B0 ), then ρ1 (·, A1 |A0 , p0 ) = ρ1 (·, A1 |B0 , p0 ) for any
A1 ∈ supp pA
0 . In general, the event that p0 is the best element of menu B0 is a subset of the
event that p0 is the best element of the smaller menu A0 ⊆ B0 ; thus, observing ĥ0 = (B0 , p0 )
may reveal more information about the agent’s possible period-0 preferences than h0 = (A0 , p0 ).
However, if we additionally know that ρ0 (p0 , A0 ) = ρ0 (p0 , B0 ), then the event that p0 is best
in A0 but not in B0 must have probability 0; in other words, we must put zero probability
on any preference that selects p0 from A0 but not from B0 . Given this, h0 and ĥ0 reveal the
same information, and hence call for the same predictions for period-1 choices. The following
example illustrates this with a concrete story in the population setting.
Example 4. There are two convenience stores, A and B, each carrying three types of milk
(whole, 2%, and 1%). Each store has a stable set of weekly customers whose stochastic process
of preferences is identical at both stores.15 Suppose that in week 0, store A’s delivery of 1%
milk breaks down unexpectedly. The purchasing shares at each store are given in Tables 1
and 2. Consider a customer of store A, Andy, and a customer of store B, Bob, who both buy
product
whole
2%
product
whole
2%
1%
market share
40%
60%
Table 1: Market shares at store A
market share
40%
35%
25%
Table 2: Market shares at store B
whole milk in week 0. If in week 1 all types of milk are available again at both stores, then
Contraction History Independence implies that Andy and Bob’s choice probabilities will be the
same. This is true because we have the same information about Andy and Bob. Since at store
14
15
0 A
0
k−1
In general this is not a history, but it is if A0k ∈ supp pA
, A0k ) > 0.
k−1 and Ak+1 ∈ supp(p )k and ρk (pk |h
For simplicity, we assume in the following that all preferences are strict.
15
A only whole and 2% milk were available in week 0, the possible week-0 preferences of Andy
are w 2 1 or w 1 2 or 1 w 2. By contrast, since store B stocked all three
types of milk, Bob’s possible preferences are w 2 1 or w 1 2. However, since we
additionally know that the share of customers purchasing whole milk in week 0 was the same
at both stores, ρ0 (w, {w, 1, 2}) = ρ0 (w, {w, 2}) = 0.4, we can also conclude that no customers
had the ranking 1 w 2 in week 0. Therefore, the analyst’s prediction is the same, since the
stochastic process that governs the transition from week-0 to week-1 preferences is the same
for Bob and Andy and in both cases the analyst conditions on exactly the same week-0 event
{w 2 1, w 1 2}.
N
The second history independence axiom takes into account that agents are expected utility
maximizers, which implies that certain convex combinations of histories must reveal the same
information. To state the axiom we will require the following notation. Given any ht , ĥt ∈ Ht
and λ ∈ [0, 1], let λht + (1 − λ)ĥt := (λAk + (1 − λ)Âk , λpk + (1 − λ)p̂k )tk=0 .16 For any finite set
of histories H t−1 ⊆ Ht−1 (At ), we can express the choice distribution at t conditional on H t−1
by
ρ(ht−1 )ρt (·, At |ht−1 )
,
t−1 )
ht−1 ∈H t−1 ρ(h
Q
k−1
).
we denote its probability by ρ(ht−1 ) := t−1
k=0 ρk (pk , Ak |h
ρt (·, At |H
where for any history ht−1
t−1
P
) :=
ht−1 ∈H t−1
P
Axiom 2 (Linear History Independence). For any ht−1 = (ht−1
−k , (Ak , pk )) ∈ Ht−1 (At ) and
H t−1 ⊆ Ht−1 with H t−1 = {(ht−1
−k , (λAk + (1 − λ)Bk , λpk + (1 − λ)qk )) : qk ∈ Bk } for some Bk
and λ ∈ (0, 1), we have ρt (·, At |ht−1 ) = ρt (·, At |H t−1 ).
To see the intuition, suppose again that T = 1. In this case, the axiom asserts that period-1
choice behavior after history h0 = (A0 , p0 ) is the same as after any set of histories of the form
H 0 = {(λA0 + (1 − λ)B0 , λp0 + (1 − λ)q0 ) : q0 ∈ B0 }. When B0 = {q0 }, this is the case because
under expected utility maximization choosing p0 from A0 and choosing λp0 + (1 − λ)q0 from
λA0 + (1 − λ){q0 } reveals the same information about the agent’s utility, see Figure 1. More
16
Again, this is not in general a history, but it is whenever λAk + (1 − λ)Âk ∈ supp(λpk−1 + (1 − λ)p̂k−1 )A
and ρk (λpk + (1 − λ)p̂k |λhk−1 + (1 − λ)ĥk−1 , λAk + (1 − λ)Âk ) > 0 for all k = 1, . . . , t.
16
generally, the choice of p0 from A0 and of an unspecified option in λ{p0 } + (1 − λ)B0 from
λA0 + (1 − λ)B0 reveals the same information.
p10
λp10 + (1 − λ)q0
q0
λp20 + (1 − λ)q0
p20
λp30 + (1 − λ)q0
p30
Figure 1: Choosing p10 from A0 = {p10 , p20 , p30 } and λp10 + (1 − λ)q0 from λA0 + (1 − λ){q0 } reveals
the same information—specifically, that period-0 preferences are represented by a utility vector in the
shaded area.
4.1.2
Limited Observability
Next, we wish to ensure that choice behavior in each period admits a static REU representation.
To this end, we will impose the known REU axioms from the static setting at each history.
Before doing so, however, we need to address the following “limited observability” problem.
Unlike in the static setting, where the analyst observes choices from all possible menus, in the
dynamic setting each history ht−1 of past choices restricts the set of current menus: ρt (·|ht−1 )
is only defined on the set At (ht−1 ) of menus that occur with positive probability after ht−1 —
typically very few menus. This severely limits the bite of standard REU axioms, which impose
consistency conditions across menus.
The inclusion of lotteries among the agent’s choice objects allows us to overcome this problem via the following “linear extrapolation” procedure: Consider any menu At and some history
ht−1 that does not lead to At . We define the agent’s counterfactual choice distribution from
At following ht−1 by extrapolating from the situation where the agent plans to make the sequence of choices captured by ht−1 , but is sometimes forced to instead make a sequence of
choices that does lead to menu At . More precisely, we consider a degenerate history dt−1 =
17
A
({q0 }, q0 , . . . , {qt−1 }, qt−1 ) such that At ∈ supp qt−1
and replace ht−1 = (A0 , p0 , . . . , At−1 , pt−1 )
with the history ĥt−1 := λht−1 + (1 − λ)dt−1 under which at every period k ≤ t − 1, the agent
faces menu λAk + (1 − λ){qk } and chooses lottery λpk + (1 − λ)qk .17 As discussed following Linear History Independence, under expected utility maximization the latter sequence of choices
reveals the same information about the agent as ht−1 . This motivates extrapolating from ĥt−1
to define choice behavior following ht−1 . Define the set of degenerate period-(t − 1) histories by
Dt−1 := {dt−1 ∈ Ht−1 : ∀k ≤ t − 1 ∃qk s.t. dk = ({qk }, qk )}.
Definition 4. For any t ≥ 1, At ∈ At , and ht−1 ∈ Ht−1 , define
t−1
ρht
(·; At ) := ρt (·; At |λht−1 + (1 − λ)dt−1 ).
(4)
for some λ ∈ (0, 1] and dt−1 ∈ Dt−1 such that λht−1 + (1 − λ)dt−1 ∈ Ht−1 (At ).
t−1
It follows from Axiom 2 (Linear History Independence) that ρht
(·; At ) is well-defined:
Lemma 14 shows that the RHS of (4) does not depend on the specific choice of λ and dt−1 .
t−1
Moreover, ρht
(·; At ) coincides with ρt (·; At |ht−1 ) whenever ht−1 ∈ Ht−1 (At ). In the following,
we do not distinguish between the extended and nonextended version of ρt and use ρt (·; At |ht−1 )
to denote both.
Extrapolation procedures similar to Definition 4 have been used in experimental and empirical work. In a recent study of temptation and self-control, Toussaert (2016) seeks to differentiate
between so-called random Strotz agents and Gul and Pesendorfer (2001) agents. Both make
the same period 0 choices between menus (given a normatively superior option n and tempting
but normatively inferior t, they have the ranking A1 := {n} B1 := {n, t} C1 := {t}), but
differ in their period 1 choices from B1 (the former will sometimes cave in to temptation, while
the latter always exerts costly self-control to choose n). In a typical incentivized experiment,
the analyst would not be able to observe either agent’s choice from B1 , since they would always
choose A1 in period 0. To overcome this problem, Toussaert (2016) confronts the subjects
with a period 0 choice between lotteries over menus: the choice of 0.5A1 + 0.3B1 + 0.2C1 over
17
If the agent is indifferent to the timing of randomization, this is equivalent to her facing menu Ak and
choosing pk with probability λ and being forced to choose qk with probability (1 − λ).
18
0.3A1 + 0.5B1 + 0.2C1 and 0.5A1 + 0.2B1 + 0.3C1 reveals the same information as the choice
of A1 over B1 and of B1 over C1 , yet it allows the analyst to observe choices from menu B1 in
period 1.
More broadly, random assignment is an important tool to obtain quasi-experimental variation in empirical research; for example, a growing literature on school choice obtains quasiexperimental variation in school assignment by exploiting the fact that many application mechanisms ration limited seats via lotteries.18 While these papers typically leverage this random
variation to identify the causal effect of entering a school on next-period outcomes such as test
scores, Definition 4 suggests exploiting it to make counterfactual inferences about next-period
choices. Figure 2 provides a toy example.
Polish
ch
(λ
ter
)
Polish
r
cha
er
art
y
ter regula
t
r (1
lo
−λ
)
man
Ger
Japa
nese
reg
reg
ula
r
ula
r
n
rma
Ge
Japa
man
Ger
Japa
nese
nese
Figure 2: Left: We cannot observe the probability ρ1 (G; {G, J}|{C, R}, C) with which students who
in period 0 chose a charter school (with Polish as its sole foreign language option) over a regular
school (offering German and Japanese) would choose to study German at the regular school in period
1. Right: If in period 0, students face a choice between a regular school, or a lottery sending them
to charter school with probability λ and regular school with probability 1 − λ, then Definition 4
extrapolates ρ1 (G; {G, J}|{C, R}, C) := ρ1 (G; {G, J}|{L, R}, L).
18
For example, Abdulkadiroglu, Angrist, Narita, and Pathak (forthcoming); Angrist, Hull, Pathak, and Walters (forthcoming); Deming (2011); Deming, Hastings, Kane, and Staiger (2014).
19
4.1.3
Static REU Axioms
The next axiom imposes the known static REU conditions on the extended choice distribution
from Definition 4 at each history. The first four conditions are the same as in Gul and Pesendorfer (2006) and the fifth is a slight modification of the finiteness condition in Ahn and
Sarver (2013). Theorem 4 in Appendix F shows that these conditions characterize static REU
representations not only for finite outcome spaces, as assumed in those papers, but for arbitrary
metric spaces of outcomes, in particular for each Xt .
For each t, ∆(Xt ) is endowed with the topology of weak convergence, which is metrized by
the Prokhorov metric π; At is endowed with the Hausdorff topology and ∆(∆(Xt )) with the
topology of weak convergence induced by π. Convergence in these topologies is denoted by
→. An additional convergence notion on ∆(Xt ) and At , convergence in mixture, is defined as
follows: For any pt ∈ ∆(Xt ) and sequence {pnt }n∈N ⊆ ∆(Xt ), write pnt →m pt if there exists
qt ∈ ∆(Xt ) and a sequence {αn }n∈N 19 with αn → 0 such that pnt = αn qt + (1 − αn )pt for all
n. Similarly, for any sequence {B n }n∈N ⊆ A, write Btn →m pt if there exists Bt ∈ At and a
sequence {αn }n∈N with αn → 0 such that Btn = αn Bt + (1 − αn ){pt } for all n. Finally, for any
At ∈ At and sequence (Ant )n∈N ⊆ At , write Ant →m At if for each pt ∈ At , there is a sequence
S
{Btn (pt )}n∈N ⊆ At such that Btn (pt ) →m pt and Ant = pt ∈At Btn (pt ) for all n.
Axiom 3 (Random Expected Utility). Fix any 0 ≤ t ≤ T and ht−1 ∈ Ht−1 . Then:
(i). Regularity: If At ⊆ A0t , then ρt (pt ; At |ht−1 ) ≥ ρt (pt ; A0t |ht−1 ) for all pt ∈ At .
(ii). Linearity: For any At , pt ∈ At , λ ∈ (0, 1), qt ,
ρt (pt ; At |ht−1 ) = ρt (λpt + (1 − λ)qt ; λAt + (1 − λ){qt }|ht−1 ).
(iii). Extremeness: For any At , ρt (ext(At ); At |ht−1 ) = 1.
(iv). Mixture Continuity: ρt (·; αAt + (1 − α)A0t |ht−1 ) is continuous in α for all At , A0t .
19
Note that we do not require αn ∈ [0, 1]. In particular, the definition allows αn < 0, as long as pn =
αn q + (1 − αn )p ∈ ∆(X).
20
(v). Finiteness: There exists K > 0 such that for all At , there is Bt ⊆ At with |Bt | ≤ K
such that for every pt ∈ At r Bt , there are sequences pnt →m pt and Btn →m Bt with
ρt (pnt ; {pnt } ∪ Btn |ht−1 ) = 0 for all n.
4.1.4
History Continuity
Our final axiom reflects the way in which tie-breaking can affect the observed choice distribution.
To state it, we first define menus and histories without ties directly from choice behavior:20
Definition 5. For any 0 ≤ t ≤ T and ht−1 ∈ Ht−1 , the set of period-t menus without ties
conditional on history ht−1 is denoted A∗t (ht−1 )21 and consists of all At ∈ At such that for any
pt ∈ At and any sequences pnt →m pt and Btn →m At r {pt }, we have
lim ρt (pnt , Btn ∪ {pnt }|ht−1 ) = ρt (pt , At |ht−1 ).
n→∞
For t = 0, we write A∗0 := A∗0 (ht−1 ). The set of period t histories without ties is Ht∗ := {ht =
0
(A0 , p0 , . . . , At−1 , pt−1 ) ∈ Ht : At0 ∈ A∗t0 (ht −1 ) for all t0 ≤ t}.
The following axiom relates choice distributions after nearby histories. To state it formally,
we extend convergence in mixture to histories by writing ht,n →m ht if ht,n = (An0 , pn0 , ..., Ant , pnt )
and ht = (A0 , p0 , ..., At , pt ) satisfy Ant0 →m At0 and pnt0 →m pt0 for each t0 .
Axiom 4 (History Continuity). For all 0 ≤ t ≤ T − 1, At+1 , pt+1 , and ht ,
ρt+1 (pt+1 ; At+1 |ht ) ∈ co{lim ρt+1 (pt+1 ; At+1 |ht,n ) : ht,n →m ht , ht,n ∈ Ht∗ }.
n
In general, if period t histories are slightly altered, we expect subsequent period t + 1 choice
behavior to be adjusted continuously, except when there was tie-breaking in the past. If the
agent chose pt from At as a result of tie-breaking, then slightly altering the choice problem can
20
Lu (2014, 2016a, 2016b) and Lu and Saito (2016) use an alternative approach by introducing sigma algebras
into their primitive. Our is in spirit similar to Ahn and Sarver (2013) by explicitly using local perturbations of
lotteries.
21
Note that A∗t (ht−1 ) 6⊆ At (ht−1 ) because the first set contains all menus without ties (we use history ht−1
here only to determine where ties could occur) while the second set contains only menus that occur with positive
probability given the sequence of choices ht−1 —typically very few menus.
21
change the set of states at which pt would be chosen and hence lead to a discontinuous change in
the private information revealed by the choice of pt . The history continuity condition restricts
the types of discontinuities ρt+1 can admit, ruling out situations in which choices after some
history are completely unrelated to choices after any nearby history. Specifically, the fact that
choice behavior after ht can be expressed as a mixture of behavior after some nearby histories
without ties reflects the way in which the agent’s tie-breaking procedure may vary with her
payoff-relevant private information.
4.2
Representation Theorem
Theorem 1. Let ρ be a dynamic stochastic choice rule. The following are equivalent:
(i). ρ satisfies Axioms 1–4.
(ii). ρ admits a DREU representation.
The proof of Theorem 1 appears in Appendix B. In Section 8, we sketch the argument for
sufficiency in the two-period setting (T = 1).
5
5.1
Special Cases with Dynamic Sophistication
Evolving Utility
Evolving Utility imposes the following additional requirements:
that the agent’s utility
Ut (zt , At+1 ) is separable across current consumption and continuation menus; that the agent
evaluates continuation menu At+1 by assuming that she will choose optimally from At+1 after
realization of next period’s utility; and that the agent’s anticipation of next period’s utility
distribution is correct. To characterize the behavioral implications of these additional properties, we find a way to translate existing axioms from the preference over menus literature to
our stochastic choice setting. The key ingredient is the following history dependent dominance
relation:
22
Definition 6. For each t ≤ T and ht = (ht−1 , At , pt ) ∈ Ht , define relation %ht , ∼ht , and ht
on ∆(Xt ) as follows. For any qt , rt ∈ ∆(Xt )
(i). qt %ht rt if there exist qtn →m qt and rtn →m rt such that ρt ( 21 pt + 12 rtn ; 12 At +
1
{qtn , rtn }|ht−1 )
2
= 0 for all n;
(ii). qt ∼ht rt if qt %ht rt and rt %ht qt ;
(iii). qt ht rt if qt %ht rt and rt 6%ht qt .
Lemma 5 in Appendix C shows that when ρ admits a DREU representation, %ht captures the
following notion of revealed stochastic preference: qt %ht rt entails that the agent weakly prefers
qt to rt at any state of the world consistent with history ht .22 To see the idea, suppose that at
some state of the world consistent with history ht = (ht−1 , At , pt ) the agent strictly prefers rt
to qt . Then conditional on history ht−1 there is positive probability that the agent both prefers
to choose pt from At and strictly prefers rt to qt ; moreover, since the latter preference is strict,
the same remains true for any close enough perturbations rtn and qtn . But then the agent would
choose 12 pt + 12 rtn from 12 At + 12 {qtn , rtn } with positive probability after ht−1 , i.e., qt 6%ht rt .
The following three axioms employ %ht to translate conditions known from the menu preference literature to our setting. The Separability axiom ensures that utility in every state of
the world has an additively separable form Ut (z, At+1 ) = ut (z) + Vt (At+1 ).23
A
Axiom 5 (Separability). For all t ≤ T − 1, ht and pt , qt with pzt = qtz and pA
t = qt , we have
pt ∼ht qt .
The next axiom translates conditions from Dekel, Lipman, and Rustichini (2001) that ensure that Vt (At+1 ) captures the option value contained in menu At+1 , i.e., that Vt (At+1 ) =
Êt [maxpt+1 ∈At+1 Ût+1 (pt+1 )] for some expectation operator Êt and some utility function Ût+1 .
Part (i) is Kreps’s (1979) preference for flexibility axiom; it says that the agent always weakly
prefers bigger menus. Part (ii) ensures that the agent cannot affect the filtration.24 Part
22
In the presence of Axioms 5 and 6 below, the converse is also true for t ≤ T − 1. See Corollary C.1 in
Appendix C.
23
This axiom can be found for example in Fishburn (1970), see his Theorem 11.1.
24
This axiom is relaxed in models of costly contemplation such as Ergin and Sarver (2010) and De Oliveira,
Denti, Mihm, and Ozbek (2016), where the agent has the ability to affect the signals she is receiving.
23
(iii) translates the continuity condition of Dekel, Lipman, and Rustichini (2001) and part (iv)
ensures that the partial relation %ht is nontrivial.25
Axiom 6 (DLR Menu Preference). For all t ≤ T − 1 and ht , the following hold:26
(i). Monotonicity: For any zt and At+1 ⊆ A0t+1 , we have (zt , A0t+1 ) %ht (zt , At+1 ).
(ii). Indifference to Timing: For any zt , At , A0t , and α ∈ (0, 1), we have (zt , αAt+1 + (1 −
α)A0t+1 ) ∼ht α(zt , At+1 ) + (1 − α)(zt , A0t+1 ).
(iii). Menu Continuity: ρt+1 (·|ht ) is continuous on A∗t+1 (ht ).
(iv). Menu Nondegeneracy: There exist A0t+1 , At+1 , and zt such that (zt , A0t+1 ) ht (zt , At+1 ).
The final axiom adapts the sophistication axiom due to Ahn and Sarver (2013). We require
that at any history ht , the agent values a menu A0t+1 strictly more than its subset At+1 if
and only if she actually chooses something from A0t+1 r At+1 with strictly positive probability
following ht . This axiom ensures that the agent correctly anticipates her future utility, that is
Êt [·] equals E[·|Ft ] and Ût+1 = Ut+1 .
Axiom 7 (Sophistication). For all t ≤ T −1, ht ∈ Ht , and At+1 ⊆ A0t+1 ∈ A∗t (ht ), the following
are equivalent:
(i). ρt+1 (pt+1 ; A0t+1 |ht ) > 0 for some pt+1 ∈ A0t+1 r At+1
(ii). (zt , A0t+1 ) ht (zt , At+1 ) for all zt .
Theorem 2. Suppose that the dynamic stochastic choice rule ρ admits a DREU representation.
The following are equivalent:
(i). ρ satisfies Axioms 5–7.
(ii). ρ admits an Evolving Utility representation.
25
In general, in the DLR-style model another Lipshitz continuity condition is needed; however, this is not
the case if the state space (or in our case the partition generating Ft ) is finite, see Dekel, Lipman, Rustichini,
and Sarver (2007). We do not need to impose an additional finiteness axiom on %ht because in the presence of
Sophistication this property is inherited from the finiteness axiom on ρ, Axiom 3 (v).
26
In the following, we identify any (zt , At+1 ) ∈ Xt with the Dirac lottery δ(zt ,At+1 ) ∈ ∆(Xt ).
24
5.2
Gradual Learning
Gradual Learning specializes Evolving Utility to the case in which the agent’s consumption
preference is time-invariant but unknown to her and she is learning about it over time. The
behavioral implications of this are captured by restrictions on the agent’s preference %ht over
streams of consumption lotteries.
Fix t ≤ T − 1. Given a consumption lottery ` ∈ ∆(Z) and menu At+1 ∈ At+1 , define
(`, At+1 ) ∈ ∆(Xt ) to be the period-t lottery that yields current consumption according to `
P
and yields continuation menu At+1 for sure; i.e., (`, At+1 ) :=
zt ∈Z `(zt )δ(zt ,At+1 ) . Given a
sequence `0 , . . . , `T −t ∈ ∆(Z) of consumption lotteries, let the stream (`0 , . . . , `T −t ) be the
period-t lottery that at every future period t + k yields consumption according to `k . Formally,
(`0 , . . . , `T −t ) := (`0 , At+1 ) ∈ ∆(Xt ), where the sequence of menus At+1 , . . . , AT is defined
recursively from period T backwards by AT := {`T } ∈ AT and As := {(`s−t , As+1 )} ∈ As for
all s = t + 1, . . . , T − 1. Finally, we write (`0 , . . . , `k , m, . . . , m ) if there is m ∈ ∆(Z) and
| {z }
T −(t+k) times
k ∈ {0, . . . , T − t} such that `
k+1
T −t
= ... = `
= m, where we do not specify the number of
m-entries when there is no risk of confusion.
The key axiom capturing learning is the following:
Axiom 8 (Stationary Consumption Preference). For all t ≤ T − 1, `, m, n ∈ ∆(Z), and ht ,
(`, n, . . . , n) ht (m, n, . . . , n) if and only if (n, `, n, . . . , n) ht (n, m, n, . . . , n).
Axiom 8 implies that at any history ht , the agent’s consumption utility today and her
expected consumption utility tomorrow induce the same preference over consumption lotteries.
The following example illustrates the connection with learning.
Example 5. Suppose the agent faces a choice between two providers of some service, e.g.,
two hairdressers or dentists. Based on her current information, she believes provider ` to be
better than m (n denotes no consumption), so will select the former if choosing between walkin appointments today. If her desired appointment date is next week, then the agent may in
general prefer to delay her decision, because she may be able to acquire more information about
` or m in the meantime. However, Axiom 8 says that if she is forced to decide today whether to
25
go to ` or m next week (say, because advance booking is required), then she must again prefer
to commit to `. This is because if the agent currently believes ` to be better than m, then by
the martingale property of beliefs she should expect her information next week to still favor `
on average.
N
The second learning axiom guarantees that the agent discounts the future at a deterministic
and time-invariant rate. There exists a weight α∗ (depending on the discount factor) that for
any consumption lotteries ` and m makes the agent indifferent between getting (` today and
m tomorrow) and the α∗ mixture of the two lotteries in both periods. The axiom ensures that
the weight α∗ (and therefore the discount factor) does not depend on the state or time period.
Axiom 9 (Intertemporal Calibration). There exists α∗ ∈ ( 12 , 1) such that for all t ≤ T − 1, ht
and `, m, n ∈ ∆(Z), we have (α∗ ` + (1 − α∗ )m, α∗ ` + (1 − α∗ )m, n, . . . , n) ∼ht (`, m, n, . . . , n).
Note that Axiom 9 has no bite if the agent is indifferent between all consumption lotteries.
Hence, we impose the following condition, which ensures that at every history the agent’s
consumption utility is nonconstant.
Condition 1 (Consumption Nondegeneracy). For all t ≤ T −1 and ht , there exist consumption
lotteries `, m, n ∈ ∆(Z) such that (`, n, . . . , n) 6∼ht (m, n, . . . , n).
Theorem 3. Suppose that ρ admits an Evolving Utility representation and Condition 1 is
satisfied. The following are equivalent:
(i). ρ satisfies Axioms 8 and 9.
(ii). ρ admits a Gradual Learning representation.
The proof is in Appendix D. The argument for sufficiency proceeds in three steps. Consider
an evolving utility representation (Ω, F ∗ , µ, (Ft , Ut , Wt , ut )) of ρ, where after adding appropriate
P
constants to each ut and Ut we can assume that z∈Z ut (ω)(z) = 0 for all ω and t. Fix any ω
and t ≤ T − 1. We first show that Axiom 8 (Stationary Consumption Preference) implies that
ut (ω) and ût (ω) := E[ut+1 | Ft (ω)] represent the same preference over consumption lotteries.
Thus, there exists a (possibly state and time-dependent) δt (ω) such that ût (ω) = δt (ω)ut (ω).
26
Next, let α∗ ∈ ( 12 , 1) be as in Axiom 9 (Intertemporal Calibration). Invoking Condition 1
(Consumption Nondegeneracy), we can find consumption lotteries ` and m such that ut (ω)(`) 6=
ut (ω)(m). Applied to these two lotteries and any third consumption lottery n, Axiom 9 yields
that δt (ω) =
1−α∗
α∗
=: δ is state and time-invariant and δ ∈ (0, 1). Finally, the above shows that
the process (δ −t ut ) is a martingale, so that δ −t ut (ω) = E[δ −T uT | Ft (ω)]. Thus, replacing Ut
with Ũt := δ −t Ut yields a gradual learning representation (Ω, F ∗ , µ, (Ft , Ũt , Wt ), δ) of ρ.
6
6.1
Properties of the Representations
Uniqueness
The following proposition, which we prove in Appendix G, summarizes the uniqueness properties
of the three representations axiomatized in the previous sections.
Proposition 1. Suppose ρ admits a DREU representation D = (Ω, F ∗ , µ, (Ft , Ut , Wt )). Consider D̂ = (Ω̂, F̂ ∗ , µ̂, (F̂t , Ût , Ŵt )), where (Ω̂, F̂ ∗ , µ̂) is a finitely additive probability space;
(F̂t ) ⊆ F ∗ is a filtration; and Ut , Wt : Ω → RXt for all t. Then D is a DREU representation of ρ if and only if (Ŵt ) is F ∗ -measurable and regular and for each t there exists a finite
partition Π̂t generating F̂t , a bijection gt : Πt → Π̂t , and Ft -measurable functions αt : Ω → R++
and βt : Ω → R such that for all ω ∈ Ω:
(i). µ(F0 (ω)) = µ̂(g0 (F0 (ω))) and µ(Ft (ω)|Ft−1 (ω)) = µ̂(gt (Ft (ω))|gt−1 (Ft−1 (ω))) if t ≥ 1;
(ii). Ut (ω) = αt (ω)Ût (ω̂) + βt (ω) whenever ω̂ ∈ gt (Ft (ω));
(iii). µ[{Wt ∈ Bt (ω)}|Ft (ω)] = µ̂[{Ŵt ∈ Bt (ω)}|gt (Ft (ω))] for any Bt (ω) such that Bt (ω) =
{w ∈ RX : pt ∈ M (M (At , Ut (ω)), w)} for some pt ∈ At ∈ At .
If (D, (ut )) is an evolving utility representation of ρ, then (D̂, (ût )) is an evolving utility representation of ρ if and only if (i)-(iii) hold and additionally
(iv). αt (ω) = α0 (ω) for all ω ∈ Ω and t = 0, . . . , T ;
27
(v). ut (ω) = α0 (ω)ût (ω̂) + γt (ω) whenever ω̂ ∈ gt (Ft (ω)), where γT (ω) := βT (ω) and γt (ω) :=
βt (ω) − E[βt+1 |Ft (ω)] if t ≤ T − 1.
If (D, δ) is a gradual learning representation of ρ and ρ satisfies Condition 1, then (D̂, δ̂) is a
gradual learning representation of ρ if and only if (i)-(iv) hold and additionally
(vi). δ = δ̂
(vii). βt (ω) =
1−δ T −t+1
E[βT |Ft (ω)]
1−δ
for all ω and t.
Points (i) and (ii) of Proposition 1 show that in DREU, the agent’s choices uniquely determine her underlying stochastic process of ordinal payoff-relevant private information, while
point (iii) shows that the (ordinal) distribution of tie-breakers is pinned down for choices featuring ties. This is the period-by-period dynamic analog of known identification results for
static REU representations (Proposition 4 in Ahn and Sarver (2013)). Point (iv) shows that
evolving utility implies strictly sharper identification than DREU of the agent’s cardinal private
information: In particular, the random scaling factor αt (ω) used to transform Ût into Ut must
be the same in all periods and measurable with respect to period-0 private information. This
allows for meaningful intertemporal comparisons such as “in state ω, the additional period-t
utility for pt over qt is greater than the additional period-(t + 1) utility for pt+1 over qt+1 ” and
cross-state comparisons such as “the additional period-t utility for pt over qt is greater in state
ω than in state ω 0 ∈ F0 (ω).” This generalizes the main identification result (Theorem 2) in
Ahn and Sarver (2013): In a two-period setting without consumption in period 0, they obtain
U1 (ω) = αÛ1 (ω̂) + β1 (ω), where α is constant since they do not allow for period-0 private information. Finally, gradual learning allows for unique identification of the discount factor (point
(vi)) and entails even sharper identification of cardinal private information (point (vii)).
6.2
Choice and Taste Persistence
In the context of evolving utility, an important special case of serial correlation is persistence
at the level of the agent’s consumption utility process ut . This section explores the relationship
between persistence of ut and observable consumption choices.
28
Our first notion of consumption persistence captures the idea that (absent ties) the agent
is more likely to choose consumption lottery p today if she consumed p yesterday than if she
consumed q yesterday, provided today’s menu does not include any new consumption options
relative to yesterday’s menu. To state this formally we focus on atemporal menus, i.e., menus
that do not feature any intertemporal tradeoffs, so that the agent’s choices are governed solely
by her preference over current consumption. Suppose t ≤ T − 1. Menu At is atemporal if for
Z
Z
A
any pt , qt ∈ At , we have pA
t = qt . We write At := {pt : pt ∈ At } ⊆ ∆(Z). All period T menus
AT are atemporal with AZT = AT .
Definition 7. ρ features consumption persistence if for any ht = (ht−1 , At , pt ), ĥt =
(ht−1 , At , qt ) ∈ Ht and pt+1 ∈ At+1 ∈ A∗t+1 (ht ) ∩ A∗t+1 (ĥt ) with At and At+1 atemporal,
AZt+1 ⊆ AZt , and pZt = pZt+1 , we have ρt+1 (pt+1 ; At+1 |ht−1 , At , pt ) ≥ ρt+1 (pt+1 ; At+1 |ht−1 , At , qt ).
If ρ admits an evolving utility representation, consumption persistence is equivalent to a
particular form of taste persistence. To obtain this characterization, we impose the assumption
that there are two consumption lotteries ` and ` such that the agent strictly prefers the former
to the latter at all histories. This condition is innocuous if, for example, the outcome space
includes a monetary dimension.
Condition 2 (Uniformly Ranked Pair). There exist `, ` ∈ ∆(Z) such that for all t, ht , and
m ∈ ∆(Z), we have (`, m, . . . , m) ht (`, m, . . . , m).
Given any set D ⊆ RZ of felicities, let [D] := {w ∈ RZ : w ≈ v for some v ∈ D}.
Proposition 2. Suppose ρ admits an evolving utility representation (Ω, F ∗ , µ, (Ft , Ut , Wt , ut ))
and Condition 2 holds. Then the following are equivalent:
(i). ρ features consumption persistence
(ii). for any u, u0 ∈ RZ , convex D ⊆ RZ with u ∈ D, and ht−1 with µ({ut ≈ u}|C(ht−1 )),
µ({ut ≈ u0 }|C(ht−1 )) > 0, we have
µ({ut+1 ∈ [D]}|C(ht−1 ) ∩ {ut ≈ u}) ≥ µ({ut+1 ∈ [D]}|C(ht−1 ) ∩ {ut ≈ u0 }).
29
An alternative notion, which is neither implied by nor implies consumption persistence, is
consumption perseverance. This says that if an agent chose a particular consumption lottery
yesterday, then (absent ties) she will continue to choose it with positive probability today, as
long as today’s menu does not include any new consumption options relative to yesterday’s
menu.
Definition 8. ρ features consumption perseverance if for any ht = (ht−1 , At , pt ) ∈ Ht and
pt+1 ∈ At+1 ∈ A∗t+1 (ht ) with At and At+1 atemporal, AZt+1 ⊆ AZt , and pZt = pZt+1 , we have
ρt (pt+1 ; At+1 |ht−1 , At , pt ) > 0.
In the presence of Condition 1 (Consumption Nondegeneracy) from Section 5.2, consumption
perseverance is equivalent to an alternative notion of taste persistence which requires that after
any history at which the agent’s period t consumption preference was given by u, her period
t + 1 consumption preference remains u with positive probability.
Proposition
3. Suppose
that
ρ
admits
an
evolving
utility
representation
(Ω, F ∗ , µ, (Ft , Ut , Wt , ut )) and Condition 1 is satisfied. Then the following are equivalent:
(i). ρ features consumption perseverance
(ii). for any u ∈ RZ and ht−1 with µ({ut ≈ u}|C(ht−1 )) > 0, we have
µ({ut+1 ≈ u}|C(ht−1 ) ∩ {ut ≈ u}) > 0.
6.2.1
Application: Markov Chain
To illustrate the preceding results, we consider an evolving utility representation in which the
felicity ut follows a finite Markov chain. Specifically, let U = {u1 , ..., um } denote a set of possible
felicities, where ui 6≈ uj for any i 6= j. Let M be an irreducible transition matrix, where Mij
denotes the probability that period t+1 utility is uj conditional on period t utility being ui . The
initial distribution is assumed to have full support, but need not be the stationary distribution.
30
Corollary 1. Consider an evolving utility representation generated by the finite Markov chain
(U, M ).
(i). Assume that for any i ∈
/ {j, k, l}, we have ui 6∈ [co{uj , uk , ul }]. Then ρ features consumption persistence if and only if the Markov chain is a renewal process, i.e., there exists
α ∈ [0, 1) and ν ∈ ∆(U) such that Mii = α + (1 − α)ν(ui ), Mij = (1 − α)ν(uj ).
(ii). ρ features consumption perseverance if and only if Mii > 0 for every i.
Point (i) above imposes a regularity condition on the structure of U. This assumption
is generically satisfied if the outcome space is rich enough relative to the number of utility
functions. The following example does not satisfy the condition. It features consumption
persistence under a non-renewal process.
Example 6 (Random walk over a line). Consider an evolving utility representation where the
felicity process is of the form ut = w + α(xt )v, where w, v ∈ RZ are fixed felicities, α : Z →
R is a strictly increasing function, and xt follows a random walk over Z: it remains at its
current value with probability p, increases by one with probability
with probability
1−p
.
2
1−p
,
2
and decreases by one
This agent features consumption persistence if and only if p ≥
consumption perseverance if and only if p > 0.
7
1
3
and
N
Comparison with Dynamic Discrete Choice
In this section we show that DREU nests the Rust model, which in turn nests a version of
evolving utility in which the agent’s private information is given by her past realizations of
consumption utility. However, the often used i.i.d. specification of the Rust model violates all
of the axioms that characterize the evolving utility model. This happens because the two models
make opposite predictions about the option value. In the evolving utility model the agent has
positive option value: she likes bigger menus, as they provide her with more flexibility and
wants to make her decisions as late as possible to condition on as much information as possible.
On the other hand, the i.i.d. Rust agent sometimes prefers to commit to smaller menus and to
make her decisions as early as possible, thus displaying a negative value of information.
31
This negative option value is a consequence of the i.i.d. specification of utility shocks, which
fails to capture Bayesian learning and is instead motivated by the need for a statistically nondegenerate econometric model. This points to a possible modeling tradeoff between between
having well-behaved likelihoods and positive value of information. We leave the exploration of
this tradeoff for future research.
7.1
The Rust Model
The general Rust model features a state variable with two components: the first one is jointly
observed by both the agent and the analyst and the second one is observed only by the agent
and unobservable to the analyst. To make this model comparable to ours we abstract away from
the jointly observable state variables (more precisely the only jointly observable state variable is
the menu of available decisions At ). The random variable t ∈ RXt is a vector of payoff shocks,
(zt ,At+1 )
each entry t
corresponding to a deterministic action (zt , At+1 ) ∈ Xt and having mean
zero ex ante. We will denote the utilities in this representation by Vt , vt to distinguish them
from our main representations. The Bellman equation in this model is
Vt (zt , At+1 ) = vt (zt ) +
(z ,A
)
t t t+1
+ δE
max Vt+1 (pt+1 )|Gt .
pt+1 ∈At+1
(5)
Here the process (Vt )t of non-constant vNM utility indexes on Xt is measurable with respect
to the filtration (Gt ) generated by (t ) and the functions vt : Z → R are deterministic. In the
remainder of the paper we will refer to this model as the Rust model.27 Let (Ω, F ∗ , µ) be a
probability space that carries all the random variables t .28
Proposition 4. The Rust model is a special case of DREU and therefore leads to an observable
distribution of choices ρ that satisfies Axioms 1–4. Moreover, the Rust model with shocks to
(zt ,At+1 )
payoffs, where t
(zt ,Bt+1 )
= t
=: zt t for all zt and At+1 , Bt+1 , yields a special case of
evolving utility, where Ut := Vt , ut (zt ) := vt (zt ) + zt t and (Ft ) is generated by (ut ).
27
Rust (1994) assumes that t have full support; to make things comparable we assume that t has finite
support. Rust (1994) defines choices only between deterministic actions; our extension of his model preserves
linearity of expected utility, ensures that the model is a special case of random expected utility, and avoids
several conceptual complications associated with other extensions (see Apesteguia and Ballester, 2017).
28
As in Section 3.2, we also define a stochastic process (Wt ) of regular tie-breakers on this space.
32
However, not all Rust models are of the evolving utility kind. The leading example that
lies outside is the Rust model with i.i.d. shocks, where the entries of t are independently and
identically distributed, an assumption that is often made in applied work.29 In this model each
option (zt , At+1 ) gets its own shock, the distribution of which is identical irrespective of the size
and nature of the continuation menu At+1 .
In the Rust model with shocks to payoffs, shocks apply only to the immediate payoffs zt .
This implies that the amount of randomness in choices from the menu {(zt , At+1 ), (zt , Bt+1 )}
depends on how much of the utility shock to payoffs in t + 1 is realized already in period 0: the
more the agent learns in period 0, the more random her choices are. In the extreme case where
ut is uncorrelated over time, choices in period 0 are deterministic. Thus, the randomness of
choices in this model is coupled with learning. On the other hand, the Rust model with i.i.d.
shocks introduces an additional source of variability of the continuation value that is purely a
mechanical effect of the variability of t across different continuation menus and is unrelated
to the agent’s expectations about fundamentals (the E[·|Gt ] part of the Bellman equation).
Choices from the menu {(zt , At+1 ), (zt , Bt+1 )} are always random, even if the menu At+1 is a
strict subset of Bt+1 . This variability leads to a violation of all of the evolving utility axioms.
Proposition 5. The Rust model with non-degenerate i.i.d. shocks violates Axiom 5, all parts
of Axiom 6, and Axiom 7.30
Note that the various instances of the Rust model may coincide on restricted choice domains,
where for each immediate outcome z there is only one continuation menu At+1 available. This
is for example the case in the setting of Pakes (1986), where the decision to renew the patent
in period t fully determines the continuation menu in period t + 1. However, many real-life
decision problems involve choices that do not affect the immediate payoffs but have delayed
consequences and therefore do not belong to this restricted domain. On such richer choice
29
See, e.g., Miller (1984), Rust (1989), Hendel and Nevo (2006), Kennan and Walker (2011), Sweeting (2011),
and Gowrisankaran and Rysman (2012), all of whom assume in addition the extreme value distribution of .
Formally speaking, since the set Xt is infinite, the iid specification violates the Finiteness axiom so such a model
is not a special case of DREU; we abstract from this technical issue to focus on more fundamental axioms.
30
A special case of the Rust model with iid shocks—the dynamic logit model—was axiomatized by Fudenberg
and Strzalecki (2015) using axioms similar to Axioms 5, 6, and 7 but with a weaker notion of the % relation,
reflecting choice more than 50% of the time.
33
domains the two models make different predictions, as illustrated by the following example.
Example 7 (Decision Timing). Suppose that there are three periods t = 0, 1, 2 and the payoff
in periods 0 and 1 is x irrespective of the decision of the agent; for simplicity assume that the
utility of x is always zero. The payoff in period 2 is either y or z, depending on the decision of
the agent. The agent can choose between y and z either in period 1 or in period 2; the decision
when to choose is made in period 0. Formally, in period 0 the agent faces the menu A0 =
early
= {(x, {y}), (x, {z})} or
), (x, Alate
{(x, Aearly
1
1 )} and in period 1 she faces either the menu A1
the menu Alate
= {(x, {y, z})}, depending on her period-0 choice. This is illustrated in Figure 3.
1
y })
(x, {
ly
,
(x
r
ea
1
A
)
(x, {
z
})
y
z
(x
, A la
y
1 te
)
(x, {y, z})
z
Figure 3: Decision Timing.
In the evolving utility model we have
) = E[max{E[u2 (y)|F1 ], E[u2 (z)|F1 ]}|F0 ]
U0 (x, Aearly
1
U0 (x, Alate
1 ) = E[E[max{u2 (y), u2 (z)}|F1 ]|F0 ].
By the conditional Jensen inequality, the agent always weakly prefers to decide late. Moreover,
this preference is strict at ω as long as there exist ω 0 , ω 00 ∈ F0 (ω) with F1 (ω 0 ) = F1 (ω 00 ) such
that u2 (y) − u2 (z) changes sign on {ω 0 , ω 00 }. In sum, absent ties, the agent chooses to make
decisions late with probability 1, because more information enables her to better tailor her
34
choice to the state.31
This preference for late decisions does not hold under the i.i.d. specification of the Rust
model, where we have
)
(x,Aearly
1
V0 (x, Aearly
) = 0
1
(x,Alate
1 )
V0 (x, Alate
1 ) = 0
(x,{y})
+ E0 [max{δ 2 v2 (y) + δε1
(x,{z})
, δ 2 v2 (z) + δε1
}]
+ E0 [max{δ 2 v2 (y) + δ 2 εy2 , δ 2 v2 (z) + δ 2 εz2 }].
The simplest case to analyze is when v2 (y) = v2 (z): In this case, the comparison of the
(x,{y})
continuation values boils down to the comparison between δE0 [max{ε1
(x,{z})
, ε1
}] and
δ 2 E0 [max{εy2 , εz2 }]. Since the shocks are i.i.d. with ex ante mean zero, the former dominates the latter, so that the agent chooses to decide early with probability greater than 0.5.
However, this choice pattern does not hinge on this assumption: for example, in the case of
extreme value shocks Fudenberg and Strzalecki (2015) showed that it holds for any values of
v2 (y) and v2 (z).
N
Thus, the Rust agent with i.i.d. shocks typically prefers to make her decisions early rather
than late. As we showed in Proposition 5, she also violates the Monotonicity axiom, that is she
does not always prefer flexibility and sometimes likes to commit. Such patterns of behavior are
typical of agents who suffer from temptation or other behavioral problems, but are inconsistent
with Bayesian rationality. This calls into question the interpretation of as “unobserved state
variable,” an understanding that is typical in the discrete choice literature.
8
Proof Sketch of Theorem 1
The proof of Theorem 1 appears in Appendix B. Here we sketch the argument for sufficiency
in the two-period setting (T = 1).
Step 1: Static random expected utility representations. Theorem 4 in Appendix F
extends the characterizations of static random expected utility in Gul and Pesendorfer (2006)
31
A related finding is Theorem 2 of Krishna and Sadowski (2016), who show that one agent prefers to commit
to a constant consumption plan more than another agent if and only if his utility process is more autocorrelated,
in other words, when he expects to learn less in the future.
35
U02
U02
q02
U01
q01
p20
U01
U03
U02
p10 p30
q03
r02
U02
U01
p0
U03
r01 r03
Â0 = 12 A0 + 12 D0
D0
U02
U03
r0
A0
Figure 4: Suppose S0 = {s10 , s20 , s30 } with corresponding utilities U01 , U02 , U03 . Menu D0 is a separating
menu from which q0i is chosen precisely in state si0 . In menu A0 = {p0 , r0 }, p0 is chosen with probability
1 in state s10 ; tied with r0 in s20 ; and never chosen in s30 . In Â0 = 21 A0 + 21 D0 , p0 is replaced with three
copies {p10 , p20 , p30 }: Each pi0 is chosen in state si0 with the same probability with which p0 is chosen at si0
and is never chosen otherwise. Step 3 shows choice probabilities following (Â0 , pi0 ) are the same as following (D0 , q0i ). Step 4 shows choice probabilities following (A0 , p0 ) are a weighted sum of choice probabilities following (Â0 , pi0 ), with weights given by µ0 (si0 |C0 (p0 , A0 )). Combined with the static represenP
si
tation of ρ1 (·|D0 , q0i ) (Step 2), this yields ρ1 (p1 , A1 |A0 , p0 ) = 3i=1 µ10 (C1i (p1 , A1 ))µ0 (si0 |C0 (p0 , A0 )).
and Ahn and Sarver (2013) to an infinite-outcome space setting. By Axiom 3, this theorem
yields a static REU representation (Ω0 , F0∗ , F0 , µ0 , U0 , W0 ) of ρ0 and for each h0 ∈ H0 , a static
0
0
0
0
0
0
REU representation (Ωh1 , F1∗h , µh1 , F1h , U1h , W1h ) of ρ1 (·|h0 ). Thus far, there is no relationship
between the period-0 and period-1 representations. In the following, we use Axioms 1, 2, and 4
to combine them into a DREU representation, which requires ρ1 (p1 , A1 |A0 , p0 ) to be represented
as a conditional probability, with respect to a single underlying probability space Ω, of the event
C(p1 , A1 ) given the event C(p0 , A0 ).
Step 2: Period-1 choices conditional on period-0 states. Let S0 be the finite partition
of Ω0 that generates F0 . We refer to cells s0 ∈ S0 as states and let Us0 = U0 (ω) for any
ω ∈ s0 ∈ S0 . For any history (A0 , p0 ), define U0 (A0 , p0 ) := {Us0 : s0 ∈ S0 and p0 ∈ M (A0 , Us0 )}
to be the set of period-0 utilities consistent with the choice of p0 from A0 . Since (U0 , F0 ) is
simple, each Us0 is nonconstant and induces a different preference, so by standard arguments
(Lemma 12 in the appendix) we can find a menu D0 = {q0s0 : s0 ∈ S0 } that strictly separates
s∗
all states, i.e., such that for any s∗0 ∈ S0 we have U0 (D0 , q00 ) = {Us∗0 }. Figure 4 shows an
example. For the remainder of this proof sketch, fix such a separating menu D0 and define
ρs10 (p1 , A1 ) := ρ1 (p1 , A1 |D0 , q0s0 ) for each A1 and p1 . The representation of ρ1 (·|D0 , q0s0 ) obtained
in Step 1 then constitutes a static REU representation (Ωs10 , F ∗s0 , µs10 , U1s0 , W1s0 ) of ρs10 .
36
Step 3: ρs10 is well-defined. We now use Linear History Independence, Contraction History Independence, and History Continuity to show that for any (B0 , q0 ) ∈ H0 such that
U0 (B0 , q0 ) = {Us0 }, we have ρ1 (·, A1 |B0 , q0 ) = ρs10 (·, A1 ).
To see this, assume first that
M (B0 , Us0 ) = {q0 }, i.e., q0 is the unique maximizer of Us0 in B0 . Define r0 :=
1
q
2 0
+ 21 q0s0 ,
B̃0 := 12 B0 + 21 q0s0 , and D̃0 := 12 D0 + 12 q0 . Then U(B̃0 , r0 ) = U(B̃0 ∪ D̃0 , r0 ) = U(D̃0 , r0 ) = {Us0 }.
From the static REU representation of ρ0 and because M (B0 , Us0 ) = {q0 }, it follows that
ρ0 (r0 , B̃0 ) = ρ0 (r0 , B̃0 ∪ D̃0 ) = ρ0 (r0 , D̃0 ) = µ0 (s0 ).
(6)
But then
ρ1 (·, A1 |B0 , q0 ) = ρ1 (·, A1 |B̃0 , r0 ) = ρ1 (·, A1 |B̃0 ∪ D̃0 , r0 )
= ρ1 (·, A1 |D̃0 , r0 ) = ρ1 (·, A1 |D0 , q0s0 ) = ρs10 (·, A1 ),
where the first and fourth equalities follow from Axiom 2 (Linear History Independence), the
second and third equalities from Axiom 1 (Contraction History Independence) and (6), and the
final equality holds by definition. Finally, using Axiom 4 (History Continuity), we can extend
this argument to the case where in state s0 , q0 is tied with other lotteries in B0 .
Step 4: Splitting histories into states. Now consider a general history h0 = (A0 , p0 ).
By mixing with the separating menu D0 , we can decompose ρ1 (·|h0 ) into a weighted sum of
choice probabilities conditional on each state s0 , where the weight on s0 is the µ0 -conditional
probability of s0 given history h0 . Concretely, let Â0 :=
1
A
2 0
+ 12 D0 , and for any s0 ∈ S0 ,
let ps00 := 12 p0 + 21 q0s0 . This is depicted in Figure 4. Note that by construction of D0 and the
representation of ρ0 , we have that ρ0 (ps00 , Â0 ) = µ0 (C0 (p0 , A0 )|s0 )µ0 (s0 ), where C0 (p0 , A0 ) is the
event in Ω0 that p0 is chosen from A0 . Moreover, whenever ρ0 (ps00 , Â0 ) > 0, then U0 (Â0 , ps00 ) =
{Us0 }, so Step 3 together with the representation of ρs10 implies ρ1 (p1 , A1 |Â0 , ps00 ) = ρs10 (p1 , A1 ) =
37
µ1 (C1s0 (p1 , A1 )), where C1s0 (p1 , A1 ) is the event in Ωs10 that p1 is chosen from A1 . Then
X
1
ρ0 (ps00 , Â0 )
1
ρ1 (p1 , A1 |A0 , p0 ) = ρ1 (p1 , A1 |Â0 , p0 + D0 ) =
ρ1 (p1 , A1 |Â0 , ps00 ) P
s0
2
2
ρ0 (p 0 , Â0 )
0
s ∈S
0
X
=
s0 ∈S0
s0 ∈S0
0
µ0 (C0 (p0 , A0 )|s0 )µ0 (s0 )
=
0
0
s0 ∈S0 µ0 (C0 (p0 , A0 )|s0 )µ0 (s0 )
s
X
µs10 (C1s0 (p1 , A1 )) P
0
0
µs10 (C1s0 (p1 , A1 ))µ0 (s0 |C0 (p0 , A0 )).
0 ∈S0
(7)
Indeed, the first equality follows from Linear History Independence, the second equality from
the definition of ρ1 conditional on a set of histories, the third from the observations of the
preceding paragraph, and the fourth from Bayes’ rule.
Step 5: Completing the proof. Now define Ω =
S
s0 ∈S0
s0 × Ωs10 . In the natural way, the
partitions S0 of Ω0 and S1s0 of Ωs10 induce a finitely generated filtration on Ω, and the random
utilities and tie-breakers on Ω0 and Ωs10 induce processes of utilities and tiebreakers on Ω.32
Define µ on Ω by µ(E0 × E1 ) = µ0 (E0 ) × µs10 (E1 ) for any measurable E0 ⊆ s0 , E1 ⊆ Ωs10 . From
the construction of Ω and (7), it is then easy to see that ρ1 (p1 , A1 |h0 ) = µ(C(p1 , A1 )|C(p0 , A0 )),
where C(pt , At ) denotes the event in Ω that pt is chosen from At . Thus, ρ admits a DREU
representation, as required.
9
10
Extension: History Dependent Utility
Conclusion
Specifically, let F0 be generated by the partition {s0 × Ωs10 : s0 ∈ S0 } and F1 by the partition {s0 ×
s1 : s0 ∈ S0 , s1 ∈ S1s0 }. For any (ω0 , ω1 ) ∈ s0 × Ωs10 , let U0 (ω0 , ω1 ) = U0 (ω0 ), U1 (ω0 , ω1 ) = U1s0 (ω1 ) and
W0 (ω0 , ω1 ) = W0 (ω0 ), W1 (ω0 , ω1 ) = W1s0 (ω1 ).
32
38
Appendix: Proofs
A
Equivalent Representations
Instead of working with probabilities over the grand state space Ω, our proofs of Theorems 1–
3 will employ equivalent versions of our representations, called S-based representations, that
look at one-step-ahead conditionals. Section A.1 defines S-based representations. Section A.2
proves the equivalence between DREU, evolving utility, and gradual learning representations
and their respective S-based analogs. Section A.3 introduces important terminology regarding the relationship between states and histories that will be used throughout the proofs of
Theorems 1–3.
A.1
S-based Representations
For any X ∈ {X0 , . . . , XT }, A ∈ K(∆(X)), p ∈ ∆(X), let N (A, p) := {U ∈ RX : p ∈ M (A, U )}
and N + (A, p) := {U ∈ RX : {p} = M (A, U )}.
Definition 9. A random expected utility (REU) form on X ∈ {X0 , . . . , XT } is a tuple
(S, µ, {Us , τs }s∈S ) where
(i). S is a finite state space and µ is a probability measure on S
(ii). for each s ∈ S, Us ∈ RX is a nonconstant utility over X.
(iii). for each s ∈ S, the tie-breaking rule τs is a finitely additive probability measure on the
Borel σ-algebra on RX and is regular, i.e., τs (N + (A, p)) = τs (N (A, p)) for all A, p.
Given any REU form (S, µ, {Us , τs }s∈S ) on Xi and any s ∈ S, Ai ∈ Ai , and pi ∈ ∆(Xi ), define
τs (pi , Ai ) := τs ({w ∈ RXi : pi ∈ M (M (Ai , Us ), w)}).
Definition 10. An S-based DREU representation of ρ consists of tuples (S0 , µ0 , {Us0 , τs0 }s0 ∈S0 ),
s
(St , {µt t−1 }st−1 ∈St−1 , {Ust , τst }st ∈St )1≤t≤T such that for all t = 0, . . . , T , we have:
s
DREU1: For all st−1 ∈ St−1 , (St , µt t−1 , {Ust , τst }st ∈St ) is an REU form on Xt such that33
33
s
For t = 0, we abuse notation by letting µt t−1 denote µ0 for all st−1 .
39
s
(a) Ust 6≈ Us0t for any distinct pair st , s0t ∈ supp(µt t−1 );
s0
s
(b) supp(µt t−1 ) ∩ supp(µt t−1 ) = ∅ for any distinct pair st−1 , s0t−1 ;
(c)
S
s
st−1 ∈St−1
supp µt t−1 = St .
DREU2: For all pt , At , and ht−1 = (A0 , p0 , A1 , p1 , . . . , At−1 , pt−1 ) ∈ Ht−1 (At ),34
P
t−1
ρt (pt , At |h
)= P
(s0 ,...,st )∈S0 ×...×St
sk−1
(sk )τsk (pk , Ak )
k=0 µk
.
Qt−1 sk−1
(sk )τsk (pk , Ak )
k=0 µk
Qt
(s0 ,...,st−1 )∈S0 ×...×St−1
An S-based evolving utility representation of ρ is an S-based DREU representation such that
for all t = 0, . . . , T , we additionally have:
EVU: For all st ∈ St , there exists ust ∈ RZ such that for all zt ∈ Z, At+1 ∈ At+1 , we have
Ust (zt , At+1 ) = ust (zt ) + Vst (At+1 ),
where Vst (At+1 ) :=
P
st+1
t
µst+1
(st+1 ) maxpt+1 ∈At+1 Ust+1 (pt+1 ) for t ≤ T − 1 and VsT ≡ 0.
An S-based gradual learning representation is an S-based evolving-utility representation such
that additionally:
GL: There exists δ ∈ (0, 1) such that for all t = 0, . . . , T − 1 and st ∈ St , we have
ust =
1 X st
µt+1 (st+1 )ust+1 .
δs
t+1
A.2
Equivalence Result
Proposition 6. Let ρ be a dynamic stochastic choice rule.
(i). ρ admits a DREU representation if and only if ρ admits an S-based DREU representation.
(ii). ρ admits an evolving utility representation if and only if ρ admits an S-based evolving
utility representation.
(iii). ρ admits a gradual learning representation if and only if ρ admits an S-based gradual
learning representation.
Proof.
34
For t = 0, we again abuse notation by letting ρt (·|ht−1 ) denote ρ0 (·) for all ht−1 .
40
A.2.1
DREU
“If ”
direction:
Suppose
ρ
admits
s
(St , {µt t−1 }st−1 ∈St−1 , {Ust , τst }st ∈St )t=0,...,T .
an
S-based
DREU
representation
We will construct a DREU representation
(Ω̂, F̂ ∗ , µ̂, (F̂t , Ût , Ŵt )).
QT
Xt
S
×
R
of all sequences of states and tie-breaking utilt
t=0
Q
s
ities. Let Ω̂ := {(s0 , W0 , . . . , sT , WT ) ∈ G : tk=0 µkk−1 (sk ) > 0}. Let F̂ ∗ be the restricQ
tion to Ω̂ of the product sigma-algebra of the discrete sigma-algebra on Tt=0 St and the
Q
product Borel sigma-algebra on Tt=0 RXt . For each K = ({s0 }, K0 , ..., {sT }, KT ) ∈ F ∗ , let
Q
Q
s
µ̂(K) = Tt=0 µt t−1 (st )τst (Kt ); by finiteness of Tt=0 St , µ̂ extends to a finitely additive probaConsider the space G :=
bility measure on Ω̂ in the natural way.
Let Πt be the finite partition of Ω̂ whose cells are all the cylinders C(s0 , . . . , st ) := {ω̂ ∈
Ω̂ : projS0 ×...×St (ω̂) = (s0 , . . . , st )}. Let F̂t be the sigma-algebra generated by Πt ; by definition
S
of Ω̂, µ(F̂t (ω̂)) > 0 for all ω̂ ∈ Ω̂. Also, F̂t (ω̂) = ω̂0 ∈F̂t (ω̂) F̂t+1 (ω̂ 0 ), so (F̂t )0≤t≤T ⊆ F̂ ∗ is
a filtration. Define Ût : Ω̂ → RXt by Ût (ω̂) = Ust where projSt (ω̂) = st . Note that (Ût )
is adapted to (F̂t ) and that Ût (ω̂) is nonconstant for each ω̂ since each Ust is nonconstant.
Finally, if Ft−1 (ω̂) = Ft−1 (ω̂ 0 ) and Ft−1 (ω̂) 6= Ft−1 (ω̂ 0 ), then projSt−1 (ω̂) = projSt−1 (ω̂) = st−1
s
and projSt (ω̂) = st 6= s0t = projSt (ω̂ 0 ) for some st−1 ∈ St−1 and st , s0t ∈ supp µt t−1 . By DREU1
(a), this implies Ût (ω̂) := Ust 6≈ Us0t =: Ût (ω̂ 0 ). Thus, (Ft , Ut ) are simple.
Define Ŵt : Ω̂ → RXt by Ŵt (ω̂) = Wt where projRXt (ω̂) = Wt . Note that for all At , µ̂({ω̂ ∈
Q
P
sk−1
T
µ
(s
)
τst ({Wt ∈ RXt : |M (At , Wt )| = 1}) = 1,
Ω̂ : |M (At , Ŵt )| = 1}) = (s0 ,...,sT )
k
k=0 k
since each τst is regular. Thus, (Ŵt ) satisfies part (i) of the regularity requirement for DREU.
Moreover, for any F̂T (ω̂) = C(s0 , . . . , sT ) and any sequence (Bt ) of Borel sets Bt ⊆ RXt , the
definition of µ̂ implies
µ̂
T
\
!
{Ŵt ∈ Bt }|C(s0 , . . . , sT )
t=0
=
T
Y
T
Y
τst (Bt ) =
µ̂ {Ŵt ∈ Bt } | C(s0 , . . . , st ) .
t=0
(8)
t=0
Since F̂T (ω̂) = C(s0 , . . . , sT ) implies F̂t (ω̂) = C(s0 , . . . , st ) for all t ≤ T , this shows that (Ŵt )
also satisfies part (ii) of the regularity requirement.
Finally, to see that (Ω̂, F̂ ∗ , µ̂, (F̂t , Ût , Ŵt )) represents ρ, fix any ht = (A0 , p0 , ..., At , pt ) ∈ Ht .
41
Then
t
µ̂(C(h )) = µ̂
T
t
k=0 {ω̂
∈ Ω̂ : pk ∈ M (M (Ak , Ûk (ω̂)), Ŵk (ω̂))} =
T
t
µ̂(C(s
,
...,
s
))µ̂
{ω̂
∈
Ω̂
:
p
∈
M
(M
(A
,
Û
),
Ŵ
)}|C(s
,
...,
s
)
=
0
t
k
k
k
k
0
t
k=0
C(s0 ,...,st )∈Πt
Tt
Qt
P
sk−1
(sk )µ̂
k=0 {ω̂ ∈ Ω̂ : pk ∈ M (M (Ak , Usk ), Ŵk )}|C(s0 , ..., st ) =
k=0 µk
(s0 ,...,st )∈S0 ×...×St
Qt
P
sk−1
(sk )τsk (pk , Ak )
k=0 µk
(s0 ,...,st )∈S0 ×...×St
P
where the third equality follows from the definition of µ̂ and Û , and the final equality follows
from (8). Thus, as required, we have
P
Qt
sk−1
t
(sk )τsk (pk , Ak )
µ̂(C(h
))
(s0 ,...,st )
k=0 µk
t−1
µ̂(C(pt , At )|C(h )) =
=P
= ρt (pt ; At |ht−1 ),
Qt−1 sk−1
t−1
µ̂(C(h )
(sk )τsk (pk , Ak )
(s ,...,s
)
k=0 µk
0
t−1
where the final equality holds by DREU2.
“Only if ” direction: Take any DREU representation (Ω, F ∗ , µ, (Ft , Ut , Wt )) of ρ. We will
s
construct an S-based DREU representation (St , {µ̂t t−1 }st−1 ∈St−1 , {Ûst , τst }st ∈St )t=0,...,T .
For each t, let St := {Ft (ω) : ω ∈ Ω} denote the partition generating Ft , which is finite
t
since (Ft ) is simple. Each µ̂st+1
is defined to be the one-step-ahead conditional of µ, i.e.,
t
µ̂0 (s0 ) := µ(s0 ) for all s0 ∈ S0 and µ̂st+1
(st+1 ) := µ(st+1 |st ) for all st ∈ St , st+1 ∈ St+1 . This is
well-defined since µ(Ft (ω)) > 0 for all ω. For each st ∈ St , define Ûst := Ut (ω) if ω ∈ st ; this is
well-defined as (Ut ) is Ft -adapted and each Ust is nonconstant since each Ut (ω) is nonconstant.
Finally, for any Borel set Bt ⊆ RXt , define τst (Bt ) := µ({Wt ∈ Bt }|st ). This is well-defined
since Wt is F ∗ -measurable. Moreover, because µ({ω ∈ Ω : |M (At , Wt (ω)| = 1} = 1 for all At
and |St | is finite, it follows that τst (N (At , pt )) = τst (N + (At , pt )) for all pt , i.e., τst is regular.
s
Thus, each (St , µt t−1 , {Ust , τst }st ∈St ) is an REU form on Xt .
s
Moreover, (a) for any distinct st , s0t ∈ supp(µt t−1 ), we have ω, ω 0 such that Ft−1 (ω) = st−1 =
Ft−1 (ω 0 ) and Ft (ω) = st 6= Ft (ω 0 ) = s0t . Thus, Ûst = Ut (ω) 6≈ Ut (ω 0 ) = Ûs0t , since (Ut , Ft ) is
simple. Also, since (Ft ) is adapted, the partition St refines the partition St−1 , so that (b) for
s0
s
any distinct st−1 , s0t−1 , we have supp(µ̂t t−1 ) ∩ supp(µ̂t t−1 ) = ∅. Since additionally µ(st ) > 0 for
S
s
all st ∈ St , we have (c) st−1 ∈St−1 supp µ̂t t−1 = St . Thus, DREU1 is satisfied.
42
To see that DREU2 holds, observe that for each ht = (A0 , p0 , ..., At , pt ) ∈ Ht , we have
P
µ(C(ht ) = sT ∈ST µ(sT )µ (C(ht )|sT )
P
T
= sT ∈ST µ(sT )µ tk=0 {ω ∈ Ω : pk ∈ M (M (Ak , Uk ), Wk )}|sT
P
T
=
µ(sT )µ tk=0 {pk ∈ M (M (Ak , Usk ), Wk )}|st
(s0 ,...,sT )
∃ω∈Ω∀t: st =Ft (ω)
=
P
(s0 ,...,sT )
∃ω∈Ω∀t: st =Ft (ω)
=
P
=
P
µ(sT )
Qt
k=0
(s0 ,...,st )
∃ω∈Ω∀k≤t: sk =Fk (ω)
(s0 ,...,st )∈S0 ×...×St
µ ({pk ∈ M (M (Ak , Usk ), Wk )}|sk )
s
Qt
k=0
µkk−1 (sk )
s
Qt
k=0
µkk−1 (sk )
Qt
k=0 τsk (pk , Ak )
Qt
k=0 τsk (pk , Ak ),
where the third equality follows from the fact that (Ut ) is Ft -adapted, the fourth equality follows
from part (ii) of the regularity assumption on (Wt ), the final equality follows from the fact that
Qt
sk−1
(sk ) = 0 whenever (s0 , . . . , st ) 6= (F0 (ω), . . . , Ft (ω)) for all ω, and the remaining
k=0 µk
equalities hold by definition. Since ρt (pt ; At |ht−1 ) =
µ(C(ht ))
µ(C(ht−1 )
by (1), this shows that DREU2
holds.
A.2.2
Evolving Utility
“If ”
direction:
Suppose
ρ
admits
an
s
S-based
evolving
utility
representation
Let (Ω̂, F̂ ∗ , µ̂, (F̂t , Ût , Ŵt )) denote the corre-
(St , {µt t−1 }st−1 ∈St−1 , {Ust , ust , τst }st ∈St )t=0,...,T .
sponding DREU representation of ρ obtained in the “if” direction for DREU. In addition, define
ût : Ω̂ → RZ for each t by ût (ω̂) := ust whenever projSt (ω̂) = st . Note that the process (ût ) is
F̂t -adapted. Moreover, for each ω̂ = (s0 , W0 , ..., sT , WT ), we have ÛT (ω̂) = UsT = usT = ûT (ω̂)
and for each t ≤ T − 1 and (zt , At+1 )
Ût (ω̂)(zt , At+1 ) = Ust (zt , At+1 )
X
= ust (zt ) +
t
µst+1
(st+1 ) max Ust+1 (pt+1 )
pt+1 ∈At+1
st+1 ∈St+1
= ût (ω̂)(zt ) +
X
µ̂(st+1 |st ) max Ust+1 (pt+1 )
st+1 ∈St+1
pt+1 ∈At+1
= ût (ω̂)(zt ) + E[ max Ût+1 (pt+1 )|F̂t (ω̂)],
pt+1 ∈At+1
where we let µ̂(st+1 |st ) := µ̂ (C(s0 , . . . , st+1 ) | C(s0 , . . . , st )).
“Only
if ”
direction:
Suppose ρ admits an evolving utility representation
43
s
(Ω, F ∗ , µ, (Ft , Ut , ut , Wt )). Let (St , {µ̂t t−1 }st−1 ∈St−1 , {Ûst , τst }st ∈St )t=0,...,T denote the corresponding S-based DREU representation obtained in the “only if” direction for DREU. In addition, for
each st , define ûst ∈ RZ by ûst = ut (ω) for any ω ∈ st ; this is well-defined as (ut ) is Ft -adapted.
Reversing the argument in the previous part, we can verify that ûsT = ÛsT for each sT and
P
t
(st+1 ) maxpt+1 ∈At+1 Ûst+1 (pt+1 ) for each st with t ≤ T − 1.
Ûst (zt , At+1 ) = ûst (zt ) + st+1 µ̂st+1
A.2.3
Gradual learning
“If ”
direction:
Suppose ρ admits an S-based gradual learning representation
s
(St , {µt t−1 }st−1 ∈St−1 , {Ust , ust , τst }st ∈St , δ)t=0,...,T . Let (Ω̂, F̂ ∗ , µ̂, (F̂t , Ût , ût , Ŵt )) denote the corresponding evolving utility representation obtained in the “if” direction for evolving utility. In
addition, define δ̂ := δ. Note that for each ω̂ = (s0 , W0 , .., sT , WT ) and t ≤ T − 1, we have
P
t
ût (ω̂) = ust = 1δ st+1 µst+1
(st+1 )ust+1 = 1δ̂ E[ût+1 |F̂t (ω̂)]. Iterating expectations, this yields
ût (ω̂) = δ̂ t−T E[ûT |F̂t (ω̂)] = δ̂ t−T E[ÛT |F̂t (ω̂)]. Replace Ût with Ût0 := δ̂ T −t Ût for each t. By
Proposition 1, (Ω̂, F̂ ∗ , µ̂, (F̂t , Ût0 , Ŵt )) is still a DREU representation of ρ. Moreover, for each
t ≤ T − 1, we have
Ût0 (ω̂)(zt , At+1 ) = δ̂ T −t ût (ω̂)(zt ) + δ̂ T −t E[ max Ût+1 (pt+1 )|F̂t (ω̂)]
pt+1 ∈At+1
0
= E[ÛT0 (zt ) | F̂t (ω̂)] + δ̂E[ max Ût+1
(pt+1 )|F̂t (ω̂)].
pt+1 ∈At+1
Thus, (Ω̂, F̂ ∗ , µ̂, (F̂t , Ût0 , Ŵt ), δ̂) is a gradual learning representation of ρ.
“Only if ” direction:
Suppose that ρ admits a gradual learning representation
(Ω, µ, (Ft , Ut , Wt ), δ). Let Ut0 := δ t−T Ut for all t. By Proposition 1, (Ω, µ, (Ft , Ut0 , Wt )) is still a
DREU representation of ρ. Moreover, let u0t := δ t−T ut , where ut (ω) := E[UT |Ft (ω)]. By (3),
for each ω, we have UT0 (ω) = UT (ω) = uT (ω) = u0t (ω) and for all t ≤ T − 1
Ut0 (ω)(zt , At+1 ) = u0t (ω)(zt ) + δ t−T δE[ max Ut+1 (pt+1 ) | Ft (ω)]
pt+1 ∈At+1
0
= u0t (ω)(zt ) + E[ max Ut+1
(pt+1 ) | Ft (ω)].
pt+1 ∈At+1
(Ω, µ, (Ft , Ut0 , u0t , Wt ))
Thus,
s
is
an
evolving
(St , {µ̂t t−1 }st−1 ∈St−1 , {Ûs0 t , û0st , τst }st ∈St )t=0,...,T
utility
representation
of
ρ.
Let
denote the corresponding S-based evolving
utility representation of ρ obtained in the “only if” direction for evolving utility. In addition,
44
define δ̂ := δ. Then for each t ≤ T − 1 and ω with Ft (ω) = st , we have
1 X st
1
û0st = u0t (ω) = δ̂ t−T E[UT |Ft (ω)] = E[u0t+1 |Ft (ω)] =
µ̂t+1 (st+1 )û0st .
δ̂
δ̂ st+1
s
Thus (St , {µ̂t t−1 }st−1 ∈St−1 , {Ûs0 t , û0st , τst }st ∈St , δ̂)t=0,...,T is an S-based gradual learning representation of ρ.
A.3
Relationship between Histories and States
Throughout the proofs of Theorems 1–3 we will make use of the following terminology concerning the relationship between histories and states. Fix any t ∈ {0, . . . , T }. Suppose that
s
0
(St0 , {µt0t −1 }st0 −1 ∈St0 −1 , {Ust0 , τst0 }st0 ∈St0 ) satisfy DREU1 and DREU2 from Definition 10 for each
t0 ≤ t.
Fix any state s∗t ∈ St .
We let pred(s∗t ) denote the unique predecessor sequence
(s∗0 , . . . , s∗t−1 ) ∈ S0 × . . . × St−1 , given by assumptions DREU1 (b) and (c), such that
s∗
k
s∗k+1 ∈ supp(µk+1
) for each k = 0, ..., t − 1. Given any history ht = (A0 , p0 , . . . , At , pt ), we
Q
say that s∗t is consistent with ht if tk=0 τs∗k (pk , Ak ) > 0.
For any k = 0, . . . , t, sk ∈ Sk , p0 ∈ A0 ∈ A0 , and pk+1 ∈ Ak+1 ∈ Ak+1 , let
k
Usk (Ak+1 , pk+1 ) := {Usk+1 : sk+1 ∈ supp µsk+1
and pk+1 ∈ M (Ak+1 , Usk+1 )};
U0 (A0 , p0 ) := {Us0 : s0 ∈ S0 and p0 ∈ M (A0 , Us0 )}.
A separating history for s∗t is a history ht ∈ Ht∗ such that Us∗k−1 (Bk , qk ) = {Us∗k } for all
k = 0, . . . , t, where we abuse notation by letting Us∗−1 (B0 , q0 ) denote U0 (B0 , q0 ). Note that
separating histories are required to be histories without ties.
We record the following properties:
Lemma 1. Fix any s∗t ∈ St with pred(s∗t ) = (s∗0 , . . . , s∗t−1 ). Suppose ht = (B0 , q0 , . . . , Bt , qt )
satisfies Us∗k−1 (Bk , qk ) = {Us∗k } for all k = 0, . . . , t. Then for all k = 0, . . . , t, s∗k is the only state
in Sk that is consistent with hk .
Proof. Note first that Us∗k−1 (Bk , qk ) = {Us∗k } implies that τs∗k−1 (qk , Bk ) = 1 for all k = 0, . . . , t,
so s∗k is consistent with hk . Next, consider any ` = 0, . . . , t and s0` ∈ S` r {s∗` }, with pred(s0` ) =
s∗
(s00 , . . . , s0`−1 ). Let k ≤ ` be smallest such that s0k 6= s∗k . Then s0k ∈ supp µkk−1 , so Us∗k−1 (Bk , qk ) =
45
{Us∗k } implies that qk ∈
/ M (Bk , Us0k ). Thus, τs0k (qk , Bk ) = 0, whence s0` is not consistent with
h` .
Lemma 2. Every s∗t ∈ St admits a separating history.
Proof. Fix any s∗t ∈ St with pred(s∗t ) = (s∗0 , . . . , s∗t−1 ). By Lemma 12 and DREU1 (a), there exs
ist menus B0 = {q0 (s0 ) : s0 ∈ S0 } ∈ A0 and Bk (sk−1 ) = {pk (sk ) : sk ∈ supp µkk−1 } ∈ Ak
for each k = 1, . . . , t and sk ∈ Sk such that U0 (B0 , q0 (s0 )) = {Us0 } for all s0 ∈ S0 and
s
Usk−1 (Bk (sk−1 ), qk (sk )) = {Usk } for all sk ∈ supp µkk−1 .
Moreover, we can assume that
Bk+1 (sk ) ∈ supp qk (sk )A for all k = 0, . . . , t − 1 and sk ∈ Sk , by letting each qk (sk ) put small
enough weight on (z, Bk+1 (sk )) for some z ∈ Z. Then ht := (B0 , q0 (s∗0 ), . . . , Bt (s∗t ), qt (s∗ (t))) ∈
Ht . Moreover, since Us∗k−1 (Bk , qk (s∗k )) = {Us∗k }, Lemma 1 implies that or all for all k = 0, . . . , t,
s∗
s∗k is the only state consistent with hk . Additionally, for all k = 0, . . . , t and sk ∈ supp µkk−1 ,
we have M (Bk (s∗k−1 ), Usk ) = {qk (sk )} by construction.
Hence, by Lemma 13, we have
Bk (s∗k−1 ) ∈ A∗k (hk−1 ). Thus ht ∈ Ht∗ , so ht is a separating history for s∗t .
B
Proof of Theorem 1
B.1
Proof of Theorem 1: Sufficiency
Suppose ρ satisfies Axioms 1–4. To show that ρ admits a DREU representation, it suffices, by
Proposition 6, to construct an S-based DREU representation for ρ.
We proceed by induction on t ≤ T . In Appendix F, we prove Theorem 4 which extends the
characterization of static random expected utility by Gul and Pesendorfer (2006) and Ahn and
Sarver (2013) to an infinite outcome setting. For t = 0, the existence of (S0 , µ0 , {Us0 , τs0 }s0 ∈S0 )
satisfying DREU1 and DREU2 from Definition 10 is then immediate from Theorem 4 since ρ0
satisfies Axiom 3.
Suppose
next
that
0
≤
t
<
T
and
that
we
have
constructed
s0
(St0 , {µt0t −1 }st0 −1 ∈St0 −1 , {Ust0 , τst0 }st0 ∈St0 ) satisfying DREU1 and DREU2 for each t0 ≤
t
We now construct (St+1 , {µst+1
}st ∈St , {Ust+1 , τst+1 }st+1 ∈St+1 ) satisfying DREU1 and DREU2.
46
t.
B.1.1
s
t
}st ∈St , {Ust+1 , τst+1 }st+1 ∈St+1 ):
Defining ρt t+1 and (St+1 , {µst+1
To this end, we first pick an arbitrary separating history ht (st ) for each st ∈ St (this exists by
Lemma 2) and define
t
(·, At+1 ) := ρt+1 (·, At+1 |ht (st ))
ρst+1
for all At+1 ∈ At+1 . Note that here ρt+1 (·, |ht (st )) is the extended version of ρt+1 (·|ht (st )) given
in Definition 4; by Axiom 2 and Lemma 14, the specific choice of λ ∈ (0, 1] and dt−1 ∈ Dt−1
used in the extension procedure does not matter.
t
,
By Axiom 3 and Theorem 4 applied to ρst+1
there exists an REU form
st
t
st ) on X
, {Ust+1 , τst+1 }st+1 ∈St+1
, µst+1
(St+1
t+1 such that Ust+1 6≈ Ust+1 for any distinct pair
st
and such that
st+1 , s0t+1 ∈ St+1
X
t
ρst+1
(pt+1 , At+1 ) =
t
µst+1
(st+1 )τst+1 (pt+1 , At+1 )
s
t
st+1 ∈St+1
s0
st
t
for all pt+1 and At+1 . Without loss, we can assume that St+1
and St+1
are disjoint whenever
S
st
t
st 6= s0t . Set St+1 := st ∈St St+1
and extend µst+1
to a probability measure on St+1 by setting
st
t
µst+1
(st+1 ) = 0 for all st+1 ∈ St+1 r St+1
.
t
By construction, it is immediate that (St+1 , {µst+1
}st ∈St , {Ust+1 , τst+1 }st+1 ∈St+1 ) thus defined
satisfies DREU1 and that
t
ρst+1
(pt+1 , At+1 ) =
X
t
µst+1
(st+1 )τst+1 (pt+1 , At+1 )
(9)
st+1 ∈St+1
for all pt+1 and At+1 . It remains to show that DREU2 is also satisfied.
B.1.2
s
ρt t+1 is well-behaved:
t
To this end, Lemma 3 below first shows that the definition of ρst+1
is well-behaved, in the sense
t
that for any history ht that can only arise in state st , ρst+1
= ρt+1 (·|ht ).
Lemma 3. Fix any s∗t ∈ St with pred(s∗t ) = (s∗0 , ..., s∗t−1 ). Suppose ht = (A0 , p0 , ..., At , pt ) ∈
Ht satisfies Us∗k−1 (Ak , pk ) = {Us∗k } for all k = 0, 1, . . . , t.
ρt+1 (·, At+1 |ht ) =
Then for any At+1 ∈ At+1 ,
s∗t
ρt+1
(·, At+1 ).
Proof. Step 1: Let h̃t = (Ã0 , p̃0 , . . . , Ãt , p̃t ) denote the separating history for s∗t used to define
47
s∗
t
. We first prove the Lemma under the assumption that ht ∈ Ht∗ , i.e, that ht is itself a
ρt+1
separating history for s∗t .
Pick (r0 , ..., rt ) ∈ ∆(X0 ) × . . . × ∆(Xt ) such that At+1 ∈ supp rtA and for all k = 0, . . . , t − 1,
supp(rkA ) ⊇ {Bk+1 , B̃k+1 , Bk+1 ∪ B̃k+1 },
where B` :=
1
A
3 `
+ 13 {p̃` } + 13 {r` } and B̃` :=
1
Ã
3 `
+ 13 {p` } + 13 {r` } for ` = 0, . . . , t. Define
q` := 13 p` + 31 p̃` + 13 r` .
Note that since ht , h̃t ∈ Ht∗ and Us∗k−1 (Ak , pk ) = Us∗k−1 (Ãk , p̃k ) = {Us∗k }, Lemma 13 implies
that M (Ak , Us∗k ) = {pk } and M (Ãk , Us∗k ) = {p̃k } for all k = 0, 1, . . . , t. By linearity of the Us ,
we then also have
Us∗k−1 (Bk , qk ) = Us∗k−1 (B̃k , qk ) = Us∗k−1 (Bk ∪ B̃k , qk ) = {Us∗k } and
M (Bk , Us∗k ) = M (B̃k , Us∗k ) = M (Bk ∪ B̃k , Us∗k ) = {qk }.
s∗
k−1
This implies that for all k = 0, . . . , t and sk ∈ supp µk−1
,
τsk (qk , Bk ) = τsk (qk , B̃k ) = τsk (qk , Bk ∪ B̃k ) =


1 if sk = s∗
k

0 otherwise
By DREU2 of the inductive hypothesis, it follows that for all k = 0, . . . , t − 1,
s∗
µt t−1 (s∗t ) = ρt (qt , Bt |B0 , q0 , . . . , Bt−1 , qt−1 ) = ρt (qt , B̃t |B̃0 , q0 , . . . , B̃t−1 , qt−1 )
= ρt (qt , Bt ∪ B̃t |B0 , q0 , . . . , Bk−1 , qk−1 , Bk ∪ B̃k , qk , . . . , Bt−1 ∪ B̃t−1 , qt−1 )
= ρt (qt , Bt ∪ B̃t |B̃0 , q0 , . . . , B̃k−1 , qk−1 , Bk ∪ B̃k , qk , . . . , Bt−1 ∪ B̃t−1 , qt−1 ),
whence repeated application of Axiom 1 (Contraction History Independence) yields
ρt+1 (·, At+1 |B0 , q0 , . . . , Bt , qt ) = ρt+1 (·, At+1 |B0 ∪ B̃0 , q0 , . . . , Bt ∪ B̃t , qt ) =
ρt+1 (·, At+1 |B̃0 , q0 , . . . , B̃t , qt ).
48
(10)
Moreover, by Axiom 2 (Linear History Independence) and Lemma 14, we have
ρt+1 (·, At+1 |ht ) = ρt+1 (·, At+1 |B0 , q0 , . . . , Bt , qt ) and
(11)
ρt+1 (·, At+1 |h̃t ) = ρt+1 (·, At+1 |B̃0 , q0 , . . . , B̃t , qt ).
s∗
t
(·, At+1 ). This
Combining (10) and (11) we obtain that ρt+1 (·, At+1 |ht ) = ρt+1 (·, At+1 |h̃t ) := ρt+1
proves the Lemma for histories ht ∈ Ht∗ .
Step 2: We now consider general histories ht ∈ Ht . Take any sequence of histories ht,n →m
ht with ht,n = (An0 , pn0 , ..., Ant , pnt ) ∈ Ht∗ for each n. Note that such a sequence exists by Axiom 4
(History Continuity).
We claim that for all large enough n, Us∗k−1 (Ank , pnk ) = {Us∗k } for all k = 0, . . . , t. Suppose
for a contradiction that we can find a subsequence (ht,n` )∞
`=1 for which this claim is violated.
Note that for all `, ρk (pnk ` , Ank ` |hk−1,n` ) > 0 for all k = 0, . . . , t (by the fact that ht,n` is a welldefined history). Hence, DREU2 for k ≤ t implies that we can find s0t,n` ∈ St with pred(s0t,n` ) =
(s00,n` , . . . , s0t−1,n` ) and (s00,n` , . . . , s0t,n` ) 6= (s∗0 , . . . , s∗t ) such that Us0k,n ∈ Us0k−1,n (Ank ` , pnk ` ) for
`
`
all k = 0, . . . , t. Moreover, since S0 × . . . × St is finite, by choosing the subsequence (ht,n` )
appropriately, we can assume that (s00,n` , . . . , s0t,n` ) = (s00 , . . . , s0t ) 6= (s∗0 , . . . , s∗t ) for all `. Pick
the smallest k such that s0k 6= s∗k and pick any qk ∈ Ak . Since Ank ` →m Ak we can find qkn` ∈ Ank `
with qkn` →m qk . For all ` we have Us0k ∈ Us0k−1 (Ank ` , pnk ` ), so Us0k (pnk ` ) ≥ Us0k (qkn` ), whence
s0
s∗
k−1
k−1
Us0k (pk ) ≥ Us0k (qk ) by linearity of Us0k . Moreover, by choice of k, s0k ∈ supp µk−1
= supp µk−1
.
Thus, Us0k ∈ Us∗k−1 (Ak , pk ) = {Us∗k }. But s0k 6= s∗k , so by DREU1 (a) of the inductive hypothesis
Us0k 6≈ Us∗k , a contradiction.
By the previous paragraph, for large enough n, ht,n satisfies the assumption of the Lemma.
s∗
t
Since ht,n ∈ Ht∗ , Step 1 then shows that ρt+1 (pt+1 , At+1 |ht,n ) = ρt+1
(pt+1 , At+1 ) for all large
enough n and all pt+1 . By Axiom 4 (History Continuity), this implies that for all pt+1
ρt+1 (pt+1 , At+1 |ht ) ∈ co{lim ρt+1 (pt+1 , At+1 |ht,n ) : ht,n →m ht , ht,n ∈ Ht∗ }
=
n
s∗t
{ρt+1 (pt+1 , At+1 )},
which completes the proof.
49
B.1.3
s
ρt+1 (·|ht ) is a weighted average of ρt t+1 :
The next lemma shows that ρt+1 (·|ht ) can be expressed as a weighted average of the statet
t
corresponds to the proba, where the weight on each ρst+1
dependent choice distributions ρst+1
bility of st conditional on history ht .
Lemma 4. For any pt+1 ∈ At+1 and ht = (A0 , p0 , ..., At , pt ) ∈ Ht (At+1 ), we have
P
t
ρt+1 (pt+1 , At+1 |h ) =
s
Qt
t
(pt+1 , At+1 )
µkk−1 (sk )τsk (Ak , pk )ρst+1
.
P
Qt
sk−1
(sk )τsk (Ak , pk )
(s0 ,...,st )∈S0 ×···×St
k=0 µk
(s0 ,...,st )∈S0 ×···×St
k=0
t
Proof. Let {s1t , ..., sm
t } denote the set of states in St that are consistent with history h (as
defined in Section A.3). For each j, let ĥt (j) = (B0j , q0j , . . . , Btj , qtj ) be a separating history for
j
state sjt . We can assume that for each k = 1, . . . , t, qk−1
puts small weight on (z, 21 Ak + 12 Bkj )
for some z, so that ht (j) := 12 ht + 21 ĥt (j) ∈ Ht (At+1 ) for all j.
Note first that for all j = 1, . . . , m, we have
t
ρ(h (j)) =
t
Y
sj
µkk−1 (sjk )τsj (pk , Ak ).
(12)
k
k=0
Indeed, observe that
t
ρ(h (j)) =
=
=
1
1
1
1
1
1
ρk ( pk + qkj , Ak + Bkj | hk−1 + ĥk−1 (j))
2
2
2
2
2
2
k=0
t
X Y
(s0 ,...,st )
=
t
Y
1
1
1
1
s
µkk−1 (sk )τsk ( pk + qk , Ak + Bkj )
2
2
2
2
k=0
t
Y
1
1
1
1
sj
µkk−1 (sjk )τsj ( pk + qkj , Ak + Bkj )
k 2
2
2
2
k=0
t
Y
sj
µkk−1 (sjk )τsj (pk , Ak ).
k
k=0
The first equality holds by definition. The second equality follows from DREU2 of the inductive
hypothesis. For the final two equalities, note that since ĥt (j) is a separating history for sjt , we
have for all k = 0, . . . , t that Usj (Bkj , qkj ) = {Usj } with {qkj } = M (Bkj , Usj ) (by Lemma 13).
k−1
Also, since
sjt
k
t
k
is consistent with h , τsj (pk , Ak ) > 0 for all k = 0, . . . , t. This implies that
k
50
sj
for every sk ∈ supp µkk−1 , τsk ( 12 pk + 12 qkj , 21 Ak + 12 Bk ) > 0 if and only if sk = sjk , yielding
the third equality. It also implies that M ( 12 Ak + 12 Bkj , Usj ) = M ( 12 Ak + 21 {qkj }, Usj ), so that
k
τsj ( 12 pk
k
+
1 j 1
q , A
2 k 2 k
+
1 j
B )
2 k
=
τsj ( 12 pk
k
+
1 j 1
q , A
2 k 2 k
1
{qkj })
2
+
k
= τsj (pk , Ak ), yielding the fourth
k
equality.
Now let H t := {ht (j) : j = 1, . . . , m} ⊆ Ht (At+1 ). Note that by repeated application of
Axiom 2, we have that
ρt+1 (pt+1 , , At+1 |ht ) = ρt+1 (pt+1 , At+1 |H t ).
(13)
Moreover, we have that
Pm
t
ρt+1 (pt+1 , At+1 |H ) =
Pm Qt
j=1
ρ(ht (j))ρt+1 (pt+1 , At+1 |ht (j))
Pm
t
j=1 ρ(h (j))
sj
µkk−1 (sjk )τsj (pk , Ak )ρt+1 (pt+1 , At+1 |ht (j))
k
=
Pm Q t
sjk−1 j
µ
(sk )τsj (pk , Ak )
j=1
k=0 k
k
P Qt
sjk−1 j
sjt
(sk )τsj (pk , Ak )ρt+1
(pt+1 |At+1 )
j
k=0 µk
k
=
P Qt
sjk−1 j
µ
(sk )τsj (pk , Ak )
j
k=0 k
k
P
Qt
sk−1
t
(sk )τsk (Ak , pk )ρst+1
(pt+1 |At+1 )
k=0 µk
(s0 ,...,st )∈S0 ×···×St
.
=
P
Qt
sk−1
(sk )τsk (Ak , pk )
(s0 ,...,st )∈S0 ×···×St
k=0 µk
j=1
k=0
(14)
Indeed, the first equality holds by definition of choice conditional on a set of histories. The
second equality follows from Equation (12). Note next that since ĥt (j) is a separating history
for sjt and sjt is consistent with ht , we have that Usj ( 21 pk + 21 qkj , 21 Ak + 12 Bkj ) = {Usj } for each
k
t
k. Hence, Lemma 3 implies that ρt+1 (pt+1 , At+1 |h (j)) =
sjt
ρt+1
(pt+1 , At+1 ),
k
yielding the third
equality. Finally, note that if (s0 , . . . , st ) ∈ S0 × . . . St with (s0 , . . . , st ) 6= (sj0 , . . . , sjt ) for all j,
j
t
then either st ∈
/ {s1t , . . . , sm
t }, or st = sj for some j but (s0 , . . . , st−1 ) 6= pred(st ). In either case,
Qt
sk−1
(sk )τsk (Ak , pk ) = 0, yielding the final equality. Combining (13) and (14), we obtain
k=0 µk
the desired conclusion.
51
B.1.4
Completing the proof:
t
in (9) yields that for any ht =
Finally, combining Lemma 4 with the representation of ρst+1
(A0 , p0 , ..., At , pt ) ∈ Ht (At+1 )
ρt+1 (pt+1 , At+1 |ht )
P
(s0 ,...,st )∈S0 ×···×St
=
=
P
s
t
(st+1 )τst+1 (pt+1 , At+1 )
µkk−1 (sk )τsk (Ak , pk ) st+1 ∈St+1 µst+1
Qt
P
sk−1
(sk )τsk (Ak , pk )
k=0 µk
(s0 ,...,st )∈S0 ×···×St
Qt+1 sk−1
P
(sk )τsk (Ak , pk )
k=0 µk
(s0 ,...,st ,st+1 )∈S0 ×···×St ×St+1
.
Qt
P
sk−1
(sk )τsk (Ak , pk )
k=0 µk
(s0 ,...,st )∈S0 ×···×St
Qt
k=0
t
}st ∈St , {Ust+1 , τst+1 }st+1 ∈St+1 ) also satisfies requirement DREU2, completing
Thus, (St+1 , {µst+1
the proof.
B.2
Proof of Theorem 1: Necessity
Suppose ρ admits a DREU representation. By Proposition 6, ρ admits an S-based DREU
representation. By Lemma 15, for each t and ht ∈ Ht , the (static) stochastic choice rule
ρt (·|ht ) : At → ∆(∆(Xt )) given by the extended version of ρ from Definition 4 also satisfies
DREU2. In other words, ρt (·|ht ) admits an REU form representation (see Definition 11). Thus,
Theorem 4 implies that Axiom 3 holds. It remains to verify that Axioms 1, 2, and 4 are
satisfied.
Claim 1. ρ satisfies Axiom 1 (Contraction History Independence).
Proof. Take any ht−1 = (h−k , (Ak , pk )), ĥt−1 = (h−k , (Bk , pk )) ∈ Ht−1 (At ) such that Bk ⊇ Ak
and ρk (pk ; Ak |hk−1 ) = ρk (pk ; Bk |hk−1 ). From DREU2 for ρk , ρk (pk ; Ak |hk−1 ) = ρk (pk ; Bk |hk−1 )
implies that
X k−1
Y
s
s
µl l−1 (sl )τsl (pl , Al )µkk−1 (sk )τsk (pk , Ak )
=
(s0 ,...,sk ) l=0
X k−1
Y
s
s
µl l−1 (sl )τsl (pl , Al )µkk−1 (sk )τsk (pk , Bk ).
(s0 ,...,sk ) l=0
(15)
Since Bk ⊇ Ak implies τsk (pk , Ak ) ≥ τsk (pk , Bk ) for all sk , the only way for (15) to hold is if
52
τsk (pk , Ak ) = τsk (pk , Bk ) for all sk . Thus,
P
t−1
ρt (pt ; At |h
)= P
(s0 ,...,st )∈S0 ×...×St
sl−1
(sl )τsl (pl , Al )
l=0 µl
Qt−1 sl−1
(sl )τsl (pl , Al )
l=0 µl
Qt
(s0 ,...,st−1 )∈S0 ×...×St−1
= ρt (·; At |ĥt−1 ),
as required.
Claim 2. ρ satisfies Axiom 2 (Linear History Independence).
Proof. Take any At , ht−1 = (A0 , p0 , . . . , At−1 , pt−1 ) ∈ Ht−1 (At ), and H t−1 ⊆ Ht−1 (At ) of the
t−1
form H t−1 = {(h−k
, (λAk + (1 − λ)Bk , λpk + (1 − λ)qk )) : qk ∈ Bk } for some k < t, λ ∈ (0, 1),
and Bk = {qkj : j = 1, . . . , m} ∈ Ak . Let Ãk := λAk + (1 − λ)Bk , and for each j = 1, . . . , m, let
j
p̃jk := λpk + (1 − λ)qkj and h̃t−1 (j) := (ht−1
−k , (Ãk , p̃k )).
By DREU2, for all pt , we have
s`−1
(s` )τs` (p` , A` )
`=0 µ`
.
P
Qt−1 s`−1
(s` )τs` (p` , A` )
(s0 ,...,st−1 )
`=0 µ`
P
t−1
ρt (pt ; At |h
)=
(s0 ,...,st )
Qt
(16)
Moreover, by definition
Pm
ρt (pt ; At |H
t−1
)=
j=1
ρ(h̃t−1 (j))ρt (pt ; At |h̃t−1 (j))
,
Pm
t−1 (j))
j=1 ρ(h̃
where for each j = 1, . . . , m, DREU2 yields
Q
s`−1
s
µ
(s
)τ
(p
,
A
)
µkk−1 (sk )τsk (p̃jk , Ãk )
` s` `
`
(s0 ,...,st )
`=0,...,t;`6=k `
t−1
Q
ρt (pt ; At |h̃ (j)) = P
.
s`−1
sk−1
j
(s` )τs` (p` , A` ) µk (sk )τsk (p̃k , Ãk )
(s0 ,...,st−1 )
`=0,...,t−1;`6=k µ`
P
and
ρ(h̃t−1 (j)) :=
Y
ρ` (p` ; A` |h̃`−1 )ρk (p̃jk ; Ãk |h̃k−1 )
`=0,...,t−1;`6=k
!
=
X
Y
(s0 ,...,st−1 )
`=0,...,t−1;`6=k
s
µ` `−1 (s` )τs` (p` , A` )
53
s
µkk−1 (sk )τsk ((p̃jk , Ãk )).
Combining and rearranging, we obtain
P
s
s`−1
j
(s
)τ
(A
,
p
)
µkk−1 (sk ) m
µ
`
s
`
`
`
j=1 τsk (p̃k , Ãk )
(s0 ,...,st )
`=0,...,t;`6=k `
t−1
Q
.
ρt (pt ; At |H ) = P
Pm
sk−1
s`−1
j
(s
)
τ
(p̃
,
Ã
)
(s
)τ
(A
,
p
)
µ
µ
k
k
` s`
` `
k
j=1 sk k
(s0 ,...,st−1 )
`=0,...,t−1;`6=k `
Q
P
(17)
But observe that for all sk ,
m
X
τsk (p̃jk , Ãk )
=
j=1
X
=
m
X
τsk ({w ∈ RXk : p̃jk ∈ M (M (Ãk , Usk ), w)})
j=1
τsk ({w ∈ RXk : pk ∈ M (M (Ak , Usk ), w) and qk ∈ M (M (Bk , Usk ), w)})
qk ∈Bk
(18)
= τsk ({w ∈ RXk : pk ∈ M (M (Ak , Usk ), w)})
= τsk (pk , Ak ),
where the second equality follows from linearity of the representation, the third equality from
the fact that τsk is a regular finitely additive probability measure on RXk , and the remaining
equalities hold by definition. Combining (16), (17), and (18), we obtain ρt (pt ; At |ht−1 ) =
ρt (pt ; At |H t−1 ), as required.
Claim 3. ρ satisfies Axiom 4 (History Continuity).
Proof. Fix any At , pt ∈ At , and ht−1 = (A0 , p0 , ..., At−1 , pt−1 ) ∈ Ht−1 . Let St−1 (ht−1 ) ⊆ St−1
s
denote the set of period-(t − 1) states that are consistent with ht−1 . Define ρt t−1 (pt ; At ) :=
P st−1
(st )τst (pt , At ) for each st−1 . By Lemma 15,
s t µt
sk−1
(sk )τsk (pk , Ak )
k=0 µk
P
Qt−1 sk−1
(sk )τsk (pk , Ak )
(s0 ,...,st−1 )∈S0 ×···×St−1
k=0 µk
P
Qt−1 sk−1
P
s
(sk )τsk (pk , Ak ) st ∈St µt t−1 (st )τst (pt , At )
(s0 ,...,st−1 )∈S0 ×···×St−1
k=0 µk
.
P
Qt−1 sk−1
(sk )τsk (pk , Ak )
(s0 ,...,st−1 )∈S0 ×···×St−1
k=0 µk
P
t−1
ρt (pt ; At |h
) =
=
(s0 ,...,st )∈S0 ×···×St
Qt
s
Hence, ρt (pt ; At |ht−1 ) ∈ co{ρt t−1 (pt ; At ) : st−1 ∈ St−1 (ht−1 )}. Fix any s∗t−1 ∈ St−1 (ht−1 ). To
prove the claim, it is sufficient to show that
s∗
t−1
∗
∈ Ht−1
}.
ρt t−1 (pt ; At ) ∈ {lim ρt (pt ; At |ht−1
→m ht−1 , ht−1
n ) : hn
n
n
54
∗
To this end, let pred(s∗t−1 ) = (s∗0 , . . . , s∗t−2 ) and let h̄t−1 = (B0 , q0 , ..., Bt−1 , qt−1 ) ∈ Ht−1
be
a separating history for s∗t−1 . By Lemma 16, for each k = 0, . . . , t − 1, we can find sequences
Ank ∈ A∗k (h̄k−1 ) and pnk ∈ Ank such that Ank →m Ak , pnk →m pk and Us∗k−1 (Ank , pnk ) = {Us∗k } for
all k = 0, . . . , t − 1. Working backwards from k = t − 2, we can inductively replace Ank and pnk
for
with a mixture putting small weight on (z, Ank+1 ) for some z to ensure that Ank+1 ∈ supp pn,A
k
all k ≤ t − 2 while maintaining the properties in the previous sentence. Then by construction
∗
ht−1
:= (An0 , pn0 , . . . , Ant−1 , pnt−1 ) ∈ Ht−1
(At ) and ht−1
is a separating history for s∗t−1 . But then,
n
n
by Lemma 15, we have
P
ρt (pt ; At |hnt−1 ) =
=
st ∈St
Q
t−1
k=0
s∗
µkk−1 (s∗k )τs∗k (pk , Ak )
Qt−1
s∗
µt t−1 (st )τst (pt , At )
s∗
k−1
(s∗k )τs∗k (pk , Ak )
k=0 µk
X
s∗
s∗
µt t−1 (st )τst (pt , At ) =: ρt t−1 (pt ; At )
st
for each n. Since hnt−1 →m ht−1 , this verifies the desired claim.
C
C.1
Proof of Theorem 2
Implications of %ht
We begin with a preliminary lemma that characterizes the implications of the revealed dominance order %ht .
Lemma 5. Suppose that ρ admits an S-based DREU representation. Consider any t ≤ T ,
ht = (A0 , p0 , . . . , At , pt ) ∈ Ht , and qt , rt ∈ ∆(Xt ).
(i). If qt %ht rt , then Ust (qt ) ≥ Ust (rt ) for all st consistent with ht .
(ii). Suppose there exist g, b ∈ ∆(Xt ) such that Ust (g) > Ust (b) for all st consistent with ht .
If Ust (qt ) ≥ Ust (rt ) for all st consistent with ht , then qt %ht rt .
(iii). If ht is a separating history for st , then qt %ht rt if and only if Ust (qt ) ≥ Ust (rt ).
Proof. (i): We prove the contrapositive. Suppose there exists st consistent with ht such that
s
Ust (qt ) < Ust (rt ). Since st is consistent with ht , we have Πtk=0 µkk−1 (sk )τsk (pk , Ak ) > 0 for
pred(st ) = (s0 , . . . , st−1 ). Moreover, Ust (qt ) < Ust (rt ) implies that for any qtn →m qt and
55
rtn →m rt , we have Ust (qtn ) < Ust (rtn ) for large enough n. But then for all large enough n, we
have τst ( 12 pt + 12 rtn ; 21 At + 12 {qtn , rtn }) = τst (pt , At ) > 0, whence by Lemma 15, ρt ( 12 pt + 12 rtn ; 21 At +
1
{qtn , rtn }|ht−1 )
2
> 0. Thus, by definition, qt 6%ht rt .
(ii): Let St (ht ) denote the set of st consistent with ht . Suppose Ust (qt ) ≥ Ust (rt ) for all
st ∈ St (ht ). Then picking g, b ∈ ∆(Xt ) such that Ust (g) > Ust (b) for all st ∈ St (ht ) and
letting qtn :=
n
q
n+1 t
rt , and Ust (qtn ) >
1
g and rtn
n+1
Ust (rtn ) for all n
+
:=
n
r
n+1 t
+
1
b
n+1
t
for all n, we have qtn →m qt , rtn →m
and st ∈ St (h ). Consider any (s0 , . . . , st−1 , st ) ∈ S0 ×
. . . × St−1 × St . Then either st ∈ St (ht ), in which case τst ( 12 pt + 12 rtn ; 12 At + 12 {qtn , rtn }) = 0
s
s
t−1
for all n, so that Πk=0
µkk−1 (sk )τsk (pk , Ak )µt t−1 (st )τst ( 12 pt + 12 rtn ; 21 At + 12 {qtn , rtn }) = 0. Or
s
s
k−1
(sk )τsk (pk , Ak )µt t−1 (st )τst (pt , At ) = 0, in which case again
st ∈
/ St (ht ), in which case Πt−1
k=0 µk
s
s
k−1
Πt−1
(sk )τsk (pk , Ak )µt t−1 (st )τst ( 21 pt + 21 rtn ; 12 At + 12 {qtn , rtn }) = 0. By Lemma 15, this implies
k=0 µk
ρt ( 12 pt + 21 rtn ; 12 At + 12 {qtn , rtn }|ht−1 ) = 0 for all n, i.e., qt %ht rt .
(iii): Finally, suppose ht is a separating history for st . If qt %ht rt , then Ust (qt ) ≥ Ust (rt )
by part (i). For the converse, note that since Ust is non-constant, there exist g, b ∈ ∆(Xt ) such
that Ust (g) > Ust (b). Since st is the only state consistent with ht (recall Lemma 1), part (ii)
implies that if Ust (qt ) ≥ Ust (rt ) then qt %ht rt .
C.2
Proof of Theorem 2: Sufficiency
Throughout this section, we assume that ρ admits a DREU representation and satisfies Axioms 5–7. We will show that ρ admits an evolving utility representation. By Proposition 6,
it is sufficient to construct an S-based evolving utility representation. Sections C.2.1–C.2.5
accomplish this.
C.2.1
Recursive Construction up to t
The construction proceeds recursively. Suppose that t ≤ T − 1. Assume that we have obs
0
tained (St0 , {µt0t −1 }st0 −1 ∈St0 −1 , {Ust0 , τst0 }st0 ∈St0 ) for each t0 ≤ t such that DREU1 and DREU2
hold for all t0 ≤ t and EVU holds for all t0 ≤ t − 1 (see Definition 10 for the statements
of these conditions). Note that the base case t = 0 is true because of the fact that ρ admits a DREU representation and by Proposition 6. To complete the proof, we will construct
t
(St+1 , {µst+1
}st ∈St , {Ust+1 , τst+1 }st+1 ∈St+1 ) such that DREU1 and DREU2 hold for t0 ≤ t + 1 and
EVU holds for t0 ≤ t.
56
C.2.2
Properties of Ust
Using Lemma 5, the following lemma translates Axioms 5 and 6 into properties of Ust .
Lemma 6. For any st ∈ St , there exist functions ust : Z → R and Vst : At+1 → R such that
(i). Ust (zt , At+1 ) = ust (zt ) + Vst (At+1 ) holds for each (zt , At+1 )
(ii). Vst is continuous
(iii). Vst is monotone, i.e., Vst (A0t+1 ) ≥ Vst (At+1 ) for any At+1 ⊆ A0t+1
(iv). Vst is linear, i.e., Vst (αA1 + (1 − α)A01 ) = αVst (A1 ) + (1 − α)Vs (A01 ) for all At+1 , A0t+1 and
α ∈ (0, 1).
0
0
) > Vst (Ct+1 ) for all st ∈ St .
, Ct+1 ∈ At+1 such that Vst (Ct+1
Moreover, there exist Ct+1
Proof. Fix any st ∈ St and a separating history ht for st , the existence of which is guaranteed
by Lemma 2.
For (i), it suffices, by standard arguments, to show that
Ust (zt , At+1 ) + Ust (zt0 , A0t+1 ) = Ust (zt0 , At+1 ) + Ust (zt , A0t+1 )
for all zt , zt0 , and At+1 , A0t+1 . Fix any zt , zt0 , and At+1 , A0t+1 . By Axiom 5 (Separability), we
have
1
(z , At+1 )
2 t
1
U (z , At+1 )
2 st t
+ 12 (zt0 , A0t+1 ) ∼ht
+ 12 Ust (zt0 , A0t+1 ) =
1 0
(z , At+1 ) + 12 (zt , A0t+1 ), whence Lemma
2 t
1
U (z 0 , At+1 ) + 12 Ust (zt , A0t+1 ). This proves
2 st t
5 implies that
that there ex-
ist functions ust : Z → R and Vst : At+1 → R such that Ust (zt , At+1 ) = ust (zt ) + Vst (At+1 ) for
each (zt , At+1 ), as required.
For (ii), observe that Axiom 6 (iii) (Menu Continuity) and Theorem 4 imply that Ust is
continuous. This implies that Vst is continuous.
For (iii), suppose At+1 ⊆ A0t+1 and fix any zt . By Axiom 6 (i) (Monotonicity), we have
(zt , A0t+1 ) %ht (zt , At+1 ). By Lemma 5, this implies that Ust (zt , A0t+1 ) ≥ Ust (zt , At+1 ), whence
Vst (A0t+1 ) ≥ Vst (At+1 ) by (i).
For (iv), fix any At+1 , A0t+1 , zt , and α ∈ (0, 1). Axiom 6 (ii) (Indifference to Timing) implies (zt , αAt+1 + (1 − α)A0t+1 ) ∼ht α(zt , At+1 ) + (1 − α)(zt , A0t+1 ), which by Lemma 5 implies
Ust ((zt , αAt+1 +(1−α)A0t+1 )) = Ust (α(zt , At+1 )+(1−α)(zt , A0t+1 )). By linearity and separability
of Ust , this implies Vst (αAt+1 + (1 − α)A0t+1 ) = αVst (At+1 ) + (1 − α)Vst (A0t+1 ), as required.
57
Finally, for the “moreover” part, again consider any s∗t and separating history ht for s∗t .
By Axiom 6 (iv) (Non-degenerate Menu Preference), there exist A0t+1 (s∗t ), At+1 (s∗t ) and zt
such that (zt , A0t+1 (s∗t )) ht (zt , At+1 (s∗t )). Thus, Lemma 5 (iii) implies Us∗t (zt , A0t+1 (s∗t )) >
0
Us∗t (zt , At+1 (s∗t )), so Vs∗t (A0t+1 (s∗t )) > Vs∗t (At+1 (s∗t )) by part (i).
Now let Ct+1
:=
P
S
1
0
∗
0
∗
∗
s∗t ∈St |St | At+1 (st ). Then for all st and st , by
s∗t ∈St At+1 (st ) ∪ At+1 (st ) and let Ct+1 :=
0
) ≥ Vst (At+1 (s0t )), where by construction this inequality
monotonicity of Vst , we have Vst (Ct+1
0
is strict whenever st = s0t . By linearity of Vst , this implies Vst (Ct+1
) > Vst (Ct+1 ).
Corollary C.1. Fix any t ≤ T − 1 and ht ∈ Ht . Then qt %ht rt if and only if Ust (qt ) ≥ Ust (rt )
for all st consistent with ht .
Proof. The “only if” direction is a restatement of part (i) of Lemma 5. For the “if” direction, let
0
0
and Ct+1 be as in the “moreover” part of Lemma 6. Pick any zt ∈ Z and let gt+1 := δ(z,Ct+1
Ct+1
)
and bt+1 := δ(z,Ct+1 ) . Then by Lemma 6, Ust (gt+1 ) > Ust (bt+1 ) for all st . Hence, the “if” direction
is immediate from part (ii) of Lemma 5.
C.2.3
Construction of Random Utility in Period t + 1
Since ρ admits a DREU representation, it admits an S-based DREU representation by
t
Proposition 6, so in particular we can obtain (St+1 , {µst+1
}st ∈St , {Ũst+1 , τst+1 }st+1 ∈St+1 ) satist
t
fying DREU1 and DREU2 at t + 1. For any st ∈ St , define ρst+1
by ρst+1
(pt+1 , At+1 ) :=
P
st
st+1 µt+1 (st+1 )τst+1 (pt+1 , At+1 ) for all pt+1 , At+1 .
C.2.4
Sophistication and Finiteness of Menu Preference
Before completing the representation, we establish two more lemmas. Using Axiom 7 (Sophist
tication), the first lemma ensures that for each st , ρst+1
and the preference over At+1 induced
by Vst satisfy Axioms 1 and 2 in Ahn and Sarver (2013).
Lemma 7. For any st ∈ St , separating history ht for st , and At+1 ⊆ A0t+1 ∈ A∗t+1 (ht ), the
following are equivalent:
t
(i). ρst+1
(A0t+1 r At+1 ; A0t+1 ) > 0.
(ii). Vst (A0t+1 ) > Vst (At+1 )
58
Proof. Pick any separating history ht for st . Note that by DREU2 at t + 1 and Lemma 15,
t
(A0t+1 r At+1 ; A0t+1 ). Moreover, by Corollary C.1 and
we have ρt+1 (A0t+1 r At+1 ; A0t+1 |ht ) = ρst+1
Lemma 6 (i), Vst (A0t+1 ) > Vst (At+1 ) if and only if (zt , A0t+1 ) ht (zt , At+1 ) for all zt . By Axiom 7,
t
(A0t+1 rAt+1 ; A0t+1 ) > 0, as claimed. this implies that Vst (A0t+1 ) > Vst (At+1 ) if and only if ρst+1
The next lemma shows that because of Lemma 7, the finiteness of each St+1 (st ) is enough
to ensure that the preference over At+1 induced by each Vst satisfies Axiom DLR 6 (Finiteness)
introduced by Ahn and Sarver (2013):
Lemma 8. For each st ∈ St , there is Kst > 0 such that for any At+1 , there is Bt+1 ⊆ At+1 such
that |Bt+1 | ≤ Kst and Vst (At+1 ) = Vst (Bt+1 ).
t
Proof. Fix any st ∈ St and a separating history ht for st . Let St+1 (st ) := suppµst+1
. We will
show that Kst := |St+1 (st )| is as required.
Step 1: First consider any At+1 ∈ A∗t+1 (ht ). Then by Lemma 13, for each st+1 ∈ St+1 (st )
S
we have |M (At+1 , Ũst+1 )| = 1. Letting Bt+1 := st+1 ∈St+1 (st ) M (At+1 , Ũst+1 ), we then have that
t
|Bt+1 | ≤ Kst and ρst+1
(At+1 r Bt+1 |At+1 ) = 0. By Lemma 7, this implies that Vst (At+1 ) =
Vst (Bt+1 ), as required.
Step 2: Next take any At+1 6∈ A∗t+1 (ht ).
By Lemma 16, we can find a sequence
n
Ant+1 →m At+1 with Ant+1 ∈ A∗t+1 (ht ) for all n. Then by Step 1, we can find Bt+1
⊆ Ant+1
n
n
). By definition of →m , for each
| ≤ Kst and Vst (Ant+1 ) = Vst (Bt+1
for all n such that |Bt+1
n
⊆
qt+1 ∈ At+1 , there exists Dt+1 (qt+1 ) ∈ At+1 and a sequence αn (qt+1 ) → 0 such that Bt+1
S
n
qt+1 ∈At+1 αn (qt+1 )Dt+1 (qt+1 )+(1−αn (qt+1 )){qt+1 } for all n. Hence, since |Bt+1 | ≤ Kst for all n,
n
→m Bt+1 and such
restricting to a subsequence if necessary, there is Bt+1 ⊆ At+1 such that Bt+1
that |Bt+1 | ≤ Kst . Finally, by continuity of Vst (Lemma 6 (ii)), we have Vst (Bt+1 ) = Vst (At+1 ),
as required.
C.2.5
Completing the representation
t
Recall that in Section C.2.3, we have obtained (St+1 , {µst+1
}st ∈St , {Ũst+1 , τst+1 }st+1 ∈St+1 ) satisfy-
ing DREU1 and DREU2 at t + 1. We now show that for each st+1 ∈ St+1 there exist αst+1 > 0
and βst+1 ∈ R such that after replacing Ũst+1 with Ust+1 := αst+1 Ũst+1 + βst+1 , we additionally
have that EVU holds at time t.
t
Fix any st and let St+1 (st ) := supp µst+1
. Note that by DREU1 at t + 1 and since we have
P
t
t
t
defined ρst+1
by ρst+1
(pt+1 , At+1 ) := st+1 ∈St+1 (st ) µst+1
(st+1 )τst+1 (pt+1 , At+1 ) for all pt+1 and At+1 ,
59
t
t
, {Ũst+1 , τst+1 }st+1 ∈St+1 (st ) ) is an REU form representation of ρst+1
it follows that (St+1 (st ), µst+1
(see Definition 11).
Since all the Ust+1 are non-constant and induce different preferences for distinct st+1 , s0t+1 ∈
St+1 (st ) and since Vst is nonconstant by Lemma 6, we can find a finite Y ⊆ Xt+1 such that
(i) Vst is non-constant on At+1 (Y ) := {Bt+1 ∈ At+1 : ∪pt+1 ∈Bt+1 supp(pt+1 ) ⊆ Y }; (ii) for each
st+1 ∈ St+1 (st ), Ũst+1 is non-constant on Y ; and (iii) for each distinct pair st+1 , s0t+1 ∈ St+1 (st ),
Ũst+1 6≈ Ũs0t+1 on Y .
Observe that by Lemmas 6 and 8, the preference %st on At+1 (Y ) induced by Vst satisfies Axioms DLR 1–6 (Weak Order, Continuity, Independence, Monotonicity, Nontriviality,
Finiteness) in Ahn and Sarver (2013) (henceforth AS), so by Corollary S1 in AS %st admits
t
a DLR representation (see Definition S1 in AS). Moreover, since ρst+1
admits an REU form
representation (what AS call a GP representation), so does its restriction to At+1 (Y ). Fit
nally, by Lemma 7, the pair (%st , ρst+1
) satisfies AS’s Axioms 1 and 2 (Sophistication) on
t
At+1 (Y ). Thus, by Theorem 1 in AS, we can find a DLR-GP representation of (%st , ρst+1
)
t
t
on At+1 (Y ), i.e., an REU form representation (Ŝt+1 (st ), µ̂st+1
, {Ûst+1 , τ̂st+1 }st+1 ∈Ŝt+1 (st ) ) of ρst+1
on At+1 (Y ) such that %st restricted to At+1 (Y ) is represented by V̂st , where V̂st (At+1 ) :=
P
st
Since Vst also represents %st restricted to
st+1 ∈Ŝt+1 (st ) µ̂t+1 (st+1 ) maxpt+1 ∈At+1 Ûst+1 (pt+1 ).
At+1 (Y ), standard arguments yield αˆst > 0 and β̂st ∈ R such that for all At+1 ∈ At+1 (Y ),
we have Vst (At+1 ) = α̂st V̂st (At+1 ) + β̂st , whence
X
Vst (At+1 ) =
t
µ̂st+1
(st+1 ) max Ust+1 (pt+1 ),
pt+1 ∈At+1
(19)
st+1 ∈Ŝt+1 (st )
where Ust+1 = α̂st Ûst+1 + β̂st .
By the uniqueness properties of REU form represen-
t
tations (Proposition 4 in AS), (Ŝt+1 (st ), µ̂st+1
, {Ust+1 , τ̂st+1 }st+1 ∈Ŝt+1 (st ) ) still constitutes an
t
REU form representation of ρst+1
on At+1 (Y ). Applying Proposition 4 in AS again, since
t
t
(St+1 (st ), µst+1
, {Ũst+1 , τst+1 }st+1 ∈St+1 (st ) ) also represents ρst+1
on At+1 (Y ), we can assume after
t
t
relabeling that St+1 (st ) = Ŝt+1 (st ), µ̂st+1
= µst+1
and that for each st+1 ∈ St+1 (st ), there exist
constants αst +1 > 0 and βst+1 ∈ R such that
Ust+1 (xt+1 ) = αst+1 Ũst+1 (xt+1 ) + βst+1
for each xt+1 ∈ Y ⊆ Xt+1 .
(20)
Since Ũst+1 is defined on Xt+1 , we can extend Ust+1 to
60
the whole space Xt+1 by (20).
Then Ust+1 and Ũst+1 represent the same preference over
t
, {Ũst+1 , τst+1 }st+1 ∈St+1 (st ) ) satisfies DREU1 and DREU2, so does
∆(Xt+1 ), so since (St+1 (st ), µst+1
t
, {Ust+1 , τst+1 }st+1 ∈St+1 (st ) ).
(St+1 (st ), µst+1
It remains to show that (19) holds for all At+1 ∈ At+1 , so that EVU is satisfied at
To see this, consider any At+1 ∈ At+1 and choose a finite Y 0 ⊆ Xt+1 such that
S
Y ∪ pt+1 ∈At+1 supp(pt+1 ) ⊆ Y 0 . As above, we can again apply Theorem 1 in AS to ob-
st .
s0
t
t
)
, {Ūst+1 , τ̄st+1 }st+1 ∈S̄t+1 (st ) ) of the pair (%st , ρst+1
tain a DLR-GP representation (S̄t+1 (st ), µ̄t+1
t
) rerestricted to At+1 (Y 0 ). But since this also yields a DLR-GP representation of (%st , ρst+1
stricted to At+1 (Y ), by the uniqueness property of DLR-GP representations (Theorem 2 in
t
t
and that there exists ᾱst > 0 and
= µst+1
AS), we can assume that S̄t+1 (st ) = St+1 (st ), µ̄st+1
β̄st+1 ∈ R such that Ūst+1 = ᾱst Ust+1 + β̄st+1 for each st+1 ∈ St+1 (st ). Since %st is repreP
t
sented on At+1 (Y 0 ) by V̄st (Bt+1 ) := st+1 ∈St+1 (st ) µst+1
(st+1 ) maxpt+1 ∈Bt+1 Ūst+1 (pt+1 ) and since
ᾱst depends only on st (and not on st+1 ), it follows that %st is also represented on At+1 (Y 0 )
P
t
(st+1 ) maxpt+1 ∈Bt+1 Ust+1 (pt+1 ). Thus, the linear functions Vst
by Vs0t (Bt+1 ) := st+1 ∈St+1 (st ) µst+1
and Vs0t represent the same preference on At+1 (Y 0 ) and coincide on At+1 (Y ), so they must also
coincide on At+1 (Y 0 ). Thus, (19) holds at At+1 .
This shows that EVU holds at t. Combining this with the inductive hypothesis, it follows
s
0
that (St0 , {µt0t −1 }st0 −1 ∈St0 −1 , {Ust0 , τst0 }st0 ∈St0 ) satisfies DREU1 and DREU2 for all t0 ≤ t + 1 and
EVU for all t0 ≤ t, as required.
C.3
Proof of Theorem 2: Necessity
Suppose that (ρt ) admits an evolving utility representation. Then by Proposition 6, (ρt ) admits
s
an S-based evolving utility representation (St , {µt t−1 }st−1 ∈St−1 , {Ust , ust , τst }st ∈St ).
We first show that for every t ≤ T − 1, there exist gt , bt ∈ ∆(Xt ) such that Ust (gt ) > Ust (bt )
0
for all st ∈ St . By separability of Ust , it is sufficient to find menus Ct+1
, Ct+1 such that
0
Vst (Ct+1
) > Vst (Ct+1 ) for all st . Note first that for any st+1 ∈ St+1 , since Ust+1 is nonconstant,
we can find gt+1 (st+1 ), bt+1 (st+1 ) ∈ ∆(Xt+1 ) such that Ust+1 (gt+1 (st+1 )) > Ust+1 (bt+1 (st+1 )). Let
0
Ct+1
:= {gt+1 (st+1 ), bt+1 (st+1 ) : st+1 ∈ St+1 }, and for every st , let At+1 (st ) := {bt+1 (st+1 )} for
0
t
some st+1 ∈ suppµst+1
. Then Vst (Ct+1
) ≥ Vst (At+1 (s0t )) for all st , s0t , with strict inequality for
P
0
) > Vst (Ct+1 ) for
st = s0t . Hence, letting Ct+1 := st ∈St |S1t | At+1 (st ), linearity implies Vst (Ct+1
all st , as required.
61
By Lemma 5, the previous paragraph implies that for all t ≤ T − 1, ht and qt , rt , we have
qt %ht rt if and only if Ust (qt ) ≥ Ust (rt ) for all st consistent with ht . Axioms 5 (Separability)
and 6 (i)–(ii) (Monotonicity and Indifference to Timing) are then straightforward to verify from
0
0
the representation. Moreover, Ct+1
and Ct+1 from the previous paragraph satisfy (zt , Ct+1
) ht
(zt , Ct+1 ) for all ht and zt , implying Axiom 6 (iv) (Nondegeneracy).
To show Axiom 7 (Sophistication), consider any t ≤ T − 1, ht , zt , and At+1 ⊆ A0t+1 ∈
A∗ (ht ). Since A0t+1 ∈ A∗ (ht ), Lemma 13 implies that ρt+1 (A0t+1 r At+1 ; A0t+1 |ht ) > 0 holds
if and only if there exists some st consistent with ht such that maxpt+1 ∈A0t+1 Ust+1 (pt+1 ) >
t
, which by the representation is equivalent to
maxpt+1 ∈At+1 Ust+1 (pt+1 ) for some st+1 ∈ suppµst+1
Vst (A0t+1 ) > Vst (At+1 ). By Lemma 5, this is equivalent to (zt , At+1 ) 6%ht (zt , A0t+1 ), which by
Monotonicity is in turn equivalent to (zt , A0t+1 ) ht (zt , At+1 ).
Finally, to show Axiom 6 (iii) (Menu Continuity), note first that for each sT −1 ∈
P
sT −1
ST −1 ,
(sT ) maxpT ∈AT UsT (pT ) is continuous in menu AT . Assuming inducsT ∈ST µT
P
sk
tively that for each k ≥ t + 1 and sk ∈ Sk ,
sk+1 ∈Sk+1 µk+1 (sk+1 ) maxpk+1 ∈Ak+1 Usk+1 (pk+1 )
is continuous in menu Ak+1 , it also follows that for each t and st ∈ St ,
P
st
st+1 ∈St+1 µt+1 (st+1 ) maxpt+1 ∈At+1 Ust+1 (pt+1 ) is continuous in menu At+1 . Then Menu Continuity follows by Theorem 4.
D
D.1
Proof of Theorem 3
Proof of Theorem 3: Sufficiency
Suppose that ρ admits an evolving utility representation and that Condition 1
and Axioms 8 (Stationary Consumption Preference) and 9 (Intertemporal Calibration) hold.
By Proposition 6, ρ admits an S-based evolving utility representation
s
(St , {µt t−1 }st−1 ∈St−1 , {Ust , ust , τst }st ∈St )t=0,...,T .
ity ust and Ust , we can ensure that
affecting that
Up to adding appropriate constants to each util-
P
z∈Z ust (z) = 0 for all t =
st−1
(St , {µt }st−1 ∈St−1 , {Ust , ust , τst }st ∈St )t=0,...,T is an
0, ..., T and st ∈ St without
S-based evolving utility rep-
resentation of ρ. We will show that this representation is in fact an S-based gradual learning
representation, i.e., that there exists a discount factor δ ∈ (0, 1) such that for all t ≤ T − 1
P
t
(st+1 )ust+1 . By Proposition 6, this implies that ρ admits a
and st , we have ust = 1δ st+1 µst+1
gradual learning representation.
62
Condition 1 implies that each ust is nonconstant:
Lemma 9. For each t = 0, .., T −1 and st ∈ St , there exist p, q ∈ ∆(Z) such that ust (p) 6= ust (q).
Proof. Consider any t = 0, . . . , T − 1, st ∈ St and separating history ht for st . By Condition 1,
there exist p, q, r ∈ ∆(Z) such that (p, r, . . . , r) 6∼ht (q, r, . . . , r). Then Lemma 5 (iii) implies
that Ust ((p, r, . . . , r)) 6= Ust ((q, r, . . . , r)), whence ust (p) 6= ust (q), as required.
For any t = 0, . . . , T − 1 and st ∈ St and p ∈ ∆(Z), let
E[ut+1 (p)|st ] :=
X
t
(st+1 )ust+1 (p)
µst+1
st+1
denote the expected period t + 1 consumption utility of p at state st . Stationary Consumption
Preference implies that ust and E[ut+1 |st ] induce the same preference over ∆(Z):
Lemma 10. For all p, q ∈ ∆(Z), t = 0, ..., T − 1, and st ∈ St ,
E[ut+1 (p)|st ] > E[ut+1 (q)|st ] ⇐⇒ ust (p) > ust (q).
Proof. Fix any p, q, r ∈ ∆(Z), t = 0, ..., T − 1, st ∈ St and separating history ht for st . Note
that ust (p) > ust (q) if and only if Ust ((p, r, . . . , r)) > Ust ((q, r, . . . , r), which by Lemma 5 (iii)
is in turn equivalent to (p, r, . . . , r) ht (q, r, . . . , r). Likewise, E[ut+1 (p)|st ] > E[ut+1 (q)|st ] if
and only if Ust ((r, p, r, . . . , r)) > Ust ((r, q, r, . . . , r)), which by Lemma 5 (iii) is equivalent to
(r, p, r, . . . , r) ht (r, q, r, . . . , r). Thus, the claim is immediate from Axiom 8.
Given Lemma 10, Intertemporal Calibration now allows us to obtain the discount factor δ:
Lemma 11. Let α∗ ∈ ( 21 , 1) be as in Axiom 9. Set δ :=
1−α∗
.
α∗
Then for all t = 0, ..., T − 1, and
st ∈ St , we have ust = 1δ E[ut+1 |st ].
Proof. Fix any t = 0, ..., T −1, st ∈ St , and separating history ht for st . Let ûst := E[ut+1 |st ]. By
Lemma 10, ust and ûst induce the same preference over ∆(Z), and moreover, ust is nonconstant
by Lemma 9. Hence, there exist constants αst > 0, βst ∈ R such that ust = αst ûst + βst . Since
P
we have normalized consumption utilities such that z∈Z ust0 (z) = 0 for any t0 and st0 , we must
have βst = 0.
Fix any r ∈ ∆(Z). By Condition 1 and Lemma 9, there exist p, q ∈ ∆(Z) such that
ust (p) 6= ust (q). By Axiom 9, we have (α∗ p+(1−α∗ )q, α∗ p+(1−α∗ )q, r, . . . , r) ∼ht (p, q, r, . . . , r).
63
By Lemma 5 (iii), this is equivalent to Ust (α∗ p + (1 − α∗ )q, α∗ p + (1 − α∗ )q, r, . . . , r))) =
Ust ((p, q, r, . . . , r)), i.e., α∗ (ust (p) + ûst (p)) + (1 − α∗ )(ust (q) + ûst (q)) = ust (p) + ûst (q). Rearranging and using the fact that ust = αst ûst , this is equivalent to
α∗ ust (p) + (1 − α∗ )ust (q) =
1
αst
ust (p) +
us (q).
αst + 1
αst + 1 t
But since ust (p) 6= ust (q), this implies α∗ (αst + 1) = αst , i.e., αst =
α∗
.
1−α∗
Thus, ust = 1δ ûst , as
claimed.
Note that since α∗ ∈ ( 21 , 1), we have δ :=
1−α∗
α∗
∈ (0, 1), so this completes the proof that ρ
admits an S-based gradual learning representation.
D.2
Proof of Theorem 3: Necessity
Suppose
that
ρ
admits
holds.
By Proposition 6,
a
gradual
learning
representation
and
Condition
1
ρ admits an S-based gradual learning representation
s
(St , {µt t−1 }st−1 ∈St−1 , {Ust , ust , τst }st ∈St )t=0,...,T
with discount factor δ ∈ (0, 1).
The same argument as in the proof of the necessity direction of Theorem 2 shows that for
all t ≤ T − 1, ht and qt , rt ∈ ∆(Xt ), we have qt %ht rt if and only if Ust (qt ) ≥ Ust (rt ) for all st
consistent with ht .
Given this, Axiom 8 is equivalent to the statement that for all st , ust and E[ut+1 |st ] represent
the same preference over ∆(Z). But this is immediate from the fact that for all st , we have
ust = 1δ E[ut+1 |st ].
Let α∗ :=
1
.
1+δ
∗
Fix any t ≤ T − 1 and p, q, r ∈ ∆(Z). Then ust = 1δ E[ut+1 |st ] implies that for
all st , we have α (ust (p)+E[ut+1 (q)|st ])+(1−α∗ )(ust (q)+E[ut+1 (p)|st ]) = ust (p)+E[ut+1 (q)|st ],
whence Ust (α∗ p + (1 − α∗ )q, α∗ p + (1 − α∗ )q, r, . . . , r))) = Ust ((p, q, r, . . . , r)). By the second
paragraph, this implies (α∗ p + (1 − α∗ )q, α∗ p + (1 − α∗ )q, r, . . . , r) ∼ht (p, q, r, . . . , r) for all ht ,
establishing Axiom 9.
E
Additional Lemmas
Lemma 12. Let Y be any set (possibly infinite) and let {Us : s ∈ S} ⊆ RY be a collection of
nonconstant vNM utility functions indexed by a finite set S such that Us 6≈ Us0 for any distinct
64
0
s, s0 ∈ S. Then there is a collection of lotteries {ps : s ∈ S} ⊆ ∆(Y ) such that Us (ps ) > Us (ps )
for any distinct s, s0 ∈ S.
Proof. By the finiteness of S, there is a finite set Y 0 ⊆ Y such that for each s the restriction
Us Y 0 to Y 0 is nonconstant and for any distinct s, s0 , Us Y 0 6≈ Us0 Y 0 (that is, there exists
p, q ∈ ∆(Y 0 ) such that Us (p) ≥ Us (q) and Us0 (p) < Us0 (q)). By Lemma 1 in Ahn and Sarver
(2013), there is a collection of lotteries {ps : s ∈ S} ⊆ ∆(Y 0 ) such that Us (ps ) = Us Y 0 (ps ) >
0
0
Us Y 0 (ps ) = Us (ps ) for any distinct s, s0 .
s
0
Lemma 13. Fix t = 0, . . . , T . Suppose (St0 , {µt0t −1 }st0 −1 ∈St0 −1 , {Ust0 , τst0 }st0 ∈St0 ) satisfy DREU1
and DREU2 for all t0 ≤ t. Take any ht−1 ∈ Ht−1 and let S(ht−1 ) ⊆ St−1 denote the set of states
consistent with ht−1 . Then for any At ∈ At , the following are equivalent:
(i). At ∈ A∗t (ht−1 )
s
(ii). For each st−1 ∈ S(ht−1 ) and st ∈ supp µt t−1 , |M (At , Ust )| = 1.
Proof.
(i) =⇒ (ii): We prove the contrapositive. Suppose that there is st−1 ∈ S(ht−1 ) and and
s
st ∈ supp µt t−1 such that |M (At , Ust )| > 1. Pick any pt ∈ M (At , Ust ) such that τst (pt , At ) > 0.
Since Ust is nonconstant, we can find lotteries r, r ∈ ∆(Xt ) such that Ust (r) < Ust (r). Fix any
sequence αn ∈ (0, 1) with αn → 0. Let pnt := αn r + (1 − αn )pt . For every qt ∈ At r {pt },
let q nt := αn r + (1 − αn )qt and q nt := αn r + (1 − αn )qt . Let B nt := {q nt : qt ∈ At r {pt }}, let
n
n
B t := {q nt : qt ∈ At r {pt }}, and let Btn := B nt ∪ B t . Then Btn →m At r {pt } and pnt →m pt .
Moreover, since |M (At , Ust )| > 1, there exists qt ∈ At r{pt } such that Ust (αn r+(1−αn )qt ) >
Ust (pnt ) for all n, so that τst (pnt , Btn ∪ {pnt }) = 0. Furthermore, note that for all s0t ∈ St r {st },
we have N (M (At , Us0t ), pt ) = N (M (B nt ∪ {pnt }, Us0t ), pnt ) ⊆ N (M (Btn ∪ {pnt }, Us0t ), pnt ), so that
τs0t (pnt , Btn ∪ {pnt }) ≤ τs0t (pt , At ) for all n. By Lemma 15, this implies that there exists35 c > 0
such that for all n,
ρt (pnt ; Btn ∪ {pnt }|ht−1 ) ≤ ρt (pt ; At |ht−1 ) − cτst (pt , At ),
whence limn→∞ ρt (pnt ; Btn ∪ {pnt }|ht−1 ) < ρt (pt ; At |ht−1 ). By definition of A∗t , this means that
At ∈
/ A∗t (ht−1 ).
35
Specifically, suppose ht−1
Qt−1 sk−1
s
( k=0 µk (sk )τsk (pk ,Ak ))µt t−1 (st )
ρ(ht−1 )
= (A0 , p0 , . . . , At−1 , pt−1 ) and pred(st−1 ) = (s0 , . . . , st−2 ).
> 0.
65
Then c =
(ii) =⇒ (i): Suppose At satisfies (ii). Consider any pt ∈ At , pnt →m pt , Btn →m At r {pt }.
s
Consider any st−1 ∈ S(ht−1 ) and st ∈ supp µt t−1 . By (ii), we either have M (At , Ust ) = {pt }
or pt ∈
/ M (At , Ust ). In the former case, Ust (pt ) > Ust (qt ) for all qt ∈ At r {pt }. But then, for
all n large enough, linearity of Ust implies Ust (pnt ) > Ust (qtn ) for all qtn ∈ Btn , i.e., τst (pt , At ) =
limn τst (pnt , Btn ∪ {pnt }) = 1. In the latter case, Ust (pt ) < Ust (qt ) for some qt ∈ At r {pt }. But
then, for all n large enough, linearity of Ust implies Ust (pnt ) < Ust (qtn ) for all qtn ∈ Btn such that
qtn →m qt , i.e., τst (pt , At ) = limn τst (pnt , Btn ∪ {pnt }) = 0.
s
Thus, for all st−1 ∈ S(ht−1 ) and st ∈ supp µt t−1 , we have τst (pt , At ) = limn τst (pnt , Btn ∪{pnt }).
Hence, the representation in Lemma 15 implies that for all n sufficiently large,
ρt (pnt ; Btn ∪ {pnt }|ht−1 ) = ρt (pt ; At |ht−1 ),
as required.
Lemma 14. Suppose that ρ satisfies Axiom 2.
Fix t ≥ 1, At ∈ At , ht−1 =
t−1
t
t−1
(A0 , p0 , . . . , At−1 , pt−1 ) ∈ Ht−1 , and λ = (λn )t−1
=
n=0 , λ̂ = (λ̂n )n=0 ∈ (0, 1] . Suppose d
ˆt−1 = ({q̂n }, q̂n )t−1 ∈ Dt−1 satisfy λht−1 + (1 − λ)dt−1 , λ̂ht−1 + (1 − λ̂)dˆt−1 ∈
({qn }, qn )t−1
n=0 , d
n=0
Ht−1 (At ), where λht−1 + (1 − λ)dt−1 := (λn An + (1 − λn ){qn }, λn pn + (1 − λn )qn )t−1
n=0 and
λ̂ht−1 + (1 − λ̂)dˆt−1 is defined analogously. Then
ρt (·; At |λht−1 + (1 − λ)dt−1 ) = ρt (·; At |λ̂ht−1 + (1 − λ̂)dˆt−1 ),
t−1
and hence, ρht
(·; At ) = ρt (·; At |λht−1 + (1 − λ)dt−1 ).
Proof. Let k := max{n = 0 . . . , t − 1 : qn 6= q̂n } be the last entry at which dt−1 and dˆt−1 differ,
where we set k = −1 if qn = q̂n for all n = 0, . . . , t − 1. We prove the claim by induction on k.
Suppose first that k = −1, i.e., that dt−1 = dˆt−1 . If λ0 > λ̂0 , then the 0-th entry of
λht−1 +(1−λ)dt−1 can be written as an appropriate mixture of the 0-th entry of λ̂ht−1 +(1−λ̂)dˆt−1
with (A0 , p0 ); if λ0 ≤ λ̂0 , then then the 0-th entry of λht−1 + (1 − λ)dt−1 can be written as
an appropriate mixture of the 0-th entry of λ̂ht−1 + (1 − λ̂)dˆt−1 with ({q0 }, q0 ). In either case,
Axiom 2 implies that ρt (·; At |λ̂ht−1 + (1 − λ̂)dˆt−1 ) is unaffected after replacing the 0-th entry
of λ̂ht−1 + (1 − λ̂)dˆt−1 with the 0-th entry of λht−1 + (1 − λ)dt−1 . Continuing this way, we can
successively apply Axiom 2 to replace each entry of λ̂ht−1 + (1 − λ̂)dˆt−1 with the corresponding
entry of λht−1 + (1 − λ)dt−1 without affecting ρt . This yields the desired conclusion.
66
Suppose the claim holds whenever k ≤ m − 1 for some 0 ≤ m ≤ t − 1. We show that the
claim continues to hold for k = m. Note first that we can assume that
1 t−1 1 t−1 1 t−1 1 ˆt−1
h + d , h + d ∈ Ht−1 (At );
2
2
2
2
2
1
1
1
A
Bm + {q̂m }, { qm + q̂m } ∈ supp qm−1
;
3
3
2
2
1
1
2
1
A
,
B̂m + {qm }, { qm + q̂m } ∈ supp q̂m−1
3
3
2
2
(21)
where Bm := 12 Am + 21 {qm },B̂m := 12 Am + 12 {q̂m }, rm := 21 pm + 12 qm , and r̂m := 12 pm + 12 q̂m .
Indeed, we can find a sequence of lotteries (`n )t−1
n=0 such that for all n = 1, . . . , t − 1
1
1
1
1
λn An + (1 − λn ){on }, An + {on }, λ̂n An + (1 − λ̂n ){ôn }, An + {ôn }, {on } ∈ supp `A
n−1 ;
2
2
2
2
2
1
2
1
1
1
Bm + {ôm }, B̂m + {om }, { om + ôm } ∈ supp `A
m−1 ,
3
3
3
3
2
2
where on :=
1
q
2 n
+ 21 `n and ôn :=
1
q̂
2 n
t−1
+ 12 `n . Letting ct−1 := ({on }, on )t−1
:=
n=0 and ĉ
t−1 t−1
({ôn }, ôn )t−1
, ĉ ∈ Dt−1 , λht−1 + (1 − λ)ct−1 , λ̂ht−1 + (1 − λ̂)ĉt−1 ∈ Ht−1 (At ),
n=0 , we have that c
and the last entry at which ct−1 and ĉt−1 differ is m. Moreover, repeated application of Axiom 2
implies
ρt (·; At |λht−1 + (1 − λ)dt−1 ) = ρt (·; At |λht−1 + (1 − λ)ct−1 );
ρt (·; At |λ̂ht−1 + (1 − λ̂)dˆt−1 ) = ρt (·; At |λ̂ht−1 + (1 − λ̂)ĉt−1 ).
Thus, we can replace dt−1 and dˆt−1 with ct−1 and ĉt−1 if need be and guarantee that (21) is
satisfied.
Given (21), 21 ht−1 + 12 dt−1 , 12 ht−1 + 12 dˆt−1 ∈ Ht−1 (At ), so the base case of the proof implies
1
ρt (·; At |λht−1 + (1 − λ)dt−1 ) = ρt (·; At | ht−1 +
2
1
ρt (·; At |λ̂ht−1 + (1 − λ̂)dˆt−1 ) = ρt (·; At | ht−1 +
2
1 t−1
d );
2
1 ˆt−1
d ).
2
(22)
Also, (21) guarantees that ((λht−1 + (1 − λ)dt−1 )−m , ( 32 Bm + 31 {q̂m }, 23 rm + 13 q̂m )) and ((λht−1 +
(1−λ)dˆt−1 )−m , ( 2 B̂m + 1 {qm }, 2 r̂m + 1 qm )) are well-defined histories in Ht−1 (A). Thus, Axiom 2
3
3
3
3
67
implies
1
ρt (·; At | ht−1 +
2
1
ρt (·; At | ht−1 +
2
1 t−1
1
d ) = ρt (·; At |( ht−1 +
2
2
1 ˆt−1
1
d ) = ρt (·; At |( ht−1 +
2
2
1 t−1
2
d )−m , ( Bm +
2
3
1 ˆt−1
2
d )−m , ( B̂m +
2
3
1
2
{q̂m }, rm +
3
3
2
1
{qm }, r̂m +
3
3
1
q̂m ));
3
1
qm )).
3
(23)
But note that
2
1
2
1
( Bm + {q̂m }, rm + q̂m )
3
3
3
3
1
2 1
1
1
2 1
1
= ( Am + { qm + q̂m }, pm + ( qm + q̂m )
3
3 2
2
3
3 2
2
2
1
2
1
= ( B̂m + {qm }, r̂m + qm ).
3
3
3
3
Thus, (( 12 ht−1 + 12 dt−1 )−m , ( 23 Bm + 13 {q̂m }, 23 rm + 31 q̂m )) is an entry-wise mixture of ht−1 with
the degenerate history et−1 = ((dt−1 )−m , ({ 12 qm + 21 q̂m }, 12 qm + 21 q̂m ) and similarly (( 12 ht−1 +
1 ˆt−1
d )−m , ( 2 B̂m + 1 {qm }, 2 r̂m + 1 qm )) is an entry-wise mixture of ht−1 with the degenerate
2
history ê
3
t−1
3
3
3
= ((dˆt−1 )−m , ({ 12 qm + 21 q̂m }, 12 qm + 12 q̂m ). But the last entry at which et−1 and êt−1
differ is strictly smaller than m. Hence, applying the inductive hypothesis, we obtain
1
2
1
2
1
1
ρt (·; At |( ht−1 + dt−1 )−m , ( Bm + {qm }, rm + qm )) =
2
2
3
3
3
3
1 t−1 1 ˆt−1
2
1
2
1
ρt (·; At |( h + d )−m , ( B̂m + {qm }, r̂m + qm )).
2
2
3
3
3
3
(24)
Combining (22), (23), and (24) yields
ρt (·; At |λht−1 + (1 − λ)dt−1 ) = ρt (·; At |λ̂ht−1 + (1 − λ̂)dˆt−1 ),
as required.
t−1
Finally, let dˆt−1 and λ̂ ∈ (0, 1] be the choices from Definition 10 such that ρht (·; At ) :=
t−1
ρt (·; At |λ̂ht−1 + (1 − λ̂)dˆt−1 ). Then the above implies that ρht (·; At ) = ρt (·; At |λht−1 + (1 −
λ)dt−1 ), as claimed.
s
0
Lemma 15. Fix t = 0, . . . , T . Suppose (St0 , {µt0t −1 }st0 −1 ∈St0 −1 , {Ust0 , τst0 }st0 ∈St0 ) satisfy DREU1
and DREU2 for all t0 ≤ t. Then the extended version of ρ also satisfies DREU2 for all t0 ≤ t,
68
0
i.e., for all p0 t , At0 , and ht −1 = (A0 , p0 , A1 , p1 , . . . , At−1 , pt0 −1 ) ∈ Ht0 −1 ,36 we have
P
t0 −1
ρt0 (pt0 , At0 |h
)= P
(s0 ,...,st0 )∈S0 ×...×St0
Qt0
(s0 ,...,st0 −1 )∈S0 ×...×St0 −1
sk−1
(sk )τsk (pk , Ak )
k=0 µk
.
Qt−1 sk−1
(s
)τ
(p
,
A
)
µ
k
s
k
k
k
k=0 k
0
Proof. If ht −1 ∈ Ht0 −1 (At0 ), the claim is immediate from DREU2.
0
So suppose ht −1 ∈
/
0
0
−1
Ht0 −1 (At0 ). Let λ ∈ (0, 1) and dt −1 = ({q` }, q` )t`=0
∈ Dt0 −1 be the choices from Definition 4 such
0
0
0
0
0
that λht −1 + (1 − λ)dt −1 ∈ Ht0 −1 (At0 ) and ρt0 (pt0 , At0 |ht −1 ) := ρt0 (pt0 , At0 |λht −1 + (1 − λ)dt −1 ).
Note that for all k ≤ t0 , sk ∈ Sk , and w ∈ RXk , we have pk ∈ M (M (Ak , Usk ), w) if and
only if λpk + (1 − λ)qk ∈ M (M (λAk + (1 − λ){qk }, Usk ), w). Hence, τsk (pk , Ak ) = τsk (λpk +
(1 − λ)qk , λAk + (1 − λ){qk }). Thus, the claim follows from DREU2 applied to the history
0
0
λht −1 + (1 − λ)dt −1 ∈ Ht0 −1 (At0 ).
s
0
Lemma 16. Fix t = 0, . . . , T . Suppose (St0 , {µt0t −1 }st0 −1 ∈St0 −1 , {Ust0 , τst0 }st0 ∈St0 ) satisfy DREU1
and DREU2 for all t0 ≤ t. Fix any st−1 ∈ St−1 , separating history ht−1 for st−1 , and At ∈ At .
Then there exists a sequence Ant →m At such that Ant ∈ A∗t+1 (ht ) for all n. Moreover, given any
s
s∗t ∈ suppµt t−1 and p∗t ∈ M (At , Us∗t ), we can ensure in this construction that there is pnt (s∗t ) ∈ Ant
with pnt (s∗t ) →m p∗t such that Ust (Ant , pnt (s∗t )) = {Us∗t } for all n.
s
Proof. Let St (st−1 ) := suppµt t−1 . By DREU1, we can find a finite Yt ⊆ Xt such that (i) for
any st ∈ St (st−1 ), Ust is non-constant over Yt ; (ii) for any distinct st , s0t ∈ St (st−1 ), Ust ≈
6 Us0t
S
over Yt ; and (iii) pt ∈At supppt ⊆ Yt . By (i) and (ii) and Lemma 12, we can find a menu
Dt := {qtst : st ∈ St (st−1 )} ⊆ ∆(Yt ) such that M (Dt , Ust ) = {qtst } for all st ∈ St (st−1 ). Define
P
bt := y∈Y |Y1 | δy ∈ ∆(Y ). For each st ∈ St (st−1 ), pick z st ∈ argmaxy∈Y Ust and let gtst := δzst .
By (i), we have Ust (gtst ) > Ust (bt ) for all st ∈ St (st−1 ). Hence, there exists α ∈ (0, 1) small
enough such that for all st ∈ St (st−1 ), we have Ust (q̂ st ) > Ust (bt ), where q̂ st := αqtst + (1 − α)gtst .
Note that setting D̂ := {q̂tst : st ∈ St (st−1 )}, we still have M (D̂t , Ust ) = {q̂tst }.
For each st ∈ St (st−1 ), pick some pt (st ) ∈ M (At , Ust ). For the “moreover” part, we can
ensure that pt (s∗t ) = p∗t . Fix any sequence (εn ) from (0, 1) such that εn → 0. For each
n and st ∈ St (st−1 ), let pnt (st ) := (1 − ε)pt (st ) + εq̂tst . And for each rt ∈ At , let rtn :=
(1 − ε)rt + εbt . Finally, let Ant := {pnt (st ) : st ∈ St (st−1 )} ∪ {rtn : rt ∈ At }. Note that Ant →m At .
Moreover, by construction, for all st ∈ St (st−1 ) and n, we have M (Ant , Ust ) = {pnt (st )}: Indeed,
36
0
0
For t0 = 0, we abuse notation by letting ρt0 (·|ht −1 ) denote ρ0 (·) for all ht −1 .
69
Ust (pnt (st )) > Ust (rtn ) for all rt ∈ At since Ust (pt (st )) ≥ Ust (rt ) and Ust (q̂tst ) > Ust (bt ); and
s0
Ust (pnt (st )) > Ust (pnt (s0t )) for all s0t 6= st , since Ust (pt (st )) ≥ Ust (pt (s0t )) and Ust (q̂tst ) > Ust (q̂t t ).
Since st−1 is the only state consistent with ht−1 , Lemma 13 implies that Ant ∈ A∗t (ht−1 ), as
required. Finally, for the “moreover” part, note that we ensured that pt (s∗t ) = p∗t . Hence pnt (s∗t )
constructed above has the desired property that pnt (s∗t ) →m p∗t and Ust (Ant , pnt (s∗t )) = {Us∗t } for
all n.
F
REU on an Infinite Outcome Space
F.1
Domain and Primitive
For any set Z we denote by ∆(Z) the set of all simple lotteries on Z and by K(Z) the set of
all nonempty finite subsets of Z. We are given a metric space (X, d) of outcomes. The space
∆(X) is endowed with the topology of weak convergence induced by d, which is metrized by
the Prokhorov metric π. We denote by A := K(∆(X)) the space of menus, which is endowed
with the Hausdorff topology induced by π. Finally, ∆(∆(X)) is endowed with the topology of
weak convergence induced by π. Convergence in these topologies is denoted by →.
We use an additional convergence notion, called convergence in mixture, on ∆(X) and A:
For any p ∈ ∆(X) and sequence {pn }n∈N ⊆ ∆(X), we write pn →m p if there exists q ∈ ∆(X)
and a sequence {αn }n∈N with αn → 0 such that pn = αn q + (1 − αn )p for all n. Similarly, for
any p ∈ ∆(X) and sequence {B n }n∈N ⊆ A, we write B n →m p if there exists B ∈ A and a
sequence {αn }n∈N with αn → 0 such that B n = αn B + (1 − αn ){p} for all n. Finally, for any
A ∈ A and sequence (An )n∈N ⊆ A, we write An →m A if for each p ∈ A, there is a sequence
{Bpn }n∈N ⊆ A such that Bpn →m p and An = ∪p∈A Bpn for all n.
Our primitive is a stochastic choice rule, i.e., a map ρ : A → ∆(∆(X)) such that
P
p∈A
ρ(p; A) = 1 for each A ∈ A. For any Y ⊆ X, let A(Y ) := {A ∈ A : ∀p ∈ A, supp(p) ⊆
Y } ⊆ A denote the space of all menus consisting only of lotteries with support in Y . Note that
for each A ∈ A, there is a finite Y such that A ∈ A(Y ). We denote by ρY the restriction of ρ
to A(Y ), which can be seen as a map from A(Y ) to ∆(∆(Y )).
70
F.2
REU form representation
Consider the space of vNM utilities RX over X, endowed with the product topology of the
Euclidean topologies. For any u ∈ RX and p ∈ ∆(X), the expected value of p under u is
P
u(p) := x∈supp(p) u(x)p(x), which we sometimes denote by u · p. For any u, u0 ∈ RX , we write
u ≈ u0 if u and u0 represent the same preference on ∆(X). Throughout, we fix some y ∗ ∈ X
and let R̃X = {0} × RXr{y
∗}
denote the set of utility functions u in RX that are normalized by
∗
u(y ∗ ) = 0. Likewise, for any Y ⊆ X such that y ∗ ∈ Y , we write R̃Y := {0} × RY r{y } .
For any A ∈ A(Y ) and u ∈ R̃Y , let M (A, u) := argmaxp∈A u(p). For any A ∈ A(Y ) and
p ∈ ∆(X), let NY (A, p) := {u ∈ R̃X : p ∈ M (A, u)} and let NY+ (A, p) := {u ∈ R̃Y : {p} =
M (A, u)}. Note that NY ({p}, p) = NY+ ({p}, p) = R̃Y and that NY (A, p) = NY+ (A, p) = ∅ if
p ∈
/ A. Let N (Y ) := {NY (A, p) : A ∈ A(Y ) and p ∈ ∆(X)}, N + (Y ) := {NY+ (A, p) : A ∈
A(Y ) and p ∈ ∆(X)}.
We will consider both the Borel σ-algebra on R̃Y and its subalgebra F(Y ) that is generated
by N (Y ) ∪ N + (Y ). A finitely additive probability measure ν Y on either of these algebras is
called regular if ν Y (NY (A, p)) = ν Y (NY+ (A, p)) for any A ∈ A(Y ) and p ∈ ∆(X). Whenever
Y = X, we omit Y from the description of NY (A, p), NY+ (A, p), N (Y ), N + (Y ), and F(Y ).
Definition 11. An REU form representation of ρ is a tuple (S, µ, {Us , τs }s∈S ) such that
(i). S is a finite state space and µ is a probability measure on S such that supp(µ) = S
(ii). for each s ∈ S, the utility Us ∈ R̃X is nonconstant and Us 6≈ Us0 for s 6= s0
(iii). for each s ∈ S, the tie-breaking rule τs is a regular finitely additive probability measure
on R̃X endowed with the Borel σ-algebra
(iv). for all p ∈ ∆(X) and A ∈ A,
ρ(p; A) =
X
µ(s)τs (p, A),
s∈S
where τs (p, A) := τs ({u ∈ R̃X : p ∈ M (M (A, Us ), u)}).
71
F.3
Characterization
Our representation theorem for REU form representations extends the characterizations for
finite outcome spaces X in Gul and Pesendorfer (2006) and Ahn and Sarver (2013) by allowing
X to be an arbitrary metric space of outcomes.
The first four conditions of our REU axiom are restatements of the conditions introduced
by Gul and Pesendorfer (2006). The fifth condition is a slight modification of the finiteness
condition in Ahn and Sarver (2013).
Axiom 10. (Random Expected Utility)
(i). Regularity: If A ⊆ A0 , then for all p ∈ A, ρ(p; A) ≥ ρ(p; A0 ).
(ii). Linearity: For any A, p ∈ A, λ ∈ (0, 1), and q, ρ(p; A) = ρ(λp + (1 − λ)q; λA + (1 − λ){q}).
(iii). Extremeness: For any A, ρ(extA; A) = 1.
(iv). Mixture Continuity: ρ(·; αA + (1 − α)A0 ) is continuous in α for all A, A0 .
(v). Finiteness: There exists K > 0 such that for all A, there is B ⊆ A with |B| ≤ K such that
for every p ∈ ArB, there are sequences pn →m p and B n →m B with ρ(pn ; {pn }∪B n ) = 0
for all n.
Note that because X may be infinite, continuity of each Us in the representation is not
directly implied by linearity. The following additional axiom yields continuity.
Let A∗ denote the collection of menus without ties, i.e., the set of all A ∈ A such that for
any p ∈ A and any sequences pn →m p and B n →m A r {p}, we have
lim ρ(pn ; B n ∪ {pn }) = ρ(p; A).
n→∞
Axiom 11 (Continuity). ρ : A∗ → ∆(∆(X)) is continuous.
Theorem 4. The stochastic choice rule ρ on X admits an REU form representation
(S, µ, {Us , τs }s∈S ) if and only if ρ satisfies Axiom 10. Furthermore, each utility Us in the
representation is continuous if and only if ρ additionally satisfies Axiom 11.
72
F.4
Proof of Theorem 4: Sufficiency
F.4.1
Outline
The proof proceeds as follows:
(i). In section F.4.2, we use conditions (i)–(iv) of Axiom 10 and Theorem 2 in Gul and Pesendorfer (2006) to construct, for each finite Y ⊆ X, a regular finitely additive probability
measure ν Y on F(Y ) representing ρY , in the sense that ρY (p; A) = ν(NY (A, p)) for all
A, p. Given the fact that each ρY is derived from the same ρ, it is easy to check that the
family {F(Y ), ν Y } is Kolmogorov consistent. We can then find a regular finitely additive
probability measure ν on F extending all the ν Y (and hence representing ρ).
(ii). The support of ν is defined by
supp(ν) :=
[
{V ∈ F : V is open and ν(V ) = 0}
c
.
In section F.4.3, we use part (v) of Axiom 10 to show that supp ν is finite (up to positive
affine transformation of utilities) and contains at least one non-constant utility function.
While Axiom 10 (v) is similar to the finiteness axiom in Ahn and Sarver (2013), this step
requires more work in our setting. A key technical challenge is that unlike in Ahn and
Sarver, it is not clear in our infinite outcome space setting how to normalize utilities to
ensure that N (A, p)-sets are compact. Compact sets C have the useful property (used
repeatedly by Ahn and Sarver) that if C ∩supp ν = ∅, then ν(C) = 0. Lemma 22 exploits
the geometry of N (A, p)-sets to show that this property continues to hold for N (A, p)-sets
in our setting, even though they are not compact.
(iii). In section F.4.4, we proceed in a similar way to the proof of Theorem S3 in Ahn and
Sarver (2013)(again using Lemma 22 to circumvent technical difficulties). Letting S :=
{s1 , . . . , sL } denote the equivalence classes of nonconstant utilities in supp ν, we find
separating neighborhoods Bs ∈ F of each s such that ν(Bs ) > 0. We then define µ(s) =
ν(Bs ) and τs (V ) =
ν(V ∩B (s))
ν(B (s))
and show that this yields an REU form representation of ρ.
(iv). Finally, section F.4.5 shows that if ρ additionally satisfies Axiom 11, then each Us is
continuous.
73
F.4.2
Construction of ν
In this section, we construct a regular finitely additive probability measure ν on F that represents ρ, i.e., such that for all A ∈ A and p ∈ A, we have
ρ(p; A) = ν(N (A, p)) = ν(N + (A, p))).
First consider any finite Y ⊆ X with y ∗ ∈ Y . By Axiom 10 (i)–(iv) (Regularity, Linearity,
Extremeness, and Mixture Continuity), Theorem 2 in Gul and Pesendorfer (2006) ensures that
there is a regular finitely additive probability measure ν Y on F Y such that
ρY (p; A) = ν Y (NY (A, p)) = ν Y (NY+ (A, p))
for all A ∈ A(Y ) and p ∈ A.
0
0
Claim 4. For any finite Y 0 ⊇ Y 3 y ∗ , (ν Y , F Y ) and (ν Y , F Y ) are Kolmogorov consistent, i.e.,
for any E ∈ F Y , we have
0
ν Y (E × RY
0 rY
) = ν Y (E).
(25)
0
0
Proof. To see this, note first that the LHS of (25) is well-defined, since E × RY rY ∈ F Y by
S
Lemma 21 (iv). Note next that by Lemma 21 (iii), E is of the form ni=1 NY (Ai , pi ) ∩ NY+ (Bi , qi )
for some finite n and Ai , Bi ∈ A(Y ). Let E 0 be obtained from E by replacing each NY (Ai , pi )
S
with NY+ (Ai , pi ). By Lemma 21 (ii), E 0 = ni=1 NY+ (Ci , ri ) for some family {Ci } ⊆ A(Y ).
0
0
Moreover, since both ν Y and ν Y are regular, we have that ν Y (E) = ν Y (E 0 ) and ν Y (E ×
RY
0 rY
0
) = ν Y (E 0 × RY
0 rY
0
). Hence, it suffices to prove that ν Y (E 0 × RY
0 rY
) = ν Y (E 0 ). For
this, it is enough to show that for any collection of sets N1+ , . . . , Nn+ ∈ N + (Y ) := {N + (A, p) :
S
0 S
0
A ∈ A(Y )}, we have ν Y ( ni=1 Ni+ ) = ν Y ( ni=1 Ni+ × RY rY ). We prove this by induction. For
the base case, note that for any N + (A, p) ∈ N + (Y ), we have
0
ν Y (N + (A, p)) × RY
0 rY
0
) = ρY (p, A) := ρ(p; A) =: ρY (p; A) = ν Y (N + (A, p)).
74
Suppose next that the claim is true whenever m < n. Then
m+1
[
Y
Ni+ )
ν (
0
i=1
m
[
0
i=1
m+1
[
νY (
νY (
Y
=ν (
m
[
Ni+ )
+ν
Y
+
)
(Nm+1
Y
−ν (
i=1
Ni+ × RY
0 rY
m
[
+
)) =
(Ni+ ∩ Nm+1
i=1
0
+
) + ν Y (Nm+1
× RY
0 rY
0
) − νY (
m
[
+
(Ni+ ∩ Nm+1
) × RY
0 rY
)=
i=1
Ni+ × RY
0 rY
),
i=1
+
∈
where the second equality follows from the inductive hypothesis and the fact that Ni+ ∩ Nm+1
N + (Y ) by Lemma 21 (ii).
Now define ν on F by setting ν(E) := ν Y (projR̃Y E) for any finite Y 3 y ∗ such that
E = projR̃Y E × RXrY and projR̃Y E ∈ F Y . By Lemma 21 (iv) such a Y exists. Moreover, given
Kolmogorov consistency of the family {ν Y }Y ⊆X , this is well-defined. Finally, it is immediate
that ν is a regular finitely additive probability measure and that ν represents ρ.
F.4.3
Finiteness of supp ν
The support of a finitely additive probability measure ν is defined by
supp(ν) :=
[
c
{V ∈ F : V is open and ν(V ) = 0} .
The next lemma invokes Axiom 10 (v) (Finiteness) to show that the support of ν constructed in
the previous section contains finitely many equivalence classes of utility functions and contains
at least one nonconstant function. We use 0 to denote the unique constant utility function in
R̃X .
Lemma 17. Let K be as in the statement of the Finiteness Axiom and let P ref (∆(X)) denote
the set of all preferences over ∆(X). Then
#{%∈ P ref (∆(X)) :% is represented by some u ∈ supp(ν) r {0}} = L,
where 1 ≤ L ≤ K.
Proof. We first show that L ≤ K. If not, then we can find utilities {u1 , ..., uK+1 } ⊆ supp(ν)
such that each ui is non-constant over X and ui 6≈ uj for all i 6= j. By Lemma 12, we can find
75
a menu A = {pi : i = 1, . . . , K + 1} ∈ A such that ui ∈ N + (A, pi ) for each i. Take any B ⊆ A
with |B| ≤ K. Then pi 6∈ B for some i.
Fix any sequences pin →m pi and Bn →m B. By definition, this means that there exists
r ∈ ∆(X) and αn → 0 such that pin = αn r + (1 − α)pi for all n, and that for each q ∈ B there
S
exists Bq ∈ A and βn (q) → 0 such that Bn = q∈B (βn (q)Bq + (1 − βn (q)){q}) for all n. Now,
B and each Bq are finite, and ui is linear with ui · pi > ui · q for all q ∈ B. Hence, there is
N such that for all n ≥ N , ui · pin > u · qn for all qn ∈ Bn . Thus, ui ∈ N + ({pin } ∪ Bn , pin )
for all n ≥ N . But since ui ∈ supp(ν) and N + ({pin } ∪ Bn , pin ) is an open set in F, the
definition of supp(ν) then implies that ν(N + ({pin } ∪ Bn , pin )) = 0 for all n ≥ N . But then
ρ(pin ; {pin } ∪ Bn ) = ν(N + ({pin } ∪ Bn , pin )) > 0 for all n ≥ N , contradicting Finiteness.
Next we show that L ≥ 1. Indeed, if L = 0, then for any A ∈ A with |A| ≥ 2 and for
any p ∈ A, we have (N (p, A) r {0}) ∩ supp ν = ∅. By Lemma 22 below, this implies that
ν(N + (p, A)) = 0 for any p ∈ A. But since ν represents ρ, ρ(p; A) = ν(N + (p, A)) for any p ∈ A,
P
so we have p∈A ρ(p; A) = 0, which is a contradiction.
F.4.4
Constructing the REU representation
Let %1 , . . . , %L denote all the preferences represented by some non-constant utility in supp(ν),
where by Lemma 17 we know that L is finite and L ≥ 1. For each i = 1, . . . , L, pick some
ui ∈ supp ν representing %i . For any u ∈ R̃X , let [u] := {u0 ∈ R̃X : u0 ≈ u}. By Lemma
12, we can find A := {p1 , . . . , pL } ∈ A such that ui ∈ N + (A, pi ) for all i = 1, . . . , L. Let
Bui := N + (A, pi ) for all i. By construction, [ui ] ⊆ Bui and Bui ∩ Buj = ∅ for j 6= i. Moreover,
by the definition of supp(ν), we have ν(Bui ) > 0 for each i, since Bui ∈ F is open and
ui ∈ Bui ∩ supp(ν) 6= ∅.
Let S := {u1 , . . . , uL } and define the function µ : S → [0, 1] by
µ(s) = ν(Bs ) for each s ∈ S.
We claim that µ defines a full-support probability measure on S. For this it remains to show
P
P
P
S
that s µ(s) = 1. Since s µ(s) = s ν(Bs ) = ν( s∈S Bs ), it suffices to prove the following
claim:
Lemma 18. ν(
S
s∈S
Bs ) = 1.
76
Proof. It suffices to prove that ν(R̃X r
S
s∈S
Bs ) = 0. Note that R̃X =
SL
i=1
N (A, pi ), since
A = {pi , . . . , pL }. Thus,
[
X
R̃ r
Bs ⊆
L
[
(N (A, pi ) r N + (A, pi )).
i=1
s∈S
By finite additivity of ν, this implies that
X
ν(R̃ r
[
Bs ) ≤
L
X
ν(N (A, pi ) r N + (A, pi )) = 0,
i=1
s∈S
where the last inequality follows from regularity of ν.
Next, we define a set function τs : F → R+ for each s ∈ S by setting
τs (V ) :=
ν(V ∩ Bs )
ν(Bs )
for each V ∈ F. Since ν(Bs ) > 0 for all s ∈ S, this is well-defined. Moreover, since ν is a
regular finitely additive probability measure on F, so is τs .
Note that for all A ∈ A and p ∈ ∆(X), {u ∈ R̃X : p ∈ M (M (A, s), u)} = N (M (A, s), p) ∈
F, so τs ({u ∈ R̃X : p ∈ M (M (A, s), u)}) is well-defined. The next lemma will allow us to
complete the representation:
Lemma 19. For each s ∈ S, A ∈ A, and p ∈ A,
ν(N (A, p)) =
X
µ(s)τs ({u ∈ R̃X : p ∈ M (M (A, s), u)}).
s∈S
Proof. Note first that for each s ∈ S, supp τs r {0} = [s]. To see that [s] ⊆ supp τs r {0},
consider any u ∈ [s] and any open V ∈ F such that u ∈ V . By Lemma 21 (iii), V is a finite
union of finite intersections of sets in N ∪ N + . Hence, since each element of N ∪ N + is closed
under positive affine transformations so is V . Thus, u ∈ V implies s ∈ V . But then V ∩ Bs ∈ F
is open and contains s, and hence ν(V ∩Bs ) > 0 since s ∈ supp ν. This proves u ∈ supp τs r{0}.
To see that supp τs r {0} ⊆ [s], consider any u 6= 0 such that u ∈
/ [s]. It suffices to show that
there exists an open V ∈ F such that u ∈ V and τs (V ) = 0. If u ≈ s0 for some s0 ∈ S r {s},
then V = Bs0 is as required since Bs0 ∩ Bs = ∅ and u ∈ Bs0 . If there is no s0 ∈ S r {s} such that
77
u ≈ s0 , then u ∈
/ supp ν. But then there exists an open V ∈ F such that u ∈ V and ν(V ) = 0,
so also τs (V ) = 0.
By Lemma 23 below, this implies that τs (N (A, p)) = τs (N (M (A, s), p)) for any A ∈ A and
p ∈ A. This implies that for any A ∈ A and p ∈ A
X
µ(s)τs ({u ∈ R̃X : p ∈ M (M (A, s), u)}) =
X
=
X
=
X
s∈S
µ(s)τs (N (M (A, s), p))
s∈S
µ(s)τs (N (A, p))
s∈S
ν(N (A, p) ∩ Bs )
s∈S
= ν(N (A, p) ∩
[
Bs )
s∈S
= ν(N (A, p),
where the last equality follows from Lemma 18.
For any s ∈ S = {u1 , . . . , uL }, we write Us := s. We claim that (S, µ, {Us , τs }s∈S ) is an REU
form representation of ρ. Indeed, by construction, Us is non-constant for all s, Us 6≈ Us0 for
any distinct s, s0 ∈ S, and µ is a full-support probability measure on S. Moreover, each τs is a
regular finitely additive probability measure on R̃X endowed with the algebra F. By standard
arguments (cf. Rao and Rao (2012)), we can extend τs to a regular finitely additive probability
measure on the Borel σ-algebra on R̃X . Finally, Lemma 19 and the fact that ν represents ρ
P
implies that for all A ∈ A and p ∈ A, we have ρ(p; A) = s∈S µ(s)τs (p, A), as required.
F.4.5
Continuity
Finally, suppose that ρ additionally satisfies Axiom 11 (Continuity). We show that each Us is
continuous.
Lemma 20. Suppose ρ is represented by the REU form (S, µ, {Us , τs }s∈S ) and satisfies Axiom 11 (Continuity). Then Us ∈ R̃X is continuous for all s.
Proof. Suppose for a contradiction that there exists some s0 ∈ S, x ∈ X, and sequence xn → x
such that limn Us0 (xn ) 6= Us0 (x). Restricting to an appropriate subsequence of {xn } if necessary,
we can assume that limn Us (xn ) exists (allowing for ±∞) for every s ∈ S.
78
Let S+ := {s ∈ S : limn Us (xn ) < Us (x)}, S− := {s ∈ S : limn Us (xn ) > Us (x)}, and
S0 := S r (S+ ∪ S− ) = {s ∈ S : limn Us (xn ) = Us (x)}. Since S is finite, there exists δ > 0 and
N such that for all n ≥ N , Us (xn ) < Us (x) − 2δ for all s ∈ S+ and Us (xn ) > Us (x) + 2δ for all
s ∈ S− . Let p = αδx + (1 − α)δxN , where we can choose α ∈ (0, 1) sufficiently close to 1 such
that
∀n ≥ N : Us (xn )
Note also that
Us (x)


< Us (p) − δ
if s ∈ S+

> Us (p) + δ
if s ∈ S− .


> Us (p)
if s ∈ S+

< Us (p)
if s ∈ S− .
(26)
(27)
Since S is finite and Us is non-constant for all s ∈ S, we can replace p with a mixture p0 that
puts large enough weight on p and small weight on an appropriate lottery q to ensure that p0
also satisfies properties (26) and (27) and additionally that for all s ∈ S, Us (p0 ) 6= Us (x). Then
there exists δ 0 > 0 and N 0 such that for all s0 ∈ S0 , either min{Us0 (xn ), Us0 (x)} > Us0 (p0 ) + δ 0 for
all n ≥ N 0 or min{Us0 (xn ), Us0 (x)} < Us0 (p0 ) − δ 0 for all n ≥ N 0 . Let S0− be the set of states in
S0 that satisfy the former inequality, and S0+ be the set of states in S0 that satisfy the latter
inequality.
Let m := |S|.
By Lemma 12, we can find distinct lotteries {q1 , ..., qm } such that
M ({q1 , ..., qm }, Usi ) = {qsi } for all si ∈ S. Define pi := (1 − )p0 + qi for each si ∈ S and
let A := {p1 , ..., pm , δx } and An := {p1 , ..., pm , δxn } for all n. By construction, we can choose
> 0 sufficiently small such that for all n ≥ max{N, N 0 }:



M (An , Usi ) = {pi } = M (A, Usi )





M (An , Us ) = {pi } and M (A, Us ) = {δx }
i
i


M (An , Usi ) = {δxn } and M (A, Usi ) = {pi }





M (A , U ) = {δ } and M (A, U ) = {δ }
n
si
xn
si
x
if si ∈ S0+
if si ∈ S+
if si ∈ S−
if si ∈ S0− .
By Lemma 24, this implies A, An ∈ A∗ for all n ≥ max{N, N 0 }. Moreover, by assumption,
there exists s0 ∈ S+ ∪ S− , say s0 = si . If si ∈ S+ (respectively, si ∈ S− ), then the above implies
that for all for all n ≥ max{N, N 0 }, ρ(pi ; An ) = µ(si ) > ρ(pi ; A) = 0 (respectively, ρ(pi ; An ) =
0 < µ(si ) = ρ(pi ; A)). Either way, limn ρ(pi |An ) 6= ρ(pi |A), contradicting Axiom 11.
79
F.5
Proof of Theorem 4: Necessity
Suppose that ρ admits an REU form representation (S, µ, {Us , τs }s∈S ). We show that ρ satisfies
Axiom 10. Observe first that for any finite Y ⊆ X with y ∗ ∈ Y , (S, µ, {Us Y , τs Y }s∈S )
constitutes an REU form representation of ρY , where Us Y denotes the restriction of Us to
Y and τs Y is given by τs Y (B) = τs (B × RXrY ) for any Borel set B on RY . Thus, by
Theorem S3 in Ahn and Sarver (2013), ρY satisfies Regularity, Linearity, Extremeness, and
Mixture Continuity.
To show that ρ satisfies Regularity, consider any p ∈ A ⊆ A0 . Pick a finite Y ⊆ X with
y ∗ ∈ Y such that A, A0 ∈ A(Y ). By definition, ρ(p; A) = ρY (p; A) and ρ(p; A0 ) = ρY (p; A0 ).
Hence, by Regularity for ρY , we have ρ(p; A) ≥ ρ(p; A0 ), as required. Similarly, we can show
that ρ satisfies Linearity, Extremeness, and Mixture Continuity by using the fact that for each
finite Y , each ρY satisfies these axioms.
To show that ρ satisfies Finiteness, let K := |S| and consider any A ∈ A. For each s ∈ S,
pick any qs ∈ M (A, Us ), and define B := {qs : s ∈ S}. Note that |B| ≤ K. If B = A, then
Finiteness is trivially satisfied. If B ( A, then pick any p ∈ A r B. We can pick a large
enough finite Y ⊆ X such that each Us is non-constant on Y and Us Y 6≈ Us0 Y for any distinct
s, s0 ∈ S. Let r ∈ ∆(Y ) be given by r(y) :=
1
|Y |
for each y ∈ Y . For each s ∈ Y , pick any
ys ∈ argmaxy∈Y Us (y). Note that Us (ys ) > Us (r). Define B n :=
n−1
p + n1 r. Then B n →m B and pn →m
n
Us ( n−1
q + n1 ys ) > Us (pn ) for each s ∈ S. Thus,
n s
pn :=
n−1
B
n
+ n1 {ys : s ∈ S} and
p. Moreover, for all large enough n, we have
ρ(pn ; {pn } ∪ B n ) = 0, proving Finiteness.
Thus, ρ satisfies Axiom 10. Suppose next that each Us is continuous. We show that ρ
satisfies Axiom 11. Consider any A ∈ A∗ and sequence {An } ⊆ A∗ such that An → A in the
Hausdorff topology. Suppose A = {p1 , ..., pm }.
Let B (pi ) denote the -neighborhood of pi in ∆(X) endowed with the Prokhorov metric.
Since A ∈ A∗ , Lemma 24 implies that for each s ∈ S, there is a unique index is ∈ {1, ..., m}
such that Us (pis ) > Us (pj ) for all j 6= is . Because each Us is continuous and S is finite, there
exists ¯ > 0 such that for all ≤ ¯, for all s, and for all q ∈ B (pis ) and q 0 ∈ B (pj ) with j 6= is ,
we have Us (q) > Us (q 0 ).
Fix any ≤ ¯. Since An → A, there exists N such that for all n ≥ N , we have An ⊆
Sm
j=1
B (pj ) and B (pj ) ∩ An 6= ∅ for all j = 1, . . . , m. Then by choice of , the representation
80
implies that for all n ≥ N and j = 1, . . . , m
ρ(B (pj ); An ) =
X
µ(s) = ρ(pj ; A).
s∈S:is =j
Thus, ρ(·; An ) → ρ(·; A) under the topology of weak convergence on ∆(∆(X)), as required.
F.6
Additional Lemmas for Section F
F.6.1
Properties of N (A, p) sets
Lemma 21. Fix any X 0 ⊆ X with y ∗ ∈ X. For any collection S, we let U(S) denote the set
of all finite unions of elements of S.
(i). If E ∈ N (X 0 ) (resp. E ∈ N + (X 0 )), then E c ∈ U(N + (X 0 )) (resp. E c ∈ U(N (X 0 )).
(ii). If E1 , E2 ∈ N (X 0 ) (resp. E1 , E2 ∈ N + (X 0 )), then E1 ∩ E2 ∈ N (X 0 ) (resp. E1 ∩ E2 ∈
N + (X 0 )).
(iii). F(X 0 ) is the set of all E such that E =
S
`∈L
M` ∩ N` for some finite index set L and
M` ∈ N (X 0 ), N` ∈ N + (X 0 ) for each ` ∈ L.
(iv). F(X 0 ) is the set of all E for which there exists a finite Y ⊆ X 0 with y ∗ ∈ Y and E Y ∈ F(Y )
such that E = E Y × RXrY .
Proof.
(i): If E = N (A, p) ∈ N (X 0 ), then E c =
S
q∈Ar{p}
N + ({p, q}, q) ∈ U(N + (X 0 )) if p ∈ A
0
and E c = R̃X ∈ U(N + (X 0 )) if p ∈
/ A. Similarly, if E = N + (A, p) ∈ N + (X 0 ), then E c =
S
0
c
X0
∈ U(N (X 0 )) if p ∈
/ A.
q∈Ar{p} N ({p, q}, q) ∈ U(N (X )) if p ∈ A and E = R̃
(ii): If N (A1 , p1 ), N (A2 , p2 ) ∈ N (X 0 ), then N (A1 , p1 ) ∩ N (A2 , p2 ) = N ( 21 A1 + 12 A2 , 21 p1 +
1
p)
2 2
∈ N (X 0 ). The same argument goes through replacing all instances of N with N + .
(iii): By standard results, F(X 0 ) can be described as follows: Let F0 (X 0 ) denote the set
of all elements of N (X 0 ) ∪ N + (X 0 ) and their complements. Let F1 (X 0 ) denote the set of
all finite intersections of elements of F0 (X 0 ). Then F(X 0 ) is the set of all finite unions of
elements of F1 (X 0 ). By part (i), F0 (X) = U(N (X)) ∪ U(N (X 0 )) is the collection of all finite
unions of elements of N (X 0 ) and of all finite unions of elements of N + (X 0 ). By part (ii),
81
F1 (X 0 ) = F0 (X) ∪ I(X 0 ), where I(X 0 ) consists of all finite unions of the form
S
`∈L
M` ∩ N` ,
0
where M` ∈ N (X 0 ) and N` ∈ N + (X 0 ) for each ` ∈ L. Note that R̃X ∈ N (X 0 ) ∩ N + (X 0 ), since
0
R̃X = NX 0 ({p}, p) = NX+0 ({p}, p) for any p ∈ ∆(X 0 ). Thus, F0 (X) = U(N (X)) ∪ U(N (X 0 )) ⊆
I(X). Hence, F1 (X) = I(X) = F(X).
(iv): Note first that for any NX 0 (A, p) ∈ N (X 0 ) (resp. NX+0 (A, p) ∈ N + (X 0 )) and any
0
finite Y ⊆ X 0 with y ∗ ∈ Y and A ∈ A(Y ), we have NX 0 (A, p) = NY (A, p) × RX rY (resp.
0
NX+0 (A, p) = NY+ (A, p) × RX rY ). Now fix any E ∈ F(X 0 ). By part (iv), we have a finite
S
index set L and M` ∈ N (X 0 ), N` ∈ N + (X 0 ) for each ` ∈ L such that E = `∈L M` ∩ N` .
By the first sentence, we can then pick a finite Y ⊆ X 0 with y ∗ ∈ Y such that for each `,
0
0
we have M` = M`Y × RX rY and N` = N`Y × RX rY , where M`Y ∈ N (Y ) and N`Y ∈ N + (Y ).
S
Then E = E Y × RXrY , where E Y := `∈L M`Y ∩ N`Y ∈ F(Y ). Conversely, if E Y ∈ F(Y ),
S
then by part (iv), E Y if of the form `∈L M`Y ∩ N`Y ∈ F(Y ) for some finite collection of
0
M`Y ∈ N (Y ) and N`Y ∈ N + (Y ). Then by the first sentence, M` := M`Y × RX rY ∈ N (X 0 ) and
S
0
0
N` = N`Y × RX rY ∈ N + (X 0 ), so E Y × RX rY = L`=1 M` ∩ N` ) ∈ F(X 0 ) by part (iv).
F.6.2
Properties of regular finitely additive probability measures on F
Lemma 22. Let ν be a regular finitely additive probability measure on F and suppose that
(N (p, A) r {0}) ∩ supp ν = ∅ for some A ∈ A and p ∈ A, where 0 denotes the unique constant
utility in R̃X . Then ν(N + (A, p)) = ν(N (A, p)) = 0.
Proof. Since (N (A, p) r {0}) ∩ supp ν = ∅, we have
N (A, p) r {0} ⊆ (supp ν)c :=
[
{V ∈ F : V open and ν(V ) = 0}.
Thus, for some possibly infinite index set I, there exists a family {Vi }i∈I , with Vi ∈ F open
and ν(Vi ) = 0 for each i such that
N (A, p) r {0} ⊆
[
Vi .
i∈I
We now show that there is a finite subset {i1 , . . . , in } ⊆ I such that
N (A, p) r {0} ⊆
n
[
j=1
82
Vij .
To see this, define C(A, p) := (N (A, p) ∩ [−1, 1]X ) r {0}. Note that since [−1, 1]X is compact
in RX (by Tychonoff’s theorem) and N (A, p) is closed in R̃X , C(A, p) is compact in the relative
S
topology on R̃X r {0}. Hence, since C(A, p) ⊆ N (A, p) r {0} is covered by i∈I Vi and each Vi
S
is open, it has a finite subcover nj=1 Vij .
S
We claim that N (A, p) r {0} is also covered by nj=1 Vij . To see this, consider any u∗ ∈
N (A, p) r {0}. We can find a finite Y ⊆ X such that y ∗ ∈ Y , u∗ Y is not constant, N (A, p) =
NY (A, p)×RXrY , and for each j = 1, . . . , n, Vij = ViYj ×RXrY for some ViYj ∈ F Y (see Lemma 21
(iv)).
Since Y is finite, there exists α > 0 small enough such that αu∗ (y) ∈ [−1, 1] for all y ∈ Y .
Define u ∈ R̃X by u Y = αu∗ Y and u(x) = 0 for all x ∈ X rY . Note that u ∈ N (A, p): Indeed,
u∗ ∈ N (A, p) = NY (A, p) × RXrY , u Y = αu∗ Y , and NY (A, p) is closed under positive scaling.
Moreover, u is not constant, since u∗ Y is not constant. Finally, u ∈ [−1, 1]X . This shows
S
u ∈ C(A, p). Since C(A, p) is covered by nj=1 Vij , there exists j such that u ∈ Vij = ViYj ×RXrY .
But note that ViYj is closed under positive scaling, since by Lemma 21 (iii) it is a finite union
of sets which are closed under positive scaling. Since u Y = αu∗ Y , this implies u∗ ∈ Vij .
S
The above shows that N (A, p) r {0} is covered by nj=1 Vij , and hence so is N + (A, p). But
since ν(Vij ) = 0 for all j = 1, . . . , n and ν is finitely additive, it follows that ν(N + (A, p)) = 0.
Moreover, by regularity of ν, this implies ν(N (A, p)) = 0.
Lemma 23. Suppose ν is a regular finitely additive probability measure on F and supp ν r
{0} = [u] for some u ∈ R̃X . Then for any A ∈ A and p ∈ A, we have ν(N (A, p)) =
ν(N (M (A, u), p)).
Proof. Fix any A ∈ A and p ∈ A.
Note first that for any q ∈ A,
q∈
/ M (A, u) ⇒ ν(N (A, q)) = 0.
(28)
Indeed, if q ∈
/ M (A, u), then ∅ = [u] ∩ N (A, q) = (N (A, q) r {0}) ∩ supp ν. But then Lemma 22
implies that ν(N (A, q)) = 0, as claimed.
Suppose now that p ∈
/ M (A, u). Then (28) implies that ν(N (A, p)) = 0. Moreover,
N (B, p) := ∅ if p ∈
/ B, so also ν(N (M (A, u), p)) = 0, as required.
83
Suppose next that p ∈ M (A, u). Then
[
N (A, p) ⊆ N (M (A, u), p) ⊆ N (A, p) ∪
N (A, q),
q∈ArM (A,u)
so that
ν(N (A, p)) ≤ ν(N (M (A, u), p)) ≤ ν(N (A, p)) +
X
ν(N (A, q)) = ν(N (A, p)),
q∈ArM (A,u)
where the last equality follows from (28).
This again shows that ν(N (A, p))
ν(N (M (A, u), p)), as required.
F.6.3
=
Menus without ties
Lemma 24. Suppose ρ is represented by the REU form (S, µ, {Us , τs }s∈S ). Then for any A ∈ A,
the following are equivalent:
(i). A ∈ A∗
(ii). For each s ∈ S, |M (A, us )| = 1
(iii). For each p ∈ A, N (A, p) ∩ {us : s ∈ S} = N + (A, p) ∩ {us : s ∈ S}.
Proof. The equivalence of (ii) and (iii) is immediate from the definitions. The proof that (i) is
equivalent to (ii) is entirely analogous to that of Lemma 13 and is thus omitted.
G
G.1
Proof of Proposition 1
“If ” directions:
DREU: Note first that because (Ut ), (αt ), and (βt ) are Ft -adapted and (Ft , Ut ) are simple, (i)
and (ii) imply (Ût ) is (F̂t )-adapted and (F̂t , Ût ) is simple. Consider any ht = (p0 , A0 , ..., pt , At ) ∈
84
Ht . Then
t
µ(C(h )) =
X
µ(FT (ω))µ
FT (ω)∈ΠT
=
t
\
!
{pk ∈ M (M (Ak , Uk ), Wk )}|FT (ω)
k=0
t
X Y
µ(Fk (ω) | Fk−1 (ω))µ ({Wk ∈ N (M (Ak , Uk (ω)), pk )}|Fk (ω))
Ft (ω)∈Πt k=0
=
t
X Y
µ̂(gk (Fk (ω)) | gk−1 (Fk−1 (ω)))µ̂ {Ŵk ∈ N (M (Ak , Uk (ω)), pk )}|gk (Fk (ω))
Ft (ω)∈Πt k=0
=
t
X Y
µ̂(F̂k (ω̂) | F̂k−1 (ω))µ̂ {Ŵk ∈ N (M (Ak , Ûk (ω̂)), pk )}|F̂k (ω̂)
F̂t (ω̂)∈Π̂t k=0
=
X
µ̂(F̂T (ω̂))
F̂T (ω̂)∈Π̂T
t
\
!
{pk ∈ M (M (Ak , Ûk ), Ŵk )} | F̂T (ω̂)
= µ̂(Ĉ(ht )),
k=0
where the second equality follows from regularity of (Wt ) and Ft -adaptedness of (Ut ), the
third equality follows from assumptions (i) and (iii), the fourth equality from the fact that
gt is a bijection and assumption (ii), the fifth equality from the regularity of (Ŵt ) and F̂t adaptedness of (Ût ), and the first and last equalities hold by definiton. Since D represents
ρ, this implies ρt (pt , At |ht−1 ) =
µ(C(ht ))
µ(C(ht−1 ))
=
µ̂(Ĉ(ht ))
µ̂(Ĉ(ht−1 ))
= µ̂[Ĉ(pt , At ) | Ĉ(ht−1 )]. Thus, D̂ is a
DREU representation of ρ as required.
Evolving utility: By the “if” direction for DREU, D̂ is a DREU representation of ρ.
Moreover, by assumption (v) and since (ut ), α0 , (βt ) are (Ft )-adapted, it follows that (ût ) is
(F̂t )-adapted. It remains to show that (D̂, (ût )) satisfies (2). From assumptions (ii), (iv), and
(v) it is immediate that ÛT = ûT . Moreover, for all t ≤ T − 1, and ω ∈ Ω, ω̂ ∈ gt (Ft (ω)), we
have
α0 (ω)Ût (ω̂)(z, At+1 ) = Ut (ω)(z, At+1 ) − βt (ω)
= ut (ω)(z) − βt (ω) + Eµ [ max Ut+1 (pt+1 ) | Ft (ω)]
pt+1 ∈At+1
= α0 (ω)ût (ω̂)(z) − Eµ [βt+1 |Ft+1 (ω)] + α0 Eµ̂ [ max Ût+1 (pt+1 ) | F̂t (ω̂)] + Eµ [βt+1 |Ft (ω)]
pt+1 ∈At+1
= α0 (ω) ût (ω̂)(z)Eµ̂ [ max Ût+1 (pt+1 ) | F̂t (ω̂)]
pt+1 ∈At+1
where the first equality follows from (ii) and (iv), the second from (2) for (D, (ut )), the third
85
from (i), (ii), (iv), and (v) (and the fact gt is a bijection), and the fourth by rearranging. Thus,
(D̂, (ût )) satisfies (2).
Gradual learning: By the “if” direction for DREU, D̂ is a DREU representation of ρ.
Since δ = δ̂ by (vi), it remains to show that (D̂, δ) satisfies (3). For all t ≤ T1 , (z, At+1 ), and
ω ∈ Ω, ω̂ ∈ gt (Ft (ω)), we have
α0 (ω)Ût (ω̂)(z, At+1 ) + βt (ω) = Ut+1 (ω)(z, At+1 )
= Eµ [UT (z)|Ft (ω)] + δEµ [ max Ut+1 (pt+1 ) | Ft (ω)]
pt+1 ∈At+1
= α0 (ω)Eµ̂ [ÛT (z)|F̂t (ω̂)] + Eµ [βT |Ft (ω)] + α0 (ω)δEµ̂ [ max Ût+1 (pt+1 ) | F̂t (ω̂)] + δEµ [βt+1 |Ft (ω)]
pt+1 ∈At+1
1 − δ T −t+1
Eµ [βT | Ft (ω)],
= α0 (ω) Eµ̂ [ÛT (z)|F̂t (ω̂)] + δEµ̂ [ max Ût+1 (pt+1 ) | F̂t (ω̂)] +
pt+1 ∈At+1
1−δ
where the first equality follows from (ii) and (iv), the second from (3) for (D, δ), the third
from (i), (ii), and (iv) (and the fact gt is a bijection), and the final equality from (vii). Since
βt (ω) =
G.2
1−δ T −t+1
E[βT
1−δ
| Ft (ω)] by (vii), this shows that (D̂, δ) satisfies (3).
“Only if ” directions:
DREU: Regularity of (Ŵt ) and the fact that each F̂t is generated by a finite partition Π̂t
is immediate from D̂ being a DREU representation. Throughout the proof, for any t and
Et = Ft (ω) ∈ Πt , we let Ut (Et ) denote Ut (ω) and likewise for Û ; this is well-defined by
adaptedness. We construct the sequence (gt , αt , βt ) inductively, dealing with the base case
t = 0 and the inductive step simultaneously.
Suppose t ≥ 0 and that we have constructed (gt0 , αt0 , βt0 ) satisfying (i)–(iii) for all t0 < t
(disregard the latter assumption if t = 0). If t > 0, fix any Et−1 = Ft−1 (ω ∗ ) ∈ Πt−1 , let
Êt−1 := gt−1 (Et−1 ), and let Πt (Et−1 ) := {Et = Ft (ω) ∈ Πt : Ft−1 (ω) = Et−1 } and Π̂t (Êt−1 ) :=
{Êt = F̂t (ω̂) ∈ Π̂t : F̂t−1 (ω̂) = Êt−1 }. As in the proof of Lemma 2, we can repeatedly
apply Lemma 12 to find a separating history for Et−1 = Ft−1 (ω ∗ ), i.e., a history ht−1 =
∗
(B0 , q0 , . . . , Bt−1 , qt−1 ) ∈ Ht−1
such that {ω ∈ Ω : qk ∈ M (Bk , Uk (ω))} = Fk (ω ∗ ) for all
k = 0, . . . , t − 1. By inductive hypothesis ht−1 is then also a separating history for Êt−1 . Thus,
by Lemma 13 (and the translation to S-based DREU in Proposition 6), C(ht−1 ) = Et−1 and
Ĉ(ht−1 ) = Êt−1 . If t = 0, then in the following we let Et−1 := Ω, Êt−1 := Ω̂, Πt (Et−1 ) := Π0 ,
Π̂t (Et−1 ) := Π̂0 , and we disregard all references to the separating history.
86
Enumerate Πt (Et−1 ) = {Eti : i = 1, . . . , m} with corresponding utilities Uti := Ut (Eti ) and
Π̂t (Êt−1 ) = {Êtj : j = 1, . . . , m̂} with corresponding utilities Û0j := Ût (Êtj ). Since (Ft , Ut ) and
0
(F̂t , Ût ) are both simple, we have µ(Eti ) > 0 for all i and Uti 6≈ Uti for i 6= i0 , and likewise
0
µ̂(Êtj ) > 0 for all j and Ûtj 6≈ Ûtj for j 6= j 0 . Note that for every j there exists a unique i(j)
i(j)
such Ut
≈ Ûtj . Indeed, if such an i(j) exists it is unique because all the Uti represent different
preferences. And the desired i(j) exists, since otherwise by Lemma 12, we can find a menu
Bt = {qti : i = 1, . . . , m} ∪ {q̂tj } such that M (Bt , Uti ) = {qti } for each i and M (Bt , Ûtj ) = {q̂tj }.
We can additionally assume (by replacing ht−1 with an appropriate mixture if need be) that
∗
(Bt ). Since D and D̂ both represent ρ, we obtain
ht−1 ∈ Ht−1
0 = µ[C(q̂tj , Bt )|Et−1 ] = ρt (q̂tj ; Bt |ht−1 ) = µ̂[Ĉ(q̂tj , Bt )|Êt−1 ) ≥ µ̂(Êtj |Êt−1 ) > 0,
j(i)
a contradiction. Similarly, for every i, there exists a unique j(i) such that Ût
j(i)
defining gt : Πt (Et−1 ) → Π̂t (Êt−1 ) by gt (Eti ) = Êt
≈ Uti . Thus,
yields a bijection. By construction,
Ut (Eti ) ≈ Ût (gt (Eti )) for all i, so we can find αt (Eti ) ∈ R++ and β(Eti ) ∈ R such that Ut (Eti ) =
αt (Eti )Ût (gt (Eti )) + β(Eti ). Defining α(ω) = α(Ft (ω)) and β(ω) = β(Ft (ω)) this yields Ft measurable maps αt , βt : Et−1 → R such that (ii) holds for all ω ∈ Et−1 . Moreover, applying
Lemma 12 again, we can find a menu Dt = {rti : i = 1, . . . , n} such that M (Dt , Uti ) = {rti }
for each i. Again, slightly perturbing the separating history ht−1 for Et−1 if need be, we can
∗
(Dt ). Then by the representation, µ(Eti |Et−1 ) = ρt (rti ; Dti |ht−1 ) =
assume that ht−1 ∈ Ht−1
µ̂(gt (Eti )|Êt−1 ) for all i, yielding (i).
∗
( 21 At + 12 Dt ).
To show (iii), consider any pt ∈ At , where we can again assume ht−1 ∈ Ht−1
Let Bti := {w ∈ RXt : pt ∈ M (M (At , Ut (Eti )), w)}. Note that by (ii), Bti = {w ∈ RXt :
pt ∈ M (M (At , Ût (gt (Eti ))), w)}. Thus, µ({Wt ∈ Bt }|Eti ) = µ(C(pt , At )|Eti ) and µ̂({Ŵt ∈
Bt }|gt (Eti )) = µ̂(Ĉ(pt , At )|gt (Eti )). But since D and D̂ both represent ρ and by choice of Dt ,
1
1 1
1
µ(Eti |Et−1 )µ[C(pt , At )|Eti ] = µ[C( pt + rti , At + Dt )|Et−1 ] =
2
2 2
2
1
1 i 1
1
ρt ( pt + rt ; At + Dt |ht−1 ) =
2
2 2
2
1
1 i 1
1
µ̂[Ĉ( pt + rt , At + Dt )|Êt−1 ] = µ̂(gt (Eti )|Êt−1 )µ̂[Ĉ(pt , At )|gt (Eti )],
2
2 2
2
which implies µ[C(pt , At )|Eti ] = µ̂[Ĉ(pt , At )|gt (Eti )], since by (i) we have µ(Eti |Et−1 ) =
µ̂(gt (Eti )|Êt−1 ). Thus, µ({Wt ∈ Bt }|Eti ) = µ̂({Ŵt ∈ Bt }|gt (Eti )), as required.
87
Finally, note that the collection {Πt (Et−1 ) : Et−1 ∈ Πt−1 } partitions Πt , and similarly
{Π̂t (Êt−1 ) : Êt−1 ∈ Π̂t−1 } partitions Π̂t . Thus, applying the above construction for every
Et−1 ∈ Πt−1 yields a bijection gt : Πt → Π̂t and Ft -measurable maps αt : Ω → R++ and
βt : Ω → R such that (i)–(iii) are satisfied.
Evolving utility: The “only if” part for DREU yields sequences (gt , αt , βt ) such that
(i)–(iii) are satisfied. It remains to show that (iv) and (v) hold. Throughout the proof, for
any Et = Ft (ω) ∈ Πt , we sometimes use Ut (Et ), αt (Et ), βt (Et ) to denote Ut (ω), αt (ω), βt (ω);
this is well-defined since Ut , αt , βt are Ft -measurable. We also let Ft−1 (Et ) := Ft−1 (ω); this is
well-defined since Ft (ω) = Ft (ω 0 ) implies Ft−1 (ω) = Ft−1 (ω 0 ), as (Ft ) is a filtration.
For (iv), fix any ω and t ≤ T − 1. Let Et := Ft (ω) and pick any At+1 , Bt+1 and zt . Then
Ut (Et )(zt , At+1 ) − Ut (Et )(zt , Bt+1 ) = αt (Et )(Ût (gt (Et ))(zt , At+1 ) − Ût (gt (Et ))(zt , Bt+1 ))
X
= αt (Et )
µ̂(Êt+1 |gt (Et ))[max Ût+1 (Êt+1 ) − max Ût+1 (Êt+1 )]
At+1
Bt+1
Êt+1 ∈Π̂t+1
X
= αt (Et )
µ̂(gt+1 (Et+1 )|gt (Et ))[max Ût+1 (gt+1 (Et+1 )) − max Ût+1 (gt+1 (Et+1 ))]
At+1
Et+1 ∈Πt+1
X
= αt (Et )
X
(29)
µ(Et+1 |Et )[max Ût+1 (gt+1 (Et+1 )) − max Ût+1 (gt+1 (Et+1 ))]
At+1
Et+1 ∈Πt+1
= αt (Et )
Bt+1
Bt+1
µ(Et+1 |Et )[max Ût+1 (gt+1 (Et+1 )) − max Ût+1 (gt+1 (Et+1 ))],
At+1
Et+1 s.t. Ft (Et+1 )=Et
Bt+1
where the first equality holds by (ii), the second equality follows from D̂ being an evolving
utility representation, the third equality from the fact that gt is a bijection, the fourth equality
from (i), and the fifth equality from the fact that µ(Ft+1 (ω 0 )|Et ) > 0 iff Ft (ω 0 ) = Et .
At the same time, we have
Ut (Et )(zt , At+1 ) − Ut (Et )(zt , Bt+1 )
=
X
µ(Et+1 |Et )[max Ut+1 (Et+1 ) − max Ut+1 (Et+1 )
At+1
Et+1 ∈Πt+1
X
=
µ(Et+1 |Et )αt+1 (Et+1 )[max Ût+1 (gt+1 (Et+1 )) − max Ût+1 (gt+1 (Et+1 ))]
Et+1 ∈Πt+1
=
X
Et+1 s.t. Ft (Et+1 )=Et
Bt+1
At+1
Bt+1
µ(Et+1 |Et )αt+1 (Et+1 )[max Ût+1 (gt+1 (Et+1 )) − max Ût+1 (gt+1 (Et+1 ))],
At+1
Bt+1
(30)
88
where the first equality follows from D being an evolving utility representation, the second
equality from (ii), and the third equality from the fact that µ(Ft+1 (ω 0 )|Et ) > 0 iff Ft (ω 0 ) = Et .
Combining (29) and (30), we have that for all At+1 and Bt+1 ,
X
µ(Et+1 |Et )αt (Et )[max Ût+1 (gt+1 (Et+1 )) − max Ût+1 (gt+1 (Et+1 ))]
At+1
Et+1 s.t. Ft (Et+1 )=Et
=
X
Bt+1
µ(Et+1 |Et )αt+1 (Et+1 )[max Ût+1 (gt+1 (Et+1 )) − max Ût+1 (gt+1 (Et+1 ))].
At+1
Et+1 s.t. Ft (Et+1 )=Et
Bt+1
(31)
0
)) for all disSince (F̂t , Ût ) is simple and gt is a bijection, Ût+1 (gt+1 (Et+1 )) 6≈ Ût+1 (gt+1 (Et+1
0
0
tinct Et+1 , Et+1
with Ft (Et+1 ) = Et = Ft (Et+1
). So by Lemma 12, we can find a menu
E
t+1
At+1 := {qt+1
: Ft (Et+1 ) = Et } such that for all Et+1 with Ft (Et+1 ) = Et we have
E∗
E
t+1
t+1
∗
M (At+1 , Ût+1 (gt+1 (Et+1 )) = {qt+1
}. Let Et+1
:= Ft+1 (ω) and let Bt+1 = At+1 r {qt+1
}. Then
∗
in (31), [maxAt+1 Ût+1 (gt+1 (Et+1 )) − maxBt+1 Ût+1 (gt+1 (Et+1 ))] 6= 0 iff Et+1 = Et+1
. Hence, (31)
∗
) = αt+1 (ω). Since this is true for all t ≤ T − 1, (iv) follows.
implies αt (ω) = αt (Et ) = αt+1 (Et+1
For (v), note that the claim for T is immediate from (ii) and the fact that UT = uT , ÛT = ûT .
Next, fix any ω ∈ Ω, ω̂ ∈ gt (Ft (ω), t ≤ T − 1, and (z, {pt+1 }). Then
Ut (ω)(z, {pt+1 }) = ut (ω)(z) + Eµ [Ut+1 (pt+1 )|Ft (ω)]
(32)
= ut (ω)(z) + α0 (ω)Eµ̂ [Ût+1 (pt+1 )|F̂t (ω̂)] + Eµ [βt+1 |Ft (ω)],
where the first equality follows from (D, (ut )) being an evolving utility representation and the
second equality from (i), (ii), (iv) (and the fact that gt is a bijection). At the same time, we
have
Ut (ω)(z, {pt+1 }) = α0 (ω)Ût (ω̂)(z, {pt+1 }) + βt (ω)
(33)
= α0 (ω)ût (ω)(z) + α0 (ω)Eµ̂ [Ût+1 (pt+1 )|F̂t (ω̂)] + βt (ω),
where the first equality follows from (ii) and (iv) and the second equality from (D̂, (ût )) being
an evolving utility representation. Combining (32) and (33) yields the desired claim.
Gradual learning: Since ρ admits a gradual learning representation and satisfies Condition 1, ρ satisfies Axiom 9. Letting α∗ be as in Axiom 9, we first show that δ =
1−α∗
α∗
= δ̂.
Indeed, combining the proofs of Proposition 6 (equivalence of gradual learning and S89
based gradual learning representations) and of the necessity direction of Theorem 3, it is
easy to see that for all t ≤ T − 1, ht , and qt , rt ∈ ∆(Xt ), we have qt %ht rt if and only
if Ut (ω)(qt ) ≥ Ut (ω)(rt ) for all ω ∈ C(ht ).
Fix any ω ∈ Ω and h0 = (A0 , p0 ) ∈ H0∗
such that {ω 0 : p0 ∈ M (A0 , U0 (ω 0 ))} = F0 (ω) (which exists by Lemma 12).
Then by
Lemma 13, C(h0 ) = F0 (ω). So by Condition 1 applied to h0 , we can find `, m, n ∈ ∆Z
such that U0 (ω)(`, n, . . . , n) 6= U0 (ω)(m, n, . . . , n). Moreover, by Axiom 9 applied to h0 , we have
U0 (ω)(α∗ `+(1−α∗ )m, α∗ `+(1−α∗ )m, n, . . . , n)) = U0 (ω)(`, m, n, . . . , n). Note that by (3) and
P
iterated expectations, we have U0 (`0 , . . . , `T ) = Tk=0 δ k u0 (ω)(`k ) for any stream of consumption
lotteries, (`0 , . . . , `T ), where u0 (ω) := E[UT |F0 (ω)]. Hence, by the preceding observations, we
have u0 (ω)(`) 6= u0 (ω)(m) and (1+δ) (α∗ u0 (ω)(`) + (1 − α∗ )u0 (ω)(m)) = u0 (ω)(`)+δu0 (ω)(m).
This implies δ =
Next, define
∗
1−α∗
. The same argument for D̂ shows δ̂ = 1−α
,
α∗
α∗
Ut0 := δ t−T Ut , Ût0 := δ̂ t−T Ût , u0t := δ t−T ut , û0t
E[UT |Ft (ω)], ût (ω̂) := E[ÛT |F̂t (ω̂)] for all ω, ω̂.
proving (vi).
:= δ̂ t−T ût , where ut (ω) :=
By the “if” part for DREU, D0 :=
(Ω, F ∗ , µ, (Ft , Ut0 , Wt )) and D̂0 := (Ω̂, F̂ ∗ , µ̂, (F̂t , Ût0 , Ŵt )) are still DREU representations of ρ.
Moreover, by (3), for each ω, we have UT0 (ω) = UT (ω) = uT (ω) = u0T (ω) and for all t ≤ T − 1
Ut0 (ω)(zt , At+1 ) = u0t (ω)(zt ) + δ t−T δE[ max Ut+1 (pt+1 ) | Ft (ω)]
pt+1 ∈At+1
0
= u0t (ω)(zt ) + E[ max Ut+1
(pt+1 ) | Ft (ω)].
pt+1 ∈At+1
Thus, (D0 , (u0t )) is an evolving utility representation of ρ and likewise for (D̂0 , (û0t )). By the“only
if” direction for evolving utility, there exists gt , αt , βt0 such that (D0 , (u0t )) and (D̂0 , (û0t )) satisfy
properties (i)–(v). Then (i) and (iii) hold for D and D̂ as well. Moreover, for any t, ω and
ω̂ ∈ gt (Ft (ω)), we have Ut0 (ω) = α0 (ω)Ût0 (ω̂) + βt0 (ω). But since Ut0 = δ t−T Ut and Ût0 = δ̂ t−T Ût
and δ = δ̂, this implies Ut (ω) = α0 (ω)Ût (ω̂) + βt (ω), where βt := δ T −t βt0 . Thus, gt , αt , βt satisfy
(i)–(iv) for D and D̂.
Finally, note that by (v) for (D0 , (u0t )) and (D̂0 , (û0t )), we have u0t (ω) = α0 (ω)û0t (ω̂) + βt0 (ω) −
0
E[βt+1
|Ft (ω)] for all t ≤ T − 1. Equivalently, ut (ω) = α0 (ω)ût (ω̂) + βt (ω) − δE[βt+1 |Ft (ω)].
Combining this with the fact that ut (ω) = Eµ [UT |Ft (ω)] = α0 (ω)Eµ̂ [ÛT |F̂t (ω̂)] + Eµ [βT |Ft (ω)]
and ût (ω̂) = Eµ̂ [ÛT |F̂t (ω̂)] yields βt (ω) = E[βT |Ft (ω)] + δE[βt+1 |Ft (ω)]. Then an easy inductive
argument shows βt (ω) =
1−δ T −t+1
E[βT |Ft (ω)],
1−δ
proving (vii).
90
H
Proofs for Section 6.2
H.1
Proof of Proposition 2
(ii)=⇒(i): Consider any (ht−1 , At , pt ), (ht−1 , At , qt ) ∈ Ht and pt+1 ∈ At+1 ∈ A∗t+1 (ht−1 , At , pt )∩
A∗t+1 (ht−1 , At , qt ) such that At and At+1 are atemporal, AZt+1 ⊆ AZt and pZt+1 = pZt =: p.
Let Ut (p) := {ut (ω) : ω ∈ C(ht−1 , At , pt )} and Ut (q) := {ut (ω) : ω ∈ C(ht−1 , At , qt )}.
Note that since At+1 features no ties, Lemma 13 implies C(pt+1 , At+1 ) = {ω : pt+1 ∈
M (At+1 , Ut+1 (ω))}, which since At+1 is atemporal is in turn equal to {ω
:
p
∈
M (AZt+1 , ut+1 (ω))}. Hence, by the representation,
ρt+1 (pt+1 ; At+1 |ht−1 , At , pt ) = µ({p ∈ M (AZt+1 , ut+1 )}|C(ht−1 , At , pt ))
≥ min µ({p ∈ M (AZt+1 , ut+1 )}|C(ht−1 ) ∩ {ut ≈ u}).
(34)
u∈Ut (p)
Likewise,
ρt+1 (pt+1 ; At+1 |ht−1 , At , qt ) = µ({p ∈ M (AZt+1 , ut+1 )}|C(ht−1 , At , qt ))
≤ 0max µ({p ∈ M (AZt+1 , ut+1 )}|C(ht−1 ) ∩ {ut ≈ u0 }).
(35)
u ∈Ut (q)
Pick u ∈ Ut (p) (respectively, u0 ∈ Ut (q)) which achieve the min (respectively max) in (34)
t−1
(respectively, in (35)). Let {u1t+1 , ..., um
, At , pt )∪C(ht−1 , At , qt ) & p ∈
t+1 } := {ut (ω) : ω ∈ C(h
Z
Z
Z
M (AZt+1 , ut+1 (ω))} and let D := co{u, u1t+1 , ..., um
t+1 }. Note that since At ⊇ At+1 and since pt =
p, we have p ∈ M (AZt+1 , u). Hence, C(ht−1 ) ∩ {ut ≈ u} ∩ {p ∈ M (AZt+1 , ut+1 )} = C(ht−1 ) ∩ {ut ≈
u} ∩ [D], and likewise C(ht−1 ) ∩ {ut ≈ u0 } ∩ {p ∈ M (AZt+1 , ut+1 )} = C(ht−1 ) ∩ {ut ≈ u0 } ∩ [D].
Thus,
µ({p ∈ M (AZt+1 , ut+1 )}|C(ht−1 ) ∩ {ut ≈ u}) = µ([D]|C(ht−1 ) ∩ {ut ≈ u}) ≥
µ([D]|C(ht−1 ) ∩ {ut ≈ u0 }) = µ({p ∈ M (AZt+1 , ut+1 )}|C(ht−1 ) ∩ {ut ≈ u0 }),
where
the
inequality
holds
by
(ii).
Combining
(34),
(35),
and
(36)
(36)
yields
ρt+1 (pt+1 ; At+1 |ht−1 , At , pt ) ≥ ρt+1 (pt+1 ; At+1 |ht−1 , At , qt ), as required.
(i)=⇒(ii): We prove the contrapositive. Suppose that for some u, u0 ∈ RZ and ht−1 with
91
C(ht−1 ) ∩ {ut ≈ u} =
6 ∅ 6= C(ht−1 ) ∩ {ut ≈ u0 }, and convex D ⊆ RZ with u ∈ D, we have
µ({ut+1 ∈ [D]}|C(ht−1 ) ∩ {ut ≈ u}) < µ({ut+1 ∈ [D]}|C(ht−1 ) ∩ {ut ≈ u0 }).
Let Ut+1 be the set of possible realizations of ut+1 conditional on event C(ht−1 ) ∩ {ω : ut ≈
u or u0 }. Likewise, Ut be the set of possible realizations of ut conditional on event C(ht−1 ).
Utility functions in these sets are all non-constant by Condition 2 (Uniformly Ranked Pair).
1
n
Z
Denote {u1t+1 , ..., um
t+1 } := Ut+1 ∩ [D] and {ût+1 , ..., ût+1 } := Ut+1 r [D]. Fix any p ∈ int∆(Z).
Note that for any j = 1, ..., n, ûjt+1 doe not belong to [co{u, u1t+1 , ..., um
t+1 }]. Thus, by Lemma
P
25, for each j = 1, ..., n, we can find vector wj ∈ RZ with z wj (z) = 0 such that ûjt+1 · wj >
0 ≥ uit+1 · wj , u · wj for any i = 1, ..., m. For each j, we construct q Z (j) ∈ ∆(Z) such that the
vector q Z (j) − pZ (in RZ ) is proportional to wj .37 Thus for each j = 1, ..., n and i = 1, .., m, we
have ûjt+1 · (q Z (j) − pZ ) > 0 ≥ uit+1 · (q Z (j) − pZ ), u · (q Z (j) − pZ ).
Take the lotteries p̄, p ∈ ∆Z from Condition 2 (Uniformly Ranked Pair). Since u·(p̄−p) > 0
and uit+1 · (p̄ − p) > 0 for each i = 1, ..., n, by setting p̃Z = (1 − )pZ + (p̄ − p), we have 0 >
uit+1 ·(q Z (j)− p̃Z ), u·(q Z (j)− p̃Z ) for any i = 1, ..., m and j = 1, ..., n. We pick sufficiently small
so that p̃Z is indeed well-defined (i.e., belongs to ∆Z) and also ûjt+1 · (q Z (j) − p̃Z ) > 0 holds for
each j = 1, ..., n. Finally, for this construction of the set of lotteries {p̃Z }∪{q Z (j) : j = 1, ..., n},
subject to perturbations of these, we can without loss require ut (q Z (j)) 6= ut (q Z (j 0 )) for each
ut ∈ Ut and distinct j, j 0 = 1, .., n while keeping the above strict inequalities.38
Construct an atemporal menu At+1 such that AZt+1 = {p̃Z } ∪ {q Z (j) : j = 1, ..., n}. Note
that since ûjt+1 · (q Z (j) − p̃Z ) > 0 > uit+1 · (q Z (j) − p̃Z ) for any j = 1, ..., n and i = 1, ..., m, we
have
µ(p̃Z ∈ M (ut+1 , AZt+1 )|C(ht−1 ), ut ≈ u) = µ({ut+1 ∈ [D]}|C(ht−1 ), ut ≈ u)
Let {[u1t ], ..., [ukt ]} denote the collection of equivalent classes of utilities in Ut , and assume
u ∈ [u1t ] without loss. By Lemma 12, we construct a collection of consumption lotteries {rZ (l) :
l = 1, ..., k} such that ult (rZ (l)) > ult (rZ (l0 )) any distinct l, l0 = 1, ..., k and any ult ∈ [ult ]. Then
we construct an atemporal menu At such that
AZt = {p̃Z } ∪ {q Z (j) : j = 1, ..., n} ∪ {p̃Z + 0 (rZ (l) − rZ (1)) : l = 2, ..., k}
37
Note that such a construction is possible because pZ is in interior of ∆(Z).
38
That is, ûjt+1 · (q Z (j) − p̃Z ) > 0 > uit+1 · (q Z (j) − p̃Z ), u · (q Z (j) − p̃Z ) for any j = 1, ..., n and i = 1, ..., m.
Note also that each ut ∈ Ut is non-constant.
92
This is well-defined if 0 is sufficiently small, as p̃Z is in interior of ∆(Z). M (p̃Z , AZt ) ∩ Ut ⊆ [u].
Note that we required ut (q Z (j)) 6= ut (q Z (j 0 )) for each ut ∈ Ut and distinct j, j 0 = 1, .., n.
Also, for any ut ∈ Ut , ut (p̃Z + 0 (rZ (l) − rZ (1))) is non-constant in 0 . Therefore, under an
appropriate value of small 0 , |M (ut , AZt )| = 1 for all ut ∈ Ut . This guarantees At ∈ A∗ (ht−1 )
(by Lemma 13). Since ult · (rZ (l) − rZ (1)) > 0 > u · (q Z (j) − p̃Z ), u · (rZ (l) − rZ (1)) for each
j = 1, .., n, l = 1, ..., k, and ult ∈ [ult ],
µ(p̃Z ∈ M (ut , AZt )|C(ht−1 )) = µ({ut ≈ u}|C(ht−1 )).
Let pt denote the lottery in At such that pZt = p̃Z . Since u 6≈ u0 , which implies u0 (p̃Z +
0 (rZ (l) − rZ (1)) > u0 (p̃Z ) for some l such that u0 ∈ [ult ], there is a lottery qt ∈ At different
from pt such that M (AZt , u0 ) = {qtZ }. Also, |M (ut+1 , AZt+1 )| = 1 for all ut+1 ∈ Ut+1 , and thus
At+1 ∈ A∗t+1 (ht−1 , At , pt ) ∩ A∗t+1 (ht−1 , At , qt ).
In case At 6∈ A(ht−1 ), by mixing ht−1 with an appropriate degenerate history, we choose
another history ĥt−1 that has the property that C(ht−1 ) = C(ĥt−1 ) and At ∈ A(ĥt−1 ). We have
AZt ⊇ AZt+1 , but by the above observations,
ρt+1 (pt+1 ; At+1 |ĥt−1 , At , pt ) = µ({ut+1 ∈ [D]}|C(ht−1 ), ut ≈ u)
< µ({ut+1 ∈ [D]}|C(ht−1 ), ut ≈ u0 )
= ρt+1 (pt+1 ; At+1 |ĥt−1 , At , qt ),
which contradicts Consumption Persistence.
Lemma 25. Take any finite X and finite set of non-constant utilities {U 1 , .., U m } ⊆ RX . Then
for any non-constant U ∈ RX , the following are equivalent:
(i). for any w ∈ RX such that
P
x
w(x) = 0,
[∀i = 1, ..., m, U i · w ≥ 0] ⇒ U · w ≥ 0
(ii). U ∈ [co{U 1 , ..., U m }]
Proof. The result follows from the utilitarian aggregation theorem, e.g., Theorem 2 in Fishburn
(1984).
93
H.2
Proof of Proposition 3
(ii)=⇒(i):
Take any t, history ht−1 and atemporal menus At ∈ At (ht−1 ) and At+1 ∈
A∗t+1 (ht−1 , At , pt ) such that AZt+1 ⊆ AZt . This implies the existence of a possible realization
of period t consumption utility ut = u conditional on C(ht−1 ) such that u(pZt ) > u(qtZ ) for all
qtZ ∈ AZt r {pZt }. By (ii) there exists a possible realization of period t + 1 consumption utility
ut+1 = u0 ≈ u conditional on C(ht−1 ) ∩ {ut ≈ u}. Note that, for any such u0 , AZt+1 ⊆ AZt ensures
Z
Z
u0 (pZt+1 ) > u0 (qt+1
) for all qt+1
∈ AZt+1 r {pZ }. This implies
ρt+1 (pt+1 ; At+1 |ht−1 , At , pt ) ≥ µ({ut+1 ≈ u}|C(ht−1 ), ut ≈ u) > 0.
(i)=⇒(ii): Suppose for a contradiction that there is some t and history ht−1 such that
µ({ut+1 ≈ u}|C(ht−1 ), ut ≈ u) = 0
for some u such that µ(ut ≈ u|C(ht−1 )) > 0. Let ([u1 ], ..., [um ]) denote the possible equivalence classes of consumption utilities that can realize in either period t or t + 1, and suppose
without loss that u ∈ [u1 ]. Note that they are all non-constant by Condition 1 (Consumption
Nondegeneracy). By applying Lemma 12 to consumption lotteries, we construct a collection
of consumption lotteries {pZ (i) : i = 1, ..., m} ⊆ ∆(Z) such that ui (pZ (i)) > ui (pZ (j)) for
any distinct pair i, j = 1, ..., m and ui ∈ [ui ]. Based on this we take an atemporal menu
At such that AZt = {pZ (i) : i = 1, ..., m}. Also we construct an atemporal menu At+1 such
that AZt+1 = {pZ (i) : i = 1, .., m}. Let pt and pt+1 denote the lotteries in At and At+1
such that pZt = pZt+1 = pZ (1). In this construction, without loss we require the support of
each lottery in At to contain an At+1 . By construction At ∈ A∗t and At+1 ∈ A∗t+1 (Lemma
13). Note in particular that, conditional on C(ht−1 ), ut ∈ M (At , pt ) only if ut ≈ u, so that
C(ht−1 , At , pt ) = C(ht−1 ) ∩ {ut ≈ u} since there is no tie at At .
In case At 6∈ A(ht−1 ), by mixing ht−1 with an appropriate degenerate history, we choose
another history ĥt−1 that has the property that C(ht−1 ) = C(ĥt−1 ) and At ∈ A(ĥt−1 )
But
ρt+1 (pt+1 ; At+1 |C(ĥt−1 , At , pt )) = µ({ut+1 ≈ u}|C(ht−1 ), ut ≈ u) = 0,
which contradicts consumption perseverance.
94
H.3
Proof of Corollary 1
first part, “only if”:
We consider the case m ≥ 2 as otherwise the desired statement trivially holds with any α.
First, take any distinct pair of indices i, j = 1, ..., m. By the consumption persistence and its
characterization (Proposition 2),
Mii = µ({u1 ∈ [ui ]}|u0 = ui ) ≥ µ({u1 ∈ [ui ]}|u0 = uj ) = Mji
(37)
(Note that by assumption both ui and uj arise with positive probability in period 0).
Next, take any distinct pair of indices i, j = 1, ..., m and let D = co{ui , uj }. Note that by
assumption there is no k 6= i, j such that uk ∈ [D]. By the consumption persistence and its
characterization (Proposition 2),
Mii + Mij = µ({u1 ∈ [D]}|u0 = ui ) = µ({u1 ∈ [D]}|u0 = uj ) = Mjj + Mji .
(38)
Then we first consider the case m = 2. Since 1 = M11 + M12 = M22 + M21 , we have
M11 − M21 = M22 − M12 := α, which is nonnegative by the above (37). Since the Markov
chain is irreducible, M21 , M12 > 0, which also ensures α < 1. By setting w(u1 ) =
w(u2 ) =
M12
,
1−α
M21
1−α
and
we obtain the desired form.
To consider the case m ≥ 3, take any distinct triple of indices i, j, k = 1, ..., m and let D0 =
co{ui , uj , uk }. By assumption there is no l 6= i, j, k such that ul ∈ [D0 ]. By the consumption
persistence and its characterization (Proposition 2),
Mii + Mij + Mik = µ({u1 ∈ [D0 ]}|u0 = ui ) = µ({u1 ∈ [D0 ]}|u0 = uj ) = Mjj + Mji + Mjk .
Combined with the above (38), this implies that Mik = Mjk for any i, j, k. Thus define βk := Mik
P
for each k. Here βk > 0, because otherwise i6=k Mik = 0, which implies that the Markov
chain is not irreducible. By the above (38), Mii − Mji = Mjj − Mij for any i, j, and thus
Mii − βi = Mjj − βj =: α for any i, j. Since the Markov chain is irreducible, α < 1 (as otherwise
Mii = 1 for any i). We set w(uj ) :=
βj
1−α
for each uj , which leads to the desired form.
first part, “if”:
Take any t and [u], [u0 ] ⊆ RZ that correspond to possible realizations of period t consumption
95
utilities. For any Et−1 ,
µ({ut+1 ∈ [D]}|Et−1 , ut ≈ u) = α + (1 − α)
X
w(uj )
uj ∈[D]
≥ αµ({ut ∈ [D]}|Et−1 , ut ≈ u0 ) + (1 − α)
X
w(uj )
uj ∈[D]
0
= µ({ut+1 ∈ [D]}|Et−1 , ut ≈ u ).
Thus ρ features choice persistence by Proposition 2.
Second part:
The proof follows immediately by applying Proposition 3 and thus is omitted.
References
Abdulkadiroglu, A., J. D. Angrist, Y. Narita, and P. A. Pathak (forthcoming):
“Research design meets market design: Using centralized assignment for impact evaluation,”
Econometrica.
Aguirregabiria, V., and P. Mira (2010): “Dynamic discrete choice structural models: A
survey,” Journal of Econometrics, 156(1), 38–67.
Ahn, D. S., and T. Sarver (2013): “Preference for flexibility and random choice,” Econometrica, 81(1), 341–361.
Angrist, J., P. Hull, P. A. Pathak, and C. Walters (forthcoming): “Leveraging lotteries for school value-added: Testing and estimation,” Quarterly Journal of Economics.
Anscombe, F. J., and R. J. Aumann (1963): “A definition of subjective probability,” The
annals of mathematical statistics, 34(1), 199–205.
Apesteguia, J., M. Ballester, and J. Lu (2017): “Single-Crossing Random Utility Models,” Econometrica.
Apesteguia, J., and M. A. Ballester (2017): “Monotone Stochastic Choice Models: The
Case of Risk and Time Preferences,” Journal of Political Economy.
Arcidiacono, P., and P. B. Ellickson (2011): “Practical methods for estimation of dynamic discrete choice models,” Annu. Rev. Econ., 3(1), 363–394.
Barberá, S., and P. Pattanaik (1986): “Falmagne and the rationalizability of stochastic
choices in terms of random orderings,” Econometrica: Journal of the Econometric Society,
pp. 707–715.
Block, D., and J. Marschak (1960): “Random Orderings And Stochastic Theories of
96
Responses,” in Contributions To Probability And Statistics, ed. by I. O. et al. Stanford:
Stanford University Press.
De Oliveira, H., T. Denti, M. Mihm, and M. K. Ozbek (2016): “Rationally inattentive
preferences and hidden information costs,” Theoretical Economics, pp. 2–14.
Dekel, E., B. Lipman, and A. Rustichini (2001): “Representing preferences with a unique
subjective state space,” Econometrica, 69(4), 891–934.
Dekel, E., B. L. Lipman, A. Rustichini, and T. Sarver (2007): “Representing Preferences with a Unique Subjective State Space: A Corrigendum1,” Econometrica, 75(2), 591–
600.
Deming, D. J. (2011): “Better schools, less crime?,” The Quarterly Journal of Economics, p.
qjr036.
Deming, D. J., J. S. Hastings, T. J. Kane, and D. O. Staiger (2014): “School choice,
school quality, and postsecondary attainment,” The American economic review, 104(3), 991–
1013.
Dillenberger, D., J. S. Lleras, P. Sadowski, and N. Takeoka (2014): “A theory of
subjective learning,” Journal of Economic Theory, 153, 287–312.
Epstein, L. G. (1999): “A definition of uncertainty aversion,” The Review of Economic
Studies, 66(3), 579–608.
Erdem, T., and M. P. Keane (1996): “Decision-making under uncertainty: Capturing
dynamic brand choice processes in turbulent consumer goods markets,” Marketing science,
15(1), 1–20.
Ergin, H., and T. Sarver (2010): “A unique costly contemplation representation,” Econometrica, 78(4), 1285–1339.
Falmagne, J. (1978): “A representation theorem for finite random scale systems,” Journal of
Mathematical Psychology, 18(1), 52–72.
Fishburn, P. (1970): Utility theory for decision making.
Fishburn, P. C. (1984): “On Harsanyi’s Utilitarian Cardinal Welfare Therem,” Theory and
Decision, 17, 21–28.
Fudenberg, D., P. Strack, and T. Strzalecki (2016): “Speed, Accuracy, and the Optimal Timing of Choices,” .
Fudenberg, D., and T. Strzalecki (2015): “Dynamic logit with choice aversion,” Econometrica, 83(2), 651–691.
Gilboa, I. (1987): “Expected utility with purely subjective non-additive probabilities,” Journal of mathematical Economics, 16(1), 65–88.
Gowrisankaran, G., and M. Rysman (2012): “Dynamics of Consumer Demand for New
Durable Goods,” mimeo.
Gul, F., P. Natenzon, and W. Pesendorfer (2014): “Random Choice as Behavioral
97
Optimization,” Econometrica, 82(5), 1873–1912.
Gul, F., and W. Pesendorfer (2001): “Temptation and self-control,” Econometrica, 69(6),
1403–1435.
(2006): “Random expected utility,” Econometrica, 74(1), 121–146.
Hendel, I., and A. Nevo (2006): “Measuring the implications of sales and consumer inventory behavior,” Econometrica, 74(6), 1637–1673.
Ke, S. (2016): “A Dynamic Model of Mistakes,” working paper.
Kennan, J., and J. Walker (2011): “The effect of expected income on individual migration
decisions,” Econometrica, 79(1), 211–251.
Kreps, D. (1979): “A representation theorem for” preference for flexibility”,” Econometrica:
Journal of the Econometric Society, pp. 565–577.
Kreps, D., and E. Porteus (1978): “Temporal Resolution of Uncertainty and Dynamic
Choice Theory,” Econometrica, 46(1), 185–200.
Krishna, R. V., and P. Sadowski (2014): “Dynamic preference for flexibility,” Econometrica, 82(2), 655–703.
Krishna, V., and P. Sadowski (2016): “Randomly Evolving Tastes and Delayed Commitment,” mimeo.
Lu, J. (2016): “Random choice and private information,” Econometrica, 84(6), 1983–2027.
Lu, J., and K. Saito (2016): “Random intertemporal choice,” Discussion paper, mimeo.
Luce, D. (1959): Individual choice behavior. John Wiley.
McFadden, D., and M. Richter (1990): “Stochastic rationality and revealed stochastic
preference,” Preferences, Uncertainty, and Optimality, Essays in Honor of Leo Hurwicz, pp.
161–186.
Miller, R. A. (1984): “Job matching and occupational choice,” The Journal of Political
Economy, pp. 1086–1120.
Natenzon, P. (2016): “Random choice and learning,” .
Norets, A. (2009): “Inference in dynamic discrete choice models with serially orrelated unobserved state variables,” Econometrica, 77(5), 1665–1682.
Pakes, A. (1986): “Patents as options: Some estimates of the value of holding European
patent stocks,” Econometrica, 54, 755–784.
Rao, K. P. S. B., and M. B. Rao (2012): Theory of Charges: A Study of Finitely Additive
Measures. Academic Press.
Rust, J. (1987): “Optimal replacement of GMC bus engines: An empirical model of Harold
Zurcher,” Econometrica, pp. 999–1033.
(1989): “A Dynamic Programming Model of Retirement Behavior,” in The Economics
of Aging, ed. by D. Wise, pp. 359–398. University of Chicago Press: Chicago.
98
(1994): “Structural estimation of Markov decision processes,” Handbook of econometrics, 4, 3081–3143.
Savage, L. J. (1972): The foundations of statistics. Courier Corporation.
Sweeting, A. (2011): “Dynamic product positioning in differentiated product markets: The
effect of fees for musical performance rights on the commercial radio industry,” mimeo.
Toussaert, S. (2016): “Eliciting temptation and self-control through menu choices: a lab
experiment on curiosity,” mimeo.
Wolpin, K. I. (1984): “An estimable dynamic stochastic model of fertility and child mortality,” The Journal of Political Economy, pp. 852–874.
99