An instrumental variable approach to dynamic models
Steven Berry and Giovanni Compiani
January 9, 2016
VERY PRELIMINARY1
1
Introduction
Empirical models of dynamic decision making play an important role in several branches of
applied microeconomics. Much of the existing empirical literature models unobserved shocks
as private information, independently distributed over time. This greatly simplifies the
estimation methods, as outlined in a series of influential papers including Rust (1987), Hotz
and Miller (1993) Bajari, Benkard and Levin (2007) (henceforth BBL), Pakes, Ostrovsky,
and Berry (2007) and Pesendorfer and Schmidt-Dengler (2008).
However, modeling econometric unobservables as purely private and identical has several
disadvantages. First, in many markets it may be unrealistic. In practice, datasets are
imperfect and frequently lack information on important variables. These variables are often
known to decision makers and market participants.
Furthermore, the assumption of independent private shocks renders the current state of
the agent (and/or industry) econometrically exogenous. This of course greatly aids estimation, but it runs contrary to applied practice in closely related problems. For example, in
1
This work is very preliminary and is not for distribution or quotation. This version is missing citations
and may contain errors
1
a model of dynamic capital formation the current state of the firm may include its capital
stock. Firms with observed high capital stocks, conditional on other observables, may be
systematically different from firms with low capital stocks conditional on the same observables. The high capital firms may, for example, be more productive and it is likely that
productivity is highly serially correlated over time, as in Olley and Pakes (1996). In that
paper, which considers production in the context of a background dynamic oligopoly model,
the serially correlated productivity shock renders capital econometrically endogenous.
In this paper, we study the identification of a broad set of dynamic models that feature unobservable shocks that are public information and serially correlated. These models
provide a complementary approach to independent private information models. In our correlated public information models, current states are econometrically endogenous because
they are partly the result of past decisions, which depended on past shocks, which are in
turn correlated with the present shocks that drive present decisions. Partly because of this
(realistic) endogeneity issue, the simple computational methods of papers like Hotz-Miller
and BBL cannot identify the parameters of the dynamic model.
To discuss identification, we rely on the results of Chesher and Rosen (2015). These
authors consider a “generalized instrumental variables” (GIV) approach to a broad set of
economic models. Like traditional instrumental variable (IV) models, model parameters in
the generalized IV approach are restricted via an assumption that unobserved shocks in the
model do not systematically vary with an observed set of instrumental variables. Unlike
classic IV models, in the GIV approach the model may not provide a map from model
parameters to a single vector of unobservables. For various reasons (including discreteness
and/or an incompletely specified model), a given parameter of the model may instead map
onto a non-trivial set of unobservables. Nonetheless, Chesher and Rosen show that the GIV
approach can still place useful bounds on model parameters.
To establish sharp bounds on parameters in our class of public information dynamic
models, we consider the sharp Chesher-Rosen bounds on the policy functions implied by
structural dynamic models. The identified parameter set of the dynamic models consists of
2
those parameters that generate policy functions in the Chesher-Rosen identified set.
Our instrumental variables identification approach is a complement to the “mixture
model” approach to serially correlated unobservables, as discussed, for example, in Arcidiacono and Miller (2011). These methods, following on Kasahara and Shimotsu (2009) and
others, use three or more periods of data and apply finite-mixture methods to identify persistent unobservables from the transition matrix of the observed data. These methods lean
heavily on assumptions about the order of Markov dependence for both observed and unobserved variables. The methods are both clever and useful, but they constrast to our methods
that make use of explicit instrumental variables. In some cases, instrumental variable arguments may have a stronger economic foundation as compared to the restrictions in finite
mixture models.
In addition, in the context of dynamic games, the finite mixture models typically assume
that while the persistent unobservables are public information, the agent and time-specific
shocks remain private information. In contrast, all of our unobservables are public information to the economic actors. Depending on context, either private or public information may
be more convincing. Certainly, in many cases, it is plausible that economic actors are aware
of more information than the econometrician.
To fix ideas, we first consider a simple example of a single-firm investment problem,
drawn from Ericson and Pakes (1995), Olley and Pakes (1996) and BBL. This example may
be particularly useful to those already familiar with two-step dynamic estimation methods.
A full description of our model follows in section 2.
1.1
Example: Single Firm Investment
Consider a simple firm investment model, in the style of the model underlying Olley and
Pakes (1996), as outlined in BBL (section 3.2.2). A wide class of models will result in an
investment policy function, at time t for firm j, of
ajt = σ(cjt , xjt , jt )
3
(1)
where ajt is investment (or some other continuous policy, such as price in a model of dynamic
pricing), xjt is capital stock (built up from past investments), cjt is an exogenous shifter and
jt represents the quantiles of a scalar shock to the profitability of investment. BBL assume
that the policy is strictly monotonically increasing in jt and that the jt are independent
over time.
The two-stage BBL approach to this problem is to first identify the policy function in
(1) directly from the data. Given the monotonicity in jt , one can invert the policy function
to obtain
jt = σ −1 (cjt , xjt , ajt ).
(2)
For example, following Matzkin (2003) the function σ −1 (cjt , xjt , ajt ) is then directly identified
as the inverse CDF of ajt given (cjt , xjt ).
As part of a second stage, BBL then “forward simulate” the dynamic investment problem,
using the policy function identified in the first stage to create the expected future value of
investment today. This forward simulation depends on the parameters of the structural
dynamic model. The BBL method then uses this forward simulation to place bounds on
parameter estimates.
The assumption that the shock t is uncorrelated over time plays a major role in the
BBL identification procedure, just as it does in the closely related Hotz and Miller (1993)
two-step estimation procedure and also in the Rust (1987) nested algorithm.
We alter the model to account for serial correlation, in t , over time. This naturally leads
to a correlation between the present shock t and the capital level xt , as current capital was
built up partly in response to past shocks to the profitability of investment. To deal with this
correlation, we propose the instrumental variables restriction that t is independent of some
vector of instruments zt . These instruments might be past values of ct (such as past demand
levels), which influenced past investments and are therefore correlated with current capital.
Alternatively, they could be measures of past policies that are still reflected in current capital
levels.
The IV approach suggests treating equation (2) as a quantile IV equation, as in Cher4
nozhukov and Hansen (2005), who give conditions for point identification under IV restrictions. Assume first that for a given data generating process the function in (2) is point
identified from the IV restrictions. In this case, that policy function can be directly used
in a BBL-style forward simulation and this forward simulation can again be used to place
bounds on the structural parameters of the dynamic model.
The discussion below will explicitly consider cases where the policy function is bounded
by the IV restrictions, but is not point-identified. Furthermore, there are important cases
(including discrete outcomes such as entry) where the policy function does not map the data
into a single point (or vector) as in (2). Often, the policy function instead maps into a set
of possible unobservables that are consistent with the data. This is exactly the Chesher and
Rosen (2015) case of set identification via generalized IV methods.
Throughout the paper we focus on the pure question of (set) identification, not estimation in finite samples. This serves to highlight the fundamental sources of data variation
that allow us to learn about model parameters, leaving the details of estimation to future
work. There is already a large literature on set identification that may provide a basis for
estimation strategies based more or less directly on the identification arguments of this paper. Where possible, we propose direct algorithms for placing sharp bounds on parameters;
these algorithms might have estimation analogs that could be explored in further research.
1.2
Additional Related Work
in progress
1.3
The Path Ahead
Through the first parts of the paper, we discuss instrumental variable methods for singleagent models. The IV methods are sufficiently different from earlier approaches that the
single-agent case is worthy of a complete treatment. In the single agent case, the public vs.
5
private information distinction is obviously not relevant and the focus is on a serially correlated shock (unobservable to the econometrician) that is independent of some instruments.
In the last section of the paper, we discuss the extension to dynamic games. Additional work
on dynamic games is a subject of our on-going research.
2
Single-agent model
We first consider the problem faced by a single agent who has to choose action a ∈ A, where
A ∈ R can be a finite, countable or uncountable set. This accommodates both discrete
and continuous actions. As mentioned above, when actions are continuous one can hope
to point identify the policy function, whereas discrete actions are typically associated with
partial identification. However, conceptually, the procedure we propose is the same. Let
(x, c) denote the observed states, where x ∈ X ⊂ RdX is a vector of dynamic variables
(i.e. their evolution depends on the current choice of a), while c ∈ C ⊂ RdC is a vector of
exogenous covariates (i.e. their evolution over time does not depend on the choice of a).
Further, let be a scalar state that is observed by the agent but not by the econometrician
whose distribution is assumed to be known up to the finite-dimensional parameter θF . Let
π (c, x, a, ; θπ ) denote the agent’s utility (or profit) function, which is assumed to be known
up to the finite-dimensional parameter θπ . Finally, let δ be the discount factor.2 Then, the
agent’s problem is associated with the following Bellman equation
V (c, x, ) = max [π (c, x, a, ; θπ ) + δE (V (c0 , x0 , 0 ) |c, x, , a, θF )]
a∈A
(3)
Note the parameters θπ and θF completely determine the dynamic problem, in the sense that,
given their value, one can obtain the value and policy functions through standard procedures
(e.g. value function iteration). We write θ ≡ (θπ , θF ) and θ ∈ Θ, for a parameter space Θ.
2
We follow a standard practice in the literature and assume that the discount factor is known or otherwise
identified by the econometrician.
6
Further, we denote the policy function resulting from (3) by
a = σ (c, x, )
(4)
We interpret as an unobserved cost and assume that the function σ is (weakly) decreasing
in . A more primitive sufficient condition for this is given by the following (see Bajari,
Benkard, and Levin (2007)).
Assumption 1. The profit function has decreasing differences in (a, ). If π is twice differentiable, this is equivalent to
∂2
π (c, x, a, ; θπ ) ≤ 0
∂a∂
The model is dynamic in that the distribution of x at time t+1 is allowed to be influenced
by the action a at time t. This implies that if is correlated over time, then xt and t will
also be correlated. This is because t−1 correlates with both t and at−1 (through equation
(4)), and at−1 in turn shifts the distribution from which xt is drawn.
As mentioned in the introduction, we depart from the existing literature in that we do
not require the unobservable to be independent over time. Furthermore, we place minimal
restrictions on the evolution of over time and on how it correlates with the endogenous
state. The next section discusses (partial) identification of the model parameters.
2.1
Identification
The identification argument consist of two steps. First, recent results on nonparametric
identification of nonseparable models with endogeneity are applied to obtain the identified
set for the policy function (4). Then, the restrictions given by the Bellman equation are used
to translate the identified set for the policy function into the identified set for the parameters
θπ and θF , which are typically the objects of interest in applications.
7
For the first step, we write
a1 = σ (c1 , x1 , 1 )
(5)
..
.
aT = σ (cT , xT , T ) ,
where T is the number of periods observed by the econometrician. Note that, unlike other
approaches in the literature, T could be as low as 2, although a higher T would typically
lead to more informative bounds on the parameters of interest.
The equations in (5) are nonseparable regression models with endogeneity. Therefore,
the results in Chesher and Rosen (2015) can be applied to obtain a sharp characterization of
the identified set for σ. To obtain nontrivial bounds, we assume an instrument z is available
such that z ⊥ (1 , ..., T ) and z has some correlation with the endogenous x.
Note that our approach applies the results in Chesher and Rosen (2015) to the system
of T equations in (5), as opposed to the single regression in (4). The key difference is that,
by looking at the system of equations, we are able to leverage the joint independence of the
vector of shocks (1 , ..., T ) and the instruments, whereas focusing on a single regression would
only exploit the marginal independence of each t and the instruments. In the simulations
below, we show that using the joint independence results in a substantially smaller identified
set for (θπ , θF ).
We now make an assumption that justifies using lagged values of c as instruments.
Assumption 2. The observed state c is strictly exogenous, i.e.
(ct )t≥1 ⊥ (t )t≥1
Under assumption 2, lagged values of c are independent of current and thus satisfy the
IV exogeneity condition. Further, by equation (3), only the current value of c affects the
8
agent’s choice today (hence lagged values of c do not enter the policy function)3 and thus
the IV exclusion restriction is also satisfied. Finally, lagged values of c affect lagged values
of a through the policy (4), which in turn shift the distribution of current x (since the model
is dynamic). This means that there is some correlation between x and lagged c built into
the model, i.e. lagged values of c are relevant instruments.
Note that the approach in Chesher and Rosen (2015) does not require the function σ to
be point-identified. Indeed, in the case where a takes a finite number of values, the function
is typically only partially identified. When a is continuously distributed, point-identification
can be achieved under certain high-level conditions (see Chernozhukov and Hansen (2005)),
but again we do not need to assume they are met in order for our set identification argument
to go through.
More precisely, let Σ denote the identified set for σ obtained in the first step. For every
θ ≡ θπ , θF , let σθ be the policy function implied by the Bellman equation (3) when θ = θ.
Further, for every σ, let Θ (σ) ≡ {θ ∈ Θ : σθ = σ}. The set Θ (σ) gives all the values of θ that
lead to the policy function σ. In other words, all θ ∈ Θ (σ) are observationally equivalent to
the econometrician. Thus, if the policy σ were known, the sharp identified set for θ would
be given by Θ (σ).
As pointed out above, the policy is only partially identified in general. Therefore, we
need to take the union over all functions in the identified set for the policy. Formally, we
have the following result.
Theorem 1. The sharp identified set for θ is
ΘID ≡
[
Θ (σ)
(6)
σ∈Σ
The set ΘID is sharp because it contains all the values of θ that (via the Bellman equation)
generate policy functions that cannot be rejected by the generalized instrumental variable
3
The first-order Markov assumption implicit in (3) can be relaxed to a k-order Markov assumption for
k ≥ 1. In this case, candidate instruments for t include (ct−k , ct−k−1 , ...).
9
conditions on the data generating process.
Therefore, given the set Σ, one can use Theorem 1 to characterize the sharp identified
set for θ. This is the second step of the identification argument. One way to operationalize
it is to perform value function iteration for each candidate value of θ and check whether the
resulting policy function belongs to the set Σ.
We now illustrate the identification argument with an example.
2.2
Illustration: monopolist dynamic discrete choice
Consider the problem faced by a monopolist who needs to choose how many stores to open
in a given market. Assume that each consumer has a linear inverse demand for the good
P = a − bQ
and that total demand is given by
r x
· s · Q,
min 1,
x
where x is the number of open stores, x is a positive integer and s is the market size. This
choice of functional form is meant to capture the idea that the quantity sold increases with
the number of open stores (e.g. more customers are reached), but at the same time the
returns to new stores are decreasing and eventually become zero for x ≥ x (e.g. because of
cannibalization across stores).
Assuming that the monopolist’s marginal cost is constant at mc, the first order conditions
for the static profit maximization problem lead to the following expression for variable profits
√
(a − mc)2
√
· s · x,
4b x
which we re-write as
α·s·
10
√
x
letting α =
(a−mc)2
√
.
4b x
Further, the monopolist incurs a fixed cost of β per open store. While β is a parameter
that is constant over time, is stochastic. Specifically, we assume t = eut , where ut =
(1 − ρ) µ + ρut−1 + νt and the innovations νt are iid N (0, 1). Note that ρ = 0 corresponds to
the case of no serial correlation in the shocks, whereas ρ 6= 0 (with ρ ∈ (−1, 1)) generates
dependence in the shocks over time. Fixed costs are modeled as stochastic and unobserved
to the econometrician because they incorporate things like rent and opportunity cost, which
are typically volatile and not directly available in the data.
Finally, opening a new store requires a sunk cost of γ. We assume that, at time t, the
monopolist both chooses how many stores will be open at time t + 1 and pays the associated
sunk costs. In our notation, at time t the monopolist has xt stores open and chooses at = xt+1 .
The market size s corresponds to the exogenous covariate (c in the notation above) and its
lagged values will be used as instrument.
In sum, the monopolist’s flow profit at time t is
πt = α · s ·
√
xt − βt xt − γ (at − xt ) I {at > xt }
(7)
We want to characterize the identified set for the parameters in the profit function, as
well as the parameters (µ, ρ) determining the distribution of , based on observing (x, a, s)
over time across a population of markets.
In the simulations, we use the following values for the parameters: α = 1, β = 0.5, γ =
1, δ = 0.95, µ = 0, ρ = 0.5. The maximum number of stores x is 2, so that the monopolist
chooses whether to have no stores, one store or two stores in the next period.
We generate data for T = 2 time periods and fix the market size s over the two periods
in any given market (but let it vary across markets). We generate lagged values of s to be
used as instruments for partial identification of the policy function. In the simulations, the
lagged s has a correlation of 0.86 and 0.53 with the endogenous variable x at time 1 and 2,
respectively.
11
2.2.1
Parametric variable profits
We first assume we know the correct functional form for the variable profits and focus on
the parameters (β, γ, µ, ρ).
First we compute the identified set Σ for the policy function by specializing the general
results in Chesher and Rosen (2015) to our setting.
Note that, in this model, the policy function is determined by a finite number of thresholds
(one for each possible number of stores, i.e. x + 1). Therefore, we can treat the policy in a
completely nonparametric fashion. Moreover, it can be shown that the sharp identified set Σ
is characterized by a finite (although large) number of inequalities and thus can be feasibly
computed. In obtaining the relevant inequalities, we use the fact that the policy function is
monotonically increasing in x, which leads to a substantial simplification.
Following the identification argument, for each candidate (β, γ, µ, ρ) in a grid, we compute
the corresponding policy function through value function iteration. Then, we check whether
this policy function lies in the identified set Σ. This step is facilitated by the fact that Σ
is convex and thus we can test whether a point lies in it by using its support function (see,
e.g., Beresteanu, Molchanov, and Molinari (2011)).
Figure 1 shows the projection of the identified set for β and γ obtained using joint
independence of (1 , 2 ) and the instrument s. The blue dots correspond to points in the
identified set, while the magenta dot indicates the true parameter value. One can see that
the bounds rule out negative values for the fixed and sunk costs and, especially for the latter,
are informative.
Similarly, Figure 2 shows the projection of the identified set for µ and ρ obtained using
joint independence. While the bounds on µ are wide, those on the key correlation parameter
ρ are narrow.
12
Figure 1: Projection of identified set for β, γ: joint independence
13
Figure 2: Projection of identified set for µ, ρ: joint independence
14
We also repeat the same exercise imposing only marginal independence of 1 and 2 with
respect to the instrument s. In practice, this means we apply the results in Chesher and
Rosen (2015) to the single equation model (4) (with one scalar unobservable) as opposed to
the system of equations (5), which has T = 2 unobservables.
Figure 3 shows the projection of the identified set for β and γ, while Figure 4 shows the
projection of the identified set for µ and ρ. One can see that not imposing joint independence
results in a significantly larger identified set for the parameters of interest.
15
Figure 3: Projection of identified set for β, γ: marginal independence
16
Figure 4: Projection of identified set for µ, ρ: marginal independence
17
3
Games
The approach discussed in section 2 can be extended to models of dynamic strategic interaction. Specifically, we consider dynamic games of complete information and allow for serially
correlated unobservables. Let J be the number of players. Then agent j 0 s policy is given by
aj = σj (c, x, )
(8)
where, as before, c denotes exogenous observable covariates and x is a potentially endogenous
covariate. On the other hand, ≡ (1 , ..., J ) is now the full vector of common knowledge
shocks unobserved to the econometrician.
As in the single agent case, identification comes from combining two types of restrictions:
the ones given by the Chesher and Rosen (2015) inequalities and the ones given by the model
via the Bellman equations. One way to operationalize this in games is to jointly search over
the profit parameter space and the space of policy functions (perhaps after parameterizing
(8) in a flexible way) and check whether any given candidate profit parameter and policy
combination is consistent with the two sets of restrictions mentioned above. Additional
assumptions may simplify the task. For example, if one assumes that the game is symmetric,
i.e. that all players have the same policy, then one only needs to check whether, given a
candidate profit parameter value and given that opponents play according to σ, the function
σ solves the Bellman equation for any given player. The Chesher and Rosen restrictions
(provided an instrument is available) would then help restrict the set of functions σ over
which one needs to search.
We are currently working on making this verbal description operational and illustrating
it by simulation.
18
References
Arcidiacono, P., and R. A. Miller (2011): “Conditional choice probability estimation
of dynamic discrete choice models with unobserved heterogeneity,” Econometrica, 79(6),
1823–1867.
Bajari, P., L. Benkard, and J. Levin (2007): “Estimating dynamic models of imperfect
competition,” Econometrica, 75(5), 1331–1370.
Beresteanu, A., I. Molchanov, and F. Molinari (2011): “Sharp identification regions
in models with convex moment predictions,” Econometrica, 79, 17851821.
Chernozhukov, V., and C. Hansen (2005): “An IV Model of Quantile Treatment Effects,” Econometrica, 73, 245–261.
Chesher, A., and A. Rosen (2015): “Characterizations of identified sets delivered by
structural econometric models,” working paper CWP 63/15, Cemmap, Institute for Fiscal
Studies UCL.
Ericson, R., and A. Pakes (1995): “Markov Perfect Industry Dynamics: A Framework
for Empirical Work,” Review of Economic Studies, 62, 53–82.
Hotz, J., and R. A. Miller (1993): “Conditional Choice Probabilites and the Estimation
of Dynamic Models,” Review of Economic Studies, 60, 497–529.
Kasahara, H., and K. Shimotsu (2009): “Nonparametric Identification and Estimation
of Finite Mixture Models of Dynamic Discrete Choices,” Econometrica, 77(1), 135–175.
Matzkin, R. L. (2003): “Nonparametric Estimation of Nonadditive Random Functions,”
Econometrica, 71(5), 1339–1375.
Olley, S. G., and A. Pakes (1996): “The Dynamics of Productivity in the Telecommunications Equipment Industry,” Econometrica, 64(6), 1263–97.
19
Pakes, A., M. Ostrovsky, and S. Berry (2007): “Simple Estimators for the Parameters
of Dynamic Games, with Entry/Exit Examples,” RAND Journal of Economics, 38(2),
373–399.
Pesendorfer, M., and P. Schmidt-Dengler (2008): “Asymptotic Least Squares Estimators for Dynamic Games,” The Review of Economic Studies, 75(3), 901–928.
Rust, J. (1987): “Optimal Replacement of GMC Bus Engines: An Empirical Model of
Harold Zurcher,” Econometrica, 55(5), 999–1033.
20