GLOBAL REGULARITY AND FLEXIBILITY OF DEMAND SYSTEMS IN THE
PRESENCE OF NON-NEGATIVITY CONSTRAINTS: A THEORETICAL AND
EMPIRICAL ANALYSIS
Nitin Mehta
Rotman School of Management
University of Toronto
105 St. George Street, Toronto, ON M5S 3E6, Canada
Email: [email protected]
Phone: (416) 978 4961
Sept 2014
______________________________________________________________________________
Nitin Mehta is an Associate Professor of Marketing at University of Toronto. This research was funded by the SSHRC
research grant # 491084. The author is indebted to Arthur van Soest for providing useful guidance. The author also
thanks Tat Chan, Ig Horstmann, David Soberman, Andrew Ching, Jian Ni, Milind Dawande and the seminar
participants at University of British Columbia, University of Alberta and Johns Hopkins University for their useful
comments. All omissions and errors are the responsibility of the author.
1
GLOBAL REGULARITY AND FLEXIBILITY OF DEMAND SYSTEMS IN THE
PRESENCE OF NON-NEGATIVITY CONSTRAINTS: A THEORETICAL AND
EMPIRICAL ANALYSIS
ABSTRACT
A vexing challenge when using the utility-maximization framework to estimate consumers’ decisions
on which set of goods to purchase and how much quantity to buy, is obtaining a functional form of
the utility that satisfies three criteria: tractability, flexibility and global-regularity. Flexibility refers to
the ability of utility function to impose minimal prior restrictions on demand elasticities. Global
regularity refers to the ability of utility function to satisfy regularity properties required by economic
theory in the entire feasible space of variables. The tractable utility functions used in prior literature
are either inflexible which could yield inaccurate estimates of underlying demand elasticities and
thereby inaccurate results in policy simulations, or do not satisfy global regularity which can result in
invalid expressions of likelihood and invalid policy simulations.
The candidates that can potentially satisfy all three criteria are flexible functional forms of
indirect utilities. The issue with flexible functional forms is that prior restrictions need to be imposed
on their parameters so that they satisfy global-regularity. To have minimal impact on their flexibility,
these restrictions should be necessary and sufficient conditions (i.e., the least restrictive conditions
on the parameter space) that ensure global-regularity. However, only sufficient conditions have been
proposed till date, which destroy their flexibility. We tackle this problem by deriving necessary and
sufficient conditions for global-regularity of Basic Translog utility, a widely used flexible functional
form. We show that as compared to alternatives used in the literature, our proposed demand system
(a) imposes much fewer restrictions on demand elasticities (in terms of allowing for inferior/normal
goods and complementarity/substitutability), and (b) yields better model fit, more accurately
captures the true demand elasticities i and yields substantially different results in policy simulations.
Key Words: utility theory, global regularity, complementarity/substitutability, inferior/normal goods
non-negativity constraints, flexible functional forms, Translog
1
SECTION 1: INTRODUCTION
With the availability of disaggregate data of consumer demand, there has been a splurge of research that
has attempted to model how consumers make purchase decisions when faced with multiple goods that
can be complements or substitutes of each other. These purchase decisions are characterized by two
simultaneous choices, where the first choice pertains to which set of goods to purchase and the second
choice pertains to how much quantity to buy for each of the purchased goods. The focus of this paper
is the stream of research that estimates these decisions using a utility maximization framework. Such
a framework requires deriving the demand equations of the purchased and non-purchased goods
from the maximization of consumer’s utility subject to the budget and non-negativity constraints.
Despite its theoretical appeal, the usefulness of this framework has been hampered by econometric
and analytical difficulties. In what follows, we first provide an overview of the challenges in this
framework and the goal of this paper. Following that, we explain each of these in greater detail.
A vexing challenge faced by researchers who have used the utility maximization framework
is obtaining a functional form of the utility that satisfies three criteria: tractability, flexibility and
global regularity. Tractability refers to the ability of the utility function to yield closed form solutions
of the demand equations for purchased and non-purchased goods. Flexibility refers to the ability of
utility function to impose minimal prior restrictions on demand elasticities. For instance, regarding
cross-price elasticities, flexibility requires the utility function to allow for complementarity as well as
substitutability between goods; and regarding expenditure elasticities, flexibility requires the utility
function to allow for normal as well as inferior goods. Global regularity refers to the ability of utility
function to satisfy regularity properties required by economic theory in the entire feasible space of
variables in the utility maximization, where the variables include prices, expenditure, quantities etc.
The tractable functional forms of utilities used in prior literature are either (a) inflexible, and thus
could yield inaccurate estimates of underlying demand elasticities in the data and thereby inaccurate
2
results in policy simulations, or (b) do not satisfy global regularity, which as we will explain later in
this section, could result in invalid expressions of the likelihood and invalid policy analysis.
The goal of this paper is to resolve this longstanding problem by constructing an appropriate
demand specification in a multi-good choice context that satisfies all three criteria. Such a demand
system will thus be easy to estimate, will yield valid expressions of the likelihood, will accurately
capture the underlying elasticities and thereby facilitate meaningful policy analysis. This completes
the overview of the challenges and our goal. We next discuss the utility maximization framework,
elaborate on the challenges and the specific tasks that we undertake to overcome those challenges.
1.1 Overview of the Utility Maximization Framework and the Challenges
In the utility maximization framework, two approaches are used to derive the demand equations of
the purchased and non-purchased goods for a given observation. The first is the primal approach in
which we specify the functional form of the random direct utility, and derive the Kuhn-Tucker
conditions that yield the demand equations. The second is the dual approach in which we specify the
functional form of the random indirect utility and derive the demand equations in a manner that
follows directly from the Kuhn-Tucker conditions. The choice of which approach to use depends
on the extent to which the functional form of the direct or indirect utility satisfies the criteria of
tractability, global regularity and flexibility. In what follows, we discuss the importance of these
criteria, the desirability of the two approaches based on these criteria, and the challenges therein.
The importance of tractability and flexibility is straightforward. Regarding global regularity, if
one uses the primal approach, then it requires the direct utility to satisfy regularity properties of strict
quasi-concavity, differentiability and monotonicity in the entire feasible space of variables. And if
one uses the dual approach, then global regularity requires the indirect utility to be such that its
corresponding direct utility satisfies the aforementioned properties in the entire feasible space.
Satisfying global regularity is important for the following two reasons. First, it ensures validity of
3
policy simulations - if the utility function fails to satisfy regularity properties in the space of variables
where simulations are performed, the simulations will be invalid since they will be inconsistent with
utility maximization behavior. Second, it ensures validity of the likelihood. To see why,1 note that the
observational likelihood consists of the joint distribution of observed quantities. To derive this joint
distribution, we derive the Kuhn Tucker conditions in terms of random errors and quantities, and
use transformation of variables from random errors to the quantities. This transformation yields a
‘valid’ joint distribution of quantities as long as the Kuhn Tucker conditions yield unique solutions
of quantities for each given set of random errors. In this regard, Van Soest et al. (1993) have shown
that when utility maximization is subject to non-negativity constraints, the Kuhn Tucker conditions
can suffer from the ‘coherency problem’, which is that they may not yield unique solutions of
quantities. Satisfying global regularity helps deal with this problem. This is because if the direct utility
is strictly quasi concave (or if the indirect utility is such that its corresponding direct utility satisfies
this property) at all values of variables in the data, then from Lagrange theory, the Kuhn Tucker
conditions will yield unique solutions of quantities, which ensures validity of the likelihood.
Given the importance of the three criteria, we next discuss the desirability of primal and dual
approaches. If we were to use the primal approach, we would have to restrict ourselves to additively
separable functional forms of direct utilities (e.g. LES, CES) in order to get tractable solutions of the
demand equations. Such additive separable functional forms impose strong restrictions on demand
elasticities. For instance, they restrict all pairs of goods to be substitutes and only allow for normal
(and not inferior) goods. The dual approach does not suffer from this problem since flexible
functional forms (e.g., AIDS, Basic Translog) lie in the realm of indirect utilities. These are second
order approximations of an arbitrary indirect utility, which are tractable and can achieve any arbitrary
demand elasticities as long their parameter space is unrestricted.
1 Note that we discuss the second reason in the context of the primal approach with non-negativity constraints. The same rationale
holds true for the dual approach with non-negativity constraints as it is derived from Kuhn Tucker conditions in the primal approach.
4
However flexible functional forms present another problem in that they only satisfy global
regularity in a restricted space of parameters. Thus if we estimate a flexible functional form without
imposing any prior restrictions of global regularity on its parameter space, and if the estimates we get
are such that regularity properties are violated at even one observation in the data, we can run into
coherency problems. And even if we do not, there is no guarantee that the parameter estimates will
be such that the regularity properties are satisfied outside the realm of the data where simulations are
performed. Thus to deal with these problems, it is crucial to impose prior restrictions on the
parameter space while estimating the flexible functional form so that it satisfies global regularity. To
have minimal impact on flexibility, these restrictions should be ‘necessary and sufficient’ conditions
(i.e., the least restrictive conditions on the parameter space) that ensure global regularity of the
flexible functional form. However, the parametric restrictions that have been proposed to date vis-àvis flexible functional forms have only been ‘sufficient’ conditions that impose strong restrictions on
elasticities. For instance, they restrict most of the goods to be substitutes, restrict expenditure
elasticities of all goods to one and restrict cross price effects between two goods to be symmetric.
The above discussion presents us with the following picture of where things stand. Prior
papers that have used the utility maximization framework have followed three routes. The first route
entails using the primal approach with additively separable direct utilities (e.g., Kao et al. 2001; Du
and Kamakura 2008). The second route entails using the dual approach with only sufficient
conditions for global regularity imposed on parameters of a flexible functional form (e.g. Lee and
Pitt 1987; Chakir et al. 2004; Song and Chintagunta 2007). The third route entails using the dual
approach in which one estimates a flexible functional form without imposing parametric restrictions
that ensure its global regularity. The first two routes are inflexible and thus could yield inaccurate
estimates of underlying elasticities and inaccurate results in policy simulations. The third route may
not satisfy global regularity and thus carries a risk of running into coherency problems and invalid
5
policy simulations. The desirable route that satisfies all three criteria is the dual approach in which
we impose necessary and sufficient conditions on parameters of a flexible functional form so that it
is globally regular. However it has not been accomplished to date. This sets the stage for what we
hope to achieve. Our goal is to derive necessary and sufficient conditions for global regularity of
Basic Translog (indirect) utility. To the best of our knowledge, it is the only flexible functional form
that yields tractable solutions when the utility maximization is subject to non-negativity constraints.
1.2 Research Objectives
To achieve this goal, we have three objectives. The first is to formally derive necessary and sufficient
conditions for global regularity of Basic Translog utility. The second is to formally derive restrictions
on elasticities implied by Basic Translog utility when we impose necessary and sufficient conditions,
and compare them with those implied by two alternatives used in the literature: (a) additively
separable direct utilities and (b) Basic Translog utility when we impose only sufficient conditions for
global regularity. We show that necessary and sufficient conditions impose much fewer restrictions
on elasticities as compared to (a) and (b). The third is to examine using simulated and scanner data,
the consequences of having a demand system that imposes fewer restrictions on elasticities. We find
that Basic Translog utility with necessary and sufficient conditions yields better model fit, more
accurately captures the true elasticities in the data and yields substantially different results in policy
simulations compared to (a)-(b). Unlike (a), it allows for complementarity and inferiority, and does
not underestimate the substitutabilities. Unlike (b), it accurately captures true expenditure elasticities,
does not underestimate the complementarities, and allows for asymmetries in cross price effects.
The rest of the paper is organized as follows. In section 2, we discuss the dual approach with
non-negativity constraints. In section 3, we derive the necessary and sufficient conditions for global
regularity of Basic Translog utility. In section 4, we derive the implications of these conditions on
the flexibility of the Basic Translog utility. In section 5, we discuss the empirical results.
6
SECTION 2: DUAL APPRAOCH WITH NON-NEGATIVITY CONSTRAINTS
To facilitate a better understanding of global regularity in the context of Basic Translog utility, we
first briefly discuss the dual approach for specifying the demand equations when we have nonnegativity constraints on quantities, and the requirements for global regularity using a general form
of an indirect utility. These follow directly from Lee and Pitt (1986) and Van Soest et al. (1993).
Consider the case where there are c=1..M goods and the consumer’s problem is to decide which
goods to purchase, and how much quantity to buy of each purchased good. Let pc be the market price
and qc be the quantity of good c. Let V(π,y; θ, ε) be the indirect utility, which is a function of the total
expenditure y, price arguments of M goods πª {πc} cM1 (whose specifications we will explain shortly),
parameters θ and random errors ε in the support Ωε. Given the preliminaries, we start with the
demand equations. For expositional ease, we will work with budget shares of goods instead of
quantities (where the two are related via sc=pcqc/y).
Consider a general observation in which the consumer purchases goods c=m+1..M with
observed (positive) budget shares as {sc} cM m 1 and does not purchase goods c=1..m with shares as sc=0
"c=1..m. For this observation, the demand equations consist of (a) the stochastic specifications of
observed budget shares of the purchased goods c=m+1..M and (b) the regime conditions of the nonpurchased goods c=1..m, which are the conditions that need to be satisfied in order for their
observed budget shares to be zero. To specify these demand equations, we start with the indirect
utility V(π, y; θ, ε) and invoke Roy’s Identity to get the optimal budget shares of all goods as
S c  , y ;  ,    
 ln V  , y ;  ,    ln  c
 ln V  , y ;  ,    ln y
 c  1..M
(1)
In the RHS of equation (1), the price argument of any good i, πi, takes the value of its market price pi
if good i is purchased and the value of its virtual price Ti (which is treated as an unknown) if good i
is not purchased. We thus set πi =Ti for non-purchased goods i=1..m and πi =pi for purchased goods
7
i=m+1..M in equation (1). To get the solutions of virtual prices of non-purchased goods, {Ti} im1 , we
set the optimal budget shares of the non-purchased goods in equation (1) to zero. Doing so yields
0 = ∑lnV({Ti} im1 , {pi} iMm 1 , y; θ, ε)/∑lnπc " c=1..m
(2)
Solving the m equations in (2) yields the solutions of virtual prices of the non-purchased goods as
functions of the market prices of purchased goods as Tc=Tc({pi} iM m 1 , y; θ, ε) c=1..m. Given the
solutions of the virtual prices, the demand equation of a non-purchased good c is given as
pc ≥ Tc({pi} iM m 1 , y; θ, ε)
c=1..m
(3)
implying that virtual price of a non-purchased good acts as its reservation price. To get the demand
equation of a purchased good c, we substitute the solutions of virtual prices into the budget share of
that purchased good in equation (1) and set its optimal budget share as its observed budget share
sc  




 ln V Ti .i 1 , pi i m1 , y ; ,   ln  c
m
M
 ln V Ti .i 1 , pi i m1 , y ; ,   ln y
m
M
 c = m+1..M
(4)
This completes the specifications of the demand equations that follow from the dual approach.
These demand equations will be identical to those derived from direct utility maximization approach
as long as there exists a unique direct utility from which the indirect utility V can be derived, and
that direct utility satisfies the properties of monotonicity, differentiability and strict quasi-concavity.
For that to be so, V needs to satisfy the following regularity properties (Van Soest et al. 1993):
1.
V(π, y; θ, ε) should be twice continuously differentiable and homogenous of degree zero in
the expenditure y and price arguments π.
2.
V(π, y; θ, ε) should be increasing in expenditure y and non-increasing in price arguments π.
3.
The M×M Slutsky substitution matrix that follows from V(π, y; θ, ε) should be negative
semi-definite with a rank of M-1.
For global regularity, the indirect utility V(π, y; θ, ε) needs to satisfy regularity properties 1-3 at all
feasible values of variables that enter the indirect utility. This consists of all values of expenditure in
8
the space Ωy ª{y |y>0}, all values of market prices in the space Ωp ª{{pc} cM1 | pc>0}, and all values of
random errors in the support, Ωε. Regarding the market prices, recall that the price argument for any
good i, πi, in the indirect utility V(π, y; θ, ε) takes the value of its market price pi only if its observed
budget share is positive; otherwise it takes the value of its virtual price. This implies the expression
of the indirect utility (in terms of which variables enter it) can change depending on the observed
values of budget shares. This in turn implies the following requirement for its global regularity.
Requirement for Global Regularity: For each vector of observed budget shares s ª [s1, s2,.., sM] in the
M
feasible space, Ωs ª{{sc} cM1 |  sc =1, sc ≥0}, the indirect utility V(π,y; θ, ε) needs to satisfy regularity
c 1
properties 1-3 at all corresponding values of yΩy, {pc} cM1 Ωp and εΩε that satisfy the demand
equations when the observed budget share vector is s ª[s1, s2,.., sM].
We next discuss the Basic Translog utility, and the necessary and sufficient conditions that
need to be imposed on its parameters so that it satisfies the above requirements of global regularity.
SECTION 3: BASIC TRANSLOG UTILITY
In section 3.1, we discuss the variables and parameters in the Basic Translog utility. In section 3.2,
we discuss the requirements for global regularity of Basic Translog utility. In section 3.3, we lay out
the necessary and sufficient conditions for satisfying those requirements. In section 3.4, we compare
those with the approaches used in the prior literature to deal with regularity of Basic Translog utility.
3.1 Parameters and Variables
The Basic Translog (indirect) utility with M goods is given as (Christensen et al. 1975):
ln V     AT ln  
1
ln T Bln 
2
(5)
In equation (5), Aª [a1,..,aM] is an M×1 vector of parameters, B {bi,c} is an M×M symmetric matrix
of parameters,  ª[  1 ,..,  M ] is an M×1 vector of normalized price arguments of the M goods, in
9
which the normalized price argument of good i is related to its price argument via  i   i y . From
our discussion in section 2,  i will take the value of the normalized market price pi (=pi /y) if the
observed budget share of good i is positive; otherwise, it will take the value of the normalized virtual
price Ti (=Ti /y). To incorporate randomness, we let the parameters in the vector A be ac=âc+εc
c=1..M, where εª [ε1,.., εM] are the econometrician’s errors with support as Ωεª(-¶,¶)M.
Invoking Roy’s Identity on equation (5), we get the optimal budget share of any good c=1..M
as S c    
 ln V    ln  c
, in which the (log) marginal utility of price of good c is given as
 ln V    ln y
 ln V    ln  c  ac   bi ,c ln  i
M
i 1
" c=1..M
(6)
and the (log) marginal utility of expenditure is given as
 ln V    ln y   ai    bi ,m ln  i
M
M M
i 1
m 1 i 1
(7)
Notice in equations (6)-(7) that the budget share of good c is homogenous of degree zero in the
parameters. Thus we have to normalize one of the parameters, say aM. Although we can choose any
non-trivial normalization for aM, the prior literature has adopted the following condition for its
normalization: aM should be normalized in a way so that Basic Translog utility can nest a homothetic
version, viz. Homothetic Translog utility, in which the (log) marginal utility of expenditure always
takes a value of one2. Observe in equation (7) that the (log) marginal utility of expenditure takes a
M
value of one if we (a) impose M homothetic restrictions,  bc ,i  0 " c=1..M, and (b) normalize the
i 1
M 1
parameter aM as aM = 1   a c . This implies the following normalization condition for parameter aM:
c 1
Normalization condition: aM should be normalized in such a way so that its normalization reduces to
M 1
M
c 1
i 1
aM = 1   ac once we impose M homotheticity restrictions,  bc ,i  0 "c=1..M, on the parameters.
2
The reason for this normalization condition is that it allows one to test whether consumer preferences are homothetic
10
In the prior literature, two different normalizations have been used for parameter aM, where both
M 1
M M
c 1
c 1 i 1
satisfy the aforementioned normalization condition: aM  1   ac    bc ,i (Pollak and Wales 1992)
M 1
and aM  1   ac (Christensen et al., 1975). In our paper, we choose the normalization for aM that
c 1
satisfies the normalization condition, but is different from the ones used in the prior literature. We
will discuss our normalization for aM in section 3.3. Although the normalization of aM per se does
not impact the flexibility of the Basic Translog utility, however as we will explain in section 3.3, the
normalization of aM that we choose has a strong impact on the extent of restrictions that need to be
imposed on the other parameters (which impact the flexibility of Basic Translog utility) in order for
the Basic Translog utility to satisfy the regularity properties. Thus we will use the normalization of aM
to our advantage so that we end up imposing minimal restrictions on other parameters.
Before we proceed, without loss of generality we represent the M×M symmetric matrix B of
parameters in equation (5) in terms of the following partition which will simplify the exposition of
requirements for global regularity of Basic Translog utility that we discuss in the next section
 Z 1
 Z 1 
B   T 1

 T Z 1   
  Z
(8)
where Z-1 is a symmetric and invertible (M-1)×(M-1) matrix, δ is an (M-1)×1 vector with elements as
δª[δ1,δ 2,..,δ M-1], and τ is a scalar. Thus Z-1 represents the elements in the first M-1 rows and M-1
columns of the matrix B, -Z-1δ represents the first M-1 elements in the Mth column of the matrix B,
and δTZ-1δ +τ represents the element in the Mth column and Mth row of the matrix B.
3.2 Requirements for Global Regularity of Basic Translog Utility
Observe in equation (5) that Basic Translog utility satisfies regularity property 1. Thus we only focus
on requirements for regularity properties 2 and 3 in the entire feasible of variables.
11
3.2.1 Regularity Property 2: Recall that as per regularity property 2, the (log) marginal utilities of
prices and expenditure should be non-positive and positive respectively. The following Lemma
states the conditions under which that would be so for the Basic Translog utility.
Lemma 1: For a given set of values of the observed budget shares, the random parameters and the
normalized price arguments of the c=1..M goods (i.e., {sc, ac,  c } cM1 ), the (log) marginal utilities of
prices and expenditure for the Basic Translog utility (as given in equations 6 and 7) will be nonpositive and positive respectively if and only if the following scalar D be positive:
M 1
D
a M    c a c   ln  M
c 1
M 1
sM    c sc
(9)
c 1
where {δ c} and τ are the elements of matrix B given in equation (8) and  M is the normalized price
argument of good M which takes the value of its normalized market price p M if sM>0, and its
normalized virtual price TM if sM=0 (whose specification follows from equation 2; and as discussed in
section 2, it will be a function of normalized market prices of all other purchased goods).
Proof: see section 1 of Web Appendix
To understand the feasible space of variables over which regularity property 2 needs to be satisfied,
M
note that D is a function of observed budget shares whose feasible space is Ωs ª{{sc} cM1 |  sc =1, sc
c 1
≥0}, the random parameters, ac=âc+εc c=1..M,whose feasible space is (-¶,¶)M, and log normalized
market prices, ln pc = ln(pc/y) c=1..M, whose feasible space is (-¶,¶)M. Thus the requirement for
regularity property 2 to be satisfied in this entire feasible space can be stated as: Requirement for regularity property 2: The scalar D in Lemma 1 should be positive for each vector
of observed budget shares sΩs, and for all corresponding values of {ln pc } cM1 (-¶,¶)M and
{ac} cM1 (-¶,¶)M that satisfy the demand equations when the observed budget share vector is s.
12
3.2.2 Regularity Property 3: Recall that as per regularity property 3, the M×M Slutsky matrix that
follow from an indirect utility V should be negative semi-definite with a rank of M-1. The following
Lemma states the conditions under which that would be so for the Basic Translog utility.
Lemma 2: For a given set of values of the observed budget shares, the random parameters and the
normalized price arguments of the c=1..M goods (i.e., {sc, ac,  c } cM1 ), the M×M Slutsky matrix of
Basic Translog utility will be negative semi-definite with rank of M-1 if and only if the following (M1)×(M-1) matrix J is positive definite
J  sˆ   sˆsˆ T 
 sˆsˆ T   I  sˆeˆ T   T Z 1 I  eˆ   sˆ T 
 ln V    ln y
where I is an (M-1)×(M-1) identity matrix, ê is an (M-1)×1 vector of ones, ŝ=[s1,s2,.., sM-1] is an (M1)×1 vector of observed budget shares of the first M-1 goods, Δ(ŝ) is a diagonal (M-1)×(M-1) matrix
with the vector ŝ on its diagonal, and  ln V    ln y is the log marginal utility of expenditure (whose
specification is given in equation 7).
Proof: see section 1 of Web Appendix
To understand the feasible space over which regularity property 3 needs to be satisfied, note
that J is a function of the observed budget shares whose feasible space is Ωs, and the log marginal
utility of expenditure, ∑lnV(  )/∑lny, which should be positive as per regularity property 2. Thus the
requirement for regularity property 3 to be satisfied in this entire feasible space can be stated as
Requirement for regularity property 3: The (M-1)×(M-1) matrix J in Lemma 2 should be positive
definite for each vector of observed budget shares sΩs, and all positive values of ∑lnV(  )/∑lny.
This completes the discussion on the requirements for global regularity of the Basic Translog
utility. We next discuss the necessary and sufficient conditions that need to be imposed on the
parameters so that the Basic Translog utility satisfies the requirements stated in this section.
13
3.3 Necessary and Sufficient Conditions for Global Regularity
Since the matrix J in Lemma 2 is a function of the (log) marginal utility of expenditure (which as per
regularity property 2 should be positive), it follows that we first need to identify the necessary and
sufficient conditions that need to be imposed so that the Basic Translog utility satisfies regularity
property 2. And conditional on these necessary and sufficient conditions, we need to then identify
the necessary and sufficient conditions that need to be imposed so that the Basic Translog utility
satisfies regularity property 3. Finally, we will combine these two sets of conditions to get the overall
necessary and sufficient conditions to satisfy regularity properties 2 and 3.
We start with the following proposition which states the necessary and sufficient conditions
required for satisfying regularity property 2 in the entire feasible space of the variables.
Proposition 1: The necessary and sufficient conditions that need to be imposed on the parameters
of Basic Translog utility so that it satisfies the requirement for regularity property 2 stated in section
3.2 are (i) the parameter τ=0, (ii) the (M-1)×1 vector δ >0 (i.e., δc >0 for all c=1..M-1), and (iii) the
M 1
parameter a M     c a c
c 1
Proof: see section 1 of the Web Appendix.
M 1
Observe that condition (iii) requires aM to satisfy the inequality: a M     c a c . Next, recall that aM
c 1
cannot be identified and needs to be normalized as per the normalization condition stated in section
3.1. This implies that we can use the normalization of aM to our advantage in such a way so that it
not only satisfies the normalization condition in section 3.1, but also results in condition (iii) of
proposition 1 to be always satisfied. Such a normalization of aM is given the following corollary:
Corollary: A convenient normalization of aM that satisfies condition (iii) in proposition 1 along with
M 1
the normalization condition given in section 3.1 is a M  1    c a c .
c 1
14
Proof: see section 1 of the Web Appendix
The normalization for aM given in the corollary imposes minimal restrictions on other parameters in
order to satisfy regularity property 2. If we normalize aM in any other way, we will end up imposing
stronger restrictions on other parameters. For instance, consider the case where we were to employ
M 1
the normalization used in prior literature, which is a M  1   a c . Substituting this normalization
c 1
value into condition (iii) of proposition 1 yields the following inequality that needs to be satisfied:
M 1
1    c  1ac  0 .
c 1
This inequality will only be satisfied for all values of random parameters a c cM11 in
the support (-¶,¶)M-1 if we impose the restrictions: δc=1 " c=1..M-1. Observe that these are more
restrictive than those stated in condition (ii) in proposition 1, which only require δc >0 " c= 1..M-1.
This completes the discussion on regularity property 2. The next proposition states the
necessary and sufficient conditions that need to be imposed on parameters to satisfy regularity
property 3 in the entire feasible space of variables, conditional on restrictions stated in proposition 1.
Proposition 2: Conditional on the restrictions stated in proposition 1, the necessary and sufficient
condition for Basic Translog utility to satisfy the requirement for regularity property 3 stated in
section 3.2 is that the (M-1)×(M-1) matrix Z in equation (8) should be positive definite.
Proof: see section 1 of the Web Appendix.
We next combine the conditions stated in propositions 1 and 2 to get the necessary and sufficient
conditions that need to be imposed on the parameters so that Basic Translog utility satisfies all the
regularity properties in the entire feasible space of variables. These are summarized as follows.
Proposition 3 (Global Regularity): The necessary and sufficient conditions that need to be
imposed on the parameters so that Basic Translog utility satisfies global regularity are (i) Z is positive
M 1
definite, (ii) τ=0, (iii)  >0, and (iv) aM is normalized as a M  1    c a c
c 1
15
3.4 Comparison with the Approaches used in Prior Literature
We compare the necessary and sufficient conditions for global regularity stated in proposition 3 with
three approaches used in the prior literature for dealing with the regularity of Basic Translog utility.
These approaches were proposed by Jorgensen and Fraumani (1981), Van Soest and Kooreman
(1990), and Mehta and Ma (2012) respectively. Starting with Jorgensen and Fraumani (1981), they
imposed the following set of conditions on the parameters of Basic Translog utility:
(a) The matrix B is positive semi-definite with all of its principal sub-matrices being positive
M
definite, and  bc ,i  0 " c= 1..M. These restrictions can be translated in terms of our block-wise
i 1
representation of B as: {Z is positive definite, τ=0, and δc =1 "c=1..M-1}.
M 1
(b) The parameter aM is normalized as a M  1   a c
c 1
Conditions (a)-(b) represent the set of ‘sufficient conditions’ that ensure global regularity of Basic
Translog utility, and have been most widely used in the literature. They are more restrictive on the
parameter space as compared to the conditions in proposition 3, since unlike the conditions in
proposition 3 in which all elements in the (M-1)1 vector δ are restricted to be positive, all elements
in δ are restricted to be unity in condition (a). Conditions (a)-(b) result in the Basic Translog utility to
reduce to the Homothetic Translog utility we discussed in section 3.1. To see why, recall that aM is
M 1
M
c 1
i 1
normalized as aM  1   ac , and parameters in matrix B are restricted as  bc ,i  0 " c=1..M in the
Homothetic Translog utility. Observe that these are reflected in conditions (a)-(b).
Van Soest and Kooreman (1990) imposed the following conditions on parameters of Basic
M
M
c 1
i 1
Translog utility: (a) matrix B is positive definite, (b)  bi ,c  0 " i =1..M, (c) 1     bi ,c ln pi   0 ,
M
 c1

M 1
and (d) aM is normalized as a M  1   a c . There are three key differences between the conditions
c 1
16
stated in proposition 3 and conditions (a)-(d): First, conditions (a)-(d) represent a set of sufficient
conditions that ensure regularity of Basic Translog utility. Second, conditions (a)–(d) do not ensure
global regularity – instead, they ensure regularity in a restricted space of expenditure and market
prices given in condition (c). Third, conditions (b) and (c) are difficult to a priori impose while
estimating the model. This is because condition (b) consists of complex set of inequalities and
condition (c) consists of parameters as well as the regressors. The only way to impose condition (c)
is to have a 0-1 indicator term in the likelihood, which takes a value of zero if condition (c) is not
satisfied. However this results in discontinuities in the likelihood, making it difficult to maximize.
Finally, Mehta and Ma (2012) imposed the following conditions on the Basic Translog utility:
(a) the matrix B is generated as a product of two Choleskies with the diagonal element of Cholesky
M 1
corresponding to the Mth good is zero, and (b) aM is normalized as a M  1   a c . Unlike the
c 1
conditions stated in proposition 3, conditions (a)-(b) do not ensure global regularity of the Basic
Translog utility. Recall from section 1 that estimating a flexible functional form without a priori
imposing restrictions of global regularity carries the risk of running into coherency problems. Thus
to ensure that their model does not suffer from coherency problems, Mehta and Ma (2012) ex-post
checked whether at the estimated values of parameters, the Basic Translog utility satisfies regularity
properties at all observations in the data. Although they found that this was indeed the case, their ex
post approach is not an ideal one. This is because if the estimates we get are such that regularity
properties are violated at even one observation in the data, we can run into coherency problems.
SECTION 4: IMPLICATIONS ON FLEXIBILITY
Since the necessary and sufficient conditions stated in proposition 3 restrict the parameter space of
the Basic Translog utility, our task in this section is to identify the restrictions on the demand
elasticities imposed by these conditions. We focus on cross-price and expenditure elasticities and not
17
the own-price elasticities. This is because in any demand system with M goods, the M own-price
elasticities can be determined from the M(M-1) cross-price elasticities. The expenditure elasticity of
any good i is ∑lnqi /∑lny, where a positive (negative) value implies that good i is a normal (inferior)
good. The cross-price elasticity is Ei,j=∑lnqi/∑lnpj, where a positive (negative) value implies that the
goods i and j are substitutes (complements). The restrictions on the demand elasticities imposed by
the necessary and sufficient conditions are summarized in the following proposition.
Proposition 4: As a result of the necessary and sufficient conditions stated in proposition 3, the
Basic Translog utility imposes the following restrictions on elasticities: Out of the total of M(M-1)
cross price elasticities across M goods, at least two of the cross price elasticities have to be positive.
Proof: see section 1 of Web Appendix.
We next compare the restrictions on elasticities stated in proposition 4 with those imposed by two
alternatives most frequently used in prior literature. The first is the Homothetic Translog utility,
which is the Basic Translog utility in which we impose sufficient conditions for global regularity
proposed by Jorgensen and Fraumani. These sufficient conditions have been almost universally
employed by papers that have used Basic Translog utility (e.g., Lee and Pitt 1987; Chiang 1991; Song
and Chintagunta 2007). The second is LES (Linear Expenditure System) used by Chintagunta (1993),
Kao et al. (2001) and Du and Kamakura (2008). It is an additively separable direct utility given by
U   ai lnqi  bi  and is globally regular as long as bi<0 and ai>0 "i=1..M. The restrictions on
M
i 1
elasticities imposed by these two alternatives are summarized in the following two propositions.
Proposition 5: The Homothetic Translog utility imposes the following restrictions on elasticities:
1) The expenditure elasticity of any good i is always equal to one.
2) The cross price elasticities between goods i and j are restricted by sj Ei,j = si Ej,i. Thus, if the
budget shares of goods i and j are equal, their cross price elasticities will be symmetric.
18
3) Out of the total of M(M-1) cross price elasticities, at least 2(M-1) cross price elasticities have
to be positive. This implies that at least M-1 pairs of goods are substitutes of each other.
Proof: see section 1 of the Web Appendix.
Proposition 6: The LES utility imposes the following restrictions on elasticities of M goods:
1) The expenditure elasticities are all positive, which implies that inferior goods are not allowed.
2) All the M(M-1) cross price elasticities are positive. This implies that all pairs of goods are
restricted to be substitutes of each other.
Proof: see section 1 of the Web Appendix.
Observe that the Basic Translog utility with necessary and sufficient conditions imposes much fewer
restrictions on demand elasticities compared to Homothetic Translog and LES utilities. This implies
that if the true demand elasticities in the data lie outside the restricted space of elasticities given in
propositions 5 and 6, then the Basic Translog utility with necessary and sufficient conditions should
do a better job in capturing the true demand elasticities as compared to Homothetic Translog and
LES utilities. We will investigate this issue in the next section using both simulated and scanner data.
SECTION 5: EMPIRICAL IMPLEMENTATION
We first discuss the operationalization of parameters in the Basic Translog utility with the necessary
and sufficient conditions imposed on its parameter space, as stated in proposition 3. Following that,
we specify the demand equations and the likelihood. And finally, we discuss the empirical results.
5.1 Operationalization of Parameters in the Basic Translog Utility
Recall that the parameters in Basic Translog utility are the ones in the MM matrix Bª{bc,j} and the
M1 vector Aª [a1, a2,..,aM]. We start with the operationalization of parameters in the matrix B.
Equation (8) yields the following expression for B after imposing the restrictions in proposition 3:
19
 Z 1
 Z 1 
B   T 1

 T Z 1 
  Z
(10)
Since the (M-1)×(M-1) matrix Z is symmetric and positive definite, we generate it as a product of
two cholesky matrices as Z = (CZ)(CZ)T, where CZ is a (M-1)×(M-1) cholesky matrix with elements as
{ ciz, j }. And since all elements of the (M-1)×1 vector δ ª [δ1, δ2,.., δM-1] are positive, we generate it as
δ=exp(r), where rª[r1, r2,.., rM-1] is an (M-1)×1 vector. Moving on to the parameters in the vector
Aª [a1, a2,..,aM], recall that ac was specified as ac=âc+εc, in which âc is the deterministic component and
εc is the econometrician’s error. Since aM is normalized, we focus on the operationalization of first M1 elements of the vector A. We define the following (M-1)1 vectors: A(-M)ª [a1,..,aM-1], Â(-M)
ª[â1,,..,âM-1], and ε(-M) ª[ε1,..,εM-1], where A(-M)= Â(-M)+ε(-M). We represent the (M-1)1 vector of the
random elements ε(-M) in terms of the following transformation
(-M)  Z-1
(11)
where η ª[η 1,..,ηM-1] is an (M-1)1 vector of the (transformed) econometrician’s errors that are IID
across observations. These are assumed to be multivariate normally distributed as η ~N(0, Γ-1). We
generate Γ as a product of two cholesky matrices as Γ=(CΓ)(CΓ)T, where CGis a cholesky matrix with
elements as { c i, j }. Similarly, we represent the (M-1)1 vector of deterministic components Â(-M) as
Â(-M) ª Z-1
(12)
where αª[α1, α2,.., αM-1] is an (M-1)1 vector of parameters. From equations (11) and (12), we can
write the (M-1) 1 vector of the random parameters, A(-M)ª [a1, a2,..,aM-1], as A(-M)ª Z-1(+). Finally,
M 1
since the parameter aM is normalized as per proposition 3 as aM  1    i ai , we get the expression
i 1
for aM from equations (11 and (12) as aM=1- TZ-1(+). This completes the operationalization of
the parameters. To summarize, there are (M+2)(M-1) parameters: (i) M(M-1)/2 parameters in the
20
cholesky CZ, which generate the matrix Z, (ii) M-1 parameters in the vector rª[r1,..,rM-1], which
generate the vector δª[δ1,.., δM-1], (iii) M-1 parameters in the vector αª[α1,..,αM-1], and (iv) M(M-1)/2
parameters in the cholesky CG, which generates the covariance matrix of econometrician’s errors, Γ-1.
5.2 Demand Equations and the Likelihood
We specify the demand equations and observation likelihood for the general observation discussed
in section 2 in which the consumer purchases goods c=m+1..M with observed positive budget shares
as [sm+1, sm+2,.., sM] and does not purchase goods c=1..m with observed budget shares as sc=0 "c=1..m.
To do so we define the following terms. We partition the M×1 vector of normalized market
prices of the M goods along the non-purchased and purchased goods as P ª[ PI , PII , pM ], where
PI ª[ p1 , p2 ,..., pm ] is a vector of the normalized market prices of non-purchased goods, PII ª
[ pm1 , pm2 ,.., pM 1 ] is a vector of the normalized market prices of purchased goods c=m+1..M-1,
and pM is the normalized market price of purchased good M. Similarly, we partition the (M-1)1
vector δ, the (M-1)(M-1) matrix Z, the (M-1)1 vector α, and the (M-1)1 vector of errors η along
the non-purchased and purchased goods as: (a) δª[δI, δII], where δIª [δ1,..,δm] and δIIª[δm+1,..,δM-1]; (b)
Z
Z   TI
Z III
Z III 
 , where ZI is an m×m matrix, ZII is an (M-m-1)×(M-m-1) matrix, and ZIII is the off
Z II 
diagonal m×(M-m-1) matrix; (c) αª[αI, αII], where αI ª[α1,..,αm] and αII ª[ αm+1,..,αM-1]; and (d) ηª[ηI, ηII],
where ηI ª[η1,.., ηm] and ηII ª[ηm+1,.., ηM-1].
Given these partitions, the specifications of demand equations can be derived by following
the steps detailed in equations (1)-(4) in section 2. Doing so yields the demand equations for the
purchased goods c=m+1..M-1 in vector form as
 II 
Z II s II
  II  ln PII   II ln pM
1  eIIT   IIT s II
(13)
21
where eII is an (M-m-1)×1 vector of ones and s II ª[sm+1,..,sM-1] is an (M-m-1)×1 vector of observed
budget shares of purchased goods. And we get the demand equations of non-purchased goods as
I 
Z III s II
  I  ln PI   I ln p M
1  eIIT   IIT s II
(14)
From the demand equations in (13) and (14), we next specify the observational likelihood. Let
f II (ηII) be the joint marginal density of errors ηIIª[ηm+1,..,ηM-1], and let FI II (ηI |ηII) be the joint c.d.f of
errors ηIª[η1,..,ηm] conditional on ηII. From (13) and (14), we get the observational likelihood as




Z II s II
 f II 
  II  ln PII   II ln pM 
T
T
 1  eII   II s II
 (15)


Z III s II
Z II s II
 F I  II 
  I  ln PI   I ln pM  II 
  II  ln PII   II ln pM 
T
T
T
T
1  eII   II s II
 1  eII   II s II

L sc  0c1 , s II  sc cm1  Z II  1  eIIT   IIT s II 
M 1
m
m M
To fully specify the observational likelihood, recall that the (M-1)1 vector of the econometrician’s
errors, ηª[ηI, ηII], is multivariate normally distributed as η ~N(0, Γ-1). We represent the inverse of the
 I
T
III
covariance matrix as   
III 
 , where ΓI is an m×m matrix, ΓII is a (M-m-1)×(M-m-1) matrix,
II 
and ΓIII is the off diagonal matrix. Thus we get the marginal joint density of the errors ηII as
f II  II  
2 
1
 M  m1 2
II  IIIT I1III
12
   T   IIIT I1III  II
exp II II
2




(16)
And the joint c.d.f of the errors ηI conditional on ηII as
FI II  I  II   
ˆ I  I
1
2 m 2
I
12
  ˆ I  I1III II T I ˆ I  I1III II  
dˆ I
exp


2


(17)
Substituting f II (ηII) and FI II (ηI |ηII) into equation (15), we get the observational likelihood. To
compute the integral in equation (17), we use the GHK simulator with R=100 random draws.
5.3 Simulated Data
We use simulated data to achieve two objectives. The first is the empirical counterpart of section 4,
which is to investigate whether and to what extent the Basic Translog utility with necessary and
22
sufficient conditions does a better job in capturing the true elasticities in the data as compared to
Homothetic Translog and LES utilities. To do so, we simulate three datasets with M=4 goods using
a direct quadratic utility. In dataset 1, most pairs are complements. In dataset 2, one of the goods is
inferior and cross-price effects are asymmetric. In dataset 3, all pairs are substitutes. The second is to
examine the efficacies of the three utilities in capturing the true elasticities over two additional facets
of the data: number of goods and number of zero purchases. To do so, we simulate datasets 4 and 5
using the quadratic utility. These are similar to dataset 3 in the sense that all pairs are substitutes. The
difference is that these datasets have M=6 goods and dataset 5 has larger number of zero purchases
compared to dataset 4. The quadratic utility used for simulating the datasets 1-5 is given as


M
1M M
M
U qi i 1    i qi     i ,k qi qk
i 1
2 i 1 k 1
(18)
In equation (18), the MM matrix of parameters bª{bi,k}is symmetric and positive definite. Since
the quadratic utility is homogenous of degree zero in the parameters, one of the parameters, γM,
needs to be normalized. The randomness is incorporated through the first M-1 parameters of the
vector γª[γ1, ..,γM-1, γM] as γi=  i  ηi " i=1..M-1 - where  i iM11 are the intercepts and {ηi} iM11 are the
random errors which are assumed to be multivariate normally distributed as {ηi} iM11 ~N(0, Ω).
The reason why we have chosen the quadratic utility to simulate the datasets is because it is
flexible. The issue however with the quadratic utility is that it does not satisfy global regularity even
in a restricted space of its parameters. Instead, as discussed in Van Soest et al. (1993), it satisfies the
regularity properties in a restricted space of the normalized market prices of M goods given by
 M  max i ,M pi 
i 1.. M
(19)
Since the parameter γM has to be normalized, we set the value of γM so that the condition in equation
(19) is satisfied for all values of normalized prices that we simulate in the five datasets. This ensures
that the regularity properties will be satisfied for all observations in the simulated datasets.
23
The true parameters of the quadratic utility as well as the distributions of normalized prices
of the M goods used for simulating datasets 1-5 are given in section 2 of the Web Appendix. All
datasets consist of 5000 observations and are constructed in a manner similar to those in marketing
in which the Mth good is the outside/composite good that is always purchased and has a
disproportionately high budget share. The summary statistics of datasets 1-5 are given in Table 1.
We estimate three models on each dataset: (a) Model I, with the Basic Translog utility with
necessary and sufficient conditions for global regularity imposed on its parameters. (b) Model II,
with the Homothetic Translog utility. (c) Model III, with the LES utility. As discussed in section 4,
the LES utility is given as U   ai lnqi  bi  which is globally regular as long as the parameters bi <0
M
i 1
and ai>0 " i=1..M. We thus operationalize the parameters {bi} and {ai} as
bi=-exp(βi) and ai =exp(αi +ηi)
(20)
where ηi is the econometrician’s error for good i. For identification, the parameter αM and the error
ηM are normalized to zero. The (M-1)1 vector of econometrician’s errors ηª[η1,.., ηM-1] is assumed to
be multivariate normally distributed as η ~N(0, Γ-1), where Γ is the inverse of the covariance matrix.
The parameter estimates of Models I-III for all five datasets are given in section 2 of Web
Appendix. The log likelihoods and BICs of the three models for all five datasets are given in Table 2.
Model I outperforms Models II-III in terms of both the log likelihood and BIC in all five datasets.
However, the extent to which Model I outperforms Models II-III decreases as we go from dataset 4
to dataset 5. This implies that although Model I yields higher log likelihood compared to Models II
and III, its relative superiority decreases as we increase the fraction of zero purchases in the data.
We next discuss the extent to which models I-III can accurately capture the true demand
elasticities in each dataset. The true demand elasticities are calculated in terms of % change in
expected purchase quantities of all goods at the true values of parameters in the quadratic utility.
24
Dataset 1 (Complements): The true demand elasticities for this dataset are given in Table 3A. The
true pairwise cross elasticities are -0.608 between goods i=1..3 which imply price complementarity,
and the true expenditure elasticity of each good i=1..3 is 1.852. The elasticities predicted by Models
I-III are reported in Tables 3B-3D. All predicted elasticities of Model I are similar to the
corresponding true elasticities. Model II’s predicted cross elasticities between goods i=1..3 range
between -0.402 to -0.423, which implies that Homothetic Translog utility underestimates the extent
of complementarity. Also, its expenditure elasticities are equal to one, which are significantly
different from the true expenditure elasticities. Regarding Model III, it yields positive cross-price
elasticities between all goods, which shows that LES utility cannot capture complementarity.
Dataset 2 (Inferiority and Asymmetry): The true demand elasticities are given in Table 4A. The true
expenditure elasticities of goods 1-3 are -0.298, 1.644 and 1.644, implying that good 1 is inferior.
Further, the true cross-price elasticities between goods 1 and 2 (which are E1,2 =1.59 and E2,1=1.33),
and between goods 1 and 3 (which are E1,3 =1.59 and E3,1=1.33) are asymmetric. The elasticities
predicted by Models I-III are reported in Tables 4B-4D. All predicted elasticities of Model I are
similar to the true elasticities. Regarding Model II, all its expenditure elasticities are one, which are
significantly different from the true expenditure elasticities. Further, its predicted cross elasticities
between goods 1 and 2 (and goods 1 and 3) are not significantly different from each other, implying
they are symmetric. The reason for it follows from proposition 5 – recall that Homothetic Translog
utility yields symmetric cross elasticities between goods with same budget shares. Since the average
budget shares of goods 1-3 are similar, the result follows. Regarding Model III, all its expenditure
elasticities are positive, which shows that LES utility cannot capture inferiority.
Datasets 3-5 (Substitutes): We start with dataset 3. The true demand elasticities are given in Table
5A. The true cross price elasticity between goods i=1..3 is 1.266 (implying substitutability), and the
true expenditure elasticity of each good i=1..3 is 1.674. The elasticities predicted by Models I-III are
25
reported in Tables 5B-5D. Starting with Model I, all of its predicted elasticities are similar to the true
elasticities. Moving on to Model II, its expenditure elasticities of all goods are equal to one, which
are significantly different from the true expenditure elasticities. Regarding Model III, its cross price
elasticities between goods i=1..3 are positive, but significantly smaller than the true cross elasticities.
This is because Model III is based on the additively separable LES utility. In additively separable
direct utilities, a price change in good i impacts the quantity of any other good only through the
budget constraint. If good i has a small budget share, a change in price of good i will have not have a
strong impact on budget left for other goods - as a result, it will not have a strong impact on
quantities of other goods. Since budget shares of goods 1-3 are small, the result follows.
We next discuss datasets 4 and 5, where both datasets have M=6 goods and dataset 5 has
larger number of zero purchases as compared to dataset 4. The true demand elasticities for these
two datasets are given in Tables 6 and 7. The elasticities predicted by Models I-III for dataset 4 are
reported in Tables 6B-6D and those for dataset 5 are reported in Tables 7B-7D. The pattern of
results in both datasets 4 and 5 is the same as that in dataset 3. This suggests that the efficacy of
Model I in capturing the true elasticities is robust to the number of goods and number of zero
purchases. Further, all three models do a better job in accurately capturing the true elasticities in
dataset 4 than in dataset 5. This implies that the efficacy of either model in capturing the true
elasticities decreases as we increase the fraction of zero purchases. The reason for it follows from
the fact that the likelihood contribution of a zero purchase consists of the joint probability that the
errors lie in a certain region (as in equation 14), which is less informative compared to the likelihood
contribution of a non-zero purchase which consists of joint densities of errors (as in equation 13).
5.4 Scanner Panel Data
We use scanner panel data to investigate whether and to what extent does Basic Translog utility with
necessary and sufficient conditions yield better goodness of fit and predictive ability, different
26
estimates of demand elasticities and different results in counterfactuals compared to Homothetic
Translog and LES utilities. To do so, we estimate Models I-III on two scanner datasets. Dataset A
consists of consumers’ purchases across M=5 goods in the store: detergents, softeners, pasta, pasta
sauce and the composite good. Dataset B consists of consumers’ purchases across M=6 goods: five
flavors of 8 oz. Dannon yogurt, viz., blueberry (BB), mixed-berry (MB), strawberry (SB), plain (P)
and pina-colada (PC), along with the composite good. Both datasets are from a large grocery store
in Pittsburgh. The goods in dataset A are similar to the ones used by Song and Chintagunta (2007)
and Mehta and Ma (2012), and goods in dataset B are similar to the ones used by Kim et al. (2002).
The two datasets help address different issues. In dataset A, we expect {detergent, softener} and
{pasta, pasta sauce} to be complements; in dataset B, we expect flavors of yogurt to be substitutes.
The summary statistics of the two datasets are given in Table 8. Dataset A consists of 16589
observations over two years with 140 consumers and dataset B consists of 10129 observations over
two years with 110 consumers. Dataset B has much greater fraction of zero purchases compared to
dataset A. We split both datasets into estimation and holdout samples, where the estimation sample
of dataset A consists of 12109 observations with 100 consumers and that of dataset B consists of
7184 observations with 80 consumers. We extend Models I-III estimated in section 5.3 by
incorporating unobserved heterogeneity and impact of inventories and feature ads/displays of each
focal good on consumers’ purchases. We do so by specifying αc (as given in equation 12 for Models I
and II, and equation 20 for Model III) for consumer h on trip t for each focal good c=1..M-1 as
αh,c,t = άc + υh,c + βPro,c Proh,c,t + βI,c Invh,h,t
(21)
In equation (21), άc is the baseline population level mean parameter for good c and υh,c is a random
error that captures unobserved heterogeneity in the preference for good c. We assume the (M-1)1
vector of these errors, υh ª[υh,1,.., υh,M-1], to be multivariate normally distributed across the population
as υh~N(0, Ωυ), where Ωυ is a full covariance matrix. We generate Ωυ as a product of two cholesky
27
matrices as Ωυ=(Cυ)(Cυ)T, where Cυ is an (M-1)(M-1) lower triangular cholesky matrix with elements
as{ ci, j }. We use 300 Halton draws to integrate out unobserved heterogeneity errors from the joint
likelihood of consumer’s purchase history. Proh,c,t is a 0-1 promotional variable that takes a value of 1
if good c was feature advertised or displayed on trip t and Invh,c,t is consumer h’s inventory at hand of
good c on trip t. To operationalize inventories, we use the flexible inventory updating proposed by
Ailawadi and Neslin (1998) that allows for consumption rates to depend on inventory at hand.
The parameter estimates of Models I-III for the two datasets are given in section 2 of the
Web Appendix. The log likelihoods and BICs of all three models for the estimation and hold out
samples for both datasets are given in Table 9. Model I outperforms Models II in terms of the log
likelihood and BIC in both samples in both datasets. And Model I outperforms Models III in terms
of log likelihood and BIC in both samples in dataset A; however in dataset B, Model I outperforms
Model III in both samples only in terms of log likelihood, but not BIC. To understand why we get
this result, note that the BIC depends on the log likelihood as well as the number of parameters.
Since dataset B has a large fraction of zero purchases, it follows from our discussion in section 5.3
that the relative superiority of Model I over Model III in terms of the log likelihood will be small.
Further, Model III has 14 fewer parameters as compared to Model I in dataset B. Since the BIC puts
a heavy penalty on the model with larger number of parameters, the result follows.
We next compare the demand elasticities predicted by the three models for dataset A. Tables
10A-10C report the price and expenditure elasticities for the three models. The results mirror the
ones we discussed for simulated dataset 1. For Model I, we get own price elasticities of detergents,
softeners, pasta and pasta sauce as -2.438, -1.880, -1.695 and -2.014 respectively, and expenditure
elasticities as 1.936, 1.881, 1.469 and 2.021 respectively. Further, we get negative cross price
elasticities between detergents and softeners (=-0.058 and -0.127) and between pasta and pasta sauce
(=-0.115 and -0.091), implying these are price complements. Moving on to Model II, its predicted
28
cross-price between detergents and softeners are -0.047 and -0.025, and between pasta and pasta
sauce are -0.043 and -0.042. These are much smaller in magnitude compared to those predicted by
Model I, implying it underestimates the extent of complementarity. And its predicted cross price
elasticities between pasta and pasta sauce are not significantly different from each other, implying
they are symmetric. The reason for it follows from our discussion in section 5.3. Its expenditure
elasticities are equal to one, which are significantly smaller than expenditure elasticities predicted by
Model I. Model III’s predicted cross elasticities between detergents and softeners, and between pasta
and pasta sauce are all positive, which shows that LES utility cannot capture complementarity. Its
predicted own price elasticities of detergents, softeners, pasta and pasta sauce are -3.348, -2.912, 2.789 and -3.306 respectively, which are significantly greater in magnitude than those predicted by
Model I; and its predicted expenditure elasticities are 1.434, 1.244, 1.197 and 1.406, which are
significantly smaller than those predicted by Model I. The promotional elasticities predicted by the
three models are reported in Tables 11A-11C. The pattern of cross-promotional elasticities across
the three models is the same as that of cross price elasticities.
Tables 12A-12C report the price and expenditure elasticities predicted by Models I-III for
dataset B.3 The results mirror those discussed for simulated datasets 3-5 in section 5.3. Model I yields
expenditure elasticities in the range of 0.926 to 1.556, and positive cross price elasticities between all
yogurt flavors. The cross price elasticities between blueberry, mixed berry and strawberry pairs are in
the range of 0.185 to 0.276, which are significantly greater than those between other pairs. This
implies that berry flavors are stronger substitutes of each other as compared to other flavors. Model
II’s predicted expenditure elasticities of four of five flavors are significantly different from those
predicted by Model I. Model III yields values of cross price elasticities between the five flavors in
the range of 0.002 to 0.022, which are significantly smaller from those predicted by Model I. This
3
We do not report promotional elasticities for dataset B since the estimate of the promotional variable was insignificant in all models.
29
once again shows that LES underestimates substitutability between goods with small budget shares.
And the reason for it follows from our discussion in section 5.3.
Finally we run a policy experiment to calculate the compensating value for each focal good,
which provides useful information for assortment decisions (Kim et al. 2002). The compensating
value of good c is the amount by which the total expenditure must be increased to compensate the
household for the deleting the good c so that its utility remains unaffected. Its magnitude depends
on budget share of the good, its marginal utility of consumption, and the extent of substitutability of
that good with the other goods (the greater the extent of substitutability, lesser is the compensation
value). Tables 13A and 13B show the per-trip compensating value for an average household of each
focal good in both datasets, as predicted by Models I-III. Model I yields significantly different
compensating values of goods in both datasets as compared to Models II-III. Comparing Models I
and III, observe that while Model III yields significantly lower compensating values of the goods as
compared to Model I in dataset A, it yields significantly larger compensating values of the three
berry flavors as compared to Model I in dataset B. The reason for this result follows from our earlier
discussion that Model III does not allow for complementarity in dataset A and it underestimates the
extent of substitutability between the three berry flavors in dataset B.
In conclusion, our proposed model yields significantly different elasticities and results in the
counterfactual compared to Models II –III. The reason for it once again follows from propositions
4-6, as per which our proposed model imposes much fewer restrictions on elasticities compared to
Models II and III; thus if the true elasticities in the data lie outside the restricted space of elasticities
given in propositions 5 and 6, then our proposed model will do a better job in capturing the true
elasticities and thereby yield significantly different results in counterfactuals compared to Models II
and III; and this is what we see in the scanner data analysis. On the other hand, if the true elasticities
in the data do in fact lie within the restricted space of elasticities given in say proposition 5 (i.e., if
30
true expenditure elasticities were equal to one), then our proposed model will yield similar values of
elasticities and results in counterfactuals as Model II. However even in that case, it is prudent to use
our proposed model since it is difficult to a priori know whether the true expenditure elasticities are
equal to one. And estimating Model II may not help as it is difficult to know whether it is accurately
recovering the true expenditure elasticities from the data or is it restricting them to one. Since our
proposed model does not impose restrictions on expenditure elasticities, it is only after estimating
our proposed model that we can conclude that true expenditure elasticities are in fact equal to one.
SECTION 6: CONCLUSIONS
A longstanding problem faced by researchers who have used the utility maximization framework
with non-negativity constraints is obtaining a functional form of the utility that satisfies the criteria
of tractability, flexibility and global regularity. In this paper, we attempt to resolve this problem by
proposing an appropriate demand system that satisfies all three criteria; as a result it is easy to
estimate, yields valid expressions of the likelihood, is flexible (in terms of allowing for
complementarity/substitutability and normal/inferior goods) which helps in accurately capturing the
underlying demand elasticities in the data, and facilitates meaningful policy analysis. We accomplish
this task by deriving the necessary and sufficient conditions for global regularity of Basic Translog
indirect utility. We formally prove that the Basic Translog utility necessary and sufficient conditions
impose much fewer restrictions on demand elasticities as compared to the two alternatives used in
the literature: (a) Homothetic Translog utility, which is the Basic Translog utility when we impose
sufficient conditions for global regularity, and (b) additively separable LES direct utility. We use both
simulated and scanner panel data to empirically examine the consequences of having a demand
system that imposes much fewer restrictions on the demand elasticities. We find that the Basic
Translog utility with necessary and sufficient conditions more accurately captures the underlying
demand elasticities and yields different results in the policy simulation compared to (a) and (b).
31
REFERENCES
Chakir, R., Bousquet, A., Ladoux, N. (2004), “Modeling corner solutions with panel data: Application to the
industrial energy demand in France,” Empirical Economics, 29, 193-208
Chiang, J. (1991), “A Simultaneous Approach to whether, what and how much to buy questions,” Marketing
Science, 10(4), 297-315.
Chintagunta, P. K. (1993), “Investigating Purchase Incidence, Brand Choice and Purchase Quantity Decisions
of Households,” Marketing Science, 12(2), 184-208.
Christensen, L.R., D.W. Jorgensen and L.J. Lau (1975), “Transcendental Logarithmic Utility Functions,”
American Economic Review, 3, 367-383.
Du, R.Y. and W.A. Kamakura (2008), “Where Did All That Money Go? Understanding How Consumers
Allocate Their Consumption Budget,” Journal of Marketing, 72 (6), pp. 109-131
Jorgenson, D.W. and B.M. Fraumani (1981), “Relative prices and technical change,” Modeling and Measuring
Natural Resource Substitution, MIT Press, Cambridge, MA
Kao, C., Lee, L. F. and M. Pitt (2001), “Simulated maximum likelihood estimation of linear expenditure
system with binding non-negativity constraints,” Annals of Economics and Finance, 2, 203-223
Kim, J., Allenby, G.M. and P.E. Rossi (2002), “Modeling Consumer Demand for Variety,” Marketing Science,
21(3), 229-25.
Lee, L.F. and M. Pitt (1986), “Micro-econometric Demand Systems with Binding Non-Negativity
Constraints: The Dual Approach,” Econometrica, 54(5), 1237-1242
Lee, L.F. and M. Pitt (1987), “Micro-econometric models of rationing, imperfect markets, and non-negativity
constraints,” Journal of Econometrics, 36, 89-110
Lan L., Ratchford, B. and B. Yang (2012), "Why We Do What We Do? A Model of Activity Consumption",
Journal of Marketing Research, forthcoming
Mehta N. and Y. Ma (2012), “A Multi Category Model of Consumers’ Purchase Incidence, Brand Choice and
Quantity Decisions,” Journal of Marketing Research, 49 (5), 435 -451
Nair, H., J.P. Dube and P.K. Chintagunta (2005), “Accounting for Primary and Secondary Demand Effects
with Aggregate Data,” Marketing Science, 24(3), 444-460
Pollak, R. and T. J. Wales (1992), Demand System Specification and Estimation, Oxford University Press
Song, I. and P.K. Chintagunta (2007), “A Discrete/Continuous Model of Multi-category Purchase Behavior
of Households,” Journal of Marketing Research, 34(4), 595-612
Van Soest, A. and P. Kooreman (1990), “Coherency of the Indirect Translog Demand System with Binding
Non-Negativity Constraints,” Journal of Econometrics, 44, 391-400
Van Soest, A., Kapteyn, A. and P. Kooreman (1993), “Coherency and Regularity of Demand Systems with
Equality and Inequality Constraints,” Journal of Econometrics, 57, 161-188
Wales, T. and A. Woodland (1983), “Estimation of consumer demand equations with binding non-negativity
constraints,” Journal of Econometrics, 21, 263-28
32
TABLES
Table 1: Summary Statistics of the Five Simulated Datasets
Dataset 1
Dataset 2
Dataset 3
Dataset 4
(M=4 goods)
(M=4 goods)
(M=4 goods)
(M=6 goods)
Dataset 5
(M=6 goods)
Panel A
Good 1
4230 (0.077)
3018 (0.066)
3210 (0.111)
4252 (0.057)
2145 (0.014)
Good 2
4267 (0.077)
2976 (0.067)
4205 (0.103)
4228 (0.056)
2154 (0.014)
Good 3
4239 (0.077)
2945 (0.065)
4196 (0.103)
4270(0.057)
2179 (0.014)
Good 4
4284 (0.057)
2160 (0.014)
Good 5
4224 (0.055)
2087 (0.013)
Panel B
0 corners
3295
599
1949
1940
11
1 corners
1229
2757
2713
2425
198
2 corners
393
1639
338
608
1280
3 corners
83
5
0
27
2536
4 corners
0
966
5 corners
0
9
Notes: Panel A reports the number of observations in which each good is consumed, with the average budget
share of each good in the parenthesis. Panel B breaks down the sample by number of non-consumed goods.
Dataset 1
Dataset 2
Dataset 3
Dataset 4
Dataset 5
Price of
Good 1
Good 2
Good 3
Good 4
Expenditure
Table 2: Goodness of Fit Tests for Simulated Datasets 1-5
Log Likelihood (BIC)
Model I
Model II
19519.0 (-38884.7)
19431.8 (-38735.7)
15788.5 (-31423.7)
15691.1 (-31254.5)
10435.1 (-20716.8)
10204.5 (-20281.2)
35939.4 (-71538.1)
35794.0 (-71289.8)
13326.7 (-26312.7)
13289.8 (-26281.5)
Model III
19439.8 (-38768.9)
14986.9 (-29861.2)
9610.0 (-19109.3)
34933.6 (-69645.7)
12990.8 (-25760.2)
Table 3A: True Price and Expenditure Elasticities for Simulated Dataset 1
Expected Quantity of
Good 1
Good 2
Good 3
-2.425
-0.608
-0.608
-0.608
-2.425
-0.608
-0.608
-0.608
-2.425
1.789
1.789
1.789
1.852
1.852
1.852
Good 4
0.272
0.272
0.272
-1.561
0.745
Price of
Table 3B: Price and Expenditure Elasticities for Simulated Dataset 1 predicted by Model I
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-2.369 (0.031)
-0.597 (0.024)
-0.600 (0.022)
0.228 (0.005)
Good 2
-0.596 (0.023)
-2.365 (0.031)
-0.592 (0.022)
0.225 (0.005)
Good 3
-0.612 (0.024)
-0.609 (0.029)
-2.406 (0.030)
0.232 (0.005)
Good 4
1.851 (0.042
1.807 (0.049)
1.850 (0.050)
-1.498 (0.010)
Expenditure
1.726 (0.069)
1.764 (0.068)
1.747 (0.066)
0.812 (0.014)
33
Price of
Table 3C: Price and Expenditure Elasticities for Simulated Dataset 1 predicted by Model II
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-2.197 (0.027)
-0.404 (0.018)
-0.411 (0.019)
0.178 (0.004)
Good 2
-0.408 (0.018)
-2.169 (0.026)
-0.402 (0.018)
0.177 (0.003)
Good 3
-0.423 (0.019)
-0.413 (0.018)
-2.227 (0.025)
0.187 (0.003)
Good 4
2.028 (0.046)
1.986 (0.044)
2.040 (0.042)
-1.542 (0.009)
Expenditure
1
1
1
1
Price of
Table 3D: Price and Expenditure Elasticities for Simulated Dataset 1 predicted by Model III
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-2.132 (0.021)
0.041 (0.003)
0.042 (0.003)
0.105 (0.002)
Good 2
0.041 (0.003)
-2.114 (0.024)
0.042 (0.003)
0.103 (0.002)
Good 3
0.043 (0.003)
0.043 (0.003)
-2.152 (0.021)
0.109 (0.002)
Good 4
1.595 (0.004)
1.582 (0.040)
1.611 (0.041)
-1.483 (0.012)
Expenditure
1.547 (0.031)
1.551 (0.031)
1.542 (0.031)
0.834 (0.001)
Price of
Good 1
Good 2
Good 3
Good 4
Expenditure
Table 4A: True Price and Expenditure Elasticities for Simulated Dataset 2
Expected Quantity of
Good 1
Good 2
Good 3
-4.619
1.337
1.337
1.591
-2.728
0.658
1.591
0.658
-2.728
1.735
0.405
0.405
-0.298
1.644
1.644
Good 4
0.149
0.138
0.138
-1.423
0.998
Price of
Table 4B: Price and Expenditure Elasticities for Simulated Dataset 2 predicted by Model I
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-4.430 (0.064)
1.247 (0.025)
1.216 (0.025)
0.150 (0.004)
Good 2
1.496 (0.034)
-2.631 (0.025)
0.579 (0.018)
0.110 (0.004)
Good 3
1.473 (0.037)
0.589 (0.018)
-2.669 (0.025)
0.109 (0.004)
Good 4
1.670 (0.070)
0.417 (0.038)
0.496 (0.039)
-1.370 (0.005)
Expenditure
-0.210 (0.095)
1.556 (0.054)
1.526 (0.051)
1.000 (0.007)
Price of
Table 4C: Price and Expenditure Elasticities for Simulated Dataset 2 predicted by Model II
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-3.953 (0.056)
1.184 (0.028)
1.153 (0.030)
0.087 (0.004)
Good 2
1.242 (0.032)
-2.497 (0.026)
0.487 (0.017)
0.114 (0.003)
Good 3
1.223 (0.033)
0.494 (0.017)
-2.528 (0.026)
0.123 (0.003)
Good 4
0.508 (0.025)
0.802 (0.020)
0.858 (0.018)
-1.325 (0.004)
Expenditure
1
1
1
1
34
Price of
Table 4D: Price and Expenditure Elasticities for Simulated Dataset 3 predicted by Model III
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-2.805 (0.061)
0.266 (0.015)
0.271 (0.015)
0.191 (0.004)
Good 2
0.239 (0.013)
-2.325 (0.050)
0.201 (0.010)
0.130 (0.002)
Good 3
0.251 (0.017)
0.207 (0.011)
-2.349 (0.042)
0.136 (0.003)
Good 4
0.750 (0.027)
0.594 (0.021)
0.602 (0.021)
-1.291 (0.006)
Expenditure
1.574 (0.066)
1.258 (0.034)
1.275 (0.031)
0.844 (0.006)
Price of
Good 1
Good 2
Good 3
Good 4
Expenditure
Table 5A: True Price and Expenditure Elasticities for Simulated Dataset 3
Expected Quantity of
Good 1
Good 2
Good 3
-4.770
1.266
1.266
1.266
-4.770
1.266
1.266
1.266
-4.770
0.564
0.564
0.564
1.674
1.674
1.674
Good 4
0.099
0.099
0.099
-1.136
0.838
Price of
Table 5B: Price and Expenditure Elasticities for Simulated Dataset 3 predicted by Model I
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-4.525 (0.055)
1.189 (0.041)
1.167 (0.041)
0.086 (0.003)
Good 2
1.240 (0.040)
-4.557 (0.066)
1.242 (0.047)
0.087 (0.003)
Good 3
1.206 (0.045)
1.290 (0.050)
-4.552 (0.066)
0.085 (0.003)
Good 4
0.457 (0.077)
0.357 (0.073)
0.515 (0.078)
-1.023 (0.003)
Expenditure
1.620 (0.097)
1.766 (0.089)
1.628 0.096)
0.845 (0.006)
Price of
Table 5C: Price and Expenditure Elasticities for Simulated Dataset 3 predicted by Model II
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-4.074 (0.062)
1.205 (0.045)
1.187 (0.045)
0.054 (0.002)
Good 2
1.248 (0.045)
-4.107 (0.064)
1.270 (0.045)
0.046 (0.002)
Good 3
1.225 (0.047)
1.330 (0.045)
-4.092 (0.064)
0.042 (0.002)
Good 4
0.606 (0.030)
0.577 (0.030)
0.638 (0.030)
-1.143 (0.003)
Expenditure
1
1
1
1
Price of
Table 5D: Price and Expenditure Elasticities for Simulated Dataset 3 predicted by Model III
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 1
-3.824 (0.094)
0.527 (0.017)
0.534 (0.016)
0.139 (0.003)
Good 2
0.503 (0.016)
-3.991 (0.070)
0.528 (0.014)
0.138 (0.002)
Good 3
0.508 (0.015)
0.529 (0.016)
-4.041 (0.062)
0.140 (0.003)
Good 4
0.242 (0.018)
0.253 (0.018)
0.257 (0.019)
-1.055 (0.004)
Expenditure
2.571 (0.042)
2.682 (0.042)
2.722 (0.042)
0.739 (0.004)
35
Price of
Good 1
Good 2
Good 3
Good 4
Good 5
Good 6
Expenditure
Table 6A: True Price and Expenditure Elasticities for Simulated Dataset 4
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 5
-2.994
0.420
0.420
0.420
0.420
0.420
-2.994
0.420
0.420
0.420
0.420
0.420
-2.994
0.420
0.420
0.420
0.420
0.420
-2.994
0.420
0.420
0.420
0.420
0.420
-2.994
0.799
0.799
0.799
0.799
0.799
0.513
0.513
0.513
0.513
0.513
Good 6
0.024
0.024
0.024
0.024
0.024
-1.316
1.124
Price of
Table 6B: Price and Expenditure Elasticities for Simulated Dataset 4 predicted by Model I
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 5
Good 6
-3.097(0.109) 0.385 (0.070) 0.466 (0.057) 0.472 (0.074) 0.459 (0.065) 0.025 (0.003)
Good 1
0.441 (0.066) -3.252(0.113) 0.455 (0.086) 0.399 (0.057) 0.547 (0.066) 0.032 (0.004)
Good 2
0.525 (0.068) 0.556 (0.083) -3.065(0.100) 0.345(0.071) 0.283 (0.083) 0.027 (0.004)
Good 3
0.438 (0.069) 0.393 (0.056) 0.370 (0.076) -3.009(0.099) 0.501 (0.078) 0.024 (0.004)
Good 4
0.408 (0.060) 0.527 (0.074) 0.379 (0.083) 0.410 (0.071) -3.220(0.083) 0.038 (0.004)
Good 5
0.735 (0.102) 0.810 (0.101) 0.820(0.092) 0.838 (0.083) 0.835 (0.078) -1.317(0.006)
Good 6
0.549 (0.147) 0.579 (0.130) 0.574 (0.111) 0.544 (0.112) 0.594 (0.109) 0.884 (0.011)
Expenditure
Price of
Table 6C: Price and Expenditure Elasticities for Simulated Dataset 4 predicted by Model II
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 5
Good 6
-3.163(0.077) 0.304 (0.078) 0.467 (0.054) 0.408 (0.074) 0.381 (0.071) 0.058 (0.004)
Good 1
0.256 (0.069) -3.334(0.106) 0.479 (0.078) 0.313 (0.062) 0.514 (0.069) 0.056 (0.004)
Good 2
0.430 (0.051) 0.469 (0.079) -3.140(0.082) 0.244 (0.059) 0.230 (0.065) 0.057 (0.003)
Good 3
0.390 (0.072) 0.318 (0.059) 0.282 (0.062) -3.168(0.112) 0.429 (0.049) 0.062 (0.004)
Good 4
0.362 (0.068) 0.525 (0.069) 0.257 (0.067) 0.427 (0.047) -3.391(0.076) 0.059 (0.003)
Good 5
0.725 (0.050) 0.720 (0.056) 0.659(0.042) 0.777 (0.056) 0.841 (0.039) -1.293(0.005)
Good 6
1
1
1
1
1
1
Expenditure
Price of
Table 6D: Price and Expenditure Elasticities for Simulated Dataset 4 predicted by Model III
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 5
Good 6
-3.395(0.073) 0.248 (0.009) 0.245 (0.010) 0.252 (0.010) 0.259 (0.011) 0.112 (0.003)
Good 1
0.245 (0.009) -3.323(0.070) 0.236 (0.007) 0.243 (0.009) 0.249 (0.009) 0.108 (0.004)
Good 2
0.241 (0.009) 0.236 (0.007) -3.274(0.056) 0.237(0.008) 0.244 (0.009) 0.106 (0.003)
Good 3
0.244 (0.010) 0.239 (0.009) 0.234 (0.010) -3.372(0.072) 0.248 (0.009) 0.107 (0.003)
Good 4
0.250 (0.010) 0.245 (0.107) 0.240 (0.098) 0.249 (0.094) -3.452(0.142) 0.110 (0.004)
Good 5
0.722 (0.015) 0.706 (0.021) 0.695(0.014) 0.716 (0.021) 0.734 (0.022) -1.281(0.005)
Good 6
1.692 (0.040) 1.649 (0.037) 1.623 (0.030) 1.673 (0.036) 1.716 (0.043) 0.836 (0.005)
Expenditure
36
Price of
Good 1
Good 2
Good 3
Good 4
Good 5
Good 6
Expenditure
Table 7A: True Price and Expenditure Elasticities for Simulated Dataset 5
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 5
-5.214
0.458
0.458
0.458
0.458
0.458
-5.214
0.458
0.458
0.458
0.458
0.458
-5.214
0.458
0.458
0.458
0.458
0.458
-5.214
0.458
0.458
0.458
0.458
0.458
-5.214
1.781
1.781
1.781
1.781
1.781
1.601
1.601
1.601
1.601
1.601
Good 6
0.035
0.035
0.035
0.035
0.035
-1.129
-1.043
Price of
Table 7B: Price and Expenditure Elasticities for Simulated Dataset 5 predicted by Model I
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 5
Good 6
Good 1
-4.965(0.191) 0.399 (0.074) 0.335 (0.079) 0.439 (0.125) 0.535 (0.121) 0.034 (0.003)
Good 2
0.393 (0.068) -5.588(0.225) 0.628 (0.129) 0.414 (0.077) 0.648 (0.163) 0.038 (0.003)
Good 3
0.328 (0.075) 0.648 ( 0.124) -5.017(0.215) 0.398(0.110) 0.348 (0.123) 0.035 (0.002)
Good 4
0.419 (0.109) 0.437 (0.086) 0.406 (0.109) -5.252(0.262) 0.450 (0.089) 0.038 (0.002)
Good 5
0.463 (0.094) 0.623 (0.153) 0.330 (0.113) 0.389 (0.075) -5.525(0.240) 0.041 (0.003)
Good 6
1.585 (0.219) 1.769 (0.268) 1.853 (0.233) 1.933 (0.213) 1.962 (0.202) -1.137(0.003)
Expenditure
1.777 (0.271) 1.713(0.319) 1.465 (0.262) 1.679 (0.264) 1.583 (0.286) -1.048(0.006)
Price of
Table 7C: Price and Expenditure Elasticities for Simulated Dataset 5 predicted by Model II
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 5
Good 6
Good 1
-4.846(0.197) 0.568 (0.111) 0.304 (0.107) 0.531 (0.140) 0.494 (0.122) 0.029 (0.002)
Good 2
0.565 (0.089) -5.328(0.205) 0.583 (0.129) 0.462 (0.087) 0.702 (0.137) 0.030 (0.002)
Good 3
0.295 (0.096) 0.608 (0.127) -4.716(0.119) 0.422(0.102) 0.378 (0.100) 0.030 (0.002)
Good 4
0.499 (0.121) 0.478 (0.089) 0.421 (0.102) -5.020(0.149) 0.465 (0113) 0.032 (0.002)
Good 5
0.433 (0.109) 0.671 (0.110) 0.355 (0.089) 0.417 (0.110) -5.205(0.209) 0.035 (0.001)
Good 6
2.054 (0.104) 2.049 (0.154) 2.049 (0.109) 2.225 (0.128) 2.158 (0.115) -1.158(0.003)
Expenditure
1
1
1
1
1
1
Price of
Table 7D: Price and Expenditure Elasticities for Simulated Dataset 5 predicted by Model III
Expected Quantity of
Good 1
Good 2
Good 3
Good 4
Good 5
Good 6
Good 1
-5.711(0.182) 0.169 (0.010) 0.183 (0.012) 0.181 (0.010) 0.181 (0.012) 0.060 (0.002)
Good 2
0.169 (0.010) -5.422(0.175) 0.171 (0.007) 0.165 (0.011) 0.171 (0.018) 0.056 (0.002)
Good 3
0.179 (0.014) 0.169 (0.008) -5.496(0.143) 0.161 (0.008) 0.168 (0.011) 0.057 (0.002)
Good 4
0.170 (0.010) 0.157 (0.009) 0.155 (0.008) -5.549(0.182) 0.182 (0.010) 0.058 (0.002)
Good 5
0.165 (0.094) 0.157 (0.009) 0.155 (0.009) 0.175 (0.008) -5.779(0.230) 0.062 (0.002)
Good 6
1.660 (0.055) 1.573 (0.070) 1.595 (0.047) 1.604 (0.078) 1.680 (0.079) -1.122(0.003)
Expenditure
3.367 (0.127) 3.197 (0.107) 3.237 (0.099) 3.263 (0.102) 3.397 (0.139) 0.827 (0.004)
37
Detergent
Softener
Pasta
Pasta sauce
Table 8: Summary Statistics of Scanner Panel Datasets
Dataset A
Dataset B
(M=5 goods, N=16589 observations)
(M=6 goods, N=10129 observations)
Number of
Avg.
Avg. price
Number of
Avg.
Avg. price
purchase
Budget
in
purchase
budget
in
obs.
Share
cents/oz.
obs.
share
cents/oz.
2607
0.0111
7.09
Blueberry
404
0.0020
9.23
1796
0.0051
6.89
Mixed-berry
198
0.0016
9.11
4243
0.0047
9.50
Strawberry
479
0.0021
9.23
2505
0.0049
9.18
Plain
339
0.0014
9.23
Pina-colada
457
0.0021
9.23
Basket Expenditure in
Number of Observations with
dollars (Std. Dev.)
0 corners 1 corner 2 corners 3 corners 4 corners 5 corners
Dataset A
55.68 (36.26)
79
513
2203
4965
8829
Dataset B
49.48 (39.50)
0
6
82
276
1056
8709
Note: the average budget share of each good in Table 7 are calculated over all purchase and non-purchase observations
Dataset A
Dataset B
Table 9: Goodness of Fit Tests for Scanner Panel Datasets
Log Likelihood (BIC)
Model I
Model II
Estimation Sample
1982.6 (-3135.2)
1674.7 (-2916.9)
Hold out Sample
650.0 (-879.7)
538.0 (-689.3)
Estimation Sample
-255.1 (1025.2)
-286.4 (1043.0)
Hold out Sample
-268.9 (1001.1)
-293.6 (1010.6)
Model III
1719.5 (-3053.5)
584.2 (-823.7)
-316.6 (1023.6)
-323.6 (998.2)
Price of
Table 10A: Price and Expenditure Elasticities for Scanner Dataset A predicted by Model I
Expected Quantity of
Detergent
Softeners
Pasta
Pasta Sauce
Comp. Good
Detergent
-2.438 (0.129)
-0.127 (0.034)
0.005 (0.019)
0.008 (0.019)
0.019 (0.003)
Softener
-0.058 (0.016)
-1.880 (0.088)
0.060 (0.016)
0.042 (0.018)
0.005 (0.001)
Pasta
0.001 (0.009)
0.060 (0.018)
-1.695 (0.065)
-0.115 (0.021)
0.005 (0.000)
Pasta Sauce
0.003 (0.009)
0.040 (0.020)
-0.091 (0.016)
-1.994 (0.084)
0.006 (0.001
Expenditure
1.936 (0.112)
1.881 (0.096)
1.469 (0.088)
2.021 (0.094)
0.939 (0.002)
Price of
Table 10B: Price and Expenditure Elasticities for Scanner Dataset A predicted by Model II
Expected Quantity of
Detergent
Softeners
Pasta
Pasta Sauce
Comp. Good
Detergent
-2.590 (0.139)
-0.047 (0.030)
0.040 (0.022)
0.005 (0.020)
0.017 (0.002)
Softener
-0.025 (0.014)
-1.667 (0.109)
0.114 (0.016)
0.075 (0.018)
0.003 (0.001)
Pasta
0.019 (0.011)
0.099 (0.022)
-1.685 (0.063)
-0.043 (0.022)
0.003 (0.001)
Pasta Sauce
0.002 (0.041)
0.061 (0.024)
-0.042 (0.022)
-2.015 (0.110)
0.005 (0.001)
Expenditure
1
1
1
1
1
38
Promo on
Table 11A: Promotional Elasticities for Scanner Dataset A predicted by Model I
Expected Quantity of
Detergent
Softeners
Pasta
Pasta Sauce
Detergent
1.141 (0.271)
0.096 (0.035)
-0.004 (0.018)
-0.006 (0.019)
Softener
0.089 (0.043)
1.614 (0.504)
-0.102 (0.045)
-0.068 (0.038)
Pasta
-0.001 (0.006)
-0.036 (0.013)
0.408 (0.086)
0.066 (0.017)
Pasta Sauce
-0.002 (0.006)
-0.024 (0.014)
0.052 (0.014)
0.598 (0.131)
Promo on
Table 11B: Promotional Elasticities for Scanner Dataset A predicted by Model II
Expected Quantity of
Detergent
Softeners
Pasta
Pasta Sauce
Detergent
1.429 (0.238)
0.039 (0.019)
-0.033 (0.014)
-0.003 (0.017)
Softener
0.063 (0.033)
1.817 (0.445)
-0.252 (0.059)
-0.162 (0.054)
Pasta
-0.014 (0.008)
-0.062 (0.016)
0.458 (0.124)
0.026 (0.013)
Pasta Sauce
-0.002 (0.005)
-0.034 (0.017)
0.023 (0.012)
0.617 (0.176)
Promo on
Price of
Table 10C: Price and Expenditure Elasticities for Scanner Dataset A predicted by Model III
Expected Quantity of
Detergent
Softeners
Pasta
Pasta Sauce
Comp. Good
Detergent
-3.348 (0.112)
0.052 (0.008)
0.026 (0.004)
0.045 (0.005)
0.023 (0.002)
Softener
0.023 (0.003)
-2.912 (0.092)
0.012 (0.001)
0.016 (0.002)
0.009 (0.001)
Pasta
0.011 (0.001)
0.010 (0.001)
-2.789 (0.056)
0.024 (0.002)
0.008 (0.000)
Pasta Sauce
0.020 (0.002)
0.016 (0.002)
0.027 (0.002)
-3.306 (0.129)
0.011 (0.001)
Expenditure
1.434 (0.078)
1.244 (0.070)
1.197 (0.059)
1.406 (0.083)
0.991 (0.002)
Table 11C: Promotional Elasticities for Scanner Dataset A predicted by Model III
Expected Quantity of
Detergent
Softeners
Pasta
Pasta Sauce
Detergent
1.069 (0.232)
-0.024 (0.005)
-0.010 (0.004)
-0.017 (0.004)
Softener
-0.017 (0.005)
2.090 (0.775)
-0.008 (0.003)
-0.011 (0.003)
Pasta
-0.002 (0.001)
-0.001 (0.098)
0.199 (0.002)
-0.003 (0.002)
Pasta Sauce
-0.004 (0.002)
-0.003 (0.002)
-0.004 (0.136)
0.341 (0.001)
Price of
Table 12A: Price and Expenditure Elasticities for Scanner Dataset B predicted by Model I
Expected Quantity of
BB
MB
SB
Plain
PC
Comp. Good
BB
-3.635(0.286) 0.228 (0.062) 0.247 (0.128) 0.119 (0.069) 0.093 (0.050) 0.004 (0.001)
MB
0.185 (0.107) -2.463(0.316) 0.215 (0.087) 0.013 (0.035) 0.053 (0.067) 0.002 (0.002)
SB
0.214 (0.085) 0.224(0.063) -3.180(0.276) 0.133 (0.068) 0.102 (0.063) 0.003 (0.001)
Plain
0.085 (0.052) 0.010 (0.020) 0.107 (0.525) -3.590(0.295) 0.049 (0.046) 0.004 (0.001)
PC
0.096 (0.034) 0.065 (0.023) 0.111 (0.040) 0.067 (0.031) -2.568(0.312) 0.003 (0.009)
Expenditure 1.480 (0.107) 0.926 (0.175) 1.400 (0.121) 1.425 (0.163) 1.556 (0.159) 0.994 (0.004)
39
Price of
Table 12B: Price and Expenditure Elasticities for Scanner Dataset B predicted by Model II
Expected Quantity of
BB
MB
SB
Plain
PC
Comp. Good
BB
-3.213(0.199) 0.225 (0.056) 0.162 (0.057) 0.035 (0.052) 0.152 (0.059) 0.004 (0.000)
MB
0.172 (0.071) -2.806(0.380) 0.172 (0.068) 0.038 (0.020) 0.087 (0.053) 0.002 (0.001)
SB
0.148 (0.061) 0.211 (0.070) -2.757(0.180) 0.005 (0.061) 0.037 (0.026) 0.003 (0.001)
Plain
0.025 (0.067) 0.030 (0.032) 0.004 (0.030) -3.439(0.223) 0.030 (0.019) 0.003 (0.001)
PC
0.161 (0.075) 0.116 (0.034) 0.040 (0.025) 0.052 (0.026) -3.140(0.268) 0.004 (0.001)
Expenditure
1
1
1
1
1
1
Price of
Table 12C: Price and Expenditure Elasticities for Scanner Dataset B predicted by Model III
Expected Quantity of
BB
MB
SB
Plain
PC
Comp. Good
BB
-3.557 (.204) 0.013 (0.002) 0.024 (0.004) 0.021 (0.004) 0.014 (0.003) 0.006 (0.001)
MB
0.008 (0.002) -2.354(0.203) 0.006 (0.001) 0.002 (0.001) 0.006 (0.001) 0.003 (0.000)
SB
0.022 (0.003) 0.007 (0.001) -2.977(0.216) 0.022 (0.003) 0.010 (0.002) 0.005 (0.000)
Plain
0.019 (0.003) 0.002 (0.001) 0.020 (0.003) -3.293(0.212) 0.008 (0.002) 0.004 (0.001)
PC
0.012 (0.003) 0.006 (0.001) 0.009 (0.002) 0.008 (0.002) -2.759(0.186) 0.004 (0.001)
Expenditure 1.037 (0.110) 0.703 (0.091) 0.904(0.104) 0.964 (0.109) 0.821 (0.012) 1.001 (0.001)
Table 13A: Compensating Values of Goods in Scanner Dataset A
Per Trip Compensating Value in Cents (Std. Error)
Detergent
Softeners
Pasta
Pasta Sauce
Model I
40.40 (4.39)
30.93 (3.54)
44.44 (5.27)
24.07 (2.71)
Model II
34.84 (4.37)
45.07 (4.82)
36.08 (4.47)
24.56 (3.79)
Model III
26.89 (3.42)
18.86 (2.21)
18.15 (1.20)
14.98 (1.97)
Model I
Model II
Model III
Table 13B: Compensating Values of Goods in Scanner Dataset B
Per Trip Compensating Value in Cents (Std. Error)
BB
MB
SB
Plain
3.45 (0.36)
6.47 (0.55)
4.32 (0.39)
3.21 (0.44)
4.14 (0.42)
5.76 (0.47)
5.56 (0.40)
2.78 (0.33)
4.50 (0.40)
7.59 (0.76)
5.87 (0.61)
4.41 (0.47)
PC
6.29 (0.72)
5.50 (0.69)
5.75 (0.58)
40
WEB APPENDIX
“GLOBAL REGULARITY AND FLEXIBILITY OF DEMAND SYSTEMS IN THE
PRESENCE OF NON-NEGATIVITY CONSTRAINTS: A THEORETICAL AND
EMPIRICAL ANALYSIS”
Table of Contents
Section
Topic
Page
Numbers
1
Proofs of Lemmas and
1-14
Propositions
2
Parameter Estimates for Simulated
15-20
Datasets 1-5 and Scanner Panel
Datasets A-B
SECTION 1: PROOFS OF LEMMAS AND PROPOSITIONS
Lemma 1: For a given set of values of the observed budget shares, the random parameters and the normalized price
arguments of the M goods (i.e., {sc, ac,  c } cM1 ), the (log) marginal utilities of prices and expenditure for the Basic
Translog utility will be non-positive and positive respectively if and only if the following scalar D be positive:
M 1
D
a M    c a c   ln  M
c 1
M 1
sM    c sc
c 1
where {δ c} and τ are elements of matrix B given in equation (8) and  M is the normalized price argument of good M
which takes the value of its normalized market price p M if sM>0, and its normalized virtual price TM if sM=0.
Proof of Lemma 1: The proof is an immediate consequence of the following two lemmas (Lemma
1.1 and Lemma 1.2):
Lemma 1.1: For a given set of values of the observed budget shares, the (log) marginal utilities of prices will be nonpositive for all goods c=1..M and the log marginal utility of expenditure will be positive if and only if the log marginal
utility of expenditure is positive.
1
Lemma 1.2: For a given set of values of the observed budget shares, the random parameters and the normalized price
arguments of the M goods (i.e., {sc,ac,  c } cM1 ), the (log) marginal utility of expenditure for the Basic Translog utility
is equivalent to the scalar D given in equation (9) in Lemma 1.
Proof of Lemma 1.1: We will only prove the claim on one direction, which is that for a given set of
{sc, ac,  c } cM1 , the log marginal utility of prices of all goods will be necessarily non-negative if the log
marginal utility of expenditure is positive4. From equations (1)-(4), the two (log) marginal utilities are
related to the observed budget share of good c as
sc  
 ln V  ln  c
 ln V  ln y
Since the observed budget share of good c, sc, is non-negative in the feasible space of budget shares,
M
Ωs ª{{sc} cM1 |  sc =1, sc ≥0}, it follows from the above equation that if the (log) marginal utility of
c 1
expenditure is positive, then the (log) marginal utility of price of all goods c =1..M will necessarily be
non-positive. This implies that regularity property 2 only requires us to ensure that the (log) marginal
utility of expenditure is positive.
Proof of Lemma 1.2: We start with a general expression of the optimal solutions of budget shares of
all c=1..M goods, S(  ), as given in equation (6) and (7) for the Basic Translog utility and set the
observed vector of the budget shares s =[s1, s2,.., sM] as the optimal solutions of the budget shares.
This yields the following expression of the observed budget shares of all goods
s
A  B ln 
 ln V    ln y
(A.1)
where Aª [a1, a2,.., aM] is the M×1 vector of the parameters,  ª [  1 , ..,  M ] are the normalized price
arguments, and  ln V    ln y is the (log) marginal utility of expenditure. Next, we pre-multiply the
RHS and LHS of equation (A.1) by an 1×M vector, [, 1]T, where  ª [1, 2,.., M-1]. Following that,
we substitute the block-wise expression for B given in equation (8) of the paper. Doing so yields the
expression of  ln V    ln y in terms of observed budget shares, s ª[s1,.., sM], as
The claim in the other direction is a tautology, since it says that if the log marginal utility of expenditure is positive and
the log marginal utility of prices are non-positive, then the log marginal utility of expenditure will be positive.
4
2
M 1
 ln V    ln y 
a M    c a c   ln  M
c 1
M 1
s M    c sc
c 1
Note that the expression for  ln V    ln y in the above equation is the same as that of the scalar D
given in equation (9). [QED]
Lemma 2: For a given set of values of the observed budget shares, the random parameters and the normalized price
arguments of the M goods (i.e., {sc ,ac,  c } cM1 ), the M×M Slutsky matrix of the Basic Translog utility will be
negative semi-definite with rank of M-1 if and only if the following matrix J is positive definite:
J  sˆ   sˆsˆ T 
 sˆsˆ T   I  sˆeˆ T   T Z 1 I  eˆ   sˆ T 
 ln V    ln y
where J is an (M-1)×(M-1) symmetric matrix, I is an (M-1)×(M-1) identity matrix, ê is an (M-1)×1 vector of
ones, ŝ=[s1,s2,.., sM-1] is an (M-1)×1 vector of observed budget shares of the first M-1 goods, Δ(ŝ) is a diagonal (M1)×(M-1) matrix with the vector ŝ on its diagonal, and  ln V    ln y is the log marginal utility of expenditure
Proof of Lemma 2: The proof is an immediate consequence of the following two lemmas (Lemma
2.1 and Lemma 2.2):
Lemma 2.1: The Slutsky matrix for Basic Translog utility will be negative semi-definite with a rank of M-1 if and
only if the following M×M matrix, G, is positive semi-definite with a rank of M-1
G  s   ss T 
B  s Be   Bes T  e T Bess T
 ln V    ln y
T
where B{bi,c} is the M×M matrix of parameters, e is an M×1 vector of ones, s [s1,.., sM] is the M×1 vector of
observed budget shares, Δ(s) is a diagonal M×M matrix with vector s on its diagonal
Lemma 2.2: The M×M matrix G in Lemma 2.1 will be positive semi-definite with a rank of M-1 if and only if the
following (M-1)×(M-1) matrix J is positive definite:
J   sˆ   sˆsˆ T 
 sˆsˆ T   I  sˆeˆ T   T Z 1 I  eˆ   sˆ T 
 ln V    ln y
where I is an (M-1)×(M-1) identity matrix, ê is an (M-1)×1 vector of ones, ŝ=[s1, s2,.., sM-1] is an (M-1)×1 vector
of observed budget shares of first M-1 goods, and Δ(ŝ) is a diagonal (M-1)×(M-1) matrix with the vector ŝ on its
diagonal.
3
Proof of Lemma 2.1: The M×M Slutsky substitution matrix for the Basic Translog utility can be
derived in terms of the observed vector of budget shares, s =[s1,., sM], as



B  s Be T  Bes T  e T Bess T 
1
K   1 s   ss T 
 



D


(A.2)
In equation (A.2), the second term on the RHS is an M×M symmetric matrix; and the first and third
bracketed terms in the RHS are inverses of the matrix Δ(  ) where Δ(  ) is an M×M diagonal
matrix that has the vector of the normalized price arguments of the M goods,  ª [  1 ,  2 ,..,  M ],
on its diagonal. Since the normalized price arguments are finite, the M×M diagonal matrix, [Δ(  )]-1
will be a full rank matrix. Note that a basic property of positive semi-definite matrices is that as long
as the MäM matrix H has a full rank, then the MäM matrix HAHT (where A is an MäM symmetric
matrix) will be positive semi-definite with a rank M-1 if and only if A is positive semi-definite with a
rank of M-1. Thus it follows that the Slutsky substitution matrix will be negative semi-definite with a
rank of M-1 if and only if the second bracketed term in the RHS of equation (A.2) is positive semidefinite with a rank of M-1, which we write as
G  s   ss
B  sBe

T
T
 Bes T  eT BessT
D 

(A.3)
where G is an M×M symmetric matrix, e is an M×1 vector of ones, s =[s1, s2,.., sM] is an M×1 vector
of budget shares of all M goods, Δ(s) is a diagonal M×M matrix that has the vector of budget shares
s on its diagonal. [QED]
Proof of Lemma 2.2: In equation (A.3), note that since eTs =1 (since the budget shares need to sum
to one), the matrix G satisfies the following two conditions: eTG=0 and Ge=0. Thus it follows from
Theorem M.D.4 on page 939 in Mas-Colell et al. (1995) that the M×M matrix G will be positive
semi-definite with rank of M-1 if and only if any principal (M-1)×(M-1) sub-matrix of G (that is, the
matrix obtained by deleting any one row and its corresponding column of G) is positive definite.
Thus, consider the (M-1)×(M-1) principal sub-matrix which is obtained by deleting the Mth row and
Mth column of matrix G. After substituting the block-wise expression of the matrix B given in
equation (8) of the main paper into equation (A.3), we get this principal sub-matrix as
J  sˆ  sˆsˆT 
sˆsˆT  I  sˆeˆT   T Z 1 I  eˆ   sˆT 
D  
4
where I is an (M-1)×(M-1) identity matrix, ê is an (M-1)×1 vector of ones, ŝ ª [s1, s2,.., sM-1] is an (M1)×1 vector of the observed budget shares of the first M-1 goods, the term Δ(ŝ) is a diagonal (M1)×(M-1) matrix that has the vector of budget shares ŝ on its diagonal. [QED]
Proposition 1: The necessary and sufficient conditions that need to be imposed on the parameter space of the Basic
Translog utility so that the requirement for regularity property 2 is satisfied in the entire feasible space of variables are
(i) The parameter τ=0
(ii) The (M-1)×1 vector  >0 (or c >0 for all c=1..M-1)
M 1
(iii) The parameter a M     c a c
c 1
Proof of Proposition 1: Recall from section 3.2.1 that the requirement for regularity property 2 is
that the expression for D in lemma 1 should be non-positive for each vector of observed budget
M
shares sª[s1,..,sM] in the feasible space Ωs ª{{sc} cM1 |  sc =1, sc ≥0}, and at all corresponding values of
c 1
{ln pc } cM1 (-¶,¶)M and {ac} cM1 (-¶,¶)M that satisfy the demand equations when the observed
budget share vector is s ª[s1,..,sM].
We first prove the necessity of conditions (i)–(iii) in proposition 1. We start with conditions
(i) and (iii). Following that, we prove the necessity of condition (ii) conditional on (i) & (iii).
Consider the observation in which the observed budget shares of the M goods are sc=0 " c=1..M-1
and sc=1 (i.e., when only good M is purchased). For this observation, the expression for D follows
form equation (9) as
M 1
D  a M    c a c   ln p M
(A.4)
c 1
And the M-1 demand equations (which are the regime conditions of non-purchased goods) for this
observation follow from equations (2) - (4) as
M 1
ln pc   ai zi ,c   c ln p M
i 1
" c=1..M-1
(A.5)
where {zi,c} are elements of matrix Z. Thus the requirement for regularity property 2 to be satisfied
for this specific observation is that the expression for D in equation (A.4) should be positive for all
values of ln p M in the space  ,   and all values of ac cM1 in the space (-¶,¶)M which satisfy the
inequalities in (A.5).
5
Given the aforementioned requirement for this specific observation, we will proceed in two
steps. In step 1, we will determine the set of feasible values of ln p M and a c cM1 that can satisfy the
inequalities in (A.5). In step 2, we determine the restrictions that need to be imposed on the
parameters so that D>0 for all feasible values of ln p M and a c cM1 identified in step 1.
Starting with step 1, recall that the log market prices of all other goods c∫M, ln pc cM11 , can
take any value in the space, (-¶,¶)M-1. This implies that the inequalities in (A.5) can be satisfied for
all values of ln p M in the space (-¶,¶) and all values of a c cM1 in the space (-¶,¶)M. Why? This is
because for any value of ln p M in the space (-¶,¶) and for any value of a c cM1 in the space (-¶,¶)M,
we can always find values of ln pc cM11 in the space (-¶,¶)M-1 that will satisfy the inequalities in (A.5).
Moving on to step 2, observe in equation (A.4) that D will be positive for all values of
ln p M in the interval (-¶,¶) and all values of a c cM1 in the interval (-¶,¶)M only if the coefficient of
M 1
ln p M in equation (A.4) is zero (i.e., τ=0) and a M     c a c . This proves the necessity of conditions
c 1
(i) and (iii).
We next prove the necessity of condition (ii) conditional on (i) and (iii). Consider the
observation in which only good i∫M is purchased– i.e., when the observed budget shares of the M
goods are sc=0 " c=1..M, c∫i and si=1 " i∫M. For this observation, the price argument of good M,
 M , in the expression for D in equation (9) takes the value of its normalized virtual price TM , whose
specification follows form equation (2). Thus substituting the expression for TM into equation (9),
we get the specification of D for this observation as
 i  a M    c ac     z i ,c ac   ln pi
M 1
D

c 1
M 1

c 1
  z i ,i
2
i
(A.6)
where pi is the normalized market price of the purchased good i and {zc,i} are the elements of the
matrix Z. Recall that as per condition (i), the parameter τ=0. Thus substituting τ=0 into equation
(A.6), we get
M 1
D   a M    c a c   i
c 1


(A.7)
Note that as per condition (iii), the numerator in equation (A.7) will be positive. Thus it follows that
the expression for D in equation (A.7) will be positive only if i >0. This proves the necessity of i
6
>0. Repeating the same exercise for all goods c=1..M-1 proves the necessity of condition (ii), which
is c >0 for all c=1..M-1.
Till now we have proven the necessity of conditions (i)-(iii). Next, we prove their sufficiency.
We start with the expression for D given in Lemma 1, which encompasses all feasible values of the
observed budget shares in the feasible space Ws. We rewrite it as
M 1
D
a M    c a c   ln  M
c 1
(A.8)
M 1
s M    c sc
c 1
Note that D in equation (A.8) will be positive as long as the conditions (i) –(iii) satisfied. To see why
M 1
that is so, first consider the numerator in the RHS of equation (A.8). Since a M     c a c and τ=0 (as
c 1
per conditions (i) and (iii)), the numerator will be positive. Next consider the denominator in the
RHS of equation (A.8). Since the observed budget shares can only take non-negative values in the
feasible space Ws and since c >0 "c=1..M-1 (as per condition ii), the denominator will be positive
for all vectors of observed budget shares in the feasible space Ws. This proves the sufficiency since it
shows that as long the conditions (i)–(iii) are satisfied, D will be positive for all values of variables in
their entire feasible space. [QED]
Corollary to Proposition 1: A convenient normalization of aM that satisfies condition (iii) in proposition 1 along
M 1
with the normalization condition given in section 3.1 is a M  1    c a c .
c 1
M 1
Proof of Corollary: It is easy to see that the normalization aM  1    c ac satisfies condition (iii) in
c 1
M 1
M 1
c 1
c 1
proposition 1 which requires aM     c ac . To see why the normalization aM  1    c ac satisfies
the normalization condition stated in section 3.1, first recall that the normalization condition in
section 3.1 requires the parameter aM to be normalized in such a way so that its normalization
M 1
M
c 1
i 1
reduces to aM = 1   a c once we impose the M homotheticity restrictions,  bc ,i  0 " c=1..M. Note
that these M homotheticity restrictions translate to ‘τ =0 and c =1 " c=1..M-1’ in the terms of our
block-wise representation of matrix B given in equation (8).5 Substituting these homotheticity
To see why that is so, note that the M homotheticity restrictions can be written in matrix form as Be=0, where Bª{bij}
is the MäM matrix of parameters and e is an Mä1 vector of ones. Substituting the block-wise representation of B given
5
7
restrictions into normalization of aM given in the corollary results in normalization to reduce to
M 1
a M  1   a c . [QED]
c 1
Proposition 2 (Regularity Property 3): Conditional on the set of restrictions stated in proposition 1, the
necessary and sufficient condition that needs to be imposed on the parameters so that the requirements for regularity
property 3 are satisfied is that the (M-1)×(M-1) matrix Z should be positive definite.
Proof of proposition 2: Recall from section 3.2 that the requirement for regularity property 3 is that
the (M-1)×(M-1) matrix J in Lemma 2 be positive semi-definite for all observed values of the budget
share vectors in the feasible space Ws and for all values of the (log) marginal utility of expenditure,
 ln V    ln y >0. To ensure that  ln V    ln y >0, we impose the conditions (i) – (iii) given in
proposition 1 into the expression for the matrix J given in Lemma 2, which yields
J  sˆ   sˆsˆ T 
I  sˆeˆ
T
 
  T  Z 1 I  eˆ   sˆ T
 ln V    ln y

" d >0
(A.9)
Next, we will show that the necessary and sufficient condition that needs to be imposed on the
parameters so that the (M-1)×(M-1) matrix J in equation (A.9) is positive definite in the entire
feasible space of the variables is that the (M-1)×(M-1) matrix Z should be positive definite.
We first prove the necessity of Z being positive definite. Consider the observation in which
only good M is purchased. For this observation, all elements in the (M-1)×1 vector, ŝ ª [s1, s2,.., sM-1],
will be zero. Substituting ŝ =0 into equation (A.9), we get J=Z-1/  ln V    ln y  . Since
 ln V    ln y >0, it follows that in order for J to be positive definite, Z has to be positive definite.
This proves the necessity.
To prove the sufficiency of Z being positive definite, consider the first term in the
expression for J in equation (A.9), which is the (M-1)×(M-1) matrix, Δ(ŝ) – ŝŝT. This term is positive
semi-definite in the space Ws. To see why that is so, note that the (M-1)×(M-1) matrix, Δ(ŝ) – ŝŝT, will
be positive semi-definite as long as for all non-zero (M-1)×1 vectors w=[w1, w2,.., wM-1], the following
in equation (8) into Be=0 yields the M homotheticity restrictions in terms of two sets of restrictions: (a) Z-1(d -ê) =0
(which consists of M-1 restrictions) and (b) dT Z-1(d -ê) + t=0 (which consists of one restriction), where ê is an (M-1)ä1
vector of ones, Z is the (M-1)ä (M-1) matrix of parameters, d is an (M-1)ä1 vector of parameters and t is a scalar. Since
Z-1 is an invertible matrix, restriction (a) implies that d = ê, which yields c =1 " c=1..M-1. And substituting d = ê into
restriction (b) yields t=0. Thus the M homotheticity restrictions reduce to ‘τ =0 and c =1 " c=1..M-1’.
8
condition is satisfied: wT(Δ(ŝ) – ŝŝT)w ≥0. Using the Lagrange’s Identity, the term wT(Δ(ŝ) – ŝŝT)w can
be written as
w T sˆ  sˆsˆT w 

1 M 1M 1
  si s j wi  w j
2 i 1 j 1
 s
M 1
2
M
 s i w i2
i 1
(A.10)
Observe in equation (A.10) that wT(Δ(ŝ)–ŝŝT)w will be non-negative since the observed budget shares
are non-negative. Thus so far we have shown that the first term in the expression for J in the RHS
of equation (A.9) is positive semi definite. Since the sum of a positive semi-definite and a positive
definite matrix is a positive definite matrix, it follows that as long as the second term in the RHS of
equation (A.9) is positive definite, the matrix J in equation (A.9) will be positive definite. Thus in
what follows, we show that as long as Z is positive definite, the second term in the RHS of equation
(A.9) will be positive definite, which will imply that the matrix J in equation (A.9) will be positive
definite
Represent the second term in the RHS of equation (A.9) by HZ-1HT/  ln V    ln y , in
which the term H is an (M-1)×(M-1) matrix given by Hª I – ŝ(êT-T). A basic property of positive
definite matrices is that as long as H has a non-zero determinant, then HZ-1HT will be positive
definite if and only if Z is positive definite. Using Sylvester’s determinant theorem, we get the
determinant of H as
M 1
H  s M   sc c
i 1
(A.11)
Since >0 (as per proposition 1) and the observed budget shares are non-negative, it follows that the
determinant of H will be positive. This implies that if Z is positive definite, then the second term in
the RHS of equation (A.9) will be positive definite. Since the first term in the RHS of (A.9) is
positive semi definite, it follows that if Z is positive definite, then the matrix J in equation (A.9) will
be positive definite. This proves the sufficiency of Z being positive definite. [QED]
Proposition 4: As a result of the necessary and sufficient conditions stated in proposition 3, the Basic Translog
utility imposes the following restrictions on the elasticities: Out of a total of M×(M-1) cross price elasticities across the
M goods, at least two cross price elasticities have to be positive.
Proof of Proposition 4: We will prove the proposition for the case when all M goods are
purchased. The proof for all other combinations will follow exactly the same line of reasoning. We
prove proposition 4 in two steps. In the first step, we prove that at least one of own price elasticities
9
of the c=1..M goods has to be less than -1. In the second step, we prove that conditional on at least
one of the own price elasticities being less than -1, at least two of the cross price elasticities have to
be positive.
For the case when all goods are purchased, the M×1 budget share vector sª[s1, s2,.., sM] of the
M goods is given as
s   A  B ln P  D
(A.12)
where D  eT A  eT B ln P is the log marginal utility of expenditure that is always positive (as per
regularity property 2), e is an M ×1 vector of ones, P ª [p1/y, p2/y,.., pM /y] is the M ×1 vector of the
normalized market prices of the M goods, and B is the symmetric M×M matrix of parameters {bi,c}.
Recall from equation (15) that matrix B for the Basic Translog utility is partitioned as
 Z 1
 Z 1 
B   T 1

 T Z 1 
  Z
(A.13)
where Z is a symmetric and positive definite (M-1)×(M-1) matrix and  ª [1, 2,..,M-1] is an (M1)×1 vector with elements i >0 " i=1..M-1. In equation (A.13), note that since Z is positive definite,
all the diagonal elements in the matrix B will be positive. We represent the M×M own and cross
price elasticity matrix of the M goods by the matrix E, in which the element in the ith row and jth
column represents the price elasticity between goods i and j as Ei,j ª ∑lnqi/∑lnpj. From equations
(A.12) and (A.13), we get the M×M matrix E as
E
s 1   Iˆ  sˆeˆ   T Z 1

D eˆT Iˆ  sˆeˆ   T Z 1
Iˆ  sˆeˆ    Z    I
 eˆ Iˆ  sˆeˆ    Z  
1
T
T
T
1
(A.14)
In equation (A.14), ŝ ª[s1, s2,.., sM-1] is an (M-1)×1 vector of the budget shares of the first M-1 goods,
ê is an (M-1)×1 vector of ones, Î is an (M-1)×(M-1) identity matrix, I is an M×M identity matrix, and
Δ(s) is a diagonal M×M matrix that has the M×1 vector of budget shares of all M goods, sª[s1, s2,..,
sM], on its diagonal. Given the price elasticity matrix in equation (A.14), we start with step 1 where
we prove that at least one of M own price elasticities in equation (A.14) will be less than -1.
Step 1: We represent the own price elasticities by an M×1 vector, Eown, in which the ith element
represents the own price elasticity of good i, ∑lnqi /∑lnpi. The own price elasticities are captured by
the diagonal of the M×M price elasticity matrix E in equation (A.14). These can be derived as
1
 1
  1  Z eˆ     
E own     s 1 B Diag  e     T 1

 D
  D   Z eˆ    
(A.15)
10
The first and the second bracketed terms in the RHS of equation (A.15) are M×1vectors. In the first
bracketed term, Bdiag is an M×1 vector that represents the diagonal of the matrix B, e is an M×1
vector of ones, and D is the (log) marginal utility of expenditure. Since all the elements in Bdiag are
positive and since D>0, it follows that all the elements in the first bracketed term will be less than 1. This implies that the own price elasticity of any good i will only be less than -1 as long as its
corresponding ith element in the second bracketed term in the RHS of (A.15) is negative. In what
follows, we will prove that at least one element in the second bracketed term in RHS of (A.15) will
be negative, which will imply that at least one of the own price elasticities will be less than -1.
 Z 1 eˆ    
 is
T
1
  Z eˆ   
In the second bracketed term, note that D>0. Further, note that the term 
an M×1 vector in which the first M-1 elements are captured by the (M-1)×1 vector ‘Z-1(ê-)’, and the
Mth element is captured by the scalar ‘-TZ-1(ê -)’. Thus in what follows, we will prove that at least
 Z 1 eˆ    
 will be negative. Consider the extreme case in
T
1
  Z eˆ   
one element in the M×1 vector 
which all the elements in the (M-1)×1 vector Z-1(ê-) are positive. Next, recall from proposition 3
that  is an (M-1)×1 vector in which all elements are positive. Since all the elements in the vectors Z(ê-) and  are positive, it follows that their dot product, TZ-1(ê-), will be positive. This implies that
1
the term ‘-TZ-1(ê-)’ will be negative. Next, note that the term ‘-TZ-1(ê-)’ is simply the Mth element
 Z 1 eˆ    
 Z 1 eˆ    
.
This
proves
that
at
least
one
element
in
the
vector
 T 1
 has

T
1
  Z eˆ   
  Z eˆ   
in the vector 
to be negative, which in turn proves that at least one of own price elasticities has to be less than -1.
This completes step 1 of the proof. In step 2, we prove that conditional on one of the own
price elasticities being less than -1, at least two of the cross elasticities in equation (A.14) have to be
positive.
Step 2: Since at least one of the own price elasticities have to be less than -1, consider the case where
the own price elasticity of good M is less than -1. Note that the own price elasticity of good M is the
diagonal element in the Mth row and Mth column in the price elasticity matrix E in equation (A.14).
Next observe that the second term in the RHS of equation (A.14) is an identity matrix. Since the
own price elasticity of good M is less than -1, it follows that the diagonal element in the Mth row and
11
Mth column of equation (A.14) (which is - ê T(I – ŝ(ê -)T)Z-1) has to be negative. Thus we get the
following inequality
ê T(I – ŝ(ê -)T)Z-1 >0
(A.16)
where the LHS of (A.16) is a scalar. In the LHS of (A.16), the first term, ê, is an 1×(M-1) vector of
ones, the second term, (I –ŝ(ê-)T)Z-1, is an (M-1)×(M-1) matrix, and the third term, , is an (M-1)×1
vector. Denote the product of the first two terms in the LHS of (A.16) by
H1 ª ê T(I – ŝ(ê-)T)Z-1
(A.17)
where H1 is an 1×(M-1) vector. Note that (A.16) and (A.17) imply H1 >0. Since all the elements in
the (M-1)×1 vector  are positive (as per proposition 3), the inequality H1 >0 implies that at least
one of the elements in the 1×(M-1) vector H1 has to be positive. Next, denote the product of the
last two terms in the LHS of (A.16) by
H2 ª (I –ŝ(ê -)T)Z-1
(A.18)
where H2 is an (M-1)×1 vector. Note that (A.16) and (A.18) imply êTH2>0. Since all the elements in
the (M-1)×1 vector ê are positive (since ê is a vector of ones), the inequality êTH2>0 implies that at
least one of the elements in the (M-1)×1 vector H2 has to be positive. Thus so far we have shown
that at least one of the elements in H1 and one of the elements in H2 has to be positive. Next, note
that H1 and H2 are simply the off diagonal vectors in the Mth row and Mth column in the price
elasticity matrix in equation (A.14). Thus it follows that at least two of the off-diagonal elements in
the price elasticity matrix have to be positive. Since the off diagonal elements in the price elasticity
matrix are simply the cross elasticities, it follows that at least two of the cross elasticities have to be
positive. [QED]
Proposition 5: As a result of the sufficient conditions for global regularity proposed by Jorgensen and Fraumani
(1981), the Basic Translog utility imposes the following restrictions on the elasticities:
1. The expenditure elasticity of any good i will always be one.
2. The cross price elasticities between goods i and j are restricted by the condition: sjEi,j = siEj,i.
3. Out of the total of M×(M-1) cross price elasticities, at least 2×(M-1) cross price elasticities will be positive.
Proof of Proposition 5: We will only prove implication 3 since the first two implications are
standard. Imposing the sufficient conditions proposed by Jorgensen and Fraumani (1981) into the
12
specification of the budget shares of the Basic Translog utility given in equation (A.12), we get the
budget share vector s ª[s1, s2,.., sM] for the Homothetic Translog utility as
s =A – B lnP
(A.19)
where Pª [p1, p2,.., pM] is the M×1 vector of the market prices of the M goods, and Bª{bi,j} is an
M×M matrix of parameters. We next represent the M×M own and cross price elasticity matrix of
the M goods by the matrix E, in which the element in the ith row and jth column represents the price
elasticity between goods i and j as Ei,j ª ∑lnqi /∑lnpj. From equation (A.19), we get the price elasticity
matrix E as
E=-[D(s)]-1B – I
(A.20)
where I is an M×M identity matrix, and Δ(s) is a diagonal M×M matrix that has the vector of budget
shares s on its diagonal. In equation (A.20), observe that the cross elasticity between goods i and j ("
i∫j) will be positive only if the off diagonal element bi,j (" i∫j) is negative. Thus in what follows, we
will prove that at least 2×(M-1) off-diagonal elements in the matrix Bª{bi,j} have to be negative,
which will in turn prove that at least 2×(M-1) cross price elasticities will be positive.
Recall from section 3.4 that in the Homothetic Translog utility, the M×M matrix B is
M
symmetric with  bc ,i  0 " c= 1..M (which means that all rows and columns of the matrix B have to
i 1
sum to zero) and all of its principal (M-1)×(M-1) sub-matrices are positive definite. Note that a
property of positive definite matrices is that all of its diagonal elements have to be positive; however,
the non-diagonal elements of a positive definite matrix can be negative or positive. Thus consider
any one of the principal (M-1)×(M-1) sub-matrices of B which is obtained by removing the ith row
and ith columns of B. We refer to this (M-1)×(M-1) sub-matrix as B(-i). Since B(-i) is positive definite, it
follows that all of its diagonal elements have to be positive. Next, consider the extreme case in
which all the (M-1)×(M-2) off-diagonal elements of B(-i) are positive. Since all the diagonal as well as
off-diagonal elements in B(-i) are positive, it follows that all rows and columns of the matrix B can
only sum to zero as long as all the 2×(M-1) off-diagonal elements in the ith row and ith columns of
matrix B are negative. This implies that that at least 2×(M-1) off-diagonal elements in the matrix B
will be negative, which will implies that at least 2×(M-1) cross price elasticities will be positive.
[QED]
Proposition 6: The LES utility imposes the following restrictions on the elasticities:
(i) The expenditure elasticity of any good i will be necessarily positive.
13
(ii) All the M×(M-1) cross price elasticities will be positive.


M
Proof: of Proposition 6: Recall that the LES direct utility given as U qi i 1   ai lnqi  bi  , which
M
i 1
satisfies the regularity properties in the entire feasible space as long as bi<0 and ai>0 " i=1..M . For
the LES utility, we get the optimal purchase quantity of good i as
xi  bi 
i y 
M p b
1   k k
pi  k 1 y



(A.21)
M
where  i  ai  ak . From equation (A.22), we get the expenditure elasticity of good i as
k 1
 ln x i  ln y   i y p i x i
(A.22)
Since qi>0, it follows from equation (A.22) that the expenditure elasticity of good i will always be
positive. This proves implication 1. Moving on the implication 2, we get the cross elasticity between
goods i and j from equation (A.21) as
 ln xi  ln p j   i
p jb j
xi q i
(A.23)
Since qi>0 and bi<0, equation (A.23) implies that all cross elasticities will be positive. [QED]
14
SECTION 2: PARAMETER ESTIMATES FOR SIMULATED AND SCANNER DATA
Table 1: True Parameters of the Direct Quadratic Utility used for simulating Datasets 1-5
Parameters
Dataset 1
Dataset 2
Dataset 3
Dataset 4
Dataset 5
(M=4 goods)
(M=4
(M=4 goods) (M=6 goods) (M=6 goods)
goods)
1.991
2.741
1.700
1.099
0.599
1
M 1
1.991
2.251
1.700
1.099
0.599
 c 
c2
M
b1,1
 
M 1
c ,c c  2
bM,M
 
 
M 1
1, m m  2
M 1
c , m c , m  2 ,c  m
b1,M
 
 
 
M 1
c,M
c2
M 1
c ,c c 1
M 1
c , m c , m 1, c  m
2.491
9.000
9.000
1.000
-2.700
2.491
4.000
6.250
1.000
1.500
2.491
4.000
4.000
1.000
1.200
0.849
4.000
4.000
0.250
1.000
0.849
4.000
4.000
0.250
1.000
-2.700
-1.875
1.200
1.000
1.000
0.000
0.000
0.160
0.120
-0.500
0.090
-0.600
-0.600
0.090
-0.050
-0.050
0.0225
-0.050
-0.050
0.0225
0.024
-0.018
-0.027
-0.0034
-0.0034
Distributions of the normalized prices for simulated datasets 1-5:
 Dataset 1-3: The normalized market price of each good i=1..3 is simulated from an IID log
normal distribution as ln pi ~ N 0,0.09 , and that of good 4 is simulated from
ln p4 ~ N 0,0.04  .
 Datasets 4-5: The normalized market price of each good i=1..6 is simulated from an IID log
normal distribution as ln p i ~ N 0,0.01
Table 2: Parameter Estimates of Model I for Simulated Datasets 1-5
Parameter
Dataset 1
Dataset 2
Dataset 3
Dataset 4
Dataset 5
Estimate (S.E.) Estimate (S.E.) Estimate (S.E.) Estimate (S.E.) Estimate (S.E.)
α1
1.363 (0.036)
0.865 (0.024)
3.317 (0.318)
0.078 (0.014)
0.293 (0.020)
α2
1.248 (0.037)
0.869 (0.025)
3.329 (0.318)
0.146 (0.016)
0.304 (0.020)
α3
1.168 (0.035)
0.860 (0.025)
3.452 (0.333)
0.128 (0.016)
0.287 (0.019)
α4
3.317 (0.323)
0.111 (0.015)
α5
3.139 (0.304)
0.135 (0.015)
0.205 (0.025)
-0.706 (0.050)
0.734 (0.071)
-0.449 (0.091)
-0.231 (0.064)
r1
-0.170 (0.040) -0.743 (0.053)
0.745 (0.071)
-0.324 (0.078)
-0.365 (0.072)
r2
-0.147 (0.038) -0.686 (0.050)
0.785 (0.071)
-0.219 (0.078)
-0.235 (0.062)
r3
0.767
(0.072)
-0.342
(0.080)
r4
0.695 (0.072)
-0.334 (0.074)
r5
z
c 1,1
3.740 (0.054)
2.376 (0.017)
3.124 (0.031)
4.638 (0.252)
2.921 (0.070)
15
cz2,1
cz2,2
cz3,1
cz3,2
cz3,3
cz4,1
cz4,2
cz4,3
cz4,4
cz5,1
cz5,2
cz5,3
cz5,4
cz5,5
cГ1,1
cГ2,1
cГ2,2
cГ3,1
cГ3,2
cГ3,3
cГ4,1
cГ4,2
cГ4,3
cГ4,4
cГ5,1
cГ5,2
cГ5,3
cГ5,4
cГ5,5
Parameter
α1
α2
α3
α4
α5
cz1,1
cz2,1
cz2,2
cz3,1
cz3,2
cz3,3
cz4,1
-1.080 (0.080)
3.607 (0.048)
-1.121 (0.076)
-1.740 (0.071)
2.875 (0.038)
1.689 (0.035)
2.543 (0.024)
1.556 (0.033)
-1.186 (0.035)
2.172 (0.021)
2.443 (0.039)
1.982 (0.022)
2.404 (0.031)
0.753 (0.034)
1.693 (0.019)
1.981 (0.047)
0.853 (0.067)
1.775 (0.052)
0.920 (0.070)
0.555 (0.076)
1.788 (0.054)
3.318 (0.059)
-1.267 (0.085)
2.899 (0.057)
-1.187 (0.090)
1.110 (0.082)
2.843 (0.058)
3.512 (0.065)
-0.684 (0.114)
3.453 (0.064)
-0.557 (0.117)
-0.825 (0.118)
3.427 (0.067)
2.971 (0.253)
3.574 (0.137)
3.242 (0.266)
1.513 (0.140)
3.208 (0.112)
3.041 (0.259)
1.375 (0.143)
0.727 (0.128)
3.218 (0.115)
2.859 (0.243)
1.449 (0.132)
0.585 (0.113)
0.720 (0.114)
2.973 (0.094)
3.063 (0.165)
-0.112 ( 0.192)
3.130 (0.154)
-0.756 (0.180)
-0.771 (0.172)
2.698 (0.149)
-0.415 ( 0.191)
-0.274 (0.185)
-0.366 (0.189)
2.857 (0.153)
-0.309 ( 0.198)
-0.649 (0.190)
-0.3006 0.2047
-0.616 (0.195)
3.140 (0.159)
0.534 (0.135)
2.915 (0.077)
0.790 (0.167)
1.236 (0.167)
2.675 (0.089)
0.903 (0.157)
0.841 (0.161)
0.415 (0.199)
2.621 (0.080)
0.945 (0.129)
1.265 (0.137)
0.195 (0.188)
0.265 (0.173)
2.384 (0.069)
3.689 (0.184)
1.315 (0.229)
3.618 (0.171)
0.846 (0.240)
-0.307 0.257)
3.247 (0.172)
0.873 (0.264)
0.278 (0.271)
0.525 (0.265)
3.481 (0.172)
0.787 (0.257)
-0.502 (0.284)
-0.502 (0.284)
0.503 (0.274)
3.814 0.170)
Table 3: Parameter Estimates of Model II for Simulated Datasets 1-5
Dataset 1
Dataset 2
Dataset 3
Dataset 4
Dataset 5
Estimate (S.E.) Estimate (S.E.) Estimate (S.E.) Estimate (S.E.) Estimate (S.E.)
0.381 (0.018)
1.309 (0.021)
1.503 (0.035)
1.286 (0.030)
0.158 (0.019)
0.420 (0.018)
1.357 (0.026)
1.517 (0.035)
1.306 (0.031)
0.244 (0.017)
0.377 (0.017)
1.272 (0.024)
1.501(0.035)
1.319 (0.032)
0.210 (0.017)
1.259 (0.030)
0.201 (0.017)
1.235 (0.028)
0.229 (0.015)
2.497 (0.036)
2.428 (0.028)
3.033 (0.044)
3.268 (0.070)
3.643 (0.058)
1.592 (0.049)
1.581 (0.038)
1.090 (0.070)
0.877 (0.154)
-1.247 (0.082)
2.496 (0.025)
1.849 (0.021)
2.850 (0.055)
3.139 (0.087)
3.419 (0.052)
1.464 (0.047)
1.547 (0.039)
1.262 (0.072)
1.088 (0.196)
-1.237 (0.079)
-1.416 (0.069)
-1.055 (0.035)
0.897 (0.039)
0.885 (0.084)
1.524 (0.191)
3.207 (0.030)
2.284 (0.020)
1.758 (0.020)
2.708 (0.058)
2.728 (0.100)
1.128 (0.073)
1.317 (0.188)
16
cz4,2
cz4,3
cz4,4
cz5,1
cz5,2
cz5,3
cz5,4
cz5,5
cГ1,1
cГ2,1
cГ2,2
cГ3,1
cГ3,2
cГ3,3
cГ4,1
cГ4,2
cГ4,3
cГ4,4
cГ5,1
cГ5,2
cГ5,3
cГ5,4
cГ5,5
1.830 (0.042)
0.468 (0.061)
1.719 (0.046)
0.567 (0.064)
0.367 (0.070)
1.804 (0.051)
3.339 (0.060)
-1.153 (0.079)
2.787 (0.051)
-1.089 (0.084)
0.855 (0.074)
2.817 (0.056)
3.371 (0.066)
-1.256 (0.105)
3.169 (0.064)
-1.159 (0.107)
-1.877 (0.099)
2.618 (0.057)
0.747 (0.085)
0.397 (0.092)
2.711 (0.061)
1.112 (0.065)
0.863 (0.077)
0.331 (0.084)
0.493 (0.085)
2.552 (0.051)
3.429 (0.135)
0.526 (0.184)
3.3865 (0.140)
0.035 (0.179)
-0.078 (0.183)
3.214 (0.134)
0.368 (0.188)
0.352 (0.192)
0.622 (0.189)
3.251 (0.145)
0.430 (0.192)
0.014 (0.200)
0.709 (0.201)
0.209 (0.214)
3.605 (0.144)
1.059 (0.194)
0.445 (0.247)
2.699 (0.097)
1.345 (0.141)
1.547 (0.150)
0.114 (0.227)
0.311 (0.208)
2.412 (0.080)
3.018 (0.147)
0.888 (0.247)
3.437 (0.170)
0.407 (0.263)
-0.645 (0.260)
2.954 (0.152)
0.199 (0.297)
0.016 (0.284)
0.203 (0.264)
3.170 (0.154)
0.095 (0.263)
-0.930 (0.278)
0.302 (0.287)
0.110 (0.275)
3.469 (0.150)
Table 4: Parameter Estimates of Model III for Simulated Datasets 1-5
Parameter
Dataset 1
Dataset 2
Dataset 3
Dataset 4
Dataset 5
Estimate (S.E.) Estimate (S.E.) Estimate (S.E.) Estimate (S.E.) Estimate (S.E.)
α1
-3.267 (0.070)
-1.155 (0.0423) -0.991 (0.021)
-1.645 (0.029)
-2.254 (0.051)
α2
-3.277 (0.070)
-1.507 (0.017)
-0.976 (0.021)
-1.662 (0.029)
-2.307 (0.051)
α3
-3.244 (0.071)
-1.479 (0.017)
-0.957 (0.022)
-1.679 (0.029)
-2.318 (0.051)
α4
-1.671 (0.029)
-2.306 (0.051)
α5
-1.633 (0.029)
-2.242 (0.050)
β1
-2.371 (0.019)
-1.238 (0.022)
-1.206 (0.022)
-1.772 (0.032)
-1.864 (0.045)
β2
-2.389 (0.019)
-1.829 (0.017)
-1.186 (0.022)
-1.790 (0.033)
-1.919 (0.046)
β3
-2.346 (0.018)
-1.723 (0.017)
-1.160 (0.022)
-1.819 (0.033)
-1.931 (0.046)
β4
1.239 (0.018)
-0.755 (0.025)
-2.370 (0.074)
-1.801 (0.033)
-1.918 (0.046)
β5
-1.758 (0.032)
-1.850 (0.045)
β6
-0.861 (0.025)
-0.746 (0.044)
cГ1,1
4.119 (0.057)
4.820 (0.072)
5.046 (0.803)
5.721 (0.132)
6.487 (0.257)
Г
c 2,1
-1.497 (0.061)
2.239 (0.088)
2.256 (0.108)
2.340 (0.094)
4.101 (0.243)
Г
c 2,2
3.800 (0.053)
4.724 (0.065)
4.567 (0.075)
5.107 (0.125)
4.775 (0.171)
cГ3,1
-1.514 (0.064)
2.233 (0.092)
2.168 (0.108)
2.314 (0.093)
3.994 (0.241)
Г
c 3,2
-2.249 (0.059)
-2.614 (0.071)
1.379 (0.087)
1.429 (0.077)
1.770 (0.114)
cГ3,3
3.259 (0.048)
4.119 (0.060)
4.371 (0.075)
4.804 (0.119)
4.331 (0.155)
cГ4,1
2.380 (0.095)
4.132 (0.252)
17
cГ4,2
cГ4,3
cГ4,4
cГ5,1
cГ5,2
cГ5,3
cГ5,4
cГ5,5
1.489 (0.079)
1.176 (0.073)
4.770 (0.120)
2.434 (0.098)
1.494 (0.081)
1.092 (0.077)
0.847 (0.074)
4.939 ( 0.124)
1.828 (0.117)
1.267 (0.093)
4.227 (0.151)
4.424 (0.261)
1.913 (0.122)
1.310 (0.100)
0.903 (0.087)
4.419 (0.155)
Table 5: Parameter Estimates of Models I and II for Scanner Datasets A and B
Notes:
1. For scanner dataset A, good 1 refers to detergents, good 2 refers to softeners, good 3 refers to
pasta and good 4 refers to pasta sauce. For scanner dataset B, good 1 refers to Blueberry, good 2
refers to mixed berry, good 3 refers to strawberry, good 4 refers to plain and good 5 refers to
pina-colada.
2. For scanner dataset B, the parameters for inventories and promotions are taken to be the same
across all five flavors of yogurt. There are two reasons for it. First, is to reduce the number of
parameters. Second, all five flavors belong to the same yogurt category.
3. For each good c, we have two inventory parameters: βI,c and fI,c. The parameter βI,c is the
coefficient of the inventory of good c at hand in the utility (equation 21) and fI,c is the inventory
flexibility parameter
Scanner Dataset A
Parameter
ά1
ά2
ά3
ά4
ά5
r1
r2
r3
r4
r5
cz1,1
cz2,1
cz2,2
cz3,1
cz3,2
cz3,3
cz4,1
cz4,2
cz4,3
Scanner Dataset B
Model I
Estimate (S.E.)
-6.447 (0.535)
-9.200 (0.267)
-6.688 (0.501)
-8.409 (0.126)
Model II
Estimate (S.E.)
-3.753 (0.098)
-5.375 (0.443)
-4.212 (0.129)
-4.123 (0.176)
-0.984 0.256)
-3.088 (1.502)
-1.056 0.805)
-3.893 (1.642)
0
0
0
0
3.007 (0.164)
-1.380 (0.331)
5.180 (0.298)
-0.356 (0.482)
1.752 (0.511)
7.379 (0.511)
-0.096 (0.327)
0.595 (0.444)
-1.626 (0.310)
3.078 (0.134)
-0.491 (0.435)
6.747 (0.345)
1.096 (0.633)
3.961 (0.609)
7.388 (0.379)
0.171 (0.370)
1.269 (0.442)
-1.052 (0.368)
Model I
Estimate (S.E.)
-4.615 (0.255)
-5.681 (0.845)
-4.655 (0.341)
-4.507 (0.354)
-6.026 (0.778)
-0.260 (0.065)
-0.059 (0.133)
-0.270 (0.082)
-0.242 (0.082)
-0.499 (0.181)
3.736 (0.176)
3.281 (0.512)
2.910 (0.625)
2.038 (0.287)
3.917 (0.850)
3.081 (0.321)
0.808 (0.415)
1.179 (1.238)
0.347 (0.519)
Model II
Estimate (S.E.)
-3.908 (0.082)
-4.701 (0.469)
-4.260 (0.159)
-3.780 (0.141)
-4.058 (0.215)
0
0
0
0
0
3.907 (0.151)
3.484 (0.408)
2.609 (0.565)
1.427 (0.247)
1.102 (0.802)
4.097 (0.322)
0.325 (0.340)
2.027 (1.110)
-0.409 (0.589)
18
cz4,4
cz5,1
cz5,2
cz5,3
cz5,4
cz5,5
cГ1,1
cГ2,1
cГ2,2
cГ3,1
cГ3,2
cГ3,3
cГ4,1
cГ4,2
cГ4,3
cГ4,4
cГ5,1
cГ5,2
cГ5,3
cГ5,4
cГ5,5
cυ1,1
cυ2,1
cυ2,2
cυ3,1
cυ3,2
cυ3,3
cυ4,1
cυ4,2
cυ4,3
cυ4,4
cυ5,1
cυ5,2
cυ5,3
cυ5,4
cυ5,5
βPro,1
βPro,2
βPro,3
βPro,4
βPro,5
βI,1
βI,2
βI,3
5.039 (0.253)
5.463 (0.329)
0.982 (0.116)
-0.250 (0.042)
0.603 (0.066)
0.063 (0.036)
-0.226 (0.041)
0.524 (0.040)
0.106 (0.039)
-0.153 (0.059)
-0.354 (0.038)
0.650 (0.054)
1.049 (0.091)
-0.150 (0.043)
0.488 (0.071)
0.026 (0.039)
-0.279 (0.039)
0.448 (0.044)
0.103 (0.045)
-0.206 (0.062)
-0.371 (0.056)
0.592 (0.071)
0.460 (0.041)
0.465 (0.070)
0.764 (0.079)
0.112 (0.056)
-0.253 (0.061)
0.993 (0.096)
0.205 (0.061)
-0.004 (0.052)
0.351 (0.071)
0.588 (0.059)
0.475 (0.036)
0.256 (0.107)
1.013 (0.146)
0.219 (0.086)
0.557 (0.1071)
0.987 (0.108)
0.066 (0.066)
0.202 (0.077)
0.294 (0.062)
0.610 (0.074)
0.575 (0.109)
1.236 (0.291)
0.493 (0.116)
0.481 (0.114)
0.574 (0.100)
1.703 (0.424)
0.623 (0.143)
0.478 (0.128)
-0.210 (0.033)
-0.407 (0.059)
-0.211 (0.042)
-0.201 (0.028)
-0.528 (0.089)
-0.2538 (0.050)
3.905 (0.435)
1.844 (0.440)
0.859 (0.858)
0.294 (0.588)
0.691 (0.591)
4.666 (0.499)
3.499 (0.319)
-0.417 ( 0.167)
1.735 (0.439)
-1.353 (0.323)
-0.900 (0.257)
2.172 (0.267)
-0.455 ( 0.155)
-0.299 (0.101)
-0.925 (0.270)
1.892 (0.214)
-0.442 ( 0.150)
-0.298 (0.126)
-0.895 0.246)
-0.751 (0.149)
0.825 (0.140)
0.695 (0.079)
0.970 (0.139)
1.276 (0.327)
0.513 (0.091)
0.355 (0.035)
0.519 (0.084)
0.125 (0.059)
-0.231 (0.049)
0.105 (0.038)
0.447 (0.073)
0.105 (0.102)
0.339 (0.098)
-0.031 (0.057)
-0.037 (0.074)
0.865 (0.167)
3.589 (0.683)
0.921 (0.254)
0.902 (0.681)
0.200 (0.369)
0.370 (0.583)
3.514 (0.278)
3.462 (0.277)
-0.957 (0.304)
2.020 (0.409)
-0.743 (0.200)
-0.750 0.196)
1.925 (0.222)
-0.325 (0.076)
-0.329 (0.089)
-0.401 (0.098)
1.831 (0.193)
-1.356 (0.278)
-1.360 (0.254)
-1.658 (0.278)
-1.140 (0.225)
1.439 0.273)
0.790 (0.064)
0.627 (0.133)
1.019 (0.205)
0.679 (0.074)
0.486 (0.039)
0.622 (0.084)
0.281 (0.073)
-0.196 (0.044)
0.152 (0.044)
0.443 (0.064)
0.587 (0.077)
0.080 (0.033)
-0.303 (0.036)
-0.175 (0.034)
0.640 (0.075)
0.417 (0.209)
0.382 (0.175)
-0.053 (0.009)
-0.048 (0.007)
19
βI,4
βI,5
fI,1
fI,2
fI,3
fI,4
fI,5
-0.101 (0.036)
-0.1104 (0.037)
0.726 (0.035)
0.959 (0.010)
0.411 (0.078)
0.282 (0.116)
0.7239 (0.041)
0.9457 (0.012)
0.4109 (0.078)
0.2974 (0.116)
0.405 (0.028)
0.362 (0.024)
Table 6: Parameter Estimates of Model III for Scanner Datasets A and B
Parameter
Scanner Dataset
Scanner
Parameter
Scanner Dataset
Scanner
A
Dataset B
A
Dataset B
Estimate (S.E.)
Estimate
Estimate (S.E.)
Estimate
(S.E.)
(S.E.)
ά1
-4.708 (0.167)
cυ1,1
0.414 (0.023)
0.345 (0.027)
-2.947 (0.067)
ά2
-5.782 (0.342)
cυ2,1
0.083 (0.038)
0.302 (0.062)
-4.008 (0.094)
υ
ά3
-5.064
(0.342)
c
0.410
(0.018)
0.861 (0.095)
-4.499 (0.055)
2,2
υ
ά4
-3.821 (0.067) -5.007 (0.179)
c 3,1
-0.004 (0.028)
0.228
υ
ά5
-5.451 (0.212)
c 3,2
-0.016 (0.025)
-0.024 (0.029)
β1
3.740 (0.092)
cυ3,3
0.323 (0.017)
0.444 (0.045)
5.428 (0.046)
υ
β2
3.612 (0.149)
c 4,1
0.117 (0.035)
0.139 (0.025)
4.682 (0.056)
υ
β3
3.521
(0.088)
c
-0.013
(0.016)
-0.229
(0.035)
4.682 (0.056)
4,2
υ
β4
4.303 (0.044)
3.630 (0.100)
c 4,3
0.182 (0.015)
0.163 (0.032)
υ
β5
4.301 (0.084)
3.452 (0.100)
c 4,4
0.363 (0.020)
0.317 (0.040)
β6
4.926 (0.153)
cυ5,1
-0.003 (0.027)
cГ1,1
2.891 (0.231)
cυ5,2
0.081 (0.040)
2.015 (0.068)
Г
υ
c 2,1
-0.242
(
0.075)
c
-0.138
(0.030)
-0.367 (0.045)
5,3
Г
υ
c 2,2
1.475 (0.059)
2.235 (0.249)
c 5,4
-0.008 (0.036)
Г
υ
c 3,1
-1.048 (0.192)
c 5,5
0.568 (0.056)
0.045 (0.042)
cГ3,2
-0.670 (0.184)
βPro,1
0.240 (0.040)
-0.120 (0.043)
cГ3,3
2.857 (0.259)
βPro,2
0.409 (0.093)
1.700 (0.036)
0.378 (0.332)
cГ4,1
0.055 (0.056)
-0.449 (0.097)
βPro,3
0.072 (0.032)
Г
c 4,2
-0.145 (0.056)
-0.281 (0.064)
βPro,4
0.087 (0.037)
Г
c 4,3
-0.580 (0.046)
-1.033 (0.173)
βPro,5
cГ4,4
1.913 (0.062)
2.308 (0.187)
βI,1
-0.108 (0.013)
Г
c 5,1
-0.493 (0.114)
βI,2
-0.167 (0.018)
-0.038 (0.005)
cГ5,2
-0.318 (0.105)
βI,3
-0.069 (0.013)
Г
c 5,3
-1.131 (0.194)
βI,4
-0.031 (0.013)
Г
c 5,4
-1.097 (0.143)
βI,5
cГ5,5
1.573 (0.110)
fI,1
0.786 (0.021)
fI,2
1.007 (0.007)
0.379 (0.025)
fI,3
0.449 (0.079)
fI,4
0.322 (0.135)
fI,5
20