RANDALL G. CHAPA^AN and RICHARD STAELIN*
The authors report on a procedure for exploiting the information content of rank
ordered choice sets to estimate efficiently the parameters of the multinomial logit
model formulation of the stochastic utility model of choice behavior. The'availability
of rank ordered choice set data leads to an "explosion" or decomposition procedure
for exploiting such extra information. This "explosion" process involves the decomposition of Q ranked choice set into a series of unranked and statistically independent choice sets. In relation to explosion strategies, severat heuristics and an analytical procedure for determining the "optimal" explosion depth are discussed in
detail. The results of a Mont6 Carlo study of the small sample properties of the
conditional logit estimation procedure (the maximum likelihood estimation procedure
used to develop parameter estimates of the multinomial logit model formulation of
the stochastic utility madel) are reported and interpreted. A college choice empirical
application illustrates the procedures developed.
Exploiting Rank Ordered Choice Set Data
Within the Stochastic Utility Model
According to a popular choice behavior modeling paradigm, the consumer decision maker evaluates each
available alternative in terms of its component parts,
assessing the relative importance of the components and
ultimately choosing the alternative with the greatest
weighted aggregate score. Marketing scientists have
posited a variety of preference and choice models of how
consumer decision makers combine the component scores
to produce a weighted aggregate score. We are concerned with one general class of consumer choice
models., the stochastic utility model. Within the choice
modeling literature, this model is commonly cited as the
Luce (1959) choice model; in actual empirical application, it is typically parameterized as the multinomial
logit model (cf. Gensch and Recker 1979). Econometricians, most notably McFadden (cf. 1974, 1890), have
made significant strides in refining statistical techniques
associated with estimating the parameters of such choice
models. Some illustrative applications of this choice behavior modeling methodology include the selection of
a college (Chapman 1979; Kohn, Manski, and Mundel
1976; Punj and Staelin 1978), a mode of transportation
(cf. Domencich and McFadden 1975), a grocery store
(Gensch and Recker 1979), a shopping center (Chapman
1980), a home (Li 1977), an occupation (Boskin 1974),
and an electric utility fuel (Joskow and Mishkin 1977).
Although the assumptions associated with the stochastic utility model do not restrict its applicability to
situations in which each consumer decision maker is
observed making just a single choice from among a set
of known alternatives, most marketing applications of
the stochastic utility model are limited to those cases.
In such situations, the analysis is cross-sectional and the
econometric problem is to assess the relative importance
of the quantifiable attributes that describe the objects of
choice, such estimation being based on (the researcher's)
knowledge of each consumer's actual choice and the
composition of each consumer's choice set.
We develop a procedure to enhance the estimation of
the parameters of the stochastic utility model. This enhancement is achieved by exploiting the additional information contained in preference rank orderings of
choice set alternatives, data often available to marketing
researchers. In the next section, several key aspects of
*RandaIl G. Chapman is Visiting Assistant Professor of Marketing,
Graduate School of Business, University of Chicago. Richard Staelin
is T. Austin Finch Professor of Business Administration, Fuqua
School of Business, Duke University.
The helpful comments and advice of Charles F. Manski, J. Edward
Russo, and an anonymous reviewer are most gratefully acknowledged. Responsibility for errors of omission and commission rests
with the authors.
288
Journal of Marketing Research
Vol. XIX (August 1982). 288-301
289
STOCHASTIC UTILITY MODEL
the stochastic utility model of choice behavior are briefly
reviewed. The principle of decomposing rank ordered
choice sets into a series of statistically independent unranked choice sets is explained. Strategies and techniques for coping with "noisy" and possibly unreliable
rank order information are then considered. The results
of a Mont6 Carlo study designed to investigate the small
sample properties of the parameter estimates of the conditional logit estimation procedure (the maximum likelihood estimation procedure used to develop parameter
estimates of the multinomial logit model formulation of
the stochastic utility model) are reported and interpreted.
Particular attention is focused on the incremental contribution of using the additional information contained
in the preference rank ordering rather than just employing knowledge of the chosen choice set alternative. We
conclude with an empirical application and a discussion
of the implications of these results for marketing resetu-chers who use rank ordered choice set data to estimate the parameters of the stochastic utility model to
draw inferences about choice behavior.
THE STOCHASTIC UTIUTY MODEL
The general nature of the choice behavior being modeled with the stochastic utility model, and the nature of
the data (assumed) available to the empirical marketing
researcher, may be described as follows. Each consumer
decision maker / (i = 1,2,...,/) has a choice set C,
consisting of 7^ alternatives (1 < 7, < oo). The choice set
alternatives are assumed to be characterized by A' quantifiable attributes. Each decision maker is observed to
choose an alternative from his or her choice set. The
decision makers are assumed to be utility maximizers
(i.e., rational) whose choices represent their most preferred alternatives at the time of choice. Also, because
the data used typically are cross-sectional, the sample
of decision makers is assumed to have homogeneous
tastes and preferences in terms of the relative importance
of the attributes characterizing the alternatives.
Let x^ denote a vector of relevant attributes of alternative j , d, denote a vector of individual decision maker
demographic attributes, and y,^ denote a vector of interactive variables relating decision maker / to alternative
} . It is assumed that a utility function V exists which
measures the unobserved desirability or attractiveness of
an alternative with attribute vector x^ to a decision maker
with demographic vector d, and associated decision
maker/altemative vector y,^.
(")
t/y = (/(x^A-y,)
Measurement error is typical in the modeling process
because the x. d, and y vectors generally do not capture
all of the factors influencing the choice process, the correct functional form for the model may not be specified,
and individuals often exhibit aspects of idiosyncratic
behavior. Thus, U,^ is assumed to be composed of two
parts—a deterministic component, V(x^,d.,y,y), representing
the systematic component of the model, and an error term e^, which captures the measurement errors in
the modeling process. If one assumes these two components are independent and additive, the model in
equation 1 may be written intheform
(2)
Vi^ = V,^ + e,^
where V,y = V(x^,d.,y,p. The presence of the stochastic
error term in equation 2 leads to this model being described as a stochastic utility model.
Suppose that individual i is observed to choose alternative j * from C,. If rational choice behavior is assumed,
revealed pre erence implies that f/,y. > f/,^ (for j =
1,2,...,/;). Because the utility function is partly stochastic, the probability of this event occurring may be
written as
(3) P,, = Prob
= Prob
^ U,, j = 1.2,....y,)
6^, < V,. -V,,j=
1.2
where P^j, is the probability that decision maker /
chooses alternative)*. Further development and simpliflcation of equation 3 require that a joint distribution
function be specified for the error terms. In principle,
any joint distribution function could be used and an
expression for the choice probabilities could be developed. Unfortunately, the choice of most distributions,
including the usual normal distribution assumption for
error terms in statistical models, necessitates the calculation of a formidable series of numerical integrations
to determine explicitly the choice probabilities. However, if the stochastic error terms are assumed to be identically and independently distributed (IID) according to
the double exponential distribution, such that
(4)
Prob (€,^ < 0 = exp [-exp (-/)!,
one can show that the choice probabilities have the following form (cf. McFadden 1974).
(5)
for;-*= 1,2
J,
The value of the double exponential distribution assumption is that a tractable closed-form expression results for the choice probabilities. This particular parametric form of the stochastic utility model is often called
the multinomial logit model because it is the multiple
choice generalization of the binary logit model.
To operationalize the choice probability expression in
equation 5, the functional form of the deterministic component of the stochastic utility model must be specified.
A linear-in-parameters specification assumption would
lead to
where Z^^ is the measured value of attribute n for alter-
290
JOURNAL OF MARKETING RESEARCH, AUGUST 1982
native j to decision maker i and Z--^ — Z-j^(Xj,Aj,y^j), and
6^ is the relative importance of attribute « to the sample
of decision makers. The 6 values in equation 6 are the
parameters of the stochastic utility model that must be
estimated from the available sample choice set data.
One particular feature of the stochastic utility model
with double exponentially distributed error terms should
be noted. Because the variance of the double exponential
distribution is a known fixed constant, not dependent on
the estimated 6 values, the magnitude of the estimated
0 values directly influences the "percent of variance explained" by the systematic component of the utility
function. As shown in the Appendix, the quantity
0)^ = ^
6^ is a measure of the "size" of the systematic
"=1
component. As co -^ '^, the systematic component dominates the error component and the choice probabilities
approach unity for the alternatives in the choice sets with
greatest utility, with all other choice probabilities approaching zero. Conversely, as (u^-> 0, the error component in the stochastic utility model dominates the systematic component and each choice set alternative has
approximately equal probability of being chosen. Other
details of these scale effects are given in the Appendix
and in subsequent discussion.
Values of the parameters of the multinomial logit
model may be estimated by maximizing the likelihood
function associated with the probabilistic choice model
in equation 5. Standard software packages (cf. Manski
1974) are available to calculate the maximum likelihood
estimates. Because maximum likelihood estimates are,
in general, consistent and asymptotically normally distributed, approximate large sample confidence bounds
on parameter estimates may be constructed and hypotheses may be tested in standard ways.
EXTENDING THE MODEL EOR
RANK ORDERED DATA
The estimation of the parameters of the stochastic
utility model requires the availability of the following
data from a representative sample of decision makers
from the population of interest: (1) the alternatives in
each decision maker's choice set; (2) the actual alternative chosen (i.e., preferred) by each decision maker;
and (3) the numerical value of each quantifiable attribute
associated with the choice set alternatives (i.e., the Z^^
values). The model operates on the principle of revealed
preference: the alternative actually chosen by a decision
maker is assumed to be preferred to all other alternatives
in the decision maker's choice set. The basic parameter
estimation methodology can be extended if the researcher has available (or could conveniently gather
along with the other required data) a complete rank ordering of all of the alternatives in the decision makers'
choice sets. To exploit the information content of a preference rank ordering of choice set alternatives, one must
relate ranking behavior to choice behavior. The theoretical justification for relating ranking behavior to choice
behavior, within the class of choice models of which the
stochastic utility model is a member, is provided by a
proof reported by Luce and Suppes (1965, p. 354-6).
Although the Luce and Suppes proof is for constant
utility models (i.e., choice models in which each alternative has a fixed utility value and the probability of
choosing one alternative over another is a function of^ the
distance between their utilities), both the constant and
stochastic (random) utility models have an econometric
specification similar to that of equation 5 (cf. Luce and
Suppes 1965, p. 332-9). Consequently, the Luce and
Suppes proof relating ranking behavior to choice behavior can be extended to the latter model.
The Luce and Suppes Ranking Choice Theorem states
that for any rank ordered preference set which has been
derived from a constant utility model,
(7)
Pvia,b,c,...)
=
where Pr(a,ft,c,...) is the probability of observing the
rank order of alternative a being preferred to alternative
b being preferred to alternative c, and so on, and Pr(a|
C) is the probability of alternative a being chosen from
the set of alternatives C = {a,b,c,...}. This Ranking
Choice Theorem enables the probability of a ranking
event, Pr(a,b,c,...), to be decomposed into the product
of two probabilities—the probability of a choice event,
Pr(a|C), and the probability of a subranking event,
Pr(6,c,...). By successively applying this Ranking Choice
Theorem to the subranking events, one can derive a
probability expression for the ranking event which is the
product of the probabilities of 7 - 1 choice events, i.e.,
(8)
?r{a,b,c, ...) = Pr(a|C)-Pr(ft|C - W)
•Pric\C-{a,b})..where C - {a} is the set of alternatives excluding alternative a. Equation 8 is equivalent to saying that the
probability of the joint ranking event of J alternatives is
composed of 7 — I statistically independent choice
events.
If one applies the Ranking Choice Theorem to the stochastic utility model, assuming that the alternative index
j is now interpreted as a serial preference index, it follows that;
(9)
Prob (t/,,
Prob {U,,.
= y*
J,).
The left side of equation 9 is the joint probability that
alternative 1 is preferred to alternative 2 which is preferred to alternative 3, and so on to alternative 7, - 1
which is preferred to alternative J. for decision maker
i. The right side of equation 9 may be interpreted as the
statistical definition of the independence of the events
(t/,, ^ U,j,j = l , 2 . . . . , y , ) . (t/,, > U,jJ = 2 , 3 , . . . , 7 , ) .
The statistical independence condition implied by
equation 9 leads to the notion of an "explosion" process
291
STOCHASTIC UTILITY MODEL
for exploiting the information content in preference rank
ordered choice sets. More formally:
Rank order "explosion rule" definition: Given rank ordered choice set data, the original rank ordered observation that decision maker / prefers the alternatives in the
order 1,2
J,—i.e.. that f/,, > t/,; > . . . > (/.^—can
be "exploded" (decomposed) into 7, - 1 statistically independent choice observations of the form (t/,, ^ t/,y, j
,,,,,
is ignored.)
To illustrate the use of the explosion rule, consider the
automobile choice process in which a decision maker is
observed to rank order his or her choice set in the following manner: Pinto is preferred to Chevette preferred
to Omni preferred to Pacer. Invoking the explosion rule,
one can form three choice sets {Pinto preferred to Chevette, Omni, and Pacer}, {Chevette preferred to Omni and
Pacer}, and {Omni preferred to Pacer}. Because these
three exploded (decomposed) rank ordered choice sets
are statistically independent, they can be thought of as
equivalently being obtained from independent decision
makers. Thus, the explosion process applied to preference rank ordered choice set data results in additional
choice observations for analysis at the estimation stage.
As the accuracy of the parameter estimates of the stochastic utility model depends on, among other factors,
the number of independent choice sets available for analysis, the explosion (decomposition) process exploits the
information content in a preference rank ordering by
generating multiple statistically independent choice observations from each decision maker's rank ordered preferences. These additional choice observations will, in
general, lead to "better" estimates of the parameters of
the stochastic utility model, "better" being interpreted
in the sense of reduced sampling variance. The explosion process is also important in terms of sampling cost.
Obtaining additional choice sets from independent decision makers is generally more expensive than asking
a smaller set of decision makers to supply the complete
preference rank ordering of all elements in their choice
sets (in addition to the chosen alternative). Hence, the
rank order explosion process can be viewed in either of
two complementary ways: (1) for a given number of
rank ordered choice sets, the explosion process leads to
incremental independent choice observations and, hence,
to decreased sampling variance of the parameter estimates, or (2) to reduce sampling costs, rank ordered
choice sets can be obtained from independent decision
makers (for a relatively small incremental cost over just
collecting data on the chosen alternative) and, by use of
the explosion process, sufficient independent unranked
choice sets can be generated from the rank ordered
choice set to yield a desired degree of precision in the
resulting parameter estimates.
^ Ujj). (Note that the trivial case
Two other implications of the explosion rule should
be noted. First, the rule states only that the J, - 1 choice
events derived from a decision maker's preference rank
ordering are statistically independent and, thus, can be
thought of as equivalently being obtained from 7, - 1
independent decision makers. (No statement apparently
is made about the error term distribution associated with
the exploded choice sets.) Because the elicitation of
preference rank ordered alternatives may involve the
decision maker in forecasting hypothetical choice behavior (which may not be identical to actual choice behavior), some "noise" may be introduced into the data
collection process. This is equivalent to observing error
terms with larger variances for lower ranked alternatives. Such a phenomenon might occur because the consumer decision makers give less thought to "choices"
associated with lower ranked alternatives. However,
such a premise is not consistent with the underlying assumptions that lead to the development of the explosion
rule, as the derivation of equation 5 is based on the assumption that the error terms are IID. Consequently, if
one suspects that the lower ranked alternatives are
"noisier" and have greater error variance than the
higher ranked alternatives, one must explicitly test for
such a condition prior to use of the explosion rule.
A second implication of note is the Luce and Suppes
Ranking Choice Theorem assumption that decision makers rank their choice set alternatives from top to bottom.
Though this behavioral model of the ranking process is
certainly plausible, choice set alternatives might be rank
ordered in other ways. Interestingly, if one assumes that
the decision makers reverse this procedure (by successively deleting inferior choice set alternatives from consideration), the choice probabilities generated from such
a bottom-to-top procedure will be equal to the top-tobottom choice probabilities only if the alternatives in
each choice set are equally likely to be chosen, a rather
restrictive situation (Luce and Suppes 1965, p. 356-8).
Therefore, it matters in a theoretical sense whether decision makers rank their choice sets from top to bottom,
because the explosion process holds only if this is true.
Consequently, if the information content of rank ordered
choice sets is to be exploited, the marketing researcher
must try to ensure that the decision makers' behavior in
ranking choice set alternatives approximates the top-tobottom behavioral model. Any violation of the top-tobottom ranking model will lead to "noisy" choice sets
being generated by the explosion process. (Two obvious
unresolved research questions flow from this discussion:
"How do consumer decision makers actually go about
the process of rank ordering a given choice set?" and
"How well are alternative ranking paradigms approximated by the top-to-bottom behavioral ranking assumption?" The latter question is of particular note in the
context of generating a ranking of choice set alternatives
from ratings data supplied by consumer decision makers.)
EXPLOSION STRATEGIES
The ideal environment for gathering rank ordered
choice set data would be a laboratory setting. The decision makers could be presented with a finite set of alternatives and asked to choose the most preferred ele-
JOURNAL OF MARKETING RESEARCH, AUGUST 1982
292
ment. This most preferred element would be assigned
rank one and then removed from each choice set. Each
decision maker would then be asked to select the best
alternative from among the remaining ones. These second choice alternatives would be assigned rank two.
This process would be iterated until the choice sets were
completely ranked from top to bottom.
Complications arise, however, in gathering such rank
ordered preference data with a survey research design.
One usually cannot control how the respondent undertakes a rank ordering task. Some respondents might indicate only the alternative actually chosen, whereas others might provide rank order information for the first
couple of alternatives but not for the rest of the elements
in the choice set. Therefore, the question arises of how
to cope with partial and/or "noisy" preference rank
ordered choice sets.
Partially rank ordered choice sets can be treated in a
straightforward manner. Denote r, as the depth of available rank order information for decision maker / on the
alternatives in choice set C, (where I :< r, < 7, — 1).
Then, the maximum number of exploded choice observations that could be made available for analysis would
be equal to S^^, min{r.,y. - 1). The practical significance
of partially complete rank order information is that the
unranked alternatives cannot be exploded, but the ranked
alternatives can still be decomposed into unranked
choice events in the usual fashion.
The conceptual problem associated with using rank
ordered data where the error terms might not be IID and/
or the ranking might have been performed in some manner other than top down is more complicated. One solution is to ignore the available rank order information
and analyze the choice set data according to the basic
stochastic utility model framework. A second, more
constructive approach might be to ask respondents to
provide ratings of the choice set alternatives. Such rating
data could then be used to derive choice set rankings.
This approach would allow respondents to provide identical ratings of alternatives they feel less sure about in
the preference rank ordering (as well as allowing the
respondent not to rate some alternatives because of lack
of relevant information or unfamiliarity). A third approach might be to elicit confidence judgments directly
from respondents on the degree of confidence associated
with the ranking (or rating) of each choice set alternative. A fourth approach is to explode only partially the
available rank ordered choice sets. Define E as the depth
of the explosion (decomposition) process as specified by
the marketing researcher. Then I{E), the number of independent choice observations available after exploding
the rank ordered choice sets to a depth of E, is defined
as
(10)
/(£•) =
.r,,y; - 1).
If the marketing researcher decides to use all of the
available rank order information (and treat all of the rank
ordered alternatives as being equally reliable), the maximum number of exploded choice observations would be
(11)
,,/, - 1).
Given a distribution of r- and J- values across the sample of decision makers, I{E) will be a monotonically
nondecreasing function of E, and I{E) will typically increase at a decreasing rate. The defmition of /(£) in
equation 10 implies that I{E)-^ I{E-\' 1), where the
equality holds when E is greater than or equal to the
maximum of r. or J- - 1 for all /.
In statistical terms, the important strategic question is
how E should be chosen. Larger values of E will generate more independent choice observations, leading to
reduced sampling variance of the parameter estimates of
the stochastic utility model, at least in principle. However, larger values of E result in explosion of observations that are farther down in the rank orders supplied
by the sample of decision makers. Because these orderings are more likely to be characterized as random or
"noisy" (or at least less systematic with respect to the
rules used by the decision makers to choose the first few
alternatives in the preference rank ordering), the effect
of adding such "noisy" orderings would be to bias the
parameter estimates toward zero. Thus, the tradeoff involves reducing sampling variance of the parameter estimates versus the possibility of less reliability (and
greater "noise") for the incremental choice observations
generated by explosions. Hence, an important factor in
tiie choice of E is the responsiveness of sampling variance to increases in the number of choice sets available
for analysis.
An heuristic approach to choosing E involves plotting
values of a likelihood ratio index versus different values
of E. McFadden (1974) has proposed the following likelihood ratio index as an overall goodness-of-fit measure
(analogous to the multiple correlation coefficient in linear statistical models).
(12)
- 6)
= 0)
where L is the log likelihood function. To the extent that
the maximum likelihood parameter estimates explain the
choice process completely, R^ will approach one in value
because L(0 = 9) will approach zero in value. If 0, the
vector of MLE parameter estimates, is essentially equal
to 0 (implying random choice among the choice set alternatives), Z-(6 = 6) will approach L(6 = 0) in value,
implying that R^ will approach zero in value. Hence,
R^ will vary between zero and one depending on the
"explanatory power" of 6. Now, because R^ does not
depend on the number of available choice observations,
the calculated R^ values should remain approximately
constant as E increases. If ^^ starts to decrease substantially at some value of £, this would imply that "noisy"
STOCHASTIC UTILITY MODEL
exploded choice observations had been added to the
available pool of choice sets, and the explosion process
should be terminated just prior to the "elbow in the
curve."
A second consideration in choosing E is related lo the
researcher's prior beliefs and knowledge about the
choice process being studied. Such beliefs and knowledge might lead the researcher to not consider values of
E beyond some practical upper bound.
Another factor to consider is the distributions of r, and
J, values which serve to restrict the possible range of
explosion depths. For example, if the mean depth of
available rank order information is small—say
r = 2.5—1(5) probably will not be much larger than
/(4). Thus, the researcher might decide not to explode
beyond a depth of 4 because few additional choice observations will be generated and the extra exploded
choice observations beyond £ = 4 might be of a priori
questionable quality.
The strength of these heuristic approaches is that they
are relatively simple to use. However, an obvious question is whether an analytical approach exists for determining the extent to which rank ordered choice sets
should be exploded (decomposed). One formal approach
involves grouping choice observations by depth of explosion and sequential hypothesis testing. Define the
first subgroup of choice observations to consist of the
/(£) choice sets generated by an explosion to a depth of
E. The second subgroup then consists of the incremental
/(£ + 1) - /(£) choice sets generated by exploding to
a depth of £ + I. If /(£ -I- 1) - /(£) is large enough
to allow reasonable parameter estimates to be developed,
the hypothesis that 9*^"' = 9'^"* can be tested by a statistical testing procedure suggested by Watson and Westin (1975). This test affords an assessment of whether
two data subgroups should be pooled for estimation purposes. To test the hypothesis that 9"* = 9'^*, the appropriate test statistic is
-2{L{% = e"*') - [L(e = e'") + UQ = e'^')]}
where 9""' is the MLE of 9 obtained by pooling the data
subgroups, and 9'" and 9'^* are the MLEs for the separate
data subgroups. This test statistic will be asymptotically
distributed chi square with N degrees of freedom (Wald
1943). In application of the Watson and Westin pooling
test, the failure to reject the null hypothesis implies that
the two data subgroups come from the same underlying
choice process, with the same error term structure. Consequently, these two data subgroups can be pooled for
the purposes of estimation. Thus, this is an exact test of
whether the assumptions underlying the explosion process hold with real data; if the null hypothesis that
Q{F) ^ Q<£+i) (;aj^j,ot be rejected, the available choice set
data are consistent with the requisite assumptions of the
constant utility model. If this null hypothesis is rejected,
no information is provided as to which assumption (or
assumptions) is violated by the data. This grouping and
sequential testing procedure can be iterated for succes-
293
sive values of £ until either the hypothesis that the
subgroup parameter vectors are equal is rejected or the
quantity /(£ + 1) - /(£) yields too few exploded choice
observations to provide meaningful parameter estimates.
The explosion process would terminate when either of
these conditions is encountered.
SMALL SAMPLE PROPERTIES OE THE
CONDITIONAL LOGIT ESTIMATION PROCEDURE
The known properties of maximum likelihood estimators have been derived only under asymptotic conditions. To assess the value of generating additional statistically independent observations by exploding rank
ordered choice sets, one must investigate the small sample properties of the multinomial logit model parameter
estimates. Of particular interest to marketing researchers
is the situation described by McFadden (1974) as "conditional logit estimation," in which there is one choice
set per decision maker and no replications. As well as
examining how the precision of the estimates is affected
by the number of choice sets available for analysis
(among other factors), we investigated several related
issues such as unbiasedness of the estimates and computational costs.
Design of the Mont^ Carlo Experiments
The general procedure used to investigate the small
sample properties of the conditional logit estimates was
Montd Carlo experimentation on artificially generated
data. Choice sets with controllable characteristics were
generated and the choice process was simulated with the
probabilistic choice model in equation 5. The simulated
rank ordered choice sets were then exploded and the
conditional logit estimation procedure was applied to the
resulting unranked choice observations. Because the true
model parameters were known, we were able to examine
the extent to which the estimation technique recaptured
the true parameter vector —i.e., the extent to which 9
approximates 9.
The experimental factors hypothesized to affect the
ability of the conditional logit estimation procedure to
recapture the true parameter values included (1) the
number of parameters to be estimated (AO, (2) the number of choice sets available for analysis after explosion
(NOBS), (3) the average number of alternatives in the
choice sets after explosion (SIZE), (4) the relative size
of the deterministic portion of the choice model in comparison with the error component as discussed in the
Appendix (SCALE), and (5) the coUinearity among the
variables in the model (COLL). SIZE and NOBS are inversely related because both depend on the depth of the
explosion chosen by the analyst.'
'Because each successive explosion generates additional choice sets
with one less alternative (because Ihe most preferred alternative is
removed from consideration in forming an exploded choice set). SIZE
decreases as E increases. Furthermore, as NOBS increases with increases in E, the indirect effect is that SIZE and NOBS are inversely
JOURNAL OF MARKETING RESEARCH, AUGUST 1982
294
The scale of the model parameters is defined as
(13)
SCALE =
Because the size of the error component was known and
the variance of the Z vectors could be determined, we
were able to select SCALE values to represent different
levels of "explained variance" ranging from about 10%
(SCALE = 0.45) to about 85% (SCALE = 3.50).'
The coUinearity among the individual components of
Z is captured by an index value designed to measure the
overall average correlation among the variables describing the choice set alternatives. The coUinearity index
used in this study is
(14)
COLL =
N(N - 1)
where X^ is the n* eigenvalue associated with the generated data (where the usual convention of ordering the
eigenvalues from largest to smallest is followed). COLL
is bounded between zero and one. If the variables are
orthogonal (i.e., completely uncorrelated), X., = X.; ~
... = Xjv = 1 and COLL = 0; if the data are completely
correlated and have rank 1, X, ^ A' and the other eigenvalues will be equal to zero and COLL = 1.
The experimental design was factorial with one replication per cell. The factorial design set four factors at
each of the following levels.
N =
/ =
E =
SCALE =
2,4, 7, and 10,
40, 100. 200. and 400,
1, 2. 3, 5, and 10.
0.45, 0.85. 1.375, and 3.50.
Hence, a total of 320 ( = 4 x 4 x 5 x 4 ) experiments
were conducted. Within this Mont6 Carlo study, NOBS
is defined as
(15)
NOBS = ^ min(EJ, - 1).
Within each cell in the experimental design, SIZE and
related. This follows because both SIZE and NOBS are functions of
E. Note that SIZE and NOBS arc only correlated within the context
of an explosion, because when E increases so does NOBS, but at the
expense of slightly decreasing the average choice set size (SIZE). Ex
ante (before explosion) measures of SIZE and NOBS are uncorrelated
within the factorial experimental design used in this study.
^These values of SCALE were chosen by noting that if the components of Z are standardized to have unit variance, it follows that
Var(t/) = SCALE' + Var(e). Because Var 6 = 1.645 for the double
exponential distribution, the usual partitioning of total variation into
explained variation and unexplained variation implies that:
"proportion of
_
SCALE
explained variation"
SCALE^ + 1.645
*""•'
COLL were chosen stochastically, but in such a way as
to reflect the usual kinds of conditions an empirical marketing researcher might encounter in real choice behavior process data. With regard to choice set size prior to
explosion, each choice set was drawn from a normal
distribution (whose mean was determined by a draw
from a uniform distribution with a range of 2 to 10 and
a standard deviation drawn from a uniform distribution
with a range of 1 to 2). The actual size of any choice
set (prior to explosion) was truncated so that 2 < 7. < 10
for each choice set /. (The lower limit on choice set size
is the absolute minimum number of alternatives that can
constitute a choice set; the upper limit of 10 was chosen
to reflect the empirical reality of bounded choice set
sizes.) CoUinearity was induced into the variables characterizing the choice set alternatives by a transformation
procedure described by Chapman (1981).
The artificial choice set data in each cell of the experimental design were generated in the following manner. A total of / choice sets were generated, each choice
set containing J^ alternatives (where the J^ values were
determined as described above). Each of the N attributes
of each alternative were generated by drawing independently distributed normal random variables with mean
zero and variance one. CoUinearity was induced into
these data. The transformed collinear data were standardized so that each attribute had mean zero and variance one. COLL was then calculated.
The next series of steps involved simulating the rank
order choice process. A true parameter vector was generated by drawing A' independent values from a uniform
distribution with a range of zero to one. Positive and
negative signs were attached to each of the parameters
(the signs being determined with a probability of 0.5)
and the resulting true parameter values were rescaled so
that relation 13 was satisfied. Next, the probabilistic
choice model in equation 5 was used to assign the choice
probabilities to each alternative. The "chosen" alternative was assigned rank one.^ The choice probabilities
of the "nonchosen" alternatives were then rescaled to
sum to 1.0, and the "choice" process was repeated to
determine the rank two alternative. This procedure was
iterated to rank all of the alternatives in each simulated
choice set. Finally, choice sets were exploded to a depth
of £, resulting in NOBS choice sets. These exploded
choice sets were input to a conditional logit estimation
program (Manski 1974) to obtain the parameter estimates.
Several additional points should be noted. First, this
type of experimental design involves an implicit assumption that each simulated decision maker employs
the same underlying choice model. Thus, there is no
heterogeneity problem to confound the estimation process. Second, reliability is not an issue in these experi'By use of a random number generator, an alternative was "chosen" with probability equal to the choice probability assigned to the
alternative.
STOCHASTIC UTILITY MODEL
295
ments because the same underlying probabilistic choice
process was employed to rank all of the choice set alternatives. Third, only the non-extreme values of
COLL ^ 0.75 were considered, reflecting the usual presence of modest degrees of interrelationships among the
attributes of the choice set alternatives. Because of these
points, the results reported in the rest of this section
probably represent an upper bound on the degree to
which the conditional logit model estimates recapture the
"true" underlying choice model. These results are "better" than might be obtained with real choice data because potential problem sources such as respondent heterogeneity, measurement and specification errors, and
unreliable rank order information are not present.
In each cell of the experimental design, several performance measures were calculated. These measures
were designed to represent the extent to which the conditional logit estimation procedure recaptured the true
parameter vector values associated with the underlying
multinomial logit model. The performance measures included the following.
1. RMSE (root mean squared error), where
N
log RELRSV = 0.1812 - 0.5753 logNOBS
(0.4270) (0.0487)
- 0.2164 \og(SIZE-\)+ 0.1849
(0.1260)
(0.1138)
3. BIAS (bias), where
1
-^(Q„-%„)
4. TIME (CPU execution time in seconds, including only
the time used for estimation and excluding data generation time requirements)
In performance measures 1-3, 6^ is the known true value
of the «'^ parameter in the multinomial logit model and
§„ is the maximum likelihood estimate obtained from the
conditional logit estimation procedure.
Results of the Mont4 Carlo Experiments
The results of these experiments were analyzed by regressing variants of the performance measures on theoretically plausible causal experimental factors.
Sampling variance. One component of the accuracy
of the conditional logit estimation procedure is sampling
variance. To reflect relative sampling variation (and to
avoid the problem associated with, ceteris paribus, large
parameter estimates having associated large standard errors), the performance measure chosen was the relative
root squared variation (RELRSV), defined as
where:
Thus. RELRSV is a measure of relative variation analogous to a coefficient of variation.
Theoretically, RELRSV = f(NOBS, SIZE, COLL,
SCALE, N) with prior beliefs that, ceteris paribus,
RELRSV should decrease with increases in NOBS, SIZE,
and SCALE and decrease with decreases in COLL and
A'. The theoretical arguments associated with the signs
on NOBS, SIZE, and A' all reflect "degrees of freedom"
concerns. The expected positive sign on COLL is due
to coUinearity being posited to lead to less precise estimates of the model's parameters. SCALE reflects the
relative magnitude of the systematic component in the
model and, thus, should be negatively related to
RELRSV—a priori, a greater magnitude for the "explained variance" component of the model should be
associated with an improvement in the precision of the
estimates (a decrease in RELRSV). An interactive functional relationship was postulated and the model was
estimated after appropriate logarithmic transformation.
The OLS results were
(18)
2. MAE (mean absolute error), where
(16)
n-i
\ 1/2
RMSE = I -
BIAS =
(17)
RELRSV=
RMSE
- 0.6863 log 5CAZ,£+ 1.6146
(0.0670)
(0.0919)
^^ = 0.65
No. ofobs. = 320
where the standard errors of the coefficient estimates are
in parentheses and log is the natural logarithm. The form
of the regression model in equation 18 reflects the theoretical relationship between COLL and RELRSV. As
COLL approaches one in value, RELRSV increases without bound regardless of the values of the other factors.
Also, note that SIZE - 1 is used to describe the choice
set sizes because the conditional logit estimation procedure is based on the number of within-choice-set comparisons available for analysis (i.e., the chosen alternative versus the other J - 1 inferior alternatives).
Given the interactive functional form, the coefficients
on NOBS, SIZE - 1, SCALE, and A' can be interpreted
as elasticities.
The estimated coefficients in equation 18 all have the
theoretically posited signs. The main conclusion to be
drawn from these results is that the explosion process
leads to important sampling variance efficiencies. The
coefficient estimate of -0.5753 for log NOBS implies
that RELRSV decreases approximately with the square
root of the number of choice sets available for analysis
after explosion, ceteris paribus. A comparison of the
coefficients on log NOBS and log (SIZE - 1) is useful
296
for assessing the marginal tradeoffs between the number
of available choice observations after explosion and the
average choice set size after explosion. For example, a
researcher who has 100 choice sets each with five ranked
alternatives could use £ = 1 (and NOBS = 100 and SIZE
= 5) or the data could be exploded to £ = 4 (where
NOBS = 400 and SIZE correspondingly declines to 3.5).
Using equation 18, we can easily show that RELRSV for
the former strategy (E = 1) is about twice that for the
latter strategy {E = 4). Further, to obtain the same level
of RELRSV as that when £ = 4 by retaining £ = 1 and
increasing the average choice set sizes, we would have
to increase SIZE from five to about 100. (In this experimental design, there was much more variation in
NOBS than in SIZE; NOBS had a range of 40 to about
2600 whereas SIZE ranged from about 2.6 to 9.2. If
SIZE had been varied much more, say to 100, it might
have exerted a stronger influence on RELRSV. Still, as
most empirical applications of the multinomial logit
model would probably involve mean choice set sizes of
fewer than 10 alternatives, this experimental framework
seems realistic. Note also that, in many applications, the
size of the choice sets is determined by the consumer
decision makers, and is not at the control of the marketing researcher. The number of sample decision makers, however, is often at the choice modeler's control.)
CoUinearity among the attributes seems to have only
a negligible impact on sampling variance. This result is
very encouraging because real choice data should be expected to exhibit patterns of coUinearity.
As was expected, the precision of the conditional logit
estimates is directly related to SCALE, a surrogate for
the degree of systematic behavior exhibited by the sample of decision makers. The coefficient estimate on log
SCALE implies that a doubling of SCALE would lead to
about a 68% decrease in RELRSV.
A doubling of ^V, the number of model parameters,
leads to about a 160% increase in RELRSV, ceteris paribus. Because theoretical considerations guide the determination of the number of parameters in the stochastic
utility model, these results are useful to a choice modeler
in assessing the appropriate sample size to obtain a desired level of precision in the resulting parameter estimates. For example, to obtain equivalent precision for
parameter estimates in a four-parameter model estimated
with 100 choice sets, the choice modeler would require
about 260 choice sets for a more theoretically complex
model with eight parameters.
The model reported in equation 18 was also estimated
with relative absolute variation as the dependent variable. The OLS results were virtually identical to those
reported in equation 18. Apparently, the relationship
between the external factors (i.e., NOBS, SIZE, and so
on) and the sampling variance of the conditional logit
estimates does not depend on whether accuracy is measured in terms of a quadratic or a linear loss function.
Although the small sample properties of the conditional logit estimates are described clearly by the regres-
JOURNAL OF MARKETING RESEARCH, AUGUST 1982
sion results reported in equation 18, the explanatory
power of the model is rather low. The reason is partly
the experimental design used in this Monte Carlo study.
Because only a single replication was performed in each
of the 320 cells in the factorial design, we could not
account for the within-cell variation of the performance
measures (the dependent variables). The source of this
within-cell variation is the probabilistic choice process
itself: the "chosen" alternative in each exploded choice
observation is determined by a probabilistic model and
a single replication of the choice process reflects but a
single point estimate of the parameter values. To measure the approximate magnitude of this replication error,
five replications of the simulated choice process were
conducted in each of the 16 cells characterized by the
factorial design with A' = 4 and 7, / = 100 and 250,
£• = 1 and 4, and SCALE ^ 0.50 and 2.00. The average
standard deviation of the log RELRSV values over these
16 cells is 0.499. Because the standard error of the estimate in equation 18 is 0.894, the replication error represents about 56% of the total unexplained variance in
the regression model in equation 18. If means of replicated choice processes had been used as the dependent
variable in equation 18, we could reasonably expect that
the total unexplained variance would have been reduced
by about 56%. The corresponding R^ value would have
been increased to about 0.85. Such resulting unexplained variance (about 15%) is much more in line with
a priori expectations.
Biasedness. Because the conditional logit estimates
are obtained by means of maximum likelihood estimation techniques, the property of unbiasedness is guaranteed only for large samples (asymptotically). Consequently, it would be useful to know the degree and
direction of any small sample bias.
For the 320 cells in the experimental design, the mean
BIAS is —0.025 with an associated standard deviation
of 0.311. A two-tailed test of the hypothesis that the
mean BIAS of the conditional logit estimates is zero
leads to the conclusion that the unbiasedness hypothesis
cannot be rejected at the conventional 5% level of significance (p = 0.15).
To test for the possible presence of conditional
bias—bias depending on the external factors—BIAS was
regressed against several theoretically plausible combinations of the external factors (NOBS, SIZE, COLL,
SCALE, and A'). None of the resulting F-statistics for the
regression models tested yielded significant values.
Though it is not possible to prove unbiasedness via
simulation, these Monte Carlo results may be viewed as
support for an hypothesis of unbiasedness. The possibility that subsets of the parameter estimates could be
biased is not precluded. That is, offsetting biases in individual parameter estimates could be present.
Computational considerations. The final performance
measure of interest was computational cost. Because the
explosion process yields additional choice sets for analysis, the decision of bow much to explode the available
STOCHASTIC UTIUTY MODEL
297
rank ordered choice set data might be influenced by the
extra associated computational costs. The performance
measure for computational costs is execution time,
TIME, which refers to the CPU execution time for the
Newton-Raphson algorithm (in seconds) used in Manski's (1974) program.
Theoretically, TIME =/(NOBS,SIZE, N) with TIME
increasing with increases in each of the external factors.
Moment matrices must be assembled at each iteration of
the algorithm and the number of elements processed at
each iteration equals NOBS(SIZE - 1)^. Also, the moment matrix must be inverted at each iteration and the
time for such an operation depends approximately on A'^.
Regressing TIME on these factors yielded the following
OLS results.
(19)
TIME = -2.2548 + 0.001991 NOBS(SIZE-\)N
(0.9866) (0.000040)
+ 0.1175 N^
(0.0185)
= 0.91
No. of obs. = 320
where the standard errors of the parameter estimates are
in parentheses. These results imply that TIME increases
linearly with increases in NOBS and SIZE, and increases
approximately quadratically with increases in N. These
results can also be used as a guide to the magnitude of
the computational costs associated with estimating the
parameters of the multinomial logit model. Naturally,
the absolute amount of execution time depends on the
specific algorithm used and the particular computer installation on which the analysis is performed. (All conditional logit estimations in the study were performed on
an IBM/360.) However, relative times for various combinations of NOBS, SIZE, and A^ should be approximately equivalent across different algorithms and computers.
For many, if not most, marketing research applications of the multinomial logit model, computational
costs apparently will not be an inhibiting force. Computational costs tend to be in the order of seconds rather
than minutes. For example, with 250 choice sets of average size four with four parameters to be estimated, the
application of the regression results reported in equation
19 yields a predicted execution time value of about 5.6
seconds. An eight-parameter model of the same choice
set configuration would require about 17.2 seconds of
execution time.
Precision and Cost Tradeoffs
To illustrate the nature of the tradeoffs between reduced sampling variance and additional computational
costs associated with analyzing exploded choice sets,
and to indicate how a choice modeler might use the results reported in equations 18 and 19, consider the following example. Suppose a researcher is working with
completely ranked choice sets of size eight for 100 consumer decision makers. Further, the variables charac-
Table 1
CALCULATIONS OF RELRSV At^D TIME (CPU EXECUTION
TIME, IN SECONDS) IN THE EXAMPLE ILLUSTRATING
PRECISION A N D COST TRADEOFFS
E
I
2
3
4
NOBS
100
200
300
400
SIZE
8
7.5
7
6.5
iV - 5
RELRSV
TIME
0.96
7.7
0.65
13.6
0.53
18.6
0.45
22.6
N = 10
RELRSV
TIME
2.93
23.4
2.00
35.4
1.61
45.3
1.39
53.3
terizing the choice set alternatives are expected to have
some correlation among them (COLL = 0.25) and a
SCALE value in the area of about l.(K) is anticipated.
Table 1 lists the predicted values of RELRSV and TIME
under various explosion depths for five- and 10-variable
multinomial logit models. The values in Table 1 were
calculated by using the results in equations 18 and 19.
Several features of these data may be noted. First, as
NOBS increases because of explosions, SIZE decreases
because choice sets of smaller size are being added to
the available pool of choice observations. Second, the
decrease in RELRSV is approximately proportional to the
square root of the increase in NOBS. Hence, to halve
RELRSV, the choice modeler requires about four times
as many choice observations. Third, even for the largest
model situation in this example (in terms of computational cost), when E = 5 and N = \0, computational
costs are not prohibitive, amounting to slightly less than
a minute in CPU time.
For a researcher deciding on the size of a choice sample study and on subsequent explosion strategy, the results in Table 1 offer some insight into the precision of
the multinomial logit model parameter estimates. Suppose a marketing researcher would like each of the coefficient estimates in a five-parameter multinomial logit
model to be statistically significant (i.e., have an estimated standard error no larger than about one-half the
size of the coefficient estimate). Given the conditions in
this illustrative example, explosion to a depth of four
would be required. Of course, the choice modeler might
choose to gather more than 100 choice sets to start with
so that a lesser explosion depth would be possible. By
using the results in equation 18, the thoughtful marketing researcher can anticipate the impact of choice study
sample sizes and possible explosion strategies on the
precision of the parameter estimates before collecting
data.
AN ILLUSTRATIVE EMPIRICAL APPLICATION OE
THE EXPLOSION PROCEDURE
To illustrate the application of the explosion procedure
for exploiting the information content of rank ordered
choice set data, we use a college choice behavior modeling study. Chapman (1977) has given the full details
JOURNAL OF MARKETING RESEARCH, AUGUST 1982
298
of this study and a summary of the empirical results
(1979).
The data base for the study consisted of admitted
freshmen applicants to Carnegie-Mellon University, a
private independent coeducational university in Pittsburgh, Pennsylvania. A survey research questionnaire
to admitted applicants was used to obtain the required
preference rank order and choice set composition information, as well as information on financial aid offered
by schools to each high school student. These data were
combined with other information contained in CarnegieMellon's applicant information system (derived from
each applicant's application forms) and other publicly
available information (college alternative characteristics
and costs) to yield the required study data. Self-reported
preference rank order and financial aid information was
subjected to consistency and reliability checks. Across
the various academic areas in this data base, average
choice set sizes ranged from 2.9 to 3.5 college alternatives, and less than 10% of the students in the sample
had more than five college alternatives in their choice
sets.
The variables in the college choice mode! were of
three categories; college characteristics variables; price,
cost, and financial aid variables; and a miscellaneous
group. To describe the attributes and characteristics of
colleges, 46 different variables were obtained from publicly available sources, such as college guidebooks. Factor analysis procedures were used to reduce these 46
variables to a set of composite indices that would serve
as suitable proxies for the college attributes. The six factors extracted accounted for about 58% of the variance
in the original 46 raw variables. The results of a varimax
rotation led to the six factors (ES^, ..., ES^,) being interpreted as quality/affluence, size/graduate orientation,
technical orientation, ruralness, fine arts orientation, and
liberalness, respectively. In the financial area, the following variables were used in the choice mode!:
DRAIN
GRANTS,
INCOME
GRANTS
, and
TOTALAID
(GRANTS is total scholarship aid, DRAIN is total out-ofpocket costs, INCOME is parental income, and TOTALAID is total financial aid.) Collectively, these variables
account for amount and kind of financial aid (mix of
financial aid), and for costs in relation to income. Within
the miscellaneous group of choice model variables are
MILES (distance from college campus to place of residence), COMMUTE (a dummy variable which equals
one if the student could commute to a college campus
and equals zero otherwise), and (SAT^ — SATj)^ (where
SAT refers to mean scholastic aptitude test score, the
subscript i refers to the student, and the subscript7 refers
to a college), which attempts to measure the impact of
"quality zoning"—the desire to attend a college that is
neither too much above nor below the student in average
student quality.
Statistical results of estimating this college choice
Table 2
COEFFICIENTS OF THE COLLEGE CHOICE MODEL
FOR " L O W " INCOME ENGINEERING A N D
SCIENCE STUDENTS (ASYMPTOTIC STANDARD
ERRORS IN PARENTHESES) FOR EXPLOSION DEPTHS
OF 1, 2, A N D 3
FS, (quality/
affluence)
FSi (size/graduate
orientation)
FSi (technical
orientation)
FSi (niralness)
FSi (fifie arts
orientation)
FS^ (liberalness)
GRANTS
1000
DRAIN
INCOME
GRANTS
TOTALAID
(MILES/IO)"^
COMMUTE
\SAT,~ SATJY
I
100
NOBS (after
explosion)
£,(9 = 0)
L(e = 9)
R'
j
Explosion strategies
E =2
E=3
E= I
0.6894
0.6100
0.6955
(0.1353)
(0.1123)
(0.1037)
0.4611
0.4104
0.3221
(0.1348)
(0.1057)
(0.0974)
0.2156
0.2351
0.2428
(0.1237)
(0.0984)
(0.0914)
-0.0690
0.0281
0.0359
(0.1237)
(0.0952)
(0.0902)
0.0205
-0.1208
-0.1503
(0.1446)
(0.1137)
(0.1057)
0.1423
0.1044
0.0811
(0.1346)
(0.1105)
(0.1020)
0.2115
0.3608
0.1839
(0.1931)
(0.1620)
(0.1526)
-1.6529
(0.8737)
-2.0704
(0,7459)
-1.3354
(0.6707)
0.3528
(0.4509)
-0.3439
(0.1895)
-0.0978
(0.3537)
-0.3863
(0.1710)
0.3701
(0.3750)
-0.1397
(0.1403)
0.0524
(0.2763)
-0.4081
(0.1425)
0.0442
(0.3501)
-0.0822
(0.1271)
0.1474
(0.2546)
-0.3188
(0.1331)
303
-309.18
-240.80
0.221
452
-448.23
-353.16
0.212
505
-496.53
-399.60
0.195
model with explosion depths of 1, 2, and 3 are reported
in Table 2. These results are for " l o w " income engineering and science students, where " l o w " is defined
to be the bottom half of the applicants in terms of parental income (for those students who had applied for
financial aid and, hence, had supplied parental income
information).
The main result to be noted in Table 2 is that the standard errors of the parameter estimates decrease as more
choice sets become available for analysis due to the explosion process for exploiting preference rank ordered
choice set data. This result illustrates the main value of
the explosion procedure: explosion of rank ordered
choice sets leads to more choice observations being
available for analysis which leads to improved precision
(reduced sampling variance) in the stochastic utility
model parameter estimates.
The key strategic issue faced by the choice modeler
is the choice of an explosion strategy. In the college
STOCHASTIC UTILITY MODEL
choice results reported in Table 2, the heuristic of examining /?' as E increases would probably lead the
choice modeler to conclude that an explosion depth of
2 is reasonably safe (as R^ only declines slightly in shifting from E = I to £ = 2, namely from 0.221 to 0.212).
but there might be some uncertainty in resolving whether
£ = 3 should be preferred to £ = 2.*
In using the grouping and sequential hypothesis testing procedure for statistically testing for an "optimal"
explosion depth, it is useful to begin by noting that the
critical chi square vaiue for a 5% level of significance
is 21.03, whereas the corresponding value for a 1% level
of significance is 27.69 (for 12 degrees of freedom, the
number of parameters in this choice model). Recall also
that the null hypothesis being tested is whether 9'^^ =
9'^*". In assessing whether a change from E = 1 to £
= 2 is appropriate, the relevant log likelihood values are
Z.(9 = 9 ' " ' ' ) = - 3 5 3 . 1 6 , L(e = 9'") = - 2 4 0 . 8 0 , and
L(Q = 9'^^) = -105.67. (In notational form, 9'"^' is the
MLE for the pooled exploded rank ordered choice sets
obtained by exploding to a depth of 2, 9'" is the MLE
for the choice sets based only on the first most preferred
alternative, and 9'*" is the MLE for the choice sets based
only on the second preferred alternative.) The relevant
chi square test statistic value is therefore equal to
- 2 ( - 3 5 3 . I 6 - {1-240.80] + [-105.67]}) = 13.38.
Comparing this calculated test statistic value with the
relevant critical values leads to the conclusion that the
null hypothesis should not be rejected on the basis of the
available sample evidence. Therefore, pooling the observations is feasible, and an explosion depth of at least
2 is warranted. In iterating this test to determine whether
an explosion depth of 3 is appropriate, the relevant log
likelihood values are L(9 = 9"^^*'*) = -399.60,
L(9 = 9"*^')= -353.16, and ^(6 = 9*'') = - 3 2 . 5 0 .
The relevant chi square test statistic value is equal to
27.88, which is above the 1% significance critical chi
square value. Thus, one could conclude that the null
hypothesis should be rejected in this situation. As the
available sample evidence seems inconsistent with the
pooling hypothesis, an explosion depth of 3 seems inappropriate. The grouping and sequential hypothesis
testing procedure for choosing an "optimal" explosion
strategy leads to the conclusion that an explosion depth
of 2 is most appropriate with these choice set data.
*R^ does not depend on the number of available choice set observations, so the value of R^ should remain approximately constant as
E increases, i/the extra choice sets generated by the explosion process
are characterized by the same 6 vector. If "noisy" choice sets are
added by the explosion process (either due to unreliable rank order
information or because the lower ranked choice sets are characterized
by some 9 vector other than that which described the higher ranked
choice sets), one may observe that the value of R^ will decline with
increases iti E. Sampling variation may explain successive values of
R' that are not identical, so an appropriate statistical test is required
to assess whether the explosion is feasible in the context of any specific empirical application.
299
CONCLUDING REMARKS
Within the context of the stochastic utility mode! of
choice behavior, the information content of rank ordered
choice sets can be exploited to estimate efficiently the
parameters of the model. The availability of rank ordered choice set data leads to an explosion (decomposition) procedure for exploiting such extra information.
This explosion process involves the decomposition of a
preference rank ordered choice set into a series of unranked and statistically independent choice observations.
Because the ability to estimate the model's parameters
depends on the number of choice sets available for analysis, the explosion process represents an intuitively simple mechanism for exploiting rank ordered choice set
data.
The choice modeler must resolve the key issue of how
much to explode the available rank ordered choice set
data. Several considerations bear on this issue. First,
because the computational costs are rather minor, the
difference in computational time from exploding to a
depth of £ + I rather than a depth of £ can probably
be ignored. Second, the problem of reliability of the
preference rank order information obtained through a
survey research design may not be a major problem, as
the precision of the conditional logit parameter estimates
is related to the square root of the number of available
choice set observations after explosion. Thus, because
each successive explosion from £ to £ + I results in
incrementally smaller improvements in the precision of
the parameter estimates, exploding beyond a depth of 3
or 4 will usually not increase accuracy greatly. This is
true partly because the precision of the estimates decreases by the square root of the number of exploded
choice sets and partly because not all decision makers
in most choice situations will have choice set sizes large
enough to allow for further explosion. Also, as larger
explosions are more likely to increase the possibility of
including "noisy" choice sets, an explosion depth of
more than about 3 would not normally be appropriate in
a survey research application.
In summary, an explosion strategy will depend on the
choice modeler's knowledge of the choice process being
studied and the likely reliability of the complete preference rank order information, the use of the grouping
and sequential hypothesis testing procedure and the heuristic tools we describe, and the Mont^ Carlo results
which suggest that large explosions (say, £ larger than
3) are normally not required.
These results also have implications for survey questionnaire designs to elicit rank ordered choice set information from decision makers. Attempting to guarantee
completely rank ordered choice set data probably costs
more than the data are worth. A better research strategy
would be to ask for the first few rank ordered alternatives (say, up to three), and then ask for all other alternatives considered by the decision maker in the choice
behavior task. As the statistical worth of more than the
JOURNAL OF MARKETING RESEARCH, AUGUST 1982
300
first couple of rank ordered choice set alternatives is
marginal at best (because of reliability and "noisy" data
problems), survey questionnaire respondents need not be
confused and antagonized by having to supply more detailed information than can be used profitably in the subsequent statistical analysis.
APPENDIX
SCALE CONSIDERATIONS AND THE STOCHASTIC
UTILITY MODEL
The rank order of a set of alternatives in a choice set
will be invariant under monotone transformations of the
form W = a t / + p, for a > 0. Hence, the most general
form of the stochastic utility model would be
(Al.l)
C/y = a(e Z,^-H e,j) + p.
The model in equations 5 and 6 follows from the normalizations a = I and p = 0. Because we are concerned
only with differences in the utilities of various choice set
alternatives, p will be identical for each alternative.
Hence, no generality is lost by assuming that p = 0.
Setting a = 1 causes no difficulties with the a 9 Z. term
because a is just a scalar, and in essence just indicates
the size (or scale) of the 8 values. We estimate the vector
a 6, the relative population taste parameters. To obtain
absolute 9 values, an estimate o f a somehow determined
independently from the estimate of 6 would be required.
We only need the relative 6 values to draw inferences
about utilities, which are only relative quantities themselves. Setting a = 1 in the term ae^ does, however,
have important ramifications for the variances of the
disturbance terms.
The most general form of the double exponential distribution is
(A1.2)
Prob(e ^ t) = exp -expl -
where TI, and TI2 are location and scale parameters, respectively (Johnson and Kotz 1970, p. 276-9). If -n,
= 0 and 1^2 = 1. then equation A1.2 is identical to equation 4. Assuming T], = 0 implies that p = 0 in A l . l ;
similarly, setting 1)2 ~ 1 implies that a - 1 in A l . l .
Because the variance of the double exponential distri1 , setting TI^ = 1 implies that
bution is Var(e) = - IT
6
Var(6) = - -n^ = 1,645. Hence, fixing a in the general
6
model in A l . l implies that the variance of the disturbances is also fixed.
The preceding discussion reveals an interesting facet
of the stochastic utility model with double exponentially
distributed error terms—the variance of the disturbance
terms is a fixed number, not dependent on the estimated
6 values. In contrast, in "regression-like" models the
estimated vector of parameters affects the error terms
and, hence, their variance.
Fixing the variance of the disturbance terms implies
that the proportion of the variance of U represented by
the systematic component of the stochastic utility model
will depend directly on the " s i z e " of the 9 values. In
this way, "large" 0 values will be associated with high
explanatory power of the stochastic utility model because the variance of the systematic component will be
large in relation to the error component. The following
theorem clarifies these relationships.
Scale theorem. In the stochastic utility model t/^ =
6Zy + e where the error terms follow the double exponential distribution in equation 4, let $„ = wB^ where
E;J=, e; = 1. This scaling implies that w^ = Si)'., 6;. The
scaling factor ti> may be interpreted as the " s i z e " of the
9 values. It follows that
(a) as 0) —• a>, p^j,
max (eZ,y), and
1 for j° alternative such that
^ 0 for all other alternatives
j
(b) as co-»0, P,^->
Proof. One version of the probabilistic choice model in
equations 5 and 6. which can be derived with some simple manipulations of equation 5, is
1
with i* being some alternative in the choice set.
Part (a); For alternative / . e(Zy - Z^-) < 0 because alternative f has been defined as the most prefeijed alternative. Hence, co ^ 'o implies that exp{o}6(Z,^ Zy")}^ 0. Therefore, w -> =c implies that P,^= -» 1.
For P^j. such that;"* / / . e(Z,y- - Z^j.) > 0, and w
-» 00 implies that exp{(»>6(Z,^o - Z,y.)} -> °° so that P^j,
Part (b): as u - * 0, expfcDOCZ,^ - Z.^^.)} ^ 1 forj* = 1,2,... ,7^.
Therefore, co ^^ 0 implies that P^j. ->• - , for j * =
In summary, as the estimated parameters become
larger, the systematic component of the stochastic utility
model dominates the stochastic component and the resulting utilities of the alternatives show greater dispersion. This follows through to the probabilistic choice
model, with the probabilities of choosing the alternatives
approaching one for the alternative with greatest utility
and approaching zero for all other alternatives. In contrast, "small" 9 values would be associated with choice
probabilities approximately equal to 1/7, for each alternative. The latter empirical result implies that the choice
process is essentially random with each alternative in a
choice set having an equivalent chance of being chosen.
STOCHASTIC UTILITY MODEL
301
REFERENCES
Boskin, Michael J. (1974), " A Conditional Logit Model of
Occupational Choice." Journal of Political Economy, 82
(April). 389-98.
Chapman. Randall G. (1977), Buyer Behavior in Higher Education: An Analysis of College Choice Decision Making
Beliavior, unpublished doctoral dissertation. Graduate School
of Industrial Administration, Carnegie-Mellon University.
(1979), "Pricing Policy and the College Choice Process," Research in Higher Education, 10(1). 37-57.
(1980), "Retail Trade Area Analysis: Analytics and
Statistics," in Proceedings. Market Measurement and Analysis, Robert A. Leone, ed. Providence, RI: TIMS College
on Marketing and The Institute of Management Sciences,
40-9.
(1981), " A Note on Generating Correlated Data."
Graduate School of Business. University of Chicago.
Domencich, T. and Daniel McFadden (1975). Urban Travel
Demand: A Behavioral Analysis. Amsterdam: North-Holland Publishing Company.
Gensch, Dennis H. and Wilfred W. Recker (1979), "The
Multinomial, Multiattribute Logit Choice Model," Journal
of Marketing Re.search, 16 (February). 124-32.
Johnson, Norman L. and Samuel Kotz (1970), Continuous
Univariate Distrihutions—I. New York: Houghton Mifflin
Company.
Joskow, Paul L. and Frederic S. Mishkin (1977), "Electric
Utility Fuel Choiee Behavior in the United States," International Economic Review, 18 (October). 719-36.
Kohn. Meir G., Charles F. Manski, and David S. Mundel
(1976), "An Empirical Investigation of Factors Which Influence College-Going Behavior," Annals of Economic and
Social Measurement, 5 (Fall), 391-419.
Li, Mingche M. (1977), " A Logit Model of Homeownership," Econometrica, 45 (July), 1081-97.
Luce. R. Duncan (1959), Individual Choice Behavior: A Theoretical Analysis. New York: John Wiley & Sons. Inc.
and P. Suppes (1965), "Preference, Utility, and Subjective Probability," in Handbook of Mathematical Psychology—Volume 3, R. Duncan Luce, Robert R. Bush, and
Eugene Galanter, eds. New York: John Wiley & Sons. Inc.,
249-410.
Manski, Charles F. (1974). "The Conditional/Polytomous
Logit Program: Instructions for Use," School of Urban and
Public Affairs, Carnegie-Mellon University.
McFadden. Daniel (1974), "Conditional Logit Analysis of
Qualitative Choice Behavior." in Erontiers in Econometrics, Paul Zarembka. ed. New York: Academic Press,
105-42.
(1980). "Econometric Mexlels for Probabilistic Choice
Among Products." Journal of Busine.ss, 53, No. 3, Pt. 2
(July). S13-29.
Punj. Girish N. and Richard Staelin (1978), "The Choice Process for Graduate Business Schools," Journal of Marketing
Research, 15 (November), 588-98.
Wald, A. (1943), "Tests of Statistical Hypotheses Concerning
Several Parameters When the Number of Observations is
Large," Transactions of the American Mathematical Society, 54, 426-82.
Watson, Peter L. and Richard B. Westin (1975), "Transferability of Disaggregated Mode Choice Models." Regional
Science and Urban Economics, 5 (May). 227-49.
FACE
Readings In The Analysis Of
Survey Data
Robert Ferber, editor
249 pp. 1980
Si 6/member S24/non member
Key pieces of the published literature
concerning applications of multivariate and related techniques to survey data, and new. Innovative approaches to the analysis of survey
data are brought together in this book
of previously published articles.
Emphasis is given to recent material
although some of the classics in the
field have also been included. Bibliographies follow each piece, to stimulate the researcher to go further in
examining the various techniques.
Readings In Survey Research
Robert Ferber. editor
604 pp. 1978
$20/member Si3/nonmember
A collection of readings which form
an extension of the special issue on
survey research of the August 1977
issue of Jourtial of Marketing Research. The articles focus on three
aspects of survey research: sampling,
questionnaire preparation and data
collection. An extensive bibliography
is Included.
AMfO
TO ORDER call or write Order
Department. Amertcan Marketing Association. 250 S. Wacker
Drive. Chicago. IL 606O6. (312)
6480536.