Giving stated preference respondents “time to think”: results from four countries
Joseph Cook a
Marc Jeuland b
Brian Maskery c
Dale Whittington d,e
a
Evans School of Public Affairs, University of Washington, USA
Sanford School of Public Policy, Duke University, USA
c
International Vaccine Institute, Seoul, Korea
d
Departments of Environmental Sciences & Engineering, and City & Regional
Planning,University of North Carolina at Chapel Hill, USA
e
Manchester Business School UK
b
Abstract
Previous studies have found that contingent valuation (CV) respondents who are given overnight to
reflect on a CV scenario have 30 - 40% lower average willingness-to-pay (WTP) than respondents who
are interviewed in a single session. This “time to think” (TTT) effect could explain much of the gap
between real and hypothetical WTP observed in experimental studies. Yet giving time to think is still rare
in CV or choice experiment studies. We review the literature on increasing survey respondents’
opportunities to reflect on their answers and synthesize results from parallel TTT studies on private
vaccine demand in four countries. Across all four countries, we find robust and consistent evidence from
both raw data and multivariate models for a TTT effect: giving respondents overnight to think reduced the
probability that a respondent said he or she would buy the hypothetical vaccines. Average respondent
WTP fell approximately 40%, and household WTP (to vaccinate all members) fell 30%. Respondents
with time to think were also more certain of their answers, and a majority said they used the opportunity
to consult with their spouse or family. We conclude with a discussion of why researchers might be
hesitant to adopt the TTT methodology.
DRAFT AUGUST 2010
Giving stated preference respondents “time to think”: results from four countries
I.
Introduction
Stated preference (SP) valuation methods--contingent valuation (CV) surveys and choice
experiments (CE) -- are now widely used to study individual and household preferences in various policy
sectors (environment, transportation, health, social welfare programs) and throughout the world. Stated
preference research from its inception has been subject to the criticism that it suffers from hypothetical
and other biases. The charge is that hypothetical responses do not predict actual behavior, and that
welfare estimates arising from these studies overestimate true willingness-to-pay. There have now been
over two decades of careful empirical work that has improved the methodology (more so for contingent
valuation than for choice experiment), including the well-known NOAA panel guidelines, the inclusion of
“cheap talk” scripts, and the recent attention to the incentive compatibility of SP scenarios (Carson and
Groves 2007)1. At virtually every turn, most SP researchers tend to make research choices that will
provide the most conservative estimates of willingness-to-pay (WTP) in order to guard against charges of
inflated welfare estimates (for policy analysis, benefit-cost analysis, etc).
Hypothetical bias probably remains, however (Harrison 2006). The rise of the behavioralists in
economics and the increasing visibility of field studies may soon focus even more criticism on this bias in
existing stated preference studies. We discuss one SP design feature – “time to think” – that has not yet
been widely accepted, yet could reduce much of the gap observed in the real vs. hypothetical field studies.
Many real-world situations, either in the context of voting for referenda or private purchasing decisions,
potentially involve some degree of deliberation. Many consumers and voters will want to think carefully
about the program or referendum with regards to their budget or their annual tax bill before reaching their
decision. They may want to confer with partners, family members or friends. They may seek out
alternative sources of information. Other people may do none of these things. Yet a dominant feature of
nearly all existing in-person stated preference interviews is that respondents are given the scenario and are
asked to quickly make their decisions during a single interview. [This is not true of mail surveys, of
course, although most SP researchers would probably agree that in-person interviews remain the gold
standard for high-quality SP studies because of the ability of a well-trained interviewer to control the
information and flow of the interview]. Beginning in the 1990’s, a handful of split-sample studies gave
some respondents overnight to think about the scenario before reaching a decision. The studies found
clear evidence that respondents who had a chance to deliberate were less likely to say yes to the
1
There is also evidence that contingent valuation studies do in fact predict actual behavior with a reasonable degree
of accuracy (Griffin et al. 1995).
hypothetical scenario, lowering average willingness-to-pay in the sample population. Some respondents
who were initially given no time to think were then given the opportunity to revise their answers
overnight. In many cases, they revised their answers downwards.
Despite this, very few SP studies have incorporated a time-to-think treatment to minimize
hypothetical bias. Researchers may be wary of the logistical and financial difficulties of implementing
TTT (because it requires two interviews). Some may feel that giving time to think does not in fact
replicate real-world situations since many or most voters (in the public good context) may pay little
attention to an issue until they enter the voting booth. Another valid theoretical objection is that giving
respondents time to think introduces the possibility of strategic behavior, especially if survey participants
have opportunities to discuss their answers with each other.
We synthesize results from four parallel TTT studies conducted in India, Mozambique, Vietnam
and Bangladesh carried out from 2002 to 2006. The studies estimated private demand for improved,
“next generation” cholera and typhoid vaccines that are unavailable in these countries. Although the
research designs differed somewhat, all four studies used the same general split-sample approach to
examine the effect of time to think on both contingent valuation (India, Bangladesh, Mozambique) and
choice experiment (Vietnam) responses. Although the full results from these studies have recently
appeared in the literature (Cook et al. 2007, Lucas et al. 2007, Whittington et al. 2008 and Islam et al.
2008), the papers were not focused on the examination of the effect of TTT (with the exception of Cook
et al 2007). Furthermore, because of their public health and development focus, they may not have come
to the attention of many SP practitioners, many of whom are environmental economists. We find
consistent and robust results that a time-to-think treatment lowers average WTP on the order of 30-40%
and increases respondents’ certainty in their answers.
The next section reviews the literature on giving respondents time to think, including related
research on test-retest studies, behavioral economics, group elicitation approaches, and two recent studies
on attention and advanced awareness in the context of choice experiments. The third section describes
the research design and the four sites where we gave respondents time to think about whether they would
buy a hypothetical cholera or typhoid vaccine for themselves or their household members. The fourth
section describes our main time-to-think results. The concluding section offers reflections on the
advantages and disadvantages of giving stated preference respondents time to think in stated preference
surveys.
II.
Literature
Since differences in preferences between respondents with and without time to think could reflect
a lack of stability of preferences, we begin with a brief review of the large number of “test-retest” studies
conducted since the mid-1980’s. Test-retest studies measure the reliability of CV by measuring
willingness to pay (WTP) at two or more points in time – either from a panel of the same respondents or
from two independent samples drawn from the same population. The earliest of these studies were
panels, administering the exact same survey to the same respondents separated by a gap of 1 month
(Jones-Lee et al. 1985) or 2 weeks (Kealy et al. 1988, Kealy et al. 1990) Both studies found no statistical
evidence for differences in responses. Because of concerns that respondents simply anchored on their
previous answers, subsequent studies lengthened the interval between interviews to several months or
years. Loomis (1989,1990) surveyed the same respondents about water quality improvements in Mono
Lake 9 months after the initial survey and found that initial and follow-up WTP were significantly
correlated. In a more recent example, Brouwer et al (2008) found that 80 percent of Bangladeshi
respondents who were contacted six months after the original in-person WTP survey said they would
answer the WTP in the same way. Four different “test-retest” studies were conducted with independent
samples from a single population at two time periods (Reiling et al. 1990,Teisl et al. 1995,Carson et al.
1997,Whitehead and Hoban 1999). All four studies found that WTP was stable over the time periods
tested, ranging from several months to several years2. Many of the test-retest studies, as well as other
early contingent valuation studies, were mail surveys, so that respondents could answer at their leisure
and in effect had time to think. Among test-retest studies, only Carson et al (1997), Jones-Lee et al
(1985), and Brouwer et al (2008) have used in-person interviews.
A series of contingent valuation studies in the early 1990’s first examined the effect of giving
stated preference respondents more “time to think” about their answers to valuation questions.
Whittington et al (1992) examined WTP for public standpipes and private water connections in Anambra
State (Nigeria). Half of respondents in the study heard the CV scenario during the first visit from
interviewers but were given overnight to think about it before responding. The other half responded
during the first interview; these respondents were subsequently given the opportunity to revise their
answers overnight. Elicitation was a double-bounded (bidding game) question, analyzed using four
different approaches. The researchers found that giving respondents time to think decreased bids,
program ratings and likelihood of connection for both public taps and private connections; the results
were robust across the four modeling approaches. Average WTP fell by approximately 37% for public
taps and 32% for private taps. Informal group interviews conducted after the interview uncovered no
2
One exception is Whitehead and Hoban (1999), who found that WTP for a generalized scenario of environmental
protection was lower at t1. They also observed, however, that the percent of the underlying population with
favorable views toward government as well as the percent who ranked environmental concerns highly had decreased
over the same period.
evidence that the extra time induced people to answer strategically. Similarly, Lauria et al (1999) found
that giving respondents time to think reduced bids for improved sanitation services in the Philippines.
They found evidence of a starting point bias in the double-bounded referendum questions (higher starting
bids increased WTP), but found that giving time to think erased that bias. In contrast, Whittington et al
(1993) did not find that time to think had a robust effect on bids for sanitation services in Ghana. Time to
think only reduced WTP for sewer connections within the subset of respondents with water connections.
There has been little additional empirical investigation on the subject since the 1990’s3.
Svedsater (2007) examined hypothetical donations to a environmental program among 111 students in
London. He used a multiple-bounded discrete choice format with explicit levels of uncertainty (i.e. “I am
90% sure I would pay”, Welsh and Poe 1998), and gave one third of students one week to consider their
responses. Svedsater found that TTT reduced respondents’ uncertainty lowered WTP. No split sample
time-to-think studies have been conducted in the general population of an industrialized country,
however, though there have been three recent and parallel lines of research that echo some of the same
findings.
The first line is in experimental economics. Baik et al (1999) explored ways to narrow the gap
between experimentally observed strategies and theoretically-optimal strategies (from a game theoretic
3
Though not a split-sample time-to-think experiment, Holmes et al (1998) had tourists in Brazil complete a conjoint
survey on their WTP for Brazilian forest protection using a laptop. The laptop showed two alternatives with varying
attributes (including entrance fees and lodging costs) and had respondents mark a strength of preference scale
varying from 1 (strongly prefer Alternative A) to 5 (indifferent) to 9 (strongly prefer Alternative B). Because the
computer recorded the time it took for respondents to complete the survey, they included this response time variable
in statistical models as an interaction with all utility parameters. In this design, it is clear that time spent thinking
about the choice is confounded with the degree to which the respondent took the task seriously and devoted time to
answering questions thoughtfully (in fact, the authors call the variable “respondent involvement”). Still, the study
provides some interesting empirical evidence. First, respondents who took longer to respond had stronger
preferences (i.e. fewer “indifferent” or “weakly prefer” responses) than those who took less time. The interactions
between response time and other utility parameters were nearly all significant, though the authors made no attempt
to explain them. Finally, Krinsky-Robb draws on the mean and standard deviations of the estimated utility
parameters were used to calculate WTP as a function of response time, which revealed that WTP fluctuated wildly
for response times below 15 minutes, but began stabilizing and decreasing for response times greater than 15
minutes.
standpoint) in endogenous timing games4. One of their split sample treatments examined the effects of
giving subjects two days to work on their strategy versus 20 minutes. The authors found that giving
respondents more time to think narrowed the gap between observed strategies and the subgame perfect
strategy. In a multivariate model, it was the only statistically significant predictor of whether subjects
used subgame perfect strategies, increasing the probability by 21%. Cherry and Kroll (2003) used
experimental subjects to measure the degree of strategic voting in open primary elections. Because they
felt the task was complex and repeating tasks would lead to learning effects not seen in real-world voting
situations, the authors gave all respondents two days with the primary ballots to consider their responses.
Similarly, many field experiments, which seek to model real behavior in a semi-controlled experiment,
give participants time to think by design (see List and Lucking-Reiley 2002 for one example from a
charitable giving experiment).
A second parallel line of research is the use of groups to elicit WTP. Alternately called the
“market stall” (Macmillan et al. 2002, Lienhoop and Macmillan 2002) or “citizen jury” approach
(Alvarez-Farizo and Hanley 2006), these surveys are typically split sample experiments where half of
respondents complete a CV or choice experiment study in one interview5. The other half of respondents
meet in groups to discuss the scenarios and ask clarifying questions of a moderator. At the end of the
hour-long session they answer a valuation question individually and confidentially. They then attend
another group session one week later after they have had a chance to discuss with family, seek out more
information, consider budget constraints, etc., and answer the same valuation question. MacMillan et al
(2002) found that respondents who were interviewed individually without time to think had mean WTP 2
4
Endogenous timing games involve players that are unevenly matched – there is a “favorite” who has greater than
50% chance of winning and an “underdog”. In such games the subgame perfect equilibrium is for the underdog to
commit to an effort level before the favorite. This allows the favorite to respond efficiently to the underdog’s nowrevealed lower strength. As cited in Baik et al, these games have been applied to defense, environmental disputes,
rent seeking and even track-and-field data from the 1992 Olympics. Subjects in Baik et al played an abstract game
with real payoffs determined by how their strategies fared in a “tournament” with other players’ strategies.
5
Another similar line of studies examined the difference in the percentage of respondents who would agree to
connect to improved water and sanitation services when the elicitation approach was an individual interview versus
a community meeting (Davis and Whittington 1998, Whittington et al. 2000, Whittington et al. 1998). Unlike the
“market stall” approach, however, these studies did not have a second follow-up community meeting to assess
whether preferences had changed, and the authors were unable to match individuals’ socioeconomic data with their
responses in community meetings (i.e. voting was by secret ballot). Furthermore, unlike the other studies mentioned
above, participants in the group meetings were not drawn from a random sample, but were voluntary participants
who had heard about the meetings. Participants in the 1-2 hour long meetings had more time (and probably more
information) to consider their responses than individually-interviewed respondents, yet the authors found that their
responses were roughly the same (Davis and Whittington 1998). They also found that respondents who participated
in the group meetings expressed more uncertainty about their answers (Whittington et al. 2000).
- 4 times higher than those who participated in this “market stall” group approach. Also, 37% of group
respondents changed their answers during the intervening week. These respondents were less certain of
their responses than those interviewed individually. Alvarez-Farizo and Hanley (2006) also found that
WTP fell when respondents answered an identical set of choice experiment tasks as a group (the group
respondents in that study were actually a subset of the individually-interviewed respondents, potentially
introducing selection problems).
The third line of research is the use of a “drop-off” protocol (Subade 2007). In a study valuing
reef protection in the Philippines, Subade randomly assigned half of respondents to complete a standard
in-person CV interview. Interviewers made personal contact with the other half of respondents, but left
the survey with the respondent to complete. They made an appointment to retrieve the completed
questionnaire and answer any questions. The author found that WTP among the sample who completed
the drop-off protocol was approximately 50% lower than WTP estimates from the sample respondents
who completed the standard in-person interview. One drawback of this method is that, like a mail survey,
it requires that respondents be literate, which may be a problem in some populations in developing
countries. Like mail surveys, the drop-off protocol also does not carefully control the flow of the
information in the questionnaire, allowing respondents to jump ahead in the survey instrument in
unobserved ways. Nevertheless, the design shares the common feature of TTT designs in that
respondents have overnight to consider the responses and discuss with family. The dropoff protocol has
been successfully implemented elsewhere in Asia (Labao et al. 2008,Nabangchang 2008).
Two new working papers in the choice experiment literature are also relevant. Bateman et al
(2008) examine the effect of “advanced awareness” of the tasks in a choice experiment. The authors
examine demand for a pure public good (drinking water quality), and split their sample by showing a
subset of respondents all of the attributes, their levels, and all of the choice tasks before they were actually
asked to make their choices on each task. The remainder of the sample completed the choice tasks in the
typical sequential fashion with no advanced awareness. Note, however, that all interviews were still done
in one session, so respondents did not have the opportunity to seek out more information or discuss their
choice with family members. The researchers were primarily interested in testing the incentive
compatibility of choice experiments compared to the standard single binary-choice, referendum question
(i.e. CV). In particular, they were concerned with ordering effects and the possibility that advanced
awareness would induce strategic behavior. Interestingly, they find that advanced awareness respondents
were more likely to agree to the highest bids for both a “large” and “small” water quality improvement,
though differences in acceptance rates were mixed at lower prices. Unlike the comparison subsample,
they found that advanced awareness respondents failed a scope test; mean (and median) WTP for a small
improvement in the public good was not statistically different from WTP for the large improvement.
Advanced awareness also did not seem to reduce the noise in responses (i.e. it had no significant effect on
the error variance of the utility function). They conclude that respondents who are shown all attribute
levels and tasks in advance seem to display a “cost-averaging” decision-making framework where they
assume that the true cost of providing the public good is the average of the costs shown in all tasks. This
type of behavior would not be incentive-compatible.
Cameron and DeShazo (2010) develop a theoretical model for stated preference and revealed
preference data that acknowledges that respondents allocate scarce cognitive resources and time to
attributes and choice tasks in a rational way. They argue that economists may incorrectly infer a low
marginal utility for an attribute that a respondent cares about but chose to pay little attention to because of
time constraints or because of small variations in that attribute’s levels. SP researchers, who often go out
of their way to ensure the salience of the cost variable to respondents, may in fact be directing an
artificially high level of attention to this attribute, lowering the estimated marginal utility of income and
artificially inflating estimated WTP. They hypothesize that respondents compare the time and cognitive
costs of allocating more attention to the marginal attribute with the costs of making a suboptimal choice
due to lack of attention. They then apply an empirical model to a large choice experiment examining
WTP for changes in health risk, and find that respondents do appear to differentially allocate attention
across attributes. In our context, their model suggests that giving respondents time to think about choice
tasks may relax the attention – or “time budget constraint”– leading to a different pattern of attention
allocation across tasks than the single-interview format.
III.
Study Sites and Research Design
As part of the Diseases of the Most Impoverished (DOMI) program, researchers from the
University of North Carolina at Chapel Hill and host-country partners conducted a series of studies on
private demand for cholera and typhoid vaccines in six countries from 2002 to 2006.6 Donors and
policymakers were interested in understanding household demand for “next generation” vaccines which
had been the target of scientific research to improve their efficacy and lower their cost. Although some
respondents had experience with older, less effective versions of these vaccines, the improved “next
generation” cholera or typhoid vaccines were generally unavailable in these countries except in limited
trials designed to evaluate the vaccines’ protective effectiveness, during which they were distributed free
of charge (Jeuland et al. 2009c). Because of this unavailability, we relied on nonmarket valuation
6
The DOMI program is administered by the International Vaccine Institute (IVI) with support from the Bill and
Melinda Gates Foundation.
techniques to measure household demand (Canh et al. 2006,Lucas et al. 2007,Islam et al. 2008,Kim et al.
2008,Whittington et al. 2008). This information was then used to evaluate the economic attractiveness of
investments in these vaccines using both benefit-cost and cost-effectiveness techniques (Cook et al.
2008,Cook et al. 2009b,Jeuland et al. 2009a,Jeuland et al. 2009b). It was also used to examine crosssubsidy schemes (where adults sales cross-subsidize free vaccines for children (Lauria et al. 2009) and
optimal vaccine subsidies in the presence of a herd immunity externality (Cook et al. 2009a). We focus
here on the subset of these studies that provided stated preference respondents time to think about their
answers. These were conducted in Hue (Vietnam), Kolkata (India), Beira (Mozambique) and Matlab
(Bangladesh).
Although the study team had to tailor surveys to local conditions and attitudes (after careful focus
groups and pretesting), the structure and content of the questionnaires in all four countries were similar.
We begin by describing the research design in Kolkata (India), and then discuss how the design in other
sites differed.
India
Stated preference surveys were conducted in 2004 in two areas of Kolkata (formerly Calcutta): a
densely-crowded low income slum (Tiljala), and a neighborhood with more diverse incomes and living
conditions (Beliaghata)7. The split-sample TTT experiment was conducted only in Beliaghata (n=559).
As in all the field sites, interviews were done in person with a team of well-trained local enumerators.
Half of contingent valuation respondents completed the interview in one session (i.e. “no time to think”).
After enumerators described the hypothetical vaccine (respondents were asked about cholera or typhoid
vaccines, but not both) and delivered a standard “cheap talk” script, respondents were asked whether they
would buy it for themselves and/or their household members if the two doses of the vaccine regimen were
available at a local clinic or pharmacy at one of four randomly selected prices. “Time-to-think”
respondents were presented with the same information about the hypothetical vaccine but were not
immediately given the price. Instead, interviewers said the following:
7
More details on the Kolkata field site, the CV design and results can be found in Whittington et al. 2008.
Similarly, details on the Bangladesh, Mozambique and Vietnam studies can be found in Islam et al. 2008, Lucas et
al. 2007, and Cook et al. 2007.
“We are almost at the end of our first interview, and I want to thank you very much for your
time. I would like to return again tomorrow to ask you more questions. I will ask you whether
you would want to buy this vaccine for yourself as well as for other members of your
household if it were sold at a certain price. I would encourage you to think overnight about
how much this new vaccine is worth to you, and the range of prices you might be willing to
pay for this vaccine for yourself and for your household members. You may also want to
discuss these decisions with other members of your household.”
The interviewer also left a “reminder” card with the respondents that summarized information
which had already been discussed. For respondents who were asked about a cholera vaccine, it read:
“In thinking about whether you might want to purchase this vaccine, please keep in mind:
•
Yours and your family’s income and economic status
•
Your risk of getting cholera;
•
We still have other ways to treat cholera such as oral dehydration solution; and
•
The vaccine is 50% effective in preventing cholera for 3 years.”
During the second interview, the interviewer asked the exact same set of contingent valuation
questions as for no time-to-think respondents, including a question which asked respondents how certain
they were of their response. The interviewer also asked several time-to-think debriefing questions,
including how long respondents spend thinking about the decision, who they discussed the decision with,
and other sources of information they consulted. Both subsamples also completed a sliding scale
payment exercise using the analogy of a traffic light where “green” prices were ones that a respondent
was completely sure she would pay, “red” prices were ones that a respondent was completely sure she
would not pay, and “yellow” prices were ones over which the respondent was uncertain (see Figure 1 and
Whittington et al 2008). Table 1 summarizes the research design in Kolkata as well as the three other
sites described below.
Bangladesh
Fieldwork was conducted in Matlab, a rural area southeast of Dhaka, in the summer of 2005. The
contingent valuation survey focused only on cholera vaccines. The time-to-think research design is
nearly identical to the one used in Kolkata. The TTT subsample was also not given a specific price to
consider overnight, but rather asked to think about the value of the vaccine to them, and to discuss the
decision with other family members. The stoplight exercise was only completed for the TTT subsample.
The final sample size was 582 respondents split among six prices.
Mozambique
This study, conducted in the coastal city of Beira in the summer of 2005, also focused on cholera
vaccines and used contingent valuation. As in the other sites, half of Beira respondents completed the
survey in one interview (no time to think), though here the format of the time-to-think experiment was
somewhat different. Unlike in India and Bangladesh, the time-to-think respondents were all given one of
five specific prices during the first interview and were told to think how many household members they
would purchase for at that price. During the second interview, respondents reported how many vaccines
they would buy for the household members. Unlike the other studies, however, the respondent did not
answer a single-price, binary choice question about their demand for themselves. Instead, respondents
completed the same stoplight exercise used in Kolkata to elicit willingness to pay for vaccines for
themselves. Furthermore, half of the time-to-think respondents completed this payment card exercise
during the first interview, were given the vaccine price, and then given overnight to think. The second
subsample of time-to-think respondents completed the payment card exercise in the second interview,
after they answered the household demand question. In addition, the starting point on the payment card
exercise (i.e. whether one starts at a high price first and moves down, or starts at the lowest price first and
moves up) was randomized. A figure summarizing the research design in Beira is provided in the
supplementary appendix (Figure A1, reprinted from Lucas et al 2007); “TL” refers to the sliding scale
“traffic light” exercise. The final sample size was 991 households.
Vietnam
Fieldwork was conducted in Hue in the summers of 2002 (contingent valuation for typhoid) and
2003 (contingent valuation for cholera and choice experiment for both vaccines). All contingent
valuation interviews were conducted during one interview (i.e. no time to think); the time-to-think split
sample experiment was only conducted for choice experiment respondents. Choice experiment
respondents were asked to complete a series of six choice tasks. Each task required the respondent to
compare a hypothetical cholera vaccine and a hypothetical typhoid vaccine. The attributes of the vaccines
were their effectiveness, its duration, and its price. Respondents could choose either vaccine or neither.
Half of these choice experiment respondents were given the set of six choice tasks (in the form of
laminated cards) to examine and mark overnight (Cook et al. 2007). The other half completed the cards
in the first interview, but were given the opportunity to revise their answers overnight if they wished,
following the approach of Whittington et al (1992) in Nigeria. The final sample size was 400 households.
IV.
Results
Nonparametric results
1. Respondents’ demand for vaccines for themselves
We first present raw results for respondent demand on the effect of giving time to think. In
Kolkata and Matlab, respondents were less likely to say they would buy a cholera or typhoid vaccine for
themselves if they were given time to think (Table 2). With the exception of respondents who were
asked about cholera vaccines at the highest price in both sites, time to think reduced the percentage of
respondents who said yes by 4 – 25%.
Figure 2 uses raw responses from the sliding scale “traffic light” exercise in Kolkata to report the
percentage of respondents who were completely certain they would buy the vaccine over a range of
prices. Giving time to think clearly reduced this percentage for typhoid vaccines, though somewhat less
so for cholera vaccines. For typhoid vaccines in Kolkata, the midpoint, lower bound and upper bound
were all lower among TTT respondents (Table 3). The midpoint is 40% lower, and all differences are
statistically significant. The length of the uncertainty interval is also shorter for TTT respondents, a result
to which we return later in the paper. Although the direction of the effect is the same for cholera vaccines
in Kolkata, none of the differences are statistically significant. In Beira, the midpoint, lower bound and
upper bound were again all lower among TTT respondents who did the sliding scale exercise after the
single-price question (Table 3). The midpoint is 30% lower. For respondents who did the sliding scale
exercise during the first interview, however, only the differences in midpoint and upper bound are
statistically significant, and the midpoint is 12% lower.
Finally, Figure 3 presents raw responses from the choice experiment survey in Vietnam. Because
we used the exact same choice tasks in the NTTT and TTT subsample, we are able to directly compare
raw responses. Each data point plotted in the graph reflects the percentage of respondents who said they
would buy neither vaccine at the offered price in the task. The solid line denotes responses where the
percentage of respondents who said “neither” on a given card is exactly the same. Points lying above this
line indicate that a higher percentage of TTT respondents bought “neither” than NTTT respondents. A
higher percentage of TTT respondents opted for “neither vaccine in 16 of 18 choice tasks.
Like the results for respondent demand, our raw response data consistently show that demand for
vaccines for household members is lower when respondents are given time to think. In the three sites
where we conducted a split-sample TTT experiment with household demand, the percentage of household
members vaccinated is lower for TTT respondents at nearly every price level (Table 4).
Multivariate models
Although exact modeling specifications differed somewhat between field sites (see Cook et al.
2007,Lucas et al. 2007,Islam et al. 2008,Whittington et al. 2008 for more details), much of our approach
was similar. We analyzed respondent demand at a single vaccine price (Kolkata and Matlab) using a
multivariate probit model and analyzed information from the stoplight sliding scale exercise using OLS.
We used a random-parameters logit model for the choice experiment data in Hue. We modeled
household demand in Kolkata, Beira and Matlab with a negative binomial model, a variant of a Poisson
count model appropriate for situations where data are over-dispersed (conditional mean and variance of
data are not equal).
Our multivariate results confirm the pattern seen in the raw data. Even after controlling for a
number of covariates for vaccine demand, a dummy variable for whether a respondent was given time to
think was negative and statistically significant in all models, including probit, OLS, and mixed logit
models of respondent demand, and negative binomial models of household demand (see the
supplementary appendix for the full multivariate results). Giving respondents time to think lowered the
average number of vaccines a household said it would purchase by -0.6 vaccines at the mean price and 1.1 vaccines at the lowest price in Beira (average marginal effects with other covariates set to their sample
means). The average number of household vaccines fell by -1.3 vaccines at US$0.50 and -1.1 vaccines at
US$1.00 in Matlab.
Willingness-to-pay
We also find that average respondent willingness-to-pay was on average 39% lower in the TTT
subsamples (Table 5). Estimates from both parametric (probit and mixed logit) models and
nonparametric models (Turnbull, Kristrom, and midpoint of uncertainty range) consistently show these
reductions. The smallest difference in WTP is for cholera respondents in Kolkata using the Turnbull
estimator of mean WTP (4% less). This is largely driven by the fact that the percent of cholera
respondents who said they would buy a vaccine for themselves was very similar at the highest price in the
two subsamples (see left panel of Figure 2). The largest difference is in predictions from the mixed logit
model between cholera subsamples in Hue. In fact, median WTP is predicted to be negative for a cholera
vaccine, leading to a very large difference between the NTTT and TTT subsamples. This negative WTP
estimate is driven by a large coefficient on vaccine effectiveness in the choice experiment model
(respondents were shown vaccines with 50, 70, and 99% effectiveness), leading to a negative predicted
WTP for the 50% effective vaccine. Average household willingness to pay drops on average 29% when
respondents are given time to think (Table 5).
Certainty and Preference Errors
We find evidence that providing time to think increases the certainty that respondents have in
their responses. In Kolkata, 86% of TTT respondents felt “very certain” about their answer about buying
a cholera vaccine for themselves, compared to 68% of NTTT respondents. This difference was
statistically significant at the 1% level. There was, however, no statistical difference in certainty among
respondents who were asked about a typhoid vaccine. In Beira, 75% of the “uncertain” respondents were
in the NTTT subsample. Giving TTT also reduced the length of the uncertainty interval of respondent
WTP (elicited from the sliding scale exercise) in some subsamples in Beira and Kolkata (described
above).
The choice experiment study in Hue also found that giving TTT reduced the number of
respondents who made preference errors (violations of stability, monotonicity, or transitivity). Of the 200
respondents in NTTT subsample, 22 (11%) made some type of preference errors. Fourteen of 200 (7%)
respondents in the TTT subsample made a preference error, a statistically significant difference (onetailed t-test, p=0.05). Furthermore, when NTTT respondents were given the opportunity to revise their
answers overnight, their revised responses showed fewer preference errors and were no longer statistically
different than the TTT responses. A multivariate probit model confirmed that giving TTT reduced the
probability of making a preference error (at the 10% level). When TTT was interacted with education,
the model showed that TTT reduced preference errors in respondents with secondary school education by
35% but only 6% for respondents with a primary school education. All TTT variables and interaction
terms were highly significant in this interaction model.
Debriefing questions: time spent thinking and discussion with others
Most respondents who were given the opportunity to think about the valuation question did so. In
Vietnam, Kolkata, Matlab and Beira, only 2%, 11%, 5%, and 5% of TTT respondents respectively
reported spending no time on the task. Figure 4 shows the distribution of time spent thinking across all
four sites, with a significant fraction reporting either 30 minutes or one hour. The average time spent
across all four sites was 37 minutes, with a median of 20 minutes (Table 6).
A strong majority in all four sites reported discussing the decision with their spouse (Table 6).
Across all four sites, 69% of respondents consulted with their spouse. Fewer consulted with someone
outside the household (23% across all sites), and, in Kolkata and Hue, even fewer reported using other
information (we did not ask questions about use of other information in Beira and Matlab).
We used these debriefing questions to explore which elements of the time-to-think protocol might
be influencing responses. First, we replaced minutes spent thinking (imputing a zero for all NTTT
respondents) for the time-to-think dummy variable in the multivariate models of respondent and
household demand. We did not find a statistically significant relationship between minutes spent on the
task and responses to the single-price respondent demand question (multivariate probit)8. There was,
however, a negative and statistically significant relationship with the midpoint of the uncertainty interval
in the OLS models of respondent demand in the stoplight exercise in Kolkata (more minutes spent
thinking lowered the midpoint). We found a similar statistically-significant relationship in household
demand in Kolkata and Matlab, but not in Beira. Next, we replaced the time-to-think dummy with a
dummy variable tracking whether the respondent had discussed the question with a household member.
This also was not a statistically-significant predictor of respondent demand in the single-price models, but
it was for OLS models of the midpoint of the uncertainty interval in Kolkata. It was also negative and
statistically-significant in household models in Kolkata, Beira and Matlab. Third, we explored whether,
among TTT respondents, those who spent longer on the task reported lower respondent or household
demand. We did not find any statistically significant relationships. Similarly, the TTT respondents who
did not discuss the decision with a household member did not have different respondent or household
demand than the TTT respondents who did.
Finally, we estimated a multivariate probit model of the length of the uncertainty interval from
the stoplight exercise in Kolkata and Beira (these results are shown in the supplementary appendix).
Controlling for other factors such as education, income, the offered vaccine price, and others, we still find
that the TTT dummy is negative and statistically significant in both Kolkata and Beira. Time spent
thinking did not have a statistically-significant effect in either site. A dummy variable indicating that the
respondent discussed the decision did have a negative and statistically-significant effect in Beira but not
in Kolkata.
V.
Discussion
In summary, we find that giving respondents overnight to think about their responses has a large
and consistent effect across all four study sites. This was true for both the contingent valuation and stated
choice methodologies. Time-to-think lowered the percentage of respondents who said they would
purchase a cholera or typhoid vaccine for themselves across a range of bid prices, and lowered the
percentage of household members for whom the respondent said they would purchase a vaccine. It
decreased average willingness-to-pay by 30-40% in both respondent and household demand models. It
reduced the likelihood that a choice experiment respondent made a preference error. We find that, given
8
Results discussed in this paragraph are available from the authors on request.
the opportunity, over half of respondents consulted with their spouse about their vaccine purchase
decision, and spent 20 – 30 minutes thinking about the decision.
In most cases, time-to-think also increased the certainty that respondents had in their answers.
Our result stands in contrast to MacMillan et al (2002), who find that respondents who were given time
and who completed a group elicitation approach were less certain than respondents who completed the
survey in one session individually. Group interactions, however, could have been the driver of increased
uncertainty rather than the effect of time to think. Our finding is similar, however, to Svedsater (2007),
who found that TTT reduced respondents’ uncertainty. Comparing real vs. hypothetical (stated)
purchases of a diabetes management program, Blumenschein et al (2008) find that dropping respondents
who said they were less than “absolutely sure” from the analysis reduced nearly all of the gap between
real and hypothetical choices, though a standard cheap talk script had no significant effect. Giving
respondents time-to-think (and reducing their uncertainty) may therefore provide a way to use more of the
preference data collected while still reducing hypothetical bias.
Why have so few TTT studies been conducted? For many researchers who are aware of the TTT
protocol, the first response to this question will be cost. In-person stated preference surveys are already
expensive endeavors. Most of this expense derives from the fact that skilled interviewers must be
employed and respondents compensated for their time in completing the interview. Although there is a
substantial fixed cost to developing the survey instrument and training enumerators, adding a second
follow-up interview will certainly increase survey expenses considerably. It is also probably no
coincidence that all existing time to think studies have been conducted in developing countries where the
cost of implementing a high-quality stated preference survey is much lower because both interviewer
salaries and any household compensation are much lower than in industrialized countries. Another way
of viewing this dilemma: for the same total cost, a researcher might implement a TTT study but be forced
to reduce the sample size. The researcher might then be concerned with the decrease in statistical
precision surrounding estimation of WTP. This should be weighed against the risk (for policy analysis)
that welfare estimates from conventional single-visit survey methodologies might be 30-40% too high.
Another potential drawback with the TTT method pointed out by Carlsson (2010) is that when
respondents have the ability to consult with other household members or friends it is not clear whose
preferences are being recorded. As he notes, however, this is not necessarily undesirable if householdlevel decisions are being studied (as was the case in our vaccine studies), and in fact may be more
representative of how decisions are made in reality.
Conducting a survey with a time-to-think protocol is also logistically complicated. As described
above, the study in Kolkata interviewed households in one of the poorest slums in Kolkata called Tiljala.
Because of safety concerns (our interviewers were not comfortable interviewing households after dark,
and there was a large riot during our fieldwork), we decided to finish the survey work quickly and did not
give respondents time to think. To avoid the potential for strategic bias and for households to receive
prior (and unobserved) information about the survey from their neighbors, we also completed the sites
where respondents were given no time to think first and then completed interviews with the second
subsample. This sort of logistical problem, however, exists for all split-sample studies, and might be less
of a concern for studies in which all respondents are given time to think (though researchers may still
want to ask households if they have heard about the study and what information they have already
learned).
Another reason why time-to-think designs may still be relatively rare may be the fact that inperson interviews for valuation work in industrialized countries are now the exception rather than the
norm. The need for such a design decreases considerably for more commonly used mail surveys, which
allow respondents to reflect longer on their choices. For other survey modes in which response rates and
selection problems are a major concern, such as telephone surveys, it may be even more difficult to
contact respondents multiple times. Researchers would have difficulty in estimating population WTP if a
sizeable number of respondents completed only the first half of the interview. We did not find this type
of sample selection problem in our four TTT studies. Representative Internet panels (like those run by
Knowledge Networks) are increasingly being used in developed countries, and have the flexibility to
allow respondents time to reflect.
Acknowledgements
We would like to thank the International Vaccine Institute and its director John Clemens, along with a
number of the DOMI project staff, including Jacqueline Deen, Lorenz von Seidlein, R. Leon Ochiai, and
Andrew Nyamete. We are also thankful for the assistance of our partners in each of the four countries,
including: India’s NICED (Dr. Susmita Chatterjee, Dr. S.K. Bhattacharya, and Dr. Dipika Sur),
Vietnam’s NIHE (Dr. Do Gia Cahn), Mozambique’s Center for Environmental Hygiene and Medical
Examination (Dr. Marcelino Lucas, Dr. Avertino Barreto, Dr. Francisco Songane, Arnaldo Cumbane,
Arminda Macamuale, and Nivalda Nazaro) and Bangladesh’s ICDDR-B (Dr. Ziaul Islam). All four
country studies were funded with support from the Bill and Melinda Gates Foundation. We also thank
Sonia Akter for thoughtful comments and suggestions on an earlier draft.
References
Alvarez-Farizo, B. and N. Hanley (2006). "Improving the process of valuing non-market benefits:
combining citizens' juries with choice modeling." manuscript.
Baik, K. H., T. L. Cherry, et al. (1999). "Endogenous timing in a gaming tournament." Theory and
Decision 47: 1-21.
Bateman, I. J., R. T. Carson, et al. (2008). Choice set awareness and ordering effects in discrete choice
experiments. European Association of Environmental and Resource Economists, Gothenburg,
Sweden.
Blumenschein, K., G. C. Blomquist, et al. (2008). "Eliciting Willingness to Pay Without Bias: Evidence
from a Field Experiment*." The Economic Journal 118(525): 114-137.
Brouwer, R., S. Akter, et al. (2008). "Economic valuation of flood risk exposure and reduction in a
severely flood prone developing country." Environment and Development Economics 14: 397417.
Cameron, T. A. and J. R. DeShazo (2010). Differential attention to attributes in utility-theoretic choice
models. Manuscript version Feb 7 2010.
Canh, D. G., D. Whittington, et al. (2006). "Household demand for typhoid fever vaccines in Hue,
Vietnam." Health Policy and Planning: 241-255.
Carlsson, F. (2010). "Design of Stated Preference Surveys: Is There More to Learn from Behavioral
Economics?" Environmental and Resource Economics 46(2): 167-177.
Carson, R. and T. Groves (2007). "Incentive and informational properties of preference questions."
Environmental and Resource Economics 37(1): 1573-1502.
Carson, R. T., W. M. Hanemann, et al. (1997). "Temporal reliability of estimates from contingent
valuation." Land Economics 73(2): 151-163.
Cherry, T. L. and S. Kroll (2003). "Crashing the party: an experimental investigation of strategic voting
in primary elections." Public Choice 114: 387-420.
Cook, J., M. Jeuland, et al. (2009a). "Re-visiting socially-optimal vaccine subsidies: an empirical
application in Kolkata, India." Journal of Policy Analysis and Management 28(1): 6-28.
Cook, J., M. Jeuland, et al. (2008). "The cost-effectiveness of typhoid Vi vaccination programs:
Calculations for four urban sites in four Asian countries." Vaccine 26: 6305-6316.
Cook, J., D. Sur, et al. (2009b). "Evaluating investments in typhoid vaccines in two slums in Kolkata,
India." Journal of Health, Population and Nutrition forthcoming.
Cook, J., D. Whittington, et al. (2007). "The reliability of stated preferences for cholera and typhoid
vaccines with time to think in Hue, Vietnam." Economic Inquiry 45(1): 100-114.
Davis, J. and D. Whittington (1998). ""Participatory" research for development projects: A comparison
of the community meeting and household survey techniques." Economic Development and
Cultural Change: 73-93.
Griffin, C. C., J. Briscoe, et al. (1995). "Contingent valuation and actual behavior: predicting connections
to new water systems in the state of Kerala, India." World Bank Economic Review 9(3): 373-395.
Harrison, G. W. (2006). "Experimental evidence on alternative environmental valuation methods."
Environmental and Resource Economics 34: 125-162.
Holmes, T., K. Alger, et al. (1998). "The effect of response time on conjoint analysis estimates of
rainforest protection values." Journal of Forest Economics 4(1): 7-28.
Islam, Z., B. Maskery, et al. (2008). "Private demand for cholera vaccines in rural Matlab, Bangladesh "
Health Policy 85(2): 184-195.
Jeuland, M., J. Clemens, et al. (2009a). "Incorporating herd protection into cost-effectiveness calculations
of new-generation oral cholera vaccines: a multi-site analysis." Value in Health forthcoming.
Jeuland, M., M. Lucas, et al. (2009b). "A cost-benefit analysis of cholera vaccination programs in Beira,
Mozambique." World Bank Economic Review 23(2): 235-267.
Jeuland, M., M. Lucas, et al. (2009c). "Estimating the private benefits of vaccination against cholera in
Beira, Mozambique: A travel cost approach." Journal of Development Economics 91: 310-322.
Jones-Lee, M. W., M. Hammerton, et al. (1985). "The value of safety: Results of a national sample
survey." The Economic Journal 95(377): 49-72.
Kealy, M. J., J. F. Dovidio, et al. (1988). "Accuracy in valuation is a matter of degree." Land Economics
64(2): 158-173.
Kealy, M. J., M. Montgomery, et al. (1990). "Reliability and predictive validity of contingent values:
Does the nature of the good matter?" Journal of Environmental Economics and Management 19:
244-263.
Kim, D., D. G. Canh, et al. (2008). "Private demand for cholera vaccines in Hue, Vietnam." Value in
Health 11(1): 119-128.
Labao, R., H. Francisco, et al. (2008). "Do Colored Photographs Affect Willingness to Pay Responses for
Endangered Species Conservation?" Environmental and Resource Economics 40(2): 251-264.
Lauria, D. T., B. Maskery, et al. (2009). "An optimization model for reducing typhoid cases in developing
countries without increasing public spending " Vaccine 27: 1609-1621.
Lauria, D. T., D. Whittington, et al. (1999). Household demand for improved sanitation services: a case
study of Calamba, Phillipines. Valuing environmental preferences: theory and practice of the
contingent valuation method. K. Willis and I. Bateman. Oxford, UK, Oxford University Press:
540-584.
Lienhoop, N. and D. Macmillan (2002). "Valuing wilderness in Iceland: New approaches in contingent
valuation." Icelandic Journal of Geography 18-19(1): 29-40.
List, J. A. and D. Lucking-Reiley (2002). "The effects of seed money and refunds on charitable giving:
Experimental evidence from a university capital campaign." Journal of Political Economy 110(1):
215-233.
Loomis, J. B. (1989). "Test-retest reliability of the contingent valuation method: a comparison of general
population and visitor responses." American Journal of Agricultural Economics February: 7684.
Loomis, J. B. (1990). "Comparative reliability of the dichotomous choice and open-ended contingent
valuation techniques." Journal of Environmental Economics and Management 18: 78-85.
Lucas, M., M. Jeuland, et al. (2007). "Private demand for cholera vaccines in Beira, Mozambique."
Vaccine 25(14): 2599-2609.
Macmillan, D. C., L. Philip, et al. (2002). "Valuing the non-market benefits of wild goose conservation: a
comparison of interview and group-based approaches." Ecological Economics 43: 49-59.
Nabangchang, O. (2008). "Mobilizing resources for marine turtle conservation in Asia: A cross-country
perspective." ASEAN Economic Bulletin 25(1): 60-69.
Reiling, S. D., K. J. Boyle, et al. (1990). "Temporal reliability of contingent values." Land Economics
66(2): 128-134.
Subade, R. F. (2007). "Mechanisms to capture economic values of marine biodiversity: The case of
Tubbataha Reefs UNESCO World Heritage Site, Philippines." Marine Policy 31(2): 135-142.
Svedsater, H. (2007). "Ambivalent statements in contingent valuation studies: inclusive response formats
and giving respondents time to think." Australian Journal of Agricultural and Resource
Economics 51(1): 91-107.
Teisl, M. F., K. J. Boyle, et al. (1995). "Test-retest reliability of contingent valuation with independent
sample pretest and posttest control groups." American Journal of Agricultural Economics 77(613619).
Welsh, M. P. and G. L. Poe (1998). "Elicitation Effects in Contingent Valuation: Comparisons to a
Multiple Bounded Discrete Choice Approach." Journal of Environmental Economics and
Management 36(2): 170-185.
Whitehead, J. C. and T. J. Hoban (1999). "Testing for temporal reliability in contingent valuation with
time for changes in factors affecting demand." Land Economics 75(3): 453-465.
Whittington, D., J. Davis, et al. (1998). "Implementing a demand-driven approach to community water
supply planning: A case study of Lugazi, Uganda." Water International 23: 134-145.
Whittington, D., J. Davis, et al. (2000). "Designing a "neighborhood" deal for urban sewers: A case study
of Semarang, Indonesia." Journal of Planning Education and Research 19: 297-308.
Whittington, D., D. T. Lauria, et al. (1993). "Household demand for improved sanitation services in
Kumasi, Ghana: A contingent valuation study." Water Resources Research 29(6): 1539-1560.
Whittington, D., V. K. Smith, et al. (1992). "Giving respondents time to think in contingent valuation
studies: a developing country application." Journal of Environmental Economics and
Management 22: 205-225.
Whittington, D., D. Sur, et al. (2008). "Rethinking cholera and typhoid vaccination policies for the poor:
Private demand in Kolkata, India." World Development 37(2): 399-409.
TABLES AND FIGURES
Figure 1. Diagram explaining the stoplight
analogy to Kolkata respondents for the
sliding scale exercise
Table 1. Summary of time-to-think studies on cholera and typhoid vaccines
Hue,
Vietnam
Cholera, Typhoid
Kolkata,
India
Cholera, Typhoid
Matlab,
Bangladesh
Cholera
Beira,
Mozambique
Cholera
Stated
preference
method in split
sample
experiment
Respondent
preference
information in
split sample
experiment
Choice experiment
CV
CV
CV
Choice experiment:
two vaccine
alternatives, three
attributes (price,
efficacy and
duration)
CV, Sliding scale
payment card,
take if free or pay
anything?
CV,
take if free or pay
anything?
Sliding scale
payment card, take
if free or pay
anything?
Household
demand
information in
split sample
experiment
Given price
overnight?
None
CV
CV
CV
Yes, given choice
tasks overnight
No
No
Yes
Subsample sizes
200 NTTT,
200 TTT
140 Cholera
NTTT
140 Cholera TTT
139 Typhoid
NTTT
140 Typhoid TTT
312 NTTT
279 TTT
499 NTTT
497 TTT
(also split on 6
vaccine prices, n =
37 -58 per price
cell)
(also split on
whether sliding
scale was before or
after household
demand, starting
point of sliding
scale, and 5 vaccine
prices, n∼50 per cell
Vaccine
(6 choice tasks
completed per
respondent)
(also split on 4
vaccine prices,
n∼35 per price
cell)
Table 2. Percent of respondents who said they would purchase a vaccine for themselves, by time to
think.
US$0.15 US$0.37 US$0.74 US$1.10 US$4.5 US$9.0
Matlab NTTT
85%
76%
65%
47%
19%
8%
Matlab TTT
82%
65%
40%
33%
9%
11%
Kolkata Cholera NTTT
Kolkata Cholera TTT
US$0.22 US$0.56 US$1.11 US$11.11
89%
68%
63%
18%
82%
60%
49%
18%
Kolkata Typhoid NTTT
Kolkata Typhoid TTT
91%
88%
86%
69%
63%
52%
20%
9%
Figure 2. Respondents in Kolkata who were certain that they would purchase the vaccine for themselves,
by price and time to think (from sliding scale exercise).
Table 3. The effect of TTT on the stoplight sliding scale exercise for respondent demand (US$)a
Subsample and variable
KOLKATAb
Cholera
Midpoint
Upper bound
Lower bound
Length of range
Typhoid
Midpoint
Upper bound
Lower bound
Length of range
BEIRAc
Sliding scale after
single-price (“TL2”)
Midpoint
Upper bound
Lower bound
Length of range
NTTT
TTT
t-statistic
3.39
5.06
1.71
3.35
n= 136
2.93
4.46
1.40
3.07
134
0.85
0.74
0.81
0.43
3.57
5.21
1.92
3.29
n= 135
2.14
3.07
1.20
1.87
131
2.90***
3.14***
1.75*
2.62***
0.93
1.36
0.49
0.87
n= 239
0.65
0.97
0.32
0.64
226
4.92***
5.06***
3.90***
4.35***
0.75
1.11
0.39
0.73
238
1.81*
2.20***
0.77
2.65***
Sliding scale before
single-price (“TL1”)c
Midpoint
Upper bound
Lower bound
Length of range
n=
a
0.85
1.29
0.42
0.87
234
Notes: * Indicates significance at the 10% level, ** at the 5% level, *** at the 1% level
b
Excludes out of market respondents as well as 17 respondents who said they might be willing to pay US$111.
c
Excludes respondents without a true upper bound (green at all prices). Excludes respondents without a lower bound (green at
all prices or red at all prices). Excludes respondents with an undefined upper or lower bound.
Figure 3. Percent of respondents, with and without time to think, who chose neither vaccine (18 tasks)
(reprinted from Cook et al 2007)
Table 4. Average percent of household members who would be vaccinated at specified price, by time to
think (TTT) in Kolkata, Matlab and Beira
Matlab NTTT
Matlab TTT
US$0.15 US$0.37 US$0.74 US$1.10 US$4.5 US$9.0
76%
75%
60%
53%
16%
4%
63%
53%
31%
27%
4%
10%
Beira NTTT
Beira TTT
US$0.2 US$0.82 US$1.64 US$2.86 US$4.08
74%
53%
38%
26%
18%
60%
30%
17%
16%
12%
Kolkata Cholera NTTT
Kolkata Cholera TTT
US$0.22 US$0.56 US$1.11 US$11.11
88%
58%
57%
33%
76%
57%
43%
27%
Kolkata Typhoid NTTT
Kolkata Typhoid TTT
85%
77%
79%
61%
64%
45%
37%
10%
Table 5. Respondent and household willingness-to-pay estimates (US$) for cholera and typhoid vaccines
Respondent WTP
Kolkata Cholera NTTT
Kolkata Cholera TTT
% reduction
Kolkata Typhoid NTTT
Kolkata Typhoid TTT
% reduction
Beira Cholera NTTT
Beira Cholera TTT
% reduction
Matlab Cholera NTTTd
Matlab Cholera TTT
% reduction
Matlab Cholera NTTTf
Matlab Cholera TTT
% reduction
Hue Cholera NTTTg
Hue Cholera TTT
% reduction
Hue Typhoid NTTT
Hue Typhoid TTT
% reduction
Turnbull
lower-bound
(mean)
2.5
2.4
4%
2.8
1.6
43%
Kristrom
mid-point
(mean)
5.2
4.5
13%
5.5
3.9
29%
Multi-variate probit
(median)
5.1
3
41%
5.2
3.6
31%
b
b
b
b
b
b
1.5
1.2
23%
2.0
1.4
29%
h
h
3.4
2.2
35%
4.8
2.4
51%
h
h
2.3
1.1
53%
2.5
1.1
58%
h
h
h
h
h
h
h
h
Household WTP
Midpoint of
uncertainty range from
sliding scale (mean)
3.39
2.93
14%
3.57
2.14
40%
Mixed logit
(median)
a
a
a
a
0.93c
0.65
30%
e
e
a
a
e
e
a
a
h
h
1.92
-0.09
105%
4.36
1.74
60%
h
h
a
a
Negative binomial
count (mean)
35
27
23%
29
23
21%
13.3
7.7
42%
14.3
10
30%
12.3
9.0
27%
Notes: a No choice experiment study carried out. b Respondent demand in Beira was only elicited through the stoplight sliding scale exercise; there is no respondent binary choice CV data to
c
analyze using the Turnbull or Kristrom estimators or multivariate probit. Estimates shown are for the Beira subsample who did the sliding scale exercise after the single-price household question.
d
For the subsample who did the sliding scale in the first interview the average midpoint was $0.85 in the NTTT subsample and $0.75 in TTT subsample. For the subsample in the government service
f
e
area (see Islam et al 2008 for details) The stoplight sliding scale exercise was only done for the TTT subset in Matlab. For the subsample in the ICDDR,B service area (see Islam et al 2008 for
more details). g The effectiveness and duration of the vaccine offered to respondents in Hue varied. Results here are for the characteristics most similar to the real “next-generation” vaccine: 50%
effective for 3 years for cholera, and 70% effective for 3 years for typhoid. h Although we elicited respondent demand for cholera and typhoid vaccines in Hue (see Canh et al. 2006 and Kim et al.
2008), we conducted the split sample TTT experiment only in the choice experiment study.
0
5
Percent
10
15
20
Figure 4. Amount of time that time-to-think respondents reported spending on the task, all four sites
pooled (n=1449 )
0
50
100
Time spent thinking about tasks (mins)
Note: the rightmost bar collapses all times longer than 150 minutes.
150
Table 6. Debriefing questions for time to think respondents.
Kolkata
Time spent thinking:
Mean (SD)a 28 (37)
Mediana 15
Percent who consulted with a
household member
Percent who consulted with
someone outside the
household
Percent who used other
information
N=
a
Hue
Beira
Matlab
All sites
32(38)
20
52 (52)
30
30 (35)b
20
37 (43)
20
74%
56%
81%
64%
69%
25%
4%
36%
27%
23%
6%
2%
n/a
n/a
3%
272
400
487
279
1438
Drops 11 records where reported time spent thinking was over 5 hours, including one respondent who reported thinking about
the task for 24 hours.
b
This data based on n= 436 for this question in Beira
SUPPLEMENTAL APPENDIX
Figure A1. Time-to-think research design in Beira, Mozambique
(Reproduced from Lucas et al 2007)
Table A1. Multivariate models of respondent and household demand: Kolkata (from Whittington et al 2008) a
Household Demand:
Respondent demand:
Negative Binomial Count
Multivariate probit
Model
Respondent yes/no to
Count of household members
“would buy for
respondent would purchase for
Dependent variable
yourself?”
(including respondent)
Price (Rs)
–4.7e–3(5.4e–4)***
–3.0e–3(4.0e–4)***
Price * Cholera
1.2e–3 (6.8e–4)*
7.8e–4 (5.4e–4)
Time to think
–0.35 (0.13)***
–0.26 (0.06)***
Cholera
–0.29 (0.15)*
–0.12 (0.07)*
Monthly per capita income (000 Rs) ‘
0.17 (0.09)*
0.038 (0.01)***
Male
–0.02 (0.15)
0.04 (0.08)
0.48 (0.22)**
0.38 (0.15)**
Education Low b
Education Mid b
0.82 (0.25)***
0.53 (0.15)***
b
0.54 (0.28)*
0.39 (0.16)**
Education High
Age
–0.01 (0.01)
0.00 (0.01)
Muslim
0.18 (0.76)
–1.15 (0.45)**
No. young children
–0.22 (0.13)*
0.04 (0.08)
No .school-aged children
–0.15 (0.08)*
–0.01 (0.05)
No. adults
0.062 (0.03)*
0.11 (0.02)***
–0.019 (0.01)**
–0.01 (0.01)
Distance to health facility
Likely to get disease – Resp.
–0.05 (0.13)
0.02 (0.08)
Likely to get disease – Child
–0.07 (0.08)
Disease is serious for adults
0.03 (0.13)
0.01 (0.07)
Disease is serious for children
–0.12 (0.08)
0.33 (0.13)**
0.11 (0.06)*
Know person with disease
Never boil drinking water
–0.35 (0.13)***
–0.17 (0.06)***
–0.14 (0.15)
–0.04 (0.08)
HH member had vaccine
Constant
0.85 (0.48)*
0.72 (0.28)***
551
551
N
Pseudo-R2
0.29
0.10
2
158
283
Wald Chi-sq (p < Wald χ )
c
Alpha (std error)
n/a
0.16 (0.05)***
a
* Indicates significance at the 10% level, ** at the 5% level, *** at the 1% level. Excluding nine scenario-rejecters. Robust standard
errors in parenthesis. b Excluded category is no education c Alpha is the overdispersion parameter of the negative binomial model, and
Stata 8.0 empirically tests whether alpha is significantly different from zero. When alpha=0 there is no overdispersion in the data and
the Poisson count model is sufficient. When alpha≠0 then the negative binomial is the appropriate model.
Table A2. Multivariate models of household demand and respondent demand (from stoplight sliding scale): Beira (from Lucas et al 2007)
Table A3. Multivariate (negative binomial) models of household demand: Matlab (from Islam et al 2008)
T-statistic in parentheses; * Indicates significance at the 10% level, ** at the 5% level, *** at the 1% level
Table A4. Multivariate (probit) models of respondent demand: Matlab
Price (Tk)
Time to think
Male
Resident from ICDDRB service area
Age
Education 1-5 years
Education 6-10 years
Education >10 years
Log income per capita
Number of household members
Practice Hinduism
Serious or very serious for children
Serious or very serious for adults
Cholera likely for respondent
Cholera likely for children
Someone in the hh has had cholera
Know someone other than hh member
who has had cholera
Respondent had prior vaccine, satisfied
Respondent had prior vaccine, not
satisfied
Treat water
Constant
Observations
Model 1
-0.00412
-0.34
0.322
-0.085
-0.0209
0.119
0.264
0.296
0.38
0.0947
-0.209
-1.798
583
(0.000501)***
(0.155)**
(0.111)***
-0.188
(0.00715)***
-0.163
-0.216
-0.208
(0.105)***
(0.0379)**
-0.192
(0.812)**
Model 2
-0.00429
-0.42
0.304
-0.134
-0.0208
0.113
0.205
0.283
0.425
0.107
-0.157
-0.289
0.433
0.327
0.099
0.0533
(0.000516)***
(0.158)***
(0.123)**
-0.184
(0.00667)***
-0.162
-0.199
-0.224
(0.101)***
(0.0392)***
-0.175
-0.184
(0.140)***
(0.134)**
-0.151
-0.161
0.0584
0.132
-0.159
-0.0959
-0.785
0.214
-2.309
583
-0.524
-0.135
(0.768)***
Robust standard errors in parentheses; * Indicates significance at the 10% level, ** at the 5% level, *** at the 1% level
Table A6. OLS model of length of uncertainty range in stoplight sliding scale exercise for respondent demand: Kolkataa
Model 1
Price (Rs)
Price * Cholera
Time to think
Cholera
TTT*cholera
Time spend thinking (mins)
1.683
-0.00263
-0.99
0.985
Model 2
(1.49)
(0.00)
(0.432)**
(0.530)*
1.716
-0.00267
-1.574
0.425
1.139
Model 3
(1.49)
(0.00)
(0.610)**
(0.68)
(0.86)
1.591
-0.00255
0.974
-0.00575
Model 4
(1.50)
(0.00)
(0.532)*
1.684
-0.00268
1.025
(1.50)
(0.00)
(0.533)*
(0.01)
Discuss with a household
member? (Yes=1)
-0.488
Monthly per capita income
(000 Rs) ‘
Male
Education Low b
Education Mid b
Education High b
(0.45)
-0.00679 (0.13)
-0.0185 (0.13)
0.00195 (0.13)
0.00479 (0.13)
-0.11 (0.48)
-0.06 (0.48)
-0.0938 (0.49)
-0.112 (0.49)
0.57 (0.77)
0.566 (0.77)
0.443 (0.78)
0.435 (0.77)
1.67 (0.829)**
1.7 (0.829)**
1.547 (0.832)*
1.511 (0.829)*
1.132 (0.93)
1.139 (0.93)
1.013 (0.93)
1.003 (0.93)
Age
-0.00671 (0.03)
-0.00794 (0.03)
-0.00222 (0.03)
-0.00344 (0.03)
Distance to health facility
-0.0586 (0.0324)*
-0.0593 (0.0324)*
-0.052 (0.03)
-0.0542 (0.0324)*
Likely to get disease – Resp.
0.697 (0.45)
0.678 (0.45)
0.711 (0.45)
0.681 (0.45)
Disease is serious for adults
0.216 (0.44)
0.202 (0.44)
0.193 (0.45)
0.19 (0.45)
Know person with disease
-0.176 (0.44)
-0.169 (0.44)
-0.106 (0.44)
-0.13 (0.44)
Muslim
-2.228 (2.95)
-2.13 (2.95)
-2.088 (2.97)
-2.182 (2.97)
Never boil drinking water
-1.172 (0.447)***
-1.165 (0.447)***
-1.172 (0.449)***
-1.182 (0.449)***
HH member had vaccine
-0.181 (0.51)
-0.215 (0.51)
-0.191 (0.52)
-0.183 (0.52)
Constant
3.139 (1.439)**
3.464 (1.458)**
2.593 (1.423)*
2.775 (1.441)*
Observations
536
536
536
536
R-squared
0.06
0.063
0.051
0.052
a
* Indicates significance at the 10% level, ** at the 5% level, *** at the 1% level. Excluding nine scenario-rejecters. Robust standard
errors in parenthesis. b Excluded category is no education Table A7. OLS model of length of uncertainty range in stoplight sliding scale exercise for respondent demand: Beira
Model 1
Price (Mts)
Time to think
Time spent thinking (mins)
0.000905
-0.0996
Model 2
(0.000491)*
(0.0327)***
Model 3
0.000894
(0.000494)*
-2.57E-05
-0.00021
Discuss with a household
member? (Yes=1)
Number of hh members
TL1, start low
TL1, start high
TL2, start high
Urban zone 3
Urban zone 4
Single
Widow
Divor
Male
Muslim
Protestant
Household cholera death
Household cholera
Diarrheal incidence in bairro
Mosquito nets (y/n)
Soap in house
Asset 4 (motor vehicle)
Income quartile 2
Income quartile 3
Income quartile 4
Primary school only
Secondary school
Post-secondary school
Traveled to Esturro
Constant
Observations
R-squared
a
-0.0229
-0.0428
0.0477
0.0873
0.153
0.0311
-0.0177
-0.00844
-0.0436
0.0438
-0.0433
0.00975
-0.088
0.0381
-0.00804
0.00627
-0.00145
0.0696
0.0493
0.0969
0.125
0.143
0.139
-0.0324
0.0582
0.33
899
0.067
(0.00645)***
-0.037
-0.0417
(0.0520)*
(0.0664)**
-0.0513
-0.06
-0.0592
-0.0647
-0.0365
-0.0537
-0.0352
-0.0942
-0.044
-0.00813
-0.0331
-0.0407
-0.0863
-0.0468
(0.0524)*
(0.0695)*
(0.0408)***
(0.0605)**
-0.0943
-0.0417
(0.114)***
-0.0222
-0.0473
0.0463
0.0902
0.154
0.0395
-0.0164
0.000763
-0.0275
0.0464
-0.061
-0.0019
-0.0737
0.0377
-0.0068
0.0199
-0.00091
0.0813
0.0503
0.104
0.134
0.151
0.148
-0.0174
0.067
0.24
899
0.057
(0.00656)***
-0.0375
-0.0416
(0.0523)*
(0.0664)**
-0.0518
-0.0582
-0.0591
-0.0628
-0.0368
-0.0536
-0.0354
-0.0981
-0.0441
-0.00818
-0.0331
-0.0409
-0.0858
-0.0472
(0.0527)**
(0.0703)*
(0.0412)***
(0.0608)**
-0.0939
-0.0415
(0.112)**
0.00092
(0.000492)*
-0.0899
(0.0326)***
-0.0227
-0.0426
0.0463
0.0823
0.148
0.0354
-0.0266
-0.0128
-0.0468
0.0464
-0.045
0.00575
-0.0818
0.0432
-0.0071
0.0134
-0.00108
0.0671
0.0526
0.102
0.135
0.149
0.147
-0.0256
0.0608
0.289
899
0.065
(0.00648)***
-0.0372
-0.0418
-0.052
(0.0661)**
-0.0514
-0.0604
-0.0598
-0.0645
-0.0365
-0.0538
-0.0352
-0.095
-0.044
-0.00811
-0.0332
-0.0409
-0.0865
-0.0469
(0.0525)*
(0.0698)*
(0.0407)***
(0.0603)**
-0.094
-0.0417
(0.111)***
* Indicates significance at the 10% level, ** at the 5% level, *** at the 1% level. Excluding nine scenario-rejecters. Robust standard
errors in parenthesis. b Excluded category is no education