White Paper #1: Stated-­‐Preference Methods An introduction to measuring the priorities and preferences of patients and other stakeholders in medicine DRAFT 09/25/14 C e n t e r f o r H e a l t h S e r v i c e s a n d O u t c o m e R e s e a r c h T h e J o h n s H o p k i n s B l o o m b e r g S c h o o l o f P u b l i c H e a l t h Stated-Preference Methods
Stated-preference methods are tools healthcare researchers use to study the priorities and
preferences of patients and other stakeholders (e.g. caregivers, providers, and healthcare professionals) to inform strategies to improve service quality and patient satisfaction. This white
paper provides an overview of stated-preference methods, their application in medicine, and
how they can guide clinical practice and improve patient experience and satisfaction. The target
audiences of this paper include healthcare providers, policy makers, and all other stakeholders
who provide services to patients. Healthcare researchers interested in applying statedpreference methods in their work may need to refer to other references for more technique details.
Contributors Research Team Members:
John F.P. Bridges, Ph.D.
Albert W. Wu, M.D., M.P.H., F.A.C.P.
Jodi Segal, M.D., M.P.H.
Karen Bandeen-Roche, Ph.D.
Lee R. Bone, M.P.H., B.S.N.
Tanjala Purnell, Ph.D., M.P.H.
Research Assistants:
Elizabeth Cummings, M.H.S.
Ellen Janssen
Mo Zhou, M.H.S., M.P.A.
Diabetes Action Board (DAB) Members:
Daniel R. Longo, Sc.D. (Chair)
Joan K. Bardsley, M.B.A., R.N., C.D.E., F.A.A.D.E.
Barri M. Blauvelt, M.B.A.
*Roger S. Clark, M.B.A.
Sherita Hill Golden, M.D., M.H.S.
*Charlotte Johnson
F. Reed Johnson, Ph.D.
Shannon D. Jones, M.L.S., M.Ed, A.H.I.P.
Marian Sue Kirkman, M.D.
*Auriela Laird
Holly Peay, M.S., C.G.C.
Darius Tandon, Ph.D.
* Local community representatives
For more information, please contact:
John F.P. Bridges, Ph.D.
Associate Professor
Director, MHS in Health Economics
624 N. Broadway, Room 689
Baltimore, MD 21205
Phone: (410) 614-9851
2 Stated-Preference Methods
Email: [email protected]
ii Stated-Preference Methods
Table of Contents I. Understanding Patient and Stakeholder Decisions ............................................................... 1 II. Role of Patients’ Priorities and Preferences in Patient-­‐Centered Outcomes Research ......... 3 III. Measuring Priorities .......................................................................................................... 5 IV. Case Study 1 – Best-­‐Worst Scaling (Object Scaling) ............................................................ 8 V. Measuring Preferences ....................................................................................................... 9 VI. Case Study 2 – Choice-­‐based Conjoint Analysis .................................................................. 8 VII. Benefit-­‐Risk Tradeoffs ...................................................................................................... 9 VIII. The Growing Application in Measuring Patients’ Priorities and Preferences................... 10 IX. Stated-­‐preference Methods Checklist .............................................................................. 11 X. Methodological Gaps........................................................................................................ 12 XI. Preference, Priorities, and Public Policy ........................................................................... 13 Glossary................................................................................................................................ 14 Reference ............................................................................................................................. 15 iii Stated-Preference Methods
I. Understanding Patient and Stakeholder Decisions A patient-centered healthcare system
requires a deeper understanding of how patients and other stakeholders (e.g. caregivers,
providers, and healthcare professionals)
make decisions.
For example, people with chronic
conditions like diabetes are faced with many
choices that affect their health and wellbeing—such as healthy eating, physical activity,
and adherence to prescribed drugs. There
are many clinical, personal, and environmental factors that can influence a patient’s
choices regarding treatment, disease management, and their health. While much is
known about how clinical and environmental
factors contribute to chronic diseases, less is
known about the impact of personal factors
on patients’ decisions.
For
decades,
researchers in economics, psychology, and
marketing have been
studying
how
consumers, companies,
and even governments
make decisions. Some
of this research is
qualitative, using indepth interviews and
focus
groups
to
understand why, how,
and
under
what
circumstances
individuals
make
decisions. And some of
the
research
is
quantitative,
using
numerical data from
large
groups
of
individuals
to
understand
real-life
trends. However, observational data alone
are often difficult to interpret and do not re-
veal the full story. For example, if we observe
a high rate of disease in a community, can we
really tell if this is caused by clinical, personal, or environmental factors?
To simplify the complexities of the real
world, researchers in economics, psychology,
and marketing use experimental methods to
understand how individuals behave in controlled settings. For example, many restaurant chains have test kitchens where they can
prepare different dishes for a group of consumers. This way they can offer consumers
many more options than are available on the
standard menu, as well as new or experimental dishes.
Researchers also use surveys to understand what individuals value and what
choices they would make under various circumstances. Just like in the test kitchen,
these surveys can include many different alternatives and explore new products or situations. By testing decisions under many different possible situations, researchers can understand what context or environmental factors drive an individual’s behavior and can
predict how this may have an impact on realworld decisions.
This
type
of
research
is
not
only
effective
for
business
decisions. Policy makers
also need to understand the
values and choices of the
people they serve. For
example, if city leaders
need to find out how a new
traffic plan may affect
commute times, they may
want to survey commuters’
transportation preferences.
Alternatively, when leaders
are considering a new
environmental policy, they
need to both predict how
the policy may affect
behaviors, and also assess
how citizens value potential
risks and benefits. For
example, a new dam may
protect a community from flooding, but it also
might impact hiking trails, resident fish popu-
1 Stated-Preference Methods
lations, and other animal habitats. Survey research can inform leaders about what people
think about these and other possible outcomes.
2 Stated-Preference Methods
on outcomes that patients care about. In tra
II. Role of Patients’ Priorities and Preferences in Patient-­‐centered Outcomes Research
Patient priorities are how patients
value and rate the importance of multiple
goals. An example is how a patient with diabetes might rate the importance of blood glucose control versus vision problems, amputations, heart attack, stroke, or quality of life,
especially when these objectives compete
with each other and one has to make tradeoffs.
In comparison,
patient preferences
What are are a reflection of the
choices that patients
priorities and make among several
preferences? alternatives based on
the
happiness,
satisfaction, gratification, or enjoyment they
get from each alternative. These preferences
reveal how patients make decisions about
disease management. Researchers usually
narrow down the decision-making process to
a few key factors (called attributes), and then
ask patients to choose or rate treatment alternatives relating to various levels of these
attributes.
For example, patients may be asked
to choose between a drug that reduces heartattack risk by 20 percent and increases
weight by 10 pounds and one that reduces
heart-attack risk by 10 percent but does not
affect weight. The choices patients make regarding different treatments reveal what end
results are most important to them and how
they make tradeoffs between 1 different outcomes.
Impact on Research Patient-centered outcome research
(PCOR) assesses the benefits and harms of
health care interventions in order to better inform decision-making. It takes into consideration patient preferences and needs, focusing
ditional clinical trials, the “primary” and “secondary” outcomes are not often relevant to
patient priorities and preferences. However,
when weighing the benefits and harms of
treatment options, patients want and need
evidence on outcomes that are most important to them, not just those that researchers
think are clinically relevant.
Ideally, patient-centered outcomes research accounts for not only the patients’
characteristics and conditions, but also for
their preferences and the tradeoffs they
would be willing to accept among various
outcomes. Information on patients’ priorities
and preferences is important and useful, and
can help guide and improve research.
Impact on Practice In addition to improving and guiding
patient-centered outcomes research, knowledge about patient priorities and preferences
may also promote better clinical practice and
patient experience2.
A greater understanding of patients’
preferences, and the tradeoffs patients are
willing to make, will allow clinicians to tailor
treatment plans to patients’ goals, influence
the decisions of payers, facilitate the approval
of therapies that are more patient-centered,
inform policy makers about the outcomes that
patients prefer, and direct the design of the
healthcare system in a way that addresses
patients’ major concerns and improves ad-
3 Stated-Preference Methods
herence
to
healthcare
services.
4 Stated-Preference Methods
III. Measuring Priorities Rating is the simplest method to measure
priorities. Using this method, researchers ask
patients to rate a set of outcomes on an ordered, “Likert-type” scale, for example, 1 =
not at all important, 2 = somewhat not important, 3 = neutral, 4 = somewhat important,
and 5 = very important. This approach allows
multiple outcomes to be equally important to
a patient. These scores reflect a patient’s
relative priorities or attitudes regarding the
outcomes (i.e., how patients feel about an
outcome and its importance) rather than preferences (i.e., how patients choose among
various options based on satisfaction gained
from each one)3.
Ranking
is another simple approach to
measuring priorities. Using this method, researchers ask patients to rank a list of outcomes or interventions from the most important to the least important. The order a patient
chooses reflects how they prioritize the outcomes or interventions. Compared to ratings
where multiple outcomes can have equal importance, this method only allows patients to
select one outcome or intervention for each
importance level. It requires that patients prioritize outcomes even when they value two
outcomes equally. Therefore, factors other
than the patient’s true priorities can affect the
ranking.
Self-­‐explicated Method includes both
rating and ranking. In self-explication, researchers ask patients to rate and score a list
of outcomes on a Likert-type scale (using 5
ordered response options from “not at all important” = 1 to “very important” = 5). Patients
are also asked to rank and score outcomes.
For example, if there are 10 outcomes, the
least important outcome is scored 1, the second least important outcome is scored 2, and
the most important is scored 10. An overall
score for each outcome is calculated by tak-
ing the product of the rating and ranking
scores. This allows the outcomes to be prioritized. By combining rating and ranking, the
self-explicated method overcomes some of
the problems associated with using rating or
ranking alone4.
2^K Conjoint Analysis is another way
to measure patients’ priorities over several
outcomes. Researchers provide patients with
a number of questions, each of which contains a choice between two outcome profiles.
Each outcome profile contains a subset of all
the outcomes that researchers are looking to
study.
The patients compare the two profiles
in each question and choose the one having
the higher overall value. By combining and
analyzing the choices made by all respondents, researchers can calculate a score (coefficient) for each outcome and rank the outcomes based on the magnitude of these coefficients. The scores not only reflect the relative value patients place on each outcome,
but also quantify how patients make tradeoffs
between various outcomes.
Best-­‐Worst Scaling (Object Scal-­‐
ing) is increasingly one of the most popular
methods researchers use to measure patient
priorities. In best-worst scaling, researchers
give a patient a number of questions, each of
which contains a subset of outcomes (for example, A, B, C, and D) among all the outcomes that researchers are interested in. The
patient is asked to choose the best and worst
outcomes for each question. If the patient
chose A as the best and D as the worst outcomes, researchers learn that the patient prefers A to B, C, and D; and the patient prefers
B and C to D. As a result, researchers learn
information about five of the six possible
paired comparisons for each question (only
the preference between B and C is unknown).
By pooling the choices patients made for all
questions, researchers can calculate a coefficient for each outcome that reflects patients’
priorities over the entire set of outcomes, as
5 Stated-Preference Methods
well as comparative information between
various outcomes.
The following table summarizes the features of the five methods and illustrates each method
with an example question. We used a scenario where a restaurant is evaluating consumers’ dining experience. The attributes include food, service, atmosphere, location, access to public
transportation, and price.
Method
Question
Format
Example
Scores
Strengths
Weaknesses
Rating
Single
outcome
Rating
scores
(e.g., 1-5
with repeated
values)
• Little burden on respondents
• Limited ability to reflect
respondents’
priorities and
preferences
• Floor and
ceiling effects
Ranking
List of all
outcomes
How important are the following restaurant features when you choose where to
eat? (1 = not at all important; 2 = somewhat not important; 3 = neutral; 4 =
somewhat important; 5 = very important)
•
Food
•
Service
•
Atmosphere
•
Location
•
Access to public transportation
•
Price
Please rank the following restaurant features based on their importance:
•
Food
•
Service
•
Atmosphere
•
Location
•
Access to public transportation
•
Price
Scores
from the
rank (e.g.,
1-5 with
unique
values)
• Better reflects respondents’ priorities
over the
outcomes
Selfexplicated
Method
List of all
outcomes
Rating
scores ×
ranking
scores
(e.g., 1-30
with
unique
values)
• Addresses
the weaknesses
associated with
using either rating
and ranking alone
2^K
Conjoint
Analysis
Pairs of
outcome
profiles
Based on
advanced
statistical
techniques
• Reveals
not only
respondents’ priorities but
also the
tradeoffs
between
outcomes
• Requires
many choice
questions
which increases the
respondents
burden
• More technical in design
BestWorst
Scaling
(Object
Case)
Subsets of
the outcomes
Please first rate (1 = not at all important;
2 = somewhat not important; 3 = neutral;
4 = somewhat important; 5 = very important) and then rank (1-6, 6 = most important) the importance of the following restaurant features:
•
Food
•
Service
•
Atmosphere
•
Location
•
Access to public transportation
•
Price
Please choose your preferred restaurant
in each pair:
•
A restaurant with great food and
great atmosphere
OR
•
A restaurant that has great food, low
prices, and is easy to get to
(Respondents receive a number of similar paired outcome profiles, each with
different combinations of features.)
Please choose the most and least important restaurant feature in each list:
•
Food
•
Service
•
Location
•
Price
(Respondents receive a number of similar lists, each with various combinations
of features.)
• More burden
on respondents
• Affected by
random factors when
two outcomes are
equivalent
• More burden
on respondents
Based on
advanced
statistical
techniques
• Reveals
priorities
and tradeoffs
• Requires
less
choice
questions
• More technical in study
design
6 Stated-Preference Methods
IV. Case Study 1 – Best-­‐Worst Scaling (Object Scaling) Best-worst scaling (object scaling) is
gaining popularity as a powerful measurement tool for patient priorities because it’s
less burdensome on respondents than other
methods that measure priorities and it provides researchers with more information.
Erdem and Rigby (2013) used bestworst scaling (object scaling) to examine individual perceptions of levels of control over a
list of risks and levels of worry about the
risks 5 . They evaluated 20 risks, which included the risks of diseases (diabetes, heart
attack, depression, E. coli, mad cow disease,
bird flu, swine flu, and the health effects of
using mobile phones), accidents (experiencing a fire at home, being run over, being
robbed, and being stuck by lighting), food
hazards (eating foods containing additives,
pesticide residues, or hormones; eating meat
or milk from a cloned animal; eating rice or
cereal that’s genetically modified; getting ill
from Salmonella; and experiencing a food allergy), and climate change.
The study randomly assigned and
surveyed 280 respondents; 142 answered
the degree of control questionnaire and 138
answered the worry questionnaire. Each
questionnaire included eight questions and
each question contained five risks. The study
designed the questionnaires such that all 20
risks were covered in the eight questions and
each risk appeared an equal number of times
among all questions. In each question, the
study asked respondents to choose the risks
for which they felt they had the most and
least control, or the risks that worried them
the most and least, as shown in the figures
on the right. Based on respondents’ choices,
researchers analyzed the overall risk perceptions of all respondents, as well as those of
subgroups, identified by various observable
socio-economic characteristics such as age,
gender, income, and education.
Respondents perceived eating foods
containing additives and experiencing a fire
at home as the most controllable risks and
being struck by lightning and climate change
as the least controllable. Diabetes was the
10th most controllable risk in the list. Respondents perceived the level of control they had
over diabetes to be approximately 3 times as
great as being struck by lightning, and about
1.5 times as great as experiencing a fire at
home. These results did not vary much
among different subgroups (e.g., males versus females).
Respondents were most worried
about experiencing a heart attack or a fire at
home. They ranked diabetes 4th following E.
coli infection. Being stuck by lightning and the
health effects of mobile phones were the risks
that the respondents worried about the least.
The differences in concerns over risks among
subgroups were not significant.
How much control do you have over events?
Please look at the events below and indicate:
• The event you think would have the most control
over (most control over preventing the event happening).
• The event you think would have the least control over
(least control over preventing the event happening).
Most
Least concontrol
trol over
over
Eating food containing pesti☐
☐ cide residues
Getting ill from Salmonella
☐
☐ Becoming depressed
☐
☐ Getting avian flu (bird flu)
☐
☐ Being run over
☐
☐ What are the events that cause most & least
worry for you?
Please look at the events below and indicate:
• The event that worries you most.
• The event that worries you least.
Most
Least
worrying
worrying
Eating food containing pesticide
☐
☐ residues
Getting ill from Salmonella
☐
☐ Becoming depressed
☐
☐ Getting avian flu (bird flu)
☐
☐ Being
run
over
☐
☐ 5
* Above tables are cited from Erdem and Rigby (2013) with
adjustments.
8 Stated-Preference Methods
V. Measuring Preferences Researchers commonly use conjoint
analysis methods to measure preferences. In
conjoint analysis, researchers decompose the
product or service of interest into several attributes known to influence patients’ decisionmaking and then determine levels for each
attribute (researchers base these attributes
and levels on previous research, patient interviews, or general accepted knowledge).
Researchers then create a series of hypothetical product or service profiles based on
these attributes and levels and present them
to patients. How researchers present these
profiles and the questions they ask vary in different conjoint analysis methods, but all these
methods consider multiple attributes jointly.
Researchers not only learn patients’ overall
valuations of a product or service (via a
choice between competing options), but also
understand what parts of the product or service patients value most.
Value-­‐based Conjoint Analysis
presents patients with a list of hypothetical
profiles and asks them to value each profile
on a given range (e.g., 1-100). Patients can
pick any number within the range that reflects
the value of the profile to them. Because profiles vary across attributes and levels, the different values patients assign to different profiles reflect their preferences across the attributes. Using statistical methods, researchers can calculate coefficients for the attributes
in order to estimate their relative values.
Rating-­‐based Conjoint Analysis
also presents patients a list of hypothetical
profiles and asks them to rate each profile on
a given scale (e.g., 1-10). Patients cannot
use decimals (e.g. 5.5) but only whole numbers (e.g. 1, 2, 3, 4, 5) on the scale to reflect
the level of the profiles values. By pooling the
values patients assign to different profiles, researchers can estimate coefficients for the
attributes to reveal patients’ preferences over
the attributes.
Take It or Leave It presents patients a
list of hypothetical profiles and asks them
whether they will or will not choose each
product or service. Compared to the previous
two methods, it imposes fewer burdens on
the respondents. These decisions patients
make regarding different profiles with various
attribute levels allow researchers to estimate
how a change in the levels of an attribute influences whether the patient chooses a product or service.
Choice-­‐based Conjoint Analysis
provides patients with several groups of hypothetical profiles. Each group contains at
least two profiles and the profiles in a group
must differ by at least one attribute. Researchers ask patients to choose the profile
with the highest overall value in each group.
The choices patients make between different
profiles with various attribute levels reveal
their preferences over the attributes and how
they make tradeoffs between attributes.
Best-­‐Worst Scaling (Profile Case)
presents patients a series of hypothetical profiles and asks them to choose the best and
worst attributes within each profile based on
the levels of the attributes. Instead of rating
the overall value of each profile, researcher
ask patients to directly compare the attributes
at various levels, which reveals their preferences for attributes at different levels.
Best-­‐Worst Scaling (Multiple Pro-­‐
file Case) presents patients with groups of
hypothetical profiles (as in choice-based conjoint analysis). However, this method asks
patients to choose the best and worst profiles
within each group. Patients need to evaluate
the profiles overall, making this method more
burdensome than best-worst scaling (profile
case). However, it provides more information
than best-worst scaling (profile case), by including multiple profiles in one choice task. It
also provides more information than the
choice-based conjoint analysis by asking pa-
9 Stated-Preference Methods
tients to choose a worst profile in addition to
the best.
The following table summarizes these
methods and provides an example question
using each method. We used a scenario
where a real estate firm is studying the market demand for high-rise apartment buildings
near an urban university. The attributes (levels) include proximity to campus (5 blocks vs.
15 blocks), security features (24-hour doorman vs. controlled access), laundry options
(in unit vs. in the basement), and cost ($800
vs. $1,000 per month). Assume all other features are the same.
Method
Question
Format
Example of A Single Question
Strengths
Weaknesses
Value-based
Conjoint
Analysis
List of single
profiles
• The values respondents can
choose from are
continuous
• More burden on
respondents
• Floor and ceiling effects
Ratingbased
Conjoint
Analysis
List of single
profiles
• Less burden on
respondents
• Values are discrete
• Floor and ceiling effects
Take It or
Leave It
List of single
profiles
• Minimum burden
on respondents
• Dichotomous
results (yes/no)
generate less
information
Choicebased
Conjoint
Analysis
List of pairs
or groups of
profiles
• Less burden than
value- and ratingbased conjoint
analyses
• Generates more
information than
take it or leave it
• More technical
and more requirements on
study design
Best-Worst
Scaling
(Profile
Case)
List of single
profiles
How much would you value the following 1-bedroom apartment over the
range of 1-100?
•
5 blocks to school
•
24-hour doorman
•
Washer/dryer in unit
•
$1,000/month
How desirable is the following 1bedroom apartment to you on a 1, 2, 3,
4, 5 scale (1 = the least desirable and
5 = the most desirable)?
•
5 blocks to school
•
24-hour doorman
•
Washer/dryer in unit
•
$1,000/month
Would you choose the following 1bedroom apartment if it is offered?
•
5 blocks to school
•
24-hour doorman
•
Washer/dryer in unit
•
$1,000/month
Which 1-bedroom apartment would
you prefer?
•
5 blocks to school, controlled access in building, washer/dryer in
unit, $1,000/month
•
15 blocks to school, 24-hour
doorman, washer/dryer in unit,
$800/month
Please choose the best and worst features in this profile:
•
5 blocks to school
•
Controlled access in building
•
Washer/dryer in unit
•
$1,000/month
• Technical on
study design
Best-Worst
Scaling
(Multiple
Profile Case)
List of pairs
or groups of
profiles
• Requires less
questions than
choice-based conjoint analysis for
same amount of
information and
less burden
• Generates the
maximum amount
of information
Please choose the best and worst
apartments in this profile:
•
5 blocks to school, controlled access in building, washer/dryer in
unit, $1,000/month
•
15 blocks to school, 24-hour
doorman, washer/dryer in unit,
$800/month
•
5 blocks to school, 24-hour doorman, laundry in basement,
$1,000/month
• More burden on
respondents
10 VI. Case Study 2 – Choice-­‐based Conjoint Analysis The most common method for assessing preferences in health care is choicebased conjoint analysis, which is also referred to as discrete choice experiment. Hauber et al. (2009) used this method to explore
treatment preferences for hypothetical therapeutic options for diabetes treatment6. Specifically, the study examined the most important features of oral glucose-lowering medication for patients and how their effectiveness
and side effects influence medical adherence
among patients with Type 2 diabetes in the
United Kingdom and the United States. The
medication features (attributes) they examined included the glycated hemoglobin
(HbA1c), the frequency of mild-to-moderate
hypoglycemia, water retention, weight gain,
mild stomach upset, and medication-related
cardiovascular risk.
The study offered patients several
pairs of hypothetical medication profiles and
asked them to choose the preferred option in
each pair, given the scenario where their current oral medication was no longer effective
and needed to be replaced. Each profile contained the six features with various levels.
Because one of the features was the improvement in HbA1c level, the study randomly
assigned patients to a pre-treatment HbA1c
level of 8.5, 9.5, or 10.5%, where a higher
HbA1c level indicates worse blood glucose
control. The study then described the hypothetical medication as decreasing HbA1c by
0.1, 0.5, 1.0, or 2.0 points if the pre-treatment
HbA1c was 8.5%; by 0.3, 0.8, 2.0, or 3.0
points if the pre-treatment HbA1c was 9.5%;
and by 0.5, 1.0, 3.0, or 4.0 points if the pretreatment HbA1c was 10.5%. The study also
described each profile by the frequency of
hypoglycemic episodes, that is, the frequency
of low blood glucose, per month (none, <1,
1~2, or >2), water retention (yes/no), weight
gain in the first 6 months (none, 2.3, 4.5, or
9.0 kg), mild stomach upset (no stomach
problems, mild nausea and vomiting or diar-
rhea that goes away after 1~2 weeks, or mild
nausea and vomiting or diarrhea that continues as long as patient takes the medicine),
and chance of a heart attack within 1 year (no
additional risk, low, medium, or high additional risk). The figure below gives an example of the questions from the study.
The researchers surveyed 204 patients from the United Kingdom and 203 patients from the United States. All patients had
physician-diagnosed Type 2 diabetes and
were currently taking oral glucose-lowering
medications. Patient preferences did not differ between the two countries. Based on the
choice model and given the choice levels,
glucose control was the most important medication feature overall, followed by medicationrelated cardiovascular risk and weight gain.
Water retention was not important to patients.
Weight gain and cardiovascular risk had significant impacts on predicted medication adherence.
Medication
Feature
Medication A
Medication B
From 8.5 to 8.3*
(poor blood glucose
control)
From 8.5 to 6.5*
(optimal blood
glucose control)
1 to 2
More than 2
Yes
No
None
10 pounds
Mild stomach
upset
Mild nausea and
vomiting or diarrhea
that continues as
long as you take
the medicine
No stomach
problems
Chance of a
heart attack
No additional person (0%) will have
a heart attack
Which medication would
you choose?
I would choose
Medication A
☐
HbA1c
change
Number of
hypoglycemic
events per
month
Water
retention
Weight gain
in first 6
months
10 additional
people out of
1,000 (1.0%) will
have a heart attack
I would choose
Medication B
☐
* HbA1c over 8.0% indicates poor blood glucose control, between 7.0% and 8.0% indicates less than optimal blood glucose control, and under 7.0% indicates optimal blood glucose
control.
** Table cited from Hauber et al. (2009)6 with adjustment. Stated-Preference Methods
VII. Benefit-­‐Risk Tradeoffs In recent years there’s been an increased interest in better understanding patient preferences for the relevant benefits and
harms of medical treatments7,8. As a result,
we’ve seen a rapid growth in patient-centered
literature examining these preferences9,10.
In spite of this growth in interest,
strong evidence is still lacking regarding patient
preferences
for
benefits
and
harms11,12,13. In order to fill this gap, a number
of new models and approaches have
emerged that facilitate quantitative benefitrisk analysis14. They vary from simple graphical techniques 15 , 16 to more comprehensive
approaches that can be beneficial in the selection, organization, summary, and communication of evidence relevant to benefits and
risks analyses17,18. A fundamental element to
many of these quantitative benefit-risk
frameworks is a mechanism to prioritize and
value the actual benefits and risks associated
with an intervention. Researchers are most
commonly using stated-preferences methods
to measuring the perspectives of patients and
other relevant healthcare professionals on
risks and benefits 19,20,21 . The United States
Food and Drug Administration, for example,
has completed a demonstration project using
conjoint analysis to assess patient preferences for emerging medical devices targeting
obesity.
Benefit-risk analyses use both priorities and preferences approaches. Priorities
approaches rank the importance of a list of
risks and benefits based on patients’ or other
healthcare professionals’ perspective. These
techniques can be as simple as combining
the ranking/rating of objects. 22 However, as
this method only ranks outcomes, researchers are increasingly using cardinal score approaches, such as best-worst scaling 23 or
conjoint analysis 24 , to generate scores that
reflect the degree of importance. Researchers also use other multiple criteria decision
analysis methods to prioritize outcomes25,26,
but these methods often focus on the weights
of experts more than patients.
Preference
approaches
explore
tradeoffs across various risks and benefits.
Researchers have applied two broad categories of stated-preference methods to examine
such tradeoffs. Threshold techniques ask patients to consider two interventions—a reference intervention and a novel intervention. By
changing the benefits or risks of the novel intervention, researchers determine when the
patients would consider the two interventions
equivalent 27 , 28 , 29 . We can use such techniques to estimate maximal acceptable risk
when a risk is varied, or minimal acceptable
benefit when a benefit is varied.
Choice-based conjoint analysis (or
discrete-choice experiments), where individuals choose among multiple interventions that
vary across multiple attributes, is the most
common approach to estimating benefit-risk
tradeoffs. Using advanced statistical techniques, this approach generates a choice
model from which we can calculate a parameter estimate (sometimes referred to as
utilities or part-worth utilities) for each attribute level. Researchers usually graph the estimates to demonstrate the preferences and
illustrate the potential tradeoffs across benefits and risks. Several good practices documents 30 and others publications 31 , 32 , 33 have
included these approaches. More recently,
investigators have applied best-worst scaling
methods to examine treatment preferences,
where patients or other healthcare professionals assess a series of treatments, selecting the best and worst aspect of each34.
We can also derive many other comparative statistics from the choice model. For
example, one could estimate the maximum
acceptable risk for incremental benefit, or the
minimum acceptable benefit for a marginal
risk. When survival is included as an attribute,
one can estimate health-year equivalents for
changes in both benefits and risks; and when
cost is included, one can similarly estimate
willingness-to-pay for an incremental benefit
or reduced risk. As part of these estimations,
one can also account for the underlying differences in preferences 35 , and cluster individuals into groups with similar preferences36.
9 Stated-Preference Methods
VIII. The Growing Application in Measuring Patients’ Priorities and Preferences
Conjoint analysis
techniques like ranking,
Measure rating,
and
choice
patients’ designs have the adpreferences vantage of measuring
preferences
for
for various goods and services
attributes of where
consumers’
do
not
intervention choices
necessarily reflect their
real preferences. Healthcare research
increasingly uses these methods to learn patients’ priorities and preferences for
healthcare interventions where patients’
choices are constrained by factors other than
their own preferences. Researchers have applied these techniques successfully to measure the preferences of a wide range of health
interventions. Some examples include:
• Prevention of HIV37 and diabetes38.
• Screening of HIV 39 and colorectal cancer40.
• Treatments for cancer 41 , HIV 42 ,
asthma 43 , diabetes 44 , depression 45 , and
Alzheimer’s disease46.
• Genetic counseling47.
• Weight-loss programs48.
Researchers have also used conjoint
analysis techniques to evaluate patients’
preferences for various health states 49 and
their willingness to take the risks associated
with potentially more effective treatments 50 .
These results are valuable in guiding tailored
treatments based on patients’ preferences. In
addition to improving patient participation in
clinical decision-making, researchers have
also used conjoint analysis to understand the
degree to which various stakeholders value
outcomes differently51.
Contingent valuation methods, such
as willingness-to-pay studies, directly measure individuals’ monetary valuation of an
item.
The
use
of
contingent
valuation
Measure methods in health economics and health
patients’ services research has
monetary grown rapidly in the
values of in-­‐
past few decades. Researchers
have
tervention conducted willingness-topay studies to evaluate
how patients value diagnostic tests in fields
such as oncology 52 , infectious diseases 53 ,
obstetrics and gynecology 54 , neurology 55 ,
musculoskeletal diseases56 , and endocrinology57. In addition to clinical diagnostic tests,
researchers also use contingent valuation to
evaluate other health interventions such as
vaccination58, pharmaceutical interventions59,
and caregiving interventions60.
The most common way researchers
measure patients’ willingness-to-pay is using
discrete-choice questions, as described
above, with one of the attributes being the
price for the intervention. Researchers commonly use all modes of administration, such
as Web-based questionnaires, in-person interviews, telephone surveys, and mail surveys, to measure willingness-to-pay, with
self-administered questionnaires being the
most popular61. In addition to discrete choice
experiments, researchers also use other
measuring techniques, such as payment
cards, bidding games, and open-ended questions, to elicit patients’ willingness-to-pay values61. Some studies use more than one contingent valuation method in the elicitation
process.
10 Stated-Preference Methods
IX. Stated-­‐preference Methods Checklist A consensus report published by the International Society for Pharmacoeconomics and
Outcome Research titled “Good Research Practices for Conjoint Analysis Task Force”30 provides broad guidance on how to conduct good conjoint-analysis research in health care. The report establishes a checklist and recommends that researcher use this checklist to conduct conjoint-analysis research. This checklist consists of 10 linked items each with a key question (for a
total of 10 key questions) associated with each item. The 10 items with their key questions are
displayed below:
1. Research Question Is the research question well defined from the perspective of different audiences and users, and are
conjoint methods appropriate to
answer the research question? 2. Attributes and Levels Is the choice of attributes and levels consistent with the research
question, supported by evidence,
and in line with patient and
stakeholder needs? 3. Construction of Tasks Is the construction of tasks (e.g.,
number of attributes and number of
options in each task) appropriate? 4. Experimental Design Did researchers justify the choice
of experimental design and evaluate the properties of the design? 5. Preference Elicitation Did researchers appropriately
measure preferences based on explanation of the tasks, elicitation
format, and other qualifying questions? 6. Instrument Design Is the data collection instrument
appropriate in collecting respondent information, defining attributes
and levels, and motivating respondents? 7. Data Collection Is the data collection plan (e.g.,
sampling strategy, task administration, and ethical considerations)
appropriate? 8. Statistical Analyses Did researchers examine respondent characteristics and quality of
respondents and conduct model
estimation appropriately? 9. Results and Conclusions Is the study presentation (e.g., research context, data-collection instrument and methods, and study
implications) clear, concise, and
complete? 10. Study Presentation Is the study presentation (e.g., research context, data-collection instrument and methods, and study
implications) clear, concise, and
complete? 11 Stated-Preference Methods
X. Methodological Gaps While we’ve seen an increased use
of stated-preference methods to assess the
priorities and preferences of patients, appropriate guidelines from the Patient Centered Outcomes Research Institute are lacking. At least four methodology gaps limit
broad acceptance of these methods.
First, there’s a lack of research
comparing the innovative methods (e.g.,
best-worst scaling) to traditional approaches
(e.g., rating and ranking). We should exercise caution when using conjoint analysis
methods that have both attributes and levels, such as choice-based conjoint analysis,
because the results obtained from these
methods are conditioned upon the levels
chosen in each study. However, this is not
commonly understood among researchers.
Comparisons between the innovative and
traditional approaches will inform researchers about the comparability of the results
from different methods and guide researchers to choose the appropriate methods.
Second, there’s a lack of research
comparing different strategies for experimental design 62 . Traditional experimental
designs for conjoint analysis have focused
on statistical efficiency (i.e., how to discover
significant statistical results when differences exist), but they can be subject to biases if respondents use rule-of-thumb
strategies (i.e., heuristics) to simplify decision-making or if they don’t tradeoff across
attributes when making choices in the
tasks63. Modern experimental design techniques use Bayesian techniques to maximize respondent efficiency64 (i.e., obtaining
more information from less questions).
While preliminary research has demonstrated that aspects of experimental design
can affect both biases and respondent efficiency 65, we need more studies that directly
compare the results from a statisticallyefficient experimental design and those from
a respondent-efficient experimental design.
Third, there’s a lack of studies that
adequately describe preference differences
among different people (i.e., heterogeneity).
Most applications of stated-preference
methods fail to explore preference heterogeneity, often citing a lack of sufficient sample size (>200 respondents per subgroup66)
to fully address subgroups. In traditional
stratification methods for comparing the
preferences of subgroups, studies can add
patient characteristics into the regression
when estimating preferences 67 . However,
they must estimate separate models for
each subgroup, which limits the number of
subgroups that a study can simultaneously
consider to two or rarely three. Given the
inability to conduct stratification on multiple
indicators simultaneously, studies can
misclassify preference differences. For example, a study may identify differences in
priorities and preferences across levels of
educational attainment, but these differences may in fact relate to levels of income.
An alternative to stratification is segmentation, where studies classify respondents into
groups based on their preferences35. Studies can then use multivariate statistics to
describe differences across the groups and
estimate the difference in preferences
across the subgroups. While current patientcentered outcomes research methods aim
to address heterogeneity of treatment effects, they do not address heterogeneity of
priorities and preferences.
Finally, there’s a lack of evidence
regarding the attitudes of patients and
healthcare professionals who participate in
stated-preference research. Anecdotal evidence from previous research suggests that
they find it both valuable and relevant.
However, we need to conduct more studies
to both qualitatively and quantitatively assess how patients and healthcare professionals feel about the relevance and importance of stated-preference methods and patient-centered outcomes research.
12 Stated-Preference Methods
XI. Preference, Priorities, and Public Policy Patient-centered outcomes research
and comparative effectiveness research aim
to generate new evidence and synthesize
existing evidence to help decision-makers
better assess the relative benefits and risks
of therapy. This requires a subjective analysis of the relative importance of different
outcomes and the acceptable tradeoffs
across outcomes. By incorporating information on patient priorities and preferences
(generated by stated-preference methods)
into this decision-making process, we can
better inform public policies at the population level. This may include assessments as
to which drugs a healthcare plan should include, what recommendations a clinical
guideline should make, and which drugs a
regulatory body should approve for a given
indication.
Recently, some policy institutions
have emphasized the importance of including patient perspectives in medical decisionmaking. Under the 5th Prescription Drug
User Fee Act, the United States Food and
Drug Administration has developed a new
benefit-risk framework for evaluating new
drugs that promises to increase transparency by placing an emphasis on soliciting
patient perspective on the severity of a disease and current treatment options. In
Europe, the 2013 Workshop of Patient’s
Voice in the Evaluation of Medicines discussed ways to involve patients in the benefit-risk assessment of medical products and
made progress in patient involvement in
medical decision-making.
Despite the increased emphasis on
incorporating patient perspectives in decision-making, there’s not sufficient evidence
about patients’ priorities and preferences to
directly inform tradeoffs across outcomes68.
As a result, decision-maker assessments of
such tradeoffs are informal and qualitative69.
To close this evidence gap, researchers are using better methods for assessing the relative balance of benefits and
harms70 and more robust techniques for assessing patient preferences for benefits and
harms7. For example, researchers increasingly use choice-based conjoint analyses to
estimate (in a systematic way) the willingness of patients to accept tradeoffs among
the various dimensions of healthcare interventions21.
However, we need further research
on preferences heterogeneity across patient
subgroups to inform policy36. For example, if
risk preferences differ across patients with
different diagnoses or stages of disease
progression, it may be appropriate to consider different recommendations for different
subgroups. In the case of type 2 diabetes,
the evidence that informs treatment guidelines most often comes from short-term randomized controlled efficacy trials in restricted patient populations. This evidence
may not reflect the reality of most patients
seen in practice, such as the elderly or
those with comorbidities who are typically
excluded from trial71. In addition to necessary changes in therapy and modification of
treatment goals due to commodities, various
patient preferences may also require an adjustment in treatment plans. For example,
research has found that while glucose control was an important therapeutic outcome
to patients, they also valued risk of heart attack, weight gain, stomach upset, and hypoglycemia72. While patients were willing to
pay for interventions to reduce the risk of
diabetes, they were also willing to accept a
higher risk if there were to be fewer restrictions on diet73. We should incorporate such
patient preference information in treatment
guidelines on type 2 diabetes to balance the
relative priorities of different treatment
goals.
A better understanding of patient
priorities, preferences, and heterogeneity
will improve the practice of medicine and
lead to better patient outcomes.
13 Stated-Preference Methods
Glossary Attribute – An attribute of a product or a
service is a characteristic or feature of the
product or service that can influence people’s choice decisions. Attributes for a mobile phone, for example, can include size,
weight, battery life, additional functions, colors, etc.
Coefficient – A coefficient is a number that
is used to multiply a variable or another
number. For example, 3 is a coefficient in
the expression 3x.
Conjoint Analysis – Conjoint analysis is a
quantitative method researchers use to
measure how people value different features that constitute a product or service.
Conjoint analysis techniques include ranking, rating, and choice-based designs.
Contingent Valuation – A contingent
valuation method measures the value a
person assigns to goods or services by directly asking the person, in a survey, how
much he or she is willing to pay for goods or
services or how much he or she is willing to
accept for compensation to give up goods
or services.
Discrete-choice Experiment – Discretechoice experiment, also called choice-based
conjoint analysis, is a quantitative method
researchers use to study people’s preferences. In a discrete-choice experiment, researchers ask respondents to choose
among hypothetical alternative goods or
services. Each of the goods or services is
described by a list of characteristics (or attributes). Based on people’s responses, researchers learn the impact of these attributes and their various levels on people’s
decision-making.
Level – A level in conjoint analysis refers to
a specification of an attribute. If color is one
attribute of a car, levels for this attribute
may include red, white, black, and silver.
Outcome Research – Outcome research
(also outcomes research) examines how
procedures, interventions, treatments, or
other health care practices affect patient
and other healthcare outcomes. It focuses
on the end result of healthcare services and
provides guidance for evidence-based practice.
Patient-centered Outcome Research –
Patient-centered outcome research is outcome research that takes patients’ experiences, preferences, and values into account. It not only focuses on outcomes that
clinicians view as clinically important, but
also examines outcomes that patients
value.
Preference – Preference reveals the process a person takes to make decisions, specifically, how a person makes tradeoffs between different options (e.g., treatments)
that vary across attributes and levels.
Priority – Priorities are related to how a
person orders and values multiple objectives, especially when these objectives are
competing with each other and the person
must make tradeoffs.
Stated-preference Methods – Statedpreference methods rely on answers to
carefully designed survey questions to learn
people’s preference. The answers can be in
the form of monetary amounts, rankings,
ratings, choices, or other indications of preferences, and are usually scaled to a measure of value using preference models. It is
one of the most effective ways to learn people’s preferences, especially those related
to nonmarket goods.
Willingness To Pay – The monetary value
a person would like to pay to obtain an object or to achieve an outcome.
14 Stated-Preference Methods
Reference 1 Bridges J. Stated preference methods in health care evalua-­‐
13 Garrison Jr LP, Towse A, Bresnahan BW. Assessing a struc-­‐
tured, quantitative health outcomes approach to drug risk-­‐
benefit analysis. Health Affairs. 2007;26(3):684-­‐95. 14 Guo JJ, Pandey S, Doyle J, Bian B, Lis Y, Raisch DW. A review tion: An emerging methodological paradigm in health econom-­‐
of quantitative risk–benefit methodologies for assessing drug ics. Applied Health Economics and Health Policy safety and efficacy—report of the ISPOR risk–benefit manage-­‐
2003;2(4):213-­‐24. ment working group. Value Health 2010;13, 657–666. 2 Brazier JE, Dixon S, Ratcliffe J. The role of patient preferences 15 Lynd L, O’Brien BJ. Advances in risk-­‐benefit evaluation us-­‐
in cost-­‐effectiveness analysis: A conflict of values? Pharmaco-­‐
ing probabilistic simulation methods: an application to the economics 2009;27(9):705-­‐12. prophylaxis of deep vein thrombosis. J Clin Epidemiol. 3 Phillips KA, Johnson FR, Maddala T. Measuring what people 2004;57(8): 795–803. value: A comparison of “attitude” and “preference” 16 Levitan, B. A Concise Display of Multiple End Points for surveys. Health Services Research 2002;37(6):1659-­‐79. Benefit–Risk Assessment, Clinical pharmacology & Therapeu-­‐
4 Kahneman D, Knetsch JL, Thaler RH. Anomalies: The en-­‐
tics, 2011;89(1): 56-­‐59. dowment effect, loss aversion, and status quo bias. The Journal 17 Coplan PM, Noel RA, Levitan BS, Ferguson J, Mussen F. De-­‐
of Economic Perspectives 1991;5(1):193-­‐206. velopment of a Framework for Enhancing the Transparency, 5 Erdem S and Rigby D. Investigating heterogeneity in the Reproducibility and Communication of the Benefit–Risk Bal-­‐
characterization of risks using best worst scaling. Risk Analysis ance of Medicines, Clinical Pharmacology & Therapeutics, 2013;33(9):1728-­‐48. 2011; 89(2): 312–315. 6 Hauber A, Mohamed A, Johnson F, Falvey H. Treatment pref-­‐
18 Noel R, Hermann R, Levitan B, Watson DJ, Van Goor K. Ap-­‐
erences and medication adherence of people with type 2 diabe-­‐
plication of the Benefit-­‐Risk Action Team (BRAT) Framework tes using oral glucose-­‐lowering agents. Diabetic Medicine in Pharmaceutical R&D: Results From a Pilot Program Drug 2009;26(4):416-­‐24. Information Journal 2012; 46(6): 736-­‐743 7 Yuan Z, Levitan B, Berlin JA. Benefit‐risk assessment: To 19 Bridges JFP, Kinter ET, Kidane L, Heinzen RR, McCormick C. quantify or not to quantify, that is the question. Pharmacoepi-­‐
Things are looking up since we started listening to patients: demiology and Drug Safety 2011;20(6):653-­‐6. Trends in the application of conjoint analysis in health 1982-­‐
8 Hauber AB, Fairchild AO, Johnson FR. Quantifying benefit-­‐
2007. The Patient: Patient-­‐Centered Outcomes Research risk preferences for medical interventions: an overview of a 2008;1(4):273-­‐82. growing empirical literature. Appl Health Econ Health Policy. 20 Ryan M, Gerard K. Using discrete choice experiments to 2013;11(4):319-­‐29. value health care programmes: Current practice and future 9 Hauber AB, Fairchild AO, Johnson FR. Quantifying benefit-­‐
research reflections. Applied Health Economics and Health Pol-­‐
risk preferences for medical interventions: an overview of a icy. 2003;2(1):55-­‐64. growing empirical literature. Appl Health Econ Health Policy. 21 Marshall D, Bridges JFP, Hauber B, Cameron R, Donnalley L, 2013;11(4):319-­‐29. Fyie K, Johnson FR. Conjoint analysis applications in health: 10 Joy M, Little E, Maruthur N, Purnell T, Bridges J. Patient How are studies being designed and reported? An update on preferences for the treatment of type 2 diabetes: A scoping current practice in the published literature between 2005 and review, Pharmacoeconomics. 2013;31(10): 877-­‐892 2008. The Patient: Patient-­‐Centered Outcomes Research. guidelines incorporate evidence on patient preferences? Med 22 Bridges J, Slawik L, Schmeding A, Reimer J, Naber D, Decis Making 2007;27: E63-­‐64. Kuhnigk O. A test of concordance between patient and psychia-­‐
12 Krahn M, Naglie G. The next step in guideline development. trist valuations of multiple treatment goals for schizophrenia, 11 Chong C, Chen I, Naglie G, Krahn M. Do clinical practice JAMA, 2008;300(4):436-­‐8. 2010;3(4):249-­‐56. Health Expectations. 2013;16(2):164-­‐76. 23 Gallego G, Bridges J, Flynn T, Blauvelt B, Niessen, L. Using Best-­‐Worst Scaling in Horizon Scanning for Hepatocellular Car-­‐
15 Stated-Preference Methods
cinoma Technologies. International Journal of Technology As-­‐
35 Hole AR. Modelling heterogeneity in patients' preferences sessment in Health Care. 2010;28(3):339-­‐46. for the attributes of a general practitioner appointment. Jour-­‐
24 Bridges J, Dong L, Joy S, Gallego G, Blauvelt B, Pawlik T. Pri-­‐
nal of Health Economics 2008;27(4):1078-­‐94. oritizing strategies for comprehensive liver cancer control in 36 Cunningham CE, Deal K, Rimas H, Campbell H, Russell A, Asia: A conjoint analysis. BMC Health Services Research. Henderson J, Matheson A, Melnick B. Using conjoint analysis to 2012;12:376. model the preferences of different patient segments for attrib-­‐
25 Hummel JM, Bridges J, Ijzerman M. Group decision making utes of patient-­‐centered care. The Patient: Patient-­‐Centered with the Analytic Hierarchy Process in Benefit-­‐Risk Assess-­‐
Outcomes Research 2008;1(4):317-­‐30. ment: A Tutorial. The Patient. 2014;7(2): 129-­‐140 37 Bridges J, Selck F, Gray G, et al. Common avoidance and de-­‐
26 Mussen F, Salek S, Walker S. A quantitative approach to terminants of demand for male circumcision in Johannesburg, benefit-­‐risk assessment of medicines – Part 1: the development South Africa. Health Policy Plan 2010; Oct 20. of a new model using multi-­‐criteria decision analysis. Pharma-­‐
38 Johnson FR, Manjunath R, Mansfield CA, et al. High-­‐risk in-­‐
coepidemiology and Drug Safety. 2007;16(Suppl. 1): S2–S15. dividuals’ willingness to pay for diabetes risk-­‐reduction pro-­‐
27 Llewellyn-­‐Thomas HA, Williams JI, Levy L, Naylor CD. Using grams. Diabetes Care 2006;29:1351–56. a tradeoff technique to assess patients’ treatment preferences 39 Phillips KA, Maddala T, Johnson FR. Measuring preferences forbenign prostatic hyperplasia. Medical Decision Making. for health care interventions using conjoint analysis: an appli-­‐
1996;16(3): 262–82. cation to HIV testing. Health Serv Res 2002;37:1681–705. 28 Palda VA, Llewellyn-­‐Thomas HA, Mackenzie RG, Pritchard 40 Marshall DA, Johnson FR, Kulin NA, et al. How do physician KI, Naylor CD. Breast cancer patients’ attitudes about rationing assessments of patient preferences for colorectal cancer postlumpectomy radiation therapy: applicability of tradeoff screening tests differ from actual preferences? Comparison in methods to policy-­‐making. J Clin Oncol. 1997;15(10):3192–
Canada and the United States using a stated-­‐choice survey. 200. Health Econ 2009;18:1420 – 39. 29 Llewellyn-­‐Thomas HA, Paterson JM, Carter JA, Basinski A, 41 Weston A, Fitzgerald P. Discrete choice experiment to de-­‐
Myers MG, Hardacre GD, Dunn EV, D’Agostino RB, Naylor CD. rive willingness to pay for methyl aminolevulinate photody-­‐
Primary prevention drug therapy: can it meet patients’ re-­‐
namic therapy versus simple excision surgery in basal cell car-­‐
quirements for reduced risk? Medical Decision Making. cinoma. Pharmacoeconomics 2004;22:1195–208. 2002;22:326–39. 42 Beusterien KM, Dziekan K, Flood E, et al. Understanding 30 Bridges JFP, Hauber AB, Marshall D, Lloyd A, Prosser LA, patient preferences for HIV medications using adaptive con-­‐
Regier DA, Johnson FR, Mauskopf J. Conjoint analysis applica-­‐
joint analysis: feasibility assessment. Value Health tions in health-­‐-­‐a checklist: A report of the ISPOR good re-­‐
2005;8:453–61. search practices for conjoint analysis task force. Value in 43 King MT, Hall J, Lancsar E, et al. Patient preferences for Health 2011;14(4):403-­‐413. managing asthma: results from a discrete choice experiment. 31 Ryan M, Hughes J. Using conjoint analysis to assess Health Econ 2006; 16:703–17. women's preferences for miscarriage management. Health 44 Hauber AB, Johnson FR, Sauriol L, et al. Risking health to Economics 1997;6(3):261-­‐73. avoid injections: preferences of Canadians with type 2 diabe-­‐
32 Ryan M, Farrar S. Using conjoint analysis to elicit prefer-­‐
tes. Diabetes Care 2005;28:2243–5. ences for health care. BMJ 2000;320(7248):1530-­‐3. 45 Wittink MN, Cary M, TenHave T, et al. Towards patient-­‐
33 Lancsar E, Louviere J. Conducting discrete choice experi-­‐
centered care for depression: conjoint methods to tailor treat-­‐
ments to inform healthcare decision making: A users guide. ment based on preferences. Patient 2010;3:145–57. Pharmacoeconomics 2008;26(8):661-­‐77. 46 Hauber AB, Johnson FR, Mohamed AF, et al. Older Ameri-­‐
34 Peay H, Hollin I, Fischer R, Bridges J. (2014) A community-­‐
cans’ risk-­‐ benefit preferences for modifying the course of Alz-­‐
engaged approach to quantifying caregiver preferences for the heimer’s disease. Alzheimer Dis Assoc Disord 2009;23:23–32. benefits and risks of emerging therapies for Duchenne muscu-­‐
47 Peacock S, Apicella C, Andrews L, et al. A discrete choice lar dystrophy. Clinical Therapeutics, 36(5):624–637. experiment of preferences for genetic counseling among Jew-­‐
16 Stated-Preference Methods
ish women seeking cancer genetics services. Br J Cancer 61 Lin PJ, Cangelosi MJ, Lee DW, et al. Willingness to pay for 2006;95:1448 –53. diagnostic technologies: a review of the contingent valuation 48 Roux L, Ubach C, Donaldson C, et al. Valuing the benefits of literature. Value Health 2013;16:797-­‐805. weight loss programs: an application of the discrete choice ex-­‐
62 Kinter E, Prior TJ, Carswell C, Bridges J. A comparison of periment. Obesity Res Clin Pract 2004;12:1342–51. two experimental design approaches in applying conjoint 49 Mohamed AF, Hauber AB, Johnson FR, et al. Patient prefer-­‐
analysis in patient-­‐centered outcomes research: a randomized ences and linear scoring rules for patient reported outcomes. trial. The Patient: Patient-­‐ Centered Outcomes Research Patient 2010;3,217-­‐ 27. 2012;5(4):279-­‐294. 50 Johnson FR, Özdemir S, Mansfield CA, et al. Crohn’s disease 63 Louviere J, Islam T, Wasi N, et al. Designing discrete choice patients’ benefit-­‐risk preferences: serious adverse event risks experiments: do optimal designs come at a price? Journal of versus treatment efficacy. Gastroenterology 2007;133:769–79. Consumer Research 2008;35:360-­‐75. 51 Shumway M. Preference weights for cost-­‐outcome analyses 64 Rose JM, Bliemer MCJ. Constructing efficient stated choice of schizophrenia treatments: comparison of four stakeholder experimental designs. Transport Reviews 2009;29(5):587-­‐617. groups. Schizophr Bull 2003;29:257–66. 65 Bridges J, Buttorff C, Groothuis-­‐Oudshoorn K. Estimating 52 Neumann PJ, Cohen JT, Hammitt JK, et al. Willingness-­‐to-­‐pay patients' preferences for medical devices: Does the number of for predictive tests with no immediate treatment implications: profiles in choice experiments matter? NBER Working Paper a survey of US residents. Health Econ 2012;21:238–251. No. 17482. National Bureau of Economic Research; 2011. 53 Goswami ND, Hecker E, Holland DP, et al. Feasibility and 66 Orme B. Getting started with conjoint analysis: Strategies willingness-­‐to-­‐pay for integrated community-­‐based tuberculo-­‐
for product design and pricing research. Madison, sis testing. BMC Infect Dis 2011;11:305. Wisconsin: Research Publishers LLC; 2006. 54 Caughey AB, Washington AE, Gildengorin E, et al. Assess-­‐
67 McFadden D. Conditional logit analysis of qualitative choice ment of demand for prenatal diagnostic testing using willing-­‐
behavior. In: P. Zarembka, editor. Frontiers in econometrics. ness to pay. Obstet Gynecol 2004;103:539–545. New York: Academic Press; 1974. 55 Kopits IM, Chen C, Roberts JS, et al. Willingness to pay for 68 Chong C, Chen I, Naglie G, Krahn M. How well do guidelines genetic testing for Alzheimer’s disease: a measure of personal incorporate evidence on patient preferences? Journal of Gen-­‐
utility. Genet Test Mol Biomarkers, 2011;15:871–875. eral Internal Medicine 2009;24(8):977-­‐82. 56 Nayak S, Roberts MS, and Greenspan SL. Osteoporosis 69 Institute of Medicine. Committee on the Assessment of the screening preferences of older adults. J Clin Densitom, US Drug Safety System. The future of drug safety: Promoting 2009;12:279–286. and protecting the health of the public. Washington: National 57 Shih HC, Chou P, Chen SJ, et al. A community-­‐based study of Academies Press; 2007. the willingness to pay associated with screening for diabetic 70 Holden WL, Juhaeri J, Dai W. Benefit-­‐risk analysis: Examples retinopathy among type 2 diabetes in Kinmen, Taiwan. J Epi-­‐
using quantitative methods. Pharmacoepidemiology and Drug demiol 2007;17:186–193. Safety 2003;12(8):693-­‐7. 58 Yeung RY and Smith RD. Can we use contingent valuation to 71 Bennett WL, Maruthur NM, Singh S, Segal JB, Wilson LM, assess the demand for childhood immunization in developing Chatterjee R, Marinopoulos SS, Puhan MA, Ranasinghe P, Block countries: a systematic review of the literature. Appl Health L. Comparative effectiveness and safety of medications for type Econ Health Policy 2005;4:165-­‐73. 2 diabetes: An update including new drugs and 2-­‐drug combi-­‐
59 Sach TH, Smith RD, and Whynes DK. A ‘league table’ of con-­‐
nations. Annals of Internal Medicine 2011;154(9):602-­‐13. tingent valuation results for pharmaceutical interventions: a 72 Hauber A, Mohamed A, Johnson F, Falvey H. Treatment hard pill to swallow? Pharmacoeconomics 2007;25:107-­‐27. preferences and medication adherence of people with type 2 60 Jutkowitz E, Gitlin LN, and Pizzi LT. Evaluating willingness-­‐
diabetes using oral glucose-­‐lowering agents. Diabetic Medicine to-­‐pay threshold for dementia caregiving interventions: appli-­‐
2009;26(4):416-­‐24. cation to the tailored activity program. Value Health 73 Johnson FR, Manjunath R, Mansfield CA, Clayton LJ, Hoerger 2010;13:720-­‐5. TJ, Zhang P. High-­‐risk individuals’ willingness to pay for diabe-­‐
tes risk-­‐reduction programs. Diabetes Care 2006;29(6):1351-­‐6. 17