Advancing stated-preference methods
for measuring the preferences of
patients with type 2 diabetes
1
WELCOME
John F. P. Bridges, Ph. D.
Principal investigator
Center for Health Services and Outcomes Research
Department of Health Policy and Management
Johns Hopkins Bloomberg School of Public Health
Project supported by the Patient-Centered Outcomes
Research Institute (PCORI)
Johns Hopkins Bloomberg School of Public Health
2
Overview of this meeting
Session1:Preferencesandpriori1esofpa1entswithtype2
diabetes
1:00-1:15pm Welcomeandintroduc6on
1:15-2:00pm Barriersandfacilitatorstoself-managementof
type2diabetes
2:00-2:45pm Treatmentpreferencesofpa6entswithtype2
diabetes
Session2:Otherapplica1onsofstated-preferencestudies
3:00-4:00pm Examplesofstated-preferencestudiesinhealth
4:00-4:30pm Currentandproposedstated-preferencestudiesin
health
4:30-5:00pm WrapupandDiscussion
5:00-6:30pm Recep6on(Carpenterroom,SchoolofNursing)
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
3
Welcome and Overview
of the PCORI Project
4
Overview
This project is funded by the Patient-Centered
Outcomes Research Institute Methods Program
Award (ME-1303-5946).
It aims to promote, advance and apply statedpreference methods to measure the priorities and
preferences of patients and other stakeholders in
medicine
The project is funded for three years at $750,000 direct
costs.
Deemed exempt of review by JHSPH Institutional
Review Board (IRB 6001)
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
5
Acknowledgements
Management team:
•  John F P Bridges (PI), Albert Wu
Patient/stakeholder engagement team:
•  Daniel Longo, Lee Bone
• Stated-preference evaluation team:
•  Karen Bandeen-Roche, Jodi Segal, Tanjala
Purnell
• Project manager
•  Karen Edwards
• Student investigators
•  Ellen Janssen, Allison Oakes, Mo Zhou
• Diabetes action board members
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
6
Diabetes action board (DAB)
The diabetes action board (DAB) is a group of local and
national stakeholders that has played and will continue to
play a role in:
•  Developing this study to measure the preference
of patients in type 2 diabetes
•  Assisting in the broad dissemination of the
research findings and in leverage further
applications and action in type 2 diabetes
•  Building personal and professional relationships to
enrich our work
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
7
Objectives of the PCORI study
1.  Demonstrate and disseminate good practices for
patient and community involvement in patient
centered outcomes research projects by applying
principles of community-based participatory research
2.  Address several key methodological questions
pertaining to the use of stated-preference methods
3.  Demonstrate and disseminate good practices for the
application of stated-preference methods in patient
centered outcomes research
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
8
Aims of the PCORI study
1.  Compare two survey methods for assessing the
priorities of patients with type 2 diabetes (rating/
ranking vs. best-worst scaling)
2.  Compare two survey methods for measuring the
preferences of patients with type 2 diabetes
(choice based conjoint/discrete choice experiment
vs. best-worst scaling)
3.  Compare stratification and segmentation methods
for analyzing preference heterogeneity
4.  Assess patients’ and stakeholders beliefs about
the relevance of our methods and results
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
9
PCORI
Study
StudyDiabetes
Overview -Preferences
Progress
Progress
FirstDAB
mee6ng
SecondDAB
mee6ng
Whitepaper
Report:focus
groups
ThirdDAB
mee6ng
FourthDAB
mee6ng
Report:aggregate
findings
FiVhDAB
mee6ng
Report:
heterogeneity
Systema6c
Pretest Na6onal
Review
(n=25) (n=1000)
Focusgroups
Pilottest
(n=25)
(n=50)
FinalDAB
mee6ng
Report:followup
findings
Followupsurvey
(n=600)
Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov
2014
2015
2016
Completed
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
10
National Survey - Overview
•  1103 people participants with self-reported type 2
diabetes.
•  Survey was administered through GfK Knowledge Panel,
a nationally representative online panel.
•  The survey was in the field for 16 days from October 10
to October 25, 2015
•  Collected preference data as well as self-reported
demographic, personality, and clinical information
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
11
Survey structure
•  A 2X2 randomized design was utilized
•  Participants were randomized to:
•  A prioritization method to measure barriers and
facilitators to diabetes self-management
•  Likert vs. BWS Case 1
•  A preference method to measure treatment
preferences for hypothetical diabetes medications
•  DCE vs. BWS Case 2
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
12
2. Barriers and facilitators to
diabetes self-management
13
Topic identification
•  The proposal called for a comparison of two methods
for prioritization
•  The topic was to be chosen through community
engagement
•  DAB members were:
•  Informed about various prioritization methods
•  Engaged in a topic selection deliberative
process involving
•  Brain storming about possible topics
•  Dot voting to identify most important topics
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
14
Brainstorming and voting results
Dailychoices
7
Lifestyle
6
Family/supportpersoninvolvement
6
Thingsthatma^ertome(health)
5
Thingsthatimpactmydecisions
4
Sources&methodsofinforma6on
4
Educa6ontechniques
3
Diet&exercise/weightcontrol
2
Barrierstotreatment
1
ThingsIunderstand
0
Barrierstolifestylechanges
0
0
(#ofvotes)
1
2
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
3
4
5
6
7
8
15
Diabetes self-management
•  The ability to manage the symptoms, treatment,
physical and psychosocial consequences and lifestyle changes inherent in living with a chronic condition.
•  Intended to curb the worsening of the disease and its
associated complications.
•  Given the complex and chronic nature of diabetes,
effective and comprehensive self-management is key to
the well-being of the patient and often cannot be
substituted by additional medical interventions.
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
16
Barriers and facilitators
•  Barriers and facilitators to self-management can be both
personal and socio-environmental.
•  Personal factors include an individual’s disease-related
beliefs, understanding, and experiences.
•  Key socio-environmental factors include geography,
socioeconomic status, family, culture, religion, work, and
access to health care.
•  Although much literature exists to identify key barriers
and facilitators to the self-management of type 2
diabetes, there has not yet been an attempt to prioritize
and weight each of the barriers and facilitators from the
perspective of the patient.
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
17
Identifying barriers and facilitators
1. Literature review
•  12 studies that examined priorities in type 2 diabetes
2. Focus groups
•  Three focus groups with patients with type 2 diabetes
in Baltimore (n=24)
3. DAB meetings
•  Presented factors to DAB members for comment
4. Pretest interviews
•  Patients with type 2 diabetes (n=25) in Baltimore
5. Pilot Testing
•  Patients with type 2 diabetes from a national online
panel (n=50)
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
18
Hypothesized facilitators
Factors
Description
My own
Do you feel you know enough about
knowledge (++) diabetes to self-manage your diabetes?
Do your healthcare providers have a
Healthcare
positive or negative impact on your ability
providers (+)
to self-manage you diabetes?
Do you have enough support from friends,
Support from
co-workers, support groups or others in
others (+)
your community?
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
19
Hypothesized neutral factors
Factors
Access to
healthy food
(+/–)
Description
Do you have regular access to healthy
food that will support your ability for
diabetes self-management?
Do you usually have the self-control to
Staying
make the best choices for managing your
motivated (+/–)
diabetes?
My ability to
Do you have enough money to
pay (+/–)
successfully self-manage your diabetes?
Physical
Does the place/location where you live and
environment
work provide you with the resources to
(+/–)
manage your diabetes?
Do your local events (e.g. cultural,
Local events
community, or religious) impact your ability
(+/–)
for diabetes self-management?
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
20
Hypothesized barriers
Factors
Family
commitments
(–)
Work
commitments
(–)
Description
Does your family have a positive or
negative impact on your ability to selfmanage your diabetes?
Does your work (or other responsibilities)
affect your ability to self-manage your
diabetes?
Do you have other health conditions
Other health
(mental and physical) that affect how you
conditions (– –)
manage your diabetes?
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
21
Prioritization methods
Likert Item
Best Worst Scaling
•  A quantified response to a
statement on a symmetric,
balanced scale according
to objective/subjective
criteria
•  Strengths: simple, intuitive
appeal, frequent use
•  Limitations: central
tendency bias, social
desirability bias,
acquiescence bias, ceiling/
floor effect
•  A repeated discretechoice response
assessing the best/worst
statement according to
objective/subjective criteria
•  Strengths: simple design
and analysis
•  Limitations: possible floor
and ceiling effects
•  Limited evidence on
strengths and limitations,
needs to be further studied
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
22
Prioritization tasks
LikertItem
•  11items
•  Rankeachitemona5pointscalefromstrong
nega6veimpact(-2)tostrongposi6veimpact
(+2)oncapacityfordiabetesself-management.
BestWorst
ScalingCase1
•  11items
•  Par6cipantpresentedwith11setsof5items
accordingtoaBIBDesign
•  Fromeachset,askedtoselectthebestandthe
worstintermsofimpactondiabetesselfmanagement
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
23
Prioritization via Likert items
-1
Neither a
positive
nor a
negative
impact
☐
☐
0
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
☐
Strong
negative
impact
-2
Local events
Support from others
Access to healthy food
Healthcare providers
Physical environment
Staying motivated
My own knowledge
Family commitments
Work commitments
My ability to pay
Other health conditions
Negative
impact
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
Positive
impact
+1
Strong
positive
impact
+2
24
Prioritization via BWS
Things impacting your own
diabetes self-management
Best
Worst
Access to healthy food
Healthcare providers
My ability to pay
Local events
Family commitments
Consider the following things that can have a positive or negative
impact on your own diabetes self-management. Which of the following
things is the best and which is the worst in terms of impact on your
own diabetes self-management?
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
25
Demographic characteristics
Total (N, prop)
Age (mean, range)
Gender
Male (N, prop)
Race
White (N, prop)
Black (N, prop)
Hispanic (N, prop)
Other (N, prop)
Education
No High school degree (N, prop)
High school degree (N, prop)
Some college (N, prop)
Bachelor’s or higher (N, prop)
Likert
549
62.29
BWS1
554
61.57
263 (.48)
290 (.52)
287 (.52)
126 (.23)
117 (.21)
19 (.03)
288 (.52)
128 (.23)
119 (.21)
19 (.03)
P-value
.313
.140
1.000
.522
44 (.08)
189 (.34)
163 (.30)
153 (.28)
38 (.07)
173 (.31)
175 (.32)
168 (.30)
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
26
Diabetes related characteristics
Years of diagnosis (mean, range)
Hypoglycemia
At least once in past 6 mo (N,
prop)
A1c level
≥ 8.0% (N, prop)
≥ 7.0%, but < 8.0% (N, prop)
< 7.0% (N, prop)
Don’t know (N, prop)
Diabetes medicine
No medicine (N, prop)
Only pills (N, prop)
Only insulin/injection (N, prop)
Pills and injections (N, prop)
Likert
11.54
BWS1
P-value
11.17
.447
270 (.50)
262 (.47)
.599
.283
75 (.14)
149 (.27)
242 (.44)
79 (.15)
88 (.16)
148 (.27)
218 (.40)
94 (.17)
.644
52 (.09)
333 (.61)
48 (.09)
115 (.21)
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
47 (.09)
333 (.60)
42 (.08)
131 (.24)
27
Self-reported personality
Standardizedscoreonascalefromstronglydisagree(-2)tostrongly
agree(+2)
1
0.8
0.6
0.4
0.2
0
Likert
-0.2
BWS1
-0.4
-0.6
Iamalways Ihavealot Iamac6vely Iconsider Iamgood
Doctors
op6mis6c
ofselfwith
should
workingto myselfarisk
aboutfuture control
improve
taker
numbers alwaysask
pa6entsfor
health
preferences.
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
28
400
Time spent per section (minutes)
300
Q1
200
Min
100
Media
n
Max
0
Q3
Likert
Task
Likert
BWS 1
N
549
554
BWS1
Median
(min)
2.3
7
Min
(min)
0.1
0.45
Q1
0.13
0.45
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
Q2
3.3
52.7
Max
(min)
218
432.5
29
Research Question
•  Are priorities estimated using Likert items the same as
priorities estimated using Best-Worst scoring?
•  Determine:
•  Correlation between methods
•  Equivalence of the methods
•  Respondent burden of the methods
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
30
Methods
•  Evaluation questions to determine respondent burden
using Likert item questions
•  Likert rating results were aggregated into a standardized
scores
•  Best-Worst responses were aggregated into “BW scores”
(Times a factor was chosen as best – Times it was
chosen as worst) / Total number of times it appeared
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
31
Evaluation of prioritization tasks
1.2000
1.0000
0.8000
0.6000
Likert
BWS
0.4000
0.2000
0.0000
Ifounditeasyto
understandthe
ques6ons
Ifounditeasyto
completetheques6ons
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
Iansweredinaway
consistentwithmy
preferences
32
Frequency of Likert item responses
500
400
300
2
1
200
0
100
-1
-2
0
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
33
BWS responses
Numberof1mesaOributewaschosenasworstorasbest
Myownknowledge
Healthcareproviders
Accesstohealthyfood
Supportfromothers
Stayingmo1vated
Best
Familycommitments
Worst
Physicalenvironment
Myabilitytopay
Workcommitments
Localevents
Otherhealthcondi1ons
-1500
-1000
-500
0
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
500
1000
1500
34
Likert item vs. BWS (rho=0.97)
0.500
0.400
0.300
BWSscore
0.200
0.100
0.000
0.000
-0.100
0.100
0.200
0.300
0.400
0.500
0.600
-0.200
-0.300
-0.400
Likertscore
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
35
Likert Scores item vs. BWS scores
0.600
0.500
0.400
0.300
0.200
0.000
BWS
Score
0.100
Likert
Scale
-0.100
-0.200
-0.300
-0.400
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
36
BWS 1 results by Gender
0.500
Best_WorstScore
0.400
0.300
0.200
0.100
0.000
-0.100
Male
-0.200
Female
-0.300
-0.400
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
37
BWS 1 results by A1c
0.500
0.400
0.300
Below
7%
0.200
Above
7%
0.100
0.000
-0.100
-0.200
-0.300
-0.400
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
38
Conclusions
•  Overall, there was a fairly strong consistency (rho
>0.9) between the methods
•  Responses to the Likert items demonstrated some of
the classical biases of the approach
•  Social desirability bias/acquiescence bias
•  Responses to the BWS were more variable, symmetric,
and consistent with hypotheses, but lacked a clear
determination neutrality (i.e. had no natural zero)
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
39
Questions?
Thank You!
©2015, Johns Hopkins University. All rights reserved.
40
3. Patient preferences for
diabetes medications
41
Preference methods
Discrete Choice
Best Worst Scaling
•  A repeated discrete-
•  A repeated discrete-
indicating preference
between two or more
profiles according to
objective/subjective criteria
•  Strengths: most frequently
used and studied statedpreference method
•  Limitations: complicated
design and analysis
assessing the best/worst
aspect of a profile
according to objective/
subjective criteria
•  Strengths: simple design
and analysis
•  Limitations: possible floor
and ceiling effects
choice response
choice response
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
42
Instrument development
Evidence
synthesis
Expert
consultation
Stakeholder
engagement
Pretest
interviews
Pilot testing
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
43
The diabetes preference literature
•  10a^ributesextractedfrom12DCEs
CVD"risk"(3)"
Monitoring"(2)"
Quality"of"Life"(2)"
Side"effects"(5)"
Burden"(15)"
Hypoglycemia"(12)"
Nausea"(7)"
Weight"(8)"
Glucose"(14)"
Cost"(5)"
0"
5"
10"
15"
20"
25"
30"
35"
40"
45"
50"
© 2014, Johns Hopkins University. All rights reserved.
Max"
Median"
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
Min"
44
Decision Framework
“Suppose that your doctor says that your current
diabetes medicine is not working to keep your
blood sugar controlled. Your doctor recommends
that you add another diabetes medicine to lower
your A1c.”
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
45
Attributes
Attributes
A1c levels go
down Stable blood
glucose Low blood
glucose
Nausea Additional
medicine Highest
benefit/
Lowest risk
Medium
benefit and
risk
Lowest
benefit/
Highest risk
1%
0.5%
0%
6 days per
week
4 days per 2 days per
week
week
During the day During the day
None
only
and/or night
30 minutes
90 minutes
None
per day
per day
1 pill and 1
1 pill per day 2 pills per day injection per
day
Additional out$10 per month $30 per month $50 per month
of-pocket costs ©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
46
Preference elicitation
Best-Worst Scaling
(BWS)
Attributes
DiscreteChoice Experiment
(DCE)
Medicine
Best Worst
A
A1c levels
1%
go down
Stable blood
4 days/wk
sugar
☐
☐
☐
☐
Attributes
A1c levels go
down
Stable blood
sugar
Medicine Medicine
A
B
1%
0.5%
2 days/wk 4 days/wk
Low blood
glucose
During the
day
☐
☐
Low blood
glucose
During the
day
None
Nausea
None
☐
☐
Nausea
None
90 min/
day
☐
☐
☐
☐
Additional
2 pills/day
medicine
Additional
out-of$50/mo
pocket costs
Additional
2 pills/day 1 pill/day
medicine
Additional out$50/mo
$30/mo
of-pocket costs
Which medicine Medicine Medicine
A
B
would you
choose?
☐
☐
© 2014, Johns Hopkins University. All rights reserved.
© 2014,
Johns
Hopkins
University.
All rights
reserved.
© 2014,
2014,
Johns
Hopkins
University.
Allrights
rights
reserved.
Johns
Hopkins
University.
All
reserved.
©2015,
©
Johns
Hopkins
University.
All
rights
reserved.
47
Preference tasks
DCE
•  6 attributes at 3 levels each
•  Bayesian efficient design:
•  48 profile pairs divided into 4 blocks
•  Added 2 holdout tasks to each block
•  18 profile pairs per participant
•  Prompt: Consider the following two diabetes
medicines. Which medicine would you prefer?
Best
Worst
Scaling
Case 2
•  6 attributes at 3 levels each
•  Orthogonal design:
•  18 profiles per participant
•  Prompt: Which of this medicine’s
characteristics is the best and which is the
worst?
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
48
Research Question
•  Are treatment preferences estimated using BWS Case 2
the same as treatment preferences estimated using
DCE?
•  Determine:
•  Correlation between methods
•  Equivalence of the methods
•  Respondent burden of the methods
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
49
Methods
•  Estimated mixed logit models for both the BWS and
DCE
•  Mixed logit models can account for preference
heterogeneity between individuals
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
50
Demographic characteristics
Total (N, prop)
Age (mean, range)
Gender
Male (N, prop)
Race
White (N, prop)
Black (N, prop)
Hispanic (N, prop)
Other (N, prop)
Education
No HS degree (N, prop)
HS degree (N, prop)
Some college (N, prop)
Bachelor’s or higher (N, prop)
BWS 2
DCE
551 (0.50) 552 (0.50)
63 (25, 89) 61 (24, 91)
P-value
274 (0.49) 279 (0.51)
.787
.985
.082
286 (0.51) 289 (0.52)
128 (0.23) 126 (0.23)
117 (0.21) 119 (0.22)
20 (0.04) 18 (0.03)
.393
39 (0.07) 43 (0.08)
174 (0.32) 188 (0.34)
182 (0.33) 156 (0.28)
156 (0.28) 165 (0.30)
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
51
Diabetes Related Characteristics
BWS 2
Years of diagnosis (mean, range)
Hypoglycemia
At least once in past 6 mo (N,
prop)
A1c level
≥ 8.0% (N, prop)
≥ 7.0%, but < 8.0% (N, prop)
< 7.0% (N, prop)
Don’t know (N, prop)
Diabetes medicine
No medicine (N, prop)
Only pills (N, prop)
Only insulin/injection (N, prop)
Pills and injections (N, prop)
DCE
P-value
13.2
12.6
(11.9, 14.5) (11.4, 13.7)
.645
273 (0.50) 259 (0.47)
.820
.169
83 (0.15) 80 (0.15)
144 (0.27) 153 (0.28)
232 (0.43) 228 (0.41)
84 (0.15) 89 (0.16)
.049
62 (0.11) 37 (0.07)
321 (0.58) 345 (0.63)
48 (0.09) 42 (0.08)
119 (0.22) 127 (0.23)
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
52
Self-reported personality
Standardizedscoreonascalefromstronglydisagree(-2)to
stronglyagree(+2)
1
0.8
0.6
0.4
0.2
0
BWS2
-0.2
DCE
-0.4
-0.6
Iamalways Ihavealot Iamac6vely Iconsider Iamgood
Doctors
op6mis6c
ofselfwith
should
workingto myselfarisk
aboutfuture control
taker
numbers alwaysask
improve
pa6entsfor
health
© 2014, Johns Hopkins University. All rights reserved.
preferences.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
53
25 50 75 100 125 150 175 200
Time spend per Section (minutes)
Q1
Min
Median
Max
0
Q3
DCE
Task
DCE
BWS 2
BWS2
N
Median
Min
Q1
Q3
Max
(minutes) (minutes) (minutes) (minutes) (minutes)
552
10.1
0.9
8.8
16.6
191.6
551
12
1.3
7.4
14.8
146.7
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
54
Evaluation of preference tasks
Standardizedscoreonascalefromstronglydisagree(-2)tostrongly
agree(+2)
1.2000
1.0000
0.8000
0.6000
DCE
BWS2
0.4000
0.2000
0.0000
Ifounditeasyto
understandthe
ques6ons
Ifounditeasyto
completetheques6ons
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
Iansweredinaway
consistentwithmy
preferences
55
Likert rating of the attributes
Standardizedscoreonascalefromnotimportantatall(-2)to
veryimportant(+2)
1.6
1.4
Importance
score
1.2
1
0.8
0.6
0.4
0.2
0
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
56
BWS 2: Results vs. priors (rho=0.93)
Designpriors
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
$50
$30
Out-ofpocketcost
$10
2pills
1pill
90minutes
30minutes
Treatment
burden
1pilland1injec6on
Finalresults
Nausea
None
Day
None
Lowblood
glucose
Dayand/ornight
Analyzedusingcondi6onal
logitandeffectscoding
2days/week
4days/week
Stableblood
glucose
6days/week
0%
0.50%
A1c
decrease
1%
1.8
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
-1.2
-1.4
-1.6
57
DCE: Results vs. priors (rho=0.92)
1.0
0.8
A1c
decrease
Stableblood
glucose
Lowblood
glucose
Nausea
Treatment
burden
Out-ofpocketcost
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
$50
$30
$10
2pills
1pill
90minutes
30minutes
Designpriors
1pilland1injec6on
Finalresults
None
Day
None
Dayand/ornight
Analyzedusingcondi6onal
logitandeffectscoding
2days/week
4days/week
6days/week
0%
0.50%
-1.0
1%
-0.8
58
DCE: Lexicographic Preferences
•  Took differences between attributes for the two
treatment alternatives
•  Participants were recorded if they always chose the
alternative with the better level for a particular attribute
Attribute
Total
A1c decrease
Stable blood sugar
Low blood glucose
Nausea
Treatment burden
Out-of-pocket cost
Final survey (n=552)
N
Proportion
70
13%
6
1%
2
0.4%
21
4%
1
0.2%
46
8%
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
Pilot survey (n=27)
N
Proportion
11
41%
1
4%
3
11%
2
7%
1
4%
3
11%
59
DCE vs. BWS Case 2 (rho = 0.91)
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
$50
$30
Out-ofpocketcost
$10
2pills
1pill
90minutes
30minutes
BWS
Treatment
burden
1pilland1injec6on
DCE
Nausea
None
Day
None
Lowblood
glucose
Dayand/ornight
Analyzedusingmixedlogit
andeffectscoding
2days/week
4days/week
Stableblood
glucose
6days/week
0%
0.50%
1%
1.4
A1c
1.2 decrease
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
-1.2
-1.4
-1.6
60
MXL: Individual coefficients (DCE)
No hypoglycemia
No nausea
1 pill
-1
0
1
2
3
-1 -.5
.5
1 1.5
Stable 4 dy/wk
0
.5
1
0
0
-.5
-2
Daytime hypoglycemia
0
2
4
-1
0
1
2
2 pills
3
2
1
1.5
-.5
0
.5
1
-1
-.5
0
.5
1
-1
-.5
0
.5
1
0
0
0
0
0
1
.5
1
1
.5
2
1
2
3
4
2
3
-2
30 minutes Nausea
4
0.5% A1c decrease
0
1.5
-2
0
0
0
.2
.2
.5
.2
1
.4
.4
1
.6
.4
2
.6
1.5
.8
.8
.6
3
1
Stable 6 dy/wk
2
1% A1c decrease
-.5
Density
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
0
.5
1
-1
0
1
2
y
61
Relative attribute importance
DCE
BWS
16%
17%
18%
12%
18%
22%
12%
17%
10%
20%
27%
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
11%
62
Standardized attribute importance
12
10
8
6
DCE
4
BWS
2
0
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
63
Comparing results with the literature
A(ributes
CVDrisk(3)
Monitoring
QualityofLife
Sideeffects
Treatment
Hypoglycemia
Nausea(7)
Weight(8)
Glucose(14)
Cost(5)
0
5
10
15
20
25
30
Rela%vea(ributeimportance(%)
Max
BWSRESULTSDCERESULTS
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
35
Median
40
45
50
Min
64
BWS score by Gender
0.5
A1c
decrease
Stableblood
glucose
Lowblood
glucose
Nausea
Treatment
burden
Out-ofpocketcost
0.3
0.1
-0.1
-0.3
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
$50
$30
$10
2pills
1pill
90minutes
30minutes
Female
1pilland1injec6on
Male
None
Day
None
2days/week
Dayand/ornight
Analyzedusingcondi6onal
logitandeffectscoding
4days/week
6days/week
0%
0.50%
-0.7
1%
-0.5
65
BWS score by A1c level
0.8
0.6
A1c
decrease
Stableblood
glucose
Lowblood
glucose
Nausea
Treatment
burden
Out-ofpocketcost
0.4
0.2
0.0
-0.2
-0.4
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
$50
$30
$10
2pills
1pill
90minutes
30minutes
>8.0%
1pilland1injec6on
<7.0%
None
Day
None
2days/week
Dayand/ornight
Analyzedusingcondi6onal
logitandeffectscoding
4days/week
6days/week
0%
0.50%
-0.8
1%
-0.6
66
Conclusion
•  Participants did not express a strong preference
towards BWS or DCE.
•  The proportion of individuals with Lexicographic
preferences was much lower in the final survey
(Bayesian D-efficient design) than in the pilot survey
(orthogonal design)
•  Preference weights obtained from BWS or DCE had
high correlation, but were on a different scale.
© 2014, Johns Hopkins University. All rights reserved.
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
©2015,
67
Questions?
Thank You!
©2015, Johns Hopkins University. All rights reserved.
68
Protecting Health,
Saving Lives—
Millions at a Time
69
Barriers and Facilitators
Factor
Access to
healthy food
Healthcare
providers
My own
knowledge
Staying
motivated
My ability to pay
Other health
conditions
Family
commitments
Physical
environment
Local events
Work
commitments
Support from
others
Description
Do you have regular access to healthy food that will support your ability for
diabetes self-management?
Do your healthcare providers have a positive or negative impact on your
ability to self-manage you diabetes?
Do you feel you know enough about diabetes to self-manage your
diabetes?
Do you usually have the self-control to make the best choices for managing
your diabetes?
Do you have enough money to successfully self-manage your diabetes?
Do you have other health conditions (mental and physical) that affect how
you manage your diabetes?
Does your family have a positive or negative impact on your ability to selfmanage your diabetes?
Does the place/location where you live and work provide you with the
resources to manage your diabetes?
Do your local events (e.g. cultural, community, or religious) impact your
ability for diabetes self-management?
Does your work (or other responsibilities) affect your ability to self-manage
your diabetes?
Do you have enough support from friends, co-workers, support groups or
others in your community?
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
70
BWS 1 results by Education
0.500
Lessthan
high
school
High
school
degree
Some
college
0.400
Best_WorstScore
0.300
0.200
0.100
0.000
-0.100
Bachelor
degree
-0.200
-0.300
-0.400
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
71
BWS 1 results by Race
0.500
0.400
Best_WorstScore
0.300
0.200
White
0.100
Black
0.000
Hispanic
-0.100
-0.200
-0.300
-0.400
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
72
BWS results by Income
0.500
0.400
Best_WorstScore
0.300
0.200
0.100
<25,000
0.000
-0.100
25,000-49,
999
-0.200
-0.300
-0.400
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
73
A1c levels
A1c levels
Doctors prescribe diabetes medicines to help lower your
A1c, or average blood glucose level during the past
three months. Keeping your A1c at the recommended
level may decrease your risk for serious health
problems such as heart attack, blindness, amputation,
and kidney failure. When taking the new medicine your
A1c level might go down by:
•  1% – this is a large decrease
•  0.5% – this is a moderate decrease
•  0% – this is no decrease
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
74
Stable Blood Sugar
Stable blood sugar
The new medicine might help keep your blood glucose
levels stable on a daily basis. Your blood glucose levels
are stable for the day if they stay in a range of 70-180
mg/dl. When taking this new medicine your blood
glucose levels might be stable for:
•  6 days per week
•  4 days per week
•  2 days per week
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
75
Low Blood Sugar
Low blood sugar
You might experience low blood glucose, also known as
hypoglycemia. This may make you feel shaky/drowsy
and have blurred vision or difficulty walking/talking. You
might pass out (if you don’t eat or drink). Low blood
glucose can also happen at night while you sleep. Then
you won’t know about it and you might be more likely to
pass out. You might experience low blood glucose:
•  None
•  During the day only
•  During the day and/or at night
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
76
Nausea
Nausea
The new medicine may cause moderate nausea. This
means you feel sick to your stomach and like you need
to vomit. When taking this new medicine you might
experience nausea for a total of:
•  None
•  30 minutes per day
•  90 minutes per day
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
77
Treatment Burden
Treatment burden
You will have to take the new medicine daily. You need
to take this medicine in addition to the medicines you
already take. We will consider three different ways of
taking the medicine. You might have to take an
additional:
•  1 pill per day
•  2 pills per day
•  1 pill and 1 injection per day
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
78
Medication Costs
Medication costs
The medicine will require out-of-pocket costs in addition
to what you already pay for other medicines. The
money you spend on this medicine cannot be spent on
other things. Your additional costs might be:
•  $10 per month
•  $30 per month
•  $50 per month
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
79
DCE results by Education
Somecollegeormore
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
$50
$30
Out-ofpocketcost
$10
2pills
1pill
90minutes
30minutes
Treatment
burden
1pilland1injec6on
HSorless
Nausea
None
Day
None
2days/week
Lowblood
glucose
Dayand/ornight
Analyzedusingcondi6onal
logitandeffectscoding
4days/week
Stableblood
glucose
6days/week
0%
0.50%
A1c
decrease
1%
1.2
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
-1.2
-1.4
80
DCE results by Race
0.6
A1c
decrease
Stableblood
glucose
Nausea
Lowblood
glucose
Treatment
burden
Out-ofpocketcost
0.4
0.2
0.0
-0.2
-0.4
-0.6
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
Hispanic
$50
$30
$10
2 pills
1 pill
90 minutes
30 minutes
Black
1 pill and 1 injection
White
None
Day and/or night
Day
None
2 days/week
Analyzedusingcondi6onal
logitandeffectscoding
4 days/week
6 days/week
0%
0.50%
1%
-0.8
81
DCE results by Language
1.2
1.0
A1c
decrease
Stableblood
glucose
Nausea
Lowblood
glucose
Treatment
burden
Out-ofpocketcost
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
-1.2
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
$50
$30
$10
2 pills
1 pill
90 minutes
30 minutes
English
1 pill and 1 injection
Spanish
None
Day
None
2 days/week
Day and/or night
Analyzedusingcondi6onal
logitandeffectscoding
4 days/week
6 days/week
0%
0.50%
1%
-1.4
82
Scale differences
•  Estimated preference weights are inversely related to
error variances.
•  Therefore estimated preference weights from groups
with different variances can differ even if the true weights
are the same.
•  These are called scale differences.
•  Can be tested for using Swait-Louviere scale test
•  No scale differences were found based self-reported
•  Skill with numbers
•  Ease of understanding the choice tasks
•  Ease of answering the choice tasks
•  Consistency in answers
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
83
DCE results by Ease of Answering
0.8
0.6
A1c
decrease
Stableblood
glucose
Nausea
Lowblood
glucose
Treatment
burden
Out-ofpocketcost
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
$50
$30
$10
2 pills
1 pill
90 minutes
30 minutes
Easy to answer
1 pill and 1 injection
Not easy to answer
None
Day and/or night
Day
None
2 days/week
4 days/week
6 days/week
0%
0.50%
1%
-1.0
84
MNL
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
90minutes
Treatment
burden
Out-ofpocketcost
$10
$30
$50
MXL
Nausea
None
30minutes
Lowblood
glucose
None
Day
Dayand/ornight
6days/week
4days/week
2days/week
1%
0.50%
0%
1.0 A1cdecrease Stableblood
glucose
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
-1.2
1pill
2pills
1pilland1injec6on
DCE: MXL vs. MNL (Rho>0.99)
85
MNL
©2015,
© 2014,Johns
JohnsHopkins
HopkinsUniversity.
University.All
Allrights
rightsreserved.
reserved.
90minutes
Treatment
burden
Out-ofpocketcost
$10
$30
$50
MXL
Nausea
None
30minutes
Lowblood
glucose
None
Day
Dayand/ornight
6days/week
4days/week
2days/week
1%
0.50%
0%
1.4 A1cdecrease Stableblood
1.2
glucose
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
-1.2
-1.4
-1.6
1pill
2pills
1pilland1injec6on
BWS: MXL vs. MNL(Rho>0.99)
86