2015 Joint Congress on
Medical Imaging and Radiation Sciences
Montreal, QC
May 30, 2015
George Wells
Comparative & Cost Effectiveness
Related to Diagnostic Testing
Learning Objectives
• To be aware of the role of comparative effectiveness in comparing
diagnostic tests
• To understand the steps in conducting a cost-effectiveness studies
• To appreciate the cautions associated with designing, conducting
and interpreting comparative and cost-effectiveness studies
Comparative
effectiveness
Cost
effectiveness
Test
performance
INTRODUCTION
Motivation
Medical imaging has transformed medical care
Evidence gap - growth in using many types of medical imaging has
occurred without convincing evidence of additional clinical benefits or
improved cost-effectiveness over existing care
Assessment need - with increasing cost pressures and wide variation in
usage, these gaps in evidence create a compelling argument for more
assessment of medical imaging to support health care decisions
Challenges – levels of evidence
A framework to evaluate diagnostic technologies was developed in 1970s
by categorizing evidence into six levels
Level of Evidence
Examples
Technical
Pixels per millimeter, Section thickness
Diagnostic accuracy
Sensitivity, Specificity, Area under ROC curve
Impact of diagnostic thinking
Percentage of cases in which the clinician believes that the test
alters the diagnosis
Impact of therapeutic actions
Impact on patient outcomes
Impact on societal outcomes
Test performance
Build upon test performance with each level
Percentage of cases in which the choice of therapies is changed
representing
a the
distinct
type of evidence that
after information from
test is provided
might,
or might not, be available when
Differences in mortality, morbidity, or quality of life between patients
evaluating
a test
diagnostic
test without it
managed with the
and those managed
Cost-effectiveness of the improvement in patient outcomes, such
as cost per life-year saved, calculated from a societal perspective
Pearson SD, Knudsen AB, Scherer RW, Weissberg J, Gazelle GS,
Assessing The Comparative Effectiveness Of A Diagnostic Technology:
CT Colonography. Health Affairs 27, no. 6 (2008): 1503–1514
Challenges – some features that complicate
creation or assessment of evidence
Level of Evidence
Examples
Constant change
• Imaging devices and patient preparation techniques evolve, and
different versions can be in use or under study at any one time
• Assessments hindered by questions of whether evidence has
much relevance to the most recent version of the technique
Role of interpretation
• Diagnostic accuracy of an imaging technique is linked with
practitioners’ experience and skill in interpreting results
Variability in published
results
• Sensitivity and specificity may vary due to differences in the
spectrum of patients – higher in patients with more typical or
severe; lower among patients with atypical or less severe
Transferability
• “Best hands” problem - how transferable are results coming
from studies of the most advanced devices, in the hands of
experts, among highly selected patients at academic health
centers TO devices, practitioners, and patients in the community
Screening intervals
• Evidence comparing the onetime use of a new technique with an
established gold standard produces no evidence to guide
decisions on intervals for screening or follow-up – clinical and
cost effectiveness may be highly dependent on frequency
Impact on patient
outcomes
• Evidence on a medical imaging technique often does not include
direct evidence of its impact on patient outcomes
Framework for levels of evidence
Level of Evidence
Examples
Technical
Pixels per millimeter, Section thickness
Diagnostic accuracy
Sensitivity, Specificity, Area under ROC curve
Impact of diagnostic thinking
Percentage of cases in which the clinician believes that the test
alters the diagnosis
Impact of therapeutic actions
Percentage of cases in which the choice of therapies is changed
after information from the test is provided
Impact on patient outcomes
Differences in mortality, morbidity, or quality of life between
patients managed with the test and those managed without it
Impact on societal outcomes
Cost-effectiveness of the improvement in patient outcomes, such
as cost per life-year saved, calculated from a societal perspective
META-ANALYSIS
Systematic reviews and meta-analysis
Systematic Review: application of scientific strategies that limit bias
to the systematic assembly, critical appraisal and synthesis of all relevant
studies on a specific topic
Meta-Analysis: a systematic review that employs statistical methods
to combine and summarize the results of several studies
Steps in a systematic review
Clearly formulated
question
Comprehensive data
search
Unbiased selection and
extraction process
Critical appraisal of data
Analysis of data
Sensitivity and subgroup
analyses
Interpretation of results
Steps in a systematic review
Clearly formulated
question
Pose question specifying operational definitions for: Population, Index tests,
Comparator tests, Outcomes
Criteria for considering inclusion of studies: PICO
P: Type of Participants / Population
I: Types of Index diagnostic tests
C: Types of comparator diagnostic tests
O: Types of outcome measures
Comprehensive data
search
Unbiased selection and
extraction process
Critical appraisal of data
Analysis of data
Sensitivity and subgroup
analyses
Interpretation of results
Steps in a systematic review
Clearly formulated question
Comprehensive data
search
Unbiased selection and
extraction process
Critical appraisal of data
Analysis of data
Sensitivity and subgroup
analyses
Interpretation of results
•
•
•
•
•
Need a well formulated and coordinated effort
Requirements for comprehensiveness of search depends on question
Usually begin with searches of bibliographic databases
Publications retrieved and references therein searched
As a step to eliminate publication bias need unpublished research
Steps in a systematic review
Clearly formulated question
Comprehensive data search
Unbiased selection and
extraction process
Study Selection
• 2 independent reviewers select studies
• Selection of studies based on PICO
• Level of agreement: kappa; differences resolved by consensus
• Specify reasons for excluding studies
Data Extraction
• 2 independent reviewers extract data using predetermined forms
• PICO characteristics, study design/methods, study results, quality items
• Level of agreement: kappa; differences resolved by consensus
Critical appraisal of data
Analysis of data
Sensitivity and subgroup
analyses
Interpretation of results
Steps in a systematic review
Clearly formulated question
Comprehensive data search
Unbiased selection and extraction process
Critical appraisal of data
Description of Studies
• Table of characteristics of included studies
• Reference, design, PICO elements, quality assessment
Quality Assessment
• Report quality assessment instrument used
(e.g. QUADAS – Quality Assessment of Diagnostic Accuracy Studies)
• Quality assessment instrument - transparent, parsimonious, reproducibility
Analysis of data
Sensitivity and subgroup
analyses
Interpretation of results
Steps in a systematic review
Clearly formulated question
Comprehensive data search
Unbiased selection and extraction process
Critical appraisal of data
Analysis of data
Effect estimates (EE): sensitivity, specificity, likelihood ratio (LR), diagnostic odds ratio (DOR)
Heterogeneity: clinical (PICO), methodological (design/conduct)
Procedures: do not pool, meta-regression, sensitivity, subgroup
Fixed effects model (FE); Random effects model (RE)
Procedures: subgroup
Overall estimates and analyses
Display: Forest Plots, Summary of Findings (SoF) tables
Summary receiver operating characteristic (SROC) curves with area under the curve
(AUC) as a measure of diagnostic accuracy - shape of curve on basis of changes in
DOR using Moses’s constant-of-linear model
Steps in a systematic review
Clearly formulated question
Comprehensive data search
Unbiased selection and extraction process
Critical appraisal of data
Analysis of data
Forest Plot
Sample ROC curve for diagnostic accuracy
Steps in a systematic review
Clearly formulated question
Comprehensive data search
Unbiased selection and extraction process
Critical appraisal of data
Analysis of data
Sensitivity and subgroup
analyses
Subgroup Analysis:
• Pre-specify hypothesis-testing subgroup analyses; label others as a posteriori
Sensitivity Analysis:
• Test robustness of results relative to key features of the studies and key
assumptions and decision
Interpretation of results
Steps in a systematic review
Clearly formulated question
Comprehensive data search
Unbiased selection and extraction process
Critical appraisal of data
Analysis of data
Sensitivity and subgroup analyses
Interpretation of results
•
•
•
•
•
•
Interpret results in context of current health care
State methodological limitations of studies and review
Consider size of effect and review consistency
Interpret results in light of other available evidence
Make recommendations clear and practical
Propose future research agenda
Illustration - Rb-82 PET
Background:
• Myocardial perfusion imaging is widely used in assessing patients with known or
suspected CAD
• PET using Rb-82 has potential advantages over SPECT that may make it more
accurate and reduce radiation exposure, has increased costs
• Comparisons of technologies are relevant for policy makers and practice guidelines
Illustration - Rb-82 PET
Objective:
To evaluate the accuracy of rubidium (Rb)-82 PET for the diagnosis of
obstructive coronary artery disease (CAD) in comparison to SPECT
Method:
• Systematic review of studies
• Where either Rb-82 PET or technetium-99m SPECT with both
attenuation correction and electrocardiography-gating were used as
a diagnostic test for obstructive CAD with invasive coronary
angiogram as a reference standard
Illustration - Rb-82 PET
Flow Diagram Showing the Results of the Systematic Review
Illustration - Rb-82 PET
Forest Plots Showing Sensitivities and Specificities of: (A) Rb-82
PET Studies and (B) Tc-99m SPECT Studies
Illustration - Rb-82 PET
SROC Curves for Diagnostic Accuracy of Rb-82 PET and
Tc-99m SPECT
With ECG-Gating and Attenuation Correction
Area under the curve (AUC) was compared for Rb82 PET (A) and Tc-99m SPECT (B), with Rb-82 PET
showing superior accuracy (p 0.001).
Illustration - Rb-82 PET
Conclusion:
• Rb-82 PET is accurate for the detection of obstructive CAD and,
despite advances in SPECT technology, remains superior
• More widespread use of Rb-82 PET may be beneficial to improve
CAD detection
Framework for levels of evidence
Level of Evidence
Examples
Technical
Pixels per millimeter, Section thickness
Diagnostic accuracy
Sensitivity, Specificity, Area under ROC curve
Impact of diagnostic thinking
Percentage of cases in which the clinician believes that the test
alters the diagnosis
Impact of therapeutic actions
Percentage of cases in which the choice of therapies is changed
after information from the test is provided
Impact on patient outcomes
Differences in mortality, morbidity, or quality of life between
patients managed with the test and those managed without it
Impact on societal outcomes
Cost-effectiveness of the improvement in patient outcomes, such
as cost per life-year saved, calculated from a societal perspective
COMPARATIVE
EFFECTIVENESS
Endpoints for diagnostic test evaluation
• Diagnostic performance:
• Measures of accuracy
Accuracy • Measures of predictive value
Affects
care
Affects
outcomes
• Intermediate process of care:
• Diagnostic thinking/decision making
• Therapeutic thinking/decision making
• Patient outcomes:
• Mortality, morbidity, quality of life, costs
Fundamental Challenge
Cost effectiveness research calls
for informing the path from
medical imaging diagnostic
information to patients outcomes
•
Diagnostic imaging tests provide
information for use in selecting course of
care
•
Both long and short-term effects of tests
materialize in context of available health
care options, including therapeutic
interventions
Diagnostic
information
Comparative
effectiveness
Diagnostic
workup
Outcomes
Treatment
decisions
Patient
outcomes
Hierarchy of Studies – levels of evidence
RCTs
Cohort studies
Case-Control studies
Cross-sectional studies
Case Series/ Case Reports
•
Concept of moving from low to high quality evidence generating
studies
RANDOMIZED CONTROLLED
TRIAL
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Outcome
Outcome
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Outcome
Outcome
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Outcome
Outcome
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Outcome
Outcome
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Outcome
Outcome
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Outcome
Outcome
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Performance bias
Attrition bias
Outcome
Outcome
Detection bias
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Compliance
Contamination
Outcome
Outcome
Cointervention
Structure: Randomized Controlled Trial
Target Population
Study Population
Participants
Non-participants
R
Imaging Test A
Imaging Test B
Outcome
Outcome
Analysis Populations
Potential Weaknesses
Limited applicability due to ethic considerations and artificial
experimental setting
Differential rates of noncompliance, withdrawals and losses to follow-up
Costly and logistically time consuming
Sample size due to recruitment (volunteers, eligibility)
External validity
NON-RANDOMIZED STUDIES
Some bias domains through which bias might be
introduced into a non-randomized study
Bias Domain
Related Terms
Bias due to confounding
Selection bias; Allocation bias; Casemix bias; Channelling
bias
Bias in selection of
participants into the study
Selection bias; Inception bias; Lead-time bias; Immortal
time bias
Bias in measurement of
interventions
Misclassification bias; Information bias; Recall bias;
Measurement bias; Observer bias
Bias due to departures
from intended interventions
Performance bias; Time-varying confounding
Bias due to missing data
Attrition bias; Selection bias
Bias in measurement of
outcomes
Detection bias; Recall bias; Information bias;
Misclassification bias; Observer bias; Measurement bias
Bias in selection of the
reported result
Outcome reporting bias; Analysis reporting bias
Making Comparisons
Exposure
Yes vs No
Outcome
Background
factors
1. Design
2. Analysis
3. Randomize
-
match
stratified sampling
restrict inclusion
standardize
stratify (subgroups)
adjust (statistical model)
Adjusting for Confounding
In conducting comparative effectiveness analysis for nonrandomized studies, concern regarding the imbalance of
underlying confounding variables between comparison groups
exists
Methods for addressing this concern:
• Covariate adjustment
• Propensity score methods
• Instrumental variable analysis
Regression Models – Covariate adjustment
General Linear Model
Y=β0 + β1X1 + β2X2 + … + βkXk + ε
Independent variables
Dependent variable
Regression Models – Covariate adjustment
Method
Dependent
Independent
General purpose
Statistical tools
Multiple
regression
analysis
Continuous
Continuous
(in theory all)
To describe the extent, direction and
strength of the relationship between
several independent variables and a
continuous dependent variable
• F-tests
• t-tests
Logistic
regression
analysis
Binary
(typically)
Continuous
Discrete
To describe the extent, direction and
strength of the relationship between
several independent variables and a
binary dependent variable
• chi-square test
• odds ratio
Cox
proportional
hazards
model
Continuous
(time-toevent)
Continuous
(in theory all)
To describe the time to occurrence of the
event of interest and the relationship
between tine to event and several
independent variables
•
•
•
Poisson
regression
analysis
Counts
(rates)
Continuous
Discrete
To describe the extent, direction and
strength of the relationship between
several independent variables and a rate
• t-tests
• chi-square test
log rank test
chi-square texts
hazard ratio
Propensity Score Analysis (PS)
A PS is a probability of receiving a specified imaging technology conditioned
on underlying known and observed confounding variables
PS analysis mimics some of the conditions of an RCT
• For example, for data matched on the PS, the distribution of measured baseline
characteristics will be similar in the imaging technologies groups compared
A PS analysis can yield unbiased estimates of the effect (impact)
PS can be calculated using logistic regression models where the imaging
technology received is the outcome and the variables on which we wish to
balance the data are the explanatory variables
Propensity Score Analysis
Once the PS is generated, the adjusted analysis can be implemented using
several different approaches
Covariate
Match
Stratify
• Consider the PS as a covariate in the regression model
• Match patients on the PS
• Use conditional logistic regression for analyzing the
matched data
• Divide the PS into strata, and separate the patients by
these strata so patients are similar on the measured
variables within each stratum
• Compare patients within strata and an overall treatment
effect can be obtained by calculating a weighted mean
across the strata
Propensity Score Analysis - pros
and cons
Pros
Cons
More similar to randomized comparison
More steps then just adding covariates to model
individually
Able to assess overlap of imaging groups
Effects of individual covariates on outcome are lost
Possible to develop score while blinded to
outcomes
Can only adjust for measured variables, and one can
only be ensured that the data is balanced on these
measured variables
Only one score needed for all outcomes
If unmeasured variables exist that are strongly
associated with the outcome, then will not be able to
adjust for them and biased estimates of the effect
(impact) may result
Avoids multicollinearity
May give false sense of security if an important
confounder is not collected
Instrumental Variable Analysis (IV)
Propensity scores can be useful, but they rely on observable and measurable
variables
• If important variables are unobservable then the effect estimates obtained from
propensity score analysis can be biased
IV method attempts to account for such unmeasured and unobserved sources of
confounding
IV method identifies variables correlated with the
imaging technology assigned but not directly
correlated to the outcome
• When such variables are found they can be
incorporated into the analysis in an attempt
to remove allocation bias
Imaging
Instrumental Variable Analysis
Once an IV is identified:
• Test if the IV is predictive of the imaging technology (can use logistic
regression)
• Next assess relationship between the IV and the outcome (can use
correlational analysis)
Once an IV is evaluated, a two-step process can be followed for the analysis
First
Second
• The IV along with other
covariates of interest are
used to predict the
imaging technology
• The predicted imaging
technology from first
stage and other
covariates are used to
estimate the outcome
Instrumental Variables –
Examples (Wall Street Journal)
Illustration – PARR-2
Background:
• Patients with severe ventricular dysfunction and suspected coronary disease, may benefit
from revascularization, but have significant perioperative morbidity and mortality
• FDG PET can detect viable myocardium that might recover after revascularization
Illustration – PARR-2
Objective:
To assess the effectiveness of FDG PET-assisted management in patients with severe
ventricular dysfunction and suspected coronary disease compared to standard care
Methods:
• Included patients with severe left ventricular (LV) dysfunction and suspected coronary
disease being considered for revascularization, heart failure, or transplantation workups or in whom PET was considered potentially useful
• Patients were stratified according to recent angiography or not, then randomized to
management assisted by FDG PET (n= 218) or standard care (n= 212)
• Primary outcome was the composite of cardiac death, myocardial infarction, or
recurrent hospital stay for cardiac cause, within 1 year.
Illustration – PARR-2
• Time to first occurring outcome of
composite event
• HR 0.78 (95% CI 0.58 to 1.1), p 0.15
• Time to first cardiac death for patients
without recent angiography
• HR 0.4 (95% CI 0.17 to 0.96), p 0.035
• Time to cardiac death for all patients
• HR 0.72 ( 95% CI 0.4 to 1.3), p 0.25
• Adherence to PET subgroup
• Time to first occurring outcome of the
composite event
• Adjusted HR 0.62 (95% CI 0.42 to
0.93), p 0.019.
Illustration – PARR-2
Conclusion:
• This study did not demonstrate a significant reduction in cardiac events in patients with
LV dysfunction and suspected coronary disease for FDG PET-assisted management
versus standard care
• In those who adhered to PET recommendations and in patients without recent
angiography, significant benefits were observed
• The utility of FDG PET is best realized in this subpopulation and when adherence to
recommendations can be achieved
Framework for levels of evidence
Level of Evidence
Examples
Technical
Pixels per millimeter, Section thickness
Diagnostic accuracy
Sensitivity, Specificity, Area under ROC curve
Impact of diagnostic thinking
Percentage of cases in which the clinician believes that the test
alters the diagnosis
Impact of therapeutic actions
Percentage of cases in which the choice of therapies is changed
after information from the test is provided
Impact on patient outcomes
Differences in mortality, morbidity, or quality of life between
patients managed with the test and those managed without it
Impact on societal outcomes
Cost-effectiveness of the improvement in patient outcomes, such
as cost per life-year saved, calculated from a societal perspective
COST EFFECTIVENESS
Concepts
• Economics is the study of unlimited needs/wants
constrained by a limited number of resources (scarcity)
• Choices need to be made
• For each choice that is made there is an opportunity cost
associated with it – something else that is given up
Opportunity Costs in Health Care
Since we are not able to pay for all therapies, we need to
make choices
In doing so, we need to know whether a therapy is worth the
cost
$
Clinical
benefits
Opportunity Costs in Health Care
To make choices we compare:
Costs
• Cost of the therapy
• Cost of health care resources (more or less because of the
therapy)
• Cost of ‘other things’ - lost time attending appointments
Benefits
• Makes me feel better
• Makes therapy more convenient
• Makes some ‘other things’ better
Example – Comparing Therapies
• When comparing two therapies, a therapy is a
good option (economically attractive) if:
• It does what the other does but costs less
• It costs the same as the other but does better
• It costs less and does better than the other therapy
• What if it:
• Costs less and does less? OR
• Costs more and does more?
Decrease in
total costs(-)
Consider
cost effectiveness
Status quo better than
new therapy

0
Increase in
total costs (+)
Comparing Therapies
?
?
O

Consider
cost effectiveness
Reduced clinical benefit (-)
New therapy better than
status quo
0
Improved clinical benefits (+)
Economic Evaluations
• Provide a measure of “value for money”
• Comprised of two concepts:
1. Cost
2. Clinical / health effects
• Systematic way to compare therapies
Types of Economic Evaluations
• Typically reported as a ratio (cost effectiveness):
Incremental
Cost Effectiveness of
=
Therapy (1) vs Therapy (2)
Total Cost (1) – Total Cost (2)
Effect (1) – Effect (2)
ICER – Incremental Cost Effectiveness Ratio
Types of Economic Evaluations
• Typically reported as a ratio (cost effectiveness):
Incremental
Cost Effectiveness of
=
Therapy (1) vs Therapy (2)
Total Cost (1) – Total Cost (2)
Effect (1) – Effect (2)
Type of analysis largely
depends on clinical effects
Cost Minimization
Focus on total cost associated with
therapies (therapies, health care cost,
administration, monitoring)
Incremental
Cost Effectiveness of
=
Therapy (1) vs Therapy (2)
Total Cost (1) – Total Cost (2)
Effect (1) = Effect (2)
Similar safety and effectiveness
established (clinical efficacy, safety,
compliance, patient satisfaction)
Costs
Opportunity cost is based on the value that would be gained from using
resources elsewhere
– In practice, assume price paid reflects the opportunity cost
Distinguish costs associated with a therapy between:
– Direct costs: medical (drugs, procedures, staff time, equipment);
patient (transportation, out-of pocket expenses)
– Indirect costs: production losses, other uses of time
– Intangibles: pain; suffering; adverse effects
Essential to specify which costs included in a cost-effectiveness analysis
Cost Effectiveness
Incremental
Cost Effectiveness of
=
Therapy (1) vs Therapy (2)
Total Cost (1) – Total Cost (2)
Effect (1) – Effect (2)
-
In terms of clinically meaningful outcomes
e.g., survival, clinical event avoided
-
Where “outcomes of uncertain clinical
significance” are used, interpretation of
results challenging
Cost Utility
Incremental
Cost Effectiveness of
=
Therapy (1) vs Therapy (2)
-
Total Cost (1) – Total Cost (2)
QALY (1) – QALY (2)
Therapy impacts patients quality of life or
meaningful outcomes that in turn affect
quality of life
What is a utility?
• A utility is a quantitative expression of an individuals’
preference for a particular health state
• Utilities can be measured by direct or indirect techniques
Reference: What is a QALY? London:
Hayward Medical Communications, 2001
What is a Quality Adjusted Life Year
(QALY)?
• Outcome measure that incorporates both quantity of life
(mortality) and health-related quality of life (morbidity)
• Quantity – how long person lives
• Quality – factor that represents a preference for a health
state
• one year of perfect health = one QALY
• one year less than perfect health < one QALY
• death = zero
Types of Analyses
Form of Analysis
Measurement
of Costs
Measurement of
Benefit
Synthesis of Costs and
Benefits
Cost Minimization
Analysis (CMA)
Dollars
None
Incremental cost
Cost Effectiveness
Analysis (CEA)
Dollars
Single dimension of
effectiveness
(e.g. life years gained)
Incremental cost
effectiveness;
Incremental cost per unit
gained
Cost Utility Analysis
(CUA)
Dollars
Utility gained
(e.g. QALYs – quality
adjusted life years)
Incremental cost
effectiveness;
Incremental cost per QALY
gained
Cost Benefit
Analysis (CBA)
Dollars
Monetary value of
benefits gained
Net benefit gained
Advantages of Economic Evaluations
• Provides a consistent framework to compare therapies
• Enables assessment or comparison of different therapies
(i.e., past therapies reviewed by CADTH, NICE etc.)
– League table - comprises a list of healthcare
interventions in ascending order (from low to high) of
their ICER (e.g., from high priority (low cost per QALY)
to low priority (high cost per QALY)).
• Reports costs and benefits in a single metric
• Provides a framework to assess uncertainty (i.e., sensitivity
analyses) to gain a better understanding of where true cost
effectiveness may fall
Rationale of CEA in diagnostic imaging
• CEA (cost-effectiveness analysis) – a comparative analysis of
alternative courses of action in terms of both their costs and
consequences
• When assessing the cost-effectiveness of DI, the initial question
is whether adding an imaging test in a medical pathway does
improve medical decision-making
• In many clinical situations imaging is already part of the
particular disease management; so CEA is used to compare
potential new imaging technologies to each other as well as to
the current reference standard
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
Selecting model input
parameters
Analysis of values and
uncertainty
Interpretation of the
results
Transferability and
validation
Sailer AM, van Zwam WH, Wildberger JE, Grutters JOC,
Cost-effectiveness modelling in diagnostic imaging: a
stepwise approach. European Radiology, Published online
May 24 2015
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
Selecting model input
parameters
Analysis of values and
uncertainty
Interpretation of the
results
Transferability and
validation
• Clear statement of imaging decision problem, modelling objectives and the
scope of the model
• Factors to consider are:
• decision maker (e.g. government or medical doctor)
• perspective (e.g. societal perspective or health care perspective)
• comparators (which imaging strategies should be compared in the analysis)
• outcomes of model (e.g. which health consequences)
• target population (for which patients is the decision relevant)
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
• Common practice in imaging studies to combine a decision tree for the shortterm diagnostic accuracy and treatment decision with a Markov model for the
longer-term disease consequences
• Decision Tree
• Flow model to visualize and calculate the effects of imaging on costs and
health outcome in a static situation and short time frame
• A hypothetical cohort of patients passes through the model and is divided
over the different pathways in the decision tree, according to the assigned
probabilities
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
• The Markov Model
• In non-static situations, such as in chronic diseases, health states of the
patients may alter depending on imaging and treatment options
• In such situations, a Markov model can be used to reflect alterations in
health states during the period of CEA
• In a Markov model, the clinical situation is described in terms of the
conditions that patients can be in (‘health states’), how they can move in
such states (‘transitions’), and how likely such moves are (‘transition
probabilities’)
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
Selecting model input
parameters
Analysis of values and
To predict costs and health outcome as effects of DI, both the direct and indirect
effects of the imaging test itself on health outcome and costs should be
considered as model input parameters
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
Selecting model input
parameters
Analysis of values and
uncertainty
• Cost-effectiveness modelling serves two purposes:
• Estimate expected costs and outcomes for medical decision-making
• Assess uncertainties around these estimates and its range of validity
• Expected costs and outcomes can be obtained by multiplying the probabilities
with their relevant costs and health outcomes
• Parameter uncertainty reflects the uncertainty of the model input parameters
because we do not know the precise values - can be addressed by a:
• Deterministic sensitivity analysis
• Probabilistic sensitivity analysis
Interpretation of the
results
Transferability and
validation
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
Selecting model input
parameters
Analysis of values and
uncertainty
Interpretation of the
results
Cost-effectiveness graph - total estimated costs of diagnostic test, treatments and
health state are compared to estimate health outcome over the chosen period of
time
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
Selecting model input
parameters
Analysis of values and
uncertainty
Interpretation of the
results
Cost-effectiveness acceptability curves (CEACs) - probability of cost-effectiveness
of investigated imaging tests is plotted against the willingness to pay for a quality
adjusted life year
Steps for CEA in Diagnostic Imaging
Defining the decision
problem
Developing the decision
model
Selecting model input
parameters
Analysis of values and
uncertainty
Interpretation of the
results
Transferability and
validation
When developing the model, it is important to consider the decision make and the
jurisdiction - the value of input parameters may well differ between countries, and
the chosen estimates should be relevant for the given society
A decision model that has been developed for one country might need adaptation
to support decision-making in another.
Important that input parameters of a model are transparently described; this
allows others to explore whether the inputs are relevant for their country or
decision problem
A transparent description of all inputs and choices is also important for reasons of
validity
Level of Evidence
Examples
Technical
Pixels per millimeter, Section thickness
Diagnostic accuracy
Sensitivity, Specificity, Area under ROC curve
Impact of diagnostic thinking
Percentage of cases in which the clinician believes that the test
alters the diagnosis
Impact of therapeutic actions
Percentage of cases in which the choice of therapies is changed
after information from the test is provided
Impact on patient outcomes
Differences in mortality, morbidity, or quality of life between patients
managed with the test and those managed without it
Impact on societal outcomes
Cost-effectiveness of the improvement in patient outcomes, such
as cost per life-year saved, calculated from a societal perspective
Questions or Thoughts