Discrete Event Simulation for
Cost-Effectiveness Models
Professor Jon Karnon
Key messages
• (most) DES for HTA ≈ Microsimulation
– DES slightly more efficient
• Use DES when “not feasible” to implement model
structure as a cohort-based model
– feasibility = f(structure, analyst)
• Value of complex model structures uncertain
– more complex to implement and to review
– nested “feasible” cohort-based model?
Key messages (cont.)
Core components: Entities
Surgery
Waiting list
Need new hip
Hospital discharge
Entities
Develop Arthritis
Core components: Attributes
Surgery
Waiting list
Need new hip
Develop Arthritis
Hospital discharge
Attributes: age, sex, arthritic
severity, pain, mobility, social
circumstances, psychological
distress, quality of life, costs.
Core components: Events
Surgery
Waiting list
Need new hip
Hospital discharge
Events
Develop Arthritis
Core components: Queues and Resources
Surgery
Waiting list
Need new hip
Hospital discharge
Queue
Resources: surgical capacity,
e.g. available daily episodes
Develop Arthritis
DES study types
• Constrained resource models
– Handful cost-effectiveness models
• Individual interaction (agent-based) models
– Vaccines…
• Non-constrained & non-interaction models
– Non-trivial minority
DES for (most) cost-effectiveness models
•
•
•
•
•
Entities
Attributes
Events
Resources
Queues
Time: Cohort state transition model
Year 0
Year 1
Year 2
Well
Ill
Dead
Well
Ill
Dead
Well
Ill
Dead
Time: Microsimulation model
Year 0
Year 1
Year 2
Well
Ill
Dead
Well
Ill
Dead
Well
Ill
Dead
Time: DES model
Well
Move to Ill after 1.5yrs
Stay in well
Move to Ill after 0.4yrs
Move to dead after 0.8 yrs
Move to dead after 1.9yrs
Ill
Stay in Ill
Move to dead
after 0.3yrs
Dead
Time to Event option 1
Well to Ill
Well to Dead
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
1
2
3
4
5
0
1
2
3
4
samples value 0.8 for Well to Ill (3yrs), 0.24 for Well to Dead
(1.9yrs)
– So moves to dead after 1.9yrs
Competing risks
– Dead is censoring event for ‘Well to Ill’
– Ill is censoring event for ‘Well to Dead’
5
Time to Event option 2
Well to Ill or Dead
pr(Ill, not Dead)
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
1
2
3
4
5
0
1
2
3
4
samples value 0.69 for Well to Ill or Dead (1.9yrs), 0.9
for pr(Ill, not Dead) (>0.77)
– So moves to dead after 1.9yrs
No competing risks
5
When to use DES?
1.
2.
3.
4.
Baseline heterogeneity
Continuous disease markers
Time varying transition probabilities
Transition probabilities = f(events experienced in
model)
Non-linear effects of baseline factors (1)
Age
Proportion
Pr(Event)
50
0.333
0.05
60
0.333
0.1
70
0.333
0.3
• Mean age = 60 years, Pr(Event) = 0.1;
Weighted mean Pr(Event) = 0.15
– Incorrect to apply 0.1 or 0.15 to single cohort of
patients mean age 60 years
Separate cohort models
Age 50yrs
Age 60yrs
Age 70yrs
Well
Well
Well
Ill
Ill
Ill
Dead
Dead
Dead
Non-linear effects of baseline factors (2)
• Chronic osteoarthritis pain model:
– age, gender, primary osteoarthritis site, sleep problem scores,
history of high potency opiate use
– too many combinations to run separate cohort models
• Value?
– robust survival model to predict times to events?
– assess extent of non-linear and interaction effects
Continuous disease markers
HbA1c attribute
HbA1c at time 0
Pr(Event)
HbA1c at time 1
Pr(Event)
HbA1c at time 2
Pr(Event)
Cohort approach (1)
HbA1c <6,
no event
HbA1c 6-7,
no event
HbA1c >7,
no event
HbA1c <6,
event
HbA1c 6-7,
event
HbA1c >7,
event
HbA1c <6,
no event
HbA1c 6-7,
no event
HbA1c >7,
no event
HbA1c <6,
event
HbA1c 6-7,
event
HbA1c >7,
event
Cohort approach (2)
no event
event
no event
event
Continuous disease markers
• Diabetes
– monitor HbA1c; clinical events = f(disease marker)
• Value?
– robust prediction of disease markers and clinical events as
f(disease marker)
Time varying transition probabilities
• Event A to Event B
– Pr(Event B, Yr1) = 0.1
– Pr(Event B, Yr2) = 0.05
– Pr(Event B, Yr3) = 0.02
Event A yr1
Event B
Event A yr2
Event B
Event A yr3
Event B
• Tunnel states
– On it’s own, time varying rarely necessitates DES
Model events = f(model events)
• Diabetes: UKPDS Outcomes Model 2
– MI = f(amputation, heart failure, IHD, stroke)
– Stroke = f(amputation, IHD); Etc.
•
no MI
no Amp
no HF
Value?
no Stroke
no IHD
MI
no Amp
no HF
no Stroke
no IHD
MI
Amp
no HF
no Stroke
no IHD
Etc.
Overall Value?
• Model validity
– Face validity
• Emphasise representation of important structural aspects
– Internal validity
• Accurate model implementation
– External validity
• More complex models more externally valid?
• Limited evidence of different ICERs
PBAC guidelines v5
• Submit nested cohort-based model alongside individual
level models…
– to demonstrate need for more complex model
– if similar results, focus review on simple model
Summary
“[cost-effectiveness models] should be made as simple as
possible, but no simpler”
– Einstein
• No simpler than DES? Demonstrate:
1. Improved model validation
2. Evidence of robust and important differences in input values