Where’s the data?
Addressing the Cost Gap
in Healthcare Simulation Research
Ryan Brydges, PhD
sim· u· la· tion
An educational tool or device with which
the learner physically interacts to mimic
an aspect of clinical care for the
purpose of teaching or assessment
10,903 Abstracts screened
8320 excluded
2583 Articles full text screened
1598 excluded
985 Articles data abstraction in full
609 Articles
comp. no
intervention
JAMA 2011; 306:978
92 Articles
comp. nonsimulation
SiH 2012;7:308
289 Articles
comp. alt.
simulation
Med Teach 2013;35:e867
10,903 Abstracts screened
8320 excluded
2583 Articles full text screened
1598 excluded
985 Articles data abstraction in full
Simulation
609 Articles>
comp.
Nothing
no
intervention
JAMA 2011; 306:978
JAMA 2011; 306:978
SimArticles
> other
92
modalities,
comp.
nonwhere you’d
simulation
expect
SiH 2012;7:308
Evidence
for
289
Articles
several
comp. Sim
alt.
Design
simulation
features
Med Teach 2013;35:e867
Largely one-off
studies, little
programmatic research
Weaker outcomes,
causal link to clinical
context largely absent
Brydges et al., 2015, Academic Medicine
Clinician
Behaviours
r = 0.51
(95% CI, 0.38 – 0.62)
N=27 studies
Direct Observation
r = 0.61
Measuring
Clinician
Behaviour
Rotation Grades
r = 0.27
OSATS/GOALS/FLS
r = 0.68 – 0.85
Measuring
Simulation
Outcomes
All Studies
r = 0.51
Lower correlations with
time and patient
outcomes
Correlation improved
with specific
conditions and specific
tools
Zendejas et al., 2013, Surgery
Cost Reporting
Sim cost only, 5%
Cost for both arms, 2%
Nothing, 93%
Recommend accounting
of Levin’s cost
components
Increased efforts towards
cost-effectiveness
analyses
Levin et al., 2001, “Waiting for Godot: Cost-effectiveness analysis in education”
Isaranuwatchai et al., 2014, AHSE
Implementation vs. total
start up costs
Cost-effectiveness:
 Unit of cost
 Unit of learning effect
“Willingness to pay”
Cost-effectiveness
analysis is more than
simple accounting
Concepts of educational
& cost effectiveness and
their integration
HILLIPS
ANALYSIS
P
ROI
KIRKPATRICK’S
ANALYSIS
Results
Impact
Learning
Satisfaction
Did the training
investment pay off?
Choosing outcomes with highquality validity evidence in
BOTH contexts
Rigorous planning of
accounting – every last detail
Collaboration,
collaboration,
collaboration
Where’s the data?
Addressing the Cost Gap
in Healthcare Simulation Research
[email protected]
http://www.nwdesignworks.com/wpcontent/themes/striking/cache/images/cost_value_1-960x400.jpg
https://www.asigovernment.com/files/images/earnedValueManagement.png
https://www.google.ca/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja
&uact=8&ved=0CAcQjRw&url=http%3A%2F%2Fwww.youtube.com%2Fwatch%
3Fv%3DbUay9DV__G0&ei=N1oAVeGXKouzggS9YCIDw&bvm=bv.87611401,d.cWc&psig=AFQjCNEgTBGgt0pTbcBiDE0XIH8Kq
Pm7XA&ust=1426172804657058