Implementation Lessons
Today’s Topics
• A story of implementation failure
• Implementation challenges
• Panel discussion
– Ashley Kay Childers
– Beth Morgan
– David Oliver
– Chris Wright
• The checklist and culture
A Story Of Implementation Failure
Urbach, D. R., Govindarajan, A., Saskin, R., Wilton, A. S., & Baxter, N. N. (2014). Introduction of surgical safety checklists in Ontario,
Canada. The New England Journal of Medicine, 370(11), 1029–38. doi:10.1056/NEJMsa1308261
• Province-wide mandate for implementation of checklists
in OR, beginning in July 2010
• Retrospective evaluation of surgical outcomes three
months before and three months after reported
implementation
• Investigators asked each site when they began using
checklist and what checklist was used
• Mortality and complications collected from pre-existing
sources
• No specific collection of data surrounding fidelity or
thoroughness of implementation beyond reports to
ministry
Results
• 130 of 133 hospitals reported implementation
• Only 9 of 92 hospitals used locally modified
checklist
• Almost all hospitals reported 99 or 100%
compliance with use within 3 months of
introduction
• 109,341 operations in pre-checklist period,
106,370 in post-checklist period
• ~60% ambulatory surgery, ~20%
ophthalmologic
Patient Outcomes
• Mortality fell from 0.71% to 0.65% (p=0.07)
• In inpatient group, mortality fell from 1.71%
to 1.58% (p=0.11)
• Adjusted complications were 3.86% before
and 3.82% after (p=0.53)
Variation in Hospitals
Authors’ Conclusions
• Mandated checklist implementation did not
affect patient outcomes after surgery in
Ontario as measured in this study
• May reflect implementation process or effect
of checklist itself
A Discussion About The
Outcomes Analysis &
Implementation Process
Challenges With Implementation
• Having the time and adequate resources to do
the work
• Engaging physicians
• Educating teams about the checklist
• Using the checklist well for every patient every
time
• Keeping the checklist going
Panel Discussion:
Challenges and Solutions
•
•
•
•
Ashley Kay Childers, SCHA, Clemson University
Beth Morgan, Palmetto Baptist
David Oliver, Palmetto Richland
Chris Wright, Greenville Memorial
The Checklist And Culture
Bill Berry
Overall, 78% of responses were positive or strongly
positive, but response varied widely by hospital
83%
Teamwork in our
operating rooms
is good
We perform
practices safely in
our ORs
75%
60%
0
20
40
60
80
0
20
40
positive response
Percent100
60
80
100
We are ready for
undertaking the
initiative
Q33: I would feel safe being Q34: If I were having surgery, I would want a checklist
treated here as a
to be used
patient
93%
85%
56.2%
Perceptions of respect and leadership
depend on discipline
Physicians provide strong
leadership in the OR
0
0
20
20
40
40
60
60
80
80
100
Percent positive response100
We feel respected in the ORs in
which we work
Physician
Physician
Non-physician
Non-physician
Physician
Physician
Non-physician
Non-physician
Perception of production pressure also
depends on discipline
80
60
40
0
20
Percent positive response
100
Patient safety isn’t threatened by
production pressure in our operating rooms
Physician
Physician
Non-physician
Non-physician
…but everyone would want a checklist
used if they were the patient
80
60
40
0
20
Percent positive response
100
If I were having an operation,
I would want a surgical checklist to be used
Physician
Physician
Non-physician
Non-physician
Teamwork is associated with outcomes
we care about in surgery
✚
Feeling safe in our ORs
Better teamwork in our
operating rooms
✚
Wanting the checklist
used for themselves
- Pressure gets in the way
of patient safety
Overall, across all hospitals and disciplines,
staff perceptions improved
+11%
Relative improvement in
average agreement
between initial and follow-up surveys
Overall average weights individual hospital averages by response rate
Communication within the team
improved substantially
“In the ORs where I work, team discussions
(e.g. briefings or debriefings) are common.”
+29%
Relative improvement in
average agreement with this statement
between initial and follow-up surveys
Overall average weights individual hospital averages by response rate
Perceptual “gap” in respect between MDs and
non-MDs declined, post v pre
We Feel Respected In The ORs In Which We Work
100
90
Average Scale Score % Positive
80
70
60
Physician
50
Non-Physican
40
30
20
10
0
Pre
Post
Perceptual “gap” in leadership also improved
100
Physicians Provide Strong Leadership in The OR
90
Average Scale Score % Positive
80
70
60
Physician
50
Non Physician
40
30
20
10
0
Pre
Post
Perceptions of safety improved
“I would feel safe being treated here as a
patient.”
+3%
Relative improvement in
average agreement with this statement
between initial and follow-up surveys
Overall average weights individual hospital averages by response rate
Proper checklist use
“In the ORs where I work, the entire surgical
team always stops at all three critical points
during the procedure to read the checklist.”
59%
agreement with this statement
at the time of the follow-up survey
Overall average weights individual hospital averages by response rate
Perceptions of safe practice differ by
professional discipline
“The entire surgical team always stops at all 3 critical points during the
procedure to read the checklist.”
Professional discipline
Tech
Nurse
CRNA
Anesthesiologist
Surgeon
0
10
20
30
40
50
Average Scale Score (% Agreement)
60
70
80
90
…but most perceive that the checklist is
averting problems and complications
80
60
40
20
0
Percent positive response
100
In the ORs where I work, problems or complications
have been averted by the checklist.
Physician
Non-physician
Take Home Messages
• This work is hard and the journey isn’t over
• SC is in the forefront of learning how to do this
work well
• For every challenge someone in South Carolina
has come up with an innovative solution
• Over the course of this program culture has
improved in many SC hospitals
• Consider taking the culture survey if your hospital
hasn’t done so. Please see Lizzie Edmondson for
more information
Thank You
For More Information
www.safesurgery2015.org