Editorial
Annals of Internal Medicine
Strategies to Improve Patient Safety: The Evidence Base Matures
I
n 2000, the Agency for Healthcare Research and Quality
commissioned a report titled, “Making Health Care
Safer: A Critical Analysis of Patient Safety Practices.” The
report analyzed and rated nearly 80 patient safety strategies
(PSSs) (1). It was heralded by many but also generated
controversy about the role of evidence in assessing the
value of PSSs (2). Since its publication, regulators, accreditors, and payers have pushed health care organizations to
adopt various “safe practices” and avoid adverse events considered largely preventable (3). Partly as a result, health
care provider organizations are striving to improve patient
safety as never before.
When “Making Health Care Safer: A Critical Analysis
of Patient Safety Practices” was published, the science supporting PSSs was immature. There was inadequate evidence to recommend interventions and how to implement
them and limited methods to measure the effect of safety
interventions. In the face of such limitations, several national programs, such as those to prevent wrong-site surgery and to implement medication reconciliation, were disseminated on the basis of face validity alone.
During the past decade, clinicians, researchers, and
policymakers gained a greater understanding of the epidemiology of errors and preventable harms. The burden is
larger than previously thought. Although we do not know
exactly how many patients experience preventable harm,
we know that, for example, 44 000 to 80 000 patients die
each year in the United States of diagnostic errors, 68 000
of decubitus ulcers, and many thousands of teamwork and
communication errors and failure to receive evidence-based
interventions (4, 5). We also learned that implementing
PSSs aimed at certain targets (for example, reducing health
care–associated infections and venous thromboembolism)
can substantially reduce errors and harm (6, 7).
Unfortunately, recent data indicate that the degree of
success in eradicating preventable harm has not matched
the investment in effort and financial resources. Studies
that examined some practices that had tremendous intuitive appeal, such as reducing resident duty hours and implementing rapid-response teams, yielded conflicting results (8, 9). Examples of unintended consequences of PSSs
emerged (10), and successful implementation was found in
some cases to be highly context-dependent (11). Three recent U.S. studies showed continuing high rates of preventable harm in hospitals (12–14)— one showed evidence of
no improvement in adverse event rates from 2003 to 2008
(12).
Against this backdrop, the Agency for Healthcare Research and Quality commissioned a team led by investigators at RAND Health; Stanford University; the University
of California, San Francisco; and Johns Hopkins University to reexamine the evidence behind key PSSs. Reexamination involved several systematic reviews that addressed
350 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 1)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
the effectiveness of particular practices, paying attention to
the importance of implementation, context, and any unintended consequences of safety interventions.
In a special supplement that accompanies this issue,
we present the evidence reviews underpinning 10 of the 41
PSSs studied in the new report. These strategies include
interventions to reduce diagnostic errors (15), in-facility
falls (16), pressure ulcers (17), and delirium (18); efforts
initiated in hospitals to improve care transitions (19) and
medication reconciliation (20); interventions in inpatient
settings to promote a patient safety culture or climate (21);
implementation of rapid-response systems (22); examination of the effect of nurse–patient staffing on patient outcomes (23); and use of simulation exercises to improve
patient safety (24). The supplement also includes an overview article that describes the entire reexamination process
and identifies 10 strongly encouraged and 12 encouraged
PSSs that are ready for adoption now (25).
In reviewing this literature, we found evidence of
progress in bringing science to the field of patient safety.
The evidence base about the effectiveness of interventions
to reduce harms grew steadily. For example, we now have
strong evidence that safety interventions have resulted in a
national reduction in 1 type of harm: central line–associated
bloodstream infections in intensive care units (6).
Although still imperfect, measures of harm have improved. Guidelines that inform the design and description
of patient safety intervention studies are available, and the
importance of context in implementing interventions is
more widely appreciated (11). Researchers recognize that
patient safety is a legitimate field of scholarly endeavor—
worthy of career focus, requiring formal training, and providing a path for academic success—although there is a
dearth of support for training programs, and applied research remains below basic research in the academic pecking order. Federal support for patient safety research has
improved, although it will need to increase even more to
meet needs.
Physicians and other health care professionals, professional societies, medical boards, and accreditation bodies
have focused efforts to reduce preventable harms. Practicing clinicians increasingly see patient safety as something
that they do rather than something that is done to them. In
the early years after “Making Health Care Safer: A Critical
Analysis of Patient Safety Practices” was published, most
PSSs were driven by external forces, such as accreditors or
insurers. Now, many of these efforts are driven by professional norms in which patient harms are viewed as a social
problem that physicians, working with others, are capable
of solving.
Although substantial gaps in the evidence base remain,
more than enough evidence exists to prompt decisive action. For example, strong evidence shows that multicomwww.annals.org
Strategies to Improve Patient Safety
ponent interventions aimed at certain safety targets, such as
prevention of falls and pressure ulcers, can significantly
reduce harm. All hospitals should implement checklistbased initiatives to prevent central line infections and have
programs aimed at improving safety culture. Certain
themes underlie successful PSSs, including the development of a motivated, trained, and resourced interdisciplinary team (16). Such teams can convert desired interventions into checklists or other system-based tools that
promote desired behaviors, focusing on the interventions
with the strongest risk reduction and lowest risks.
Researchers will note the need for additional study of
many PSSs, such as interventions for care transitions and
medication reconciliation and how best to engage patients
and families in improving safety. Research is also needed to
develop better measures of harm and context. Furthermore, we need additional studies to identify the best models for training, organizing a safety program, integrating
systems engineering approaches into clinical environments,
and taking full advantage of information technology while
avoiding unintended negative consequences. To accomplish this goal, patient safety research will need to move
from interdisciplinary (diverse researchers working on diverse problems) to multidisciplinary (diverse researchers
working on common problems while maintaining their
own conceptual models) to transdisciplinary (diverse researchers working on common problems by using common
theories) methods.
A decade ago, our early enthusiasm for patient safety
was accompanied by a hope, and some magical thinking,
that finding solutions to medical errors would be relatively
straightforward. It was believed that by simply adopting
some techniques drawn from aviation and other “safe industries,” building strong information technology systems,
and improving safety culture, patients would immediately
be safer in hospitals and clinics everywhere. We now appreciate the naivety of this point of view. Making patients
safe requires ongoing efforts to improve practices, training,
information technology, and culture. It requires that senior
leaders supply resources and leadership while simultaneously promoting engagement and innovation by frontline
clinicians. It will depend on a strong policy environment
that creates appropriate incentives for safety while avoiding
an overly rigid, prescriptive atmosphere that could sap providers’ enthusiasm and creativity.
Although we have become more sophisticated about
the challenges of keeping patients safe over the past decade,
the fundamentals have not changed. We need competent,
well-trained providers equipped with high-quality evidence
and working with talented, strong leaders using welldesigned and integrated technologies and sound policies.
We hope that the evidence reviews published in the supplement contribute to those efforts by identifying PSSs
that will help keep patients safe.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Editorial
Robert M. Wachter, MD
University of California, San Francisco
San Francisco, California
Peter J. Pronovost, MD, PhD
Johns Hopkins Medicine Patient Safety and Quality
Baltimore, Maryland
Paul G. Shekelle, MD, PhD
West Los Angeles Veterans Affairs Medical Center and
RAND Corporation
Santa Monica, California
Disclaimer: The authors declare that no competing financial interests
exist. All statements expressed in this work are those of the authors and
should not in any way be construed as official opinions or positions of
the University of California, San Francisco; Johns Hopkins University;
RAND Corporation; U.S. Department of Veterans Affairs; Agency for
Healthcare Research and Quality; or U.S. Department of Health and
Human Services.
Financial Support: This work was supported by funding from the Agency
for Healthcare Research and Quality, U.S. Department of Health and Human Services (contract HHSA-290-2007-10062I).
Potential Conflicts of Interest: Disclosures can be viewed at www
.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum⫽M12
-2570.
Corresponding Author: Robert M. Wachter, MD, Room M-994, University of California, San Francisco, 505 Parnassus Avenue, San Francisco, CA 94143; e-mail, [email protected].
Current author addresses are available at www.annals.org.
Ann Intern Med. 2013;158:350-352.
References
1. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ.
Making health care safer: a critical analysis of patient safety practices. Evid Rep
Technol Assess (Summ). 2001:i-x, 1-668. [PMID: 11510252]
2. Leape LL, Berwick DM, Bates DW. What practices will most improve safety?
Evidence-based medicine meets patient safety [Editorial]. JAMA. 2002;288:
501-7. [PMID: 12132984]
3. Wachter RM. Patient safety at ten: unmistakable progress, troubling gaps.
Health Aff (Millwood). 2010;29:165-73. [PMID: 19952010]
4. U.S. Department of Health and Human Services. The Surgeon General’s
Call to Action to Prevent Deep Vein Thrombosis and Pulmonary Embolism.
Washington, DC: U.S. Department of Health and Human Services; 2008.
5. Newman-Toker DE, Pronovost PJ. Diagnostic errors—the next frontier for
patient safety. JAMA. 2009;301:1060-2. [PMID: 19278949]
6. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, et
al. An intervention to decrease catheter-related bloodstream infections in the
ICU. N Engl J Med. 2006;355:2725-32. [PMID: 17192537]
7. Streiff MB, Carolan HT, Hobson DB, Kraus PS, Holzmueller CG, Demski R,
et al. Lessons from the Johns Hopkins Multi-Disciplinary Venous Thromboembolism (VTE) Prevention Collaborative. BMJ. 2012;344:e3935. [PMID: 22718994]
8. Shetty KD, Bhattacharya J. Changes in hospital mortality associated with
residency work-hour regulations. Ann Intern Med. 2007;147:73-80. [PMID:
17548403]
9. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams:
a systematic review and meta-analysis. Arch Intern Med. 2010;170:18-26.
[PMID: 20065195]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 1) 351
Editorial
Strategies to Improve Patient Safety
10. Sittig DF, Singh H. Defining health information technology-related errors:
new developments since to err is human. Arch Intern Med. 2011;171:1281-4.
[PMID: 21788544]
11. Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, et al.
Advancing the science of patient safety. Ann Intern Med. 2011;154:693-6.
[PMID: 21576538]
12. Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA,
Sharek PJ. Temporal trends in rates of patient harm resulting from medical care.
N Engl J Med. 2010;363:2124-34. [PMID: 21105794]
13. Classen DC, Resar R, Griffin F, Federico F, Frankel T, Kimmel N, et al.
‘Global trigger tool’ shows that adverse events in hospitals may be ten times
greater than previously measured. Health Aff (Millwood). 2011;30:581-9.
[PMID: 21471476]
14. Levinson DR; Department of Health and Human Services. Adverse Events in
Hospitals: National Incidence Among Medicare Beneficiaries. November 2010.
15. McDonald KM, Matesic B, Contopoulos-Ioannidis DG, Lonhart J,
Schmidt E, Pineda N, et al. Patient safety strategies targeted at diagnostic errors.
A systematic review. Ann Intern Med. 2013;158:381-9.
16. Miake-Lye IM, Hempel S, Ganz DA, Shekelle PG. Inpatient fall prevention
programs as a patient safety strategy. A systematic review. Ann Intern Med.
2013;158:390-6.
17. Sullivan N, Schoelles KM. Preventing in-facility pressure ulcers as a patient
safety strategy. A systematic review. Ann Intern Med. 2013;158:410-6.
18. Reston JT, Schoelles KM. In-facility delirium programs as a patient safety
strategy. A systematic review. Ann Intern Med. 2013;158:375-80.
19. Rennke S, Nguyen OK, Shoeb MH, Magan Y, Wachter RM, Ranji SR.
Hospital-initiated transitional care interventions as a patient safety strategy. A
systematic review. Ann Intern Med. 2013;158:432-40.
20. Kwan JL, Lo L, Sampson M, Shojania KG. Medication reconciliation during transitions of care as a patient safety strategy. A systematic review. Ann Intern
Med. 2013;158:397-403.
21. Weaver SJ, Lubomski LH, Wilson RF, Pfoh ER, Martinez KA, Dy SM.
Promoting a culture of safety as a patient safety strategy. A systematic review. Ann
Intern Med. 2013;158:369-74.
22. Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM. Rapidresponse systems as a patient safety strategy. A systematic review. Ann Intern
Med. 2013;158:417-25.
23. Shekelle PG. Nurse–patient ratios as a patient safety strategy. A systematic
review. Ann Intern Med. 2013;158:404-9.
24. Schmidt E, Goldhaber-Fiebert SN, Ho LA, McDonald KM. Simulation
exercises as a patient safety strategy. A systematic review. Ann Intern Med. 2013;
158:425-31.
25. Shekelle PG, Pronovost PJ, Wachter RM, McDonald KM, Schoelles K, Dy
SM, et al. The top patient safety strategies that can be encouraged for adoption
now. Ann Intern Med. 2013;158:365-8.
Congratulations to Octavian Toma, MD, DESA, winner of the 2012 Annals Personae prize. Dr. Toma’s photogragh was published on the cover of the 4 December 2012 issue (vol. 157, no. 11) and is reprinted below.
For more information on the Annals Personae prize and to view a list of past winners, go to www.annals.org/public
/personaephotographyprize.aspx.
352 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 1)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Annals of Internal Medicine
Current Author Addresses: Dr. Wachter: Room M-994, University of
California, San Francisco, 505 Parnassus Avenue, San Francisco, CA
94143.
Dr. Pronovost: 600 North Wolfe Street, Meyer 295, Baltimore, MD
21287-7294.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Dr. Shekelle: RAND Corporation, 1776 Main Street, PO Box 2138,
Santa Monica, CA 90407-2138.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 1) W-173
Editorial
Annals of Internal Medicine
Patient Safety Strategies: A Call for Physician Leadership
T
he American health care enterprise, by far the most
expensive in the world, continues to deliver, on average, a mediocre product (1). At its best, health care in the
United States can be superb. It is fraught, however, with
uneven quality, extremely expensive interventions, and an
environment that continues to be remarkably unsafe for
the patients it serves (2).
Work highlighted in the supplement on patient safety
strategies that accompanies this issue (3) shows that there
has been some progress since the publication in 2000 of
the landmark Institute of Medicine report “To Err is Human: Building a Safer Health System” (4). A project team
(supported by the Agency of Healthcare Research and
Quality, from the RAND Corporation; Stanford University; the University of California, San Francisco; Johns
Hopkins University; and ECRI Institute) examined the evidence base underpinning particular patient safety practices. With the help of an international panel of experts,
they created a framework for reviewing studies about patient safety strategies. The group scrutinized 158 patient
safety topics and selected 41 for either an in-depth systematic review (18 topics) or a brief review (23 topics) that
focused on emerging data or new insights about implementing the strategy. Reviews entailed current literature
searches; a priori– determined selection criteria; assessments
of the quality of studies of safety interventions; and evaluation of context, implementation, and adoption issues. On
the basis of the reviews, the expert panel rated the strength
of evidence for each safety strategy: They recommended
that 10 patient safety strategies should be “strongly encouraged” for adoption and 12 strategies should be “encouraged” for adoption.
The methods and findings of this review process are
thoughtfully constructed and should be useful in future
evaluations of evidence about strategies. In many cases,
further evidence and validation are needed before the patient safety intervention can be accepted as proven.
Regardless, the analysis raises questions about why it
has been so difficult over the past 12 years to obtain meaningful safety data and more substantially improve patient
safety. The report reveals important clues and possible
remedies. Physicians have been notoriously unwilling to
consider treatment protocols, checklists, and, more recently, “bundles” that describe approaches to patient care
(5). These have been criticized and labeled as “cookbook
medicine,” while physicians argued for tailored care, emphasizing the uniqueness of every patient. The data that
were reviewed, however, clearly demonstrate that checklists
and bundles can substantially improve patient safety and
quality of care. In training and in practice, it is essential
that health care professionals understand that bundles of
care provide a touchstone on which reproducible quality
can be achieved while making the modifications that may
be required for each patient. Some electronic health records have begun to incorporate this kind of information
for the practitioner.
The enormous variability of care provided is a major
challenge to the identification of evidence-based, safe care.
Increasingly, it is important that physicians in each hospital and practice identify the steps that they will take in the
diagnosis and care of the most common clinical problems
they confront and follow these protocols with appropriate
individual variation. Sometimes, they can build on work of
professional societies and other organizations that provide
such guidelines, but there is nothing to prevent the orthopedic surgeons in a given hospital from agreeing on the
fundamental approaches that they will take in the care of a
patient requiring a knee replacement, which will impact
everything from prophylactic antibiotic use to protocols for
increasing physical activity. Agreement among colleagues
on a standard approach to community-acquired infection,
hospital-acquired infection, or initial treatment of hypertension would not only provide a much higher consistency
in patient care but also provide the basis for serious clinical
investigation to compare outcomes when variations in
these protocols are considered. In many cases, the evidence
is unclear about the proper protocol, but until existing
approaches are systemized, it will be very difficult to obtain
the evidence.
Another lesson from these reviews is the importance of
the system of care provided by a health care team. Some of
the most successful programs have been those that minimize the roles of the physicians and maximize those of
nursing or respiratory staff. It is ironic but often the case
that the less the physician is required to do in the course of
maintaining a bundle of care, the more likely it is that
protocols will be followed and outcomes improved. As
health care insurance coverage increases and the population
grows with increasing amounts of chronic illness, the roles
of nonphysician health professionals, including nurses and
pharmacists, to provide quality and timely care can only
expand. They are essential parts of an effective team that
requires physicians to understand team function and their
own leadership roles. Teams will be central to newer delivery models.
Hand hygiene represents another extraordinary example of the importance of physician behavior in reducing
hospital-acquired infections. Despite all of the evidence
about the germ theory of infection, many physicians somehow believed that their hands and stethoscopes were immune from the transmission of these organisms. Creating
situations in which physicians demonstrate the key role
they have as models for other members of the health care
team is critical. Once again, the physician must be a leader.
We are very proud of the health sciences research enterprise, which has produced remarkable results in improv© 2013 American College of Physicians 353
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Editorial
Physician Safety Strategies
ing outcomes in many aspects of disease, such as cancer
and congestive heart failure. However, it is remarkable
that, despite the loss of life from lapses in quality of care
and patient safety, so little money is invested in research in
this area. The project team in the supplement calls for
enhanced education and training for those interested in
health care and patient safety research. Providing adequate
funding for well-designed research in health care delivery
would be an important magnet to bring the best and the
brightest health care professionals and researchers into
these studies. To improve survival from illness through
biomedical research only to lose patients because of poor
quality of care is unacceptable.
The articles in the supplement provide important opportunities for the future. Success depends on multiplicity
of factors, including policies, reimbursement models, health
care delivery models, systems, education, and financing.
Critical to all of them is the role of the individual physician
who is fully committed to the effort and is adaptable in a
changing world in which bundles, protocols, team care,
electronic data collection, and evidence-based treatment
paradigms are undertaken. Historically, when physicians
have not responded to the key elements of health care, such
as access, cost, and quality, others have imposed solutions
on them. Quality of care remains the arena in which physician leadership can still provide direction, innovation,
and enhanced satisfaction for patients and caregivers.
Kenneth I. Shine, MD
University of Texas System
Austin, Texas
Potential Conflicts of Interest: Disclosures can be viewed at www
.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum⫽M13
-0111.
Requests for Single Reprints: Kenneth I. Shine, MD, University of
Texas System, 601 Colorado Street, Austin, TX 78701; e-mail,
[email protected].
Ann Intern Med. 2013;158:353-354.
References
1. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al.
The quality of health care delivered to adults in the United States. N Engl J Med.
2003;348:2635-45. [PMID: 12826639]
2. Mehtsun WT, Ibrahim AM, Diener-West M, Pronovost PJ, Makary MA.
Surgical never events in the United States. Surgery. 2012. [PMID: 23257079]
3. Shekelle PG, Pronovost PJ, Wachter RM, McDonald KM, Schoelles K, Dy
SM, et al. The top patient safety strategies that can be encouraged for adoption
now. Ann Intern Med. 2013;158:365-8.
4. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a
Safer Health System. Washington, DC: National Academies Pr; 2000.
5. Gawande A. The Checklist Manifesto: How to Get Things Right. New York:
Metropolitan Books; 2010.
READER’S COMMENTS
Readers may comment on published articles at www.annals.org. While
this service is free to Annals subscribers, readers without subscriptions
who wish to comment on articles may purchase temporary access.
354 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 1)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Supplement
Annals of Internal Medicine
The Top Patient Safety Strategies That Can Be Encouraged for
Adoption Now
Paul G. Shekelle, MD, PhD; Peter J. Pronovost, MD, PhD; Robert M. Wachter, MD; Kathryn M. McDonald, MM; Karen Schoelles, MD, SM;
Sydney M. Dy, MD, MSc; Kaveh Shojania, MD; James T. Reston, PhD, MPH; Alyce S. Adams, PhD; Peter B. Angood, MD;
David W. Bates, MD, MSc; Leonard Bickman, PhD; Pascale Carayon, PhD; Sir Liam Donaldson, MBChB, MSc, MD; Naihua Duan, PhD;
Donna O. Farley, PhD, MPH; Trisha Greenhalgh, BM BCH; John L. Haughom, MD; Eileen Lake, PhD, RN; Richard Lilford, PhD;
Kathleen N. Lohr, PhD, MA, MPhil; Gregg S. Meyer, MD, MSc; Marlene R. Miller, MD, MSc; Duncan V. Neuhauser, PhD, MBA, MHA;
Gery Ryan, PhD; Sanjay Saint, MD, MPH; Stephen M. Shortell, PhD, MPH, MBA; David P. Stevens, MD; and Kieran Walshe, PhD
O
ver the past 12 years, since the publication of the
Institute of Medicine’s report, “To Err is Human:
Building a Safer Health System,” improving patient safety
has been the focus of considerable public and professional
interest. Although such efforts required changes in policies;
education; workforce; and health care financing, organization, and delivery, the most important gap has arguably
been in research. Specifically, to improve patient safety we
needed to identify hazards, determine how to measure
them accurately, and identify solutions that work to reduce
patient harm. A 2001 report commissioned by the Agency
for Healthcare Research and Quality, “Making Health
Care Safer: A Critical Analysis of Patient Safety Practices”
(1), helped identify some early evidence-based safety practices, but it also highlighted an enormous gap between
what was known and what needed to be known.
For the past 4 years, with support from the Agency for
Healthcare Research and Quality, our group (a project
team from the RAND Corporation; Stanford University;
the University of California, San Francisco; Johns Hopkins
University; and ECRI Institute) and an international panel
of 21 stakeholders and evaluation methods experts conducted an evidence-based assessment of patient safety strategies (PSSs). Our efforts involved 3 phases. In the first
phase, we developed a framework for reviewing existing
studies and prospectively evaluating new PSS implementation studies (2). This framework identified several key
points about the importance of theory, context, and implementation (Table 1) (2).
The second phase was a review of current patient
safety strategies. We started with the 79 topics in Making
Health Care Safer and added practices from the National
Quality Forum’s 2010 update, the Joint Commission, and
the Leapfrog Group; those we identified in an initial scoping search; and those suggested by experts. From this list of
158 potential topics, we used several rounds of voting with
our stakeholders to narrow the scope to 41 PSSs that the
expert panel judged to be most important to the largest
audience. Given limited time and resources, we prioritized
topics as needing either a traditional systematic review or
only a “brief review.” The latter generally focused on a
specific aspect of the PSS, such as emerging data or new
insights about implementation.
Ann Intern Med. 2013;158:365-368.
For author affiliations, see end of text.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
We chose 18 topics for in-depth reviews. As a first step
for the reviews, we searched for existing relevant systematic
reviews. To assess the potential utility of such reviews, we
followed procedures proposed by Whitlock and colleagues
(3) and asked the following questions: Is the existing review sufficiently “on topic” to be of use? Is the review of
sufficient quality to foster confidence in the results? If we
determined that the existing systematic review was sufficiently on topic and of acceptable quality, we took 1 of 2
further steps. In some cases, we did an “update” search
(that is, we searched databases for all new relevant evidence
published since the search end date in the existing systematic review); in others, we conducted searches for “signals
for updating.” Such searches generally followed the criteria
proposed by Shojania and colleagues (4), which involved a
search of high-yield databases and journals for pivotal studies that could signal that a systematic review is out of date.
A pivotal study is one that may call into question the results of a previous systematic review. We added any evidence identified in either the update search or signals
search to the evidence base from the existing systematic
review. Some PSSs had no existing systematic reviews and
others had previous reviews that were not of sufficient relevance or quality to be used. In those situations, we conducted new searches using existing guidance (5).
Evidence about context, implementation, and adoption was a key focus of our reviews. We searched for evidence on these aspects of primary studies in 2 ways. First,
we sought and extracted data about context, implementation, and unintended harms from articles that evaluated
the effectiveness of PSSs. Second, we identified “implementation studies” from our literature searches. These
studies focus on the implementation processes, particularly
elements demonstrated or hypothesized to be of special
importance for the success, or lack of success, of the intervention. To be eligible, implementation studies needed to
See also:
Web-Only
CME quiz (Professional Responsibility Credit)
Annals of Internal Medicine
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 365
Supplement
The Top Patient Safety Strategies That Can Be Encouraged for Adoption Now
Table 1. Recommendations for Evaluating the Effectiveness
of Patient Safety Strategies and High-Priority Contexts to
Include in Reports of Patient Safety Research*
Recommendations for evaluating the effectiveness of patient safety
strategies
Explicitly describe the theory behind the chosen intervention components
or an explicit logic model for why this patient safety practice should
work
Describe the patient safety practice in sufficient detail so it can be
replicated, including the expected effect on staff roles
Measure high-priority contexts in the 4 domains described below
Detail the implementation process, the actual effects on staff roles, and
how the implementation or intervention changed over time
Assess the effect of the patient safety practice on outcomes and possible
unexpected effects, including data on costs, when available
For studies with multiple intervention sites, assess the influence of context
on the effectiveness of the intervention and implementation
High-priority contexts to include in reports of patient safety research
External factors, such as regulatory requirements, public reporting, or
pay-for-performance, and local sentinel events
Organization structural characteristics, such as size, complexity, and
financial status or strength
Teamwork, leadership, and patient safety culture
Management tools, such as training resources, internal organization
incentives, audit and feedback, and quality improvement consultants
* From reference 2.
either report or be linked to reports of effectiveness
outcomes.
The 23 brief reviews were explicitly designed not to be
full systematic reviews or updates. The goals of each brief
review varied by PSSs, according to needs identified by
technical experts and stakeholders. The brief review could
focus primarily on information about the effectiveness of
an emerging PSS or implementation of an established PSS.
Alternatively, the review could explore whether new evidence calls into question the effectiveness of an existing
PSS or identifies unintended consequences of safety interventions. In general, a content expert on the topic, working
with the project team, conducted the brief reviews. The
methods involved focused literature searches for evidence
relevant to the specific need. Typically, the author narratively summarized the evidence in a format tailored to the
particular goal of the brief review.
We used standard instruments, such as the Cochrane
Effective Practice and Organisation of Care criteria (6), the
U.S. Preventive Services Task Force criteria (7), and the
Cochrane Risk of Bias criteria (8), to assess the quality or
risk of bias for individual studies of safety interventions.
We developed criteria to evaluate strength of evidence
across studies of effectiveness (9) that were informed by
existing methods (10, 11) and incorporated criteria about
the use of theory and description of implementation.
All of the reviews can be found in the Agency for Healthcare Research and Quality evidence report, “Making Health
Care Safer II: An Updated Critical Analysis of the Evidence
for Patient Safety Practices” (9). In this supplement issue, we
present the reviews for 10 PSSs. In an upcoming issue of BMJ
366 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Quality & Safety, we will present several more. A summary of
the evidence for all 41 PSSs is available in Table 1 of Chapter
44 in that report (9). It categorizes each PSS according to the
following: the scope of the underlying problem that the PSS
addresses (its frequency and severity); the strength of evidence
about the effectiveness of the safety strategy; the evidence or
potential for harmful consequences of the strategy; a rough
estimate of the cost of implementing the strategy (low, medium, or high); and an assessment of the difficulty of implementing the strategy.
In the last phase of our effort, the expert panel explicitly considered the strength and quality of evidence about
effectiveness and implementation for each PSS and concluded that 22 PSSs are ready to be encouraged for adoption by health care providers (Table 2). The first 10 are
those that the expert panel believed should be “strongly
encouraged” for adoption. The remaining 12 are ones they
“encouraged” for adoption. Future implementation and
evaluation will further our understanding of how best to
implement these 22 practices to make them most effective
and help health care organizations become learning health
care systems. In the meantime, our expert panel believes
that providers should not delay adopting these practices,
Table 2. Patient Safety Strategies Ready for Adoption Now
Strongly encouraged
Preoperative checklists and anesthesia checklists to prevent operative and
postoperative events
Bundles that include checklists to prevent central line–associated
bloodstream infections
Interventions to reduce urinary catheter use, including catheter reminders,
stop orders, or nurse-initiated removal protocols
Bundles that include head-of-bed elevation, sedation vacations, oral care
with chlorhexidine, and subglottic suctioning endotracheal tubes to
prevent ventilator-associated pneumonia
Hand hygiene
The do-not-use list for hazardous abbreviations
Multicomponent interventions to reduce pressure ulcers
Barrier precautions to prevent health care–associated infections
Use of real-time ultrasonography for central line placement
Interventions to improve prophylaxis for venous thromboembolisms
Encouraged
Multicomponent interventions to reduce falls
Use of clinical pharmacists to reduce adverse drug events
Documentation of patient preferences for life-sustaining treatment
Obtaining informed consent to improve patients’ understanding of the
potential risks of procedures
Team training
Medication reconciliation
Practices to reduce radiation exposure from fluoroscopy and CT
The use of surgical outcome measurements and report cards, such as
those from ACS NSQIP
Rapid-response systems
Use of complementary methods for detecting adverse events or medical
errors to monitor for patient safety problems
Computerized provider order entry
Use of simulation exercises in patient safety efforts
ACS ⫽ American College of Surgeons; CT ⫽ computed tomography; NSQIP ⫽
National Surgical Quality Improvement Program.
www.annals.org
The Top Patient Safety Strategies That Can Be Encouraged for Adoption Now
particularly the strongly encouraged ones. Enough is
known now to permit health care systems to move ahead.
From the RAND Corporation, Santa Monica, Veterans Affairs Greater
Los Angeles Healthcare System, Los Angeles, University of California,
San Francisco, San Francisco, Stanford Center for Health Policy and
Center for Primary Care and Outcomes Research, Stanford, Kaiser Permanente, Oakland, and University of California, Berkeley, Berkeley,
California; Johns Hopkins Medicine Patient Safety and Quality, Johns
Hopkins University, and John’s Hopkins Children’s Center, Baltimore,
Maryland; ECRI Institute, Plymouth Meeting, RAND Corporation,
Pittsburgh, and University of Pennsylvania, Philadelphia, Pennsylvania;
Centre for Patient Safety, University of Toronto, Ottawa, Ontario, Canada; National Quality Forum, Washington, DC; Harvard University,
Brigham and Women’s Hospital, Boston, Massachusetts; Vanderbilt
University’s Peabody College, Nashville, Tennessee; University of
Wisconsin-Madison, Madison, Wisconsin; Imperial College London and
Queen Mary, University of London, London, University of Birmingham, Edgbaston, Birmingham, and Manchester Business School, University of Manchester, Manchester, United Kingdom; New York State
Psychiatric Institute, New York, New York; PeaceHealth Medical
Group, Eugene, Oregon; Research Triangle Institute International, Research Triangle Park, North Carolina; Dartmouth Institute for Health
Policy and Clinical Practice, Lebanon, North Hampshire; Case Western
Reserve University, Cleveland, Ohio; and Veterans Affairs Ann Arbor
Healthcare System and University of Michigan, Ann Arbor, Michigan.
Note: The Agency for Healthcare Research and Quality reviewed con-
tract deliverables to ensure adherence to contract requirements and quality, and a copyright release was obtained from the Agency for Healthcare
Research and Quality before submission of the manuscript.
Disclaimer: All statements expressed in this work are those of the authors
and should not be construed as official opinions or positions of the
organizations where any of the authors are employed, the Agency for
Healthcare Research and Quality, the U.S. Department of Health and
Human Services, or the U.S. Department of Veterans Affairs.
Acknowledgment: The authors thank Aneesa Motala, BA.
Financial Support: From the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services (contract HHSA290-2007-10062I). Dr. Lilford was supported by the National Institute
of Health Research Collaborations for Leadership in Applied Health
Research and Care for Birmingham and the Black Country.
Potential Conflicts of Interest: Dr. Shekelle: Consultancy: ECRI Institute;
Employment: Veterans Affairs; Grants/grants pending: Agency for Healthcare
Research and Quality (AHRQ), Veterans Affairs, Centers for Medicare &
Medicaid Services, National Institute of Nursing Research, Office of the
National Coordinator; Royalties: UpToDate. Dr. Pronovost: Board membership: Cantel Medical Group; Consultancy: Association for Professionals in
Infection Control and Epidemiology, Hospitals and Health Care Systems;
Grants/grants pending (money to institution): AHRQ, National Institutes of
Health; Payment for lectures: Leigh Bureau (speaking on quality and safety);
Royalties: Penguin Group. Dr. Wachter: Grant, support for travel to meetings,
payment for writing or reviewing the manuscript, grants/grants pending (money
to institution): AHRQ; Board membership: American Board of Internal Medicine, Salem Hospital; Payment for lectures: More than 100 health care organizations (such as hospitals, health care systems, state medical, and hospital
associations); Royalties: Lippincott, Williams & Wilkins, McGraw-Hill; Payment for development of educational presentations (money to institution): QuantiaMD, In-Patient Consulting—The Hospitalist Company; Stock/stock opwww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
tions: PatientSafe Solutions, CRISI, EarlySense; Other: John Wiley and Sons,
Marc and Lynne Benioff, United States–United Kingdom Fulbright Commission. Ms. McDonald: Grant (money to institution): AHRQ. Dr. Schoelles:
Support for travel to meetings and support of work on publication of “Making
Health Care Safer II” (money to institution): RAND Corporation (funded by
AHRQ). Dr. Dy: Grant (money to institution): AHRQ. Dr. Reston: Grant
(money to institution): AHRQ. Dr. Adams: Support for travel to meetings:
RAND Corporation. Dr. Bates: Consulting fee and support for travel to meetings: RAND Corporation; Consultancy: PatientSafe Solutions; Royalties:
Medicalis; Stock/stock options: Calgary Scientific. Dr. Bickman: Support for
travel to meetings and fees for participation in review activities: RAND Corporation. Dr. Carayon: Support for travel to meetings: RAND Corporation;
Employment: University of Wisconsin-Madison; Grants/grants pending:
AHRQ, Office of the National Coordinator; Royalties: Taylor & Francis. Dr.
Donaldson: Consulting fee and support for travel to meetings: RAND Corporation. Dr. Farley: Grant and support for travel to meetings: AHRQ; Consultancy: RAND Corporation, World Health Organization; Employment:
RAND Corporation. Dr. Greenhalgh: Consulting fee and support for travel to
meetings: RAND Corporation. Dr. Lake: Consulting fee and support for travel
to meetings: RAND Corporation. Dr. Lilford: Grant: National Institute of
Health Research Collaborations for Leadership in Applied Health Research
and Care for Birmingham and the Black Country; Consulting fee: AHRQ;
Support for travel to meetings: AHRQ. Dr. Lohr: Consulting fee: RAND Corporation. Dr. Meyer: Grant: RAND Corporation; Support for travel to meetings (money to institution): RAND Corporation; Expert testimony: Winston
Straw. Dr. Miller: Consulting fee: RAND Corporation. Dr. Neuhauser: Consulting fee and support for travel to meetings: RAND Corporation. Dr. Ryan:
Grant, consulting fee, support for travel to meetings, fees for participation of
review activities, and payment for writing or reviewing the manuscript (money to
institution): AHRQ. Dr. Saint: Consulting fee and support for travel to meetings: RAND Corporation (funded by AHRQ); Payment for lectures: Various
hospitals, academic medical centers, group-purchasing organizations (for example, Veterans Health Administration and Premier), professional societies
(for example, Society of Hospital Medicine), and nonprofit foundations (for
example, Institute for Healthcare Improvement and Michigan Health and
Hospital Association); Stock/stock options: Doximity. Dr. Shortell: Support for
travel to meetings: AHRQ. Dr. Stevens: Consulting fee and support for travel to
meetings: RAND Corporation (funded by AHRQ). All other authors have
no disclosures. Disclosures can also be viewed at www.acponline.org/authors
/icmje/ConflictOfInterestForms.do?msNum⫽M12-2931.
Requests for Single Reprints: Paul G. Shekelle, MD, PhD, RAND
Corporation, 1776 Main Street, Santa Monica, CA 90401; e-mail,
[email protected].
Current author addresses and author contributions are available at www
.annals.org.
References
1. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ.
Making health care safer: a critical analysis of patient safety practices. Evid Rep
Technol Assess (Summ). 2001:i-x, 1-668. [PMID: 11510252]
2. Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, et al.
Advancing the science of patient safety. Ann Intern Med. 2011;154:693-6.
[PMID: 21576538]
3. Whitlock EP, Lin JS, Chou R, Shekelle P, Robinson KA. Using existing
systematic reviews in complex systematic reviews. Ann Intern Med. 2008;148:
776-82. [PMID: 18490690]
4. Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How
quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med.
2007;147:224-33. [PMID: 17638714]
5. Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ publication no. 10(11)5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 367
Supplement
The Top Patient Safety Strategies That Can Be Encouraged for Adoption Now
EHC063-EF. Rockville, MD: Agency for Healthcare Research and Quality;
2011. Accessed at http://effectivehealthcare.ahrq.gov/ehc/products/60/318
/MethodsGuide_Prepublication-Draft_20120523.pdf on 20 July 2012.
6. Cochrane Effective Practice and Organisation of Care Group (EPOC) Reviews. Accessed at http://epoc.cochrane.org/epoc-reviews on 20 July 2012.
7. Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM,
et al; Methods Work Group, Third US Preventive Services Task Force. Current
methods of the US Preventive Services Task Force: a review of the process.
Am J Prev Med. 2001;20:21-35. [PMID: 11306229]
8. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al;
Cochrane Bias Methods Group. The Cochrane Collaboration’s tool for assessing
risk of bias in randomised trials. BMJ. 2011;343:d5928. [PMID: 22008217]
368 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
9. Shekelle PG, Wachter RM, Pronovost PJ, Schoelles K, McDonald KM, Dy
SM, et al. Making Health Care Safer II: An Updated Critical Analysis of the
Evidence for Patient Safety Practices. (Prepared by the Southern CaliforniaRAND Evidence-based Practice Center under contract HHSA290200710062I.)
Rockville, MD: Agency for Healthcare Research and Quality; 2013. [Forthcoming].
10. Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al.
AHRQ series paper 5: grading the strength of a body of evidence when comparing
medical interventions—agency for healthcare research and quality and the effective
health-care program. J Clin Epidemiol. 2010;63:513-23. [PMID: 19595577]
11. Grading of Recommendations Assessment, Development and Evaluation
(GRADE) Working Group. Accessed at www.gradeworkinggroup.org on 20 July
2012.
www.annals.org
Current Author Addresses: Dr. Shekelle: Veterans Affairs Greater Los
Angeles Healthcare System, 11301 Wilshire Boulevard, Los Angeles, CA
90073.
Dr. Pronovost: Johns Hopkins University School of Medicine, 1909
Thames Street, 2nd Floor, Baltimore, MD 21231.
Dr. Wachter: University of California, San Francisco, 505 Parnassus
Avenue, San Francisco, CA 94143.
Ms. McDonald: Stanford University, 117 Encina Commons, Stanford,
CA 94305-6019.
Drs. Schoelles and Reston: ECRI Institute, 5200 Butler Pike, Plymouth
Meeting, PA 19462-1298.
Dr. Dy: Johns Hopkins University, Room 609, 624 North Broadway,
Baltimore, MD 21205.
Dr. Shojania: Sunnybrook Health Sciences Centre, Room H468, 2075
Bayview Avenue, Toronto, Ontario M4N 3M5, Canada.
Dr. Adams: Kaiser Permanente, Division of Research, 2000 Broadway,
Oakland, CA 94612.
Dr. Angood: American College of Physician Executives, 400 North Ashley Drive, Suite 400, Tampa, FL 33602.
Dr. Bates: American College of Physician Executives, 400 North Ashley
Drive, Suite 400, Tampa, FL 33602.
Dr. Bickman: Center for Evaluation and Program Improvement, Vanderbilt University’s Peabody College, Peabody #151, 230 Appleton
Place, Nashville, TN 37203.
Dr. Carayon: University of Wisconsin-Madison, 3126 Engineering Centers Building, 1550 Engineering Drive, Madison, WI 53706.
Dr. Donaldson: Department of Surgery & Cancer, Division of Surgery,
Imperial College London, Room 1090a, 10th Floor, QEQM Building,
St Mary’s Hospital, Praed Street, London W2 1NY, United Kingdom.
Dr. Duan: New York State Psychiatric Institute, 1051 Riverside Drive,
Unit 48, New York, NY 10032.
Dr. Farley: RAND Corporation, 4570 5th Avenue #600, Pittsburgh, PA
15213.
Dr. Greenhalgh: Global Health, Policy and Innovation Unit, Centre
for Primary Care and Public Health, Blizard Institute, Barts and The
London School of Medicine and Dentistry, Yvonne Carter Building,
58 Turner Street, London E1 2AB, United Kingdom.
Dr. Haughom: PeaceHealth, 770 East 11th Avenue, Eugene, OR 97401.
Dr. Lake: University of Pennsylvania School of Nursing, Room 302
Fagin Hall, 418 Curie Boulevard, Philadelphia, PA 19104-4217.
Dr. Lilford: University of Birmingham, Room 110, 90 Vincent Drive,
Edgbaston, Birmingham B15 2TT, United Kingdom.
Dr. Lohr: RTI International, 3040 Cornwallis Road, PO Box 12194,
Research Triangle Park, NC 27709-2194.
Dr. Meyer: Dartmouth-Hitchcock, One Medical Center Drive, Lebanon, NH 03756.
Dr. Miller: Johns Hopkins Children’s Center, 200 North Wolfe Street,
Room 2094, Baltimore, MD 21287.
Dr. Neuhauser: Case Western Reserve University, 10900 Euclid Avenue,
Cleveland, OH 44106-4945.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Dr. Ryan: RAND Corporation, 1776 Main Street, Santa Monica, CA
90401.
Dr. Saint: Veterans Affairs Ann Arbor Healthcare System, 2215 Fuller
Road, Ann Arbor, MI 48105.
Dr. Shortell: University of California, Berkeley, 50 University Hall, MC
7360, Berkeley, CA 94720-7360.
Dr. Stevens: Dartmouth Institute for Health Policy and Clinical Practice, 30 Lafayette Street, Lebanon, NH 03766.
Dr. Walshe: University of Manchester, Booth Street West, Manchester
M15 6PB, United Kingdom.
Author Contributions: Conception and design: P.G. Shekelle, P.J. Pronovost, R.M. Wachter, K.M. McDonald, K. Schoelles, S.M. Dy, K.
Shojania, J.T. Reston, A.S. Adams, P.B. Angood, D.W. Bates, L. Bickman, P. Carayon, L. Donaldson, N. Duan, D.O. Farley, T. Greenhalgh,
J.L. Haughom, E. Lake, R. Lilford, K.N. Lohr, G.S. Meyer, M.R. Miller,
D.V. Neuhauser, G. Ryan, S. Saint, S.M. Shortell, D.P. Stevens, K.
Walshe.
Analysis and interpretation of the data: P.G. Shekelle, P.J. Pronovost,
R.M. Wachter, K.M. McDonald, K. Schoelles, S.M. Dy, K. Shojania,
J.T. Reston, A.S. Adams, P.B. Angood, D.W. Bates, L. Bickman, P.
Carayon, L. Donaldson, N. Duan, D.O. Farley, T. Greenhalgh, J.L.
Haughom, E. Lake, R. Lilford, K.N. Lohr, G.S. Meyer, M.R. Miller,
D.V. Neuhauser, G. Ryan, S. Saint, S.M. Shortell, D.P. Stevens, K.
Walshe.
Drafting of the article: P.G. Shekelle, P.J. Pronovost, R.M. Wachter.
Critical revision of the article for important intellectual content: P.G.
Shekelle, P.J. Pronovost, R.M. Wachter, K.M. McDonald, K. Schoelles,
S.M. Dy, K. Shojania, J.T. Reston, A.S. Adams, P.B. Angood, D.W.
Bates, L. Bickman, P. Carayon, L. Donaldson, N. Duan, D.O. Farley, T.
Greenhalgh, J.L. Haughom, E. Lake, R. Lilford, K.N. Lohr, G.S. Meyer,
M.R. Miller, D.V. Neuhauser, G. Ryan, S. Saint, S.M. Shortell, D.P.
Stevens, K. Walshe.
Final approval of the article: P.G. Shekelle, P.J. Pronovost, R.M. Wachter, K.M. McDonald, K. Schoelles, S.M. Dy, K. Shojania, J.T. Reston,
A.S. Adams, P.B. Angood, D.W. Bates, L. Bickman, P. Carayon, L.
Donaldson, N. Duan, D.O. Farley, T. Greenhalgh, J.L. Haughom, E.
Lake, R. Lilford, K.N. Lohr, G.S. Meyer, M.R. Miller, D.V. Neuhauser,
G. Ryan, S. Saint, S.M. Shortell, D.P. Stevens, K. Walshe.
Statistical expertise: K. Shojania, N. Duan, D.V. Neuhauser.
Obtaining of funding: P.G. Shekelle, P.J. Pronovost, R.M. Wachter,
K.M. McDonald, K. Schoelles.
Collection and assembly of data: P.G. Shekelle, P.J. Pronovost, R.M.
Wachter, K.M. McDonald, K. Schoelles, S.M. Dy, K. Shojania, J.T.
Reston, A.S. Adams, P.B. Angood, D.W. Bates, L. Bickman, P. Carayon,
L. Donaldson, N. Duan, D.O. Farley, T. Greenhalgh, J.L. Haughom, E.
Lake, R. Lilford, K.N. Lohr, G.S. Meyer, M.R. Miller, D.V. Neuhauser,
G. Ryan, S. Saint, S.M. Shortell, D.P. Stevens, K. Walshe.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) W-175
Supplement
Annals of Internal Medicine
Promoting a Culture of Safety as a Patient Safety Strategy
A Systematic Review
Sallie J. Weaver, PhD; Lisa H. Lubomksi, PhD; Renee F. Wilson, MS; Elizabeth R. Pfoh, MPH; Kathryn A. Martinez, PhD, MPH;
and Sydney M. Dy, MD, MSc
Developing a culture of safety is a core element of many efforts to
improve patient safety and care quality. This systematic review
identifies and assesses interventions used to promote safety culture
or climate in acute care settings. The authors searched MEDLINE,
CINAHL, PsycINFO, Cochrane, and EMBASE to identify relevant
English-language studies published from January 2000 to October
2012. They selected studies that targeted health care workers practicing in inpatient settings and included data about change in patient safety culture or climate after a targeted intervention. Two
raters independently screened 3679 abstracts (which yielded 33
eligible studies in 35 articles), extracted study data, and rated study
quality and strength of evidence. Eight studies included executive
walk rounds or interdisciplinary rounds; 8 evaluated multicomponent, unit-based interventions; and 20 included team training or
communication initiatives. Twenty-nine studies reported some improvement in safety culture or patient outcomes, but measured
outcomes were highly heterogeneous. Strength of evidence was
low, and most studies were pre–post evaluations of low to moderate quality. Within these limits, evidence suggests that interventions can improve perceptions of safety culture and potentially
reduce patient harm.
THE PROBLEM
Patient safety climate is a related term—often inadvertently used interchangeably with culture—that refers specifically to shared perceptions or attitudes about the norms,
policies, and procedures related to patient safety among
members of a group (for example, care team, unit, service,
department, or organization) (11). Climate provides a
snapshot of clinician and staff perceptions about the observable, surface-level aspects of culture during a particular
point in time (10, 15). It is measured most often using a
questionnaire or survey. Clinicians and staff are asked
about aspects of their team, work area, or hospital, such as
communication about safety hazards, transparency, teamwork, and leadership. Because climate is defined as a characteristic of a team or group, individual responses to survey
items are usually aggregated to form unit-, department-, or
higher-level scores. The difference between culture and climate is often reduced to a difference in methodology.
Studies involving surveys of clinicians and staff are categorized as studies of safety climate, and ethnographic studies
involving detailed, longitudinal observations are categorized as studies of safety culture. The terms are often used
interchangeably in practice, but it is important to remember that there are conceptually meaningful differences in
their scope and depth. For the purpose of this review, studies of both patient safety culture and climate were included. We use the term patient safety culture in discussion
only to simplify the reporting of results.
Given that safety culture can influence care processes
and outcomes, efforts to evaluate patient safety climate
Developing a culture of safety is a core element of
many efforts to improve patient safety and care quality in
acute care settings (1, 2). Several studies show that safety
culture and the related concept of safety climate are related
to such clinician behaviors as error reporting (3), reductions in adverse events (4, 5), and reduced mortality (6, 7).
Accreditation bodies identify leadership standards for
safety culture measurement and improvement (8), and promoting a culture of safety is a designated National Patient
Safety Foundation Safe Practice (9). A search of the Agency
for Healthcare Research and Quality (AHRQ) Patient
Safety Net (www.psnet.ahrq.gov) yields more than 5665
articles, tips, and fact sheets related to improving safety
culture. Although much work has focused on promoting a
culture of safety, understanding which approaches are most
effective and the implementation factors that may influence effectiveness are critical to achieving meaningful improvement (10).
Drawing on the social, organizational, and safety sciences, patient safety culture can be defined as 1 aspect of an
organization’s culture (11, 12). Specifically, it can be personified by the shared values, beliefs, norms, and procedures related to patient safety among members of an organization, unit, or team (13, 14). It influences clinician and
staff behaviors, attitudes, and cognitions on the job by
providing cues about the relative priority of patient safety
compared with other goals (for example, throughput or
efficiency) (11). Culture also shapes clinician and staff perceptions about “normal” behavior related to patient safety
in their work area. It informs perceptions about what is
praiseworthy and what is punishable (either formally by
work area leaders or informally by colleagues and fellow
team members). In this way, culture influences one’s motivation to engage in safe behaviors and the extent to which
this motivation translates into daily practice.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Ann Intern Med. 2013;158:369-374.
For author affiliations, see end of text.
www.annals.org
See also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 369
Supplement
Promoting a Culture of Safety as a Patient Safety Strategy
Key Summary Points
Safety culture is foundational to efforts to improve patient
safety and may respond to intervention.
Bundling multiple interventions or tools is a common strategy to improve safety culture.
Many programs include a form of team training or implementation of communication tools, executive walk rounds
or another form of interdisciplinary rounding, or unitbased improvement strategies that target clinical microsystems (for example, teams, units, or service lines) and are
owned by front-line clinicians and staff.
Low-quality, heterogeneous evidence derived primarily
from pre–post evaluations suggests that bundled, multicomponent interventions can improve clinician and staff
perceptions of safety culture.
Low-quality, limited evidence derived primarily from
pre–post evaluations suggests that multifaceted interventions aimed at improving patient safety can also improve
care processes and patient outcomes.
Future research should consider investigation of safety culture as a cross-cutting contextual factor that can moderate
the effectiveness of other patient safety practices.
over time are being widely implemented (16). Measurement and feedback are necessary—although likely
insufficient—means to effectively promote a culture of
safety. One previous systematic review found strong face
validity for interventions to promote safety culture in
health care, but heterogeneity among studies, measures,
and settings limited conclusions about intervention effectiveness (17). Results suggested possible positive effects
for leadership walk rounds and multifaceted, unit-based
interventions on survey measures of safety climate. However, the review did not assess effects on patient outcomes
or care processes. Another review done by the Cochrane
Collaboration (18) examined organizational culture–
change interventions designed to improve patient outcomes and quality of care. Only 2 studies were identified
for inclusion, both of which evaluated different outcomes,
and results were inconclusive. We attempted to address
these gaps by conducting a systematic review of the peerreviewed literature to identify interventions used to promote safety culture in health care, assess the evidence for
their effectiveness in improving both safety culture and
patient outcomes, and describe the context and implementation of these interventions.
PATIENT SAFETY STRATEGIES
Promotion of patient safety culture can best be conceptualized as a constellation of interventions rooted in
370 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
principles of leadership, teamwork, and behavior change,
rather than a specific process, team, or technology. Strategies to promote a culture of patient safety may include a
single intervention or several interventions combined into
a multifaceted approach or series. They may also include
system-level changes, such as those in governance or reporting structure. For example, team training, interdisciplinary rounding or executive walk rounds, and unit-based
strategies that include a series of interventions have all been
labeled as interventions to promote a culture of safety.
Team training refers to a set of structured methods for
optimizing teamwork processes, such as communication,
cooperation, collaboration, and leadership (19, 20). Previous reviews show that the term has been applied to a range
of learning and development strategies, but the critical defining element is a focus on attaining the knowledge, skills,
or attitudes that underlie effective teamwork (20).
Executive walk rounds is an interventional strategy
that engages organizational leadership directly with frontline care providers. Executives or senior leaders visit frontline patient care areas with the goal of observing and discussing current or potential threats to patient safety, as well
as supporting front-line staff in addressing such threats (21,
22). Walk rounds aim to show leadership commitment to
safety, foster trust and psychological safety, and provide
support for front-line providers to proactively address
threats to patient safety. However, walk rounds have been
operationalized in diverse ways, making comparison across
studies difficult (21). For example, not all rounding interventions use a structured format, and time intervals between rounds vary widely across studies.
Improvement strategies that combine several intervention techniques have also been used to promote safety culture. For example, the Comprehensive Unit-Based Safety
Program (CUSP) is a multifaceted strategy for culture
change that pairs adaptive interventions (such as continuous learning strategies or team training) with technical interventions (such as translation and use of best available
evidence-based clinical care algorithms) to improve patient
safety and quality (23, 24). The CUSP methodology includes elements of executive engagement and team training, along with specific strategies for translating clinical
evidence into practice. Other interventions have combined
unit-based interventions with broader organizational
changes, including restructuring patient safety governance
(25, 26).
REVIEW PROCESSES
This review examines the evidence for interventions
that articulate improvement in patient safety culture as a
primary outcome and intervention goal. We identified relevant articles through searches of 5 databases from 1 January 2000 through 31 October 2012: PubMed, CINAHL,
Cochrane, EMBASE, and PsycINFO. Key search terms
included patient safety culture, safety climate, and safety atwww.annals.org
Promoting a Culture of Safety as a Patient Safety Strategy
titudes (see the Supplement, available at www.annals.org,
for a description of the search strategies, an article flow
diagram, and evidence tables). The searches found 3679
records, all of which were independently screened by 2
reviewers. One hundred sixty-two articles were identified
for full screening. Of these, 33 studies (in 35 articles) were
identified for final inclusion. Two studies each contributed
2 papers to the review (26 –29).
Studies were included if they targeted health care professionals or paraprofessionals practicing in adult or pediatric inpatient settings, explicitly indicated that the purpose of the intervention was promoting or improving a
culture or climate of patient safety, used a psychometrically
valid measure to assess patient safety culture that had previous evidence of sound psychometric properties published
in a peer-reviewed outlet (15, 30, 31), assessed culture over
at least 2 time points, and included adequate data to assess
change in patient safety culture or climate. Only Englishlanguage studies conducted in the United States, the
United Kingdom, Canada, or Australia were included. Although a growing number of studies have translated
English-language surveys of culture into other languages,
evidence that their construct validity is comparable across
samples remains limited. Studies were excluded if they examined interventions aimed at medical or nursing students, targeted other aspects or types of culture (for example, general organizational culture), or were primarily
focused on survey development or establishing the psychometric properties of a culture assessment. Qualitative studies were also excluded. Each article was abstracted by a
primary reviewer and checked by a second reviewer.
Strength of evidence, including risk of bias, was evaluated by both reviewers using the Grading of Recommendations Assessment, Development and Evaluation Working Group criteria adapted by AHRQ (32). Interventions
and reported outcomes were highly heterogeneous, and
meta-analyses were not done. We present results from thematic analysis and qualitative summaries of individual
studies.
This review was supported by the AHRQ, which had
no role in the selection or review of the evidence or the
decision to submit the manuscript for publication.
BENEFITS
AND
HARMS
Study Characteristics
Of the 33 studies reviewed, 24 were pre–post studies;
3 were concurrent control or pre–post with concurrent
control studies; 3 were time-series studies; 2 were cluster
randomized, controlled trials (RCTs); and 1 had a quasistepped wedge design. The clinical care areas studied
included intensive care, perioperative, labor and delivery,
radiology, and general medical and surgical floors. Twentyone studies measured patient safety culture or climate with
the Safety Attitudes Questionnaire (33), 10 studies used
the AHRQ Hospital Survey on Patient Safety (34), and 2
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
studies used the Patient Safety Climate in Healthcare Organizations survey (35). Most studies operationalized culture at the level of the hospital unit or work area; that is,
individual survey responses from clinicians and staff in a
given work area were aggregated to form group-level patient safety climate scores for each work area surveyed.
Survey sample sizes ranged from 5461 persons working in
144 units in a single hospital to 28 individuals working
within a single hospital unit. The response rate—the number of individuals who complete and return surveys out of
the total invited to complete the survey—is an important
factor influencing the validity of survey results. Survey response rates ranged from 23% to 100%.
Intervention Types
Heterogeneity among interventions was substantial.
Most (19 studies) were multicomponent interventions
combining several improvement strategies under a single
overarching initiative to promote safety culture. For example, Blegen and colleagues (36) used a 3-component approach that included team training, unit-based safety
teams, and strategies for engaging patients in daily goal
setting. Thematic analysis identified 3 broad categories of
intervention that emerged across multiple studies: 20 studies explicitly included team training or tools to improve
team communication processes, 8 explicitly included some
form of executive walk rounds or interdisciplinary rounding, and 8 explicitly used CUSP.
Benefits
Team Training
Twenty studies explicitly examined team training or
tools to support team communication as interventions to
promote safety culture. Of these, 10 were conducted in
perioperative care areas, 5 in labor and delivery or pediatrics, 2 in medical general floors or intensive care, and 3 in
other care areas or a mix of care areas. Seventeen had pre–
post or pre–post with concurrent control designs. One
study was a quasi-cluster RCT; however, only 3 organizations were randomly assigned to 3 conditions. Sixteen of
the 20 studies reported statistically significant improvement in staff perceptions of safety culture. In addition, 5
reported improvements in care processes (for example, decreased care delays or increased use of structured communication) and 7 reported improvements in patient safety
outcomes (for example, errors resulting in harm or reductions in adverse outcomes index).
Executive Walk Rounds
Eight studies evaluated walk rounds (either executive
or interdisciplinary), including 1 cluster RCT. All reported
improvement in staff perceptions of safety culture. One
study, however, showed improvement on only 2 of 30 survey items and did not report domain scores (37). Three
reported improvements in perceptions of care processes
(for example, quality of collaboration) or patient safety
outcomes (for example, improvement in mean number of
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 371
Supplement
Promoting a Culture of Safety as a Patient Safety Strategy
days since last event). One study (27, 28) found that adjusted care costs were $24.01 lower for intervention work
areas despite an adjusted length of stay that was 0.19 days
longer. However, neither of these indices were statistically
significantly different from control work areas. The study
included only 4 units (2 intervention, 2 control) and was
underpowered to detect differences in these outcomes.
CUSP
Eight studies specifically evaluated the effects of
CUSP. Most used medium- to larger-sample pre–post designs in intensive care unit settings, although 1 used a
quasi-stepped wedge design. Overall, 6 of the 8 studies
reported statistically significant improvements in staff perceptions of safety culture, including perceptions of teamwork. Two studies reported improvements in care processes, such as second-stage labor care (38) and timely
resolution of safety concerns (39). Two studies reported
improvements (although statistically nonsignificant or not
statistically tested) in nursing turnover (40, 41), 1 reported
a reduction in length of stay (41), and 1 reported greater
reductions in infection rates (although not statistically significant) (42). Other studies of CUSP have shown sustained improvements in infection rates and mortality after
implementation (23, 27).
Outcomes
Regarding effectiveness, 23 of 32 reviewed studies reported a statistically significant effect of the intervention
on the overall safety culture score, the safety climate score,
or at least half of reported survey domains or items (if
analyzed at the item level). Several studies reported improvements in teamwork climate but did not find similar
improvements in safety culture or safety climate (27, 43).
Additional outcomes included changes in care processes, patient outcomes (for example, indices of harm),
and clinician outcomes (for example, turnover or burnout).
Nineteen studies also reported the effect of interventions
on such outcomes. Statistically significant improvements
were reported in 6 of 11 studies reporting on patient outcomes. Five studies found reductions in indices of patient
harm (25, 26, 43– 45), and 1 study reported improvements
in length of stay (41). One study found a decrease (0.56 vs.
0.15; P ⬍ 0.01) in the rate of reported errors that resulted
in patient harm after a multifaceted suite of interventions
that included both cultural (for example, feedback on errors in the form of posters) and system-focused changes
(for example, medication management protocols) (43). A
cluster RCT that found a marginal increase in teamwork
culture (45) also found that the experimental unit’s
weighted adverse outcome score (an index of patient harm)
decreased by 37% after implementation of a team training
program designed to promote patient safety culture, compared with a 43% increase in a control unit (P ⬍ 0.05).
372 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Two studies also reported reductions in nurse turnover after interventions to promote safety culture (40, 41).
Overall, the strength of evidence was low. Risk of bias
was generally high because of study design issues; for example, we identified only 1 true cluster RCT (22). Core
issues affecting risk of bias for reviewed studies included
low survey response rates and incomplete reporting (not
reporting full results for all units or hospitals where interventions were conducted, or not reporting results for all
domains measured as part of culture surveys). Results were
inconsistent, with 56% of studies reporting statistically significant findings. Regarding directness, or the extent to
which findings generalize to different organizations or populations, few studies discussed the logic model or conceptual foundation underlying the intervention design. Only 2
studies comparatively evaluated the effects of different intervention strategies, and patient safety outcomes were infrequently and heterogeneously reported. Regarding precision, many survey instruments were used across reviewed
studies and results were often reported differently.
Harms
We did not identify any data on patient harms.
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
Studies differed in the characteristics of the organizations in which they were implemented, the level of leadership support and engagement reported, and the tools and
strategies used to support implementation into daily care
processes. Thirteen studies were done in academic hospital
settings, 4 in community-based hospitals, 6 in a mix of
academic and community hospitals, and several did not
address the hospital mix in their sample. One study reported that the gain in safety climate scores was larger for
faith-based hospitals (14%) than for non–faith-based hospitals (8%) but reported no direct statistical test of these
findings (46). Only 1 study (28) examined costs of care
among intervention and control work areas. No statistically
significant differences in mean care costs between control
and intervention work areas at follow-up were found.
DISCUSSION
Our review identified 33 studies in 35 articles that
evaluated interventions to promote safety culture in inpatient care settings. Although these interventions varied
greatly and often included multiple components, 3 common types of intervention emerged: team training and
team communication tools, executive walk rounds and interdisciplinary rounding, and CUSP. These interventions
were implemented across various care areas in both academic and community hospital settings. Most were evaluated in either perioperative or intensive care areas.
Overall, results suggest evidence to support the effectiveness of such interventions in improving clinician and
staff perceptions of elements of safety culture (for example,
www.annals.org
Promoting a Culture of Safety as a Patient Safety Strategy
general perceptions of safety climate and teamwork). A few
studies provide evidence that interventions aiming to improve safety culture may meaningfully improve clinical
care processes (28, 47– 49) and suggest the potential to
improve aggregate indices of patient harm (29, 45). However, these conclusions are tempered by the limitations of
the current evidence. Although 1 true cluster RCT was
identified (22), most studies had pre–post designs with
relatively small to moderate samples (particularly at the
unit or work area level of analysis) that did not include
control participants. In addition, few studies examined potential variation in perceptions of safety culture by care
provider type.
Although this review offers a systematic analysis of
strategies to promote safety culture, clear limitations must
be considered. Only studies in acute care settings using
established survey measures were included. Although qualitative studies of safety culture may offer insight into nuances of implementation, they were outside the scope of
this review. Because several studies in outpatient settings
were not included, results may not generalize beyond inpatient settings. Relevant studies may also have been inadvertently excluded despite extensive searches. Publication
bias and selective reporting of positive findings also may
limit conclusions about the effectiveness and generalizability of the interventions evaluated. Finally, traditional criteria for evaluating the effectiveness of clinical interventions
for individual patients are not well-suited to assessing the
effectiveness of quasi-experimental study designs conducted at the unit level of analysis. This may have introduced systematic bias into our ratings for strength of evidence. As noted by Pizzi and colleagues in the original
“Making Health Care Safer” report (50); “the threshold for
evidence may need a different yardstick than is typically
applied in medicine.”
In summary, this review suggests that evidence to support the potential effectiveness of interventions to promote
safety culture is emerging. In particular, the best evidence
to date seems to include strategies comprising multiple
components that incorporate team training and mechanisms to support team communication and include executive engagement in front-line safety walk rounds. Organizations should consider incorporating these elements into
efforts to promote safety culture but also robustly evaluate
such efforts across multiple outcomes. Future research
should also consider thorough investigation of safety culture as a cross-cutting contextual factor that can moderate
the effectiveness of other patient safety practices, such as
implementation of rapid response systems. The strength of
evidence for patient safety culture would be improved if
theoretical models (31, 51, 52) were meaningfully used in
the development of interventions for improvement and
those interventions were robustly evaluated. Finally, work
is needed to better understand the contextual role that
safety culture plays in implementation of other patient
safety practices, as well as how efforts to promote safety
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
culture can best be implemented to enhance the effectiveness of complementary or supplementary interventions for
safety and care quality.
From Johns Hopkins University, Baltimore, Maryland, and University of
Michigan, Ann Arbor, Michigan.
Note: The AHRQ reviewed contract deliverables to ensure adherence to
contract requirements and quality, and a copyright release was obtained
from the AHRQ before submission of the manuscript.
Disclaimer: All statements expressed in this work are those of the authors
and should not in any way be construed as official opinions or positions
of the Johns Hopkins University, the AHRQ, or the U.S. Department of
Health and Human Services.
Financial Support: From the AHRQ, U.S. Department of Health and
Human Services (contract HHSA-290-2007-10062I).
Potential Conflicts of Interest: Dr. Weaver: Grant (money to institution): AHRQ, U.S. Department of Health and Human Services; Travel/
accommodations/meeting expenses unrelated to activities listed (money to author): Improvement Science Research Network. Dr. Lubomski: Grant
(money to institution): AHRQ. Ms. Wilson: Grant (money to institution):
AHRQ. Ms. Pfoh: Grant (money to institution): AHRQ. Dr. Martinez:
None disclosed. Dr. Dy: Grant (money to institution): AHRQ. Disclosures can be also viewed at www.acponline.org/authors/icmje
/ConflictOfInterestForms.do?msNum⫽M12-2567.
Requests for Single Reprints: Sallie J. Weaver, PhD, Johns Hopkins
University School of Medicine, Department of Anesthesiology and Critical Care Medicine and Armstrong Institute for Patient Safety and Quality, 750 East Pratt Street, 15th Floor, Room 1544, Baltimore, MD
21202; e-mail, [email protected].
Current author addresses and author contributions are available at www
.annals.org.
References
1. Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer
Health System. Washington, DC: National Academies Pr; 2000.
2. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ.
Making health care safer: a critical analysis of patient safety practices. Evid Rep
Technol Assess (Summ). 2001:i-x, 1-668. [PMID: 11510252]
3. Braithwaite J, Westbrook MT, Travaglia JF, Hughes C. Cultural and associated enablers of, and barriers to, adverse incident reporting. Qual Saf Health
Care. 2010;19:229-33. [PMID: 20534716]
4. Singer S, Lin S, Falwell A, Gaba D, Baker L. Relationship of safety climate
and safety performance in hospitals. Health Serv Res. 2009;44:399-421. [PMID:
19178583]
5. Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf.
2010;6:226-32. [PMID: 21099551]
6. Estabrooks CA, Tourangeau AE, Humphrey CK, Hesketh KL, Giovannetti
P, Thomson D, et al. Measuring the hospital practice environment: a Canadian
context. Res Nurs Health. 2002;25:256-68. [PMID: 12124720]
7. Sexton JB. A Matter of Life or Death: Social, Psychological, and Organizational Factors Related to Patient Outcomes in the Intensive Care Unit. Austin:
Univ of Texas; 2002.
8. The Joint Commission. Revisions to LD.03.01.01. Oakbrook Terrace, IL:
The Joint Commission; 2012. Accessed at www.jointcommission.org/assets/1/6
/Pre-Pubs_LD.03.01.01_HAP.pdf on 8 September 2012.
9. National Patient Safety Foundation Safe Practices. Accessed at www.npsf.org
/for-healthcare-professionals/resource-center on 8 September 2012.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 373
Supplement
Promoting a Culture of Safety as a Patient Safety Strategy
10. Singer SJ, Vogus TJ. Safety climate research: taking stock and looking forward. BMJ Qual Saf. 2012. [PMID: 23112287]
11. Zohar D, Livne Y, Tenne-Gazit O, Admi H, Donchin Y. Healthcare climate: a framework for measuring and improving patient safety. Crit Care Med.
2007;35:1312-7. [PMID: 17414090]
12. Flin R. Measuring safety culture in healthcare: A case for accurate diagnosis.
Saf Sci. 2007;45:653-67.
13. Schein EH. Organizational Culture and Leadership. 4th ed. Hoboken, NJ:
Jossey-Bass; 2010.
14. Pronovost PJ, Berenholtz SM, Goeschel CA, Needham DM, Sexton JB,
Thompson DA, et al. Creating high reliability in health care organizations.
Health Serv Res. 2006;41:1599-617. [PMID: 16898981]
15. Jackson J, Sarac C, Flin R. Hospital safety climate surveys: measurement
issues. Curr Opin Crit Care. 2010. [PMID: 20827181]
16. Sorra J, Famolaro T, Dyer N, Nelson D, Smith SA. Hospital Survey on
Patient Safety Culture: 2012 User Comparative Database Report. AHRQ
publication no. 12-0017. (Prepared by Westat under contract HHSA
290200710024C.) Rockville, MD: AHRQ; 2012.
17. Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C.
Strategies for improving patient safety culture in hospitals: a systematic review.
BMJ Qual Saf. 2012. [PMID: 22849965]
18. Parmelli E, Flodgren G, Beyer F, Baillie N, Schaafsma ME, Eccles MP.
The effectiveness of strategies to change organisational culture to improve
healthcare performance: a systematic review. Implement Sci. 2011;6:33. [PMID:
21457579]
19. Salas E, DiazGranados D, Weaver SJ, King H. Does team training work?
Principles for health care. Acad Emerg Med. 2008;15:1002-9. [PMID:
18828828]
20. Weaver SJ, Lyons R, DiazGranados D, Rosen MA, Salas E, Oglesby J, et al.
The anatomy of health care team training and the state of practice: a critical
review. Acad Med. 2010;85:1746-60. [PMID: 20841989]
21. Frankel A, Grillo SP, Pittman M, Thomas EJ, Horowitz L, Page M, et al.
Revealing and resolving patient safety defects: the impact of leadership WalkRounds on frontline caregiver assessments of patient safety. Health Serv Res.
2008;43:2050-66. [PMID: 18671751]
22. Thomas EJ, Sexton JB, Neilands TB, Frankel A, Helmreich RL. The effect
of executive walk rounds on nurse safety climate attitudes: a randomized trial of
clinical units[ISRCTN85147255] [corrected]. BMC Health Serv Res. 2005;5:28.
[PMID: 15823204]
23. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S,
et al. An intervention to decrease catheter-related bloodstream infections in the
ICU. N Engl J Med. 2006;355:2725-32. [PMID: 17192537]
24. Romig M, Goeschel C, Pronovost P, Berenholtz SM. Integrating CUSP
and TRIP to improve patient safety. Hosp Pract (Minneap). 2010;38:114-21.
[PMID: 21068535]
25. Muething SE, Goudie A, Schoettker PJ, Donnelly LF, Goodfriend MA,
Bracke TM, et al. Quality improvement initiative to reduce serious safety events
and improve patient safety culture. Pediatrics. 2012;130:e423-31. [PMID:
22802607]
26. Pettker CM, Thung SF, Raab CA, Donohue KP, Copel JA, Lockwood CJ,
et al. A comprehensive obstetrics patient safety program improves safety climate
and culture. Am J Obstet Gynecol. 2011;204:216.e1-6. [PMID: 21376160]
27. O’Leary KJ, Wayne DB, Haviley C, Slade ME, Lee J, Williams MV.
Improving teamwork: impact of structured interdisciplinary rounds on a medical
teaching unit. J Gen Intern Med. 2010;25:826-32. [PMID: 20386996]
28. O’Leary KJ, Haviley C, Slade ME, Shah HM, Lee J, Williams MV. Improving teamwork: impact of structured interdisciplinary rounds on a hospitalist
unit. J Hosp Med. 2011;6:88-93. [PMID: 20629015]
29. Pettker CM, Thung SF, Norwitz ER, Buhimschi CS, Raab CA, Copel JA,
et al. Impact of a comprehensive patient safety strategy on obstetric adverse
events. Am J Obstet Gynecol. 2009;200:492.e1-8. [PMID: 19249729]
30. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety
climate: a review of surveys. Qual Saf Health Care. 2005;14:364-6. [PMID:
16195571]
31. Flin R, Burns C, Mearns K, Yule S, Robertson EM. Measuring safety
climate in health care. Qual Saf Health Care. 2006;15:109-15. [PMID:
16585110]
32. Schünemann HJ, Schünemann AH, Oxman AD, Brozek J, Glasziou P,
Jaeschke R, et al; GRADE Working Group. Grading quality of evidence and
374 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336:
1106-10. [PMID: 18483053]
33. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, et al.
The Safety Attitudes Questionnaire: psychometric properties, benchmarking
data, and emerging research. BMC Health Serv Res. 2006;6:44. [PMID:
16584553]
34. Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. AHRQ
publication no. 04-0041. Rockville, MD: AHRQ; 2004.
35. Singer S, Meterko M, Baker L, Gaba D, Falwell A, Rosen A. Workforce
perceptions of hospital safety culture: development and validation of the patient
safety climate in healthcare organizations survey. Health Serv Res. 2007;42:19992021. [PMID: 17850530]
36. Blegen MA, Sehgal NL, Alldredge BK, Gearhart S, Auerbach AA, Wachter
RM. Improving safety culture on adult medical units through multidisciplinary
teamwork and communication interventions: the TOPS Project. Qual Saf Health
Care. 2010;19:346-50. [PMID: 20693223]
37. Tiessen B. On the journey to a culture of patient safety. Healthc Q. 2008;
11:58-63. [PMID: 18818531]
38. Simpson KR, Knox GE, Martin M, George C, Watson SR. Michigan
Health & Hospital Association Keystone Obstetrics: a statewide collaborative for
perinatal patient safety in Michigan. Jt Comm J Qual Patient Saf. 2011;37:54452. [PMID: 22235539]
39. Saladino L, Pickett LC, Frush K, Mall A, Champagne MT. Evaluation of a
Nurse-Led Safety Program in a Critical Care Unit. J Nurs Care Qual. 2012.
[PMID: 23052353]
40. Timmel J, Kent PS, Holzmueller CG, Paine L, Schulick RD, Pronovost PJ.
Impact of the Comprehensive Unit-based Safety Program (CUSP) on safety culture in a surgical inpatient unit. Jt Comm J Qual Patient Saf. 2010;36:252-60.
[PMID: 20564886]
41. Pronovost PJ, Weast B, Rosenstein B, Sexton JB, Holzmueller CG, Paine
LA, et al. Implementing and validating a comprehensive unit-based safety program. J Patient Saf. 2005;1:33-40.
42. Vigorito MC, McNicoll L, Adams L, Sexton B. Improving safety culture
results in Rhode Island ICUs: lessons learned from the development of
action-oriented plans. Jt Comm J Qual Patient Saf. 2011;37:509-14. [PMID:
22132663]
43. Abstoss KM, Shaw BE, Owens TA, Juno JL, Commiskey EL, Niedner MF.
Increasing medication error reporting rates while reducing harm through simultaneous cultural and system-level interventions in an intensive care unit. BMJ
Qual Saf. 2011;20:914-22. [PMID: 21690249]
44. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP,
et al; Safe Surgery Saves Lives Study Group. Changes in safety attitude and
relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf. 2011;
20:102-7. [PMID: 21228082]
45. Riley W, Davis S, Miller K, Hansen H, Sainfort F, Sweet R. Didactic and
simulation nontechnical skills team training to improve perinatal patient outcomes in a community hospital. Jt Comm J Qual Patient Saf. 2011;37:357-64.
[PMID: 21874971]
46. Sexton JB, Berenholtz SM, Goeschel CA, Watson SR, Holzmueller CG,
Thompson DA, et al. Assessing and improving safety climate in a large cohort of
intensive care units. Crit Care Med. 2011;39:934-9. [PMID: 21297460]
47. McCulloch P, Mishra A, Handa A, Dale T, Hirst G, Catchpole K. The
effects of aviation-style non-technical skills training on technical performance and
outcome in the operating theatre. Qual Saf Health Care. 2009;18:109-15.
[PMID: 19342524]
48. Weaver SJ, Rosen MA, DiazGranados D, Lazzara EH, Lyons R, Salas E,
et al. Does teamwork improve performance in the operating room? A multilevel
evaluation. Jt Comm J Qual Patient Saf. 2010;36:133-42. [PMID: 20235415]
49. Wolf FA, Way LW, Stewart L. The efficacy of medical team training: improved team performance and decreased operating room delays: a detailed analysis of 4863 cases. Ann Surg. 2010;252:477-83. [PMID: 20739848]
50. Pizzi LT, Goldfarb NI, Nash DB. Promoting a culture of safety. In: Making
Health Care Safer: A Critical Analysis of Patient Safety Practices. Rockville, MD:
AHRQ; 2001.
51. Reiman T, Pietikäinen E, Oedewald P. Multilayered approach to patient
safety culture. Qual Saf Health Care. 2010;19:e20. [PMID: 20724396]
52. Kirk S, Parker D, Claridge T, Esmail A, Marshall M. Patient safety culture
in primary care: developing a theoretical framework for practical use. Qual Saf
Health Care. 2007;16:313-20. [PMID: 17693682]
www.annals.org
Annals of Internal Medicine
Current Author Addresses: Drs. Weaver and Lubomski: Johns Hopkins
Author Contributions: Conception and design: S.J. Weaver, L.H.
University School of Medicine, Department of Anesthesiology and Critical Care Medicine and Armstrong Institute for Patient Safety and Quality, 750 East Pratt Street, 15th Floor, Baltimore, MD 21202.
Ms. Wilson: Johns Hopkins University, 1830 East Monument Street,
Room 8061, Baltimore, MD 21287.
Ms. Pfoh: Johns Hopkins University, 624 North Broadway, Baltimore,
MD 21205.
Dr. Martinez: University of Michigan, Department of General Medicine, 2800 Plymouth Road, Building 16, 4th Floor, Ann Arbor, MI
48109-2800.
Dr. Dy: Johns Hopkins University, Health Services Research and Development Center, 624 Broadway, Room 609, Baltimore, MD 212051901.
Lubomksi, K.A. Martinez, S.M. Dy.
Analysis and interpretation of the data: S.J. Weaver, E.R. Pfoh, K.A.
Martinez, S.M. Dy.
Drafting of the article: S.J. Weaver, E.R. Pfoh, S.M. Dy.
Critical revision of the article for important intellectual content: S.J.
Weaver, L.H. Lubomksi, S.M. Dy.
Final approval of the article: S.J. Weaver, R.F. Wilson, K.A. Martinez,
S.M. Dy.
Obtaining of funding: S.M. Dy.
Administrative, technical, or logistic support: L.H. Lubomksi, R.F.
Wilson, K.A. Martinez.
Collection and assembly of data: S.J. Weaver, L.H. Lubomksi, R.F.
Wilson, K.A. Martinez, S.M. Dy.
W-176 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Supplement
Annals of Internal Medicine
In-Facility Delirium Prevention Programs as a Patient Safety Strategy
A Systematic Review
James T. Reston, PhD, MPH, and Karen M. Schoelles, MD, SM
Delirium, an acute decline in attention and cognition, occurs among
hospitalized patients at rates estimated to range from 14% to 56%
and increases the risk for morbidity and mortality. The purpose of
this systematic review was to evaluate the effectiveness and safety
of in-facility multicomponent delirium prevention programs. A
search of 6 databases (including MEDLINE, EMBASE, and CINAHL)
was conducted through September 2012. Randomized, controlled
trials; controlled clinical trials; interrupted time series; and controlled
before–after studies with a prospective postintervention portion
were eligible for inclusion. The evidence from 19 studies that met
the inclusion criteria suggests that most multicomponent interventions are effective in preventing onset of delirium in at-risk patients
in a hospital setting. Evidence was insufficient to determine the
benefit of such programs in other care settings. Future comparative
effectiveness studies with standardized protocols are needed to
identify which components in multicomponent interventions are
most effective for delirium prevention.
THE PROBLEM
ing. Among studies that evaluated cognitive impairment or
dementia, 84.6% found a significant association between
this factor and incidence of delirium. Depression was
found to have a significant association with delirium occurrence in only 40% of the studies that evaluated it as a
potential risk factor.
Other patient-specific risk factors that showed a significant association with delirium in more than 1 study include male sex, multiple medications, comorbid conditions
(for example, diabetes), pneumonia, various anesthetics,
neuropsychiatric drugs (for example, benzodiazepines), anticholinergics, blood transfusions, abnormal serum chemistry (for example, blood urea nitrogen levels or creatinine
levels), apolipoprotein E4, atrial fibrillation, heavy alcohol
intake, volume depletion (dehydration), hypoxia, complications, restraints (rendering patients immobile), and visual
impairment. Several studies evaluated patients having specific surgical procedures (for example, hip repair or replacement or cardiac surgery); some of these studies focused on
surgery-specific risk factors (for example, blood transfusions or intraoperative anesthesia) and evaluated few nonsurgical factors.
Given the multifactorial nature of delirium, a patient
safety strategy designed to assess and remediate multiple
factors is considered likely to be effective for delirium prevention. The purpose of this systematic review was to assess
the benefits and harms of multicomponent interventions,
including system-level changes, that are designed to prevent delirium in hospitals, palliative care centers, and longterm care facilities.
Delirium (also known as acute confusional state) is an
acute decline in attention and cognition that constitutes a
serious problem for older hospitalized patients and longterm care residents. Estimated hospital occurrence rates
range from 14% to 56% and vary depending on the reason
for hospitalization (for example, urgent surgery, intensive
care, or general medical admission) and the patient’s risk
for the condition (1).
Delirium is associated with an increased risk for death,
postoperative complications, longer hospital and intensive
care unit stays, and functional decline (1, 2), and it presents a substantial burden in terms of short- and long-term
health care costs. A study of 841 patients (aged ⱖ70 years)
admitted to non–intensive care general medical units over
a 3-year period at Yale-New Haven Hospital found that
daily costs were more than 2.5 times higher for patients
with delirium than for those without it. The total cost
estimates associated with delirium ranged from $16 303 to
$64 421 per patient, which the authors extrapolated to
national costs ranging from $38 billion to $152 billion
each year (1). Because these estimates were based on data
from 1995 to 1998, the costs would be even higher today.
Accordingly, prevention of delirium is extremely important
for improving patient outcomes and decreasing health care
costs.
Evidence from risk-factor studies suggests that delirium has a multifactorial cause (more information on these
studies appears in the full report, available at the Agency
for Healthcare Research and Quality [AHRQ] Web site
[www.ahrq.gov]). No 2 studies evaluated the same set of
factors or found the same combination of significant factors associated with delirium. Age was the most commonly
evaluated factor—58.8% of studies that evaluated age
found it to be significantly associated with occurrence of
delirium. Some studies may have lacked adequate power to
find statistical significance, although this was clearly not
the case in all studies that did not have a significant findwww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Ann Intern Med. 2013;158:375-380.
For author affiliations, see end of text.
www.annals.org
See also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 375
Supplement
In-Facility Delirium Prevention Programs as a Patient Safety Strategy
Key Summary Points
Because delirium has multiple risk factors, multicomponent
interventions targeting several risk factors represent promising patient safety strategies for delirium prevention.
Most of the evidence suggests that most multicomponent
interventions are effective in preventing onset of delirium
in at-risk patients in a hospital setting. These interventions
do not seem to have significant associated harms.
The evidence is insufficient to identify which multicomponent interventions are most beneficial, and the studies do
not address the question of which components within a
program provide the most benefit for delirium prevention.
The evidence is insufficient to determine the benefit of
delirium prevention programs in palliative care or longterm care settings.
PATIENT SAFETY STRATEGIES
Several delirium prevention programs consist of multifactorial intervention bundles. In general, the components
of the bundle vary across each published evaluation, and
the same bundle is rarely evaluated in more than 1 application. Therefore, the best that can be done is to describe
the components most commonly included in bundles that
have been found to reduce incident delirium. The most
common components of successful bundles are shown in
the Table.
Additional components have been reported in successful multifactorial bundles. An intervention used for patients with hip fracture in a Swedish university hospital
included increased physiologic monitoring, avoidance of
delays in transfer through different areas of the hospital,
daily delirium screening, and avoidance of polypharmacy
(as well as several components from the Table, including
extra nutrition, intravenous fluid supplementation, pain
management, and perioperative or anesthetic period protocols) (3). A multifactorial intervention used for patients
with hip fracture at another Swedish university hospital
included treatment of sleep apnea, prevention and treatment of decubitus ulcers, and measurement of blood pressure along with components from the Table, although it is
not clear that all of these components were specifically
designed to prevent delirium (4).
The Hospital Elder Life Program (HELP), or modified versions thereof, has been evaluated in 3 studies (5–7).
This program typically consists of 6 components:
orientation, therapeutic activities, vision and hearing protocols, sleep enhancement, and early mobilization. Two
studies (1 in the United States and 1 in Australia) used
proactive geriatric consultation with targeted recommendations (several from the Table) based on a structured protocol (8, 9).
376 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
REVIEW PROCESSES
We conducted a systematic review of 6 databases (including MEDLINE, EMBASE, and CINAHL) for 1999
to September 2012. A total of 673 titles were identified, of
which 309 were reviewed in detail. The Supplement (available at www.annals.org) provides a complete description of
the search strategies, an article flow diagram, and evidence
tables. Randomized, controlled trials; controlled clinical
trials; interrupted time series; and controlled before–after
studies with a prospective postintervention portion were
eligible for inclusion to address effectiveness and harms.
Studies were required to have at least 20 patients per intervention group. Methods for assessing risk of bias and
strength of evidence are described in the full report on the
AHRQ Web site.
Of 309 studies retrieved from our literature searches
and reviewed in detail, we identified 35 that addressed
single or multicomponent interventions. Of these, 19 evaluated the efficacy of multicomponent interventions and are
the subject of this review (see Table 2 of the Supplement).
Most of these studies reported the incidence of delirium
after the intervention compared with a control group of
usual care patients treated concurrently or during a period
immediately before adoption of the new intervention. Because few studies used the same intervention, comparison
group, study design, or patient population, meta-analyses
were not done.
This review was supported by the AHRQ, which had
no role in the selection or review of the evidence or in the
decision to submit the manuscript for publication.
BENEFITS
AND
HARMS
Benefits
Hospital Inpatient Care
Two studies used HELP, and a third used a modification of HELP. One was a controlled before–after study
with a concurrent control group consisting of patients
from usual care units (7, 10); this study had a moderate
risk of bias. The remaining 2 studies were before–after
studies where the usual care group consisted of patients
treated before implementation of HELP (historical control) (5, 6); these studies had a high risk of bias. All 3
studies found a substantial reduction in incident delirium
after implementation of HELP compared with usual care.
Although the findings of the studies were consistent, the
average risk of bias was high, mainly because of lack of
randomization and blinding.
Two studies used proactive geriatric consultation with
targeted recommendations based on a structured protocol
for patients with hip fracture. One was a single-blind, randomized, controlled trial with a usual care control group
(8), and the other was a before–after study with a historical
usual care control group (9). Both studies reported a reduction of incident delirium for the geriatric consultation
group compared with the usual care group; however, the
www.annals.org
In-Facility Delirium Prevention Programs as a Patient Safety Strategy
findings from the randomized, controlled trial were no longer statistically significant after adjustment for baseline imbalances. The risk of bias for the studies was high and
moderate, respectively. A nonrandomized, controlled study
used an inpatient geriatric consultation team that made
targeted recommendations (although a list of potential recommendations was not reported) to prevent delirium in
patients with hip fracture. Although delirium incidence
was lower in the intervention group, the difference in incidence rates did not reach statistical significance (28% vs.
44%; P ⫽ 0.067). This study had a high risk of bias due to
lack of randomization and a low adherence rate to inpatient geriatric consultation team recommendations (one
third of recommendations was not implemented) (11).
Of the remaining multicomponent studies (3, 4, 12–
20), all but 1 reported a statistically significant reduction in
delirium by at least 1 measure in the intervention group
versus the control group. The exception was a study of a
system-wide quality improvement project (17). A study of
nurse-facilitated family participation reported substantially
fewer patients with a diagnosis of delirium (defined as a
score ⱖ4 on the Intensive Care Delirium Screening
Checklist) in the intervention group but also no statistically significant between-group difference in mean scores;
this study placed more emphasis on the latter measure
(18). Overall, the findings are consistent with those from
studies of the HELP intervention, although the risk of bias
was high—again, because of lack of randomization and
blinding.
Palliative Care
One multicenter controlled trial assessed a multicomponent intervention intended to prevent delirium in patients with terminal cancer (21). In this population, delirium stems from such risk factors as metastatic brain
lesions, high opioid intake, and metabolic disturbances;
these are not the typical risk factors found in the general
geriatric population, which include older age, cognitive impairment, visual impairment, and multiple medications.
Two centers (with 674 patients) implemented the intervention, and the remaining 5 centers (with 842 patients)
performed usual care. The intervention involved training
nurses to orient the patient each day, recognize risk factors
for delirium, and send this information to physicians so
that preventive actions could be taken (for example, changing medication). The closest family member was also educated by nurses on delirium and its symptoms, as well as
American College of Physicians recommendations for
avoiding symptoms of confusion in this patient population. Delirium symptoms were assessed by nurses using the
Confusion Rating Scale. During the 3-year study, 49% of
patients in the intervention group and 44% in the usual
care group developed delirium; after adjustment for confounding factors, there was no significant between-group
difference in incident delirium (odds ratio, 0.94; P ⫽
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Table. The Most Common Components of Successful
Delirium Prevention Programs
Anesthesia protocols
Assessment of bowel/bladder functions
Early mobilization
Extra nutrition
Geriatric consultation
Hydration
Medication review
Pain management
Prevention and treatment of medical complications
Sleep enhancement
Staff education
Supplemental oxygen
Therapeutic cognitive activities/orientation
Vision and hearing protocols
0.66). The risk of bias was high because of lack of randomization; inadequate blinding; and failure to obtain a systematic, formal diagnosis of delirium.
Long-Term Care
The single study done in a nursing home setting reported that homes randomly assigned to use pharmacistled Geriatric Risk Assessment MedGuide reports and automated medication monitoring plans had a significant
reduction in delirium onset among newly admitted residents compared with those randomly assigned to usual care
(22). However, it is unclear how much of this is due to
delirium prevention or resolution of new-onset delirium.
Complications did not differ significantly between the
groups.
Harms
Most trials of multicomponent delirium prevention
programs have not reported any harms. However, it is not
clear whether the possibility of harms was explicitly assessed in all of these trials. One study based on a structured
quality improvement model reported 4 unexpected minor
events (rectal or feeding tube displacement or removal that
did not lead to any true complications) but no major complications (and no statistically significant difference compared with usual care, although the study lacked the statistical power to detect meaningful differences) (12). One
other study reported no statistically significant differences
in total complication rates between intervention (50.4%)
and usual care (53%) groups; this study was adequately
powered to detect a meaningful difference in complication
rates (3).
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
Structural Organizational Characteristics
Multicomponent delirium prevention programs have
been successfully implemented in acute care hospitals (17
studies), palliative care centers (1 study), and nursing
homes (1 study). Five of the acute care hospital studies
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 377
Supplement
In-Facility Delirium Prevention Programs as a Patient Safety Strategy
were conducted in the United States; 3 in the United
Kingdom; 3 in Sweden; and 1 each in Australia, Spain,
Italy, Belgium, Chile, and Taiwan. Twelve studies were
from academically affiliated urban hospitals, 2 were conducted in urban hospitals that were not described as teaching hospitals, 2 were set in community hospitals (in 1
study, the participating community hospitals were part of a
larger health system), and the remaining study was set in a
naval hospital. No studies have been reported from rural
hospitals. The single study of palliative care was conducted
in Canada, and the study set in nursing homes was done in
the United States.
Existing Infrastructure
Only 1 study reported minimal information on patient safety culture at the organizational level. The authors
stated merely that “SHS [Summa Health System] maintains a strong commitment to patient safety and quality”
(17).
External Factors
External factors or motivators were not mentioned in
any delirium study.
Implementation
All multicomponent intervention studies provided at
least minimal information about teamwork or leadership at
the level of the unit where the intervention was implemented. Thirteen of 19 studies specifically identified the
study leaders, and 17 of 19 studies identified the team
members by job status (for example, nurses and geriatricians) or at least stated that all staff in the intervention
ward or unit was part of the team. All of these studies
reported multidisciplinary teamwork that included clinical
experts, nurses, and other staff (for example, physical therapists or volunteers). One study reported minimal information on teamwork or leadership at the hospital level
(17).
Eight studies described multiprofessional implementation, 1 had the intervention performed by the ward staff, 1
involved ward staff plus physical therapists (during home
visits), 1 involved ward staff plus ambulance workers, 1
involved unit staff plus volunteers, 2 involved the nursing
staff only, 1 involved nursing staff plus consultant pharmacists, 2 involved nurses assisting family members with the
intervention, and 1 involved elder life specialists plus
volunteers.
Fifteen studies reported on staff education and training if this was part of the intervention, and 9 studies reported the persons responsible for implementation. Most
of these studies reported that all staff involved in the implementation had some type of education or training.
Thirteen studies reported the type of training, and only 4
reported the length of training.
378 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Four studies reported a change in the implementation
process due to local tailoring or an iterative process. Only 1
study reported that internal incentives were used to promote implementation (5). Allen and colleagues published
the only study that provided a table summarizing an actual
implementation instrument (a scorecard used to track process and outcome variables) (17).
Eighteen studies outlined the intended intervention
and the general sequence in which the components were
implemented; only 13 studies included enough detail to
determine the roles of the various team members. Most
studies generally described how the intervention was supposed to be implemented and did not describe any modifications or failures of adherence that might have occurred
during the actual implementation. Only 2 studies actually
measured adherence to targeted recommendations (8, 11),
respectively reporting adherence rates of 77% and approximately 67% for implementation of geriatric consultant
recommendations for patients after hip fracture. Fifteen
studies reported patient characteristics.
Although implementation of multicomponent delirium prevention programs has not been well-described in
most studies, a few themes seem sufficiently consistent to
report here. First, engagement of front-line clinical staff in
the design of the intervention helps ensure that it will mesh
with existing clinical procedures. Second, a multidisciplinary team comprising clinical experts, nurses, and additional staff is helpful for implementation of a complex intervention. Finally, education and training of clinical staff
are necessary to help ensure that compliance does not wane
over time.
Context
Two studies reported on the effect of context on outcomes. One study of an educational package for medical
and nursing staff reported that it was effective at preventing delirium in hospitalized men but not women (12, 23).
A study of proactive geriatric consultation with target recommendations based on a structured protocol for patients
with hip fracture reported a “trend” toward more effectiveness among patients without prefracture dementia or impairment in activities of daily living, but the differences
were not statistically significant (8).
One study assessed the somewhat related concept of
patient adherence and its effect on outcomes of a multifactorial intervention (HELP). Based on a composite adherence score for the 3 components assigned to all patients
(orientation, mobility, and therapeutic activities), increased
adherence scores were associated with a reduction in delirium incidence rates (odds ratio, 0.69 [95% CI, 0.56 to
0.87]) (7).
Costs
Two studies in the evidence base reported information
on costs or cost savings associated with multicomponent
delirium prevention programs. Rizzo and colleagues (24)
calculated the total intervention costs of HELP over a
www.annals.org
In-Facility Delirium Prevention Programs as a Patient Safety Strategy
3-year period (1995–1998) at Yale-New Haven Hospital
as $257 385 (personnel plus equipment). In a costeffectiveness analysis, they found that the intervention was
cost-effective for patients at intermediate risk for delirium
but not for patients at high risk (lack of effectiveness and
higher overall costs). However, these findings may be due
to inadequate power based on their relatively small sample
size of higher-risk patients, leading to uncertainty in the
results for this subgroup (24). Rubin and colleagues (5)
calculated that implementation of HELP at their hospital
led to estimated cost savings of more than $2 million per
year from prevention of delirium cases. In addition, more
than $2.2 million per year of estimated revenue was generated by shorter hospital stays for patients without
delirium.
Supplement
most effective for delirium prevention. Identification of the
most effective bundle of components might encourage hospitals to adopt a more standardized approach to delirium
prevention.
From ECRI Institute, Plymouth Meeting, Pennsylvania.
Note: The AHRQ reviewed contract deliverables to ensure adherence to
contract requirements and quality, and a copyright release was obtained
from the AHRQ before submission of the manuscript.
Disclaimer: All statements expressed in this work are those of the authors
and should not in any way be construed as official opinions or positions
of ECRI Institute, the AHRQ, or the U.S. Department of Health and
Human Services.
Acknowledgment: The authors thank Allison Gross, MS, MLS, for per-
DISCUSSION
Moderate-strength evidence suggests that most multicomponent interventions are effective in preventing onset
of delirium in at-risk patients in a hospital setting. These
interventions have not been reported to have important
associated harms, although most studies did not explicitly
assess this possibility. In general, successful delirium prevention programs involved a multidisciplinary team of
clinical experts, nurses, and other staff (for example, physical therapists or volunteers) and included protocols for
early mobilization of patients, volume repletion (for hydration and electrolyte balance), and addressing visual or hearing deficits; a few programs included elimination of unnecessary medications. Other components reported in more
than 1 study included staff education, geriatric consultation, therapeutic cognitive activities or orientation, extra
nutrition, sleep enhancement, pain management, anesthesia protocols, supplemental oxygen, assessment of bowel or
bladder functions, and prevention and treatment of medical complications.
This review has several limitations, the most notable of
which is that most of the included studies were rated as
having a high risk of bias due to lack of randomization and
blinding, as well as other shortcomings. Although a few
studies were rated as having a moderate risk of bias, none
of the studies was considered to be at low risk of bias. In
addition, although the findings of benefit were consistent
across most studies, the heterogeneity of multicomponent
interventions and the low number of studies evaluating
each specific intervention preclude identifying a particular
program as being the most beneficial, and these studies do
not address the question of which particular program components are most beneficial. Finally, the evidence was insufficient to determine the benefit of delirium prevention
programs in palliative care or long-term care settings.
Future comparative effectiveness studies with standardized protocols are needed, particularly to identify
which components in multicomponent interventions are
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
forming literature searches and Katherine Donahue and Lydia Dharia for
editing and formatting the manuscript.
Financial Support: From AHRQ, U.S. Department of Health and Human Services (contract HHSA-290-2007-10062I).
Potential Conflicts of Interest: Dr. Reston: Grant (money to institution):
AHRQ, U.S. Department of Health and Human Services. Dr. Schoelles:
Support for travel to meetings for the study or other purposes (money to
institution): Rand Corporation. Other (money to institution): Rand Corporation. Disclosures can also be viewed at www.acponline.org
/authors/icmje/ConflictOfInterestForms.do?msNum⫽M12-2566.
Requests for Single Reprints: James T. Reston, PhD, MPH, ECRI Institute, 5200 Butler Pike, Plymouth Meeting, PA 19462-1298; e-mail,
[email protected].
Current author addresses and author contributions are available at www
.annals.org.
References
1. Leslie DL, Marcantonio ER, Zhang Y, Leo-Summers L, Inouye SK. Oneyear health care costs associated with delirium in the elderly population. Arch
Intern Med. 2008;168:27-32. [PMID: 18195192]
2. Rudolph JL, Marcantonio ER. Review articles: postoperative delirium: acute
change with long-term implications. Anesth Analg. 2011;112:1202-11. [PMID:
21474660]
3. Björkelund KB, Hommel A, Thorngren KG, Gustafson L, Larsson S, Lundberg D. Reducing delirium in elderly patients with hip fracture: a multi-factorial
intervention study. Acta Anaesthesiol Scand. 2010;54:678-88. [PMID:
20236093]
4. Lundström M, Olofsson B, Stenvall M, Karlsson S, Nyberg L, Englund U,
et al. Postoperative delirium in old patients with femoral neck fracture: a randomized intervention study. Aging Clin Exp Res. 2007;19:178-86. [PMID:
17607084]
5. Rubin FH, Neal K, Fenlon K, Hassan S, Inouye SK. Sustainability and
scalability of the hospital elder life program at a community hospital. J Am
Geriatr Soc. 2011;59:359-65. [PMID: 21314654]
6. Chen CC, Lin MT, Tien YW, Yen CJ, Huang GH, Inouye SK. Modified
hospital elder life program: effects on abdominal surgery patients. J Am Coll Surg.
2011;213:245-52. [PMID: 21641835]
7. Inouye SK, Bogardus ST Jr, Williams CS, Leo-Summers L, Agostini JV. The
role of adherence on the effectiveness of nonpharmacologic interventions: evidence from the delirium prevention trial. Arch Intern Med. 2003;163:958-64.
[PMID: 12719206]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 379
Supplement
In-Facility Delirium Prevention Programs as a Patient Safety Strategy
8. Marcantonio ER, Flacker JM, Wright RJ, Resnick NM. Reducing delirium
after hip fracture: a randomized trial. J Am Geriatr Soc. 2001;49:516-22. [PMID:
11380742]
9. Wong DM, Niam T, Bruce JJ, Bruce DG. Quality project to prevent delirium after hip fracture. Aust J Ageing. 2005;24:174-7.
10. Inouye SK, Bogardus ST Jr, Charpentier PA, Leo-Summers L, Acampora
D, Holford TR, et al. A multicomponent intervention to prevent delirium in
hospitalized older patients. N Engl J Med. 1999;340:669-76. [PMID:
10053175]
11. Deschodt M, Braes T, Flamaing J, Detroyer E, Broos P, Haentjens P, et al.
Preventing delirium in older adults with recent hip fracture through multidisciplinary geriatric consultation. J Am Geriatr Soc. 2012;60:733-9. [PMID:
22429099]
12. Needham DM, Korupolu R, Zanni JM, Pradhan P, Colantuoni E, Palmer
JB, et al. Early physical medicine and rehabilitation for patients with acute respiratory failure: a quality improvement project. Arch Phys Med Rehabil. 2010;91:
536-42. [PMID: 20382284]
13. Tabet N, Hudson S, Sweeney V, Sauer J, Bryant C, Macdonald A, et al. An
educational intervention can prevent delirium on acute medical wards. Age Ageing. 2005;34:152-6. [PMID: 15713859]
14. Vidán MT, Sánchez E, Alonso M, Montero B, Ortiz J, Serra JA. An
intervention integrated into daily clinical practice reduces the incidence of delirium during hospitalization in elderly patients. J Am Geriatr Soc. 2009;57:202936. [PMID: 19754498]
15. Harari D, Hopper A, Dhesi J, Babic-Illman G, Lockwood L, Martin F.
Proactive care of older people undergoing surgery (‘POPS’): designing, embedding, evaluating and funding a comprehensive geriatric assessment service for
older elective surgical patients. Age Ageing. 2007;36:190-6. [PMID: 17259638]
16. Lundström M, Edlund A, Karlsson S, Brännström B, Bucht G, Gustafson
Y. A multifactorial intervention program reduces the duration of delirium, length
380 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
of hospitalization, and mortality in delirious patients. J Am Geriatr Soc. 2005;
53:622-8. [PMID: 15817008]
17. Allen KR, Fosnight SM, Wilford R, Benedict LM, Sabo A, Holder C, et al.
Implementation of a system-wide quality improvement project to prevent delirium in hospitalized patients. J Clin Outcomes Manag. 2011;18:253-8.
18. Black P, Boore JR, Parahoo K. The effect of nurse-facilitated family participation in the psychological care of the critically ill patient. J Adv Nurs. 2011;67:
1091-101. [PMID: 21214624]
19. Colombo R, Corona A, Praga F, Minari C, Giannotti C, Castelli A, et al.
A reorientation strategy for reducing delirium in the critically ill. Results of an
interventional study. Minerva Anestesiol. 2012;78:1026-33. [PMID: 22772860]
20. Martinez FT, Tobar C, Beddings CI, Vallejo G, Fuentes P. Preventing
delirium in an acute hospital using a non-pharmacological intervention. Age Ageing. 2012;41:629-34. [PMID: 22589080]
21. Gagnon P, Allard P, Gagnon B, Mérette C, Tardif F. Delirium prevention
in terminal cancer: assessment of a multicomponent intervention. Psychooncology. 2012;21:187-94. [PMID: 22271539]
22. Lapane KL, Hughes CM, Daiello LA, Cameron KA, Feinberg J. Effect of a
pharmacist-led multicomponent intervention focusing on the medication monitoring phase to prevent potential adverse drug events in nursing homes. J Am
Geriatr Soc. 2011;59:1238-45. [PMID: 21649623]
23. Tabet N, Stewart R, Hudson S, Sweeney V, Sauer J, Bryant C, et al. Male
gender influences response to an educational package for delirium prevention
among older people: a stratified analysis. Int J Geriatr Psychiatry. 2006;21:493-7.
[PMID: 16676296]
24. Rizzo JA, Bogardus ST Jr, Leo-Summers L, Williams CS, Acampora D,
Inouye SK. Multicomponent targeted intervention to prevent delirium in hospitalized older patients: what is the economic value? Med Care. 2001;39:740-52.
[PMID: 11458138]
www.annals.org
Annals of Internal Medicine
Current Author Addresses: Drs. Reston and Schoelles: ECRI Institute,
5200 Butler Pike, Plymouth Meeting, PA 19462-1298.
Author Contributions: Conception and design: J.T. Reston, K.M.
Schoelles.
Analysis and interpretation of the data: J.T. Reston, K.M. Schoelles.
Drafting of the article: J.T. Reston, K.M. Schoelles.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Critical revision of the article for important intellectual content: J.T.
Reston, K.M. Schoelles.
Final approval of the article: K.M. Schoelles.
Obtaining of funding: K.M. Schoelles.
Administrative, technical, or logistic support: K.M. Schoelles.
Collection and assembly of data: J.T. Reston.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) W-177
Supplement
Annals of Internal Medicine
Patient Safety Strategies Targeted at Diagnostic Errors
A Systematic Review
Kathryn M. McDonald, MM; Brian Matesic, BS; Despina G. Contopoulos-Ioannidis, MD; Julia Lonhart, BS, BA; Eric Schmidt, BA;
Noelle Pineda, BA; and John P.A. Ioannidis, MD, DSc
Missed, delayed, or incorrect diagnosis can lead to inappropriate
patient care, poor patient outcomes, and increased cost. This systematic review analyzed evaluations of interventions to prevent
diagnostic errors. Searches used MEDLINE (1966 to October 2012),
the Agency for Healthcare Research and Quality’s Patient Safety
Network, bibliographies, and prior systematic reviews. Studies that
evaluated any intervention to decrease diagnostic errors in any
clinical setting and with any study design were eligible, provided
that they addressed a patient-related outcome. Two independent
reviewers extracted study data and rated study quality.
There were 109 studies that addressed 1 or more intervention
categories: personnel changes (n ⫽ 6), educational interventions
(n ⫽ 11), technique (n ⫽ 23), structured process changes (n ⫽ 27),
THE PROBLEM
The family of patient safety targets that includes diagnostic errors has unclear boundaries. An operational definition includes diagnoses that are “unintentionally delayed
(sufficient information was available earlier), wrong (another diagnosis was made before the correct one), or
missed (no diagnosis was ever made), as judged from
the eventual appreciation of more definitive information”
(1, 2).
Although the definition is a bit fluid, there is no doubt
that the scope of the problem is large. A systematic review
of 53 series of autopsies reported a median antemortem
error rate of 23.5% (range, 4.1% to 49.8%) for major
errors (clinically missed diagnoses involving a principal underlying disease or primary cause of death) and 9.0%
(range, 0% to 20.7%) for incorrect diagnoses that are likely
to have affected patient outcomes (3). Disease-specific
studies show that 2% to 61% of patients experience missed
or delayed diagnoses (4). In a survey of pediatricians, 54%
admitted making a diagnostic error at least once per
month, and 45% noted making diagnostic errors that
harmed patients at least once per year (5). Lack of pertinent historical or clinical information and team processes
(for example, inadequate care coordination) contributed to
errors (5).
Furthermore, research on variation in patient outcomes related to diagnosis timing suggests that there is
room for improvement for some high-risk conditions. For
example, early identification of sepsis may decrease mortality in surgical intensive care (6).
Problems in care related to diagnosis are particularly
prevalent among precipitating causes for lawsuits; 25% to
59% of malpractice claims are attributable to diagnostic
errors (4, 7, 8). A recent study of 91 082 diagnosis-related
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
technology-based systems interventions (n ⫽ 32), and review
methods (n ⫽ 38). Of 14 randomized trials, which were rated as
having mostly low to moderate risk of bias, 11 reported interventions that reduced diagnostic errors. Evidence seemed strongest for
technology-based systems (for example, text message alerting) and
specific techniques (for example, testing equipment adaptations).
Studies provided no information on harms, cost, or contextual
application of interventions. Overall, the review showed a growing
field of diagnostic error research and categorized and identified
promising interventions that warrant evaluation in large studies
across diverse settings.
Ann Intern Med. 2013;158:381-389.
For author affiliations, see end of text.
www.annals.org
malpractice claims from 1986 to 2005 estimated payments
summing to $34.5 billion (inflation-adjusted to 2010 U.S.
dollars) (9). Among 10 739 malpractice claims from the
2005–2009 National Practitioner Data Bank, diagnosisrelated problems accounted for 45.9% of paid claims from
outpatient settings and 21.1% of paid claims from inpatient settings (10).
Some authors have asserted that diagnostic errors are
both more likely to result in patient harms and more preventable than treatment-related errors (such as wrong-site
surgery or incorrect medication dose), making the problem
particularly important to address (11). Given this potential, the purpose of this review is to assess the multitude of
interventions to prevent diagnostic errors and better understand their effectiveness.
PATIENT SAFETY STRATEGIES
There is a broad array of patient safety strategies
(PSSs) that could affect diagnostic errors. Approaches
might involve technical, cognitive, and systems-oriented
strategies, usually tailored to specific conditions or settings.
Strategies might address specific types of diagnostic
error, root causes of the error, or particular technologies
that are available. Strategies might target clinician errors
related to assessment (for example, failure or delay in considering an important diagnosis) or laboratory and radiology testing (including failure to order needed tests, techniSee also:
Web-Only
CME quiz (Professional Responsibilty Credit)
Supplement
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 381
Supplement
Patient Safety Strategies Targeted at Diagnostic Errors
Key Summary Points
Missed, delayed, or incorrect diagnosis can lead to
inappropriate patient care, poor patient outcomes,
and increased cost.
Patient safety strategies targeting diagnostic errors have
only recently been studied.
Approaches to reduce errors may involve technical, cognitive, and systems-oriented strategies tailored to specific
conditions or settings.
A framework that organizations might use to classify intervention strategies aimed at reducing diagnostic errors includes technique, personnel, education, structured process,
technology-based systems, and review methods.
Limited evidence from randomized, controlled trials shows
that some interventions, such as text messaging—a
technology-based systems strategy— can reduce diagnostic
errors in certain situations.
Very few studies of interventions to reduce diagnostic
errors have examined clinical outcomes (for example,
morbidity, mortality) or evaluated the utility of engaging
patients and families in prevention of diagnostic errors.
cal errors in processing specimens or tests, or erroneous
reading of tests) (2). Interventions that target such failure
areas might include tools that generate differential diagnosis lists based on algorithms and checklists; electronic monitoring of test result follow-up; and redesigned documentation systems that efficiently aggregate relevant evidence
and aid cognitive interpretation (2). Broad-based strategies
might target changes in residency training, board certification, and even patient and family engagement in diagnostic
problem solving.
Finally, many strategies could incorporate advances
in medical problem solving (including heuristics and
metacognition), decision analytic or normative decision
making, and clinical diagnostic decision support
(12–14). Strategies in this area— computerized diagnosis
management— could include computerized physician order entry with clinical decision support.
REVIEW PROCESSES
We captured relevant literature for review through 2
main mechanisms. First, we identified 2 key systematic
reviews that summarized data on system-related interventions addressing organizational vulnerabilities to diagnostic
errors (15) and cognitively related interventions that could
affect diagnosis (16). Then, we used broad search strategies
to identify additional literature. We searched MEDLINE
(1966 to October 2012), the Agency for Healthcare Research and Quality (AHRQ) Patient Safety Network (www
382 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
.psnet.ahrq.gov/), and bibliographies of background articles and previous systematic reviews to identify literature on
effects of interventions targeting diagnostic errors and/or diagnostic delays. The major Medical Subject Heading terms
were “diagnostic errors” and “delayed diagnosis.”
Eligible studies were those that evaluated any intervention to decrease diagnostic errors (incorrect diagnoses or
missed diagnoses) in any clinical setting and with any study
design, provided that they addressed patient-related outcomes (that is, the correct diagnosis was eventually confirmed through patient follow-up testing, surgery, autopsy,
or other means) or proxy measures of patient-related outcomes. We also considered studies that evaluated interventions intended to affect the time to correct diagnosis or
appropriate clinical action. We excluded studies in which
there was no intervention or no real patients (for example,
simulations), the intervention was not aimed to reduce diagnostic errors, or there were no patient outcomes or proxies thereof.
Two independent investigators screened articles for eligibility at the title and abstract level, and any discrepancies
about selection were resolved through discussion with the
entire research team. We also screened all of the studies
included in the reviews by Singh and colleagues (15) and
Graber and associates (16) and identified 23 studies that
were evaluations of interventions.
In total, we identified 109 articles that met inclusion
criteria. The Supplement (available at www.annals.org)
provides a complete description of the search strategies,
article flow diagram, and evidence tables.
We used AMSTAR, a tool that addresses such items as
the comprehensiveness of the search, the assessment of the
quality of included studies, and the methods for synthesizing the results, to assess the methodological quality of the 2
key systematic reviews (17). We used a standard risk of bias
assessment to evaluate quality of the randomized trials (Table 3 of the Supplement) (18). We developed and used a
categorization scheme to classify, from an organizational
perspective, interventions that target diagnostic errors
(Table). Categories included changes that an organization
might consider generically to reduce errors. Such changes
include techniques investment; personnel configurations;
additional review steps for higher reliability; structured
processes; education of professionals, patients, and families;
and information and communications technology– based
enhancements.
This review was supported by the AHRQ, which had
no role in the selection or review of the evidence or the
decision to submit this manuscript for publication.
BENEFITS
AND
HARMS
Benefits
Prior Systematic Reviews
Singh and colleagues (15) considered 43 diagnostic
error studies of systems interventions related to provider–
patient encounters, diagnostic test performance and interwww.annals.org
Patient Safety Strategies Targeted at Diagnostic Errors
pretation, follow-up and tracking, referral-related issues,
and patient-related issues. Their high-quality review (score
of 9 out of 9 relevant AMSTAR criteria) identified only 6
evaluations of interventions that met eligibility criteria for
our review. Three of the 6 reported diagnostic outcomes,
such as incidence of delayed diagnosis of injury, incidence
of missed injuries, or misdiagnosis rates. None provided
information on patients’ downstream clinical course.
Graber and colleagues (16) summarized 141 articles
on improving cognition and human factors affecting diagnosis. Their high-quality review (score of 9 out of 9 relevant AMSTAR criteria) included 42 evaluations of interventions. These investigators classified interventions in 3
dimensions. For interventions to increase knowledge and
expertise, only 1 (19) of 7 studies provided information on
diagnostic outcomes and clinical course for actual patients.
For interventions to improve intuitive and deliberate considerations, none of the 5 identified studies reported effects
on documented diagnoses with actual patients during
clinical course of care. In the largest group of studies—
interventions on getting help from colleagues, consultants,
and tools—16 of the 28 identified studies evaluated diagnostic outcomes in actual patients (20 –35).
Graber and colleagues noted the current scarcity of
evidence for any single intervention targeting cognitive and
human factors in reducing diagnostic error. They highlighted potential for interventions that target contentfocused training, feedback on performance, simulationbased training, metacognitive training, second opinion or
group decision making, and the use of decision support
tools and computer-aided technologies.
Studies of PSS Evaluations
We identified 109 studies, including 14 randomized
trials, of interventions that targeted diagnostic errors and
addressed patient-related outcomes (see Tables 1 to 4 of
the Supplement). Of the 6 categories of interventions,
most studies pertained to interventions in the categories of
technology-based systems and additional review methods
(Figure 1). Figure 2 shows increases over time in available
evidence related to the categories of additional review
methods, structured process changes, technique, and
technology-based systems interventions.
Patient-related outcomes and their proxies can be categorized as diagnostic accuracy outcomes (for example,
false-positive and false-negative results), management outcomes (for example, use of further diagnostic tests or therapeutic interventions), and direct patient outcomes (for
example, death, disease progression, or deterioration). An
intervention that leads to better diagnosis does not automatically change management or improve patient outcomes. Management change depends on treatment options
and the feasibility of implementing those options. Improvements in direct patient outcomes depend also on effectiveness of treatment or management. Outcomes that
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Table. Categories of Organizational Interventions to
Decrease Diagnostic Errors
Category
Example
Technique
Changes in equipment, procedures, and clinical
approaches that target diagnostic
performance in clinical practice
Introduction of additional health care members
and replacing certain professionals with
others
Implementation of educational strategies,
residency training curricula, and maintenance
of certification changes
Implementation of feedback loops or additional
stages in the diagnostic pathway
Implementation at the system level of
technology-based tools, such as computer
assistive diagnostic aids, decision-support
algorithms, text message alerting, and pager
alerts
Introduction of additional independent reviews
of test results, from reporting through
interpretation
Personnel changes
Educational
interventions
Structured process
changes
Technology-based
system interventions
Additional review
methods
were assessed in the 109 studies varied markedly, but few
studies (5 randomized, controlled trials and 8 other designs) evaluated direct patient-level clinical outcomes (6,
31, 36 – 46).
Results of Randomized, Controlled Trials
Primary and secondary outcomes that were assessed in
the 14 randomized trials are summarized in Table 2 of the
Supplement. Eight trials (9 comparisons) addressed diagnostic accuracy outcomes, and 3 trials (5 comparisons) addressed outcomes related to further diagnostic test use. Six
trials (8 comparisons) addressed outcomes related to further therapeutic management. Five trials (7 comparisons)
addressed direct patient-related outcomes. Three trials addressed composite outcomes (diagnostic accuracy and therapeutic management, and therapeutic management and
patient outcome). One trial addressed time to correct therapeutic management, and another trial addressed time to
diagnosis.
Trials evaluated various interventions. The control
group used most often was usual care. No trials had high
risk of bias, whereas 9 and 5 trials had moderate and low
risk of bias, respectively.
Statistically significant improvements were seen for at
least 1 outcome in all but 3 trials. Of the 3 trials with
non–statistically significant improvements, 1 was a noninferiority trial that showed no more diagnostic errors occurred during work-up of abdominal pain among patients
given morphine and those not given morphine (47). Two
trials that involved patients with mental conditions (46,
48) reported no beneficial diagnostic error effects from
computerized decision-support systems. Only 1 trial (42)
reported improvements in direct patient outcomes;
whether improvements were related to the comparison
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 383
Supplement
Patient Safety Strategies Targeted at Diagnostic Errors
or adding another professional to the care team. The 3
studies (71–73) in which a specialist was added to examine
the interpretation of a test result reported an increase in
case detection, although the studies were quite small and
targeted narrow patient populations. There was only 1 randomized trial, showing that emergency nurse practitioners
perform better than junior physicians (45).
Figure 1. Interventions, by type.
Education
8%
Technique
18%
Technology-based
systems
22%
Educational Interventions
Structured process
20%
Eleven studies (19, 43, 74 – 82) used educational interventions for various targets: patients, parents, community doctors, and intensive care unit doctors and nurses.
Strategies targeted at professionals produced improvements, but the studies were nonrandomized. Two randomized trials that targeted consumers found that parent education improved discrimination of serious symptoms
necessitating physician diagnosis and patient education improved the performance of breast cancer screening (74,
78).
Personnel
4%
Additional review
methods
28%
The percentage of studies as categorized by the 6 types of interventions.
against the randomized concurrent control group or a preintervention period was unclear.
Structured Process Changes
Twenty-seven studies (43, 44, 46, 48, 56 –59, 73, 77,
79, 83–98) examined interventions that added structure to
the diagnostic process. Structure included, among other
things, triage protocols, feedback steps, and quality improvement processes. Most interventions included the addition of a tool, often a checklist or a form (for example, to
guide and standardize physical examination of a patient).
Some of the studies centered on laboratory processes,
whereas others occurred during clinical management, often
in situations related to trauma patients. Beneficial effects
on diagnosis-related outcomes were seen in most nonrandomized studies, but of the 3 randomized trials, 2 did not
show benefit for improving diagnosis of mental illness (46,
48) and 1 had mixed results for a protocol for ordering
radiography in injured patients (84).
Technique
There were 23 studies of interventions related to medical techniques (39, 47, 49 – 69). Most of these studies,
including 3 randomized trials (47, 49, 55), found that
these interventions can enhance diagnosis (for example,
visual enhancements via ultrasonography-guided biopsy,
changes to number of biopsy cores, and cap-fitted colonoscopy) or not make it worse (for example, medical interventions for pain relief in patients with abdominal pain).
Personnel Changes
Six studies (44, 45, 70 –73) compared the effect on
diagnosis of substituting 1 type of professional for another,
Figure 2. Intervention studies, by year.
Studies, n
14
Additional review methods
12
Technique
10
Technology-based systems
Structured process
8
6
4
Education
Personnel
2
0
19
70
19
–1
76
97
5
19
–1
81
98
0
19
–1
86
98
5
19
–1
91
99
0
19
–1
96
99
5
20
–2
01
00
0
20
–2
06
00
5
–2
01
1
Published Year of Study
Timeline of the included studies categorized by the 6 types of interventions.
384 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Patient Safety Strategies Targeted at Diagnostic Errors
Technology-Based Systems Interventions
Thirty-two studies (6, 29 –36, 40 – 42, 44, 46, 60, 71,
78, 80, 97, 99 –111) included computerized decision support systems and alerting systems (for example, for abnormal laboratory results), most of which were associated with
improvements to processes on the diagnostic pathway (for
example, relaying a critical laboratory value to the clinician
in a more timely manner). Some interventions related to
specific symptoms (for example, a computer-aided diagnostic tool for abdominal pain interpretation), whereas others
intervened at the level of a particular test (for example, an
electronic medical record alert for a positive result on a
fecal occult blood screen for cancer). All 4 randomized
trials (31, 36, 42, 100) reported beneficial diagnostic error
effects (see Table 2 of the Supplement).
Additional Review Methods
The most common type of intervention that was evaluated was the introduction of redundancy in interpreting
test results (6, 20 –28, 34, 37–39, 72, 73, 76, 78, 79, 81,
95, 96, 109, 112–126). Most studies showed that an additional review step (usually by a separate reader, from the
same specialty or from another specialty) had a positive
effect on diagnostic performance. However, in some cases,
false-positive results also increased. Tradeoffs between sensitivity and specificity were reported erratically. Some studies targeted higher-risk patients for enriched review. However, the systems to support such targeting were neither
described nor evaluated. Randomized evidence was weak,
based on 1 group of 1 trial showing statistically significant
benefit (no effect size reported) for an audit and feedback
approach (78).
Studies With Interventions That Corresponded to Multiple
Categories
Twenty-four studies (6, 34, 39, 43, 44, 46, 56 – 60,
71–73, 76 – 80, 95–97, 109, 127) combined approaches in
a variety of ways and covered diverse clinical areas, with
mixed results. These studies are also included in the categories covered above. Twenty of the 24 studies combined 2
categories of intervention in almost every permutation possible (11 of 15 combinations). With only 1 to 4 studies for
any combination set, it is not possible to draw conclusions
about whether benefits are enhanced with more complex
interventions. Moreover, complex approaches may be more
costly, but this information was not reported.
Notifying Patients of Test Results
Another potential grouping of PSSs focuses on the
interface between the system and the patient, such as strategies that involve patient notification of test results (128).
No studies with comparative designs evaluated this intervention. The review by Singh and colleagues (15) identified 7 studies of patient preferences or satisfaction with
different options for receipt of test results. They also found
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
no studies that tested ways to reduce error using an intervention that affected test notification.
Casalino and colleagues (129) found a 7.1% rate of
apparent failures to inform patients of an abnormal test
result and identified a positive association between use of
simple processes by physician practices for managing results and lower failure rates. A systematic review that examined failures to follow up test results with ambulatory
care patients reported that failed follow-up ranged from
1.0% to 62.0%, depending on the type of test result, including failures associated with missed cancer diagnoses
(130). None of the studies included in that systematic review evaluated patient-oriented interventions.
Harms
No studies in our review evaluated direct patient
harm. Studies generally did not assess unintended adverse
effects, although some reported false-positive rates.
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
The context in which a safety strategy is implemented
depends on the specific type of diagnostic error and practice being examined. The studies that we reviewed covered
a range of subspecialties, settings, patient populations, and
interventions. Context varied greatly. Most interventions
were not tested in more than 1 site. Many studies were
small, early proof-of-concept evaluations. No information
was reported on the cost of implementing the reviewed
PSSs; costs would probably vary greatly, depending on the
particular strategy or practice.
DISCUSSION
This review identified over 100 evaluations of interventions to reduce diagnostic errors, many of which had a
reported positive effect on at least 1 end point, including
statistically significant improvements in at least 1 end point
in 11 of the 14 randomized trials. Mortality and morbidity
end points were seldom reported.
We also identified 2 previous systematic reviews of
cognitive and systems-oriented approaches to improve diagnostic accuracy that mostly found proof-of-concept strategies not yet tested in practice. Our review built on the
previous systematic reviews by grouping PSSs targeting diagnostic errors from an organizational perspective into
changes that an organization might consider more generically (techniques investment; personnel configurations; additional review steps for higher reliability; structured processes; education of professionals, patients, families; and
information and communications technology– based enhancements), as opposed to individual clinicians looking
for ways to improve their own cognitive processing in specific diagnostic contexts. Although many of the PSSs tested
thus far target diagnostic pathways for specific symptoms
or conditions, grouping interventions into common leverage points will support future development in this field by
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 385
Supplement
Patient Safety Strategies Targeted at Diagnostic Errors
the various stakeholders who seek to reduce diagnostic
problems. Involvement of patients and families has received minimal attention, with only 2 studies addressing
education of consumers.
Data synthesis is difficult because few studies have
used randomized designs, comparable outcomes, or similar
interventions packages. The existing literature may be susceptible to reporting biases favoring “positive” results for
different interventions. It is expected that with heightened
awareness of the problem, the number of studies in this
field will increase further in the future, including more
randomized trials and studies testing different approaches:
for example, policy-level efforts. However, the range of
outcomes assessed in the studies that we reviewed highlights the known lack of tools to routinely measure the
effect of interventions to decrease diagnostic errors. Additional work is needed on appropriate measurements of diagnostic errors and consequential delays in diagnosis. A
final limitation, especially for synthesis, is the diversity of
interventions that are reverse-engineered on the basis of
the many diagnostic targets; the diverse tailored needs for
each clinical situation (for example, protocols designed for
specific work-up pathways); and the variety of specialized
personnel, and even patients, receiving educational or
cognitive-support approaches.
Evidence is also lacking on the costs of interventions
and implementation, particularly how to reduce diagnostic
errors without producing other diagnostic problems, such
as overuse of tests. Eventually reaching the correct diagnosis with inefficient testing strategies (for example, some
sequences of multiple test ordering) is not the appropriate
pathway to improved diagnostic safety. Our review found a
paucity of studies that assessed both sensitivity and specificity of interventions addressing diagnostic performance in
the context of mitigating diagnostic errors. Thus, although
we found several promising interventions, evaluations need
to be strengthened before any specific PSSs are scaled up in
this domain.
In conclusion, our review demonstrates that the nascent field of diagnostic error research is growing, with new
interventions being tested that involve technical, cognitive,
and systems-oriented strategies. The framework of intervention types developed in the review provides a basis for
categorizing and designing new studies, especially randomized, controlled trials, in these areas.
From Stanford Center for Health Policy/Center for Primary Care and
Outcomes Research; Stanford University School of Medicine; Stanford
Prevention Research Center; School of Humanities and Sciences, Stanford University, Stanford, California; and Palo Alto Medical Foundation
Research Institute, Palo Alto, California.
Note: The AHRQ reviewed contract deliverables to ensure adherence to
contract requirements and quality, and a copyright release was obtained
from the AHRQ before the manuscript was submitted for publication.
386 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Disclaimer: All statements expressed in this work are those of the authors
and should not in any way be construed as official opinions or positions
of Stanford University, the AHRQ, or the U.S. Department of Health
and Human Services.
Financial Support: From the AHRQ, U.S. Department of Health and
Human Services (contract HHSA-290-2007-100621).
Potential Conflicts of Interest: Ms. McDonald: Grant (money to insti-
tution): AHRQ. Mr. Schmidt: Grant (money to institution): AHRQ. All
other authors had no disclosures to report. Disclosures can also be viewed
at www.acponline.org/authors/icmje/ConflictOflnterestForms.do?msNum
⫽M12-2571.
Requests for Single Reprints: Kathryn M. McDonald, MM, Stanford
University, 117 Encina Commons, Stanford, CA 94305-6019; e-mail,
[email protected].
Current author addresses and author contributions are available at www
.annals.org.
References
1. Graber ML. Next steps: envisioning a research agenda. Adv Health Sci Educ
Theory Pract. 2009;14 Suppl 1:107-12. [PMID: 19669917]
2. Schiff GD, Hasan O, Kim S, Abrams R, Cosby K, Lambert BL, et al.
Diagnostic error in medicine: analysis of 583 physician-reported errors. Arch
Intern Med. 2009;169:1881-7. [PMID: 19901140]
3. Shojania KG, Burton EC, McDonald KM, Goldman L. Changes in rates of
autopsy-detected diagnostic errors over time: a systematic review. JAMA. 2003;
289:2849-56. [PMID: 12783916]
4. Schiff GD, Kim S, Abrams R, Cosby K, Lambert B, Elstein AS, et al.
Diagnosing diagnosis errors: lessons from a multi-institutional collaborative project. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient
Safety: From Research to Implementation. vol 2. Rockville, MD; Agency for
Healthcare Research and Quality: 2005.
5. Singh H, Thomas EJ, Wilson L, Kelly PA, Pietz K, Elkeeb D, et al. Errors of
diagnosis in pediatric practice: a multisite survey. Pediatrics. 2010;126:70-9.
[PMID: 20566604]
6. Moore LJ, Jones SL, Kreiner LA, McKinley B, Sucher JF, Todd SR, et al.
Validation of a screening tool for the early identification of sepsis. J Trauma.
2009;66:1539-46. [PMID: 19509612]
7. Phillips RL Jr, Bartholomew LA, Dovey SM, Fryer GE Jr, Miyoshi TJ,
Green LA. Learning from malpractice claims about negligent, adverse events in
primary care in the United States. Qual Saf Health Care. 2004;13:121-6.
[PMID: 15744204]
8. Selbst SM. Pediatric emergency medicine: legal briefs. Pediatr Emerg Care.
2005;21:214-8.
9. Tehrani AS, Lee H, Mathews S, Shore A, Frick KD, Makary M, et al.
20-year summary of U.S. malpractice claims for diagnostic errors from 19852005 [Abstract]. 33rd Annual Meeting of the Society for Medical Decision Making, Chicago, Ilinois, 22–26 October 2011.
10. Bishop TF, Ryan AM, Casalino LP. Paid malpractice claims for adverse
events in inpatient and outpatient settings. JAMA. 2011;305:2427-31. [PMID:
21673294]
11. Ely JW, Graber ML, Croskerry P. Checklists to reduce diagnostic errors.
Acad Med. 2011;86:307-13. [PMID: 21248608]
12. Cosby KS. A framework for classifying factors that contribute to error in the
emergency department. Ann Emerg Med. 2003;42:815-23. [PMID: 14634609]
13. Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases.
Science. 1974;185:1124-31. [PMID: 17835457]
14. Metcalfe J, Shimamura AP. Metacognition: Knowing About Knowing.
Cambridge, MA: MIT Press; 1994.
15. Singh H, Graber ML, Kissam SM, Sorensen AV, Lenfestey NF, Tant EM,
et al. System-related interventions to reduce diagnostic errors: a narrative review.
BMJ Qual Saf. 2012;21:160-70. [PMID: 22129930]
www.annals.org
Patient Safety Strategies Targeted at Diagnostic Errors
16. Graber ML, Kissam S, Payne VL, Meyer AN, Sorensen A, Lenfestey N,
et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ
Qual Saf. 2012;21:535-57. [PMID: 22543420]
17. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al.
Development of AMSTAR: a measurement tool to assess the methodological
quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID:
17302989]
18. Assessing risk of bias in included studies. In: Higgins JP, Green S, eds.
Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.1. The
Cochrane Collaboration; September 2008. Accessed at www.cochrane-handbook
.org. on 6 September 2012.
19. Fridriksson S, Hillman J, Landtblom AM, Boive J. Education of referring
doctors about sudden onset headache in subarachnoid hemorrhage. A prospective
study. Acta Neurol Scand. 2001;103:238-42. [PMID: 11328195]
20. Raab SS, Stone CH, Jensen CS, Zarbo RJ, Meier FA, Grzybicki DM, et al.
Double slide viewing as a cytology quality improvement initiative. Am J Clin
Pathol. 2006;125:526-33. [PMID: 16627263]
21. Raab SS, Grzybicki DM, Mahood LK, Parwani AV, Kuan SF, Rao UN.
Effectiveness of random and focused review in detecting surgical pathology error.
Am J Clin Pathol. 2008;130:905-12. [PMID: 19019767]
22. Manion E, Cohen MB, Weydert J. Mandatory second opinion in surgical
pathology referral material: clinical consequences of major disagreements.
Am J Surg Pathol. 2008;32:732-7. [PMID: 18360282]
23. Nordrum I, Johansen M, Amin A, Isaksen V, Ludvigsen JA. Diagnostic
accuracy of second-opinion diagnoses based on still images. Hum Pathol. 2004;
35:129-35. [PMID: 14745735]
24. Hamady ZZ, Mather N, Lansdown MR, Davidson L, Maclennan KA.
Surgical pathological second opinion in thyroid malignancy: impact on patients’
management and prognosis. Eur J Surg Oncol. 2005;31:74-7. [PMID:
15642429]
25. Espinosa JA, Nolan TW. Reducing errors made by emergency physicians in
interpreting radiographs: longitudinal study. BMJ. 2000;320:737-40. [PMID:
10720354]
26. Duijm LE, Groenewoud JH, Fracheboud J, de Koning HJ. Additional
double reading of screening mammograms by radiologic technologists: impact on
screening performance parameters. J Natl Cancer Inst. 2007;99:1162-70.
[PMID: 17652282]
27. Kwek BH, Lau TN, Ng FC, Gao F. Non-consensual double reading in the
Singapore Breast Screening Project: benefits and limitations. Ann Acad Med
Singapore. 2003;32:438-41. [PMID: 12968545]
28. Canon CL, Smith JK, Morgan DE, Jones BC, Fell SC, Kenney PJ, et al.
Double reading of barium enemas: is it necessary? AJR Am J Roentgenol. 2003;
181:1607-10. [PMID: 14627582]
29. Pozen MW, D’Agostino RB, Selker HP, Sytkowski PA, Hood WB Jr. A
predictive instrument to improve coronary-care-unit admission practices in acute
ischemic heart disease. A prospective multicenter clinical trial. N Engl J Med.
1984;310:1273-8. [PMID: 6371525]
30. Selker HP, Beshansky JR, Griffith JL, Aufderheide TP, Ballin DS, Bernard
SA, et al. Use of the acute cardiac ischemia time-insensitive predictive instrument
(ACI-TIPI) to assist with triage of patients with chest pain or other symptoms
suggestive of acute cardiac ischemia. A multicenter, controlled clinical trial. Ann
Intern Med. 1998;129:845-55. [PMID: 9867725]
31. Bogusevicius A, Maleckas A, Pundzius J, Skaudickas D. Prospective randomised trial of computer-aided diagnosis and contrast radiography in acute
small bowel obstruction. Eur J Surg. 2002;168:78-83. [PMID: 12113275]
32. Ramnarayan P, Winrow A, Coren M, Nanduri V, Buchdahl R, Jacobs B,
et al. Diagnostic omission errors in acute paediatric practice: impact of a reminder
system on decision-making. BMC Med Inform Decis Mak. 2006;6:37. [PMID:
17087835]
33. Olsson SE, Ohlsson M, Ohlin H, Dzaferagic S, Nilsson ML, Sandkull P,
et al. Decision support for the initial triage of patients with acute coronary syndromes. Clin Physiol Funct Imaging. 2006;26:151-6. [PMID: 16640509]
34. Peldschus K, Herzog P, Wood SA, Cheema JI, Costello P, Schoepf UJ.
Computer-aided diagnosis as a second reader: spectrum of findings in CT studies
of the chest interpreted as normal. Chest. 2005;128:1517-23. [PMID:
16162752]
35. Kakeda S, Moriya J, Sato H, Aoki T, Watanabe H, Nakata H, et al.
Improved detection of lung nodules on chest radiographs using a commercial
computer-aided diagnosis system. AJR Am J Roentgenol. 2004;182:505-10.
[PMID: 14736690]
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
36. Kuperman GJ, Teich JM, Tanasijevic MJ, Ma’Luf N, Rittenberg E, Jha A,
et al. Improving response to critical laboratory results with automation: results of
a randomized controlled trial. J Am Med Inform Assoc. 1999;6:512-22. [PMID:
10579608]
37. Dudley M, Channer KS. Assessment of the value of technician reporting of
electrocardiographs in an accident and emergency department. J Accid Emerg
Med. 1997;14:307-10. [PMID: 9315933]
38. Nam YS, Pikarsky AJ, Wexner SD, Singh JJ, Weiss EG, Nogueras JJ, et al.
Reproducibility of colonic transit study in patients with chronic constipation. Dis
Colon Rectum. 2001;44:86-92. [PMID: 11805568]
39. Beigi B, Uddin JM, McMullan TF, Linardos E. Inaccuracy of diagnosis in
a cohort of patients on the waiting list for dacryocystorhinostomy when the
diagnosis was made by only syringing the lacrimal system. Eur J Ophthalmol.
2007;17:485-9. [PMID: 17671919]
40. Major K, Shabot MM, Cunneen S. Wireless clinical alerts and patient outcomes in the surgical intensive care unit. Am Surg. 2002;68:1057-60. [PMID:
12516808]
41. Etchells E, Adhikari NK, Wu R, Cheung M, Quan S, Mraz R, et al.
Real-time automated paging and decision support for critical laboratory abnormalities. BMJ Qual Saf. 2011;20:924-30. [PMID: 21725046]
42. Fitzgerald M, Cameron P, Mackenzie C, Farrow N, Scicluna P, Gocentas
R, et al. Trauma resuscitation errors and computer-assisted decision support.
Arch Surg. 2011;146:218-25. [PMID: 21339436]
43. Chern CH, How CK, Wang LM, Lee CH, Graff L. Decreasing clinically
significant adverse events using feedback to emergency physicians of telephone
follow-up outcomes. Ann Emerg Med. 2005;45:15-23. [PMID: 15635301]
44. Vernon DD, Furnival RA, Hansen KW, Diller EM, Bolte RG, Johnson
DG, et al. Effect of a pediatric trauma response team on emergency department
treatment time and mortality of pediatric trauma victims. Pediatrics. 1999;103:
20-4. [PMID: 9917434]
45. Sakr M, Angus J, Perrin J, Nixon C, Nicholl J, Wardrope J. Care of minor
injuries by emergency nurse practitioners or junior doctors: a randomised controlled trial. Lancet. 1999;354:1321-6. [PMID: 10533859]
46. Rollman BL, Hanusa BH, Lowe HJ, Gilbert T, Kapoor WN, Schulberg
HC. A randomized trial using computerized decision support to improve treatment of major depression in primary care. J Gen Intern Med. 2002;17:493-503.
[PMID: 12133139]
47. Thomas SH, Silen W, Cheema F, Reisner A, Aman S, Goldstein JN, et al.
Effects of morphine analgesia on diagnostic accuracy in emergency department
patients with abdominal pain: a prospective, randomized trial. J Am Coll Surg.
2003;196:18-31. [PMID: 12517545]
48. Schriger DL, Gibbons PS, Langone CA, Lee S, Altshuler LL. Enabling the
diagnosis of occult psychiatric illness in the emergency department: a randomized,
controlled trial of the computerized, self-administered PRIME-MD diagnostic
system. Ann Emerg Med. 2001;37:132-40. [PMID: 11174229]
49. Attard AR, Corlett MJ, Kidner NJ, Leslie AP, Fraser IA. Safety of early pain
relief for acute abdominal pain. BMJ. 1992;305:554-6. [PMID: 1393034]
50. Resnick NM, Brandeis GH, Baumann MM, DuBeau CE, Yalla SV. Misdiagnosis of urinary incontinence in nursing home women: prevalence and a
proposed solution. Neurourol Urodyn. 1996;15:599-613. [PMID: 8916113]
51. Borgstein PJ, Gordijn RV, Eijsbouts QA, Cuesta MA. Acute
appendicitis—a clear-cut case in men, a guessing game in young women. A
prospective study on the role of laparoscopy. Surg Endosc. 1997;11:923-7.
[PMID: 9294274]
52. Vermeulen B, Morabia A, Unger PF, Goehring C, Grangier C, Skljarov I,
et al. Acute appendicitis: influence of early pain relief on the accuracy of clinical
and US findings in the decision to operate—a randomized trial. Radiology. 1999;
210:639-43. [PMID: 10207461]
53. Prieto VG, Argenyi ZB, Barnhill RL, Duray PH, Elenitsas R, From L, et al.
Are en face frozen sections accurate for diagnosing margin status in melanocytic
lesions? Am J Clin Pathol. 2003;120:203-8. [PMID: 12931550]
54. Kokki H, Lintula H, Vanamo K, Heiskanen M, Eskelinen M. Oxycodone
vs placebo in children with undifferentiated abdominal pain: a randomized,
double-blind clinical trial of the effect of analgesia on diagnostic accuracy. Arch
Pediatr Adolesc Med. 2005;159:320-5. [PMID: 15809382]
55. Hewett DG, Rex DK. Cap-fitted colonoscopy: a randomized, tandem
colonoscopy study of adenoma miss rates. Gastrointest Endosc. 2010;72:775-81.
[PMID: 20579648]
56. Brössner C, Madersbacher S, Bayer G, Pycha A, Klingler HC, Maier U.
Comparative study of two different TRUS-guided sextant biopsy techniques in
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 387
Supplement
Patient Safety Strategies Targeted at Diagnostic Errors
detecting prostate cancer in one biopsy session. Eur Urol. 2000;37:65-71.
[PMID: 10671788]
57. Naughton CK, Miller DC, Mager DE, Ornstein DK, Catalona WJ. A
prospective randomized trial comparing 6 versus 12 prostate biopsy cores: impact
on cancer detection. J Urol. 2000;164:388-92. [PMID: 10893592]
58. Presti JC Jr, Chang JJ, Bhargava V, Shinohara K. The optimal systematic
prostate biopsy scheme should include 8 rather than 6 biopsies: results of a
prospective clinical trial. J Urol. 2000;163:163-6. [PMID: 10604337]
59. Ravery V, Goldblatt L, Royer B, Blanc E, Toublanc M, Boccon-Gibod L.
Extensive biopsy protocol improves the detection rate of prostate cancer. J Urol.
2000;164:393-6. [PMID: 10893593]
60. Weatherburn G, Bryan S, Nicholas A, Cocks R. The effect of a picture
archiving and communications system (PACS) on diagnostic performance in the
accident and emergency department. J Accid Emerg Med. 2000;17:180-4.
[PMID: 10819379]
61. Johnson AJ, Zywiel MG, Stroh A, Marker DR, Mont MA. Serological
markers can lead to false negative diagnoses of periprosthetic infections following
total knee arthroplasty. Int Orthop. 2011;35:1621-6. [PMID: 21181540]
62. Larson EM, O’Donnell M, Chamblee S, Horsburgh CR Jr, Marsh BJ,
Moreland JD, et al. Dual skin tests with Mycobacterium avium sensitin and PPD
to detect misdiagnosis of latent tuberculosis infection. Int J Tuberc Lung Dis.
2011;15:1504-9, i. [PMID: 22008764]
63. Maclean JE, Solomon M, Corey M, Selvadurai H. Cystic fibrosis newborn
screening does not delay the identification of cystic fibrosis in children with
negative results. J Cyst Fibros. 2011;10:333-7. [PMID: 21536503]
64. Bachur RG, Hennelly K, Callahan MJ, Chen C, Monuteaux MC. Diagnostic imaging and negative appendectomy rates in children: effects of age and
gender. Pediatrics. 2012;129:877-84. [PMID: 22508920]
65. Zheng Y, Hawkins L, Wolff J, Goloubeva O, Goldberg E. Detection of
lesions during capsule endoscopy: physician performance is disappointing.
Am J Gastroenterol. 2012;107:554-60. [PMID: 22233695]
66. Garcia EA, Lopez JR, Meier JL, Swislocki AL, Siegel D. Resistant hypertension and undiagnosed primary hyperaldosteronism detected by use of a computerized database. J Clin Hypertens (Greenwich). 2011;13:487-91. [PMID:
21762361]
67. Piliouras P, Allison S, Rosendahl C, Buettner PG, Weedon D. Dermoscopy
improves diagnosis of tinea nigra: a study of 50 cases. Australas J Dermatol.
2011;52:191-4. [PMID: 21834814]
68. Leufkens AM, DeMarco DC, Rastogi A, Akerman PA, Azzouzi K, Rothstein RI, et al; Third Eye Retroscope Randomized Clinical Evaluation
[TERRACE] Study Group. Effect of a retrograde-viewing device on adenoma
detection rate during colonoscopy: the TERRACE study. Gastrointest Endosc.
2011;73:480-9. [PMID: 21067735]
69. Kline JA, Hogg MM, Courtney DM, Miller CD, Jones AE, Smithline HA.
D-dimer threshold increase with pretest probability unlikely for pulmonary embolism to decrease unnecessary computerized tomographic pulmonary angiography. J Thromb Haemost. 2012;10:572-81. [PMID: 22284935]
70. de Lacey G, Barker A, Harper J, Wignall B. An assessment of the clinical
effects of reporting accident and emergency radiographs. Br J Radiol. 1980;53:
304-9. [PMID: 7378697]
71. Jacobs MJ, Edmondson MJ, Lowry JC. Accuracy of diagnosis of fractures by
maxillofacial and accident and emergency doctors using plain radiography compared with a telemedicine system: a prospective study. Br J Oral Maxillofac Surg.
2002;40:156-62. [PMID: 12180212]
72. Trotter MJ, Bruecks AK. Interpretation of skin biopsies by general pathologists: diagnostic discrepancy rate measured by blinded review. Arch Pathol Lab
Med. 2003;127:1489-92. [PMID: 14567717]
73. Tsai JJ, Yeun JY, Kumar VA, Don BR. Comparison and interpretation of
urinalysis performed by a nephrologist versus a hospital-based clinical laboratory.
Am J Kidney Dis. 2005;46:820-9. [PMID: 16253721]
74. McCarthy PL, Sznajderman SD, Lustman-Findling K, Baron MA, Fink
HD, Czarkowski N, et al. Mothers’ clinical judgment: a randomized trial of the
Acute Illness Observation Scales. J Pediatr. 1990;116:200-6. [PMID: 2405140]
75. Thaler T, Tempelmann V, Maggiorini M, Rudiger A. The frequency of
electrocardiographic errors due to electrode cable switches: a before and after
study. J Electrocardiol. 2010;43:676-81. [PMID: 20591441]
76. Seltzer SE, Hessel SJ, Herman PG, Swensson RG, Sheriff CR. Resident
film interpretations and staff review. AJR Am J Roentgenol. 1981;137:129-33.
[PMID: 6787863]
388 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
77. Gleadhill DN, Thomson JY, Simms P. Can more efficient use be made of
x ray examinations in the accident and emergency department? Br Med J (Clin
Res Ed). 1987;294:943-7. [PMID: 3107669]
78. McPhee SJ, Bird JA, Jenkins CN, Fordham D. Promoting cancer screening.
A randomized, controlled trial of three interventions. Arch Intern Med. 1989;
149:1866-72. [PMID: 2764657]
79. Kundel HL, Nodine CF, Krupinski EA. Computer-displayed eye position as
a visual aid to pulmonary nodule interpretation. Invest Radiol. 1990;25:890-6.
[PMID: 2394571]
80. Linver MN, Paster SB, Rosenberg RD, Key CR, Stidley CA, King WV.
Improvement in mammography interpretation skills in a community radiology
practice after dedicated teaching courses: 2-year medical audit of 38,633 cases.
Radiology. 1992;184:39-43. [PMID: 1609100]
81. Thomas HG, Mason AC, Smith RM, Fergusson CM. Value of radiograph
audit in an accident service department. Injury. 1992;23:47-50. [PMID:
1541500]
82. Itri JN, Kang HC, Krishnan S, Nathan D, Scanlon MH. Using focused
missed-case conferences to reduce discrepancies in musculoskeletal studies interpreted by residents on call. AJR Am J Roentgenol. 2011;197:W696-705.
[PMID: 21940542]
83. Enderson BL, Reath DB, Meadors J, Dallas W, DeBoo JM, Maull KI. The
tertiary trauma survey: a prospective study of missed injury. J Trauma. 1990;30:
666-9. [PMID: 2352294]
84. Klassen TP, Ropp LJ, Sutcliffe T, Blouin R, Dulberg C, Raman S, et al. A
randomized, controlled trial of radiograph ordering for extremity trauma in a
pediatric emergency department. Ann Emerg Med. 1993;22:1524-9. [PMID:
8214829]
85. Biffl WL, Harrington DT, Cioffi WG. Implementation of a tertiary trauma
survey decreases missed injuries. J Trauma. 2003;54:38-43. [PMID: 12544897]
86. Soundappan SV, Holland AJ, Cass DT. Role of an extended tertiary survey
in detecting missed injuries in children. J Trauma. 2004;57:114-8. [PMID:
15284560]
87. Perno JF, Schunk JE, Hansen KW, Furnival RA. Significant reduction in
delayed diagnosis of injury with implementation of a pediatric trauma service.
Pediatr Emerg Care. 2005;21:367-71. [PMID: 15942513]
88. Ursprung R, Gray JE, Edwards WH, Horbar JD, Nickerson J, Plsek P,
et al. Real time patient safety audits: improving safety every day. Qual Saf Health
Care. 2005;14:284-9. [PMID: 16076794]
89. Raab SS, Andrew-Jaja C, Condel JL, Dabbs DJ. Improving Papanicolaou
test quality and reducing medical errors by using Toyota production system
methods. Am J Obstet Gynecol. 2006;194:57-64. [PMID: 16389010]
90. Raab SS, Grzybicki DM, Sudilovsky D, Balassanian R, Janosky JE, Vrbin
CM. Effectiveness of Toyota process redesign in reducing thyroid gland fineneedle aspiration error. Am J Clin Pathol. 2006;126:585-92. [PMID: 16938657]
91. Raab SS, Tworek JA, Souers R, Zarbo RJ. The value of monitoring frozen
section-permanent section correlation data over time. Arch Pathol Lab Med.
2006;130:337-42. [PMID: 16519561]
92. Raab SS, Jones BA, Souers R, Tworek JA. The effect of continuous monitoring of cytologic-histologic correlation data on cervical cancer screening performance. Arch Pathol Lab Med. 2008;132:16-22. [PMID: 18181668]
93. Mueller CA, Klaassen-Mielke R, Penner E, Junius-Walker U, HummersPradier E, Theile G. Disclosure of new health problems and intervention planning using a geriatric assessment in a primary care setting. Croat Med J. 2010;
51:493-500. [PMID: 21162161]
94. de Vries EN, Eikens-Jansen MP, Hamersma AM, Smorenburg SM,
Gouma DJ, Boermeester MA. Prevention of surgical malpractice claims by use of
a surgical safety checklist. Ann Surg. 2011;253:624-8. [PMID: 21209590]
95. Ross PD, Huang C, Karpf D, Lydick E, Coel M, Hirsch L, et al. Blinded
reading of radiographs increases the frequency of errors in vertebral fracture detection. J Bone Miner Res. 1996;11:1793-800. [PMID: 8915788]
96. Goodyear N, Ulness BK, Prentice JL, Cookson BT, Limaye AP. Systematic
assessment of culture review as a tool to assess errors in the clinical microbiology
laboratory. Arch Pathol Lab Med. 2008;132:1792-5. [PMID: 18976017]
97. Lewis G, Sharp D, Bartholomew J, Pelosi AJ. Computerized assessment of
common mental disorders in primary care: effect on clinical outcome. Fam Pract.
1996;13:120-6. [PMID: 8732321]
98. Meier FA, Varney RC, Zarbo RJ. Study of amended reports to evaluate and
improve surgical pathology processes. Adv Anat Pathol. 2011;18:406-13.
[PMID: 21841408]
www.annals.org
Patient Safety Strategies Targeted at Diagnostic Errors
99. Wexler JR, Swender PT, Tunnessen WW Jr, Oski FA. Impact of a system
of computer-assisted diagnosis. Initial evaluation of the hospitalized patient.
Am J Dis Child. 1975;129:203-5. [PMID: 1091140]
100. Wellwood J, Johannessen S, Spiegelhalter DJ. How does computer-aided
diagnosis improve the management of acute abdominal pain? Ann R Coll Surg
Engl. 1992;74:40-6. [PMID: 1736794]
101. Poon EG, Kuperman GJ, Fiskio J, Bates DW. Real-time notification of
laboratory data requested by users through alphanumeric pagers. J Am Med
Inform Assoc. 2002;9:217-22. [PMID: 11971882]
102. Gur D, Sumkin JH, Rockette HE, Ganott M, Hakim C, Hardesty L, et al.
Changes in breast cancer detection and mammography recall rates after the introduction of a computer-aided detection system. J Natl Cancer Inst. 2004;96:
185-90. [PMID: 14759985]
103. Cupples TE, Cunningham JE, Reynolds JC. Impact of computer-aided
detection in a regional screening mammography program. AJR Am J Roentgenol.
2005;185:944-50. [PMID: 16177413]
104. Fenton JJ, Taplin SH, Carney PA, Abraham L, Sickles EA, D’Orsi C,
et al. Influence of computer-aided detection on performance of screening mammography. N Engl J Med. 2007;356:1399-409. [PMID: 17409321]
105. Park HI, Min WK, Lee W, Park H, Park CJ, Chi HS, et al. Evaluating the
short message service alerting system for critical value notification via PDA telephones. Ann Clin Lab Sci. 2008;38:149-56. [PMID: 18469361]
106. Piva E, Sciacovelli L, Zaninotto M, Laposata M, Plebani M. Evaluation of
effectiveness of a computerized notification system for reporting critical values.
Am J Clin Pathol. 2009;131:432-41. [PMID: 19228648]
107. Singh H, Wilson L, Petersen LA, Sawhney MK, Reis B, Espadas D, et al.
Improving follow-up of abnormal cancer screens using electronic health records:
trust but verify test result communication. BMC Med Inform Decis Mak. 2009;
9:49. [PMID: 20003236]
108. David CV, Chira S, Eells SJ, Ladrigan M, Papier A, Miller LG, et al.
Diagnostic accuracy in patients admitted to hospitals with cellulitis. Dermatol
Online J. 2011;17:1. [PMID: 21426867]
109. Jiang Y, Nishikawa RM, Schmidt RA, Toledano AY, Doi K. Potential of
computer-aided diagnosis to reduce variability in radiologists’ interpretations of
mammograms depicting microcalcifications. Radiology. 2001;220:787-94.
[PMID: 11526283]
110. Leaper DJ, Horrocks JC, Staniland JR, De Dombal FT. Computerassisted diagnosis of abdominal pain using “estimates” provided by clinicians.
Br Med J. 1972;4:350-4. [PMID: 4629240]
111. Nishikawa RM, Schmidt RA, Linver MN, Edwards AV, Papaioannou J,
Stull MA. Clinically missed cancer: how effectively can radiologists use computeraided detection? AJR Am J Roentgenol. 2012;198:708-16. [PMID: 22358014]
112. Ciatto S, Del Turco MR, Morrone D, Catarzi S, Ambrogetti D, Cariddi
A, et al. Independent double reading of screening mammograms. J Med Screen.
1995;2:99-101. [PMID: 7497164]
113. Howard J, Sundararajan R, Thomas SG, Walsh M, Sundararajan M.
Reducing missed injuries at a level II trauma center. J Trauma Nurs. 2006;13:
89-95. [PMID: 17052086]
114. Singh P, Warnakulasuriya S. The two-week wait cancer initiative on oral
cancer; the predictive value of urgent referrals to an oral medicine unit. Br Dent
J. 2006;201:717-20. [PMID: 17159958]
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
115. Bruner JM, Inouye L, Fuller GN, Langford LA. Diagnostic discrepancies
and their clinical impact in a neuropathology referral practice. Cancer. 1997;79:
796-803. [PMID: 9024718]
116. Carew-McColl M. Radiological interpretation in an accident and emergency department. Br J Clin Pract. 1983;37:375-7. [PMID: 6671078]
117. Galasko CS, Monahan PR. Value of re-examining x-ray films of outpatients
attending accident services. Br Med J. 1971;1:643-4. [PMID: 5548841]
118. Lind AC, Bewtra C, Healy JC, Sims KL. Prospective peer review in surgical
pathology. Am J Clin Pathol. 1995;104:560-6. [PMID: 7572817]
119. Lufkin KC, Smith SW, Matticks CA, Brunette DD. Radiologists’ review of
radiographs interpreted confidently by emergency physicians infrequently leads to
changes in patient management. Ann Emerg Med. 1998;31:202-7. [PMID:
9472181]
120. Murphy R, Slater A, Uberoi R, Bungay H, Ferrett C. Reduction of perception error by double reporting of minimal preparation CT colon. Br J Radiol.
2010;83:331-5. [PMID: 19651707]
121. Parameswaran L, Prihoda TJ, Sharkey FE. Diagnostic efficacy of additional
step-sections in colorectal biopsies originally diagnosed as normal. Hum Pathol.
2008;39:579-83. [PMID: 18289637]
122. Robson N, van Benthem PP, Gan R, Dixon AK. Casualty X-ray reporting:
a student survey. Clin Radiol. 1985;36:479-81. [PMID: 4075715]
123. Thiesse P, Ollivier L, Di Stefano-Louineau D, Négrier S, Savary J, Pignard K, et al. Response rate accuracy in oncology trials: reasons for interobserver
variability. Groupe Français d’Immunotherapie of the Fédération Nationale des
Centres de Lutte Contre le Cancer. J Clin Oncol. 1997;15:3507-14. [PMID:
9396404]
124. Westra WH, Kronz JD, Eisele DW. The impact of second opinion surgical
pathology on the practice of head and neck surgery: a decade experience at a large
referral hospital. Head Neck. 2002;24:684-93. [PMID: 12112543]
125. Buchner AM, Shahid MW, Heckman MG, Diehl NN, McNeil RB,
Cleveland P, et al. Trainee participation is associated with increased small adenoma detection. Gastrointest Endosc. 2011;73:1223-31. [PMID: 21481861]
126. Swanson JO, Thapa MM, Iyer RS, Otto RK, Weinberger E. Optimizing
peer review: a year of experience after instituting a real-time comment-enhanced
program at a children’s hospital. AJR Am J Roentgenol. 2012;198:1121-5.
[PMID: 22528902]
127. Thomas DC, Spitzer WO, MacFarlane JK. Inter-observer error among
surgeons and nurses in presymptomatic detection of breast disease. J Chronic Dis.
1981;34:617-26. [PMID: 7309826]
128. Davis Giardina T, Singh H. Should patients get direct access to their
laboratory test results? An answer with many questions. JAMA. 2011;306:
2502-3. [PMID: 22122864]
129. Casalino LP, Dunham D, Chin MH, Bielang R, Kistner EO, Karrison
TG, et al. Frequency of failure to inform patients of clinically significant outpatient test results. Arch Intern Med. 2009;169:1123-9. [PMID: 19546413]
130. Callen JL, Westbrook JI, Georgiou A, Li J. Failure to follow-up test results
for ambulatory patients: a systematic review. J Gen Intern Med. 2012;27:133448. [PMID: 22183961]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 389
Annals of Internal Medicine
Current Author Addresses: Ms. McDonald, Ms. Lonhart, and Mr.
Schmidt: Stanford Center for Health Policy/Center for Primary Care
and Outcomes Research, Stanford University, 117 Encina Commons,
Stanford, CA 94305-6019.
Mr. Matesic and Ms. Pineda: School of Medicine, Stanford University,
291 Campus Drive, Stanford, CA 94305.
Dr. Contopoulos-Ioannidis: Department of Pediatrics, Division of Infectious Diseases, Stanford University School of Medicine, 300 Pasteur
Drive, G312, Stanford, CA 94305.
Dr. Ioannidis: Stanford Prevention Research Center, Department of
Medicine, School of Medicine, Stanford University, 1265 Welch Road,
X306, Stanford, CA 94305.
W-178 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Author Contributions: Conception and design: K.M. McDonald,
B. Matesic, D.G. Contopoulos-Ioannidis, J. Lonhart, J.P.A. Ioannidis.
Analysis and interpretation of the data: K.M. McDonald, B. Matesic,
D.G. Contopoulos-Ioannidis, J. Lonhart, E. Schmidt, J.P.A. Ioannidis.
Drafting of the article: K.M. McDonald, B. Matesic, D.G. ContopoulosIoannidis, J. Lonhart, E. Schmidt, J.P.A. Ioannidis.
Critical revision of the article for important intellectual content: K.M.
McDonald, B. Matesic, D.G. Contopoulos-Ioannidis, J.P.A. Ioannidis.
Final approval of the article: K.M. McDonald, B. Matesic, D.G.
Contopoulos-Ioannidis, J. Lonhart, E. Schmidt, N. Pineda, J.P.A.
Ioannidis.
Provision of study materials or patients: J. Lonhart.
Statistical expertise: D.G. Contopoulos-Ioannidis, J.P.A. Ioannidis.
Obtaining of funding: K.M. McDonald.
Administrative, technical, or logistic support: K.M. McDonald, B.
Matesic, J. Lonhart, E. Schmidt, N. Pineda.
Collection and assembly of data: K.M. McDonald, B. Matesic, J. Lonhart, E. Schmidt, N. Pineda, J.P.A. Ioannidis.
www.annals.org
Supplement
Annals of Internal Medicine
Inpatient Fall Prevention Programs as a Patient Safety Strategy
A Systematic Review
Isomi M. Miake-Lye, BA; Susanne Hempel, PhD; David A. Ganz, MD, PhD; and Paul G. Shekelle, MD, PhD
Falls are common among inpatients. Several reviews, including 4
meta-analyses involving 19 studies, show that multicomponent programs to prevent falls among inpatients reduce relative risk for falls
by as much as 30%. The purpose of this updated review is to
reassess the benefits and harms of fall prevention programs in acute
care settings and to identify factors associated with successful implementation of these programs. We searched for new evidence
using PubMed from 2005 to September 2012. Two new, large,
randomized, controlled trials supported the conclusions of the existing meta-analyses. An optimal bundle of components was not
identified. Harms were not systematically examined, but potential
harms included increased use of restraints and sedating drugs and
decreased efforts to mobilize patients. Eleven studies showed that
the following themes were associated with successful implementation: leadership support, engagement of front-line staff in program
design, guidance of the prevention program by a multidisciplinary
committee, pilot-testing interventions, use of information technology systems to provide data about falls, staff education and training, and changes in nihilistic attitudes about fall prevention. Future
research would advance knowledge by identifying optimal bundles
of component interventions for particular patients and by determining whether effectiveness relies more on the mix of the components or use of certain implementation strategies.
THE PROBLEM
ications, and postural hypotension. The latter include poor
lighting; “trip” hazards, such as uneven flooring or small
objects on the floor; suboptimal chair heights; and limited
staff availability or skills. Because in-facility falls can be
precipitated by many factors and patients who fall often
have several risk factors, multicomponent interventions are
believed to be necessary for prevention. The purpose of this
updated review is to reassess the benefits and harms of
multicomponent inpatient programs for fall prevention
and to assess the factors associated with successful implementation of such programs.
The reported rate of falls in acute care hospitals ranges
from 1.3 to 8.9 per 1000 bed-days (1). Higher rates are
reported in neurology, geriatrics, and rehabilitation wards.
Because falls are probably underreported, most estimates
may be overly conservative (1). Defining a “fall” is a challenge in itself (2, 3). For example, the National Database of
Nursing Quality Indicators defines a fall as “an unplanned
descent to the floor with or without injury” (4), whereas
the World Health Organization defines a fall as “an event
which results in a person coming to rest inadvertently on
the ground or floor or some lower level” (5).
Regardless of the definition, falls occur frequently and
can have serious physical and psychological consequences.
Between 30% and 50% of in-facility falls result in injuries
(6, 7). Falls are associated with increased health care use,
including increased length of stay and higher rates of discharge from hospitals into long-term care facilities. Even a
fall that does not cause an injury can trigger a fear of
falling, anxiety, distress, depression, and reduced physical
activity. Family members, caregivers, and health care professionals are susceptible to overly protective or emotional
reactions to falls, which can affect the patient’s independence and rehabilitation.
A fall is often the result of interactions between
patient-specific risk factors and the physical environment.
The former risk factors include patient age (particularly
older than 85 years), history of a recent fall, mobility impairment, urinary incontinence or frequency, certain medSee also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
390 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Ann Intern Med. 2013;158:390-396.
For author affiliations, see end of text.
www.annals.org
PATIENT SAFETY STRATEGIES
All of the multicomponent fall prevention strategies in
recent meta-analyses included an assessment of fall risk (often the Morse Fall Scale [8] or St. Thomas’s Risk Assessment Tool in Falling Elderly Inpatients [9] is used). Table 1
lists additional components commonly included in multicomponent interventions. These typically include staff and
patient education, a bedside risk sign or an alert wristband,
attention to footwear, a toileting schedule, medication review, and a review after the fall to identify causes. Although most in-facility fall prevention programs are multicomponent interventions, none of the controlled trials
explicitly articulated a conceptual framework underpinning
its intervention. Individual components of published strategies varied in type, intensity, duration, and targeting, and
none of the trials that evaluated multicomponent interventions used the same combination of components. Table 1
of the Supplement (available at www.annals.org) shows
data about components of fall prevention strategies from
studies addressed in this review.
REVIEW PROCESSES
We identified 4 recent existing reviews that were relevant to the topic of inpatient fall prevention. Reviews of
www.annals.org
Inpatient Fall Prevention Programs as a Patient Safety Strategy
fall prevention in community-based settings were excluded.
To identify relevant reviews, we used methods described by
Whitlock and colleagues (10). See the Supplement for a
complete description of the search strategies, evidence tables, and a literature flow diagram. These reviews were
supplemented with the results of a search by Hempel and
colleagues (11), which was done for a report that addressed
prevention of inpatient falls. Hempel and coworkers used
16 existing reviews and reports to identify additional pertinent sources and searched PubMed, CINAHL, and the
Web of Science for relevant literature not yet covered in
reviews. The search included randomized, controlled trials;
nonrandomized trials; and before-and-after studies in
English-language publications that addressed falls in the
acute care hospital setting. Searches were conducted from
2005 through September 2012.
Previous Studies and Reviews
The 4 systematic reviews are a 2008 review from the
Cochrane Collaboration by Cameron and colleagues (12),
a 2008 review by Coussement and coworkers (13), a review
by Oliver and colleagues originally published in 2007 (14)
and then updated in 2010 as a narrative review (1), and a
2012 review by DiBardino and colleagues (15). All 4 reviews scored well on the assessment of multiple systematic
reviews (AMSTAR) criteria for systematic reviews (11 out
of 11, 10 out of 11, 10 out of 11, and 8 out of 11,
respectively), which evaluates such items as comprehensiveness of the search, assessment of the quality of included
studies, and methods for synthesizing the results (16). The
Cochrane review searched for randomized trials to assess
the effectiveness of fall reduction interventions for older
adults in nursing care facilities and hospitals (12). Of the
41 included trials, 11 were conducted in hospital settings,
4 of which addressed multicomponent interventions. The
review by Coussement and coworkers identified 4 multicomponent studies, 2 of which were included in the Cochrane review (13). The review by Oliver and colleagues
used broader inclusion criteria than the Cochrane review,
which led to the inclusion of 43 trials, case– control studies, and observational cohort studies (14). Thirteen of these
studies were classified as multicomponent inpatient interventions. Oliver and coworkers’ updated narrative review
focused directly on hospital fall prevention and discussed
17 multicomponent studies spanning from 1999 to 2009,
which include the 6 trials in the Cochrane and Coussement and colleagues’ reviews (1, 13). The recent review by
DiBardino and coworkers (15) identified 6 primary research studies in the acute care inpatient setting, 3 of
which were included in the Oliver and colleagues’ 2010
update.
Supplement
Key Summary Points
The rate of falls in acute care hospitals ranges from
approximately 1 to 9 per 1000 bed-days.
High-quality evidence shows that multicomponent interventions can reduce risk for in-hospital falls by as much
as 30%.
The optimal bundle of components is not established, but
common components include risk assessments for patients,
patient and staff education, bedside signs and wristband
alerts, footwear advice, scheduled and supervised toileting,
and a medication review.
Harms of multicomponent interventions are unclear because they have not been studied systematically, but they
may include the potential for increased use of restraints
and sedating drugs and decreased efforts to mobilize
patients.
Evidence about successful implementation of multicomponent interventions suggests that the following are important factors: leadership support, engagement of front-line
clinical staff in the design of the intervention, guidance by
a multidisciplinary committee, pilot-testing the intervention, and changing nihilistic attitudes about falls.
hospitals, in the general population or older adult population. We were looking for “pivotal studies,” defined by
Shojania and colleagues (17) as trials that may call into
question the results of an existing review. Studies were
screened by a clinician and nonclinician, each of whom
was experienced in systematic reviews. This search identified 2 new relevant studies, both of which showed statistically significant improvements in intervention groups
when compared with control groups and which we discuss
briefly later. We also describe a third study because of its
unique design. Because Oliver and coworkers’ 2010 update
used Downs and Black (18) to assess the quality of individual studies, we did the same for the 2 new studies. We
assessed the strength of evidence across studies using a
framework developed for the Agency for Healthcare Research and Quality patient safety review (19). To identify
studies in which a principal goal was reporting on implementation, we surveyed the results of our updated search
and queried experts for additional studies.
This review was supported by the Agency for Healthcare Research and Quality, which had no role in the selection or review of the evidence or the decision to submit the
manuscript for publication.
Supplemental Search
Our supplemental search started with studies identified by Hempel and coworkers, focusing on individual and
cluster randomized, controlled trials with large sample sizes
that assessed multicomponent interventions in acute care
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
BENEFITS
AND
HARMS
Benefits
Table 2 presents details about the 21 effectiveness
studies included in previous reviews or our updated search.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 391
Supplement
Inpatient Fall Prevention Programs as a Patient Safety Strategy
Table 1. Intervention Components in Studies of Inpatient
Falls Prevention Programs
Component
Studies Including This Component, n
Patient education
Bedside risk sign
Staff education
Alert wristband
Footwear
Review after fall
Toileting schedules
Medication review
Environment modification
Movement alarms
Bedrail review
Exercise
Hip protectors
Urine screening
Vest, belt, or cuff restraint
11
10
9
7
7
7
7
6
5
5
4
4
3
2
1
The 4 reviews we identified reached similar conclusions.
The reviews by Cameron and colleagues (12) and Oliver
and coworkers (14) found that multicomponent in-facility
prevention programs result in statistically and clinically significant reductions in rates of falls. Cameron and colleagues included 6478 older adults from 4 randomized trials in a pooled analysis that found a 31% decrease in the
rate of falling (pooled rate ratio [RR], 0.69 [95% CI, 0.49
to 0.96] and a 27% decrease in the incidence of falls when
compared with usual care among 3 trials involving 4824
participants (RR, 0.73 [CI, 0.56 to 0.96]) (12). Oliver and
coworkers (14) included 5 randomized trials and 8 beforeand-after studies in a pooled analysis that found an 18%
decrease in the rate of falling (RR, 0.82 [CI, 0.68 to 1.00]).
Coussement and colleagues (13) included 2 randomized
trials, 1 before-and-after study, and 1 cohort study and
found a pooled RR similar to that of Oliver and coworkers;
however, this effect was not quite statistically significant
(RR, 0.82 [CI, 0.65 to 1.03]). DiBardino and colleagues’
review (15) pooled data from 6 studies (including 1 randomized trial, 1 quasi-experimental study, and 4 beforeand-after studies) and found a pooled odds ratio of 0.90
(CI, 0.83 to 0.99). The studies included in these reviews
used interventions with 3 to 7 components and compared
them with control participants who received usual care (for
example, “control ward had no trial intervention” [23] and
control participants who “followed conventional routines” [33]).
We rated the first trial identified in our update search
as having a low risk of bias. In this cluster randomized trial,
Dykes and coworkers (24) compared the fall rates in 8
units at 4 urban U.S. hospitals over a 6-month period.
Control units in each hospital received usual care, which
included fall risk assessments, signage for high-risk patients, patient education, and manual documentation in
patient records. The intervention units at each hospital
tested the Falls Prevention Tool Kit, which was developed
by the study team. This kit is a health information tech392 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
nology application that includes a risk assessment and tailored signage, patient education, and plan-of-care components. Adjusted fall rates in the intervention units (3.15 per
1000 patient days [CI, 2.54 to 3.90]) were lower than
those of control units (4.18 per 1000 patient days [CI,
3.45 to 5.06]), yielding a rate difference of 1.03 (CI, 0.57
to 2.01). A particularly strong effect was found in patients
aged 65 years or older (rate difference, 2.08 per 1000 patient days [CI, 0.61 to 3.56]).
In the second study, which we also judged to have low
risk of bias, Ang and colleagues (20) randomly assigned
patients in 8 medical wards of an acute care hospital in
Singapore to a target intervention or usual care. An assessment tool was used to match high-risk patients with
appropriate interventions, in addition to an educational
session tailored to patient-specific risk factors, in the intervention group. Both groups received usual care, which included environmental modifications, review of medications
and fall history, and generic fall prevention advice. The
proportion of patients with at least 1 fall in the intervention group was 0.4% (CI, 0.2% to 1.1%), whereas in the
control group it was 1.5% (CI, 0.9% to 2.6%), for a relative risk reduction of 0.29 (CI, 0.1 to 0.87).
One other study worth noting, by van Gaal and colleagues (39, 40), evaluated a program that targeted 3 patient safety practices (pressure ulcers, urinary tract infections, and fall prevention) simultaneously. They found an
overall positive effect on development of any adverse event,
a composite measure of pressure ulcers, urinary tract infections, and falls. The study was not powered to assess falls
separately, but it is worth noting that the point estimate for
the relative risk reduction in falls was 0.69, which is within
the range of results reported in other studies and metaanalyses. The value of this study is the demonstration of
simultaneous improvements in several safety intervention targets that may be relevant to the same patient population.
Harms
Most trials of fall prevention programs did not report
any harms, although 1 reported constipation from intake
of vitamin D (13). Whether trials explicitly assessed the
possibility of harms was mostly unclear. Despite little empirical evidence, concern exists that some fall prevention
interventions may lead to harms. For example, Oliver and
colleagues (1) detailed many potential harms, including
those that would result from increased use of restraints or
sedating medications.
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
Structural organizational characteristics, existing quality and safety infrastructure, patient safety culture, teamwork, and leadership are believed to be important contexts
for understanding the effectiveness of fall prevention programs (41, 42).
www.annals.org
Inpatient Fall Prevention Programs as a Patient Safety Strategy
Supplement
Table 2. Abridged Evidence Tables*
Study, Year
(Reference)
Study Design
Setting
Participants
Quality
Score†
Outcome
Ang et al, 2011 (20)‡
Barker et al, 2009 (21)
Barry et al, 2001 (22)
RCT
Before-and-after
Before-and-after
Significantly fewer falls
Significantly fewer injuries
Significantly fewer injuries
Before-and-after
11
Nonsignificantly fewer falls
Cumming et al,
2008 (23)
Dykes et al,
2010 (24)‡
Cluster RCT
1822 patients
271 095 patients
All patients admitted to
95 beds for 3 y
All patients admitted to
500 beds for 2 y
3999 patients
25
16
15
Brandis, 1999 (7)
8 medical wards; acute care; Singapore
Small; acute care; Australia
Small; long-stay and rehabilitation;
Ireland
Acute; Australia
27
Nonsignificantly fewer falls
All patients admitted or
transferred to units over
6-mo study period
3961 patients
27
Significantly fewer falls
20
Significantly fewer falls
Cluster RCT
Fonda et al, 2006 (25)
Before-and-after
Grenier-Sennelier et al,
2002 (26)
Haines et al, 2004 (27)
Before-and-after
RCT
Healey et al, 2004 (28)
Cluster RCT
Koh et al, 2009 (29)
Krauss et al, 2008 (30)
Cluster RCT
Before-and-after
Mitchell and Jones,
1996 (31)
Oliver et al, 2002 (32)
Before-and-after
Schwendimann et al,
2006 (6)
Stenvall et al,
2007 (33)
Udén et al, 1999 (34)
van der Helm et al,
2006 (35)
Vassallo et al,
2004 (36)
von Renteln-Kruse and
Krause, 2007 (37)
Williams et al,
2007 (38)
Before-and-after
Before-and-after
RCT
Before-and-after
Before-and-after
Cohort
Before-and-after
Before-and-after
24 wards; acute and rehabilitation;
Australia
8 units; medical; urban United States
4 wards; elderly acute and
rehabilitation; Australia
400 beds; rehabilitation; France
3 wards; subacute, rehabilitation, and
elderly; Australia
8 wards; acute and rehabilitation;
3 hospitals; United Kingdom
2 hospitals; acute; Singapore
General medicine; acute academic
hospital; United States
1 acute and 1 subacute ward; 32 beds;
Australia
Elderly medical unit; acute hospital;
United Kingdom
300 beds; internal medical, geriatric,
and surgical; Switzerland
3 wards; orthogeriatric, geriatric,
orthopedic; Sweden
Geriatric department; acute hospital;
Sweden
Internal medical and neurology wards;
acute hospital; the Netherlands
3 wards; rehabilitation; United
Kingdom
Elderly acute and rehabilitation wards;
Germany
3 medical wards and 1 geriatric unit;
Australia
All admitted patients over
4y
626 patients
11
Significantly fewer falls
26
Significantly fewer falls
3386 patients
26
Nonsignificantly fewer falls
All admissions over 1.5 y
All admissions over 18 mo
14
18
Nonsignificantly fewer falls
Nonsignificantly fewer falls
All patients admitted to
32 beds for 6 mo
3200 patients admitted
annually; data over 2 y
34 972 admissions
16
Nonsignificantly fewer falls
15
Nonsignificantly fewer falls
199 patients
25
Significantly fewer falls
379 patients
12
Nonsignificantly greater falls
2670 patients
11
Nonsignificantly greater falls
825 patients
25
Nonsignificantly fewer falls
7254 patients
17
Significantly fewer falls
1357 admitted patients
during 6-mo
intervention
17
Significantly fewer falls
8
Nonsignificantly greater falls
RCT ⫽ randomized, controlled trial.
* From reference 1.
† Downs and Black Quality Score (18), evaluated by Oliver and colleagues (1)— except for entries in italics, which were evaluated by Ms. Miake-Lye and Dr. Shekelle.
‡ New studies added from updated search.
Structural Organizational Characteristics
Existing Infrastructure
Studies evaluating fall prevention programs were done
in various geographic areas and settings, including the
United States, Australia, the United Kingdom, Sweden,
Singapore, France, Switzerland, the Netherlands, and Germany (see Table 3 of the Supplement). Several were conducted in an academically affiliated or teaching hospital.
Sizes of hospitals varied from small (fewer than 100 beds)
to large (greater than 500). Some studies encompassed several hospitals (for example, 4), and others involved multiple wards. These data show that fall prevention programs
have been implemented in hospitals of varying size, location, and academic or teaching status. No studies reported
on financial concerns (for example, how patient care or the
interventions were financed), although 1 U.S. study mentioned the potential effect of reimbursement on the emphasis on fall prevention (24).
Existing organizational infrastructure was described
rarely, with only 5 of the 21 studies describing this for their
settings. In 4 studies, this description was limited to their
usual fall prevention care. The fifth study provided a more
explicit statement, namely, “prior to this study none of the
wards carried out specific fall assessments or interventions . . . there was no specialist falls clinic or other falls
service available at this hospital” (28). Two studies reported on the presence or absence of information systems
that could be used in fall prevention programs (24, 26).
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Patient Safety Culture, Teamwork, and Leadership
Although some studies briefly mentioned patient
safety culture, teamwork, or leadership, only 4 studies presented expanded explanations of those factors. GrenierSennelier and colleagues (26) used a framework from Shor5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 393
Supplement
Inpatient Fall Prevention Programs as a Patient Safety Strategy
tell and coworkers (43) and Gillies and colleagues (44) to
analyze culture at the unit level, teamwork at both the
organizational and unit levels, and leadership at the organizational and unit levels. Stenvall and colleagues (33) discussed teamwork at the unit level. Koh and coworkers (29)
discussed leadership on the organizational and unit levels.
van der Helm and colleagues (35) made several observations addressing leadership on both the organizational and
unit levels.
Implementation
Implementation details are also considered to be important in understanding the effectiveness of fall prevention programs (41). The most commonly reported implementation details in the 21 studies were patient
characteristics and the initial plan, or the intended intervention components. Some studies reported the intended
roles of project staff, or by whom the intended intervention
components were to be completed. Most studies reported
the recipients of any training component, with slightly
fewer reporting the type of training or giving a description
of the training and even fewer studies reporting the length
of training. Thus, the context and duration of training
needed to implement fall prevention programs need better
descriptions.
Several studies provided the materials used in program
implementation, and some reported on adherence or fidelity to the designed initiative and how and why the plan
evolved. Adherence or fidelity was most often characterized
in a qualitative statement. According to Brandis (7): “The
strategies implemented . . . had high acceptance by staff.”
Williams and colleagues (38) found staff involvement crucial to fidelity: “[I]nvolving ward staff . . . so that they take
ownership of the project and do not perceive it as being
driven by middle management were important strategies.”
Dykes and coworkers (24) provided a strong example of
adherence reporting, in which protocol adherence was
measured by completion of components in both control
(81%) and intervention (94%) wards. Such quantitative
data on protocol adherence should be encouraged in future
evaluations of fall prevention programs. Measures of adoption and reach were usually provided in the form of a flow
chart— 6 studies presented these data for providers, and 8
presented the data for patients.
In addition to the studies previously discussed, we
identified 11 studies that focused primarily on implementation. None were randomized, clinical trials and all studies
had either pre–post or time-series designs. Six studies were
poststudy evaluations of fall prevention implementations
that reported detail about the potential reasons for effectiveness or lack thereof. Nine of the 11 studies assessed
implementation at only 1 or 2 facilities. Four studies reported no beneficial effects of the fall prevention program
and highlighted potential implementation factors that may
account for the lack of success. One study explicitly assessed the effect of some contextual factors on intervention
394 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
success across 34 facilities (described later) (45). One study
explicitly assessed sustainability. From these 11 studies, we
identified the following 7 themes about effective implementation: leadership support is critical, both at the facility
level (for example, hospital director) and at the unit level
(for example, unit director or “clinical champions”); engagement of front-line clinical staff in the design of the
intervention helps ensure that it will mesh with existing
clinical procedures; use of multidisciplinary committees is
needed to guide or oversee the interventions; the intervention should be pilot-tested to help identify potential problems with implementation; information systems that are
capable of providing data about falls can facilitate evaluation of the causes and adherence to the intervention components and potentially be a crucial facilitator of the intervention; changing the prevailing nihilistic attitude that falls
are “inevitable” and that “nothing can be done” is required
to get buy-in to the goals of the intervention (46, 47); and
education and training of clinical staff are necessary to help
ensure that adherence does not diminish. Table 5 of the
Supplement presents evidence from the 11 studies supporting each theme.
Costs
The Cochrane review found no economic evaluations
of the fall prevention programs that met inclusion criteria
(12). Oliver and colleagues (1) estimated the cost for specific combinations of components in terms of environment
and equipment and in terms of staff; most costs were low
or inconsequential.
The Effects of Context on Effectiveness
We identified only 1 study that explicitly assessed the
effect of context on effectiveness (45). Across 34 Veterans
Affairs health centers (a mix of acute care and long-term
care facilities), leadership support was cited as one of the
strongest factors for success. At 1-year follow-up, highperforming sites reported greater agreement with questions
assessing leadership support, teamwork skills, and useful
information systems than low-performing sites.
DISCUSSION
The evidence base indicates that inpatient multicomponent programs are effective at reducing falls and that
consistent themes are associated with successful implementation. However, there is no strong evidence about which
components are most important for success. The effects of
context have not been well-studied; however, multicomponent interventions have been effective in hospitals that vary
in size, location, and teaching status. The cost of implementing fall prevention programs has not been rigorously
assessed but generally does not involve capital expenses or
hiring new staff.
Our results about effectiveness are consistent with previous reviews on inpatient fall prevention programs. Our
review additionally identifies 7 themes associated with sucwww.annals.org
Inpatient Fall Prevention Programs as a Patient Safety Strategy
cessful implementation. Some themes, such as education or
training and leadership support, are often included in general lists of factors for successful implementation of any
intervention, whereas themes that may be more specific to
fall prevention programs include development and guidance by a multidisciplinary committee and changing the
prevailing attitudes of nihilism with respect to falls.
Our findings that multicomponent fall prevention
programs are effective in inpatient settings may seem at
odds with recent U.S. Preventive Services Task Force recommendations not to automatically do a multifactorial fall
assessment in community-dwelling adults aged 65 years or
older (48). However, there is no contradiction because,
although the goal is to prevent falls in both communitydwelling and hospitalized patients, the settings are different. The hospital environment is more tightly controlled
than the outpatient setting, where it is more difficult to
ensure that risk factors for falls are appropriately managed.
In fact, as Tinetti and Brach (49) note, community-based
multifactorial programs achieve greater reduction in falls
when identified risk factors are actually managed.
Our review has several limitations. Like all reviews, we
are limited by the quality and quantity of the original research articles. Also, we did not do an exhaustive update of
existing reviews. With several previous reviews reaching
consistent results, including a total of 19 effectiveness studies, we focused instead on identifying “pivotal studies” that
may call into question the conclusions of previous reviews.
None were found; additional large randomized, controlled
trials supported the conclusions of existing reviews. Our
assessment of implementation themes is novel and deserves
prospective evaluation (for example, one that could measure the degree of leadership support or staff attitudes
about fall prevention before and during an intervention).
For multicomponent inpatient fall programs, our review provides both evidence that such programs reduce
falls and insight into how facilities can successfully implement them. Future research would most effectively advance the field by determining whether an “optimal”
bundle of components exists or whether effectiveness is
primarily a function of successful implementation.
From the Veterans Affairs Greater Los Angeles Healthcare System and
David Geffen School of Medicine at the University of California, Los
Angeles, Los Angeles, and the RAND Corporation, Santa Monica,
California.
Note: The Agency for Healthcare Research and Quality (AHRQ) re-
viewed contract deliverables to ensure adherence to contract requirements and quality, and a copyright release was obtained from the AHRQ
before submission of the manuscript.
Disclaimer: All statements expressed in this work are those of the authors
and should not in any way be construed as official opinions or positions
of the RAND Corporation; U.S. Department of Veterans Affairs; University of California, Los Angeles; the AHRQ; or U.S. Department of
Health and Human Services.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Acknowledgment: The authors thank Aneesa Motala, BA; Sydne New-
berry, PhD; and Roberta Shanman, MLS.
Financial Support: From the AHRQ, U.S. Department of Health and
Human Services (contracts HHSA-290-2007-10062I, HHSA-290-201000017I, and HHSA-290-32001T). Dr. Ganz was supported by a Career
Development Award from the Veterans Affairs Health Services Research
& Development Service, Veterans Health Administration, U.S. Department of Veterans Affairs through the Veterans Affairs Greater Los Angeles Health Services Research & Development Center of Excellence
(project VA CD2 08-012-1).
Potential Conflicts of Interest: Dr. Hempel: Grant (money to institution): AHRQ. Dr. Ganz: Grant (money to institution): AHRQ, Veterans
Affairs Health Services Research and Development Service. Dr. Shekelle:
Consultancy: ECRI Institute; Employment: Veterans Affairs; Grants/grants
pending: AHRQ, Veterans Affairs, Centers for Medicare & Medicaid
Services, National Institute of Nursing Research, Office of the National
Coordinator; Royalties: UpToDate. Ms. Miake-Lye: None disclosed.
Disclosures can also be viewed at www.acponline.org/authors/icmje
/ConflictOfInterestForms.do?msNum⫽M12-2569.
Requests for Single Reprints: Paul G. Shekelle, MD, PhD, RAND
Corporation, 1776 Main Street, Santa Monica, CA 90401; e-mail,
[email protected].
Current author addresses and author contributions are available at
www.annals.org.
References
1. Oliver D, Healey F, Haines TP. Preventing falls and fall-related injuries in
hospitals. Clin Geriatr Med. 2010;26:645-92. [PMID: 20934615]
2. Zecevic AA, Salmoni AW, Speechley M, Vandervoort AA. Defining a fall and
reasons for falling: comparisons among the views of seniors, health care providers,
and the research literature. Gerontologist. 2006;46:367-76. [PMID: 16731875]
3. Schwenk M, Lauenroth A, Stock C, Moreno RR, Oster P, McHugh G, et al.
Definitions and methods of measuring and reporting on injurious falls in randomised controlled fall prevention trials: a systematic review. BMC Med Res
Methodol. 2012;12:50. [PMID: 22510239]
4. National Database of Nursing Quality Indicators. Guidelines for Data Collection on the American Nurses Association’s National Quality Forum Endorsed
Measures: Nursing Care Hours per Patient Day, Skill Mix, Falls, Falls with
Injury. National Center for Nursing Quality; March 2012. Accessed at www.odh
.ohio.gov/~/media/ODH/ASSETS/Files/dspc/health%20care%20service/nurse
staffingmaterials8-2-2010.ashx on 7 January 2013.
5. World Health Organization. Violence and Injury Prevention: Falls. 2012.
Accessed at www.who.int/violence_injury_prevention/other_injury/falls/en on 25
July 2012.
6. Schwendimann R, Bühler H, De Geest S, Milisen K. Falls and consequent
injuries in hospitalized patients: effects of an interdisciplinary falls prevention
program. BMC Health Serv Res. 2006;6:69. [PMID: 16759386]
7. Brandis S. A collaborative occupational therapy and nursing approach to falls
prevention in hospital inpatients. J Qual Clin Pract. 1999;19:215-20. [PMID:
10619149]
8. Morse JM. Preventing Patient Falls. Thousand Oaks, CA: Sage; 1997.
9. Oliver D, Britton M, Seed P, Martin FC, Hopper AH. Development and
evaluation of evidence based risk assessment tool (STRATIFY) to predict which
elderly inpatients will fall: case-control and cohort studies. BMJ. 1997;315:104953. [PMID: 9366729]
10. Whitlock EP, Lin JS, Chou R, Shekelle P, Robinson KA. Using existing
systematic reviews in complex systematic reviews. Ann Intern Med. 2008;148:
776-82. [PMID: 18490690]
11. Hempel S, Newberry S, Wang Z, Shekelle PG, Shanman RM, Johnsen B,
et al. Review of the Evidence on Falls Prevention in Hospitals: Task 4 Final
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 395
Supplement
Inpatient Fall Prevention Programs as a Patient Safety Strategy
Report. Santa Monica, CA: RAND Corporation; 2012. Accessed at www.rand
.org/pubs/working_papers/WR907.html on 7 January 2013.
12. Cameron ID, Murray GR, Gillespie LD, Robertson MC, Hill KD, Cumming RG, et al. Interventions for preventing falls in older people in nursing care
facilities and hospitals. Cochrane Database Syst Rev. 2010:CD005465. [PMID:
20091578]
13. Coussement J, De Paepe L, Schwendimann R, Denhaerynck K, Dejaeger
E, Milisen K. Interventions for preventing falls in acute- and chronic-care hospitals: a systematic review and meta-analysis. J Am Geriatr Soc. 2008;56:29-36.
[PMID: 18031484]
14. Oliver D, Connelly JB, Victor CR, Shaw FE, Whitehead A, Genc Y, et al.
Strategies to prevent falls and fractures in hospitals and care homes and effect of
cognitive impairment: systematic review and meta-analyses. BMJ. 2007;334:82.
[PMID: 17158580]
15. DiBardino D, Cohen ER, Didwania A. Meta-analysis: multidisciplinary fall
prevention strategies in the acute care inpatient population. J Hosp Med. 2012;
7:497-503. [PMID: 22371369]
16. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al.
Development of AMSTAR: a measurement tool to assess the methodological
quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID:
17302989]
17. Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How
quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med.
2007;147:224-33. [PMID: 17638714]
18. Downs SH, Black N. The feasibility of creating a checklist for the assessment
of the methodological quality both of randomised and non-randomised studies of
health care interventions. J Epidemiol Community Health. 1998;52:377-84.
[PMID: 9764259]
19. Shekelle PG, Wachter RM, Pronovost PJ, Schoelles K, McDonald KM, Dy
SM, et al. Making Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient Safety Practices. (Prepared by the Southern California-RAND
Evidence-based Practice Center under contract HHSA290200710062I.) Rockville,
MD: Agency for Healthcare Research and Quality; 2013. [Forthcoming].
20. Ang E, Mordiffi SZ, Wong HB. Evaluating the use of a targeted multiple
intervention strategy in reducing patient falls in an acute care hospital: a randomized controlled trial. J Adv Nurs. 2011;67:1984-92. [PMID: 21507049]
21. Barker A, Kamar J, Morton A, Berlowitz D. Bridging the gap between
research and practice: review of a targeted hospital inpatient fall prevention programme. Qual Saf Health Care. 2009;18:467-72. [PMID: 19955459]
22. Barry E, Laffoy M, Matthews E, Carey D. Preventing accidental falls among
older people in long stay units. Ir Med J. 2001;94:172, 174-6. [PMID:
11495234]
23. Cumming RG, Sherrington C, Lord SR, Simpson JM, Vogler C, Cameron
ID, et al; Prevention of Older People’s Injury Falls Prevention in Hospitals
Research Group. Cluster randomised trial of a targeted multifactorial intervention to prevent falls among older people in hospital. BMJ. 2008;336:758-60.
[PMID: 18332052]
24. Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, et al. Fall
prevention in acute care hospitals: a randomized trial. JAMA. 2010;304:1912-8.
[PMID: 21045097]
25. Fonda D, Cook J, Sandler V, Bailey M. Sustained reduction in serious
fall-related injuries in older people in hospital. Med J Aust. 2006;184:379-82.
[PMID: 16618235]
26. Grenier-Sennelier C, Lombard I, Jeny-Loeper C, Maillet-Gouret MC, Minvielle E. Designing adverse event prevention programs using quality management
methods: the case of falls in hospital. Int J Qual Health Care. 2002;14:419-26.
[PMID: 12389808]
27. Haines TP, Bennell KL, Osborne RH, Hill KD. Effectiveness of targeted
falls prevention programme in subacute hospital setting: randomised controlled
trial. BMJ. 2004;328:676. [PMID: 15031238]
28. Healey F, Monro A, Cockram A, Adams V, Heseltine D. Using targeted risk
factor reduction to prevent falls in older in-patients: a randomised controlled trial.
Age Ageing. 2004;33:390-5. [PMID: 15151914]
29. Koh SL, Hafizah N, Lee JY, Loo YL, Muthu R. Impact of a fall prevention
programme in acute hospital settings in Singapore. Singapore Med J. 2009;50:
425-32. [PMID: 19421690]
396 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
30. Krauss MJ, Tutlam N, Costantinou E, Johnson S, Jackson D, Fraser VJ.
Intervention to prevent falls on the medical service in a teaching hospital. Infect
Control Hosp Epidemiol. 2008;29:539-45. [PMID: 18476777]
31. Mitchell A, Jones N. Striving to prevent falls in an acute care setting—action
to enhance quality. J Clin Nurs. 1996;5:213-20. [PMID: 8718053]
32. Oliver D, Martin F, Seed P. Preventing patient falls [Letter]. Age Ageing.
2002;31:75-6. [PMID: 11850313]
33. Stenvall M, Olofsson B, Lundström M, Englund U, Borssén B, Svensson
O, et al. A multidisciplinary, multifactorial intervention program reduces postoperative falls and injuries after femoral neck fracture. Osteoporos Int. 2007;18:
167-75. [PMID: 17061151]
34. Udén G, Ehnfors M, Sjöström K. Use of initial risk assessment and recording as the main nursing intervention in identifying risk of falls. J Adv Nurs.
1999;29:145-52. [PMID: 10064293]
35. van der Helm J, Goossens A, Bossuyt P. When implementation fails: the
case of a nursing guideline for fall prevention. Jt Comm J Qual Patient Saf.
2006;32:152-60. [PMID: 16617946]
36. Vassallo M, Vignaraja R, Sharma JC, Hallam H, Binns K, Briggs R, et al.
The effect of changing practice on fall prevention in a rehabilitative hospital: the
Hospital Injury Prevention Study. J Am Geriatr Soc. 2004;52:335-9. [PMID:
14962145]
37. von Renteln-Kruse W, Krause T. Incidence of in-hospital falls in geriatric patients before and after the introduction of an interdisciplinary team-based fallprevention intervention. J Am Geriatr Soc. 2007;55:2068-74. [PMID: 17971140]
38. Williams TA, King G, Hill AM, Rajagopal M, Barnes T, Basu A, et al.
Evaluation of a falls prevention programme in an acute tertiary care hospital.
J Clin Nurs. 2007;16:316-24. [PMID: 17239067]
39. van Gaal BG, Schoonhoven L, Hulscher ME, Mintjes JA, Borm GF, Koopmans RT, et al. The design of the SAFE or SORRY? study: a cluster randomised
trial on the develpment and testing of an evidence based inpatient safety program
for the prevention of adverse events. BMC Health Serv Res. 2009;9:58. [PMID:
19338655]
40. van Gaal BG, Schoonhoven L, Mintjes JA, Borm GF, Hulscher ME, Defloor T, et al. Fewer adverse events as a result of the SAFE or SORRY? programme in hospitals and nursing homes. part i: primary outcome of a cluster
randomised trial. Int J Nurs Stud. 2011;48:1040-8. [PMID: 21419411]
41. Shekelle PG, Pronovost P, Wachter R, Taylor S, Dy S, Foy R, et al; PSP
Technical Expert Panel. Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria. (Prepared under
contract HHSA-290-2009-10001C.) AHRQ publication no. 11-0006-EF.
Rockville, MD: Agency for Healthcare Research and Quality; 2010. Accessed at
www.ahrq.gov/qual/contextsensitive on 7 January 2013.
42. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ.
Making health care safer: a critical analysis of patient safety practices. Evid Rep
Technol Assess (Summ). 2001:i-x, 1-668. [PMID: 11510252]
43. Shortell SM, O’Brien JL, Carman JM, Foster RW, Hughes EF, Boerstler
H, et al. Assessing the impact of continuous quality improvement/total quality
management: concept versus implementation. Health Serv Res. 1995;30:377401. [PMID: 7782222]
44. Gillies GL, Reynolds JH, Shortell SM, Hughes EF, Budetti P, Huang CF,
et al. Implementing continuous quality improvement. In: Kimberly JR, Minvielle
E, eds. The Quality Imperative Measurement and Management of Quality in
Healthcare. London: Imperial Coll Pr; 2000.
45. Neily J, Howard K, Quigley P, Mills PD. One-year follow-up after a collaborative breakthrough series on reducing falls and fall-related injuries. Jt Comm
J Qual Patient Saf. 2005;31:275-85. [PMID: 15960018]
46. Semin-Goossens A, van der Helm JM, Bossuyt PM. A failed model-based
attempt to implement an evidence-based nursing guideline for fall prevention.
J Nurs Care Qual. 2003;18:217-25. [PMID: 12856906]
47. Dempsey J. Falls prevention revisited: a call for a new approach. J Clin Nurs.
2004;13:479-85. [PMID: 15086634]
48. Moyer VA; U.S. Preventive Services Task Force. Prevention of falls in
community-dwelling older adults: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2012;157:197-204. [PMID: 22868837]
49. Tinetti ME, Brach JS. Translating the fall prevention recommendations into
a covered service: can it be done, and who should do it? [Editorial]. Ann Intern
Med. 2012;157:213-4. [PMID: 22868841]
www.annals.org
Annals of Internal Medicine
Current Author Addresses: Ms. Miake-Lye and Drs. Ganz and
Shekelle:
Veterans Affairs Greater Los Angeles Healthcare System, 11301 Wilshire
Boulevard, Los Angeles, CA 90073.
Dr. Hempel: RAND Corporation, 1776 Main Street, Santa Monica, CA
90401.
Author Contributions: Conception and design: P.G. Shekelle.
Analysis and interpretation of the data: I.M. Miake-Lye, S. Hempel,
D.A. Ganz, P.G. Shekelle.
Drafting of the article: I.M. Miake-Lye, P.G. Shekelle.
Critical revision of the article for important intellectual content: I.M.
Miake-Lye, S. Hempel, D.A. Ganz, P.G. Shekelle.
Final approval of the article: I.M. Miake-Lye, S. Hempel, D.A. Ganz,
P.G. Shekelle.
Provision of study materials or patients: P.G. Shekelle.
Obtaining of funding: P.G. Shekelle.
Administrative, technical, or logistic support: I.M. Miake-Lye, P.G.
Shekelle.
Collection and assembly of data: I.M. Miake-Lye, S. Hempel, P.G.
Shekelle.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
50. Browne JA, Covington BG, Davila Y. Using information technology to assist
in redesign of a fall prevention program. J Nurs Care Qual. 2004;19:218-25.
[PMID: 15326991]
51. Capan K, Lynch B. A hospital fall assessment and intervention project. J Clin
Outcomes Manag. 2007;14:155-60.
52. Gutierrez F, Smith K. Reducing falls in a Definitive Observation Unit: an
evidence-based practice institute consortium project. Crit Care Nurs Q. 2008;31:
127-39. [PMID: 18360143]
53. Kolin MM, Minnier T, Hale KM, Martin SC, Thompson LE. Fall initiatives: redesigning best practice. J Nurs Adm. 2010;40:384-91. [PMID:
20798621]
54. McCollam ME. Evaluation and implementation of a research-based falls
assessment innovation. Nurs Clin North Am. 1995;30:507-14. [PMID:
7567575]
55. O’Connell B, Myers H. A failed fall prevention study in an acute care setting:
lessons from the swamp. Int J Nurs Pract. 2001;7:126-30. [PMID: 11811315]
56. Rauch K, Balascio J, Gilbert P. Excellence in action: developing and implementing a fall prevention program. J Healthc Qual. 2009;31:36-42. [PMID:
19343900]
57. Weinberg J, Proske D, Szerszen A, Lefkovic K, Cline C, El-Sayegh S, et al.
An inpatient fall prevention initiative in a tertiary care hospital. Jt Comm J Qual
Patient Saf. 2011;37:317-25. [PMID: 21819030]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) W-179
Supplement
Annals of Internal Medicine
Medication Reconciliation During Transitions of Care as a Patient
Safety Strategy
A Systematic Review
Janice L. Kwan, MD*; Lisha Lo, MPH*; Margaret Sampson, MLIS, PhD; and Kaveh G. Shojania, MD
Medication reconciliation identifies and resolves unintentional discrepancies between patients’ medication lists across transitions in
care. The purpose of this review is to summarize evidence about
the effectiveness of hospital-based medication reconciliation interventions. Searches encompassed MEDLINE through November
2012 and EMBASE and the Cochrane Central Register of Controlled
Trials through July 2012. Eligible studies evaluated the effects of
hospital-based medication reconciliation on unintentional discrepancies with nontrivial risks for harm to patients or 30-day postdischarge emergency department visits and readmission. Two reviewers evaluated study eligibility, abstracted data, and assessed study
quality.
Eighteen studies evaluating 20 interventions met the selection
criteria. Pharmacists performed medication reconciliation in 17 of
the 20 interventions. Most unintentional discrepancies identified
had no clinical significance. Medication reconciliation alone probably does not reduce postdischarge hospital utilization but may do
so when bundled with interventions aimed at improving care
transitions.
THE PROBLEM
reconciliation on unintentional discrepancies with the potential for harm (“clinically significant discrepancies”) and
hospital utilization after discharge, as assessed by unplanned emergency department visits and readmission to
the hospital within 30 days.
Transitions in care, such as admission to and discharge
from the hospital, put patients at risk for errors due to poor
communication and inadvertent information loss (1–5).
Unintentional changes to patients’ medication regimens
represent 1 well-studied category of such errors (6 –9).
Medication regimens at hospital discharge often differ
from preadmission medications. Some differences reflect
deliberate changes related to the conditions that led to
hospitalization (for example, withholding antihypertensive
medications from patients with septic shock). However,
other discrepancies are unintentional and result from incomplete or inaccurate information about current medications and doses.
Up to 67% of patients admitted to the hospital have
unintended medication discrepancies (9), and these discrepancies remain common at discharge (7, 10). As in
other areas of patient safety, errors are more common than
actual harms. Reported proportions of unintended discrepancies with the potential for harm range from 11% to 59%
of all discrepancies (9). Of note, approximately 40% to
80% of patients have no clinically significant unintended
medication discrepancies (8, 10 –16). Thus, although unintended medication discrepancies are common, clinically
significant discrepancies may affect only a few patients.
Nonetheless, medication reconciliation, the formal
process for identifying and correcting unintended medication discrepancies across transitions of care, has been
widely endorsed (17, 18) and is mandated by health care
accreditation bodies in both the United States (19) and
Canada (20). One previous systematic review (21) looked
broadly at the effect of medication reconciliation on various processes and outcomes related to medication safety.
We sought to focus specifically on the effect of medication
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Ann Intern Med. 2013;158:397-403.
For author affiliations, see end of text.
* Dr. Kwan and Ms. Lo contributed equally to this manuscript.
www.annals.org
PATIENT SAFETY STRATEGY
The best possible medication history (BPMH) provides the cornerstone for medication reconciliation. More
comprehensive than a routine primary medication history,
the BPMH involves 2 steps: a systematic process for obtaining a thorough history of all prescribed and nonprescribed medications by using a structured patient interview, and verification of this information with at least 1
other reliable source of information (for example, a government medication database, medication vials, patient medication lists, a community pharmacy, or a primary care
physician) (17, 22) (Figure).
At a minimum, medication reconciliation refers to the
completion of a BPMH and the act of correcting any unintended discrepancies between a patient’s previous medication
regimen and the proposed medication orders at admission
(from home or a health care facility, such as a nursing home),
inpatient transfer (to or from other services or units, such as
the intensive care unit), or discharge (to home or a health care
facility). More advanced medication reconciliation involves
See also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 397
Supplement
Medication Reconciliation During Transitions of Care
Key Summary Points
Medication reconciliation is widely recommended to avoid
unintentional discrepancies between patients’ medications
across transitions in care.
Clinically significant unintentional discrepancies affect only
a few patients.
Medication reconciliation alone probably does not reduce
postdischarge hospital utilization within 30 days but may
do so when bundled with other interventions that improve
discharge coordination.
Pharmacists play a major role in most successful
interventions.
Commonly used criteria for selecting high-risk patients
do not consistently improve the effect of medication
reconciliation.
interprofessional collaboration (for example, a physician and
nurse or pharmacist conducting medication reconciliation as a
team), integration into discharge summaries and prescriptions, and provision of medication counseling to patients (22).
Medication reconciliation has also been bundled with other
interventions to improve the quality of transitions in care,
such as patient counseling about discharge care plans, coordination of follow-up appointments, and postdischarge telephone calls (23–26).
Recommendations for medication reconciliation in
ambulatory settings have begun to appear (27, 28). However, most studies still focus on medication reconciliation
across hospital-based transitions in care, which is the focus
of our review.
REVIEW PROCESSES
The Supplement, available at www.annals.org, includes a complete description of the search strategies, summary of evidence search and selection, and evidence tables.
We searched MEDLINE to 5 November 2012,
EMBASE between 1980 and July 2012, and the Cochrane
Central Register of Controlled Trials to July 2012 for
English-language articles (Figure 1 of the Supplement).
We also scanned reference lists of all included studies and
review articles and directly communicated with study authors as required to obtain details not included in published reports. We included randomized, controlled trials
(RCTs); before-and-after evaluations; and postintervention
studies.
Eligible studies reported emergency department visits
and hospitalizations within 30 days of discharge or evaluated the severity or clinical significance of unintentional
discrepancies. For studies reporting unintended discrepancies, we required that at least 1 clinician independent from
398 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
the medication reconciliation process assess severity or clinical significance. Thus, we excluded studies in which the
person conducting medication reconciliation provided the
sole assessment of clinical significance for identified discrepancies. We also required that studies explicitly distinguish unintentional discrepancies from other (intentional)
medication changes through direct communication with
the medical team.
Although studies varied in their definitions of categories of severity for the potential harm associated with medication discrepancies, most reported a category that
amounted to “trivial,” “minor,” or “unlikely to cause
harm.” We applied the term “clinically significant” to all
unintended discrepancies not labeled as such. This definition of clinically significant unintentional discrepancies
corresponds to the concept of potential adverse drug events
(ADEs), although only a few studies explicitly used this
term (25, 29 –31).
Two of 3 reviewers independently screened each citation for inclusion. Information was abstracted about clinical setting, study design, number of participants, components of the intervention, transitions of care targeted, and
outcomes. Disagreements between the 2 reviewers were resolved by discussion and involved a third reviewer when
necessary to achieve consensus. The full data extraction
form (available on request) included questions directed at
general methodological features (for example, sample size
and study design), details about the components of the
medication reconciliation intervention (for example, components of the BPMH and the method for confirming that
medication discrepancies were unintended), and the process for assessing the clinical significance of identified
discrepancies.
Two reviewers independently applied the Cochrane
Collaboration’s tool for assessing risk of bias (32) to each
of the 5 included RCTs, assessing patient selection bias,
selective reporting, patient attrition, and other biases by
using this standardized tool. Meta-analysis was performed
with Comprehensive Meta-Analysis (Biostat, Englewood,
New Jersey). For results from studies of disparate designs,
we calculated the median effect and interquartile range by
using Microsoft Excel (Microsoft, Seattle, Washington).
This approach was first used in a large review of guideline
implementation strategies (33) and has since been applied
in other systematic reviews of quality improvement interventions (34 –37).
This review was supported by the Agency for Healthcare Research and Quality, which had no role in the selection or review of the evidence or the decision to submit the
manuscript for publication.
BENEFITS
AND
HARMS
Overview of Studies
Of 1845 screened citations, 18 studies (reporting 20
medication reconciliation interventions) met the inclusion
www.annals.org
Medication Reconciliation During Transitions of Care
criteria (Figure 2 of the Supplement). All 18 were from
hospitals in the United States or Canada. Studies about
medication reconciliation from other countries met prespecified exclusion criteria, such as not distinguishing intended from unintended medication discrepancies (38 –
40) or basing the assessment of clinical severity solely on
judgments by the personnel conducting medication reconciliation (41, 42).
Five studies (reporting 7 medication reconciliation interventions) used randomized, controlled designs (23–25,
30, 31). All 5 were assessed as having low risk of bias. One
study used a quasi-experimental design (intervention delivered in alternating months) (26), 3 had a before-and-after
design, and 9 reported postintervention data only (Appendix Table, available at www.annals.org). Seven interventions focused on “high-risk patients” based on advanced
age, presence of chronic illnesses, or use of multiple medications (Appendix Table).
Seven studies compared medication reconciliation
with “usual care” (23, 26, 30, 31, 43– 45), whereas 2 studies (24, 25) compared 2 forms of medication reconciliation. All but 2 of the studies (15, 44) were done in academic medical centers, although 1 study involved both
teaching and nonteaching settings (43). Five of the interventions targeted admission to a hospital (8, 11, 14, 16,
46), 7 targeted discharge home (10, 23, 26, 29, 31, 43,
45), 1 targeted in-hospital transfer (13), and 7 targeted
multiple care transitions (15, 24, 25, 30, 44).
Our 2 outcomes of interest— clinically significant unintentional discrepancies and 30-day postdischarge hospital
utilization— corresponded to the primary outcome in 9 of
18 included studies (15, 23–26, 29, 30, 43, 45). The primary outcome for most of the remaining studies involved
variations of our outcomes of interest, such as all unintentional discrepancies rather than the subset of clinically significant unintentional discrepancies (8, 14, 16, 46). Only 1
study (44) reported a primary outcome substantially different from our outcomes of interest. This study evaluated the
feasibility of implementing an electronic system for targeted pharmacist- and nurse-conducted admission, but it
included sufficient information to abstract data for our
outcomes of interest.
Benefits
Clinically Significant Unintended Medication Discrepancies
The number of clinically significant unintentional discrepancies per patient varied greatly across the 12 included
medication reconciliation interventions (Table 1 of the
Supplement). The median proportion of all unintended
discrepancies judged as having clinical significance was
34% (interquartile range, 28% to 49%). The median proportion of patients with at least 1 clinically significant discrepancy was 45% (interquartile range, 31% to 56%).
Two of the interventions that reported clinically significant unintended discrepancies focused on “high-risk
patients” based on number of medications (8) and medical
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Figure. Overview of medication reconciliation in acute care.
Home
Patient/
Family
Interview
Medication
Vials/List
Government
Medication
Database
Previous
Patient
Health
Records
Sources of Medication Information
Admission
Reconciliation
Health care
Facility
Best Possible
Medication History
(BPMH)
Medications ordered during
admission and internal transfer
Decision to discharge
Best Possible
Medication
Discharge Plan
(BPMDP)
Discharge
Reconciliation
Home
Reconciled
Discharge
Prescriptions
Physician
Discharge
Summary
Patient
Medication
Schedule
BPMDP communicated to patient
and next provider of care
Adapted, with permission, from Fernandes OA. Medication reconciliation. Pharmacy Practice. 2009;25:26.
complexity (14). One intervention identified 0.36 clinically significant discrepancies per patient (8), whereas the
other reported a much higher value of 0.91 per patient (14).
Only 2 RCTs (30, 31) evaluated the effect of medication reconciliation on clinically significant unintended discrepancies. One trial (31) randomly assigned 178 patients
being discharged from the medical service at a teaching
hospital in Boston, Massachusetts, to an intervention that
included medication reconciliation and counseling by a
pharmacist, as well as a follow-up telephone call within 5
days. For patients in the control group, nurses provided
discharge counseling and pharmacists reviewed medication
orders without performing a formal reconciliation process.
Fewer patients in the intervention group experienced preventable ADEs (1% vs. 11%; P ⫽ 0.01). Total ADEs did
not differ between the 2 groups.
A subsequent cluster randomized trial from the same
research group involved 14 medical teams at 2 teaching
hospitals in Boston (30). The intervention included a
Web-based application using the hospital’s electronic medical record (which included ambulatory visits) to create a
preadmission medication list to facilitate the medication
reconciliation process. This study reported a relative reduc5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 399
Supplement
Medication Reconciliation During Transitions of Care
tion in potential ADEs (equal to clinically significant unintended medication discrepancies) of 0.72 (95% CI, 0.52
to 0.99). Of note, the intervention’s effect achieved statistical significance at only 1 of the 2 participating hospitals,
with an adjusted relative risk for potential ADEs of 0.72
(CI, 0.52 to 0.99), but not at the other (0.87 [CI, 0.57 to
1.32]).
Emergency Department Visits and Readmission Within 30 Days
Nine interventions reported emergency department
visits and readmission within 30 days per patient (Table 2
of the Supplement). Of these interventions, 5 applied selection criteria for high-risk patients (24, 26, 47, 48). Again,
however, focusing on high-risk patients did not consistently increase the effect of medication reconciliation.
Across 3 RCTs, readmissions and emergency department visits were reduced by 23% (CI, 5% to 37%; I2 ⫽
24%) (Figure 3 of the Supplement). This pooled result was
driven by the statistically significant reduction achieved by an
intensive intervention (23) that included additional components beyond medication reconciliation that were specifically aimed at reducing readmissions.
One other RCT (47) met inclusion criteria but was
excluded from meta-analysis because it reported hospital
utilization at 12 months rather than 30 days after discharge. This study showed that reconciliation led to a significant 16% reduction in all visits to the hospital. The
intervention consisted of a fairly intensive medication reconciliation strategy in which pharmacists identified drugrelated problems beyond unintended discrepancies, counseled patients at admission and discharge, and telephoned
patients 2 months after discharge to ensure adequate home
management of medications.
Harms
Mistakes in the medication reconciliation process may
become “hard-wired” into the patient record. Once medication reconciliation has occurred, clinicians assessing a
given patient may rely exclusively on the documented
medication history and be less likely to confirm its accuracy
with the patient or other sources.
The larger concern with medication reconciliation pertains to the reliance on pharmacists. Pharmacists have
proven roles in the prevention of ADEs (48 –50); however,
they are in short supply in most hospitals. Thus, involving
pharmacists in medication reconciliation, as most published studies have done, risks taking these personnel away
from other important activities related to patient safety.
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
Effect of Context on Effectiveness
Conceptually, 3 categories of contextual factors probably affect the impact of medication reconciliation: the
degree to which patients can directly provide up-to-date
medication histories, which reflects patients’ knowledge of
400 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
their medications, health literacy, and language; availability
of medication data sources (for example, electronic medical
records in an ambulatory setting and regional prescription
databases) to facilitate the medication reconciliation process; and possibly the clinical informatics milieu, including the degree to which medication reconciliation can
be integrated into such applications as computerized
physician order entry and electronic medical records. We
had hoped to explore the impacts of these factors on effectiveness, but the number of included studies and the studies’ descriptions of context were insufficient to permit such
analyses.
Costs
Medication reconciliation has become mandatory for
hospital accreditation in the United States (19) and Canada (20). Thus, it has been implemented in hospitals of
varying types and sizes and across a broad range of clinical
services. However, most published studies evaluating the
effect of medication reconciliation come from academic
settings (Appendix Table). Moreover, in routine practice,
medication reconciliation is probably done by physicians
and nurses, especially outside of academic centers. By contrast, pharmacists played a major role in conducting
medication reconciliation in 17 of the 20 interventions
included in this review (Appendix Table). Nurses or physicians delivered only 3 interventions (23, 25, 45) without
substantial support from pharmacists, and one of these interventions used a nurse discharge advocate assigned to
deliver the intervention (23).
A clinical informatics milieu (computerized physician
order entry or electronic medical record) was noted for 13
interventions, but electronic medication reconciliation occurred in only 9 interventions. The medication reconciliation process generated new medication orders in only 3
interventions (25, 44), 2 of which came from 1 study (25)
(Table 3 of the Supplement).
One model-based study (51) considered the costeffectiveness of 5 pharmacist-led strategies for reducing
ADEs. Pharmacist-led medication reconciliation carried a
reasonable probability of cost-effectiveness (compared with
no reconciliation) at £10 000 ($16 240 as of 31 December
2012) per quality-adjusted life-year. The authors estimated
the cost for implementing pharmacist-led medication
reconciliation at £1897 ($3200) per 1000 prescription orders (51). A systematic review of economic analyses of patient safety strategies (52) judged this study as having acceptable quality features for economic analyses of patient
safety strategies. The main limitation identified was the
uncertainty surrounding assumptions about expected reductions in ADEs as a result of reductions in potential
ADEs.
DISCUSSION
Medication reconciliation addresses the conceptually
plausible and well-documented problem of unintended
www.annals.org
Medication Reconciliation During Transitions of Care
medication discrepancies introduced across transitions in
care. This review suggests that only a few unintended discrepancies have clinical significance. Furthermore, most
patients have no unintentional discrepancies. Therefore,
the actual effect of medication reconciliation on reducing
clinically significant discrepancies in the inpatient setting
remains unclear.
Medication reconciliation has attracted interest because of its potential effect on reducing postdischarge utilization. The pooled results of 3 RCTs showed that interventions significantly reduced emergency department visits
and readmissions within 30 days of discharge. However,
this finding was driven by the results of a single trial—a
robust intervention that included several additional facets
aimed at improving the discharge process and coordinating
postdischarge care (23). The degree to which medication
reconciliation contributed to the result is unclear.
The lack of effect of medication reconciliation alone
on hospital utilization within 30 days of discharge may
reflect the need to consider a longer window of observation
to demonstrate benefit. The inadvertent discontinuation of
cholesterol-lowering medications, antiplatelet or anticoagulant agents, thyroid hormone replacement, antiresorptive
therapy for osteoporosis, and gastric acid suppression
agents—all commonly encountered examples of unintended discrepancies— carry risks for adverse clinical effects that may require hospital utilization in the long term
but not usually within 30 days of discharge. It is thus
noteworthy that a trial of medication reconciliation alone
(that is, with no additional discharge coordination interventions) that used a longer postdischarge follow-up (12
months) reported a significant reduction in emergency department visits and readmissions (47).
Given limited resources, the paramount issue becomes
how to target medication reconciliation to direct resources
most efficiently. This is especially important given that
most studies involve pharmacists to conduct medication
reconciliation, which requires substantial investment of resources beyond usual care. Our review suggests that common selection criteria for high-risk patients showed no
consistent correlation with the prevalence of clinically significant unintentional discrepancies.
The absence of apparent effect from focusing on highrisk patients could reflect the limited number of studies.
However, the high-risk criteria that are used also have
plausible limitations. For example, even though elderly patients and patients with multiple chronic conditions may
receive many medications, their medication regimens may
remain stable for some time or may be well-known to the
patients or their caregivers. These risk factors for unintended medication discrepancies do not account for such
nuances. A more direct risk factor is probably frequent or
recent changes to medication regimens. This risk factor
unfortunately cannot be ascertained reliably without conducting a thorough medication history, not unlike that
required by the BPMH for medication reconciliation.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Our findings have some similarities with a previous
review of hospital-based medication reconciliation (21) in
that we found that most successful interventions relied
heavily on pharmacists and that, on the whole, medication
reconciliation remains a potentially promising intervention. The previous review found inconsistent reductions in
postdischarge health care utilization and indicated greater
success from targeting high-risk patients. These differences
may reflect the methodological differences between our
studies. We explicitly selected for studies that assessed the
clinical significance of unintentional discrepancies, required a clear distinction between intentional and unintentional medication changes through communication with
the medical team, and required that assessments of clinical
significance be performed by at least 1 clinician independent from the reconciliation process.
Our review has several limitations. Although we conducted a comprehensive literature search, we had no way of
identifying unpublished research. One of our outcomes of
interest, clinically significant unintentional discrepancies,
was not always the primary outcome in included studies.
In addition, this outcome is subjective and open to individual interpretation. Lastly, in most of the included studies, the interventions were described with relatively little
detail and frequently omitted potentially important contextual features (for example, patients’ understanding of
their medications and the interprofessional culture at the
institution).
Hospital-based medication reconciliation at care transitions frequently identifies unintended discrepancies, but
many have no clinical significance. Pharmacists play important roles in most published interventions. Most studies
have assessed patient outcomes during or shortly after hospitalization, but the benefits of resolving unintended discrepancies may not become apparent for months after discharge. Perhaps for this reason, medication reconciliation
alone does not seem to reduce emergency department visits
or readmission within 30 days.
Bundling medication reconciliation with other interventions aimed at improving care coordination at hospital
discharge holds more promise, but the specific effect of
medication reconciliation in such multifaceted interventions may not become apparent until much later than 30
days after discharge. Future research should examine the
effect of medication reconciliation on postdischarge hospital utilization at time points extending past the traditional
30-day mark and identify patient features that more consistently increase the risk for clinically significant unintended discrepancies.
From the University of Toronto, Toronto, and Children’s Hospital of
Eastern Ontario, Ottawa, Ontario, Canada.
Note: The Agency for Healthcare Research and Quality reviewed contract deliverables to ensure adherence to contract requirements and quality, and a copyright release was obtained from the Agency for Healthcare
Research and Quality before submission of the manuscript.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 401
Supplement
Medication Reconciliation During Transitions of Care
Disclaimer: All statements expressed in this work are those of the authors
and should not be construed as official opinions or positions of the
organizations where any of the authors are employed, the Agency for
Healthcare Research and Quality, or the U.S. Department of Health and
Human Services.
Financial Support: From the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services (contract HHSA290-2007-10062I).
Potential Conflicts of Interest: Dr. Shojania: Other: Agency for Healthcare Research and Quality as a subcontract from the University of California, Los Angeles-RAND Evidence-Based Practice Centre. All other
authors have no disclosures. Disclosures can also be viewed at
www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum
⫽M12-2634.
Requests for Single Reprints: Kaveh G. Shojania, MD, Department of
Medicine, Sunnybrook Health Sciences Centre, Room H468, 2075
Bayview Avenue, Toronto, Ontario M4N 3M5, Canada; e-mail,
[email protected].
Current author addresses and author contributions are available at
www.annals.org.
References
1. Coleman EA, Berenson RA. Lost in transition: challenges and opportunities
for improving the quality of transitional care. Ann Intern Med. 2004;141:533-6.
[PMID: 15466770]
2. Forster AJ, Clark HD, Menard A, Dupuis N, Chernish R, Chandok N, et al.
Adverse events among medical patients after discharge from hospital. CMAJ.
2004;170:345-9. [PMID: 14757670]
3. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence
and severity of adverse events affecting patients after discharge from the hospital.
Ann Intern Med. 2003;138:161-7. [PMID: 12558354]
4. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW.
Deficits in communication and information transfer between hospital-based and
primary care physicians: implications for patient safety and continuity of care.
JAMA. 2007;297:831-41. [PMID: 17327525]
5. van Walraven C, Taljaard M, Etchells E, Bell CM, Stiell IG, Zarnke K, et al.
The independent association of provider and information continuity on outcomes after hospital discharge: implications for hospitalists. J Hosp Med. 2010;
5:398-405. [PMID: 20845438]
6. Bell CM, Brener SS, Gunraj N, Huo C, Bierman AS, Scales DC, et al.
Association of ICU or hospital admission with unintentional discontinuation of
medications for chronic diseases. JAMA. 2011;306:840-7. [PMID: 21862745]
7. Coleman EA, Smith JD, Raha D, Min SJ. Posthospital medication discrepancies: prevalence and contributing factors. Arch Intern Med. 2005;165:1842-7.
[PMID: 16157827]
8. Cornish PL, Knowles SR, Marchesano R, Tam V, Shadowitz S, Juurlink
DN, et al. Unintended medication discrepancies at the time of hospital admission. Arch Intern Med. 2005;165:424-9. [PMID: 15738372]
9. Tam VC, Knowles SR, Cornish PL, Fine N, Marchesano R, Etchells EE.
Frequency, type and clinical importance of medication history errors at admission
to hospital: a systematic review. CMAJ. 2005;173:510-5. [PMID: 16129874]
10. Wong JD, Bajcar JM, Wong GG, Alibhai SM, Huh JH, Cesta A, et al.
Medication reconciliation at hospital discharge: evaluating discrepancies. Ann
Pharmacother. 2008;42:1373-9. [PMID: 18780806]
11. Coffey M, Mack L, Streitenberger K, Bishara T, De Faveri L, Matlow A.
Prevalence and clinical significance of medication discrepancies at pediatric hospital admission. Acad Pediatr. 2009;9:360-365. [PMID: 19640822]
12. Kwan Y, Fernandes OA, Nagge JJ, Wong GG, Huh JH, Hurn DA, et al.
Pharmacist medication assessments in a surgical preadmission clinic. Arch Intern
Med. 2007;167:1034-40. [PMID: 17533206]
13. Lee JY, Leblanc K, Fernandes OA, Huh JH, Wong GG, Hamandi B, et al.
Medication reconciliation during internal hospital transfer and impact of com402 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
puterized prescriber order entry. Ann Pharmacother. 2010;44:1887-95. [PMID:
21098753]
14. Stone BL, Boehme S, Mundorff MB, Maloney CG, Srivastava R. Hospital
admission medication reconciliation in medically complex children: an observational study. Arch Dis Child. 2010;95:250-5. [PMID: 19948664]
15. Vira T, Colquhoun M, Etchells E. Reconcilable differences: correcting medication errors at hospital admission and discharge. Qual Saf Health Care. 2006;
15:122-6. [PMID: 16585113]
16. Gleason KM, McDaniel MR, Feinglass J, Baker DW, Lindquist L, Liss D,
et al. Results of the Medications at Transitions and Clinical Handoffs (MATCH)
study: an analysis of medication reconciliation errors and risk factors at hospital
admission. J Gen Intern Med. 2010;25:441-7. [PMID: 20180158]
17. World Health Organization. Action on Patient Safety-High 5s. 2006. Accessed at www.who.int/patientsafety/implementation/solutions/high5s/en/index
.html on 13 September 2012.
18. Institute for Healthcare Improvement. Overview of the 100,000 Lives Campaign. 2006. Accessed at www.ihi.org/offerings/Initiatives/PastStrategicInitiatives
/5MillionLivesCampaign/Documents/Overview%20of%20the%20100K%20
Campaign.pdf on 13 September 2012.
19. The Joint Commission. 2011 National Patient Safety Goals. 2011. Accessed
at www.jointcommission.org/standards_information/npsgs.aspx on 13 September 2012.
20. Accreditation Canada. Required Organizational Practices 2012. Ottawa, Ontario, Canada: Accreditation Canada; 2012. Accessed at www.accreditation.ca
/uploadedFiles/ROP%20Handbook.pdf Accessed on 15 September 2012.
21. Mueller SK, Sponsler KC, Kripalani S, Schnipper JL. Hospital-based medication reconciliation practices: a systematic review. Arch Intern Med. 2012;172:
1057-69. [PMID: 22733210]
22. Fernandes O. Medication reconciliation in the hospital: what, why, where,
when, who and how? Healthc Q. 2012;15 Spec No:42-9. [PMID: 22874446]
23. Jack BW, Chetty VK, Anthony D, Greenwald JL, Sanchez GM, Johnson
AE, et al. A reengineered hospital discharge program to decrease rehospitalization:
a randomized trial. Ann Intern Med. 2009;150:178-87. [PMID: 19189907]
24. Koehler BE, Richter KM, Youngblood L, Cohen BA, Prengler ID, Cheng
D, et al. Reduction of 30-day postdischarge hospital readmission or emergency
department (ED) visit rates in high-risk elderly medical patients through delivery
of a targeted care bundle. J Hosp Med. 2009;4:211-8. [PMID: 19388074]
25. Kripalani S, Roumie CL, Dalal AK, Cawthon C, Businger A, Eden SK,
et al; PILL-CVD (Pharmacist Intervention for Low Literacy in Cardiovascular
Disease) Study Group. Effect of a pharmacist intervention on clinically important medication errors after hospital discharge: a randomized trial. Ann Intern
Med. 2012;157:1-10. [PMID: 22751755]
26. Walker PC, Bernstein SJ, Jones JN, Piersma J, Kim HW, Regal RE, et al.
Impact of a pharmacist-facilitated hospital discharge program: a quasiexperimental study. Arch Intern Med. 2009;169:2003-10. [PMID: 19933963]
27. Bayoumi I, Howard M, Holbrook AM, Schabort I. Interventions to improve medication reconciliation in primary care. Ann Pharmacother. 2009;43:
1667-75. [PMID: 19737997]
28. Schnipper JL, Liang CL, Hamann C, Karson AS, Palchuk MB, McCarthy
PC, et al. Development of a tool within the electronic medical record to facilitate
medication reconciliation after hospital discharge. J Am Med Inform Assoc.
2011;18:309-13. [PMID: 21486889]
29. Pippins JR, Gandhi TK, Hamann C, Ndumele CD, Labonville SA,
Diedrichsen EK, et al. Classifying and predicting errors of inpatient medication
reconciliation. J Gen Intern Med. 2008;23:1414-22. [PMID: 18563493]
30. Schnipper JL, Hamann C, Ndumele CD, Liang CL, Carty MG, Karson
AS, et al. Effect of an electronic medication reconciliation application and process
redesign on potential adverse drug events: a cluster-randomized trial. Arch Intern
Med. 2009;169:771-80. [PMID: 19398689]
31. Schnipper JL, Kirwin JL, Cotugno MC, Wahlstrom SA, Brown BA, Tarvin
E, et al. Role of pharmacist counseling in preventing adverse drug events after
hospitalization. Arch Intern Med. 2006;166:565-71. [PMID: 16534045]
32. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD,
et al; Cochrane Bias Methods Group. The Cochrane Collaboration’s tool for
assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. [PMID:
22008217]
33. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L,
et al. Effectiveness and efficiency of guideline dissemination and implementation
strategies. Health Technol Assess. 2004;8:iii-iv, 1-72. [PMID: 14960256]
www.annals.org
Medication Reconciliation During Transitions of Care
34. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD.
Audit and feedback: effects on professional practice and health care outcomes.
Cochrane Database Syst Rev. 2006:CD000259. [PMID: 16625533]
35. Ranji SR, Steinman MA, Shojania KG, Gonzales R. Interventions to reduce
unnecessary antibiotic prescribing: a systematic review and quantitative analysis.
Med Care. 2008;46:847-62. [PMID: 18665065]
36. Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J.
Effect of point-of-care computer reminders on physician behaviour: a systematic
review. CMAJ. 2010;182:E216-25. [PMID: 20212028]
37. Steinman MA, Ranji SR, Shojania KG, Gonzales R. Improving antibiotic
selection: a systematic review and quantitative analysis of quality improvement
strategies. Med Care. 2006;44:617-28. [PMID: 16799356]
38. Midlöv P, Bahrani L, Seyfali M, Höglund P, Rickhag E, Eriksson T. The
effect of medication reconciliation in elderly patients at hospital discharge.
Int J Clin Pharm. 2012;34:113-9. [PMID: 22207271]
39. Abdel-Qader DH, Harper L, Cantrill JA, Tully MP. Pharmacists’ interventions in prescribing errors at hospital discharge: an observational study in the
context of an electronic prescribing system in a UK teaching hospital. Drug Saf.
2010;33:1027-44. [PMID: 20925440]
40. Steurbaut S, Leemans L, Leysen T, De Baere E, Cornu P, Mets T, et al.
Medication history reconciliation by clinical pharmacists in elderly inpatients
admitted from home or a nursing home. Ann Pharmacother. 2010;44:1596-603.
[PMID: 20736427]
41. Norris CM, Thomas V, Calvert PS. An audit to evaluate the acceptability of
a pharmacist electronically prescribing discharge medication and providing information to GPs. Pharmaceutical Journal. 2001;267:857-9.
42. Climente-Martı́ M, Garcı́a-Mañón ER, Artero-Mora A, Jiménez-Torres
NV. Potential risk of medication discrepancies and reconciliation errors at admission and discharge from an inpatient medical service. Ann Pharmacother. 2010;
44:1747-54. [PMID: 20923946]
43. Dedhia P, Kravet S, Bulger J, Hinson T, Sridharan A, Kolodner K, et al. A
quality improvement intervention to facilitate the transition of older adults from
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
three hospitals back to their homes. J Am Geriatr Soc. 2009;57:1540-6. [PMID:
19694865]
44. Kramer JS, Hopkins PJ, Rosendale JC, Garrelts JC, Hale LS, Nester TM,
et al. Implementation of an electronic system for medication reconciliation.
Am J Health Syst Pharm. 2007;64:404-22. [PMID: 17299180]
45. Showalter JW, Rafferty CM, Swallow NA, Dasilva KO, Chuang CH. Effect
of standardized electronic discharge instructions on post-discharge hospital utilization. J Gen Intern Med. 2011;26:718-23. [PMID: 21499825]
46. Gleason KM, Groszek JM, Sullivan C, Rooney D, Barnard C, Noskin GA.
Reconciliation of discrepancies in medication histories and admission orders of
newly hospitalized patients. Am J Health Syst Pharm. 2004;61:1689-95. [PMID:
15540481]
47. Gillespie U, Alassaad A, Henrohn D, Garmo H, Hammarlund-Udenaes M,
Toss H, et al. A comprehensive pharmacist intervention to reduce morbidity in
patients 80 years or older: a randomized controlled trial. Arch Intern Med. 2009;
169:894-900. [PMID: 19433702]
48. Kaboli PJ, Hoth AB, McClimon BJ, Schnipper JL. Clinical pharmacists and
inpatient medical care: a systematic review. Arch Intern Med. 2006;166:955-64.
[PMID: 16682568]
49. Kucukarslan SN, Peters M, Mlynarek M, Nafziger DA. Pharmacists on
rounding teams reduce preventable adverse drug events in hospital general medicine units. Arch Intern Med. 2003;163:2014-8. [PMID: 14504113]
50. Leape LL, Cullen DJ, Clapp MD, Burdick E, Demonaco HJ, Erickson JI,
et al. Pharmacist participation on physician rounds and adverse drug events in the
intensive care unit. JAMA. 1999;282:267-70. [PMID: 10422996]
51. Karnon J, Campbell F, Czoski-Murray C. Model-based cost-effectiveness
analysis of interventions aimed at preventing medication error at hospital admission (medicines reconciliation). J Eval Clin Pract. 2009;15:299-306. [PMID:
19335488]
52. Etchells E, Koo M, Daneman N, McDonald A, Baker M, Matlow A, et al.
Comparative economic analyses of patient safety improvement strategies in acute
care: a systematic review. BMJ Qual Saf. 2012;21:448-56. [PMID: 22523319]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 403
Annals of Internal Medicine
Current Author Addresses: Dr. Kwan: Department of Medicine,
Mount Sinai Hospital, Room 427, 600 University Avenue, Toronto,
Ontario M5G 1X5, Canada.
Ms. Lo: University of Toronto Centre for Patient Safety, 525 University
Avenue, Room 630, Toronto, Ontario M5G 2L3, Canada.
Dr. Sampson: Children’s Hospital of Eastern Ontario, 401 Smyth Road,
Ottawa, Ontario K1H 8L1, Canada.
Dr. Shojania: Department of Medicine, Sunnybrook Health Sciences
Centre, Room H468, 2075 Bayview Avenue, Toronto, Ontario M4N
3M5, Canada.
W-180 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Author Contributions: Conception and design: J.L. Kwan, M. Samp-
son, K.G. Shojania.
Analysis and interpretation of the data: J.L. Kwan, L. Lo, M. Sampson,
K.G. Shojania.
Drafting of the article: J.L. Kwan, L. Lo, M. Sampson, K.G. Shojania.
Critical revision of the article for important intellectual content: J.L.
Kwan, L. Lo, K.G. Shojania.
Final approval of the article: J.L. Kwan, M. Sampson, K.G. Shojania.
Statistical expertise: L. Lo, K.G. Shojania.
Obtaining of funding: K.G. Shojania.
Administrative, technical, or logistic support: L. Lo, K.G. Shojania.
Collection and assembly of data: J.L. Kwan, L. Lo, M. Sampson, K.G.
Shojania.
www.annals.org
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) W-181
Medical ward in U.S.
community teaching
hospital
Medical and cardiology
wards in 2 U.S.
academic medical
centers
Medical and cardiology
wards in 2 U.S.
academic medical
centers
Kramer et al,
2007 (44)
Kripalani et al,
2012 (25)
Schnipper
et al,
2006 (31)
Schnipper
et al,
2009 (30)
Pippins et al,
2008 (29)
Lee et al,
2010 (13)
Inpatient wards and critical
care units in 2 academic
medical centers in
Canada
Medical wards in 2 U.S.
academic medical
centers
Medical ward in U.S.
academic medical
center
Medical wards in 2 U.S.
academic medical
centers
Medical ward in U.S.
academic medical
center
Koehler et al,
2009 (24)
Kripalani et al,
2012 (25)
Medical ward in U.S.
academic medical
center
Koehler et al,
2009 (24)
Jack et al,
2009 (23)
Gleason et al,
2010 (16)
Gleason et al,
2004 (46)
Dedhia et al,
2009 (43)
Surgical and medical wards
in U.S. academic
medical center
Medical ward in U.S.
academic medical
center
Medical ward in U.S.
academic medical
center
Pediatric ward in academic
medical center in
Canada
Medical ward in academic
medical center in
Canada
Medical wards in U.S.
academic medical
center, community
teaching hospital, and
urban community
hospital
Coffey et al,
2009 (11)
Cornish et al,
2005 (8)
Setting
Study, Year
(Reference)
Discharge home
Age ⱖ65 y
None
None
RCT (162)§
None
None
None
ⱖ1 of the following: ⱖ7 medications,
substantial comorbid condition,
previous admission for ADR, ⱖ4
drug allergies
None
Admission to
hospital, discharge
home
Age ⱖ70 y, ⱖ5 medications, ⱖ3
chronic comorbid conditions,
requirement for assistance with
ⱖ1 ADL
Age ⱖ70 y, ⱖ5 medications, ⱖ3
chronic comorbid conditions,
requirement for assistance with
ⱖ1 ADL
Admission to
hospital, discharge
home
Discharge home
Discharge home
In-hospital transfer
At time of enrollment
in study, discharge
home, and
in-hospital transfer
At time of enrollment
in study, discharge
home, and
in-hospital transfer
Admission to
hospital, discharge
home
Admission to
hospital, discharge
home
Discharge home
Admission to hospital
None
None
Admission to hospital
Admission to hospital
ⱖ4 medications
None
Admission to hospital
Transition of Care
Targeted
None
Selection for High-Risk Patients
RCT (92)㛳
Prospective postintervention
study (180)§
Prospective postintervention
study (129)‡
RCT (423)
RCT (428)
Prospective before-and-after
study (136)
RCT (20)†
RCT (21)†
RCT (373)
Prospective postintervention
study (651)
Postintervention study (204)*
Prospective before-and-after
study (185)
Prospective postintervention
study (151)
Prospective postintervention
study (272)
Study Design
(Sample Size, n)
Physician with confirmation by
pharmacist or nurse
(enhanced by preadmission
medication list builder in
electronic medical record)
Pharmacist
Pharmacist
Pharmacist
Pharmacist
Physician and nurse
Pharmacist and physician
Pharmacist
Nurse and reviewed by
pharmacist
Nurse discharge advocate
Pharmacist
Pharmacist
Physician followed by
pharmacist
Pharmacist, pharmacy student,
or medical student
Pharmacy student
Person Performing Medication
Reconciliation
None
None
None
Pharmacist intervention,
including inpatient
pharmacist
counseling,
low-literacy
adherence aids, and
postdischarge
telephone call
None
Discharge counseling
Supplemental elderly
care bundle
(counseling by
pharmacist,
postdischarge
telephone call, and
discharge letter to
PCP)
None
Nurse discharge
advocates created a
posthospitalization
care plan and
postdischarge
telephone call
Counseling by registered
nurse
None
Safe STEPS intervention,
including admission
assessment,
communication
with PCP, and
multidisciplinary
discharge meeting
None
None
None
Additional Interventions
Beyond Medication
Reconciliation
Continued on following page
Emergency department visits and
hospitalizations within 30 d of
discharge
Clinically significant unintentional
discrepancies
Emergency department visits and
hospitalizations within 30 d of
discharge
Clinically significant unintentional
discrepancies
Clinically significant unintentional
discrepancies
Clinically significant unintentional
discrepancies
Clinically significant unintentional
discrepancies
Emergency department visits and
hospitalizations within 30 d of
discharge
Emergency department visits and
hospitalizations within 30 d of
discharge
Emergency department visits and
hospitalizations within 30 d of
discharge
Emergency department visits and
hospitalizations within 30 d of
discharge
Clinically significant unintentional
discrepancies
Clinically significant unintentional
discrepancies
Emergency department visits and
hospitalizations within 30 d of
discharge
Clinically significant unintentional
discrepancies
Clinically significant unintentional
discrepancies
Outcome
Appendix Table. Studies of Medication Reconciliation, Including Assessment of Clinically Significant Unintended Discrepancies and Emergency Department Visits and
Hospitalizations Within 30 Days of Discharge
W-182 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Pediatric ward in U.S.
academic medical
center
Acute care units in urban
community hospital in
Canada
Medical ward in U.S.
academic center
Stone et al,
2010 (14)
Prospective postintervention
study (150)
Prospective quasi-experimental
study (358)**
Retrospective postintervention
study (60)
Prospective postintervention
study (23)¶
Retrospective before-and-after
study (17 516)
Study Design
(Sample Size, n)
None
ⱖ1 of the following: ⱖ5 medications,
ⱖ1 targeted medications††,
medication requiring monitoring,
ⱖ2 changes to regimen, dementia
or confusion, or inability to
manage medications
Identification of medically complex
conditions based on published
guidelines
None
None
Selection for High-Risk Patients
Discharge home
Admission to
hospital, discharge
home
Discharge home
Admission to hospital
Discharge home
Transition of Care
Targeted
Pharmacist or pharmacy
resident
Pharmacist
Pharmacist
Pharmacist
Physician
Person Performing Medication
Reconciliation
Pharmacist-facilitated
discharge program,
including counseling,
provision of
medication
reconciliation list
to PCP, and
postdischarge
telephone call
None
None
Standardized mandatory
electronic discharge
instructions
document with
embedded
computerized
medication
reconciliation
None
Additional Interventions
Beyond Medication
Reconciliation
Clinically significant unintentional
discrepancies
Emergency department visits and
hospitalizations within 30 d of
discharge
Clinically significant unintentional
discrepancies
Clinically significant unintentional
discrepancies
Emergency department visits and
hospitalizations within 30 d of
discharge
Outcome
ADL ⫽ activity of daily living; ADR ⫽ adverse drug reaction; PCP ⫽ primary care physician; RCT ⫽ randomized, controlled trial; Safe STEPS ⫽ Safe and Successful Transition of Elderly Patients Study.
* 12 adult medical–surgical units.
† 2 hospital medicine groups.
‡ 10 patient care units.
§ 7 medical teams.
㛳 4 medical teams.
¶ On 2 medical teams.
** 2 medical teams and 1 hospitalist service.
†† Targeted medications included digoxin, diuretics, anticoagulants, sedatives, opioids, asthma or chronic obstructive pulmonary disease medications, and angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers.
Wong et al,
2008 (10)
Walker et al,
2009 (26)
Medical ward in academic
medical center in
Canada
All admitted patients
through emergency
department in U.S.
academic medical
center
Showalter
et al,
2011 (45)
Vira et al,
2006 (15)
Setting
Study, Year
(Reference)
Appendix Table. —Continued
Supplement
Annals of Internal Medicine
Nurse–Patient Ratios as a Patient Safety Strategy
A Systematic Review
Paul G. Shekelle, MD, PhD
A small percentage of patients die during hospitalization or shortly
thereafter, and it is widely believed that more or better nursing care
could prevent some of these deaths. The author systematically
reviewed the evidence about nurse staffing ratios and in-hospital
death through September 2012. From 550 titles, 87 articles were
reviewed and 15 new studies that augmented the 2 existing reviews were selected. The strongest evidence supporting a causal
relationship between higher nurse staffing levels and decreased
inpatient mortality comes from a longitudinal study in a single
hospital that carefully accounted for nurse staffing and patient
comorbid conditions and a meta-analysis that found a “dose–
response relationship” in observational studies of nurse staffing and
death. No studies reported any serious harms associated with an
increase in nurse staffing. Limiting any stronger conclusions is the
lack of an evaluation of an intervention to increase nurse staffing
ratios. The formal costs of increasing the nurse–patient ratio cannot
be calculated because there has been no evaluation of an intentional change in nurse staffing to improve patient outcomes.
THE PROBLEM
ing care and reductions in hospital mortality, potentially in
addition to or instead of a simple nurse–patient ratio.
These factors include measures of nursing burnout, job
satisfaction, teamwork, nurse turnover, nursing leadership
in hospitals, and nurse practice environment.
Several research groups have proposed conceptual
frameworks to explain why more effective nursing care
may reduce inpatient mortality (5– 8). Underlying all of
these conceptual frameworks is the belief that surveillance
is a critical factor that can be improved with more staff,
better-educated staff, or a better working environment (9).
A representative framework by Aiken and colleagues (8)
posits that nurse–patient ratios, along with staffing skill
mix, can lead to better surveillance, which, along with
many other factors, can influence the process of care and
lead to better patient outcomes (Figure 1).
A small percentage of hospitalized patients die during
or shortly after hospitalization. Evidence suggests that
some proportion of these deaths could probably be prevented with more nursing care. For example, in 1 early
study of 232 342 surgical discharges from several Pennsylvania hospitals, 4535 patients (2%) died within 30 days of
hospitalization; the investigators estimated that the difference between 4:1 and 8:1 patient–nurse ratios may be approximately 1000 deaths in a group of this size (1). Other
studies have produced roughly similar estimates, namely
approximately 1 to 5 fewer deaths per 1000 inpatient days
with more nurse staffing per patient (2– 4). The rationale
for suggesting that increasing the ratio of registered nurses
(RNs) to patients will lead to decreased illness or mortality
rates rests on the belief that improved attention to patients
is the critical factor. This systematic review examined the
evidence on the effects of interventions aimed at increasing
nurse–patient ratios on patient illness and death.
PATIENT SAFETY STRATEGIES
There has been no evaluation of an intentional change
in RN staffing to improve patient outcomes; therefore, the
patient safety strategy referred to in this article remains
somewhat unclear. Most studies have been cross-sectional
or longitudinal assessments of differences in nursing staff
variables, with the most commonly assessed measure being
the proportion of RN time per some measure of inpatient
load and the most commonly assessed outcome being mortality. However, many other factors have been proposed as
being causal with respect to the relationship between nursSee also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
404 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Ann Intern Med. 2013;158:404-409.
For author affiliation, see end of text.
www.annals.org
REVIEW PROCESSES
Two existing reviews relevant to the topic were identified, by using methods described by Whitlock and colleagues (10). These reviews were supplemented by searching the Web of Science for articles published from 2009
(the end date of the search from the most recent review) to
September 2012 that cited any of 4 key articles in this
field, including the older of the 2 reviews, and was limited
to studies published in English. For a complete description
of the search strategies, literature flow diagram, and evidence tables, see the Supplement (available at www.annals
.org). The update search identified 546 titles, and 4 articles
came from reference mining. Titles and abstracts were reviewed and selected if they reported empirical data on the
relationship between nurse staffing ratios and mortality or
nursing-sensitive outcomes, such as pressure ulcers and failure to rescue. Because several cross-sectional studies have
assessed this relationship, only 1 additional cross-sectional
study was included for detailed review. The exception was
a cross-sectional study that evaluated a quasi-intervention
(11). Nine longitudinal studies were identified (12–20).
www.annals.org
Nurse–Patient Ratios as a Patient Safety Strategy
Key Summary Points
Cross-sectional studies, mostly in intensive care unit or
postsurgical settings, support a relationship between
the number of nurses staffed per patient and inpatient
mortality.
The strongest evidence supporting a causal relationship
between higher nurse staffing levels and decreased inpatient mortality comes from a longitudinal study in a single
hospital that carefully accounted for nurse staffing levels
and found decreases in mortality of 2% to 7%.
Limiting any stronger conclusions is the lack of an evaluation of an intervention to increase nurse staffing ratios.
Four simulation studies reported on costs, and 1 systematic
review article was included (21–25). Two frameworks
were also included (6, 7). No experimental studies were
identified.
The assessment of multiple systematic reviews
(AMSTAR) criteria was used to assess the quality of the
systematic reviews (26). Only criteria relevant to a particular review were applied; for example, 2 of the 11
AMSTAR criteria are only applicable to reviews that involve meta-analysis. In addition, the AMSTAR criteria requiring a list of all excluded studies were not applied. New
studies were not formally assessed for study quality, but
their strengths and limitations are discussed later.
This review was supported by the Agency for Healthcare Research and Quality, which had no role in the selection or review of the evidence or the decision to submit
this manuscript for publication.
BENEFITS
AND
HARMS
Benefits
Two recent relevant systematic reviews on this topic, a
meta-analysis (27) and a narrative review (28), respectively
Supplement
scored 10 out of 10 relevant criteria and 7 out of 9 relevant
criteria according to AMSTAR.
The meta-analysis included 28 studies, of which 17
were cohort studies, 7 were cross-sectional studies, and 4
were case– control studies (no experimental studies were
identified). Most were U.S. studies, and the average level of
staffing was 3.0 patients per RN for the intensive care unit
(ICU) setting, 4.0 patients per RN in the surgical setting,
and 4.4 patients per RN for the medical setting. It found a
consistent relationship between higher RN staffing and
lower hospital-related mortality: An increase of 1 RN fulltime equivalent (FTE) per patient day was related to a 9%
reduction in the odds of death in the ICU, a 16% reduction in the surgical setting, and a 6% reduction in the
medical setting. With respect to other outcomes, lower
rates of hospital-acquired pneumonia, pulmonary failure,
unplanned extubation, failure to rescue, and nosocomial
bloodstream infections were related to higher RN staffing
in pooled analyses of several studies. However, several
other outcomes that were presumed to have strong sensitivity to nurse staffing levels did not show consistent relationships, including falls, pressure ulcers, and urinary tract
infections.
The authors also conducted an indirect analysis of the
potential for a dose–response relationship. This analysis
assessed the effect across studies of additional RNs per
shift. In each case, comparisons of quartiles of nurse staffing levels showed the expected relationship (Figure 2). In
other words, if the relationship between nurse staffing and
mortality is causal, the difference in the risk for death
should be greater between the first and third quartiles of
nurse staffing than it is between the first and second quartiles because the difference in staffing between the first and
third quartiles is greater than that between the first and
second quartiles.
The authors of the meta-analysis concluded that a
consistent relationship has been shown but identified several limitations in the literature with respect to establishing
Figure 1. Hospital organization, nursing organization, and patient outcomes.
Hospital organization
Nurse–patient ratios/
staffing skill mix
Organizational support for
nursing care
Resource adequacy
Nurse autonomy
Nurse control
Nurse–physician relations
Nurse
outcomes
Patient
outcomes
Surveillance/early
detection of complications
Process of care
Medical staff qualifications
From reference 8, with permission.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 405
Supplement
Nurse–Patient Ratios as a Patient Safety Strategy
Figure 2. Pooled odds ratio of quartiles of nurse staffing levels.
Odds Ratio
of Death*
(95% CI)
Quartiles of Patients/RN per Shift
All patients
1 vs. 2
0.94 (0.92–0.95)
1 vs. 3
0.76 (0.71–0.81)
1 vs. 4
0.62 (0.59–0.66)
2 vs. 3
0.81 (0.76–0.87)
2 vs. 4
0.66 (0.63–0.70)
3 vs. 4
0.82 (0.76–0.88)
Intensive care units
0.94 (0.92–0.97)
2 vs. 3
Medical patients
0.94 (0.92–0.95)
1 vs. 2
Surgical patients
1 vs. 3
0.76 (0.70–0.82)
1 vs. 4
0.62 (0.58–0.66)
2 vs. 3
0.80 (0.74–0.87)
2 vs. 4
0.65 (0.61–0.70)
3 vs. 4
0.81 (0.75–0.88)
0.5
1
Odds Ratio of Death*
Odds ratios are based on pooled analysis consistent across the studies
(heterogeneity not significant). From reference 27, with permission.
RN ⫽ registered nurse.
that this relationship is causal. The authors ultimately concluded that the arguments for a causal relationship are
“mixed,” and they called for future research to address the
role of nurse staffing and competence on the effectiveness
of patient care, “taking greater cognizance of other relevant
factors such as patient and hospital characteristics and
quality of medical care” (27).
The narrative review identified literature published
through 2009 and was restricted to studies that used
hospital-related mortality as the outcome; the authors
identified 17 studies (10 of which were not included in the
first review and 7 that were published since 2007) (28).
Although this review was narrative, the 2 reviews had
broadly similar results: 14 of 17 studies found a statistically
significant relationship between nurse staffing variables and
lower mortality rates. In addition, the narrative review
identified mixed findings for mortality among 5 studies
assessing the characteristics of the nurse work environment
and work relationships, 3 studies assessing nurses’ responses to work and the work environment (for example,
burnout), and 7 studies assessing nurses’ educational preparation and experience. Only 1 study, which had a crosssectional design, assessed nursing process-of-care variables;
it found a relationship between the use of care maps and
lower hospital-associated mortality, with an estimated effect size of 10 fewer deaths per 1000 acute medicine dis406 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
charged patients. Like the meta-analysis, the narrative review concluded that a strong relationship exists but more
research is needed to understand the reasons why this relationship between higher nurse staffing and lower hospital
mortality may be causal (that is, they called for a theoretical model that explains the relationship in ways that can
be tested and refined).
Thus, these 2 reviews came to broadly similar conclusions. Mostly cross-sectional studies consistently report
that higher RN staffing is related to lower hospital-related
mortality.
However, many factors can confound the observed relationship. In cross-sectional studies, hospitals that are
“better” in other ways may also be better staffed with more
RNs. For example, 1 published study of electronic health
record implementation showed that hospitals with electronic health records have higher nurse staffing ratios and
lower patient mortality (29). If the cross-sectional relationship is confounded, then critics worry that adoption of
fixed nurse–patient ratios will not necessarily lead to better
health outcomes, that such a policy is “an inflexible solution that is unlikely to lead to optimal use of resources” (30).
The results of the updated search are as follows. Nine
longitudinal studies and one new systematic review (12–
20, 25) were identified. The systematic review included
studies that assessed nurse staffing ratios and outcomes restricted to adult ICU settings (25) and reached conclusions
similar to the previous reviews: a consistent relationship
between increased nurse staffing and better patient outcomes in observational studies, evidence that falls short of
causality. One longitudinal study narratively reported that
increased nurse staffing was related to “significantly (P ⱕ
0.01) decreased rates of decubiti, pneumonia, and sepsis,”
but data were not presented (20). The cross-sectional study
addresses the effect of an “intervention” to change nurse
staffing ratios, implemented in response to a 2004 California law requiring minimum nurse–patient ratios in acute
care hospitals (11). This legislation mandated patient–
nurse staffing levels of 5:1, 4:1, and 2:1 for medical or
surgical units, pediatric units, and ICUs, respectively. The
California legislative mandate does not require nurse staffing to be met with RNs (that is, licensed vocational [practical] nurses can also meet the mandate).
Aiken and colleagues (11) assessed the relationship between nurse staffing and mortality in 2006, 2 years after
the California mandate, comparing data from California
with those of 2 states without mandates, New Jersey and
Pennsylvania. Data about workloads were drawn from a
survey of RNs in the 3 states (22 336 nurses in total); the
response rate was 35.4%. Hospital data came from the
American Hospital Association, and patient and outcome
data came from state hospital discharge databases.
The authors reported that their survey data showed
substantial adherence to the California mandate, with 88%
of medical or surgical nurses, 85% of pediatric nurses, and
85% of ICU nurses reporting that the staffing of their last
www.annals.org
Nurse–Patient Ratios as a Patient Safety Strategy
shift was within the mandated ratio. In logistic regression
analyses adjusted for many patient characteristics and 3
hospital characteristics (such as bed size, teaching status,
and technology use), Aiken and colleagues found statistically significant relationships between the estimation of the
average number of patients per nurse and 2 outcomes: 30day mortality and failure to rescue (11).
Although the study collected data after implementation of the California staffing mandate, it did not test the
effect of that mandate per se because it had no comparison
data from the period before the mandate went into effect.
The possibility that the relationship is causal is blunted by
longitudinal studies that examined measures from before
and after the California mandate, which showed the expected changes in nurse staffing and proportion of licensed
staff per patient but no improvement in other patient outcomes believed to be nursing-sensitive (such as falls, pressure ulcers, and failure to rescue) (16, 17, 19). In fact, an
unexpected statistically significant increase in pressure ulcers was related to a greater number of hours of care for the
patient (which may have been because of greater detection). These studies did not assess mortality.
Five additional longitudinal studies add further information to this picture. The first is a longitudinal assessment of nurse staffing and hospital mortality and failure to
rescue in 283 California hospitals between 1996 and 2001,
which had access to direct measures of nurse staffing (14).
In multivariable models that included many hospital market characteristics as well as risk adjustment using the
Medstat Disease Staging methodology to produce a predicted probability for complications or death, the authors
found that an increase of 1 RN FTE per 1000 inpatient
days was related to a statistically significant decrease in
mortality of 4.3%.
The second longitudinal study assessed care at 39
Michigan hospitals between 2003 and 2006; it included
adults admitted through the emergency department with
acute myocardial infarction, heart failure, stroke, pneumonia, hip fracture, or gastrointestinal bleeding (15). This
study simultaneously controlled for 4 factors—high hospital occupancy on hospitalization, weekend hospitalization,
seasonal influenza, and nurse staffing levels—each of which
had a statistically significant effect on in-hospital mortality.
Each additional RN FTE per patient day was related to a
0.25% decrease in mortality.
The third longitudinal study assessed the effect of a
mandate in 3 Western Australia public hospitals to implement a new staffing method, the Nursing Hours per Patient Day (12). The study assessed 3 periods: 20 months
before implementation, 7 months of a “transition period,”
and 2 months after implementation. The authors found
that the total nursing hours and RN hours increased during the observation period. However, the percentage of
total nursing hours provided by RNs decreased (from 87%
to 84%). Also, the article stated that “although the nursing
hours increased for all three hospitals (in the postwww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
implementation period), the changes were not statistically
significant” (12). Mortality rates were reduced during
this period. Among many other outcomes, some improved,
others did not, and some changes were inconsistent
across hospitals. Although the study was described as an
interrupted time series, it was analyzed as a before–after
study.
The fourth longitudinal study assessed changes in
nurse staffing over 9 years in 124 Florida hospitals and
related these to changes in Agency for Healthcare Research
and Quality Patient Safety Indicators (18). The study used
both initial staffing ratios and changes in staffing ratios.
Results were mixed but generally favored better patient
safety outcomes with higher RN staffing levels.
The methodologically strongest longitudinal study is
that of Needleman and colleagues (13). The researchers
used data over time from a single hospital to assess the
relationship between natural differences in levels of RN
staffing in the same hospital and inpatient mortality. The
study is further characterized by a careful matching of
nurse staffing on a shift-by-shift basis with the actual patients cared for during that shift. Knowing the actual patients cared for allowed for more sophisticated adjustments
of risk for death at the patient level. The study was done at
a tertiary academic hospital between 2003 and 2006 and
included 197 691 hospitalizations and 176 696 nursing
shifts across 43 hospital units. The patients themselves averaged 60 years of age, and approximately 50% were covered under Medicare. The variable of interest was exposure
of the patient to nursing care that was below the target
level (for that type of unit) for that shift (that is, the proportion of shifts below target level staffing on a per-patient
basis). An additional exposure variable was a “highturnover” shift (that is, a shift with many hospitalizations,
discharges, or transfers). The authors found that exposure
to each shift of below-target staffing or high turnover was
related to a 2% to 7% increase in mortality, with higher
levels of risk if the high-turnover or below-target shift occurred in the first 5 days after hospitalization. For patients
who were not in an ICU, this risk was increased by 12%
and 15% during below-target and high-turnover shifts,
respectively.
The data from Needleman and colleagues contribute
to the “causality” determination because the study is longitudinal in 1 hospital, thus controlling for the “hospital
effect” potentially present in all cross-sectional studies, and
has detailed measures of exposure and confounding variables. These results and the dose–response analysis from
the meta-analysis provide the strongest evidence in support
of causality.
Harms
The survey administered as part of the cross-sectional
study previously described, which collected data 2 years
after the California mandate for minimum nurse staffing
ratios (11), found that some California nurses perceived
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 407
Supplement
Nurse–Patient Ratios as a Patient Safety Strategy
that they had less support from the use of licensed vocational nurses, unlicensed personnel, and nonnursing support services (such as housekeeping and unit clerks) after
implementation of the mandate. For example, 25% of RNs
reported that they perceived that they had decreased use of
licensed vocational nurses after the mandate, whereas 10%
perceived that they had increased use and 56% reported
that use remained the same.
The longitudinal assessments from California (16 –19)
and Western Australia (12) reported an increase in pressure
ulcers related to increased nurse staffing, although this development may reflect increased detection. Few other studies mentioned an explicit assessment of potential unexpected adverse outcomes.
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
Implementation Contexts
Because no published studies of an assessment of an
“implementation” were found, the contexts in which interventions have been implemented cannot be directly assessed. However, the cross-sectional and longitudinal studies that have been published and have consistently shown a
relationship between staffing levels and patient outcomes
have included a broad array of hospitals, often all or nearly
all of the hospitals (except for very small ones) in a state.
Therefore, if the relationship between increased RN staffing and inpatient mortality is a causal one, it very likely
applies to most hospitals and contexts. This strategy is
most likely to be implemented when mandated by state or
federal policy.
As previously noted, the relationship between staffing
and mortality that underpins this strategy has been seen in
various hospitals and contexts. The effect, if causal, is probably relatively insensitive to the usual effects of contexts
considered in other patient safety strategy reviews. Of note,
the recent study by Needleman and colleagues was conducted in a tertiary medical center that has a lower-thanexpected in-hospital mortality rate and a reputation for
excellence. Therefore, the relationship between increased
RN staffing and lower mortality, if causal, is potentially
applicable even to high-performing hospitals.
Costs
Four simulation studies reported information about
costs. The first used 2003 data from 28 Belgian cardiac
surgery centers to assess the costs and outcomes of increasing nurse staffing. Assuming a causal relationship between
this staffing increase and an outcome of 5 fewer patient
deaths per 1000 elective hospitalizations, the authors concluded that the incremental cost-effectiveness ratio was
€26 372 (approximately $35 000) per avoided death and
€2639 (approximately $3500) per life-year gained (21).
The second simulation study was conducted by the
University of Minnesota Evidence-based Practice Center,
which produced the systematic review on nurse staffing
(22). It used its own meta-analysis as the basis for estimat408 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
ing the potential monetary benefits of increased RN staffing. Assuming that those relationships were causal and taking a societal perspective, the authors concluded that
increasing RN staffing by 1 FTE per patient day was related to positive savings– cost ratios across a broad range of
clinical settings. For example, the net cost of adding 1 RN
FTE per 1000 hospitalized ICU patients was an estimated
$590 000, whereas the net benefit (in terms of life-years
saved and productivity) was an estimated $1.5 million, for
a benefit– cost ratio of 2.51. However, hospitals did not
save money because the net cost of adding an extra RN
FTE was not offset by the expected 24% decrease in length
of stay.
A third simulation study (24) used data from studies
by Aiken and colleagues and Needleman and colleagues to
estimate benefits in mortality and length of stay, respectively, and estimated an incremental cost-effectiveness ratio
between $25 000 and $136 000 per life saved as patient–
RN staffing ratios decreased from 8:1 to 4:1. The model
was most sensitive to the estimate of effect on mortality.
Lastly, 1 additional study from Portugal estimated that
increasing neonatal nurse staffing to “adequate” would increase staff costs more than 30% of the current rate (23).
DISCUSSION
Nurse staffing ratios have a relationship with reductions in hospital-related mortality in most published studies. However, lack of a published evaluation of an intentional change in RN staffing from some initial value (for
example, 6 patients to 1 RN on general medical wards) to
some lower patient–RN staffing value (such as 5:1 or 4:1)
limits conclusions on increasing nurse staffing ratios as a
patient safety strategy. All longitudinal published studies to
date have assessed natural variations in RN staffing. The
concern also remains that mortality is not reduced by increased nurse staffing but by something the nurses do. Determining what this is and how it can best be facilitated
should be the goal of an effective patient safety strategy.
Limitations of this review include those of the original
articles, such as lack of rigorous evaluations of an intentional intervention, low response rates to surveys that collect explanatory variables (such as RN staffing), potentially
poor matching of RN staffing to actual patients cared for
and their risk for death, and lack of replication of the 1
high-quality longitudinal study that has been published;
and the possibility that some relevant evidence was not
found, either because it was not identified during the
search or because some completed evaluations have not
been unpublished.
To further advance this field, studies assessing an intentional change in nurse staffing ratios are needed. It may
be impractical for such a study to be a randomized, controlled trial, but high-quality evidence could come from a
time series analysis or a controlled before-and-after study,
particularly if it included the necessary process variables to
www.annals.org
Nurse–Patient Ratios as a Patient Safety Strategy
serve as a test of a conceptual framework for how increased
staffing can influence outcomes.
From the RAND Corporation, Santa Monica, and Veterans Affairs
Greater Los Angeles Healthcare System, Los Angeles, California.
Note: The Agency for Healthcare Research and Quality reviewed contract deliverables to ensure adherence to contract requirements and quality, and a copyright release was obtained from the Agency for Healthcare
Research and Quality before submission of the manuscript.
Disclaimer: All statements expressed in this work are those of the author
and should not in any way be construed as official opinions or positions
of the RAND Corporation, Veterans Affairs, the Agency for Healthcare
Research and Quality, or the U.S. Department of Health and Human
Services.
Acknowledgment: The author thanks Robert Kane, MD; Eileen Lake,
PhD, RN; Aneesa Motala, BA; Sydne Newberry, PhD; and Roberta
Shanman, MLS.
Financial Support: From the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services (contract HHSA290-2007-10062I).
Potential Conflicts of Interest: Consultancy: ECRI Institute; Employ-
ment: Veterans Affairs; Grants/grants pending: Agency for Healthcare Research and Quality, Veterans Affairs, Centers for Medicare & Medicaid
Services, National Institute of Nursing Research, Office of the National
Coordinator; Royalties: UpToDate. Disclosures can also be viewed at www.
acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum⫽M12
-2574.
Requests for Single Reprints: Paul G. Shekelle, MD, MPH, RAND
Corporation, 1776 Main Street, Santa Monica, CA 90401; e-mail,
[email protected].
Author contributions are available at www.annals.org.
References
1. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse
staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA.
2002;288:1987-93. [PMID: 12387650]
2. Aiken LH, Clarke SP, Sloane DM, Lake ET, Cheney T. Effects of hospital
care environment on patient mortality and nurse outcomes. J Nurs Adm. 2008;
38:223-9. [PMID: 18469615]
3. Tourangeau AE, Giovannetti P, Tu JV, Wood M. Nursing-related determinants of 30-day mortality for hospitalized patients. Can J Nurs Res. 2002;33:7188. [PMID: 11998198]
4. Person SD, Allison JJ, Kiefe CI, Weaver MT, Williams OD, Centor RM,
et al. Nurse staffing and mortality for Medicare patients with acute myocardial
infarction. Med Care. 2004;42:4-12. [PMID: 14713734]
5. Tourangeau AE, Doran DM, McGillis Hall L, O’Brien Pallas L, Pringle D,
Tu JV, et al. Impact of hospital nursing care on 30-day mortality for acute
medical patients. J Adv Nurs. 2007;57:32-44. [PMID: 17184372]
6. Thornlow DK, Anderson R, Oddone E. Cascade iatrogenesis: factors leading
to the development of adverse events in hospitalized older adults. Int J Nurs Stud.
2009;46:1528-35. [PMID: 19643409]
7. Despins LA, Scott-Cawiezell J, Rouder JN. Detection of patient risk by
nurses: a theoretical framework. J Adv Nurs. 2010;66:465-74. [PMID: 20423428]
8. Aiken LH, Clarke SP, Sloane DM. Hospital staffing, organization, and quality
of care: cross-national findings. Nurs Outlook. 2002;50:187-94. [PMID:
12386653]
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
9. Aiken LH, Sochalski J, Lake ET. Studying outcomes of organizational change
in health services. Med Care. 1997;35:NS6-18. [PMID: 9366875]
10. Whitlock EP, Lin JS, Chou R, Shekelle P, Robinson KA. Using existing
systematic reviews in complex systematic reviews. Ann Intern Med. 2008;148:
776-82. [PMID: 18490690]
11. Aiken LH, Sloane DM, Cimiotti JP, Clarke SP, Flynn L, Seago JA, et al.
Implications of the California nurse staffing mandate for other states. Health Serv
Res. 2010;45:904-21. [PMID: 20403061]
12. Twigg D, Duffield C, Bremner A, Rapley P, Finn J. The impact of the
nursing hours per patient day (NHPPD) staffing method on patient outcomes: a
retrospective analysis of patient and staffing data. Int J Nurs Stud. 2011;48:
540-8. [PMID: 20696429]
13. Needleman J, Buerhaus P, Pankratz VS, Leibson CL, Stevens SR, Harris
M. Nurse staffing and inpatient hospital mortality. N Engl J Med. 2011;364:
1037-45. [PMID: 21410372]
14. Harless DW, Mark BA. Nurse staffing and quality of care with direct measurement of inpatient staffing. Med Care. 2010;48:659-63. [PMID: 20548254]
15. Schilling PL, Campbell DA Jr, Englesbe MJ, Davis MM. A comparison of
in-hospital mortality risk conferred by high hospital occupancy, differences in
nurse staffing levels, weekend admission, and seasonal influenza. Med Care.
2010;48:224-32. [PMID: 20168260]
16. Burnes Bolton L, Aydin CE, Donaldson N, Brown DS, Sandhu M, Fridman M, et al. Mandated nurse staffing ratios in California: a comparison of
staffing and nursing-sensitive outcomes pre- and postregulation. Policy Polit Nurs
Pract. 2007;8:238-50. [PMID: 18337430]
17. Donaldson N, Bolton LB, Aydin C, Brown D, Elashoff JD, Sandhu M.
Impact of California’s licensed nurse-patient ratios on unit-level nurse staffing
and patient outcomes. Policy Polit Nurs Pract. 2005;6:198-210. [PMID:
16443975]
18. Unruh LY, Zhang NJ. Nurse staffing and patient safety in hospitals: new
variable and longitudinal approaches. Nurs Res. 2012;61:3-12. [PMID:
22166905]
19. Cook A, Gaynor M, Stephens M Jr, Taylor L. The effect of a hospital nurse
staffing mandate on patient health outcomes: evidence from California’s minimum staffing regulation. J Health Econ. 2012;31:340-8. [PMID: 22425767]
20. Duffield C, Diers D, O’Brien-Pallas L, Aisbett C, Roche M, King M, et al.
Nursing staffing, nursing workload, the work environment and patient outcomes.
Appl Nurs Res. 2011;24:244-55. [PMID: 20974086]
21. Van den Heede K, Simoens S, Diya L, Lesaffre E, Vleugels A, Sermeus W.
Increasing nurse staffing levels in Belgian cardiac surgery centres: a cost-effective
patient safety intervention? J Adv Nurs. 2010;66:1291-6. [PMID: 20546363]
22. Shamliyan TA, Kane RL, Mueller C, Duval S, Wilt TJ. Cost savings associated with increased RN staffing in acute care hospitals: simulation exercise.
Nurs Econ. 2009;27:302-14, 331. [PMID: 19927445]
23. Fugulin FMT, Lima AFC, Castilho V, Bochembuzio L, Costa JA, Castro L,
et al. Cost of nursing staffing adequacy in a neonatal unit. Revista da Escola de
Enfermagem da Usp. 2011;45:1582-8.
24. Rothberg MB, Abraham I, Lindenauer PK, Rose DN. Improving nurse-topatient staffing ratios as a cost-effective safety intervention. Med Care. 2005;43:
785-91. [PMID: 16034292]
25. McGahan M, Kucharski G, Coyer F; Winner ACCCN Best Nursing Review Paper 2011 sponsored by Elsevier. Nurse staffing levels and the incidence
of mortality and morbidity in the adult intensive care unit: a literature review.
Aust Crit Care. 2012;25:64-77. [PMID: 22515951]
26. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al.
Development of AMSTAR: a measurement tool to assess the methodological
quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID:
17302989]
27. Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. The association of
registered nurse staffing levels and patient outcomes: systematic review and metaanalysis. Med Care. 2007;45:1195-204. [PMID: 18007170]
28. Tourangeau AE. Mortality rate as a nurse-sensitive outcome. In: Doran DM.
Nursing Outcomes: The State of the Science. Sudbury, MA: Jones & Bartlett;
2011.
29. Furukawa MF, Raghu TS, Shao BB. Electronic medical records, nurse staffing, and nurse-sensitive patient outcomes: evidence from California hospitals,
1998-2007. Health Serv Res. 2010;45:941-62. [PMID: 20403065]
30. Griffiths P. RN⫹RN⫽better care? What do we know about the association
between the number of nurses and patient outcomes? [Editorial]. Int J Nurs Stud.
2009;46:1289-90. [PMID: 19647533]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 409
Annals of Internal Medicine
Author Contributions: Conception and design: P.G. Shekelle.
Analysis and interpretation of the data: P.G. Shekelle.
Drafting of the article: P.G. Shekelle.
Critical revision of the article for important intellectual content:
P.G. Shekelle.
Final approval of the article: P.G. Shekelle.
Obtaining of funding: P.G. Shekelle.
Administrative, technical, or logistic support: P.G. Shekelle.
Collection and assembly of data: P.G. Shekelle.
31. Stone PW, Pogorzelska M, Kunches L, Hirschhorn LR. Hospital staffing
and health care-associated infections: a systematic review of the literature. Clin
Infect Dis. 2008;47:937-44. [PMID: 18767987]
32. Cummings GG, MacGregor T, Davey M, Lee H, Wong CA, Lo E, et al.
Leadership styles and outcome patterns for the nursing workforce and work environment: a systematic review. Int J Nurs Stud. 2010;47:363-85. [PMID:
19781702]
33. Butler M, Collins R, Drennan J, Halligan P, O’Mathúna DP, Schultz TJ,
et al. Hospital nurse staffing models and patient and staff-related outcomes. Cochrane Database Syst Rev. 2011:CD007019. [PMID: 21735407]
34. Flynn M, McKeown M. Nurse staffing levels revisited: a consideration of key
issues in nurse staffing levels and skill mix research. J Nurs Manag. 2009;17:75966. [PMID: 19694919]
35. Cho SH, Hwang JH, Kim J. Nurse staffing and patient mortality in intensive
care units. Nurs Res. 2008;57:322-30. [PMID: 18794716]
36. Kiekkas P, Sakellaropoulos GC, Brokalaki H, Manolis E, Samios A, Skartsani C, et al. Association between nursing workload and mortality of intensive
care unit patients. J Nurs Scholarsh. 2008;40:385-90. [PMID: 19094155]
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
37. Hamilton KE, Redshaw ME, Tarnow-Mordi W. Nurse staffing in relation
to risk-adjusted mortality in neonatal care. Arch Dis Child Fetal Neonatal Ed.
2007;92:F99-F103. [PMID: 17088341]
38. Mark BA, Harless DW, Berman WF. Nurse staffing and adverse events in
hospitalized children. Policy Polit Nurs Pract. 2007;8:83-92. [PMID: 17652626]
39. Rafferty AM, Clarke SP, Coles J, Ball J, James P, McKee M, et al. Outcomes of variation in hospital nurse staffing in English hospitals: cross-sectional
analysis of survey data and discharge records. Int J Nurs Stud. 2007;44:175-82.
[PMID: 17064706]
40. Stone PW, Mooney-Kane C, Larson EL, Horan T, Glance LG, Zwanziger
J, et al. Nurse working conditions and patient safety outcomes. Med Care. 2007;
45:571-8. [PMID: 17515785]
41. Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P.
The impact of hospital nursing characteristics on 30-day mortality. Nurs Res.
2005;54:74-84. [PMID: 15778649]
42. Halm M, Peterson M, Kandels M, Sabo J, Blalock M, Braden R, et al.
Hospital nurse staffing and patient mortality, emotional exhaustion, and job dissatisfaction. Clin Nurse Spec. 2005;19:241-51. [PMID: 16179855]
43. Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational
levels of hospital nurses and surgical patient mortality. JAMA. 2003;290:161723. [PMID: 14506121]
44. Sasichay-Akkadechanunt T, Scalzi CC, Jawad AF. The relationship between
nurse staffing and patient outcomes. J Nurs Adm. 2003;33:478-85. [PMID:
14501564]
45. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nursestaffing levels and the quality of care in hospitals. N Engl J Med. 2002;346:171522. [PMID: 12037152]
46. Tarnow-Mordi WO, Hau C, Warden A, Shearer AJ. Hospital mortality in
relation to staff workload: a 4-year study in an adult intensive-care unit. Lancet.
2000;356:185-9. [PMID: 10963195]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) W-183
Supplement
Annals of Internal Medicine
Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy
A Systematic Review
Nancy Sullivan, BA, and Karen M. Schoelles, MD, SM
Complications from hospital-acquired pressure ulcers cause 60 000
deaths and significant morbidity annually in the United States. The
objective of this systematic review is to review evidence regarding
multicomponent strategies for preventing pressure ulcers and to
examine the importance of contextual aspects of programs that aim
to reduce facility-acquired pressure ulcers. CINAHL, the Cochrane
Library, EMBASE, MEDLINE, and PreMEDLINE were searched for
articles published from 2000 to 2012. Studies (any design) that
implemented multicomponent initiatives to prevent pressure ulcers
in adults in U.S. acute and long-term care settings and that reported pressure ulcer rates at least 6 months after implementation
were selected. Two reviewers extracted study data and rated quality of evidence. Findings from 26 implementation studies (moderate
strength of evidence) suggested that the integration of several core
components improved processes of care and reduced pressure ulcer
rates. Key components included the simplification and standardization of pressure ulcer–specific interventions and documentation,
involvement of multidisciplinary teams and leadership, use of designated skin champions, ongoing staff education, and sustained
audit and feedback.
THE PROBLEM
PATIENT SAFETY STRATEGIES
Pressure ulcers are largely preventable, but pressure ulcer rates continue to escalate at an alarming rate. Between
1995 and 2008, incidence increased by as much as 80%
(1). An estimated 2.5 million patients will develop a pressure ulcer annually in the United States (2); more than 1
million patients are affected annually in U.S. acute and
long-term care settings (3). Because of the forecasted increase in populations most at risk for pressure ulcers (for
example, obese, diabetic, and elderly patients), rates are
predicted to continue to increase.
Preventing this problem is important not only to protect patients from harm but also to reduce costs of caring
for them. Morbidity caused by pressure ulcers can lead to
requirements for more care and resources and a longer
inpatient stay. In some cases, late-stage pressure ulcers can
even lead to life-threatening infections. In fact, 60 000
U.S. patients die annually of complications related to
hospital-acquired pressure ulcers (2).
The objective of this review is to review the evidence
on implementation of multicomponent strategies for preventing pressure ulcers, focusing on the importance of contextual aspects of programs to reduce the likelihood of
facility-acquired pressure ulcers. We focus on implementation of multicomponent initiatives because a patient safety
strategy designed to address multiple factors is believed to
be more effective than single-component initiatives in preventing this condition.
Strategies aimed at preventing pressure ulcers may
consist of individual or multicomponent interventions or
a series of interventions and may include system-level
changes. A systematic review by Reddy and colleagues (4)
included 59 prevention studies that addressed impaired
mobility, impaired nutrition, or impaired skin health,
mostly in patients in acute care settings. The authors concluded that using support surfaces, regularly repositioning
the patient, optimizing nutritional status, and moisturizing
sacral skin are appropriate strategies for preventing pressure
ulcers. Other reviews and guidelines stress the importance
of initial and repeated assessment of patients’ risk, tailored
care for individuals found to be at increased risk, and regular skin examinations (5–17).
Many organizations endorse the concept of bundling
care practices (for example, standardized risk assessment
and regular repositioning), which typically include 3 to 5
evidence-based practices that “when performed collectively
and reliably, have been proven to improve patient outcomes” (18). Some recommend having an identifiable
theme (such as “Save Our Skin”) (1, 19). Besides bundling
care practices, experts recommend that attention be paid to
organizational and care coordination components (1, 20).
Organizational components include selecting lead team
membership, establishing policies and procedures, evaluating quality processes, educating staff, using skin champions, and communicating written care plans. Care coordination components include creating a culture of change
and establishing regular meetings to facilitate communication, collegiality, and learning.
Ann Intern Med. 2013;158:410-416.
For author affiliations, see end of text.
www.annals.org
See also:
REVIEW PROCESSES
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
This review was done in parallel with another Agency
for Healthcare Research and Quality (AHRQ)–sponsored
systematic review on specific interventions for preventing
pressure ulcers (for example, different kinds of support
surfaces, heel supports, nutritional supplementation, and
410 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy
repositioning). We searched CINAHL, the Cochrane
Library, EMBASE, MEDLINE, and PreMEDLINE for articles published from 2000 to September 2012 and the
gray literature by using keywords related to the concepts of
pressure ulcer prevention efforts, barriers, and settings.
Searches were restricted to English-language literature. We
identified 587 abstracts, from which 95 full-text articles
were reviewed in more detail, yielding 51 articles contributing data to this review. We selected studies of any design
that implemented multicomponent initiatives in acute and
long-term care settings in the United States. Studies were
included if they considered multicomponent pressure ulcer
preventive measures (such as evidence-based clinical decision tools combined with training and education), targeted
adult populations, and reported pressure ulcer rates 6
months after implementation.
Two independent reviewers screened publications for
inclusion; 26 studies (18 acute care, 8 long-term care) met
inclusion criteria. The reviewers extracted information on
context, including influence of external factors (such as
state survey deficiencies); descriptions of teamwork, leadership, and safety culture; and implementation tools (such as
ongoing performance monitoring). They detailed descriptions of the implementation efforts (such as processes, barriers, and sustainability) in the studies and extracted information about our main (pressure ulcer rates) and secondary
(process-of-care measures) outcomes.
We assessed study quality using the 19-item Standards
for Quality Improvement Reporting Excellence (SQUIRE)
guidelines (21). We paid particular attention to a subset of
the items we thought were important for implementation
studies, such as the following: 1) describes the intervention
and its component in sufficient detail that others could
reproduce it, 2) presents data on changes observed in the
care delivery process and changes observed in measures of
patient outcomes, 3) reports on study limitations, and 4)
interprets possible reasons for differences between observed
and expected outcomes. Our assessment did not consider
other requirements in the SQUIRE guidelines such as including an abstract, describing the local problem, or reporting funding. We considered a study to be high quality
if it reported 8 to 10 items, moderate quality if it reported
5 to 7 items, and low quality if it reported fewer than 5
required items.
The Supplement (available at www.annals.org) completely describes the search strategies, provides an article
flow diagram, and provides evidence tables.
This review was supported by AHRQ, which had no
role in the selection or review of the evidence or the decision to submit the manuscript for publication.
BENEFITS
AND
HARMS
Benefits
Twenty-six studies met inclusion criteria. Eighteen
studies were conducted in acute care settings and 8 in longwww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Key Summary Points
Despite being largely preventable, pressure ulcer rates are
escalating in the United States.
Moderate-strength evidence suggests that implementing
multicomponent initiatives for pressure ulcer prevention in
acute and long-term care settings can improve processes
of care and reduce pressure ulcer rates.
Key components of successful implementation efforts include: simplification and standardization of pressure ulcer–
specific interventions and documentation, involvement of
multidisciplinary teams and leadership, designated skin
champions, ongoing staff education, and sustained audit
and feedback.
term care settings. Study designs were mostly time series
assessments of changes before, during, and after implementation of the intervention. Other designs included randomized, controlled trials (22–24) and a controlled before-andafter (24). Several of the studies were identified from a
2011 review of nurse-focused quality improvement interventions in hospitals (25) and a 2012 review of comprehensive programs for preventing pressure ulcers (5). Of the
26 studies, 9 were high-quality, 14 were moderate-quality,
and 3 were low-quality.
Nine core components of programs for pressure ulcer
prevention, in addition to specific patient care practices,
have been associated with a reduction in incidence or prevalence of pressure ulcers. Appendix Tables 1 and 2 (available at www.annals.org) show which components and patient care practices were used in the 18 studies in acute care
settings and the 8 studies in long-term care settings. Studies showed that most organizations educated and trained
staff (96%), developed or revised their protocols for assessment and documentation of wounds (96%), performed
quality audits and provided feedback to staff (81%), adopted the Braden Scale for Predicting Pressure Sore Risk
(61%), and redesigned documentation processes and reporting (58%).
In the 18 studies of pressure ulcer prevention programs in U.S. hospitals, study authors described multiple
patient care interventions or cited clinical practice guidelines or resources that describe specific interventions to
reduce patients’ risk for pressure ulcers. The hospital caregivers performed initial and repeated risk assessments
(such as the Braden Scale), followed by tailored interventions chosen from a menu of options based on a risk category or specific risk factors. These interventions included
support surfaces (for example, specialized mattresses and
heel supports), getting patients out of bed or frequently
repositioning those who were bed-bound, moisture management (including incontinence interventions and skin
care products), mechanical means of reducing friction and
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 411
Supplement
Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy
shear forces on body areas at greatest risk, nutritional assessments or interventions, and hydration. Pressure ulcer
prevention programs that were used in the 8 studies in
long-term care facilities typically referenced guidelines or
other resources developed by their state’s quality improvement organizations.
Twenty-four studies reported at least some improvement in pressure ulcer rates. Two additional studies reported that process-of-care quality measures improved but
that pressure ulcer rates did not (26, 27). Statistically significant reductions in pressure ulcer rates were reported in
11 (42%) of 26 studies (median reduction, 82% [range
67% to 100%]) (24, 28 –37). Of the 13 studies with improvements not reaching statistical significance, 5 reported
improvements in both pressure ulcer rates and process-ofcare measures (19, 38 – 41).
The implementation of a multicomponent strategy by
Walsh and colleagues (2009) reduced pressure ulcer prevalence (12.8% to 0.6%), increased focused communication
among patient caregivers, and improved clinician behavior
and clinical processes once other improvements were recognized (38). Young and colleagues streamlined online
policies (from 7 to 1) and reduced time spent documenting
skin care, which resulted in “clinically relevant reductions”
in development of nosocomial pressure ulcers (19). In 1
year, pressure ulcer rates were reduced by 82.8% (from
2.8% to 0.48%) at 1 rehabilitation hospital. Lynch and
Vickery (39) reported that streamlining documentation increased timely and accurate completion from 60% to 90%
in 90 days. Delmore and colleagues (41) reported a reduction in incidence (from 7.3% to 1.3%) and reduction in
time for collection of prevalence and incidence data (from
8 hours to 2.5 hours).
In the long-term care setting, implementation of the
on-time approach in 10 participating facilities led to reductions in prevalence of pressure ulcers for 7 facilities, reductions in the average number of in-house pressure ulcers (all
stages) for 8 facilities, and reductions in the average number of certified nursing assistant documentation forms for
10 facilities (35). Another study (37) reported a statistically
significant reduction in pressure ulcer incidence (28.3% vs.
9.3%) and improvements in identifying patients as “highrisk” (increase from 22.3% and 28.0%). Milne and colleagues (40) reported reducing prevalence from 41% to
4.2% after increased monitoring of patients with nasal cannulas (pulmonary unit) and increased attentiveness to heel
offloading, support surfaces, and proper positioning (spinal
cord injury and trauma unit). Of the 396 charts reviewed
after implementation, fewer than 1% had missing data. A
review of 45 patient charts showed that wound teams consistently determined staging and wound cause in more than
90% of cases.
Harms
No harms were reported for the patient safety strategies that were used to prevent pressure ulcers.
412 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
Use of a Model or Theory
Of the 26 studies, 6 programs described a model or
theory as the basis of their implementation strategy. Several
quality improvement approaches were described. The
PDSA (Plan, Do, Study, Act) framework used in a 17hospital-initiative (26) involves 4 improvement cycles: 1)
identifying the problem and designing an intervention
(Plan), 2) implementing change (Do), 3) evaluating collected data (Study), and 4) implementing what was learned
(Act). Courtney and colleagues (32) integrated Six Sigma
methods called DMAIC into treatment processes developed for a multisite, not-for-profit facility. Described as a
data-driven quality strategy for improving processes,
DMAIC consists of 5 interconnected steps: (1) Defining
the problem, (2) Measuring the performance, (3) Analyzing the data, (4) Improving the process, and (5) Controlling change (42). Young and colleagues (19) and Chicano
and Drolshagen (43) empowered staff at the point of care,
which “suggests a model of shared governance where decisions are made at the point of service” (44). Two studies
described use of failure mode and effects analysis (40) and
Havelock’s (1974) model of effective research utilization
(24). Of these 6 studies, 2 reported statistically significant
reductions in pressure ulcers (24, 32); 1 reported improvements in processes of care (26).
External Factors Motivating Attention to Pressure
Ulcer Prevention
Most studies in acute care facilities reported feeling
pressure from impending changes in U.S. Centers for
Medicare & Medicaid Services reimbursement to implement pressure ulcer prevention strategies. Specifically, subsequent to passage of the Deficit Reduction Act of 2005,
the Centers for Medicare & Medicaid Services no longer
allows higher diagnosis-related group payments for patients
with stage 3 and 4 hospital-acquired pressure ulcers. Additional positive and negative external motivators are described below.
Positive motivators included a stakeholder’s commitment to improve patient outcomes and a goal “to be recognized as a quality provider of patient services” (19). The
emergence of new guidelines from the American Nurses
Association and AHRQ’s “revitalized interest” in preventing and treating pressure ulcers was cited by Courtney and
colleagues (32). One facility, at which prevalence of
hospital-acquired pressure ulcers was lower than national
norms, set out to eliminate hospital-acquired pressure ulcers completely (33).
Negative motivators for 1 cancer hospital included the
identification of 2 stage 4 pressure ulcers and evidence that
pressure ulcer prevalence exceeded the national benchmark
by nearly 50% (31). Two facilities reported influence from
a G-level citation (a deficiency judged to cause actual harm
to residents) (36) and other citations from the Department
of Health (37). Two critical incidents (not specified) and
www.annals.org
Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy
inconsistent documentation were listed as external motivators by Dibsie (45). Additionally, “the frequency with
which concerns and incidents were discussed, but went
unreported within the internal reporting system” was of
concern (45).
Teamwork/Leadership
A majority of studies used multidisciplinary teams
with skin champions being described as key team members. Studies typically designated 1 individual (for example,
a certified wound ostomy continence nurse) (28, 30, 46) to
coordinate prevention efforts.
Two studies provided detailed descriptions of leadership support. Stier and colleagues (34) described support
provided to multidisciplinary teams at 1 health care system. Teams consisting of clinical experts from 18 facilities
convened to discuss the various risk assessment tools and
facility protocols already in place. Multidisciplinary teams
then agreed to develop a uniform protocol, skin care formulary, and specialty bed contract. “System leadership
(e.g., nurse executives, quality management directors, and
senior physicians) provided support to the team at both the
system and facility level” vis-à -vis “resources, ensured staff
orientation and education, maintained quality control programs, and continually assessed actions to improve performance through system-wide care committee meetings”
(34). Dibsie (45) described broadening teamwork from
nursing management to a larger group of managers and
clinical specialists after “it became evident that serious
skin-related issues crossed many areas and could be better
handled by the group together.”
Implementation Tools
More than 21 initiatives provided examples of unique
tools used for audit and feedback, education and training,
and streamlining products and processes. For a complete
listing of implementation tools, see the Data Supplement
(available at www.annals.org). Audit and feedback (positive
and negative) were mentioned as key elements in 20 (80%)
preventive initiatives. Hiser and colleagues (46) reported
that providing frequent feedback to clinical staff on unit
progress helped engage staff members and “allowed them
to take credit for the improved clinical outcomes.” Certificates for the most improved units were used as reinforcements. While providing feedback to nursing staff in 1
study, the certified nurse specialists balanced compliments
for a job well done with recommendations for improvement (47). In 1 long-term care study, facilitators provided
direct feedback to certified nursing assistants regarding data
inconsistencies by unit and by shift to help track progress
(35). Real-time management feedback in Rosen and colleagues’ study (37) consisted of a prominently displayed
thermometer tracking weekly pressure ulcer incidence and
positive ($10 reward) or negative (termination) reinforcement. Weekly informal feedback by nursing supervisors
(36), formal weekly walk-rounds (39), and frequent patient
positioning audits were also used during implementation
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
(36). One rehabilitation hospital posted report cards unitwide, allowing staff to track progress against other units
and unit goals (39).
Unique tools used during education and training sessions included enrollment of guest speakers to educate physicians about the role of the certified wound ostomy continence nurse and best-practice interventions for wound
care (46). In another study, participants sat on bedpans
during 30-minute mandatory sessions as a reminder that
pressure ulcers can occur in less than 1 hour (19). This
same study tailored educational content for multilevel
staffing and measured effectiveness of presentations by
posttest survey. Finally, Delmore and colleagues described
the involvement of perioperative services in establishing an
educational newsletter for the facility’s Skin and Wound
Care Web site and hosting a Skin Fair Day (41).
Barriers Solved
Reported barriers to implementation included unmotivated staff (28, 31, 43), staff turnover (23, 24, 27, 35),
staff and physician resistance (19, 26, 27), inconsistent
documentation (27, 28, 47), difficulties in exporting data
(35), and miscommunication between electronic systems
(47). Staff disruption of implementation initiatives was the
most commonly reported barrier. One study described staff
as relatively uninvolved in planning (43), whereas another
study described staff members focusing more on the role of
wound care products and specialty beds than on nursing
care when patients developed in-facility pressure ulcers
(31). The launching of monthly to quarterly campaigns
(28); perseverance by leadership (43); and use of additional
education, mentoring, and support at the unit level (31)
were solutions given for motivating staff. Staff reverting to
previously unsuccessful practices (27), staff turnover (24,
27, 35), and variations in new staff orientation also slowed
program momentum. The development of a strong multidisciplinary team (35), assignment of responsibility for
processes to multiple nurses (23), and monthly visits by a
state quality improvement organization (27) helped address
these issues.
To address concerns regarding inconsistent reporting
and documentation, Horn and colleagues (35) worked
with long-term care facilities to simplify and standardize
certified nursing assistant documentation and translate the
information into reports that were used in weekly care
planning meetings. Bales and Padwojski (28) responded by
recognizing and awarding nursing units in which patients
had 0 hospital-acquired pressure ulcers. Initiatives were
also challenged by limited resources. Finally, LeMaster (47)
indicated that 2 different electronic documentation systems were causing shortfalls in pressure ulcer risk reporting. Transition to 1 universal electronic record system resolved this issue (47).
Sustainability
Several acute and long-term care facilities reported on
sustainability or long-term maintenance of prevention ef5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 413
Supplement
Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy
forts. Conducting quarterly prevalence studies (33), requiring registered nurses and licensed practical nurses to demonstrate competency annually (19), and providing monthly
updates via intranet to staff of product changes (19) were
key to sustaining improvements in 2 studies. McInerney
(29) indicated that publicizing improvements in pressure
ulcer rates kept staff focused on prevention efforts. One
rehabilitation hospital printed quarterly newsletters and attached them to paychecks. The newsletters described findings, results, and new initiatives in pressure ulcer management (39). Other studies describe basing staff bonuses on
pressure ulcer incidence (32), establishing a wound care
coordinator position (36) and a wound care committee
(24), and keeping current regarding “initiatives for improved patient safety, changes in regulatory mandates, and
changes in EBP [evidence-based practices]” (38) helped
maintain gains.
Cost-Savings
Four studies reported on cost-savings. Two studies
(36, 37) referenced a secondary analysis by Xakellis and
Frantz (48) that evaluated long-term care and hospital
costs for healing 45 pressure ulcers from 30 patients. Rosen
and colleagues stated that “based on a mean cost of $2,700
to treat a single stage II pressure ulcer, reducing the incidence of ulcers by approximately 15 over 12 weeks would
yield savings of approximately $40,000” (37, 48). In 2009,
a 151-bed Midwest skilled-nursing facility described costsavings 4 years after program implementation. After adjusting (using the Consumer Price Index) the 1996 mean cost
of treating a patient with a pressure ulcer ($1115 per
month), the authors estimated their cost-savings at $1617
per pressure ulcer per month, $10 187 in total monthly
savings, and greater than $122 000 in yearly savings (36).
Estimated cost-savings in the remaining 2 studies
(based on an additional cost per case of approximately
$3000) were also significant (29, 32). In 2006, Courtney
and colleagues reported that a reduction of hospitalacquired pressure ulcers by 50% to 5% would reduce overall costs by $2 438 000 (32). In 2008, a 2-hospital system
(548 beds) in Naples, Florida (29), estimated cost-savings
of approximately $11.5 million annually as a result of statistically significant reductions in pressure ulcer prevalence.
Effects of Context
Authors of studies in long-term care (27) and acute
care (26) settings agreed that the most sustainable interventions were those that were institutionalized. For example,
interventions that were less dependent on sufficient staffing
(for example, changing to pressure-relieving mattresses and
using risk assessment tools) were easier to sustain than interventions that were more dependent on sufficient staffing
(such as ensuring that every resident is turned every 2
hours). Horn and colleagues (35) found that full integration of clinical reports derived from documentation by
front-line staff (certified nursing assistants) was key to success. Studies also specifically mentioned that nurses taking
414 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
ownership (45), as well as promotion and support by leadership (28, 43), were significant factors in achieving goals.
DISCUSSION
Moderate-strength evidence from 26 implementation
studies suggests that the integration of a common set of
components in pressure ulcer prevention programs could
lead to reductions in pressure ulcer rates. Key issues were
the simplification and standardization of pressure-ulcerspecific interventions and documentation, involvement of
multidisciplinary teams and leadership, designated skin
champions, ongoing staff education, and sustained audit
and feedback for promoting both accountability and recognizing successes.
Two recent systematic reviews of quality improvement
programs to prevent pressure ulcers found improvements
in process or ulcer outcomes that were similar to our findings (5, 25). Nurse-focused initiatives led to improvements
“on at least one nursing process or patient health outcome
measure in the intended direction” in 36 of 39 acute care
studies in a 2011 review by Soban and colleagues (25). In
a 2012 review by Niederhauser and colleagues (5), 17 of 20
studies reporting on process-of-care measures and outcomes reported improvements in acute and long-term care
settings. Both reviews included a listing of core components integrated during implementation. Our review adds
to the previous reviews by providing details on implementation of prevention programs, lessons learned (see the
Supplement), solutions to barriers, and potential costsavings.
Neither our review nor those by Soban (25) and Niederhauser (5) and their colleagues discussed the effectiveness of individual components included in preventive bundles because the included studies did not focus on the
effectiveness of specific intervention components. Nevertheless, most studies included certain aspects of direct patient care: initial and repeated risk assessments and skin
examinations; the use of specialized support surfaces (such
as special mattresses and overlays); repositioning or mobility protocols; moisture, friction and shear management;
and nutrition and hydration. Most studies cited clinical
practice guidelines that informed the choice of interventions. Additional limitations of our review included the
exclusion of non-U.S. studies, possible selective reporting,
and no formal evaluation of the possibility of publication
bias. Niederhauser and colleagues (5) speculated that publication bias explains the positive results in most published
studies.
All 3 reviews agree on the need for future research to
delve deeper into daily care processes to better understand
their influence on outcomes. Limitations of the evidence
include the lack of information on processes of care and
their measurement. In fact, in this review, only 9 of 26
studies included information on both processes and outcome measures. Studies also did not describe study limitawww.annals.org
Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy
tions or summarize successes and barriers to implementation, items listed by the SQUIRE guidelines (21) as key to
reporting in a discussion.
In 2000, a review of measures to prevent pressure ulcers in older patients in Making Health Care Safer (49)
included a brief discussion on implementation of pressurerelieving devices specifically noting cost, time, and difficulty in assessing change in pressure ulcer rates after implementation. Since that time, guidance provided by such
organizations as the Institute for Healthcare Improvement
(6), National Pressure Ulcer Advisory Panel (50), and
AHRQ (51) has resulted in successful implementation of
bundled evidence-based practices throughout the United
States. Although we identified 26 implementation studies
(published since 2000), we are concerned about the possibility of publication bias. To continue to understand the
influence of context on implementation of strategies to
prevent pressure ulcers, we encourage clinicians to report
findings regardless of success level and to provide detail on
the patient care processes, staff education and training initiatives, and system-level interventions. In addition, future
research should report strategies to sustain momentum of
preventive programs, a topic rarely discussed in the implementation studies we reviewed. Given the persistent significant morbidity and mortality resulting from pressure ulcers, further study of both system-level and patient care
interventions aimed at preventing pressure ulcers is still
needed for clinicians and managers to choose the most
effective and efficient practices.
From ECRI Institute Evidence-based Practice Center, Plymouth Meeting, Pennsylvania.
Note: The AHRQ reviewed contract deliverables to ensure adherence to
contract requirements and quality, and a copyright release was obtained
from the AHRQ before submission of the manuscript.
Disclaimer: All statements expressed in this work are those of the authors
and should not in any way be construed as official opinions or positions
of ECRI Institute, the AHRQ, or the U.S. Department of Health and
Human Services.
Acknowledgment: The authors thank Allison Gross, MS, LIS, for performing the literature searches; Lydia Dharia and Katherine Donahue for
preparing the manuscript for publication; and Paul G. Shekelle, MD,
PhD, for his review and suggestions on earlier versions of the manuscript.
Financial Support: From the AHRQ, U.S. Department of Health and
Human Services (contract HHSA-290-2007-10062I).
Potential Conflicts of Interest: Ms. Sullivan: None disclosed. Dr.
Schoelles: Support for travel to meetings for the study or other purposes
(money to institution): RAND Corporation; Other (money to institution):
work done by several ECRI staff on Making Health Care Safer II: An
Updated Critical Analysis of the Evidence for Patient Safety Practices for
the AHRQ supported by RAND. Disclosures can also be viewed at
www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum
⫽M12-2655.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Requests for Single Reprints: Nancy Sullivan, BA, ECRI Institute
Evidence-based Practice Center, 5200 Butler Pike, Plymouth Meeting,
PA 19462-1298; e-mail, [email protected].
Current author addresses and author contributions are available at
www.annals.org.
References
1. Lyder CH, Ayello EA. Annual checkup: the CMS pressure ulcer present-onadmission indicator. Adv Skin Wound Care 2009;22:476-84. [PMID:
20026923]
2. Lyder CH. The benefits of a multi-disciplinary approach to the prevention and
treatment of pressure ulcers. Infection Control Today. 10 August 2011. Accessed
at www.infectioncontroltoday.com/news/2011/08/the-benefits-of-a-multi-disciplinary-approach-to-the-prevention-and-treatment-of-pressure-ulcers.aspx on 15
November 2012.
3. Sharkey S, Hudak S, Horn SD, Spector W. Leveraging certified nursing
assistant documentation and knowledge to improve clinical decision making: the
on-time quality improvement program to prevent pressure ulcers. Adv Skin
Wound Care. 2011;24:182-8; quiz 188-90. [PMID: 21407045]
4. Reddy M, Gill SS, Rochon PA. Preventing pressure ulcers: a systematic review. JAMA. 2006;296:974-84. [PMID: 16926357]
5. Niederhauser A, VanDeusen Lukas C, Parker V, Ayello EA, Zulkowski K,
Berlowitz D. Comprehensive programs for preventing pressure ulcers: a review of
the literature. Adv Skin Wound Care. 2012;25:167-88. [PMID: 22441049]
6. Institute for Healthcare Improvement. How-to Guide: Prevent Pressure Ulcers. Cambridge, MA: Institute for Healthcare Improvement; 2011.
7. Advancing Excellence in America’s Nursing Homes. Implementation Guide:
Goal 4: Reducing High Risk Pressure Ulcers. NH Quality Campaign; 2008.
8. Institute for Clinical Systems Improvement. Pressure ulcer prevention and
treatment protocol. Health care protocol. Bloomington, MN: Institute for Clinical Systems Improvement; 2012.
9. Registered Nurses’ Association of Ontario. Risk assessment & prevention of
pressure ulcers 2011 supplement. Toronto, Ontario, Canada: Registered Nurses’
Association of Ontario; 2011.
10. Wound, Ostomy, and Continence Nurses Society. Guideline for prevention
and management of pressure ulcers. Mount Laurel, NJ: Wound, Ostomy, and
Continence Nurses Society; 2010.
11. Association for the Advancement of Wound Care. Association for the Advancement of Wound Care guideline of pressure ulcer guidelines. Malvern, PA:
Association for the Advancement of Wound Care; 2010.
12. National Pressure Ulcer Advisory Panel, European Pressure Ulcer Advisory
Panel. Pressure ulcer prevention recommendations. In: Prevention and treatment
of pressure ulcers: clinical practice guideline. Washington, DC: National Pressure
Ulcer Advisory Panel; 2009.
13. Lyder CH, Ayello EA. Pressure ulcers: a patient safety issue. In: Patient Safety
and Quality: An Evidence-Based Handbook for Nurses. Rockville, MD: Agency
for Healthcare Research and Quality; 2008.
14. American Medical Directors Association. Pressure ulcers in the long-term
care setting. Columbia, MD: American Medical Directors Association; 2008.
15. Ayello EA, Sibbald RG. Preventing pressure ulcers and skin tears. In: Capezuti E, Zwicker D, Mezey M, Fulmer T, eds. Evidence-Based Geriatric Nursing
Protocols for Best Practice. 3rd ed. New York: Springer; 2008.
16. Registered Nurses’ Association of Ontario. Risk assessment & prevention of
pressure ulcers. Toronto, Ontario, Canada: Registered Nurses’ Association of
Ontario; 2005.
17. Pressure Ulcer Guideline Panel, Agency for Health Care Policy and Research. Pressure ulcers in adults: prediction and prevention. Rockville, MD: U.S.
Department of Health and Human Services, Public Health Service, Agency for
Health Care Policy and Research; 1992.
18. Institute for Healthcare Improvement. What is a bundle? 26 April 2011.
Accessed at www.ihi.org/knowledge/Pages/ImprovementStories/WhatIsaBundle
.aspx on 18 November 2011.
19. Young J, Ernsting M, Kehoe A, Holmes K. Results of a clinician-led
evidence-based task force initiative relating to pressure ulcer risk assessment and
prevention. J Wound Ostomy Continence Nurs. 2010;37:495-503. [PMID:
20736858]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 415
Supplement
Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy
20. Jankowski IM, Nadzam DM. Identifying gaps, barriers, and solutions in
implementing pressure ulcer prevention programs. Jt Comm J Qual Patient Saf.
2011;37:253-64. [PMID: 21706985]
21. Ogrinc G, Mooney SE, Estrada C, et al. The SQUIRE (Standards for
Quality Improvement Reporting Excellence) guidelines for quality improvement
reporting: explanation and elaboration. Qual Saf Health Care 2008 Oct;17 Suppl
1:i13-32. [PMID: 18836062]
22. Rantz MJ, Hicks L, Petroski GF, Madsen RW, Alexander G, Galambos C,
et al. Cost, staffing and quality impact of bedside electronic medical record
(EMR) in nursing homes. J Am Med Dir Assoc. 2010;11:485-93. [PMID:
20816336]
23. Rantz MJ, Popejoy L, Petroski GF, Madsen RW, Mehr DR, ZwygartStauffacher M, et al. Randomized clinical trial of a quality improvement intervention in nursing homes. Gerontologist. 2001;41:525-38. [PMID: 11490051]
24. Ryden MB, Snyder M, Gross CR, Savik K, Pearson V, Krichbaum K, et al.
Value-added outcomes: the use of advanced practice nurses in long-term care
facilities. Gerontologist. 2000;40:654-62. [PMID: 11131082]
25. Soban LM, Hempel S, Munjas BA, Miles J, Rubenstein LV. Preventing
pressure ulcers in hospitals: a systematic review of nurse-focused quality improvement interventions. Jt Comm J Qual Patient Saf. 2011;37:245-52. [PMID:
21706984]
26. Lyder CH, Grady J, Mathur D, Petrillo MK, Meehan TP. Preventing
pressure ulcers in Connecticut hospitals by using the plan-do-study-act model of
quality improvement. Jt Comm J Qual Saf. 2004;30:205-14. [PMID:
15085786]
27. Abel RL, Warren K, Bean G, Gabbard B, Lyder CH, Bing M, et al. Quality
improvement in nursing homes in Texas: results from a pressure ulcer prevention
project. J Am Med Dir Assoc. 2005;6:181-8. [PMID: 15894247]
28. Bales I, Padwojski A. Reaching for the moon: achieving zero pressure ulcer
prevalence. J Wound Care. 2009;18:137-144. [PMID: 19349933]
29. McInerney JA. Reducing hospital-acquired pressure ulcer prevalence through
a focused prevention program. Adv Skin Wound Care. 2008;21:75-8. [PMID:
18349734]
30. Ballard N, McCombs A, Deboor S, Strachan J, Johnson M, Smith MJ,
et al. How our ICU decreased the rate of hospital-acquired pressure ulcers. J Nurs
Care Qual. 2008;23:92-6. [PMID: 18281882]
31. Catania K, Huang C, James P, Madison M, Moran M, Ohr M. Wound
wise: PUPPI: the Pressure Ulcer Prevention Protocol Interventions. Am J Nurs.
2007;107:44-52. [PMID: 17413732]
32. Courtney BA, Ruppman JB, Cooper HM. Save our skin: initiative cuts
pressure ulcer incidence in half. Nurs Manage. 2006;37:36, 38, 40 passim.
[PMID: 16603946]
33. Gibbons W, Shanks HT, Kleinhelter P, Jones P. Eliminating facilityacquired pressure ulcers at Ascension Health. Jt Comm J Qual Patient Saf. 2006;
32:488-96. [PMID: 17987872]
34. Stier L, Dlugacz YD, O’Connor LJ, Eichorn AM, White M, Fitzpatrick J.
Reinforcing organization wide pressure ulcer reduction on high-risk geriatric inpatient units. Outcomes Manag. 2004;8:28-32. [PMID: 14740581]
35. Horn SD, Sharkey SS, Hudak S, Gassaway J, James R, Spector W. Pressure
ulcer prevention in long-term-care facilities: a pilot study implementing standard-
416 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
ized nurse aide documentation and feedback reports. Adv Skin Wound Care.
2010;23:120-31. [PMID: 20177165]
36. Tippet AW. Reducing the incidence of pressure ulcers in nursing home
residents: a prospective 6-year evaluation. Ostomy Wound Manage. 2009;55:
52-8. [PMID: 19934464]
37. Rosen J, Mittal V, Degenholtz H, Castle N, Mulsant BH, Hulland S, et al.
Ability, incentives, and management feedback: organizational change to reduce
pressure ulcers in a nursing home. J Am Med Dir Assoc. 2006;7:141-6. [PMID:
16503306]
38. Walsh NS, Blanck AW, Barrett KL. Pressure ulcer management in the acute
care setting: a response to regulatory mandates. J Wound Ostomy Continence
Nurs. 2009;36:385-8. [PMID: 19609158]
39. Lynch S, Vickery P. Steps to reducing hospital-acquired pressure ulcers.
Nursing. 2010;40:61-2. [PMID: 20975436]
40. Milne CT, Trigilia D, Houle TL, Delong S, Rosenblum D. Reducing
pressure ulcer prevalence rates in the long-term acute care setting. Ostomy
Wound Manage. 2009;55:50-9. [PMID: 19387096]
41. Delmore B, Lebovits S, Baldock P, Suggs B, Ayello EA. Pressure ulcer
prevention program: a journey. J Wound Ostomy Continence Nurs. 2011;38:
505-13. [PMID: 21860330]
42. Terry K. What is DMAIC? iSixSigma. 13 March 2010. Accessed at www
.isixsigma.com/methodology/dmaic-methodology/what-dmaic on 10 November
2011.
43. Chicano SG, Drolshagen C. Reducing hospital-acquired pressure ulcers.
J Wound Ostomy Continence Nurs. 2009;36:45-50. [PMID: 19155823]
44. Porter-O’Grady T. Shared governance. J Nurs Adm. 1995;25:8-9. [PMID:
7636580]
45. Dibsie LG. Implementing evidence-based practice to prevent skin breakdown. Crit Care Nurs Q. 2008;31:140-9. [PMID: 18360144]
46. Hiser B, Rochette J, Philbin S, Lowerhouse N, Terburgh C, Pietsch C.
Implementing a pressure ulcer prevention program and enhancing the role of the
CWOCN: impact on outcomes. Ostomy Wound Manage. 2006;52:48-59.
[PMID: 16464994]
47. LeMaster KM. Reducing incidence and prevalence of hospital-acquired pressure ulcers at Genesis Medical Center. Jt Comm J Qual Patient Saf. 2007;33:
611-6, 585. [PMID: 18030863]
48. Xakellis GC, Frantz R. The cost of healing pressure ulcers across multiple
health care settings. Adv Wound Care. 1996;9:18-22. [PMID: 9069752]
49. Agostini JV, Baker DI, Bogardus ST. Chapter 27: prevention of pressure
ulcers in older patients. In: Making Health Care Safer: A Critical Analysis of
Patient Safety Practices. AHRQ publication no. 01-E058. Evidence Report/
Technology Assessment no. 43. Washington, DC: Agency for Healthcare Research and Quality; 2001.
50. Pressure ulcer prevention points. Washington, DC: National Pressure Ulcer
Advisory Panel; 2007. Accessed at www.npuap.org/wp-content/uploads/2012/03
/PU_Prev_Points.pdf on 8 January 2013.
51. On-Time Prevention of Pressure Ulcers: Partnering With Quality Improvement Organizations. Final Report. Rockville, MD: Agency for Healthcare Research and Quality; 2007.
www.annals.org
Annals of Internal Medicine
Current Author Addresses: Ms. Sullivan and Dr. Schoelles: ECRI In-
stitute Evidence-based Practice Center, 5200 Butler Pike, Plymouth
Meeting, PA 19462-1298.
Author Contributions: Conception and design: N. Sullivan, K.M.
Schoelles.
Analysis and interpretation of the data: N. Sullivan, K.M. Schoelles.
Drafting of the article: N. Sullivan.
Critical revision of the article for important intellectual content: N.
Sullivan, K.M. Schoelles.
Final approval of the article: K.M. Schoelles.
Obtaining of funding: K.M. Schoelles.
Administrative, technical, or logistic support: N. Sullivan, K.M.
Schoelles.
Collection and assembly of data: N. Sullivan, K.M. Schoelles.
52. Kelleher AD, Moorer A, Makic MF. Peer-to-peer nursing rounds and
hospital-acquired pressure ulcer prevalence in a surgical intensive care unit: a
quality improvement project. J Wound Ostomy Continence Nurs. 2012;39:
152-7. [PMID: 22415120]
53. Ackerman CL. ‘Not on my watch:’ treating and preventing pressure ulcers.
Medsurg Nurs. 2011;20:86-93. [PMID: 21560959]
54. Institute for Healthcare Improvement. Prevent Pressure Ulcers. Cambridge,
MA: Institute for Healthcare Improvement; 2007.
W-184 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Appendix Table 1. Components of Pressure Ulcer Prevention Studies in U.S. Hospitals, 2000 –2012
Study
Multidisciplinary
Team
Skin
Champion
Education/
Training
Risk
Assessment
Tool
Review
Wound
Care
Products
Kelleher et al, 2012 (52)
⫻
⫻
⫻
⫻
⫻
Ackerman 2011 (53)*
⫻
⫻
⫻
⫻
Delmore et al, 2011 (41)
⫻
⫻
⫻
⫻
Lynch and Vickery,
2010 (39)
Young et al, 2010 (19)
⫻
Bales and Padwojski,
2009 (28)†‡§
Chicano and Droishagen,
2009 (43)‡
⫻
⫻
⫻
⫻
⫻
⫻
⫻
Walsh et al, 2009 (38)
⫻
⫻
⫻
⫻
⫻
Dibsie, 2008 (45)
McInerney, 2008 (29)†
Ballard et al. 2008 (30)†
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
Catania et al,
2007 (31)*†
⫻
⫻
⫻
⫻
⫻
⫻
LeMaster, 2007 (47)‡§
⫻
⫻
Courtney et al,
2006 (32)†
Gibbons et al,
2006 (33)†§
Hiser et al, 2006 (46)
⫻
Lyder et al, 2004 (26)*
⫻
⫻
Stier et al, 2004 (34)†
⫻
⫻
⫻
Upgrade
Automated
Systems
(Information
Technology)
Patient Care Integrate
Interventions New
Reporting
⫻
RA; MM;
F, S; N;
RP; SS
RA; MM;
F, S; N;
RP; SS
RA; SE; MM;
F, S; N;
SS; PEd
RA; SE; MM;
RP; SS
RA; SE; MM;
F, S; N;
RP; SS
RA; SE; MM;
N; RP; SS
RA; MM;
F, S; N;
RP; SS
RA; SE; MM;
N; RP; SS
MM;SS
RA; RP; SS
RA; MM;
F, S; RP;
SS
RA; SE; MM;
F, S; N;
RP; SS
RA; SE; MM;
F, S; N;
RP; SS
RA; MM; N;
RP; SS
RA; MM; N;
RP; SS
RA; MM;
F, S; N;
RP; SS
RA; SE; N;
RP; SS
RA; SE; MM;
F, S; RP;
SS
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
Implement
Protocol
⫻
⫻
⫻
⫻
⫻
⫻
⫻
Audit
and
Feedback
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
F, S ⫽ interventions to reduce friction and shear on at-risk body areas; MM ⫽ moisture management (includes incontinence interventions and skin care products); N ⫽
nutrition; PEd ⫽ patient and family education; RA ⫽ risk assessment (usually Braden scale, typically with repeated assessments during hospital stay); RP ⫽ repositioning or
increasing activity/time out of bed when possible; SE ⫽ frequent skin examinations; SS ⫽ support surfaces (includes specialty beds and heel supports or heel elevation).
* Audit only.
† Reported a statistically significant reduction in pressure ulcer rates.
‡ Reduced prevalence/incidence to 0.
§ Describes use of incentives.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) W-185
Appendix Table 2. Components of Pressure Ulcer Prevention Studies of Long-Term Care, 2000 –2012
Study
Multidisciplinary
Team
Use of
Outside
Consultants
Skin
Champion
Education/
Training
New
Assessment
Tool
Upgrade
Automated
Systems
(Information
Technology)
Implement
Protocol
Featured
Patient Care
Interventions
Integrate
New
Reporting
Audit
and
Feedback
Horn et al,
2010 (35)*†
Rantz et al,
2010 (22)*
Milne et al,
2009 (40)‡
Tippet,
2009 (36)†
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
RA; SE; MM;
N; RP
RP
⫻
⫻
RA; MM; N;
RP; SS
RA; SE; MM;
F, S; N; RP;
SS
RA; SE; RP;
AHCPR
RA; SE; MM;
N; RP; SS;
PEd
RA; AHCPR
⫻
⫻
⫻
⫻
Rosen et al,
2006 (37)†
Abel et al,
2005 (27)
Rantz et al,
2001 (23)§
Ryden et al,
2000 (24)†㛳
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
⫻
RA; MM; F, S;
RP
AHCPR ⫽ Agency for Health Care Policy and Research clinical practice guideline on pressure ulcer prediction and prevention; F, S ⫽ interventions to reduce friction and
shear on at-risk body areas; MM ⫽ moisture management (includes incontinence interventions and skin care products); N ⫽ nutrition; PEd ⫽ patient and family education;
RA ⫽ risk assessment (usually Braden scale, typically with repeated assessments during hospital stay); RP ⫽ repositioning or increasing activity/time out of bed when possible;
SE ⫽ frequent skin examinations; SS ⫽ support surfaces (includes specialty beds and heel supports or heel elevation).
* Study focused on improving communication of observations by nursing assistants using electronic documentation tools.
† Reported a statistically significant reduction in pressure ulcer rates.
‡ Long-term acute care hospital setting.
§ Study focused on use of minimum data set– derived quality indicators for quality improvement efforts.
储 Study focused on involvement of advance practice nurses to improve a variety of quality issues.
W-186 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Supplement
Annals of Internal Medicine
Rapid-Response Systems as a Patient Safety Strategy
A Systematic Review
Bradford D. Winters, MD, PhD; Sallie J. Weaver, PhD; Elizabeth R. Pfoh, MPH; Ting Yang, PhD; Julius Cuong Pham, MD, PhD;
and Sydney M. Dy, MD, MSc
Rapid-response systems (RRSs) are a popular intervention in
U.S. hospitals and are supported by accreditors and quality
improvement organizations. The purpose of this review is to
evaluate the effectiveness and implementation of these systems in acute care settings. A literature search was performed
between 1 January 2000 through 30 October 2012 using
PubMed, PsycINFO, CINAHL, and the Cochrane Central Register of Controlled Trials. Studies published in any language
evaluating outcome changes that occurred after implementing
an RRS and differences between groups using and not using
an RRS (effectiveness) or describing methods used by RRSs
(implementation) were reviewed.
A single reviewer (checked by a second reviewer) abstracted data
and rated study quality and strength of evidence. Moderatestrength evidence from a high-quality meta-analysis of 18 studies
and 26 lower-quality before-and-after studies published after that
meta-analysis showed that RRSs are associated with reduced rates
of cardiorespiratory arrest outside of the intensive care unit and
reduced mortality. Eighteen studies examining facilitators of and
barriers to implementation suggested that the rate of use of RRSs
could be improved.
THE PROBLEM
ent limb defines the variables that indicate deterioration
and democratizes that knowledge to all clinicians. It also
empowers bedside clinicians to trigger the response team
(or “efferent limb,” the team of clinicians that respond to
an event) when the clinician has a suspicion that a patient
is deteriorating (2). As such, most RRSs rely on clinicians
to proactively identify deteriorating patients rather than
solely on continuous monitoring technology, which is
common in the intensive care unit (ICU).
2) The response team (efferent limb). The response team
most frequently comprises ICU-trained personnel and
equipment. Team composition varies on the basis of local
needs and resources but generally uses one of the following
models: medical emergency teams (METs), which include
a physician; rapid-response teams, which do not include a
physician; and critical care outreach teams, which follow
up on patients discharged from an ICU but also respond to
all ward patients.
3) An administrative and quality improvement component. This team collects and analyzes event data and provides feedback, coordinates resources, and ensures improvement or maintenance over time.
Many hospitals have implemented RRSs to remedy
the failure of our current system to adequately monitor
patients in the general ward, recognize the signs and symptoms of deterioration, rescue deteriorating patients, and
deliver optimal care rapidly through escalation and triage.
That RRSs should be able to improve outcomes has strong
face validity. Given the rapid pace of RRS literature since the
Patients in the general ward often experience unrecognized deterioration that may progress to cardiorespiratory
arrest. Patients commonly show signs and symptoms of
deterioration for hours or days before cardiorespiratory arrest (median time, 6 hours) (1). Such arrests are associated
with a poor prognosis (mortality up to 80%).
Almost all cardiorespiratory arrests have a common set
of antecedents that are often poorly recognized secondary
to the low sensitivity and fidelity of periodic assessments by
staff. Improving this process should lead to earlier recognition and intervention. Many approaches have been devised
(for example, single- and multiple-track and trigger systems
and weighted early warning scoring systems), but none has
been shown to have a clear advantage.
Even when recognition of deterioration is prompt, intervention may lag because of such barriers as a physiciancentric medical culture that discourages speaking up or
bypassing the chain of command, and imbalances between
patient and clinician needs and resources. Improving recognition and overcoming the barriers to an effective and
timely response should reveal problems before they become
life-threatening.
PATIENT SAFETY STRATEGY
Rapid-response systems (RRSs) were created to improve recognition of and response to deterioration of patients on general hospital wards, with the goal of reducing
the incidence of cardiorespiratory arrest and hospital mortality. An RRS generally has 3 components.
1) Criteria and a system for notifying and activating the
response team (known as an “afferent limb,” the mechanism
by which team responses are triggered). Activation criteria
usually include vital signs (single-trigger criteria vs. aggregate and weighted early warning scoring) or general concern expressed by a clinician or family member. The afferwww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Ann Intern Med. 2013;158:417-425.
For author affiliations, see end of text.
www.annals.org
See also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 417
Supplement
Rapid-Response Systems
Key Summary Points
Many hospitals have implemented rapid-response systems
(RRSs) over the past 15 years to improve recognition of
and response to deteriorating patients in the general ward.
Moderate-strength evidence suggests that RRSs are associated with reduced rates of cardiorespiratory arrest and
mortality.
Important components of successful RRSs include criteria
and a system for notifying and activating the response
team; a response team; and an administrative and quality
improvement component to train staff, collect and analyze
event data, provide feedback, coordinate resources, and
ensure improvement or maintenance over time.
Implementation issues are critical in RRSs because rates of
use are often suboptimal secondary to various barriers that
could be improved.
last systematic review on the subject, done in 2010, we conducted this systematic review to update the current state of the
evidence for RRS effectiveness and implementation.
REVIEW PROCESSES
PubMed, PsycINFO, CINAHL, and the Cochrane
Central Register of Controlled Trials were searched between January 2000 through October 2012. The Supplement (available at www.annals.org) describes the search
strategies and includes the summary of evidence search and
selection. Effectiveness articles were restricted to studies
that used a MET, rapid-response team, or critical care outreach team model; had a comparison group; and were published after November 2008 (the end date for the highquality systematic review described later) (3).
Studies of RRS implementation selected for inclusion
could be either qualitative (that is, studies using interviews,
focus groups, or ethnographic methods) or quantitative
(that is, studies examining implementation strategy on use
of the RRS or patient outcomes that provided numerical
outcome data). There were no exclusions based on country
or language.
Two reviewers independently screened all abstracts.
Full articles identified for inclusion had outcome data abstracted by a single reviewer and checked by a second reviewer. We did not abstract data on nursing satisfaction,
which was rarely reported. The strength of evidence, including risk of bias, was evaluated using the Grading of
Recommendations Assessment, Development and Evaluation Working Group criteria adapted by the Agency for
Healthcare Research and Quality (4). We evaluated the
quality of systematic reviews by using the assessment of
multiple systematic reviews criteria (5).
418 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Of 2560 abstracts captured by the search strategy,
2321 were excluded during screening and 232 articles were
excluded in 2 rounds of article screening. Forty-three articles met the inclusion criteria (26 studies for effectiveness;
17 studies for implementation).
For effectiveness, the main outcome variables were
adult and pediatric non-ICU cardiorespiratory arrest and
adult and pediatric total hospital mortality. Studies that
provided complete raw data (sample sizes and number of
events both before and after the intervention periods) or
relative risk (RR) estimate and its 95% CI or RR estimate
with its associated accurate P value for any of these main
outcomes were included. Studies that did not provide sufficient quantitative data were excluded.
Using data from each study, we were able to present or
calculate a risk ratio estimate, its logarithm, and the associated SE. The results are summarized as risk ratios with
95% CIs and are shown in Figures 1 to 4, which include
the data from the high-quality systematic review (3) The
95% CIs were computed and plotted in the log scale but
labeled in the original scale. We used R, version 2.15.1 (R
Foundation for Statistical Computing, Vienna, Austria),
for these analyses. For implementation, Table 2 of the
Supplement shows qualitative summaries of individual
studies.
This review was supported by the Agency for Healthcare Research and Quality, which had no role in the selection or review of the evidence or the decision to submit
this manuscript for publication.
BENEFITS
AND
HARMS
Benefits
We identified 7 systematic reviews of RRSs; however,
only 1 was rated as high quality and is described in detail
here (3). A second review addressed implementation and is
discussed in the next section. The highest-quality systematic review (3) (assessment of multiple systematic reviews
criteria score, 10 of 11) identified 16 studies (6 –21)
through November 2008 involving nearly 1.3 million hospital admissions.
The meta-analysis concluded that, among adults, implementation of an RRS was associated with a statistical
reduction in non-ICU cardiorespiratory arrest (RR, 0.66
[95% CI, 0.54 to 0.80]) but not with lower total hospital
mortality (RR, 0.96 [CI, 0.84 to 1.09]). In children, implementation of an RRS was associated with statistical reductions in both non-ICU cardiorespiratory arrest (RR,
0.62 [CI, 0.46 to 0.84]) and total hospital mortality (RR,
0.79 [CI, 0.63 to 0.98]).
The review rated studies as high quality if they adjusted for confounders and for time trends by using either
concurrent control groups or an interrupted time series
design. Studies were rated as fair quality if they adjusted
only for confounding. Five of the 18 studies were rated as
high quality, 2 as fair quality, and the rest as low quality.
www.annals.org
Rapid-Response Systems
Supplement
Figure 1. Studies that reported the outcome of non–intensive care unit adult cardiorespiratory arrest.
Risk Ratio (95% CI)
Study, Year (Reference)
Bristow et al, 2000 (6) (first hospitalization)
0.88 (0.62–1.24)
Bristow et al, 2000 (6) (second hospitalization)
1.00 (0.73–1.37)
Buist et al, 2002 (7)
0.50 (0.35–0.72)
Bellomo et al, 2003 (8)
0.35 (0.22–0.56)
DeVita et al, 2004 (9)
0.81 (0.71–0.93)
Kenward et al, 2004 (10)
0.92 (0.72–1.17)
Hillman et al, 2005 (19)
0.94 (0.79–1.12)
Jones et al, 2005 (11)
0.47 (0.35–0.63)
Dacey et al, 2007 (13)
0.39 (0.26–0.58)
Jolley et al, 2007 (29)
0.86 (0.69–1.08)
Offner et al, 2007 (30)
0.32 (0.16–0.63)
Thomas et al, 2007 (31)
0.52 (0.12–2.28)
Baxter et al, 2008 (14)
0.61 (0.40–0.94)
Bosch and de Jager, 2008 (47)
0.50 (0.41–0.61)
Chan et al, 2008 (20)
0.59 (0.40–0.88)
Hatler et al, 2009 (37)
0.68 (0.36–1.28)
Gerdik et al, 2010 (35)
0.26 (0.12–0.57)
Beitler et al, 2011 (33)
0.49 (0.40–0.61)
Sarani et al, 2011 (44) (adjusted)
0.57 (0.47–0.69)
Shah et al, 2011 (45)
0.97 (0.72–1.31)
0.05
0.10
0.25
0.50
1.00
2.00
4.00
Risk Ratio
In this updated review, we identified 26 additional
effectiveness studies (22– 47) that met our inclusion criteria and were published since the previous high-quality systematic review. None used randomization in its methodology or had a concurrent control group; 2 studies included
multiple centers. Three studies were done in pediatric hospitals. Most took place in the United States, Australia, or
Canada, with only a few in Europe or Asia. Most studies
were conducted in teaching hospitals.
Almost no studies included any information on context, and no studies reported a theoretical or logic model.
The number of included hospital admissions or discharges
in the studies ranged from 1920 to 277 717. All were rated
as low or moderate quality.
Most studies reported the main outcomes of non-ICU
cardiorespiratory arrest and total hospital mortality; some
studies also reported variations on these outcomes, such as
unexpected cardiorespiratory arrest or unexpected mortality. From our included studies, Figures 1 to 4 show studies
providing adequate data (29 – 47) along with the studies
included in the high-quality review.
Figure 1 shows studies describing adult non-ICU cardiorespiratory arrest. Nineteen of 20 studies reporting this
outcome had point estimates favoring the intervention, 12
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
of which reached statistical significance. Figure 2 shows
adult total hospital mortality. Eighteen of 23 studies
showed favorable point estimates, 7 of which were
significant.
Figure 3 shows pediatric non-ICU cardiorespiratory
arrest; all point estimates favored the RRS, and 3 of 7 were
significant. Figure 4 shows pediatric total hospital mortality, where all but 1 study (21) had point estimates favoring
RRSs; however, only 2 of these findings were significant.
More recent studies more often showed positive results. Although outcomes were heterogeneous—number of
hospital discharges during the study period, type of hospital (size and teaching vs. nonteaching status), and RRS
model—there was no clear correlation between intervention effectiveness and these characteristics (data not
shown).
The overall strength of evidence was moderate when
the high-quality systematic review (3) was included, but
strength of evidence in the additional studies identified
since 2008 was low to moderate. Risk of bias was high for
all additional studies because of the before-and-after design. Only a few studies accounted for differences in patient populations over time or reported characteristics of
providers in the 2 periods.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 419
Supplement
Rapid-Response Systems
Figure 2. Studies that reported the outcome of total hospital adult mortality.
Risk Ratio (95% CI)
Study, Year (Reference)
Bristow et al, 2000 (6) (first hospitalization)
0.93 (0.77–1.12)
Bristow et al, 2000 (6) (second hospitalization)
1.20 (1.00–1.43)
Buist et al, 2002 (7)
0.87 (0.76–1.01)
Bellomo et al, 2003 (8)
0.74 (0.70–0.79)
Kenward et al, 2004 (10)
0.99 (0.91–1.07)
Priestley et al, 2004 (12)
0.52 (0.32–0.85)
Hillman et al, 2005 (19)
1.03 (0.83–1.27)
Jones et al, 2005 (11)
1.18 (1.10–1.27)
Dacey et al, 2007 (13)
1.07 (0.87–1.31)
Jolley et al, 2007 (29)
1.00 (0.74–1.36)
Baxter et al, 2008 (14)
0.99 (0.85–1.15)
Chan et al, 2008 (20) (adjusted)
0.95 (0.81–1.11)
Campello et al, 2009 (34) (adjusted)
0.83 (0.64–1.07)
Konrad et al, 2010 (39) (adjusted)
0.90 (0.84–0.97)
Lighthall et al, 2010 (42)
0.82 (0.62–1.09)
Santamaria et al, 2010 (43)
0.48 (0.41–0.56)
Beitler et al, 2011 (33) (adjusted)
0.89 (0.82–0.97)
Laurens and Dwyer, 2011 (41)
0.76 (0.66–0.87)
Sarani et al, 2011 (44) (medicine)
0.74 (0.68–0.80)
Sarani et al, 2011 (44) (surgery)
0.92 (0.80–1.05)
Shah et al, 2011 (45)
0.89 (0.79–1.00)
Howell et al, 2012 (38) (adjusted)
0.91 (0.82–1.01)
Tobin and Santamaria, 2012 (46) (adjusted)
0.90 (0.88–0.92)
0.05
0.25
0.50
1.00
2.00
4.00
Risk Ratio
Few studies attempted to control for secular trends
over time that could have affected outcomes. The 1 study
that accounted for such trends found that benefits in mortality and cardiorespiratory arrest rate remained after they
were adjusted for (33). No studies reported on or ac-
counted for other safety initiatives that may have influenced the outcomes.
No studies conducted blinded outcome assessment.
Although mortality is an objective outcome, the other key
outcome, incidence of cardiorespiratory arrest, can be de-
Figure 3. Studies that reported the outcome of non–intensive care unit pediatric cardiorespiratory arrest.
Risk Ratio (95% CI)
Study, Year (Reference)
Brilli et al, 2007 (15)
0.41 (0.20–0.86)
Sharek et al, 2007 (16)
0.29 (0.13–0.65)
Zenker et al, 2007 (21)
0.64 (0.47–0.87)
Hunt et al, 2008 (18)
0.49 (0.20–1.20)
Hanson et al, 2009 (36)
0.35 (0.10–1.24)
Tibballs and Kinney, 2009 (17)
0.91 (0.50–1.64)
Anwar-ul-Haque et al, 2010 (32)
0.52 (0.12–2.26)
0.05
0.25
0.50
1.00
4.00
Risk Ratio
420 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
www.annals.org
Rapid-Response Systems
Supplement
Figure 4. Studies that reported the outcome of total hospital pediatric mortality.
Risk Ratio (95% CI)
Study, Year (Reference)
Brilli et al, 2007 (15)
0.55 (0.14–2.10)
Sharek et al, 2007 (16)
0.82 (0.71–0.95)
Zenker et al, 2007 (21)
1.05 (0.74–1.50)
Hanson et al, 2009 (36)
0.08 (0.01–1.03)
Tibballs and Kinney, 2009 (17)
0.65 (0.56–0.75)
Kotsakis et al, 2011 (40)
0.97 (0.84–1.12)
0.05
0.25
0.50
1.00
4.00
Risk Ratio
fined in numerous ways (for example, calling the code
team vs. documented use of cardiopulmonary resuscitation) and is subject to bias, as are some of the other
outcome variations reported (for example, unexpected
mortality).
Studies ideally should not report rates for the ICU and
emergency departments because these patient populations
are rarely part of the exposure group; however, hospitalwide rates were often reported. One study (20) included
ICU cardiorespiratory arrest in the analysis (total hospital
cardiorespiratory arrest); this study concluded that there was
no effect, although data presented on the non-ICU cardiorespiratory arrest rate showed a statistical improvement.
Such arrest rates are also affected by changes in patient
case-mix over time, frequency of do-not-resuscitate orders,
and terminal illness. However, most studies did not account for these potential confounders. Other outcomes reported, such as unanticipated ICU admissions, are indirect
outcomes. In addition, no studies compared RRSs with
other interventions that may affect these outcomes, such as
enhanced nurse–patient ratios or hospitalists. An unexpected beneficial consequence was improved frequency and
quality of end-of-life discussions with patients and their
families.
Harms
Potential harms included “deskilling” of ward staff because of dependence on the RRS, inappropriate patient
care for other patients (decreased responsiveness of the
usual team), staff conflict, diversion of ICU staff from
usual care in the ICU, and communication errors caused
by introducing additional providers (2). Despite several articles discussing potential harms and unexpected consequences, neither the high-quality systematic review nor any
of the additional studies reported any quantitative data for
these variables.
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
Seventeen studies (10, 48 – 63) met our inclusion criteria for studies of the implementation processes surroundwww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
ing RRSs. Eleven of these used quantitative methods primarily for evaluating the effect of a change in the
implementation process for an RRS program and 7 used
primarily qualitative methods. Most implementation studies were conducted in academic hospitals; however, several
studies specifically detailed implementation in community
hospitals (10, 13, 14, 23).
Rapid-response systems have been implemented in
several contexts and vary in composition, activation criteria, and implementation process. Strong external factors
have driven the implementation of RRSs in U.S. hospitals:
The 2009 Joint Commission’s National Patient Safety
Goal 16 (64) and the Institute for Healthcare Improvement (65), as well as numerous other organizations, have
created toolkits to help implement RRS interventions. Despite these attempts to reduce variability in the implementation process, our review found that implementation processes differed widely and that local needs and resources
tended to dominate the processes.
Education and promotion of the new service was often
a factor in preparing for implementation. For staff training
and education, several studies introduced new staff, such as
a nurse educator. Most studies indicated that implementation processes explicitly included educational activities;
however, such activities varied in the degree to which they
were strictly information-based or included dedicated
training and practice opportunities for RRS members or
staff. Such activities as simulation education and training
were uncommon. Most studies explicitly noted that cognitive aids, such as posters listing activation criteria, were
included.
During development of the afferent limb, various objective criteria were used for calling the team and many
interventions depended on nurses’ clinical judgment to
activate the team on the basis of subjective “worry” or
“general concern” (58, 59). One study found that MET
hospitals in the MERIT (Medical Emergency Response Intervention and Therapy) trial were 35 times more likely to
activate their emergency response team based on the “wor5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 421
Supplement
Rapid-Response Systems
ried” criteria than control (activated by vital signs) hospitals (P ⬍ 0.001) (58).
A few studies also included systems for family or patient initiation of the RRS team. One study showed improved outcomes only after family-initiated activation was
implemented (35). No studies reported specific use of technology (such as computerized alerts) to enhance RRS implementation. Some studies used single vital sign triggers,
whereas others used early warning scoring systems. In most
reports, activation of the efferent limb was voluntary, although 1 study changed its program to mandatory activation on the basis of alert criteria (59).
Most studies used METs as the efferent limb model,
but several studies examined rapid-response teams or critical care outreach teams (23); however, none were directly
compared. One of the very few studies to compare these
models studied a resident-led team and an attending-led
MET and found no difference in outcomes (24).
Many RRS implementation efforts have low utilization rates; that is, ward staff do not activate the team despite criteria for activation being met. Although the systematic reviews that we identified did not address this
issue, several did so individually. Jones and colleagues (66)
examined the barriers and facilitators that affecting nurses’
activation of the RRS. The following 5 major themes
emerged: adequate education on the RRSs’ purpose and
role, clinical expertise, support by medical and nursing
staff, nurses’ familiarity with and advocacy for the patient,
and nurses’ workload.
Other studies found changes in culture (that is, development of strong support for calling for help and lack of
criticism or punishment for activating the team), knowledge of activation criteria, communication, teamwork, and
perceptions about the team’s helpfulness to nurses and patients to be important influences on utilization. Another
factor affecting utilization is the time since initial implementation or duration of the RRS. For example, 1 study
(67) specifically examined RRS processes over time and
found that the proportion of patients with delayed RRS
activation decreased as the RRS matured (40.3% vs.
22.0%; P ⬍ 0.001). Other programs have tried various
strategies to improve utilization (education, mandatory activation, and changing the activation criteria) (59 – 63).
Team structure may also influence utilization. For example, 1 study reported the effects of separating the overall
emergency response system into 2 teams with different activation criteria and processes. Utilization increased sharply
(15.7 to 24.7 activations per 1000 admissions; P ⬍ 0.001)
after the changes were implemented (61).
Patient populations may also benefit differently from
RRSs. The high-quality systematic review concluded that
RRSs were associated with significantly reduced hospital
mortality in pediatric patients but not in adults (3). One
study that included 2 separate RRS teams showed an effect
in a medical but not a surgical population (44). Most studies to date were conducted in academic centers, although
422 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
nonacademic hospitals also frequently reported RRS success. Earlier studies that the high-quality systematic review
reported were mainly from Australia and the United States;
2 were in England, and 1 was in Canada (3). Since then,
the number of countries reporting effectiveness data for
RRSs has increased, but how differing national medical
cultures affect implementation and effectiveness of the intervention is unclear.
Finally, cost was not evaluated in the high-quality review (3) or in the additional articles that we reviewed.
DISCUSSION
The previous high-quality systematic review and metaanalysis found that, although RRSs were associated with a
statistical reduction in rates of cardiorespiratory arrest outside of the ICU among pediatric and adult patients, total
hospital mortality was not reduced in adults (3). Our update supports the previous conclusions, although the most
recent studies were more likely to show positive results.
The high-quality review found the opposite in its cumulative analysis: Early studies tended to have more positive results. In fact, in 7 sequentially published studies,
starting with Kenward and associates’ study (10) in 2004
and continuing to Chan and coworkers’ study (20) in
2008, the point estimate of effect did not decrease below
0.95.
After Chan and coworkers’ study (20) in 2008, all
point estimates were less than 0.95. Potential explanations
for this result include maturation of the intervention and
improved implementation strategies that may have led to
improved results in and across institutions. In addition,
secular trends of total hospital mortality may have decreased over time unrelated to the intervention, and few
studies controlled for this, although 1 study that did found
it to not be the case (33).
Although the beneficial effects of RRSs are becoming
clearer as the intervention is more universally applied, not
all RRS programs realize these benefits. There are several
potential explanations for this. An archaic model of patient
monitoring on general wards limits the afferent limb (the low
sensitivity of periodic visits by clinicians to identify deteriorating patients) (1). Automating the identification of a deteriorating patient through continuous monitoring and a
directly activated response team potentially would both improve sensitivity and fidelity and mitigate cultural barriers.
Optimal team composition and structure are unknown. Restricted financial resources may also affect the
RRS’s ability to self-audit, evaluate events, and improve
systematically. Utilization rates are often reported to be
low. Creativity and maturation of the intervention are necessary to achieve ultimate long-term goals. Other factors
may affect commonly measured outcomes, and several
metrics (that is, total cardiorespiratory arrest) count patients who are not exposed to the intervention. Staff and
education themes mainly focus on information rather than
www.annals.org
Rapid-Response Systems
training. Barriers to effective recognition and response ingrained in the culture of medicine persist.
Given that 80% of patients who have an inpatient
cardiorespiratory arrest die, several potential reasons may
explain why results for mortality are less robust than those
for arrests. For example, the measurement of cardiorespiratory arrest can be subjective. In addition, many arrests
occur in terminally ill patients, and some studies of RRSs
have found evidence for increased rates of do-notresuscitate orders after RRS implementation. Some studies
have tried to account for this factor by measuring rates of
cardiorespiratory arrests that are “unexpected” or that occur in patients without do-not-resuscitate orders, but accurately defining terminally ill patients is challenging and
may be subject to bias or measurement error.
Our review and the literature have limitations. The
updated literature since 2008 includes low- to moderatequality studies, and several studies have inconsistent findings across outcomes. The elements of the RRS, sample
size, and reporting of outcomes varied among these studies.
All of the most recent studies have used a before-and-after
historically controlled design, which needs to be considered
carefully because a recent evaluation of a multifaceted patient safety program in the United Kingdom found statistical improvements in the before-and-after comparison but
not in the concurrent cohort controlled comparison (68),
as the MERIT study (19) did.
In addition, we reviewed only “effectiveness” studies
that reported raw data for mortality and cardiorespiratory
arrest. Also, the possibility of selective reporting and publication bias cannot be excluded. For implementation studies, few used formal qualitative methods, and these also
addressed various RRS types and study populations. Finally, the relative effectiveness of RRSs compared with
other interventions to identify and treat deteriorating patients is unknown.
In summary, we found moderate strength of evidence
that RRSs improve outcomes from both a high-quality systematic review through November 2008 and the additional
literature published through October 2012. Our review
also identified key barriers and facilitators of effective RRS
implementation, which included staff acceptance and leadership of the RRS, rates of calling the RRS, and trigger
mechanisms.
Rapid-response systems have been described as a
“band-aid” for a failed model of managing patients in the
general ward in hospitals (69). Although this intervention
is beginning to help many hospitals increase recognition of
patient deterioration and reduce preventable deaths, they
are unlikely to more universally improve these outcomes
until we address the culture and system defects that contribute to the root of the problem. For now, RRSs seem to
be the best option.
From Johns Hopkins School of Medicine and Johns Hopkins University,
Baltimore, Maryland.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Note: The Agency for Healthcare Research and Quality reviewed contract deliverables to ensure adherence to contract requirements and quality, and a copyright release was obtained from the Agency for Healthcare
Research and Quality before submission of the manuscript.
Disclaimer: All statements expressed in this work are those of the authors
and should not in any way be construed as official opinions or positions
of the Johns Hopkins University, Agency for Healthcare Research and
Quality, or U.S. Department of Health and Human Services.
Financial Support: From the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services (contract HHSA290- 2007-10062I).
Potential Conflicts of Interest: Dr. Winters: Grant (money to institution): Agency for Healthcare Research and Quality; Employment: Johns
Hopkins University; Expert testimony: several law firms; Payment for lectures including service on speakers bureaus: 3M; Royalties: Lippincott. Dr.
Weaver: Grant (money to institution): Agency for Healthcare Research
and Quality; Travel/accommodations/meeting expenses unrelated to activities
listed: Improvement Science Research Network. Ms. Pfoh: Grant (money
to institution): Agency for Healthcare Research and Quality. Dr. Dy:
Grant (money to institution): Agency for Healthcare Research and Quality.
All other authors have no disclosures. Disclosures can also be viewed at
www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum
⫽M12-2568.
Requests for Single Reprints: Bradford D. Winters, MD, PhD, De-
partment of Anesthesiology and Critical Care Medicine and Armstrong
Institute for Patient Safety and Quality, Johns Hopkins University
School of Medicine, Zayed 9127, 1800 Orleans Street, Baltimore, MD
21287; e-mail, [email protected].
Current author addresses and author contributions are available at
www.annals.org.
References
1. Buist MD, Jarmolowski E, Burton PR, Bernard SA, Waxman BP, Anderson
J. Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care. A pilot study in a tertiary-care hospital. Med
J Aust. 1999;171:22-5. [PMID: 10451667]
2. Jones DA, DeVita MA, Bellomo R. Rapid-response teams. N Engl J Med.
2011;365:139-46. [PMID: 21751906]
3. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid Response
Teams: A Systematic Review and Meta-analysis. Arch Intern Med. 2010;170:1826. [PMID: 20065195]
4. Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al.
AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—Agency for Healthcare Research and Quality and the
Effective Health-Care Program. J Clin Epidemiol. 2010;63:513-23. [PMID:
19595577]
5. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al.
Development of AMSTAR: a measurement tool to assess the methodological
quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID:
17302989]
6. Bristow PJ, Hillman KM, Chey T, Daffurn K, Jacques TC, Norman SL,
et al. Rates of in-hospital arrests, deaths and intensive care admissions: the effect
of a medical emergency team. Med J Aust. 2000;173:236-40. [PMID:
11130346]
7. Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, Nguyen
TV. Effects of a medical emergency team on reduction of incidence of and
mortality from unexpected cardiac arrests in hospital: preliminary study. BMJ.
2002;324:387-90. [PMID: 11850367]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 423
Supplement
Rapid-Response Systems
8. Bellomo R, Goldsmith D, Uchino S, Buckmaster J, Hart GK, Opdam H,
et al. A prospective before-and-after trial of a medical emergency team.
Med J Aust. 2003;179:283-7. [PMID: 12964909]
9. DeVita MA, Braithwaite RS, Mahidhara R, Stuart S, Foraida M, Simmons
RL; Medical Emergency Response Improvement Team (MERIT). Use of medical emergency team responses to reduce hospital cardiopulmonary arrests. Qual
Saf Health Care. 2004;13:251-4. [PMID: 15289626]
10. Kenward G, Castle N, Hodgetts T, Shaikh L. Evaluation of a medical
emergency team one year after implementation. Resuscitation. 2004;61:257-63.
[PMID: 15172703]
11. Jones D, Bellomo R, Bates S, Warrillow S, Goldsmith D, Hart G, et al.
Long term effect of a medical emergency team on cardiac arrests in a teaching
hospital. Crit Care. 2005;9:R808-15. [PMID: 16356230]
12. Priestley G, Watson W, Rashidian A, Mozley C, Russell D, Wilson J, et al.
Introducing Critical Care Outreach: a ward-randomised trial of phased introduction in a general hospital. Intensive Care Med. 2004;30:1398-404. [PMID:
15112033]
13. Dacey MJ, Mirza ER, Wilcox V, Doherty M, Mello J, Boyer A, et al. The
effect of a rapid response team on major clinical outcome measures in a community hospital. Crit Care Med. 2007;35:2076-82. [PMID: 17855821]
14. Baxter AD, Cardinal P, Hooper J, Patel R. Medical emergency teams at The
Ottawa Hospital: the first two years. Can J Anaesth. 2008;55:223-31. [PMID:
18378967]
15. Brilli RJ, Gibson R, Luria JW, Wheeler TA, Shaw J, Linam M, et al.
Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care
unit. Pediatr Crit Care Med. 2007;8:236-46. [PMID: 17417113]
16. Sharek PJ, Parast LM, Leong K, Coombs J, Earnest K, Sullivan J, et al.
Effect of a rapid response team on hospital-wide mortality and code rates outside
the ICU in a children’s hospital. JAMA. 2007;298:2267-74. [PMID: 18029830]
17. Tibballs J, Kinney S. Reduction of hospital mortality and of preventable
cardiac arrest and death on introduction of a pediatric medical emergency team.
Pediatr Crit Care Med. 2009;10:306-12. [PMID: 19307806]
18. Hunt EA, Zimmer KP, Rinke ML, Shilkofski NA, Matlin C, Garger C,
et al. Transition from a traditional code team to a medical emergency team and
categorization of cardiopulmonary arrests in a children’s center. Arch Pediatr
Adolesc Med. 2008;162:117-22. [PMID: 18250234]
19. Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al;
MERIT study investigators. Introduction of the medical emergency team
(MET) system: a cluster-randomised controlled trial. Lancet. 2005;365:2091-7.
[PMID: 15964445]
20. Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA.
Hospital-wide code rates and mortality before and after implementation of a
rapid response team. JAMA. 2008;300:2506-13. [PMID: 19050194]
21. Zenker P, Schlesinger A, Hauck M, Spencer S, Hellmich T, Finkelstein M,
et al. Implementation and impact of a rapid response team in a children’s hospital. Jt Comm J Qual Patient Saf. 2007;33:418-25. [PMID: 17711144]
22. Benson L, Mitchell C, Link M, Carlson G, Fisher J. Using an advanced
practice nursing model for a rapid response team. Jt Comm J Qual Patient Saf.
2008;34:743-7. [PMID: 19119728]
23. Bader MK, Neal B, Johnson L, Pyle K, Brewer J, Luna M, et al. Rescue me:
saving the vulnerable non-ICU patient population. Jt Comm J Qual Patient Saf.
2009;35:199-205. [PMID: 19435159]
24. Karvellas CJ, de Souza IA, Gibney RT, Bagshaw SM. Association between
implementation of an intensivist-led medical emergency team and mortality.
BMJ Qual Saf. 2012;21:152-9. [PMID: 22190540]
25. Medina-Rivera B, Campos-Santiago Z, Palacios AT, Rodriguez-Cintron
W. The effect of the medical emergency team on unexpected cardiac arrest and
death at the VA Caribbean Healthcare System: a retrospective study. Critical Care
and Shock. 2010;13:98-105.
26. Rothberg MB, Belforti R, Fitzgerald J, Friderici J, Keyes M. Four years’
experience with a hospitalist-led medical emergency team: an interrupted time
series. J Hosp Med. 2012;7:98-103. [PMID: 21998088]
27. Scherr K, Wilson DM, Wagner J, Haughian M. Evaluating a new rapid
response team: NP-led versus intensivist-led comparisons. AACN Adv Crit Care.
2012;23:32-42. [PMID: 22290088]
28. Scott SS, Elliott S. Implementation of a rapid response team: a success story.
Crit Care Nurse. 2009;29:66-75. [PMID: 19487782]
424 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
29. Jolley J, Bendyk H, Holaday B, Lombardozzi KA, Harmon C. Rapid response teams: do they make a difference? Dimens Crit Care Nurs. 2007;26:25360. [PMID: 18090145]
30. Offner PJ, Heit J, Roberts R. Implementation of a rapid response team
decreases cardiac arrest outside of the intensive care unit. J Trauma. 2007;62:
1223-7. [PMID: 17495728]
31. Thomas K, VanOyen Force M, Rasmussen D, Dodd D, Whildin S. Rapid
response team: challenges, solutions, benefits. Crit Care Nurse. 2007;27:20-7.
[PMID: 17244856]
32. Anwar-ul-Haque, Saleem AF, Zaidi S, Haider SR. Experience of pediatric
rapid response team in a tertiary care hospital in Pakistan. Indian J Pediatr.
2010;77:273-6. [PMID: 20177830]
33. Beitler JR, Link N, Bails DB, Hurdle K, Chong DH. Reduction in hospitalwide mortality after implementation of a rapid response team: a long-term cohort
study. Crit Care. 2011;15:R269. [PMID: 22085785]
34. Campello G, Granja C, Carvalho F, Dias C, Azevedo LF, Costa-Pereira A.
Immediate and long-term impact of medical emergency teams on cardiac arrest
prevalence and mortality: a plea for periodic basic life-support training programs.
Crit Care Med. 2009;37:3054-61. [PMID: 19770754]
35. Gerdik C, Vallish RO, Miles K, Godwin SA, Wludyka PS, Panni MK.
Successful implementation of a family and patient activated rapid response team
in an adult level 1 trauma center. Resuscitation. 2010;81:1676-81. [PMID:
20655645]
36. Hanson CC, Randolph GD, Erickson JA, Mayer CM, Bruckel JT, Harris
BD, et al. A reduction in cardiac arrests and duration of clinical instability after
implementation of a paediatric rapid response system. Qual Saf Health Care.
2009;18:500-4. [PMID: 19955465]
37. Hatler C, Mast D, Bedker D, Johnson R, Corderella J, Torres J, et al. Implementing a rapid response team to decrease emergencies outside the ICU: one hospital’s experience. Medsurg Nurs. 2009;18:84-90, 126. [PMID: 19489205]
38. Howell MD, Ngo L, Folcarelli P, Yang J, Mottley L, Marcantonio ER,
et al. Sustained effectiveness of a primary-team-based rapid response system. Crit
Care Med. 2012;40:2562-8. [PMID: 22732285]
39. Konrad D, Jäderling G, Bell M, Granath F, Ekbom A, Martling CR.
Reducing in-hospital cardiac arrests and hospital mortality by introducing a medical emergency team. Intensive Care Med. 2010;36:100-6. [PMID: 19760206]
40. Kotsakis A, Lobos AT, Parshuram C, Gilleland J, Gaiteiro R, Mohseni-Bod
H, et al; Ontario Pediatric Critical Care Response Team Collaborative. Implementation of a multicenter rapid response system in pediatric academic hospitals
is effective. Pediatrics. 2011;128:72-8. [PMID: 21690113]
41. Laurens N, Dwyer T. The impact of medical emergency teams on ICU
admission rates, cardiopulmonary arrests and mortality in a regional hospital.
Resuscitation. 2011;82:707-12. [PMID: 21411218]
42. Lighthall GK, Parast LM, Rapoport L, Wagner TH. Introduction of a rapid
response system at a United States veterans affairs hospital reduced cardiac arrests.
Anesth Analg. 2010;111:679-86. [PMID: 20624835]
43. Santamaria J, Tobin A, Holmes J. Changing cardiac arrest and hospital
mortality rates through a medical emergency team takes time and constant review. Crit Care Med. 2010;38:445-50. [PMID: 20029341]
44. Sarani B, Palilonis E, Sonnad S, Bergey M, Sims C, Pascual JL, et al.
Clinical emergencies and outcomes in patients admitted to a surgical versus medical service. Resuscitation. 2011;82:415-8. [PMID: 21242020]
45. Shah SK, Cardenas VJ Jr, Kuo YF, Sharma G. Rapid response team in an
academic institution: does it make a difference? Chest. 2011;139:1361-7.
[PMID: 20864618]
46. Tobin AE, Santamaria JD. Medical emergency teams are associated with
reduced mortality across a major metropolitan health network after two years
service: a retrospective study using government administrative data. Crit Care.
2012;16:R210. [PMID: 23107123]
47. Bosch FH, de Jager CPC. Number of resuscitations for in-hospital cardiopulmonary arrests decreases after introduction of a medical emergency team.
“The Arnhem experience.” Netherlands Journal of Critical Care. 2008;12:256-9.
48. Calzavacca P, Licari E, Tee A, Egi M, Downey A, Quach J, et al. The
impact of Rapid Response System on delayed emergency team activation patient
characteristics and outcomes—a follow-up study. Resuscitation. 2010;81:31-5.
[PMID: 19854557]
49. Williams DJ, Newman A, Jones C, Woodard B. Nurses’ perceptions of how
rapid response teams affect the nurse, team, and system. J Nurs Care Qual.
2011;26:265-72. [PMID: 21248644]
www.annals.org
Rapid-Response Systems
50. Shapiro SE, Donaldson NE, Scott MB. Rapid response teams seen through the
eyes of the nurse. Am J Nurs. 2010;110:28-34. [PMID: 20505459]
51. Donaldson N, Shapiro S, Scott M, Foley M, Spetz J. Leading successful
rapid response teams: A multisite implementation evaluation. J Nurs Adm. 2009;
39:176-81. [PMID: 19359889]
52. Adelstein BA, Piza MA, Nayyar V, Mudaliar Y, Klineberg PL, Rubin G.
Rapid response systems: a prospective study of response times. J Crit Care. 2011;
26:635.e11-8. [PMID: 21703813]
53. Cretikos MA, Chen J, Hillman KM, Bellomo R, Finfer SR, Flabouris A;
MERIT Study Investigators. The effectiveness of implementation of the medical
emergency team (MET) system and factors associated with use during the
MERIT study. Crit Care Resusc. 2007;9:206-12. [PMID: 17536993]
54. Mackintosh N, Rainey H, Sandall J. Understanding how rapid response
systems may improve safety for the acutely ill patient: learning from the frontline.
BMJ Qual Saf. 2012;21:135-44. [PMID: 21972419]
55. Shearer B, Marshall S, Buist MD, Finnigan M, Kitto S, Hore T, et al.
What stops hospital clinical staff from following protocols? An analysis of the
incidence and factors behind the failure of bedside clinical staff to activate the
rapid response system in a multi-campus Australian metropolitan healthcare service. BMJ Qual Saf. 2012;21:569-75. [PMID: 22626737]
56. Soo S, Berta W, Baker GR. Role of champions in the implementation of
patient safety practice change. Healthc Q. 2009;12 Spec No Patient:123-8.
[PMID: 19667789]
57. Chen J, Bellomo R, Hillman K, Flabouris A, Finfer S; MERIT Study
Investigators for the Simpson Centre and the ANZICS Clinical Trials Group.
Triggers for emergency team activation: a multicenter assessment. J Crit Care.
2010;25:359.e1-7. [PMID: 20189754]
58. Genardi ME, Cronin SN, Thomas L. Revitalizing an established rapid response team. Dimens Crit Care Nurs. 2008;27:104-9. [PMID: 18434864]
59. Jones CM, Bleyer AJ, Petree B. Evolution of a rapid response system from
voluntary to mandatory activation. Jt Comm J Qual Patient Saf. 2010;36:26670, 241. [PMID: 20564888]
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
60. Peebles E, Subbe CP, Hughes P, Gemmell L. Timing and teamwork—an
observational pilot study of patients referred to a rapid response team with the
aim of identifying factors amenable to re-design of a rapid response system. Resuscitation. 2012;83:782-7. [PMID: 22209834]
61. Jones DA, Mitra B, Barbetti J, Choate K, Leong T, Bellomo R. Increasing
the use of an existing medical emergency team in a teaching hospital. Anaesth
Intensive Care. 2006;34:731-5. [PMID: 17183890]
62. Foraida MI, DeVita MA, Braithwaite RS, Stuart SA, Brooks MM,
Simmons RL. Improving the utilization of medical crisis teams (Condition C) at an urban tertiary care hospital. J Crit Care. 2003;18:87-94.
[PMID: 12800118]
63. Jones D, Bates S, Warrillow S, Goldsmith D, Kattula A, Way M, et al.
Effect of an education programme on the utilization of a medical emergency team
in a teaching hospital. Intern Med J. 2006;36:231-6. [PMID: 16640740]
64. Joint Commission on Accreditation of Healthcare Organizations.
2008 National Patient Safety Goals. Joint Commission Perspectives. 2007;
27:10-22.
65. Institute for Healthcare Improvement. 5 Million Lives Campaign: Overview. Accessed at www.ihi.org/offerings/Initiatives/PastStrategicInitiatives
/5MillionLivesCampaign/Pages/default.aspx on 28 November 2012.
66. Jones L, King L, Wilson C. A literature review: factors that impact on nurses’
effective use of the Medical Emergency Team (MET). J Clin Nurs. 2009;18:
3379-90. [PMID: 20487489]
67. Buist M, Harrison J, Abaloz E, Van Dyke S. Six year audit of cardiac arrests
and medical emergency team calls in an Australian outer metropolitan teaching
hospital. BMJ. 2007;335:1210-2. [PMID: 18048504]
68. Benning A, Dixon-Woods M, Nwulu U, Ghaleb M, Dawson J, Barber N,
et al. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ. 2011;342:d199. [PMID: 21292720]
69. Litvak E, Pronovost PJ. Rethinking rapid response teams. JAMA. 2010;304:
1375-6. [PMID: 20858881]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 425
Annals of Internal Medicine
Current Author Addresses: Drs. Winters, Weaver, Yang, and Pham:
Department of Anesthesiology and Critical Care Medicine and Armstrong Institute for Patient Safety and Quality, Johns Hopkins University School of Medicine, 750 East Pratt Street, 15th Floor, Baltimore,
MD 21231.
Ms. Pfoh and Dr. Dy: Department of Health Policy and Management,
Johns Hopkins University, Hampton House, Room 609, 624 North
Broadway, Baltimore, MD 21205.
Author Contributions: Conception and design: B.D. Winters, S.J.
Weaver, J.C. Pham, S.M. Dy.
Analysis and interpretation of the data: B.D. Winters, T. Yang, J.C.
Pham, S.M. Dy.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Drafting of the article: B.D. Winters, S.J. Weaver, E.R. Pfoh, J.C. Pham,
S.M. Dy.
Critical revision of the article for important intellectual content: B.D.
Winters, J.C. Pham, S.M. Dy.
Final approval of the article: B.D. Winters, S.J. Weaver, E.R. Pfoh, J.C.
Pham, S.M. Dy.
Provision of study materials or patients: B.D. Winters,
Statistical expertise: T. Yang, J.C. Pham.
Obtaining of funding: S.M. Dy.
Administrative, technical, or logistic support: E.R. Pfoh.
Collection and assembly of data: B.D. Winters, S.J. Weaver, E.R. Pfoh,
S.M. Dy.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) W-187
Supplement
Annals of Internal Medicine
Simulation Exercises as a Patient Safety Strategy
A Systematic Review
Eric Schmidt, BA; Sara N. Goldhaber-Fiebert, MD; Lawrence A. Ho, MD; and Kathryn M. McDonald, MM
Simulation is a versatile technique used in a variety of health care
settings for a variety of purposes, but the extent to which simulation may improve patient safety remains unknown. This systematic
review examined evidence on the effects of simulation techniques
on patient safety outcomes. PubMed and the Cochrane Library
were searched from their beginning to 31 October 2012 to identify
relevant studies. A single reviewer screened 913 abstracts and selected and abstracted data from 38 studies that reported outcomes
during care of real patients after patient-, team-, or system-level
simulation interventions. Studies varied widely in the quality of
methodological design and description of simulation activities, but
in general, simulation interventions improved the technical performance of individual clinicians and teams during critical events and
complex procedures. Limited evidence suggested improvements in
patient outcomes attributable to simulation exercises at the health
system level. Future studies would benefit from standardized reporting of simulation components and identification of robust patient safety targets.
THE PROBLEM
mance (7) and knowledge acquisition and clinical reasoning (8). Second, simulation can replicate rare, complex, or
high-stakes scenarios known to affect individual and team
performance (9, 10). Third, mistakes are not only allowed
in simulation, they enhance learning through reflection
and debriefing (11, 12). Fourth, new technologies or procedures may be tested in simulation before implementation
in real time with real patients (13). Finally, teams can simulate patient care flow in situ for critical events (14) or
adequacy of new facilities and equipment (15).
However, studies evaluating the relationship between
these benefits and patient safety outcomes, including potential harms, have not been thoroughly evaluated. The
purpose of this systematic review is to examine evidence on
the benefits and harms of using simulation to improve
patient safety in medicine.
It is well-known, both in medical practice and in other
professions, that error rates decrease with experience (1).
Yet, an important challenge is how to train physicians and
ensure that they maintain competency while minimizing
the potential for patient harm. Many medical educators
now regard the traditional medical training model “see
one, do one, teach one” as unstructured and inadequate
(2). In contrast, simulation exercises allow patient-safe clinician training. Although all physicians must eventually
perform procedures on and manage critical events for an
actual patient for the first time, simulation can make initial
interactions with patients safer.
Clinical expertise and mastery within a specialty do
not increase simply as a function of experience (3), and
likewise, patient safety issues are not likely to decrease simply as a function of more practice hours. Deliberate practice, or practice that includes reflection on performance,
increases mastery (4), and simulation exercises offer opportunities for deliberate practice with flexibility to adjust procedure complexity and provide regular practice for rare
treatments. Experienced clinicians also must maintain proficiency in a wide array of skills, most of which are known
to deteriorate over time without practice (5). Simulation
can serve to maintain clinical skills and may be part of
maintenance of board certification, as is the case for the
American Board of Anesthesiology (6).
The versatility of simulation techniques affords many
potential benefits to those working to improve patient
safety. First, simulation is designed to match user needs
and has been associated with increased technical perforSee also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
426 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Ann Intern Med. 2013;158:426-432.
For author affiliations, see end of text.
www.annals.org
PATIENT SAFETY STRATEGIES
Research demonstrating the benefits of simulation
comes from studies about simulation as well as studies using simulation (16). Research about simulation directly examines the effect of a simulation technique as an intervention on behaviors and actions at the health professional or
team level that could directly improve patient safety if that
training were widely implemented. In contrast, studies using simulation harness these techniques as a laboratory to
investigate new technologies and human performance for
insights into potential causal pathways to improve safety.
Simulation is used along the translational pathway
from health care provider actions in simulated “laboratory”
contexts to similar actions in clinical settings to patient
outcomes (17). As such, simulation is considered a technique rather than any 1 specific technology (18).
Simulation to enhance patient safety has 4 general
purposes: education (for example, in transitioning trainees
from content knowledge to experiential practice, and in
continuing education); assessment (for example, in quality
control or quality improvement, or usability testing); rewww.annals.org
Simulation Exercises as a Patient Safety Strategy
search (for example, regarding clinician behaviors) and
health system integration (for example, team processes)
(19). These purposes are not mutually exclusive, and each
may span a range of complexity. A classic low-fidelity example of partial task training is simulation of intramuscular medication administration by inserting a needle into an
orange. Individual dynamic medical management exercises
may include high-fidelity simulations that utilize anatomically accurate mannequins and vital sign monitors. Patient
safety may also be enhanced through full scenario team
management, in which a human patient simulator and a
fully simulated care environment, such as entire operating
rooms or emergency department bays, are utilized.
On a basic level, simulations improve patient safety by
allowing physicians to become better trained without putting patients at risk and, importantly, by providing protected time for reflection and debriefing—where most of
the learning takes place (11, 12). The challenge is matching the best simulation method to the desired learning
objectives while recognizing the costs of each method (18).
Because simulation is a broad technique, faculty training
and time are often a more important investment than are
specific expensive simulation equipment. Practitioners
must be appropriately trained to effectively use simulation
techniques, as well as any specific technologies, to accomplish the relevant training, assessment, or systems probing
goals.
The simulation needs to feel real enough for participants to be able to suspend disbelief, enabling them to feel,
think, and act much as they would in a real scenario (12,
19). If the learning objective is mainly to practice cognitive
skills for diagnosis or treatment, a verbal simulation, such
as, “What would you do if . . . ,” may be sufficient. In
contrast, if development of management skills, such as situational awareness or team communication, is the focus, a
more accurate replication of the actions and team presence
become important for the simulation experience.
REVIEW PROCESSES
Methodology to capture literature in this review involved 3 mechanisms. First, we used structured search
strategies to search PubMed and the Cochrane Library
from their beginning to 31 October 2012. These searches
were limited to meta-analyses; systematic reviews; and randomized, controlled trials (RCTs) or observational studies
published in English. Second, practitioners with expertise
in simulation provided recommendations on key articles,
including issues in implementation and empirical research
on simulation and patient safety. Finally, the reference lists
of articles captured by using the first 2 methods were
scanned for relevant literature. Abstracts of references captured by these searches (n ⫽ 913) were screened by a single
reviewer.
Studies, including systematic reviews and metaanalyses, were included in the review if they reported evalwww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Key Summary Points
Simulation is a versatile technique that may be applied in
patient safety strategies across a variety of factors that
contribute to patient harm.
Heterogeneous evidence across multiple topic areas shows
that training with simulation-based exercises increases
technical and procedural performance.
Heterogeneous evidence shows that simulation-based
exercises can improve team performance and interpersonal dynamics.
Limited evidence suggests that improvements in patient
outcomes attributable to simulation exercises can occur
at the health system level.
uative results of patient outcomes or changes in clinician
actions in patient care. Studies that only provided
laboratory-based results were excluded.
Data from the 38 studies that met the inclusion criteria were abstracted by a single investigator. Given the varied nature of the included studies and the broad area of
simulation, quality assessment of individual studies was
limited to reporting study design. Selected studies are described with narrative synthesis. The Supplement (available at www.annals.org) provides a complete description of
the search strategies, article flow diagram, and evidence
tables.
This review was supported by the Agency for Healthcare Research and Quality, which had no role in the selection or review of the evidence or the decision to submit the
manuscript for publication.
BENEFITS
AND
HARMS
Of the 38 included studies, 22 were RCTs, 11 were
prospective observational studies, and 5 were retrospective
analyses of previous simulation interventions. Table 1
shows the distribution of study methodology and aspects of
simulation interventions by targeted areas for improvement
in patient safety. Thirty-four studies reported patient outcomes from care provided by trainees at varied levels of
education or specialties; postgraduate residents and fellows
were highly represented. Of the 27 studies that specified a
setting for the simulation, academic medical settings predominated (n ⫽ 23).
Diagnostic Procedures
Five RCTs and 1 prospective before–after study on
training for colonoscopy and upper gastrointestinal endoscopy found better initial performance in actual patients
when physicians received simulation-based training (20 –
25). Studies generally reported a similar training period
requirement to ultimately reach desirable levels of procedure mastery (20 –25). Safety outcomes focused primarily
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 427
Supplement
Simulation Exercises as a Patient Safety Strategy
Table 1. Number of Studies, by Study Characteristics and Intervention Components
Study Category, n
Study Characteristic
Total
Studies, n
Diagnostic
Procedures
Surgical
Procedures
Central Venous
Catheterization
Other Procedures
and Processes
Team and
Systems Studies
Total studies
9
8
12
3
6
38
Study design
RCT
Prospective*
Retrospective*
5
3
1
5
2
1
6
5
1
3
0
0
3
1
2
22
11
5
Primary target of simulation†‡
Procedural
Medical management
Team training
9
0
0
8
0
1
12
0
0
2
1
0
4
5
5
35
6
6
Components of simulation‡§
Didactic component
Demonstrate technique
Deliberate practice
Practice time
Debriefing
Feedback
7
5
5
4
3
5
6
3
7
1
4
1
11
11
8
2
5
4
3
3
2
0
1
1
4
4
4
1
5
0
31
26
26
8
18
11
RCT ⫽ randomized, controlled trial.
* Includes pre–post, before–after, case– control, and other mixed methods or observational designs.
† Categories represent a set of general categories created to represent targets that could be abstracted with confidence from the literature base. Categories are not considered
exhaustive of possible applications of simulation techniques.
‡ Characteristics are not mutually exclusive in this category.
§ Components of simulation-based interventions were developed post hoc as properties common to all studies included in the review. These are not considered exhaustive
of the possible components of simulation. The Supplement (available at www.annals.org) contains more detailed information on each intervention component category. The
Discussion section of the article includes recommendations on future reporting of components.
on patient discomfort (for example, insufflation). Simulation training was associated with less discomfort in 1 study
(20), no difference in another (21), and greater patient
discomfort in a third study (24). No critical patient safety
events or major complications were reported. A systematic
review (26) that addressed virtual reality– based simulation
for endoscopy also found no studies that reported major
complications or critical patient safety events.
In a prospective randomized mixed-methods study on
simulation training for bronchoscopy, there was no observed difference in procedure time between participants
who did and did not have simulation training (27). Another study showed that training for thoracentesis was associated with fewer pneumothoraces and procedures advancing to thoracostomy when coupled with simulation
(28). Finally, cordocentesis procedure time was shorter and
success rate was higher with simulation training, although
there were no statistically significant differences in
procedure-related fetal loss or overall fetal loss (29).
Surgical Procedures
A meta-analysis of laparoscopic training with virtual
reality simulators reported that procedure time was no
faster but was more accurate among simulation-trained clinicians than traditional video-trained clinicians (standardized mean difference, 0.68 [95% CI, 0.05 to 1.31]) (30).
Simulation training for laparoscopic cholecystectomy was
associated with improved performance, 3-fold fewer errors,
428 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
an 8-fold decreased variation in error making (31), and
increased “respect for tissue” during the procedure (32–
34). Laparoscopic simulation practice improved global
scores on the Objective Structured Assessment of Technical Skills (OSATS) during cholecystectomies (35). Simulation training for extraperitoneal hernia repair was associated with increased individual OSATS item scores for
knowledge of procedure, knowledge of instruments, and
use of assistants, but this association was not significant
when these individual item scores were aggregated into a
global OSATS score (36). Cataract surgeries performed by
residents trained with simulation had a lower rate of sentinel complications than did surgeries performed by residents who were trained before simulation was implemented (37). Finally, faster procedures and improved
performance during prostate resection was observed among
physicians trained with simulation (38).
Central Venous Catheterization
A recent meta-analysis of RCTs and observational
studies (39) showed that simulation-based education in
central venous catherization techniques improved learner
outcomes and performance during actual procedures. For
example, simulation-based education resulted in fewer needle passes (standardized mean difference, ⫺0.58 [CI,
⫺0.95 to ⫺0.20]) and reduced pneumothoraces (relative
risk, 0.62 [CI, 0.40 to 0.97]) (39).
www.annals.org
Simulation Exercises as a Patient Safety Strategy
Several RCTs and observational studies that we reviewed confirmed that simulation-based education improved performance (40 –51). Two prospective studies and
1 RCT reported that simulation training decreased rates of
catheter-related bloodstream infection (41, 46, 49), but 1
prospective controlled cohort study reported no difference
in rates (48) attributable to simulation-based training.
Studies showed mixed results for other major complications and critical patient safety events.
Other Procedures and Processes
Three RCTs reported data on other procedures and
processes. In 1 RCT, a simulation-based training curriculum for pediatric residents using high-fidelity models was
associated with non–statistically significant increases in
performance of basic clinical procedural skills, such as bag–
mask ventilation, venipuncture, peripheral venous catheter
placement, and lumbar puncture (52). Bachelor’s-level
nursing students made fewer medication administration errors in external training rotations when simulation training
was added to coursework (53). Among paramedic students,
simulation-based training did not lead to improved performance during their first 15 intubations in terms of overall
success rate, success rate on first attempt, or complications
(54).
Team and Systems Performance
Researchers retrospectively investigated the effect of an
annual mandatory 1-day workshop and training program
for all midwifery (including community-based practitioners) and obstetric emergency staff in a tertiary care center
(55). The workshop used simulation exercises for 7 common obstetric emergencies: shoulder dystocia, postpartum
hemorrhage, eclampsia, delivery of twins, breech presentation, adult resuscitation, and neonatal resuscitation. Compared with the 2-year period before the training program
was implemented, there was a statistically significant decrease in the rate of births with 5-minute Apgar scores of 6
or less and hypoxic–ischemic encephalopathy in the 2-year
period after implementation. The decrease in rate of moderate to severe hypoxic–ischemic encephalopathy only approached statistical significance.
In an RCT (56), primary care physicians in a large
multidisciplinary medical group were randomly assigned to
1 of 3 groups: no simulation control, simulation alone, or
simulation combined with a physician leader program. The
simulation training provided a series of interactive virtual
encounters with patients who had newly diagnosed diabetes or who had indicated or contraindicated adjustments to
their insulin regimen. When combined for comparison
with the control group, physicians in the simulation groups
prescribed renal-contraindicated metformin significantly
less often to patients with diabetes. In another RCT (57),
residents who participated in full-scenario simulation training for elective coronary artery bypass graft surgery had
increased Anesthesiologists’ Nontechnical Skills Assesswww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
ment scores, and this difference was observed at 5-week
follow-up.
Three studies reported patient outcomes after
simulation-based training for resuscitation teams (14, 58,
59). An RCT (58) reported no differences attributable to
simulation training for actual team performance on rates of
ventilation, return of spontaneous circulation, or survival
to discharge. However, a prospective before–after study examined resuscitation outcomes after implementation of the
TeamSTEPPS team-building program coupled with simulation (59). This study reported several improvements in
communication, as well as reductions in time to computed
tomography, intubation, and the operating room. Finally,
in a retrospective case– control study (14), simulation
training was associated with a higher correct response rate
based on the American Heart Association standards for
resuscitation.
Harms
Studies generally provided additive or supplemental
interventions to training as usual, and no study reported
data indicating increased potential for or actual harm to
patients that resulted from implementing simulation techniques. However, it is conceivable that simulation exercises
would place demand on valuable resources that could be
applied elsewhere in patient safety efforts. We found no
evaluations of such considerations.
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
The Context for Simulation
A meta-analysis (8) of simulation in education programs for health professionals found that 564 of 609 studies (92.6%) examined techniques provided through dedicated simulation centers. Thirty-four studies (5.6%)
examined simulation in situ, and 11 studies (1.8%) reported from both contexts. Among studies cited in our
review, academic medical systems and academically affiliated hospitals predominated (21–23, 27–29, 35, 36, 38,
41– 46, 49 –54, 58, 60, 61). However, studies also reported
outcomes of use of simulation in tertiary care facilities (25,
45, 50, 58), trauma centers (44, 59), and multispecialty
medical groups (55, 56). We found no reported data on
the effect of context on the effectiveness of simulation exercises for improving patient safety.
Implementing Simulation
Gaba (18) conceptualized a framework for simulation
techniques that may aid implementation and ultimately
enhance patient safety (Table 2). The framework includes
11 dimensions that form a comprehensive set of considerations proposed to enhance the development and the effectiveness of simulation exercises. Application of each dimension guides specification and decision making on critical
choices about the simulation exercise. In practice, objectives of implementing simulation are aligned with the
needs of learners and the goals of trainers from level of
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 429
Supplement
Simulation Exercises as a Patient Safety Strategy
Table 2. Eleven Dimensions to Consider When Designing
and Setting Up Simulation Exercises*
Purpose and Participants
Setup
Purpose and aims of simulation
activity
Type of knowledge, skills, attitudes,
or behaviors addressed in
simulation
Unit of participation in the simulation:
individuals or teams
Experience level of simulation
participants
Health care domain in which the
simulation is applied
Health care disciplines of personnel
participating in the simulation
Age of the patient being
simulated
Technology applicable or
required for simulations
Extent of direct participation
Site of simulation: in situ
clinical setting vs.
dedicated simulation center
Feedback method
accompanying simulation
* Adapted from reference 18.
participation to training in the particular simulation technique. In addition, sufficient time for creating meaningful
exercises with debriefing, equipment matched to the simulation need that recreates sufficient realism, and adequate
space or storage for in situ simulations will increase the
likelihood of success (18). Resources used in simulations
must be available when needed and kept safe from being
used inappropriately for patient care (for example, expired
medications). Technical support and maintenance may be
required for complex or high-tech simulators.
Rosen and colleagues (19) highlighted the importance
of cognitive fidelity (vs. physical fidelity) in a simulated
exercise: Simulations that engage the participant in ways
that cognitively best reflect the actual task are likely to be
more effective. Debriefing is considered crucial when implementing simulation requires instructor training (11, 12)
and is considered a best practice in simulation-based medical education (62).
Costs
The cost of implementing simulation exercises ranges
from low to high, depending on the type of exercise and
personnel and equipment resources involved (18). Instructor and learner time are likely to be the most expensive and
crucial aspects of simulation in the long run. Start-up costs
for a comprehensive simulation center may be accounted
for differently from ongoing costs for exercises, which
complicates the ability to categorize the expected cost for
simulation as a patient safety strategy. Unfortunately, research addressing cost savings attributable directly to simulation remains sparse, although some studies have reported up to a 7-to-1 return on simulation costs through
reduction in hospital days for bloodstream infections (21,
49).
DISCUSSION
Simulation has continued to gain momentum in patient safety efforts in the past decade because it allows for
exercising and improving aspects of health care delivery
430 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
without any known risks to patients. Simulation has been
used in patient safety for the purposes of education, assessment, research, and integration of system-level strategies.
These efforts have been reported in the literature as research about simulation: that is, research evaluating the
translation of simulation-based education to enhanced patient safety. In contrast, other research has focused on using simulation as a laboratory to discover potential leverage
points for patient safety (16).
Our review found that studies reporting patient outcomes or systems of care have been done primarily in academic settings, although researchers have used simulation
in diverse clinical specialties, experience levels, and care
settings. These studies varied in terms of individual quality,
but the majority were randomized or had methodologically
sound controlled prospective designs. Researchers have
replicated standardized simulation training for central venous catherization, and although this approach is promising for patient safety in that area, we did not find other
examples of replication studies in our review. We also did
not find analysis of contextual effects on the validity of
simulation to improve patient safety. The generalizability
of any one technique is likely to vary according to many
factors, such as those in Gaba’s 11-dimensional framework
(18), and the adequacy of resources dedicated to simulation (for example, debriefing).
At this juncture, simulation seems to have a favorable
effect on quicker acquisition and improved performance of
technical skills. Although not yet thoroughly studied, simulation of complex or high-stakes procedures seems to be a
promising technique to increase patient-safe behavior at
the clinician and team levels. Simulation has the potential
to enhance patient safety through structured assessment
and debriefing in quality improvement initiatives. It has
been used to assess practices that would be difficult or
unsafe to study empirically in real time with actual patients. Likewise, simulation has been endorsed for ongoing
competency and continuing education, as well as advancement to mastery-level practices.
A previous systematic review (7) reported that simulation contributes to enhanced knowledge acquisition and
improved clinical performance. Simulation techniques
have been used in translating results from the withinsimulation laboratory to patient- and health care system–
level outcomes (17). Another systematic review (4) suggested that protected time for debriefing in a learning
experience is a crucial component of simulation techniques. To our knowledge, our review is the first to examine the effects that simulation exercises have on patient
safety outcomes, and in particular outcomes in patients
outside of simulation laboratory settings (that is, during
clinical care).
Our review has limitations. First, it is possible that the
broad search strategies missed studies that may be captured
with targeted and comprehensive strategies dedicated to
each simulation technique, clinical specialty, or applicawww.annals.org
Simulation Exercises as a Patient Safety Strategy
tion. Second, given the relative infancy of the research on
simulation exercises, the field may be prone to selective
reporting of studies with positive findings, leading to potential publication bias. Finally, we limited our assessment
of quality of evidence to study design and did not perform
a structured assessment of the strength of evidence. Therefore, the overall strength of the evidence for simulation
exercises to improve patient safety should be interpreted
with caution.
In conclusion, simulation is a versatile technique that
continues to gain momentum in a variety of clinical settings and applications, including patient safety strategies.
Although evidence is largely heterogeneous at this time,
our review suggests the potential for simulation exercises to
contribute to patient safety through increased technical
and procedural performance and improved team performance. Limited research using health system–level observations suggests that simulation may enhance patient
safety, although more research is needed on the potential
for simulation to contribute to system-level differences in
patient safety outcomes. Systematic reviews of simulation
for specific procedures have begun reporting patient safety
outcomes (26, 30); more reviews of this nature would enhance our understanding of the overall contribution of
simulation techniques to patient safety. Future systematic
reviews would benefit from investigators using a consistent
framework, such as that developed by Gaba (18), to describe the intervention and its context and implementation.
From Stanford Center for Health Policy/Center for Primary Care and
Outcomes Research and Stanford University School of Medicine, Stanford University Hospital and Clinics, Stanford, California.
Note: The Agency for Healthcare Research and Quality reviewed
contract deliverables to ensure adherence to contract requirements and
quality, and a copyright release was obtained from the Agency for
Healthcare Research and Quality before the manuscript was submitted
for publication.
Disclaimer: All statements expressed in this work are those of the authors
and should not in any way be construed as official opinions or positions
of Stanford University, the Agency for Healthcare and Quality, or the
U.S. Department of Health and Human Services.
Financial Support: From the Agency for Healthcare and Quality, U.S.
Department of Health and Human Services (contract HHSA-290-200710062I).
Potential Conflicts of Interest: Mr. Schmidt: Grant (money to institu-
tion): Agency for Healthcare Research and Quality. Ms. McDonald:
Grant (money to institution): Agency for Healthcare Research and Quality. All other authors have no disclosures. Disclosures can also be viewed at
www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum
⫽M12-2572.
Requests for Single Reprints: Kathryn M. McDonald, MM, Stanford
University, 117 Encina Commons, Stanford, CA 94305-6019; e-mail,
[email protected].
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
Current author addresses and author contributions are available at www
.annals.org.
References
1. Meltzer D, Manning WG, Morrison J, Shah MN, Jin L, Guth T, et al.
Effects of physician experience on costs and outcomes on an academic general
medicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137:86674. [PMID: 12458986]
2. Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an
ethical imperative. Acad Med. 2003;78:783-8. [PMID: 19088599]
3. Ericsson KA. Deliberate practice and acquisition of expert performance: a
general overview. Acad Emerg Med. 2008;15:988-94. [PMID: 18778378]
4. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. Effect of practice on
standardised learning outcomes in simulation-based medical education. Med
Educ. 2006;40:792-7. [PMID: 16869926]
5. Semeraro F, Signore L, Cerchiari EL. Retention of CPR performance in
anaesthetists. Resuscitation. 2006;68:101-8. [PMID: 16325986]
6. Park CS. Simulation and quality improvement in anesthesiology. Anesthesiol
Clin. 2011;29:13-28. [PMID: 21295750]
7. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does
simulation-based medical education with deliberate practice yield better results
than traditional clinical education? A meta-analytic comparative review of the
evidence. Acad Med. 2011;86:706-11. [PMID: 21512370]
8. Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health
professions education: a systematic review and meta-analysis. Acad Med. 2010;
85:1589-602. [PMID: 20703150]
9. Blum RH, Raemer DB, Carroll JS, Dufresne RL, Cooper JB. A method for
measuring the effectiveness of simulation-based team training for improving communication skills. Anesth Analg. 2005;100:1375-80. [PMID: 15845689]
10. Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb Y. Simulation-based
training in anesthesia crisis resource management (ACRM): a decade of experience. Simul Gaming. 2001;32:175-93.
11. Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Simul Healthc. 2007;2:115-25. [PMID: 19088616]
12. Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing
with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin. 2007;25:361-76. [PMID: 17574196]
13. Lipman S, Daniels K, Cohen SE, Carvalho B. Labor room setting compared
with the operating room for simulated perimortem cesarean delivery: a randomized controlled trial. Obstet Gynecol. 2011;118:1090-4. [PMID: 22015877]
14. Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghie
WC. Simulation-based education improves quality of care during cardiac arrest
team responses at an academic teaching hospital: a case-control study. Chest.
2008;133:56-61. [PMID: 17573509]
15. Lighthall GK, Poon T, Harrison TK. Using in situ simulation to improve
in-hospital cardiopulmonary resuscitation. Jt Comm J Qual Patient Saf. 2010;36:
209-16. [PMID: 20480753]
16. Gaba DM. The future’s here. We are it. Simul Healthc. 2006;1:1-2. [PMID:
19088617]
17. McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis D. Evaluating the impact of simulation on translational patient outcomes. Simul Healthc.
2011;6 Suppl:S42-7.
18. Gaba DM. The future vision of simulation in healthcare. Simul Healthc.
2007;2:126-35.
19. Rosen MA, Salas E, Tannenbaum SI, Provonost P, King HB. Simulationbased training for teams in health care: designing scenarios, measuring performance, and providing feedback. In: Carayon P, ed. Handbook of HFE in Health
Care and Patient Safety. 2nd ed. Boca Raton, FL: CRC Press; 2011:571-92.
20. Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D. Virtual
reality colonoscopy simulation: a compulsory practice for the future colonoscopist? Endoscopy. 2005;37:1198-204. [PMID: 16329017]
21. Cohen J, Cohen SA, Vora KC, Xue X, Burdick JS, Bank S, et al. Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc. 2006;64:361-8.
[PMID: 16923483]
22. Ferlitsch A, Schoefl R, Puespoek A, Miehsler W, Schoeniger-Hekele M,
Hofer H, et al. Effect of virtual endoscopy simulator training on performance of
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 431
Supplement
Simulation Exercises as a Patient Safety Strategy
upper gastrointestinal endoscopy in patients: a randomized controlled trial.
Endoscopy. 2010;42:1049-56. [PMID: 20972956]
23. Park J, MacRae H, Musselman LJ, Rossos P, Hamstra SJ, Wolman S, et al.
Randomized controlled trial of virtual reality simulator training: transfer to live
patients. Am J Surg. 2007;194:205-11. [PMID: 17618805]
24. Sedlack RE, Kolars JC. Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study.
Am J Gastroenterol. 2004;99:33-7. [PMID: 14687137]
25. Thomson M, Heuschkel R, Donaldson N, Murch S, Hinds R. Acquisition
of competence in paediatric ileocolonoscopy with virtual endoscopy training.
J Pediatr Gastroenterol Nutr. 2006;43:699-701. [PMID: 17130753]
26. Walsh CM, Sherlock ME, Ling SC, Carnahan H. Virtual reality simulation
training for health professions trainees in gastrointestinal endoscopy. Cochrane
Database Syst Rev. 2012;6:CD008237. [PMID: 22696375]
27. Blum MG, Powers TW, Sundaresan S. Bronchoscopy simulator effectively
prepares junior residents to competently perform basic clinical bronchoscopy.
Ann Thorac Surg. 2004;78:287-91. [PMID: 15223446]
28. Duncan DR, Morgenthaler TI, Ryu JH, Daniels CE. Reducing iatrogenic
risk in thoracentesis: establishing best practice via experiential training in a zerorisk environment. Chest. 2009;135:1315-20. [PMID: 19017865]
29. Tongprasert F, Wanapirak C, Sirichotiyakul S, Piyamongkol W, Tongsong
T. Training in cordocentesis: the first 50 case experience with and without a
cordocentesis training model. Prenat Diagn. 2010;30:467-70. [PMID:
20440735]
30. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual reality
training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev.
2009:CD006575. [PMID: 19160288]
31. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C,
McClusky DA 3rd, et al. Proficiency-based virtual reality training significantly
reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007;193:797-804. [PMID: 17512301]
32. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J,
Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004;91:146-50. [PMID: 14760660]
33. Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, et al.
Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg. 2000;191:272-83. [PMID: 10989902]
34. Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM.
Fundamentals of laparoscopic surgery simulator training to proficiency improves
laparoscopic performance in the operating room—a randomized controlled trial.
Am J Surg. 2010;199:115-20. [PMID: 20103076]
35. Calatayud D, Arora S, Aggarwal R, Kruglikova I, Schulze S, Funch-Jensen
P, et al. Warm-up in a virtual reality environment improves performance in the
operating room. Ann Surg. 2010;251:1181-5. [PMID: 20485133]
36. Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV,
et al. Improving operative performance using a laparoscopic hernia simulator.
Am J Surg. 2001;182:725-8. [PMID: 11839347]
37. Rogers GM, Oetting TA, Lee AG, Grignon C, Greenlee E, Johnson AT,
et al. Impact of a structured surgical curriculum on ophthalmic resident cataract
surgery complication rates. J Cataract Refract Surg. 2009;35:1956-60. [PMID:
19878829]
38. Källström R, Hjertberg H, Svanvik J. Impact of virtual reality-simulated
training on urology residents’ performance of transurethral resection of the prostate. J Endourol. 2010;24:1521-8. [PMID: 20677993]
39. Ma IW, Brindle ME, Ronksley PE, Lorenzetti DL, Sauve RS, Ghali WA.
Use of simulation-based education to improve outcomes of central venous catheterization: a systematic review and meta-analysis. Acad Med. 2011;86:1137-47.
[PMID: 21785310]
40. Andreatta P, Chen Y, Marsh M, Cho K. Simulation-based training improves
applied clinical placement of ultrasound-guided PICCs. Support Care Cancer.
2011;19:539-43. [PMID: 20306091]
41. Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of
simulation-based education to reduce catheter-related bloodstream infections.
Arch Intern Med. 2009;169:1420-3. [PMID: 19667306]
42. Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use
of simulation-based mastery learning to improve the quality of central venous
catheter placement in a medical intensive care unit. J Hosp Med. 2009;4:397403. [PMID: 19753568]
432 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
43. Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB.
Simulation-based mastery learning reduces complications during central venous
catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37:2697701. [PMID: 19885989]
44. Britt RC, Novosel TJ, Britt LD, Sullivan M. The impact of central line
simulation before the ICU experience. Am J Surg. 2009;197:533-6. [PMID:
19249739]
45. Evans LV, Dodge KL, Shah TD, Kaplan LJ, Siegel MD, Moore CL, et al.
Simulation training in central venous catheter insertion: improved performance
in clinical practice. Acad Med. 2010;85:1462-9. [PMID: 20736674]
46. Khouli H, Jahnes K, Shapiro J, Rose K, Mathew J, Gohil A, et al. Performance of medical residents in sterile techniques during central vein catheterization: randomized trial of efficacy of simulation-based training. Chest. 2011;139:
80-7. [PMID: 20705795]
47. Martin M, Scalabrini B, Rioux A, Xhignesse MA. Training fourth-year
medical students in critical invasive skills improves subsequent patient safety. Am
Surg. 2003;69:437-40. [PMID: 12769219]
48. Miranda JA, Trick WE, Evans AT, Charles-Damte M, Reilly BM, Clarke
P. Firm-based trial to improve central venous catheter insertion practices. J Hosp
Med. 2007;2:135-42. [PMID: 17549773]
49. Sherertz RJ, Ely EW, Westbrook DM, Gledhill KS, Streed SA, Kiger B,
et al. Education of physicians-in-training can decrease the risk for vascular catheter infection. Ann Intern Med. 2000;132:641-8. [PMID: 10766683]
50. Smith CC, Huang GC, Newman LR, Clardy PF, Feller-Kopman D, Cho
M, et al. Simulation training and its effect on long-term resident performance in
central venous catheterization. Simul Healthc. 2010;5:146-51. [PMID:
20651476]
51. Velmahos GC, Toutouzas KG, Sillin LF, Chan L, Clark RE, Theodorou D,
et al. Cognitive task analysis for teaching technical skills in an inanimate surgical
skills laboratory. Am J Surg. 2004;187:114-9. [PMID: 14706600]
52. Gaies MG, Morris SA, Hafler JP, Graham DA, Capraro AJ, Zhou J, et al.
Reforming procedural skills training for pediatric residents: a randomized, interventional trial. Pediatrics. 2009;124:610-9. [PMID: 19651582]
53. Sears K, Goldsworthy S, Goodman WM. The relationship between simulation in nursing education and medication safety. J Nurs Educ. 2010;49:52-5.
[PMID: 19810664]
54. Hall RE, Plant JR, Bands CJ, Wall AR, Kang J, Hall CA. Human patient
simulation is effective for teaching paramedic students endotracheal intubation.
Acad Emerg Med. 2005;12:850-5. [PMID: 16141019]
55. Draycott T, Sibanda T, Owen L, Akande V, Winter C, Reading S, et al.
Does training in obstetric emergencies improve neonatal outcome? BJOG. 2006;
113:177-82. [PMID: 16907952]
56. O’Connor PJ, Sperl-Hillen JM, Johnson PE, Rush WA, Asche SE, Dutta
P, et al. Simulated physician learning intervention to improve safety and quality
of diabetes care: a randomized trial. Diabetes Care. 2009;32:585-90. [PMID:
19171723]
57. Bruppacher HR, Alam SK, LeBlanc VR, Latter D, Naik VN, Savoldelli GL,
et al. Simulation-based training improves physicians’ performance in patient care
in high-stakes clinical setting of cardiac surgery. Anesthesiology. 2010;112:98592. [PMID: 20234305]
58. Weidman EK, Bell G, Walsh D, Small S, Edelson DP. Assessing the impact
of immersive simulation on clinical performance during actual in-hospital cardiac
arrest with CPR-sensing technology: a randomized feasibility study. Resuscitation. 2010;81:1556-61. [PMID: 20724057]
59. Capella J, Smith S, Philp A, Putnam T, Gilbert C, Fry W, et al. Teamwork
training improves the clinical care of trauma patients. J Surg Educ. 2010;67:43943. [PMID: 21156305]
60. Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Mastery
learning of temporary hemodialysis catheter insertion by nephrology fellows using
simulation technology and deliberate practice. Am J Kidney Dis. 2009;54:70-6.
[PMID: 19376620]
61. Griswold-Theodorson S, Hannan H, Handly N, Pugh B, Fojtik J, Saks M,
et al. Improving patient safety with ultrasonography guidance during internal
jugular central venous catheter placement by novice practitioners. Simul Healthc.
2009;4:212-6. [PMID: 21330794]
62. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of
simulation-based medical education research: 2003-2009. Med Educ. 2010;44:
50-63. [PMID: 20078756]
www.annals.org
Annals of Internal Medicine
Current Author Addresses: Mr. Schmidt and Ms. McDonald: Stanford
Center for Health Policy/Center for Primary Care and Outcomes Research, Stanford University, 117 Encina Commons, Stanford, CA
94305-6019.
Drs. Goldhaber-Fiebert and Ho: Stanford University School of Medicine, Stanford University Hospital and Clinics, 291 Campus Drive,
Stanford, CA 94305.
Author Contributions: Conception and design: E. Schmidt, S.N.
Goldhaber-Fiebert, K.M. McDonald.
W-188 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Analysis and interpretation of the data: E. Schmidt, S.N. GoldhaberFiebert, L.A. Ho, K.M. McDonald.
Drafting of the article: E. Schmidt, S.N. Goldhaber-Fiebert, L.A. Ho,
K.M. McDonald.
Critical revision of the article for important intellectual content: E.
Schmidt, S.N. Goldhaber-Fiebert, L.A. Ho, K.M. McDonald.
Final approval of the article: S.N. Goldhaber-Fiebert, K.M. McDonald.
Obtaining of funding: K.M. McDonald.
Administrative, technical, or logistic support: E. Schmidt, K.M. McDonald.
Collection and assembly of data: E. Schmidt, S.N. Goldhaber-Fiebert.
www.annals.org
Supplement
Annals of Internal Medicine
Hospital-Initiated Transitional Care Interventions as a Patient
Safety Strategy
A Systematic Review
Stephanie Rennke, MD; Oanh K. Nguyen, MD; Marwa H. Shoeb, MD; Yimdriuska Magan, BS; Robert M. Wachter, MD;
and Sumant R. Ranji, MD
Hospitals now have the responsibility to implement strategies to
prevent adverse outcomes after discharge. This systematic review
addressed the effectiveness of hospital-initiated care transition strategies aimed at preventing clinical adverse events (AEs), emergency
department (ED) visits, and readmissions after discharge in general
medical patients. MEDLINE, CINAHL, EMBASE, and Cochrane Database of Clinical Trials (January 1990 to September 2012) were
searched, and 47 controlled studies of fair methodological quality
were identified. Forty-six studies reported readmission rates, 26
reported ED visit rates, and 9 reported AE rates. A “bridging”
strategy (incorporating both predischarge and postdischarge interventions) with a dedicated transition provider reduced readmission
or ED visit rates in 10 studies, but the overall strength of evidence
for this strategy was low. Because of scant evidence, no conclusions
could be reached on methods to prevent postdischarge AEs. Most
studies did not report intervention context, implementation, or cost.
The strategies hospitals should implement to improve patient safety
at hospital discharge remain unclear.
THE PROBLEM
ever, little evidence supports their effect on readmissions or
other important markers of postdischarge patient safety,
such as emergency department (ED) visits and AEs occurring shortly after discharge. Moreover, a recent review (15)
identified no interventions proven to reduce 30-day readmission rates in general patient populations, although it
did not focus on hospital-initiated interventions. Because
financial penalties place the onus on hospitals to be primarily responsible for implementation of strategies to prevent
adverse outcomes after discharge, we conducted a systematic review of the effectiveness of hospital-initiated care
transition interventions on reducing AEs, ED visits, and
readmissions after discharge in general medical patients.
Nearly 1 in 5 Medicare patients is readmitted within
30 days of discharge from the hospital (1). This proportion
has not changed substantially over the past several years (2)
despite intense efforts to improve the discharge process.
Patients are vulnerable to a wide range of adverse events
(AEs) after discharge, with more than 20% of medical patients sustaining a preventable AE within 3 weeks of discharge (3). Multiple issues contribute to ineffective care
transitions, including poor communication between inpatient and outpatient clinicians (4); medication changes
during hospitalizations (5); inadequate patient understanding of diagnoses, medications, and follow-up needs (6);
discharging patients with incomplete diagnostic work-ups
(7); and other, more general patient-related and health care
system–related factors (8 –10).
Several policy initiatives have recently been implemented to encourage improvements in transitional care.
The Centers for Medicare & Medicaid Services publicly
reports hospitals’ risk-adjusted 30-day readmission rates for
patients hospitalized with pneumonia, acute myocardial infarction, or congestive heart failure (11). The Centers recently announced that more than 2000 hospitals will suffer
financial penalties of up to 1% of Medicare reimbursements because of high readmission rates (12). The Partnership for Patients initiative aims to decrease preventable readmissions by 20% by the end of 2013 and has identified
improving transitional care as an opportunity to reduce
health care expenditures (13). Together, these policies constitute a mandate to hospitals to improve transitional care
at hospital discharge.
Little information is available on effective transitional
care strategies for general medical inpatients. Prominent
national organizations have recommended a range of interventions (14), which are being implemented widely. Howwww.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Ann Intern Med. 2013;158:433-440.
For author affiliations, see end of text.
www.annals.org
PATIENT SAFETY STRATEGIES
We defined a “transitional care strategy” as 1 or a
group of interventions initiated before hospital discharge
with the aim of ensuring the safe and effective transition of
patients from the acute inpatient setting to home. To synthesize a variety of published interventions, we classified
specific interventions on the basis of an existing taxonomy
of transitional care interventions (16 –21). We grouped
transitional care strategies into 3 categories according to
the timing and setting of intervention components: predischarge, postdischarge, and “bridging” (including both preand postdischarge components) (Table 1) (15).
We defined postdischarge AEs as any of the following
patient experiences—all representing clinically meaningful
See also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 433
Supplement
Hospital-Initiated Transitional Care Interventions
Key Summary Points
Hospitals are charged with implementing transitional care
strategies—interventions initiated before hospital discharge
to facilitate the safe transition of patients across health
care settings—to prevent adverse events, emergency
department visits, and readmissions after discharge.
Hospital-based or bridging (including in-hospital and postdischarge components) strategies to prevent adverse clinical outcomes after discharge can involve patient engagement, use of a dedicated transition provider, medication
reconciliation, and facilitation of communication with outpatient providers.
Low-strength evidence shows that use of a bridging intervention incorporating a dedicated transition provider, who
contacted patients before and after discharge, reduced
emergency department visits and readmission rates in
10 fair-quality studies.
Evidence on the effectiveness of strategies to prevent
postdischarge adverse events is scant and inconclusive.
Few studies provide information on contextual factors,
cost, or implementation of transitional care strategies.
Although hospitals may be penalized for excessive readmission rates, strategies to improve the quality of care
transitions at hospital discharge for general medical
patients remain undefined.
injuries from medical care— occurring after hospital discharge: new or worsening symptoms, laboratory abnormalities (such as elevated international normalized ratio) necessitating a change in clinical management, and injuries
(such as adverse drug events, falls, or hospital-acquired infections) attributable at least in part to hospital care. This
definition was based on classifications (3, 22) used in previous studies that analyzed the epidemiology of postdischarge AEs.
REVIEW PROCESSES
As part of this supplement on patient safety, our purpose was to evaluate the effect of transitional care strategies
initiated in the hospital on adverse outcomes after discharge compared with usual discharge care. We searched
MEDLINE, CINAHL, EMBASE, and the Cochrane Database of Controlled Trials from January 1990 through
September 2012 using a search strategy developed with the
assistance of a medical librarian. We included Englishlanguage, randomized, controlled trials (RCTs) and nonrandomized, controlled clinical trials that evaluated the effect of a transitional care strategy initiated before hospital
discharge on postdischarge AE rates, ED use, or readmission rates after discharge home. To be included, studies
434 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
must have enrolled an undifferentiated population of adult
general medical patients. We excluded studies conducted
in disease-specific populations, studies of other formal care
programs (such as disease management programs) that
were not initiated in the hospital or did not explicitly target
care transitions, and studies focusing on transition from
hospitalization to another acute or subacute care setting.
We included studies that reported intervention costs only
if one of the main outcomes was also reported.
Study investigators screened 20 248 titles identified by
the search strategy for relevance and rereviewed a sample of
excluded titles for accuracy. Two investigators independently reviewed the full text of potentially relevant studies
(n ⫽ 762) to determine study eligibility. Two investigators
independently reviewed the 47 studies that met inclusion
criteria. They extracted data on the following domains:
study design, methodological quality, study setting, participants (type of health system, target population), details of
the intervention components, and outcomes. Disagreements on specific fields were resolved by consensus and
discussion with a third investigator if necessary. Reviewers
rated the quality of individual studies using the Cochrane
Collaboration Effective Practice and Organisation of Care
checklist; they also rated the overall strength of evidence
supporting specific strategies according to the method used
for the Agency for Healthcare Research and Quality evidence report for which this project was performed (23).
The main outcomes extracted were AE rates and ED and
readmission rates within 30 days after hospital discharge.
Additional outcomes included readmissions, ED visits, and
AE rates up to 1 year after discharge. Given the heterogeneity of interventions, study settings, and patient populations, we chose not to perform a meta-analysis. See the
Supplement (available at www.annals.org) for a complete
description of the search strategies; the detailed article flow
diagram; and evidence tables, including quality ratings.
This review was supported by the Agency for Healthcare Research and Quality, which had no role in the selection or review of the evidence or the decision to submit
this manuscript for publication.
BENEFITS
AND
HARMS
Of 47 eligible studies, 28 were RCTs (24 –51) and 19
were controlled clinical trials (52–70). Most were rated as
having fair methodological quality (see Table 3 of the
Supplement).
Benefits
Patient Populations, Risk Factors, and Settings
About half of the studies (n ⫽ 24) were conducted
within the United States. The majority (n ⫽ 27) targeted
older adult populations, although definitions of “elderly”
varied widely (enrolling patients older than age 55 years in
1 case [25]). Twelve studies targeted individuals at “high
risk” for readmissions or AEs, although definitions of “high
risk” were inconsistent across studies. Seven studies tarwww.annals.org
Hospital-Initiated Transitional Care Interventions
Table 1. Taxonomy of Interventions to Improve Transitional
Care at Hospital Discharge
Predischarge interventions
Assessment of risk for adverse events or readmissions
Patient engagement (e.g., patient or caregiver education)
Creation of an individualized patient record (customized document in lay
language containing clinical and educational information for patients’
use after discharge)
Facilitation of communication with outpatient providers
Multidisciplinary discharge planning team
Dedicated transition provider (who has in-person or telephone contact
with patient before and after discharge)
Medication reconciliation
Postdischarge interventions
Outreach to patients (including follow-up telephone calls,
patient-activated hotlines, and home visits)
Facilitation of clinical follow-up (including facilitated ambulatory provider
follow-up)
Medication reconciliation after discharge
Bridging interventions
Inclusion of at least 1 predischarge component and at least 1
postdischarge component
geted individuals according to medication-related indications, including polypharmacy or receipt of a “high-risk”
medication; again, these definitions varied across studies.
The most common exclusion criteria used in individual
studies were the presence of cognitive impairment or dementia (n ⫽ 15) and lack of fluency in the dominant language of the country in which the intervention took place
(n ⫽ 17). The exclusion of these individuals may limit the
generalizability of study findings to specific groups generally considered to be at lower risk for readmission and AEs
and may have biased the study toward null results in some
cases.
Characteristics of Transitional Care Strategies
Studies used a median of 4 separate interventions
(range, 1 to 8) (Table 2 of the Supplement). Thirty studies
(21 RCTs) used a bridging strategy with both pre- and
Supplement
postdischarge intervention components, and 17 studies (7
RCTs) included only hospital-based, predischarge interventions. The strategies included a variety of separate interventions. The most commonly used interventions included patient engagement (n ⫽ 37), ranging from general
patient education to more specific instruction on symptom
management and medication counseling. Twenty-eight
studies included postdischarge outreach to patients by telephone (n ⫽ 10), home visit (n ⫽ 8), or both telephone
contact and at least 1 home visit (n ⫽ 10). Of the 30
studies that included a bridging intervention, 20 included
a designated transition provider who had contact with the
patient in the hospital and in the outpatient setting after
discharge (Table 2).
Effect of Transitional Care Strategies on Postdischarge AEs
Nine studies reported AE rates after discharge (29 –32,
38, 40, 44, 59, 70) (Table 4 of the Supplement). Of these,
3 reported statistically significant reductions in postdischarge AE rates (31, 44, 70). Gillespie and colleagues
(31) found that a pharmacist-led intervention reduced
medication-related readmissions within 12 months of hospital discharge. The intervention targeted elderly patients
and involved inpatient monitoring, counseling, discharge
teaching and medication reconciliation, and postdischarge
telephone follow-up. Schnipper and colleagues (44) reported that a similarly comprehensive pharmacist-led intervention reduced preventable drug AEs and reduced a
composite outcome of medication-related ED visits and
hospital readmissions within 30 days of hospital discharge.
Another pharmacist-led study (70) that included discharge
medication counseling without postdischarge follow-up reduced adverse drug events in a Saudi Arabian population.
Two additional studies (30, 59) reported reductions in
postdischarge AEs with pharmacist-led medication safety
interventions; findings were not statistically significant, but
both studies were underpowered to detect important differences between intervention and control groups.
Table 2. Summary Strength of Evidence and Findings
Intervention and Strategies
Total
Studies, n
Mean EPOC
Score
Studies Reporting ED
Visit or Readmission
Rate (at Any Time
Point), n
Hospital-only
17
3.53
16
6
Bridging strategy
Dedicated transition provider
30
20
4.83
4.95
30
20
12
10
10
4.6
10
2
No dedicated transition provider
Statistically Significant
Reduction in
Readmissions or ED
Visits
Findings
Wide variation in types of interventions and
providers involved
Most transition providers were nurses;
postdischarge patient contact was via
telephone call or home visit; probably
resource-intensive, but little information provided on cost or ease of
implementation
Wide variation in types of interventions
and providers involved
ED ⫽ emergency department; EPOC ⫽ Effective Practice and Organisation of Care.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 435
Supplement
Hospital-Initiated Transitional Care Interventions
Effect of Transitional Care Strategies on 30-Day Readmission
and ED Visit Rates
Forty-six studies reported readmission rates at intervals
ranging from 15 days to 1 year after the index hospital
discharge; 22 of these studies (12 RCTs) reported readmission rates or ED visit rates 30 days or less after discharge
(Table 5 of the Supplement). Eight studies (4 RCTs) reported statistically significant reductions in 30-day readmission rates, ED visits, or a composite of the 2 outcomes.
Six of the 8 studies used a bridging intervention that included a dedicated provider who had primary responsibility for ensuring safe transitions (26, 27, 33, 34, 55, 67).
Transition providers met with patients before discharge to
provide patient education and conducted posthospital outreach to patients via telephone or home visits. Transition
providers also created individualized, patient-centered
health records and communicated information about the
hospitalization to the patient’s primary care provider.
Three studies that evaluated the Care Transitions Intervention (CTI)—an intervention with a “transition coach” who
performed postdischarge home visits that emphasized patient education and self-management—reported reductions
in 30-day readmissions (26, 55, 67) when conducted in
managed care systems, capitated delivery systems, and
Medicare fee-for-service populations. Another similar intervention, Project RED, reduced 30-day ED visits at an
urban safety net hospital (33). A nurse discharge advocate
was responsible for patient education and communication
of clinical information to the patient’s primary care provider, and a clinical pharmacist reviewed the discharge plan
and medication management by telephone with the patient
after discharge.
Fourteen studies (8 RCTs) reported no statistically significant reductions in 30-day readmission or ED visit rates.
These studies were broadly similar to the successful studies
in terms of sample size and methodologic quality. Four
used a bridging intervention with a dedicated transition
provider. One, which evaluated the CTI in a Medicare
fee-for-service population, reported a reduction in readmissions at 90 days after discharge (43).
ED Visits and Readmission Rates Beyond 30 Days
After Discharge
Twenty-six studies reported ED visit rates, readmission rates, or a composite of the 2 outcomes at intervals
ranging from 45 days to 1 year after the index discharge.
Seven studies reported statistically significant reductions in
readmission rates, including 4 studies (39, 40, 43, 47) that
used a bridging intervention with a dedicated transition
provider.
Harms
None of the studies reported any harms associated
with transitional care interventions.
436 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
IMPLEMENTATION CONSIDERATIONS
AND
COSTS
Although a majority of studies (n ⫽ 26) reported a
detailed timeline of the implementation of each component of the transitional care strategy, fewer than one third
explicitly described the resources needed to implement the
strategy or the training protocols used in the intervention.
No studies reported a plan for sustainability or long-term
incorporation of the intervention into current clinical practice. Studies also generally failed to include information
about the health care system context in which the intervention was conducted. No studies reported on the local
quality improvement infrastructure, safety culture, or other
important contextual elements that could have influenced
the success of the intervention.
The CTI was the only transitional care strategy that
was “successfully” implemented and evaluated in multiple
settings, including many types of hospitals and integrated
and nonintegrated health care systems (26, 43, 55, 67). All
other investigations of interventions that reduced 30-day
readmissions or ED visits were single-center studies that
were not replicated in multiple settings or diverse
populations.
Sixteen studies reported comparisons of health care
utilization and associated costs for patients in the intervention group and patients receiving usual care. These costs
were measured over varying intervals after discharge and
used cost estimates from different sources. No studies reported the costs of the intervention itself. We therefore
could not draw any firm conclusions on the effect of transitional care interventions on overall health care costs.
Contextual factors probably play a significant role in
determining the effectiveness of a transitional care strategy.
These contextual factors may operate at the patient level
(for example, an individual patient’s readmission risk), the
organizational level (such as a hospital’s quality improvement infrastructure and ability to support transitional care
interventions), and the health care system level (such as
access to primary care). Unfortunately, the studies we identified did not describe these factors. Because CTI was the
only strategy evaluated in different patient populations and
health care systems, we could not draw conclusions on the
effect of context on effectiveness.
DISCUSSION
In this systematic review, we examined 47 studies involving 44 distinct hospital-initiated strategies aimed at reducing postdischarge AEs, ED visits, and readmissions. We
identified 15 studies showing that interventions successfully reduced readmission or ED visit rates after discharge,
including 8 studies showing that interventions reduced 30day readmission rates. Nearly all studies used a bridging
intervention, and 10 of the 15 used a dedicated transition
provider who contacted patients before and after discharge.
One of these strategies, the CTI, has been successfully implemented and evaluated in multiple patient populations
www.annals.org
Hospital-Initiated Transitional Care Interventions
and health care systems; a similar intervention, Project
RED, has been implemented in a safety net system. Although these strategies are relatively intensive and probably
require considerable resources, information on costs of
transitional care strategies was lacking. Because few studies
specifically addressed the problem of postdischarge AEs, we
could not reach firm conclusions regarding effective strategies in this area.
Two recent systematic reviews (71, 72) also attempted
to identify interventions to improve the quality of care
transitions at hospital discharge. One of these focused
on the clinical handover from hospital to primary care, and
the other evaluated transitional care interventions for patients with stroke and acute myocardial infarction. These
reviews identified many flaws in the care transitions evidence base that we found as well. These flaws included
possible selective reporting; heterogeneity in intervention
types, patient populations enrolled, and outcomes measured; limited description of implementation processes;
and failure to report on important contextual aspects that
may have influenced the success or failure of the transitional care strategy being studied.
Within our classification of interventions, the manner
in which the studies carried out specific interventions varied widely. For example, studies that deployed a dedicated
transition provider used different types of providers (primarily nurses, but also pharmacists) who had varying levels
of contact with patients after discharge (ranging from single telephone calls to multiple home visits). Although
many studies enrolled elderly patients or patients considered to be at high risk for readmission, these definitions
were also inconsistent. Strategies that involve adding dedicated transition providers probably require considerable
resources to implement and sustain effectiveness. However,
fewer than one third of studies described the training protocols or resources needed to implement a transitional care
strategy, and no studies reported a plan for intervention
sustainability.
Although readmission risk is known to be linked to
access to primary care and the overall level of health care
resources within a community (73), most studies did not
include information on the health system context in which
the intervention was implemented. In addition, even
among the most comprehensive intervention strategies reviewed, there was little evidence of active engagement of
primary care providers in the transitional care planning
process. Primary care providers and the medical home may
be best positioned to detect and prevent AEs before an ED
visit or readmission, and thus active engagement of outpatient providers in discharge safety efforts may prove
fruitful.
Despite the rapid proliferation of transitional care
strategies in the race to reduce hospital readmissions, there
has been a notable lack of attention to the potential additional benefit of strategies to reduce specific postdischarge
AEs. Postdischarge AEs should also be targeted in quality
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
improvement efforts because they still represent significant
failures to ensure patient safety, even if they do not ultimately lead to ED visits or readmissions. Medication safety
interventions led by clinical pharmacists seem to be a
promising approach, indicating a need for larger trials with
an explicit plan to measure clinically significant AEs. Further research in this field should also follow recently published recommendations (74) to standardize intervention
nomenclature and reproducibility, identify target populations most likely to benefit from specific interventions,
measure patient-centered outcomes, and rigorously report
and evaluate cost and implementation factors.
Our study has several limitations. We focused on transitional care strategies initiated during hospitalization for
general medical patient populations, and we excluded studies conducted in disease-specific populations. Because current policy initiatives emphasize the role of hospitals in
preventing readmissions in all patients, we therefore aimed
to identify strategies that hospitals could apply to broad
patient populations. Prior systematic reviews (18, 21, 72,
75) have identified interventions that can reduce readmission risk in patients with congestive heart failure, acute
myocardial infarction, or stroke, but these conditions collectively account for only about 10% of Medicare hospital
admissions per year (2). Thus, a successful disease-specific
approach may not translate to reductions in overall readmission rates. Proven disease-specific strategies, such as disease management programs, often rely on customized
patient self-management or medication adherence interventions that may be less relevant for other disease
processes.
We also included only studies that measured clinically
significant AEs, in an effort to emphasize patient-centered
outcomes. This led to exclusion of some studies that measured surrogate outcomes, such as studies of discharge
medication reconciliation that measured medication discrepancies but did not report data on clinical AEs (76, 77).
Some of these strategies may yet prove to be effective at
preventing clinical AEs. Finally, publication bias may have
affected the results of our review because the national focus
on readmissions has catalyzed many efforts to improve
transitional care that have yet to be published in the peerreviewed literature.
Hospitals are now faced with the challenge of reevaluating their current transitional care practices in order to
reduce 30-day readmission rates. Although emphasizing readmissions may have good face validity, we believe that
policymakers’ focus on 30-day readmissions is problematic.
Only a small proportion (approximately 20% from published studies) (78) of readmissions at 30 days are probably
preventable, and much of what drives hospital readmission
rates are patient- and community-level factors, such as
mental illness, poor social support, and poverty, that are
well outside the hospital’s control (79, 80). Furthermore,
high readmission rates can be the result of low mortality
rates, improved access to hospital care, and high admission
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 437
Supplement
Hospital-Initiated Transitional Care Interventions
rates (81) and therefore may not always represent care transitions failures. Because there are currently no reliable
methods to predict an individual patient’s readmission risk
(82), hospitals face significant difficulties in determining
which patients should be targeted for transitional care interventions. Finally, because hospitals are expending resources on reducing readmissions, they may not be able to
address other, more pressing patient safety issues. In this
context, our finding that only a few resource-intensive interventions seem to reduce readmission rates is especially
problematic.
In summary, we found that only a limited number of
bridging interventions involving a dedicated transition provider seems to reduce readmissions and ED visits after hospital discharge to home. Among these, only the CTI has
been implemented in multiple settings and patient populations. Few studies specifically targeted AEs after discharge, and the studies we identified provided little information about implementation factors, contextual factors,
or cost. Although hospitals are now being penalized for
excessive readmission rates, the strategies that an individual
hospital can implement to improve transitional care remain
largely undefined.
From the University of California, San Francisco, San Francisco,
California.
Note: The Agency for Healthcare Research and Quality reviewed contract deliverables to ensure adherence to contract requirements and quality, and a copyright release was obtained from the Agency for Healthcare
Research and Quality before submission of the manuscript.
Disclaimer: All statements expressed in this work are those of the authors
and should not in any way be construed as official opinions or positions
of the University of California, San Francisco; the Agency for Healthcare
Research and Quality; or the U.S. Department of Health and Human
Services.
Financial Support: From the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services (contract HHSA290-2007-10062I).
Potential Conflicts of Interest: Dr. Rennke: Grant (money to self and to
institution): AHRQ; Support for travel to meetings for the study or other
purposes: AHRQ; Payment for writing or reviewing the manuscript (money
to self and to institution): AHRQ; Provision of writing assistance, medicines,
equipment, or administrative support (money to institution): AHRQ; Consultancy: Society Hospital of Medicine. Dr. Ranji: Grant (money to institution): AHRQ. Dr. Magan: Grant (money to institution): AHRQ. Dr.
Wachter: Grant (money to institution): AHRQ; Support for travel to meetings for the study or other purposes (money to institution): AHRQ; Board
membership: Chair of the American Board of Internal Medicine; Grants/
grants pending (money to institution): AHRQ; Payment for lectures including service on speakers’ bureaus: honorarium for lectures from more than
100 health care organizations, mostly on patient safety, health care quality, and hospitalists; Royalties: Lippincott Williams & Wilkins, McGrawHill; Payment for development of educational presentations: QuantiaMD;
Payment for development of educational presentations (money to institution):
IPC-The Hospitalist Company; Stock/stock options: PatientSafe Solutions,
CRISI, EarlySense; Other: Compensation from John Wiley & Sons for
438 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
writing “Wachter’s World” blog, Benioff endowed chair in hospital medicine, funded by the US-UK Fulbright Commission for a sabbatical at
Imperial College London from July to December 2011, unpaid member
of the Board of Directors, Quality Committee of Salem Hospital. All
other authors have no dislosures. Disclosures can also be viewed at www
.acponline.org/authors/icmje/ConflictOfInterestForms
.do?msNum⫽M12-2573.
Requests for Single Reprints: Stephanie Rennke, MD, University of
California, San Francisco, UCSF Mount Zion Medical Center, 1600
Divisadero Street, San Francisco, CA 94115-1945; e-mail, srennke
@medicine.ucsf.edu.
Current author addresses and author contributions are available at
www.annals.org.
References
1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in
the Medicare fee-for-service program. N Engl J Med. 2009;360:1418-28.
[PMID: 19339721]
2. Jha AK, Joynt KE, Orav EJ, Epstein AM. The long-term effect of premier pay
for performance on patient outcomes. N Engl J Med. 2012;366:1606-15.
[PMID: 22455751]
3. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence
and severity of adverse events affecting patients after discharge from the hospital.
Ann Intern Med. 2003;138:161-7. [PMID: 12558354]
4. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW.
Deficits in communication and information transfer between hospital-based and
primary care physicians: implications for patient safety and continuity of care.
JAMA. 2007;297:831-41. [PMID: 17327525]
5. Gleason KM, McDaniel MR, Feinglass J, Baker DW, Lindquist L, Liss D,
et al. Results of the Medications at Transitions and Clinical Handoffs (MATCH)
study: an analysis of medication reconciliation errors and risk factors at hospital
admission. J Gen Intern Med. 2010;25:441-7. [PMID: 20180158]
6. Makaryus AN, Friedman EA. Patients’ understanding of their treatment plans
and diagnosis at discharge. Mayo Clin Proc. 2005;80:991-4. [PMID: 16092576]
7. Moore C, McGinn T, Halm E. Tying up loose ends: discharging patients with
unresolved medical issues. Arch Intern Med. 2007;167:1305-11. [PMID:
17592105]
8. Hasan O, Meltzer DO, Shaykevich SA, Bell CM, Kaboli PJ, Auerbach AD,
et al. Hospital readmission in general medicine patients: a prediction model.
J Gen Intern Med. 2010;25:211-9. [PMID: 20013068]
9. Kirby SE, Dennis SM, Jayasinghe UW, Harris MF. Patient related factors in
frequent readmissions: the influence of condition, access to services and patient
choice. BMC Health Serv Res. 2010;10:216. [PMID: 20663141]
10. Robinson S, Howie-Esquivel J, Vlahov D. Readmission risk factors after
hospital discharge among the elderly. Popul Health Manag. 2012;15:338-51.
[PMID: 22823255]
11. U.S. Department of Health & Human Services. Hospital Compare. 11
October 2012. Accessed at www.hospitalcompare.hhs.govon 29 November 2012.
12. Rau J. Medicare to penalize 2,211 hospitals for excess readmissions. Kaiser
Health News. 13 August 2012. Accessed at www.kaiserhealthnews.org/Stories
/2012/August/13/medicare-hospitals-readmissions-penalties.aspx on 28 December 2012.
13. HealthCare.gov. Partnership for patients: better care, lower costs. Accessed at
www.healthcare.gov/center/programs/partnership/index.html on 29 November
2012.
14. Maynard GA, Budnitz TL, Nickel WK, Greenwald JL, Kerr KM, Miller
JA, et al. 2011 John M. Eisenberg Patient Safety and Quality Awards. Mentored
implementation: building leaders and achieving results through a collaborative
improvement model. Innovation in patient safety and quality at the national
level. Jt Comm J Qual Patient Saf. 2012;38:301-10. [PMID: 22852190]
15. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions
to reduce 30-day rehospitalization: a systematic review. Ann Intern Med. 2011;
155:520-8. [PMID: 22007045]
www.annals.org
Hospital-Initiated Transitional Care Interventions
16. Chiu WK, Newcomer R. A systematic review of nurse-assisted case management to improve hospital discharge transition outcomes for the elderly. Prof Case
Manag. 2007;12:330-6; quiz 337-8. [PMID: 18030153]
17. Mistiaen P, Francke AL, Poot E. Interventions aimed at reducing problems
in adult patients discharged from hospital to home: a systematic meta-review.
BMC Health Serv Res. 2007;7:47. [PMID: 17408472]
18. Naylor MD, Aiken LH, Kurtzman ET, Olds DM, Hirschman KB. The
care span: The importance of transitional care in achieving health reform. Health
Aff (Millwood). 2011;30:746-54. [PMID: 21471497]
19. Parker SG, Peet SM, McPherson A, Cannaby AM, Abrams K, Baker R,
et al. A systematic review of discharge arrangements for older people. Health
Technol Assess. 2002;6:1-183. [PMID: 12065067]
20. Richards S, Coast J. Interventions to improve access to health and social care
after discharge from hospital: a systematic review. J Health Serv Res Policy. 2003;
8:171-9. [PMID: 12869344]
21. Shepperd S, McClaran J, Phillips CO, Lannin NA, Clemson LM, McCluskey A, et al. Discharge planning from hospital to home. Cochrane Database Syst
Rev. 2010:CD000313. [PMID: 20091507]
22. Forster AJ, Clark HD, Menard A, Dupuis N, Chernish R, Chandok N,
et al. Adverse events among medical patients after discharge from hospital.
CMAJ. 2004;170:345-9. [PMID: 14757670]
23. Shekelle PG, Wachter RM, Pronovost P. Chapter 2. Methods. In: Making
Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient
Safety Practices. Comparative Effectiveness Review. Prepared by the Southern
California-RAND Evidence-based Practice Center under contract no.
HHSA290200710062I. Rockville, MD: Agency for Healthcare Research and
Quality. [In Press].
24. Balaban RB, Weissman JS, Samuel PA, Woolhandler S. Redefining and
redesigning hospital discharge to enhance patient care: a randomized controlled
study. J Gen Intern Med. 2008;23:1228-33. [PMID: 18452048]
25. Bolas H, Brookes K, Scott M, McElnay J. Evaluation of a hospital-based
community liaison pharmacy service in Northern Ireland. Pharm World Sci.
2004;26:114-20. [PMID: 15085948]
26. Coleman EA, Parry C, Chalmers S, Min SJ. The care transitions intervention: results of a randomized controlled trial. Arch Intern Med. 2006;166:
1822-8. [PMID: 17000937]
27. Courtney M, Edwards H, Chang A, Parker A, Finlayson K, Hamilton K.
Fewer emergency readmissions and better quality of life for older adults at risk of
hospital readmission: a randomized controlled trial to determine the effectiveness
of a 24-week exercise and telephone follow-up program. J Am Geriatr Soc. 2009;
57:395-402. [PMID: 19245413]
28. Dellasega CA, Zerbe TM. A multimethod study of advanced practice nurse
postdischarge care. Clin Excell Nurse Pract. 2000;4:286-93. [PMID: 11858450]
29. Forster AJ, Clark HD, Menard A, Dupuis N, Chernish R, Chandok N,
et al. Effect of a nurse team coordinator on outcomes for hospitalized medicine
patients. Am J Med. 2005;118:1148-53. [PMID: 16194647]
30. Gallagher PF, O’Connor MN, O’Mahony D. Prevention of potentially
inappropriate prescribing for elderly patients: a randomized controlled trial using
STOPP/START criteria. Clin Pharmacol Ther. 2011;89:845-54. [PMID:
21508941]
31. Gillespie U, Alassaad A, Henrohn D, Garmo H, Hammarlund-Udenaes M,
Toss H, et al. A comprehensive pharmacist intervention to reduce morbidity in
patients 80 years or older: a randomized controlled trial. Arch Intern Med. 2009;
169:894-900. [PMID: 19433702]
32. Graumlich JF, Novotny NL, Stephen Nace G, Kaushal H, Ibrahim-Ali W,
Theivanayagam S, et al. Patient readmissions, emergency visits, and adverse
events after software-assisted discharge from hospital: cluster randomized trial.
J Hosp Med. 2009;4:E11-9. [PMID: 19479782]
33. Jack BW, Chetty VK, Anthony D, Greenwald JL, Sanchez GM, Johnson
AE, et al. A reengineered hospital discharge program to decrease rehospitalization:
a randomized trial. Ann Intern Med. 2009;150:178-87. [PMID: 19189907]
34. Koehler BE, Richter KM, Youngblood L, Cohen BA, Prengler ID, Cheng
D, et al. Reduction of 30-day postdischarge hospital readmission or emergency
department (ED) visit rates in high-risk elderly medical patients through delivery
of a targeted care bundle. J Hosp Med. 2009;4:211-8. [PMID: 19388074]
35. Lim WK, Lambert SF, Gray LC. Effectiveness of case management and
post-acute services in older people after hospital discharge. Med J Aust. 2003;178:
262-6. [PMID: 12633482]
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Supplement
36. Lipton HL, Bird JA. The impact of clinical pharmacists’ consultations on
geriatric patients’ compliance and medical care use: a randomized controlled trial.
Gerontologist. 1994;34:307-15. [PMID: 8076871]
37. Martin F, Oyewole A, Moloney A. A randomized controlled trial of a high
support hospital discharge team for elderly people. Age Ageing. 1994;23:228-34.
[PMID: 8085509]
38. Marusic S, Gojo-Tomic N, Erdeljic V, Bacic-Vrca V, Franic M, Kirin M,
et al. The effect of pharmacotherapeutic counseling on readmissions and emergency department visits. Int J Clin Pharm. 2012. [PMID: 23007693]
39. Naylor MD, Brooten D, Campbell R, Jacobsen BS, Mezey MD, Pauly
MV, et al. Comprehensive discharge planning and home follow-up of hospitalized elders: a randomized clinical trial. JAMA. 1999;281:613-20. [PMID:
10029122]
40. Naylor MD. Comprehensive discharge planning for hospitalized elderly: a
pilot study. Nurs Res. 1990;39:156-61. [PMID: 2188217]
41. Nazareth I, Burton A, Shulman S, Smith P, Haines A, Timberal H. A
pharmacy discharge plan for hospitalized elderly patients—a randomized controlled trial. Age Ageing. 2001;30:33-40. [PMID: 11322670]
42. Nikolaus T, Specht-Leible N, Bach M, Oster P, Schlierf G. A randomized
trial of comprehensive geriatric assessment and home intervention in the care of
hospitalized patients. Age Ageing. 1999;28:543-50. [PMID: 10604506]
43. Parry C, Min SJ, Chugh A, Chalmers S, Coleman EA. Further application
of the care transitions intervention: results of a randomized controlled trial conducted in a fee-for-service setting. Home Health Care Serv Q. 2009;28:84-99.
[PMID: 20182958]
44. Schnipper JL, Kirwin JL, Cotugno MC, Wahlstrom SA, Brown BA, Tarvin
E, et al. Role of pharmacist counseling in preventing adverse drug events after
hospitalization. Arch Intern Med. 2006;166:565-71. [PMID: 16534045]
45. Scullin C, Scott MG, Hogg A, McElnay JC. An innovative approach to
integrated medicines management. J Eval Clin Pract. 2007;13:781-8. [PMID:
17824872]
46. Siu AL, Kravitz RL, Keeler E, Hemmerling K, Kington R, Davis JW, et al.
Postdischarge geriatric assessment of hospitalized frail elderly patients. Arch Intern Med. 1996;156:76-81. [PMID: 8526700]
47. Stewart S, Pearson S, Luke CG, Horowitz JD. Effects of home-based intervention on unplanned readmissions and out-of-hospital deaths. J Am Geriatr Soc.
1998;46:174-80. [PMID: 9475445]
48. Thomas DR, Brahan R, Haywood BP. Inpatient community-based geriatric
assessment reduces subsequent mortality. J Am Geriatr Soc. 1993;41:101-4.
[PMID: 8426028]
49. Weinberger M, Oddone EZ, Henderson WG. Does increased access to
primary care reduce hospital readmissions? Veterans Affairs Cooperative Study
Group on Primary Care and Hospital Readmission. N Engl J Med. 1996;334:
1441-7. [PMID: 8618584]
50. Finn KM, Heffner R, Chang Y, Bazari H, Hunt D, Pickell K, et al.
Improving the discharge process by embedding a discharge facilitator in a resident
team. J Hosp Med. 2011;6:494-500. [PMID: 22042739]
51. Legrain S, Tubach F, Bonnet-Zamponi D, Lemaire A, Aquino JP, Paillaud
E, et al. A new multimodal geriatric discharge-planning intervention to prevent
emergency visits and rehospitalizations of older adults: the optimization of medication in AGEd multicenter randomized controlled trial. J Am Geriatr Soc.
2011;59:2017-28. [PMID: 22091692]
52. Adler A, Lipkin C, Cooper L, Agolino M, Jones V. Effect of social work
intervention on hospital discharge transition planning in a special needs population. Manag Care. 2009;18:50-3. [PMID: 19999254]
53. Al-Rashed SA, Wright DJ, Roebuck N, Sunter W, Chrystyn H. The value
of inpatient pharmaceutical counselling to elderly patients prior to discharge.
Br J Clin Pharmacol. 2002;54:657-64. [PMID: 12492615]
54. Brand CA, Jones CT, Lowe AJ, Nielsen DA, Roberts CA, King BL, et al.
A transitional care service for elderly chronic disease patients at risk of readmission. Aust Health Rev. 2004;28:275-84. [PMID: 15595909]
55. Coleman EA, Smith JD, Frank JC, Min SJ, Parry C, Kramer AM. Preparing patients and caregivers to participate in care delivered across settings: the Care
Transitions Intervention. J Am Geriatr Soc. 2004;52:1817-25. [PMID:
15507057]
56. Cowan MJ, Shapiro M, Hays RD, Afifi A, Vazirani S, Ward CR, et al. The
effect of a multidisciplinary hospitalist/physician and advanced practice nurse
collaboration on hospital costs. J Nurs Adm. 2006;36:79-85. [PMID: 16528149]
57. Einstadter D, Cebul RD, Franta PR. Effect of a nurse case manager on
postdischarge follow-up. J Gen Intern Med. 1996;11:684-8. [PMID: 9120655]
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) 439
Supplement
Hospital-Initiated Transitional Care Interventions
58. Gow P, Berg S, Smith D, Ross D. Care co-ordination improves quality-ofcare at South Auckland Health. J Qual Clin Pract. 1999;19:107-10. [PMID:
10408752]
59. Hellström LM, Bondesson A, Höglund P, Midlöv P, Holmdahl L, Rickhag
E, et al. Impact of the Lund Integrated Medicines Management (LIMM) model
on medication appropriateness and drug-related hospital revisits. Eur J Clin Pharmacol. 2011;67:741-52. [PMID: 21318595]
60. Hogan DB, Fox RA. A prospective controlled trial of a geriatric consultation
team in an acute-care hospital. Age Ageing. 1990;19:107-13. [PMID: 2337005]
61. Makowsky MJ, Koshman SL, Midodzi WK, Tsuyuki RT. Capturing outcomes of clinical activities performed by a rounding pharmacist practicing in a
team environment: the COLLABORATE study [NCT00351676]. Med Care.
2009;47:642-50. [PMID: 19433997]
62. Mudge A, Laracy S, Richter K, Denaro C. Controlled trial of multidisciplinary care teams for acutely ill medical inpatients: enhanced multidisciplinary
care. Intern Med J. 2006;36:558-63. [PMID: 16911547]
63. Palmer HC Jr, Halperin A, Elnicki M, Powers R, Kolar M, Evans K, et al.
Effect of a patient care partnership project on cost and quality of care at an
academic teaching hospital. South Med J. 2002;95:1318-25. [PMID: 12540000]
64. Scullin C, Hogg A, Luo R, Scott MG, McElnay JC. Integrated medicines
management— can routine implementation improve quality? J Eval Clin Pract.
2012;18:807-15. [PMID: 21504517]
65. Steeman E, Moons P, Milisen K, De Bal N, De Geest S, De Froidmont C,
et al. Implementation of discharge management for geriatric patients at risk of
readmission or institutionalization. Int J Qual Health Care. 2006;18:352-8.
[PMID: 16861721]
66. Styrborn K. Early discharge planning for elderly patients in acute
hospitals—an intervention study. Scand J Soc Med. 1995;23:273-85.
[PMID: 8919370]
67. Voss R, Gardner R, Baier R, Butterfield K, Lehrman S, Gravenstein S. The
care transitions intervention: translating from efficacy to effectiveness. Arch Intern
Med. 2011;171:1232-7. [PMID: 21788540]
68. Walker PC, Bernstein SJ, Jones JN, Piersma J, Kim HW, Regal RE, et al.
Impact of a pharmacist-facilitated hospital discharge program: a quasiexperimental study. Arch Intern Med. 2009;169:2003-10. [PMID: 19933963]
69. Wilkinson ST, Pal A, Couldry RJ. Impacting readmission rates and patient
satisfaction: results of a discharge pharmacist pilot program. Hospital Pharm.
2011;46:876-83.
70. Al-Ghamdi SA, Mahmoud MA, Alammari MA, Al Bekairy AM, Alwhaibi
M, Mayet AY, et al. The outcome of pharmacist counseling at the time of
440 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2)
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
hospital discharge: an observational nonrandomized study. Ann Saudi Med.
2012;32:492-7. [PMID: 22871618]
71. Hesselink G, Schoonhoven L, Barach P, Spijker A, Gademan P, Kalkman
C, et al. Improving patient handovers from hospital to primary care: a systematic
review. Ann Intern Med. 2012;157:417-28. [PMID: 22986379]
72. Prvu Bettger J, Alexander KP, Dolor RJ, Olson DM, Kendrick AS, Wing
L, et al. Transitional care after hospitalization for acute stroke or myocardial
infarction: a systematic review. Ann Intern Med. 2012;157:407-16. [PMID:
22986378]
73. Epstein AM, Jha AK, Orav EJ. The relationship between hospital admission
rates and rehospitalizations. N Engl J Med. 2011;365:2287-95. [PMID:
22168643]
74. Bray-Hall ST. Transitional care: focusing on patient-centered outcomes and
simplicity [Editorial]. Ann Intern Med. 2012;157:448-9. [PMID: 22986380]
75. Takeda A, Taylor SJ, Taylor RS, Khan F, Krum H, Underwood M. Clinical service organisation for heart failure. Cochrane Database Syst Rev. 2012;9:
CD002752. [PMID: 22972058]
76. Karapinar-Carkit F, Borgsteede SD, Zoer J, Smit HJ, Egberts AC, van den
Bemt PM. Effect of medication reconciliation with and without patient counseling on the number of pharmaceutical interventions among patients discharged
from the hospital. Ann Pharmacother. 2009;43:1001-10. [PMID: 19491320]
77. Schnipper JL, Hamann C, Ndumele CD, Liang CL, Carty MG, Karson
AS, et al. Effect of an electronic medication reconciliation application and process
redesign on potential adverse drug events: a cluster-randomized trial. Arch Intern
Med. 2009;169:771-80. [PMID: 19398689]
78. van Walraven C, Jennings A, Taljaard M, Dhalla I, English S,
Mulpuru S, et al. Incidence of potentially avoidable urgent readmissions
and their relation to all-cause urgent readmissions. CMAJ. 2011;183:
E1067-72. [PMID: 21859870]
79. Joynt KE, Orav EJ, Jha AK. Thirty-day readmission rates for Medicare
beneficiaries by race and site of care. JAMA. 2011;305:675-81. [PMID:
21325183]
80. Philbin EF, Dec GW, Jenkins PL, DiSalvo TG. Socioeconomic status as an
independent risk factor for hospital readmission for heart failure. Am J Cardiol.
2001;87:1367-71. [PMID: 11397355]
81. Fisher ES, Wennberg JE, Stukel TA, Sharp SM. Hospital readmission rates
for cohorts of Medicare beneficiaries in Boston and New Haven. N Engl J Med.
1994;331:989-95. [PMID: 8084356]
82. Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman
M, et al. Risk prediction models for hospital readmission: a systematic review.
JAMA. 2011;306:1688-98. [PMID: 22009101]
www.annals.org
Annals of Internal Medicine
Current Author Addresses: Dr. Rennke: University of California, San
Francisco, UCSF Mount Zion Medical Center, 1600 Divisadero Street,
San Francisco, CA 94115-1945.
Dr. Nguyen: University of California, San Francisco, UCSF Laurel
Heights, Campus Box 1211, 3333 California Street, San Francisco, CA
94143.
Drs. Shoeb and Ranji: Department of Medicine, University of California, San Francisco, 533 Parnassus Avenue, Box 0131, San Francisco, CA
94143.
Dr. Magan: Division of Hospital Medicine, University of California, San
Francisco, 533 Parnassus Avenue, Box 0131, U-129, San Francisco, CA
94143.
Dr. Wachter: Department of Medicine, University of California, San
Francisco, 533 Parnassus Avenue, Box 0120, San Francisco, CA 94143.
www.annals.org
Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013
Author Contributions: Conception and design: S. Rennke, O.K.
Nguyen, M.H. Shoeb, S.R. Ranji.
Analysis and interpretation of the data: S. Rennke, O.K. Nguyen, M.H.
Shoeb, Y. Magan, S.R. Ranji.
Drafting of the article: S. Rennke, O.K. Nguyen, M.H. Shoeb, Y.
Magan, S.R. Ranji.
Critical revision of the article for important intellectual content: S.
Rennke, O.K. Nguyen, M.H. Shoeb, Y. Magan, R.M. Wachter, S.R.
Ranji.
Final approval of the article: S. Rennke, O.K. Nguyen, M.H. Shoeb,
R.M. Wachter, S.R. Ranji.
Obtaining of funding: R.M. Wachter.
Administrative, technical, or logistic support: Y. Magan.
Collection and assembly of data: S. Rennke, O.K. Nguyen, M.H. Shoeb,
Y. Magan, S.R. Ranji.
5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) W-189