ARTICLE
Interns’ Success With Clinical Procedures in Infants
After Simulation Training
AUTHORS: David O. Kessler, MD, MSc, RDMS,a Grace
Arteaga, MD,b Kevin Ching, MD,c Laura Haubner, MD,d
Gunjan Kamdar, MD,e Amanda Krantz, MS,f Julie Lindower,
MD,g Michael Miller, MD,h Matei Petrescu, MD,i Martin V.
Pusic, MD,a Joshua Rocker, MD,h Nikhil Shah, MD,c
Christopher Strother, MD,j Lindsey Tilt, MD,a Eric R.
Weinberg, MD,c Todd P. Chang, MD,k Daniel M. Fein, MD,l
and Marc Auerbach, MD, MSce
aColumbia
University College of Physicians and Surgeons, New
York, New York; bMayo Clinic Children’s Hospital, Rochester,
Minnesota; cWeill Cornell School of Medicine, New York, New York;
dUniversity of South Florida College of Medicine, Tampa, Florida;
eYale University School of Medicine, New Haven, Connecticut; fNew
York University/Bellevue Hospital Center, New York, New York;
gUniversity of Iowa Children’s Hospital, Iowa City, Iowa; hCohen
Children’s Medical Center, New Hyde Park, New York; iTulane
University School of Medicine, New Orleans, Louisiana; jMount
Sinai School of Medicine, New York, New York; kChildren’s
Hospital Los Angeles, Los Angeles, California; and lChildren’s
Hospital at Montefiore, Bronx, New York
KEY WORDS
checklist, child, clinical skills, clinical competence/standards,
competency-based education/methods, educational
measurement/methods, education/medical/graduate methods,
humans, infant, internship and residency/methods, manikins,
models, anatomic, pediatrics/education, practice (psychology),
prospective studies, outcome assessment (health care), patient
simulation, randomized controlled trial, spinal puncture
ABBREVIATIONS
CIV—child intravenous line
CSF—cerebrospinal fluid
ILP—infant lumbar puncture
IV—intravenous line
LP—lumbar puncture
SBME—simulation-based medical education
Drs Kessler, Arteaga, Ching, Haubner, and Kamdar, Ms Krantz,
Drs Lindower, Miller, Petrescu, Pusic, Rocker, Shah, Tilt,
Weinberg, Chang, Fein, and Auerbach contributed substantially
to the conception and design of this study; Drs Kessler, Arteaga,
Ching, Haubner, Kamdar, Lindower, Miller, Petrescu, Pusic,
Rocker, Shah, Strother, Tilt, Weinberg, Chang, and Auerbach
contributed to the data acquisition and enrollment of study
subjects; and Drs Kessler, Auerbach, and Pusic contributed to
the analysis and interpretation of the data. All authors
contributed to the drafting, editing, and preparation of the
manuscript, and all authors approved of the final version of the
manuscript and are responsible for the reported research.
www.pediatrics.org/cgi/doi/10.1542/peds.2012-0607
doi:10.1542/peds.2012-0607
Accepted for publication Nov 19, 2012
(Continued on last page)
WHAT’S KNOWN ON THIS SUBJECT: Pediatric training programs
use simulation for procedural skills training. Research
demonstrates student satisfaction with simulation training,
improved confidence, and improved skills when retested on
a simulator. Few studies, however, have investigated the clinical
impact of simulation education.
WHAT THIS STUDY ADDS: This is the first multicenter, randomized
trial to evaluate the impact of simulation-based mastery learning
on clinical procedural performance in pediatrics. A single
simulation-based training session was not sufficient to improve
interns’ clinical procedural performance.
abstract
BACKGROUND AND OBJECTIVE: Simulation-based medical education
(SBME) is used to teach residents. However, few studies have evaluated
its clinical impact. The goal of this study was to evaluate the impact of
an SBME session on pediatric interns’ clinical procedural success.
METHODS: This randomized trial was conducted at 10 academic medical centers. Interns were surveyed on infant lumbar puncture (ILP)
and child intravenous line placement (CIV) knowledge and watched
audiovisual expert modeling of both procedures. Participants were
randomized to SBME mastery learning for ILP or CIV and for 6 succeeding months reported clinical performance for both procedures. ILP
success was defined as obtaining a sample on the first attempt with
,1000 red blood cells per high-power field or fluid described as clear.
CIV success was defined as placement of a functioning catheter on the
first try. Each group served as the control group for the procedure for
which they did not receive the intervention.
RESULTS: Two-hundred interns participated (104 in the ILP group and
96 in the CIV group). Together, they reported 409 procedures. ILP success rates were 34% (31 of 91) for interns who received ILP mastery
learning and 34% (25 of 73) for controls (difference: 0.2% [95% confidence interval: –0.1 to 0.1]). The CIV success rate was 54% (62 of
115) for interns who received CIV mastery learning compared with
50% (58 of 115) for controls (difference: 3% [95% confidence interval:
–10 to 17]).
CONCLUSIONS: Participation in a single SBME mastery learning session was insufficient to affect pediatric interns’ subsequent procedural success. Pediatrics 2013;131:e811–e820
PEDIATRICS Volume 131, Number 3, March 2013
e811
Downloaded from by guest on July 12, 2017
The Accreditation Council for Graduate
Medical Education mandates pediatric
residency programs to provide “sufficient training” for providers to develop
competency in 16 procedures. Skills
training has traditionally involved the
apprenticeship model of “see one, do
one, teach one.”1 There is concern that
this paradigm of skills training is inefficient and not safe for patients.2
Furthermore, patients and parents are
uncomfortable with trainees learning
while performing procedures.3 However, patients report they would be
more likely to allow trainees to perform a procedure after demonstrating
mastery on a simulator.4 New medical
school graduates have minimal exposure to and lack confidence in common
procedures such as lumbar puncture
(LP).5–14 In addition, training programs
struggle to provide sufficient opportunities for procedural training in the
setting of restricted work hours, increased attending oversight, and greater
midlevel providers competing for experience.10–12
A growing number of training programs
are using simulation-based medical
education (SBME) to meet the challenges of procedural training. SBME
allows for controlled training experiences and standardized learning outcomes. Experiences traditionally left to
chance, such as when and where a
procedure will occur, are standardized.
Teaching is customized to individual
learner needs.15–17 SBME provides opportunities for novice providers to
practice procedural skills and make
errors without harming patients. Research has demonstrated that SBME
improves clinical skills and patient
outcomes associated with procedures
in adults,18 including laparoscopic
surgery,19 bronchoscopy,20 emergency
airway management,21 advanced cardiac life support,22–24 thoracentesis,25
and central venous catheter insertion.26,27 However, studies demonstrating
e812
improved clinical impact on pediatric
patients are limited to obstetrical literature showing improved neonatal
outcomes for shoulder dystocia after
an SBME intervention.28,29
of pediatric training programs. We
hypothesized that a single SBME mastery
learning intervention improves clinical
procedural success rates on patients.
One of the components of a successful
SBME intervention is incorporation
of a sound educational theory. The
majority of SBME studies with positive
outcomes apply mastery learning, a
form of competency-based education.30
Mastery learning starts with a baseline
skill assessment by using a checklist
to identify the trainee’s individual
learning needs. Next, the learner participates in repetitive cycles of deliberate practice until he or she
achieves a predefined passing score on
the checklist. An ideal deliberate practice session consists of a highly motivated learner pursuing a well-defined
learning objective via focused repetitive practice while receiving informative feedback from an expert.31,32
The duration of a learning session will
vary based on the time it takes each
learner to achieve the predefined outcome.33
METHODS
In a recent study,34 interns randomized
to SBME procedural training (LP, intravenous line [IV], and bag valve
mask) demonstrated higher skill levels than interns randomized to conventional procedural training when
retested on a simulator. However, participation in SBME was not associated
with improved procedural success
rates on patients. In contrast, our previous research at a single center
demonstrated that residents randomized to SBME mastery learning for the
infant lumbar puncture (ILP) procedure were more likely to be successful with their next patient ILP (71%)
than the control group (27%).35 The
current study aims to evaluate the impact of a single SBME mastery learning
session on pediatric interns’ ILP and
child intravenous line insertion (CIV) performance on patients across a network
KESSLER et al
Downloaded from by guest on July 12, 2017
Study Design and Setting
This was a randomized controlled trial
of an SBME mastery learning intervention conducted in 10 tertiary care
urban academic medical centers at
the start of the 2009–2010 academic
year. These institutions are members
of the Patient Outcomes in Simulation
Education Network, a pediatric SBME
research network consisting of 10 institutions during the study period
(currently known as INSPIRE).36 Investigators conducted a 4-month studyplanning phase involving biweekly
conferences. The network supported
the purchase of simulator equipment,
video development, and centralized
data collection. Institutional review
boards at each center approved this
study, and participants provided the
requisite consent before enrollment
at each site.
Participants
We enrolled incoming postgraduate
year 1 trainees from pediatric categorical or combined training programs.
Intervention
Using a block randomization scheme,
participants were randomized within
each site to either the ILP mastery
learning or CIV mastery learning group.
Each group served both as an experimental group for the mastery learning
intervention that it received as well
as a control group for the procedural
training that it did not receive. To keep
outcome assessors blinded, each participant was assigned an anonymous
study identification number for all data
collection. Next, participants viewed
a 40-minute set of LP and IV videos
ARTICLE
including both procedure videos published by New England Journal of
Medicine and content developed by the
study authors.37 The video content included indications, contraindications,
complications, necessary equipment,
and key steps (positioning, landmarks,
preparation, early stylet removal technique, and use of local anesthesia and
analgesia). Additional video content developed by the study authors covered
pediatric-specific elements of these
procedures along with expert modeling
of both procedures on simulators.38
baseline information on knowledge,
attitudes, and experience with the ILP
and CIV procedures. Knowledge was
assessed by using multiple choice questions developed and pilot tested on nonstudy senior residents, fellows, and
faculty. Attitudes were assessed on a 4point Likert scale of confidence. ILP and
CIV procedural experience was assessed
based on numbers of procedures observed and performed.
The SBME mastery learning session
occurred at the start of internship and
used bench-top simulators and trained
facilitators who guided participants in
deliberate practice until they achieved
a predefined level of mastery of the skill
being taught. All facilitators were clinician educators who participated in
a standardized 30-minute train-thetrainer session before the study. The
ILP group completed their mastery
learning session by using a Laerdal
Baby Stap ILP simulator (Laerdal Medical, Stockholm, Sweden). The CIV
group completed their session by using a Laerdal Multivenous IV training
arm simulator. Mastery was defined as
achievement of a predetermined level
of performance on a subcomponent
skills checklist that was developed
through a modified Delphi process
among the network site directors
(Appendix). The training sessions continued until learners demonstrated independent mastery performance on the
checklist (ie, sessions ranged from 20–
60 minutes depending on the learner).
At the study’s conclusion 6 months later,
each group received mastery learning
for the alternate procedure to close the
educational gap.
Throughout the 6-month study period,
participants completed an online selfreport questionnaire for all ILP procedures on patients aged ,365 days and
all CIV placements on patients 1 to 18
years of age performed as part of
their patient care duties. Participants
were sent monthly e-mail reminders
with links to the online procedure logs.
Site directors used varied methods,
including checking computer medical
records, reviewing local procedure
logs, and holding conversations with
staff, to provide feedback and emphasize the importance of reporting the
procedures. Data collected for ILP included fluid acquisition, cell count,
fluid appearance (bloody, blood-tinged,
purulent, or clear), number of attempts,
patient age, holder, clinical environment
where the procedure was performed,
use of local anesthetic, use of early
stylet technique, and presence of supervisor. ILP success was defined as
obtaining an adequate sample on the
first attempt that had ,1000 red blood
cells per high-power field on microscopic examination or that was described as clear. Data were excluded
from analysis if another provider had
attempted the procedure before the
subject’s attempt (thereby increasing
the risk for a traumatic specimen). Data
collected for CIV included success,
gauge of needle, number of attempts,
age of patient, clinical environment, and
presence of supervisor. CIV success
Baseline Assessment
At the start of internship, participants
completed an online, self-administered,
28-item questionnaire that collected
Primary Outcome Measures and
Potential Confounders
PEDIATRICS Volume 131, Number 3, March 2013
was defined as placement of a functioning intravenous catheter on the first
attempt. Physicians supervising the interns during the procedures were unaware of the participants’ study group,
as were the patients and parents.
Assessing for Reporting and
Other Bias
We assessed reporting behaviors in
a follow-up questionnaire at 6 months
that assessed reporting behaviors
and surveyed participants for any additional procedural training received
during the study period.
Analysis
Participant characteristics were analyzed by using descriptive statistics, and
clinical outcomes were compared by
using the Fisher exact test. Knowledge
and confidence levels were compared
between groups by using the Mann–
Whitney U test and x2 analysis. Other
secondary variables were analyzed by
using either x 2 tests for proportions or
Mann–Whitney U tests for continuous
variables. We powered the study to find
an improvement from 60% success at
obtaining cerebrospinal fluid (CSF)
from the patient to 80%, estimating
a sample size of 73 residents in each
group to have 80% power to detect this
difference at an a level of 5% (G*Power
for Mac, version 3.0.10).39 Sample sizes
were based on ILP frequency, expected
to be rarer than IV insertions. Target
enrollment was increased by 20% to
accommodate interns who did not have
an opportunity to do an ILP within the 6month study period. All analyses were
conducted by using SPSS version 19
(IBM SPSS Statistics, IBM Corporation,
Armonk, NY).
RESULTS
A total of 210 interns were randomized
to either the ILP or CIV group, and 200
interns completed the study (Fig 1).
e813
Downloaded from by guest on July 12, 2017
Baseline knowledge, attitudes, and
experience collected on the initial
survey were similar in both groups
(Table 1).
A total of 203 participants achieved
mastery during the session according
to a predefined set of criteria (Appendix). Across study groups, clinical data
were reported for 178 LPs performed by
89 participants and 230 IVs performed
by 65 participants from all sites.
Participants who reported procedures
were similar in terms of experience,
knowledge, and attitudes to those who
did not report procedures throughout
the year. Groups who performed an ILP
or CIV were similar with regard to
hospital location, number of attempts,
and whether another provider had
made previous attempts in the preceding 24 hours, in addition to other
characteristics (Table 2).
Primary Outcome Measures:
Clinical Outcomes
In the ILP mastery learning group, 31
(34%) of 91 LPattempts were successful
on the first try. In the CIV mastery
learning group, 25 (34%) of 73 LP
attempts were successful (difference:
0.2% [95% confidence interval: –0.1 to
0.1]). Due to another provider’s previous LP attempt, 10 reports from the
ILP group and 4 from the CIV group
were excluded. There were no differences between groups for rates of CSF
acquisition, success after multiple
attempts, or success with the first ILP
performed after training (Table 3).
There were also no statistical differences in success between groups on an
individual hospital basis.
In the ILP mastery learning group, 58
(50%) of 115 IV attempts were successful on the first try compared with
62 (54%) of 115 attempts in the CIV
mastery learning group. There were no
differences between groups for rates of
success after multiple attempts or
success with the first IV performed on
a child after the SBME training. There
were also no statistical differences in
success between groups on an individual hospital basis.
FIGURE 1
CONSORT study flow diagram.
e814
KESSLER et al
Downloaded from by guest on July 12, 2017
Sixteen subjects who did not complete
demographic data were included in this
intention-to- treat analysis for both
outcome measures. A reanalysis with
these subjects removed also did not
result in any statistical differences
between groups for either outcome.
Process Measures
Median elapsed days from training to
reporting of LPs in the ILP mastery
learning group were less (70 days
compared with 90 days in the control
group, P = .001 [Mann–Whitney U test]),
whereas median elapsed days from
training to reporting of IVs in the ILP
mastery learning group were longer
(91.5 days compared with 64 days in
the CIV mastery learning group, P = .03
[Mann–Whitney U test]) (Fig 2).
A total of 137 participants responded to
the 6-month follow-up questionnaire
(69% response rate). Underreporting of
procedures was not statistically different between groups. Self-reported
experience and reporting rates are
presented in Table 4. There were no
ARTICLE
TABLE 1 Participant Characteristics: Baseline Experience, Knowledge, and Confidence With the LP and IV Procedures
Characteristic
Received ILP Mastery Learning
(n = 96 [8 forms missing])
LP experience
Previous LP didactic training (yes)
Previous LP simulator experience (yes)
Observed an ILP (yes)
Mean ILPs observed
Performed an ILP (yes)
Mean ILPs performed
LP knowledge
% Correct on 5-item knowledge quiz
LP confidence
% Agreement with the following statement: “I feel confident in my ability to perform
an LP on an infant” (responses dichotomized from 4-point Likert scale)
IV experience
Previous IV didactic training (yes)
Previous IV simulator experience (yes)
Observed an infant IV (yes)
Mean number of infant IVs observed
Performed an infant IV (yes)
Mean number of infant IVs performed
IV knowledge
% Correct on 6-item knowledge quiz
IV confidence
% Agreement with the following statement: “I feel confident in my ability to place
an IV on a child” (responses dichotomized from 4-point Likert scale)
statistically significant differences in
additional procedural training obtained
between the 2 groups.
DISCUSSION
We investigated whether a single SBME
mastery learning session for a procedure could result in measurably
higher success rates with the procedure on patients. Although we did not
detect differences in procedural success rates in participants completing
an SBME session, our work demonstrates the feasibility of testing the
impact of SBME on clinical outcomes in
a multi-institutional setting.
These results differ from our previous
single-center study of ILP success after
SBME (71% for intervention and 27% for
controls).35 That cohort included residents from all 3 years and only 3
trainers, whereas our current study
included only interns and 50 trainers.
To reduce variability in the SBME intervention, we designed a train-thetrainer module that was viewed by all
22 (23%)
15 (16%)
71 (74%)
2.3 (SD: 2.5)
28 (29%)
0.5 (SD: 1)
19 (22%)
14 (16%)
58 (66%)
2.3 (SD: 3)
19 (22%)
0.8 (SD: 2.3)
56% (SD: 23%)
59% (SD: 22%)
Agree, 13 (14%)
Disagree, 83 (86%)
Agree, 12 (14%)
Disagree, 76 (86%)
53 (55%)
51 (53%)
86 (90%)
11 (SD: 10)
48 (50%)
2.2 (SD: 3.5)
49 (56%)
47 (53%)
83 (94%)
9.9 (SD: 10.7)
44 (50%)
2.3 (SD: 4.4)
79% (SD: 16%)
73% (SD: 17%)
Agree, 33 (34%)
Disagree, 63 (66%)
trainers; however, variability in learner–
trainer interactions or other institutional
practices may have led to differences in
the intervention’s effectiveness.
In this study, we report success rates
for ILP (34% for intervention and 34%
for controls) that are lower than the
descriptive reports of novice success
rates in the literature (45%–63%).34,40
These differences may be due to the
populations studied, variations in data
acquisition, or the definitions of success. Our decision to define success as
a single, atraumatic attempt was based
on the observation that the CSF is
more likely to be bloody after multiple
attempts.41 This patient-centered definition was chosen because success
on the first attempt would cause the
least harm to patients. Other than our
aforementioned single-center study,35
no research has demonstrated an
SBME intervention leading to improved
clinical success with the ILP procedure.
A randomized trial of an ILP SBME intervention for pediatric residents at
the start of their emergency medicine
PEDIATRICS Volume 131, Number 3, March 2013
Received CIV Mastery Learning
(n = 88 [8 forms missing])
Agree, 25 (28%)
Disagree, 63 (72%)
rotation did not find any significant
difference in clinical success (70% for
participants and 62% for controls).34
The high success rates reported in that
study are likely due to the more liberal
definition of success, comprising any
CSF sample that was suitable for culture. A more recent single-center study
found that 62% (13 of 21) of interns
were successful on their first ILP attempt after an SBME intervention; however, conclusions were limited due to
a lack of comparison group.42
The success rates reported in this study
for CIV (62% for intervention and 58%
for control) are similar to success rates
in the literature. A previous SBME study
of 53 interns reported no improvement
in CIV success (66% for intervention and
50% for controls).34
Participants in both groups reported
few procedures, even when accounting
for underreporting. This shortage of
clinical opportunities and the timelapse between mastery learning and
clinical performance may have contributed to the lack of improvement in
e815
Downloaded from by guest on July 12, 2017
TABLE 2 Comparison of Procedural Factors Between Groups
Factor
No. of LP procedures performed
Days from training to ILP performance
(median)
Site ILP performed
ILP patient median age in days (IQR)
Holder for ILP
Attending supervision (yes)
Early stylet removal used (yes)
Analgesia used (yes)
Procedures reported by hospital
No. of CIV procedures performed
Days from training to CIV performance
(median)
Site CIV performed
CIV patient median age in years (IQR)
Attending supervision (yes)
Needle gauge
Procedures reported by hospital
ILP Mastery
Learning Group
(n = 63)
CIV Mastery
Learning Group
(n = 47)
102
69 (25–113)
76
93 (44–136)*
ED, 29 (28%)
NICU, 52 (51%)
Inpatient, 17 (17%)
Other, 4 (4%)
15 (3–47)
Attending, 6 (6%)
Nurse, 55 (54%)
Resident, 21 (21%)
Other assistant, 19 (19%)
Child life, 1 (1%)
28 (28%)
53 (52%)
12 (12%)
Hospital A, 5 (5%)
Hospital B, 10 (11%)
Hospital C, 15 (16.5%)
Hospital D, 6 (7%)
Hospital E, 1(1%)
Hospital F, 12 (13%)
Hospital G, 5 (5.5%)
Hospital H, 5 (5.5%)
Hospital I, 26 (29%)
Hospital J, 6 (7%)
115
94 (45–134)
ED, 30 (40%)
NICU, 28 (37%)
Inpatient, 17 (22%)
Other, 1 (1%)
20 (4–44)
Attending, 6 (8%)
Nurse, 47 (62%)
Resident, 9 (12%)
Other assistant, 14 (18%)
24 (32%)
24 (32%)
17 (22%)
Hospital A, 9 (12%)
Hospital B, 2 (3%)
Hospital C, 11 (15%)
Hospital D, 9 (12%)
Hospital E, 3 (4%)
Hospital F, 6 (8%)
Hospital G, 10 (14%)
Hospital H, 3 (4%)
Hospital I, 10 (14%)
Hospital J, 10 (14%)
115
66 (28–125)*
ED, 35 (30%)
NICU, 8 (7%)
Inpatient, 71 (62%)
Other, 1 (1%)
8 (2–16)
6 (5%)
24 G, 32 (28%)
22 G, 48 (42%)
20 G, 19 (16%)
18 G, 8 (7%)
No answer, 8 (7%)
ED, 36 (31%)
NICU, 5 (4%)
Inpatient, 73 (64%)
Other, 1 (1%)
5 (2–14)
6 (5%)
24 G, 52 (45%)
22 G, 52 (45%)
20 G, 9 (8%)
18 G, 0 (0%)
No answer, 2 (2%)*
Hospital A, 16 (14%)
Hospital B, 35 (30%)
Hospital C, 1 (1%)
Hospital D, 36 (31%)
Hospital E, 2 (2%)
Hospital, F, 19 (16.5%)
Hospital G, 5 (4%)
Hospital H, 1 (1%)
Hospital I, 0 (0%)
Hospital J, 0 (0%)
Hospital A, 19 (16.5%)
Hospital B, 27 (23.5%)
Hospital C, 1 (1%)
Hospital D, 37 (32%)
Hospital E, 1 (1%)
Hospital F, 22 (19%)
Hospital G, 8 (7%)
Hospital H, 0 (0%)
Hospital I, 0 (0%)
Hospital J, 0 (0%)
ED, emergency department; IQR, interquartile range.
* P , .05 (all other P values in table were ..05).
clinical success rates. Studies on skill
retention for other procedures, such
as neonatal intubation, have demonstrated a significant deterioration of
skills within months after simulator
e816
training.43–46 Current research into
skills training suggests that distributed practice and just-in-time skills
refreshers may be a better strategy to
confront skill deterioration over time.47,48
KESSLER et al
Downloaded from by guest on July 12, 2017
Participants who received SBME for
a skill were more likely to report that
procedure earlier in the year compared
with the control group. This finding may
have been due to increased confidence
of participants or their supervisors.
Educational interventions often improve confidence more rapidly than
skills.49,50 In our study, premature performance of the procedure may have
occurred before an intern was truly
competent due to increased confidence of participant or supervisor.
Another possibility is that participants
lost interest in reporting once they
became proficient in a procedure. Both
scenarios could have falsely deflated
the success rates of the intervention
group. However, the fact that first-time
success rates did not differ between
groups speaks against this variable
being a major confounder.
An important limitation of our study
was the reliance on self-report to
document the primary outcome. We
chose self-report in part due to limited
feasibility of alternative data collection
methods that could reliably capture all
events and be standardized across
multiple centers. More robust methods
such as direct field observation were
considered less feasible based on the
heavy resources and the logistics of
around-the-clock coverage that would
have limited site involvement and significantly reduced our power to detect
a clinical difference. Despite the propensity of underreporting seen with
self-reported outcomes, we still had
a sufficient number of reports to have
detected a clinically significant difference had it existed. Another potential
problem with self-report is recall bias in
which reliance on memory leads to
systematic errors (eg, reporting only
successes); however, the success rate
we observed is plausible and had sufficient variability to have allowed us
to detect differences between groups.
We cannot exclude the possibility that
ARTICLE
TABLE 3 Procedural Success Rates as a Function of Training Received
Group
ILP Success
CIV Success
ILP mastery learning group
CIV mastery learning group
31/91 (34%)
25/73 (34%)
58/115 (50%)
62/115 (54%)
recall interacts with the group assignment. Future studies should consider
verifying self-report data through chart
review or supervisor-reporting forms.
use of multiple trainers is an inherent
limitation to any multicenter educational study that we sought to mitigate
through the use of standardized training protocols and faculty development. Although this inevitably increases
variability (despite best efforts for
standardization), it also improves the
generalizability of our findings.
Participants in both groups reported
low levels of experience with ILP and CIV
procedures before starting internship,
confirming that current medical school
experiences are not sufficient to prepare incoming interns to perform these
procedures.14 However, we were also
missing baseline data from 16 participants, thus limiting our ability to
draw conclusions in this regard. The
CONCLUSIONS
This study demonstrated the feasibility
of disseminating a standardized SBME
intervention at multiple institutions.
However, participation in a single SBME
mastery learning session did not seem
to impact pediatric interns’ clinical
success rates with ILP or CIV. Despite
training, interns were not adequately
prepared for either procedure in the
clinical setting as measured by low
success rates. Although mastery learning may be a necessary component of
skills education, it was not sufficient
to affect clinical success rates. Future
studies should focus on providing
more time for deliberate practice
and distributed training to deal with
skill development and skill retention,
respectively.
ACKNOWLEDGMENTS
The authors thank Jillian Olsen for her
help in editing and preparing the
FIGURE 2
Number of procedural reports per group over time.
PEDIATRICS Volume 131, Number 3, March 2013
e817
Downloaded from by guest on July 12, 2017
TABLE 4 Response to 6-Month Follow-up Questionnaire
Response
Received ILP Mastery
Learning (n = 66)
Received CIV Mastery
Learning (n = 71)
Statistics
“I attempted an infant LP this year (,1 year old)”
“How many of your ILP attempts did you log online this year by
using the POISE survey link?”
Yes, 54 (82%)
None, 15 (28%)
Some, 10 (18.5%)
Half, 2 (4%)
Most, 10 (18.5%)
All, 17 (31%)
Yes, 40 (67%)
None, 14 (35%)
Some, 11 (27.5%)
Half, 3 (7.5%)
Most, 9 (22.5%)
All, 3 (7.5%)
Yes, 61 (86%)
None, 25 (41%)
Some, 10 (16%)
Half, 2 (3%)
Most, 6 (10%)
All, 18 (30%)
Yes, 47 (66%)
None, 18 (38%)
Some, 10 (21%)
Half, 4 (8.5%)
Most, 12 (25.5%)
All, 3 (6%)
P = NS, x2
P = NS, x2
“I attempted an IV in a child this year (1 to 18 years old)”
“How many of your IV attempts did you log online this year by
using the POISE survey link?”
P = NS, x2
P = NS, x2
NS, not statistically significant; POISE, Patient Outcomes in Simulation Education.
manuscriptforpublication;KarlSantiago,
who provided technical assistance with
organization of several orientation train-
ings and manikin distribution; and Laura
Santry, for help with editing the figures.
The authors also acknowledge the many
faculty educators and interns who participated in making this educational intervention possible.
residents in the performance of lumbar
punctures. Acad Emerg Med. 2005;12(7):
622–628
Huang GC, Smith CC, Gordon CE, et al.
Beyond the comfort zone: residents assess
their comfort performing inpatient medical
procedures. Am J Med. 2006;119(1):71.e17–
71.e24
Gaies MG, Landrigan CP, Hafler JP, Sandora
TJ. Assessing procedural skills training in
pediatric residency programs. Pediatrics.
2007;120(4):715–722
Gozzo YF, Cummings CL, Chapman RL, Bizzarro
MJ, Mercurio MR. Who is performing medical
procedures in the neonatal intensive care
unit? J Perinatol. 2011;31(3):206–211
Nadel FM, Lavelle JM, Fein JA, Giardino AP,
Decker JM, Durbin DR. Assessing pediatric
senior residents’ training in resuscitation:
fund of knowledge, technical skills, and
perception of confidence. Pediatr Emerg
Care. 2000;16(2):73–76
Kamdar G, Kessler D, Tilt L, et al. Qualitative
evaluation of just-in-time simulation-based
learning: the learners’ perspective. Simul
Healthc. 2013, In press
Auerbach MA, Chang T, Krantz A, Pratt A,
Gerard J, Quinones C, Pusic M, Kessler D.
Are pediatric interns prepared to perform
infant lumbar punctures? A multi-institutional
descriptive study. Pediatric Emergency Care.
[accepted 2012, publication pending]
McGaghie WC, Issenberg SB, Petrusa ER,
Scalese RJ. A critical review of simulationbased medical education research: 20032009. Med Educ. 2010;44(1):50–63
16. Issenberg SB, McGaghie WC, Petrusa ER,
Lee Gordon D, Scalese RJ. Features and uses
of high-fidelity medical simulations that lead
to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10–28
17. Scalese RJ, Obeso VT, Issenberg SB. Simulation technology for skills training and competency assessment in medical education.
J Gen Intern Med. 2008;23(suppl 1):46–49
18. McGaghie WC. Medical education research
as translational science. Sci Transl Med.
2010;2(19):cm8
19. Andreatta PB, Woodrum DT, Birkmeyer JD,
et al. Laparoscopic skills are improved with
Lap Mentor™ training: results of a randomized, double-blinded study. Ann Surg.
2006;243:854–863
20. Blum MG, Powers TW, Sundaresan S. Bronchoscopy simulator effectively prepares junior residents to competently perform basic
clinical bronchoscopy. Ann Thorac Surg.
2004;78(1):287–291, discussion 287–291
21. Mayo PH, Hackney JE, Mueck JT, Ribaudo V,
Schneider RF. Achieving house staff competence in emergency airway management:
results of a teaching program using a computerized patient simulator. Crit Care Med.
2004;32(12):2422–2427
22. Wayne DB, Didwania A, Feinglass J, Fudala
MJ, Barsuk JH, McGaghie WC. Simulationbased education improves quality of care
during cardiac arrest team responses at
an academic teaching hospital: a casecontrol study. Chest. 2008;133(1):56–61
23. Wayne DB, McGaghie WC. Use of simulationbased medical education to improve patient
REFERENCES
1. Rodriguez-Paz JM, Kennedy M, Salas E,
et al. Beyond “see one, do one, teach one”:
toward a different training paradigm. Qual
Saf Health Care. 2009;18(1):63–68
2. Walter AJ. Surgical education for the
twenty-first century: beyond the apprentice
model. Obstet Gynecol Clin North Am. 2006;
33(2):233–236, vii
3. Williams CT, Fost N. Ethical considerations
surrounding first time procedures: a study
and analysis of patient attitudes toward
spinal taps by students. Kennedy Inst Ethics
J. 1992;2(3):217–231
4. Graber MA, Wyatt C, Kasparek L, Xu Y. Does
simulator training for medical students
change patient opinions and attitudes toward medical student procedures in the
emergency department? Acad Emerg Med
2005;12(7):635–639
5. Kilbane BJ, Adler MD, Trainor JL. Pediatric
residents’ ability to perform a lumbar
puncture: evaluation of an educational intervention. Pediatr Emerg Care. 2010;26(8):
558–562
6. Hicks CM, Gonzalez R, Morton MT, Gibbons
RV, Wigton RS, Anderson RJ. Procedural
experience and comfort level in internal
medicine trainees. J Gen Intern Med. 2000;
15(10):716–722
7. Coberly L, Goldenhar LM. Ready or not,
here they come: acting interns’ experience
and perceived competency performing
basic medical procedures. J Gen Intern
Med. 2007;22(4):491–494
8. Lammers RL, Temple KJ, Wagner MJ, Ray D.
Competence of new emergency medicine
e818
9.
10.
11.
12.
13.
14.
15.
KESSLER et al
Downloaded from by guest on July 12, 2017
ARTICLE
24.
25.
26.
27.
28.
29.
30.
31.
care quality. Resuscitation. 2010;81(11):
1455–1456
Wayne DB, Butter J, Siddall VJ, et al.
Simulation-based training of internal medicine residents in advanced cardiac life
support protocols: a randomized trial.
Teach Learn Med. 2005;17(3):202–208
Wayne DB, Barsuk JH, O’Leary KJ, Fudala
MJ, McGaghie WC. Mastery learning of
thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med. 2008;3(1):48–54
Barsuk JH, McGaghie WC, Cohen ER, O’Leary
KJ, Wayne DB. Simulation-based mastery
learning reduces complications during central venous catheter insertion in a medical
intensive care unit. Crit Care Med. 2009;37
(10):2697–2701
Barsuk JH, Cohen ER, Feinglass J, McGaghie
WC, Wayne DB. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med. 2009;169
(15):1420–1423
Draycott T, Sibanda T, Owen L, et al. Does training in obstetric emergencies improve neonatal outcome? BJOG. 2006;113(2):177–182
Draycott TJ, Crofts JF, Ash JP, et al. Improving neonatal outcome through practical shoulder dystocia training. Obstet
Gynecol. 2008;112(1):14–20
McGaghie WC, Issenberg SB, Cohen ER,
Barsuk JH, Wayne DB. Does simulationbased medical education with deliberate
practice yield better results than traditional clinical education? A meta-analytic
comparative review of the evidence. Acad
Med. 2011;86(6):706–711
Ericsson KA. Deliberate practice and the
acquisition and maintenance of expert performance in medicine and related domains.
Acad Med. 2004;79(suppl 10):S70–S81
32. Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med. 2008;15(11):988–994
33. Airasian PWBB, Carroll JB. Mastery Learning: Theory and Practice. New York, NY:
Holt, Rinehart, and Winston; 1971
34. Gaies MG, Morris SA, Hafler JP, et al.
Reforming procedural skills training for pediatric residents: a randomized, interventional trial. Pediatrics. 2009;124(2):610–619
35. Kessler DO, Auerbach M, Pusic M, Tunik MG,
Foltin JC. A randomized trial of simulationbased deliberate practice for infant lumbar
puncture skills. Simul Healthc. 2011;6(4):
197–203
36. International Network for Simulation-Based
Pediatric Innovation, Research and Education (INSPIRE). Available at: www.InspireSim.com. Accessed November 1, 2012
37. Ellenby MS, Tegtmeyer K, Lai S, Braner DAV.
Videos in clinical medicine. Lumbar puncture. N Engl J Med. 2006;355(13):e12
38. Auerbach M, Chang T, Krantz A, et al. Infant
lumbar puncture: POISE pediatric procedure video. Available at: https://www.
mededportal.org/publication/8339.
Accessed January 14, 2013
39. Faul, F., Erdfelder, E., Lang, A.-G. & Buchner,
A. (2007). G*Power 3: A flexible statistical
power analysis program for the social,
behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191
40. Nigrovic LE, Kuppermann N, Neuman MI.
Risk factors for traumatic or unsuccessful
lumbar punctures in children. Ann Emerg
Med. 2007;49(6):762–771
41. Glatstein MM, Zucker-Toledano M, Arik A,
Scolnik D, Oren A, Reif S. Incidence of
traumatic lumbar puncture: experience of
a large, tertiary care pediatric hospital.
Clin Pediatr (Phila). 2011;50(11):1005–1009
42. White ML, Jones R, Zinkan L, Tofil NM.
Transfer of simulated lumbar puncture
training to the clinical setting. Pediatr
Emerg Care. 2012;28(10):1009–1012
43. Patel J, Posencheg M, Ades A. Proficiency
and retention of neonatal resuscitation
skills by pediatric residents. Pediatrics.
2012;130(3):515–521
44. Stefanidis D, Korndorffer JR Jr, Sierra R,
Touchard C, Dunne JB, Scott DJ. Skill retention following proficiency-based laparoscopic simulator training. Surgery. 2005;
138(2):165–170
45. Mancini ME, Kaye W. The effect of time since
training on house officers’ retention of
cardiopulmonary resuscitation skills. Am J
Emerg Med. 1985;3(1):31–32
46. Kaczorowski J, Levitt C, Hammond M, et al.
Retention of neonatal resuscitation skills
and knowledge: a randomized controlled
trial. Fam Med. 1998;30(10):705–711
47. Kahn CE Jr, Ehlers KC, Wood BP. Radiologists’
preferences for just-in-time learning. J Digit
Imaging. 2006;19(3):202–206
48. Stoler GB, Johnston JR, Stevenson JA,
Suyama J. Preparing emergency personnel
in dialysis: a just-in-time training program
for additional staffing during disasters
[published online ahead of print April 28,
2011]. Disaster Med Public Health Prep.
DOI: 10.1001/dmp.2011.34
49. Cook DA, Hatala R, Brydges R, et al.
Technology-enhanced simulation for health
professions education: a systematic review
and meta-analysis. JAMA. 2011;306(9):978–
988
50. Garrood T, Iyer A, Gray K, et al. A structured
course teaching junior doctors invasive
medical procedures results in sustained
improvements in self-reported confidence.
Clin Med. 2010;10(5):464–467
(Continued from first page)
Address correspondence to David O. Kessler, MD, MSc, RDMS, Columbia University Medical Center/New York Presbyterian Morgan Stanley Children’s Hospital of New
York, 622 West 168th St, PH-137, New York, NY 10032. E-mail: [email protected]
PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).
Copyright © 2013 by the American Academy of Pediatrics
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This work was supported in part by the R Baby Foundation.
PEDIATRICS Volume 131, Number 3, March 2013
e819
Downloaded from by guest on July 12, 2017
APPENDIX Critical Skills Checklist
LP critical steps checklist
1. Planning insertion site:
Palpate iliac crest (ridge on hip area of model) and follow iliac crest landmarks medially to spine and palpate interspace.
2. Preparation:
Don gloves, open tray, select appropriate needle (22 G 3 1.5 inch), cleanse the area by making circular motions starting at the planned site with each of 3 swabs (or
use chlorhexedine scrub for 30 seconds), and prepare tubes.
3. Draping:
Place a sterile drape under the infant and another on top to create a sterile field.
4. Positioning:
Ask resident how they want you to hold the infant. Make sure they can instruct you with correct technique (not obstructing the airway).
5. Needle insertion:
At planned insertion site place the needle, bevel up, at the center of interspace in the midsagittal plane, perpendicular to the skin and parallel to the floor.
6. Needle advancement:
Slowly advance through the skin and soft tissue at a cephalad angle toward the umbilicus (teacher must identify umbilicus to the learner). Remove the stylet
when through the skin and advance slowly 1 to 2 mm at a time until fluid is noted or resistance is encountered.
6a. No fluid obtained after needle advancement:
If resistance is encountered, withdraw the needle but do not remove it from the skin, and re-evaluate the planes. Redirect the needle in a new plane, advancing
the needle slowly 1 to 2 mm at a time until fluid is noted or resistance is encountered.
7. Fluid acquisition:
Fill each of the tubes to ∼0.5 mL.
8. Needle removal:
Replace the stylet in the needle and withdraw the needle in 1 motion. Apply pressure to the insertion site with gauze. Safely discard the needle in a Sharps
container or the foam on the LP tray.
IV Critical Steps Checklist
1. Planning insertion site:
Apply the tourniquet and palpate the antecubital fossa for a vein.
2. Preparation:
Don gloves; select appropriate tourniquet, catheter, connector piece, flush, tape, and gauze. Cleanse the area with antiseptic solution.
3. Positioning:
Proper positioning of the arm for antecubital IV insertion approach.
5. Needle insertion:
At the planned insertion vein, place the needle into the skin with bevel up and the needle at an angle of ∼15 to 20 degrees.
6. Needle advancement:
Slowly advance the needle through the skin and soft tissue into the vein until flashback is noted. When flashback is noted, lower the angle of insertion closer to
the skin (0–10 degrees). Advance the catheter into the vein and retract needle. Apply pressure onto the vein to prevent efflux of blood.
6a. No fluid obtained after needle advancement:
If no flashback is noted, withdraw the catheter to the skin entry point and redirect the catheter toward the vein. When flashback is noted, advance the catheter
into the vein and retract needle. Apply pressure onto the vein to prevent efflux of blood. DO NOT reinsert needle into catheter once it has been withdrawn because
of risks of shearing and thrombus.
7. Flush and secure:
Connect the catheter hub to the flush/connector piece. Continue to hold pressure proximal to the point of insertion and remove the tourniquet. Flush the catheter
with saline while palpating the entry point for flow or infiltration. Secure the catheter and connector to the skin.
8. Needle removal:
Discard the needle into a Sharps container.
e820
KESSLER et al
Downloaded from by guest on July 12, 2017
Interns' Success With Clinical Procedures in Infants After Simulation Training
David O. Kessler, Grace Arteaga, Kevin Ching, Laura Haubner, Gunjan Kamdar,
Amanda Krantz, Julie Lindower, Michael Miller, Matei Petrescu, Martin V. Pusic,
Joshua Rocker, Nikhil Shah, Christopher Strother, Lindsey Tilt, Eric R. Weinberg,
Todd P. Chang, Daniel M. Fein and Marc Auerbach
Pediatrics 2013;131;e811; originally published online February 25, 2013;
DOI: 10.1542/peds.2012-0607
Updated Information &
Services
including high resolution figures, can be found at:
/content/131/3/e811.full.html
References
This article cites 36 articles, 5 of which can be accessed free
at:
/content/131/3/e811.full.html#ref-list-1
Citations
This article has been cited by 8 HighWire-hosted articles:
/content/131/3/e811.full.html#related-urls
Post-Publication
Peer Reviews (P3Rs)
2 P3Rs have been posted to this article
/cgi/eletters/131/3/e811
Permissions & Licensing
Information about reproducing this article in parts (figures,
tables) or in its entirety can be found online at:
/site/misc/Permissions.xhtml
Reprints
Information about ordering reprints can be found online:
/site/misc/reprints.xhtml
PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly
publication, it has been published continuously since 1948. PEDIATRICS is owned, published,
and trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk
Grove Village, Illinois, 60007. Copyright © 2013 by the American Academy of Pediatrics. All
rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.
Downloaded from by guest on July 12, 2017
Interns' Success With Clinical Procedures in Infants After Simulation Training
David O. Kessler, Grace Arteaga, Kevin Ching, Laura Haubner, Gunjan Kamdar,
Amanda Krantz, Julie Lindower, Michael Miller, Matei Petrescu, Martin V. Pusic,
Joshua Rocker, Nikhil Shah, Christopher Strother, Lindsey Tilt, Eric R. Weinberg,
Todd P. Chang, Daniel M. Fein and Marc Auerbach
Pediatrics 2013;131;e811; originally published online February 25, 2013;
DOI: 10.1542/peds.2012-0607
The online version of this article, along with updated information and services, is
located on the World Wide Web at:
/content/131/3/e811.full.html
PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly
publication, it has been published continuously since 1948. PEDIATRICS is owned,
published, and trademarked by the American Academy of Pediatrics, 141 Northwest Point
Boulevard, Elk Grove Village, Illinois, 60007. Copyright © 2013 by the American Academy
of Pediatrics. All rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.
Downloaded from by guest on July 12, 2017