AJCP / Special Article
Senior Pathology Resident In-Service Examination
Scores Correlate With Outcomes of the American Board
of Pathology Certifying Examinations
Henry M. Rinder, MD,1 Margaret M. Grimes, MD, MEd,2 Jay Wagner, MBA, MLS(ASCP),3
and Betsy D. Bennett, MD, PhD4; for the RISE Committee of the American Society for Clinical
Pathology and the American Board of Pathology
Key Words: In-training examination; In-service examination; Board certification; Medical knowledge; Board passing rates; Pathology
graduate medical education
DOI: 10.1309/AJCPA7O4BBUGLSWW
Abstract
The Resident In-Service Examination (RISE)
addresses 1 area of the Accreditation Council for
Graduate Medical Education Outcome Project;
RISE results demonstrate progressive attainment of
pathology knowledge during training. We compared
RISE scores with primary pathology board certification
success for residents graduating in 2008 and 2009.
Overall RISE and nearly all sectional scores in
anatomic and clinical pathology were significantly
higher for residents passing all certifying examinations
at the first attempt vs residents who failed any
examination. The risk of failing increased with each
lower quartile of overall RISE score, such that 34%
(2009) and 54% (2008) of residents in the lowest
quartile failed at least 1 certifying examination. Two
thirds of graduating residents with lowest quartile
scores had a similar quartile ranking in the previous
RISE, identifying them as at risk. Residents passing the
American Board of Pathology certifying examinations
have a higher level of medical knowledge in general
and specific pathology disciplines as assessed by senior
RISE scores.
The American Society for Clinical Pathology (ASCP)
Resident In-Service Examination (RISE) has been given
annually since 1983; 100% of United States residency
programs currently participate, and Canadian and other
international pathology training programs have participated.
All residents in all years of pathology residency training take
the examination, which is available during a 2-week period
in early spring. There are more than 350 multiple-choice
questions in a “one-best-answer” format, and each year’s
RISE is unique.1 The RISE is developed in collaboration with
the Academy of Clinical Laboratory Physicians and Scientists,
the Association of Directors of Anatomic and Surgical
Pathology, the American Association of Blood Banks, the
American Association of Neuropathology, the American
Pathology Foundation, the American Society for Apheresis
Graduate Medical Education Resources Subcommittee, the
National Association of Medical Examiners, the Society for
Hematopathology, and the Society for Pediatric Pathology.1
The usefulness of the RISE has expanded from selfassessment after the Accreditation Council for Graduate
Medical Education introduced its Outcome Project in 2000,2
and in 2003, ASCP converted the RISE to a Web-based
format. The focus of accreditation by the Accreditation
Council for Graduate Medical Education is now centered
on a training program’s accomplishments, including board
passing rates and in-service examination scores. Thus, since
2003, the RISE has been used by program directors as
an assessment tool for medical knowledge gained during
pathology residency training.1 The RISE tests medical
knowledge in anatomic pathology (AP), clinical pathology
(CP), and special topics (ST) common to AP and CP. AP/CP
residents take all 3 sections, whereas AP-only residents take
© American Society for Clinical Pathology
Am J Clin Pathol 2011;136:499-506
499
DOI: 10.1309/AJCPA7O4BBUGLSWW
499
499
Rinder et al / RISE Correlates With Board Exam Success
the AP and ST sections and CP-only residents take the CP and
ST sections. The 3 sections of the RISE comprise 10 content
areas. The AP section is allotted 2 hours and 15 minutes for
3 content areas: surgical pathology (20% of total content),
cytopathology (12%), and autopsy/forensic pathology (10%).
The CP section is allotted 2 hours and 45 minutes and consists
of 4 content areas: clinical chemistry (11%), hematology
(11%), microbiology (11%), and transfusion medicine
(11%). The ST section is allotted 1 hour and consists of
3 content areas common to AP and CP resident training:
hematopathology (4%), laboratory administration (5%), and
molecular pathology (5%).
The examination is scored using a Rasch norm-referenced
model of analysis, and the measurement is then converted to
a scaled score. This scaled score is a linear transformation
of the raw measure that is comparable across years and
examinees with 999 being the highest and 100 the lowest
reportable score. There are no passing scores. Scores are
based only on the sections attempted. Residents receive
scores and percentile equivalents in comparison with their
peer training group (postgraduate year 1, 2, 3, or 4) for each
of the 10 content areas and for the total examination. A list
of descriptors for each question (which details the specific
knowledge being tested) and a reference are provided along
with each resident’s corresponding list of missed questions.
Program directors receive each individual resident’s report
plus programmatic mean scores, standard deviations, trending
reports, and percentile equivalents for each content area
and for each year of training, all with comparisons with the
national peer group.
The RISE has consistently demonstrated the overall
progression of attainment of medical knowledge during
pathology residency training similar to other disciplines,3 but
RISE results have not been formally compared with future
accomplishments of pathology trainees, eg, success in attaining
board certification.4 The spring RISE is given in March
to ensure that results are available to graduating residents
taking their primary pathology certifying examinations in the
following May or June, partly to highlight any knowledge gaps
for concentrated study, but also to reassure graduating seniors
before the boards that their RISE-assessed progress is keeping
pace with nationally measured peers. This reassurance would
be reinforced if RISE results were shown to correlate with
success on the pathology certifying examinations.
The American Board of Pathology (ABP) administers
2 primary certification examinations, 1 in AP and 1 in
CP. Applicants may certify in both (AP/CP certification)
or in either area alone (AP only or CP only). Applicants
choosing to certify in AP/CP may take both examinations
during the same administration or may take the AP and CP
examinations at separate administrations. AP/CP candidates
are not certified until both parts of the examination have been
500
500
Am J Clin Pathol 2011;136:499-506
DOI: 10.1309/AJCPA7O4BBUGLSWW
passed and all other requirements have been met. All AP and
CP examinations are graded in the same manner and have the
same pass-fail standard, regardless of whether the candidate is
taking one or both tests.
The AP examination consists of 324 questions and is
separated into 2 parts, written and practical (images, glass
slides, and virtual slides). A candidate must pass both parts
of the examination at the same administration. The written
portion (2 hours and 15 minutes) consists of 141 questions,
while the microscopy portion has 75 slides (50 glass and
25 virtual) for which 3 hours and 30 minutes are allotted.
The image portion has 108 questions to be completed in
1 hour and 45 minutes. The CP examination is composed
of 350 questions. The written portion (130 questions) is
administered over 2 hours. The practical portion is divided
into an image section (110 questions in 2 hours) and a
practical examination (eg, interpretive questions and cell
panels) with 110 questions given in 3 hours and 15 minutes.
As with the AP examination, a candidate must pass the
written and practical portions of the CP examination at the
same administration. All questions are in the one-best-answer
format, and both certifying examinations are graded using the
criterion-reference method. Scores are scaled, and a score of
500 is required to pass. Candidates who pass the examination
are not given their scores. Candidates who fail are given
their scores and a breakdown of performance on the various
portions of the examination.
We correlated RISE scores for graduating residents in
2008 and 2009 with the corresponding outcomes of their first
attempt at passing the primary certifying examinations in AP
and/or CP.
Materials and Methods
This study was judged by the investigators to be exempt
from institutional review board review based on federal
regulation 45 CFR 46.101(b)(2), which covers research
involving the use of educational tests. In compliance with that
regulation, none of the recorded or published information can
be linked to any human subject, directly or through identifiers.
The database of 2008 and 2009 individual ASCP RISE
results for graduating residents, ie, the data from each
resident’s final RISE examination, was encoded in a passwordprotected format and delivered to 1 trustee of the ABP. These
data comprised residents completing AP/CP, AP-only, and
CP-only training. Results from the ABP database were added
as “pass” or “fail” for all first-time attempts at the AP and/
or CP certifying examinations in 2008 and 2009. Residents
who elected to take their AP and CP certifying examinations
at different times during this period were included. Second or
later attempts at the certifying examinations were excluded.
© American Society for Clinical Pathology
AJCP / Special Article
The final database had all identifying information erased
permanently before saving in Microsoft Excel (Microsoft,
Redmond, WA) format with password protection.
Certifying examination outcomes were grouped in 2
ways: (1) passing all examinations attempted vs failing any
certifying examination and (2) passing vs failing the specific
AP or CP certifying examination. RISE results were similarly
grouped as follows: (1) the overall score for the entire RISE
examination and (2) the individual RISE section scores specific
for the AP (autopsy/forensics, cytopathology, and surgical
pathology) and CP (chemistry, hematology, microbiology,
and transfusion medicine) disciplines. Comparisons between
groups were examined for statistical significance (P < .05)
using the unpaired t test. Besides mean scores, overall RISE
results were also analyzed by their percentile ranking to
determine the risk for failing a certifying examination by
overall RISE quartiles. To correlate a single comparison of
year-year lowest quartile performance, a blinded post hoc
analysis was performed to examine the 2007 and 2008 RISE
results for all 2008 graduating residents.
1❚. A similar difference in the overall 2008 and 2009 RISE
scores was noted when the data were separated by residents
who specifically failed the AP (60 and 61 points, respectively)
or the CP (54 and 61 points, respectively) component of the
certifying examination (Table 1). Although mean scores
within passing and failing groups differed somewhat from
year to year, the percentile for that mean value was remarkably
similar between 2008 and 2009, further supporting the RISE
emphasis on percentiles for comparing residents with peers.
When mean sectional 2008 RISE scores were examined
for the 3 AP-specific disciplines ❚Figure 1A❚, the mean scores
for the cytopathology and surgical pathology sections were
significantly higher for the residents who passed the AP
certifying examination vs residents who failed. By contrast,
there was no significant difference in autopsy/forensic
pathology scores between the pass and fail groups. The mean
difference in cytopathology scores (65 points) for passing vs
failing the AP certifying examination was similar to the mean
difference in the overall RISE (55 points), but mean surgical
pathology section scores were 100 points higher for passing
residents than for residents who failed (Figure 1A).
Similar differences in mean scores were noted for 2009
❚Figure 1B❚, with surgical pathology again having the largest
difference between groups, followed by cytopathology and,
finally, autopsy/forensic pathology demonstrating a marginal
(albeit significant) difference. The mean surgical pathology
and cytopathology scores in 2008 and 2009 for the resident
group failing the certifying examination were consistently
below the range of 2nd quartile scores for passing residents,
suggesting that residents in the bottom quartile of scores are at
greater risk for failing (Table 1 and ❚Table 2❚).
Similarly, residents who passed vs failed the CP certifying
examination demonstrated consistently significant differences
in all 4 CP-specific disciplines of the 2008 ❚Figure 2A❚ and
2009 ❚Figure 2B❚ RISE. The differences between the mean
scores for the 4 sections ranged from 47 to 77 points.
Results
A total of 454 residents graduating in 2008 and 424
graduating in 2009 were first-time takers of the 2008 and/or
2009 ABP certifying examinations in AP and/or CP and had
evaluable data from their senior RISE in the spring of 2008
or 2009.
Mean RISE Scores and Certifying Examination Outcome
Mean overall RISE scores were significantly higher (by
55 and 59 points) for graduating residents in 2008 and 2009,
respectively, who subsequently passed all of their attempted
ABP examinations (including AP and CP, AP only, and CP
only) vs residents who failed any ABP examination ❚Table
❚Table 1❚
RISE Scores for Graduating Residents and Their Corresponding ABP Certifying Examination Results*
ABP Certifying
Examination Outcome
Passed all examinations
Failed any examination
AP examination
Passed
Failed
CP examination
Passed
Failed
2008 RISE Score
Percentile for
the Mean Score
2009 RISE Score
Percentile for
the Mean Score
545 ± 39† (364 [80])
490 ± 35 (90 [20])
65th
20th
506 ± 40† (382 [90])
447 ± 25 (42 [10])
65th
20th
540 ± 41† (392 [91])
480 ± 36 (40 [9])
60th
15th
504 ± 41† (365 [96])
443 ± 28 (17 [4])
60th
20th
545 ± 39† (352 [84])
491 ± 36 (67 [16])
65th
20th
508 ± 38† (351 [92])
447 ± 26 (30 [8])
65th
20th
ABP, American Board of Pathology; AP, anatomic pathology; CP, clinical pathology; RISE, Resident In-Service Examination.
* Data are given as mean ± SD (number [percentage] of residents).
† P < .001 compared with failed.
© American Society for Clinical Pathology
Am J Clin Pathol 2011;136:499-506
501
DOI: 10.1309/AJCPA7O4BBUGLSWW
501
501
Rinder et al / RISE Correlates With Board Exam Success
A
1,000
Q1
Min
Median
Max
Q3
RISE Score
800
600
400
200
Passed
Failed
Passed
Failed Passed
Failed
Autopsy Forensic
Cytopathology
Surgical
Pathology
Pathology
2008 RISE AP Sections
B
1,000
Q1
Min
Median
Max
Q3
RISE Score
800
600
400
200
Passed
Failed
Passed
Failed Passed
Failed
Autopsy Forensic
Cytopathology
Surgical
Pathology
Pathology
2009 RISE AP Sections
❚Figure 1❚ Mean sectional 2008 (A) and 2009 (B) Resident In-Service Examination (RISE) scores for the anatomic pathology
(AP)-specific disciplines for residents who passed vs failed the AP certifying examination. For the 2008 and 2009 RISE,
passing residents had significantly higher cytopathology and surgical pathology scores (P < .001); by contrast, autopsy/forensic
pathology scores did not differ between groups (P = .066) in 2008 and, although significantly different in 2009 (P = .03), were
much less disparate than the scores for cytopathology and surgical pathology. Max, maximum; Min, minimum; Q1, quartile 1;
Q3, quartile 3.
502
502
Am J Clin Pathol 2011;136:499-506
DOI: 10.1309/AJCPA7O4BBUGLSWW
© American Society for Clinical Pathology
AJCP / Special Article
❚Table 2❚
Quartiles for Overall 2008 and 2009 RISE Scores for Graduating Residents vs Their Rate of Passing All 2008 and 2009 American
Board of Pathology Certifying Examinations
2008 Graduates (n = 454)
2009 Graduates (n = 424)
RISE Score Quartile
Examination Pass Rate (%)
RISE Score Range
Examination Pass Rate (%)
RISE Score Range
1st
2nd
3rd
4th
97
92
86
46
≥565
532-564
505-531
<505
100
99
94
66
≥533
500-532
473-499
<472
RISE, Resident In-Service Examination.
Analogous to the RISE surgical pathology and
cytopathology scores, the mean CP section scores for the
resident group that failed the certifying examination were
consistently below the range of 2nd quartile scores for passing
residents. Only the microbiology mean score for failing
residents in 2008 approximated the lower scoring range of the
2nd quartile for passing residents.
Passing Rates for the ABP Certifying Examinations and
Overall RISE Percentile Results
When individual residents were ranked according to
their overall 2008 RISE score quartile, the rate of passing
all certifying examinations declined with each lower quartile
ranking (Table 2), such that fewer than half of 2008 graduating
residents in the bottom quartile passed all of their certifying
examinations. In a post hoc analysis, we found that slightly
more than two thirds (69%) of 2008 graduating residents who
were in the lowest quartile for the 2008 RISE were also in the
lowest quartile for their 2007 RISE scores.
A decline in passing rates with lower RISE quartiles
was also noted for 2009 graduating seniors (Table 2), but the
certifying examination passing rates in each quartile were
somewhat higher than those for 2008. This difference between
graduating years 2008 and 2009 is important to note in light
of the difference in passing rates for the ABP certifying
examination in CP during those 2 years: 82% in 2009
compared with 75% in 2008; the AP passing rate was similar
for both years. Still, the data demonstrate that graduating
residents scoring in the bottom quartile of the RISE are at
particular risk of failing a certifying examination; based on
our post hoc analysis, it may be possible to identify residents
who fall into this at-risk group based on their previous “junior
year” RISE percentile rankings.
Discussion
There is a body of work that has assessed whether
the results of in-training examinations of residents’
medical knowledge predict success on the corresponding
certifying examination,5-18 ranging from single department
reviews6,11,12,16,18 to studies conducted by the board or
specialty society.5,14 Most have limited their correlation to
the in-training score with the written portion of a multipart
certification exam,8,13,17 whereas some have incorporated
training demographics as additional potential predictors of
success.10,13,15 Of the 14 studies we identified in the literature,
a significant correlation between in-service scores and
certifying examination results was found in all but 1.18 This
correlation tended to be strongest for the composite or overall
in-service score9,15 and weaker for sectional or specialty
scores within the in-service test.12,15 When the predictive
value of in-training examination results for certifying success
was studied, there was clear evidence that the strongest values
were found in later years of training.5,6,12 For these reasons,
the current study was designed to examine the correlation for
overall and sectional RISE scores for graduating residents on
their first attempt at either or both of the 2 primary pathology
certifying examinations.
Our analysis of RISE scores for 2008 and 2009 graduating
residents and the outcomes of their 2008 and 2009 ABP
certifying examinations revealed a clear difference in both
total and nearly all of the discipline-specific RISE section
mean scores for residents who passed vs failed any primary
pathology certifying examination. Overall RISE and most
sectional mean scores were between 45 and 75 points
higher for residents passing the certifying examinations.
These findings confirm that residents passing their certifying
examination(s) have a higher level of medical knowledge
in general pathology and within the AP- and CP-specific
disciplines of the RISE.
We cannot rule out the possibility that the relatively
consistent margins in the overall and most sectional scores
between board outcome groups may additionally reflect
general test-taking skills.19 However, RISE scores in the
autopsy/forensic pathology section of AP did not differ
significantly between 2008 graduates (and were very similar
for 2009 graduates) who passed vs failed their first attempt at
the AP certifying examination. The training experience of AP/
CP and AP-only residents may be most uniform in this area
© American Society for Clinical Pathology
Am J Clin Pathol 2011;136:499-506
503
DOI: 10.1309/AJCPA7O4BBUGLSWW
503
503
Rinder et al / RISE Correlates With Board Exam Success
A
1,000
Q1
Min
Median
Max
Q3
RISE Score
800
600
400
200
Passed
Failed
Clinical
Chemistry
Passed
Failed Passed
Hematology
Failed Passed
Microbiology
Failed
Transfusion
Medicine
2008 RISE CP Sections
B
1,000
Q1
Min
Median
Max
Q3
RISE Score
800
600
400
200
Passed
Failed
Clinical
Chemistry
Passed
Failed Passed
Hematology
Failed Passed
Microbiology
Failed
Transfusion
Medicine
2009 RISE CP Sections
❚Figure 2❚ Mean sectional 2008 (A) and 2009 (B) Resident In-Service Examination (RISE) scores for the clinical pathology
(CP)-specific disciplines for residents who passed vs failed the CP certifying examination. For the 2008 and 2009 RISE, passing
residents had significantly higher clinical chemistry, hematology, microbiology, and transfusion medicine scores (P < .001). Max,
maximum; Min, minimum; Q1, quartile 1; Q3, quartile 3.
504
504
Am J Clin Pathol 2011;136:499-506
DOI: 10.1309/AJCPA7O4BBUGLSWW
© American Society for Clinical Pathology
AJCP / Special Article
because of the ABP eligibility requirement of 50 autopsies
or more; the vast majority of residents have participated in
numbers of autopsies very close to the required number.
In addition, nearly all pathology training programs include
medical examiner experience or didactic forensics lectures
by a medical examiner; we speculate that the impact of
“interesting” and classic forensics images may leave a lasting
impression reflected in board performance.
By contrast, resident training in surgical and cytopathology
varies widely and is based solely on programmatic and
not ABP requirements. This variation may partly explain
why the surgical pathology and cytopathology AP sections
consistently demonstrated large and significant differences in
mean scores between residents passing and residents failing
the AP certifying examination; indeed, surgical pathology
demonstrated the largest gap between groups of all the
RISE subdisciplines. The number of surgical/cytopathology
specimens submitted on the ABP application varies by
several-fold among applicants, especially in cytopathology.
This range, when compared with the more uniform autopsy
experience, could account for the differences in surgical/
cytopathology performance compared with autopsy/forensic
pathology. Thus, a relatively poor RISE performance in the
surgical pathology and/or cytopathology sections (scores of
50-100 points lower than the mean for peer senior residents)
could be a signal of forthcoming difficulty with the AP
certifying examination.
Graduating residents who scored above the overall mean
for the senior RISE had very high passing rates for all
attempted certifying examinations. This result is not surprising
given the first-time passing rates of 85% for AP and 75% for
CP certifying examinations in 2008 and higher passing rates
in 2009. Passing rates for the certifying examinations declined
with each quartile of overall RISE scoring, especially when
RISE scores were less than the 50th percentile and even more
so for the bottom quartile of RISE results. This finding is
consistent with our data demonstrating that the mean overall
scores for failing residents (Table 1) fall into the bottom
quartile. The analysis of overall RISE scores from 2007 for
residents graduating in 2008 further suggest that significant
attention be given to residents who score in the bottom quartile
of the overall RISE in their last 2 years of training.
One important caveat for this study is that our RISE
results (except for the specific post hoc analysis) were taken
only from the senior examination, a snapshot of medical
knowledge just before graduation, rather than a longitudinal
assessment of RISE scoring. We chose this approach not only
because of previous studies5,6,12 correlating later years of
in-training results with board success but also because there
is wide variation between pathology training programs with
respect to overall exposure to each of the specific pathology
disciplines of AP and CP, as noted in the discussion of
surgical and cytopathology scores. Moreover, the timing
of those experiences between programs can be completely
disparate within the 3 or 4 years of residency training. Thus,
with respect to predicting board outcomes, we specifically
focused on graduating resident scores because the residents
were closest to fulfilling specific and general eligibility
requirements for the ABP certifying examination(s). Still,
junior residents should be examining their RISE scores within
the context of their individual program’s curriculum and also
with their national colleagues at a similar training level; for
the latter, particular attention should be given to residents
scoring in the lowest quartile overall and, perhaps, residents
with significantly lower surgical pathology and cytopathology
sectional scores.
Finally, we cannot rule out that resident study habits,
which are known to affect in-training examination outcomes,20
may have accelerated before the senior RISE in anticipation
of taking the certifying examinations. However, our finding of
low overall RISE quartile performance as an at-risk predictor
suggests that a consistent knowledge deficit is an important
target for training programs to identify. Future studies of RISE
results and certifying examination outcomes may yield more
specific targets for training improvements by prospectively
examining longitudinal RISE trends within individuals;
another prospective venue might include correlating RISE
scores and outcomes with specific training differences
between pathology residency programs. These studies could
add to the current data for identifying at-risk residents and
perhaps targeting the areas of medical knowledge where
focused training may improve outcomes in the ABP certifying
examinations.
From the 1Department of Laboratory Medicine, Yale University
School of Medicine, New Haven, CT; 2Department of Pathology,
Virginia Commonwealth University, Richmond; 3Examination
Development & Outcomes Assessment Products, American Society
for Clinical Pathology, Chicago, IL; and 4American Board of
Pathology, Tampa, FL.
Address reprint requests to Dr Rinder: Dept of Laboratory
Medicine, Yale University School of Medicine, PO Box 208035,
New Haven, CT 06520-8035.
References
1. McKenna BJ. The American Society for Clinical Pathology
Resident In-Service Examination: does resident performance
provide insight into the effectiveness of clinical pathology
education? Clin Lab Med. 2007;27:283-291.
2. Accreditation Council for Graduate Medical Education.
ACGME outcome project. http://www.acgme.org/outcome/.
Accessed April 22, 2011.
3. Garibaldi RA, Subhiyah R, Moore ME, et al. The in-training
examination in internal medicine: an analysis of resident
performance over time. Ann Intern Med. 2002;137:505-510.
© American Society for Clinical Pathology
Am J Clin Pathol 2011;136:499-506
505
DOI: 10.1309/AJCPA7O4BBUGLSWW
505
505
Rinder et al / RISE Correlates With Board Exam Success
4. Spellacy WN, Carlan SJ, McCarthy JM, et al. Prediction
of ABOG written examination performance from the thirdyear CREOG in-training examination results. J Reprod Med.
2006;51:621-622.
5. Althouse LA, McGuinness GA. The in-training
examination: an analysis of its predictive value on
performance on the general pediatrics certification
examination. J Pediatr. 2008;153:425-428.
6. Crawford CH III, Nyland J, Roberts CS, et al. Relationship
among United States Medical Licensing Step I, orthopedic
in-training, subjective clinical performance evaluations, and
American Board of Orthopedic Surgery examination scores:
a 12-year review of an orthopedic surgery residency program.
J Surg Educ. 2010;67:71-78.
7. Fish DE, Radfar-Baublitz L, Choi H, et al. Correlation
of standardized testing results with success on the 2001
American Board of Physical Medicine and Rehabilitation
Part I Board Certificate Examination. Am J Phys Med Rehabil.
2003;82:686-691.
8. Goodman JC, Juul D, Westmoreland B, et al. RITE
performance predicts outcome on the ABPN Part I
examination. Neurology. 2002;58:1144-1146.
9. Grossman RS, Fincher RM, Layne RD, et al. Validity of the
in-training examination for predicting American Board of
Internal Medicine certifying examination scores. J Gen Intern
Med. 1992;7:63-67.
10. Herndon JH, Allan BJ, Dyer G, et al. Predictors of success
on the American Board of Orthopaedic Surgery examination.
Clin Orthop Relat Res. 2009;467:2436-2445.
11. Johnson GA, Bloom JN, Szczotka-Flynn L, et al. A
comparative study of resident performance on standardized
training examinations and the American Board of
Ophthalmology written examination. Ophthalmology.
2010;117:2435-2439.
12. Juel VC, Johnston KC. Predict Resident Exam Performance
(PREP) study. Neurology. 2003;60:1385-1387.
506
506
Am J Clin Pathol 2011;136:499-506
DOI: 10.1309/AJCPA7O4BBUGLSWW
13. Juul D, Schniedman BS, Sexson SB, et al. Relationship
between Resident-In-Training Examination in psychiatry
and subsequent certification examination performances. Acad
Psychiatry. 2009;33:404-406.
14. McClintock JC, Gravlee GP. Predicting success on the
certification examinations of the American Board of
Anesthesiology. Anesthesiology. 2010;112:212-219.
15. Replogle WH, Johnson WD. Assessing the predictive value
of the American Board of Family Practice In-Training
Examination. Fam Med. 2004;36:185-188.
16. Shellito JL, Osland JS, Helmer SD, et al. American Board
of Surgery examinations: can we identify surgery residency
applicants and residents who will pass the examinations on
the first attempt? Am J Surg. 2010;199:216-222.
17. Webb LC, Juul D, Reynolds CF III, et al. How well does
the Psychiatry Residency In-Training Examination
predict performance on the American Board of Psychiatry
and Neurology Part I examination? Am J Psychiatry.
1996;153:831-832.
18. Withiam-Leitch M, Olawaiye A. Resident performance on
the in-training and board examinations in obstetrics and
gynecology: implications for the ACGME Outcome Project.
Teach Learn Med. 2008;20:136-142.
19. de Virgilio C, Stabile BE, Lewis RJ, et al. Significantly
improved American Board of Surgery in-training
examination scores associated with weekly assigned reading
and preparatory examinations. Arch Surg. 2003;138:11951197.
20. Derossis AM, Da Rosa D, Schwartz A, et al. Study habits
of surgery residents and performance on American Board of
Surgery in-training examinations. Am J Surg. 2004;188:230236.
© American Society for Clinical Pathology