Quality Assurance of Histopathologic Diagnoses:
A Prospective Audit of Three Thousand Cases
MYRON E. WHITEHEAD, M.D., JAMES E. FITZWATER, M.D., SHELDON K. LINDLEY, M.D.,
SUSAN B. KERN, M.D., RUDOLF C. ULIRSCH, M.D., AND WILLIAM F. WINECOFF III, M.D.
The purpose of this study was to examine a system of quality
assurance in which microscopic slides from all surgical cases
were subject to peer review before completion of the final pathology report. Three thousand surgical pathology cases were
evaluated and the diagnoses entered on preliminary diagnosis
sheets. Microscopic slides were reviewed and coded reviewer
opinions entered on photostatic copies of preliminary diagnosis
sheets. Reviewer comments were incorporated into the final pathology reports at the discretion of the responsible pathologist.
Two hundred thirty-three (7.8%) cases were identified in which
at least one reviewer disagreed with the preliminary diagnosis;
in 67 of the 233 cases (2.2%) the final diagnosis was modified
as a result of the review process. Changes in final diagnoses
were judged to be significant in 29 (0.96%) cases. These results
suggest that systematic review of surgical pathology diagnoses
prior to preparation of final pathology reports may be a useful
tool in assuring quality and consistency in diagnoses. (Key words:
Quality assurance; Quality control; Surgical pathology) Am J
Clin Pathol 1984; 81: 487-491
INTERPRETATION OF HISTOPATHOLOGIC CHANGES
by the surgical pathologist traditionally has been the standard against which most diagnostic and therapeutic interventions are judged. Quality of diagnoses has been
monitored by outside consultation, continuing education,
personal experience, and clinical observations of disease
progression.
Interest in more comprehensive approaches to quality
assurance in surgical pathology has been sparked by observing the advantages of such programs in the clinical
laboratory, interest of quality assurance procedures by
accrediting agencies, and increased numbers of malpractice suits involving surgical pathology and its subspecialties.
We describe a system of quality assurance in which we
subjected microscopic slides from all surgical cases to
peer review before completing the final pathology report.
We discuss our findings in 3,000 consecutively reviewed
surgical pathology cases and present the advantages and
disadvantages of this system.
Received June 29, 1983, received revised manuscript and accepted
for publication August 22, 1983.
The opinions and assertions contained herein are the private views
of the authors and are not to be construed as official or as reflecting the
views of the Department of the Army of the Department of Defense.
Address reprint requests to Technical Publications Editor, Letterman
Army Medical Center, Presidio of San Francisco, California 94129.
487
Department of Pathology, Letterman Army Medical Center,
Presidio of San Francisco, California
Materials and Methods
Microscopic slides obtained from all surgical pathology
cases during the period August 16, 1982, to January 28,
1983, were entered into the study. Of 3,569 cases accessioned, 3,000 (84%) had documentation of review by at
least one pathologist. On initial interpretation by the responsible pathologist (a senior resident in anatomic pathology with sign-out privileges or a staff pathologist), the
diagnoses were dictated and written on preliminary diagnosis sheets. On the following day, the microscopic
slides and preliminary diagnosis sheets were reviewed by
staff pathologists. Each reviewing pathologist made and
recorded an independent judgment regarding the accuracy
and completeness of the preliminary diagnosis and recorded his/her results on individual photostatic copies of
the preliminary diagnosis sheet. The reviewing pathologists examined all microscopic slides, using both doubleand single-headed microscopes, and indicated essential
concurrence with the responsible pathologist's provisional
diagnosis by printing "1A" next to the patient's name
on the preliminary diagnosis sheet. The responsible pathologist was available for comments, clarification, and
discussion.
All review results other than " 1 A" were tabulated as
"nonconcurrence." Cases in which one or more reviewing
pathologists nonconcured were designated as "nonconsensus." In cases of nonconcurrence, each reviewer was
Table 1. Cases Reviewed by Specific
Number of Reviewers
No. of
Reviewers
No. of Cases
Percentage of Total
Reviewed Cases
1
2
3
4
5
6
204
522
652
715
679
228
6.8%
17.4%
21.7%
23.8%
22.6%
7.6%
WHITEHEAD ET AL.
488
A.J.C.P. • April 1984
Table 2. Summary of Changes in Final Diagnosis as a Result of the Review Process
Biopsy Specimen
Number
1
2
3*
Skin
Soft tissue of shoulder
Uterus
4
5*7
Ovary
Prostate gland
Stomach
7t
Skin with clinical suspicion of
Hansen's Disease
Original Diagnosis
Benign lichenoid keratosis
Descriptive diagnosis
Endometrial adenocarcinoma invasion
1/3 of myometrium
Simple cysts
Well differentiated adenocarcinoma
Inflammatory changes. Cannot
exclude well differentiated
adenocarcinoma
Descriptive diagnosis
Epiglottic fold
Epithelial dysplasia
9
10
Soft tissue of knee
Endometrium
11
12*
Skin
Endometrium
13t
Skin
14
15
16
17*
18
19*t
Vas deferens
Skin
Skin
Appendix
Skin
Uterine contents
20
21*-t
Skin
Endometrium
Descriptive diagnosis
Secretory phase with apocrine
metaplasia
Actinic keratosis
Disordered proliferative endometrium
with cystic change and apocrine
metaplasia
Consultation needed prior to
completion of report
No significant pathologic change
Neurofibroma
Inflammation, granulation tissue
No significant pathologic changes
Adnexal tumor
Insufficient tissue submitted for
microscopic examination
Foreign body reaction
Adipose tissue present
22*
23
Endometrium
Stomach
Cystic hyperplasia
Acute inflammation and necrosis
24
25
26*
27
28
29
30
31t
Skin
Cervix
Skin
Skin
Uterus
Skin
Skin
Posterior fossa brain tumor
Basosquamous carcinoma
Koilocytotic change
Lentigo maligna
Neuroma
Adenomyosis
Basal cell carcinoma
Basal cell carcinoma
Acoustic neuroma
32*-t
Frontal lobe brain
Oligodendroglioma
33
34
35*
Colon
Soft tissue peritoneal cavity
Scalene lymph node
36
37
38
39
40
41*
42
Skin
Skin
Skin
Skin
Skin
Prostate gland
Esophagus
Tubular adenoma
Blood clot
Well-differentiated lymphocytic
lymphoma
Epidermal inclusion cyst
Inverted follicular keratosis
Descriptive diagnosis
Lichen nitidus
Atypical fibroxanthoma
Atypical hyperplasia
Invasive carcinoma
43
Skin
Solar elastosis and venous lake
Final Diagnosis
Descriptive diagnosis
Ganglion cyst
Endometrial adenocarcinoma invasion
1/2 of myometrium
Follicular cysts
Sent for consultation. (Consultant
opinion—"highly suspicious" for
adenocarcinoma. Suggest additional
biopsy.)
Inflammatory changes
Descriptive diagnosis. Added comment
about absence of features suggestive
of leprosy
Invasive squamous cell carcinoma
(carcinoma also present on other
biopsy fragments collected at the
same time from different anatomic
site, e.g., pharyngeal wall)
Chronic synovitis
Secretory phase
Solar elastosis
Disordered endometrium with atypical
glands
Nodular melanoma
Mild chronic inflammation
Scar with foreign body reaction
Thrombosed dermal vessel
Acute appendicitis
Chondroid syringoma
Products of conception chorionic villi
present
Ruptured epidermal inclusion cyst
Comment about presence of adipose
tissue deleted
Menstrual phase
Comment added documenting yeast
colonization
Basal cell carcinoma
Koilocytotic atypia
Dermatofibroma
Hypertrophic scar
No adenomyosis
Actinic keratosis
Actinic keratosis
Acoustic neuroma with comment
about small size of biopsy specimen
Oligodendroglioma with comment
about extreme cellularity and high
grade of the lesion
Villotubular adenoma
Blood clot with granulation tissue
Mixed small and large cell lymphoma,
diffuse
Pilar cyst
Actinic keratosis
Actinic keratosis
Descriptive diagnosis
Consistent with atypical fibroxanthoma
Well differentiated adenocarcinoma
Invasive carcinoma and incidental
leiomyoma
Solar elastosis
489
QUALITY ASSURANCE OF PATHOLOGIC DIAGNOSES
Vol. 81 - N o . 4
Table 2. (Continued)
Number
Biopsy Specimen
Original Diagnosis
Final Diagnosis
44*
45*
46*
47*"f
Endometrium
Endometrium
Cervix
Uterus
Basalis endometrium
Atypical hyperplasia
Condyloma latum
Chronic cervicitis
48
49*
50
Skin
Urinary bladder
Endometrium
Molluscum contagiosum
No significant pathologic change
Proliferative phase
51*
52*
53
54*
55
56*7
Breast
Myometrium
Skin
Endometrium
Vulva
Liver
Fibrocystic disease
Leiomyoma
Sebaceous tumor
Atypical hyperplasia
Moderate dysplasia
Metastatic adenocarcinoma
57*
Uterine contents
58*7
Lung
59*7
Rectum
Necrotic decidua and chorionic villi
seen
Squamous cell carcinoma 7/18 nodes
positive
Adenocarcinoma
60
61t
62*7
Skin
Endometrium
Cervix
Seborrheic keratosis
Progesterone effect
Moderate dysplasia
63*
64*
Endometrium
Colon
65*
66
67
Skin
Skin
Bladder
Benign endometrium
Moderately differentiated
adenocarcinoma
Basal cell carcinoma margins free
Basal cell carcinoma
Epithelial atypia
Undifferentiated carcinoma
Well-differentiated adenocarcinoma
Condyloma acuminatum
Sent for consultation. Mesonephric
rests present
Verruca vulgaris
Severe epithelial atypia
Proliferative phase with
pseudoactinomycotic radiate granules
Gynecomastia
Leiomyoma and endometriosis
Clear cell hidradenoma
Endometrial adenocarcinoma
Mild dysplasia
Adenocarcinoma with comment about
primary versus metastasis
Necrotic decidua with no chorionic villi
seen
Added comment about lymph node
involvement by local extension
Poorly differentiated adenocarcinoma
with comment regarding question of
primary or secondary lesion
Skin tag
Proliferative phase with comment
Dysplasia with comment that biopsy
was inadequate for grading of
dysplastic process
Chronic endometritis
Poorly differentiated adenocarcinoma
• Indicates a case in which Ihe change in final diagnosis was defined as "significant.'
asked to note the discrepancy and using letter codes A,
B, or C, to designate if the discrepancy had "no patient
care implication" (A); "possible patient care implication"
(B); or "definite patient care implication" (C). Each reviewer was asked to note the time spent for review at the
bottom of the preliminary diagnosis sheet. At the conclusion of the study, one of us (MW) examined copies
of all final surgical pathology reports from nonconsensus
cases. Final reports were compared with discrepancies
noted by reviewers. Nonconsensus reports were divided
into two groups: (1) Those exhibiting evidence of change
infinaldiagnosis corresponding to discrepancies identified
by the peer review process; and (2) those that showed no
evidence of change in the final pathologic diagnosis.
Changes made in the final reports were judged to be "significant" or "not significant."
In all, 112 review sessions were held, with an average
of 27 cases reviewed per session. Of the 569 cases not
reviewed, thefinalsurgical pathology reports for 243 cases
were constructed on gross findings only, i.e., no microscopic slides were made. The remaining 326 nonreviewed
Basal cell carcinoma margin involved
Bowenoid actinic keratosis
Qualitative changes in grade of atypia
X2
t Indicates cases in which comments or pathologist work-up were modified.
cases required special processing, e.g., decalcification,
submission of additional material for histologic staining,
special stains, electron microscopy, and documentation
of formal review was not accomplished. These cases generally were discussed widely within the department, and
many were presented during formal conferences, but because of the delay in diagnosis formal staff review was
not documented. Obviously, under ideal conditions, every
effort should be made to include such cases in any formal
review process.
A senior resident was the responsible pathologist for
1,608/3,569 (45%) of the accessioned cases, and a staff
pathologist was responsible for the remaining 2,161 cases.
Reviewer data were kept in separate notebooks and were
available in the surgical pathology area of the laboratory.
Results
The frequency distribution of cases reviewed by specified number of reviewers is given in Table 1. Of the 3,000
cases with documented review, 233 (7.8%) were noncon-
490
WHITEHEAD ET
A.J.C.P. 'April 1984
AL.
Table 3. Reviewer Specific Performance Data
Reviewer
No.
Reviewer
Experience
(yrs)
Nonconsensus
Rate
1
2
3
4
5
6
7
2
18
1
3
3
3.3%
4.8%
5.1%
4.1%
5.3%
3.2%
Nonconsensus
Cases Identified
(n = 233)
51
84
76
75
97
58
sensus cases. Final reports were changed in 67 of the 233
nonconsensus reports; 29 of these changes were judged
to be "significant." Thus, "significant" changes represented 12.4% of the nonconsensus cases and 0.96% of all
cases reviewed. Changes in final pathology reports corresponding to discrepancies noted by reviewers are listed
in Table 2. Individual reviewer performance is documented in Table 3.
The marginal utility of reviewers was assessed by observing the number and rate at which nonconsensus cases,
cases with changed final surgical pathology reports, and
cases in whichfinalreports were both changed and judged
to be significant, were identified by specified numbers of
reviewers (Table 4). In generating these data, reviewers
were ordered randomly; when fewer reviewers than specified actually reviewed a case, fewer actual viewer opinions, as available, were ordered randomly and used.
Reviewers spent approximately one hour per session
for review (82% of sessions for which time was noted
were between 40-80 minutes in length). The mean time
for review was 2.2 minutes per case.
Discussion
Quality assurance of individual surgical pathologic interpretations has been described previously in the literature.1"7 However, these reports deal with review of only
a portion of surgical materials accessioned over any given
period of time,3 retrospective review of previously diagnosed material,4 or review of quality assurance material
submitted for the purpose of quality control.5 One article6
briefly describes a prospective review process, but results
Table 4. Cases Identified by Specified Numbers
of Randomly Ordered Reviewer Evaluations
No. of
Reviewers
Nonconsensus
Cases
Identified
(n = 233)
Changed
Final Reports
Identified
(n = 67)
"Significant"
Changed Reports
Identified
(n = 29)
125 (53.6%)
168(72.1%)
202 (86.7%)
45 (67.2%)
56 (83.6%)
63 (94.0%)
19 (65.5%)
24 (82.8%)
27 (93.1%)
(22%)
(36%)
(33%)
(32%)
(42%)
(25%)
Changed Final
Reports Identified
(n = 67)
20
32
25
26
46
15
(30%)
(48%)
(37%)
(39%)
(69%)
(22%)
"Significant" Changed
Reports Identified
(n = 29)
8 (28%)
14 (48%)
7 (24%)
9(31%)
24 (83%)
5(17%)
are not quantitative. We are aware of no descriptions of
a prospective system of peer review in which a large percentage of surgical cases accessioned over a given period
of time have been subject to peer review and results quantitatively described.
The prospective, systematic audit of diagnoses described
in this report yielded a nonconsensus rate of 7.8%. As
shown in Table 3, nonconcurrence rates among individual
reviewers varied little, ranging from 3.2% to 5.3%. The
peer review process resulted in changed diagnoses in 67
reports, i.e., the diagnosis in most cases identified as
"nonconsensus" were not altered. Moreover, most of the
modifications made in diagnoses of nonconsensus cases
were of minor clinical significance.
Factors affecting the likelihood that the responsible
pathologist would modify the final diagnoses in accordance with peer commentary were not studied, although
many variables play a role in the process. For example,
the number of nonconcurrent peers, personal feelings between nonconcurrent peer and the responsible pathologist,
type of discrepancy noted (i.e., continuous vs. discrete),
degree of confidence in the original diagnosis, and clinical
significance of the nonconcurrent opinion probably all
play a role in the decision. Disagreements along continuous diagnostic categories (i.e., grading of cervical dysplasia) were associated less often with altered final diagnoses than discrete ("I did not see that piece" or "I
did not think of that") disagreements.
Although we included all available staff pathologists in
the review process, more than one-half of the nonconsensus cases and more than two-thirds of the cases with
altered diagnoses were identified by a single randomly
selected reviewer. Two randomly selected reviewers identified 72% of nonconsensus cases and 84% of cases with
modified final diagnoses.
Since no attempt was made to document total professional time expended on each case, the fraction of total
professional time used by the review process is unknown;
however, mean time was slightly over 2 minutes per case
per reviewer. Assessment by nonconcurring reviewers of
the significance of the disagreement was related to the
probability of the nonconcurrence resulting in a modified
final diagnosis. Discrepancies rated as Category A resulted
Vol. 81 -No. 4
QUALITY ASSURANCE OF PATHOLOGIC DIAGNOSES
in changing the final diagnosis in 20% of cases; those
rated as Category B resulted in changes of 40% of cases;
and those rated as Category C resulted in changes in 64%
of cases.
We believe that our system of peer review offers several
advantages: (1) The process is prospective with respect
to patient care, i.e., input by peer reviewers is made before
the final report leaves the laboratory. (2) Responsibility
for each case remains with the responsible pathologist;
peer consultations are incorporated into the final report
at the discretion of the responsible pathologist. (3) Senior
residents in anatomic pathology are allowed the flexibility
of sign-out privileges with the knowledge that cases are
subject to the staff-review process. When all staff pathologists at an institution are subjected to an identical
review process, the review is not looked upon by senior
residents as a lack of staff confidence in their ability. (4)
Essentially all cases are reviewed. We are not aware of
any case selection criteria that have been shown to be
efficacious in identifying cases for which review was most
beneficial. It seems likely that criteria could be found to
identify cases for which the professional time spent in
peer review might lead to maximum improvement in
diagnostic results. In large surgical pathology laboratories
or laboratories unable to expend the professional time
needed to review all cases, such criteria would provide a
better alternative to selection review than random selection. (5) Existence of the peer review process itself probably
has beneficial effects on the quality of diagnoses by sharpening diagnostic criteria, gaining uniformity, and assuring
that responsible pathologists are prepared to defend all
diagnoses rendered.
Several disadvantages require comment: (1) Professional time required for review is substantial and may be
translated into increased turnaround time for surgical
pathology reports. Appropriate scheduling of review can
minimize any increase in turnaround time; our clinical
colleagues have accepted well the required small increase
in turnaround time necessary to conduct our review. (2)
The system, as described, deals only with the interpre-
491
tation of histopathologic changes and does not take into
account other variables impacting on the quality of service
in surgical pathology. (3) Since, in our study, peer comments were incorporated into the final surgical pathology
report at the discretion of the responsible pathologist, it
is possible that diagnoses with significant nonconcurrence
among reviewers were not altered. (4) The legal implications of such a formal peer review system are not clear,
i.e., to what degree are reviewers accountable for incorrect
diagnoses missed during the review process? (5) Resident
pathologists may not develop skills as readily in determining which cases should be submitted for consultant
opinions.
In summary, we have described a prospective peer review system for quality assurance of histopathologic interpretations. The essential elements of this system have
been used at our institution since 1979. Although such
a system is perhaps not applicable to all surgical pathology
practitioners, our experience with it has been professionally beneficial, and we are encouraged that a comprehensive meaningful audit mechanism can be devised that
will benefit pathologists, clinical colleagues, and patients.
Acknowledgment. The idea and organization of the authors' quality
assurance program was formulated by William R. Starke, M.D. The
authors gratefully acknowledge his leadership and foresight in this regard.
References
1. Dudley HAF: Audit and the pathologist. Proc R Soc Med 1975;
68:634-637
2. Langley FA: Quality control in histopathology and diagnostic cytology. Histopathology 1978; 2:3-18
3. Murthy MSN, Derman H: Quality assurance in surgical pathology—
personal and peer assessment. Am Soc Clin Pathol 1981; 75:462466
4. Owen DA, Tighe JR: Quality evaluation in histopathology. Br Med
J 1975; 1:149-150
5. Penner DW: Quality control and quality evaluation in histopathology
and cytology. Pathol Annu 1973; 8:1-9
6. Wilkerson JA: New developments in surgical pathology IV. Quality
Control. Ala J Med Soc 1982; 19:199-202
7. Wright EA: Quality control in histopathology. Proc R Soc Med
1975; 68:619-622