-
.-.
S
“?‘!‘
#{149}‘,,“
.:
4;5:,1
.
‘
CLIN. CHEM. 36/9, 1602-1604
The Pursuit
of Quality
(1990)
in Clinical
Laboratory
In this issue Westgard
and Burnett
(1) and Fraser et al.
(2) present
two thought-provoking
approaches
to the derivation
of the maximum
error
allowable
in an analytical
process. Fraser
et al. focus on allowable
analytical
error
(AAE) in serial measurements,
and Westgard
and Burnett
examine
the AAE from a quality-control
perspective.
Although the two views initially
appear divergent,
they are
complementary
and demonstrate
that many previously
published
estimates
of A.AE are too accepting
of current
analytical
performance.
Analytical
error can be thought
of as a combination
of
two types of error-inaccuracy
and imprecision.
The inaccuracy of an analytical
method
is determined
by analyzing
a series of patients’
specimens
by the method
in question
and by an alternative
established
method.
The resulting
method-comparison
data are evaluated
with
regression
analysis
to derive
a measure
of the constant
and proportional
errors
(y-intercept
and slope, respectively).
Use of
the y-intercept
and slope allows calculation
of the inaccuracy at any concentration.
Estimates
of imprecision
are
determined
from the replicate
analysis of several patients’
specimens
and control samples over at least 20 days. The
magnitudes
of the inaccuracy
and imprecision
are them
compared
with the AAE to determine
acceptability
of the
analytical
method.
Inaccuracy
and imprecision
can be evaluated
either
separately
or together,
as a total
error term
(3). Different
strategies
have been used to define AAE, including
the use
of multiples
of the reference
range, physician
interviews,
state-of-the-art
precision,
consensus
conferences,
and derivation
of formulas
based
on interand intra-individual
variation.
These
approaches
can produce
very divergent
values for AAE.
Physician
interviews
and the study
of inter- and mtraindividual
variation
account for most of the AAE estimates
published
in the scientific
literature.
Elion-Gerritzen
(4)
provided the best perspective
on the use of physician
interviews to establish
AAE. Through
a series of interviews
with
European
and North
American
physicians,
she demonstrated that the test usage dictates
the magnitude
of AAE.
She showed that physicians
would tolerate only very small
amounts
of analytical
error
when they used their
own
reference
ranges
to classify
a patient’s
results
as normal
or
abnormal.
This observation
has been ignored
by most instrument
manufacturers,
because
near-zero
imprecisions
have been largely
unattainable.
Moreover,
Elion-Gerritzen
showed that
significantly
larger
error was tolerated
in
situations
where
serial
results
were being
evaluated
for
significant
changes.
Skendzel
has published
the two most
comprehensive
assessments
of A.AE based on physician
interviews
(5, 6), but his rather tolerant
AAE values have
been criticized
by both Elion-Gerritzen
(7) and Fraser (8).
Cotlove
was one of the first to base AAE
values
on
biological
variation.
He suggested
that
the interand
intra-individual
CVs could be used to calculate
an optimal
analytical
CVa
(9):
1602
CLINICAL
CHEMISTRY,
Vol.36,
No. 9, 1990
Analyses
CVa
=
0.5
X
E(CVintra.individual)2
For monitoring
serial
mal CVa is reduced to:
CVa
These
=
0.5
results
X
+
from
(CVinter.indivjduai)2]V2
one patient,
the opti.
CVintra.indivjdual
two formulas
have formed the basis for statementi
analytical
goal setting by both the 1976 College
of American
Pathologists
Aspen Conference
(10) and the
1978 World Association
of Societies
of Pathology
Subcom.
mittee
on Analytical
Goals in Clinical
Chemistry
(11).
Fraser
has been instrumental
in developing
analytics]
goals based on physiological
variation,
and has even de
rived allowable
error goals for therapeutic
drug monitoring
(12) and common
hematology
tests (13). As a rule, his AAE
goals have been far tighter
than those of Skendzel.
All analytical
goals should
be compared
with what
it
currently
achievable.
Gilbert
(14) derived
estimates
ol
state-of-the-art
precision
from summaries
of the College d
American
Pathologists’
survey
data and then compared
these with
analytical
goals. Others,
including
Ross and
Fraser
(15), have derived
estimates
of state-of-the-art
pre.
cision
of large
numbers
of chemistry
analyzers
from the
statistical
analysis
of regional
quality-control
data.
Fraser’s
latest work (2) involves the use of two factors tc
obtain
the AAE for detecting
significant
changes
in labo.
ratory
test results:
(a) physiological
variation
and (b) phy.
sician-derived
AAE.
It is Fraser’s
thesis that, because
physicians’
assessment
of AAE incorporates
both physiological variation
and analytical
variation,
the actual analytical
component
of AAE can be obtained
from the statistical difference
of physiological
variation
and physician
AAE.
Fraser
et al. present
several
examples
in which
physiological
variation
is subtracted
from Skendzel’s
AAE
to provide
values
for AAE
that compare
favorably
wi
state-of-the-art
performance.
Their approach
reconciles
th
differences
between
Skendzel’s
work and Fraser’s
earlie
work.
Although
the paper
of Fraser
et al. presents
a use
approach
to the derivation
of analytical
goals, it should no
be considered
the ultimate
approach.
It still depends o
clinical
scenarios
in which an initial
analyte
concentratio
is presented
and a second,
clinically
different,
value
i
requested.
The difference
between
the initial
and secon
values
varies
with the numerical
difference
between
th
initial
value and upper (or lower) reference
value.
Secon
this approach
assumes
that quality-control
systems
perfect
and will detect the first occasion
that a clinicall
important
error occurs.
Westgard
and Burnett
(1) show that
moderate-sO
analytical
errors, e.g., shifts as great as 2 to 3 s (standar
deviations)
or a doubling
or even tripling
of the rando
error, are not readily
detected
and may require
multipl
analytical
runs before being discovered.
On the other han
regarding
larger
errors
can be detected with almost a 100% certainty.
It is advantageous
to reduce the imprecision
of a method so
that the magnitude
of the AAE
becomes equal to the
magnitude
of statistical
error that can be easily detectable.
Westgard
and Barry
(16) have provided
a formula
for
transforming
AAE into estimates
of critical random error
and critical systematic
error, estimates
of the size of error
that would result in a significant
proportion
of clinical
misclassifications.
If the critical
error
is large (e.g., 4 s,
implying
a precise assay), then simple
control
rules with
expanded
control
limits can detect these errors. In a recent
publication,
Westgard
and colleagues
have shown
that
quality-control
rules such as the 1358 can be used for many
analytes
measured
on the Hitachi
737 (17). Statistically
large, but clinically
unimportant
errors can be tolerated
with such a quality-control
system.
Clinically
important
errors can be detected
with a very high probability.
Carey
and I performed
a similar
critical
error analysis
with the
Du Pont aca and Kodak
Ektachem
and showed that clinically derived
quality-control
procedures
can be used for
many analytes
measured
by these instruments
(18).
Even with these highly
precise systems,
for some analytes the critical
errors are still relatively
small, i.e., <2 s
to 3 s. Even
complex
quality-control
procedures
cannot
detect the presence
of analytical
error in these analytes
before clinically
incorrect
data are produced.
In this issue
Westgard
and Burnett
(1) demonstrate
that, for qualitycontrol
systems
to be most effective,
the values
for AAE
recommended
by Skendzel
and others
should
be halved.
With such reductions
in AAE, the precision
of quantifying
certain
analytes,
e.g., calcium
and chloride,
would
no
longer be borderline
but rather would be unacceptable.
If
manufacturers
could decrease the imprecision
of such analytes,
the laboratory
could consistently
produce excellent
data without
exerting
extraordinary
effort.
There
is an intricate
relationship
between
analytical
performance
and physician
requirements
for analytical
accuracy.
As analytical
imprecision
decreases,
the expectations of the physician may change, as illustrated
in the
following
example:
For the past seven years the physicians
at Park Nicollet Medical
Center
have been using hemoglobin A1 (HbA1)
results
measured
by high-performance
liquid chromatography
(HPLC).
The standard
deviation
of
the HPLC assay at the upper range of normal is approximately 0.1%, compared
with 0.35% for an affinity chromatography
assay. Our endocrinologists
define good glycemic
ontrol as having
HbA1 between
the upper
limit of normal
6%) and 1-1.5%
above
normal
(19). On at least two
casions,
our endocrinologists
have noted systematic
inreases
in the HbA1 values of multiple
patients.
(Not only
o physicians
monitor
individual
patients,
they also subonsciously
or consciously group individual
patients’ data.)
sulting
investigations
revealed
significant
variations
tween analytical
columns.
We have modified our qualityontrol
practices
and now analyze
10 previously
analyzed
atients’
specimens
on new analytical
columns
before
puting the columns
into service.
Columns
for which results
e biased
by more than 0.3% from those of the current
olumn
are classified
as unacceptable.
A subsequent
study
of allowable
analytical
error
in
A1 analyses
confirmed
our physicians’
low tolerance
for
alytical
error
in this assay (20). We interviewed
14
ndocrinologists,
14 family practitioners,
15 internists,
16
iabetes
nurse educators,
and 30 type I and 20 type II
iabetics. All subjects stated their upper desirable
limit for
HbA1
and the value beyond which therapeutic
intervention was required.
All subjects also listed the value of
HbA1 that they considered too high and a HbA1 value that
indicated
significant
improvement
in a patient’s
condition.
Differences
between
the paired HbAIC values were used to
estimate
the AAE.
The endocrinologists
demanded
the
smallest
AAE, a median
value of 0.2% at the upper desirable HbA1 value of 7%. Because
these AAE values correspond to total imprecision
plus bias, they may be converted
to standard
deviations
by dividing
by 2 or 3 if there is
negligible
bias in the assay.
Our AAE
values
based
on physician
interviews
are
significantly
smaller
than those of Hyltoft
Petersen
et al.,
quoted
in Fraser’s
current
article
in this issue (optimal
analytical
standard
deviation
of 0.2% and a maximum
acceptable
standard
deviation
of 0.5%). These differences
arise from at least two related causes. First, the patients at
our institution
appear to have better glycemic control than
did Hyltoft
Petersen’s
cohort; second, we use the HPLC
assay of HbA1, whereas Hyltoft
Petersen
probably
uses the
affinity
chromatography
assay. Possibly
with more precise
HbA1
assays and with intensive
insulin
treatment
regimens, Hyltoft
Petersen’s
AAE might approach
ours.
Improved
analytical
precision
greatly
simplifies
the task
of the laboratorian.
It also allows the clinician
to focus on
patient
treatment
and not to be distracted
by analytical
variation.
The differences
between
today’s
analytical
systems are becoming
blurred.
Even the use of complex
checklists of quality
characteristics
for instrument
evaluation
may not simplify
instrument
selection.
Perhaps
two other
quality
characteristics
should be added to the checklist:
the
critical random error and the critical
systematic
error. These
numbers
will more readily indicate
the analytically
superior
system. The inclusion
of these critical
errors might provide
new incentives
for industry
to improve
borderline
assays.
Such improvements
in analytical
quality
will not occur
overnight;
they will require
the persistence
of the manufacturer in the design and implementation
stages. They will
also require
a sophisticated
laboratory
community
that
recognizes
quality
and is willing
to pay for it.
The editorial
assistance
of Ernest
J. Kiser
is gratefully
acknowl-
edged.
References
1. Westgard
JO, Burnett
RW. Precision
requirements
for costeffective
operation
of analytical
processes.
Clin
Chem
1990;36:1629-32.
2. Fraser CG, Hyltoft Petersen
P, Larsen ML. Setting analytical
goals
for random
analytical
error in specific clinical
monitoring
situations.
Clin Chem 1990;36:1625-8.
3. Westgard JO, Carey RN, Wold S. Criteria for judging precision
and accuracy
in method development
and evaluation.
Clin Chem
1974;20:825-33.
4. Elion-Gerritzen
WE. Analytic
precision
in clinical
chemistry
and medical decisions.
Am J Clin Pathol
1980;73:183-95.
5. Skendzel
LP. How physicians
use laboratory
tests. J Am Med
Assoc 1978;239: 1077-80.
6. Skendzel LP, Barnett RN, Platt R. Medically
useful criteria for
analytic
performance
of laboratory
tests.
Am J Clin Pathol
1985;83:200-5.
7. Elion-Gerritzen
WE. How physicians
use laboratory
tests. J Am
Med Assoc 1978;240:2246.
8. Fraser CG. Better criteria for desirable
laboratory
performance
exist. Am J Clin Pathol 1986;88:251.
9. Cotlove E, Harris EK, Williams
GZ. Biological
and analytic
components
of variation in long-term
studies of serum constituents
in normal
subjects.
Ill. Physiological
and medical
implications.
Clin Chem 1970;16:1028-32.
CLINICAL
CHEMISTRY,
Vol. 36, No. 9, 1990
1603
10. Elevitch
FR, ed. Analytical
goals in clinical chemistry.
Proc,
CAP Aspen Conference,
1976. Skokie,
IL: College of American
Pathologists,
1977.
11. World Association of Societies of Pathology.
Proceedings
of the
Subcommittee
on Analytical
Goals in Clinical
Chemistry.
Analytical goals in clinical chemistry:
their relationship
to medical
care.
Am J Clin
12. Fraser
Pathol
1979;71:624-30.
CG. Desirable standards
of performance
for therapeutic
drug monitoring.
Clin Chem 1987;33:1436-8.
13. Fraser CG. Desirable performance
standards
for hematology
tests: a proposal. Am J Cliii Pathol 1987;88:667-9.
14. Gilbert RK. Progress and analytic goals in clinical chemistry.
Am J Clin Pathol 1975;63:960-73.
15. RossJW,
Fraser MD. Analytical
clinical chemistry precisionstate of the art for thirty-one
analytes.
Am J Cliii Pathol
1980;74:521-30.
16. Westgard
JO, Barry PL. Cost-effective
quality control: managing the quality and productivity
of analytical
processes.
Washington, DC: AACC
Press,
1986.
1604
CLINICAL
CHEMISTRY,
Vol. 36, No. 9, 1990
17. Koch DD, Oryall JJ, Quam EF, et al. Selection
of medically
useful quality-control
procedures
for individual tests done in a
multitest
analytical
system.
Clin Chem 1990;36:230-3.
18 Cembrowski
GS, Carey RN. Considerations
for the implemen.
tation of clinically
derived quality control procedures.
Lab Med
1989;20:400-5.
19. Hollander P. Approaches to the treatment
of type I diabetes.
Lab Med August
1990 (in press).
20. Hollander P, Weiss 5, Cembrowski
G, Mama R. Determination of the accuracy
required
for hemoglobin A1 testing:
perceptions
and realities
[Abstract].
Diabetes
1990;39(Suppl
1):175A.
George
Clinical
Labomtory
Park Nicol let Medical
5000 West 39 St.
Minneapolis,
MN
55416
Center
S. Cembrowski