Designing QC Rules in the Presence of Laboratory Bias: Should a QC Rule be Centered on the Instrument’s Mean or the Reference Mean? Designing QC Rules in the Presence of Laboratory
Should
a QC Rule be
Centered
on thePlano,
Instrument’s
Mean or the Reference Mean? C. A. Parvin, Bias:
L. Kuchipudi,
J. Yundt-Pacheco,
Bio-Rad
Laboratories,
TX
C. A. Parvin, L. Kuchipudi, J. Yundt-Pacheco, Bio-Rad Laboratories, Plano, TX
Abstract
Introduction
Reference Mean
E(NU)
Max
AUC
Instrument Mean
0.015E(NU) 0.053
Max
AUC
Reference Mean
0.008E(NU) 0.046
Efficacy of Using Instrument Mean vs. Reference Mean
Bias(%)
Instrument 0.5
1.6
Sigma
5.9
CV(%)
1.6
Bias(%)
1.0
Sigma
5.6
Max
0.038
AUC
0.123
Max
0.014
AUC
0.088
1.6
1.6
0.5
1.5
5.9
5.3
0.015
0.091
0.053
0.302
0.008
0.032
0.046
0.192
1.6
2.0
1.0
0.5
5.6
4.8
0.038
0.404
0.123
2.237
0.014
0.244
0.088
2.107
1.6
2.0
1.5
1.0
5.3
4.5
0.091
0.762
0.302
3.670
0.032
0.355
0.192
3.049
2.0
2.0
0.5
1.5
4.8
4.3
0.404
1.420
2.237
6.627
0.244
0.587
2.107
4.441
2.0
2.4
1.0
0.5
4.5
4.0
0.762
2.898
3.670
21.694
0.355
1.919
3.049
21.020
2.0
2.4
1.5
1.0
4.3
3.8
1.420
4.804
6.627
29.398
0.587
2.489
4.441
25.816
2.4
2.4
0.5
1.5
4.0
3.5
2.898
7.965
21.694
43.597
1.919
3.583
21.020
29.196
2.4
1.0
3.8
4.804
29.398
2.489
25.816
2.4
1.5
3.5
7.965
43.597
3.583
29.196
Conclusion:
Using the reference mean for the QC rule reduces the risk of reporting unreliable
patient results. The reduction in the worst-case E(NU) ranges from 34% to 65% when
Conclusion:
the QC rule mean is set to the reference mean instead of the instrument’s mean for the
casesthe
studied.
The reduction
all rule
out-of-control
conditions
by the
Using
reference
mean foracross
the QC
reduces the
risk of characterized
reporting unreliable
) curves ranges
3% to 37%
thefrom
reference
mean
used.
area under
theThe
E(NUreduction
ranges
34% to
65%iswhen
patient
results.
in the from
worst-case
E(NU) when
In QC
general,
the improvement
in QC rule
the reference
the
rule mean
is set to the reference
meanperformance
instead of theusing
instrument’s
mean formean
the
increases
as
the
ratio
of
instrument
bias
to
imprecision
increases.
cases studied. The reduction across all out-of-control conditions characterized by the
area under the E(NU) curves ranges from 3% to 37% when the reference mean is used.
In general, the improvement in QC rule performance using the reference mean
increases as the ratio of instrument bias to imprecision increases.
Methods
• Allowable total error of 10% (TEa = 10%)
Sigma = (TEa – Bias) / CV
Instrument
The following combinations of instrument bias and imprecision are evaluated:
CV(%)
Bias(%)
Results
The probability of rejection when SE = 0 (false rejection rate) increases if the QC mean
is set to a value different from the instrument’s mean.
“s” for the QC Rule When the QC Mean is Set to the Reference Mean
Figure 1 shows the probability of error detection for the 1:3s/2:2s/R:4s multirule when
The
rejection when
SE =there
0 (false
rejection
rate)
if the
QC mean
“s” probability
is set to theofinstrument’s
SD and
is a 1%
bias in
the increases
instrument’s
mean.
is set to a value different from the instrument’s mean.
Figure
shows the
probability
of error
detection
for the
1:3s/2:2s/R:4s
multirule
Fig. 1: 1
Probability
of QC
rule rejection
for the
1:3s/2:2s/R:4s
multirule
when the QC
mean iswhen
set to
“s”
set to the instrument’s
SD and
is a 1% mean
bias in(green
the instrument’s
mean.
theisinstrument’s
mean (blue curve)
or there
the reference
curve), assuming
instrument
1 rule rejection for the 1:3s/2:2s/R:4s multirule when the QC mean is set to
Fig. 1: Probability of QC
the instrument’s mean (blue curve) or the reference mean (green curve), assuming instrument
CV = 2% and instrument bias = 1%.
0.8
1
Sigma
1.6
5.9
0.5
Instrument
1.6
1.0
5.6
1.6
1.5
5.3
CV(%)
Bias(%)
Sigma
2.0
0.5
4.8
1.6
5.9
0.5
2.0
1.0
4.5
1.6
1.0
5.6
2.0
4.3
1.5
1.6
1.5
5.3
2.4
0.5
4.0
2.0
0.5
4.8
2.4
1.0
3.8
2.0
1.0
4.5
2.4
1.5
3.5
2.0
4.3
1.5
2.4
0.5
4.0
2.4
1.0
3.8 of error detection (P )
The 1:3s/2:2s/R:4s QC multirule is assumed. The probability
ed
2.4
1.5using numerical3.5
for the rule was derived
mathematically
integration and validated
by computer simulation.
The
QC multirule
The is
probability
error detection
(Ped)
The1:3s/2:2s/R:4s
standard deviation
(s) usedisinassumed.
the multirule
set to theofinstrument’s
standard
for
the rulewhen
was the
derived
mathematically
using numerical
validated
deviation
QC rule
uses the instrument’s
mean asintegration
the QC ruleand
mean.
by computer simulation.
When the QC rule uses the reference mean as the QC rule mean, “s” is determined
The
standard
deviationrate
(s) of
used
the
multirule
is on
setthe
to the
instrument’s
so the
false rejection
the in
QC
rule
centered
reference
mean isstandard
equal to
deviation
the rate
QC rule
uses
as the QC rule
mean.
the false when
rejection
of the
QCthe
ruleinstrument’s
centered onmean
the instrument’s
mean.
When
the QC rule
uses the
reference patient
mean as
the QC
rule mean,
is determined
The expected
number
of unreliable
results
reported
due “s”
to the
occurrence
so
rejection rate
of the E(N
QC Urule
on the
reference using
mean the
is equal
to
), forcentered
each case
is computed
formula
of the
an false
out-of-control
condition,
the
falseinrejection
rate of of
the aQC
rule centered
the instrument’s
mean.
given
the appendix
recent
paper byonParvin
(Parvin C.A.
“Assessing the
impact
of the number
frequency
quality control
the quality
of the
reported
patient
The
expected
of of
unreliable
patient testing
results on
reported
due to
occurrence
Clinical Chemistry
2008
), for each case is computed using the formula
ofresults,”
an out-of-control
condition,
E(N;U54:2049-54).
given in the appendix of a recent paper by Parvin (Parvin C.A. “Assessing the
impact of the frequency of quality control testing on the quality of reported patient
results,” Clinical Chemistry 2008 ; 54:2049-54).
Probability
Probability
0.2
0.4
Instrument
−10
−5
0
10
15
0
−15
−10
probability
(SE =−5
0)
Bias(%) QC
Sigma
QCSet
Rule
Neg.
SE
Pos. SE
Rule Mean
toCV(%)
Instrument
Mean
0.0022
1.6
0.0154
0.5
5.9
1.6
Max E(NU0.0381
)
0.0008
1.6Instrument
1.0
5.6
1.6
0
15
is 0.0097
for5 both 10
QC rules.
SE (%) mean the rule has greater power for
When the QC rule mean is set to the reference
detecting an out-of-control condition in the same direction as the instrument’s bias, but
The
false
rejection
probabilityan(SE
= 0) is 0.0097condition
for both QC
rules.
lower
power
for detecting
out-of-control
in the
opposite direction of the
instrument’s
to the
QC rule
that ismean
centered
thehas
instrument’s
mean.for
When
the QCbias
rulecompared
mean is set
to the
reference
the on
rule
greater power
detecting
an out-of-control
condition
in mean
the same
direction
but2.
around the
reference
which
occursas
at the
SE =instrument’s
-1% in Figs.bias,
1 and
ΔPE is symmetric
lower power for detecting an out-of-control condition in the opposite direction of the
A positive systematic
out-of-control
condition
a larger
in themean.
likelihood
instrument’s
bias compared
to the QC
rule thatproduces
is centered
on theincrease
instrument’s
of producing unreliable patient results than a negative systematic out-of-control
around
the reference
meanofwhich
occurs1%
at SE
= -1% in bias.
Figs. 1 and 2.
ΔP
E is symmetric
condition
of the same
magnitude
because
the positive
instrument
0.6
0.8
0.4
0.6
for various
instrument
bias and imprecision.
The green
curves
Fig.each
3: E(Ncase
U) curves
For
studied,
E(NU)combinations
is computedoffrom
the appropriate
QC rule power
function
show
U) when the QC rule is centered on the reference mean. The blue curves show E(NU)
and
ΔPE(N
E curves.
when the QC rule is centered on the instrument mean.
0.2
0.4
Sigma=5.9
CV=1.6%
Sigma=5.6
CV=1.6% Bias=1.5%The
Sigma=5.3
curvesBias=0.5%
for various
combinations
ofBias=1%
instrument
bias and imprecision.
green curves
Fig. 3: E(NU) CV=1.6%
0.1
0.1
show E(N0.1
U) when the QC rule is centered on the reference mean. The blue curves show E(NU)
when the0.08
QC rule is centered on the instrument
mean.
0.08
0.08
−10
−5
0
5
10
15
SE (%)
0
−15 to the
−10instrument’s
−5
0mean: 5
is set
10
15
SE= (%)
• The false rejection probability (SE
0) is 0.0097
• The power curve is symmetric around SE = 0
If the QC mean is set to the instrument’s mean:
If the QC mean is set to the reference mean:
•• The
Thefalse
falserejection
rejectionprobability
probability(SE
(SE==0)0)isis0.0097
0.0194
•• The
curveofiserror
symmetric
around
SEsymmetric
=0
Thepower
probability
detection
is not
around SE = 0
If the QC
mean
is
set
to
the
reference
mean:
– The probability of a QC rule rejection is increased for out-of-control conditions
• Thethat
falseadd
rejection
(SE
= 0) is 0.0194
to the probability
instrument’s
bias
• The
probability
of error
is not symmetric
around
SE = 0
– The
probability
of a detection
QC rule rejection
is decreased
for out-of-control
conditions
– The
of a the
QCinstrument’s
rule rejectionbias
is increased for out-of-control conditions
thatprobability
subtract from
that add to the instrument’s bias
In order to match false rejection rates, the value for “s” in the 1:3s/2:2s/R:4s
– Thewhen
probability
of amean
QC rule
rejection
is decreased
for out-of-control
conditions
multirule
the QC
is set
to the
reference mean
is increased
until the
that subtract
from the instrument’s
bias
probability
of false rejection
equals the probability
of false rejection for the multirule
when
the
QC
mean
is
set
to
the
instrument’s
mean.
In order to match false rejection rates, the value for “s” in the 1:3s/2:2s/R:4s
multirule
whenshown
the QC
set to“s”
thetoreference
mean isgives
increased
until theof
For the case
in mean
Fig. 1, issetting
a CV of 2.165%
a probability
probability
of false
probability
false
rejectionprobability
for the multirule
false rejection
(SErejection
= 0) of equals
0.0097 the
matching
the of
false
rejection
for the
when
the
QC
mean
is
set
to
the
instrument’s
mean.
multirule when the QC mean is set to the instrument’s mean (fig. 2).
For
case shown
in Fig. 1,
setting
“s” toresult
a CV contains
of 2.165%
gives a probability
of
Thethe
change
in probability
that
a patient
measurement
error that
false
rejection
(SEa =function
0) of 0.0097
thesystematic
false rejection
probabilitycondition,
for the
of the matching
size of the
out-of-control
exceeds
TEa as
multirule
when
the QCinmean
is set to the instrument’s mean (fig. 2).
SE, is also
displayed
Fig. 2.
The change in probability that a patient result contains measurement error that
exceeds TEa as a function of the size of the systematic out-of-control condition,
SE, is also displayed in Fig. 2.
0.06CV=1.6% Bias=0.5% Sigma=5.9
0.1
0.04
0.08
0.02
0.06
0
0.04
−15 −10 −5
0
5 10 15
0.02
CV=2% Bias=0.5% Sigma=4.8
0
1.5
−15 −10 −5
1
0
5
10
15
0.06 CV=1.6% Bias=1% Sigma=5.6
0.1
0.04
0.08
0.02
0.06
0
0.04
−15 −10 −5
0
5 10 15
0.02
0
1.5
CV=2% Bias=1% Sigma=4.5
−15 −10 −5
1
CV=2% Bias=0.5% Sigma=4.8
0
5
10
15
0.06CV=1.6% Bias=1.5% Sigma=5.3
0.1
0.04
0.08
0.02
0.06
0
0.04
−15 −10 −5
0
5 10 15
0.02
0
1.5
10
10
10
0.5
−0.5
0.5
−0.5
−15 −10 −5
6
9
6
0
5
10
15
CV=2.4% Bias=0.5% Sigma=4.0
0
5
10
15
3CV=2.4% Bias=0.5% Sigma=4.0
0
5
10
15
CV=2.4% Bias=1% Sigma=3.8
−15 −10 −5
6
6
0
5
10
15
3 CV=2.4% Bias=1% Sigma=3.8
−6
0 −15 −10 −5
0
5
SE (%)
0
5
SE (%)
10
15
10
15
−6
−15 −10 −5
−15 −10 −5
−0.5 9
10
15
0
5
10
15
CV=2.4% Bias=1.5% Sigma=3.5
−15 −10 −5
6
6
0
5
10
15
3CV=2.4% Bias=1.5% Sigma=3.5
0
−3
3
−6
0 −15 −10 −5
−3
0
9
0
−3
3
−3
3
−6
−15 −10 −5
−15 −10 −5
−0.5 9
9
0
−3
0
5
1.5
0.5
0.5
−0.5
−0.59
0
CV=2% Bias=1.5% Sigma=4.3
1.5
0.5
−15 −10 −5
0
CV=2% Bias=1.5% Sigma=4.3
−15 −10 −5
1
CV=2% Bias=1% Sigma=4.5
1.5
0.5
0
5
SE (%)
0
5
SE (%)
10
15
−6
0 −15 −10 −5
−3
10
15
−6
−15 −10 −5
Max E(NU)
CV(%)
AE(N
positive
condition
produces
a larger
increase
in the likelihood
QC Rules out-of-control
Centered on the
Instrument
Mean
versus
the Reference
Mean
U) forsystematic
of producing unreliable patient results than a negative systematic out-of-control
)
is
computed
from
the
appropriate
QC
rule
power function
For
each
case
studied,
E(N
U
condition of the same magnitude because of the positive 1% instrument bias.
and ΔPE curves.
E(NU) for QC Rules Centered on the Instrument Mean versus the Reference Mean
0
0.2 −15
If the QC mean
5
SE (%)
The false rejection
If the QC rule mean is set to the instrument’s mean the risk of an unreliable patient
result in the presence of a systematic out-of-control condition in the same direction as
the instrument’s bias is greater than the risk if the out-of-control condition is in the
of is
theset
instrument’s
bias.
Ifopposite
the QC direction
rule mean
to the instrument’s
mean the risk of an unreliable patient
result
therule
presence
a systematic
out-of-control
the same patient
direction
as
If the inQC
mean isofset
to the reference
mean thecondition
risk of anin unreliable
result
the
instrument’s
bias is greater
thansystematic
the risk ifout-of-control
the out-of-control
condition
is same.
in the
associated
with positive
or negative
conditions
are the
opposite direction of the instrument’s bias.
Note that a systematic out-of-control condition in the direction opposite to the
Ifinstrument’s
the QC rule bias
meancan
is set
reference in
mean
the risk ofnumber
an unreliable
patient patient
result
leadtotothe
a decrease
the expected
of unreliable
associated
positive
or negative
systematic
conditions are the same.
results duewith
to the
out-of-control
condition
(E(Nout-of-control
U) < 0). The effect is more pronounced if
the
QC
rule
mean
is
set
to
the
reference
mean.
Note that a systematic out-of-control condition in the direction opposite to the
instrument’s
biasvalue
can lead
to Ua) and
decrease
in the
expected
number
of unreliable patient
the area
under
the E(N
The maximum
for E(N
U) curve were determined in
The effect is more pronounced if
results
due to
condition
U) < 0).tables:
each case
andthe
theout-of-control
results are shown
in the(E(N
following
the QC rule mean is set to the reference mean.
) andMean
the area
under
the E(NMean
The maximum value for QC
E(NURule
Set to
Instrument
U) curve were determined in
each case and the results are shown in the following tables:
“s” for the QC Rule When the QC Mean is Set to the Reference Mean
CV = 2% and instrument bias = 1%.
• Number of analyte examinations between QC events = 100
Without
a loss of generality,
assume
the examined
following: at each QC event
• 2 concentration
levels we
of QC
material
Bracketedtotal
QC error
testing
patient
results are not reported until
•• Allowable
of where
10% (TE
a = 10%)
a QC event
is accepted
• Number
of analyte
examinations between QC events = 100
• 2 concentration
of QC material
examined
each QC event
Instrument
bias and levels
imprecision
combinations
wereatinvestigated
that span a range
•
Bracketed
QC
testing
where patient
results
not of
reported
of process capabilities represented
by the
sigmaare
value
the testuntil
process:
a QC event is accepted
Sigma = (TEa – Bias) / CV
Instrument bias and imprecision combinations were investigated that span a range
following
combinations
of instrument
bias and
imprecision
evaluated:
ofThe
process
capabilities
represented
by the sigma
value
of the testare
process:
0.4
0.6
E(NU )
Instrument Mean
E(NU)
0.6
0.8
0
0.2 −15
E(NU )
Without a loss of generality, we assume the following:
E(N0) is the expected number of patient results produced between the time an
Results condition occurs and the next QC event (E(N0) = E(NB)/2 = 50 here).
out-of-control
E(NU )
CV(%)
If instrument bias exists then the reference mean for the QC material can be defined as:
An instrument’s stable mean value for a quality control material is generally used as
Reference
Mean
= Instrument
Mean
– Instrument
the target QC mean
when QC
rules
are evaluated
on the
instrument. Bias
reference
the laboratory’s
of the
correctly
defined
IfThe
instrument
biasmean
existsrepresents
then the reference
mean fordesignation
the QC material
can
be defined
as:
value for the QC material on the laboratory’s instrument platform.
Reference Mean = Instrument Mean – Instrument Bias
When instrument bias exists should a QC rule be centered on the instrument’s
The
reference
mean represents
mean
or the reference
mean? the laboratory’s designation of the correctly defined
value for the QC material on the laboratory’s instrument platform.
A related question arises when multiple instruments or analytical units evaluate the
QC
on the all
instrument’s
When
instrument
bias
exists
should aset
same analyte. Should each instrument
itsrule
ownbe
QCcentered
mean or should
instruments
mean
or the
reference
mean?
use the
same
QC mean?
Using a single QC mean for multiple analytical units that
same arises
analytewhen
appears
to be instruments
a common practice.
Aevaluate
related the
question
multiple
or analytical units evaluate the
same
analyte.risk
Should
each instrument
set its
ownrule
QCon
mean
or should all mean
instruments
The patient
implications
of centering
a QC
the instrument’s
versus
use
same mean
QC mean?
Using
a formally
single QC
mean for multiple analytical units that
the the
reference
have not
been
evaluated.
evaluate the same analyte appears to be a common practice.
We investigate the patient risk characteristics of centering a QC rule on the
The
patient risk
implications
of centering
QC rule
the instrument’s
mean number
versus
instrument’s
mean
versus the
reference amean
by on
comparing
the expected
the
have
not been
formally
evaluated.
of reference
unreliablemean
patient
results
reported
due
to the occurrence of an out-of-control
the risk
QC characteristics
rule mean is centered
on the
instrument
condition,
E(NU),
We
investigate
thewhen
patient
of centering
a QC
rule onmean
the
versus the reference
meanthe
for various
amounts
of instrument
imprecision.
instrument’s
mean versus
reference
mean by
comparingbias
the and
expected
number
of unreliable patient results reported due to the occurrence of an out-of-control
condition, E(NU), when the QC rule mean is centered on the instrument mean
versus
the reference mean for various amounts of instrument bias and imprecision.
Methods
Fig. 2: Ped for 1:3s/2:2s/R:4s multirule when the QC mean is set to the instrument mean (blue curve)
and when QC mean is set to the reference mean and “CV” is set at 2.165% (green curve), assuming
instrument CV = 2% and instrument bias = 1%. ΔPE (black curve) is the increase in probability of a
patient result with measurement error that exceeds TEa = ±10%
Fig. 2: Ped for 1:3s/2:2s/R:4s multirule when the QC mean is set to the instrument mean (blue curve)
and when QC mean is set1to the reference mean and “CV” is set at 2.165% (green curve), assuming
instrument CV = 2% and instrument bias = 1%. ΔPE (black curve) is the increase in probability of a
patient result with measurement error that exceeds TEa = ±10%
0.8
1
E(NU )
Instrument
E(NU) = ΔPE{(ARLed – 1)E(NB) – (1 – P1)[E(NB) – E(N0)]}.
ΔPE is the increase in the probability of producing an unreliable result given an
out-of-control
condition.
E(N
U) = ΔPE{(ARLed – 1)E(NB) – (1 – P1)[E(NB) – E(N0)]}.
ARLed is the average number of QC rule evaluations from the occurrence of an
ΔP
is
the
increase
in the
of producing
an unreliableQC
result
given with
an
E
out-of-control condition
to aprobability
QC rejection.
For the 1:3s/2:2s/R:4s
multirule
out-of-control
is the inverse of the probability of error detection = 1 / Ped.
2 QCs ARLed condition.
is the
theaverage
averagenumber
numberof of
QC rule
evaluations
from (100
the occurrence
of an
ARL
patient
specimens
in a batch
here).
E(Ned
B) is
out-of-control condition to a QC rejection. For the 1:3s/2:2s/R:4s QC multirule with
of a QC rejection at the first QC event after the out-of-control
1 is the
2PQCs
ARLprobability
ed is the inverse of the probability of error detection = 1 / Ped.
condition occurs. For the 1:3s/2:2s/R:4s QC multirule with 2 QCs, P1 = Ped.
E(NB) is the average number of patient specimens in a batch (100 here).
E(N0) is the expected number of patient results produced between the time an
of aoccurs
QC rejection
theQC
first
QC (E(N
event
Pout-of-control
1 is the probability
E(NBthe
)/2 =out-of-control
50 here).
condition
and the at
next
event
0) =after
condition occurs. For the 1:3s/2:2s/R:4s QC multirule with 2 QCs, P1 = Ped.
E(NU )
Results: ResultsEfficacy
are shown
in theInstrument
table.
of Using
Mean vs. Reference Mean
An instrument’s stable mean value for a quality control material is generally used as
the target QC mean when QC rules are evaluated on the instrument.
Introduction
E(NU )
Objective – Evaluate and compare performance of QC strategies centered on the
individual instrument mean versus a reference mean.
Background:
Relevance – Using a reference mean rather than the individual instrument mean for
Objective
– Evaluate
anda common
compare practice
performance
QC strategies
centered
on the
QC rules appears
to be
whenof
multiple
analytic units
evaluate
the
individual
instrument
versus aefficacy
reference
same analyte.
The mean
comparative
of mean.
this approach has not been formally
evaluated.
We
compare
the
expected
number
of
unreliable
results
reported
due
Relevance – Using a reference mean rather than the individual instrument mean forto
theunits
QC evaluate
rule mean
therules
occurrence
condition,
E(NU), when
QC
appears of
to an
be aout-of-control
common practice
when multiple
analytic
theis
centered
on the
instrument
meanefficacy
versus the
mean.has not been formally
same
analyte.
The
comparative
of reference
this approach
evaluated. We compare the expected number of unreliable results reported due to
the
occurrence of an out-of-control condition, E(NU), when the QC rule mean is
Methods:
centered on the instrument mean versus the reference mean.
We assume an allowable total error (TEa) of 10%, 100 patient examinations between
QC events, 1:3s/2:2s/R:4s QC Rule, 2 QC levels, and instrument CV and bias
Methods:
combinations that give sigma values, (TEa-bias)/CV, ranging from 3.5 to 5.9.
published
equations
(Clinical Chemistry 2008, page 2051)
Weassume
compute
U) usingtotal
We
anE(N
allowable
error (TE
a) of 10%, 100 patient examinations between
overevents,
a range1:3s/2:2s/R:4s
of systematic error
out-of-control
conditions
when the QC
mean
QC
QC Rule,
2 QC levels,
and instrument
CVrule
and
biasis
set to the instrument’s
the reference
mean. In
each case
ranging
from we
3.5determine
to 5.9.
combinations
that give mean
sigmaand
values,
(TEa-bias)/CV,
)
and
area
under
the
E(N
)
curve.
the
maximum
value
for
E(N
U
U
We compute E(NU) using published equations (Clinical Chemistry 2008, page 2051)
over
a range–ofThe
systematic
error out-of-control
conditions
QC rule mean
is
Validation
mathematically
derived E(N
werethe
recomputed
using
U) curveswhen
set
to the instrument’s
mean
and matched
the reference
mean.
In each case we determine
simulation.
The simulation
results
the direct
computations.
the maximum value for E(NU) and area under the E(NU) curve.
Results: Results are shown in the table.
Validation – The mathematically derived E(NU) curves were recomputed using
simulation. The simulation results matched the direct computations.
Probability
Probability
Background:
Abstract
0
5
SE (%)
0
5
SE (%)
10
10
15
15
1.6
1.5
CV(%)
Bias(%)
2.0
0.5
1.6
0.5
2.0
1.0
1.6
1.0
2.0
1.5
1.6
1.5
2.4
0.5
2.0
0.5
2.4
1.0
2.0
1.0
2.4
1.5
2.0
1.5
2.4
0.5
2.4
1.0
2.4
1.5
Instrument
CV(%)
Bias(%)
1.6
0.5
1.6Instrument
1.0
1.6
1.5
CV(%)
Bias(%)
2.0
0.5
1.6
0.5
2.0
1.0
1.6
1.0
2.0
1.5
1.6
1.5
2.4
0.5
2.0
0.5
2.4
1.0
2.0
1.0
2.4
1.5
2.0
1.5
2.4
0.5
2.4
1.0
2.4
1.5
Conclusions
0.0003
0.0911
5.3
Sigma
QC Rule1.6
CV(%)
Neg.
SE
Pos.
SE
0.1079
0.4036
4.8
2.0
0.0022
0.0154
5.9
1.6
0.0542
0.7620
4.5
2.0
0.0008
0.0381
5.6
1.6
0.0266
1.4201
4.3
2.0
0.0003
0.0911
5.3
1.6
1.0394
2.8981
4.0
2.4
0.1079
0.4036
4.8
2.0
0.6138
4.8037
3.8
2.4
0.0542
0.7620
4.5
2.0
0.3571
7.9648
3.5
2.4
0.0266
1.4201
4.3
2.0
1.0394
2.8981
4.0
2.4
0.6138
3.8Rule Mean Set
2.4 to Reference
QC
Mean 4.8037
0.3571
7.9648
3.5
2.4
Max E(NU)
Sigma
QCSet
RuletoCV(%)
Neg.
SE
QC
Rule Mean
Reference
Mean
5.9
5.6
5.3
Sigma
4.8
5.9
4.5
5.6
4.3
5.3
4.0
4.8
3.8
4.5
3.5
4.3
4.0
3.8
3.5
1.6560
1.7942
1.9725
QC Rule
CV(%)
2.0458
1.6560
2.1653
1.7942
2.3279
1.9725
2.4386
2.0458
2.5431
2.1653
2.6913
2.3279
2.4386
2.5431
2.6913
Pos. SE
0.0077
0.0077
Max E(NU0.0145
)
0.0145
0.0316
0.0316
Neg.
SE
Pos.
SE
0.2438
0.2438
0.0077
0.0077
0.3555
0.3555
0.0145
0.0145
0.5867
0.5867
0.0316
0.0316
1.9188
1.9188
0.2438
0.2438
2.4891
2.4891
0.3555
0.3555
3.5829
3.5829
0.5867
0.5867
1.9188
1.9188
2.4891
2.4891
3.5829
3.5829
E(NU) AUC
0.0529
0.1230
E(N0.3020
U) AUC
2.2373
0.0529
3.6696
0.1230
6.6269
0.3020
21.6936
2.2373
29.3983
3.6696
43.5970
6.6269
21.6936
29.3983
43.5970
E(NU) AUC
0.0456
0.0879
E(N0.1916
U) AUC
2.1070
0.0456
3.0490
0.0879
4.4406
0.1916
21.0203
2.1070
25.8160
3.0490
29.1960
4.4406
21.0203
25.8160
29.1960
• The expected number of unreliable patient results reported is reduced using the
reference mean, regardless of the method’s CV and bias.
Conclusions
• Centering the QC rule on the reference mean balances the risk of reporting unreliable
• The
expected
number
of unreliable
patient
results
reported is reduced using the
patient
results due
to out-of-control
shifts
in either
direction.
mean,
regardless
of the
CV and bias.
• reference
The reduction
in the
worst-case
E(Nmethod’s
U) ranged from 34% to 65% when the QC rule mean
• Centering
the
QC
rule
on
the
reference
mean
balances
themean
risk offorreporting
unreliable
was set to the reference mean instead of the instrument’s
the cases
studied.
results due
to out-of-control
shifts inconditions
either direction.
• patient
The reduction
across
all out-of-control
as characterized by the area under
• The
in theranged
worst-case
ranged
34%
65%
when
thewas
QC rule
from E(N
3%U) to
37%from
when
thetoQC
rule
mean
set mean
to the
the reduction
E(NU) curves
was
set to mean
the reference
mean
instead of the
instrument’s mean for the cases studied.
reference
instead of
the instrument’s
mean.
•• The
reduction
across all out-of-control
conditionsusing
as characterized
by therather
areathan
under
In general,
the improvement
in QC rule performance
the reference mean
the
ranged as
from
3% of
tothe
37%
when bias
the to
QC
rule mean
was setincreases.
to the
the
E(NU) curves
instrument
mean increases
the ratio
instrument
instrument
imprecision
reference
mean
instead
of
the
instrument’s
mean.
• Centering a QC rule’s rejection limits on the reference mean is effectively equivalent to
• Insetting
general,
theQC
improvement
in QC limits
rule performance
the reference
mean
the
the
rule’s rejection
around theusing
instrument’s
mean
but rather
with athan
shorter
instrument
as limit
the ratio
instrument
instrument imprecision
distance mean
to theincreases
rejection
in of
thethedirection
ofbias
thetoinstrument’s
bias andincreases.
a longer
distance
to
the
rejection
limit
in
the
direction
opposite
the
instrument’s
bias.
• Centering a QC rule’s rejection limits on the reference mean is effectively equivalent to
QC suggest
rule’s rejection
limits the
around
the instrument’s
but withmean
a shorter
• setting
These the
results
that setting
QC rule’s
mean to themean
instrument’s
puts
distance
to the
rejection
limit in the
of the
instrument’s
and amean
longerto
the primary
focus
on controlling
thedirection
instrument
while
setting a bias
QC rule’s
distance
to the rejection
in the
direction bias
opposite
bias.on controlling
the instrument’s
meanlimit
minus
instrument
putsthe
theinstrument’s
primary focus
the reliability
of patient
results.
• These
results suggest
that
setting the QC rule’s mean to the instrument’s mean puts
the primary focus on controlling the instrument while setting a QC rule’s mean to
the instrument’s mean minus instrument bias puts the primary focus on controlling
the reliability of patient results.