RATIONAL DIAGNOSTIC RESTING IN FOOD ANIMAL PRACTICE
D R B RIAN A LDRIDGE , BVS C , P H D, D IP ACVIM (L ARGE A NIMAL), MRCVS
D R J AMES L OWE , DVM, MS, D IP ABVP (F OOD A NIMAL)
Integrated Food Animal Medicine Systems, Department of Veterinary Clinical Medicine,
College of Veterinary Medicine, 1008 Hazelwood Drive, Urbana, Illinois 61802. USA
Key Mathematical Principles Underlying Test Selection
In advance of our consideration of specific test selection approaches for specific clinical
scenarios it is useful to revisit (or visit for the first time) the mathematical principles that
underpin optimal test selection and application. These principles have been clearly and
expertly covered elsewhere but it is important to incorporate them into any discussion that
aims to convert sound scientific theory in to best clinical practice. The key concepts, which
are important to grasp, are:
•
When a test result is positive how confident am I that the animal has the disease
•
When a test result is negative how confident are you that the animal does not have
the disease?
The principles underlying the answers to these questions lie in an understanding of
sensitivity, specificity and predictive values.
Sensitivity, Specificity and Predictive Values
Sensitivity, specificity and positive and predictive values are measures used to express the
usefulness of a test in identifying an individual person with the disease or recognizing an
individual without the disease.the very basis of the diagnostic endeavour.
The following figures will be used to illustrate and describe these values:
2X2 Table
TRUE DISEASE STATUS
TEST RESULT
Diseased
Healthy
Test Positive
TOTAL IDENTIFIED BY
TEST AS POSITIVE
Test Negative
TOTAL IDENTIFIED BY
TEST AS NEGATIVE
TOTAL ANIMALS WITH
TOTAL ANIMALS WITHOUT
DISEASE
DISEASE
The following example will be used to illustrate the key principles:
1. Assume that the Disease Prevalence is 4%
2. Assume that of the 4 cattle with the disease, 3 are picked up by the test (True positives)
2X2 Table
TEST RESULT
Test Positive
Test Negative
TRUE DISEASE STATUS
Diseased
Healthy
3
1
4
3. Assume that of the test is positive for a further 7 cattle who don’t have the disease (False
Positives)
The remainder of the sample are negative on the test (True Negatives)
2X2 Table
TEST RESULT
Test Positive
Test Negative
TRUE DISEASE STATUS
Diseased
Healthy
3
1
4
7
89
96
10
90
100
Sensitivity of the diagnostic test is the proportion of diseased animals correctly identified
by the test = True Positives/Total of Diseased Animals = 3/4 = 75%
i.e. Sensitivity measures the proportion of FALSE NEGATIVES
Specificity of the diagnostic test is the proportion of non-diseased animals correctly
identified by the test = True Negatives/Total of Healthy Animals = 89/96 = 93%
i.e. Specificity measures the proportion of FALSE POSITIVES
Question 1: If an animal is positive on the test what is the probability that they have the
disease?
= 3/10 = 30% - this is the Positive Predictive Value (PPV) of Test
Question 1: If an animal is negative on the test what is the probability that they do not have
the disease?
= 89/90 = 99% - this is the Negative Predictive Value (NPV) of Test
2X2 Table
TRUE DISEASE STATUS
TEST RESULT
Test Positive
Test Negative
Diseased
Healthy
TRUE POSITIVE (TP)
FALSE POSITIVE (FP)
FALSE NEGATIVE (FN)
TRUE NEGATIVE (TN)
TOTAL ANIMALS WITH
TOTAL ANIMALS
DISEASE
WITHOUT DISEASE
TOTAL IDENTIFIED BY
TEST AS POSITIVE
TOTAL IDENTIFIED BY
TEST AS NEGATIVE
Summary Formulae
Sensitivity = (TP + FN)
Specificity = (FP + TN)
Dz –ve
High sensitivity = low false negatives
High specificity = low false positives
Dz +ve
Practical Application #2
An understanding of these principles is essential for the correct application of diagnostic
tests, knowing when to use what kind of test.
•
•
High probability of detecting an animal with the disease
•
“rule out” tests if test is negative (SnNout)
•
high sensitivity (low false negative)
High probability of indentifying an animal without the disease
•
“rule in” tests if result is positive (SpPin)
•
high specificity (low false positive)
Use tests with high sensitivity to rule out diseases (SnNout) and tests with high
specificity to rule in diseases (SpPin)
Clinical Conundrum: how to decide which test to use?
Q: Do I use haematology to rule out or rule in infection?
A1: What is the relative sensitivity or specificity of haematology in detecting infections?
A2: If used proficiently (differential counts, absolute numbers, cytopathological changes in
leukocytes) haematology is more likely to give you false negatives than false positives in the
detection of infection. i.e. some animals with infection will not show haematological changes.
So false negatives are more of a problem than false positives; haematology has more
problems with sensitivity than specificity. So, the principle applied agrees what we already
know, haematology is better for ruling in than for ruling out infection.
Limitations of Sensitivity and Specificity
Predictive Values are, in many ways, a much more useful value to the clinician in diagnostic
test selection than sensitivity and specificity in that they indicate the probability that the test
result will give the correct diagnosis.
The Positive Predictive Value (PPV) is the proportion of animals with a positive test that
have the disease.
The Negative Predictive Value (PPV) is the proportion of animals with a negative test that do
not have the disease.
The problem is that predictive values are affected by disease prevalence: one of the most
important factors in using these parameters in gauging the usefulness of a test in diagnosis
is an understanding of the influence of disease prevalence. An exercise to illustrate the
importance of prevalence in diagnostic test performance will be presented on Blackboard.
For each diagnostic test, these are the key test characteristics that the clinician needs to
know, understand and apply. Another problem is that sensitivity and specificity cannot be
used to estimate probability of disease in an individual animal
The main diagnostic question of interest to the clinician is what is the probability of disease
given a positive test? The answer lies in combining the information provided by sensitivity
and specificity into a measure called Likelihood Ratio (LR).
•
LRs provide a comparison of the likelihood of a particular test result in diseased
animals over non-diseased animals*
LR+ =
probability of diseased animal having positive test
probability of healthy animal having positive result
LR- = probability of diseased animal having negative test
probability of healthy animal having negative result
LR+ =
sensitivity of test
LR- =
1-sensitivity of test
1- specificity of test
specificity of test
Likelihood Ratios: Case Example
TRUE DISEASE STATUS
Diseased
Healthy
9
18
1
72
10
90
TEST RESULT
Test Positive
Test Negative
Disease Prevalence = 10%
Sensitivity = 90%
LR+ = probability of diseased animal having positive test
27
73
Specificity = 80%
= 90/20 = 4.5
probability of healthy animal having positive result
i.e.
An animal with the disease is 4.5 times more likely to have a positive test than an
animal without the disease
LR- = probability of diseased animal having negative test
= 10/80 = 0.13
probability of healthy animal having negative test
i.e.
An animal without the disease is 7.5 times more likely to have a negative test than an
animal with the disease
Practical Application #2
•
Likelihood Ratios can be used to adapt sensitivity/specificity of tests to achieve a rule-out
or a rule-in
•
•
Increase LR+ for rule in (because false positives less of a problem)
•
Increase LR- for rule out (because false negatives less of a problem)
Sensitivity/Specificity adapted by altering cut-off values (see references on ROC curves)
Use Likelihood Ratios (>10) to adapt sensitivity/specificity to maximize rule in or rule
out potential of test
IN SUMMARY
•
Use tests with high sensitivity to rule out diseases (SnNout) and tests with high
specificity to rule in diseases (SpPin)
•
Use Likelihood Ratios (>10) to adapt sensitivity/specificity to maximize rule in or rule
out potential of test
•
Use variable cut-offs in your diagnostic tests depending on the sensitivity/specificity
balance required (i.e. the Likelihood Ratios)
•
To optimize the performance of expensive tests, select inexpensive screening tests
that increase the prevalence of your disease in your sample population
•
Adopt problem-oriented clinical reasoning for difficult cases
•
Use lab investigations for diagnostic, therapeutic and prognostic decision-making
REFERENCES
A significant proportion of the above information is adapted from a series of articles titled
Understanding Diagnostic Tests by Anthony Akobeng in Acta Paediatrica (2006).
Participants are directed to these excellent articles for additional background in these topics.