HSS4303B Intro to Epidemiology
Feb 4, 2010 – Screening Tests
Student Obesity Conference
•
•
•
•
•
www.studentobesitymeeting.ca
June 9-12 at uOttawa
Abstract deadline is Feb 12
Registration fee is $95 (includes meals, etc)
200 students + 25 mentors
CSEB Student Conference
• May 27-28, 2010
• Kingston
• Details will be posted on www.cseb.ca
Your Abstracts
•
•
•
•
Marks are now posted
3 people did not submit
Min = 5.7/10 Max=9.5/10 Mean = 7.6/10
No one failed (except the above 3)
Your abstracts – Erin’s comments
• The students who got >85% were clear about their research
topics and provided intros and conclusions.
• Students who got between 70 and 85% followed the
instructions, but there is some variance in marks due to
style/grammar, and the quality of their references.
• Students who received a grade of <70% did not follow the
instructions
– ie. all of their references were web-based (PHAC, StatsCan, etc.)
– they went way over the word count (some over 400 or 500
words)
– and/or there was nothing at all related to epidemiology in their
abstract.
Erin’s Office Hours
• Erin can be available during reading week.
Does anyone intend to come by?
• She will not be available March 4
• Always available by appointment
Tuberculosis
• What is it?
• We apply tuberculin skin test (also called PPD
– purified protein derivative) test
• Positive response is an “induration”
– a hard, raised area with clearly defined margins at
and around the injection site
What type of curve is this?
Bimodal curve
• ________________ identifies two types of traits in a
population
• _________________ separates individuals ho had not prior
experience with tuberculosis from those who had prior
experience
• Bimodal distribution allows to separate people on the basis of
the trait, characteristic or disease
• However, for many of the conditions and diseases people fall
under uni-modal distribution
What’s it called when there’s only one hump?
Distribution of systolic blood pressure for men
(unimodal distribution)
Unimodal curve
• _________________ has a single peak with normal
distribution or tailed distribution
• Since it does not categorize people an arbitrary cutoff has to
be used to separate people as hypertensive or normotensive
• Cutoff is usually based on statistical evidence, however,
biological, genetic and other information also need to be
considered
• Which men are at a higher risk of stroke, myocardial infraction
• Unimodal or bimodal there will still be people in the grey zone
and there is uncertainty about these cases
So….
• We are concerned about TB and High BP in
the population, and we have screening tests
for both
• But you can see that the challenges are
different for both types of screening tests
What is a Screening Test?
• A test given to persons who do not show
clinical signs of a disease to nonetheless test
for that disease
•Validity
•Reliability
•Sensitivity
•Specificity
Examples of screening tests?
•
•
•
•
•
PSA
CT scans
Pregnancy tests
DRE
Phenylketonuria (PKU) Test
Validity
• ability to distinguish between those who have
the disease and those who do not have the
disease
• i.e., is it detecting what it says it’s detecting?
e.g. PPD Test
• The PPD test purports to test for TB infection
• However, it is possible to get a reaction from
the BCG TB vaccine (which is not available in
North America)
• With respect to distinguishing between actual
TB exposure and vaccine exposure, the PPD
test has poor validity
There are many types of validity
• Internal vs External
– Refers to the validity of a study
– Not relevant for screening tests
– We’ll revisit this dude
There are many types of validity
• Construct validity
– The extent to which the measurement
corresponds to theoretical concepts
– "Are we actually measuring (are these means a
valid form for measuring) what (the construct) we
think we are measuring?"
– IQ test
Validity
• Content validity
– Also known as “logical validity”
– The extent to which the test incorporates all that
is known about the disease
– Eg. If test purports to measure “functional health”
then it should include measurements of social
happiness, etc, and not just biological markers
Validity
• Criterion validity
– The extent to which the test correlates with an
external criterion of the thing you’re studying
• Concurrent validity
– The measurement and the criterion refer to the same point in
time
– If visually looking at a wound is your test for injuries in a
battle, how do you know if the would was inflicted during the
battle?
• Predictive validity
– The measurement can predict the criterion
– SAT scores are a good predictor of freshman marks
Reliability
• Can you repeat the test and get the same
result?
– Let’s say you’re measuring nose length to
determine cancer risk --will you get different
results everytime you measure the same nose?
– Blood pressure has poor reliability because it
changes every few minutes
Reliability of Screening Tests
RELIABILITY:
The extent to which the screening test will
produce the same or very similar results each
time it is administered.
--- A test must be reliable before it can be valid.
--- However, an invalid test can demonstrate
high reliability.
Reliability of Screening Tests
Sources of variability that can affect the
reproducibility of results of a screening test:
1. Biological variation (e.g. blood pressure)
2. Reliability of the instrument itself
3. Intra-observer variability (differences in
repeated measurement by the same
screener)
4. Inter-observer variability (inconsistency in
the way different screeners apply or
interpret test results)
Measures of Validity
• Sensitivity
• Specificity
Measure of Validity
• Sensitivity
– The probability of correctly diagnosing a case
(case= person with the disease)
– i.e. the proportion of truly diseased people who
are identified as diseased by the test
• Specificity
– The probability of correctly rejecting a case
– i.e., “true negative rate”
Sensitivity/Specificity
Screening test
results
Truly diseases
(cases)
Truly nondiseases
Totals
Positive (thinks
it’s a case)
a
b
a+b
Negative (thinks
it’s not a case)
c
d
c+d
totals
a+c
b +d
a+b+c+d
Sensitivity/Specificity
Screening test
results
Truly diseases
(cases)
Truly nondiseases
Totals
Positive (thinks
it’s a case)
a
b
a+b
Negative (thinks
it’s not a case)
c
d
c+d
totals
a+c
b +d
a+b+c+d
Sensitivity = a/(a+c)
Sensitivity/Specificity
Screening test
results
Truly diseases
(cases)
Truly nondiseases
Totals
Positive (thinks
it’s a case)
a
b
a+b
Negative (thinks
it’s not a case)
c
d
c+d
totals
a+c
b +d
a+b+c+d
Sensitivity = a/(a+c)
Specificity = d/(b+d)
How do you compute prevalence from these data?
All cases / total pop
=(a+c) / (a+b+c+d)
Can you fill in the blanks?
Example: Assume a population of 1,000 people, of whom 100 have a
disease and 900 do not have the disease
Screening Test to Identify the 100 People with the Disease
True Characteristics in the
Population
Results of Screening
Disease
No Disease
Total
Positive
180
Negative
820
Total
100
900
1,000
Example: Assume a population of 1,000 people, of whom 100 have a
disease and 900 do not have the disease
Screening Test to Identify the 100 People with the Disease
True Characteristics in the
Population
Results of Screening
Disease
No Disease
Total
Positive
80
100
180
Negative
20
800
820
100
900
1,000
Total
So….
• What’s a false positive?
• What’s a false negative?
So….
• What’s a false positive?
– Test says positive but in reality it’s a negative
• What’s a false negative?
– Test says it’s negative but in reality it’s a positibe
Sensitivity/Specificity
Screening test
results
Truly diseases
(cases)
Truly nondiseases
Totals
Positive (thinks
it’s a case)
a
b
a+b
Negative (thinks
it’s not a case)
c
d
c+d
totals
a+c
b +d
a+b+c+d
Sensitivity = a/(a+c)
Specificity = d/(b+d)
Which are the false positives?
Which are the false negatives?
Sensitivity/Specificity
Screening test
results
Truly diseases
(cases)
Truly nondiseases
Totals
Positive (thinks
it’s a case)
a
b
a+b
Negative (thinks
it’s not a case)
c
d
c+d
totals
a+c
b +d
a+b+c+d
Sensitivity = a/(a+c)
Specificity = d/(b+d)
Which are the false positives?
Which are the false negatives?
False positives and false negatives
• False positives
– Burden on health care for follow tests
– Anxiety and worry for the people
– Psychosocial aspects of the label
• False negatives
– Missed being diagnosed and provided with the
timely treatment has compromised prognosis
– Shock and disbelief upon diagnosis in advanced
stage of the disease
So….
• We define two more concepts:
– Positive Predictive Value (PV+ or PPV)
– Negative Predictive Value (PV- or NPV)
These are measures of “performance yield”
PV+
• Also called “precision rate”
• Also called “post-test probability of disease”
• the proportion of patients with positive test
results who are correctly diagnosed
• Sounds like sensitivity, right?
PV• the proportion of patients with negative test
results who are correctly diagnosed
Performance Yield
•
People with positive screening test
results will also test positive on the
diagnostic test:
Predictive Value Positive (PV+)
•
People with negative screening test
results are actually free of disease
Predictive Value Negative (PV-)
Sensitivity/Specificity
Screening test
results
Truly diseases
(cases)
Truly nondiseases
Totals
Positive (thinks
it’s a case)
a
b
a+b
Negative (thinks
it’s not a case)
c
d
c+d
totals
a+c
b +d
a+b+c+d
Sensitivity = a/(a+c)
Specificity = d/(b+d)
Sensitivity/Specificity
Screening test
results
Truly diseases
(cases)
Truly nondiseases
Totals
Positive (thinks
it’s a case)
a
b
a+b
Negative (thinks
it’s not a case)
c
d
c+d
totals
a+c
b +d
a+b+c+d
Sensitivity = a/(a+c)
Specificity = d/(b+d)
PV+ = a/(a+b)
Sensitivity/Specificity
Screening test
results
Truly diseases
(cases)
Truly nondiseases
Totals
Positive (thinks
it’s a case)
a
b
a+b
Negative (thinks
it’s not a case)
c
d
c+d
totals
a+c
b +d
a+b+c+d
Sensitivity = a/(a+c)
Specificity = d/(b+d)
PV+ = a/(a+b)
PV- = d/(c+d)
Relationship between Sens/Spec and PV-/PV+
Performance Yield
True Disease Status
+
Results of +
Screening
Test
-
400
995
100
98905
Compute sensitivity, specificity, PV+ and PV-
Performance Yield
True Disease Status
+
Results of +
Screening
Test
-
400
995
100
98905
Sensitivity: a / (a + c) = 400 / (400 + 100) =
80%
Specificity: d / (b + d) = 98905 / (995 + 98905) =
99%
PV+:
a / (a + b) = 400 / (400 + 995) =
29%
PV-:
d / (c + d) = 98905 / (100 + 98905) =
99%
Performance Yield
True Disease Status
+
Results of +
Screening
Test
PV+:
400
995
100
98905
a / (a + b) = 400 / (400 + 995) = 29%
Among persons who screen positive, 29% are found
to have the disease.
Performance Yield
True Disease Status
+
Results of +
Screening
Test
PV-:
400
995
100
98905
d / (c + d) = 98905 / (100 + 98905) = 99.9%
Among persons who screen negative, 99.9% are found
to be disease free.
Performance Yield
Factors that influence PV+ and PV1.
The more specific the test, the higher
the PV+
2.
The higher the prevalence of preclinical
disease in the screened population, the
higher the PV+
3.
The more sensitive the test, the higher
the PV-
Performance Yield
Prevalence (%)
Sensitivity
Specificity
PV+
0.1
90%
95%
1.8%
1.0
90%
95%
15.4%
5.0
90%
95%
48.6%
50.0
90%
95%
94.7%
Relationship between prevalence and positive predictive value of a test
Performance Yield
Thus, the PV+ is maximized when used in “high
risk” populations since the prevalence of preclinical disease is higher than in the general
population….
screening a total population for a relatively
infrequent disease can be very wasteful of
resources and may yield few previously
undetected cases.
Homework
•
Geenberg p. 105, question 1-13:
– 13786 Japanese patients underwent CT scans to detect first signs of cancer, then had
pathology tests 2 years later to confirm whether or not they actually had cancer
Compute:
•Prevalence of cancer
•Sensitivity & specificity
•% of false positives
•% of false negatives
•PV+ and PV-
CT result
Cancer
present
Cancer absent
Positive
56
532
negative
4
13194