Medical Screening
The government has a screening program for a potentially fatal
medical condition which is thought to affect 1 in 1000 of the
population.
100 000 people are tested.
The test gives the correct result in 98 out of
100 patients.
People will naturally worry if they have a
result which suggests they have the condition,
but how much of this concern is misplaced?
Where do we start?
• How worried should someone be if they have a positive
result?
– Extremely worried,
– Very worried,
– A little worried,
– Not at all worried.
• Why?
• Do you think many will get a FALSE positive result?
Explain why.
Some calculations that might help
(i) How many of the 100 000 people tested would you
expect to have the condition?
(ii) How many of the 100 000 people tested would you
expect to NOT have the condition?
(iii) How many people would be expected to get a positive
test result?
False Positives
• A ‘false positive’ is a result which suggests a person has
a condition when they actually don’t have it at all
• Fill in the ‘tree diagram’ on the following slide, with
numbers, to see how many people will receive genuine
positive results and how many will receive false positive
results
• Remember that the test result is correct 98% of the time:
– 98% of the time it says you do have it when you do
– 98% of the time it says you don’t have it when you
don’t
False Positives
Have the
medical
condition
???
Number tested
100000
Don't have the
medical
condition
???
Test Positive
???
Test Negative
???
Test Positive
???
Test Negative
???
False Positives
Have the
medical
condition
100
Number tested
100000
Don't have the
medical
condition
99900
Test Positive
98
Test Negative
2
Test Positive
1998
Test Negative
97902
False Positives
• What percentage of the people who have received
positive results actually have the condition?
• Does this surprise you?
• Should doctors use this test?
• Is it better to worry people who don’t really have the
condition, or to miss people who do have it?
Hit and Run
The police receive a report of a hit and run accident involving a
taxi. Although it is dark, an eye witness is 90% sure it was a
green cab.
There are 1000 taxis in the city, 10 are green and the rest are
red.
Is it more likely the accident involved
a red or green taxi?
Hit and Run
Involved ??
Green ??
Taxis ???
Not
involved ??
Involved ??
Red ???
Fill in the blanks:
• Eye witness is 90% sure
• There are 1000 taxi’s in the city
• 10 are green and the rest are red
Not
involved ??
Hit and Run
Involved 9
Green 10
Not involved
1
Taxis 1000
Involved 99
Red 990
Not involved
891
Is it more likely the accident involved a red or green taxi?
Drug Testing
In the USA, 20 000 air traffic controllers
undergo random drug testing.
The test is good but not perfect:
• 96% of those who use drugs test positive
• 93% of those that do not use drugs test negative
Based on previous figures the Federal Airline Authority believe
that 99% of air traffic controllers are drug free.
Do you think it’s likely that people identified as positive by the
test are guilty of taking drugs?
Drug Testing
How is this problem different to the previous ones?
Assuming that the Federal Aviation Authority are correct – that
99% of pilots are drug free – what percentage of those testing
positive are actually drug free?
Complete the tree diagram on the next slide and discuss your
results.
Drug Testing
??%
Number
expected to be
innocent of
taking drugs
???
??%
??%
Number tested
20000
??%
Number
expected to be
guilty of taking
drugs
???
??%
??%
Number expected
to test Positive
???
Number expected
to test Negative
???
Number expected
to test Positive
???
Number expected
to test Negative
???
Drug Testing
99%
Number
expected to be
innocent of
taking drugs
19800
7%
93%
Number tested
20000
1%
Number
expected to be
guilty of taking
drugs
200
96%
4%
Number expected
to test Positive
1386
Number expected
to test Negative
18414
Number expected
to test Positive
192
Number expected
to test Negative
8
Lie Detector Test
A TV show uses a lie detector test to try to establish
which one of two people is telling the truth.
Assume that just one of the two people in the dispute is
not being honest, and it is equally likely to be either
guest.
The American Polygraph Association claim the tests are
89% accurate for a single issue response.
Out of 1000 shows, how many people will be
falsely accused of lying?
How many guests will receive the wrong result?
Lie Detector Test
• Draw out a diagram to help you ascertain how
many inaccurate results there are likely to be.
• What will happen on the shows if one or other or
both of the results are inaccurate?
Lie Detector Test
Truthful
1000
Pass Lie Detector
890
Fail Lie Detector
110
Number tested
2000
Not Truthful
1000
Pass Lie
Detector
110
Fail Lie Detector
890
Down’s Syndrome Occurrence
In an article in 2009, a “top doctor” called for changes to
the pre-natal screening for Down’s Syndrome.
To consider the concerns expressed by the doctor here are
the most recent figures from the Office for National
Statistics which show that there were approximately
730,000 babies born in the UK in 2012 and gives the
approximate age of their mothers.
2012 Live Births In England and
Wales
All ages Under 20 20-24
25-29
30-34 35-39 40-44 45 and over
729,674 33,815 132,456 202,370 216,242 114,797 28,019
1,975
As stated in the article, screening for Down’s Syndrome is
currently offered to all prospective mothers.
Down’s Syndrome Occurrence
The current test has a false positive rate of about 3% according to
information from the NHS.
To complicate matters further the chances of having a baby with
Down’s Syndrome increases with the age of the mother.
(Figures from the NHS)
• 25 years of age has a risk of 1 in 1,250
• 30 years of age has a risk of 1 in 1,000
• 35 years of age has a risk of 1 in 400
• 40 years of age has a risk of 1 in 100
• 45 years of age has a risk of 1 in 30
How many false positives are there likely to be for the different age
groups?
Use this evidence to decide whether you think pre-test counselling is a
good idea, explaining your response.
Down’s Syndrome Occurrence
Births:
The current test has a false positive rate of about 3%
Chances of having a baby with Down’s Syndrome
increases with the age of the mother.
• 25 years of age has a risk of 1 in 1,250
• 30 years of age has a risk of 1 in 1,000
• 35 years of age has a risk of 1 in 400
• 40 years of age has a risk of 1 in 100
• 45 years of age has a risk of 1 in 30
Teacher notes: It’s a Risk
Pupils will be familiar with situations in which ‘risk’ is used, such as the
use of ‘Lie Detector tests’ on day-time TV shows and athletes testing
positive for the use of banned substances, but may be less familiar with
the impact of test accuracy in medical tests.
Through these activities, pupils will gain a better understanding of false
positives and how to interpret ‘risk'.
Teacher notes: It’s a Risk
There are 5 activities with a decreasing amount of support and
structure.
It is important to give students time to think, discuss and absorb. Many
of the results will challenge what they believe or suspect to be true.
Attaining an understanding of why there are so many false negatives is
helpful.
It is suggested that students work in pairs or small groups in order to
discuss their initial thoughts and then to make sense of the outcomes.
Activities could all be tackled within a lesson or each could be used as
a starter activity in a series of lessons.
Teacher notes: Medical Screening
Where do we start
Most students in the trials said very or extremely worried. The cause of this
worry was generally stated as the 98% accuracy of the test.
Some calculations that might help
• Leave a little time between each question for the students to think and
respond.
• Students should attempt to produce a figure by calculation… the answers
will be confirmed in the next section ‘False Positives’
False Positives
• Of the 2096 positive results, only 98 are genuine – that’s less than 5%
• Why, with such an accurate test, are there so many false positives?
• Because 98% of small amount (in this case 100) < 2% of much larger
amount (in this case 99900).
• This is a key point so take some time to ensure that the students fully
appreciate it.
Teacher notes: Hit and Run
Eye witness testimony is notoriously inaccurate. In this case, the eye-witness
is 90% sure it was a green taxi, which means that there’s a 90% chance of it
being one of the green taxis and a 10% chance of it being one of the red taxis.
This is similar in structure to the ‘Medical Screening’ question, where there are
a lot more items in the ‘non-target’ group. In this case there’s a lot more red
taxis than green ones.
Out of the 108 taxis that could have been involved, 9 are green and 99 are red,
so it’s far more likely that the taxi involved was actually red.
Teacher notes: Drug Testing
What is different in this problem? The probabilities are conditional this
time. 96% for correctly identifying a drug user, but only 93% chance of
correctly identifying a non-user.
Ask the students to fill in the blanks in the diagram and discuss the
outcomes.
Pose the question: “If an air traffic controller has a positive result, how
likely are they to actually have taken drugs?”
Of the 1578 likely to test positive, only 192 are likely to have done.
That means approximately 88% of those testing positive in this test are
actually innocent.
Teacher notes: Lie Detector Test
This is a relatively straight forward problem.
If students have tried the previous problems with some structure, then
this would be a good one to let them try without teacher input and
without being given a blank tree diagram.
In the 1000 shows, there are likely to be 110 false negatives and 110
false positives.
This would mean that on some shows the following could happen:
• If one result is accurate and one is erroneous, then either both
people are found to be telling the truth or both people are found to
be lying
• Occasionally, the results are completely the wrong way round – the
liar is found to be telling the truth and the honest person is found to
be lying
Teacher notes: Down’s Syndrome
Sensitivity must be shown with this content; this activity is simply about
identification of a condition and allowing prospective mothers to be prepared.
Students should be encouraged to explore the information for themselves and
justify their responses to the questions. It might be helpful to print out copies of
the information slide for students to refer to more easily. Working with a partner
or in a small group would be helpful for students to share the workload and to
interpret their answers.
If students struggle to get started, encourage them to think about the following:
• Number of Down’s Syndrome babies expected for each age group
• Number of non-Down’s Syndrome babies expected for each age group
Then consider how many babies would fall into each of the following categories
for each age group:
Correct test result
Down’s baby
Non-Downs baby
Incorrect test result
Teacher notes: Down’s Syndrome
Numbers
Risk of Down's (1 in ……..)
Expected Number of Children born with Down's
Expected Number of children Without Down's
Test False indication rate
False Positives
False Negatives
Correct positives
Correct negatives
Probability of a positive result being false
All
ages
729,674
Under
20
33,815
20-24
25-29
30-34
35-39
40-44
45+
132,456
202,370
216,242
114,797
28,019
1,975
1144
1250
27
33,788
1250
106
132,350
1250
162
202,208
1000
216
216,026
400
287
114,510
100
280
27,739
30
66
1,909
1014
1
26
32,774
3971
3
103
128,380
6066
5
157
196,142
6481
6
210
209,545
3435
9
278
111,075
832
8
272
26,907
57
2
64
1,852
3%
21856
34
1110
706,674
97.5%
97.5%
97.5%
96.9%
92.5%
75.4%
47.3%
Teacher notes: Down’s Syndrome
•
•
•
Looking at the results it’s possible to conclude that mothers below 35 years
of age are much more likely to get a false positive result.
However, about half the children with Down’s Syndrome are born to
mothers aged under 35; this is because they make up the largest proportion
of mothers.
Perhaps mothers should be made aware of the probability of a false
positive, for their age group, and they can then make an informed choice of
whether they want the test.
References
American Polygraph Association accessed 3-2-2014
http://www.polygraph.org/section/resources/polygraph-validity-research
Lie Detector Test accessed 3-2-2014
http://www.dailymail.co.uk/femail/article-1203070/Jeremy-Kyle-nearly-killed-Thehorrifying-story-womens-decision-daytime-TVs-notorious-show.html
Down’s Syndrome article accessed 3-2-2014
http://www.dailymail.co.uk/health/article-1223406/Doctor-calls-mothers-opt-DownsSyndrome-test.html
Data from the ONS accessed 3-2-2014
http://www.ons.gov.uk/ons/taxonomy/index.html?nscl=Births+by+Mother%27s+Area+of+
Residence#tab-data-tables
NHS information accessed 3-2-2014
http://www.nhs.uk/news/2013/06June/Pages/New-Downs-syndrome-blood-test-morereliable.aspx
http://www.nhs.uk/Conditions/Downs-syndrome/Pages/Causes.aspx