Statistical Deception
 Lying with statistics
 Putting a “positive spin” on
the facts
•
Statistical deception is not
necessarily a bad thing,
but you need to be aware
of it rather than just
accepting statistics at face
value.
Many problems with statistics
involve problems with
gathering the data:
1. Non-representative
samples
 too small
 too large
 not randomly chosen
* convenience
sample
* purposely chosen
wrong
2. Generalizing to the
wrong population
 sample is drawn from a
different population than
the results imply
3. Comparing apples and
oranges
 groups being compares
were different to begin
with
 difference is due to
something other than the
results imply
4. Survey bias
 leading questions
 “Good boy”
Effect
People will
give the
answer they
think you want to hear.
 “NOYB” effect
The more personal a question is
(the more it is “none of your
business”), the more likely
people are to lie.
5. Placebo effect
 In medicine a placebo is a
fake treatment actually
helps because people
think it will work.
 Doesn’t have to deal with
medicine.
 In general, when people
think they are being
watched or treated, they
often act differently than
they would otherwise.
Other issues can come up in
interpreting and publicizing
statistical results:
1. Convenient averages
• Choosing the average that
makes you look the best
(or your opponent look the
worst), even if it’s not
really a “typical” average
• Ask …Is the number they
give really “average”?
2. Assuming everybody is
average (or close to it)
•
…How spread out are
things?
3. Not adjusting for different
sample sizes
•
Comparing raw numbers
instead of percentages
Which is true?????
 Raw crimes are going up,
mostly because the
country’s population is
increasing.
 Your risk of being a crime
victim has never been
lower.
4. Not adjusting money
amounts for inflation
5. Screwing up the math
•
Classic example: Is
spinach a high-iron food?
No—in the initial report a
decimal point was misplaced.
• Spinach was reported to
have 1000 times as much
iron as it actually does.
• In fact, spinach is pretty
much equivalent to any
other leafy green
vegetable.
6. Extrapolating from a
partial result.
 Does a low-salt diet lower
blood pressure?
Sometimes—but mostly not.
• In otherwise healthy
people, there is no
correlation at all between
salt consumption and
blood pressure.
•
In patients with serious
hypertension, reduced salt
has been shown to be one
of many treatments that
may lower blood pressure.
•
It doesn’t always work,
though—and some patients
BP actually increases with
less salt consumption.
•
About seventy years of
research have been at best
inconclusive.
7. Assuming cause and effect
•
Remember: correlation
just means “relationship”
•
A confounding variable
may be skewing the
results.
8. Ignoring Occam’s Razor
• complex or unbelievable
explanations
• Occam’s Razor says the
simplest explanation is
generally the best.
•
Always consider the
simplest explanation first.
Was there a conspiracy to
cover up the fact that
President Obama was not
born in the United States?
•
Some people have
claimed that the President
Obama’s birth certificate is
fake.
•
Many independent experts
have examined the birth
certificate. They
determined the certificate
was authentic, had a
raised seal, and was of the
same format as others
issued in Hawaii at the
time it was requested.
•
Linda Lingle, a Republican
who was the governor of
Hawaii when Obama ran
for office, verified that the
birth certificate was
genuine.
•
In addition, it has been
noted that birth
announcements were
published in the Honolulu
papers the day after
Obama’s birth:
•
A nurse who worked with
the doctor who attended
the birth remembers the
doctor commenting on the
birth, because having a
black father was very
unusual in Hawaii at the
time.
•
Real estate and tax
records show Stanley
Ann Dunham Obama and
Barack Obama, Sr. lived in
Honolulu at the time
Barack Obama, Jr. was
born.
•
Though some critics still
claim the President is not
a “native-born” American
citizen, this claim doesn’t
pass Occam’s Razor—
which is the reason courts
have refused to consider
them.
9. Changing the subject
 a.k.a. “Moving the bullseye
to fit the arrows”
 saying a result means
something different than it
really does
 putting a “good spin” on
the data
 finding one small thing
about the results that
supports what you want to
find
Suppose you were the police
chief in a town with a crime
problem …
1993 – 94 … 100 more
1994 – 95 … 75 more
1995 – 96 … 50 more
1996 – 97 … 25 more
10.
Reporting information
from biased sources
that have a vested
interest.
Always ask “Who says so”?
Try to get information from
neutral parties who don’t
have a stake in the outcome.
11.
Using a non-standard
significance level
 deciding after the fact on a
level that guarantees
significance
12. Misuse of the word
“significant”
 implying significant means
big, important, or dramatic
 REMEMBER: it just
means “unlikely to have
happened by chance”
13. Discounting significance
because something is “just
statistics”