Section 8.4 - Decision
Making with Data
NOT ALL DATA IS GOOD DATA!
“Do not put faith in what statisticians say
until you have carefully considered what
they do not say.” ---William W. Watt
Evaluating data collection
procedures
• Sample size
– Was the sample size adequate and representative of
the population studied?
• Random assignment
– Were the subjects randomly selected?
• Validity
– Did the test measure what it was supposed to
measure?
• Reliability
– Would the test provide the same results over and
over?
Famous example of polling
mistake:
In 1938, the magazine “Literary Digest” polled 10 million people to
predict the winner of the 1938 Presidential election (Franklin
Roosevelt vs. Alf Landon). The names of the 10 million people were
taken from phone books and club membership lists. The sureys
were mailed and the people had to return them by mail.
The magazine’s published results: Landon (57%) beat
Roosevelt (42%).
Election Day Results: Roosevelt (57%) beat Landon (43%).
Why was the magazine’s prediction so off? Was it sample size,
bias, etc…? (Think about life back in the 1930’s)
Interpreting Graphs
Ask yourself these questions…
• Conclusions
– What conclusions can I draw from the graph?
– Do the conclusions that I read seem reasonable?
• Construction of graph
– Are the scales and units clear or are they misleading?
– Would another graph be more appropriate?
• Reliability/validity
– Do I have questions about how the data were obtained
that could affect the accuracy of the data?
Making Valid Conclusions From
Data
• Beware of vague or undocumented
statements of comparison
– “More people are killed in an average airplane accident than an average
car accident. Thus, driving is safer than flying.”
• Beware of percents in claims
– “50% of NAU math graduates prefer beer to wine at Charlie’s.” – Yet,
only 4 NAU math graduates visited Charlie’s.
• Beware of conclusions based on
correlations that claim causation
– “Older people have bigger feet.” – based on data collected from
1 to 12 year olds!
Example: Misleading because 1950s (Post
WWII) was an abnormality. Median age of
marriage before that was higher.
What is missing?
Misleading Conclusions from
Graphs
• Make sure comparing LIKE UNITS.
• Make sure the graph is scaled correctly.
• Make sure visual area and volume are
increased correctly (i.e. doubled, not
quadrupled)
The US uses more oil than any
other country…
B
a
r
r
e
l
s
p
e
r
d
a
y
20 million
5.6 million
Japan
USA
Connecting data analysis and
problem solving
• Understand the question/problem
– Is it qualitative or quantitative?
• Develop a plan
– What data is needed? How to collect it?
– Sample (randomness and size)
• Implement the plan
– Does data collection technique match data needed?
– What kind of data? What kind of plot and/or statistics?
• Analyze results
– Analyze the statistics; draw conclusions only from what data tells you
– Remember: correlation ≠ causation