Unit 3 SAS 10
III.C Student Activity Sheet 10:
Statistical Bias in Research Studies
and Polls
high statistical bias,
low variability
low statistical bias,
high variability
high statistical bias,
high variability
low statistical bias,
low variability
1. What are some reasons for deliberately using
a nonrepresentative sample in an experimental
study? What problems could result?
• Experimental studies on humans use volunteers, and it
is not possible to take a random sample of the
population of interest and force them to participate.
• Just the fact that the participants have some condition
for which a treatment is to be tested implies that there
is something special about the participants.
• Smokers, for example, may be more likely to have
health issues (beyond those clearly related to smoking)
that nonsmokers do not have.
• Smokers may not eat as well or exercise as much as
nonsmokers, so using smokers in a study may result in
bias that cannot be removed with statistical analysis.
Question 2
2. Rather than spending a great deal of time and
money to ensure a representative sample,
researchers often use the techniques listed
below to try to eliminate any statistical bias
introduced by the sampling method.
• Discuss how each method can reduce
sampling method bias and thus increase the
accuracy of a study’s results.
a. Random assignment of treatments
• This approach helps ensure that the treatment
group and the control group are similar to
each other.
• Therefore, if different outcomes are detected,
researchers can be more confident that this
result is due to the treatment, not to
differences in the participants.
b. Blind/double-blind studies
• Participants sometimes inaccurately report on
their improvement.
• Likewise, researchers sometimes
subconsciously try harder to find
improvement in treatment-group participants
versus control-group participants.
• Conducting the study blind, or preferably
double-blind, can ensure more accurate
reports.
c. Use of control groups
This approach is related to the first two
techniques, as the three work together to
improve result accuracy.
Randomly assigning participants to receive a
fake treatment helps control for the placebo
effect—hence the name control group .
d. Replication
If a study is run again with a new sample (for
example, a larger population, different ethnic
group, or different gender) and still yields similar
results, the researchers can feel more confident
that the treatment is working.
3. Some of the same issues (nonrepresentative samples,
undercoverage, replication) affect observational studies. Other
issues discussed (response bias, wording of questions) can also
specifically affect these studies. What problem could be
occurring in each study described?
a. Jade is so tired of sales calls that she refuses
to answer the phone if she does not recognize
the name or number on the Caller ID.
• This is a very common situation and can result
in a poor sample due to response bias or
being a nonrepresentative sample.
3. Some of the same issues (nonrepresentative samples,
undercoverage, replication) affect observational studies. Other
issues discussed (response bias, wording of questions) can also
specifically affect these studies. What problem could be
occurring in each study described?
b. Serena knows the caller will probably keep
calling, so she answers the phone. If the call’s
subject is an opinion poll, she politely tells the
surveyor that she does not want to participate.
• This is an example of nonresponse in the
sample obtained.
3. Some of the same issues (nonrepresentative samples,
undercoverage, replication) affect observational studies. Other
issues discussed (response bias, wording of questions) can also
specifically affect these studies. What problem could be
occurring in each study described?
c. DeShawn does not worry about sales calls
since his family uses cell phones and no longer
has a land line.
• This situation is also becoming more common.
Households that do not have land lines and
thus cannot be surveyed by telephone cause
undercoverage in samples.
3. Some of the same issues (nonrepresentative samples,
undercoverage, replication) affect observational studies. Other
issues discussed (response bias, wording of questions) can also
specifically affect these studies. What problem could be
occurring in each study described?
d. At the mall, an older man is interviewing
people about their consumer spending habits.
Maria does not want to tell him she just bought
a swimsuit, so she says her shopping bag
contains a new shirt.
• This is response bias in the sample.
3. Some of the same issues (nonrepresentative samples,
undercoverage, replication) affect observational studies. Other
issues discussed (response bias, wording of questions) can also
specifically affect these studies. What problem could be
occurring in each study described?
e. Mrs. Gibbs normally gets a hamburger for lunch.
When she notices that someone who looks like a
nurse is sitting in the corner writing down what
each person orders, Mrs. Gibbs orders a salad with
fat-free dressing.
• This is observer effect—Mrs. Gibbs acts
differently because she is being observed. It could
also be a form of response bias.
3. Some of the same issues (nonrepresentative samples,
undercoverage, replication) affect observational studies. Other
issues discussed (response bias, wording of questions) can also
specifically affect these studies. What problem could be
occurring in each study described?
f. The local television station runs a viewer poll on the
nightly news: “Call us or log on to our website to give
your opinion about the new skate park.”
• This approach is not really a research study and is not
scientifically valid.
• It is strictly voluntary, and generally only people with a
really strong opinion (and possibly only those with a
really strong opinion in one direction) call or log on,
which exemplifies response bias.
4. Several television shows have a survey component in
which viewers call in and vote for their favorite
singer/dancer/entertainer/competitor. Discuss the
issues you perceive with this method.
• There could be differences between the people who watch
these shows and the people who do not, so the winner may
not reflect the preferences of the entire country.
• People vote for a particular contestant based on different
criteria than the ones intended. For example, respondents
like a contestant’s clothes when they are supposed to vote
for his or her singing ability.
• One respondent votes according to how the contestants did
on one specific show, while another votes according to the
contestants’ overall performances on the past month’s
worth of shows.
• People vote more than once.
• People vote when they have not even watched the show.
5. Some shows have viewers vote for the contestant
whom they want to leave the show. Discuss the issues
you perceive with this method.
• Many of the same issues exist as in Question 4.
• In addition, some viewers vote strategically. Rather
than voting for the worst contestant, these viewers
vote to eliminate the person they perceive as the
strongest competition for their favorite.
Where in the research cycle is
statistical bias most likely to occur?