Lesson 11-1



Rebecca and Tova have math class right after
lunch.
Rebecca always eats a hot lunch on days
when she has an exam, because she has a
theory that people score a little higher on
math exams when they have eaten a hot
meal.
Tova is very skeptical of this theory. How can
Rebecca and Tova test whether the theory is
valid?


Rebecca and Tova write down as many of their
own exam scores as they can remember, and
they record whether they had a cold lunch or hot
lunch on the days of those exams. They ask a few
of their friends to give them the same
information. Then they find the average of all the
scores associated with a cold lunch and all the
scores associated with a hot lunch.
This type of study, in which data are collected
from a sample convenient to the investigators, is
called anecdotal. Though it is not an accurate or
scientific kind of study, this approach is used
very frequently. Here are three better ways to
collect sample data for a study.
The diagram shows the stages of an
experimental study.

After using any of these methods to collect
information about the type of lunch students ate,
Rebecca and Tova would need to learn the exam
scores for the students who participated in the
study. For this study it would be best if they gave
the names of the students in each lunch-type
group to the teacher, who could provide the
mean and standard deviation of the exam scores
of each group. Each type of study has advantages
and disadvantages. In the investigation you will
use two of the methods to conduct a study.


Consider this hypothesis: Listening to music
while doing math homework will shorten the
time it takes to complete the assignment.
Your group will design two studies that could
test this hypothesis. Choose two of the types
of study described on the previous page, and
describe how you would collect sample data
for the study. Provide enough detail so that
another group could use your description to
collect data in the same way. Keep in mind
these points as you design your study:



If you design an experiment, you must
describe what treatments will be used and
how subjects will be assigned to a treatment.
To design an observational study, you must
describe what is to be observed and
measured and how you’ll decide which
subjects to observe.
A survey plan must address who will be
surveyed, the method of survey (interview,
paper questionnaire, e-mail, and so on), and
the exact questions to be asked.




When the assignment of treatments in an experiment is
done so that the groups are not biased, then the conclusions
of the experiment may indicate causation.
If Rebecca implements a carefully designed experimental
study, she might find that students having a hot lunch did
much better on exams than those eating a cold lunch. If so,
she would have evidence that eating the hot lunch “caused”
the students to perform better on the exam.
However, if Rebecca used either an observational study or a
survey, she could conclude only that there’s an association
between the type of lunch and exam scores. Careful
experimental studies are the primary way to learn about
causation. Only a huge observational study over a long
period of time can indicate causation.
On the other hand, observational studies and surveys are
often more practical than experimental studies, and they do
not require subjects to give up control of their own
treatment.


In all types of studies, a major concern is to
minimize bias that will invalidate the results.
In experimental studies, the way subjects are
assigned to a treatment may affect the data.
Observational studies may omit consideration
of some subjects.
Surveys may be biased by the selection of the
participants, by the truthfulness of the
subjects, and by how the questions are
phrased.

Brandon surveyed some students and found
that the average amount of sleep for female
students was much higher than that for male
students. Later, a thorough follow-up study
concluded that the average amount of sleep
was the same for males and females. Give
some possible reasons for the discrepancy in
the results. How could Brandon design a
better survey?


The original survey may have been biased by how
Brandon selected the students to be sampled. For
example, the only males he sampled might have
been friends who had similar sleeping patterns to
his own. Or students who were reluctant to talk
about their sleep habits may have declined to
participate. Or the participants may have
misunderstood the questions.
One way Brandon could minimize bias is to
choose subjects randomly. In addition, Brandon
should ask some other people to review the
questions in his survey to make sure the
phrasing of the questions does not influence the
answers given or the likelihood that a person
would choose not to respond.

No matter how the data are collected, a
researcher must be concerned about over- or
under-representing subgroups of the
population. For instance, Brandon might have
overrepresented his male friends.

As another example, suppose you want to design
a study about the effectiveness of a new
fertilizer. If all of your evidence comes from
using the fertilizer on tomato plants, then your
conclusion can be only about tomato plants, not
about all plants. If your data come from several
types of plants, but most of your data are from
tomato plants, then the data will be biased
because one segment of the population is
overrepresented. A study based on biased data
will not necessarily have incorrect conclusions,
but the methods of data collection should be
described so that those who read about the study
know of the sampling bias.

Newspapers or other sources do not always
include enough information about a study to
determine whether the statistics are reliable.
As an educated consumer, before accepting a
conclusion from a study you need to know
how the data were collected and whether any
bias or other errors could be present.

Evaluate each of these statistics.
◦ Arthur says that 71.43% of students hate the new school
dress code. He interviewed seven people in the parking
lot after school and asked them, “Do you like the way the
school is forcing us to dress?”
◦ Betty claims that the typical student spends $19.93 on
fast food every month. She passed out a survey to the 30
students in her algebra class, and 20 of the surveys were
returned. The survey question read, “How much did you
spend last week at fast-food restaurants?”
◦ Carl states that the number of students with multiple
suspensions has increased by 67% in the last 10 years.
He found out from school records that last year there
were 5 students with multiple suspensions and 11 years
ago there were only 3 students.
Although none of these students have lied about
their results, all of them are using biased data or
misrepresenting a statistic to make a point.
a) Arthur’s conclusion does not match his
question. A participant may object to being
forced to dress in a certain way, but find
nothing objectionable in the present dress
code. Or subjects may have one small issue
with the current code but overall think that it is
fair.
Arthur’s sample is biased because it didn’t
include students who weren’t in the parking lot
(such as those involved in after-school
activities). Therefore, the students that he
sampled may not share the same view about
uniforms as the other students in the school.
b) Betty multiplied her one-week figures to
obtain a monthly value. This assumes that
spending is uniform every week. You do not
know whether the value is a mean or a
median value. You do not know why 1/3 of
the people did not return the survey. If
participants had similar reasons for not
returning the survey, it may be they had
similar spending habits also. Because Betty
chose to sample only students from algebra
class, her coverage of students is biased.
c) c. Carl’s claim may sound as if he has many
years of data instead of only two years. Also,
the population of the school needs to be
considered. Suppose there are 600 students at
the school. Although it is true that there was an
increase of 67% in the actual number of
students with multiple suspensions, the ratio
of students with multiple suspensions in the
student body increased by only 0.3% from
3/600 to 5/600. Also, if the population of the
school has increased in the last ten years, the
suspension rate may have actually decreased.