Chapter 4
Gathering Data
Section 4.3
Good and Poor Ways to Experiment
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Elements of an Experiment
Experimental units: The subjects of an experiment; the
entities that we measure in an experiment.
Treatment: A specific experimental condition imposed
on the subjects of the study; the treatments correspond to
assigned values of the explanatory variable.
Explanatory variable: Defines the groups to be
compared with respect to values on the response variable.
Response variable: The outcome measured on the
subjects to reveal the effect of the treatment(s).
3
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Experiments
 An experiment deliberately imposes treatments on the
experimental units in order to observe their responses.
 The goal of an experiment is to compare the effect of
the treatment on the response.
 Experiments that are randomized occur when the
subjects are randomly assigned to the treatments;
randomization helps to eliminate the effects of lurking
variables.
4
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
3 Components of a Good Experiment
1. Control/Comparison Group: allows the researcher to
analyze the effectiveness of the primary treatment.
2. Randomization: eliminates possible researcher bias,
balances the comparison groups on known as well as
on lurking variables.
3. Replication: allows us to attribute observed effects to
the treatments rather than ordinary variability.
5
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Component 1: Control or Comparison Group
 A placebo is a dummy treatment, i.e. sugar pill. Many
subjects respond favorably to any treatment, even a
placebo.
 A control group typically receives a placebo. A control
group allows us the analyze the effectiveness of the
primary treatment.
 A control group need not receive a placebo.
Clinical trials often compare a new treatment for
a medical condition, not with a placebo, but with
a treatment that is already on the market.
6
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Component 1: Control or Comparison
Group
Experiments should compare treatments rather than
attempt to assess the effect of a single treatment in
isolation.
 Is the treatment group better, worse, or no different
than the control group?
Example: 400 volunteers are asked to quit smoking and
each start taking an antidepressant. In 1 year, how many
have relapsed? Without a control group (individuals who
are not on the antidepressant), it is not possible to gauge
the effectiveness of the antidepressant.
7
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Placebo Effect
Placebo effect (power of suggestion). The “placebo
effect” is an improvement in health due not to any
treatment but only to the patient’s belief that he or she will
improve.
In some experiments, a control group may receive an
existing treatment rather than a placebo.
8
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Component 2: Randomization
To have confidence in our results we should randomly
assign subjects to the treatments. In doing so, we
9

eliminate bias that may result from the researcher
assigning the subjects.

balance the groups on variables known to affect the
response.

balance the groups on lurking variables that may be
unknown to the researcher.
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Component 3: Replication
Replication is the process of assigning several
experimental units to each treatment.
10

The difference due to ordinary variation is smaller with
larger samples.

We have more confidence that the sample results
reflect a true difference due to treatments when the
sample size is large.

Since it is always possible that the observed effects
were due to chance alone, replicating the experiment
also builds confidence in our conclusions.
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Blinding the Experiment
 Ideally, subjects are unaware, or blind, to the treatment
they are receiving.
 If an experiment is conducted in such a way that neither
the subjects nor the investigators working with them
know which treatment each subject is receiving, then
the experiment is double-blinded.
 A double-blinded experiment controls response bias
from the subject and experimenter.
11
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Define Statistical Significance
If an experiment (or other study) finds a difference in two
(or more) groups, is this difference really important?
If the observed difference is larger than what would be
expected just by chance, then it is labeled statistically
significant.
Rather than relying solely on the label of statistical
significance, also look at the actual results to determine if
they are practically significant.
12
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
Generalizing Results
Recall that the goal of experimentation is to analyze the
association between the treatment and the response for
the population, not just the sample.
However, care should be taken to generalize the results of
a study only to the population that is represented by the
study.
13
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
SUMMARY: Key Parts of a Good
Experiment

A good experiment has a control comparison group,
randomization in assigning experimental units to treatments,
blinding, and replication. The experimental units are the
subjects—the people, animals, or other objects to which the
treatments are applied.
 The treatments are the experimental conditions imposed on
the experimental units. One of these may be a control (for
instance, either a placebo or an existing treatment) that
provides a basis for determining if a particular treatment is
effective. The treatments correspond to values of an
explanatory variable.
14
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.
SUMMARY: Key Parts of a Good
Experiment
 Randomize in assigning the experimental units to the
treatments. This tends to balance the comparison groups
with respect to lurking variables.
 Replicating the treatments on many experimental units
helps, so that observed effects are not due to ordinary
variability but instead are due to the treatment. Repeat
studies to increase confidence in the conclusions.
15
Copyright © 2013, 2009, and 2007, Pearson Education, Inc.