8. Designing experiments
The Practice of Statistics in the Life Sciences
Third Edition
© 2014 W. H. Freeman and Company
Objectives (PSLS Chapter 8)
Designing experiments

Experimental terminology

Comparative, randomized experiments

Completely randomized designs

Block designs

Matched pairs designs

Ethics and experimentation
Terminology

The individuals in an experiment are the experimental units. If
they are human, we call them subjects.

The explanatory variables in an experiment are often called factors.

A treatment is any specific experimental condition applied to the
subjects. If an experiment has several factors, a treatment is a
combination of specific levels of each factor.

The factor may be the administration of a drug.

One group of people may be placed on a diet/exercise program for 6
months (treatment), and their blood pressure (response variable) would
be compared with that of people who did not diet or exercise.
Comparative, randomized experiments
Experiments compare the response to a given treatment versus:

another treatment

the absence of treatment (often called a control)

a placebo (a fake treatment)
Experiments randomize the assignment of subjects to treatments.
Experiments use replication: several or many individuals are studied.
Inventing experimental design
Fisher was sent to a UK agricultural station to evaluate the effect
of fertilizers. He found decades worth of bad data.

Fertilizer had been applied to a field one year and not in another to compare
the yield of grain produced in the two years.

Fertilizer was applied to one field and not to a nearby field in the same year.
Satellite and soil recordings show that soil types
and moisture vary, even within a single field.
F

F
F F F F
Solution: Randomized comparative experiments.
He selected many fields and randomly assigned
the fields to receive fertilizer or not. Grain yield was
then compared for the two conditions.
F
F
F
F F
F
F
F F
F
F F F
F
F F
F
F
F
F F
F F F F
F F F
F
F
Importance of design
Gastric freezing was once a recommended treatment for peptic
ulcers. Patients would swallow a balloon through which a refrigerated
liquid was pumped for an hour to cool the stomach. The treatment was shown to
be safe and significantly reduce ulcer pain and it was widely used for years
(2500 gastric freeze machines sold and 15,000 patients chilled).
A randomized comparative experiment was later performed to compare the
effect of gastric freezing with that of a placebo:
- 28 of the 82 patients subjected to gastric freezing improved
- 30 of the 78 patients in the control group improved
Gastric freezing was then abandoned.
About the “placebo effect”
Improvement in health or perceived condition due not to any active treatment,
but only to the patient’s belief that he or she is being cared for or helped.

therapeutic results on up to 35% of patients

famous placebo: kiss/blow/hug/Band-aid
to help kids cope with minor injuries

neural response to the placebo effect
seen as early as the spinal cord
Science (2009), DOI: 10.1126/science.1180142

“negative placebo effect” also observed;
labeling an active drug a placebo lowers
its effectiveness
Sci Transl Med (2014), DOI: 10.1126/scitranslmed.3006175
Design issue: Bias and blinding
Bias, conscious or unconscious, from the placebo effect (subjects) or
the experimenter is a challenge. “Blinding” can help against bias.
A double-blind experiment is one in which neither the subjects nor
the experimenter(s) know which individuals received which treatment
until the experiment is completed.
However, subjects must be informed that they will get one of a
number of treatments, and must consent to that condition (it
would be unethical otherwise).
doi:10.2519/jospt.2008.2791
doi: 10.1016/j.pain.2011.04.006
Lack of realism
Is the sample representative of the target
population? Random sampling is meant to
gain information about a larger population.
population
sample
Is the treatment appropriate for the response you want to study?

FDA requiring lower recommended dose for certain sleep drugs containing
zolpidem (Jan. 10, 2013): “Since women eliminate zolpidem from their bodies
more slowly than men, the FDA has notified the manufacturers that the
recommended dose should be lowered for women.”

Carcinogenicity studies administer high doses of a
potential carcinogen to lab rats. Results don’t always
apply to humans (e.g., saccharin delisted in 2000).
Some common experimental designs
In a completely randomized experimental design individuals are
randomly assigned to groups, then the groups are assigned to
treatments completely at random.
In a block design, subjects are divided into blocks (groups sharing a
given characteristic) before the randomization, in order to account
for possible differences between the blocks.
The blocks are NOT random.
Randomization occurs inside each block.
A block design lets us choose how many individuals of each
block will receive each treatment.
Repeated measures and matched pairs designs
 Choose pairs of subjects that are closely matched (like
twins). Within each pair, randomly assign who will receive
which treatment.
 Or give the two (or more) treatments to each subject over time, in
random order, so we have repeated measures for each subject.
Each subject is given two chili bowls: version A and version B. Subjects eat
and rate both versions.
Half of all subjects are given version A first then version B.
The other half are given version B first then version A.
What experimental design?
Six male runners participated in two running trials
separated by one week. In random order, they
consumed either chocolate milk or a Gatorade-like
sports drink after their run. Protein synthesis was
assessed three hours later.
In a study of sickle-cell anemia, 152 patients were given the
drug hydroxyurea, and 147 were given a placebo (dummy pill),
based on random assignment. The researchers counted the
episodes of pain in each subject at the end of the study.
Ethics and experimentation

Biology deals with life. Experimentations have an impact on live
subjects and ecosystems. What rights do human subjects, animals,
ecosystems have?

There is a difference between what can physically be done and what
can be done ethically. When is it okay/not okay to include a placebo
group? When should an experiment be interrupted?

Personal standards vary, and extreme
experimentations have occurred.
Committees have been established
to review all research proposals.
Subjects must give “informed” consent.