ACMS 20340
Statistics for Life Sciences
Chapter 8:
Designing Experiments
Fisher’s Experiments
Experiment Terminology
individuals (if human)
explanatory variable
→
→
subjects
factors
In conducting an experiment, we try to keep as many variables
constant while only changing the designated factor variables.
A treatment is a specific combination of factors applied to the
subjects.
The experiment compares the response to a given treatment to
other treatments, no treatment, or a fake treatment.
Experiments versus Observational Studies
Experiments are in many ways preferable to observational studies.
I
With an experiment we can study the specific treatments we
are interested in.
I
With an experiment we can study the combined effects of
many factors simultaneously.
Corn and chicks: an example
How do two specific varieties of genetically modified corn and
various protein-level diets affect the growth of newborn chicks?
The factors are
I corn type: Opaque-2, Floury-2, Normal Corn
I protein level: 12%, 16%, 20%
There are 9 different treatment groups between the two factors.
Control is important
In medical experiments, there are usually at least two groups
considered.
One group serves as a baseline against which we compare the
treatment we are interested in. This is called the control group.
We would rather the subjects didn’t know which group they belong
to, so a placebo is usually used.
A placebo is a treatment given to the control group which is fake
but otherwise indistinguishable from the treatment given to the
experimental group.
The placebo effect occurs when subjects feel the treatment
worked just from the psychological impact of having a treatment.
Placebo A-Go-Go
Principles of Experimental Design
Here are some basic principles of experimental design.
I
Use a control group to keep lurking variables at bay.
I
Randomize as much as possible to head off selection bias.
I
Use enough subjects to reduce the effect any one subject has
on the results.
The goal is to see a difference between the experimental groups
large enough to not be a chance variation.
By using enough subjects and assigning enough to each group, the
average response observed in each group becomes more likely to be
representative the population with that treatment.
An observed effect which is large enough to occur only rarely by
chance is called statistically significant.
How do we know what counts as chance variation? We use
probability (chapter 9).
Types of experimental design
These basic principles for experimental design describe
randomized comparative experiments.
A randomized comparative experiment uses both the comparison of
two or more treatments and chance assignment of subjects to
treatments.
Three sub-categories of randomized comparative experiments
which we will discuss are:
I
Completely randomized design
I
Matched pairs design
I
Block design
Completely randomized design
Each subject in a completely randomized design is assigned one of
any of the possible groups at random.
I
We always determine in advance how large each group should
be.
The chicks experiment is an example of a completely randomized
design.
Gastric freezing: an example
Matched Pairs Design I
Completely randomized designs are useful, but they don’t always
give the most precise results.
Sometimes it helps to match the subjects in various ways.
For instance, a matched pairs design combines matching with
randomization.
A matched pairs design compares only two treatments.
We choose pairs of subjects that are as closely matched as
possible.
I
E.g., individuals of the same sex, age, weight, etc.
I
E.g., genetically related individuals such as twins or animals
born in the same litter.
Matched Pairs Design II
We assign one treatment to one member of the pair, and the other
treatment to the other.
How do we decide which member gets which treatment?
CHANCE!
Why do we randomize the order of treatment?
The order of the treatment might influence the response of the
subjects, so we randomize to rule out a possible bias.
The pairs in matched pairs designs need not always consist of two
distinct individuals.
We can consider the same subject twice, applying one treatment
after the other (in a randomized order, of course).
Drugs: an example
Generics are brand-name drugs manufactured by a different
company but with identical active ingredients and properties.
Individuals are given either Brand X or its generic version on a
given day so that drug absorption can be measured.
One week later, each individual receives the other drug to measure
drug absorption.
A difference in absorption extent between Brand X and its generic
is then calculated for each individual.
Block design
Closely related to a matched pairs design is a block design.
Here’s an example:
Block design
The idea is to test blocks of individuals.
A block is simply a group of individuals that are known before the
experiment to be similar in some way that is expected to affect the
response to the treatments.
In a block design, the random assignment of individuals to
treatments is carried out separately within each block.
Soybeans: an example
We would like to study the
effects of two different types
of tillage and three different
types of pesticides on soybean
yields.
Soybeans: an example
To do so, we first divide the
area we are testing into
smaller blocks.
Soybeans: an example
Next, we subdivide each of
these blocks into six smaller
plots.
Soybeans: an example
Lastly, we randomly apply
each of the six treatments to
one of the six plots in each
block.
Some Cautions
According to the text,
“The logic of a randomized comparative experiment depends on
our ability to treat all the subjects identically in every way except
for the actual treatments being compared.”
We can do better:
“In performing a randomized comparative experiment, we should
do our best to ensure that the subjects are treated identically in
every way except for the actual treatments being compared.”
How can we achieve this?
One way to treat the subjects identically in every way except for
the treatments we give to them is by means of a double-blind
experiment.
In a double-blind experiment, neither the subjects nor the people
who administer the experiment know which treatment each subject
is receiving.
Why be this careful?
We don’t want the way the experiment is administered to give rise
to some bias.
For instance, a patient’s perception of his or her prognosis can be
influenced by how he or she interacts with a doctor. This might
create or reinforce a placebo effect.
Double-blind experiment: an example
A study was conducted to
determine whether the herbal
supplement ginkgo biloba can
help alleviate tinnitus, a
ringing noise in the ears that
doesn’t have an effective
pharmaceutical treatment.
Double-blind experiment: an example
978 healthy adults with tinnitus were matched by age and sex.
Within each pair, one individual received the ginkgo biloba
treatment, while the other was given a placebo.
Moreover, the experiment was double-blind, as neither the subjects
nor the investigators knew who was receiving the ginkgo biloba and
who was receiving the placebo, as the tablets were
indistinguishable.
Double-blind experiment: an example
The tablets were distributed to the subjects in coded bottles, and
the code was revealed only after the experiment had concluded and
the data had been collected.
Other issues: Replication
Convincing evidence usually requires more than just a
well-designed experiment that takes into account a number of
potential sources of bias.
It requires that the study be successfully replicated by the
investigators as well as investigators from different locations.
The environment of an experiment might influence the outcome of
that experiment.
Other issues: Realism
“The most serious potential weakness of experiments”:
lack of realism
That is, the subjects or treatment or setting of an experiment may
not realistically duplicate the situations we want to study.
For example, saccharin causes bladder cancer in rats, but it turns
out that this cancer is specific to the rat urinary system and is
related to saccharin consumption at concentrations higher than
what is realistic for human consumption.
Other issues: Ethics
Should we consider as acceptable all experiments that are
performed in the name of science, or for the sake of knowledge, or
for the sake of the public’s greater good?
Is it morally right to include a placebo in a study design when
there are safe and efficient treatments already available?
Should we continue an experiment if we find that one of the
treatments shows early signs of adverse effects? And what if one of
the treatments shows early signs of clear superiority?
Is it ever acceptable to leave subjects in the dark about the
purpose and expected results of an experiment?
Nope!
Other issues: Ethics
“The interests of the subject must always prevail over the interests
of science and society.”
I
The 1964 Helsinki Declaration of the World Medical
Association
All studies funded by the U.S. government must now obey the
following principles:
I
Those carrying out the study must have an institutional
review board to protect subjects from possible harm.
I
All subjects must give their informed consent in writing before
data is collected.
I
Subjects must be informed about the nature of a study and
any risk of harm it may bring.
I
All individual data must be kept confidential.
Ethics: Tuskeegee syphilis study
“For 40 years, the U.S. Public Health Services has conducted a
study in which human guinea pigs, not given proper treatment,
have died of syphilis and its side effects. The study was conducted
to determine from autopsies what the disease does to the human
body.”
Ethics: Tuskeegee syphilis study
Beginning in 1932, nearly 400 poor black men with syphilis were
enrolled in the study.
Originally intended to be run for six months, the study lasted until
a whistle-blower turned to the press in 1972.
The subjects of the study were never told they had syphilis, and
were also never told that they were part of a medical experiment.
Penicillin was known to be a highly effective cure in 1943, and
became widely available by 1947.
It was not given to any of the subjects in the study.