Experimental Design
Step 1: Choose treatments
Identify all factors and levels
A Control group is always needed
Step 2: Assign the experimental units to the treatments
Randomization (randomly assign units to
each treatment group)
Well-designed experiment makes it possible to
draw conclusions about cause/effect.
Sources of Bias
 Experiment should compare treatments rather than
assess each individual treatment
 Placebo Effect and other lurking variables will then
operate on both groups
 Control Group - used to control effect of lurking variables
(First Principle of Statistical Design - Control of the
Effects of Lurking Variables).
 Comparison of several treatments is simplest form of
control.
Completely Randomized Experiments
 Randomization in dividing experimental units into
groups- 2nd Major Principle of Statistical Design of
Experiments
 Logic: produces groups of experimental units who should
be similar in all respects before treatment is applied;
 Influences other than experimental treatments operate
equally on all groups;
 Differences in response variable must then be due to the
effects of the treatments
Randomization in Experiments
When you want to make inferences about a population
(e.g. estimating the total yield of Farmer Brown’s
fields from a 10 plot sample) you need to start with a
representative sample (SRS, stratified sample, etc.).
But that's not the goal of an experiment. In an
experiment, we are trying to see if different
treatments lead to differences in the responses. The
inference is about the treatment, not about any
population. We needn't start with a random sample.
Instead we need to randomly allocate subjects to
treatments, thus balancing any unknown sources of
variability and creating independent groups to test our
factor on.
GOAL
Random
Sampling
To produce a sample that is
representative of the population of
interest.
Random
Allocation/
Assignment
To produce treatment groups that
are similar in all respects except for
the treatment imposed.
Statistical Significance
An observed effect too large to attribute
plausibly to chance.
Replication
 The more subjects used in an experiment,
the more likely that randomization will
create groups that are alike on average.
 When differences are averaged out, only
the effects of the different treatments
remain.
Principles of Experimental Design
 Control the effects of lurking variables on
the response, most simply by comparing
several treatments.
 Randomization, the use of impersonal
chance to assign subjects to treatments.
 Replication of the experiment on many
subjects to reduce chance variation in the
results.
Hidden Bias
 Unequal Conditions – all conditions must be the
same other than the assigned treatments
 Researchers knowledge of treatment may
effect how he/she records response variable
(example: amount of pain after treatment)
 Lack of Realism - subjects or treatments don’t
realistically duplicate conditions we want to
study. (Also, subjects know they are part of a
study).
Controlling Hidden Bias
 Blind Experiment - subjects don’t
know which treatment they are
receiving
 Double-Blind Experiment - neither
the subjects nor the people who
have contact with them know which
treatment a subject received.
Randomized Comparative
Experiment
Group 1
(# of subjects)
Treatment 1
(Control)
Compare
Treatments
Subjects
Random Assignment
Group 2
Treatment 2
(# of subjects)
(experimental)
Example:
A food company assesses the nutritional
quality of a new “instant breakfast” by
feeding it to newly weaned male rats.
The response variable is a rat’s weight
gain over a 28-day period. A control
group of rats eats a standard diet but
otherwise receives exactly the same
treatment as the experimental group.
Draw a grid to illustrate a completely randomized experiment.