26. Follow-up tests and
two-way ANOVA
The Practice of Statistics in the Life Sciences
Third Edition
© 2014 W.H. Freeman and Company
Objectives (PSLS Chapter 26)
Follow-up tests and two-way ANOVA

Comparing several means

Tukey’s pairwise multiple comparisons

Contrasts

Two-way designs

The two-way ANOVA model

Interaction

Inference for two-way ANOVA
Comparing several means
When comparing >2 populations, the question is not only whether each
population mean µi is different from the others, but also whether they
are significantly different when taken as a group.
If we split a class of 200 into 20 random samples of 10 students and found
the 20 mean heights, the 2 most extreme samples could very well be
“significantly different” --- but that would be a Type 1 error.
We should ask instead if all 20 samples might have come from a single
population or not.
Handling multiple comparisons statistically

The first step in examining multiple populations statistically is to test
for an overall statistical significance as evidence of any difference
among the parameters we want to compare.
ANOVA F test (Chapter 24)

If that overall test showed statistical significance, then a detailed
follow-up analysis can examine all pair-wise parameter comparisons
to define which parameters differ from which and by how much.
 pairwise multiple comparisons
Tukey’s pairwise multiple comparisons
Pairwise multiple comparisons among k groups are variants of the two-
sample t procedures. However,

they use the pooled standard deviation sp =√MSE

the pooled degrees of freedom DFE = N – k

and they compensate for the multiple comparisons.
Simultaneous confidence intervals
We can calculate simultaneous level C confidence intervals for all
pairwise differences (µi – µj) between population means:
CI : ( xi  x j )  m * s p
1 1

ni n j
m* depends on the confidence level C, the number of populations k,
and the total number of observations N. Use technology or Table G.
Table G
Simultaneous tests of hypotheses
To carry out simultaneous Tukey tests of the hypotheses
H0: μi = μj
Ha: μi ≠ μj
for all pairs of population means, reject H0 for any pair whose Tukey
level-C confidence interval does not contain 0.
These tests have an overall significance level no less than 1 − C.
That is, 1 − C is the probability that, when all of the population means
are equal, any of the tests incorrectly rejects its null hypothesis.
Male lab rats were randomly assigned to 3 groups differing in access
to food for 40 days. The rats were weighed (in grams) at the end.
•
•
•
Group 1: access to chow only.
Group 2: chow plus access to cafeteria food restricted to 1 hour every day.
Group 3: chow plus extended access to cafeteria food (all day long every day).
H0: µchow = µrestricted = µextended
Ha: H0 is not true
One-way ANOVA: Chow, Restricted, Extended
Source
Factor
Error
Total
DF
2
47
49
SS
63400
139174
202573
MS
31700
2961
F
10.71
P
0.000
Pooled StDev = 54.42
With Table G,
m* ≈ 2.434 (df 3 and 47≈ 40),
Tukey 95% CI for µextended – µrestricted:
691.13  657.31  2.43454.42
1
1

15 16
33.82  47.61, or  13.79 to 81.43 g
Chow
Restricted
Extended
N
19
16
15
Mean
605.63
657.31
691.13
Tukey 95% Simultaneous Confidence Intervals
All Pairwise Comparisons
Individual confidence level = 98.05%
Chow subtracted from:
Restricted
Extended
Lower
7.03
40.05
Center
51.68
85.50
Upper
96.33
130.95
----+---------+---------+---------+----(-------*------)
(------*-------)
----+---------+---------+---------+-----60
0
60
120
Restricted subtracted from:
Extended
Lower
-13.47
Center
33.82
Upper
81.12
----+---------+---------+---------+----(-------*-------)
----+---------+---------+---------+-----60
0
60
120
 There are two significant differences at simultaneous significance level 5%:
μrestricted – μchow and μextended – μchow
Two-way designs
In a two-way design, 2 factors are studied in conjunction with the
response variable. There is thus two ways of organizing the data, as
shown in a two-way table.
Color
Two-way table
Texture
Red
Green
Blue
Smooth
14
14
14
(3  3 design to test the
Grooves
14
14
14
attractiveness of
Bumps
14
14
14
sticky boards to beetles)
When the response variable is quantitative, the data are analyzed with
a two-way ANOVA procedure. A chi-square test is used instead if the
response variable is categorical.
Advantages of a two-way ANOVA model

It is more efficient to study two factors at once than separately.
Smaller samples sizes per condition are needed because the samples
for all levels of factor B contribute to sampling for factor A.

Including a second factor thought to influence the response variable
helps reduce the residual variation in a model of the data.
In a one-way ANOVA for factor A, any effect of factor B is assigned to
the residual (“error” term). In a two-way ANOVA, both factors contribute
to the fit part of the model.

Interactions between factors can be investigated.
The two-way ANOVA breaks down the fit part of the model between two
main effects and an interaction effect.
The two-way ANOVA model

We record a quantitative variable in a two-way design with R levels
of the row factor and C levels of the column factor.

We have independent SRSs from each of R x C Normal populations.
Sample sizes do not have to be identical (although some software only
carry out the computations for equal sample sizes  “balanced design”).

All parameters are unknown. The population means may be different
but all populations have the same standard deviation σ.
Main effects and interaction effect

Each factor is represented by a main effect: this is the impact on the
response variable of varying levels of that factor, regardless of the
other factor (i.e., pooling together the levels of the other factor).
There are two main effects, one for each factor.

The interaction of both factors is also studied and is described by the
interaction effect.

When there is no clear interaction, the main effects are enough to
describe the data. In the presence of interaction, the main effects
could mask what is really going on with the data.
Major types of two-way ANOVA outcomes
In a two-way design, statistical significance can be found for each factor,
for the interaction effect, or for any combination of these.
Neither factor is
significant
Only 1 factor is
significant
Both factors are
significant
No interaction
Interaction effect is
significant
No interaction
With or without
significant interaction
Levels of factor A
Levels of factor A
Levels of factor B:
B1
B2
Dependent var.
Dependent var.
Neither factor is
significant
Levels of factor A
Levels of factor A
Interaction
Two variables interact if a particular combination of variables leads to
results that would not be anticipated on the basis of the main effects of
those variables.

Drinking alcohol increases the chance of throat cancer. So does
smoking. However, people who both drink and smoke have an even
higher chance of getting throat cancer. The combination of smoking and
drinking is particularly dangerous: these risk factors interact.
An interaction implies that the effect of one variable differs depending
on the level of another variable.

The effect of smoking on the probability of getting throat cancer is greater
for people who drink than for people who do not drink.
Interpreting plots
Main effects,
Important interaction effect: the
no interaction
main effects may be misleading
Inference for two-way ANOVA

A one-way ANOVA tests the following model of your data:
Data (“total”) = fit (“groups”) + residual (“error”)
so that the sum of squares and degrees of freedom are:
SST = SSG + SSE
DFT = DFG + DFE

A two-way design breaks down the “fit” part of the model into more
specific subcomponents, so that:
SST = SSR + SSC + SSRC + SSE
DFT = DFR + DFC + DFRC + DFE
where R and C are the 2 main effects from each of the 2 factors, and
RC represents the interaction of factors R and C.
The two-way ANOVA table
Source of
variation
DF
Sum of squares
SS
Mean
square MS
F
P-value
Factor R
DFR = r -1
SSR
SSR/DFR
MSR/MSE
for FR
Factor C
DFC = c - I
SSC
SSC/DFC
MSC/MSE
for FC
DFRC = (r-1)(c-1)
SSRC
Error
DFE = N - rc
SSE
SSE/DFE
Total
DFT = N – 1
SST
=DFR+DFC+DFRC+DFE
=SSR+SSC+SSRC+SSE
SST/DFT
Interaction
SSRC/DFRC MSRC/MSE
for FRC

Main effects: P-value for factor R, P-value for factor C.

Interaction: P-value for the interacting effect of R and C.

Error: It represents the variability in the measurements within the groups.
MSE is an unbiased estimate of the population variance s2.
Nematodes and plant growth
Do nematodes affect plant growth? A botanist prepares 16 identical
planting pots and adds different numbers of nematodes into the pots.
Seedling growth (in mm) is recorded two weeks later. We analyzed
these data with a one-way ANOVA in the previous chapter.
We also have data for another plant species. We can study the effect of
nematode amounts (4 levels) on seedling growth for both plant species (2
levels, species A and B).
Plant species A was also grown with pesticide. We can analyze seedling
growth for combinations of nematodes and pesticide conditions.
Nematode levels and plant species
Source
SS
NEMATODES
254.645
SPECIES
2.101
NEMATODES*SPECIES 34.124
Error
65.710
df
3
1
3
24
MS
84.882
2.101
11.375
2.738
F-ratio
31.002
0.767
4.154
P
0.000
0.390
0.017
All plants suffer from the presence of nematodes (main effect P < 0.001), but plant
A and plant B do not have significantly different growth (main effect P = 0.39).
The third plot shows that the effect of nematodes is less pronounced for plant A,
shown in red (interaction effect P = 0.017).
Nematode levels and pesticide
Source
NEMATODES
PESTICIDE
NEMATODES*PESTICIDE
Error
SS
68.343
29.070
36.711
41.438
df
3
1
3
24
MS
22.781
29.070
12.237
1.727
F-ratio
13.195
16.837
7.087
P
0.000
0.000
0.001
Both main effects are very significant (P < 0.001).
The interaction is significant (P = 0.001): We see from the third plot that the
detrimental effect of nematodes is much more pronounced in pesticide-free pots,
shown in red.
Dietary manipulations in fruit flies
An experiment assessed the percent of body fat in female fruit flies fed one of four
diets enriched with yeast (a high-protein food). Wild-type fruit flies and mutants with
a longer reproductive cycle were used.
Only one main effect is
significant: yeast amount
(P < 0.001)
There is no significant
interaction effect.
Better corn for heavier chicks?
A study compared the effect of 3 corn varieties (“norm,” “opaq” and “flou”) and 3 feed
protein levels on the growth of chicks. Male chicks were randomly allotted to
treatments at birth and their weight gain was recorded 21 days later.
The interaction effect is
significant (P < 0.001),
meaning that mean weight
gain differs for different
combinations of corn
variety and protein level.
Both main effects are also
significant (P ≤ 0.001).