Check your calendar…
If you are qualified, Fill out make-up
exam form by 5pm this Friday, attach
necessary documents.
3/23/12
Lecture 20
1
Interpreting the ANOVA
results
Chapter 9
3/23/12
Lecture 20
2
Assumptions (prior to ANOVA)
•  Two important assumptions for ANOVA
1.  Constant variance: The variances of the k
populations are the same.
– 
Check this with the ratio of the largest and smallest
standard deviations, the ratio must be < 2
2.  Each of the k populations follows a normal
distribution.
– 
3/23/12
Check this by looking at QQplots for each group
Lecture 19
3
Quick sidenote
•  ANOVA is a good example of a situation where often a
nonsignificant test is actually useful.
–  Suppose we are comparing a new drug to several standard
drugs already used
–  Suppose also that the new drug is less expensive to produce
–  In this case, mostly what we’d like to show is that the new
drug is at least effective as the other standard drugs used
–  So in this situation, a non-significant ANOVA is a great result!
•  Remember:
statistical significance ≠ practical significance
3/23/12
Lecture 19
4
9.3
Interpreting ANOVA results
•  If the results are significant in ANOVA, we’d
like to know explicitly which means are
different
•  Remark: If insignificant in ANOVA, we don’t
have to try further steps…
•  Two benefits of ANOVA
1.  Single test with single chance (α) of type I error
2.  Better estimation of error among all groups
• 
3/23/12
By comparing all the groups simultaneously, we get a
better picture of the overall error among groups.
Lecture 20
5
How do we know which means are
different?
•  We need some "Supplementary Analysis" to tell that.
–  One way is to do a check visually, using the "effect plots"
•  Scatter plot of means
•  Side-By-Side Boxplots
–  Another way is to perform multiple comparison of means
•  Tukey's method
•  Dunnett's method
•  many more: LSD, Scheffe, Bonferroni, FDR, etc.
Actually, We can compare the means pairwise and keep the two
benefits of ANOVA, we do this by adjusting the T value we use to
compare the two means.
3/23/12
Lecture 20
6
Visual Check - Boxplots
3/23/12
Lecture 20
7
Visual Check – Scatterplot of
Means
3/23/12
Lecture 20
8
Multiple comparisons (General Concepts)
•  How far apart do two means need to be to
be statistically significant?
–  This value can be calculated directly similar to
what we did with confidence intervals
•  It functions like a t from a t-test
–  Generally, the critical value (t for example) is
modified to “correct” the inflated type I error
rate to keep it at the desired α level (like 0.05)
•  So instead of an α error rate for each test, we get an
“family” α error rate—one rate for the entire
comparison
3/23/12
Lecture 20
9
Tukey’s Method
Controls the type I error rate directly by modifying
the T value
• 
T = qα
• 
– 
– 
• 
3/23/12
MSE 1 1
( + ) or
2 ni n j
T = qα
MSE if n = n
i
j
ni
qα comes from the studentized range tables
Table IX on pages 577-578
Df of qα is (k, n – k) for single-factor ANOVA
If the difference in two means is greater than this
critical value, we say those two means are
statistically significantly different
Lecture 20
10
More simply
•  Calculate T, calculate the differences xi − x j
•  If xi − x j < T, means are not significantly different
•  If xi − x j > T, means ARE significantly different
•  Remark: Tukey’s method is conservative,
sometimes its conclusion will be inconsistent with
that by using ANOVA test results.
If we took a less stringent alpha level we might see
some of the significant differences.
3/23/12
Lecture 20
11
Example—generic
•  Let’s say we are comparing 4 means with equal sample
sizes of ni = 5 for all i. With an MSE of 10.
•  Looking at Table IX, we have k = 4, and Error df = n – k
= 16
–  qα = 4.05
•  So, T = qα
MSE
10
= 4.05
= 5.73
ni
5
•  Any difference of means more than 5.73 apart would be
significant different
3/23/12
Lecture 20
12
Example—generic
•  Suppose that you have four treatment groups
and the treatment means are:
• TRT 1:
• TRT 2:
• TRT 3:
• TRT 4:
52
63
58
54
•  Which pairs are significantly different?
3/23/12
Lecture 20
13
Example (from Monday’s Class)
•  For the cereal example, let’s use Tukey’s method using
α = 0.01
•  The means are (arranged in descending order):
n4 = 5, x4 = 27.2
n3 = 4, x3 = 19.5
n1 = 5, x1 = 14.6
n2 = 5, x2 = 13.4
•  Note, group 3’s sample size, what effect will that have on
the comparisons?
3/23/12
Lecture 20
14
Another Method: Using SAS code
proc glm data=cereal alpha=0.01;
class design;
model cases = design;
means design / tukey cldiff;
run;
3/23/12
Lecture 20
15
Example (cont) using SAS
•  Notice it stars the pairs
that are significantly
different.
•  So the only pairs that are
significantly different are:
1 and 4
2 and 4
3/23/12
Lecture 20
16
Alternate SAS code
proc glm data=cereal alpha=0.01;
class design;
model cases = design;
means design / tukey lines;
run;
3/23/12
Lecture 20
17
Example (cont) using SAS
• 
• 
3/23/12
Lecture 20
Same information as before,
differences are:
1 and 4
2 and 4
Notice the nice “groupings”
though
18
Dunnett’s Multiple
Comparison
3/23/12
Lecture 20
19
3/23/12
Lecture 20
20
Multiple Comparison – Dunnett’s
Method
3/23/12
Lecture 20
21
Dunnett’s Method
Functions like a Tukey, just uses a different T
• 
1 1
T = tα (k − 1, n − k ) MSE ( + )
ni nC
• 
tα comes from the Dunnett’s t table
–  Table X on page 579
–  Only use when one of the groups is a control group
–  Only interested in comparing the “other” groups to
the control group
• 
3/23/12
Again we take pairwise differences,
Lecture 20
xi − xC
22
Example (Cereal Design) using SAS
•  Dunnett’s SAS code, pretend design 1 was the
regular design already used
proc glm data=cereal alpha=0.01;
class design;
model cases = design;
means design / dunnett(“1”) cldiff;
run;
Note: lines doesn’t work with Dunnett’s
3/23/12
Lecture 20
23
Example (cont) using SAS
Dunnett's t Tests for cases
NOTE: This test controls the Type I experimentwise error for
comparisons of all treatments against a control.
Alpha
Error Degrees of Freedom
Error Mean Square
Critical Value of Dunnett's t
0.01
15
10.54667
3.43026
•  So if group1 was the
control, only group
4 is significantly
different
Comparisons significant at the 0.01 level are indicated by ***.
Difference
design
Between Simultaneous 99%
Comparison
Means Confidence Limits
4-1
3-1
2-1
3/23/12
12.600
4.900
-1.200
5.554 19.646 ***
-2.573 12.373
-8.246 5.846
Lecture 20
24
Summary about ANOVA and
Multiple Comparison
•  ANOVA (Analysis of Variances)
–  Check the assumptions (constant variance/normality)
–  Be able to do most of ANOVA by hand or by SAS both
•  Lots of hand calculations
•  Be able to read and interpret SAS output
–  For Hw, do it either way you like, but for the exam be prepared to
do both!
•  Multiple Comparison methods (ONLY when ANOVA result is
significant)
–  are useful in other situations, but they all involve calculating a T
value and using it to compare pairs of means
–  Tukey’s is approprirate if there’s no control group; try Dunnett’s
if there is any control(s)
3/23/12
Lecture 20
25
After Class
•  Hw#8, due by 5pm next Monday
•  Start review of Exam 2 (Ch.7, 8 and
Monday’s complete notes)
–  Practice Test 2
–  Hw5-8, Lab 3 and 4
3/23/12
Lecture 20
26