Repeated Measures ANOVA
How to test and
“control” for individual
differences when
performing an ANOVA
http://www.pelagicos.net/classes_biometry_fa16.htm
What do I want You to Know
• What is repeated measures (RM) ANOVA?
• What is the theory behind RM ANOVA?
• What are the mechanics of RM ANOVA?
• However, you will not need to do this test;
either using SPSS, or by hand
Reading - Field:
Chapter 13
AIMS
• Rationale of Repeated Measures ANOVA
• Advantages and Disadvantages
• One-way and two-way
• Partitioning Variance
• Statistical Problems with Repeated Measures Designs
• Defining and Quantifying Sphericity
• Testing and Overcoming this Assumption
• Interpretation of Repeated Measures ANOVA
Rationale for Repeated Measures
Group 1
Group 2
Lecturing
Skills
• Paired Samples “control” individual variation:
• Variance created by our manipulation
– Brain Transplant (systematic variance)
•
–
Variance created by unknown factors
Differences in ability (unsystematic variance)
Dependent vs Independent Samples
Approach 1. Randomly assign individuals to one treatment:
Group
1
Group
2
Outcome: Independent Samples – Not linked in any way
Approach 2. Each individual measured repeatedly (control):
Outcome: Dependent Samples – Linked by individual
Repeated Measures ANOVA
• Enhances Sensitivity of ANOVA:
– Unsystematic variance is reduced.
– More sensitive to experimental effects.
• Facilitates larger Experiments:
– Less participants needed for experiments.
– But, beware of potential biases
(e.g. subject fatigue – randomize order).
Repeated Measures ANOVA - Theory
Basic Idea:
We perform a standard ANOVA
(compare the group means using one factor).
But include another factor (individual) – to
account for the lack of independence in the
repeated measurements from same subjects.
For Example: Ice Skating Competition Scores
Participant 1
Judge
Judge
Judge
Judge
Judge
1
2
3
4
5
Participant 2
Participant 3
Repeated Measures ANOVA - Example
• Question: Are certain foods from the
Australian bush more revolting than others?
• Four Foods tasted by 8 celebrities:
–
–
–
–
Stick Insect
Kangaroo Testicle
Fish Eyeball
Witchetty Grub
• Outcome:
– Time to throw up (seconds).
Repeated Measures ANOVA - Example
Celebrity
Stick
Insect
Testicle
Fish Eye
Witchetty
Grub
Mean
Variance
Df
1
8
7
1
6
5.50
9.67
3
2
9
5
2
5
5.25
8.25
3
3
6
2
3
8
4.75
7.58
3
4
5
3
1
9
4.50
11.67
3
5
8
4
5
8
6.25
4.25
3
6
7
5
6
7
6.25
0.92
3
7
10
2
7
2
5.25
15.58
3
8
12
6
8
1
6.75
20.92
3
Mean
8.13
4.25
4.13
5.75
Grand Mean = 5.56
Degrees of Freedom:
Treatments (levels): 4 – 1 = 3
Overall: 8 * 3 = 24
NOTE: Total observations is 32 (but we loose 8 df – one per person)
24
Repeated Measures ANOVA - Example
SST
Variance between all scores
SSW
SSBetween
Variance Within Individuals
SSM
Effect of Experiment
SSR
Error
ANOVA – Assumptions
• ANOVA is a parametric test based on normal
distributions. Therefore, it assumes:
Observations are randomly sampled.
Data are on the interval or ratio scale .
Sampling distributions normally distributed.
• Because ANOVA compares different groups
of observations, it assumes:
– Variances in these populations are equal
(homogeneity of sample variances).
– Scores in different treatment conditions
are independent (from different individuals).
Repeated Measures ANOVA - Problems
• Same participants in all treatments
– Scores across treatments are correlated.
– Thus, violates assumption of independence.
• Assumption of Sphericity.
– Correlation across conditions should be equal.
– Different groups (levels) have equal variances.
Repeated Measures ANOVA - Problems
• Same participants in all treatments
YES – but this dependence is in the core of
Repeated Measures. Test deals with this
lack of independence by removing one degree
of freedom for each participant (subject).
• Assumption of Sphericity.
YES – This is a new RM ANOVA assumption.
We need to test for this assumption.
If test significant, adjust Model and Error df.
Sphericity - Introduction
Sphericity is a new mathematical assumption
unique to the repeated measures ANOVA designs.
Consider a one-way independent measures ANOVA:
One of the mathematical assumptions is that the variances
of the populations that groups are sampled from are equal.
This homogeneity of variance assumption follows from
the null hypothesis being tested in ANOVA, as follows:
If the treatment has no effect on the dependent variable,
we can consider all the groups to be sampled from the same
population. However, sampling makes it difficult to observe
exactly equal variances (even if assumption perfectly met).
Sphericity - Definition
The sphericity assumption (in repeated measures ANOVA)
is an extension of the homogeneity of variance assumption
(in independent measures ANOVA).
What is sphericity?
Sphericity relates to the equality of the variances of the
differences between levels (treatments) of the repeated
measures factor. Sphericity requires that the variances
for each set of difference scores are equal.
Food 1 – GrandMean
Food 2 – GrandMean
Food 3 – GrandMean
Food 4 – GrandMean
Food 1
The variances from these differences
should be equal (e.g., we can calculate the
mean and the variance of the differences)
Values
0, 0, 0, 0, 0, 0, 8, -8
Mean of difference = 0
Variance of difference = 18.3
Sphericity - Example
Testicle Stick
Eye –
Stick
Witchetty
– Stick
Eye –
Testicle
Witchetty Witchetty
– Testicle
– Eye
1
-1
-7
-2
-6
-1
5
2
-4
-7
-4
-3
0
3
3
-4
-3
2
1
6
5
4
-2
-4
4
-2
6
8
5
-4
-3
0
1
4
3
6
-2
-1
0
1
2
1
7
-8
-3
-8
5
0
-5
8
-6
-4
-11
2
-5
-7
Variance
5.27
4.29
25.70
11.55
14.29
26.55
Null Hypothesis:
All treatments (levels) have the same variance
Sphericity – Testing
• Three measures of Epsilon:
– Lower-bound Estimate
– Greenhouse-Geisser Estimate
– Huynh-Feldt Estimate
NOTE: L-b conservative, G-G meddium, H-F liberal.
• One statistical test: Mauchly’s W.
P < 0.05, Reject Null Hypothesis: Sphericity violated.
P > 0.05, Accept Null Hypothesis: Sphericity is met.
Sphericity - Correction
Df = Total is 24 = 3 (levels), 21 (error)
Mauchly's Test of Sphericity
Measure: MEASURE_1
Epsilon
Within Subjects Ef f ect Mauchly 's W
Animal
.136
Approx.
Chi-Square
11.406
df
5
Sig.
.047
Greenhouse
-Geisser
.533
Huy nh-Feldt
.666
Lower-bound
.333
Tests the null hy pothesis that t he error cov ariance matrix of the orthonormalized transf ormed dependent v ariables is
proportional to an identity mat rix.
Outcome: Variances of Different levels Not Equal
Implication: Modify df – multiply by Epsilon
Sphericity – Corrected Testing
Animal refers to different levels experienced by each participant (subject)
RM ANOVA compares variability within each participant with the error
Tests of Within-Subjects Effects
Measure: MEASURE_1
Source
Animal
Error(Animal)
Sphericity Assumed
Greenhouse-Geisser
Huy nh-Feldt
Lower-bound
Sphericity Assumed
Greenhouse-Geisser
Huy nh-Feldt
Lower-bound
Ty pe II I Sum
of Squares
83.125
83.125
83.125
83.125
153.375
153.375
153.375
153.375
df
3
1.599
1.997
1.000
21
11.190
13.981
7.000
Mean Square
27.708
52.001
41.619
83.125
7.304
13.707
10.970
21.911
F
3.794
3.794
3.794
3.794
Sig.
.026
.063
.048
.092
For each scenario: uncorrected (green) or three corrected values
(red), the test calculates the MS square terms (using different dfs)
and the resulting F ratio = MS model / MS error.
Note: Only uncorrected and “liberal” correction (H-F) significant
Repeated Measures ANOVA Interpretation
We can plot
the mean +/SD for each
group (level)
Bushtucker Trials
10
9
8
Time to Retch (s)
Same eight
subjects
measured for
each “animal”
treatment
(the four
test levels)
7
6
5
4
3
2
1
0
Stick Insect
Kangaroo Testicle
Animal
Fish Eye
Witchetty Grub
Repeated Measures ANOVA – Post-Hoc
• When the RM ANOVA yields significance,
proceed with a series of post-hoc tests.
• Compare each group (level) mean against all
other means using t-tests.
• Use stricter criterion to determine significance.
– Hence, control the familywise error rate.
– Simplest example is the Bonferroni method:
Bonferroni  

Numberof Tests
Repeated Measures ANOVA - SPSS
4 treatments (levels)
8
participants
(subjects)
Null
Hypothesis:
At least one
treatment has a
different mean
What test do we use to analyze these data?
- First Question:
How many factors are we testing? 1 or MORE
- Second Question:
Are the data independent or paired?
Repeated Measures ANOVA - SPSS
One-Way ANOVA
(factor1: Food – 4 categories)
(dependent data – 8 subjects)
Dependent: Time to Barf (secs.)
Repeated Measures ANOVA - SPSS
Go to: Analyze >
GLM > Repeated Measures
One-Way ANOVA
(factor: animal to eat)
Dependent (paired) Data
(8 subjects; measured 4 times)
What other settings do we need to decide on?
Post-hoc Tests OR Planned Comparisons
Repeated Measures ANOVA - SPSS
You can select the following:
- Contrasts
- Plots
- Post-hoc Tests
- And other Options
Options for Contrasts:
Make sure FACTORS
selected as “Repeated”
Use CHANGE to modify selection
Repeated Measures ANOVA - SPSS
Select plots to visualize
the effects of the factors
Typically, there are three options:
Each Animal gets a separate line
Each Animal gets a separate entry
Each Animal gets a separate plot
NOTE: Some only work for “levels”
Repeated Measures ANOVA - SPSS
Select Post-Hoc to perform
group comparisons after ANOVA
NOTE: Other
factors listed if
multi-way ANOVA
NOTE:
In this case, there are no
additional factors to test
post-hoc after ANOVA.
Levels tested already.
Repeated Measures ANOVA - SPSS
Show Means for
Levels (Animal)
Compare Effects
(add Bonferroni)
Select Display
(add Ticks)
Output: Descriptive Stats, Residuals
Tests: Homogeneity (variances)
Repeated Measures ANOVA - SPSS
You can select the following:
Predicted Values
Diagnostics
Residuals
Repeated Measures ANOVA - SPSS
Output: Descriptive Statistics, Level Definitions
Repeated Measures ANOVA - SPSS
Output: Sphericity Test Result
Repeated Measures ANOVA - SPSS
Output: Differences within and between subjects
Repeated Measures ANOVA - SPSS
Output: Group means
Output: Post-hoc Tests
for levels; Grand Mean (comparing level pairs)
1 different from 2 and 3
2 different from 1
3 different from 1
4 different from none
Repeated Measures ANOVA - SPSS
Output: Plot of Level Means
NOTE:
SPSS does
not provide
error bars,
only means
NOTE: Other plot formats for actual factors
(not levels)
Repeated Measures ANOVA – Summary
• One-way ANOVA cannot deal with dependent (paired)
observations from the same subjects.
• Repeated Measures ANOVA is able to incorporate
these paired observations, explicitly controlling for
their dependence (decreasing model and error df).
• Repeated Measures ANOVA uses Post-Hoc tests to
determine which treatments (levels) are different.
•
NOTE: Repeated Measures ANOVA can be used in a
One-Way (only that factor) or a Multi-Way design
(repeated measure level and one other factors).