Statistical Design of Experiments‐Part IV Analysis of Variance
Joseph J. Nahas
© Joseph J. Nahas 2012
10 Dec 2012
1
References
• NIST ESH 1.3.5.4 One‐Factor ANOVA and 1.3.5.5 Multi‐factor Analysis of Variance
• NIST ESH 7.4.3 Are the means equal? and subsections
–
–
–
–
–
–
–
–
7.4.3.1 1‐Way ANOVA overview
7.4.3.2 The 1‐way ANOVA model and assumptions
7.4.3.3 The ANOVA table and tests of hypotheses about means
7.4.3.4 1‐Way ANOVA calculations 7.4.3.5 Confidence intervals for the difference of treatment means
7.4.3.6 Assessing the response from any factor combination 7.4.3.7 The two‐way ANOVA 7.4.3.8 Models and calculations for the two‐way ANOVA • NIST ESH 7.4.4 What are variance components?
10 Dec 2012
© Joseph J. Nahas 2012
2
Outline
•
•
•
•
Analysis of Variance (ANOVA) Basics
2 Way ANOVA
2 Way ANOVA Example
Significance Testing using the F distribution
10 Dec 2012
© Joseph J. Nahas 2012
3
Analysis of Variance
• In an Analysis of Variance, the variation in the response measurements is partitioned into components that correspond to different sources of variation.
• The primary components are:
– Input factors
– Random variation
10 Dec 2012
© Joseph J. Nahas 2012
4
Sum of Squares
• The variance of n measurements is given by
n
s2 =
2
(
y
−
y
)
∑ i
i =1
n−1
• Analysis of Variance concentrates on the numerator of the variance calculation.
– This term is called the Total Sum of Squares or SS(Total)
• The numerator or SS(Total) is factored into component parts.
10 Dec 2012
© Joseph J. Nahas 2012
5
Single Factor Experiment
• For a single factor experiment with k treatments and ni
samples at each level:
SS(Total) = SST + SSE
k
ni
k
k
ni
2
2
2
(
y
−
y
)
=
n
(
y
−
y
)
+
(
y
−
y
)
∑ ∑ ij
∑ i i
∑ ∑ ij i
i =1 j =1
i =1
i =1 j =1
where
–
–
–
–
–
–
SS(Total) is the Total Sum of Squares
SST is the Treatment Sum of Squares
SSE is the Error Sum of Squares
k is the number of treatments
ni os the number of observations at each treatment level.
A treatment is a specific combination of factor levels whose effect is to be compared to other treatments.
10 Dec 2012
© Joseph J. Nahas 2012
6
Outline
•
•
•
•
Analysis of Variance (ANOVA) Basics
2 Way ANOVA
2 Way ANOVA Example
Significance Testing using the F distribution
10 Dec 2012
© Joseph J. Nahas 2012
7
Two Factor Model
• Two Factor Analysis of Variance
– Two factors A and B
ƒ A has levels 1, 2, . . . a
ƒ B has levels 1, 2, . . . b
• Model each sample data point, Yijk as made up of four parts:
–
–
–
–
The mean value: μ
The response to factor A: αi
The response to factor B: βj
An error term associated with that sample: εijk
Yijk = μ + α i + β j + ε ijk
i = 1,2,L a; j = 1,2,L b;k = 1,2,L r
NIST ESH 1.4.3.7
10 Dec 2012
© Joseph J. Nahas 2012
8
The Sample Data Structure
10 Dec 2012
© Joseph J. Nahas 2012
9
The Model (cont.)
• We can define the deviations from the mean so that they sum to zero.
a
∑α
i
=0
j
=0
i =1
b
∑β
j =1
• We can estimate the value of αi from the average over the sample containing that factor:
r
b
∑∑ y
αˆ i =
k =1 j =1
rb
ijk
− y = yi − y
10 Dec 2012
© Joseph J. Nahas 2012
10
Contributions to the Total Variation
• Because the deviations are defined around the mean, the variation in the sampled data from the global mean can be made up of three parts:
– The variation due to Factor A,
– The variation due to Factor B,
– The variation due to noise, i.e. the error.
• We can break the variation down into its component parts by looking at a sum of the squares:
SS(Total) = SS( A) + SS(B) + SSE
where – SS(A) is the weighted sum of squares for factor A,
– SS(B) is the weighted sum of squares for factor B, and
– SSE is the sum of squares for the error
10 Dec 2012
© Joseph J. Nahas 2012
11
Sum of Square Formulation
• The four components of the Sum of Squares are:
a
a
SS( A) = rb∑ αˆ = rb∑ ( y i − y ) 2
i =1
b
i =1
b
SS(B) = ra ∑ βˆ = ra ∑ ( y j − y ) 2
j =1
r
b
j =1
where rb is the number
of samples in yi
where ra is the number
of samples in yi
a
SSE = ∑ ∑ ∑ ( y ijk − y ij ) 2
k =1 j =1 i =1
r
b
a
SS(Total) = ∑ ∑ ∑ ( y ijk − y ) 2
k =1 j =1 i =1
Note the similarities
in the formulations
10 Dec 2012
© Joseph J. Nahas 2012
12
Expansion of a Variance Calculation
n
s2 =
Raw Sum of Squares
RSS
Corrected
Sum of Squares
CSS
2
(
y
−
y
)
∑ i
i =1
n−1
n
⎛
⎞2
⎜
∑ yj ⎟
n
⎜ y − j =1 ⎟
∑⎜ i n ⎟
i =1
⎜
⎟
⎝
⎠
=
n−1
⎛ n ⎞2
⎜∑ yi ⎟
n
⎝ i =1 ⎠
2
∑ yi − n
= i =1
n−1
Correction
for the mean
CM
Degrees of Freedom
DF
10 Dec 2012
© Joseph J. Nahas 2012
13
Calculating SS
⎛ r b a
⎞2
⎜⎜ ∑ ∑ ∑ y ijk ⎟⎟
2
(SumOfAllObservatios)
⎝ k =1 j =1 i =1 ⎠
CM =
=
rab
rab
SStotal = ∑ (EachObservation) 2 − CM
a
∑A
2
i
SS( A) =
r
− CM
i =1
rb
where
b
Ai = ∑ ∑ y ijk
k =1 j =1
a
∑B
SS(B) =
j =1
2
j
r
− CM
where
a
B j = ∑ ∑ y ijk
ra
SSE = SStotal − SS( A) − SS(B)
k =1 i =1
10 Dec 2012
© Joseph J. Nahas 2012
14
ANOVA Table
Source
Sum of Squares
(SS)
Degrees of Freedom
(df)
Mean Square
Variation
(MS)
Factor A
SS(A)
(a‐1)
SS(A)/(a‐1)
Factor B
SS(B)
(b‐1)
SS(B)/(b‐1)
Error
SSE
(N‐a‐b‐1)
SSE/(N‐a‐b‐1)
Total
SS(Total)
(N‐1)
Relative contribution
to the total variation
10 Dec 2012
© Joseph J. Nahas 2012
15
Outline
•
•
•
•
Analysis of Variance (ANOVA) Basics
2 Way ANOVA
2 Way ANOVA Example
Significance Testing using the F distribution
10 Dec 2012
© Joseph J. Nahas 2012
16
An Example
NIST ESM 7.4.3.8
10 Dec 2012
© Joseph J. Nahas 2012
17
Summations
Summations in each box on previous slide
Summation
over
rows
Summation
over
Columns
NIST ESH 7.4.3.8
Total Sum
10 Dec 2012
© Joseph J. Nahas 2012
18
Summations
Calculations
based on
Slide 14
ANOVA Table
NIST ESH 7.4.3.8
10 Dec 2012
© Joseph J. Nahas 2012
19
Outline
•
•
•
•
Analysis of Variance (ANOVA) Basics
2 Way ANOVA
2 Way ANOVA Example
Significance Testing using the F distribution
10 Dec 2012
© Joseph J. Nahas 2012
20
Significance of the Relative Contribution • Are the various factor contributions significant relative to the
noise in the measurement.
• Look at variance of factor relative to variance of the noise.
10 Dec 2012
© Joseph J. Nahas 2012
21
ANOVA Table
Source
Sum of Degrees Mean Square
Squares
of Variation
(SS)
Freedom
(MS)
(df)
F
Statistic
F Reference
Factor A
SS(A)
(a‐1)
SS(A)/(a‐1)
MS(A)/MSE
F0.05, a‐1, N‐a‐b‐1
Factor B
SS(B)
(b‐1)
SS(B)/(b‐1)
MS(B)/MSE
F0.05, b‐1, N‐a‐b‐1
Error
SSE
Total
SS(Total
)
(N‐a‐b‐1) SSE/(N‐a‐b‐1)
(N‐1)
Variance of Factor
relative to
Variance of Error
F Distribution
Comparing
Factor Variance
to
Error Variance
10 Dec 2012
© Joseph J. Nahas 2012
22
Expanded ANOVA Table
>
>
In Excel: Fref
= FINV(Alpha, a– 1, N – a – b – 1)
= FINV(0.05, 2 – 1, 18 – 2 – 3 – 1)
= FINV(0.05, 1, 12)
= 4.75
10 Dec 2012
© Joseph J. Nahas 2012
23