VARIANCES OF REGRESSION COEFFICIENTS FOR SPLIT
PLOI' MULTI-FACTOR EXPERDiJENTS
Prepared Under Contract No. DA-)6.034-0RD-1517 (RD)
(Experimental Designs for Industrial Research)
by
R. J. Hader
Institute of Statistics
Mimeo Series No. 11)
July, 1954
•
1.
INTRODUCTION
Previous technical reports in this series (Numbers 1, 3,
4, , and 6) have
considered the design and analysis of multi-factor experiments.
ments it has been postulated that a response variable
independent continuous variables
relationship ~
m
~(xl,x2,
~,
••• ,xk)'
l
For such experi-
depends on several
x ' • u, x according to an unknown functional
2
k
It is convenient to regard this relationship
as the equation of a hyper-surface over the k dimensional space whose coordinates
are xl' x2 ' ... , x k • In order to investigate the nature of this surface a number,
N, of experiments are performed in which the response variable is measured at
selected points in the k space, the over-all configuration of such points being
called the experimental designo
e.e (xl'
x ' ... , ~) is unknown, in practice it
2
may often be replaced by a Taylor Series to terms of some order. For example, in
Though the functional form,
three variables Xl' x and x ' and to terms of second order only, such a series is
2
3
written
(1)
1
m
~o + ~lXl + ~2.X2 + ~3~ + ~J2XlX2 + ~l3XlX3 + ~23~x3
2
2
2.
+ ~llxl + ~22X2 + ~J3x3
The problem then reduces to determining the unknown coefficients in this series.
In the simplest case it is next assumed that each measured response y.,.
where
&.,.
variance
represents a random experimental error having expected value zero and
i.
Each such error is assumed completely independent of all other errors.
The least Squares cr!terion is then used to obtain estimates of the
let
."'l. -< + s.,.
!
~. s.
be the N x 1 matrix of observed values of y-< and let! be an N x L
matrix of the values taken in the N experiments by the independent "variables ll in
the series.
The tem "variables" is here used for the basic factors, xl'
~ ..
x
J
-2and also for products and powers thereof,
variable, x : 1, l'lith the constant term
O
~ IS are given by the L x 1 matrix
It is convenient to include a dUlll1llJ
~O'
The Least Squares estimates of the
It is further easy to show that the variance-covariance matrix for the bls"
that is" the Lx L matrix whose elements are E(bi - ~i)(bj - ~j)' is given by
Equation (J) has been derived on the specific assumption that the random
experimental errors, 8-<" associated with the measured values of the response
variable have average value zero" common variance, (12" and are independent of each
One common type or multi-factor experimentation for which these assumptions
other.
must be modified is the so-called split plot design.
This report will be concerned
with the variance-covariance pattern of the regression coefficients (i,e.
E(!! -
It) (!! -
I!)') when the data have been obtained by a split plot type experiment,
It is felt that the matrix techniques by which the appropriate formulae are
developed can readily be applied to a considerable number of designs differing
from those specifically considered but having other types of split plot aspects.
2•
SPLIT PLCYl EXPERIMBlILTS
For the present discussion the essential characteristic of the split plot
type experiment is its error pattern.
of Xl
CI
soaking temperature, x
2
cutting veneer on a rotary lathe.
surface smoothness.
lD
Consider an experiment to study the effects
knU'e angle and x III pressurebar·setting in
J
The response variable might be some measure of
A number of veneer "bolts" are soaked at a series of diffezent
-3A bolt is then put on the lathe and veneer is cut under as many
temperatures.
of the knite angle-pressure bar settings as possible before changing to a new
bolt.
The error pattern tor such an experimental procedure is that shown in
Figure 1.
5oaki np; Temnera t ure
Knite Angle
Pressure Bar
X2 • X2l
. - --
b2 + "'21
--. -
b
+
b .... w
1
12
b +w
22
2
----
b
+ W2
b:1. + wl )
b 2 + w2)
.. - - -
b
X3 • X31
X2 .. ~2
X) .. X)2
~. ~)
X) •
Xl • X12
(Bolt 2)
Xl • Xu
(Bolt 1)
1.;)
l
•
•
•
•
Xz .. ~q
X) • X3q
Figure 1:
+ wll
•
!
,
P
P
•
•
•
•
I
1
P
bp + litp)
•
•
lit
P
•
•
•
Xl • Xl
(Bolt p)
•
•
,
'1.
i
oft
I
wlq
~ + ~q
i
I
II
bp +
lIt
pq
I
Error Pattern for Simple Split Plot Experiment
Each cell of the table contains the random error for the observation made
at the factor levels indicated.
The important point to note is that the errors
in any column have a common component, b , associated with bolt to bolt variation.
i
In addition they have a second canponent, wij ' which differs from one observation
-4to the next.
Following terminology carried over from field experimentation the
bolts would often be referred to as "whole plots ll and the seotions of veneer on
which the individual measurements were taken would be called "sub-plots"'.
The above example has been chosen for simplicity.
In general the dis-
tinguishing characteristics of these experiments are
(1)
The error affecting each observation is regarded as the sum ot two or
more random oomponents.
(2)
One component varies trom one observation to the next.
(3)
The other components vary only from one group of observations to another
group, the groups being determined by the experimental procedure followed •
.An actual example from the field of ceramic engineering involving three random
components in a rather unusual pattern will be included in this report.
3. SINGIE RANDCM CCMPONENT
For completeness 't'1e begin by considering the simple experiment in which there
is only a single random component.
The N x 1 matrix of observations for the
response vari able J YJ is po stuJa. ted to be of the torm
(4)
Where
!
!
II
! l!.
+!
is the N x L matrix made up of the values taken by the "independent
variables" in the Taylor Series equation and @. the Lx 1 matrix of unknown coeffioients in this equation (e.g. equation (1) ).
random errors.
The N x 1 matrix! consists of the
Its elements are assumed to have expected value zero, variance a2
and to be independent ot each other.
where I is the N x N identity matrix.
In matrix notation
-,As indicated in (2)
-
whence on substitution for Y we get
(6)
~. (!,!)-l !I(!! + !)
. l
+
(~I!)-l !'!
The variance-covariance matrix of the bls (elements of
(1)
E(~ - ~)(! -
!!)
is therefore given by
l)' • (!I!)-l !'E ! !,!(!,!)-l
• 02 (!I!)·l
4.
SIMPIE SPLIT PLOI' (TWO RANDOM COMPONENt'S)
Consider next. an experiment of the type whose error pattern is illustrated
in Figure 1.
The k variables may be divided into tl'lO groups, those "'hich require
"whole plots" and those which require "'sub-plots". We shall assume a second order
series is sufficient and for convenience will write this series with sub-plot
variable terms J the whole plot variable tems and the mixed terms each grouped
together,
It will further be convenient to write the pure quadratic terms in
2
N
2
their orthogonal form, i.e. Xi - Ci , where Ci is chosen so that j~l XO(JC..Lj • Ci )
Also each of the x variables will be measured from the centroid of the design so
N
that j~l ~j • O.
In partitioned form then the
!
matrix is (see next page)
I:
O.
-6Sub-Plot
Variables
Whole Plot
Variables
Mixed
Variables
Xo
xl,x 2"etc
x5"x6,etc
x 5" etc
1
!s
.~
!sWl
1
~
~
~wa
-1
~
~
~.3
••
•
••
••
•
••
••
•
•
•••
•
•
•••
•
•••
•
•••
Hhole
Plot
-1
!s
~p
Xswp
lrJhole
Plot
Whole
Plot
lrJhole
Plot
x- •
••
••
lJ
where! is a q x 1 column vector of liS;
2!S
r
is the matrix of values taken on by
the sub-plot variables wi thin each whole plot; ~i is the matrix of values taken
th
on by the whole plot variables in the i
whole plot and !sWi the matrix of
interaction variables between the whole plot and sub-plot variables.
has been assumed to be the same in each whole plot.
though it will not always be true.
Note that
!s
This is ordinarily the case"
It will also be assumed that the sub-plot
design and the whole plot design are such that the interaction variables are
N
orthogonal to the mean" i.e. j~l x ij X1tj • 0 (1 r} it). The conventional factorials"
the central composite designs and the rotatable designs all have this property.
With these assumptions and conventions the following matrix relations hold.
(8) ! ' ! s . 0
(9)
(10)
!wJ.
+ ~ + ... +
l' X~T.f
• 0
-ovvi
-
!wp
III
0
-7(11)
~Wl ...
(12 )
~ ~1· 0
(1.3) ~Wl +
(14 )
!am
+ ••• +~WpaO
!sw2
+'''+~p.O
f
~11 !swi
0
a
The four columns of the partitioned matrix
!
are therefore orthogonal to each other
and we have
--
P 1'1
I
0
(15)
where
~
and
columns of
!
!otv'
--
XIX ...
P
0
0
0
~!s
0
0
0
0
~~
0
0
0
,
0
~W
!svl
'Without subscripts, represent the Ilwho1e plot lf• and "mixed"
respectively.
The inverse matrix is therefore
!(lll)-l
p-0
(16) (!I!)·l.
0
0
0
0
0
!(XI X ,-1
0
0
p -S :.:s
0
0
(~ !w)-l
0
0
(!ow !sw)-l
Now the error pattern associated with the experiment under consideration is
that indicated earlier in Figure 1, where the error for the j th sub-plot in the
b + wij • We assume b has expectation zero" variance a~
i
i
and that the bi are independent of each other. Similarly we assume the w have
ij
i th whole plot is &ij
CI
expectation zero" variance
CJ~ and. are indep~dent of each other and further the wij
-8are independent of the b • If then we let!. be the matrix of variances and
i
covariances of all the observations, i.e, the matrix corresponding to E ! 1', we
have
(17)
V•
A
0
• • • •
0
0
A
•
• • •
0
0
0
4-
• • • •
0
••
•
••
•••
•
•••
••
•
•••
•
••
••
•
0
0
~+
Jw
0
A
• • • •
where
O'b
2
O'b
2 2
O'b + a w
2
O'b
•
•
•
2
(18)
~
II
~
2
O'b
,
,
•
•
• • • •
2. 2
O'b + a w
• • • •
~
,
•
2
O'b
2
2
O'b
2
O'b
•
•
~
2
c{
,•
•
O'b
• • • •
• • • •
•
•
2.
O'b +
~
2
that is, each observation has variance O'b + aw' any two observations in the same
whole plot have covariance CT~1 and any tl-l0 observations in different whole plots
have covariance zero,
(19)
!
=
It will be convenient to write
CT~ ! ! 1
of,
0:; !
Now analogous to equation (7) we ha va the variance-covariance rnatrix of the
coefficients b given by
i
-9Performing the matrix multiplication
V X •
-X' --
we get
!'!!o
!'A.~:!wi
!rA~~Wi
p ~!!
p~!~
X~ !~~fi
~ !~!sV1i
(~~1i! !
(~~~1i)~
Z~i! ~li
]~i! !sWi
(~!6Wi)g
(~!oWi)!!a Z~W1~i
- --
p l'A
(21)
! I Y. !
1
P
~~Wi! !swi
- --
By virtue of equations (8) to (14) all the off-diagonal tams in X' V X are zero.
Substituting for A we get
•
(22)
- --
X' V X
•
•
•
II
•
In getting the third element advantage is taken of the fact that within any given
whole plot the q elements of any column of
X1ra
are constant" therefore
(2.3)
2 (XI
<Tw
:;SW!oW
)-1
-10-
>'
The elements are respectively the varieJlce of the mean (i.e. bO the variancecovariance matrices of the coefficients of the sub-plot variables, the whole plot
variables and the mixed variables
5. AN EXAHPIE
('l'\'10 RANDOM OQJ:·1PONllJTS)
Suppose the veneer cutting experiment described earlier is BElt up as a
3 x 3 x .3 factorial and that for each ot the te:nperature levels H'e use two bolts
maldng a total of
,4
observations.
let the three levels of each variable be coded
to -1, 0, +1 and l'l1"ite the second order series to be fitted in the form
y
:II
bO+ b1x l + b2xl + b1l (xi - 01) •
b22(~
.. 02) + b12x1"2 + b3x3
(24)
where
~
• knife angle,
X
2
orthogonality 01 ,. 02 = 03
=:
pressure bar setting, x
3
=:
sx~
N
For this experiment p = 6, q
(9C1~ + '{; )/54.
rl...·1
-6
temperature and for
=2/3.
= 9 hence
the variance of b is given by
O
The sub-plot design is a 3 x 3 factorial hence the variance of the
coefficients of t:.le sub-plot variables is given by
(25)
lIlI
'b1
b2
bll
b22
6
0
0
0
0
0
6
0
0
0
0
0
2-
0
0
0
0
0
2
0
0
0
0
0
4
b
12
-1
-11-
The variances of the whole plot coefficients are
The variances of the mixed coefficients are
(21)
Estimates ot a; and of
9c{" + ce
can be found from the following analysis
of variance:
Source
£!
Variation
Mean
1
Whole Plots
5
Temp, Coefficients
2
Bolts within Temps
3
48
Sub-Plots
Pre ssure and Knife }
Angle Coefficients
Mixed Coefficients
Higher Order Coefficients
17
Sub-Plot Error*
24
Total
*
2
54
The sub-plot error is made up ot Bolts x p(6 d.f')1 Bolts x K(6 d.f.) and
Bolts x K x p(12 d,f.).
..12In this analysis of variance the mean square for II·Bolts within Temps" is a direct
estimate of
9(1~ + (1; and the mean square for "Sub-Plot Error" is an estimate of
2
0:\1'
6.
A FURTHER EXAMPIE (THREE RANDOM COMPONENTS)
As a furtl'.er and somewhat more complex example we consider an experiment in
the field of ceramic engineerinB*'
The experimenter ldshed to study the etfect
on y .. absorption" ot the following variables:
Xl .. r a ti 0 of clay to nepheline syenite plus feldspar
"2 ..
x
3
ratio of clay to flint
.. ratio of nepheline syenite to feldspar
x4 = firing
temperature
It was decided to set the three composition variables up in a central composite
design (see references 1" 3" 6) and to repeat this design at three firing tern"
peratures (equally spaced).
tures.
Two firings were made at each of the three tempera-
For each firing the kiln contained duplicate test specimens for all of the
fifteen different compositions required by the three variable central composite
design.
Two batches of the mixed ral"1 materials l-lere prepared and the
specimens actually consisted of a representative from each batch.
dup1ic~te
It was te1t
that three distinct components of randan error would be involved" namely
W
ijkt
the specimen to specimen variation" bkt the batch to batch variation (for fixed
composition) and f
ture).
the firing to firing variation (foX' nominally fixed tempera..
ij
These are postulated to have zero expectation, variance
(1~ and (1~
respectively and to be independent of each other.
0:;,
The experimental arrangement
is shown in Figure 2 l'
*
W. C, Hackler If'The Effect of Raw Nateria1 Ratios on the Absorption of 'Whiteware
Compositions", Ph.D. Thesis, N. C. State College (1954).
...1)-
Compositions
ITemperature
Firing
.
1S
1
2
18 1b 28 2b
ISa .l$b
I
-
1
II
I
2
II
I
3
II
Figure
2:
Arrangement of Ceramic Experiment
In Figure 2 the firing component~ f
component~
, is constant in a given row; the batch
ij
bk.e' is constant in a given co1wnn; the specimen component Wijk.e varies
from one observation to the next.
The variance-covariance matrix of the observations has the form
<!+12)
(28)
V •
- - D
D
-- -
D
12
(A+D)
D
Q
-D
D
(A+D)
-D
D
(!+Q)
D
D
-
-
-D
-D
-D
--
-
D
D
D
D
D
-
-D
D
-
D
D
D
D
-D
(~+~)
-
-D
-D
<!+£)
D
where
c4 + O'2.w
(29)
!-
a
2
O'f
(1£ +
<{
2
C\r
• • • •
2
•
0'£
•
•
•
•
2
(1£
2
f
Cl
2 ... 0'.2
W
f
Cl
-14and
0
• • • • •
o
0
2
O'b
• • •• •
o
•
•
(1.2.
b
(30)
-
D•
•
•
•
•
•
•
0
0
•
•
•
2
O'b
• • • ••
222
The variance of each observation is O'f + O'b + O'w'o
c{;
firing have covariance
Two observations in the same
two observations in the same batch have covariance ~;
observations are otherwise independent.
We note that
(31)
y may
V + -=-b
V
-a
s
be written as
!
0
0
0
0
0
0
-A
0
0
0
0
0
0
-A
0
0
0
0
0
0
A
0
0
0
0
0
0
~
0
0
0
0
0
0
!
-- - -- --- -- -----D D D D D D
D D D D D D
+
D
D D D D D
- D- D- -D -D D- - -D -D -D D
D
D
D
-D -D
D D D D
- - -
The matrix (xtX)~lx.v X'(x'X)-1 18 that already evaluated in the two random
-- --a---
component problem (though with
0'; substituted for O'~).
We therefore merely
evaluate (!'!)-~IYb !(!,!).l to be added to the earlier results.
36 !'Q!
(32.)
!'!t,!.
36! !2 ~
6!
36 ~!2!s
6
symmetrio
Q~~i
~ g ~!wi
We find first
6
!'Q ~!swi
6
.!t!2
~~Wi
~ ~~li~)e\'l1 ~~~iP.)~Wi
~ ] ~WiQ)!sva
'-
-1;Again by virtue of equations (8) to (14) only the first two elements of the
diagonal are non-zero.
and
Hence
Upon substitution for
36 !'(c~
!)! • (36)(30)C~
36:%(c{
!)~ • 36c~ ~ ~
~
these become
(!'!)-~I !b !(!,!)-1 is given by
0
c{/30
0
(33)
2 (t
-1
b Xs XS)
C7
0
o
0
o
o
o
0
0
0
0
0
0
Finally adding this to the previously obtained (XIX)-!x' V X(XIX)-l we get
-- - -a----
•
The elements are respectively the varianoe of the mean, bO' the variance and
covariance of the coeffioients of the composition variables, the variance of
temperature coefficients and the variance of the mixed coefficients.
To estimate
the variance components involved we use the following analysis of variance,
-16-
-d.f.
Source
E(m.s.l
~
~ 2
2
2
aw+30O'f+30 2 t:i
Temperatures
2
1179.9889
Firings wi thin Temps
3
.1521
Compositions
14
lO~3360
Batches within Compositions
15
.7405
Temps x Comps.
28
1.1130
(Comps) x (Firings wi thin Temps)
42
.0818
~
(Temps) x (Batches within Comps)
30
.0857
2
0',.,
(Batches win COMps)(Firings win Temps)
45
.0631
Gt
Total
O'~
+ 30t{
~+60'~+12 ~ C~/14
2
2
O'w ... 60'b
ce,+4 ~
~tC)ij28
2
w
179
The expected values o:t the mean squares follow :trom the model
where t , C and (tC)ik are parameters for temperature, composition and temperaturei
k
composition interaction respectively. This model dif:ters somewhat :trom the second
degree Taylor Series model but the difference is confined to the non-random portion
of the model.
The analysis of variance based on it furnishes the necessary
22222
estimates of Gtw' O'w + 60'b and O'~l + 300f as indicated. The last three lines may
be pooled to estimate
0';..
7.
SUMI'1ARY
In this report we have presented a matrix development of formulae for
variances and covariances of regression coefficients in split plot multi-factor
experiments. We have considered the simplest type of split plot experiment plus
one more complex example which arose in the field of ceramic engineering.
It is
felt, however, that the techniques by which these formulae are derived can readily
be applied to many other designs having the characteristics outlined on page four
of this report.
REFERENCES
OOR Technical Reports prepared under Contract Nos. DA-36-034-0RD-1177 and 1517.
(1)
Tech. Report No.1: "Experimental Designs for Multi-Factor Experiments:
Preliminary Report", G. E. p. Box, R. J. Hader, J. S. Hunter.
(2)
Tech. Report No.3: "A Confidence Region for the Solution of a Set of
Simultaneous Equations with an Application to Experimental Designl~,,,
G. Eo P. Box, J. S. Hunter.
(3)
Tech. Report No.4: II'The Exploration and Exploitation of Response Surfaces",
G. E. P. Box. Also published in Biometrics, 10, 1 (1954).
(4)
Tech. Report No.5: II'The Effect of Inadequate Models in Surface Fittingll "
G. E. P. Box, J. S. Hunter, R. J. Hader.
(5)
Tech. Report No.6:
J. So Hunterc
"Ml1lti.-Factor Experimental Designs", G. E. P. Box,
other References:
(6)
Box, G. Eo P. and Hilson, K. Bo (1952) "On the Experimental Attainment of
Optimum Conditions". ll,o~. Statist. 22£. B, 13, 1.
(7)
Box, G. Eo P.
(8)
Hackler, U. C. "The Effect of Raw Material Ratios on the Absorption of
'IrJhiteware Compositions". Ph.D. Thesis, N. C. State College, 1954
(unpublished) •
•
•
"IvIu1ti-Factor Designs of F.irs.t Order".
Biomet.rika 39:49, 1952.