•
MULTI-FACTOR EXPERIMENTAL -DESIGNS
Prepared Under Office of Ordnance Research
Oontract No. DA-36-o34-0RD-1177 (RD)
by
Os E. P.Box
J. S.Hunter
•
Institute of Statistics
Mimeo Series No. 92
J~nuary, 1954
TECHNICAL REPORT NO.6
MULTI-FACTOR EXPERIMENTAL DESIGNS
Prepared Under Contraet No. DA-36-034-0RD-1177 (RD)
(Experimental Designs for Industrial Research)
Ordnance Projeot No, TBa·DOOl (632)
Dept. of Army Project No, 599-01-004
Philadelphia Ordnanoe District
Department of the Army, Department of Defense
with
Institute of Statistics
North Carolina State College of
The University of North CarolL~a
Raleigh l North Carolina
•
Technioal Supervisor
Ballistics Research Laboratories
Aberdeen Proving Ground
Aberdeen, Maryland
G. E. P, Box
J. S. Hunter
Authors of Report
. •
'...
.~
•
1. INTRODUCTION
&uppose we have k quantitative factors whose levels are denoted by
Xl" X2 " ... " Xk on which depend the level of' some response in accordance with an
unknown relationship
(1)
Suppose that in order to explore this relationship" N experiments are performed.
th
The u of these experiments consists in adjusting the factor levels to a certain
set of k pre-decided values.. Xlu " X2u ' ••• " \.u and of observing the response Yu'
The problem of experimental design is that of deciding for given assumptions
concerning the function t{ what is the best arrangement of N sets of levels to use.
Following the convention adopted in previous papers we shall define a set of
standardized factor levels
x
iu
..
(Xi
-
U
.. X.)
J.
where 51
51
&I
f~
Lu-l
(2)
For these standardized levels therefore
•
0
and
N 2
~ x
.. N
@ iu
u=1
-2-
'4t
various types of assumptions concerning
the function f
II'
In given circumstances",
the experimenter could select the appropriate design matrix and (by deciding on
~2'
... " Xk and units 5 p 52" ., ." Sk whioh would
cause the design to cover that region of the factor space in which he was interested)
suitable averages values ill
covert the standardized variables xl'
x2~ e u" ~
of the design to the real levels
Xl' X2' "" ~ of the variables with lhich he is experimenting. The level of the
1th factor to be used in the uth trial would then be X • Xi + Sixiu' We shall
iu
assume 1n what follows that we can approximate the function 't by means of its
Taylor's Series in which tarms up to degree dare inoluded.
represent
r
That is to say, we will
by a polynomial of degree d so that the response at the uth point will
be assumed to be given by
with suitable choioe of the coefficients
~O'" ~l.t
etc.
We shall obtain estimates
b bl " etc. of these coefi'icients by fitting the equation to the N observed values
O'
of y at the N experimental points by the method of least squares.
th
We call ~ i the i
linear effect, and Xi the i th linear independent variable",
~ii and xi the i th quadratic effect and the i th quadratic variable respectively,
and ~ ij" Xixj the linear x linear interaction effect and variable for the 1th and
j th factors respectively" and so on.
as
~OXOu
rather than as
It is conveniQt to write the constant tel"11l
~O
defining x as equal to unity for all values of u.
Ou
A design which includes k factors and allows all constants up to order d to be
determined will be called
k dimensional design of order d. In a polynomial
k +d
. equation of degree d there are ( k ) terms so that a k-dimensional design of
1:\
order d must contain at least this number of points" ie" N2(k : d),
One arrangement of experiments that might be employed uses the points of inter-
o
-3-
'~
section of a cubio lattioe as the experimental points. Such arrangements are called
factorial designse
A faotorial design in which were determined all the effects of
order d or less would require the performance of all combinations of d + 1 levels
of the factorsl thus (d + l)k experiments would be required. The number of
The number of observations needed by the factorial design may sometimes be considerably reduced by fraotional replication (particularly for first order designs when
two-level fractional factorials may be employed). However this device is less
effective for designs of higher order.
There is also some doubt as to whether the
relative emphasis placed on different terms in the series, as measured by the
variances of the effects, is an ideal one with such designs. Since for quantitative
factors there seems no prior reason for basing experimental arrangements on the
factorial pattern some more fundamental approach may be attempted.
·4·
~0
ORTHOGONAL DESIGNS
The problem of determining most efficient designs of order one has been
discussed elsewhere (Box, 1952.). The most important practical problem outstanding
is that of investigating designs of order two which are of great importance in the
study of near-stationary regions of the
f.a~tor
space, that is, regions in which the
first order effects are small o We shall not, however, for the moment, limit the
discussion to this special case.
Suppose the observed values found at N experimental points are represented
by a vector
!
and
then on the supposition that the mathematioal model (4) exactly represents the
true situation the estimates
(i.e.,
t: (!) = ~)
~
of
~
linear in the observations which are unbiased
and have jointly the smallest possible variances, are those which
-
(1.. - -Y)I(1. ...y)
reduce to a mimimum the sums of squares of descrepancies
the observed values
!
and the values
! ,. ~ "predicted"
between
by the fitted equation.
These are the 1I1east squares" estimates and are given by
(6)
The variances and co-variance of these effects are
,.
-1
'Where.9.
0-1 (j2
-
may be called the "precision" matrix•
.An unbiased estimate of
i
is provided by the quantity
(7)
•
-5The expressions in (6), (7), and (8) contain the matrix
£ .. l.-x'x_7
of sums of
squares and products of the independent variables. We also notice that N- l I.X'!7
is a matrix of moments of the design,
For example" if there were k .. 2 variables,
and we were considering a design of order t1iO so t hat the equition to be fitted
was
then
1
1.-1.7
1.·2..7
1.-11.7
[22-.7
1..12..7
1
{t.7
,-ll?:?
L·J2.~7
/.·11~7
[2_7
1.-2'2)
['1127
1.- 12'iJ
--
l.i1g7
-(in]]-
(22.gJ
N-ll.! 1_ff=11
--L'1117
.-
,-lg7
2
L-117
/-127
Li1227
--
Li1127
0
/i17
--
- .
22.
L"}.~7
L'1.22.7
L'J2.~7
l.il2~7
1.2.2'l'{J
1.i'l2'{J
12
Li2.7
Lilg7
Li2 'f.7
{il1g7
Li22g7
Lil2~7
(10)
·where the quantities in square brackets denote the moments of the design. For
N
N
l
l
example, N- ~ xl .. 1.-1.7, N- ~ xiux2u= 1:112,.7 and so on. It will be noted
u-l
u=l u
that in a fitted expression of the form of (4).lin which terms of order greater than
the first are included,many of the independent variables are related to each other,
thus we have not only xl' but
xi,
and xl x2 ocouring in equation (9). Consider for
a moment an expression of the form
•
Yl ..
~OzO + ~lzl + ~2z2 + H' + ~LzL
N
(ll)
and suppose the sum of squaIee for the pth variable ~ '1. 2 is denoted by S • Then
u=l pu
P
it is readily shown, (see for example Box 1952) that if the Sp are regarded as
•
-6fixed the smallest possible varianoe for everyone of the elements b " b , etc.
O l
is obtained if the moment matrix and hence of oourse the precision matrix are
diagonal" assuming that such a diago~al form is possible. In suoh a case the
th
-1 2
varianoe ot the p
effect is given by Sp a. In the oase of first order designs
in which the terms in (11) are unrelated this fact supplies the neoessary oonditions
for an optimal design.
To satisfy the oondition we ohose the levels 1 0 " Zl' .,." I L
so that the L veotors formed by the oolumns of the matrix of independent variables
have zero inner produots one with another.
-
When fitting an equation of degree higher than the first" provided X is of full
rank L" we may still, of oourse, use the procedure of least squares to estimate the
effects even though the independent variables are reJa ted to one another.
However,
when an equation of degree greater than the first is fitted" these relationships
make impossible the attainment of a diagonal matrix for
!'!.o
For example" both
,-11_7, ,-22.7 whioh
appear in non-diagonal positions in (lO~ must ot necessity
q
be positive unless all the x.; are to be zero. In general ~
x will produoe
.u
u~l u i u
essentially non-zero element,s where p plus q is an even number" and these terms oan
xi
ocour in non-diagonal positions when prq? We oan however rewrite equation (4) in
terms of new variables x2,,· x3j x4, whioh are (odd (polynOmials;.
1 even)
4
.. -<_xPetc. orthogonal to all ot.'ler (odd
1 powers of lower degree
~
( even)
x? •
-<lxP- 2
and
(12)
where r is an integer.
For example we require
= 0
(13)
•
-7-
-
(14)
whence usin! (3) ..< .. 1 and xi .. xiu - 1
Similarly
xi . xi - L-iiii.7 Xi
(1,)
The equations written in terms of these new variables would be
(16)
where T is a matrix transforming the old independent variables to the new, thus in
-
tems of the new variables (9) could be written in the form
'L • (~O
-
and T ..
'eli
),),c,"
-.( Q,"\ ~
~
boo
2
2
+ Pu + ~22)xO + ~lxl + ~2x~ + Pll (xl - 1) + ~22 (~ - 1) + ~12xlx2
• •
• 1 •
-1
-1
•
•
•
•
• 1
@ • •
•
•
•
1
•
•
•
1
•
1
•
• •
• •
•
T- l ..
1
1
.-
t
•
•
(18)
•
•
• 1
• • 1 • • •
\ci @ • • 1 • •
':;..f'"
{e-~ ~f)t (i)
• 1 •
•
•
II
~
1
II
1
(17)
•
II
cD
1
For the new independent variables it is now possible to attain an orthogonal matrix
for any permissable choice of diagonal elerrents.
The Gauss-Markoff theorem,
(Guass, 1831, Markoff, 1912,) ensures that the least squares estimates obtained for
these new variables will also be least squares estimates for the old variables.
We have agreed to define the elements in the design matrix so that
N
~ x~
u-l
N
u
.. N
, and
~
u=l
xiu .. N (i - 1"2, ... ,k),, therefore, the
first k + 1 diagonal elements of the matrix
remaining diagonal elements of
!'! will
!'! will
not be fixed
example, it is easy to show that the sum of squares
be fixed and equal to N.
b~
our definition,
The
For
~ (x~ - 1)2 corresponding
u-l
J.U
.8...
to the i th quadratic variable can take any value between zero
and H(N • 2)2/(N ... 1),
,
This is what is meant by a "permissable" choioe of diagonal elements.
SUIMI
The remaining
of squa.res would likewise be at our choice wi thin certain wide ranges.
Consider the Cas8 ot a design of order two.
The choice of the quantity
N
2
N- 1 .~ (x ... 1)2 • q • i-iiii 7 - 1 corresponds to the choice of the fourth
iu
u·1
moment tor the i th variable in the design, tha.t is, at a fourth marginal moment
at the distribution of design points. Since
'Yo')'
.
of kurtosis
q •
t 2 + 2..
L-11_7 • 1,
'-iiii.7 - .3 i. the measure
the standardized fourth cU1llulan t ot the marginal distribution and
Suppose
tix q at some particular value. then tor example wi th k • 2
W8
factors, the design will be such that the moment matrix tor the orthogonal variables
will be
1
.'
N- 1,-!ol!;;}
•
•
•
•
•
1
,
•
1
1
1
•
•
•
1
•
•
• •
•
• •
• •
•
•
1
•
•
q
•
•
1
Q+l
1
q
•
1
•
•
•
1
•
•
•
1
a
The diagonal element in N·
l
••
L-! IL7
H-1(X 1X]a
•
I
1 q+1
•
(19)
•
,
1
correspondinc to the interaction term
'-l12.2....i is tixed automatically, since to ensure orthogonalit;r~ 0'1 the quadratic
effects
N
N.. ~ (X~u
l
e
-1) (x~u ... 1) • (1122:7 - ,-11) •
Henoe [-U22.J • 1,
(12;.7
+1
•
/:1l22.7 . .
1 • 0
The corresponding mOlll8nt matrix N- l (!,!) for d • 2 and any
'Y&1u.& of k will be exactly similar in pattern, that is to say" the element ,-ii_7
-9-
III
corresponding to the sum of squares of the i th linear variable and the sum of pro·
ducts between
X
o and the i th quadratic variable will always be equal to one. The
element L-iiii.7 corresponding to the i th quadratio effect will always be equal to
q + 1 and the element L-iijj.7 corresponding to the sum of produots between.the
i th and j th quadratic variable, and to the sum of squares of the ij th interaotion
variable will be unity.
All other elements are zeroo
Similarly, the variance·
covariance matrix written on a "per·observationll basis would be
l+kq-1 •
and for the
1
original non-
1
orthogonal
1
q
x'!7-l •
Nt
-1
variables
q·1
'-1
1
•
•
•
1
•
o
-1
•
-1
-q
-q
1
• -q-1-1
-q
•
q
•
• •
•
q
•
•
•
•
-1
•
•
•
•
1
For the 32 faotorial (and in general for the .3 P factorial design) q .. 1/2 that is
r 2 • • ~.
Thus" using these designs" the varianoe of the quadratic effects is
twice that for the interaction effect{s)~ As was pointed out in an earlier paper,
(Box and Wilson, 1951, pg. 18) if we regard (9) as representing a Taylor's Series
of the function
example
if
1\ ,~
12 ...
Y<xl x2 ) then
O'\'l /'J Xl dX 2 '
~ll .. ~ If 11: ~22
..
f te 22; ~
a
l( 12
where for
Thus the variance of the quadratic derivatives
is eight times that of the interaotion derivatives.
.
This relative weighting for
quadratic and interaotion terms found for the faotorial design seems intuitively to
be unsatisfaotory from the point of view of acourate estimation of the fitted
surface, 1'0 r it appears that too little weight is placed on the quadratic terms as
oompared to the interaction terms • This fact may be the reason for a mis-conception
concerning the relative importance of quadratic and interaction effects.
It is
",
-10commonly found in the analysis of variance of three level factorial designs that
two-factor interactions are significant whereas quadratic effects are not.
This has
led to a supposition that conditions frequently occur in which two-factor interactions are important but quadratic effects are unimportant, which apparently
conflicts with the common sense view that for a smooth surface effects of the same
order ought to be of equal importance.
That this contradiction is apparent rather
than real can be seen if we remember that these expected values of the mean squares
in the analysis of variance are of the form
(21)
It will be noted that the second term in (21), whioh will cause the mean square to
be inflated when real effects occur, is a function not only of the size of the effeot
~
but also of the variance of the estimate of this quantityQ
Thus if real quadratic
and interaction derivatives of equal magnitude ocoured the inflation of the mean
squares for the interaotion would be eight times as large on the average as the
inflation of the mean squares for the quadratic effects.
When d
estimated.
=3
we have tha cubic design in Which all effeots up to order three are
When k
=2
the equation
'tt,. ~ to
be fitted is
y)=A
t:l
2
2
3
1
2.
2
l t'OXO + t'lx1 + ~2x2 + ~llxl '" ~22X2 + P12x l x2 + ~lllxl + ~222X2i{3l12xlx2~l22xlx2
(22)
which, proceeding as before, may be written in the
alt~rnative
form
~. (! '1..7/'-T- l ~_7
f
~. (~o 1f3 U -tf3 22 )xO+ {~l+(q+l)~lll '*i3 l22 } Xl+ ~2 +(q+l)~222 +f3 112Jx2 ~ll (xi-l)~12 (X~-l)
of{)12X:i.~ "'i3 111{xf-(q+l)Xl ) ""13 222 {x~- (q+l)X2]
+{3ll2 (xi-l)~ '*f3 122 x l
(X~-l)
(2,3)
where the diagonal terms corresponding to the quadratic x linear interactions are
necessarily fixed by the orthogonality conditibns. For we require that
N
N-l~
fx?i -(Q+l)x. II x. x~ ..xi .~
J.u!
J.U JU
u~
u=l r U
But
IS
L-iiiijj 7·(q+l) • 0" i.e." '-iiiijj.7 .. q + 1
-
~-
N-
l
~ 1(X~u~)x.1 2 • ( i i U j j ) - 1 • q,
u=l
JU!
..12-
22.2
111
112 122.
q+l
I
q+l
1
1
q+l
1
1
q+1
I
1
?'
. J'+(et+1)2
~+J
q+1
r+(q+l)2
q+l
q+1
q+1
J
J
q+1
q+1
N times tile precision matrix N"'/-XIX )-1 for the origina1variab1ss is
o
1
2
l+2q-l
11
-q
-1
222
22 12 JIll
-q
112
-1
1+(q+1)2 r -1+q-1
- { q+l)r
..I
-q
.. (q+l)r-1-1
-q
1+(q+1)2 r -1+q-l
q
-1
-1
q
1
-1
-(q+1)r
r
.. ( q-l-l)r -1
-q
·1
122
-1
-r
-1
q
-1
-1
..13-
2,..71€
For
fLur
level factorial, for e"ample, q
0.61
e
l'
and r
0.1.304
t
=~-:2~?6)
thus
q/r )- ~~n so that in this design" the v.3.riance of the cubic effects b iii is
({.4oVtimes as large as that for the quadratic times linear interaction bijj •
'---
terms of derivatives the variance of
l.f
l.f iii
is over
In
48 times as large as that for
iij.
3. DEPENDENCE OF THE PROPERTIES OF DESIGNS ON THEIR ORIENTATION
Whereas, withthe designs ot order one" the principal of minimizing the
variance of the effects tor a given spread of the design points leads to the
criterion of orthogonality (Which gives uniquely a simple class of designs), for
designs of order greater than one this principal does not lead to such a unique
class.
If we decide what the relative variance of the effects should be, and hence
what the diagonal elements of the moment matrix will be, we may, by using an
orthogonal design, obtain smallest possible variance for this ohoice of diagonal
elements.
elements we should choose.
sion.
Some further principal is required to make such a deci-
However this approach gives no clue as to what values for the dia.gona1
Now we wish to use the designs to explore a 100a1 response surface of which
little is known.
design is unknown.
It particular the orientation of the surface with respeot to the'
For example" suppose the surface could be represented locally
by an equation of second degree" then the response contours would be a set of conics
which could be referred to their principle axes.
The orientation ot these axes and
the direction of the center of the system relative to the axes of the factors would
differ from one problem to another.
In these circumstances it would seem
unsatisfactory if the accuracy with which the constants of the surface were estima.ted
depended on the orientation of these axes.
-J.43.1 EFFECT OF ROTATING AN ORTHOGONAL DESIGN
In the developments which follow we need to use some properties of derived
power and product vectors and the corresponding Schlaflian matrices (Aitken: 1948,
1949, Wedderburn; 1934).
If 2£'"
derived power vector of degree p p
(xl' x ' ... , ~) then we denote by
2
For example if k 2
-7 the
x'£~
III
and in general a£,£E7 will contain as elements all the powers and. products of degree
alil~
p and less
that !
,1.€7 ~p7 . . L-!,t a£.7£§,7.
z• H--x,
-
epee!'! et·1fte elemen'e !i:1'! !' with suitable multipliers attached so
If a vec'tox' ! is transformed to a vector ! by
the pth Schfaflian matrix
readily confirmed that
-HI.E7
is defined such that
I.-!! !.-71.i7 !!LE7 tJ.p.7.
III
zZE7.. HI.€7-xl.~7.
-
It is
Also if H is orthogonal then so
also is IJ!.E7.
We may now consider the effect of rotating an orthogonal de sign 0
Consider in
partioular the case k .. 2, d ... 2., and let us write the equation to be fitted in the
form ~ ... !. ~ .. that is
Suppose we have selected. some orthogonal arrangement for which the design matrix is
~ and. q .. Y' 2 + 2 has some specific value 0 The matrix
0
0
N-
l
1
'-!:L7=
2
1
2
11
22
N-
1
L-!'!.7 is
:.2
1
1
(28)
1
11
22
12
then
2 + 'Y'2
2.
-15If the design is orthogonallly rotated through an angle Q so that the
matrix is
D
-
III
D Hand H
-
-
III
_
,c
/l§
~
where s is sine
-8
c
"
Q
ne~
design
~_CUIO'
and c is Ifons~ Q" then the
~ , .:;)
uth row vector in D x' • r xl L. 7 will be transformed to a new row vector w
-j-U " Ulii::U-u
by the transformation WI H x' • The transformation which carries over
-u - - U
~!.~7
X~UI Ji.:r.1U 2u.7 to~L~7 L·wiu}w~u )v2wlu w2u_7 is
III
III
,-xiu'
x
III
0
H,~7
s
III
sc
2
s
2
0
J2
2
-so {2
2
so fa
- sc {2
0
2
- s
2
This transformation likewise carries over the modified vector
L-xiu
- 1"
L-wiu • 1"
~U
w~u
a latent vector of
.. 1" V2x lUx2u.7
to the modified vector
- 1" {2wlUw2u_7
-HL~Z
; for the veotor (1, 1" 0) is always
After rotating" the 3 x 3 matrix P in the lOvIer right hand corner of the
-
transformed moment matrix becomes
e ,.J}
P • !!. ,Lgl f(2 • \)13 •
III
(2 + "(,) I
.... -3
• ((. a a t
2..- -
0
III
(2 + Yo) ) I _
'l:,
l-
!!.LV
(30)
..I!...
2+\
(31)
't
. i/WI \
where a is the 3x 1 vector with elements
-•
matrix
f will
contain off-diagonal terms.
e/ sc ,r:
~ 2,,-so Y2
2
J
Its reciprocal may be readily found using
a formula given by Toohar (1951), who shows that (! + ML)-llll
Where L and Mare not necessarily square matrices.
-
-
2
c - s • In general this
! - tl(! _ ~)-l ~.
'Y
Putting....:.J!.c. a
(2+~ -
III
N· and a l
-
-
III
L
-
we obtain
(32)
From elemerltary trigonometry!.!,' may be expressed in terms of cos
only (written as ~ and
'13)
•..
" , ..
and finally we obtai
A
-A
A
-B
.,,>,-
49 and
0.06
89
J
~~ 1
(33)
where
therefore
1
1
(34)
1
The linear effects have the same variance in all orientations and are uncorrelated
with all other terms.
However for second order terms the variances change markedly
as the design is rotated and the effects became correlated.
As an example, the effect of rotation on the varia.nces and correlations of the
second order effects in the three level factorial design is shown in Table 2 and
illustrated in Figure 1.
Angle of
Rotation 0
0
0
15°
22oS
o
30 0
45°
Variance
e
tJ,adraticB
2.
1.8
1.63
1.44
1.25
Variance
Interaction
1
1.75
2.50
3.25
4.00
Correlation
Quadr.atic x
Quadratic
0
0.10
0.23
0.39
Q.6O
Correlation
Quadratic x
Interaction
0
..0.32
-0.)7
-0.26
Table 2.:
0
2
Change in Standardi~ed Varianoes N V(b)/a of
Effects as Factorial Design is Rotated.
. -18-
Changing Varianoes
of
Interaction terms
bij
2,0 -~;~~.::=-==_---.-u.==~'.::.-:;-~::.:~"'=--=:."'-~~::::=.~~~=..------1.0
.
O4
-........ . .::.~_
.. ",
~.--......__....__._.... -------:-::.::.
. --------------.-...--------- ..__.. __...
n.
...
__
--_._ _
--~--_
--
_.. _-_
_._ _ _-_
-
..
~.-
...... ......
. -_.
Changing Variances
of
Quadratio Terms
b
ii
. - .-....-.. ~-.-.... -_.. _-....
. . . _ ~ ...
_.~.~
-
/'
~""
0.2 ---.-...._.. .- ---.----..--.-.. --.L~-.- .---..-~~"'_ . - ---....-.. -- ....--..--.. -.- .---.-.- Changing CorreI
O
\
-0,2 -
!
39
!.
/'
/.
'-
6?
"-
,
!.~
I
~o
,
~
i
/
~---_;/_--._._--..-- -··-----····-·..··---\..,-··----·-......--·-··---·7/.....·. ·
'..
-
....
'.'._~
,,.-""
. .-/
-0.4 -.-----
.-./'
.. "
. .- . - - -...--.-....-.-.--.---..-.-..-- ----..- .--------
:::~~=-~=L/~~~~ ----_==-:=~=~_=--:,/0.2. -
'"
,.
-.-.-J----------.----.-.
_~ .
//
o
lations between
Quadratic by
Interaction Terms
...L-_....L_ _, .-..-_... .l.__
0
o
30
60
0
""""
!
. _ ._._ - --',__
.,J.,...- /-/~
!
.......
90
0
'/,1'
/
__..
"
Changing CorreIa tiona between
Quadratic Terms
L _ ...-l-I-,,---1-_ _
1200
Figure 1
2
3 Factorial Design Under Rotation Standardized Variances of Effects
We see that our oondition of orthogonality refers to orthogonality in a particular
orientation and that this property will usually be lost on rotation of the design.
We notice also that since the variance and covariance of the effects may change
markedly from one
ori~ntation
to another, the apparent efficiency of the design"
as judged by inspection of the variances in one particular orientation, may be
deooptive.
-20-
4. INFOBMA'l'ION DISTRIBUTIONS
The objeot of our experimentation is to gain lmowledge of a response surfaoe
which it is assumed may be represen'b#ed by an equation of a certain formG
We are
interested in the individual terms in the equation and their varianoes only in so
far as they supply us information about this surfaoe.
Suppose that some surface had been fitted.
point xl"
"2" ..." ~
The predicted response
y at
a given
would be provided by the equation
The varianoe of this predioted value would also be a function of xl'
"2' •""
~
and hence so would the distribution of information in the space of the factors.
Writing I(y) for the information per observation we have
ICy)
where
j
(Xl' ~.t
•
u,
CI
l
NV(y)
i
-1
II
t(xl» x2" ...
J
(36)
xk )
x k ) may be called the information distribution.
using the three level factorial design in the oase d
1:1
2, k
1:1
For example
2, we see £rom
equation (19) that the information distribution is
I(y) •
(S.
3xi - 3~ + ~ + ~ + xi~.rll a2
Information contours for ttds distribution are shown in Figure 3.
(31)
-21-
p
~
distance out from center of design
Figure 3
Contours of Information Distribution for 32
Factorial Design Assuming a Second Degree Equation
\
ri,5
would be
~x'pectec1
from the earlier discussion and indeed f:r;'om the placement
01'
the point.s we flee that at a given distance from the origin a greater concentration
01' information exists in some direct:tons than in others.
It seems of some importa.nce to consider designs, if such exist, which have
the property that the information is constant at a given distance from the origin,
in
ov~erwords$
'the
~~crmation
SuCh arrangements wilL
~e
contours are circles, Bphe:-'es or hyperspheresc>
called rotatable designs.
, $•
CONDITIONS FOR ROTATABILITY
We shall need to consider some properties of spherical distribution functions,
(Box 19.53).
These distribution functions are of aome importance in basic statisticcd
theory, and especially in randomization theory.
left to a later publication.
A discussion of these aspects is
Here we shall need +llem for a 'somewhat different
purpose.
If the joint
d~stribution function
of a set of variates, zl' z2'
SGY,
Zk'
whioh may be regarded as the elements of a vector !O, and each of which has zero
mean and unit variance, can be written in the form
where Wmay be infinite and k is taken so that the integral over the whole space
is unity, then since the density will be constant on hyper-spheres centered at the
origin of the z' s, we shall say that the, variatas
have a spherical di atribution •
. :.
If all the moments of a distribution exist, and the m.g.f. Cf (t) oan be expanded
in an infinite series we can write this
Ger~es
.
~ (t)
~.,
(39)
-24then it has been shown, (.J. Clarke Maxwell" 1820"
fil. S.
Bartlett" 1934) that the
only spherical distribution possible is the multi-variate normal with equal variances
-
and zero covariances, which may be called the spherical multi-normal distribution.
For this distribution the
is
m~g,f.
(n(t) :: exp ! (tit)
'\
- 2 --
and all the AI s are equal to unity.
(45)
We see therefore that for any spherical distri-
bution" the moments of the same order bear the same relationship to one another as
do the moments for the spherical multi-normal. HOl<rever the moments of different
orders will depend on the
hiS
Consider ·the response
and hence on the function
y predicted
f(!l~).
by the regression equation at the particular
point whose co-ordinates are given by the last k elements of the vector
defined as
~l
:: (1" xl" x 2J
~I
now
). Since the equation fj.tted is a polynomial
k
in the k factors of degl'ee d then the polynomial contains (k ~ d) :: L terms and the
H
iJ X
fitted equation
(46)
may be written
(note y is a single
predicted value,)
where the LxI vector
~ conta~~s
that (47) is equivalent to (46).
all the bls with
6uitab]~
(47)
multipliers attached so
Suppose also that the true value at
t~is
point is
given by
(a scalar)
(48)
,.
Then the variance of this predicted value is
The variance at a second point which is the same distfance p from the origin whose
co-ordinates are the last k elements of the vector Z
- -R -x where -R is an orthogonal
II
(k + 1) x (k + 1) matrix consisting of an arbitrary orthogonal matrix!! bordered by
a first row u'
II
(1, 0, 0,
tt.,
0) and a first column~. Making the substitution in
(49) we have
($0)
($1)
To satisfy the condition that the variance is constant on a sphere centered at the
7-1~
origin of the design we require therefore that the precision matrix 0-1 ... "r !tL
-1
-1
.
and hence also the moment matrix N C" N XfX I remain invarien t when the design
-
is rotated,
--
"
This means of course that every variance and covariance of
~~e
b r sand
all the moments and mixed moments of the design remain constant under rotation,
We now have to find the form of the matrices C and C- l for which this is so.
.
Consider the quadratic form
-
-
($2)
Q is a generating function for the moments of order 2d and less of the design" for
($4)
,
,
.23where
!!!s
is the vector of mommts
ClxL~71.
But for a spherical distribution the
mog.f. is
(0
\ (t)
&I
e (e-tlz-)
C
for any orthogonal matrix!!o
,1' this implies that
~
(40)
&I
Regarding now the matrix!! as transforming the matrix
(t) is unchanged by afrl transfonnation on
!:. which
This m.g.f. is then a function of !:,t!:,
unchanged.
~ (t)
c.o
&I
1 +
~ lU
pllll 2p
(ttt)p
(41)
where the w~ are real constants depending on the function f in (38).
~P
&I
leaves !:,I!:,
Writing
W2P(pD2P the mDg.f. for the spherical distribution can be written in this
(\.2(>'
form
Qo
L( (t)
&I
1 +
,.
~ C\ 1 - (tlt)p
pal ~ pt 2.P - .,(2
2. J
(42)
.,(k
•••
-7 for the moment
,k
(43)
'2..<12. n
k
(1
-.,( )'•
i=l 2
i
k
where .,(
&I
~ .,(. will be called the order of the moment.
i=l J.
If the zlsare independent so that
P (z)
k
III
1£
i-1
P (S1)
(44)
-26..
..
"
~
, in this expression is
~N
k k 7 is the moment If-1 ~
u-l
Now we require that C should be such that
where ,-iJ'l, 2..(2 I
(56)
••• ,
-
(58)
--
that is to say, any transformation whioh leaves tit unchanged does not ohange Q.
--
Hence Q is some funoti on of tit and since it is a polynomial in the t' sit must be
.
of the form
~l
~
.,(k
The coefficient of t l ' t 2 ' ••• , t k in this expression is aero if any of the
are odd integers. If the ~i are even integers this ooeffioient is
1
(~
we
~1
(60)
I
-\:).
now equate ooefficients to obtain specific values for the moments up to order
~ • 2d
(61)
-27Write
1
2',,( 1
I
2
(~ -<).
.-,;.--...;;..------ ... A
2dl
~
11",(
(62)
Then finally the moments of a rota tabJe design of order dare
... ,
if one or more of the
are odd
~i
(63)
if all of the ..(i are even
which (in equation (43) ) are the moments up to order 2.d of the ~pherioal distribution.
Thus a design of order d will have the proposrty that when a polynomial of
degree d in the variables xl' x2 ' ""
~
is fitted by the method of least squares
all points at the same distance p from the origin of the design are estimated with
equal aocuracy if and only if the moments of the de sign up to order ad are those
of a spherica 1 distribution.
That is" those given by equation (43) where the AI s
are arbitrary. With these values the information distribution will be spherical
and as we have seen" the moments of the design and the varianoes and oovariances
will remain constant whatever its orient,tion.
Since the dummy variable X is always unity" and we have selected the design
o
l
so that NII 1" A
and
are such equal to unity by assumption.
O
u-l
~ X~u
~
5.1
RC11'ATABLE DESIGNS OF ORDER 1
Suppose we have k variables Xl" x2" ... , ~ and we £it a polynomial of degree
d .. 1" that is to say the fitted equation represents a plane,
(64)
-2.8·
then using (63)
= 1,2, ••• ,k) of order 1
/-ij.7 (i f j, = 1,2, ••• ,k) of
•
all moments '-i.7 (i
are zero,
mixed moments
order 2 are zero,
quadratio moments
,-1i_7
(i .. 1,2, ••• ,k) of order 2 are equal to ~ = 1.
Thus the moment matrix is
,-1_7 ~-2_7
1
N-l
,-1.7
{iJJ
('12)
,-!tV .. ,-2..7
LJ2.7
(';.2..7
••
•
•
••
~
and
Conae
.-. - ••
.. ,
l~k7
•
••
.. !
(65)
k+l
0
'-k.7 {ik.,7 L"'J.k_7
.\
••• L·k_7
••• (.1k)
{k~7
first order rotatable design is one whose moment matrix is the unit matrix.
ently such a design is obtained by writing down any k vectors mutually
orthog al to a column vector of ones, and each of which is standardized so that
~
.~ x~
~u.l u
III
1, We notice that this is the same conclusion (Plackett and
BurmanJ 1946, Box; 1952) which is reached if we pursue an apparently different
objective, namely if we select a design which supplies estimates bO,b1, •••,b
haVing smallest variance.
ICY.)
The information distribution for such a design is
fl4
2 .-lI1
a
k
(1
~-~
k
where
2
.~ 2
p " ~ ~,
(66)
i~
l+p
In Figure (4) below the information distribution is shown, I(y) being plotted against
p, and we see that in this example the fall-off in information as we move away from
the origin follows the Cauchy
distribution~
1.0
r
.29-
D·8 ~
I
o
Figure
4
Information per observation at a distance p from the
center of a 1st order rotatable design.
,.2
ROTATABLE DESIGNS OF ORDER TWO '
Suppose we have k variables, and fit a pol;ynomial of degree two.
if k
= 2 we
For example,
have
(67)
Then using (63) we know that all moments are zero in which any of the .,(.1 ~re odd.
The remaining moments are then
l.:i1.7 = ~ • 1, l.-iijj.7
A4' '-iiii.7
=
= 3A4•
Thus, for example, if k • 2
,-lk7 L-~_7 ,-11.7 ,-22.7 1.- 12 .7
L\7 ,~1!7 ,-l'{I ,MUg '·12g7 ,rll~7
1.~~7 ,-157 ,-2~7 l.- l1g7 1.·22~7 1.-12~7
(1.!7 /.'11; /.'1~7 {1ll]J tl12~7 {hl~7
LTJg7 t!2.g7 ,22g7 /.i12g7 L1J.22~7 lJ.22~7
L'!:f.7 I.n~7 1.~1 I.Ill~7 {r.22~7 I.I12~7
where A is written for A4'
• ,
1
1
• 1 •
• • 1
•
• •
1
1
•
•
1
•
1
• •
• • •
•
• • •
3A
A-
I
A 3A-
•
•
A
•
(68)
-30In general" for every value of k the matrix N-
l
l~ Il.7 will be or the same
form" in which the only mixed moments that occur are those correllponding to the
variables Xo' xi..
x~ ....... x~.
The measure of kurtosis for this design pattern
is '\2 = .3 (A - 1). We notice in particular that if the design is to be orthogonal
in the sen·se of
§
(2.) as well as rotatable then using (19) '\ • 1, 1'"2
moments of the design to order 2d
:I
a
0 and the
4 are the same as those as the spherical multi- _
normal distribution.
'1'0 determine the inverse matrix
·
sub ma t rl.X
!!
NL-!"l.,7-1 we
partition off a (k + 1) x (k + 1)
of N-lr
c. !,'!..7 corresponding to the variables
U
-
:I
1
1
1
• ••
1
1
.3>"
A
1.
1
>..
3A
•••
• ••
•
•
•
•
•
•
•
•
•
•
•
•
>..
>..
L~
• ••
2 ~,
2 ' ' ' ' ~.
2
Xl'
xo'
A
(69)
3>"
for which the inverse is readily shown to be
2>..2 (k+2)
U- l
=A
II
t
e
-2.>"
•
•
•
-2),
-2>"
(k+l)A-(kilil)
l-A
•
•
1.>..
-2).
l-'A
(k+l)>..-(k-l)
•
•
1.>..
•
•
•
•
•
•
•
•
•
•
•
•
l-'A
1-).
L -2>"
where A • ,-2.>"
-2).
f (k + 2) )" -
k
i.7
-1
•
•
(70)
(k+l»"- (k-l)
(71)
-.31·
The remaining elements in N- l
L-!f!..7
are the diagonai elements corresponding to
The reciprocal N (!f!..7-1 consists of the elements
linear and interaction termsQ
Q-l with the reciprocals of these remaining diagonal elements.
For the general
second order rotatable design therefore we have
2
NV(bO) • 2A (k+2)Ai; NV(b!) =
NCov(bO,b ii ) •
i;
NV(bi i ) • ,tk+l)A - (k-l).lAf}; NV(bij )
-2.M,/·;
,\-1 2
•
1\
"
(72)
NOov(biibj j ) • (l-A)Ai
and all the remaining covariances are zero, We see that for any rotatable second
order design, all first and second degree effects will be un-correlated except the
quadratic effects whioh are correlated with each other, with coefficient of
Q~t!,.l
~(1
2 _ (k + 1) (-1. We see that for any value of A the matrix
- A
j
is of a simple form and experiments carried out with such designs would be
correlation
readily analysed.
If the design is rotatable and orthogonal so that A = 1 these correlations
vanish and we have
Ni(b ) •
O
.~(k~),,2J
)If"
(7.3)
NV(b ) . ,,2;
i
NV(b
ii
)
2
-1 2
NV(bij ) • " ; NCov(bObii ) • 2 "
The condition of rotatability and orthogonality fixes the relative values for effects
of different orders.
In particular for designs of this sort the variances of the
quadratic effects bii are 1/2 those of the two-factor interaction effects bij • They
may be compared with three-level factorial designs for which the variance of the
quadratic effects is twice that for the interaction.
Compared with the factorial
the rotatable orthogonal design thus places four times as much emphasis on the
quadratic effects relative to the interaction effects.
~
The information distribution for any the second order rotatable design is given
by
(74)
•
In partioular, for the rotatable orthogonal design this is
1m
.f~ ('k
+
Jt;
2 + P4
(TJ.
In Figure S ICy) is plotted against p for various values of A.
We notioe that
whatever value of A is ohosen the information talls otf rapidly when p exoeeds unity.
If we chose >..
'=
1 the design will be orthogonal.
For this and other higher values
ot >.. the distribution has a large value in the center and the information will
generallY,' be slightly greater than it is for the lower values of ). even when p
exceeds one.
It is perhaps well to remember at this point, however.. that we are
comparing designs for which the 11 spreadllof points, as measured by the marginal
2
-1 J1,
- 2'
second' moments 8 i CI N
.2 (X.2 • Xi)
is oonstant. Such a convention is bound
ullll
J.U
to favor designs with a high value of'Y , so that. although the general shape ot
2
the information distribution will be meaningful, the relative heights of the ourves
will be to some extent an outoome of this convention,
It seems reasonable to ask
for a relatively uniform distribution of information in the immediate vioinit,y ot
In partioular, 1£ the information at p ... 1 is to equal the informati0ll;
the design.
at p •
0.1 the. following values of \
2
.3
0.7844
008385
k
~
Table .3:
will be needed
4
0.8704
S
0.8918
678
0.9070
0.9184
0.9274
Values of \ Required to Make the JlJrlount ot Information at p
Equal to the Amount ot Information at p ... O.
III
1
6. DERIVATION OF 2nd ORDER ROTATABLE DESIGNS
The above discussion had been direoted to deciding what type of design we
should be seeking.
It has appea red that a second order rotatable design wi th a
value of >.. which gave a fairly uniform distribution of information between p equals
.
-33-
til
OM
til
! 0.5
a
OM
~M
OJ
!Il
.0
0
L
0.4
Q)
A.
t1l
§
$:I
0
OM
+l
0.3
E
e
.:i
0.2
A = 3/5
0.1
7 points at
5 design
3 center
1
0.2
0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2
p
= distance
2.4 2.6
from design center
Figure
5
Information per observation at a distance p from the origin
for second order rotatable designs with various values of A
-34..
zero and p equals one would be satisfactory,
We now consider how designs of this
type may be obtained,
We have seen
th~t
for a spherical distribution of information the moments of
the design must be the same as those of a spherical probability distribution up to
order 2d,
If such arrangements are obtainable, therefore, we might expect to
construct them by trying, as nearly as is possible with a finite number of points,
to form a spherical distribution, that is, a distribution in which the density of
the points is constant on spheres.
The nearest we might expect to come to this with
a finite number of points would be an arrangement with the points equally spaced
over the spheres.
We shall need, therefore, to consider the moment properties of
the regular figures in k dimensional space.
We shall find that it is often convenient to regard the designs as built up
from component sets of points.
an "arrangement".
In what follows suoh a component set shall be called
If the arrangement satisfies the moment conditions for rotata-
bility up to order 2d, we shall say that it is a rotatable arrangement of order d.
It is important to remember that n points at the origin provide a rotatable
arrangement of infinite order.
6.1
ARRANGEMENTS WITH ALL POINTS EQUI-DISTANT FROM THE ORIGIN
We shall find that it is possible to obtain rotatable arrangements, i.e.
arrangements that satisfy the moment conditions (63), by using n points, eaoh of
which is the same distance p from the origin.
oannot provide seoond order designs.
Suoh arrangements by themselves
This is seen as follows:
The sum of squares of elements in the u th row of the design matrix for suoh
arrangement is p2, whenoe
(76)
.J
-35..
(77)
Also
(78)
But from
(69), if the arrangement is rotatable (78) must also be equal to
(79)
3kA + k(k • 1)",
Whence
A
4
III
P
(80)
/k(k .". 2)
If we attempt to use such an arrangement as a second order design then putting
2
'·ii.7 III 1, (i III 1,2, ... ,k) we have p III k, whenoe
)..
III
'Y 2
III
k/k
or:_
-6/ (k
(81)
2.
(82)
+ 2)
Using (71) for such a value of A, A is infinite and the quadratic effects are not
estimable.
By combining two or more suoh arrangements, however, we shall see that designs
may be obtained for which the value of '" is not pathological.
6.2 OOMBINATION OF ARRANGEMENTS
e
Suppose we oombine I, k-dimensional arrangements (not necessarily rotatable
arrangements of order two) to form a rotatable design of order two.
Suppose the wth
~
A.
points all at a distanoe p (and ~ nw - N) 1 and that
w
w
w-l
the marginal seoond moments are all equal" (and henoe are given by ,r~7wl:lp;;k).
Supp oe8 finally that for the wth arrangement the measu~e of kurtosis of the
suoh arrangement oontains
11
I
variable xi is
(J + 'r.
Then for the entire design
.)p
k~W1 w
'6 2wi" so that '-iiii.7w A
@/'r
'-ii.7 .. (kN)-l ] n rl
w-l ww
,";V'l
(8)
-""
(";Ii
,-iiii 7
•
1:1
(k2N)-1 ~ n p4 (3 + 6" i)
w-l w W
cW
Putting '-ii.7 • 1 we h~ve for the measure of kurtosis
the oomplete design
N
~.
'J'?'
1r2i
(84)
for the variable xi in
A
n
0
4
w-l WW
()
+
'Y2wi)
(85)
Whenoe" if by oombining such arrangements we oan attain a second order rotatable
design" this expression is equal to )A, that is
We note that for n points in the center the term p~() + 1r ) in (85) equals
1
2li
2
k ,-iiii.71 which is zero,.
7. TWO DIMENSIONAL DESIGNS
Consider n equally spaoedpoints on a circle of radius p in the complex plane"
and suppose the first point makes an angle t.f with the real axis" then if Mp is the
pth moment of the projeotions of the n points on the real axis
(87)
-37where Q is the angle between radii from the origin to suocessive points l
(88)
where
e
i2n/n
i9
• e
= W,
(89)
that is
(90)
(91)
If ~
101
0.. a
101
1 and substituting this value in (91) and substracting the result from
(91) we obtain an expression for the change in the pth moment on rotating through
an angle
<f ,
(93)
n-l
Now
~ W (p-2t)u • n whenever p - 2t is
0 or mn, where m is any positive or
u=O
negative integer
(94)
lIJ n(p-2t) _ 1
101
p-2t
l.A)
e
•
0 otherwise •
-1
We see therefore, since for t
101
0.. 1"2", .. ,p,, we have -p~p .. 2.t~PI then if p(n
all moments on the real axis are invariant l'nder rotation. For if p - 2t • 0,
2t
so that (94) yields a non-zero product l (aP• 1) in (93) is zero. We notice
-38however, that for p )n the moments on the real axis will not in general be invariant
under rotation.
Thus for a set of equally spaced points on a circle the marginal moments of
the projections on any axis through the center remain constant.
It follows that
the moments and the mixed moments of the two-dimensional arrangements up to order
n - 1 are constant for any set of orthogonal axes through the origin and consequently they are of the form
(68). Hence n equally spaced points on a circle
provide a rotatable arrangement of order (n arranged in a
~gular
1)/2.
In particular
$ or more points
polygon gives a rotatable arrangement of order 2.
Since the points are all equidistant from the origin, however, we know from
§
6.1 that we could not use such an arrang~nent as a second order design.
these points alone" the quadratic effects are not estimable.
For, with
This is readily con-
firmed for the particular case k .. 2 for if p is even, t he only non-zero element in
(91) 18
(~)p (:12 In thus
(!n
,p
2t"1
M •
p
(9$)
which using equation (43), is the p th marginal moment for the spherical distribution
for which
A
=
p
If
,-11.7 c: ,-22) .. 1"
'( 2
1:1
p •
V2 and >"p '"
-1.$, a pathological value.
substituting
'r 2wi
1
(~) ~
+
(2P)~
((..)~
• \
(96)
t;
and in particular
'4 .. 1/2"
For more than one circle' of points however
= -1,$ in (8$) we have
S
2
.
S
(
W~l nwP;)
>...
~ N r~l nw P~
(97)
-39a formula which clearly applies also When n
Pl equals zero).
l
of the points are at the center (when
Substituting any desired value of h in
(91) produces an equation
any solution of which with n1)O and nw~5" (w.,= 2,,3 ..... ,s) produces the desired
£econd order rotatable design.
will exist.
of n
Those which involve the fewest number of points will have one circle
= five points with n
2
It is clear that an infinity of solution of (97)
l
points at the oenter.
®/"I
The value of ). for any value of n l is then
h
II
!(n + n)
2 1
2
n 'p. 4
n +n
2"w • 1 1 ow!
2
2 n2
4
P
n2 w
II
n, )
l};t
!(l + ..,g)
n1
2
(98)
Thus putting n = 5 we have.. in particular..
2
Number of points in center
of pentagon (n )
l
Value of ).
Table
It is seen
4:
o
1
1/2
3/5
5
4/5
Values of h for Points in Center of Pentagon.
tha~ a very satisfactory design" in a sense discussed in ~ 5) is obtained
with n
= 3. This arrangement with five points at the vertices of a pentagon and
l
with three points at the oenter provides a design (see Table 3) giving about the
same amount of information at pliO as at pill.
For orthogonality and rotatability
we require). = 1 Which gives 5 points in the oenter. '
Orthogonal 'arrangements which however involve more than ten points may be
obtained using two concentric circles containing nl?r 5, n2 ~5, at distances Pl and
P2 from the center which satisfy (97)
-40l
5
5
5
5
6
6
6
7
7
8
2
5
6
7
8
6
7
8
7
8
8
Pl
2..000
2.047
2..089
2.128
2.000
2,040
2.076
2.000
2 0034
2.000
P2
0
0.417
00557
0.647
0
0.385
0.518
0
0.359
0
n
n
Designs in Two Concentrio Circles
Table 5:
•
8. SECOND ORDER DESIGNS IN
MORE THAN TWO DIMENSIONS
There exist only a limited number of regular figures in k dimensions.
In
= 4; octahedron,
dodeoahedron with n = 20 points.
particular" in three dimensions there are only the tetrahedron, n
n
a
6; cube, n
= 8;
ioosahedron, n
= 121
and the
The tetrahedron" octahedron and cube are not capable of forming alone a basis for
a second order design, but there remains the possibility that two or more such
arrangements suitably combined, with suitable values of p" might provide seoond
order arrangements.
We should, therefore" consider the moment matrices for regular
-
figures submitted to a general rotation defined by the k x k orthogonal matrix H.
Given any N x k design matrix
rotated by post-multiplying
£,
the points making up the design may be generally
£ by!!~
Similarly the moment matrix N-
l
!'! associated
wi th a polynomial model of order d is transformed by rotation to a new moment matrix
by pre and post-multiplying N-
l
!'! by
an orthogonal matrix derived from
!i.
For a
second degree k dimensional model the moment matrix after the rotation of the design
is
"...-
1
1
-1
N
-XIX
H
H,~7
-'
-
(99)
H
'--
H,~7
-
where
!!L~7iS
the seoond Soh1at1ian matrixJ i,e., if
'-H •
1tle ·seoond Soh1rflian
HL~7m8Y be
au
a12 _
8
a21
~2
a23
&31
832
833
13
(100)
written in abbreviated tom as
(101)
where the partition is made at'terthe kth row and oolumn and where
,.(
.
2
&11
82
21
2.
~2
2
8
23
2
aU
2
8
13
~
III
2
8
31
2
a
32
2
8
33
58118
21
.Ji811831
{2~1831
[2a 8
12 22
·[.2.a 8
12 32
/2.11
[2a13 8?3
,[28 8
13 33
.J28 8
23 33
22 8 32
(102)
'rill
-
6
•
[2:a a
11 12
.J2'8
21822
.J'2a 8
31 32
~2a11813
fia21a2J
{28.318.33
~812813
{2822 8.
23
128328.33
(a11822. + 8218 )
12
(8. 8
+8 8 )
11 32
12 31
(~1832 + 822 831 )
(8118 23 + 813~1)
(an8 + 8 8 )
33
13 31
(8
12 8 23 + 813 8 22 )
(8 8
+8 8 )
12 33
13 32
(822 8 + 8 23 8.32)
33
(8
21833 + 823 8 31 )
---.J
-42Since
~
is orthogonal, it can be easily €hown that
To correspond to our definition of derived power vectors and Schlaf1ians we
write the model in the form in which the product terms xixj are written with a
coefficient {2, for example for k = 3
We may now investigate the moment matrices of the regular figures under any
rotation.
It has been shown previously,
§ 5,
that the moment matrix
N-~I! for
any rotatable design of second degree in k dimensions must be, (with the present
definition of the interaction variables) of the form
1
1
1...
1
1
1
••
•
1
(103)
••
3A • • •
•
Symmetric
• 3'A.
2>"
2>'
•
o
•
2>' j
-
-43Substituting
!t! for
the tetrahedron in (99) and solving for the moment matrix of
the generally rotated matrix of the tetrahedron, all of whose points are on a
sphere of radius
{3,
we have
1
-
-j
a
-
! XIX·
4--
1
1
1
a
I
I
.1
I
where
~
- -
~ 'r'J 'H~ 1-*.' +2-n,_-_
I
I
I
I
- - -
~'~I!!,oc
J •
!
0
I
is a k x k matrix such that
and the vector
1 1 1
- .- - _f-
'r/i H'~:r
- !~ - - - - - ~~~~~
-- - - -
0
I
I
_-.1-
28 1'(
~'=
(104)
2Y'8
-
--
-
-
28 1 8
!, thus for k = 3 we have
o
o
1
o
1
o
1
o
o
(105)
is a column vector all of whose elements are unity so that the
matrix 11' is a k x k matrix all of whose elements are ones, that is when k
11' •
1
1
1
1
1
1
1
1
1
=3
(106)
We see that the moment matrix for the generally rotated tetrahedron satisfies the
moment requirements up to order two, but not up to order four required for a
rotatable arrangement of second order.
The moment matrices for the generally rotated octahedron and cube with p •
are
Vi
~...
-441
.-
o
_
I
0
j
I
I
1 1 Ii
I
o
i
:-~11_
I
I
L _
o
1._
3~'~
I
3~~
-
._1-
__
-
I
3.£'~
0
I
0
I
o : ~.3
j
0
I
1
!rx'x
7=
8" - - .
.+
0
I
o
-I .- -IIf,
I
1 I 0 I 11' .. 2"('y
1 I
--- I
(107)
- -I o
3~'~
I
o
l' 1
-':--1--
0
1 I
~L!!.!l· i
1
0
-1-
I
0
CUBE
OCTAHEDRON
These matrioes both satisfy the moment requirements for rotatability up to order
two~
but not up to order four required for a second order rotatable arrangement.
The moment matrices for the generally rotated icosahedron and dodecahedron
with p ..
{3 are
of the form
1 ' 0_ ;_1_ 1
1
.-
_·1_
_. -I _
o
!N-X'X
..
I
I
0
I
-3 ,
I
--1_--
1
I
I
1 I 0
I
1
-
1
-'1- -
o
I
0
I
0
where N .. 12 for the icosahedron
-,
I
~I.3 + ~ ±l'
1;'-;)
r
0
-
- -
0
'.
I
-
-
-
N .. 20 for the dodecahedron
0
-6 . .- -
5' 13
The moment matrices for the genera lly rotated icosahedron and dodecahedron do oblige
all the moment requirements up to order four neoessary for a second order rotatable
arrangement. However" as outlined in
8 6.1
any arrangement of points" all of which
are the same distance from the origin, cannot provide a seoond order design since
the mean and quadratic effects are not estimable. In fact we note that
~
• 315 .. k/(k
+ 2) agreeing with (72).
However, as indicated in ~ 6.1, different arrangements of points may be combined to giva workable rotatable designs. The probler.n now is to change the 't2
(i,e., the
~)
of the designs without affecting the relative magnitudes of the
-45-
e
moments of the same order 0 This is most economically accomplished by adding n1" new
points at the center of the design.
Adding n l new points at the center yields for
the complete arrangement
3h
=
In particular, to guarantee an approximately uniform distDibution of information
throughout the interior of the design; we note from Table 3 that we require A.
=
= 0.84,
=8
5 points for the icosahedron, and n l
1
points for the dodecahedron, thus giving designs requiring a total of 17 and 2-8
which is most nearly attained with n
points
respective~.
For an orthogonal rotatable design A.
n • 8 for the icosahedron and n
l
l
dodecahedron.
= 40/3
a
1 which requires
(or 13 to the nearest integer) for the
It can be shown (R.S,M. Coxeter, Regular Polytopes, Methuen and Co., Ltd.
1948) that the twenty points forming the dodecihedron may be divided into five sets
of four points, each of these sets being a tetrahedron, that is, five tetrahedra
may be inscribed in a dodecahedronll
As indicated in
§
6.1, given that for each of
the arrangements all the points are equidistant from the origin, rotatable designs
are possible by combining suoh arrangements,
defined by suitable rotations
~l" ~,:I . . . ,
~,
ThUG
if we imagine five tetrahedra
the sum of the five resultant matrices
(104) will give the moment matrix of a rotatable arrangement.
A further example of a combination of sets of points not themselves second
order rotatable arrangements whioh together form such a configuration is provided
by the octahedron and the cube.
e
Taking advantage of the faot that
~r~
III
-
'tt.§.
we may add the two moment matrices of the cube and octahedron together, (107), so
that the quadratic by interaction elements in the resultant matrix become zero,
-46making sure to
~arantee that l8p~~t~ = l6p~ !.'§. where
P1 is the radius of the
ootahedron and P is the radius of the oube. Thus" if the points of the oube are
2
on a sphere of radius \f3" the points of the ootahedron must lie on a sphere of
3
radius 2. / 4 • The resultant moment matrix is
1 I
_
1
I1
-···1-
1
1
I
l~iand A. =
______
7
1
1
L -1'-
3
I
-
I
-
14
7
+
I
J
2 (3 +2{2:)
-
-
-
• 11'
2. (3 +2.f2 )
l-
I
-I
_
T
7
2 (3+2'12)
I
3
• 0.60050
2 (3+2.{2)
This value of A. is very olose to the pathological value of A III k/ (k + 2) = 0.6,
thus although the quadratio effects are theoretically estimable, their variances
would be extremely large.
Proceeding as beforel a design with an approximately
III 5.6 (i.e.,
6 to the
l
nearest integer) points at the origin~ Similarly, a nearly orthogonal rotatable
uniform information distribution can be obtained by adding n
=
9.3 (i,e., 9 points) to the origin.
l
The oombining of the octahedron and the cube to form a seoond order rotatable
design is obtained using A. • 1, which gives n
design is of particular importance sinoe figures analagous to the ootahedron and
the oube are available in k dimensions.
In four dimensions the regular figures comprise that formed from tive points,
the analogue of the tetrahedron; the next in the series is that formed from 8 points"
the four dimensional analogue of the
e
oota~edron;
that formed from 16 points, the
four dimensional analogue of the oube; and a further figure formed from
24
points
mich" as we shall see later) oan be formed by combining the f1icube" and lIootahedronl/.
....
-47Other regular figures oocur but these have at least 120 points and are not of
value for the present purpose.
Proceeding as before ~ the moment matrix for the
generally rotated 8 point figure is
1
I
I
1
1
1 1 1
- -l - _I __ - - - _ ! -
\
4
----r---1 I
'
1
-_..I
I -
! X'X =
, 8 --
1
1
1
i
I
I
_~---
I 4~IA
4 ~'~
I
(111)
_..I::
- - 1'- - .l- I
'4 ~t~
The moment matrix for the 16 point figure is
1,
-T
I
11.11'j
_.L
!4
-_ _I_
_._._
I
I
--- - - 1 - - - - -
1.. X'X
1
1
1
1
eo
16 - -
-- - - - - - I
I
I
+. 11 1
2."'('1"
I
... -
2
I
--
1
')-16
-
-
-i·- -
-
2.§., 1:
(112)
2. 6 1 6
while that for the 24 point figure is
1
- - , -J;..
X''\;
24 - ;;
1
1
1
1
==
-
!
!
I
-\ I
"--
4-
1
1
1
I
-----
--,
2.
_l
I
3=4
+ - 1] i
3=-=
I
- f-
I
_. _. -
-
-
(113)
I
-4
_1-
3~
-48This latter configuration is seen to form a rotatable arrangement of second
order with A. .. 2/3 from which may be formed a rotatable design,
Prooeeding as before"
we find by adding points to the center a suitable value of A. may be obtained,
particular, we find to attain the value of
In
A '" 0.87, (Table 3) required for a rea-
sonably urdform distribution of inf'ormati. on wi. thin the design requires n1 III 7.32,
i.e,7 points. The number of points required for A III 1, necessary for orthogonality)
is n l '" 12.
The 8 point regular figure and the 16 point reguJa r figure are clearly not
rotatable arrangements of order two J but following the device recorded in the last
section" may be combined to form such an arrangement,
Proceeding as before we
find that the values of the radii of the two figures must be equal, and that the
arrangement is the same as that obtained from the
8.1.
24
point regular figure.
DESIGNS IN MORE THAN FOUR DIMENSIONS
In more than four dimensions only three regular figures exist, these are the
regular simplex involving k + 1 points, whioh 1s.,the analogue of the tetrahedron;
the cross polytope whioh is the analogue of the octahedron with 2k points; and the
k
measure polytope, the hypercube> with 2 points.
Configurations corresponding to
the icosahedron and the dodecahedron are not now available.
We may, however, always
form a rotatable arrangement by combining the cross-polytope and the measure
polytope~
It can further be shown that if the points of a k-dimensional measure polytope are
on a hyper-sphere of radius
'{k,
the points of the k-dimensional cross polytope
neoessary to form a rotatable seoond order design will be on hyper-ephere of radius
k/4
2.
•
The table below S1 ows the number of points required at the center of the
design to attain a rotatable arrangement fur values of A required to give approximately uniform informa.tion, and for orthogonality.
-494
24
42
6
76
added points at oenter for
approximately uniform
information
7
10
16
added points at oenter for
brthogonality
1.2.
17
24
k
Table
6:
8
2.7.3
7
4.3
48
35
Uniform Information and Orthogonal Designs for k ~ 4.
The number of points required in such designs" although far fewer than the
three level factorial arrangement, nevertheless rapidly becomes much greater than
the number of constants to be estimated. It is often possible
hcw~ver
to produce
k .
the sarne moment matrix up to order 2d as is given by the hypercube (the 2. design)
using a fraction of the points of the hypercube, Such arrangements are the £ractional two-level factorials which may be combined with the points of the crosspolytope to give second order arrangements, In particular, for k .. 5, 6 and 7" a
1/2 replicate of the ak design may be used" and for k .. 8~ a 1/4 replicate may be
used. These arrangements with appropriate values of A are given in Table 7.
Whenever a 1/2 replicate of the hypercube is used, i f all the points on the hypercube lie on a sphere of (radius}2 .. k" then all the points of the cross-polytope
.
2
neoessary for a second order rotatable design lie on a sphere of (radius) •
Whenever a
J/4
a
li;~
~
,
replicate of the hypercube is used then the (radius)2 of the required
orOSB polytope is .. 2.
k~2
-r , and in
general a (~)p repUca.teof a h;ypercube in k
~
dimensions requires a. cross polytope in k dimensions with a. (radius)2 .. 2. ~~.
These arrangements" with appropriate values of nl required for approximately
uniform information di stributi ons wi thin the de sign" and for orthogonality are
given in the table below
.'"
""O~
1/2 replicate
k
5
6
D
26
44
6
8
10
17
4
-2fi
3
2
n
l
n
l
added points at the center for
approximately uniform information
added points at the center for
orthogonality
(radius)2 of Cross Polytope
(radius)2 of HyPercube
Table
7:
:5
1/4 replica te
8
7
78 144
8
80
J2
20
4
22
33
20
8/7 {2
1
Designs Using Fractional Factorials for k ~ S.
9.
CONFOUNDING
Situations frequently occur in praotioe where it is desirable to perform
experiments in lI'blocks".
This situation arises for example where an insufficient
quantity of uniform material is available for use in all the experimental combina..
tiona, but smaller amounts of more uniform material are at hand.
If it oan be
assumed that the effect of a change from one batch of material to another is to
add a constant amount to the response at all levels of the variables, then by a
suitable arrangement of the design it is possible to fit the response surfaoe free
of the disturbing influence of the quantity of the raw material.
For m blocks" the
mathematical model may be written
(114)
where the 8i is the increase in the response in the i th block" the value 8 being
m
m
determined by the remaining (m .. 1) block constants so that ~ 8 • O. The
i-l i
ai(i = 1,2, ••• ,n-l) oonstitute an N x 1 column vector containing unit elements £or
those points in the i th block and zeros elsewhere.
If
@~t is possible
to obtain an arrangement such that the z IS are orthogonal
po~om1al
with the independent variables in the
equation then the effects of the
blocks may be eliminated wi thout reduoing the accuracy with which the constants ot
the surface are determined.
When this is not possible it will be our obj ect to
obtain arrangements in which the block vectors have smallest possible inner products
with the vectors of the independent variables.
blocking is possible is the
tl~~ee
A.'1. example in w}'lich orthogonal
dimensicnal design forMed from a dodecahedron,
As has been already pointed out, the dodecahedron contains five tetrahedra which
may be used as a basis for the arre.ngement.
For example" if we employ a dodecahedron
Hith 10 points ill the cente:J:' we m.ay carry out the experiment in
eac~
block containing
4 poi~ts
5 blocks of 6 po::.nts,
of a tetrahedron plus two points from the center.
The
fol1ows~
analysis of variance for the design would appear as
df
30
Total
4
:n5
D~e to Blocks
C::,r..s ~E!lts
LP.e:'.;
10
c::: ri t
--~~_.-
Erro~
For the icosahedron no orthogonal blocking arrangement has been found.
ever, for this desi&n
obtained by
wi~~
6 points in the
spl~.tt~ngthe ico5ahed~on
c~nter,
How-
one useful blocking scheme is
into its three component rectangles and using
these together with pairs of poin'bs in the center to form the blocks.
The linear
and interaction effec'ljs are orthogrmal to the block effects$ but the quadratic
effects are not orthogonal.
The variances of the quadratic effects when blocking is
us~d is O.074~2 compared with O.033cr2 without blocking. The efficiency of these
quadratic estimates is thus reduced by
56.3%.
10. ALIASES
If a function can be desoribed using a third order polynomial model, but only
a second order polynomial model has been fitted, then the estimates of the coefficients in the fitted second degree model will be biassed by the uneetimated third
order effects.
It is of real interest therefore to investigate second order rota-
table designs from the point of view of biasses contributed by third order effects.
Let !l be the N x
~
be an N x
~
~
matrix of the L]. original independent variables l and let
matrix of the
~
independent variables not admitted to the original
model but suspected of being required if the unknown response surface is to be
properly estimated.
Thus, in estimating an unk..'1own response surfaoe with a second
order polynomial model the Ll independent variables would be the mean, linear and
.....
second order (both quadratic and two factor interaction) effects. The ~ independent
variables would be the third order effects 0
It is originally assumed that the proper
model is
(115)
and the least squares estimates of the coefficients are then
(116)
If however the correct model is
(117)
then the estimates of the coefficients
~i
will be biassed since
(118)
".53thus
(119)
where
(120)
is called the "alias" matrix.
the original
~
Thus E(hi )
j-l
. th
th
coefficients and a ij is the element in the i
row and j
column
is
Thus only those coefficients in
~l
X2 are those with non-zero elements in
A.
of!.
biassed by the added independent variables
biassed by the added independent variables
~
ill!! Matrix
~
~i + ~ aij~j where hi is anyone of
~
Rotatable Design
Thus only those coefficients in
~
~l
are those with non-zero elements in
f:.o
!i!.1..
lie observe that the alias matrix can be written in the form
(121)
For any second order rotatable design it has been fh own that
N'-!i!1.7-1 is
the
.
In addJ.tion
N-1;'" Xt X _7 is simply a
1 2
matrix of moments} and since we are concerned with rotatable designs all moments up
inverse of the moment matrix of the design.
to order four in this matrix are invarient order rotation.
l
N- '-!1!2.7 is of the general form
Thus for k • 3 1
-54·
222
III
0
'.
1
_.~.
U
..
0
--_ .....- - ..
....
..
i
--..
3~
.. -
.~._~
133
- _·0_·.. .. , . . _.o
.
233
112
~_
.~
. . _.,.
113
.._ .__..__.._.. _.
223
123
,
.~
~
"' - -..-
-- -
•
-.-
3A
+
- .._._
-..•.
_.~.
.
A
.
.
._". - .. _. .
•
LIlll!7 /J.122:f.7 1J.133'JlL!1l2~7 {1l13t/ (11132) 1J.123~7 L!lll'Jl Li122~7 '/J.U2J7
j
I
22.
o._.
•
3A
2
3
-
122
333
Lill~
/J222gJ
L~233gILi222~
L'I.223t/ {J.122fJ /J223'Jl Li122J} L'J.222'iJ't.'I22237
i
i
(122)
•
33 Lil13~7 tJ.223~7 L'333~7IL'I22.3~7(J.333t1 LT.123J.7 ~333:V LT.133~7 L~2.33~7ILt233~
i
1.:1112) L~22~~ /)233:[/ iL!122~7 Li123~7 t.Il12~7 L'!223~7 Lil12~7 L!222J7 [!'122:iJ
13 LllIIJ7 L1222J1 [i333J.'! !Li122J7 Li133~7 Lil123) /J.233iJ LilJJ~7 {1223i/ i1.'J.123i!
23 ,il12a7 L'J.222~7 L'J.333~71L!222~ L!233t! ,i122~7 1.2233V LI123"J7 1.~223~7't.!223a7
12
Now" provided the design is such that, all S]b. order moments are zero, then
.
7 is aero except for the terms in the rows corresponding to
1- ,
everything
in N-1"!l~.
the linear effects, and these are the terms that are constant in every orientation:
For any two dimensional rotatable arr~ we have seen ~ 7 that all moments
of order p <n" where n is the number of equally spaced points on a circle" a re
I
/~
~
invarj@nt under rotation, and that for p ~n the moments are not in general invarj(jnt
to rotation. Thus provided there are at least six points on every oirole of nonzero radius, all the fifth order moments are zero.
For the icosahedron" dodecahedron
and the cube plus octahedron (in all dimensions) all the fifth order moments are
zero.
* I
•..---... -t···
0
--~-
o I
I
... - . ~ .-:k.
* I
0
.....-_-1--_ _
t-. 0 . . I/-..0
(123)
~~ i- ~-i-;--t-; ic
(The Bub-matrices indicated by
* can be derived using
equation (70) )
.Thus, in forming the alias matrix the elements in the rows corresponding to the
..
linear effects are multiplied only by
rotation,
~
11
so that for all these designs, under all
is of the form
o
o
o
.-j-_ ..
,
o
-------.··.-·
3X ·1.·
,
-_
o...._!
... _n·I_ ...
..._.. _......_.....
--'
t (bi ) .. ~i ... X(3~iii
'"
-
X---L._
I
..
0
- .... '. ..... -- -. -_ ..- ...
O!
Thus
..- '
(124)
~ 0.
-"r-! 0
0
I
k
-It
~ ~ ••• )
irj
(125)
J.JJ
If a fractional replicate of a cube is used with a cross polytope, the alias
will be like those above, but in addition aliases of the two factor interactions with
three factor interactions will also occur.
For example, if k ..
5 and we use the
comfounding arrangement I .. 1°2°3.4.5 to form a half replicate of the cube then
N-IL-Xix2J for the design in its ordina~ position is
111 222 333 444 555 122 133 144 155 112 233 244 255
0
o
o
I
,i.
1
f
i
2
'"
X
X
•
55
e
:
•
45
o
I
I
A-
o
.. - - ' ' ' ' - . '---' _.
i
I.
._n
o
Astl
I
_--0
---I'~ _._-_ _..--'-_. -.
I
0
I
-----t--"-- ..- .....
I
12.
13
345 245 125 123 125 etc
!
X
4
I
5 --..-.--.--------+-..-.3>" I
----
I
I
I
'j-" --_.
3
11
22.
t.
o
o
AI
• •
..56The matrix
!,
in this orientation is the same, except that the diagonal submatrix
of Als is replaced by a diagonal matrix of ones due to multiplYing by ~ from
NL-X~Xl.7-l.
To determine the generally rotated alias matrix we first pre and post multiply
~l
the oombined moment matrix of the
and
!e
independent variables by some orthogonal
transforming matrix, that is,
1
HI
-
N- l
H,~71 !
I
!
~
I
!l~l
;
-.-_..-.-. -----....-.. t-._._._where
1
I
-H,~7'
I
I ~l~
L~~ +~~
is some arbitrary k x k orthogonal matrix,
matrix and
-HL~7
the third Schlaflian matrix.
-H
(127)
HI.g71
...._ - - - - - .. _.j- --"";
! HI.~7
1-
~L~7 is
the second Schla£lian
To correspond to the definition of
.
derived power vectors and S,chlaflians, see
I
I
§a,
the model is written so·'bhat··the .
coefficient of the terms xix j is {[ and' for the third order interaotion terms
X~Xj
and xixjxk is
form, is
-{j.
The seoond order Schlarlian, written as before in
abbreviat~l
r.~ ~l
--.... ,--
!!L27
- = ~r': --'6 •
The third order Schlaflian can be similarly written in abbreviated form as
---"--1'""'.,._ .. -
!l !
1!
-; ..... _--
~ -i'-"
i .2
........, ...-
where the sub-matrices in
•
again after the
~(k
~L~7 are
~
:
2
partitioned after the kth row and column and
+ 1) (k + 2) row and column.
(128)
.
•<
-$7The generally rotated alias matrix, obtained from the expansion of equation
(127) is
-1 -2.
1
Rotated
!
H'
III
k 2- .2
!
"(,,.\'",
£'
(129)
-
-8
1
--
~'
:!
9
Remembering now that rotation cannot affect moments up to order four I the rows
!
corresponding to the mean and linear terms in
remain constant so that" upon
expanding (12.9)
111 222 33.3 444 55$ 122 1.33 144 155 112 233
o
0
...._
n O " • _ _ ..
. _ ...... _
•
__
1 .3A
o
j"I ...
;)..
345 245 125 12.3 125 •••
••••
o
A
a
o
A etc
3
4
5
~
._~
.•.•
'._.
•
__ ••
~
_ _ .._
•••••
• • •__ • __ • _ _ •
_.
_.'
...
~._
••••
••
_.
_4
'
__
__ ·_"'
·_
.--I·_._·~"
11
a2
o
33
44
55
•
. _ . _ .. ...
~
_.'~
o
o
__
._u·u
...... __..• _...•......• '-" .... - ....-,
'-'"
-._.-'
.........
12.
13
f
•
•
45
We note now that the interaction terms are biased by the unestimated tfudrd order
effects.
..58..
One criteria advanoed for determining the efficienoy
~f
any partioular
orientation of a design is to oonsider the sums of squares of the bias ooeffioients
for eaoh of the
(k =2) coeffioients in the model.
by the diagonal elements of
AAi.
Order, Biometrika, Vol. 39,
195~).
(Box~
These sums of squares are given
G. E. P., Multi£aotor Designs of First'
The sums of squares of the alias ooeffioients
for the interaotion effects in any orientation are given by
2' ,- 1 §. 2,.7 If
.§. '" 2'1 2 '" 2'2
§.,
2'
Thus, since 5 depends on the orientation, the magnitUdes of the biases and their
sums of squares also depend on the orientation of the design.