\,
.
BIAS AND VARIANCE CRITERIA FOR ESTIMATORS
AND DESIGNS FbR FITTmG POLYNOMIAL RESPONSES
by
M. J. Karson" R. J. Hader and A. R.
Mal~on
This reseutch wus supported by a SheU
Foundation FellowshipInstitute of Statistics
•
Mimeograph Series No. 510
Rulei,gh" N. C.
1967
•
't' ,"
iv
.e
TABLE OF CONTENTS
Page
".
LIST OF TABLES
•
•
•
•
0'
•
..
I)
...
• . .
·.. •
LIST OF ILLUSTRATIONS •
1.
INTRODUCTION
•
2 • REVIEW OF LITERATURE
...
It
(it
til
•
•
•
•
4.
vi
5
0
8
24
..
19
·.·.
19
21
24
26
DERIVATION OF DESIGN MOMENT CONDITIONS FOR ONE DD1ENSIONAL
CASE
.0
••••••
0
29
•••••••••••••
A Criterion for Choice of Design . . • • • . •
Design Moment Conditions for d=2; k=1,2,3,4
Design Moment Conditions for d=3, k=1,2,3
Design Moment Conditions for d=4, k=1,2 • . • •
•
..
5 . SOME DESIGNS
· ..
SUMMARY • • • • • • • •
8.
LIST OF REFERENCES
9.
APPENDICES
.~.DO
76
...
..
••
59
63
• . . 66
71
The p-Dimensional Case • . . . . . • • .
Two Examples of the Design Problem.
7.
51
58
6 • GENERALIZATION TO HIGHER DIMENSIONS •
6.1
6.2
29
34
42
·..
Designs for d=2
Designs for d=3, k=1,2 •
Designs for d=4, k=1,2 •
d=3, k=1,2,3 . •
•
5
·.-.
Set-up for the One-Dimensional Case
Bias Criterion • . • • • • . • •
Variance Criterion • • • • • • • • •
Some Properties of the Estimator
•
. •
·...
...
DERIVATION OF THE ESTIMATOR .
v
1
. • •
2.1 Response Surface Methodology .
2.2 Criteria Useful in Response Surface Fitting
. . • •
2·3 Minimum Bias Designs
3.
·.
O
99
· •• 104
•••
9.1 Some Algebra Leading to (4.82) • •
9.2 Some Algebra Leading- to (4.104)
9.3 Some Algebra Leading to (4.127)
76
79
·.•
•
0
•
•
•
•
•
•
• 106
• • • 106
• 107
108
v
.e
LIST OF TABLES
Page
.
4.1.
Summary of design moment requirements for linear, quadratic
and cubic estimators and protection against certain
higher order effects
0.......... 0. . . .
5.1.
Designs for d=3, k=1,2 with N/6 experiments at each of
.± '£1' .± .£2' .±.£3 . . •
0
5.2.
•
0
•
•
•
•
•
•
•
•
•
•
Designs for d=4, k=1,2 with N/8 experiments at each of
.±.£1' .± '£2' .± '£3''±'£4 . . . • • • •
0
•
•
•
•
•
•
•
•
•
6.1.
Designs for the configuration of design 2, and moments
6.2.
Designs and moments for design 5
•
III
•
•
•
•
•
•
•
•
57
67
•
72
•
91
•
•
•
•
96
vi
LIST OF ILLUSTRATIONS
Page
5·1.
5.2.
,
•
Values of £1 and .£2 with N/4 observations at .± .£1 and .± £2'
for d=2, k=1,2 (X: the least squares design)
. • • • • 64
4
Initial admissible region on [12 ] and [1 ] for d=4, k=1,2,
N/8 observations at .± .£1' '±'£2' .± £3' .± £4 (shaded area
shows final admissible region)
• • . . . . • • • • • • 70
6.1.
Design 1, experimental cOnf'iguration''£l = .698''£2 = .793.
6.2.
Typical design 2, experimental conf'iguration
•
•
9
•
•.•
0
89
•
90
.e
1.
INTRODUCTION
It is generally recognized that response surface analysis is a
..
statistical technique which uses the methodology of the design and
analysis of experiments, to detect and describe unknown functional
relationships between a dependent response variable and several independent controllable variables.
as a surface built by response
Commonly the relationship is expressed
conto~s
in the space of the controllable
factors, properly scaled to dimensionless variables.
The manner in
which the underlying response surface is expkored relies on the experimental levels of the controlled variables and the estimation procedure
for fitting the response surface.
l
These two aspects are obviously
intimately related.
At some stage of a response surface investigation it often is
necessary and convenient to assume a true underlying functional re1ationship,
exists between the response
~
and p independent variables
~i'
over some
large region of operability in the space of the_.,contro11ab1e factors,
where the C¥j represent unknown parameters.
Over some smaller region of
interest lying in the region of operability, N observations of the
response
~
are taken at a set of N corresponding levels of each
these levels constituting an experimental design.
A
used to fit an estimating function y to
••
~
~i;
The data are then
over the region of interest •
2
.e
This thesis is concerned with
(1) criteria for choosing the estimator and
(2) criteria for choosing an experimental design.
Clearly the application of criteria to these problems must be consistentwith the underlying problem.
These criteria are used to derive
estimating functions, experimental design moment requirements and some
experimental designs.
More specifically, suppose the original variables
.
~.
~
are standard-
ized in a natural way to transformed variables xi and denote by R the
scaled region of interest.
In most applications R is the p-dimensional
cube defined by -1 < x. < 1.
-
.
~-
Suppose that over the wider region of
operability the true functional response
~
is a polynomial of degree
A
d+k-l and the estimating function y, desired to fit
is a polynomial of degree .d-l.
~
over the region R,
This is the practical or experimehtal
way that the unknown form of the true model is taken into consideration.
To account for the model inadequacy in fitting and from a result discussed by BOX and Draper (1959, 1963), the primary criterion adopted
for choosing an estimator is minimizing the squared bias-, arising from
A
'
the failure of y to represent the true model exactly, integrated over
the region of interest.
Sufficient . conditions on the form of the
estimator are derived and subject to the estimator satisfying these
minimum bias requirements, estimators are found which satisfy the
A
secondary criterion of minimizing the variance of y integrated over R.
The bias criterion is
(1.2)
.
3
and the variance criterion is
•
where cl and c are constants.
2
Given that the estimator satisfying these criteria is to be used
for fitting, conditions on the design moments are determined which make
the experimental design satisfy the
folla~dng
protection criterion.
Protection is against specified higher order effects being present in
the true model.
These effects are guarded against by selecting experi-
mental designs which in turn yield estimators which are constant for
specified values of k.
This means, for example, that if the experiment-
er is fitting a linear estimator, the estimator depends on the degree
of the true model assumed, and protection against different values of k,
say against k::l and k:=:t2 for quadratic and cubic effects in the true
model, is obtained by choosing designs which make the estimators identical for these values of k.
It should be noted that the plan followed for finding estimators
and experimental designs is different in certain aspects from the more
or less traditional approaches to these problems.
In most cases
variance type criteria have been preferred for choosing estimators (for
example the least squares procedure assumes the fitted function is
identical to the true model).
However, when the underlying problem
supposes a certain inadequacy in the fitted response model, satisfying
..
variance criteria should not necessarily be the primary objective in
choosing estimators.
e
"
Both variance type criteria
(~.~.
rotatability)
and bias type criteria (~o~. min max I E[y(~») - ~(~)I) have been
4
applied to the design problemJ but essentially independently of the
motivations for the criteria used in :find:i,ng estimators
0
Section 2 constitutes a survey of the pertinent literature and
includes additional remarks in the above vein.
Section 3 derives the
estimator for the single controllable variable case and Section
rives design moment
conditio~s
4 de-
for fitting linear :functions and guard-
ing against quartics and less, :fitting quadratic functions and guarding
against quintics and less J and fitting cubic and protecting against
quintics and less.
Section 5 generates some experimental designs for
these situations,
Chapter 6 o:ffers an extension of the structure and
principles to higher dimensions J generalizing the estimator and giving
some design conditions and designs.
5
.e
2
0
REVIEW OF LITERATURE
This survey of pertinent literature :is divided into three parts .
The first reviews general response surface methodology and is mostly
concerned with the underlying motivation for many response surface
investigations" .!.!:. finding optimum operating conditions.
The second
part covers different criteria proposed both for fitting response
surfaces and
se~ecting
experimental designs.
These works have arisen
in the traditional regression analysis context as well as in the
response surface framework.
The last section specifically reviews
those articles concerned with the bias criterion given in (1.2) and
discusses the motivation for the present work.
2.1
Response Surface Methodology
Hotelli!l& (1941) considered the problem of' arranging an experiment
for determining the value of x which maximizes a function
whose coefficients t30 ' t31 J t32 " ••• are unknown •. An early exploratory
study is initially performed to provide sufficient preliminary approximate knOWledge of' the neighborhood of' the maximum.
Advance knowledge
of certain higher order parameters are also required and may be obtained
from small experiments in the neighborhood of the maximum.
When a fair-
ly good knowledge of the response function is available the function is
fitted with a quadratic equation
2
A
y=b
o
+bx+bx
1
2
by least squares and the estimate, xO' of the maximizing level is
6
-b /2b • The er!'ors arising from the inadequacy of the quadratic
1
2
approximation and from experimental error are examined and are used as
criteria to determi.ne an eXl'erimental design.
Letting B:.; be the co./
efficient of 13 and B be t.he coefficient of 13
3
4 in E(bl -l3l ),
4
.!.~.
E(bl -131 ) = B3133 + B4/3 +
~ and assuming the variance of b is fixed,
l
4
Rotelling shows how to allocate the experimental points to make B
v ••
3
vanish and to minimize B
4
0
Three distinct levels of x are required and
they must satisfy
Re also briefly consid.ered. the two variable problem.
It is important to note that this early work was concerned directly
wi th model inadequacy in experimental design.
Friedman and Savage (1947) also restrict themselves to problems
where the primary object is to find that combination of the controllable
factors at which the maximum occurs.
For this purpose they reject the
factorial design because 1) it devotes observations to exploring regions
that may be of no interest because they are far from the maximtml, 2) it
either explores a small region or explores a large region very superfi cially and 3) it does not t_ake advantage of the -fact that some of the
variables are continuouso
The authors suggest a sequential, one factor-
at-a-time scheme, as follows:
Choose in advance the best starting
combination of factors, using the best preliminary estimate of the
optimtml combination.
Somehow order the independent variables and per-
form a series of experiments varying the first factor holding the others
constant.
Estimate the level of the first factor yielding maximum
.e
7
response.
Holding this fact.or fixed at its maximizing value, find the
optimal level of the second factor in the same manner.
all variables, making a
fli
round.
Ii
of ,experiments
0
Continue for
The process is
re~
peated until convergence to a maximum is attained.
Box and Wilson
(1951) laid the foundation for present response
surface analysis using a strategy involving var,ying more than one
factor at a time in order to find optimum operating conditions.
entire k-dimensional factor space, there is a region,
R,
In the
called the
experimental region where experimentation is to be performed to find
that combination of factors yielding maximum response.
method is to begin with a full or fractional 2
k
The general
factorial design.
These designs provide estimates of first order effects and of certain
interaction effects.
If the first order effects are large compared to
second order interactions, then further
ex~eriments
are performed along
the path of steepest ascent determined from the first order effects.
When-first order effects are small it is inferred that a near-stationary
region has been reached and a full second order model is fitted using a
composite design, formed by adding points which allow for determination
of quadratic effects to the factorial.
Box and Wilson were also concerned with the bias due to fitting an
inadequate polynomial and introduced the alias matrix to reflect the
bias in the least squares estimates of the unknown parameters.
A further discussion of the general considerations in response
surface methodology is given
by Box (1954) who included in the
state~
ment of the problem the experimenter's concern with elucidating certain
aspects of the underlying functional relationship, and not simply
8
finding maximum or min:tmlUD. producing operating conditions.
This paper
represents a cursory review of the previous article J discussing factor
dependence.ll experimental designs, ridge systems (see Draper.ll 1963) and
further exposition of the method of steepest ascent.
It should also <be mentioned that Anderson (1953) reviews the first
three articles of this section.
202
Criteria Useful in Response Surface Fitting
Since regression analysis is an important part of response surface
methodology, criteria have been applied to regression problems for both
Prior to 1958 variance type
designs and estimation of parameterso
criteria were employed.ll while after 1958 bias type criteria were introducedo
This section is primarily concerned with criteria that may
b~
useful in exploring a response surface.
El.fving (1952) considered the general linear model with two independent variables Xl and x2J r possible combinations of the two factors,
a total of n observations with nP replicates of the i th experiment J
i
i = 1,2,.0.,r.
The problem is to find the optimum weights Pl,P2,o.o,Pr.
The sense of optimality depends on the estimation problemo
Let the two
unknown parameters be /31 and ~2 in the model Yi ,." xil /31 + xi2~2 + €i'
and let Yo? be the means of the npo responses. The errors are independ~
.J.
ent with mean zero and variance
e = ~~l
and
A
~~
~
+ ~~2.'1 by {j
= al~l
2
(J
0
The problem of est:lmating
+ a2~2.'1 where a l and a2 are known and gl
are the least squares estimates of
Since -~l and
g2
~l
and
~~
minimize the weighted squared sum
they depend on Pl,9P2,000,9Pr.
c
I;
is considered.
Pi (Yi -Xil/3l-Xi2~2)2,
Elf'ving introduces the criterion of choosing
.e
9
the Pi which minimize t.he variance of the estimateo
For estimating
e
he found that experiments were required at two and only two points.
For estimating 131 and 13 experiments at three pointsjl and in general
2
for estimating s l3 i sjl at most 8(8+1)/2 e:x:periments are required for
optimality
U..;
.LuS
0
approach
was~:v'.La
a purely geometric argument
0
Cherno:ff (1953) generalized. the "-'York of El.fv.iJ.~ and introduced the
criterion of
the
~mum
m1nimizip~
the vBxiance of the asymptotic distribution of
likelihood estimate of the parameterso
De 1s. Garza (1954) studied spacing of e:'iC,Periments for a single
factor when the relation 1.8 a polynomial of degree mo
Using the cri-
terion of minimizin€f'the ma..'timum variance of' least squ.ares estimators,
he showed that no more than m+l distinct values of the factor were required over a certain interval
u
Box and Hunter (195'7) int·roduced the important concept of' rotatable
designs
0
A design in which the variance of an estimated response at a
given point in the factor space is dependent only on the distance of
the point from the ori.gin of the design is called a rotatable design.
With a standardized set of' k factorsjlthe unknown relationship
requiring exploration is assumed to be represented by a polynomial of
degree d in the k factorso
A
The estimated response y is fitted by least
squares and the criterion of rotatability requires that
vax (y(;~)
=:
va!' (y(~», where ~ and ~ are two points in the :factor
space at equaJ. distanceE'. from the origin
0
By
using design moment
generating functions it is shown that the moments of a rotatable design
are the moments up to order 2d. of a spherical. distribution.
Let
10
.e
o
I't
(J:J
k
et.R]
In particular the moment requirements for rotatability are
(a) all odd order moments are zero (~o~o if any ao is odd)
J.
for all a i even,
where
depends on
A,a
d~
the degree of the approximating polynomial, and
generating function coef.ficientso
Hoel (1958), (1961) used the criterion of
ized variance of least
s~uares estima~es
minimizj~~
the general-
in the general linear. model
wi th uncorrelated erTors to choose experimental designs
0
He
also
applied this criterion to models with specific non-diagonal covariance
matrices
0
Folks (1958) considered the design problem for approximating true
polynomial responses with a lower degree polynomial, using least squares
estimates over an experimental region Ro
He considered eight different
optimality criteriao
Let
Y(2!"J
be the estima.ted response at the point 2;.
"" be the
Let 1?
vector of least squares estimates for the coefficj.ent,s, and let B(2;) be
r " ) to represent the true
the bias at ?£ due to the inadequacy of' E,y
model.
were
.Also let D denote the design ma.trixo
Then the eriteria studied
.e
11
1)
"" .
m.n 'max VaT' y(x)
D
2)
x
J
min
D
Var Y{2£) dx
R
3)
min
D
4)
min max B(~)
6)
D
x
min
J
D
7)
min
R
J
B2C~)~
MSE(y)~
DR
8)
min max MSE (Y) •
D
x
Criteria (1)=(3) are variance type criteria 3 (4)-(6) are bias type cr1ter1a and the last two involve both types.
Folks showed the invariance of optimum designs using these criteria
under simple linear transformatiofiS J enabling restricting attention to
cubical regions =1 :S xi ::; 1 and spherical regions of radius unity and
still generalizing the des:ign to any rectangular or ellipsoidaJ. regions.
He developed design
conditior~
for fitting linear approximations to
quadratic models in p variables for the variance and bias criteria over
cubical regions as well as for other specific situations and criteria.
12
.e
David and Arens (1959) used mean square error type criteria to
find an optimal spacing of a single controllable factor when fitting a
straight line to a true quadratic responseo
each of 2 locations
'1.
-
Using n/2 observations at
and x2Ji with corresponding PJ-ean responses Y1 and
Y2' the fitted line is
~',
= Co +
where ~O
A(-)
C1 \ ....
v =X
Ji
= Y, ~1 = (Y2~'Y1)/(x2-Xl)' The observed response is assumed
to be y(x)
variance 0-:2
= rex)
0
+
€J
where f(x) is quadratic and
€
has mean zero and
.As an extension of the non-random problem of fitting
polynomials by Legendre and Tchebysheff spacing,lJ the authors introduce
the criteria
1)
minimum expected squared error integrated over [-l J l],
2)
minimUJ.'1l maximum expected squared error
The first leads to the symmetric spacing
4 2
of r (3r -1)
= (20-2 /9nc22 )Ji
~
0
= -x
2
where c is the coefficient of the Legendre
2
polynomial P2(x) in the expression fO(x)
:0::
Co + cl P1 (x) + c2P2 (x) ° The
second criterion leads to the symmetric spacing Xl
X
2
= min
rH 1
~
with x being a root
2
2
1
= -x2
with
1
+ (1 + 320-£::: /9nc'2, )'2J2 j> 1 Jolt is also showYi that when
fitting a least squares line there is no improvement by spacing at more
than 2 distinct levelso
Kiefer (1959) offered various criteria for optimum experimental
designs under the setting of inference about s given linearly independent parametric functions
~.
J
=L
c ..e.,lJ in the parameters eio
~J
~
Let V
be the covariance matrix of the best linear unbiased estimates of the
for a given design.
~.
J
The problem is considered to be that of testing the
·e
13
hypothesis that the if. are all zero, under the normality assumption,
J
against alternatives for wb:ich L: if~ ; : :
J
eel-
Let
0
¢.
represent a test and
let ~$ (c) be the infirmm of' the corresponding power functiono
Different
kinds of optimum designs and their relationships are discussed depending
on the following criteria
1)
0
Designs are
M~ optimum if sup ~ IP ( c) is a ma,ximum,
$
2)
L~optimum
3)
D-optimum if"
4)
E-optimum if the maxilrrum characteristic root of V is a
if they are locally M-optimum,
i vi
is a minimum,
minimum,
A-optimum if trace (V) is a minimum,
G-optimum (for regression problems) if the supremum of the
variance of the difference between the estimated regression
function, assumed correct, and the true model, when fitted by
least squares, is a minimum,
This work was continued by Kiefer (1961) and applied to rotatable
designs and systematic designs (which arise when a time variable
affects the experimental procedure)
0
Kiefer and Wolfowitz (1959) consider optimum designs in regression
analysis for the regression functions ~(x) ""
k
L:
aof,(x)o They allow
i=l ~ ~
for three inference problems in choosing designs, These are
(1) estimation of a
k
assuming the other a
i
are known, (2) estimation of
s < k of the a, and (3) estimation of the whole regression functiono
~
For the first situation designs are sought which minimize the variance
.e
of the best linear unbiased estimator of a ,
k
These designs depend on
·e
14
the best Tchebyche:U approximation to i'k{x)
0
For the second inference
situation justification is given for using as a design criterion Doptimality as previously defined 3
!o~o
minimizing the generalized
variance of the best linear unbiased estimates ot the s aio
The cri-
terion used for the third situation is to choose designs which minimize
the maximum expected squared difference between TJ(x) and the best
linear unbiased estimator of TJ(x).
For polynomial regression on the
interval -1 ~ x ~ 1 3 the best design places 11k of the observations at
-1,+1 and at the zeros of the derivative of the Legendre polynomials.
They also show that if a design putting mass 11k on each of k points is
optimum for set-up (3) then the design 1s D-optimuID. for s=ko
The final criterion mentioned in this section was
•
Hoel and Levine (1964)
discusse~
by
They employed the criterion of minimizing the
0
variance of the predicted value of a polynomial response at a specified
point beyond the interval of observations, defined by -1 $ :x:
~
10
Minimum Bias DesignS
20'
Box and Draper (1959) motivated and set the framework for the
present study
0
It is requi!-ed to fj.t a response function
polynomial y(!) of degree
Ro
T)
= g(x)
by a
d:J.
over some standardized region of interest
In R it is assumed that T) can be expressed exactly as a polynomial
;,
of degree
~
> d13 but that the lower degree polynomial y will be ade-
quate for practical purposes.
They assumed that Y(;!f) is fitted by
least squares and recognized both variance error (due to sampling error)
and bias error (due to the inadequacy of' the polynomial
..e
fit
T)
0
As
y to
exactly
a result of their primary objective of requiring the design
.e
15
'" to represent
to allow Y
~
as well as possible within R, they considered
the problem of choosing designs which minimized J, the expected mean
square error averaged over R.
where
.Q -1
:=
J ~ and (J'2
Th'~,
is the experimental error variance.
This can
R
be partitioned into two meaningful components, the integrated variance
and the integrated bias yielding
J
:=
V + B,
where
They showed, even though designs minimizing J depended on certain unknown parameters, that the bias contribution generally far outweighed
the variance contribution.
the
II
all-bias
If
Hence it is useful to consider designs for
case.
Designs which minimize B can be found by equating the moments of the
design up to order
~.+~
distribution over R.
to the corresponding moments of a uniform
Thus, if Y(2f) ::: ~!?l and Tj(2£) ::: ~.@l + ~~ where
£1 is the vector of least squares estimates, and if Xl is the design
matrix and X is the matrix representing the bias type terms, then the
2
necessary and sufficient conditions for a design to minimize Bare
16
!J.
2 = n fR~l~~'
Particular examples of designs are given for fitting
linear functionB to true quadratic surfaces within a spherical region,
In a second paper Box and Draper (1963) extend their earlier work
by applying their development to fitting quadratic polynomials when the
true model is cubico
They considered second order rotatable designs
and found, for example, that with wo factors and R being tl).e unit
sphere, that the design must satisfy.E xi =.E x~ ;::: N(,2652)
/I.
:=
3/1.4/c
2
4
4
=::
Nc and
.;>'.?
and /..4 :; .E x l /3 N "" .E X2/3N = .E X1X2/N
0
Lawrence (1964), applied the Box and Draper all-bias results to
different kinds of regions
He let R be a triangle in two dimensions,
0
a tetrahedron in three dimensions and a k-dimensional parallelepiped,
A further contribution was added to the Box and Draper all-bias
development by Manson (1966)
y = .E b .e
jz
J
0
He considered the problem of fitting
to a true exponential response Tl
:=
ex + 13e/z
0
'When trans-
formed to the problem of fitting .E b.xJ to ex + 13xl the Box and Draper
J
framework is apparent
0
Manson applied some aspects of' numerical
analysis pertinent to polynomial equations to actually deriving minimum
B designso
At this point it is necessary to consider an experimental outlook
which ties together the design problem and the estimation problemo
Clearly without an estimation goal (or more generally, an inference
goal) there is no meaningful design criterion
0
This viewpoint is re-
fleeted in Kiefer (1959) and more explicitly in E1:fving (1952) where it
is recognized that the sense of design optimality depends on the
17
estimation problem which the design is to help solveo
It follows that
it is also necessary to consider the estimation criteria along with the
design criteriao
problem
0
These criteria must relate to the entire underlying
Optimization with respect to both design and estimation
jointly can be achieved by first finding the optimum estimation procedure for any given design and then choosing the design(s) which, toII
gether with the optimum estimation procedure, gives the best
for some appropriate over-all criterion
H
results
It is, of course, necessary
0
that criteria of optimality u$ed in each of the two stages be consistento
From the foregoing more general viewpoint the Box and Draper work
•
leading to designs which minimize bias alone (or those which minimize
some combination of variance and 'bj,as) is r~ally an optimization in a
somewhat restricted sense, the restriction being that only least squares
estimation is allowed
0
Traditionally the justification of least squares
estimation is that minimum variance unbiased estimators are obtained if
the model fitted is correcto
In the Box and Draper workJl however» it
is assumed that the model fitted may well not be correct
0
The least
squares estimates are then biased, and it is not at all clear that the
minimum variance property is at all relevant
0
The Box-Draper results indicate rather conclusively that the bias
contribution to mean square error exerts much more influence on the
choice of design than does the variance contribution
0
Since least
squares estimation essentially optimizes with respect to variance» it
appears that its use may be an unnecessary restrictiono
which follows this restriction is dropped
0
In the work
An estimation method is
18
used which, for a fixed deBigu y minimizes the integrated bias due to
specified terms not inc111ded in the
fitte~.equation,
and subject to
this, also minimizes the integrated variance of the estimator.
It is
found that using this method of estimation the same miIUmum bias is
achieved for any design (subject to easily satisfied conditions).
For
designs satisfying certain special conditions the estimation method
reduces to the familiar least squares method but it is emphasized that
j
this is only a special case of the more general method.
Since the primary criterion of mintmizing bias and a secondary
criterion of minimizing variance subject to the minimum bias are
satisfied essentially by choice of the estimator alone) an additional
criterion is introduced to select among designs.
19
30
DERIVATION OF THE ESTINATOR
In this chapter the bias and variance criteria are developed and
applied to deriving the estimating polynomial.
Suf£icient conditions
are first found which minimize the bias criterion.
This is followed by
determining the exact representation of the estimator arising £rom
minimizing the variance criterion subject to these conditlonso
3.1
Set-yP for the One-Dimensional Case
Over an entire region of operability, the true model
~
at the
point x is assumed to be a polynomial of degree d+k-l given by
(3.1)
e
where
(3.2 )
~l
(3.3 )
~
(3.4)
~~
(3.5)
~
I
I
= (l,x,x2 , .• o,xd-l')
;:::
::;
(xd,xd+l,o .• ,xd+(k-l))
(~O'~1'~2""'~d-l)
;::: (~d'~d+l""'~d+k-l)
.
It is clear that ~1 and ~l are (dxl) vectors and that ~2 and ~ are
(kxl) vectors J and that ~l and ~ represent the unknOlvu parameters in
the assumed true model,
Over some region o£ interest R, scaled to the closed interval
-1
~
x
~
1, it is desired to fit
the observations on
~
~
by taking N observations on
are represented by y ::;
~
~,
where
+ error in the usual
manner, and using the approximating£unction, called the estimator
20
where the (diL) vector b 2s to De determined
0
c.~,~o,
Thus;. (d.-lj gives the degree of the fitted equation
d=2
implies fitting cwith a fi.rst order or linear equation and (d+k-l) gives
'I
the degree of the true model; so that k represents the "number of
degrees
I[
omitted in fitting.
The observational Btructu:re is
~i
:=
the N observations on
~
where
(3,8)
with the usual assumptions on.§.j i. e, E,.€
necessarily assuming normality.
:=
2,
Var
H
,;.;
2
x;2
1
:2
N xN
1
X
and
x-
d
.1.
x
e
(3.10)
X
2
d
2
XI
x
d+l
d+1
2
x1
x:2
d+k-l
d+k-l
::
....
d+l
xNd xN
=I
The "X matrices are
xi
~
1.
§.
x
d+k-l
N
2
(j
J
but not
21
Restricting attention to those ~iS wrdch are linear functions of
the observations, set
-b
:=
Ttv
a/.,'
where the (NXd) matrix T is to be determinedo
The problem is now to
determine T so that the desired optimality criteria are satisfied 0
choice of T defines the estimator, which can now be written as
Thus~
Obviously, if the familiar least squares estimator is required then
Tt
(~X1)-lXi, where Xl is as in (309)0
:=
302 Bias Criterion
At the point x, the "bias contribution is defined to be
Bias Cx)
:=
E[~(x)] - ~(x) ,
where
(3·14)
Similarly
In terms of the model (3.1), and the fitted equation (3.6) and using
(3.11) and
(3.14),
(3·16)
Hence, (3.15) becomes
22
.e
-~
(3.18 )
[
3(·T 1
.~\
X1~1
A
+ T,tX 8
2~ -
A)'
~l
-
'8_]2 .
~2~
Define the (dxl) vector ~ as
(3·19)
Then (3.18) becomes, using (3.19), and remembering that ~ is a function
of T,
The bias criterion for choice of T is (3.21) Blaveraged'over the
region of interest, and therefore, it is required to choose T, independently of §.1 and
(3·22)
~
to minimize
B = B(T) = (~)
1
J
[E[y(x)] - T)(x)} 2 dx
J
-1
where the integrand is given by (3.21) and
t
= (
J1 dx) -1
-1
Thus, from (3.22) and (3.21)
(3.23 )
(3.24)
where
.
·e
23
(3.25)
K
::::
(1.
')
,2,
(3·26)
W
2
::::
(1'
.'2)
W
3
::::
J.
1
J-1 ~,~ ~dx
f
1
-1
'1)
\'2
-~xidx
....2
1
[1 ~dx
It is understood that where the integrand is a matrix in
and
(3 ~25), (3.26)
(3.27), the integration is performed element by element.
The three
matrices Wl , W , W are constants independent of T.
2
3
The (dxd) matrix Wl is symmetric, positive definite (Manson,
1966)
and therefore, B can be written
•
Since T appears in B only
through~,
B :::: B( T) will be a minimum with
respect to T when
g :::: W~~2~
Using
•
(3.19), the definition of g, (3.29) becomes
Hence, sufficient conditions on T for satisfying
~l
e
..
and
~2.:1
and hence for minimizing B are
(3.30), independent of
Conditions (3,31) and (3.32)~ wtdch can be 'written as
T'X
~
A,
where
().34)
x "'" (XI
A "'" (Id
are the mininrum B (min B) cond1"tioM
The conditions
0
(,.33) on the elements of the T matrix mean that if
a T matrix satisfies (3.33) for a given X matrix then using the estimator
y( x)
,; ~~ T':l. will give min B, 't-rhen
T}
is t.he true model,
For example,
if least squares is reqv.ired then T' is automatically fi,xed at
(Xi.lS. rl~
= Wilw2
and then the conditions (3 .32) which yield (X~Xl rlx:i.X
2
are forced ~ the desigg.
Furthermore, the value of minimum B is, for
a~
design,
().36)
).3
Variance Criterion
Equation (:3.33) represents a set of suf'ficient conditiona on the
elements of T which generate a. min B estimator.
Subject to these condi-
tions the secondary criterion, a variance criterion, is introduced to
specify T, still independently of the choice of design.
In this manner
min B, and subject to min B, minimization of V is obtained for any
experimental design.
e
•
.e
;:: -,.L
~T'l
.'
'rhUS:J
rtF,...
minimum
'.,rr: Band
--
,,~.:_.
-,
-...--.~~
'With res;pect to T, and sUbJect;to (.3 .3,3).
;It
is nowsuggestive to wrii:;!;; the variance
yar
~
I
" .
,
y. ';JeT)
\. ~.
'i
(3,1nce E(~) "" ~, var
:£ ;::
I
;:: ~f!1 :
="
I2.2
Q;fl as
2
.
x I T IfJ'<,.
(J
......1-'-~
).
•
y\ x; T)
will b e minimi~~,dwi t}l r.eSllect
to
T
tf
Ti'tl. is the :m1nimum vari9J1<:e unbiased estipate of ~J in a m9,~el
w;lp,;q!h includes all the parameters
0
This.; of course j follows from the
Gauss"Markov theorem and y'lelds theref'ore
26
.e
Hence the T matrix which mir~mizes (3040)J fOT any x, is given by
I f this minimizing T is ienoted by TO' then clearly
for all x and To
(3.li8)
Therefore,
for any T ,
and hence (3047) gives the T matrix which produces the (min V
estimator.
Thus, the (min V
h()
x
Y
Note that since
I'
:=
I
.
~.A~X X)
-1
I
I
min B)
min B) estimator y(x) is given by
I
X 'Z •
(3.50) requires that
(XIX) be non-singular, a require-
ment is
N> d + k - 1
3.4
:=
~
degree of true model.
Properties of the Estimator
By using the es't·imator given by (3.50) minimum B and subject to
minimum B, minimum V is realized for any experimental design.
It is
important to note that the y(x) so obtained depends on the degree of
the assumed true mod.el as reflected in the A and X matrices.
applying (3.50)
ev~bles
Thus
an experimenter to fit an estimator satisfying
certain cri teria of optim91.i ty, while at the same time
against possible higher order effects.
II
guarding
II
Section 4 will give an addi-
tional kind of protection in terms of experimental design conditions
and for the present certain properties of the estimator are stated.
27
E(~)
=:
E(TiZ)
::::; T' E(Z)
=:
T' (Xl~ + X~)
=:
T'X11::::1
A + T'X A
2J:::2
=:
~l
+
W~lw2~
•
Similarly,
=:
T'T CT2
(3.55 )
In terms of the estimator, (3.52) and (3.54) become
E[y(x)] :: E(~~)?)
(3.57)
=:
Var
~i(~l
+
W~~#2) ,
y(x) :: Var (x'b)
-1-
(3.59)
It is of interest, both from computational and certain design considerations, to develop a partitioned representation of the T matrix,
and hence of the estimatoro
(3.60 )
28
.e
Let
s "'"
XiX
Then (3.61) becomes
-1
Assuming S
exists~
it can be written as
-1
_
S
-1
-1
,-1
(Sl '-Sl S2 t 8i.? Sl S2=83
IslS~lS
\ 2 1
2
-s
3
)-1
I
-1
S2 S1
)~lS'S-1
:2 1
Then T' becomes
+ (W- L - ~-lS )(Sls-l~ S )'-l S 1 S-1] x!
T' -- [S-l
1
1 w 2 -o1 :2
2 1 :)2= 3
2 1
--:1
From (3.65)~ the partitioned representation of the estimator is
29
4.
DERIVATION OF DESIGN MOMENT CONDITIONS
FOR ONE DIMENSIONAL CASE
For a given degree of an e?timating polynomial, represented by
(d-l), and a given assumed degree of the true model, represented by
(d+k-l), the estimator y(x) given by (3.50), or equivalently the matrix
T' given by (3.47) guarantee the attainment of' minimum B, and minimum V
for this minimum B, for any experimental design.
!.~.
Additional criteria,
criteria other than the bias and variance type criteria already
satisfied by the estimation procedure, can now be introduced which may
be satisfied by imposing certain conditions on the moments of the experimental design.
For example, by requiring the condition that all of the
odd moments vanish, symmetric designs are obtained.
Similarly, from the
partitioned representation of the T matrix as in (3.65), it is clear
that if in addition to a (min V
I
min B) estimator a least squares esti~
mator was a requirement, then it would be sufficient, if (X'X)
to have the design moments satisfy W~~2
-1
exists,
= S~lS2; because in this- case
T' would reduce to s~lx~ and y(x) would then be the least squares esti'
)-1XJ.;x:.
I
mator, ~lI (Xi
~
It should be mentioned that in all that follows in this chapter,
and the next, the estimator used is the (min V
I
min B) estimator.
4.1 A Criterion for Choice of Desi@
The criterion introduced to select experimental designs is related
to offering protection against different values of k.
the (min V
I
The derivation of
min B) estimator requires assUll1l?tions about the degree of
the true model.
For a given d, different estimators are obtained for
.e
30
different values of k.
It is felt that a criterion consistent with and
pertinent to the development of the (min V
I
min B) estimator is one
that will generate the same estimator for various values of k protected
against.
Thus, protection against different values of k means that a
particular estimator can be used, by choice of experimental design,
which will be (min V
I
min B) for these different values of k.
example, if d=3 and k=l,
!.~D'
For
if a quadratic estimator is used and the
true model is assumed to be cubic, then applying (3.47) yields a certain
T matrix, say T1D
I f d=3 and k=2 is assumed, (3.47) yields another
T
matrix, say T . Protection against k=l and k=2 simultaneously means
2
choosing experimental designs which force T =T , and hence make the
1 2
....
"
corresponding estimators Yl and Y2 identical. Alternatively, in the
response surface setting protection can be defined by supposing it is
desired to approximate a particular response surface over a specified
A
region of interest with a polynomial of degree d-l, denoted by y.
a
y given by (3.50)
Then
can be determined, by choice of design, which will
minimize B and subject to this will also minimize V for different "true
1)I S "
of higher degree than d-l.
It is important to note the relation of the above concept of
protection to the requirement of least squares estimators, since the
latter will be referred to and maintained as a familiar reference frame.
As previously discussed, the design conditions leading to the least
-1
1
-L_
squares estimator are X X non-singular and 81 8 = Wl w 2 where W2 and
2
8 depend on k, the number of terms assumed in the model but not in2
eluded in the estimator y. Consider the case k=r and partition the W
2
A
matrix, defined in (3.26), as
.e
31
(4.1)
where }fl'~' • ",:!fr are yhe (d.xl) columns qf W2 • Partition the (Nxr)
X2 matrix of (3.10) as
Therefore, when k=r,
t~e ~mposition
ot the least squares estimator as
~
requirement leads to the following ponditions on the design moments,
-1
.. '
S1 S2 = T.7
;'::.. ~"T;1 2 .
(4.4)
(4.1) and (4.3), these conditiqps are
From
which means
(4.6 )
This means that a design
y = 2£~(X~Xlrlx~Z'
for k =
1~2,3,
1e~dfpg
to the least squares estimator,
as a (min V
I
••• ,r ,
min B) estimator for a true model with
X2 = (~l'~"" '~r) will automat:f,.cal1y also be a least squares for any
true model of lower degree. Since the e~timator would be the same
l
; = 2£~ (X~Xl r ~Z we would automatically have 1'1=T2 =T =... =Tr . That
3
is, any design satisfying (4.5) will not only nave a least squares estimator but will be a (min V
I
min B) design for which T1 =T2 =T =·· .=Tr ,
3
However, recognizing and appreciatipg the result of the previous
paragraph, it also shall be demonstrated that to realize (min V
I
min B),
the "least squares" requirement may be more restrictive than ne.cessary.
32
It is possible to satietY
squares
If
designs.
Tl~T2~~f'~~r w~thout
If
necessarily using least
In somEl eaSElS this additiqnal flexiblli ty may be use-
ful to an experimenter.
The remainder of this cnapter derives the protection conditions on
the design moments for fitting linear, quadratic and cubic estimators,
in the one dimension
and
cas~,
prote~ting ag~inst
up to and includipg quintic effeqtp
~o~
One requirement init:telly iml)osed
If
in a particular d and k
II
sit~tion
be
eaQh
;1.a
higher order models
~stimator.
that
ze~O~
/3.1],.
odd moments arising
This requirement of
symmetric designs is made mostly as a matf;er of making much of the
algebra which
follow~
mOre trl;l.ctable,
that symmetric designs are :frnttj.1tively
It is convenient at f;h1s
tim~
It is standard to refer to the
The (Nxd+k)
~
and~.
X matrix
4 :;: The Nx4
t' matr!:x:
~
:;: The Nx5
:(C
to most experimenters.
to introQ,uce some additional notation.
(~.;34) as
X ;;:
t;l.S
the design matrix.
(~IX2)' is frequently
Let
:;: The N:l0
X
it should also be noted
a,pp~~ing
(Wxd) Xl matrix
X matrix, defined iij
used for d+k :;: 3,4,5
HQW~ver,
matrix
X6 :;: The Nx6 X matrix,
That is, for example, X4 is a matrix wit;n the l'f columns (1
i :;: 1,2, .•• ,N. Also let
xi
x~ ~),
33
(4.8)
M6 = X X .
66
It is also standand to denqte the jth design mpment by
N
I: x j
[l J]
(4.9)
= u=lNt u;
j
Note that symmetric designe ~av~ f1 ] = 0 a.nd j odd.
the values of k for
th~
given d
~d~~ st~dy.
Also let i index
Then T and Ai will denote
i
the correspondingT and A mat:r1cel3'T
Of interest and
singularity of
(a)
~
and M6 •
'nles~
cQnditiona are
[1 4] ~ [12]~ ;
M non-singular it
4
(4.11)
(c)
M4,
~ non-singular if
(4.10)
(b)
~,
conditions for non-
UBe~ ~n t~e seq4~1 ar~
[1 4] ~ [12]2,
~12)[16) ~ [14~2
J
~ non-singular is
(4.12)
[12][16] ~ (14)~~
(4.13)
[18][14] + 2[~4)[19)~12~ ~ [12)2(+8) - [14]3 - [16]2 ~ 0;
(d)
M6 non-singular if
(4.14)
[18][1 4] + 2[14][16~(12) ~ [12]2(18] - [1 4]3 - [1 6]2 ~ 0;
(4.15)
[12][16][110 ) + 2[14)[16l(+8}
Of
{12](18)2 _ [1 4]2[110]_[16)3
~
a .
Design Moment
CondiJions
tor i d=2; k=1,2,3,4
\'
;.
"
4.2
When d=2, a linear estimator ot the torm y'" = b O + b x is being
1
:fitted, where b o and b1 are deterndn~<t trom (3.11) and (3.47) depending
upon the assumed vaJ,ue ot k.
Protection against quadratic (k=l) and cubic (k=2) effects means
choosing designs which yield T =T ,
l 2
(i) For k=l, trom (3.25), (3.26) and (3.27)
(4.16 )
W1 =
[~ 1~3 J •
=[
1/3, ]
W2
.
e
(4.18)
0
'
,
W = 1/5
3
and
(4.19 )
T' = ~M31x·
1
3 '
where
(4.20 )
0
1
~ = [~
1/3 ]
0
,
(4.21)
(4.22)
e
.
=
[~l
,
]
~l
x;=
1
1
xl
x
~
2
• •
2
2
x
2
•
·..
•
.
1
xN
X
2
N
35
.e
0
~~2)
~121
0
1
(4.23)
1'3
=N
~14)
sYlJlIlletr1~)
and
[12)[1 4)
(4.24)
-1
l')
=
[J,.4]_[12]2
~
N[ 12] <£':1-- ). [ 1~) g j
tsymme"j;~1c)
It is convenient to write (4.19)
T'
1
IiLS
=
where from (4.18)
(4.26 )
'
t' -=01
- ~1
t'
-11
= a'
.::;02
'
l')~l,
X3 - ~~1 rs t row 0 f T
1
"'T
:r(lX' = s~cond row
3
,
(11) For k=2,
(4.28 )
where
o
1/3
1
o
0
pf
T'
1
_[12]2
0
[12]
36
(4.30 )
(4.31)
a'
(4.32 )
= [~2
~2
e
(4.33 )
(4.34)
with
and
x'4 --
J
,
1
1
··
xl
x2
·. .
2
~
2
x2
•
xi
~
•
= [~]
z
'
-3
1
~
x2N
·.
·• ~
,
37
~e
1
ClgJ
4 =N
(4.36)
[12~
0
f14]
0
M
0
(1 4]
0
[1 6]
(symmet;rio)
where
E:i
Q) •
= N(O
Let
and note that c is well defined if
1
c1 = -
I~ I / IM41
Then, from
(4.34)
-1 =
M4
~
~nd
since
(4.35{
I
"1)
137"~Cl~~
M3l~ol
,
°l~M31
where (4.11 ) gives the conditions
combining
r ~12J[16],
•
'1"1 (
(4.40)
~~4J2
(4.28), (4.31),
(4.~4)
"Pl
to~
and
-1
the existence of M4
(4.40),
. Therefore,
-
38
I
0
~1
(4.41)
T'
2
=
,
M3;t( I , .,u
~M3;L)
,.,.1 c1-.
Xl
-c
~3
cl~~;l
3/5
~11
-1
~ J:!1 c1
1
3
I
From (4.41)
= first
row of T~ ,
= second row ot' 'r~ ,
•
This leads to
Result I:
A sufficient cOJ1d1tion for
'l'i = 'l'~
is
(4.44)
Proof:
The two matrices T:lr and Tg ar~ eq~ if and only if ~1
and !i1 = !i2'
First it 1$ ShOIDl that ~1
l'l
~2'
= ~2
From (4.26) and
(4.39)
t' = ~1::>
'. l£lx!.
-=01
._~
From (4.24) and (4.38), (M31~), :1,8 a v~otq;r of th~ form (0, element, 0),
and from (4.17) and (4.16), !hl ;;: (1
therefore ~1
..
e
= ~2'
0
1/3).
Hence,
~1M31E::I. = 0 and
The reIj1aindel" of the proOf is to show that
4
!i1 = !i2 if [1 ] = (3/5) (12L
~fll
(4,2'7) and (4.43)
39
I
I
-1
~11 = ~11~
I
X3
and
From (4.20), (4.21), (4.24) and (4.38),
I
-1
~11~ ~
=
(
1
0
2
= [1 4] /(1 ] .
4
2
Therefore, if [1 ] = (3/5)[1 J, then
~il ~ !i2'
which establishes
result 1.
Combining (4.44) with
~he
protection against k=1,2, when
conditionl:j f9r the non-singular matrices,
d~
is obtained when
4
[1 ] = (3/5 )[12 ]
[1 2 ]"
r 3/5,
[1
It is of interest to
for fitting linear
6] f (9/25)[+2] ,
pot~
estimato~s
that the least squares design conditions
to asaumed cubic models are, from
with d=2, r=2 and a synnnetriq design, [12 ] :;: 1/3 and [1 4]
(4.4)
= 1/5; a
special case of result 1.
( iii) For k=3, in ordeJ;' now to equa.te T " T and T3' along with
1 a
(4.45) additional conditions are peeded.
As before
(4.46)
where
e
.
-
40
(4.47)
~
_1
=[ 0
I
~2
0
1/3
0
1
0
3/5
0
1/5
= [~
(4.48)
1/5 ]
0
-2
]
at
=[ ~3 J,
(4.49)
~3
Xl
(4.50 )
X;=[~J ,
with
e
( 4.51)
I
~4
=(4
~,
4) '
x24, ... , xN
and
1
(4.52 )
~
0
[+2]
(12 ]
0
[1 4]
=N
0
(1 4]
0
[1 6]
(syIIlIlletr1q)
=
.e
where
Ee
N[lf?]] ,
[1 4]
0
[1 6 ]
0
[18 ]
41
o
(4.54)
In a manner similar to
th~
proof of
o )
r~sult
1 it is seen that the addi-
tional condition for equality of T , T and T is
l
2
3
However, (4.55) violates (4.45). Hence, the least squares solution,
4
[12 ] = 1/3, [1 ] = 1/5 may be Uf3 ed with the additional restraint that
6
8
[1 ] f 3/25, so that M is non-singular, aqq ~so (4/45)[1 ] +
4
6
[1 ]((2/15) - [16]~ f (1/5)3, so that ~ is non-singular. These additional restraints are necessary becaus~ the (win V
I
min B) estimator
requires that (X'X)-l exist and to show that designs for which
-1
SI S2
= WI-L_
-W lead to the least squares estimator it is also necessary
2
to have a non-singular (Xl X) matrix, for the full model.
Thus, in summary, protection is obtained for quadratic, cubic and
quartic effects if
[12 ]
= 1/3
4
= 1/5
6
f
[1 ]
[1 ]
3/25
8
6
6
(4/45)[1 ] + ((2/15) - (1 ])[1 ]
f
1/125 •
(iv) The k=4 case is included for completeness, because if T =T =T
l 2 3
can only be satisfied by the least squares conditions, then obviously it
is not possible to satisfy T =T =T =T for other designs.
l 2 3 4
••
The conditions
are
2
[1 ]
= 1/3
4
[1 ] = 1/5
6
[1 ] = 1/7
8
[1 ]
r 1~~; (so that ~1 ~xists)
4.3
Design
Mo~ent
Conditions for d=3,_ k=1,2,3
A
2
When d=3, a quadratic estimator 9f the form y = b O + b x + b x is
1
2
being used. Again, bO' b1 and b are determined from (3·11) and (3.47)
2
depending upon the assumed values of k.
(i) For k=l,
(4.58 )
where
(4.59 )
~ =
10
o
o
0
1
o
3/5
o
o
1
o
at
=01
(4.60)
=
t
~1
f
~1
••
,
4
and X is given by (4.,30), M4 is given by (4.,3,3) and
o
o
o
(4.61)
1
(syzmnetric)
Writing
(4.58) as
~1
(4.62)
~1
means
(4.6,3 )
(4.64)
I
_
I
-1
I
~1 - ~lM4 X4
(ii) For k=2
(4.66)
••
where
T'
2
= -"2,
A_M:1X'.
5'
-
h
I
- t :1,rd row of' T1 •
o
44
100
A2
=
0
1
0
001
-3/35
(4.68 )
o
=
6/7
t
-3/35
~1
(4.69)
t
= !!J.1
0
t
6/7
~1
e
•
t
~2
(4.70 )
=
,
,
~2
I
~2
(4.71)
where
.e
x'
X5=[~J
'
o
-3/35
3/5
0
o
6/7
45
1
0
[1 2]
(4.73 )
[12]
0
[1 4]
l~ = N
0
[1 4]
(1 4]
0
0
[1 6]
[16]
0
[18]
(symmetric)
(4.74)
where
o
o) .
Let
(4.76)
noting that e1 =: - IM4 / /I~I . Then, :from (4.61), (4.62), and assuming
(4.12) and (4.13)
-1
~
=
-e1
Combining (4.66), (4.69), (4.71) and (4.77) yields
(4.78)
.e
T'
2
=
~1
I
-3/35
~11
f
0
f
6/7
~1
-1(
I
-1)
4 1 4-Yl el!J.M4
-1
M4 Yl el
X'4
Zl
-4
46
Result 2:
(4.82 )
Proof:
[1 ]
4
= (3/35)(10
6
= (3/245)(53
T;
if and only if ~1 = ~,
[1 ]
Ti
Sufficient conditions for
=
Ti = T;
are
[12 ] - 1)
2
[1 ]) - 6)
1i1
=
1i2
and ~1 =~2
First, conditions ensuring that ~1 = ~2 are derived.
shown that
ii1 = ii2
and then conditions giving ~1
From
(4.63)
From
(4.58), (4.59), (4.61)
and
Then it is
= ~2
(4.79)
and
(4.75)
Hence, the condition
(4.83 )
yields ~1
.e
=:
~2·
From
(4.64)
and
(4.80)
are found.
47
From (4.58), (4.59), (4.61) and (4.75)
Hence the condition
(4.84)
yields ~~1
= ~2'
Equations (4.83) and (4.84), after some algebra (see
Appendix 9.1) reduce to the first two equations of (4.82), while the
6
inequality ensures 0 < [1 4], [1 ] < 1. Hence result 2 is established.
Result 2 along with (4.13) yields the constraint
(4.85 )
2
2
[18] ~ (828/1715)[1 ]3_(28179/60025)[12 ]2+(5346/60025)[1 ]-(1431/300125) •
[12 ]2 - (6/7) [12 ] + 3/35
'
It is again of interest to note that the least squares protection
4
2
conditions are from (4.4) with d=3 and r=2, [1 ] = 1/3, [1 ] = 1/5,
[1 6] = 1/7; a special case of result 2.
(iii) For k=3, additional conditions are needed to equate T , T
1 2
and T and thereby protect against quintic effects as well as cubic and
3
quartic. Now
(4.86)
where
48
1
0
0
0
-3/35
0
o
1
o
3/5
o
3/7
o
0
1
0
6/7
0
o
(4088 )
::
3/7
o
(4.89 )
::
~2
r
0
~2
I
3/7
r
0
~2
e
r
~3
(4090)
::
I
§:13
,
I
~3
( 4.91)
where
(4092)
(4093 )
~e
X'6 --
X;
[~J
,
is g1ven by (4.71) and
~
:: (xi, ~, ..., ~) ,
~
M6
:: [ v'
~
~
N[110] ]
,
where
(4.94)
Letting
e2
from
,~l
[lO])~l
= (:Y2~
:Y2~N 1
=a
scalar,
(4.93), (4.94), (4.95) and assuming (4.14) and (4.15),
(4.96)
-e
2
and hence
I
~2
o
T'3 = !!12'
3/7
I
~2
o
, -1
e~~
-e
2
Also
o
[12 ][16]_[1 4]2~
o
o
(symmetric)
..
e
o
50
where
~
(4.100 )
:=
[1 4 ][18 ] + 2[1 4 ][16 ][12 ] _ [1~]2[18] _ [1 4 ]3 _ [16 ]2
h = [1 6][12] - [1 4 ]2
2
.
Thus, from (4.97)
( 4.101)
(4.102)
= second row of
(4.103 )
T"; •
r _ I -1 t t -1 (
t -1 t
r) _
•
t
~3 ~ ~2~ X5-~2~ .Y2 e2~~ x5-e2~ - third row of T3 .
Result 3 ~
Sufficient conditions for T~ =T~ =T"; are
[1 4] = 3/35 (10 [12] - 1)
(4.104 )
6
[1 ]
= 3/245
8
[1 ]
= 9/8575
2
(53 [1 ] - 6)
2
(460 [1 ] - 53)
2
53/460 < [1 ] < 1
2
[1 ]
r 1/3
.
It is sufficient to find conditions yielding T~ >=T"; and then
Proof:
including result 2.
lit
First ~3
:=
~2 because (4.98) and (4.79) imply
-1.::Q ( e~~
t -1X5-e~
t
t) and from ( 4
) ('4 ·70"
) ( 4.98) and
.67,
~3 >= ~2-~~
(4.94) ~2
(
:=
(1
0
0
0
)
0, element, 0, element, O.
-3/35) and (M51~)t is of the form
Hence
t
-1
!!o2~ ~
= o.
Furthermore, from
d (j 66 ) t
i
-1 t
d I
r -1 t
('4 • 1rY'»
vc.
an
'+.
~12 = ~2~ X5 an 113 = ~2~ X 5
(!i2~~1~-3/7)(e2~M5~) + (~:i.2M51~-3/7)e#5' From (4.70), (4.98)
51
and (4.94), ~~2
:=
~i3 if
since the left hand side of (4.105) is
I
(4
·90) yield ~2
:=
I
-1 I
I
~2~ X'5 and ~3
:=
~2M5\~2. Similarly, (4.53) and
I
-1 I I -1 (
I
-1 I
I )
~2~ X5-a22~ :Y2 e~~ X5-e2~ ,
and (4.59), (4.60) and (4.61) imply ~2M51:Y2
:=
Hence ~2
O.
It remains to make (4.105) compatible with result 2.
:=
~3 .
Alte:dng (4.105)
yields
(4.106)
Notice that if [12 ]
:=
4
1/3, then result 2 implies [1 ]
:=
1/5, which makes
8
the left hand side of (4.106) zero, and hence [1 ] and [110] may take
on any value staisfying (4.14) and (4.15).
In this case, then
6
4
but if [12 ] = 1/3 and [1 ) = 1/5 then (4.107) yields [1 ) = 1/7 or
[16 ] = 3/25.
[12 ]
:=
6
Since [1 ]
:=
3/25 is not compatible with result 2 when
2
1/3, this value of [1 ] is excluded when using result 2.
Simplifying (4.107) and using result 2, yields after some algebra (see
Appendix 9.2), the third equation of (4.104), where the inequality on
[12 ] ensures the moments lying between 0 and 1.
This establishes
result 3.
4.4
Design Moment Conditions for d=4; k=1,2
When d=4, a cubic estimator of the form y'" = b
.e·
is being used where the b I S
+ b l x + b x2 + b X"'':Ii
2
3
are determined from (3 .11) and (3.72)
depending upon the value of k .
O
and
x;
is given
by
(4071) and
Writing (40108) as
t'
--II
( 40111)
yields
T' :::
1
~1
M51 is
given 'by (4098).
53
(4.112 )
(11) For k"'2"
(4.116)
where
(40117)
~=
1
0
0
0
-3/35
0
0
1
0
0
0
-5/21
0
0
1
0
6/7
0
0
0
0
1
0
10/9
0
-5/21
(40118)
""
~
0
10/9
(40119 )
~1
I
0
I
-5/21
~1
:=
~r
~1
h
e
I
!!31
0
10/9
i
§;12
,
(4.120)
(4.121)
Xl =
6
[~
]
~
,
61 is given by (4.96).
as in (4.91) and (4.92) and M
Thus, f'rom (4.116),
(4.119), (4.96) and (40121)
I
e
i
§;11
(4.122 )
l
T
2
=
~ (I5-~e~~
-1
~ ~e2
I
-1
e2~~
-e
2
-1
0
~1
I
-1)
Xl
5
-5/2
I
0
~1
I
.§:31 10/9
Zl
~
and theref'ore
= f'irst
(4.124)
row of' T~ ,
I
i -1, ( ' -1
~) (
I
-1 I ) (i - 1 / )
I
~12 = ~1~ X5 - !!11~ ~+5/21 e2~~ X5 + ~1~ .Y2+5 21 e~
= second row of' T; ,
(4.125 )
i
i
-1
= third
(4.126)
I
i
-1
(
I
-1
I
I )
~2 = ~1~ X5-~1~ ~ e2~~ X5-e2~
row of' T~ ,
i_I
-1 i ( I -1
/ )(
I
-1 I )+( I -1
1 /)
I
~32 - ~1~ X5 - ~1~ ~-10 9 e~~ X5 .!!31~ ~- 0 9 e2~
= f'ourth
row of' T~ .
55
Suf'ficient conditions for T~ ::; T~ are
Result 4:
4
[16] :: (5/21)«(14/3) [1 ]
c~
[12 ])
8
4
[1 ] "" (50/189)«113/3 0 ) [1 ]
~
[12 ] ,
Since T~ :: T; if' and onl.y if ~1 :: ~2' !~1 ::: .t~2'
Pl'oof:
~l :" .!.;2 and !31 "" !32' the proof' shall follow the same procedure as
in results 1, 2 and 3.
First.tbl
=:
~2' because, from (4.123) and
(4.112) it is sufficient to show that ~lM;l~
=:
From (4.98) and
O.
,
(4.94), as seen previously (M5~~)i is of the form (0, element, 0,
element, 0) and from (4.109) and (4.110), ~l
Hence .t;1
=:
~l ;: (0
~2'
0
=:
(1
0
0
0
-3/35).
Similarly, from (~.•109) and (4.110),
1
0
6/7) and therefore ~1M51~
~1 "" ~2 from (4.125) and (4.114).
=:
0, which implies
It remains to find +.he conditions
which will equate the second and fourth rows of Ti and T;.
From (4.113)
and (4.124)
i i - I
!11
=:
i
~11~ X5
i
i ~1 I . I -1
/) (
I -1 I ) ( '
-1
/)
I
!12 "" .!!:Ll~ X5- (~11~ ~ T5 21 e~~ X; + !!ll~ ~ T5 21 e:#5.
From (4.109), (4.l10)J (4.98) and (4.94)
Thus, ~~l
=:!i2
if
Similarly, :from (40115) and (40126)
f
i
-1 i (i
!32 "" ~l~
Again, :from
X5-
-1
/ ).
i -1 i) (i -1
/)
i
~1~ !2-10 9 (ei!2~ X5 + E!:31~ Yz- IO 9 e~.
(4.109), (4.110), (4.98)
and
(4.94)
,
After some algebra. (see Sppend.1.x
the two equations o:f (40114).
903), (4.128) and (4.129) reduce to
From these equations, the inequa1.ity
8
6
ensures that [1 ] and [1 ] are between 0 and L
Hence result 4 is
established.
It is convenient and useful :for the :following section to summari,ze
the important results ot this section as in Table 4.10
57
Table 4010
of' design moment requirements :for linear~ quadratic
and cubic estima,to:rs and protection against certain higher
SUUl1IIarjr
order ef':fects
d
Values of' k
Moment condition5§:/
Protected against
4 ~ (3/5)[12 ]
[1 ]
2
[1:2]
4
[1 ]
:=;
1/3:; [1 4] "" 1/5:; [1 6] "" 1/7
:=;
(3/35)(10[12 ]-1)
£16]
:=;
(3/245)(53[12 ]-6)
2
3
(6/53) < £1 ] < 1
[1 4] (3/35)(10[12 ]-1)
[16] (3/245)(53[12 ]-6)
:=;
:=;
f1
8]
:=;
(9/8575)(460[12 ]=53)
'c
2
(53/460) < [1 ] < 1
4
(5/21)(14/3)[1 4]-[12])
[18] ~ (5 0 /189)«113/30)[14]-.[12])
(30/113)[12] < [14] < (9/10) + (3/14)[12]
6
[1 ] ""
~/The condition:; to be found in the text:; for non-singular (X'X)
matrices are not included hereo
50
SOME DESIGNS
For the design moment conditions
gen~rated
in the previous section,
actual designs are derived for specified experimental situations,
Designs are developed for certain numbers of experimental levels presumed of interest to experimenters.
Aside from the particular designs
much of this material is to exhibit techniques, approaches, interpreta=
tiona and meanings of reBol ving the design moment condi.tions.
It is
clear that the admissible values of sol.utions to equations involving
the experimental levels 'W1.11 lead to additional requirements on the
design moments 0 Some of the design possibilities will offer the
experimenter more design moment flex:i.bili ty than the earlier approaches.
An important solution technique is the application of the elementary symmetric functions, see Hou,seholder (1953 L to finding experimental levels satisfying the design moment requirements.
T'nis technique
was exploited for similar problems and discussed in detail by Manson
(1966)0
Since this methodology is very useful, the particular type of
problem involved and the corresponding method of solution is described
"below
0
Consider the problem of finding
~,u2".0,Ut
such that the equations
= Sp(Y)
are satisfied for given y and known ~o
Newton's identities relate the
sp to the elementary symmetric functions ~j of the roots of the tth
degree polynomial p( u) •
Thus, given y, the
~j
are determined and
represent the coefficients of ut~j in p(u), i.e.
59
p(u)
:= U
t
.- 0-, U
t-l +
.J..
0- U
2
t-2 +
0
_,
•
Hence, the u. satisfying (5.1) are the roots of p(u)
1.
=0
•
For deriving the designs of this chapter 0"1' 0"2' 0"3 and 0"4 are
needed.
They are given by, see Manson (1966),
0"1 - 8 1
2
- s2
1
=
c
0"2
2•
8
8
3 - 3 8 8 + 26
1 2
3
1
0"3 =
6
4
1
0"4 =
3:
-
2
6s 1 s
2
+ 8s1 8 + 3 s22
3
4'.
-
6s
4
It is important also to note that a necessary condition for all the
roots of p(u) to be real and non-negative is that all the 0". be nonJ
negative.
For convenience of notation, denote [12 ] by a, [li+] by b, [1 6 ] by
8
c, [1 ] bye.
Most of the numerical results presented in this chapter and the
next were obtained using an IBM 360 computer system.
5.1
Designs for d=2
In this section a class of designs is derived for d=2 with protection against k=l and 2.
The conditions which must be satisfied are
60
class of designs is defined by consideration of experiments involving
five levels £lJ£2J£3J£4JO.
= -£1
Letting £3
and £4
= -£2satisfies
and it is also convenient to write N
a : -.
(4m+n) 1m = Him
= 4m+n,
the symmetry" requirements
where m "" lJ2,3,.o.'
= 4+n/m •
The design "rill then be completely specified by plac::i.ng m observations
at each of the four experimental values x
=.::t.ep x ""
where £1 and £2 are to be determined.
By" the definition of a and from
N
Z
a::-.
x
u=l
: -. m.ei
2
(5.7)
IN
U
+
m.e~
+
m.e~
+
m.e~
+ n( 0 )
N
or
Similarly
I
lJ.!.j.
"'1
+
•
lJ4 ::;
"'2
g b
2
.
The condition b "" 3a/5 and (5.12) yield
±.£2 and n at x=O,
61
(~4 + ~4) = 2 a
. 1
25'
= ga
b
which means
.e14 +.e24 = 3a
(5·14)
a/10 .
Thus, (5.11) and (5.14) are the two equations
2 + 02
0
....1
....2
= g2
a
and the elementary symmetry functions can now be introduced.
Setting
= ....12
+ Ai/12
(5.15 )
8
1
(5.16)
8
_ /14 + /14
Ai
2 '
2 .- Ail
0
2
it is clear that (5.11), (5.14), (5.15), (5.16) can be represented by
the equations ( 5.1) where t
g2(Y)
= 2,
~
= .e21 ,
~
= .e22 ,
y
= a,
gl(Y)
= ay ; 2,
= 3ar/10 • Hence using (5.3) and (5.4)
~
2
and therefore
u2
= ~~
.ei and .e~ are the two roots of the equation
-~u+~
The necessity for
constraint
a (g a - 2)
2
5'
1
~1
2
and
=0.
~2
to be non-negative yields the additional
.e
62
(5,20 )
a
2: 6/5a .
substituting (5.17) and (5.18) in (5.19) and solving the quadratic
equation yields
a
a
2 faa
(5 - 4 a)]
Since the two solutions are
£i and £~,
(5·21)
u =
4a
±
1
':<
.l2
•
it is necessary that
3/5 - aa/4 ~ 0 and this implies
a < l2/5a .
Hence, it is sufficient to set
£i and £~ at the values given in (5.21).
To ensure that c " 9a/25, note
(5·23)
:2
a
Q
= a a(25 - 8
which means that c
= 9a/25
a) ,
if a = 6/5a
Or l2/5a, and therefore the
equalities in (5.20) and (5.22) are not allowed.
Hence, summarizing the above development, the desired class of
designs is given by:
N = 4m+n,
= N/m,
m = 1,2,3, .•• ,; n
m observations at +J
m observations at -J
l
l
m observations at +J
2
m observations at -£2
n observations at
°,
= 0,1,2,3.
.e
63
where
(5·25 )
(5·26)
6
12
5a<a<5a'
An alternative way of looking at the design is by the explicit
For simplicity take a = 4,
relation between £1 and £2'
N is divisible by 4:
Since £~+
£~ = ~)(£i + £~),
!.~.
suppose
£~
solving for
yields
(5.28)
Furthermore
(~!25)
-
4£~
+ l2£i/5
~
0
~mplies
(5.29)
(This can also be found from max £i in (5.25))·
a
Also, from (5.27), 3/5 < £i + £~ <: 6/5 and hence all admissible values
1
1
of £1 and £2 must lie between the two circles of radius (.6)2 and (1.2)2.
Figure 5.1 shows the admissible values of £1 and £2'
5.2
Designs for d=3 z k=1,2
From result 2, the conditions
b
=~
(10 a-I)
c = 2t (53 a - 6)
5
6
53 < a < 1
w~ich
must be satisfied are
64
.e-
.1
o. 0 '---r---t----t------1r----t---~-__+_--!-r__-__r_-_+£1
000
Figure 5.10
01
.2
.3
.4
.5
.6
.7
.8
Values of .£1 and £2 with N/4 observations at
for d=2, k=1,2 (X: the least squares design)
.:t
.9
£1 and
1.0
.:t
'£2'
.e
65
The class of designs is defined for experiments where N is divisible by
six or six levels £1'£2""'£6 are involved, with N/6 experiments at
each level.
Consider symmetric designs with £4
= -£1'
£5
= -~,
£6
= -£3'
As
in the preceding section, the second, fourth and sixth moments can be
written
(5.30)
(5.31)
(5.32)
Substitution of the requirements on band c in terms of a yields the
three equations
In order to solve these equations for £1' £2' £3 in terms of the second
moment, the elementary symmetric functions are used, where now the
application of (5·1) is for t = 3, ui =
£i, r
= a and gl'
the right hand sides of (5.33), (5.34), (5.35).
(5.4) and (5.5)
~, g3 are
Hence, from (5.3),
66
.e
0"1
= s 1 = 3a
0"2
2
s1 - s2
21
=
8
0"3
=
= 22
1)
(a2 - -2 a +7
35
3
s s + 28
1 - 3 1 2
3
31
= 22 (a3 - -6 a2
7
For a > 6/53, 0"1' 0"2' 0"3 are greater than zero.
+ 169
4
735 a - 245)
The requires values of
£i, £~, £~ are therefore the roots of the cubic equation
where 0"1' 0"2' 0"3 are given by (5.36), (5.37) and (5.38) in terms of a.
Varying a over the range (6/53,1) in increments of .01, and solving
(5.39) leads to the results in Table 5.1.
a
= .33
Note that only the range of
= .53 is admissible, since all other values generated 2
to a
complex roots.
It is also of interest to observe that if minimum bias
subject to least squares estimation is desired, then for this experimental set-up, only the levels £1 = .86624682, £2 = .26663540, £3 =
.42251865, corresponding to a ~ 1/3, could be used.
5.3
Designs for d=4, k=1,2
When fitting a cubic and protecting against quartic and quintic
true models, result 4 gives the conditi<;ms
c =
e
i
(14b - 3a)
= 5g7 (113b - 30a) ,
.-
67
Designs ~or d~3, k~1,2 with N/6 experiments at each o£
Table 5.1.
~ £1' ~ £2' ± £3
a
e
_
.
£1
£2
£3
·33
.86441137
.28063902
.40501201
.34
.86978621
.24632224
.45033021
·35
.874773 45
.22563634
.48359037
.36
.87938347
.21185459
·51166624
.37
.88362364
.20295407
·53667393
.38
.88749837
.19796810
.55960278
·39
.89100901
.19628338
.58101272
.40
.89415364
.19742269
.60125996
.41
.89692670
.20097117
.62059091
.42
.89931834
.20655578
.63918795
.43
.9013134$
.21384387
.65719465
.44
·90289038
.22254580
,67473130
.45
.90401836
.23241593
.69190580
,46
.90465420
.24325064
·70882290
.47
.90473611
.25488431
,72559393
.48-
.90417304
.26718429
.74235009
.49
.90282402
.28004567
.75926492
·50
.90045340
.293'38651
.77660030
·51
.89661579
·30714374
.79482252
·52
.89025754
.32136974
.81500139
.53
.87683336
.33572962
.84169405
68
with 0 < a,b,c,e < 1.
Consider de$igns using eight non-zero levels
£1'£2""'£8' placing N/8 observation at each level and symmetrized
by setting £5
= -£1'
£6 = -£2' £7
= -£3'
£8 = -£4'
The second, fourth, sixth and eighth moments are
(5.41)
(5.43 )
e
= (£~
+
J~
+
£~
+
J~)/4 .
In terms of the protection conditions, these equations are
(5.44)
For specified values of a and b, equations (5.44) - (5.47) represent a
system of four equations in the four unknowns £1' £2' £3' £4'
tion of (5.1) is now for t
= 4,
ui
= £~,
r
,Applica-
= (a,b)
andgl , g2' g3' g4
the respective right hand sides of (5.44), (5.45), (5.46) and (5.47).
Thus the four elementary symmetric functions are, from (5.3), (5.4),
e
•
69
.e
=
4a
(5.4)
2
= 2( 4a -b)
6
0"3
=
3
s
1 - 3s l 2 + 2s 3
3!
= 32 a3 _ 8ab +
3
40 b - 20 a
27
63
422
_ sl - 6s l 6 2 + 86 1 6 + 3s 2 - 6s 4
0"4 41 3
= 32
3 a
4 _ 16
2_
ao
_
80 2... 2b2 ... 1120 b + -.2Q _ 2.§L b
63 a
JB9 a 189 a 567 .
Solving the quartic equation
U
4 _ rT"v u3 + rT"2
v U2 - rT"
v u + v4 l
3
rr
-
0
222 2
yields the required values of t l , ~2' ~3' ~4' Since solutions are
desired for various values of a and b in order to generate a family of
designs, the region of admissible (a,~) values is at first confined to
that region where 0"1' 0"2' 0"3 and 0"4 are non-negative; necessary for all
the roots of (5.52) to be real.
This, of course, is not sufficient for
all the roots to be real and does not ensure that the roots lie between
zero and one, but the conditions reduce the region of admissible designs in the (a, b) plane.
region.
Figure 5.2 shows this initially admissible
(It is of interest to note that the minimum bias conditions on
70
·25
.20
.10
.05
0.0
0.0
Figure 5.2.
.e
.10
.20
.40
4
Initial admissible region on [12 ] and [1 ] for d=4, k=1,2,
N/8 observations at ± £1' .± £2' ± £3' ± £4 (shaded area
shows final admissible region)
71
the design to have a least squares estimator lead to an inadmissible
result for this class of designs.
requirements a
This is because the least squares
= 1/3, b = 1/5, c ;::: 1/7, e = 1/9, mean that 0'"4 = - .00101
and hence minimum bias designs cannot be attained, for this class, using
the least squares estimator).
Table 5.2 presents designs, obtained by
"sweeping out" the admissible region of Figure 5.2, which satisfy the
requirements, along with the corresponding values of the moments a and
b.
Designs for this situation must satisfy, from result 3,
b = .2 (10 a-I)
35
c =
e
2i5
(53 a - 6)
= 8;75 (460 a-53) ,
with (53/460) < a < L
Thus, for example, with N experiments, 2m
distinct non-zero levels £1'£2' ""£m'£m+l
= -£m,
= -£l'£m+2 = -£2""'£2m
and N divisible by 2m so that t:Q.ere are N/2m experiments at each
£., the problem is to solve
~
th~
system of four equations in the m
£2 + £2 +
1
2
.-
£4 + £4 +
1
2
+ £4 =
£6 + £6 +
1
2
+ £6
£8 + £8 +
1
2
+
m
m
£~
~ (lOa-I)
35
= 25
3m (53a-6)
4
;:::
8§~5
(460a-53) •
.-
e
«
e
72.
Designs for d=4, k=1,2. with N/8 experiments at each of
~ J2.' ~ J ,± J
4
3
Table 5.2.
± J1,
a
b
.400
.2.4
.63962.956
.405
.2.4
.410
J1
J
3
J4
·53607600
.91665500
.2.5147596
.62.677741
.55653767
.91584868
.2.8042.306
.2.4
.60713511
.58092670
.91497807
.31100843
.42.5
.2.4
.6114562.1
.5532.4569
.91169107
.43458014
.400
·2.5
.68408360
.487595 60
.92.196057
.2.1040169
.405
·2.5
.67802.394
·50505011
.92.12.8764
·2.3756475
.410
·2.5
.67154472.
.52.055505
.92.05842.0
.2.6565931
.415
·2.5
.665012.13
.533442.59
.91982.855
·2.9514963
.42.0
·2.5
.6952.3584
.542.2.3378
.91899549
·32.686072.
.42.5
·2.5
.65571176
·54391114
.91805546
.362.45954
.430
·2.5
.6562.7060
·532.072.81
.91697346
.40665353
.400
.2.6
.71535163
.44873534
·92.647406
.16898061
.405
.2.6
.71102.403
.468162.66
.92.589535
.19490088
.410
.2.6
.70631837
.48561049
.92.531553
.2.2.155806
.415
.2.6
.70135707
·50097799
.92.471612.
.2.4903697
.420·
.2.6
.69636526
.51387679
.92.407580
.27765085
.425
.2.6
.69173659
·52347907
.92337002
.30799019
.430
.2.6
.68810545
.52817935
.92257036
.34117648
.435
.2.6
.68634597
·52462531
.92.164349
.37982.993
.440
.2.6
.687352.76
.5002.4504
.92.054976
.43576281
.400
·2.7
.74157156
.41115062.
.9302.1857
.12.53802.2.
.405
.27
.7382.3905
.43369316
.92.97012.5
.1502.2.959
J2.
Table continued on next page
73
Table 5.2 (continued)
a
e
•~
b
.£1
.£2
.£3
.£4
.410
.27
.73452774
.45384940
.92921024
.17623299
.415
.27
.73049120
.47188822
.92872925
.20289359
.420
.27
.72622119
.48784667
.92823957
.23017323
.425
.27
.72186731
·5015332;1.
.92771981
.25827888
.430
.27
.71767022
.51246153
.92714564
.28763451
.435
.27
.71399825
·51967945
·92648916
.31899469
.440
.27
.71137240
·52132255
.92571814
·35386159
.445
.27
.71043226
.51278242
.92479472
·39635202
.400
.28
.76502898
·37172152
.93317941
.07569649
.405
.28
.76241015
.39871545
.93269370
.09919282
.410
.28
.75940922
.42231280
.93226172
.12584764
.415
.28
.75605404
.44330222
.93186875
.15325163
.420
.28
.75238953
.46208039
.93149773
.18084158
.425
.28
.74848615
.47879925
·93112923
.20990154
.430
.28
.74445267
.49340897
·93074129
.23676671
.435
.28
.74045435
.50564013
·93030922
.26566927
.440
.28
.73673625
·51492129
.92980516
.29586844
.445
.28
.73364586
·52017211
.92919734
.32894129
.450
.28
.73163883
.51918886
.92844900
.36486985
.455
.28
.73123857
·50535362
.92751670
.41185028
.400
.29
.78683340
.32727363
.93529479
.03148292
.410
.29
.78235124
.38949268
·93441003
.05567666
.415
.29
.77954263
.41402384
·93407917
.09161686
.420
.29
.77638695
.43578513
.12380788
·93380203
Table continued on next page
~e
74
Table 5.2 (continued)
a
e
e
b
£1
£2
£3
$4
.425
.29
·77292301
.45527657
.93356001
.15420395
.430
.29
·76920819
.47273869
.93333243
.18365040
.435
.29
.76532664
.48822369
.93309589
.21270829
.440
.29
·76140109
·50160689
.93282479
.24186118
.445
,29
.75760848
.51255226
·93249063
.27162620
.450
.29
.75419794
.52041477
.93206154
·30268006
.455
.29
.75150472
·52399643
·93150125
.33611578
.420
·30
.79914275
.40839097
.93502042
.01801430
.425
·30
.79606953
.43078313
.93488949
.08173641
.430
·30
.79268143
.45Q884~2
.93480883
.12038282
.435
.30
.78902817
.46899659
.93475501
.15397987
.440
.30
.78517905
.48524339
.93470258
.18537477
.445
·30
·78123100
.49959724
.93462397
.21577478
.450
·30
·77721976
·51187191
.93448934
.24594881
.455
·30
.77363367
·52167642
.93426601
.27657128
.435
·31
.81216194
~44846262
.93505383
.07034634
.440
·31
.80845103
.46710577
.93523142
,11645309
.445
·31
.80452150
.48398292
·93541922
.15328593
.450
·31
.80045470
.49913147
.93558625
.18659561
.455
·31
.79636032
.51246231
·93570110
.21830274
.445
·32
.82801485
.46671702
.93445284
.05800463
.450
·32
.82385947
.4840~700
.93498633
.11272187
.455
·32
.81954550
.49988351
.93549674
.15266795
.455
·33
.84390157
.48532938
.93286994
.04515757
75
222
The solution must be such that £1' £2' .•• , £m are all real
numbers lying between zero and one, for some values of a lying between
(53/460) and 1. Attempts at solving (5.53), using the previously
discussed solution techniques, have been made for m=4 and 5, and for
values of m from 6 to 10 allowing experiments at zero with four nonzero levels, and no admissible solutions have yet been found.
76
6.
GENERALIZATION TO HIGHER DIMENSIONS
This chapter includes extension of the previous development to
fitting polynomial responses involving p controllable factors.
Given
the new set-up, the optimum estimator is the "natural extension" of the
estimator derived in section 3, where the derivation did not explicitly
depend on the dimension of the factor space.
Examples of the design
problem are also presented in this section.
6.1
The p-Dimensional Case
As in section 3.1, an entire region of operability is assumed,
over which the true model ~ at the point ~t
= (~,x2' ... ,xp )
polynomial of degree d+k-l in the p factors xl ,x ' •.. ,xp '
2
model is again written
(6.1)
~(x)
-
= -11:::.1
x tR +
xt~
==2 2
is a
The true
,
where now
(6.2 )
~i = (1;xl ,x2 ,···,xp; xi,x~""'~_lxp; ~,~, ... ,xp_2xp_lxp;
• • . ; •.• ;xl~l ' x2~l ,... , xp _d -t2 • • •xp _l xp )
o
0
•
.
J
(6.4)
~~~'~P-d+.2,. o,p-l,p)
0
d-l
d-l
77
~ = (13~~2'" .,l3p - d +l , .•. ,p-l,p,···j
d
d
... , 1311. .. 1' 1322 ...2' ... ,13p- d- k+2, ... , p )
•
~~
d+k-l
Note that equations
tions
d+k-l
(6.2), (6.3), (6.4)
(3.2), (3.3), (3.4)
and
and
(6.5)
correspond to equa-
(3.5).
Since a polynomial of' degree q in r variables contains (q+r) terms,
q
1') (2£)
contains
(6.6)
el =
p+d+k-l
(d+k-l )
terms, ~ and ~l are (e 2xl) vectors, where
..
and ~ and ~ are (e 3xJ) vectors, where
(6.8)
Over the region of' interest R,
where the
e2
elements of'
structure is as in
(3.7)
:£
1')
is to be fitted by the estimator
are to be determined.
and
(3.8)
The observational
and the "X" matrices are now
78
I
~1
,
~ 21
(6.10)
Xl
=
I
~Nl
and
,
~12
,
~2
(6.11)
X
2
=
,
~2
e
JS.
and X2 is (N x e )· In (6.10) and (6 .11) ~~1
3
th
and ~~2 represent the i
observation on vectors of the type in (6.2)
so that
is(N x
e2 )
and (6.3).
As in (3.11) attention is restricted to those £'s of the form
£
= T'~, where now T is the (N x e2 ) matrix to be determined. Remember-
ing the generalized set-up, the derivation of the (min V
I
min B)
estimator is exactly as in section 3, noting that integration is now in
p-space over the region R, and the Wmatrices in (3.25), (3.26) and
(3.27) are extended to integrals over R with respect to x ,x , ..• ,xp '
1 2
The problem is now to choose T to minimize
(6.12 )
where
n-1
=
J
R
~,
and subject to this, to minimize
79
v = n f Var
y (2S)~
•
R
This leads to the T matrix given by (3.47), in the generalized form,
and the (min V
I
min B) estimator, as in (3.50),
(6.14)
6.2
Example 1.
Two Examples of the Design Problem
Suppose there are two controllable factors
and x
2
and suppose that yt~) is a linear function and that protection is required against the true model being quadratic or cubic.
~
Thus, p=2, d=2
and it is necessary to choose a design which equates the (min V
estimators for k=l and k=2.
defined by -1
~~,
When k=l; $1
$2
= 3,
(6.16)
~
2 2
)
= (Xl'~'Xlx2
(31
= (130 , (31' (32)
Le.
(6.19 )
(6.20)
1
-1
$3
xl ,x )
2
~ = ((311,(322,(312)
min B)
It is also assumed that R is the region
1.
~ = (1;
(6.18)
e
= 6,
~
(6.15)
(6.17)
•
x2
I
,
=3
and.
j
I
j
80
The estimator is
I
where
-b ,: T'v
1.aL. ,
and
T'1
(6.24)
= -l
JL(X'X
)-l '
a a Xa '
where -l
JL and Xla and (X'X
f1 are defined below.
a a
In this case
.
e
(6.25 )
(6.26)
W1 =
W =
2
1
0
0
0
1/3
0
0
0
1/3
1/3
1/3
0
0
0
0
0
0
0
,
,
and therefore
(6.27)
which means
W~~2
=
, \
\....
(6.21)
/3
1/3
0
0
0
0
0
0
0
,
/
.
'\
~
81
(6.28 )
~
=
1
o
o
1/3
1/3
o
0
1
o
o
o
o
o
o
1
o
o
o
a'
"'"'01
,
=
(6.29 )
~1
f
~1
Furthermore, here
e
(6.30 )
Xl
a
=
1
1
1
~1
~2
x1N
x21
x22
x
2
~1
2
~2
2
~N
x21
2
x22
x
x
~2x22
~r2N
2N
2
x
11 21
,
2
2N
so that, assuming all odd moments are zero,
(6.31)
e
XiX.
a a
=N
1
0
0
[12 ]
[22]
0
0
[12 ]
0
0
0
0
0
0
[22]
0
0
0
[12 ]
0
0
[1 4]
(22]
0
0
4
2
[1 2 2 ] (2 ]
0
0
0
0
(12 22 ]
0
0
0
[12 2 2 ] .
,
.-
82
and
(6.32)
(XIX
a a
r1
[1 4)[2 4)_[1222 )2
h
=
0
0
1
2
[1 )
0
1
(1222 )[22)_[12 )[2 4)
h
1
[22)
N
[1222 )[12 )_[1 4)[22)
0
h
0
0
0
0
0
0
(12 )[22] _ [1222 ]
(symmetric)
o
h
1
where
Thus, using (6.29), (6.24) can be written
~1
(6.34)
~1
a' (XiX
-=21
a a r~'a
83
(6·36)
and ~i and ~i are as in (6.15) and (6.17).
The estimator is
y(~) = ~ T;Z where
where A and ~ are defined below.
2
In this case WI is as in (6.25) and
W2
=
/3
1/3
o
o
o
o
o
o
o
o
1/5
o
1/9
o
o
o
o
o
1/5
o
1/9
so that
(6.40)
1/3
1/3
o
o
o
o
o
o
o
o
3/5
o
1/3
o
o
o
o
o
3/5
o
1/3
and hence
(6.41)
~
=
1
0
0
1/3
1/3
0
o
o
o
o
0
1
0
o
o
o
3/5
o
1/3
0
001
o
o
o
o
3/5
0
1/3
84
·e
(6.42)
~=
A:t
I
(6.43 )
:::
0
0
0
0
3/5
0
1/3
0
0
1/3
0
3/5
~1
0
0
0
0
~1
t
3/5
0
1/3
0
t
0
1/3
0
3/5
~1
t
~2
(6.44)
:::
I
§;12
I
~2
e
Also
1
1
1
~1
x12
x
22
2
x1N
x2N
x21
2
xlI
(6.45 )
e
x;,:::
...
X
2
x22
x21N
2
x2N
x11x
21
x12x22
xl~2N
xiI
xi2
xiN
~I
~2
~N
x11x212
2 x
x11
21
x12X222
2
X x
12 22
2
x1rt'2N
2
2
21
~2
xI~2N
85
.e
(6.46)
where U' is the (4xN) matrix of the last four rows
(6.48)
E
= XIX
a a
F
= x'u
a
G = UIU
Of~.
Let
.
Then
(6·50 )
~~
E
=
[F
,
l
and
(6.51)
-B
]
,
where
and E- 1 is assumed to exist.
(6.53 )
where
~2
§ol
!~2
= ~~1
~2
~1
l
I
From (6.38), (6.44), (6.47), and (6.51),
0
0
3/5
0
0
0
-0
E- 1 _E- 1FBF ' E- 1 E- 1FB
Xl
a
,
1/3 0
1/3 0 3/5
BI F'E- 1
-B
ul
86
.e
(6·54)
The problem is now to find sufficient conditions for equating the three
rows of (6.53) and the three rows of (6.34).
Result 2~
Sufficient conditions for T~
[1 ]
4
= 3/5
[12 ]
[1222 ]
= 1/3
[12 ]
[12 ]
= [22]
4
=
T; are
= [2 4] ,
[1 ]
and the XIX matrices in (6.32) and (6.51) are non-singular.
Proof ~
row.
As in the previous results conditions are derived row by
From (6.35) and (6.47)
,
~1
,-1 I
E Xa
= §:ol
and f!'om (6.53)
but from (6.28), (6.29), (6.32), (6.45), (6047), (6.48), ~lE-1F
where
2 is a
(4xl) null vector.
Therefore ~2
= 2',
= ~1· Similarly, from
(6.35) and (6.47)
and fro~ (6.53), and noting that B = B' in (6.52),
87
.e
where ~I
!i2
o
= (3/5
=! i l '
1/3
0) •
Hence if
-1
I
~ll E
F:o
~
r
, then
From (6.28) and (6.29) and the definition of ~t, this
I
means the second row of E-1F must equal s.
From ( 6.32), (6.45 ) , (6.47)
and (6.48), the second row of E-1F is
o
4
2
Thus, if ([1 ]/[1 ]) = 3/5
Finally, as above, ~2
(0
1/3
0
and
2 2
2
([1 2 ]/[1 ])
= 1/3,
then !i2
= !il'
= ~l if the third row of E-1F is the same as
3/5).
([1222 ]/[22]) = 1/3.
0) •
4
This will occur if ([2 ]/[22])
Since this means
[12~2]
=
[1 ]
2
and further, [2 4 ] = 3/5 [22] = 3/5 [1 ]
= [1 4] .
[12~2]
= 3/5 and
, then [12 ] = [22],
[2 ]
Hence result 5 is
established.
It is again of interest to note that the least squares minimum
bias conditions force [12 ] = [2 2 ] = 1/3, [1 4] = [2 4] = 1/5 and
[1222 ] = 1/9; a special case of (6.55).
For the design moment conditions given in result 5, actual designs
are generated for different configurations of levels of the two factors.
All the examples will be for symmetric designs and comparisons with the
least squares minimum bias designs will be made, if the latter exist.
Combinations of factor levels shall be denoted by (.8 ,.8 j) say, meaning
i
Xl is at level .8 i and X2 is at level .8 j •
.
Design 1:
This is a twelve point configuration made up of four
experiments at (.8 ,0), (0,.81 ) (-.8 ,0), (0,-.8 ) and a 32 factorial with
1
1
1
-'£2' 0 and .£2 as the levels, with the center point deleted (center
88
points may, however, be added with no change in the resulting design).
For this design
(6.56 )
(6.57)
12 4
E x
u=l 1u
12
(6.58)
E
2
=
12 4
E x
u=l 2u
2
= 4.£4
u=l ~u~u
'
+ 6.£i}
1
2
= 2.£
4
2 '
and conditions (6.55) yield
(6.60)
The solution of (6.59) and (6.60) is
(6.61)
(6.62)
.£
2
= -
1
-{2
1
1/2
<::; 1/4
(1 + - - )
= (,,-)
.£,
-{l5
3
1
or
.e1
::;.69806757
.£2 =.79315789
Note that there is no least squares minimum bias design for this configuration.
Figure 6.1 shows the design.
x
x
_ _ _ _ _Xll...X.ll.-_ _
O
x
x
Figure
6.1.
xl
XA...A.X
Design 1, experimental configuration, J
Design 2:
l
= .698,
J
2
=
.793
This is also a twelve point configuration (center points
again mayor may not be added with no change in the other levels) with
eight points at (J ,J ), (-J
l
1 2
(-J
2
,-.el
,.e2 ),
(-J ,-J ), (J ,-J ),(J
l
2
l
2
) and (J ,-J ), and four at (J ,.e ), (-J ,J ),
l
2
3 3
3 3
U3'-J ).
3
Figure 6.2 shows a typical configuration.
Here
(6.64)
and applying conditions (6.55) yields
2
(-.e
,.el
3
), (-J ,J ),
2 l
,-J ),
3
90
x
x
x
x
x
x
x
Figure
6.2.
Typical design
x
2,
experimental configuration
(6.66)
(6.67)
A class of admissible solutions is defined as those solutions such that
.e1 , .e2 , .e3
are all real and such that 0
~ .ei, .e~, .e~ ~ 1.
Varying the
quantity (.ei-.e~) over the range .50 to .61, and thus ensuring admissible
solutions, in steps of .005 yields the designs in Table 6.1.
•
(6.68)
The least
~e
91
Table 6.1.
J)1
Designs for the configuration of design 2, and moments
J)2
J)3
12 2
E x
u=l u
12
E
u=l
x
4
u
12 2 2
E~x
u=l
u 2u
,46658851
,84717462
.04572007
3,750000
2,250000
1.250000
,46884063
,85135864
.10826207
3.825375
2,295225
1.275125
.47097069
,85546092
.14747279
3·901500
2.340900
1.300500
.47296960
,85947673
.17942483
3,978375
2.387025
1.326125
.47482705
,86340068
,20755371
4.056000
2.433600
1·352000
.47653127
,86722664
.23330164
4.134375
2.480625
1.378125
.47806868
.87094756
.25744063
4.213500
2.528500
1.431125
,48057743
.87804024
.30263951
4,374000
2.624400
1.458000
.48150850
,88139119
,32425742
4.455375
2,673225
1,485125
.48219063
.88459471
.34548430
4,5375 00
2·722500
1·512500
.48259206
.88763455
.36647451
4,620375
2.772225
1·540125
,48267362
.89049078
,38736591
4.704000
2.822400
1·568000
.48238606
.89313846
.40829048
4.788375
2.873025
1.596125
.48166601
.89554573
.42938410
4.873500
2.924100
1.624500
.48042966
.89767068
.450 79756
4,959375
2,975625
1.653125
.47856220
.89945638
.47271179
5.046000
3·027600
1.682000
,47589923
.90082189
.49536209
5·133375
3.080025
1·711125
.47219054
.90164511
,51908303
5·221500
3.132900
1,740500
.46702127
.90172549
.54440428
5·310375
3.186225
1,770125
.45960553
.90068710
.57229845
50400000
3,240000
1,800000
.44804725
.89763374
.60506287
5,490375
3.294225
1.830125
042315481
.88829049
,65364747
50581500
30348900
1.860500
.J
e
e
92
16
(6.69)
16 4
:xi
=
x
= 8ci +£2)
u=l u u=l' 2u
1
2
L;
L;'
(6.70 )
Result 5 yields the system
(6.71)
However (6.72) can be written
(6.73)
or
(6.74)
which means, from (6.71)
Equations (6.74) and (6.75) are exactly the same conditions as the
least squares requirement yields, since least squares requires
(6.76)
(6.77)
(6.78)
where (6.78) is simply (6.76) squared.
Hence for this 42 design the
93
least squares solution is also the non-least squares solution.
This
solution is
or
£1
= .79465447
£2
= .18759248
However, suppose one (or more than one) center point at (0,0) is added
to the design.
Then the above result will, of course, still satisfy
(6.55) but is no longer a least squares solution;
i.~.
adding a center
point (s) makes a least squares design impossible since the least
squares equations (for one center point)
(6.80)
8(£i +
£~) = 17/5
(6.81)
are inconsistent.
Design 4:
.(6.82)
Consider adding a zero level to the 42 yielding a 52
94
(6.84)
and the conClitions to be satisfied are
(6.86)
As for the 42 factorial, this design, given by
Jl
=~(1/l2)(5 +l/li)
J2
=~(1/12)(5
J
=:
.69305207
=:
.14028127 ,
--vu)
or
J
1
2
coincides with the least squares minimum bias design.
However, if an
additional center point is included the design satisfies result 5 but
there is no least squares design in this case either.
Design 5:
Suppose four experiments, at (£3,0), (0'£3)' (-£3,0)
and (0'-£3) are added to the 52 factorial.
must be satisfied are
(6.88)
Then the equations which
95
In order for a solution to represent an admissible design, as previously defined, the solution must satisfy
5/6 < 1,2 + 1,2 < 1 ,
-
2 -
1
since
Table 6.2 gives admissible designs, and moments, obtained by varying
J,i + J,~ from .84 to 1.0 in steps of .01. The least squares design
requires J,i + J,~ = -{2§/6 and here 1,1 = .85184513,
= .41459298,
1,2
1,3
= .58795918 is the design.
Example 2:
•
This example is presented to show the effect of a
change in the region of interest on the protection design conditions •
Consider the same set-up as in the first example with R now being
defined by the circle of radius r and center at the origin, so that the
region of interest is 0
222
+ x ~ r .
2
~ ~
The only changes in the estimators for k=l and k=2 occur in the
"A" matrices. In this case
(6.90)
Wl
=
1
0
0
0
2
r /4
0
0
0
2
r /4
,
and when k=l
r 2 /4
(6·91)
so that
2
r /4
0
o
o
o
o
o
o
,
96
.e
Table 6.2.
Designs and moments for design 5
29[1 4]
29!1 222]
8.4672
5.08032
2.8224
.29154759
8.6700
5·20200
2.8900
.38483912
·37094474
8.8752
5.32512
2.9584
.84683347
·39098906
.43749286
9.0828
5.44968
3.0276
.84927605
.39840958
.49638695
9.2928
5.57568
3.0976
.8510222.3
.40713778
.55009092
9·5052
5·70312
3.1684
.85201806
.41721125
.60000000
9.7200
5.83200
3.2400
.85219420
.42867825
.64699304
9.9372
5.96232
3·3124
.85146084
.44160440
.69166466
10.1568
6.09408
3.3856
.84969984
.45608134
.73443856
10.3788
6.22728
3.4596
.84675224
.47224002
.77562878
10.6032
6.36192
3.5344
.84239699
.49027269
.81547532
10.8300
6.49800
3·6100
.83631231
.51047205
.85416626
11.0592
6.63552
3.6864
.82799841
.53330914
.89185201
11.2908
6.77448
3.7636
.81659967
.55961146
.92865494
11.5248
6.91488
3.8416
.80039637
·59107161
.96467611
11.7612
7.05672
3.9204
.77459667
.63245553
1.0000000
12.0000
7·20000
4.0000
£1
£2
£3
.83569958
.37630600
.18330303
.84001712
·37996216
.84373643
29[1 2]
6
e
I-
97
(6.92)
A:L=
r 2 j4
1
0
0
lj4
0
1
0
0
0
0
0
0
1
0
0
0
0
t
~l
=
,
~ll
,
~l
When k=2
2
(6.94)
W2 =
j4
r 2 j4
0
0
0
0
0
0
0
0
r 2 j8
0
4
r j24
0
0
0
0
0
r 4 j24
0
r 4 j8
and therefore,
0
(6.95)
A2 = r 2 j2
0
0
0
0
0
r 2 j6
0
r 2 j6
0
r 2 j2
I
~2
=
t
~12
I
~2
II
II
Then proceeding as in example 1, and using the new A matrices, leads
to:
Result 6:
oS
Su:f'fici~nt conditions for T~
= T~
when R is
xi + x~ S r 2 are
[12 ]
= [22]
[1 4 ]
= [2 ]
[1 4]
=!... [12 ]
[1222 ]
=~
4
2
2
2
2
[1 ]
,
and the XiX matrices in (6.32) and (6.51) are non-singular.
Note that the least squares conditions [12 ] = [22], [1 4 ] = [2 4],
4
[12 ] = r 2 /4, [1 ] = r 4/8, [1222 ] = r 4/24 are a special case of (6.96)
99
7.
SUMMARY
The problem of exploring polynomial response surfaces in the
presence of assumed model inadequacies has been considered.
Over the
entire region of factor operability it was assumed that the true
response is a polynomial of degree d+k-l.
of experimental interest it was
ass~ed
However, within the region
that a polynomial of degree d-l
would be used as an estimator of the true response.
In this setting the
important statistical proplems which arise are those of designing experiments and choosing estimators for fitting the response surface.
It was
argued, in Section 2, that optimality criteria, to be useful in attacking these problems, must be meaningful in terms of the underlying problem and also reflect the association between the two problems.
The criteria chosen were 1) minimum squared bias integrated over
the region of interest, 2) minimum variance integrated over the region
of interest and 3) protection against higher order effects, meaning
satisfaction of 1) and 2) for various higher order true models.
Because of the relationship between the design problem and the
estimating procedure, it was decided to initially apply the criterion
considered most important, the bias criterion, to the estimation problem.
Subject to satisfying the bias criterion independently of the
unknown parameter values in the true model, estimators are considered
optimum if the variance criterion is also satisfied.
developed in Section 3, to the (min V
I
This led, as
min B) estimator.
This esti-
mator, given the degree of the assumed true model, satisfies the bias
and variance requirements for any experimental design.
'Ihis result
100
enables minimum B, and subject to this, minimum V, to be attained
without any restrictions on the design moments.
Prior to this outcome
minimum B could be reached, but only if certain design moment requirements were met.
This is because the previous works conf'ined attention
to the least squares estimation procedure, a somewhat questionable
criterion in the presence of
re~ognized
bias, as
o~scussed
in Section 2.
Using least squares estimators, the bias criterion can be satisfied
only by imposing restrictions on the design.
It was shown that this
arises as a special case of the more general development.
It is at this stage,
!.~.
after an estimation procedure has been
derived, that additional criteria should be applied to distinguish
between different experimental designs.
Given that the (min V
I
min B)
estimator is to be used, the protection criterion was introduced in
Section
4. When applied to the experimental design problem, this pro-
tection criterion yields, by choice of design, the same estimator for
different higher order effects in the true model.
This procedure in-
volved determining design moment conditions which resulted in the same
estimator being used for different bigher order models.
In Section
4,
for one independent variable, design moment conditions were derived for
using linear, quadratic and cubic polynomial approximating functions and
protecting against certain higher order effects up to quintic.
sons with the least squares minimum
b~as
Compari-
design conditions were made and
it was found that greater flexibility for the values of the pertinent
moments could be achieved by using the non-least squares estimator.
Where, for example, the second moment must always be 1/3 to attain
minimum B using least squares, no such restriction was required for the
101
(min V
I
min B) procedure.
Similar results also pertained to higher
moments, reflecting greater freedom in the design moment conditions.
In Section 5, using the elementary symmetric functions of the roots
of polynomials, actual experimental designs were generated for the design moment conditions derived in Section 4.
Some of the difficulties
inherent in resolving the resulting systems of non-linear equations into
actual experimental designs were evident in this section.
Complex
valued factor levels and levels lying outside the region of interest are
inadmissible as applied to experimental designs.
However, for the
experimentally meaningfUl situations, detailed results giving combinations of design levels satisfying the requirements were obtained and
displayed in Figure 5.1 and Tables 5.1 and 5.2.
This section was of
importance in demonstrating how the derivations may be applied to experimental situations.
Generalizations of both the estimator and design conditions were
made, in Section
6,
for the multi-factor case, with examples of the de-
sign problem given for two factors with a linear estimator and protection against quadratic and cubic effects.
When the design problem is
considered for more than one independent variable the choice of the
region of interest and the experimental design configuration both offer
additional flexibility.
In the exam.ple considered, the effect on the
moment requirements, of changing the region of interest from a unit
cube to a unit sphere was seen by comparing results 5 and 6, in Section
6.
For the cube the fourth moment must equal 3/5 of the second moment
and the fourth order product moment must be 1/3 the second moment.
..
the sphere these values were 1/2 and 1/6 respectively.
For
The effect on
102
the problem, of different configurations was considered, and specific
designs were tabulated.
In summary then, the development presented in this work has
employed an approach to a particular response surface problem involving
model inadequacies which enables a certain amount of flexibility over
the earlier approaches.
The experimenter can guard, in the (min V
I
min B) sense, against a particular higher order efi'ect by using the
appropriate estimator with any experimental design, and can protect
against a number of higher order effects by choice of design.
It is hoped that future work in this problem domain will closely
relate both the design and estimation problems and
problems to underlying problem Objectives.
rel~te
both these
More specifically future
work can be suggested in the following areas:
1)
Extensions of the derivation of protection conditions on the
design moments to multi-factor set-ups involving more than two factors.
It is conjectured that the results for two factors will extend "naturally " •
2)
Relaxation of certain symmetry requirements.
With modern
computing facilities available it may be desirable to explore the
effect of not requiring certain odd moments to vanish.
All of the
theory and general approaches in this work will still be applicable in
such non-symmetric designs.
3)
Changing the form of the true model and/or estimator from
polynomials to other kinds of funct+ons.
In this realm also it would
be interesting to change the regions of interest and apply the procedures to different combinations of true models, estimators, and regions.
103
.e
4)
Introduce prior information on the unknown parameters in the
Equation (3.36) gives the value of minimum B actually
true model.
attained, as a quadratic form in these parameters.
a
If it were pertinent,
prior distribution could be attacbed to these parameters and addi-
tional optimality criteria introduced.
5)
Applying other forms of bias, variance and protection criteria.
ClearlyJ the procedures and approaches in this work could be used with
other criteria for estimators and designs.
The literature review, in
Section 2, contains many criteria which may be useful and of interest
in a particular problem setting.
6)
Relating optimality criteria to cope with the wide choice of
design configurations.
This would be appropriate in the multi-
dimensional set-up and for example could lead to certain classifications
of designs, based on the criteria used.
7)
Weighting functions emphasizing different subspaces of the
factor space.
This essentially incorporates into the optimality
criteria the concept of prior information on the relative importance of
different sub-regions in the resion of interest.
8)
obta+ned.
Extend the values of d and k for which designs were actually
This suggestion may be of mQre a.cademic or numerical interest
than of experimental value.
The question of convergence of the derived
design moment conditions to the lea.st squares requirements for larger
values of d and k is included here.
104
8.
t
LIST OF REFERENCES
Anderson, R. L. 1953. Recent advances in finding best operating
conditions. J. Am. Stat. Assoc. 48:789-798.
Box, G. E. P. 1954. The exploration and exploitation of response
surfaces: some general considerations and examples. Biometrics
10:16-60.
Box, G. E. P., and N. R. Draper. 1959. A basis for the selection of a
response surface design. J. Am~ Stat. Assoc. 54:622-654.
Box, G. E. P., and N. R. Draper. 1963. The choice of second order
rotatable designs. Biometrika 50 :335-352.
Box, G. E. P., and J. S. Hup.ter. 1957. Multi-factor experimental
designs for exploring respQnse surfaces. Ann. Math. Stat. 28:195241.
Box, G. E. P., and K. B. Wilson. 1951. On the experimental attainment
of optimum conditions. J. Roy. St~t. Soc. B13:1-45.
Chernoff, H. 1953. Locally optimal 4.esigns for estimating parameters.
Ann. Math. Stat. 24:586~602,
David, H. A., and B. E. Arens. 1959. Optimal spacing in regression
analysis. Ann. Math. St$.t. ;30:1072-1081.
de la Garza, A. 1954. Spacing of information in polynomial regression.
Ann. Math. Stat. 25:123-130.
Draper, N. R. 1963.
5:469-479.
Ridge t;l.nalysis of response surfaces.
Technometrics
Elfving, G. 1952. Optimum allocatio~ in linear regression theory.
Ann. Math. Stat. 23:255-262.
Folks, J. L. 1958. Comparisons of designs for exploration of response
relationships. Unpublished Ph.D. thesis, Department of Statistics,
Iowa State College, Ames, Iowa.
Friedman, M., and L. J. Savage. 1947. Planning experiments seeking
maxima, pp. 363-372. In C. Eisenhart, M. W. Hastay, and
W. A. Wallis (ed.), Selected ~echniques of Statistical Analysis.
McGraw-Hill Book Co., Inc., New York.
<
Hoel, P. G. 1958. Efficiency problems in polynomial regression.
Math. Stat. 29:1134-1145.
Ann.
105
Hoel, P. G. 1961. Some properties of optimal spacing in polynomial
estimation. Ann. Iust. Stat. Math. (Tokyo) 13:1-8.
Hoel, P. G., and A. Levine.
polynomial prediction.
1964. Optimal spacing and weighting in
Ann. Math. Stat. 33:1553-1560.
Hotelling, H. 1941. The experimental determination of the maximum of
a function. Ann. Math. Stat. lZ:20-45.
Householder, A. s. 1953. Principles of Numerical Analysis.
Hill Book Co., Inc., New York.
Kiefer, J. 1959. Opti~ experimental designs.
(London) B2l:272-304.
McGraw-
J. Roy. Stat. Soc.
Kiefer, J. 1961. Optimum experimental designs v, with applications
to systematic and rota~able designs, pp. 381-405. In J. NeYman
(ed.), Proceedings of the Fourth Berkeley Symposium-on Mathematical
Statistics and Probability, Vol. r, University of California Press,
Berkeley and Los Angeles.
Kiefer, J., and J. Wolfowitz. 1959. Optimum designs in regression
problems. Ann. Math. Stat. 30 :271-294.
Lawrence, W. E. 1964. Experimental designs which minimize model
inadequacies. Unpublished Ph.D. thesis, Department of Statistics,
University of Wisconsin, Madison, Wisconsin.
Manson, A. R. 1966. Minimum bias designs for an exponential response.
Oak Ridge National Laboratory, Oak Ridge, Tennessee, ORNL-3873.
106
9. APRENDICES
9.1 Some Algebra Leadins to (4.82)
Equations (4.83) and (4.84) are
For each of handling, let a
~
[12 ], b
= [1 4],
(4.84) become
(9.1)
2
(b -ac)/(b-a2 )
(9.2 )
(c-ab)/(b-a2 )
= -3/35
= 6/7
which yield
or
ac - c/10 = b2
-
ab/10
or upon solving for c,
(9.3)
c = b(10b-a)/(10a-1) J
Putting (9.3) into (9.1) yields
which reduces to
e
,
(9·4)
b = (3/35)(10a-1)
a
i
~O.
c
= [1 6].
Then (4.83) and
107
substituting (9.4) in (.93) yields
c = (3/35)(10a-1)(lQb-a)/(10a-1)
= (3/35 )(10b-a)
= (3/35)[(6/7)(10a-l)-aJ ,
which becomes
9.2 Some Algebra Leadins to (4.104)
i
i
8
As in 9.1, set d = [1 J.
Then (4.105) is
This becomes
Substituting (9.4) and (9.5) into the terms of (9.7) yields
(9·8)
(3a/5)-b = (3a/5) :: (3/35)(10a-1)
= "'(3/35)(1-3a)
(9.9)
ac-b 2
= (3a/g45)(53a-6) - (3/35)2(10a-1)2
= -(3/35)(a2 - 6a/7
(9.10)
+ 3/35)
c(c-3b/5)= (3/245)(53a-6)[(3/245)(53a-6) - (9/l75)(10a-1)J
=
[9/(245)(1225)] (2915a2 - 807a + 54) ,
and combining (9.8), (9.9) and (9.10) yields, from (9.7)
108
d = -(317)(3/35)(a2-6a/7+3f31.)~(t/(24i)(1225)](2915a2-807a+54)
. 3 35) 1-3a .
-
(1380a2- 61 9a453)
3a-i
=
(9/8575)(3a-1)(460a-53)/(3a-1)
_ 19/(7)(1225)]
= (9/8575)( 460a-53) •
(9.11)
9.3 Some Algebra Leading to (4.127)
As in 9.1 ~nd 9.2, with a = (12], b = [1 4], c = [16], d
.
;
i
=
(18],
(4.128) and (4.129) are
(c2-bd)/(ac-b2 )
=-5/21
(ad-bc)/(ac-b2 )
= 10/9
,
= c(14c-3b)/(14b-3a)
•
which yield
or
and solving for d yields
d
Substituting (9.14) in (9.12) yields
c2-bc(14c-3b) /(14b"3a)
=-(5/~1)( aC-b2 )
,
which reduces to
e
.,
Solving for d in terms of a and b, from (9.15) and (9.14) yields
I
•
109
which reduces to
t
••