F
'F
MINIMJM BIAS ESTIMATION .AlfD EXPERIMENTAL DESIGN APPLIED
TO UNIVARIATE roLYNOMIAL M)DELS
by
A. R. Manson, R. J. Hader, M. J. Karson
Institute of Statistics
Mimeograph Series No. 826
Raleigh - 1972
•
MINDIJM BIAS ESTIMATION AND EXPERIMENTAL DEEIGN APPLIED
TO UNIVARIATE POLINOOIAL MODELS
A. R. MANSON AND R. J. HADER
Department of Statistics
North Carolina State University
Raleigh, North Carolina
M. J. KARSON
Graduate School of Business Administration
Universtty of Michigan
Ann Arbor, Michigan
Minimum. bias estimation, as introduced by the authors in [4], is
applied to those combinations of univariate polynomial responses and
polynomial approximating functions of lower degree which are of primary
practical importance.
R:
Designs in the experimental region of interest
(-1, +1] which give smallest integrated variance V are presented.
Examples of contour plots of constant V versus design levels: show
regions of experimental designs which give smaller integrated mean
square error than is obtained by using standard least squares
estil!lcltion for designs which attain minimum. bias B via choice of design
moments.
KEY WORDS
f
Minimum. Bias Estimation
Experimental Design
Polynomial Models
Polynomial Approximating Functions
•
... 2 1.
INTRODUCTION
In the approximation of response relationships, several authors
have taken both variance and bias errors into consideration (for
example, see
[11, (21, [3], and (4). In a previous paper [41,
minimum bias estimation of a polynomial response model was introduced.
The present paper concerns itself with the design implications
associated with minimum bias estimation in the univariate case.
It is necessary to review briefly the notation and results of r41,
where it is assumed that the response variable
~
is related to an inde-
pendent variable via a polynomial of degree d + k - 1,
in which lS~ is a row vector containing p0l<TerS of x up through order
d - 1; lS~ is row vector containing powe:cs of x from order d through
d + k - 1; and
~l
and
~2
regression coefficients.
are column vectors of the corresponding
Equation (1.1) is to be approximated by
using observations of the response variable taken at selected levels
of the single, ind.ependent variable x to fit the polynomial of degree
d - 1
y(~)
.-
= ,...,);:,1
x'b
.
(1.2)
The estimation criterion adopted is that of minimiZing the squared
bias integrated over a specific region of interest R in the independent
variable, i.e., it is desired to minimize
(1.3 )
•
- 3 \
Furthermore, subject to minimizing B, the estimation method used should
then minimize the integrated variance of the fitted polynomial y(x),
NO S
,.
V =:2
R Var[Y(~ldx
,
(1.4)
(j
for any fixed experimental design.
In the univariate case presented here, the region of interest R
will be the interval [-1, IJ on x.
(1.3)
and
(1.4)
The symbol
a
used in equations
is defined via its inverse as
(1
..1
= JRdx.
The primary criterion selected is motivated by a desire to use
simple low order polynomial approximations to estimate response
relationships but to achieve some measure of robustness or protection
against the possible existence of higher order polynomial terms.
The
expected mean square error integrated CNer R is simply the sum of V
and B (1. e., J
=V
+ B).
Minimization of this sum requires advance
knowledge of the regression coefficients which is not generally
available.
Results obtained by Box and Draper in (11 and [2J
indicate that unless V is quite large with respect to B, optimum
designs for minimizing the sum of V and B are very close to those
obtained by assuming that V
= O.
Their approach used standard least
squares estimation in the approximating polynomial Y(x).
In view
of the dominating importance of B, we have chosen to use a method of
estimation aimed directly at minimization of B.
Another reason for
desiring to minimize B comes from a numerical analysis viewpoint.
one approximates a polynomial response by
another polynomial of
If
•
- 4lower degree, then it seems reasonable to use the best approximating
polynomial, apart from experimental error considerations.
In (4J it is shown that an estimator of the form
b1 = A(X'XrX'1,
......
(1.6)
will minimize B and subject to giving minimum B will minimize V for any
fixed design.
In equation (1.6) the matrix X contains the full set of
powers of x in (1.1) for the given design; ......
y is the column vector of
observations on the response variable; and
where I
d
is a d by d identity matrix,
(1.8)
and
By the integration of a matrix, we in:g;>ly integration in an element by
element sense.
The notation
(x'xf
used in equation (1.6) designates
the ordinary inverse of XiX if it is non-singular and the generalized
inverse if XiX is singular.
If XiX is singular, then·it is required
that
be estimable for a given design to be "admissible".
inverse satisfying the single requirement
Any generalized
•
- 5 -
(x/x){x'xf (XiX) .. XiX
will suffice to make the estimator
.21
,.
Furthermore" for Y' and
unique.
11
which are polynomials 1n x it can be shown that W 1s a positive ..
1
definite matrix and therefore bas an ordinar1
exists for all
adaiasib~e
inv~rsej
thus
w~lw2
designs.
,.
The estimator Y' = ~~l with.el given in (1.6) is referred to as the
-
mini.Jm.Jm bias estimator for
11- It obviously does not
depend on advance
knowledge of the true regression coefficients of (1.1), it is unique"
minimizes B and subject to giving Min B gives min.:1.mum V for any fixed
design for which
~
is estimable" and contains the method of standard
least squares estimation as a speciaJ. case when
The same minimum B is attained for any design for which .AfJ
,.., is
estimable"
name~
(l.lO)
where
(l.ll)
By the standard least squares estimator we mean
(J..12)
where Xl is that part of the X .. (X :X ) matrix which involves only the
1 2
values of terms in ~i.
Box and Draper [1] showed that Min B is attained
:.".",
: ".
•
- 6 using the standard least squares estimator and requiring that the
experimental design used have moments satisfying the requirement
Designs which satisfy (1.1.3) are special cases of the larger class of
./
designs for which ~
,..., is estimable, for which the same Min B is attained
using minimum. bias estimation.
The additiona! design flexibility obtained is one of the main
advantages of the minimum bias estimation method.
One of the more
obvious ways of using this flexibility is to choose designs which will
minimize V aver all admissible designs.
obtain a smeller value of
intei~rated
In addition, it is possible to
mean sCJuo.ce error J using minimum
bias estimation than is obtained using standard least squares
estimation for those designs which satisfy the Box-Draper requirement
given in equation (1.1.3).
In the following sections we shall treat the design problem of
minimizing V aver the class of admissible designs for the 8i tuations
where (d
= 2;
k
= 1,
2,
The model situation (d
3),
= 2;
(d
k
= 3;
k
= 1,
= 1,2,3)
2), and (d
= 4;
k
= 1).
implies that y(x)
,..., is a
linear polynomial in x while 11 is either a quadratic polynomial in
x (k = 1), a cubic polynomial in x (k = 2), or a quartic polynomial in
x (k =
3).
Our general procedure shall be:
using the minimum bias
estimator find the design which minimizes V aver the class of
admissible designs, i. e.,. those for which JfJ
,..., is estimable.
•
- 1 2.
LINEAR APPROXIMATING POLYNOMIAL
This section is concerned with situations where the linear
approximating polynomial (d
= 2)
(2.1)
is expected to give a reasonable fit to the true response.
Three polynomial responses will be investigated; quadratic (k
cubic (k
= 2),
and quartic (k
= 3).
= 1),
To specify a particular
approximating polynomial, true response, and design setting, a triple
of the form (d, k, N) will be used, e.g., (d, k, N)
= (2,
1, 4) will
,.
be used to designate a situation where y is of degree d - 1
(2.1),
~
is of degree d + k - 1
=1
as in
=2
(quadratic) where k represents the
,.
difference in degree of the polynomials for y and ~, and N = 4 is the
total number of observations of the response.
2.1
Quadratic Response
(k = 1)
Tbe true response is assumed to be of degree 2 (quadratic) of tbe
form
We shall restrict our attention to designs symmetric about the
origin in terms of the x , in determining those designs which will
u
minimize V given in equation (1.4).
The expression for V may be
simplified since
,.
I
I
2
Var y = ~lA(X X) A/~CT
."
to give
•
- 8 V =
2'N
f1
I
( ,
)- I
-1 ~1A X X A ~1dx
= N trace
If XiX is non-singular, then for (d, k, N)
I
-
I
[A(X X) A W1 ) •
= (2,
1, N)
v = 9c-6a+l + 1.. .
2
9(c-a )
(2.1.4)
3a '
where
a
=
N
t
u=1
2
x u/N
and
c
=
N
4
1: xu /N
u=1
.
Note that non-singularity of XiX requires that 0 < a
real valued x.
u
2
2
< c s Na /2 for
The expression for V given in (2.1.4) may be verified
by evaluating equation (2.1. 3) using the appropriate matrices, A,
(X/X,-~ and WI which are given in Section 1 of Appendix A.
To determine the Xu for an N point design so as to minimize the
V of equation (2.1.4) one simply minimizes V over a and c.
the equation
~: = 0
However,
gives (3a - 1)2 =·0, which does not contain c.
Therefore, V is minimized (for any value of a) when c is either at its
maximum value or at its minimum.
and will in general give V
a
= 1/3
= co
If c
= Max
Designs which have c
value of x u at +.1"
c
then XiX will be singular
(except in the special case of
which wi 11 be treated below).
a and c occurs when c
= a2
Hence, }liin V with respect to
= lile?"! 2..
= Na 2/2
must have N. - 2 of. the Xu at zer~, 0r...e
and one value of x u at -t where t 2=Na '2.
I
I.n fa.ct
•
- 9 Min V
c
1
fN-61 -1
= M[N
+31 a
+
9"2
a
-2
1,
which can be minimized with respect to a by the usual techniques to
give
for 2 ~ N ~ 5
Min V = (18-N)/8
a,c
(2.1.8 )
which occurs when a = 4/(18-3N).
For N ~ 6, V possess no relative minima for finite values of a, in
which situation one should choose the maximum. possible value of a in
order to decrease V.
In the limit as a approaches infinity
Min V
a,c
= HI (N-2)
•
If one wishes to confine the experimental points to the region of
interest R: (-1, 1], then
Min V = 4(:~N-4 )/9(N-2)
a,c
for N ~ 6 .
In situations where XiX is singular (for non-trivial designs),
the singularity occurs when c = a
when N
=2
2
(which will always be the case
and occurs for N > 2 whenever only two of the design points
are distinct).
These situations require a =
I/J,... and such a value of a gives V = 2.
1/3
for estimabili ty of
Designs having \(C' = a =
i
1/3
must have all Xu equal in magnitude and thus have all non-zero levels
of x
u
at +.t.
-
These designs will satisfy the Box-Draper condition of
(1.3.3) and hence the minimum bias estimator .21 will coincide with the
*
,.
standard least squares estimator Eol for this y, 11 combination whenever
XIX is singular.
•
- 10 Table 1 gives the values of Min V for 2 s: N s: 7, and all designs
gi ven in Table 1 have N - 2 values of x
and one value of Xu at
at zero, one value of x
u
u
at
+1.,
-1..
Figure 1 shows how V varies with the single non-zero design level,
TABLE 1
Design Parameter Values Which Minimize V
for (d, k, N)
c
= «&2/2
N
a
,
1/3
1/q
4/9
81Z(
2
4
5
6
7
2/3
4/3
co
co
8/9
40/9
ro
co
1.2
= (2,
1, N)
Absolute Min V
2.000000
1.875000
1·750000
1.625000
1·500000
1.400000
1/3
2/3
4/3
10/3
co
co
Min V in R:
2.000000
1.875000
1.777778
1.629630
1.555556
1·5ll1ll
[-1, 1]
(1.=1)
(1.=1)
(1.=1)
(1.=1)
Note that for N ~ 4, the absolute Min V does not occur for a design in
R, in which case the design having smallest V in R is given.
For N s: 5, designs which were non-symmetric in the x
u
were
investigated to see if smaller values of V than those given in Table 1
could be obtained.
The results showed that no non-symmetric designs
exist which will give smaller V than the best obtained for the
corresponding symmetric designs of N observations.
If one were to use the standard least squares estimator
l
)-lx1: 1
11..
b* = (XiX
-1
and
...*
,*
y =!Jl>J.'
•
- II -
2.6
2.4
v
o
2.0
7
N=2
1.8
1.6
!
0·5
J..l
J,
Figure 1
V versus the design level t for (d, k, N) = (2, 1, N)
•
- 1.2 -
then
which can be shown by wri ting out the elements of the above matrices in
summation form.
Designs which satisfy the Box-Draper conditions for
Min B given in equation (1.13), via the sufficient conditions
always have
For d
=2
if equation (1.13) is satisfied, in any way, then V*
= 2.
However, for d > 2 it is possible to achieve V* < d for designs satisfying
equation (1.13).
(d, k, N)
= (2,
For the particular estimator-model situation
1, N), it is necessary that a
design to have moments satisfying (1.13).
= 1/3
in order for a
Such designs lie outside R
for any N ~ 4 and hence no design exists in R for N
Min B using standard least squares estimation.
~
4 which will give
For (d, k, N)
1t is always possible to find designs in R with V < (V*
= 2)
= (2,
1, N)
for N ~ 3.
In fact, Figure 1 shows an infinity of designs for each N ~ 3 for
which V < 2, so that is is not necessary to choose the design level
t so as to minimize V.
v<
Rather, one should choose t in a range to give
2, and to satisfy whatever other design cri teria one might have.
This additional flexibility is available in addition to attaining Min B
and V < 2 which means that it 1s possible to obtain smaller, integrated
mean square error, J
= V + B,
than is possible using any design which
•
-
~
-
will satisfy the requirements of (1.13) wi. thout requiring any advance
knowledge of the parameters
2.2
Cubic Response (k
f:.1
or
~.
= 2)
The true response is now assumed to be a polynomial in x of degree
, (cubic), of the form
Methods analagous to those used in Section 2.1 give
5 ;9
v = 2c -6a;1 + 2 e-30c a = N tr[A(X/X)-A/W )
9-(c.•a. )
for (d, k, N)
=
l
75(ae-c)
(2, 2, N) when XiX is non-singular, where a and c are
defined in (2.1.5) and (2.1.6) and
N
6
e = 1: x uIN
u=l
See Section 2 of Appendix A
equation (2.2.2).
•
(2.2.3 )
for the appropriate matrices used in
The method of minimizing V over choice of design is
similar to that used in Section 2.1.
It is desired to determine the
values of the design moments a, c, and e which minimize V.
For the x
u
levels of the design to be real valued, the range of c is dependent on
the value of a and similarly, the range of e is a function of both a
and c.
However,
contain e.
~~
= 0 implies that (5c ..... 3a)2
=0
which does not
Hence, Min V occurs either when e is at its maximum value
(in terms of a and c) or at its minimum value.
maximum e gives Min V.
one must maximize e.
A check shows that
Thus, to minimize V for any values of a and c,
This can be accomplished by symmetric designs
which have one value of Xu at each of
.:!: 1- 2, m/2 values of Xu at each
•
- 14 -
2 = 2m
N {
tt.
(2.2.4)
a-
and
m
= (N-2)/2
if N"is an even integer
~
= (N-3)/2
if
N is
an ~~d integer.
At this point minimization of V over a and c becomes intractible
by analytic methods.
AccordinglY, numerical optimization methods were
used to determine the values of a and c (or alternativelY t
which give Min V for N
=4
Min V designs outside of R:
through N
= 15.
l
and t )
2
For values of N which have
[-1, IJ, the design in R which gives
smallest V is listed in Table 2, although such a design may not have the
"optimum allocation" of design points required for absolute Min V.
Table 2 also lists the designs in R which have singular XiX (1. e.,
rank
= 3),
allow estimation of Af', and give smallest V among such
~
* = 2.
designs, provided such smallest V is less than V
Those designs
which have XiX of rank 3 would not allow fitting the assumed true,
cubic response by a full cubic pOlYnomial and hence a pOlYnomial of
lesser degree would be required.
Minimum bias estimation allows
pOlYnomial modela whi ch can not be approximated by pOlYnomials of the
same degree to be fitted by lower degree polynomials and still obtain
•
- 15 Min B, a value of V < (V* = 2), and a lower value of integrated mean
square error J than is obtained using standard least squares estimation
for designs satisfying the conditions of equation (1.1-').
TABLE 2
Designs in R with Smallest V for (d, k, N)
H
NO
N
1
.t1
N
2
1,2
4
5
6
6
7
7
8
8
9
9
10
10
11
11
12
12
1-'
1-'
14
14
15
15
0
1
2
0
3
1
2
0
3
.1
4
0
3
1
4
2
5
1
4
0
5
1
1
1
0
2
0
2
0
3
0
3
0
4
0
4
0
4
0
4
0
5
0
5
0.349742
0·534446
0.000000
0.451919
0.000000
0.54li666
0.000000
0.491261
0.000000
0·553197
0.000000
0.51-'658
0.000000
0.56<:691
0.000000
0.6<:6025
0.000000
0.498492
0.000000
0.474860
0.000000
0.;12170
1
1
2
1
2
1
3
1
3
1
3
1
4
1
4
2
4
2
5
2
0.833343
0.856963
0·774597
0·907502
0·774597
0.916919
0·774597
0.962052
0·774597
0·965593
0.774597
1.000000
0·774597
1.00000o
0·774597
0.997966
0·774597
0.914725
0·774597
0·936400
0·774597
0·941-'00
*
5
2
= (2,
2, H)
V
1.886561
1.886601
1.888889
1.885022
1·973251
1.885494
1.94~87
1.883965
1.888889
1.884574
1.934156
1.883485
1·912551
1.884187
1.888889
1.884689
1.917901
1.885170
1·90l235
1.884461
1.888889
1.884672
Rank of X'X
4
4
3*
4
'*
4
3*
4
3*
4
3*
4
3*
4
3*
4
3*
4
3*
4
3*
4
Designs having singular XiX matrices
While Table 2 lists the single designs in R having smallest V,
there are, for each value of H, an infinite number of designs located
in a region in the Cel' £2) plane surrounding the optimal design for
which V < (V*
for N = 4.
= 2).
Figure 6 of [4] shows the contours of constant V
Any design characterized by values of Cel' £2) ic.aide the
•
- 16 -
v =2
contour will give a
s~er
integrated. ill.ean square error· thani!
standard least squares estimation were used and the design were required to satisfy the Box-Draper conditions for Min B given in
equation (1. ~). See Table Bl of Appendix B for addi tiona"l Hin B design.
If one desired to use the add.! tional flexibility obtained using
the minimum bias estimator to achieve equal spacing of design levels
(for example) then it is possible to minimize V along the line
t
2
=
3t
l
(for N even) or along t
2
= 2.t
l
(for N odd).
See [4J for an
example.
Table 2 presents only the optimal designs for Min V.
There are
alternative allocations ot design points which lead to values of V
* = 2.
which are less than V
For example, if N
= 8,
could have been obtained by taking two points at x
each ot x
= !tl = 0.640560,
which would have given V
trom Min V
= 1.883~5
a symmetric design
= 0,
and one point at each of x =
= 1.886635
three points at
!t 2
=
0.899906
which does not differ appreciably
given in Table 2.
Hence, it is possible to use
non-optimal allocation ot the N design observations and still obtain
a
large class of designs for which V < 2 and which therefore will have
smaller integrated mean square" error than designs satisfying (l.~).
",
2.3
Quartic Responses (k
= 3)
The fitting ot a quartic po).ynomial response of the form
(2.3.1)
,.
by a linear y is ot somewhat less interest from a practical viewpoint.
However 1 it is presented tor completeness.
•
- 17 The expression obtained for V is quite lengthy and will be
om1 tted since minimization of V was accomplished numerically.
gives the matrices (X /
3 of ~AppendiX A
xrl,
Section
A, end W required to
1
evaluate V for non-singular XiX.
Table 3 gives the symmetric designs in R: [-1, 1] of the form NO
= 0,
observations at x
N observations at each of x
l
observations at each of x
= .! 1,2'
XiX matrices of full rank (5).
= !.1IJ! and
N
2
which minimize V and which have
The table also contains designs in R
giving smallest V for singular XiX matrices of rank ; and 4 provided
* = 2.
such smallest V is less than V
The conditions on the non-zero
levels of x for estimabili ty of }fJ
,... in cases of singular X' X matrices
are:
ti =0
Note that for N
and
= 4,
* =2
compared to V
.t~ = 3/5
for XiX of rank; •
the best four point design gives V
= 1.904555
.
for the umque four point design which will
satisfy equation (l.l,}) having 1,1
= 0.18759;
and 1.
2
= 0.7g(654.
If one
were to use minimum bias estimation and desire equal spacing of design
levels for N =
1,2 =
4,
then the optimal design would have 1,1
0.823311 and V
= 1.914148.
= 0.2744;7,
See Table B2 of Appendix B for
additional Min B designs.
2.4
Comparison
If one denotes by Vl' V2' and V;, the values of V obtained when
k
= 1,
2, and ; respectively, then it is easy to show that V1 s; V2
for any fixed design in the admissible cJ.ass (i. e., ~ estimable).
:l:
V;
•
- 18 -
TABLE 3
Designs in R with Smallest V for (d, It, N) = (2, 3.. N)
B
NO
IV1
4
5
6
6
6
7
0
1
2
0
2
1
3
2
0
2
3
3
2
0
4
3
3
4
0
4
3
5
4
0
4
5
5
1
1
1
1
0
1
0
1
2
0
1
0
1
2
0
1
0
1
2
0
1
0
1
3
0
1
0
7
8
8
8
9
9
10
10
10
11
11
12
12
12
13
13
14
14
14
15
15
t
1
0.303491
0.496521
0·767378
0.043174
0.00000o
0·3CX)CX)5
0.00000o
0.499186
0.303491
0.000000
0·773660
0.000000
0.310515
0.2041f66
0.000000
0·507998
0.00000o
0·770555
0.043174
0.000000
0.316107
0.000000
0·519581
0·239053
0.00000o
0·773158
0.00000o
N2
2
1
1
2
2
2
2
2
2
3
2
3
3
3
3
3
4
3
4
4
4
4
4
4
5
4
5
0.837747
0.869878
0.781555
0.775564
0.7745g-f
0.808836
0·774597
0.827043
0.837747
0·774597
0·775CX)3
0.774597
0·798464
0.798900
0·7745g-f
O.81cm1
0·774597
0·775925
0·775564
0.774597
0·793188
0·774597
0.802419
0.809430
0·7745gT
0.774955
0·7745g-f
Rank of X'X
V
1,2
1·904555
1·902200
1.888940
1.888820
1.888889
1.897086
1.973251
1.895975
1·904555
1·942387
1.888889
1.888889
1.894343
1.895888
1·934156
1.893447
1·912551
1.888894
1.888820
1.888889
1.892912
1·917901
1.892058
1.898540
1·90l235
1.888889
1.888889
4*
5
5
4*
3*
5
3*
5
4*
3*
5
3*
5
4*
3*
5
3*
5
4*
3*
5
3*
5
4*
3*
5
3*
* Designs having singular X'X matrices
This is true because in adding terms of higher degree to the assumed
polynomial model one produces positive covariances between models
differing in degree.
Hence, VarG)
increase as
k
increases for a
fixed design, and therefore.. V must also increase as
k
increases.
will
It is possible to achieve a measure of robustness when fitting a
response by a linear polynomial, in the sense that designs exist which
•
.. 19 A
for the same linear y will simultaneously g1 ve minimum B whether the
true response model is the quadratic polynomial (k = 1) of (2.1.1) or
the cubic polynomial. (k = 2) of (2.2.1).
Of course, Min B could be
obtained for both k = 1 and 2 using different estimators ~l)' but this
will in general increase V since V
2
,.
2
~
V •
l
The condition for which a
linear polynomial y may be used to achieve simultaneous Min B for k
or k
=2
=1
is
c
= 3a/5
or
N
1:
u=l
4 3 N 2
xu =-5 1: xu •
u=l
Such designs will of course have V = V (which is as small as V
l
2
2
can be for a given design). That partiCUlar design having smallest
V1 = V2 for a N = 4 point design occurs when the design has one point
at each of x = ,!.t
x =
l
.:.tt2 = 0.828616
=
o.3539ll and, cne point at each of
giving V = V
l
2
= 1.888059·
This value of V = V
l
2
is still less than V* = 2, which is the best that can be achieved by
designs satisfying the conditiona of (1..13) and using standard least
squares estimation. iee [51 for an alternative derivation of (2.4.1).
By carrying the logic one step further it is possible to determine
...
the conditions for which the same linear y may be used to simultaneously
protect against the true model being quadratic, cubic, or quartic (i.e.,
k = 1, 2, or
3).
However, it is felt that such a requirement is not as
realistic in a practical sense as obtaining simultaneous protection
against the response being a polynCDial of It
than the. approximating pol;ynomiaL
= 1. or
2 degrees higher
•
- 20 -
3·
QUADRATIC APPROXIMATING POLYNOMIAL (d
= 3)
us now extend the results of the previous section to the
Let
situaion whe-re a response is approximated by a quadratic polynomial in
x, of the form
Attention will be confined to true response models which are either
= 1)
cubic (k
or quartic (k
3.1 Cubic Response (k
= 2)
polynomials.
= 1)
When the true response is a polynomial of k
=1
degree higher than
"-
that of y, of the form
(3.1.1)
we may evaluate V using the appropriate matrices from Section
4
of -
Appendix :A -to obtain
V
= N tr(A(X'x)-lA'W
1
]
= !5c-].0~+3
15(c-a)
+ g?e-30c ;9a •
75(ae-c)
This expression is valid for non-singular X/X (L e., c .; a
(3.1.2)
2
ano.- ae .; c 2 ).
Using techniques identical to those used in Section 2.2, it can be
shown that
9Y = 0
oe
its maximum value.
does not contain e and that Min V occurs when e is at
e
Hence minimization of V was then accomplished
numerically over the levels of a and c or equivalently over design
levels t
l
and
t • The resulting designs were frequently located outside
2
of R: [-1, 11; so the I'best" symmetric designs in R, in terms of
smallest V, are given in Table 4.
The symmetric allocation
of design
•
- 21 observations considered for the entries of Table 4 is No points at
= 0,
x
N1 points at each of x
= .!ll'
.~ ad
N points at each of
2
x = .:!:l2; so that N = No + 2N + 2N as in Section 2.~.
1
2
Note that each of the designs listed in Table 4 have V ~ (V* = ~) ..
.
*
which is the value of V obtained by the standard least squares
estimator for designs satisfying the sufficient conditiona for Min B
glven in equation (2.1.9).
2. 961012 rather than V* =~.
For d = ~ the minimum value of V* is
This value of Min V* is attained for
designs which satisfy the necessary and sufficient conditions for
Min B given in equation (l.13).
These designs have a
= 0.~76544
and
= 0.220527.
c
TABLE 4
Designs in R with Smallest V for (d, k, N)
N
NO
N
1
4
0
l
0
2
l
3
2
l
l
2
0
2
0
2
2
0
0.~J.6012
0.47004~
~
0·~96~~~
g
6
7
7
8
9
9
lO
lO
11
12
12
13
13
14
14
l5
l5
*
~
~
0
4
1
0
4
l
5
0
6
1
5
0
~
4
0
4
0
5
0
5
0
tl
0.422003
0.00000o
0.491342
0.000000
0.56754l
0.6~04l0
0.000000
0.000000
0.452430
0.42200~
0.000000
0.455871
0.000000
0.42682~
0.000000
0.456746
0.000000
N
2
t2
l
1
1
2
l
2
l
l
3
2
0.878025
~
2
2
4
2
4
2
4
2
5
0·929~2
1.00000o
0.774597
l.OOOOOO
0·774597
l.OOOOOO
l.OOOOOO
0·774597
0.565500
0·774597
0.990878
l.OOOOOO
0·774597
l.OOOOOO
0.774597
1.000000
0·774597
1.00000o
0.774597
Designs having singul.ar X'X matrices
= (3,
l, N)
V
Rank of XiX
2·795520
4
4
3.00000o
~*
2·R~4l
2. 1 97
4
2. 62809l
2.981481
2. 65891,}
2.699964
4
3*
4
4
~.oooooo
~*
2. 682120
2·962963
2.6596~8
4
~*
2.615597
4
4
~.OOOOOO
~*
2.6l7360
2·96l111
2.618401
2.981481
2.636~18
~.OOOOOO
4
3*
4
~*
4
~*
•
- 22 -
Each design in Table 4 having non-singular XiX may be specified as
a point in (t , .t ) space for a given allocation of the N observations.
2
l
Such a point is surrounded by a region of points corresponding to designs
(of the same allocation) which have V < V* , so that additional
*
flexibill ty is available if one merely requires that V be less than V
rather than requiring Min V.
(d, k, N)
= (;,
1, 5).
Figure 2 shows contours of constant V for
Should equal spacing of design levels be desired,
the best such design would have NO
t2
= 0.92855;,
and V
= Nl = N2 = 1
with t
= O. 464z77,
l
= 2.7;;991.
It should be noted that designs for which neither t
l
nor t 2 equal
one are, in fact, l\1in V designs since absolute Min V is attained in R.
Also, some of the l1best" V allocations of design points differ from the
"optimall1 allocation for obtaining Min V because absolute Min V is not
attained in R. See Table B3 of Appendix B for addi ti onal Min B designs.
3.2
Quartic Response (k
= 2)
The true response is assumed to be a polynomial in x of degree 4
(quartic), of the form
(;.2.1)
The expression Obtained for V when XiX is non-singular is extremely
. messy and is not presented here since minirti7.ation of V over choice of
design was done numerically.
The appropriate matrices required to
evaluate
are given in Section 5 of Appendix: A.
The type of symmetric design
•
- 23 -
r----------------------..
o. 'r
0.6
0·5
0.4
0.2
0.1
0.8
0·9
1.0
1.1
£'2
Figure 2
Contours of constant V for (d, k, N)
= (3,
1,
5)
•
- 24 considered was again restricted to that used in Section '.1, namely NO
Nl observations at each of x = !.ll' and N2
observations at x = !./'2. For XiX to be non-singular (of rank 5) for
observations at x
= 0,
such designs, it is necessary that NO' N , N2" t and 1,2 be non-zero
l
l
and that /'11: 1. • When X'X is singular, ~ is estimable only when XiX
2
is of rank 4, i.e., when NO = 0 with all other requirements for nonsingul.arity holding.
This requires an even number of total
observations N for symmetric designs of the type considered.
Table
5 gives the designs in R having smallest V for both singular and nonsingular X'X matrices provided such smallest V obtained is less than or
equal to d
= 3,
the value of V* which would be obtained by the standard
least squares estimator for any design satisfying the sufficient
condi tions for Min B given in equation (2.1.9)
However, it is of interest to note that it is not possible to
satisfy even the necessary and sufficient conditions for Min B given in
equation (1.13) for (d, k" N)
= (3,
designs of the type considered.
2, N) using 5 point, symmetric
ThUS, the minimum bias estimation
results have no direct comparison with the standard least squares
,.
approach for this y, 'Tl combination.
Once again, it should be pointed out that for brevity, only the
designs having smallest V are shown in Table 5.
The design co-
ordinates (ll' 1. ) for each design shown in Table 5 are surrounded by a
2
res!0n of designs each having V
are quite extensive.
~
d, and in most cases these regions
Furthermore, nearly all symmetric allocations of
the N design observations to the levels 0, !.L , !.1. have smallest
2
l
values of V
~
d which are in turn surrounded by regions of designs
•
• 25 -
TABLE 5
Designs in R with Smallest V for (d" k" N)
N'
No
4
5
6
6
1
8
8
9
10
10
1f
0
1
0
2
1
0
2
3
0
2
3
0
4
5
0
4
5
11
12
12
13
1.4
14
15
N
1
.11
1
0.339981
0.414192
0.339981
0.593542
0.443900
0.339981
0.633619
0.661552
0.339981
0.414192
0·558129
0.339981
0.693903
0.699432
0.339981
0.615956
0.111865
1
2
1
1
2
2
2
3
2
2
3
3
3
4
3
4
= (3"
0.8611,36
0.885511
0.86ll36
0·903043
0.866452
0.86l.J.36
0·950746
0.951885
0.861136
0.885511
0.901824
0.861136
0·993209
0.996184
0.861l36
0·931558
1.000000
1
1
1
1
2
2
1
1
2
2
2
3
1
1
3
2
1
Rank of XiX
V
"2
2
2" N)
4*
5
4*
5
5
4*
5
5
4*
5
5
4*
5
5
4*
5
5
2.833333
2.802313
3.050569
2.841842
3.024199
2.833333
2. 83;?980
2·184341
2.875316
2.80""13
2·1971.45
2.833333
2·114136
2.810097
2.839110
2·186130
2·182480
* Designs for which X'X is singular
haVing V ~ d.
Only that symmetric allocation producing the smallest
value of V is shown in Table 5.
=5
NO
x
observations at x
= !,t1 = 0.633969"
= 0"
and N
2
For example" for N
N
1
=2
=4
= 15
if one takes
observations at each of
observations at each of x 2
then the value of V obtained is V
= 2.191816
Q
!,t2 = 0·935381"
which is not appreciably
larger than the smallest value. of V listed in Table 5 for N = 15.
Thus"
a wide range of choices is available in allocating the N observations to
the five design points.
See Table B4 of Appendix B for additional Min B
designs.
3.3
,
Comparison of k
=1
and k
=2
Let V be the value of V when 11 is cubic (k
l
= 1)
as in equation
(3.1.1) and V2 be the value of V when 11 is quartiC (k = 2) as in
equation (3. 2.. ~). Then it can be shown tha.t" for any fixed design"
•
- 26 that V
l
S;
V •
2
Conditions for equality of V and V are not simple, as
2
l
was the 8i tuation for d = 2.
(t , t ) plane for which V
l
2
= Nl = N2 = 1,
No
(t l , t ) plane.
then V
l
Indeed, there is no continuum in the
= V2 •
l
= V2
For example, if N
Vl
= V2 = 2.8220
V1
= V2
= 3.1442.
and at (ll
with
at only two isolated points in the
These are at (ll
2
=5
= 0.4892,
= 0.6164,
l2
l2
= 0.8181)
= 0.9048)
giving
giving.,,: ..
The former of the two points is reasonably close to
the design giving Min V in R, namely (£1 = 0.4100,
l
t 2 = 0.9292) with
V1 = 2·1331 and is very close to the design giving Min V2 in R, namely
(t l
= 0.4142, t 2 = 0.8855)
with V
2
= 2.8023.
Hence, if one were
fi tting a response with a quadratic approximating polynomial (d = 3)
and wanted to obtain minimum bias B for true models of degree 3(k
and 4(k
(t l
= 2)
= 0.4892~
using the same estimator, then the 5 point design with
t
2
= 0.8181)
would be the best choice of such designs.
Figures 2 and 3 illustrate the contours of constant V for k
k
= 2,
= 1)
respectively.
These two figures have the Min
=1
and
V design locations
in the (tl' l2) plane marked with small circles as well as the two
points at which V1 and V2 are equal.
4.
CUBIC APPROXIMNrING POLYNOMIAL
This section is concerned with the si tuation where a third order
( cubi c) approximati ng polynomi. al (d = 4),
is expected to fit the true response 11, with reasonable closeness.
true response
The
11 will be assumed to be a quartic polynomial in x(k = 1),
•
- 'Z7 -
0·7
0.6
0·5
0
0
.t1
0.4
0.2
0.1
0.8
Figure 3
1.0
Contours of constant V for (d, k, N)
= (3,
2,
5)
•
- 28 of the form:
(4.2)
Once again the expression
is quite 1.engthy and canplicated in terms of design moments and will be
ami tted.
See Section 6 of Append;ix A
eval.uating V when X'X is non-singu1.ar.
for the ap:propriate matrices for
Minimization of V over admissib1.e
designs in R which are symmetric and have five 1.evels of the form
considered in Sections 301. and 3.2 for both singular (rank 4) and nonsingu1.ar XiX was accomplished numerical.1Y, with the results shown in
Tab1.e 6.
The design moment conditions required for estimability of /fJ
,..,
when XiX is singu1.ar (rank 4) are:
(4.3)
and
(4.4)
These conditions are also the requirements for designs to satis1'y
equation (1..13) when standard 1.east squares estimation is used, and
therefore give V = V* when X'X is singular of rank four.
"
•
- 29 TABLE 6
Designs in R with Smallest V for (d, k, N)
N
NO
4
5
6
6
7
8
8
9
10
10
0
1
0
2
1
0
2
3
0
2
3
0
4
3
0
4
3
11
12
12
1.3
14
14
15
N
N
1
l1
1
1
2
1
2
2
2
2
3
3
3
3
3
3
4
3
4
0·;;9981
0·518972
0·;;9981
0·591999
0.479d>9
0·;;9981
0.619822
0.633348
0.339981
0.613550
0.638934
0.339981
0.641029
0·588300
0.339981
0.617d>8
0.602426
.t
2
1
1
1
1
1
2
1
1
2
1
1
3
1
2
3
2
2
2
0.86l.l36
0.919464
O. 86 J..J.,36
0.955847
0.921945
0.861J.,36
1.000000
1.00000o
0.861J.,36
1.00000o
1.00000o
0.86lJ.,36
1.00000o
o. gr01B5
0.86W6
0·98>698
0.991821
= (4,
1, N)
V
Rank of XiX
4.000000
3.749758
4.043565
3.870498
3.868120
4.00000o
3.492849
3·520392
3.91.3091
3.637839
3.505884
4.00000o
3·548900
3.582986
3.905831
3·562382
3.607998
4*
5
4*
5
5
4*
5
5
4*
5
5
4*
5
5
4*
5
5
* Designs for which X'X is singular.
These are also the designs which
give Min B when the standard least squares estimator is used.
5.
SUMMARY
In this paper we have presented the application of "minimum bias
estimation" to those
,.
where both y and
y,
11 combinations of primary practical importance
11 are polynomials in a single, independent variate.
,.
For fitting a linear polynanial (y) to ei ther a quadratic or a cubic
polynomial resonse (11), the symmetric designs which give absolute
minimum integrated va:dance (V) are determined.
For all other
y.,
11
combinations considered, the type of design considered was limited to
a five point design having NO observations at x
I
each of x
= .:.t1'
= 0, N1 observations
x = .!.l2" When the
and N observations at each of
2
designs gi. ving absolute minimum V are outside of R: [-1,. 1] those
at
•
- 30 designs in R which give smaJ.lest V are presented.
It is seen that
minimum bias estimation in combination with proper experimental. design:
(i)
l.eads to smaller integrated mean square error than is
obtained by standard l.east squares estimation for designs
which satisfy the Box-Draper condi tiona for Min B as given
in equation (1.13) j
(ii)
does not require any prior information concerning the unknown regression coefficients;
(iii)
allows a great deal. of flexibility with respect to design
levels and with respect to the allocation of observations
to the design levels j
(iv)
allows a type of robustness to be achieved, in that one may
A
use a single estimator (y) and obtain Min B simul.taneously
for each of several. possible "true models" (11);
(v)
A
allows Min B to be attained for many y,
11
combinations by
designs having a minimal. number of l.evels, for which the
true model coul.d not be fitted by standard least squares
I
estimation because of singularity of the design matrix, X X;
(vi )
and is reasonably simple to apply.
REFERENCES
[11
Box, G. B. P. and Draper, N. R., A Basis for the Selection of a
Response Surface Design. Jour. ~ ~ M$OC. 54:622-654,
(1959).
(21
Box, G. E. P. and Draper, N. R., The Choice of a Second order
Rotatable Design. Biometrika 50:335-352, (1963).
•
- 31 -
[3J David, H. A. and Arens, B. E., Optimal Spacing in Regression
Analysis.
Annals of
~.
stat.
30:J.072-1081, (1949).
[4J
Karson, M. J., Manson, A. R., and Hader, R. J. Minimum Bias
Estimation and Experimental Design for Response Surfaces.
Technometrics 11:461-475, (1969).
[5J
Karson, M. J., Design Criterion for Minimum Bias Estimation of Response Surfaces. Jour. Amer. Stat. Assoc. 65:1565-1572, (1970).
APPENDIX A
1.
(d, k, N)
1
W
1
=
= (2, 1, Nj
1
0
0
A
1
0
3
0 1
1
3
c
--2'
c-a
,
and (X'Xr
1
= N- 1
0
-a
0,
c-a
1
,
0
a'
0
-a
1
0,
~,
~
c-a
2.
(d, k, N)
A
c-a
2
= (2, 2, N)
c
--2'
c-a
1
2
0
=
0 1
1
3
0
0
and ex'xfl
-a
-'-2'
c-a
0
,
e
ae-c
= N- 1
1
0 5
0
0
2'
,
-c
ae-c
2'
-a
--2'
c-a
0
-c
0
ae-v
1
--2'
c-a
0
,
a
0
,
ae-c
2'
,
2'
For W see Section 1 of Appendix A.
1
,
•
- 32 3.
(d, k, N)
1
A
(2, 3, N)
=
0
1
-
3
1
0
5
=
and
0
1
0
L
5
0
cf-e 2
,
b.
0
,
e
0
ae-c
(x'xf
1
= N-
1
ce-af
,
0
,
-c
b.
0
ae-c
ae-c
2
0
b.
ce-af
b.
,
,
0
2'
0
-c
ae-c
2
f-c
-b.
0
2'
,
0
,
a
ae-c
ac-e
,
0
b.
,
2'
,
2'
,
ae-c
b.
2
0
ac-e
b.
0
c-a
b.
2
where
See Section 1 of Appendix A for WI
4.
(d, k, N) = (3, 1, N)
1
W-=
1
0
1
3"
0
1
3
0
1
3
0
1
5
and
A =
1
0
0
0
0
1
0
L
0
0
1
0
See Section 2 of Appendix A for (X'xf
1
5
.
•
- 33 5.
Cd, k, N) = (3 , 2, N)
A=
For
1
0
0
0
3
- 35
0
1
0
~
0
0
0
1
0
-6
5
7
see Section 4 of Appendix A and for
WI
(XiX)
see Section 3
of Appendix A.
6.
(d, k, N) = (4, 1, N.l
1
0
W1
=
1
3
0
0
1
3"
0
1
5
1
3
0
1
5
0
0
1
5
0
1
7
and
1
0
0
0
_L
0
1
0
0
0
0
0
1
0
6
7
0
0
0
1
0
A=
/
1
See Section 3 of Appendix A for (x Xf
,
35
.
•
- 34 -
, ..
-
APPENDIX B
TABLE Bl
Alternative Designs in R Having V
N
6
7
8
8
8
9
9
10
10
10
10
10
11
11
11
11
11
12
12
12
12
12
12
12
13
13
13
13
13
13
13
13
14
14
14
14
14
14
14
14
14
14
14
14
NO N1
0
1
2
2
0
1
1
2
0
0
2
0
3
3
1
3
1
2
2
0
2
0
0
0
3
3
1
3
1
3
1
3
4
4
2
4
2
0
4
2
0
2
0
6
1
1
1
2
2
1
2
1
2
2
3
3
1
2
2
3
3
1
2
2
3
3
4
5
1
2
2
3
3
4
5
0
1
2
2
3
3
3
4
4
4
5
6
0
~
(V* = d) for (d, k, N) = (2, 2, N)
1,1
N2
1,2
V
0.095776
0.352715
0.532174
0.640566
0.349741
0.117212
0.443585
0.3588,6
0·5344 5
0.246155
0.615359
0.416412
0·535844
0.631954
0.350473
0.682177
0.478399
0.135566
0.442208
0.095776
0.539642
0.349741
0.451919
0.530472
0.365725
0·531963
0.250521
0.602690
0.413025
0.653154
0·564406
0.000000
0.541389
0.630295
0.352715
0.673090
0.473820
0.282389
0·704813
0·544665
0.399428
0.612953
2
2
2
1
2
3
2
3
2
3
1
2
3
2
3
1
2
4
3
0·777837
0.806560
0.819963
0.899905
0.833344
0·777805
0.849949
1. 887966
1. 887264
0·539555
0.000000
4
2
3
2
1
4
3
4
2
3
1
1
5
4
3
4
2
3
4
1
2
3
1
1
4
0.~91021
o. 56963
0·798919
0·954379
0.874137
0.806029
0.848126
0.815774
0·917991
0.884790
0·777787
0.827695
0·777837
0.888772
0.833344
0·907562
1. 000000
0·792104
0.833009
0·793685
0.883635
0.845341
0·975015
1.000000
0.774597
0·798661
0.826738
0.806560
0.862069
0.853517
0.808322
0.926915
0·916919
0.861567
1. 000000
1. 000000
0·774597
* Designs having singular XiX matrices
1.887275
1. 886636
1. 886561
1. 887967
1.886389
1. 8~511
1. 8 601
1. 887387
1.885571
1. 885690
1. 887529
1.887218
1. 887025
1. 886876
1.885730
1. 887968
1.886900
1. 887966
1.886017
1.886561
1.885022
1. 897732
1. 887637
1.887038
1. 887536
1.886548
1.886405
1. 885824
1. 897326
1. 979424
1. 887663
1. 887469
1. 887264
1.887313
1.886416
1. 887149
1. 887087
1.885494
1.885950
1. 892440
1·925018
1·973251
Rank of XiX
4
4
4
4
4
4
4
4
4
4
4
4
4.
4
4
4
4
4
4
4
4
4.
4
4
4
4
4
4
4
4
4
3*
4
4
4
4
4
4
4
4
4
4
4
3*
•
- 35 TABLE B2
Alternative Designs in R Having V ~ (V* = d) for (d, k, N) = (2, 3, N)
N
NO
N1
1,1
N
2
1,2
V
Rank of X'X
7
7
8
8
9
9
9
9
10
10
10
11
11
11
11
11
12
12
12
12
12
12
13
13
13
13
13
13
13
14
14
14
14
14
14
14
3
1
2
0
1
3
1
1
2
2
0
1
3
1
3
1
2
2
2
2
0
0
3
1
3
1
3
1
3
4
2
4
4
2
2
6
1
2
2
3
1
2
2
3
2
3
4
1
2
2
3
3
1
2
3
4
5
3
2
2
3
3
4
4
0
2
2
3
4
3
4
0
0·734832
0·503241
0.621306
0.441121
0.175578
0·757718
0·397235
0·509827
0.496521
1
1
1
1
3
1
2
1
2
1
1
4
2
3
1
2
4
3
2
1
1
3
3
4
2
3
1
2
5
3
4
2
1
3
2
4
0.806237
0·939678
0·921194
0·936912
0·785414
0.804501
0.858899
Ou996532
0.869878
0·987182
1.000000
1·930904
1·911037
1.899703
1·923918
1.895845
1.889411
1·905392
1·917252
1·902200
1·907978
1·944444
5
5
5
4*
5
5
5
5
5
5
4*
5
5
5
5
5
5
5
5
5
4*
4*
5
5
5
5
5
5
3*
5
5
5
5
5
5
3*
*
o
o
58l';~)'4
./.j'
0.447214
0.107596
00617404
0.299629
0.674648
0.432001
0.107486
0·397107
0.499643
0·565296
0.447214
0·303491
0.496571
0.229103
0·5151)2
0·365753
0.636666
0.451826
0.000000
0.621393
0·306065
0.669513
0·705087
0.429112
0·503241
0.000000
00778609
10926938
0.859125
0.817300
0·937252
1.895310
1.899808
1.896720
1·910761
1.889095
1·900435
1·907065
1·930959
2·000000
1·904555
1.898246
1.895578
1·901340
1, 905580
1·905677
1·914845
1·979424
1.893169
1.897086
1.893819
1.894519
1·904757
1,,911037
1·973251
00899410
0·777569
0.834022
0·906883
10000000
1.000000
0.837747
0.842120
0·798073
0·902506
0.852666
1.000000
0·934347
0·774597
0.833430
0.808836
0.871075
0·938335
0.863941
00939678
ou 774597
Designs which have singular X'X lnatrices
•
- 38 TABLE B5
Alternative Designs in R Having V ~ (V* = d) for(d, k, N) = (4, 1, N)
N
NO
N1
1,1
N2
9
10
10
11
11
11
12
12
12
13
13
13
14
14
14
14
15
15
15
15
15
15
1
4
2
5
3
1
4
2
2
5
1
3
6
2
4
2
7
5
3
5
1
3
2
2
2
2
2
3
2
3
4
3
3
4
3
3
4
4
3
3
3
4
4
5
0.417495
0.632435
0·518972
0.643081
0·570652
0·393808
0·591999
0·500552
0.609249
0.644288
0·388631
0.635662
0.637584
0.450885
0.634605
0.479069
0.650007
0.630316
0·518972
0.629830
0·377101
o· 620724
2
1
2
1
2
2
2
2
1
1
3
1
1
3
1
2
1
2
3
1
3
1
V
1,2
0.882031
1.000000
0·919464
1.000000
0·943865
0.879760
0·955847
0·923720
1.000000
1.000000
0.873324
1.000000
1.000000
0.893183
1.000000
0·921945
1.000000
0·999780
0·919464
1.000000
0.872558
1.000000
3·856579
3· 682551
3· 749758
3·896177
3· 759195
3·854017
3·870498
3·723359
3·926611
3·663404
3·899921
3·673386
3·817058
3.815495
3·633879
3·868120
3·986267
3·614337
3·749758
3·689707
3·858180
3·911113
All the designs in this table have non-singular X'X matrices
•