ON CERTAIN TYPES OF BIAS IN CURRENT METHODS
OF RESPONSE SURFACE ESTD1ATION
v
,,•"
by
H. R. van der Vaart
Prepared under Contract No. DA-J6-034=ORD-1517 (RD)
(Exper:iJnental Designs for Industrial Research)
':,,,,,
."
,.
~
-
Institute of Statistic~
: .c.
Mimeo. Series No. 205
July" 1958
•
ON CERTAIN TYPES OF BIAS IN CURRENT METHODS
OF RESPONSE SURFACE ESTIMATI orql
by
H. R. van der Vaa.rt2
1.
Introduction
Let ~l' ... , ~ k represent a set of k simultanecus conditions (f .i. tempel'a-
ture, humidity', pressure, etc.) under which some process, biological or chemical,
is made to take place.
Let
'l represent the yield or response of th~ process.
Experiments will be made at different sets of levels for ea.ch of the k conditions.
Interest is in
'1
as a function of ~l' ••• , ~ k' or equivalently in
'<. as
a scalar
function of the k x 1 matrix (column-vector) ~. We shall assume throughout that
the response function, or response surface, is guadrW,Q", i.e. that
Here the k x 1 matrix
Ii and the k x k matrix ~ represent constants which the
experimenter will want to determine; a prime denotes transposition of any matrix,
so
~r
is a row-vector.
Of course there are interesting problems because of the
fact that this quadratic model will only rarely be co:rrect (cf. for instance Box
and Hunter, 1957, p. 216), but this paper is restricted to certain problems which
arise even though this model would be correct.
Evidently, if the different levels of the k components of ~ could be experimentally produced with exactitude, and if the corresponding yield 1( could be
observed with exactitude, there would be little need for statistics in the problem.
1 A report on work sponsored by the Office of Ordnance Research, United States
Army, under contract DA-36-034.. CRD-1517(RD) with the Institute of Statistics, North
Carolina State College of the University of North Carolina, Raleigh, N. C.
2 Present address: Institute for Theoretical Biology of the University,
Leiden, Netherlands.
- 2As is done in most statistical work on the theory of estimation of response surfaces (cf. Box and V{jlson,. 1951, p. 2; Blo.em~m, 1956, p. 8; ~ and Hurte;;., 1957,
P .198; Cochran and Cox, 1957, p. 335), we will assume that ~ can be experimentally
produced with exactitude, but "1'\ cannot be so observed: instead of the "true ll
response
'l
we observe realizations of a random variable y with
Ey='"1.,
var y
= (]2
varY=(J2,
being assumed not to depend on the correspo!lding value of ~, and
depending on
?;,
according to (1).
'l
Observe that we denote random variables by Latin
letters, pa:rameters by Greek letters, square matrices 'b-J capital letters, vectors
(k x 1 matrices) by underlined small letters.
The general procedure now is to choose wisely a set of'
}J,
say, sets of values
of the k conditions being studies, i.e., a set of N points in (~l' ... , ~k)-space,
and for each of these to observe y, finally to estimate
data.
"'la' !i and
J from these
This est:imation is usually done by 'the method of least squares (cf. Box and
Wilson. 1951, p. 5: ~ and Hunter~ 1957, p. 198), wi1:i.ch yields expectationunbiased estimators of
'70 '
~ and
P provided
(a) the random varia,bles y corres-
ponding to the different points in the above-mentioned set of N points in
(c;l' •• to, ~k) -space are uncorrelated a..'1d (b) the p,~lt,tarn of these N points in
(~l' ...,
et k)-space
satisfies a certain condition (mentioned in Section 3, p.
of ~ and Wilson., 1951) which is easy to comply with.
nonnally distributed, the estimators of
1. 0
5
If in addition Y' is
and of the elements of
!i and ~
are
multinormally distributed', their covariance matrix depending on the pattern (the
design) of the N points in (~l' ... , ~k)-space.
The present paper will investigate a few questions arising with respect to
e
current methods of estimating the type 2f
9J1§fl..:r.flli19. surface: i.e. estimating
whether it has a maximum, or a minimum, or a saddle-point, or a stationary ridge,
or a rising ridge, eto.
As is well known, if one wants to investigate the type of quadratic sl.l.rface
defined by equation (I), the thing to do is to redu.ce this equation to canonical
form, i.e., to rotate the (~l' ••• , ~k)"axes in suoh a w9¥ that the transform of
~
'l. ~
no longer oontains cross-product terms.
This means that one seeks to
construct an orthogonal matrix Y such that if' one effects the transformation
~ = Yt"
whioh throws equation (1) in the form
"">(
&:I
the matrix ye ~Y is diagonalo
'>to
+
Ii'Y ~
+ ~ tye py f; ,
Denote this diagonal matrix by
Y'~Y~A,
then (312) reads
'1-
~o
~he elements of Ll, "'1 ~ "'2 ~~
eo.
.l-
~gy ~ +
t,ali.!Z.
{3d>
~ "'k" tlrS the late:a roots of the matrix
J
.;
!.
It is well known that suoh an orthogonal ma.trix Y can a.lways be cons·li!'ucted.
Obviously the type of e. quadratio ourface can be assens3d
tion like (3g) than from (1).
if k
= 2)
....
m1~.oh
easier from an equa-
Note that plotting contour ....
lines (which is feasible
.
directly from equation (1) (cf. for instanoe CocP..rS!! and Fox, 1957,
p. 352) constitutes eS'3ent::'[~:.ly the same method --..
as reducing equation (1) to
.-:.c.t:.~~~.....-
-=-:&~
___
canonical form: one may regard reduc tion to canonical form as a device by whioh to
plot contour lines.
Now, as we set forth in the second and third paragra.ph of this introduction,
Y(O'~ and ~ are unknown; only their (expects,tion-unbiased) estimates, to,be
denoted by YO' !l and F, are available.
Yet ~ BIld Wj.!!QU, 1951, p. 23- 24; ~S
1954, p. 35; ~ and Youle, 1955, p. 289; ~ a..'1d ~!., 1957, p. 239 (IIA fitted
_
second degree equation can be interpreted most readily by writ:i:ng it in the
- 4canonioal form") proceed in just the same way: they oonstruct an orthogonal matrix
U such that the transformation
throws the £1tted equation
~
III
YO + Jl'
t
+
~ ~F ~
in the form
where
L .. U'FU
is the diagonal matrix consie.ting of the latent roots $1
~
$2
~
•••
~
t k of F.
Then they use these latent roots of the estimatj.ng matrix F in their inference
concerning the type of qua.dratic response surface.
We shall report on some properties of the distribution of the latent roots of
F which seem to explain (at least partly) some difficulties cited in the literature
on applications, and which make clear tha.t ~tS long as the present method (of
estimating.1\) has not been replaced by a better one, it. I:Jhculd :::.t least be applied
with oaution. Unfortunately this caution calls for more experimental points (i.e.,
for larger N) than othe~ri.se would be neoessary, thus counteracting the third re~
quirement concerning experimental designs for estimating response surfaces which
was laid down by
~
and
I!m~~
1957, on p. 197: IIIt should not contain an
excessively large nUlilber of experimental points."
Before carrying out this program, we shall have to make a few general remarks
on the effects of scaling.
2. A few remarks on scaling
-
e
--
When writing down an equtl,tion like (1) we implicitly assume thAt the units in
which ~l' 'S2' ... , ~k are to be measured have been chosen. A change in the units
.. 5 ..
of some or all ~, will bring about a change in the values of ~ and ~, and by
choosing these units judiciously one can make the elements of
1i and of ~ ha.ve
almost any value one wants -. except for changes of sign. An analogous conclUsion
holds for the estimators
B. and
F of
1i and
~.
Obviously a sphere in (tl' ... , ~k)­
space will no longer remain a sphere if the unites) of at least one of the variables
~l' ..., E;k are changed.
Thus it is understandable that certain aspects of the
theory of estimation of response surfaces have been'criticized as depending too much
on scaling. Specifically, it has bem frequently asked whether there is a:ny sense
in rotatable designs.
In rotatable desi@s the varianoe of the estimated response
YO + h' ~ + ~ 'F~
(5)
in the experimental point represented by the veotor
1. dePGnds only on the distance
~,~
- between this experimental point and the origin (cf. Box and Hunter, 1957,
~
e
p. 204), henoe after any change of soale a rotatable design is no longer rotatable.
The values of the Is.tent ro0't! of
but fortunately, the m:mbers of
respeotively, are
!!21:
~
and F, too, are
E.~$,g.!!~, ~,
and
~~cte~ by 2.,hanges
J?£siti!.~
1atel1!
of scale,
~,
affected by these ohanges.
Because of all this it is important to see what scaling really means.
There
are some differences in the ways in which Box and his associates have handled this
matter in different papers.
at all; on page 8 a kind of normalization is introduced with the purpose of being
Part of the paper by Box and" W:.U.son, 1951, is given without mentioning units
.
able to compare two designs I lItwo designs are regarded as of comparable size when
they are measured so that the spread for each of the (k) fac tors (~. ) is the same
J.
in the two designs,3 the spread being some variance-type quantity used to judge if
3 Incidentally, at the Ames regional meeting of the Institute of Mathematical
Statistios, April, 1950, Le Roy Folks gave an interesting alternative definition of
comparable designs: he wanted to compare all designs which have all their points in
the region of immediate experimental interest; hence he considered the range of
experimental points rather than their spread.
e
.. 6 ..
the N dif1'erent levels 01' the 1'actor ~i are widely di1'1'erent or not (i .. 1, "" k).
In ~ and !.oule,
1955, p. 292, the variables
"1'or convenience. 1I
~i are being coded, apparently only
-
In Box and Hunter, 1957, p. 196, standardized levels
~*.
= ;iu" ~~
:.
J.u
~i F
(u .. 1, ..., N; i
= 1,
0 H,
(6a)
k)
are introduced so that
(6JV
•
Here ~ :LU
, is tr..e value which the exper:imantal • condition represented by ~ i takes
in the uth experimental point (u
~
J.'
(i
III
1, 0'"
CI
N); ~ i is the mean 01' ~iU over u: the
1, ""
k) determine the center 01' the design, i.e., the canter 01' the
.
N
.
region 01' ilTlmediate experimental interest; ~i :=u~(~iu - ~ i}2/N; c is an arbitrary constant, independent 01' i, hence not playing an essential role in our
present problem.
(u
1:1
1,
a ..,
01' course, not only the experimental po:ints
(E51u'
""
~ku)
N) are transformed as a oonsequence 01' (6~), lr..1t the vlhole 01'
~,
(~l' "'J ~ k)-spaoe is trans1'ormed into (~i:,
?s.*
J.
'.
Now, on the
0
n'G
~.
=].
-
~i
~,
F
J.
o.
0'
*
~k) -space according to
(i.. 1, ..., k)
(6~
hand, as ~ and &mter, 1957, p. 196, write, equation (6!.>
allows us to derive designs (Le' J sets 01' N values 01' the vector ~= (~lJ
a .. ,
~~»
which cover the region 01' :ilIImediate interest in any given experimental problem J
from one fixed design4 described in terms of the variables
~ ~J •• 0' ~=. This is
done by choosing the quantities '1:'i and 'Si so that the points (Slu' ""
~ku)
4 Of' course more than one fixed design in (~~J "', ~:) ...space exists, according
to the properties which one desires the design to have, but (5!.) can transform each
1'ixed design in
~, o •• , ~:) -space in an infinite number of w~s so that it may
serve on an :infinite number 01' regions to be explored in (~l' 0", ~k) ..space 0
(s
- 7(u .. 1, ... , N) cover the region of iInmediate interest as fully as possible.
On
the other hand, however, equation (6~ means that designs whioh are rotatable with
respeot to (~~, ... , ~:)-spaoe ha.ve the property that the variance of the estik
mated response (,5) is oonstant i f .~ g~2
iul
ellopsoids of the family
~.1
k
~
t;:1
(-
-
-
5.] .).
~i ..[C
.. oonstant,
i.e., on the surface of the
2
(651)
.. oonstant,
where 'C'i and ~i reportedly are ohosen in suoh a way that the ellipsoid (of. (6~.»
follows the boundary of the experir.lentally iIlt3rest:L"'1g region as closely as possible.
The only result of scaling according to (6§) and (6s.) is to adapt the design to the
region the experimenter wants to
e~~lorej
a·s for the region to be explored, the
experimenter has to make up h:i.s m:i.nd on thi.s poi..Tlt, Emyho'W
Q
In case of rotatable
designs the oontours of equally exaot prediction are -chus mac.e to follew the boundary of this region as closely (in a oertain sense) as possible -- which does not
seem too unreasonable a procedure in a number of contexts.
The importance of the
region to be explored (or the region of immediate experimental interest) oan easily
be seen by considering one response surface (for k
= 2)
and two different regions
of interest, one of the."Il happening to be elongated along one prinoipal axis of
~ t ~ E; and short along the other aris, the other region happening to be elongated
along the second axis and short along tl:e first.
Examples of this kind show that
in this problem it is not to be taken for granted that scale-invariant procedures
are necessarily preferable.
•
The results to be disoussed in the next section are described for a system
of ooordinates ~l' ••• , 'Sk-
One may assume if one wishes that the ~ .i stand for
e
- 8-
~~ in the sense of equation (6£,).
3. Some properties of
~e
distribution of
t~e
la.tent roots of symmetrio random
matrioes
Part of the proofs of the results oontained in this seotion oan be found in
!!!l. Q&:r !9-art• 1958. The other proofs are yet to be published.
The k x k matrix F introduoed in seotion 1 is real, symmetrio, has random
variables for its elements. We shall assume throughout that the joint distribution
of its elements f ij (1 ~ i ~ j ~k) is oontinuous in the usual sense (some results
will hold under a somewhat weaker assumption, for instanoe that the latent roots of
F are all different with probability· one -- which of course they are i f the f
continuously distributed).
ij
are
Because of the least squares E';stimation
Under these rather weak conditions
(.e l <.e 2 < •eo <.ek
being the latent roots
of F, 'A. l ~ "'2 ~ .•• ~ A. k the latent roots of ~):
k .
~ C' 0
",G.'fI
g=1
=
g
k
~"I.
~ I\.
g:oJ1 g
k
= k""V P,i "
1':1 ~J.
If in addition the ~(k+l)~variable probability density function of the f
(1 ~ i ~ j ~ k) is symmetrioal with respect to the point with ooordinates
(1
(8)
,
ij
9fij
= t:fij
~ i ~ j ~k), then
(10)
Under certain weak oonditions on the distribution of the f ij , whioh are satisfied
for instanoe by the mu1tinormal distribution, the inequalities (10) are striot.5
5 In the abstract No. 41 in Annals Mathematioal Statistios. g§" 1957, p. 1069 it is
erroneously stated that the first two inequalities in ho) hold under the oonditions
described in the second paragraph of this section.
e
.. 9 -
Evidently a much larger class of distributions of the f ij will allow just one of the
inequalities (10) to hold.
Again if the joint distribution of the f ij is continuous and if (7) holds, the
equations (11) and (12) obtain
k
k
k
k
~ var t = ~ ":' var f .. + ~A.2
~
g:r.fl J~
J.J
g
#1
If k
Il'I
2 and the "amount of eA'Pectation"bias"
.,(!3
III
C(t 2-A. 2 )
III
k'
-
~ (c,t )2
~~ g
(11)
.,(!3 is defined by
-~(tl"'A.1) > 0
(11§)
then (11) yields
2
2
2
~
>varoflD a:.:
"" var f i . - 2.,(~ - 2(A. 2-A. l )"'A •
J
t'
t'
g ~ ~
~
Ft
Concerning covariances
k
~
k
~ cov(t ,th)
g.;'l niil
g.
Ie
'=
~
k
":> cOV(f,pf j .)
J.~31 J~
As a particular case of a general theorem given in
-
J
~ ~t y'aa~..t,
1958, we have
that for k .. 2
if f ll , £12' f 22 are tri.normally distributed with
under this condition (llW determines var t 1 *'= var t 2 in terms of .,(~.
Equations ell) gi~1T8 some information on the variances of the latent roots of
F in terms of the variances of the f ij 9
on this po:1nt.
~
and
Youl~.
The existing litera.ture does not elaborate
1955, p. 295 state that "appropriate standard errors
for these constants (i.e., the t g) can be shown to be of the same order of magnitude
as those of the original quadratic and interaction terms". Box and Hunter, 1957,
- 10 -
e
p. 240 state that for any rotatable design the variances of the coefficients are
the same in every orientation and since the latent roots
It
are simply the 'quadratic
effects r in the directions of the canonical variables they have the same standard
errors as have the quadratic effects (i.e., our f .. ) before transformation. 1t
J.J.
Equation (11!V shows that if this were true, then for k
",,2 + (1.. -1.. )""13
2 1
13
III
var f 12
III
2
,
(14)
hence
""13 .. Jvar £12
if 1.. -1..
2 1
III
0
•
(14~)
That for any distribution of the f ij (1 ~ i ~ j ~ 2) the amount of e:A'Pectation-bias
should depend solely on the latent roots of ~ and on var f , not on any higher
12
moment, seems unlikely_In case the f ij (1 ~ i oS j ~ 2) are multinormally distributed our equations (18~ and (18g) below show that (14) and (14.!> do not hold
generally.
The flaw in the argument of BQA. and Hunter, l.c., seems to be that
though it is derived from a statement (at the top of p. 208 of the same paper) to
the effect that in rotatable designs every variance and covariance of the coefficients (our f . .) ••••••• u%!lust remain constant under rotation lt of the design of N
J.J
experimental points, yet there is a difference between the rotation on p_ 208 and
C:J
/-the_one on p _ 240: the first is nonrandom, does not depend on the estimate F, the
/
~second is random, does depend on F,reduces the off-diagonal elements of F to zero
would equal var 0 III o.
12
If the observational error (y-"l() is normally distributed, the distribution
(L is a diagonal matrix). By their argument var f
~ f ij (1 ~ i
s:
.$ j ~. k) is !!luItinormal. This will be asped in the remaining EU..1
of section 3- Then we can explicitly compute the probability density function of
.el' ... , .e k
e
~
= yAy',
.L1 and
.As the distribution of the f ij depends on the parameters contained in
the probability density of.el' .. _, .ek will in general depend both on
Y_ In a sense the elemen ts of Yare nuisance parameters.
However,· they do
-11~
'.0' $k if we restriot attention to those
not ooour in the distribution of $1'
distributions of the f .. for whioh the oovarianoe matrix oan be desoribed in terms
~J
of two parameters . <. a.'tld ~ as follows
C1ii ,ii a
I3 + {k-l)""
(i
~(~+k..<.l
-..<.
a
= oov(fij ,
1:1
1
=2~
(i, j
=1,
=a
(i, j, p, q
III
C1
.
a.~ j ,pq
~(~+k"")
f pq) ):
1, .. 0' k)
(i, P
C1ii ,pp
ij,ij
ij ,pq
(C1
1,
..., k·, if p)
.. " k·, i " j)
= 1,
p .;. q, and i
... , k; i r) j or
r p or
j .;. q)
Here
a)-p/k,
e
~>O
It is interesting that all seoond order rotatable designs lead to the set of
oov(fij , f pq) satisfying (15).
§;)
Observe that
In our oonditions (15) only the quadratic coefficients f .. of our
~J
equation (4~ are involved.
In ~ and Hunter's (1957, p" 213) oonditions for
second order rotatability the ooefficients of the terms of degree zero and one
ooour as well.
£) When trying to make the two sets of oonditions oorrespond note that _
boo
u
= :roo,
II
5V
bij
III
2f .. (the b .. are Box and Hunter's notation, the f .. are ours).
~
.
~
For k = 2 the oovarianoe matrix (15) satisfies oondi tions (1312), hence
if (15) holds, var $1
si>
~
= var
$2'
The l-l-oorrespondence between our oovariance matrix (15) and Box and
Hunter f S oovariance matrix oorresponding to second order rotatable designs is
described by
(16g)
- 12 -
(16:g)
for any k ~ 2. Here N is the number of experimental points, 0' is defined in (2),
A depends on the design of the N experimental points and influences the function by
which the variance of the estimated yield in the point (~l' ...., ~k) depends on
the distance between (~1' ••• J ~ k) and the center of the design.
!2aE. and
According to
Hunter, 1957, p. 214, values of A somewhat less than unity (small negative
values of .,I../f»
are satisfactory in various respects.
If in our formulae below we
let @ - ..> co, we assume. A, hence:::11... to remain constant~
.,I../f> constant, then f>
--> co
corresponds to N 0'-2 --~
We shall now give some results for k
covariance matrix (15)
e
G
1>1
12 = A2-A1'
Here IJ.
CD
2., f ~J
.. multinormally distributed with
I Fl (aj c; x) is the confluent hyper-
geometric function (of. Higher transoendental'functions, vol.
!, 1953,
p. 248),
I (x) is the modified Bessel tunc,tion of order zero (of. Higher transcendental
O
functions, vol. 2, 1953, p. 5) •
f,(~2-A2)
-£($1-).1)
a
a
-
1;" ~ff 'll(- t;l; - ~)
(18§)
For IJ. and/or f3 --) co we find:
(18,2)
Note that the amount of expectation-bias
is independent of
~(~l-Al)
=~(~2-A2)
• 12Jf>fj[ i f
II
c-
=0
.,1..,
(18~
,
IS
decreases as IJ. increases,
(18iV
decreases as f3 increases,
(18!)
is
o( ~)
as IJ. --) co and/or f> --) co (IJ.
r 0)
(18~
- 13 Furthermore
var .e
= var.e 2 .. ~
+ L + lt~ 1
~\fl+2~2~
2
For 1J. and/or
13 -->
....a r 'CI/)
VI
l
Q)
!!...
4~
[F1 1 (- 1_2"1-_&:]
2
2)
•
we find
= ""ar /) ~~ + ..< 1/ 2
I3(13+2..<)
V
O(..L-)
4
13
12 __
13 4 2
4~ 1J.
2~ 1J.
6
1J.
•
(19'b'
!U
Note that
..: var f 11
an
var f
D
22
&01
var f ll + var f 12 -
~T~:21r
1T
413
'
(19£>
&I;
if 1J.
yet
+
var f
11
= 0,
for certain combinations of
~-
and 1J.-vs,lues
We ought to remark that we have as yet no q.1ite rigorous proof that (18.~), (la!>,
(19s) hold good for all values of"<, ~, l.l. although the evidence available seems
pretty strong_
Finally i f W
uJ a
- 1..1
D
a
and
-X a/
1J.'r .. 1J..ffi" .. (1..
2
J2,
W
b are two constants and if
b - 1..1
VJ
- AI)
Jf,
= -Xb/ J2, X~
then
= Xa
Ii,
X~
.. Xb
fi3 ,
(20!.>
(20.Q)
Applying Hsu, 1951" starting from (20:g) or from the expression which follows from
(20lV by the Bubstitution m = IJ.tX" one can obtain an asymptotio result for IJ. and/or
~
--> (1):
laok of time has prevented me from doing so.
for A and B in (20~ show thtlt for IJ.
X0
p ( - ~ < tl-A. l
\J 2~·
tit
If'
x~
III
co,
X~ = 0,
X0
b
<- --,..
II:
J2f
III
0, the expressions
0
)
0
:ll
If IJ.
0
constant it' X and "b are constant
a
-
(20d)
the result is
(20~
4. Discussion of results
Our investigation has been restrioted
(iJ
to k = 2, (9) to quadratic response
surfaces: possible oonsequences of third degree terms on the effeotsdescribed here
are not considered, (2.) to designs satisfying (15) -- whioh inolude second order
rotata.ble designs.
N.a-2
--> 0Il>,
Our convention (17) entails that ~ ..... > 00 oorresponds to
i.e., mostly to N -->
00
( C1 vill not be easily ohanged in the course
of' one experiment).
< var f ll (of. (19~), certainly
l
var t l r) var f ll (of. (19~, (19!V). This oontradicts ~ and !funter, 1957, p. 240,
of'. our section 3. Yet (19~ and (19D show that under oertain conditions
Let us f'irst observe that probably var t
e
var t
l
~var f n asymptotically.
• 1$ __
.As to types
2! E!!.! in
the estimation of the latent roots of ~ (the canonical
coe.fficients of the quadratic)" our resuJ.ts are much more oomplete for !.L
').1
III
III
0" i.e.
X2" than .for arbitrary !.L.
For
, !.L • 0 there is oonsiderable expectation-bias (18g) and median-bias (20~) z
O"design both rotatable and orthogonal, P(t < A ) I=! .85 .. P(t >>"2); for
l
l
2
pentagonal designs P(tl <Xl) 1:11 .75 • P (t 2 >>"2); upper limit of this probability is
if' .,(
III
1.00). If
13 --> co, median-bias remains oonstant, expeotation-bias --> O. Both
.. var .&2 being asymptotically "-' 13-1/ 2 (times some
l
oonstant) suggests that as 13 increases tlle 60ale of the probability distribution of
.& .contracts 'td th r.. as a center (g .. 1, 2), at' the rate of 13-1/ 2 . TJ1is is confirmed
-i(tl->"l) .. &(t ->"2) and
2
g
Val'
t
g
by (20g).
For J.+
>0
there is still expectation-bias and median-bias.
bias appears to deorease with !.L
--> 00
and/or f3
-->
<Xl.
The expectation-
As we could not yet explore
the expression~ {20gJ and (20lV we do not know for certain how the median-bias
varies with !.L arrl
13. As now
-G'{ t 1->"1) / Jvar
is not asymptotically independent
l
of 13, the contraction of the probability distribution will probably be somewhat more
t
Considering ±~(tg-A g)/~ar t g (g 1:11 1" 2) suggests that
for !.L > 0 the median-bias might decrease with f3 --> <Xl and/or !.L --> 00 (some part of
intrioate than for!.L
a
O.
the probability being pulled over to the other side of the Ag> •
One general oonclusion from the fact that the t g are expectation-biased and
median-biased estimators of the Ag is that a oonsiderable part of the probability
distribution of t l ('&2) is found on t l <Al (t 2 >A2) (the marginal distributions of
the t g, not oited in this paper, are quite skew). This suggests that if Al > 0 is
e
small its estimate will be neative lti92. freguentJ.y," if A2 < 0 is small its
estimate will be positive u~ frequently.1I Of' course" without a comparing est!mator (cf. van dar Vaart, 1957" p. 4) the meaning of lltoo frequentlylt cannot be
- 16 made precise.
One conclusion is warranted, though.
the above-mentioned contraction of the
proba'bi~ity
Ii' m.2!! observations are made,
distribution will eventua.lly pull
the la'l:'ge bulk of it over the zero point to A.
...
of.....e...r...r...o..n..e..
ou;;;;s-.,;s;;,;i;;.llgn;:;,;"",._e;;;"st_J.l;;;;·m;;~.
or A. , thus decreasing the freguencl
2
l
There is some evidence that for large differences
(A.2-A. l ) .the frequency of erroneous sign estimates will be smaller than for small
happens to be zero the contraction
I
of the probability distribution to A. will pull nothing over the zero point (as for
I
A. l > 0), but it will eventually concentrate a. large portion of it so close to zero
that the difference becomes materially uni.mportant (cf. Hodges and Lehma."m IS, 1954,
(A. 2-A. l ); this seems intuitively plausible.
If A.
concept of material significance as opposite to statistical significa.'l'lce):
The
fact that the frequency of errcneous estimates of the signs of the latent roots can
be decreased by tak:L'l'lg more observations is grati.fying, but oounteracts (as long as
e
no better estimators are known) the third requirement on an ei>.."Perimental design for
estimating response surfaces, as was laid down by ~ and
!2.ntll,
1957, p. 197:
IIIt should not contain an excessively large number of experimental points."
There is experimental
1)
e~r.idence
for the
tteoretic~l
results
he~e
presented.
On p. 548 of the book on design and analysis of industrial experiments, edited
by Da.vies, 1956, an example is given of an analysis uhich in the first step yields
one negative and one s!!l.8.11er positive latent root (besides a quite small negative
one). After the acctl!l!ulationof a larger body of data the positive root is
described as IJnegligibly small. 1I
2)
Mason. 1956, p. 94, Fig. 5e6, presents a diagram of contour lines of a fitted
second degree response surface which clearly shows a saddle-point (= col = minimax).
(As we said before, plotting contour lines and computing latent roots are equivalent
~th respect to the present problem.)
His cautious remark concerning this occurrence
is in direct agreement with our theory: IISuch a surface would appear to be difficult
to interpret agronomically.
One would certainly like some substantiation of this
.. 17 -
---.........:;,,----..:.-
type of pattern before extending its applioation too far. A more oomplete sampling
by observation points in th!:, oriti2,al region .is perhaps in order. n
On the session on applioations of response surfaoe designs at the Atlantio City
3)
Meeting of the Amerioan Statistioal Assooiation (September, 1957) there were reports
of smilar experienoes.
I wish
to thank Dr. R. J. Hader of the Institute of Statistios, North Carolina
State College of the University of North Oarolina, Raleigh, for his suggestion that
it might be interesting to investigate some questions in the theory of response
surfaoe estimation during my stay at this Institute. A seminar I gave on the
subjeot at the Department of Statistios of the Uniwrsity of Chioago during my stay
there led to plans for oomputing on high speed maohines oertain problems whioh
seemed analytioally untraotable.
,/
,/
RESUME
On ~tudie la loi de probabilite des val-eurs prop!'es d 'une matrioe re'elle,
/
......
s;ymetrJ.que, a
//
eleme~ ts
/
..
/
,
/
alea to ires. L ! applJ.O atJ.on des resultats a. la theorie de
Itest:imation des surfaoes de r~gression quadratiques semble expliquer pourquoi les
e~rimentateurs ont quelquefois pu trouver un
001
ou minimax au lieu du maximum
auquel ils stattendaient - et d'autres exp~rienoes analogues. De plus, on fait
/
/
quelques remarques sur l'ecart moyen centre quadratique des valeurs propres et sur
les dispositions dites
It
rotatable ll des observations.
LITERATURE OITED
Bloemena, A. R. (1956). Experimentele bepaling van optimale oondities (Overziohtsrapport). Rapport S 204 (Ov 6) van de Statistisohe Afdeling van het
Mathematisoh Oentrum, Amsterdam (Netherlands).
Box, G. E. P. (1954).
Biometrios ~ 16-60.
i_
... 18 Box, G. E. P. and Hunter, J. S. (1957) • Ann. Math. Stat.
~,
195-241.
Box, G. E. p. and !!;i.lsoIl" K. B. (1951) • Jour. Roy. Stat. Soo., Ser.
v.
~, G. E. p. and Youle, p.
(1955).
~
JJ., 1..45.
Biometrios 11, 281-323.
Cochran, W. G. and Cox, G. M. (1957). Experimentel Designs, 2nd ed., New York,
Wiley, and London, Chapman and F..all, xiv + 611 p.
Davies, O. L. (Editor) (1956).
The design and analysis of industrial experiments;
pUblished for Imp. Cham. Ind. Lim. by Oliver and Boyd, London and Edinburgh,
and Hafner, New York, 2nd ed., xiii'" 636 p.
Higher Transcendental Functions, (Bateman Manuscript Projeot), Vol. 1, 1953,
--:~------------...,
xxvi + 302 po, VoL.
~
1953, xvii + 396 p., McGraw Hill.
!!2.~~, J. L. and. fehmaEE., E. L. (1954).
Jour, Roy. Stat. Sooo, Sere
11
~
261-268.
Hau, L.
o.
(1951). Amer. Jour. Math. ll, 625...634.
!1ason , D. D. (1956). Funoticnal models and ex,perimental des:tgns for oharac terizing
response curves and surfaces j> p. 76-98 in: Methodologj.cal prooedures in the
economic analysis of fElrt:i.li~:.e!' use data (bused on a SYMPosium held in 1955
and edited by Baum, E. L., Heady, E. 0., Blaokmore, J.).
Ioym, State College
Press, xx + 218 p.
Vaart, H, R. van dar (1957)
0
Of biased estimators.
Yurt, H. R. van der (1958).
Some general remarks on the definition of the conoept
Institute of Statistics Mimoo. Series No.
~
18 p.
Soma results on the probability distribution of the
latent roots of a symmetric matrix of contil1uously distributed elements, and
some applioations to the theory of response surf'ace estimation.
Statistics Mimeo. Series No. 189, 1 +
40 + 2 p.
Institute of