7/29/2017
12:41 AM
Efficient Product Designs for Quadratic Models on the Hypercube
by
Rainer Schwabe
Weng Kee Wong
Otto von Guericke University
Department of Biostatistics
Institute for Mathematical Stochastics
School of Public Health
PF 4120
UCLA
39016 Magdeburg
Los Angeles, CA 90095
Germany
USA
Abstract
We propose a technique for obtaining improved efficiency lower
bounds for the best product design under a multifactor regression
model.
The method is illustrated for the general quadratic model
using the integrated mean squared error (IMSE) as our optimality
criterion.
Key words and phrases:
additive models, continuous designs,
equivalence theorems, integrated mean squared error.
1
7/29/2017
12:41 AM
1.
Introduction
The purpose of this paper is to provide improved efficiency
lower bounds for product designs in multi-factor experiments.
We
focus on the general quadratic models because they are widely used in
industry.
For example, Snee (1975) used the general quadratic model
and found efficient designs in mixture experiments, and Lucas (1976)
compared various competing designs for estimating the response
surface of the quadratic model.
Recently, this model was also used
to analyze a Phase II anti-anginal clinical trial (Peace, 1990).
Using the integrated mean-squared error as our optimality criterion,
we show that our bounds are substantially better than currently
available bounds for product designs.
In addition, we show that the
best product designs are very efficient designs.
Throughout, we consider a multifactor experiment with k factors
k
defined on a set T = X Ti, where each Ti is a compact subset of R1.
i1
An
element in T is denoted by tT = (t1, ..., tk), ti ΠTi, i = 1, ..., k.
The model of interest is
E y(t) = b0 + bTf(t),
with an explicit constant term b0.
t ΠT,
(1)
The vector of parameters is (b0,
bT) and f(t) is a known vector of regression functions. Additionally,
we assume that var(y(t))=1 and all observations are uncorrelated.
It is instructive to first consider the special case when we
have an additive model with f(t)T = (f1(t1)T,..., fk(tk)T)
and the
components of each fi(ti) are linearly independent. The additive
model is popular (Cook and Thibodeau, 1980, Rafajlowicz and Myszka,
2
7/29/2017
12:41 AM
1992, and Wong, 1994) because it does not permit interactions among
factors
and is one of the simplest models for analysis.
In practice, each factor in a multi-factor experiment may affect
the response differently.
The marginal models are given by
E y(ti) = b0,i + biT fi(ti),
ti ΠTi,
where bi is a pi x1 vector of parameters.
i = 1, ..., k,
In what is to follow,
these marginal models play a crucial role in the construction of
optimal designs for estimating the parameters in (1).
k
Let p = 1 +
p
i
and consider the px1 vector F(t)T = (1, f(t)T).
i1
The normalized information matrix of any design x on T for model (1) is
given by M(x) = ÚT F(t)F(t)T x(dt).
Likewise, if Fi(ti)T = (1, fi(ti)T),
the normalized information matrix of any desgin xi on Ti for the ith
marginal model is Mi(xi) = ÚTi Fi(ti)Fi(ti)T xi(dti).
Let µ(t) = Ey(t)

be the mean response in model (1) and let (t) be the best linear
unbiased estimate of µ(t).





We have (t) =  0 +  Tf(t), where  0 and 
are the best linear unbiased estimates of the parameters b0 and b
respectively.

The normalized variance of the predicted response (t) at

the point t using design x is varx (t) = F(t)TM(x)-1F(t).
If l(t) is the
normalized weighting measure on T, i.e. l(T) = 1, the integrated mean
square error (IMSE) of using design x can be variously written as


IMSE(x) = ÚT varx( (t) )l(dt) = ÚT Ex( (t) -µ(t))2l(dt) = tr M(x)-1M(l).
The IMSE-optimal design x* is the one that minimizes IMSE(x) over the
set of all designs x on T.
3
7/29/2017
12:41 AM
In what is to follow, we consider only continuous designs, in
the sense that designs are treated as probability measures on T
(Kiefer, 1959). There are many advantages in this approach.
For
example, in our setup, for fixed l, the IMSE-criterion is convex in
M(x) and so the optimality of a design x* can be verified by checking
if x* satisfies
F(t)TM(x*)-1M(l)M(x*)-1F(t) - tr M(x*)-1M(l) ≤ 0
for all t ΠT.
Technical details leading to this inequality comes from an
equivalence theorem, see Kiefer (1974).
The IMSE-optimal design x*
can be used to measure the worth of any non-singular design x
calculating the value of tr M(l)M(x*)-1/tr M(l)M(x)-1 .
by
This ratio is
called the IMS-efficiency of the design x and is denoted by
effIMSE(x).
2.
Efficiency Lower Bounds
Efficiency lower bounds are useful when the optimal design is
unknown or is too laborious to compute.
Designs with high
efficiencies signify that the designs are close to the optimum.
The
simplest way of obtaining efficiency lower bounds is from directional
derivatives arguments, see Kiefer (1974) for example.
For the IMSE-
optimality criterion with weighting function l, one obtains for model
(1),
effIMSE(x) ≥
2 - max F(t)T M(x)-1M(l)M(x)-1F(t)/tr M(x)-1M(l).
tT
A drawback of this approach is that the bound can give negative
values.
Numerical examples are given in Schwabe and Wong (1999)
4
7/29/2017
12:41 AM
where they considered ANOVA types of situations and the criterion is
A-optimality, corresponding to the case when M(l) is the identity
matrix.
Dette (1996) proposed a different efficiency lower bound
based on a concavity argument :
effIMSE(x) ≥
tr M(x)-1M(l) / max F(t)T M(x)-1M(l)M(x)-1F(t).
tT
We will call the above bounds Kiefer's and Dette's respectively.
3.
Efficiencies of the Best Product Designs for Additive Models
Our focus is on product designs which have the form x = x1 x ...
x xk and each xi is a design defined on Ti, i = 1, 2, ..., k. These
designs are easy to construct since they are pieced together from
designs from the 'smaller' models. An increasingly popular design
strategy is to restrict the search for the optimal design within the
class of product designs (Lim and Studden, 1988, Schwabe, 1996).
The
resulting optimal design is the best product design.
It is self-evident that product designs will perform well only
if the model assumptions are correct.
Thus, to improve the
efficiency lower bound for an additive model, we assume that the
weighting measure l has a product structure itself with l = l1 x... x
lk.
This assumption is essential for the validity of efficiency
lower bounds to be derived.
Let r(x) = ÚT f(t) x(dt) and let C(x) = ÚT f(t)f(t)T x(dt) r(x)r(x)T be the information matrix for estimating the parameter b.
If x has a non-singular information matrix, we have
5
7/29/2017
12:41 AM
 1

r( )T


M(x) = 

r( )  T f (t) f (t)T  (dt)
1 r( )T C( )1 r( ) r()T C( )1

.


1
1
C( )
 C( ) r( )

and M(x)-1 =
When x is a product design, it can be shown that C(x) is block
diagonal and its ith diagonal block Ci(xi) = ÚTi fi(ti)fi(ti)T xi(dti)
- ri(xi)ri(xi)T is the information matrix for  i in the ith marginal
model and ri(xi) = ÚTi fi(ti) xi(dti), i = 1, ..., k.
Because the IMSE-criterion depends only on the mean response
function, it is invariant with respect to scale transformations.
Thus, we may, without loss of generality, assume that li(Ti)=1, i =
1, ..., k. For the k-factor additive model and product design x = x1
k
x .... x xk, we then have IMSE(x) = 1 +

((ri(xi)- ri(  i))T Ci(xi)-
i1
(ri(xi)- ri(  i)) +
1
k

tr Ci(xi)-1Ci (li) .
Consequently, the
i1
following properties apply:
k
1.
IMSE(x) =
 IMSE ( )  (k 1) for
i
i
any product design x = x1 x .... x xk,
i1
where IMSEi(xi) = ÚTi varxi i (t i )i (ti ) = tr Mi(xi)-1Mi(li) is the marginal
IMSE-criterion and µi(ti) = E(yi(ti)) = b0,i + biTfi(ti) is the mean
response for the ith marginal model, i = 1, ...., k.
If xi* is IMSE-optimal with respect to  i in the ith marginal
2.
model, then x1* x .... x xk* is the best product design with

respect to l = l 1 x .... x lk under the IMSE-criterion.
To develop similar efficiency lower bounds as in Schwabe and
Wong (1999) for the A-criterion, consider the adjusted IMSE defined
6

7/29/2017
12:41 AM




by IMSEadj(x) = ÚT varx( (t) -  0 )l(t), where  0 = ÚT (t) l(dt).
This is
the IMSE for the estimate of the regression function µ-µo adjusted for
the general mean µ0 = ÚT µ(t)l(dt)
The concept here is the
counterpart to that of taking contrasts in ANOVA models and is
similar to using A-optimality for a set of orthogonal contrasts.



From (t) -  0 -(µ(t)-µ0) = (  -b)T(f(t)-r(l)), it follows that
k
IMSEadj(x) = tr C(x)-1C(l) =
 IMSE
adj
i
( i) , where IMSE i (xi) = tr Ci(xi)adj
i1
1
Ci(li) is the marginal adjusted IMSE-criterion.
Let xi* be the IMSE-
optimal design and let ˜ i be the IMSEadj-optimal design for the ith
The latter design ˜ i minimizes tr Ci(xi)-1Ci(li), i =
marginal model.
1, ..., k.
For any design x, let x1, ..., xk be the one dimensional
projections (marginals) onto its components.
Then by a refinement
argument, we have
IMSE(x) = tr M(x)-1M(l)

≥ tr M1(x1)-1M1(  1)+ tr C2(x2)-1C2(l2) + … + tr Ck(xk)-1Ck(lk)
= IMSE1(x1)+
k
k
i 2
i 2
 IMSE iadj( i) ≥ IMSE1 (1*) +  IMSE iadj (˜ i ) .
Replacing the first factor by any other yields
k
IMSE(x) ≥

i1

adj
IMSE i ( ˜ i)+ max
1≤i≤k
{IMSEi(x i )-IMSE i ( ˜ i)}.
*
adj
This implies for the best product design x1* x ... xk*, we have
k

effIMSE(x1* x.... x xk*) ≥
˜ )  max {IMSE ( * )  IMSE adj ( )}
IMSE iadj (
i
i
i
i
i
1 ik
i1
.
k
 IMSE (
i
i1
Here the denominator comes from property 1.
7
*
i
)  (k 1)
7/29/2017
12:41 AM
4.
Application to the Additive Quadratic Model
We now apply the results in Section 3 to the case when all
marginal models are quadratic, i.e. fi(ti) = (ti, ti2)T, Ti
and li is uniform on Ti, i = 1, ...., k.
= [-1, 1]
Our choice of the IMSE
criterion is motivated by the fact that it is invariant with respect
to linear transformations in a regression setup when a uniform
weighting measure is used.
In addition, explicit orthogonality of
the regression functions is not required because the IMSE criterion
depends only on the shape of the response function and not on the
particular parameterization.
0 1/3
 1
Mi(li) =  0 1/3 0 
1/3 0 1/5
A direct calculation shows
and
1/3
0
Ci(li) =  0 4/45
, i = 1,...,
k.
Note that both the IMSE-optimal design xi* and the IMSEadj-optimal
design ˜ i are symmetrically supported at ±1 and 0 with weight a/2 at
±1 and 1-a at 0.
For such symmetrical designs, we have
1 0  


Mi(xi) = 0  0 


 0  
and

0 

Ci(xi) = 

, i = 1,..., k.
0  (1  )
A direct calculation shows that IMSEi(xi) = 8/{15a(1-a)} and
adj
IMSE i (xi) = (19-15a)/{45 a(1- a)}.
It follows that the optimal
weights for the IMSE-optimal marginal design xi* and the adjusted
IMSE-optimal design ˜ i are a = 1/2 and a =
respectively.
19 /(2+ 19 )= 0.6855
adj
Note that IMSEi( ˜ i) = 2.4737, IMSE i ( ˜ i) = 0.8986,
8
7/29/2017
12:41 AM
IMSEi(xi*) = 2.133 and IMSE i (xi*) = 1.0222, implying that the IMSE
adj
and the adjusted IMSE optimal designs have different performances.
If we substitute these values into the above lower bound, we obtain,
for example, 0.9281 and 0.8286 for k = 2 and 10 respectively (Table
1).
Table 1 Here
It should be noted that the new lower bound decreases to as low
as 0.7929 when k gets large, while the traditional bounds decline to
zero.
To assess the precision of these lower bounds, the actual
optimal designs have to be determined.
Using a symmetry and a
majorization argument, we can search for the optimal design among
designs x, whose information matrices have the form
 1

M(x) =  0

aI
 k
0
aI k
0


.
0

T
(a  b)Ik  b1k1k 

T
a1k
Here, a = ÚT ti2d x = ÚT ti4d x and b = ÚT ti2tj2d x, (i ≠j) are
determined by the one and two-dimensional marginals of x.
This means
that the design x is concentrated on the corner points, midpoints of
vertices and faces, etc. of the hyper-cube and have at most 3k
support points of the full 3 factorial.
A direct calculation shows
that the
integrated mean-squared error for the additive quadratic model is
IMSEadd(x) = 1 +
4(k  1)
45ka2  30ka 5k  4
k
+
+
.
2
45(a  b)
45(a  b  bk  ka )
3a
9
7/29/2017
12:41 AM
This expression is minimized numerically by choices of a and b to
yield the desired moments of the optimal design.
the designs here for space consideration.
We do not present
Table 1 compares the
values of the new efficiency lower bound of the best product designs
for the additive quadratic model with corresponding values obtained
from Kiefer's and Dette's bounds.
Numerical calculations show that
for large values of k, the actual efficiency decreases also to 0.7929
showing that the above bound is asymptotically sharp.
5. Efficiencies of the Best Product Designs for the General
Quadratic Model
When interactions are expected to occur among factors, a popular
model is the general quadratic model given by
k
E(y(t)) = b0 +
 t
i i
k
+
i1
 t
ii
2
i
k
k
   tt .
+
ij i j
i1 j i 1
i1
Nalimov, Golikova and Mikeshina (1970) provides a detailed discussion
of design issues and applications of this model in practice.
Again,
we assume that l = l1 x .... x lk on [-1,1]k and each li is uniform.
The IMSE criterion is invariant with respect to sign changes and
permutations of the factors.
Together with its convexity property,
we can use a symmetry argument and restrict our search for the
optimal design to designs x with information matrices of the form
 1

 0
M(x) = 
a1k

 0
0
a1Tk
aI k
0
0
(a  b)Ik  b1k1k
0
0
T
10


0 
.
0 

bIk (k1) / 2 
0
7/29/2017
12:41 AM
Here again, a = ÚT ti2d x = ÚT ti4d x and b = ÚT ti2tj2dx, i ≠j. We now
apply a similar argument as in Lim and Studden (1988) for D-optimality
to the IMSE-criterion, i.e. the directional derivative occurring in
the corresponding equivalence theorem (see Kiefer, 1974) is a quartic
polynomial in each factor.
The optimal design x* is supported at the
maxima of that function and so x* is concentrated on the corner
points, midpoints of vertices and faces of the hyper-cube. Because of
the convexity of the moment space, the number of design points can be
reduced to three sets of barycenters.
It can be shown that these
points consist of the origin, the corner points and a third set of
points with exactly j entries from ±1 and k-j entries equal to 0,
where |j-ka| ≤ 1.
For the uniform weighting measure l = l1 x .... x
lk on [-1, 1]k,

 1


 0
M(l) = 
1 1
3 k

 0

0
1 T
1k
3
1
Ik
3
0
0
4
1
Ik  1k1Tk
45
9
0
0




0 
.
0 


1
I k(k 1)/ 2 

9
0
The forms of the above two matrices imply that the IMSE-optimal
design minimizes IMSEadd(x) + k(k-1)/18b, where IMSEadd is the
criterion function in the additive quadratic model of the previous
section.
Table 2 below shows characteristics of the IMSE-optimal
product designs along with their efficiencies.
The minimum
efficiency of the product design in the table is about 77% and this
occurs when k = 49.
The values of a* and b* are the optimal first and
second order moments of the IMSE-optimal design x* and a˜ , b˜ denote
the corresponding
11
7/29/2017
12:41 AM
moments for the best product design.
a˜ and b˜ depend on the dimension k.
It is interesting to note that
For k large, the limiting
marginal designs become concentrated on the boundaries of the
interval (a>1) as the k(k-1)/2 interaction terms titj dominate the
model.
The efficiency lower bounds of the best product designs for the
general quadratic model are obtained in a similar manner as in
Section 3.
It is straightforward to show that the efficiency lower
*
bound of the best product design is the ratio of IMSE1(x 1 ) + (k-1)
IMSE 1 ( ˜ 1) + k(k-1)/18 and the IMSE of the best product design.
adj
Table 3 lists the values of the various lower bounds and again, the
superiority of the new bound is evident.
For large k, the
traditional bounds decrease to 0 but the new bound tends to 1 and so,
again, asymptotically sharp.
Table 2 Here
Table 3 Here
6.
Concluding Remarks
Product designs can be used as building blocks for constructing
highly efficient designs.
If one additionally wishes to restrict the
number of design points in product designs, they can easily be
emulated by suitable fractional factorials or orthogonal arrays.
The proposed method in this paper is not limited to the IMSE
optimality criterion. Various extensions are possible. For instance,
if we wish to find a design on T to minimize the optimality criterion
12
7/29/2017
12:41 AM
ÚT' varx(µ(t))l(dt) where the sets T' and T are different, the method
also applies provided T and T' both have product structures and the
weighting measure on T' is a product measure.
Other generalizations
include replacing the L2-norm in the IMSE-criterion by the more

flexible criterion { ÚT{var( (t) )}p/2 l(dt)}1/p. The case p = •
corresponds to the maximum norm, which is the same as G-optimality
discussed in Schwabe and Wong (1997).
Acknowledgements
Essential parts of this work were done while Schwabe was
visiting
the Technical University of Darmstadt.
His work was partly supported
by grant Ku719/2-2 of the Deutsche Forschungsgemeinschaft.
The
research of Wong is partially supported by a NIH research grant R29
AR44177-01A1.
We thank the referees for their helpful comments.
References
Cook, R. D. and Thibodeau, L. A. (1980).
optimal designs.
Marginally restricted D-
Journal of the American Statistician Association,
Vol. 75, 366-371.
Dette, H. (1996).
Lower bounds for efficiencies with applications.
In Research Developments in Probability and Statistics.
Festschrift
in Honor of Madan Puri on the Occasion of his 65th Birthday Editors:
E. Brunner, M. Denker), 11-124, VSP, Utrecht.
13
7/29/2017
12:41 AM
Kiefer, J. (1959).
Optimum experimental designs.
J. Roy. Statist.
Soc., Ser B, Vol. 21, 272-319.
Kiefer, J.
(1974). General equivalence theory for optimum designs
(approximate theory).
The Annals of Statistics, Vol. 2, 849-879.
Lim, Y. B. and Studden, W. J. (1988).
Efficient Ds-optimal designs
for multivariate polynomial regression on the q-cube.
The Annals of
Statistics, Vol. 16, 1225-1240.
Lucas, J. M. (1976).
Which response surface design is best: a
performance comparison of several types of quadratic response surface
designs in symmetric regions. Technometrics, Vol. 18, 411-418.
Nalimov, V. V., Golikova, T. and Mikeshina, N. (1970).
use of the concept of D-optimality.
Peace, K. E. (1990).
development.
On practical
Technometrics, Vol. 12, 799-812.
Statistical Issues in drug research and
Marcel Dekker, Inc.
New York and Basel.
Rafajlowicz, E. and Myszka, W. (1992).
experimental design is optimal?
When product type
Brief survey and new results.
Metrika, Vol. 39, 321-333.
Schwabe, R. (1996). Optimum designs for multi-factor models.
Notes in Statistics, Vol. 113.
14
Springer.
Lecture
7/29/2017
12:41 AM
Schwabe, R. and Wong, W. K. (1997).
A relationship between
efficiencies of marginal designs and product designs.
Metrika, Vol.
45, 3, 253-258.
Schwabe, R. and Wong, W. K. (1999).
designs in linear models.
Efficiency bounds for product
Annals of the Institute of Statistical
Mathematics, Vol. 51, 723-730.
Snee R. D. (1975).
Experimental designs for quadratic models in
constrained mixture spaces.
Wong, W. K. (1994).
Technometrics, Vol. 17, 149-160.
Multifactor G-optimal designs with
heteroscedastic errors.
Journal of Statistical Planning and
Inference, Vol. 40, 127-133.
15
7/29/2017
k
2
3
4
5
6
7
8
9
10
100
1000
Kiefer’s
0.9319
0.8484
0.7590
0.6666
0.5726
0.4776
0.3818
0.2857
0.1891
0
0
12:41 AM
Dette’s
0.9363
0.8684
0.8058
0.7500
0.7005
0.6568
0.6180
0.5833
0.5522
0.0941
0.0101
new
0.9281
0.8932
0.8727
0.8591
0.8495
0.8423
0.8367
0.8323
0.8286
0.7967
0.7932
actual
0.9954
0.9901
0.9847
0.9795
0.9745
0.9698
0.9655
0.9613
0.9575
0.8676
0.8189
Table 1: Efficiency bounds (columns 2-4) and efficiencies of the best
product design for the additive quadratic regression model with k factors.
k
2
3
4
5
6
10
49*
50
100
1000
a*
0.5467
0.5838
0.6137
0.6381
0.6584
0.7153
0.8595
0.8609
0.9019
0.9704
b*
0.3643
0.4093
0.4458
0.4760
0.5013
0.5731
0.7678
0.7698
0.8310
0.9440
IMSE(x*)
3.5862
5.2754
7.1635
9.2326
11.4732
22.0561
245.7242
254.1757
841.6582
62673.00
a˜
0.5407
0.5683
0.5894
0.6064
0.6207
0.6619
0.7873
0.7888
0.8352
0.9372
2
b˜ = a˜
0.2923
0.3230
0.3474
0.3678
0.3853
0.4381
0.6198
0.6221
0.6976
0.8784
IMSE
efficiency
3.6752
0.9757
5.5538
0.9498
7.7342
0.9262
10.1942
0.9056
12.9181
0.8884
26.2447
0.8404
318.865
0.7706
329.825
0.7706
1076.95
0.7815
71250.0
0.8796
Table 2: Characteristics of the best product IMSE-optimal designs for the general
quadratic model.
k
2
3
4
5
6
7
8
9
10
50
100
1000
Kiefer's
0.7857
0.4940
0.1484
0
0
0
0
0
0
0
0
0
Dette's
0.8235
0.6640
0.5401
0.4462
0.3748
0.3196
0.2764
0.2419
0.2139
0.0276
0.0114
0.0007
new
0.8552
0.7677
0.7105
0.6708
0.6419
0.6202
0.6035
0.5904
0.5799
0.5526
0.5952
0.7915
actual
0.9757
0.9498
0.9262
0.9056
0.8881
0.8732
0.8605
0.8497
0.8404
0.7706
0.7815
0.8796
Table 3: Efficiency bounds (columns 2-4) and efficiencies of the best
product designs for the general quadratic model with k factors.
16