Estimating Variance Components Using
Alternative Minqe's in Selected Unbalanc
Designs
John Clare Brocklebank
lnst. of Statistics Mimeo Series # 1394
iv
TABLE OF CONTENTS
Page
LIST OF TABLES . .
.........
..
LI ST OF FIGURES
1.
INTRODUCTION AND REVIEW OF THE LITERATURE
1
1.1 Introduction . . . . . • . . • • .
1.2 Reyiew of the Literature . . . •
1
1
1.2.1 Methods of Estimation . .
. •.
1.2.2 Comparison of Estimators . . . . . . . •
1.2.3 A Comparison of Designs and Estimators.
1.3 Outline of This Research
2.
3.
..
...
SOME BASIC DEFINITIONS AND RESULTS •
18
18
20
2.1 Introduction . . . • . . . .
2.2 The Form of the General Model . . • .
2.3 Minimum Norm Quadratic Estimation .
2.4 Quasi Inner Products . . . . . . . • . . .
20
20
22
23
MINIMUM NORM QUADRATIC UNBIASED TRANSLATION INVARIANT
ESTIMATORS . . . • . . . .
. . . .
26
3.1 Introduction . . . . . . . . . .
3.2 ~lINQE(U,I) Sol utions . . . . . . .
3.3 Estimation in the Nested ~bdel
3.4 The Risk Function for MINQE(U,I) . . • • . . . .
3.5 A Combined Estimator Using MINQE(U,I) and
Henderson's Method III for the Error Component.
3.6 The Risk Function for MINQH .
4.
.3
16
DESIGN EVALUATION
... . . . . . .
4.1 Introduction . . . . . . . .
4.2 Design Fundamental Structures
4.3 Designs of Historical Interest Included in the
Enumeration . . . . . . . . . . .
.
4.4 Risk Studies for the MINQE(U,I), ANOVA, and the
Combined MINQH Estimators
.
4.5 Design Evaluation for ANOVA, MIVQE(U,I) and MINQH
Estimators . . . . . . . . . . . . . .
4.6 Variance Comparison of MINQE(U,I), ANOVA, and
MINQH Estimation
. . . .
4.7 ~lINQE(U, 1,1)
4.8 r~HIQE(U,I ,0)
26
26
29
40
47
50
53
53
53
61
62
63
67
90
100
v
TABLE OF CONTENTS (continued)
Page
5. MINIMUM NORM QUADRATIC UNBIASED ESTIr~TION WITHOUT
INVARIANCE • • . •
. . • • • . • • •
111
5.1 Introduction .•
• • • •.
\ j' 5.2 MI NQE (U) • • . . . . . . • . • • . . . . . . • •
\ 5.3 Explicit MINQE(U) for the One-Way Classification.
111
113
116
6. MINIMUM NORM QUADRATIC ESTIMATION UNDER SOME
SPECIALIZED CONDITIONS • . . . . .
. . • . . . • ..
124
6.1
Introduction • . . • • • • • . • . • • . . • •
6.2 MINQE, MINQE(O), and MINQE(I) . . • . . . .
6.3 Explicit MINQE without Unbiasedness for the
124
125
.
One-Way Classification . . • • • •
. .
Explicit MINQE . . . . • • • • . • . . . • •
6.5 Explicit MINQE(O) . • . . . . . • . . • . •
6.6 Explicit MINQE(I) . . . . . . • . • . . . . . . • • .
. 6.4
7. MINH1UM NORM QUADRATIC NON-NEGATIVE DEFINITE UNBIASED
ESTIMATORS . . • .
.
• . • .
7.1 Introduction . . . . . • . . . • . . . • •
7.2 MINQE(U,D) . . . . . . . . . . • . . . .
.v/ 7.3 Explicit MINQE(U,D) of e for the One-Way
Classification . . . : . . . . . . . .
8. SENSITIVITY OF THE VARIANCE AND MEArl SQW\RE ERROR FOR
MINQE( ) USING DIFFERENT A PRIORI AND UNKNO~JN VARIANCE
COMPONENTS . . . . .
. . • • .
8.1 Introduction . . . . . . . . .
8.2 MINQE(U)
8.3 Sensitivity of the Estimators .
129
132
137
144
146
146
146
151
153
153
153
154
9. SUMMARY OF THE RESULTS.
160
LIST OF REFERENCES .
162
10.
11. APPENDIX . . . . .
Analysis
Nested
11.2 Variance
(under
JILl
. . . .
. . . .
of Variance Estimators fl)r the TWO-Way
Classification . . . . . . . . . . .
of Analysis of Variance Estimators
Normality) . . . . . . . . . . . . . . .
167
167
167
vi
LIST OF TABLES
Page'
4.2.1 Fundamental structures •.
54
4.2.2 Two-stage nested designs •
56
4.2.3 Similarities between enumerated designs and those
suggested by other workers • . • • • • • • • • •
57
4.2.4 Correspondence between the degrees of freedom and
the design codes
••.....•••••.
4.2.5 ANOVA table
•••••••
...
...
...
59
60
4.5.1 Optimal designs (codes) for MIVQE(U,I) when all
16 designs are compared • • • • • • • • • . •
65
4.5.2 Optimal designs (codes) for ANOVA estimators when
all 16 designs are compared . • . • • . • . • •
66
4.6.1 Priors that maximize the percentage of points where
the variance (MINQE(U,I))/variance (ANOVA estimates)
~ lover all unknown components
..•••.••..
69
4.6.2 Priors that maximize the percentage of points where
the variance (MINQH)/variance (ANOVA estimates) ~ 1
over all unknown components . • . . . . • . . .
80
4.7.1 The effects of priors equaling 1 on the ratio of
the variances over all unknown components
91
4.8.1 The effects of zero priors on the ratio of the
variances over all unknown components . . • . . . .
101
8.3.1 Net weights (oz) of vegetable oil fills
155
8.3.2 Estimates of 6a and ee using a priori values based on
ANOVA calculations of Sa = .0010365, 8e = .00773381,
and u = 15.67 . . . . . . . . . . . . . . . . . . .
157
8.3.3 Sensitivity of the variance and mean square error
when a priori values are exactly the true unknown
parameters for (ea = 0.0010365, 6 e = 0.0077381) /
(Sa = 0.0028, 6 = 0.0025) . . . . . . . . . . . .
e
158
A
vii
LIST OF FIGURES
Page
4.6.1 Ratio of the variance (MINQE(U,I))/viiriance (ANOVA)
72
for estimating the A component . • . • • . .
4.6.2 Ratio of the variance (MINQH}/varianc:e (ANOVA) for
estimating the A component . • . . • • • • • • .
....
4.6.3 Ratio of the variance (MINQH}/variance (MINQE(U,I))
for estimati ng the A component • ••
..•• .
4.6.4 Ratio of the variance (MINQE(U.I}}/var-iance (ANOVA)
for estimating the B component . • • • • . .
76
estima ti ng the B component . . • . • • . • • . .
4.6.6 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimating the B component . • • . • . . • • •
for estimating the E component . • . . • • .
74
75
4.6.5 Ratio of the variance (f4INQH)/varianc:e (ANOVA) for
4.6.7 Ratio of the variance (MINQE(U,I))/variance (ANOVA)
73
. ... .
4.6.8 Ratio of the variance (MINQH)/variance (ANOVA) for
estima ti ng the A component . . • . . • . . .
4.6.9 Ratio of the variance (MINQE(U,I))/variance (ANOVA)
for estimating the A component. . . . . . . . . .
4.6.10 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimating the A component . . . . . . •
4.6.11 Ratio of the variance (MINQH)/variance (ANOVA) for
estimating the B component . . . . . . . . . . .
77
78
82
83
84
85
4.6.12 Ratio of the variance (MINQE(U,I))/valriance (ANOVA)
for estimati ng the B component . . . . . • . . . .
4.6.13 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimating the B component . . . . . . . . . .
4.6.14 Ratio of the variance (MINQE(U,I))/variance (ANOVA)
for estimating the E component . . . . . . . . . .
4.6.15 Ratio of the variance (MINQH)/variance (MINQE(U,I))
4.7.1
86
87
88
for estima ti ng the E component . . .' . . . . . . .
89
Ratio of the variance (MINQE(U,I))/variance (ANOVA)
for estimating the A component . . . . . . . . . .
92
viii
LIST OF FIGURES (continued)
Page
4.7.2 Ratio of the variance (MINQH)/variance (ANOVA) for
es tima ti ng the A component • . • • • • . . • ••
. ' . ' 93
4.7.3 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimating the A component . . • • • . • . • .
4.7.4 Ratio of the variance (MINQE(U,I))/variance (ANOVA)
.
for estimating the B component • . . . . . .
4.7.5 Ratio of the variance (MINQH)/variance (ANOVA) for
es tima ti ng the B component . . . • • . . . • • .
94
...
95
....
96
4.7.6 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimating the B component . • • . • • . • • .
97
4.7.7 Ratio of the variance (MINQE(U,I))/variance (ANOVA)
for es tima ti ng the E component • • . . . . . . . .
98
4.7.8 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimating the E component . • . . . . • . • •
99
4.8.1 Ratio of the variance (MINQE(U,I))/variance (ANOVA)
for estimating the A component . . • . . . .
103
4.8.2 Ratio of the variance (MINQH)/variance (ANOVA) for
estimating the A component . . . . . . . . . • .
104
4.8.3 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimating the A component. . . . . . . . • .
105
4.8.4 Ratio of the variance (MINQE(U,I))/variance (ANOVA)
for estimating the B component . . . . . . .
106
4.8.5 Ratio of the variance (MINQH)/variance (ANOVA) for
estimating the B component . . . . . . . . . . . .
107
4.8.6 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimati ng the 8 component . . . . . . . . . .
108
4.8.7 Ratio of the variance (MINQE(U,I))/variance (ANOVA)
for estimating the E component. . . . . . . . . .
109
4.8.8 Ratio of the variance (MINQH)/variance (MINQE(U,I))
for estimating the E component . . . . . . . . . .
110
1.
INTRODUCTION AND REVIEl1 OF THE LITERATURE
1. 1 Introducti c~
During the past 30 years the estimaticln of variance components
in random and mixed linear models has received considerable attention
in the literature.
For mixed models the emphasis has been on finding
methods for estimating the variance components that are not biased
by the presence of the fi xed effects.
In r'andom effects mode1s ,
the major concern has been on finding procedures to estimate the
variance components.
While numerous procedures have been suggested
for obtaining estimates of the variance components and finding the
variances of these estimates, little work has been done in determining
the relative merits of some of the recent estimation procedures with
regards to various classes of design structures.
1.2 Review of the Literature
"
In this section we make no attempt to review all aspects of
variance component estimation theory since this has already been ably
done by Anderson (1979), Harville (1969, 1975, 1977) and Searle
(1971b).
Thus instead, the most significant contributions to the
following areas of variance component estimation will be discussed,
since they have some bearing on the results obtained in this study,
namely:
comparisons of the estimation
procl~dures,
estimators of variance components, and the
E~ffect
variance of the
of different design
structures.
R. D. Anderson (1979) provides an insi9ht into the evolutionary
process of variance component methodology up through 1966.
with a discussion of works
by
He begins
19th century astronomers and the early
2
development of analysis of variance (ANOVA) methods of variance
component estimation.
Some of the major contributions of R. A. Fisher
are highlighted and those early methods for estimating variance
components from unbalanced data (unequal number of subclasses) are
considered.
A discussion of the optimal properties of ANOVA and
Henderson's methods (1953) is followed by a brief description of
maximum likelihood (ML) and restricted maximum likelihood (REML)
approaches to variance component estimation.
Harville (1969) presents a comprehensive review and analysis of .
the literature dealing with the one-way random classification.
He
discusses the estimation of variance components when the data are
either balanced or unbalanced, and the effects on the estimation of
the variance components when some of the usual analysis of variance
assumptions are relaxed.
Se~rle
estimation.
(1971b) provides an extensive review of variance component
The article is broken into three parts.
The first
part discusses the models underlying variance component estimation.
The second part deals with the analysis of variance methods of
estimating variance components for balanced data, with prime considerations being unbiased estimators, negative estimates, minimum
variance, and the consequences of normality assumptions.
The third
part is primarily concerned with estimating variance components when
the data are unbalanced using analysis of variance method, adjusting
for the bias with mixed models, the fitting of constants method,
analysis of means method, symmetric sums, maximum likelihood, and
best quadratic unbiased estimation.
3
Harville 0975, 1977) describes recent developments in maximum
likelihood theory as applied to the estimation of variance components.
He examines the relationship between the mclximum likelihood estimation
of variance components and the estimation or prediction of the
model's fixed and random effects.
This
re~lationship
is' further ex-
ploited for the purposes of computation and approximation, and numerical algorithms for computing maximum likellihood estimates of variance
components are given.
The use of maximum li'kelihood as a vehicle for
relating various methods is proposed.
1.2.1
Methods of Estimation
Estimation of variance components from unbalanced data was done
using a standard method up until 1967.
In 1953 C. R. Henderson gave
the fi rst systemati c approach to the estima,tion of vari ance components
in random or fi xed mode 1s when the data are' unbalanced.
He sugges ted
the following three methods.
1.2.1.1
Henderson Methods.
In r1ethod I the sums of squares for
each source of variation in the analysis of variance table are
calculated as though the design were balancl:!d. Estimates of variance
components are Obtained by equating the various sums of squares to
their expected values and solving the resulting system of equations.
Searle (1968 ) examines this procedure usin'g a general linear model
approach and shows that Method I gives biased estimates of the
variance components if the model contains any fixed effects other
than the overall mean.
Method II ;s designed to correct these deficiencies by obtaining
the 1eas t squa re es ti rna tes
0f
the fi xed efft:!cts from the data and
4
adjust the data for these fixed effects. The estimates of variance
components are then obtained by applying Method I to the adjusted
data.
Searle (1968 ) points out that if the model does contain fixed
effects, then there are many ways in which the data can be corrected
for these effects depending on the choice of a generalized inverse
used in the estimation of the fixed effects.
This lack of specificity
results in the method being not uniquely defined and hence impractical.
Moreover, he shows that if there are non-zero interactions among the
fixed effects and random effects, then the method does not work.
In Method III the estimates of variance components are obtained
by computing the reduction in the sum of squares due to fitting
appropriate sub-models of the model being considered, and then equating these reductions to their expected values.
Searle (1968 )
points out, however, that due to the arbitrariness in the choice of
quadratic forms this procedure does not always lead to a unique set
of unbiased estimates of the variance components free of the fixed
effects.
This problem together with the major computing task involved
forced many people to use Henderson Hethod I.
During the years following Henderson (1953) the bulk of the
literature concerning variance component estimation was directly
related to Henderson1s Methods.
This includes many applications and
explanations of his methods, together with some exploration into
thei r properti es.
Not unti 1 Hartley and Rao (1967) descri bed
maximum likelihood (ML) methodology, based on normality assumptions,
did a new development in the area of variance components estimation
theory emerge.
5
The concept of maximum likelihood was not new but the application
to estimating variance components in the most general mixed model
was new.
Actually, Crump (1947) in his thesis had considered a
maximum likelihood approach for the simple one-way random model but
it was Hartley and Rao who first used it fi)r the general mixed model
with unbalanced data.
1.2.1.2 Seely Methods.
Seely (1970a .• 1970b) developed a
least squares theory for the quadratic
functions of the form Li
So
j
est~imability
for parametric
Aij SiBj + Lk ,lkak and procedures for
obtai ni ng quadratic estimators for such pa,"ametri c functi ons for a
random vector Ywith expectation XS and covariance L aiV i (a 1 , •• .,a m,
and 6 denote parameters). Special interest is given to parametric
functions of the form LkAk6k.
His results concern sufficient and nec:essary conditions for the
existence of unbiased estimators of the form Y'AY where A is a real
matrix.
If one denotes x1, ... ,X p as the columns of the matrix X
and defines
(1.2.1)
= X·X'·
+ X·X~
B..
lJ
1 J
J 1
He shows that from
6
Ii
-, ,/
tr(B11B\~) tr(B 11 V1)
(1.2.2)
the parametric function Z = r
and only if there exists a
p
i
~ j
A.. B.S· + rkAkek 'is estimable if
lJ 1
J
such that H*Hp = A or similarly if
one can show the system of matrices B11 , B12 , ... , Bpp ' V1, ... , Vm
are linearly independent, the resulting system is
(H*H)E
= H*U
(1.2.3)
for (H*U)' = (Y'B 11 Y, ... , Y'B pY, Y'V 1Y, ... , Y'VmY) and
2
2
P
E = (B , B B , .. ., B ' 8 , ... , 8 ),
p 1
1 12
m
For estimating Lk Ake k he works with the matrix
tr(VIP~~}
Q*~~
(1.2.4)
=
)
tr(V mpJ-V
xm
7
and shows that necessary and sufficient conditions for Lk Ak6k to be
estimable are the existence of a
p
such that Q*Wp = A.
The estimates are obtained by solving the following:
(Q*W ) a = \~*U
where (W~)'
(1.2.5)
= [Y'P~lY'
•.. , Y'P~VmY]'
This second method reduces the number of equations but both
estimates lack invariance.
Immediately following this came the minimum norm quadratic
estimators (MINQE) with several variations
(~.~.,
MINQE(U,I), MINQF(U,I,O),
I-MINQE(U,I), MIVQE(U,I)), and also restricted maximum likelihood
(REMl) which is just ML under a transformed model.
Thus, today we have not only Henderson's three methods but at
least a half dozen other named methods as wlell.
As we shall see,
not all of these methods are different from one another and the
latter are all part of the dispersion mean model.
In order to
briefly discuss these estimators we will neled some notation as
follows.
Let Y = vector of N observations having mean Xu and variancecovariance (dispersion) matrix Var(Y)
y
= Xu
= V,
~Lo~.,
Y - (Xu, V):
+ Ub + e is the equation of the model,
(1.2.6)
u = a vector of unk nown fi xed effects pa rameters ,
X = known matrix, often an incidence matrix of O's and lis,
but sometimes includes columns of covariates,
b
= rbi,
b~,
o.
0' b~J' is the vector of random effects,
8
bi - (O, ail n;) is the vector of n; effects for the levels of
the ith random factors occurring in the data, i.~., main
effect or interaction factor,
= 0,
cov{b i , bj)
U
for i .,. j ,
= [U 1 ,
U2 , ••• , Uh] is a known incidence matrix, partitioned
conformably with b for the product Ub,
e - (0, aeI N) is a vector of random errors,
V
= Var{Y} = ae IN
e
= [6 1 ,
h
+
r
i=l
V.e. ,
1 1
6 , ... , eh' ee] ,
2
Pv = X(X'V-1X}-X'V- 1 ,
R = v-I (I - PV) ,
V
P~= I-X(X'X}-X' .
1.2.1.3 Maximum Likelihood (ML).
Hartley and Rao (1967)
rewrote the model (1.2.6) as
h+1
Y=Xu+
')
i~1
h+1
U.b.-N(Xu,
1 1
L
i=1
a.V.)
(1.2.7)
1 1
= 6e
and Vh+1 = IN' then simply derived equations for
obtai ni ng r·1L estimators. ,Although these !'1L equati ons have a vari ety
for sh+1
of forms, one such form suggests solving
(1.2.8)
9
and for i,j
= 1,
••• , h+1,
(1.2.9)
-
-
-
where a is a solution to (1.2.9) corresponding to a,and V and RV
are V and RV with a in place of a.
-
Solutions to (1.2.9) are obtained analytically for all completely
nested models and for certain crossed classification models (see
-
Szatrowski (1980)); otherwise, a must be solved iteratively.
the solution of (1.2.9) for
e is
Also
not necessarily the ML estimator.
Maximum likelihood theory demands that the ML equations be solved in
the parameter space.
Since the parameters in a are variances, the
ML estimator for a is a solution to (1.2.9) constrained so that 6h+1
and 6i ~ 0, i = 1, .•. , h. This can produce difficulties when
>
computing, depending on what algorithms are used for the iterative
process as described by Hemmerle and Hartley (1973) and Jennrich
and Sampson (1976).
Several alternative expressions for iterative
computing techniques are also available (see Harville (1977)).
1.2.1.4 Restricted Maximum Likelihood (RErL).
Patterson and
Thompson (1971) considered restricted maximum likelihood or REML
by extending the ideas of Thompson (1962) who suggested estimating
variance components by maximizing that portlion of the likelihood
which is invariant to the mean.
block information . . .
11
Their article IIRecovery of inter-
considers maximizing the likelihood on just
that portion of the set of sufficient statistics which is locationinvariant.
One form of the REML equations indicates solving
(1.2.10)
0
10
One should observe that the REML of
e can
be obtained from
(1.2.10) only by iteration similar to the ML solution of
(1.2.9).
There is no estimator of
u implicit
e from
in the REML method.
The right-hand sidesof the REML and ML equations are the same. The
left-hand side of the REML equations are the same as the ~L equations
when RV is replaced with.V- 1; and when X = 0 they are equal.
Numerical experience in using various iterative algorithms to
compute ML or REMl estimates of variance components seems to be very
limited and is largely confined to a variation of the methods of
steepest ascent by Hartley and Vaughn (1972), the Newton-Raphson
procedure
(Hemmerle and Hartley (1973); Corbeil and Searle (1976 );
. Jenrich and Sampson (l976))-; the methods of scoring (Jennrich and Sampson
(1976); Giesbrecht and Burrows (1978); Kleffe (1980)); and Anderson's
method (Miller (1973)).
Most of the recent literature on the problems of estimating
variance components has centered on the derivation of estimators
that have minimum mean squared error at some point in the parameter
space, ar that they are locally best when the estimator is restricted
to certain conditions.
Early work done by Townsend (1968) and by
Townsend and Searle (1971) explicitly described estimators of variance
components for the one-way model that are unbiased, quadratic functions of the observations, and are locally best under the assumptions
that Y is normal and has mean vector
o.
Thus by selecting a sym-
metric matrix A so that Y'AY is unbiased for the variance components
being estimated one wishes that the
var(Y'AY)= 2 tr(AV)2 + 4u'X ' AVAXu
(1.2.11)
11
is minimized.
Harville (1969) considered the same setting but dropped
the assumption that the mean vector is null.
He gave some results on
estimators that are locally best in the class of quadratic unbiased
.
estimators and in the class of translation invariant quadratic un- .
biased estimators, though his results were in an inconvenient.form.
These ideas have been greatly improved in papers by La Motte (1970,
1971, 1973b), C. R. Rao (1970, 1971a, 1971b, 1972, 1973, 1979) and Rao
and Kleffe (1979).
The extensions also include translation invariance
which is very similar to the restricted aspe1:t of REMl, namely confining attention to location invariant statistil:s.
For cases without
norma 1i ty the second term in (1. 2.11) is rep'laced wi th tenns in
kurtosis parameters as indicated in Pukelshe"im (1977) and Anderson,
Quaas and Searle (1977).
1.2.1.5
r-1inimum
Invariant Estimation.
~form
Quadratic Unbiase:d and Translation
In Rao's work on variance component estimation
he considers the effects of minimizing the Euclidean norm rather than
minimizing a variance which can be done without concern for the
distributional form of the model of interest.
j·1INQE(lJ,I) theory
leads to minimizing the tr(A Va )2 where Va is V with the variance
components 3 replaced by pre-assigned values of the variance
components represented as
(The E in
~.
estimation or their pluralS.)
milQE
can stand for estimator,
The resulting equations for
estimating a are then for i ,j = 1, ... , h+l:
{tr(R a V.R
V")}~
, a J
a
= {YIR
3
V.R
Y}
1 a
where R is R'f with Va in place of V.
~
( 1. 2.12)
C. R. Kao (1979 ) shows the
12
solution of (1.2.12) is not unique unless a is identifiable on the
basis of the distribution of ~Y (or unless the matrices ~Vi~ ,
= 1,
i
... , p, are linearly independent).
Because a is a vector of
known priors (1.2.12) can be solved non-iteratively for the
estimator
ea
and thus
ea
is a function of a. Thus MINQE(U,I} is not
an iterative process and is dependent upon the a vector.
tions have also been derived by setting ai
= 1 and
The equa-
considering the
residuals from a weighted least squares analysis as shown by Pringle
(1974).
Of special interest is the fact that equations (1.2.12) are
exactly the same as the REML equations (1.2.10) with RV replaced by
R. The REML estimators are obtained from solving the REML equations
a
for some initial value given to a, and thus to V and R . Hence, the
V
solution obtained from the first iteration of REML will be exactly
the same as solving the equations using values for a. Thus correspondingly we have that the first iterate of
REr~
under the
normality assumption is a MINQE(U,I}.
1.2.1.6 Iterated MINQE(U,I) (I-MINQE(U,I)}. After obtaining
ea
from the MINQE(U,I} equations corresponding to a, C. R. Rao
(1972) suggests the use of the values e
a
as new prior weights a and
calculate the system of equations again to obtain possibly different
A
values for ea
This process could be be repeated until two
successive values of
aa
are equal to within some degree of approx-
imation. The resulting values are
iterative MINQE(U,I), named
I-MINQE(U,I) by Brown (1976) who also shows that MINQE(U,I} and
I-MINQE{U,I) are asymptotically normal.
Thrum and Schmidt (1979)
have developed a general asymptotic theory of rtINQE(U,I} and
13
I-MINQE(U,I) that is based on quite arbitrary sequences of variance
components models. They found surprising1j' simple and general conditions for strong and weak consistency of MINQE(U,I) if V = L aiVi is
such tha t ai
>
0 for all i
= 1,
•.• , h.
Brown (1976) proved the asymptoti c
normality of MINQE(U,I) for n replicated models by assuming a
universal normalizing factor n~ and similarly for his proof of
I-MINQE(U,I). Thrum and Schmidt (1979) suggest that this case is
an excepti on and that in general the di ffer'ent vari ance components
ai are estimated at di fferent degrees of fr'eedom and. hence need
different normalizing factors to achieve an asymptotic normal
distribution. They investigate the limiting distribution of 0n(e-e)
e is
the vector of MINQE(U,I)'s or I-MINQE(U,I)'s for aI' ... , ah
and On is a given diagonal matrix with elements di(n) which all tend
where
to i nfi ni ty .
1.2.1.7 MIVQE(U,I) under Normality.
When one considers transla-
tion invariant quadratic forms Y'AY the matrix A must satisfy AX = 0;
and if normality is assumed then (1.2.11) blecomes var(Y'AY)
2 tr(AV)2.
=
Hence we obtain the translation invariant minimum variance
quadratic unbiased estimator by minimizing tr(AV)2 subject to AX
= o.
Searle (1979) shows that the resulting systl:m of equations is exactly
the same as the REML equa ti ons gi ven in
(1.:~.
10) and thus ; ntroduces
the concept of I-rlIVQE(U,I) for the solution analogous to I-rUNQE(U,I)
of the preceding section.
It should be noted that these solutions
are rarely referred to as MIVQE(U,I) or I-MIVQE(U,I) because under
the iterative procedure the MIVQE(U,I) prOpE!rty is destroyed.
~
14
1.2.1.8 Local MIVQE(U,I) under Normality.
If V is replaced
by V in (1.2.10) with a representing some pre-assigned values for
a
e;
then the solution (1.2.12) can be calculated explicitly and can be
called "Local-MIVQE(U,I)" in this context.
It is an estimator with
the properties of being locally best at e = a, as discussed by'
La Motte (1970, 1971, 1973b).
1.2.1.9 MINQE(U,I,O).
If a priori knowledge about e is rare,
one possible set of values for a is to put all of its elements zero
except that corresponding to e& , for which a choice of any positive
number p can be made.
Then V = IN·p and R = [I - X(X'X)-X'J/p
a
a.
so that the fUNQE(U,I) equations (1.2.12) become what can be called
the MINQE(U,I,O) equations for i,j = 1, ••• , h+1:
(1.2.13)
The solution e(O) from this system of equations is the MINQE(U,I)
using prior values ai = 0, i=l, .•. , h and any positive value for e&.
Seely (1971) suggests this as a method of estimation using a least
squares approach to variance component estimation.
Corbeil and
Searle (1976 ) use it as a starting value for the
iterative REHL equations and it has
been considered by Hartley,
J. N. K. Rao, and La Motte (1978) because of its computational ease,
a feature promoted by Goodnight (1979).
1.2.1.10 Dispersion Mean Models.
Pukelsheim (1976, 1977 )
considers unbiased translation invariant quadratic forms Y'AY
satisfying AX =
° and he shows this is equivalent to defining a
vector Z that contains all squares and products two at a time of the
15
observations such that
E(Z)
= X6
(1. 2 .14)
for a known form X.
Pukelsheim identifies this as the dispersion
mean model and Brown (1978)· calls it the dE!rived model because of
its linearity in
6.
Pukelsheim (1976) appliies the theory of ordinary
least squares (OLS) and generalized least squares (GLS) to the above
model and obtains MINQE(U,I,O) and MINQE(U"I) equations respectively
for estimating e.
In surnnary, MINQE( U, I) theory for estimati ng variance and
covariance components in linear models was the first successful
attempt to create a general estimation proc:edure which yields optimal
quadratic unbiased estimates whenever quadr'atic unbiased estimates
exist at all.
Regrettably, however, it does not overcome a dis-
advantage shared with ANOVA and REML methods of sometimes producing
negative estimates that is observed to happen most frequently in
cases of only few observations.
The method also involves numerous
multiplications and inversions of nxn matrices for data sets with n
observations.
The possible occurrence of negative estimates together
with the enormous numerical efforts required to calculate MINQE(U,I)
has led many practitioners to the few alternative methods that are
available.
Those fears of the computational burden that may arise when
r1INQE(U,I) becomes applied to real life data have been mitigated with
the continuing infiltration of surprisingly simple explicit formulae
for the MINQE(U,I) under a variety of unbalanced random models.
16 .
An iterative method for obtaining REMl estimates has been
developed by Corbeil and Searle (1976) using a W-transformation
described by Hemmerle and Hartley (1973) and similarly extended to
the MINQE(U,I} equations by Liu and Senturia (1977).
These procedures
require inversions of q x q matrices where q is the total number of
random levels in the structure. The calculation of the estimates
by these methods can still be cumbersome even though q is much
smaller than n generally.
On the other hand, Ahrens (1978) and Swallow and Searle (1978)
give straightforward methods for computing f1INQE(U,I} for the one-way
classification random model.
Giesbrecht and Burrows (1978) give an
elegant method for computing MINQE(U,I} for the three-level nested model
illustrating a direct extension to more levels of nesting.
The
author has corrected the explicit equations of Kleffe (1980) describing the MINQE(U,I} formulae for the variance components and sample
variances of the estimated variance components for the two-stage
random model.
Recent developments in MINQE theory also include a modified
MINQE principle to produce non-negative estimates of which those
suggested by P. S. R. S. Rao and Chaubey (1978) are even easier to
compute.
1.2.2 Comparison of Estimators
Ahrens (1978) makes a risk function comparison of MINQE(U,I)
and ANOVA estimators for different "j
patt~rns
A special nj pattern and parameter value
is uniformly better than ANOVA.
p
for the one-way design.
is found for which MINQE(U,I)
17
Similarly Swallow and Searle (l978) examine the variance of
the estimates of local MIVQE(U,I) and
ANO~~
for thirteen different
unbalanced one-way design configurations. They suggest that even an
inaccurate choice of prior weights may provide local MIVQE(U,I)
with a smaller variance than the ANOVA est'imate for the non-error
variance component, whereas the ANOVA estimate of the error variance
component generally has a small variance and is preferable to the
local
~lIVQE(U,I)
unless prior values can
b~!
accurately specified.
Hess (1979) graphically displays that for prior weights in a
neighborhood of the true variance component values that the
variances of the MINQE(U,I) estimates are quite stable.
For compari-
son using a random one-way model, the variclnces of the corresponding
Henderson Method III or ANOVA type estimators of variance components
are also computed.
His results are very similar to those of Swallow
and Searle (1978).
Finally, P. S. R. S. Rao, Kaplan, and Cochran (1981) compare the
variances of five different variance component estimators for the
one-way random effects model with unequal error variances.
The two
estimators in their study not previously described include an unweighted sum of squares procedure and an ASR (average of squared
residual) method suggested by J. N. K. Rao (1973) and P. S. R. S. Rao
and Chaubey (1978).
They found that when the true variance for the
non-error component was different from 0, then the mean squared error
of the ASR residual was smaller than the MSE1s of the other estimators
considered.
When this variance component was 0, however, ANOVA had in
general smaller MSE than ASR, MINQE(U,I) and USS provided that the
variation among ni was small.
18
1.2.3 A Comparison of Designs and Estimators
The problem o"f selecting a design and estimation procedure for
estimating (with minimum mean squared error (MSE)) the variance
components of unbalanced data has not been assessed with the MINQE
type of estimators.
Crump (1954) and Anderson and Crump (1967) compared various
designs using ANOVA and MlE estimators for estimating the variance
parameters in the one-way random model.
The Thompson-Anderson (1975)
article considered one-way designs with k1 classes having one sample
per class and k2 classes having two samples per class for MLE, ANOVA
and iterated least squares types of estimators.
Thitakomal (1977)
extended this investigation with the use of REML estimators for the
(k 1 , kZ) designs.
Several classical designs are suggested for use by Goldsmith and
Gaylor (1970) when variance components are estimated from two-stage
nested designs.
Based on ANOVA estimates they propose methods for
determining the optimal design for 49 different variance component
configurations and ten different sample sizes.
1.3 Outline of This Research
In this study the problem of estimating the variance components
in some basic random effects models is investigated.
For the two-stage nested design model explicit equations are
developed for obtaining estimates of the variance components and the
variances of their estimates using MINQE(U,I) or REML types of
estimators.
Explicit forms for a combined r1INQE-Henderson Method III
type of estimator for variance components are also given.
19
A risk function comparison of MINQE(U,I), ANOVA and the combined
estimator mentioned above is considered for certain optimal two-staged
nested designs considered in Goldsmith and (;aylor (1970) using different priors and true values for the variance components.
Explicit equations for minimum nonn qucidratic estimators under
a variety of specialized conditions are considered and applied to
data from a one-way unbalanced design.
20
2. SOME BASIC DEFINITIONS AND RESULTS
2.1 Introduction
In this chapter the form of the general model used throughout
the remaining discussions is described. The minimum norm quadratic
estimator is defined for the general model and the Euclidean norm
identified.
Finally, a minimization theorem is described with an
application in the use of quasi inner products.
2.2 The Form of the General Model
Let Y be an n-vector of random variables with a linear structure
{2.2.1}
where Xis a !;Jiven nxmmatri x,
6 ;s
m-vector of unknown parameters,
Ui is given n x ci matrix, and €i is a ci-vector of uncorrelated
random variables with zero mean value and dispersion matrix eil c .'
,
= 1,
i
2, •.. , p, where ei are unknown.
correlated.
Further Ei and Ej are un-
For the general model form Y = XB + U€ we have
(2.2.2)
and F.F.
, ,
= 6 1V1 +
= F,.
and E{Y)
••• + epv p '
= X6,
D(Y)
= e1UF 1U'
+ •.• +
ep UF p U
When Ei is known then a natural estimator of
,
e. is
e.,
= E~e../C
.•
, 1
1
(2.2.3)
21
Let aI' ••• , a p be a priori (or approximate) positive values of
e , ••• , ep • (If no a priori information is available, then Rao
1
(l979a) suggests each ai should be taken to be unity.) Using these
pri or va1ues we may transform e: to F-~e: whE~re F = al F1 + •.. + a pFp,
and U to U~ denoted €* and U*,respectivel)" and rewrite (2.2.1) as
Definition (2.2.1):
Under the general model (2.2.4) a natural
estimator of fIe is
(2.2.5)
where N = I uiF i and ui are determined such that
E{e:~Ne:*)
= fIe.
Definition (2.2.2): A quadratic form Y'AY used as an estimator of
fIe is said to be translation invariant if it is not affected by
changes in the fixed effects 8.
S + 0, then YIAY is unchanged;
YIAY
= (XB
+
This means that if S becomes
l.~.,
UE)IA (XB + UE)
= (Xs
+
Xo + UE)IA (Xs + Xo + UE)
so giving
olX'AXo + 2s'X'A(Xs + UE)
=0
(2.2.6)
and we want this to be true for all 0 and Y.
said to be a translation invariant estimatOl",
then necessary suffi ci ent condi ti on for
thE~
When it is, Y'AY is
If A is symmetric,
above is to choose A so
that
AX :: 0 .
(2.2.7)
22
Definition (2.2.3):
If Y'AY is to be an unbiased estimator of f'e,
L fie i = E(Y'AY) = tr(AE(YY')) = S'X'AXS + L ei tr(AV i )
sand 6i . This implies X'AX = 0 and tr (AV i ) = f i , and thus
then
for all
the term
S'X'AXs does not enter into any calculations.
2.3 Minimum Norm Quadratic Estimation
Consider the general linear model described in (2.2.1), and a
quadratic estimator y
= Y'AY
of f'e.
Now using the previous
transformation we have
U~X
)
X'AX
The difference
(
£*
).
Y'AY and the natural estimator
bet~~en
(2.3.1)
B
€~N£*
as defined in (2.2.5) is
U~X
X'AX
}
(
£*
S
).
(2.3.2)
The minimum norm quadratic estimator is the one obtained by
minimizing an appropriately chosen norm of the matrix of quadratic
form in (2.3.2):
U~X
=
(2.3.3)
X'AX
We shall consider the Euclidean norm
(2.3.4)
23
2.4 quasi Inner Products
The concept of quasi inner products has been discussed by
Pukelsheim (1974) in a simplified form.
Definition (2.4.1):
Let
cS
denote the lineiir space of all symmetric
Nx Nmatrices over the real number field.
Then the inner product
<A, B> is defined as tr(AVBV) for some pos"itive definite matrix V.
This mapping from
0
x cS to R1 is called a quasi inner product since
the following conditions hold:
(i)
<
A, A >
~
(ii)
<
A, B >
=<
(iii)
0
(2.4.1)
for any Aeo
B, A >
(2.4.2)
for any A and Beo .
< A, B > - < C, B > = < A-C, B >
(2.4.3)
Theorem (2.4.1)
(i) Let Mbe a convex set in 0; then the minimum of < A, A > subject
to Ae~l is attained at A*eM if and only if
for any Be M .
(ii) Let
00
be a subspace of
0,
define U = {T + B : BeoO}
(2.4.4)
T be a fixed element of
Then the minimum of
0,
<
and
A, A >
subject to AeU is attained at A*eU if and only if
for any Ceo 0 .
Proof of (i):
(2.4.5)
24
If (2.4.4) holds, then
<
B, B
> ~ <
A*, A* > which proves
sufficiency.
Now if A* minimizes
<
A, A
>
and A*€M we must have
for all B € Mand A € [0, 1] since
t~
is convex.
Letting A
approach 0 we have that
<
A*, B-A*
> ~
0 which gives the necessity.
Proof of (ii):
Simi 1arly for any matri x Hc: U
°
Now H = T + B1 and A* = T + B2 for some B1 , B2 £ 0 • So
H - A* = B1 - B2 € 00 since 00 is a subspace. Thus if (2.4.5)
holds,
<
H, H >
~<
A*, A*>, proving sufficiency.
Nm'/ if A* minimizes
A*, A* > ~
<
A*
=T +
<
A, A > and A*
£
U, then
A* + A(H-A*), A* + A(H-A*)
81 and H - A*
A* + A(H - A*)
<
= B1
- 82
~ 00;
> ,
but
thus
=T +
83 for some 83 £°0 since 50 is a subspace.
Similar to the proof of (i) we must have < A*, H-A* > ~ 0 or
<
A*, B1-B >
2
any C € 00.
~
0 for any B1 - 62 £ 00
or
<
A*, C >
2.
0
for
Since Co is a subspace - C E: cO' thus \'/e must have
25
<
A*, -C
> ~
0 and hence the desired result holds
for any Ce: 00.
<
A*, C >
=0
26
3. MINIMUM NORM QUADRATIC UNBIASED TRANSLATION INVARIANT ESTIMATORS
3.1 Introduction
In this chapter MINQE(U,I) will be derived from first principles.
For the two-stage nested design explicit equations for the MINQE(U,I)
and their variances will be given under normality assumptions. A
combined MINQE(U,I) and Henderson Method III (ANOVA) estimator
denoted MINQH will be described and expressions similar to the above
reported.
3.2 MINQE(U,I) Solutions
The above is sometimes simply denoted Minimum Norm Quadratic Unbiased
Estimation.
estimators,
Consider the class of invariant unbiased quadratic
i.~.,
of the form Y'AY where A belongs to the class
C~I' = {A: A is symmetric, AX = 0, tr(AV i )= f i , i = 1, ... , p}
(3.2.1)
where Xand Vi are defined for the general model (2.2.1) and use the
following assumptions and notation:
(3.2.2)
Then we have the following.
Theorem (3.2.1)
If C01 is non-empty, then under the Euclidean norm (2.3.4),
the MINQE(U,I) of f'e is
27
Y= L A.VIA.V,
1
1
A.1
= Ra V.R
1 a
(3.2.3)
where A = (AI' ••• , Ap) is any solutions of QA
= f,
Proof:
Under conditions '3.2.1) and (3.2.2) the square of the
Euclidean norm in (2.3.4) becomes
(3.2.4)
=L. u.1
1
a·
1
tr(AV.)
1
= L. u.a.f.
1 1 1
1
and thus this term does not depend on A.
So we need only minimize
the expression
{3.2.5}
f
where A E: CUI
.
Now defining A*
= t..\
\.R
V.R where the A's satisfy
1 Ct 1 a
28
QA
= f,
one easily observes that A*X
f
A* € CUI'
o
CUI
=0
and tr(A*V i )
=fi ,
and hence
Oefi ne
={ 0
: 0 is symmetric, OX
= 0 and
Note that C31 is non-empty since 0 € c3r
can write A = A* + O.
tr(OV i )
=0
}.
(3.2.6)
Hence given any A€ C~I we
Since C~I is a subspace of the set of all n x n
symmetric matrices we can apply Theorem 2.4.1,
part~i),
which con-
cludes that A* minimizes tr(AVCla
AV ) if and only if tr(OVClA*VCl )
=0
for
any 0 e: C~1 • Now if 0 e: C~I' then 0 can be wri tten as
o = pol' Epol
a
(3.2.7)
a
for some symmetric matrix E.
(Note:
Choose E = D.)
Hence,
l
tr(polCl EP'"
\' A,.RCl'
V.R a Va )
a a~
= ~,
\' A.
tr(pol'
Epol V,.)
Cl Cl
=
0
(3.2.8)
and thus A* satisfies the above.
It should be noted that the general development of the MINQE(U,I)
described above precludes the use of zero a priori values when
trans fo rmi ng the genera 1 mode 1 wi th the rnatri ces F-~ and F\
Discussions in the literature fail to mention this fact but rather
haphazardly suggest the method of plugging in 0 a priori values (with
the exception of the error) into the MINQE(U,I) system of equations.
The MINQE(U,I) system can, however, be derived using a least squares
estimation procedure by premultiplying the model by the matrix V~~
29
and computing the appropriate residuals.
Using this technique one
can set a priori values to 0 and solve the equivalent system of
equations.
3.3 Estimation in the Nested Model
The earlier development has been in terms of a relatively
general linear model.
Henceforth, we will specialize to the
level nested model with only one fixed
This
parameter~.
~NO­
t~~-level
nested model is written
y. ok
1J
=~
(3.3.1)
+ a 1° + b•.
+ e.1J° k
1J
.•. , ni j' j = 1, .•. , mi , i = 1, ... , h, and where
{Yijk} are the observations, ~ is an unknown constant, {ail are
for k
= 1,
independent N(O, Sa)' {b ij } are independent N(O, Sb)' {e ijk } are
independent N(O, 0 ) and the latter three sets are also mutually
e
independent.
The convention of denoting a summation by replacing the
corresponding subscript with a "+" symbol will be followed.
The
connection between (3.3.1) and (2.2.1) is established by noting that
Y corresponds to the "++ elements {Yijk} in dictionary order, X is
a column of l's, S the scalar].1, U1 is an n++xh matrix of D's and
115, U2 is an n++ x m+matrix of D's and l's and U the n++ x n++
3
identity matrix.
The general form of V0. is clear.
It 'is a diagonal matrix with
submatrices Vi of order ni +, i = 1, ... , h. Thus, V is the direct
h +
h
sum of matri ces denoted V = LVi and hence V- = L + 'I ~ 1 .
0.
i=l
0.
0.
i=l
0.1
Denote prior estimates of the unknown parameters
t
30
e:
= ae
(3.3.2)
•
Then V . may be written as
al
Val'
= dP(B)J ni . X n..
J
for i
= 1,
... , h, j
1J
= 1,
+ In
ij
} + {p(A)J n x n
i+ i+
... , m.,
where.
1
I n .. x n.. indicates a matrix of l's nij x nij'
lJ
lJ
.
J n••
indicates a vector of l's (n . ,
l
lJ
Lemma (3.3.1)
Given a matrix K of the form
for a given constant 9
K- 1
=In
F -lIn,
- gl (ng + 1) J
then
nxn .
Lemma (3.3.2)
Gi ven a ma tri x W of the form
W =B+gJ
nxn
for non-singular B of the form
o
o
J
~1) •
}
(3.2.3)
31
and 9 as described above; then
for
w=
9
1-1
g[ L.J.B. J.] + 1
1 1 1
1
Applying Lemmas (3.3.1) and (3.3.2) to equa.tion (3.3.3) we obtain
v-~
a1
= 1/£
[ {I
J
p(B)
n · - n::PrBJ+T n .. x n ..
1J
1J
1J
}
1J
o(A)
mi
peA)
{.L
J=1
mi
niJ· - pCB)
.L
J=l
n~.
{ J
nijx·nijl
[ 1 -
n.. (~)+l} + 1
1JP
n .. n ..• ~B) + n.· + n" 1
1J
'J}]}]
(n~.;p(B)+l)(n.
,J
, J"P{B)+lT
pCB) { ..:..u...J.L
(3.3.4)
In terms of (i, j, k, ii, jl, k' ) we have the elements of V described
a
by
(3.3.5)
for ;
= 1, ... , h,
j =
1, ... , mi , k
where
o~ m =
:..
{
1
if
2 = m
0
if
~;:m
= 1, ... , "i j ,
32
Define
(3.3.6)
m·1 9
= L 5 ..
j=I P lJ
=
p(A)
p(A)[SJ i - p( B)Sf i ] + 1
,
~~A)i + 1
- p (A)
0
H
i
The corresponding elements of V-I
are
a
1/ e: [0 .. I {o
11
JJ
0
0
I
a
(0 kk' - p ( B) 51
lJ
0
0
)
o,
- ToO-p(B)[Slloo+Slloo.-p(B)SlloO
1
lJ
lJ
lJ Slll·J ]}
].
(3.3.7)
Summing over i 'j'k ' yields elements of the vector V-IX corresponding
a
to Yijk in
Y:
v-IXo Ok
a
1J
=
l/dl - P(B}Sll
lJ
00-
To(SOI
1
0
1
(3.3.8)
The scalar XlV-IX is obtained as
a
33
h
lie: oL
,=1
1
2
12
21
[50; - pCB} 51; - T;{ {50;} - pCB} [25 1 ; 50;
{3.3.9}
Define the scalar,
{3.3.10}
and
C(p(B}
= a} = C1
The general element of Ra. follows as
r,oJ"k"'J"k'
= lIe:[o",Ooo,okk'
11
JJ
- o,,,ooo,P(B)50100 - ooo,To(l
"JJ
'J
11'
1
1
1
1
- p(B}{5 .0+ 51 001 - p(B}5 .. 5 .01 })]
lJ
1 lJ
1 lJ 1 lJ
2
1
2
.
1
+ 51 i - 0(B)5 1 ij 51 i })][.L - 0(B)5 1 i'j'
- T'I(501"
1
- 0
1
1
- p(B){5
2
1
+ 51 1
1 1·, J'I SOl'l
1
'1
2
(B ) 51 i I j' 51 i
1
})
]
and the typical element of Z = Ra Y is then
(303.11)
34
Zi 'k
J
= 1!E:[Y"lJ k
1
- p(B)50 · .y .. + -T.y,++ + p(B)T.5 .. y.++
1 1
1 1 lJ 1
1 lJ lJ
.
1
1
1
1
2
· - p(B){5 1 lJ
- C/d1 - p(B)5 1 lJ
.. - T.(5
.. 50'1 + 51'1
1 01
(3.3.12)
Thus quadratic forms corresponding to the right hand side of
the MINQE(U,I) equations for the two-level nested design denoted
yl R V R Y
CL 2 CL
and
can be obtained as
h
h 2
r
L z, '+
1=1 j=l lJ
h
and
mi nij
r r r
i=l j=l k=l
2
Z"k
lJ
(3.3.13)
where the + symbol indicates summation over subscript before squaring.
Note at this point that these quadratic forms can be obtained by one
pass through the data file and are not effected by the size of the
structure.
35
Simplifying the left hand side of the
done as follows.
(1
(1
(1
~
equations can be
We require the inverse of a matrix of nine tenns
of which six are distinct.
R V R
~tINQE(U,I)
Using the identity
R
(3.3.14)
(1
and
(3.3.15)
and the fact that
h mi
= L L n.. - rank
i=1 j=1
=n -
lJ
(X)
(3.3.16)
1
we obtai n
(3.3.17)
Now
2
2
1
2
1
- p(8)(Sl i) ]}][l - (SO i - 0(8)Sl i - Ti{(SO i)
2
1
- p ( 8)[ 2S 1 i So i -
2
P ( 8)( S1
2.
i ) j})] C
2
(3.3.18)
36
and
3
- p(B)S2' })] - [
1
mi
I
j=1
1
SO"
1J
1
[1 - p(B)Sl"
1J
1
11
2
12
2
p(B){S1"
SO'
+
Sl'
p(B)
S1"
S1'
})]J
C
- T.(SO·
1
1
lJ
1
1
lJ
1
(3.3.19)
and by (3.3.17)
(3.3.20)
Now for the special case p(B)
= 1/ e ~L.
~i
L.
;=1 j=l
1
=a
we have
1
1
[SO; j - H; (SO i j )
1
1,2
J ]
- C1 {SO"
- H.1 So lJ
.. So 1.
lJ
It should be noted that although the
e
2
(3.3.21)
appears in the denominator of
these equations, when calculating the MINQE(U,I) and the variance of
these estimators the e'S disappear and we are left with terms a
function of p(A) and 0(8).
Using (3.3.14), (3.3.15), multiplying by VI' V and V , and
2
3
taking traces we obtain the following system of equations:
37
(3.3.22)
(3.3.23)
and
(3.3.24)
Q
= II e: 2 i h=1
L
1
1 ,2
2
[SO· - p (B) S1 . - T. ( (SO· )
1
1
1
1
2
1
2 2
2
- p(B){2S 1 i SO; - p(B)(SI; ) })] •
(3.3.25)
Now
tr(R
Ct.
VI R VI) = lIe:
Ct.
2 h
12
1 2
L
[SO· - p(B)Sl' - T.{(SO' )
;=1
1
1
1
1
1 2
2
2,
- p(B)[2S 0 i SI i - p(B)(5 1 ;) L'][l -
mi
2.L
J=l
1
50 iJ' [1
1
1
1
1
2
- 0(B)S1"lJ - T.{SO·
- o(B}[S,.J. lJ
,. SO·,'I + Sl'1
1
1
12,2
221
2
12
- p(8)Sl ij S1 i J}] C + C Qe: [SO i - p(B)S1 i - Ti{(SO i)
(3.3.26)
38
2 h
tr(R
VI R V3 )
a
a
mi
= liE ,=1
.L J=l
.L
1
1
So iJ· [1 - p(B)S1 iJ·
1
11
2
12
2
- T.(SO· - p(B){Sl·· SO· + S1· - p(B)S1·· S1· })]
1
1
'J'
1
lJ
1
•
22
1
2
12
[1 + C e: Q - 2C[SOi - p{B)S1i - Ti{{SOi)
1
2
2
2
- p{B)[2S 0i S1 i - p(B){Sl i)]}]]
(3.3.27)
,
and because of the symmetric form of Ra we can obtain the following
with the aid of (3.3.11):
tr ( R V3 R V3 )
a
a
2
= l~ J~ k) lJ' JI' kL, 1
(r J..1
k • , •, k' )
J
.
. (3. 3.28)
By (3.3.24) we obtain
(3.3.29)
By (3.3.22)
(3.3.30)
and by (3.3.23)
39
(3.3.31)
When prior values for Q(B) are given as 0
WE!
have the following
alternative for (3.3.29) - (3.3.31):
tr(R
V R V)
a 2 a 3
r
iJk i'J'k'
= Ih: 2 4
I
{{Ooo,{Ooo, - H.S · .. ,}]
11
JJ
"OJ
1
1
- Cl [(l - H., So ., )( 1 - H. So . )]} ;
"
"
.
(3.3.32)
_
2
1
1
1
tr( R VI R V2 ) - 1/ c: L )' SO· . So
[(l - H. So . )( 15 •••
a
a
ij il-j'
'J
11 J
""
':I ••
1
1
2
1
- Cl[SO·'
••• - H.. S .,.}
, - H.•
" (SO·' ) ])}{O ,.., I{O JJ
, O, J
1
- CHI - H.,SO
"
I
I
I
{SO·· - H.S O ·· SO·}]
lJ
1 lJ
1
·I}
tr(Ra V2 Ra V2 ) = Ih:
2
I ..L.. s6iJos 6i ' J·'
lJ 1 J
(3.3.33)
l
[ooo.{o ..• - H.S O ..• }
11
JJ
1 lJ
(3.3.34)
40
Thus we are now able to solve the MINQE(U,I) equations without
calculating any large order n inverse matrices and without storing
large matrices of order n2.
3.4 The Risk Function for MINQE(U,I)
By the previous section the estimates of ea , eb, and 0 e are
functions of peA), pCB) only and have the following form:
A
°a
A
0b
=
A
ee
all
al2
a l3
VI Ra VI Ra Y
a21
a22
a23
YI R V R V
a 2 a
a31
a32
a33
yl R V R Y
a 3 a
(3.4.1)
where the elements aij are functions of peA), p(B) and €.
It is well known (Rao and Kleffe (1979))that for Y-N(XS,V) we have
that
Var(yIAV)
= 2 tr(AV)2
+ 4S ' X' AVAXS
and
Cov(V'AV, Y'BY)
= 2 tr(AVBV)
for A and B symmetric matrices.
+
4S ' X' AVBXe
With invariance the latter terms are
dropped and using the fact that matrices commute under the trace
operator we obtain the following.
41
+ 20a0b tr {Ra VI Ra VI Ra VI Ra V,,}
~
{3.4.2}
Var{Y' Ra V2 Ra Y} = 2[0a2 tr{R a VI Ra V2 Ra VI Ra V2 }
+ 20a \
+
tr(R a VI Ra V2 Ra V2 Ra V2 }
~ tr{R a V2 Ra V2 Ra V2 Ret. "3}] ;
{3.4.3}
42
(3.4.4)
Cov(Y' Ra VI Ra Y, Y' Ra V2 Ra Y)
+ 0
2
V?)
b tr(RCt. VI RCt. V2 RCt. V2 RCt.-
(3.4.5)
43
Cov(V' RCt VI RCt V, V' RCt V3 RCt V)
(3.4.6)
Cov(V' RCt V2 RCt V, VI RCt V3 RCt V)
+
ee2
tr( R V R V R V R V ) ]
a 2 a 3 a 3 a 3
(3.4.7)
44
In order to simplify the calculation of the above variance and
covariance terms above, we will use the following explicit forms.
The general element of Ra Vi follows as:
0 0k o, O'k'
rv 1 lJ
1 J
= Iff; [(1 -
p(B)SII 00- To{SOl 0 - p(B)[S1 00 SOlo
lJ
2
1
2
+ S1°1 - p(B)S1°lJ0S1 10 D)(<5oo.
11
1
1 lJ
1
e[so1 101
1
2
- p(B)S1 1o.
1
1
2
1
2
- p(B)[SI" So 0o. + S1 0o. - p(B)Sl 00 Sl 001]}}]
lJ
lJ
lJ
lJ
lJ
- elF:. [1 - p(B)S11 1J
00 - To{S01 0 - p(B)[S1100S010 -+ S2110
1
1
1J
1
2
1
2
1
B [S1 10I So 1J
o. o. + S1 1J
0I •• So 10I
- p ()
-
)2
2
,
p(B S1 1o. S1 1J
0I 0I J}]
,
(3.4.9)
(3.4.10)
Hence the elements of tr(R Vi R Vj R Vk R V,) can be obtained by
multiplying the matrices previously calculated in explicit form.
45
For notational convenience designate
(3.4.11)
Thus
by
(3.4.1) and the previous explicit foms we have the following
variances for our .MINQE(U,I).
VA(M) indicates the variance of the MIN.QE(U,I) of the 8 a
variance component; likewise for VB(M) and VE(M):
2 t
2 t 2222 + 2a 13
all 1212 + 2a12
2323 + 4a 11 a 12 t 1222
+ 2 {2 2 t
8b
46
(3.4.12)
The equation for VB(M) and VE(N) follow from (3.4.12) by changing
ali to a2i and a3i , respectively.
The analysis of variance estimators for the two-way nested
classification (Searle, 1961) are given in the appendix.
Since it is well known that MINQE(U,I) for balanced data are
equal to an ANOVA estimator, it must hold that
VA(M)
= VA(N)
VB (M)
= VB (N) ,
VE(M)
= VE(N) ,
,
and
when n;j
= n for
all; and
estimator was used.
j
and where
~);ndicates
that an ANOVA
This requirement gives us a good check for any
computer programs and for comparing the two estimators and their
variances.
47
3.5 A Combined Estimator USin y MINQE{U,I) and
Henderson i s Nethod I It for the
Error Component
For the two-stage nested design the Henderson Method III
estimator of 0 e has the following form,
IIp {Y'Y - R{x : u1 : u2)} (see
Searle~
1968) ,
(3.5.1)
R{x
(3.5.2)
U2) , or (3.5.1) can be described as
for
(3.5.3)
for Px u u' the projection operator which projects on to the space
, 1'2
spanned by the columns of Z
.
x,ul,u2
Lemma (3. 5. 1) :
The Henderson Method III estimator of 0 e described above is
exactly the within estimator in the analysis of variance.
Proof:
For the two-stage nested design, consider the within estimator
h
rn;
(3.5.4)
h
L L ni' - L m.
i=1 j=1
J
i=1
'
48
where the projection operator Pu
2
fonn:
= U2(U 2U2)-1 U2 has the following
(3.5.5)
I n.. xn .. ]
lJ
lJ
n++
It should be noted that like the projection operator used in the
Henderson Method III estimate we know these operators are
idempotent having eigenvalues O's and l's and thus estimates are
non-negative in both cases.
This condition is not necessarily true
for the MINQE(U,I) estimate of se' however.
Since U1 has the fonn
n1+ 1
0
0
0
n2+ 1
0
0
0
0
0
0
0
0
0
nh+ 1
(3.5.6)
we obtain
(3.5.7)
and since X is a vector of lis of dimension n++xl,
p X= X
u2
.
By the definition of a projection operator
{3.5.8}
49
(3.5.9)
= tr{P~
Now E{Y'P~ Y}
u2
V} + a'x'p~ Xa
u2
u2
where
~
= tr{P u2 } ee
(3.5.10)
and
h
= [r
m;
r n •.
;=1 j=1
Thus, YIP~ Y/[
u2
V3
=I
and the above,
h
-
lJ
h m;
r
L n. . i=l j=l' J
I
;=1
m.]G
1
e
(3.5.1l)
h
I
m.]
i=l'
is an unbiased estimator of Ge •
To show the ANOVA and Henderson r·1ethod I II E!stimators for the Ge
component are the same, one only needs to show the equivalence of the
But by equations (3.S.7) - (3.5.9) we obtain
projection operators.
p
p
u2 x,u 1,u 2
= Px,u
,u
1 2
(3.5.12)
and
p
p
x,u 1 ,u 2 u2
= Pu
2
{3.5.13}
50
and by the fact that these projectors are symmetric we obtain that
(3.5.14)
So in terms of a combined MINQE(U,I) and Henderson Method III
(ANOVA) estimators denoted MINQH we must solve the following system
of equations:
tu
t 12
t 13
ea
t 21
t 22
t 23
eb
0
0
1
ee
=
yl
RIX VI RIX Y
yl
RIX V RIX Y
2
yl
pol Ip
u2
(3.5.15)
y
h
mi
h
where p =(L L niJ· - L m.}andthelefthandsidehaspreviouslybeen
i=1 j=1
i=1 1
discussed. The consistency and unbiasedness of the MINQE(U,I) is
also shared with estimators obtained by solving the above.
3.6 The Risk Function for MINQH
Now the variance of the ANOVA estimate of ee will be
Var(Y'Pol Yip}
u2
= JL2 tr(Pol V}2
p
Uz
+
0
(3.6.1)
51
Considering the covariance terms we have
Cov(Y' Ra VI Ra Y,
=2
y l p~
u
Y)
2
tr(Ra VI Ra V p~u V) + 0 •
(3.6.2)
2
Now
(3.6.3)
for
and if V-I exists, then
a
(3.6.4)
Now since
the above
= 28e2 tr(R~ VI R~
pL}
Uz
=0
by (3.5.7) and (3.5.8).
(3.6.5)
(3.6.6)
Similarly,
52
.
Cov(V' Ra V R,a V
2
VI pJ. V)
u2
=0 .
(3.6.7)
Let
bU
b12
b13
b21
b22
b23
b31
b32
b33
t
=
t
u
t
12
-1
13
t 21
t 22
t 23
0
0
1
(3.6.8)
The variance of our combined estimators MINQH have the following
fonn:
2
+ 2b 13
/
n
(L
mi
L
i=l j=l
n ° JO
1
and in a similar fashion VB(MH) and
h
-
L
i=l
VE(~~)
changing b1i to b2i and b3i , respectively.
m° )
}
1
are constricted by
(3.6.9)
53
4.
DESIGN
4.1
EVALUA~ION
Introduction
In this chapter 16 designs are enumerated from five fundamental
structures such that the designs permit ANOVA, MINQE(U,I) and MINQH
estimates of all three variance components based on a completely
random two-stage nested model.
Using the :minimization of the trace
of the covariance matrix as a criterion for design selection, consideration is given as to which design should be selected when prior
ratios equal unknown ratios of true variance components. A priori
values are suggested where the variance of the MINQE(U,I) and 11INQH
estimates are less than the variance of the ANOVA estimates for a
large proportion of those unknown variance component values considered.
The topological features of the
l~atios
of the variances
of the different estimators are considered for design configuration
17.
Finally, the effects of using some a priori values considered
as being "reasonable choices" in the literature are examined.
4.2
Desiqn Fundamental Structures
The designs to be considered in this study are made up of five
fundamental structures similar to those described in C. H. Goldsmith
and G. W. Gaylor (1970).
The five fundamental structures are formed
by taking all possible combinations of structures formed when splits
of either one or two are allowed when passing from one stage to the
next lower stage in the design.
The five fundamental structures so
formed are pictorially represented by the stick diagrams shown in
Table 4.2.1.
54
Table 4.2.1 Fundamental structures
Structure Number
Stage
1
2
3
4
5
m.1
2
2
2
1
1
n •.
2,2
2,1
1,1
2
1
4
3
2
2
1
1
2
lJ
n.
1•
55
The first structure is used when two first-stage samples are
taken and in turn, two second-stage samples; are taken from each
first-stage sample.
Similarly, the second structure is used when two
first-stage samples are taken and in one of the first-stage samples,
two second-stage samples are selected, while in the other first-stage
sample, only one second stage sample is selected. . • .
Likewise,
the fifth structure is used when only one second-stage sample is
selected from each first-stage sample.
The designs were restricted to contain: 121' observations,
l'
= 1
or 5, and as many as three fundamental structures found in Table
4.2.1 such that each of the three variance components could be
estimated by means of the standard analysis of variance estimators.
The number 12 was chosen as a multiplier of r since it is the lowest
common multiple of the n.1· found in Table 4.2.1.
Originally it was
thought that letting r go as high as 10 would be necessary; however,
preliminary investigations showed that ther,e was very little change
in the results between r
= 5 and
r
= 10.
sample sizes 12 and 60 are encountered.
enumeration of 61 different designs.
By letting
l'
= 1 and 5,
These restrictions enable the
Of these 61, 16 optimal design
structures (see Goldsmith and Gaylor, 1970) are considered in the
study and they are identified in Table 4.2 . .c~.
Included in the 16
design structures are 11 designs previously suggested by other workers
and described in Table 4.2.3.
In Table 4.2.2. the designs are systemcltically numbered according
to the identified code used in Goldsmith.
The numbers in the body
56
Table 4.2.2 Two-stage nested designs
Structure No.
Coefficient Nos.
Design
No.
1
2
3
4
5
Code
c
d
e
g
1
1
0
0
0
8
(9,1,2)
60
8
12
2
2
1
0
1
0
6
(8,2,2)
54
8
13
3/2
4
1
0
3
0
2
(6,4,2)
42
8
15
5/4
5
1
0
4
0
0
(5,5,2)
36
8
16
2
6
2
0
0
0
4
(6,2,4)
48
10
18
2
8
2
0
2
0
0
(4,4,4)
36
10
20
3/2
9
3
0
0
0
0
(3,3,6)
36
12
24
2
10
0
1
0
0
9
(l0,1,1)
64
7
9
4/3
13
0
1
3
0
3
(7,4,1)
46
7
12
13/12
17
0
2
2
0
2
(6,4,2)
44
8
14
7/6
18
0
2
3
0
0
(5,5,2)
38
8
15
17/15
21
0
4
0
0
0
(4,4,4)
40
10
18
4/3
53
0
0
2
2
4
(8,2,2)
60
8
10
1
55
0
0
2
4
0
(6,2,4)
60
10
12
1
60
0
0
4
2
0
(6,4,2)
48
8
12
1
61
0
0
5
1
0
(6,5,1)
42
7
12
1
57
Table 4.2.3 Similarities between enumerated designs and those
suggested by other workers
Design No.
Code
Previous Names
2
(8,2,2)
Prairie (8:01)
4
(6,4,2)
Prairie (7:01)
5
(5,5,2)
r4bdified Anderson
6
(6,2,4)
Prairie (4:01)
8
(4,4,4)
Anderson; Prairie (3:01)
9
(3,3,6)
Balanced; Prairie (1)
17
(6,4,2)
Ba'j nbri dge
21
(4,4,4)
Ca" vi n-Mi 11 er; Prairie (3:02)
53
(8,2,2)
Prairie (8:02)
55
(6,2,4)
Prairie (4:02)
60
(6,4,2)
Prairie (7:02)
58
of the table below the structure indicate, when multiplied by r,
how many times that structure is to be used in a design.
The code
that accompanies each design is a triplet (u, v, w) which serves as
a design code as well as a way of identifying the degrees of freedom .
structure in the ANOVA table of a two-stage nested design. The
degrees of freedom can be determined by means of the formulas in
Table 4.2.4.
The coefficient numbers c, d, e, and g in Table 4.2.2 are used
to determine coefficients for the expected mean squares in the ANOVA
table of the two-stage nested design, as k1 = (cr-d)/6(h-l),
k2 = (72r-e)/6(h-l), k 3 = g, where k1, k2, and k3 are identified in
Table 4.2.5.
As an example, suppose an experimenter decided to run an
experiment with design 13.
The integers 01303 beneath the structure
numbers indicate in Table 4.2.2 that the experimenter would select
3r replicates of the structures 3 and 5 and r replicates of the
structure 2.
Pictorially, the experimental plan for r = 1 would be
the appropriate randomization of the following pattern:
I
I
I
~nn
59
Table 4.2.4 Correspondence between the degrees of freedom and the
design codes
Source
A
D. F. Fonnu'la
h - 1
=
ur - 1
h
Bin A
l
. 1
1=
h
Error
m.
1
m'
l l1
;=1 j=1
- h
=
vr
h
n..
1J
l m.1
- i=1
=
wr
60
Table 4.2.5 ANOVA table
Source
A-classes
B ;n A-classes
df
h - 1
E(t<lS)
'·15A
ee2 + k1e2b + k2ea2
'·15B
ee2 + k3ab2
r·1SE
ee2
h
L
;=1
h
Error
MS
m. - h
1
m;
L L
h
n.. - . L1 m.1
;=1 j=1 1J
1=
61
The design code (7, 4, 1) indicates that there are 7r - 1 df for the
A-classes, 4r df for the B in A-classes and r df for the error
source in the ANOVA table.
From the coeff"icient numbers, the
expected mean square coefficients are:
k1
= (46r-7)/6(7r-l),
k2
= (72r-12)/6(7r-l),
k3
= 13/12
.
4.3 Designs of Historical Interest Included in the Enumeration
Anderson and Bancroft (1952), Anderson (1960) and Anderson (1966)
suggest a "staggered" design, denoted an Anderson design in this study,
made up of an equal number of replications of the two structures 1
and 3. Also included is a modified Andersotn design which is made up
of four replications of structure 3 for each replication of structure
1. A Calvin-Miller (1961) design is made up entirely of replications
of structure 2 by itself.
A Bainbridge (1965) design is made up of
equal replications of each of the structures 2, 3 and 5.
Prairie's (1962) designs are formed by the following schemes.
Suppose we let A identify the broad selection class and Band E
classes at the first and second sampling stlage, respectively.
What Prairie calls 02 designs are constructed as follows.
(i) Choose n, the total number of observations, and a, the number
of A-classes.
(i i)
vlri te
and assign 91 + 1 experimental units to each of r 1 of the
A-classes and gl units to each of the remaining a - r 1
A-classes.
62
(iii) Choose b > a, the number of B-classes,and write
b
= ag 2 + r 2
'
Assign g2 + 1 B-classes to each of r 2 of the A-classes and
g2 B-classes to the remaining a - r 2 A-classes.
(iv) Within each A-class, assign the E-classes or units to the,
B-classes as equally as possible.
Prairie's 01 designs are constructed by dividing the A-classes into
groups and then assigning the B-classes and E classes such that
balance is maintained within each group and such that the design
constructed does not constitute a 02 design.
Prairie used sample sizes of 48.
Unlike our study,
The exception to this coding rule
is the design denoted by Prairie (1), which is the balanced design.
Expressions for the variances of these different estimators
are too complicated to allow analytic comparison and so numerical
comparison must be used.
Unfortunately numerical comparison must
involve evaluation of the variances under a variety of n-patterns
for different design structures and their replications, a variety of
values of unknown components
a , 9 b , Ge ), and for the MINQE(U,I)
and MINQH estimators unlike the ANOVA estimators, a variety of prior
values denoted
(e a , eb ,
(G
eel . . For convenience we will scale the
unknown components and prior values by Ge and ee' respectively, to
give the scaled configurations (Rho (A) ,Rho (B),I) for the unknown
components and (o(A), o(B), 1) for the prior values where
Rho (.A)=
(3
a/G e ' Rho(B)= '3 b/e e , p(A) = 0a/e e' and n(B) = eb/e e .
63
As Rao (1972) notes when a, b, and e alre nonna lly di stri buted,
the MINQE(U,I) procedures have minimum variance in the class of
translation invariant, unbiased quadratic estimators only when the
ratio of the prior weights equal the ratio of the variance
components,
i.~.,
peA) =Rho(A) and p(S) =Rho(S), and are denoted
MIVQE(U,I) estimates.
It would seem that this is a very severe
requirement to assure a desired optimal property of unbiased quadratic estimators.
Since ANOVA and the combined MINQH estimators fall into the
class of translation invariant, quadratic,
unbiased estimators,
a risk comparison of the three estimation procedures was natural.
In order to obtain numeri ca1 va lues fOI'" va ri ances, it was
necessary to specify a sample size, 12r, fo,'" r = 1 or'S, the scaled
unknown variance component configuration and the scaled prior valued
configuration. Sixty-four different scaled prior value configurations
were used and they were fonned by setting o(A) := 0 or Zk and
p(S) := 0 or 2m, where k = -3, -2, •.. ,3 and m = -3, -2, ••• ,3.
Four thousand two hundred and twenty-five unknown component configurations were considered and these were formed by setting Rho (A) = .125k
and Rho (8) = .125m where k
= 0,
1, ... , 64 and m = 0, 1, ... , 64.
4.5 Design Evaluation for ANOVA, MIVQE(U,I) and MINCH Estimators
In order to evaluate the optimality of the designs enumerated, a
trace criterion suggested by Goldsmith and Gaylor (1970) is used where
the value is simply
obtain~d
by taking the trace of thecovarianre matrix
of the variance component estimators. This criterion is considered
optimal since it tends to concentrate the samoling at the stage for
64
which the variance component is large, relative to others, and is
sensitive to changes in sample size.
Hence, given a design structure we would calculate
(4.5.1)
for the special case when p(A) = Rho (A) and p(B) = Rho (B).
A design is said to be optimal for a given estimated procedure
from the group of designs considered if its trace value is the
smallest of all the designs in the group for a given sample size.
Tables 4.5.1 and 4.5.2 can be interpreted in the following
fashion.
When Rho (B) =p(B)=4and Rho (A) = p(A) = 4 and the basic
structure of the designs is replicated once, the design 5 with
code (5, 5, 2) was selected as mi"nimizing the trace criterion for
the MIVQE(U,I) whereas design 18 with a similar code (5, 5, 2) was
selected for the ANOVA estimators.
Both estimators suggest a balanced design when Rho (A) and Rho (B)
are less than one for replications 1 and 5.
The majority of differ-
ences between the two tables are similar to the example previously
described.
Although the optimal designs selected are structurally
different for the two estimation procedures, the design codes or
degrees of freedom in the analysis of variance are very similar if
not identical.
Optimal designs 5, 4, 1 and 2 in the MIVQE(U,I)
estimation are repeatedly replaced by designs 18, 60, 10, and 53,
respectively, in the ANOVA estimation.
Replicating the designs five times for the MIVQE(U,I) estimation
increased the region of optimality for the balanced design
Table 4.5.1 Optilnal designs (codes) for MIVQE(U,I) when all 16 designs are compared
•
_ _ _ _ _ • __ .•
_ . __ • •
___•
_ _ _ . _ . _ _ ._ _ n
Ilep 1
----
----- ._-
Rei' 5
Rep 5
Rep I
RepS
Rep I
Rep 5
Rep I
Rep 5
Rep I
Rep 5
Rep I
Rep 5
Rep I Rep 5
Ii
5
(5,5.2)
5
5
(5.5.2)
5
5
(5.5.2)
5
5
(5.5.2)
5
5
(5,5,2)
5
5
(5,5,2)
5
5
(5,5,2)
5
61
61
(6,5,1)
4
5
(5.5.2)
5
5
(5,5.2)
5
5
(5.5.2)
5
5
(5.5,2)
5
5
(5,5,2)
5
5
(5.5,2)
5
5
(5.5,2)
5
61
61
(6.5,1)
8
8
8
(4.4.4)
8
8
8
8
(4,4,4)
8
5
8
(5,5,2) (4.4.4)
5
(5,5,2)
5
5
(5.5,2)
5
9
9
5
8
(5,5.2) (4.4,4)
4
(6.4.2)
(4,4.4)
1
a
Rei' 1
--------~-~------~--
2
ttl
..... _ _ _ • _ _ _ _ _ _ _ _ ~ _ _ _ _ _ _ _ ~
"
9
9
(3.3.6)
(4.4.4)
13
13
4
2
(8.2,2)
2
(7.4,1)
9
0.3.6)
9
8
9
(4.4.4) 0.3.6)
(4,4.4)
9
9
(3.3.6)
9
9
(3.3.6)
9
8
9
(4,4.4) 0.3,6)
8
6
(6.4.2) (4,4.4)
2
(8.2.2)
2
1
(9.1.2)
!l
9
9
(3.3.6)
9
9
(3.3.6)
9
8
9
(4,4.4) (3.3.6)
6
(6.4.2)
6
2
(8.2,2)
2
1
(9.1.2)
!I
9
9
9
9
(3,3.6)
9
8
9
(4.4,4) O,3.6}
6
(6.4,2)
6
I
1
9
0.3.6)
9
9
(3.3.6)
9
8
9
(4,4.4) (3.3.6)
6
(6,4,2)
6
I
I
(3.3.6)
8
8
,.-.
~".j
u
.5
~
c:
.25
.12!!
0
---.
9
0.3.6}
9
9
9
(3,3.6)
9
(3.3.6)
9
9
0.3.6)
9
-.-.---- ----
+-'-
9
0.3.6}
(3,3.6)
(3.3.6)
9
(3.3.6)
9
(3.3.6)
(9,1.2)
(9.1,2)
I
(9.1,2)
1
(9.1.2)
---------._---------_._-------_._-----_._-
0
.125
.25
.5
I
2
4
8
Rho (A) • p(A}
0'0
01
-
e
e
--
-Table 4.5.2
e
Optima 1 designs (codes) for ANOVA es t ima tors when all 16 designs are compared
-._--.. _- ... ---------------;-Rep 5
Rep 5
Ilel) I
Ilep I
Rep I
Rep I Rep 5
Rep 1 Rell 5
. - .- - -. -- - -_.-- --_. ---------- ----- - ------------------II
18
18
Iii
Iii
18
18
18
18
III
(5,5,2)
(5,5,2)
(5,5,2)
(5.5.2)
(5.5.2)
4
2
18
(5,5,2)
18
8
8
(4,4.4)
18
18
(5,5.2)
18
8
II
8
(4,4,4)
8
9
9
(4,4,4)
9
9
_.
(3,3,6)
r.:
Iii
(5,5,2)
9
9
0,3,6)
(3,3,6)
- ... ------------------_...
Rep 5
Rep I
Rep I
Rep 5
Rep 5
Rep I
Rep 5
18
18
(5,5,2)
18
18
(5,5,2)
18
61
(6,5,1)
61
61
(6,5,1)
61
18
(5.5.2)
18
18
(5.5.2)
18
18
(5,5,2)
18 .
18
(5,5,2)
18
II
8
5
(5,5,2)
5
5
(5,5,2)
5
18
(5,5.2)
18
5
(5,5,2)
8
(4.4,4)
60
(6,4,2)
(4,4,4)
(1.4,1)
16
(6,5.1)
18
(5,5,2)
53
(8.2,2)
60
(6,4,2)
lJ
8
(4,4,4)
( 3.3,6)
9
8
(4.4.4)
8
9
9
8
(4,4,4)
9
(3,3,6)
6
21
(6.2,4) (4.4,4)
2
(8,2.2)
60
(6.4,2)
10
(lO,1.1)
53
(8,2,2)
9
(3.3,6)
9
8
(4.4,4)
9
(3,3,6)
6
(6.2,4)
21
(4,4.4)
I
I
10
(10,1. I)
10
(9.1.2)
9
( 3,3,6)
9
9
0,3.6)
9
6
(6.2.4)
6
I
(9,1.2)
I
10
(10,1.1)
10
9
9
6
(6.2,4)
9
6
(6.2.4)
6
I
1
10
(10,1,1)
10
(3,3.6)
u
&.
.5
9
9
(3,3,6)
.25
.Il!l
9
9
13,3,6)
9
9
9
9
9
9
9
9
9
9
9
13.3,6)
• • _0
0
_ ••
__
•
_______ •
____ •
9
(3,3.6)
9
13,3,6)
9
9
9
9
9
(3.3.6)
13.3.6)
(3,3.6)
9
(3.3,6)
0,3,6)
(3,3.6)
II
9
(3.3,6)
(3.3,6)
_ _ _ _ _ _ _ _ _ _ _ _ •• _ _ _ _ •
.125
(l,l.6)
(9.1,2)
_ _ _ _ _ _ _ _ • _ _ _ _ _ _ _ _ _ _ _
.25
.5
I
2
------------ ------------4
8
Rho (A)
0\
0\
67
configuration and design 8 with code (4,4,4) occasionally replaced
designs 5 (5,5,2) and 6 (6,4,2). Simil,arly for the ANOVA
estimation, increasing the number of repliciations to 5 increases the
region of optimality for design 9 and three new designs are introduced, namely designs 16 (6, 5, 1),60 (6, 4, 2), and 21 (4,4,4).
A table describing those optimal designs for the combined
estimator MINQH was not included since those optimal designs selected
were identical to those selected for the MI"QE(U,I) estimator seen
in Table 4.5.l.
When es timati ng vari ance components frClm balanced data such as
design 9 (3, 3, 6), the estimators and their' variances are identical
for all three estimators.
When design 9 was excluded from the
selection process it was consistently replac:ed with design 8 with
code (4, 4, 4) as the optimum design in Tables 4.5.1 and 4.5.2.
ANOVA,
4.6
The most common method for estimating variance components is
through equating observed and expected mean squares in the analysis
(ANOVA) and solving the resulting equations for the estimators.
are known to have a number of desirable properties:
they are
unbiased, easy to compute, have a positive estimate of the error
component and under normality for balanced data they have minimum
variances among all unbiased estimators.
Hm"ever, for unbalanced
data, optimal properties beyond unbiasedness and a positive mean
squared error term are largely unknown and lacking.
MINQE(U,I) on the other hand because of their dependence on
prior weights have locally minimum variance, meaning the same as
They
68
"best in Townsend and Searle (1971) and La Motte (1973b)and "1oca11y
lf
best" in Harville (1969).
correctly specify peA)
The fact that in application one cannot
=Rho (A)
and pCB)
=
Rho (B) and so cannot (given
unbalanced data) use estimators which are truly minimum variance begs
the question "'Why use
r~INQE(U,I)?".
First, no matter what values are
used for peA) and pCB), so long as the values are chosen independently
of the data, the estimators based on them are unbiased, which is
all one can claim for the ANOVA estimates with unbalanced data.
Second, if the peA) and pCB) are not specified "correct1y," it would
be of interest to see for whi ch unknowns Rho (A) and Rho (B), the resu1 ti n9
estimators, have smaller variances than ANOVA estimators.
Simi 1arly, by construction, the MINQH estimators are unbi ased; but
unlike the MINQE(U,I), the variance of the ee estimate is identical
to the variance of ee using the ANaVA estimator.
We would also like
to see if peA) and pCB) are incorrectly specified, for which unknowns
Rho (A) and Rho (B), the resul ti n9 rHNQH estimators, have smaller vari ances
than ANaVA and MINQE(U,I).
Since 4,225 combinations of Rho (A) and Rho (B) are considered
rather than enumerating those points where the variance of one
estimator is smaller than another, we chose to consider the proportion
of scaled unknown components where the variance of one estimator is
smaller than another estimator of interest.
Table 4.6.1 can be interpreted as follows.
For design 5, of
those 64 combinations of priors pCB) and peA) of interest, when
8 and 2, respectively were chosen we found the ratio of the variance
of the MINQE(U,I) of 0 a divided by the variance of the ANaVA estimate
Table 4.6.1
Priors that maximize the percentage of points where the variance (MINQE(U,I))/variance
(ANOVA estimates) < lover all unknown components
-_.-_._----------------_._--IIlf1iCAHU
,'nIGH
IllS IliN
VA'II£S
,.(11)
,,(A)
2
~ml ~
~i\(m ~ I
VACII) -
I
VU(It)
vo(iif ~
oller 1
grids
I
V£{"}
1
timn
~
Vj!{!l
fil..r
~
oller 1
1
VA(tIU < 1
VAni) -
V8(ltI!l
volii)
o
78
60
71
Ii
8
89
94
74
~JW ~ 1
YM~!I)
<
VAnl, _.
over 1
grids
grids·
Ii
~ 1
1
~!!m!l
VOl"" ~
1
~wm
VE{Nr ~
1
o
o
o
o
a
o
58
0
0
2-1- - - - 6 8 - - - - < 1 - 46
46 ----------0
17
-
--. -,.
III
II
--.
8
------
<',
8
91
---'._--.----
I!
8
9!!
9
0
43
12
o
99
74
100
99
68
100
89
99
66
11
100
o
o
:J4
100
lOG
96
94
100
16
46
0
0
_._---- - - - -
Ii
89
-_._-_. ---------
61
4---8 ------9r-----loo------Y-t"
foo
97
100
92
14
26
15-----0-5J
ir---- 8---- --87-------0"2 -----80-----1oo----lii----8r---aj)--~9----SO__---]8---a----,------ jj ----0-----99-----100------100------97---100
100
100----<1
0
0
1
•
m
5"
a- ---- 0------- ar---m--ilr----ar
100 ---------.-n--if-----9a--------97--JiilY
96
611
a- - 11---- -A9 - - - --Sr------n5""----lOO
- - - Ii
-1 ------98 ---- - 9'-----100
96
61
~~~::::::=89-~::::~=~~B2------85"
100U
!_-:-::::
n
87
81
98
99
~--
82
99
89
Al
100
86
30
1
46
6
§II
100
2 g - - - - 4A
]
6
81·
86
zg----48
4"2----0<)
lD
lil
<I
4
0-4
.L-
0\
\0
e
e
e
70
of ea denoted VA(N)/VA(N) was less than or equal to one for 98 percent
of those 4,225 unknown scaled components of interest. Similarly,
for the estimate of 6 b the ratio of the variances denoted VB(M)/VB(N)
indicates the variance of the MINQE(U,I) was less than (not in the
strict sense) the variance of the ANOVA estimate for 71 percent of
those unknown components considered.
Finally, for the indicated priors o(B) and o(B) we find the
variance of the MINQE(U,I) of ee is less than the variance of the
ANOVA estimate of ee for all unknown components and is described by
VE(M)/VE(N)
L
1.
The average of these three percentages has a value of 90 and is
denoted by V(M)/V(N)
~
1. This value gives some indication of how
well the MINQE(U,I) does in relation to the ANOVA estimates based on a
risk function comparison over all unknowns.
Those priors maximizing
this average are then adopted and included in Table 4.6.1 for the
15 different design structures excluding the balanced design.
Table 4.6.1 also includes the percentages V(MH)/V(N)
V(MH)/V(M)
~
~
1 and
1 where (MH) indicates the combined MINQH estimator.
Remember VE(MH)
= VE(N)
for all unknown components and is not recorded
in the following tables.
results between r
= 1,
r
There was very little difference in the
= 5,
and hence no separate breakdown is
recorded.
In terms of applications, perhaps an experimenter has unbalanced
data like structure 21 and is without prior knowledge of estimates
of components determined independently of the data other than the
predetermined fact they are probably smaller than 8.0. Selecting
7i
priors p(A) = p(B) = 8.0 and using r1INQE(U,I) would guarantee him a
smaller risk function than using ANOVA estimators on the average 98
percent of the time over the three ratios of variances where the
proportions are less than or equal to one.
This procedure is somewhat restricted by the fact that .only 64
combinations of priors are considered.
It does appear, however,
that of these 64 priors, several can be found whereupon implementing
rUNQE(U,I), the variance of the estimates w'ill be smaller than those
of ANOVA estimates for a majority of those 4,225 scaled unknown
components.
Table 4.6.1 reveals that for priors
proportion where V(M)/V(N)
~
p(J~)
= p(B) = 8.0
the
1 ranges from j'8 percent for design 1
to 98 percent for design 21 and VE(M) is uniformly smaller than VE(N)
for all unknown components and all designs. Although the above
priors are not selected as those maximizing the proportion where
V(M)/V(N)
~
1 for designs 5, 10, 53, 55, 60, and 61, these priors
appear to do well in a risk comparison as indicated in Table 4.6.1.
The proportion where V(MH)/V(N)
of V{M)/V{N)
~
~
1 is almost identical to that
1 for allIS designs. A closer examination of the
proportion of V{MH)/V{M)
~
1, however, suggests that the variance of
the MINQE(U,I) is somewhat smaller than the variance of the MINQH
estimates for a large proportion of unknowns.
In fact only in the
estimation of 0 a does. the variance of the combined estimator rUNQH
seem to be smaller than the variance of the HINQE(U,I) for approximately 50 percent of the unknowns over all 1!5 desi gns.
Figures 4.6.1 - 4.6.7 examine the topological features of the
ratio of the variances of the different estimators over the scaled
~
72
1.G31
1.001
a.1l
0.911
Figure 4.6.1 Ratio of the variance (MItIQE(U.ll )/variance (ANOVA) for
estimating the A component (using design 17 the a priori
values are p(A) : 8 and p(B) : 8 (99 percent of region
< :
1.0))
73
1. 031
l.I)!)1
a.O
0.911
Figure
~.6.2
(AMO~A)
Ratio
of thethevariance
(MIMQH)/variance
estimatino
A component
(Using design 17 thefora priori
values are ptA) a a and p(B) a a (99 percent of regiOn
<::1.0))
~
..
74
1.OQOOE+OO
a.O
9.9998E-Ol
2.1
9.9991E-Cl
Ratio of the variance (MINOH)/variance (MINQE(U.l)) for
Figure 4.6.3 estimating the A component (using Design 17 the
a priori values are ptA) = 8 and p(B) = 8 (43 percent
of reg; on
<
=1. 0 ) )
75
RVSI'W
1.25
1.15
8.1)
1. t)5
Rhc(SJ
Figure 4.6.4 Rat; 0 of the varl ance (r·lINQE(U,I))/vari ance (ANOVA) for
estimating the a component (Using Design 17 the
a priori values are p(A) = 8 and p(B) = 8 (74 percent
of region < = 1.0))
e·
76
RV8HN
1. 25
1.15
8.0
1. ~5
5.3
~ho(B)
Figure 4.6.5 Ratio of the variance (MINQH)/variance (ANOVA) for
estimating the 8 component (Using Design 17 the
a priori values are p(A) = 8 and 0(8) = 8 (74 percent
of region < = 1. 0))
77
RVBHM
1.1)03/f
1. '1022
8.0
1. 0011
0.9999
Figure 4.6.6 Ratio of the variance (MINQH)/variance (MINQE(U,I)) for
estimating the 8 component (Using Design 17 the
a priori values are p(A) = 8 and 0(8) = 8 (l~ percent
of region < =1.0))
78
':1/
.;.:t:1
~~.
~.).fJ
79
unknowns for design 17 with priors p(A)
= p(B) = 8.0.
Identical
surfaces for the ratios VA(M)/VA(N) and VA(MH)/VA(N} are depicted
in Figures 4.6.1 and 4.6.2 and further illustrated in Figure 4.6.3
by examining the ratio VA(MH)/VA(M) for all scaled unknowns considered.
Likewise for the estimation of 9 b Figures 4.. 6.4 and 4.6.5 reveal the
two ratios VB(M)/VB(N) and VB(MH}/VB(N) are also very similar.
However, Figure 4.6.6 indicates VB(M)
~
VB(MH) for 88 percent of
those scaled unknown components considered.
By examining Figure
4.6.7 one would conclude the variance of thE! MINQE(U,I) is only
slightly smaller than the variance of the ANOVA estimate of Se
universally over the unknowns considered.
Table 4.6.2 depicts those priors maximizing the average of the
3 percentages VA(MH}/VA(N)
VE(MH)
~
~
1, VB(MH)/VB(N)
~
1 and 100 since
VE(N) for all scaled unknowns and is denoted V(MH)/V(N)
~
1.
For designs 2, 4, 5, 8, 13, and 17, the recommended priors are
p(B} = 1 and p(A)
=
2 when using the combined estimator MINQH and
no prior knowledge component estimates determined independently of
the data.
Designs 1, 6, 10, 18, 21, 53, 55, 60, and 60 suggest
alternative priors; however, with the exception of design 1, use of
priors 1 and 2, respectively, appear to do an adequate job in a risk
comparison with the ANOVA estimators.
Although the proportion V(r1}/V(N)
V(MH}/V(N)
~
~
1 is universally less than
1 for all designs ccnsidered, a close examination of
the proportions VA(M)/VA(N)
~
1, VA(MH)/VA(N)
~
1, and VA(MH)/VA(M)
indicates the variance of the MINQE(U,I) estimate of 0 a is less than
the variance of the combined estimator for a rather large proportion
~
1
e
e
e
Table 4.6.2 Priors that maxilnize the percentage of points where the variance (MINQH)/variance
(A"OVA estimates) < lover all unknown components
-------- ----- ------_.-._------InJicdted
Prior
Values
Ilesign
pIA)
p(U)
'!(~Jl • I
VCNr over 3
grids
~!lJ~!l '" I
VA1N,-
-\----.125--..---0-1----64
_ _-_1. ---. i-==lr-- 15
2
I
4
I
VBTh, ~
86
82
I
~f~ ~
over 3
grids
52
I
VA(~
VJ\lt'T ~
I
VB (H.)
VBCHT ~ I
VE{IIl
vWiT
55
60
64
91
81
11
4
~ I
WfTl ~ 1
VAJ!tJl
VAlItr ~
I
VB(~
varMT ~
I
VE!ffil
VETM') ~ 1
over 3
grids
- - , - 3 . - - - -..
40.----"'82.----....9"""6-6]
36
10
83
2
91
96
82
66
97
83
18
62
25
80
82
2
91
91
81
42
91
IJ
23
61
22
83
80
99
82
45
56
54
99
65
60
15
100
84
21
]
18
60
63 - - - 0 9
18
51
41
84
89
70
79
97
82
38
87
83
22
11
16
61
65
63
20
22
78
86
5
I
2
93
-------1l----1---84
___ ~___
------._-------------_.--------------------------VB(t~tl
--Y--OI
60
82
79
65
94
99
94----91
86
-76
85
89
82-
51
62
58
99
89
75
34
85
86
14
94
99
83
47
99
23
19
65
32
83
81
1
2
95
1
4
99
1
2
96
2 - ; - - - - 0 - - 8--100
99
99
99
100
85
100
54
49
99
99
44
19
24
69
73
41
60
84
82
81
76
10
100
86
24
40
0
46
100
94
U
I
-10-----:5-__ u
13
2
2
~~C=--t
I
2
17
if1-
--~-
1
I
~--~
--ga----l00
2
gO88
-S98
97
46
57
50
95
79
99
96
19
61
14
88
80
17
97
16
27
53
7
53
12
24
14
14
50
57
83
11
5]
27
0
4
84
0
49
0
10
83
10
55
26
57
8]
43
98
&2
9'i
86
100
M
97
100
88
100
100
90
89
92
56
89
62
17
-1--~-94
80
97
80
66
55
--1--y-----gO----S6
-------0--8---100
100
60
-T--- 2
92---~
--------0---- 8--1iiO
100
--n-2-
88
97
86
]]
\9
61
97
61
97
_~I___
.1
2
2
4
88
100
22
\8
100
15
90
3
3
32
26
0
87
84
81
7r3
86
00
o
81
of cases. On the other hand, for those pri10rs considered in
Table 4.6.2 the proportion of unknown components where VE(M}/VE(N}
~ 1
is significantly less than the proportion of unknown components
where VE(MH}/VE(N}
~
1.
This fact is further illustrated by
examining the percentages in the VE(HH)/VE(M)
~
1 column of
Table 4.6.2.
Like those figures for design 17 descr'ibed earlier, Figures
4.6.8 - 4.6.15 display the ratios of the valriances for priors
pCB) = 1 and peA) = 2. Similar surfaces fOI" the ratios VA(MH)/VA(N)
and VA(M}/VA(N} are displayed in Figures 4.6.8 and 4.6.9.
Figure
4.6.10 shows the ratio VA(MH)/VA(M} is less than or equal to 1.0 for
43 percent of the unknowns considered and the region around pCB)
and peA)
= 2 has
=1
a value larger than one as expected since MINQE(U,I)
is a locally best estimator. The surfaces describing the ratios for
the eb component are 1ess uni form than thOSE! for the ea component
and are depicted in Figures 4.6.11 and 4.6.12. The ratio VBU·1H}/VB{~1}
described in Figure 4.6.13 is less than or equal to 1.0 for 84 percent
of the unknowns considered, a value almost
t~vo-fold
the amount for
the 0 a component. Figures 4.6.14 and 4.6.15 are interesting in the
fact that
.. since VE(HH) = VE(M) for all unknowns considered, the
graph of Figure 4.6.14 is just the inverse of the graph of Figure
4.6.15 as indicated by examining the percentage of unknowns less than
or equal to one.
82
RVAHN
1.1)19
0.99&
8.1)
C.973
0.9"9
Figure 4.6.8 Ratio of the variance (HINQH)/variance (ANOVA) for
estimating the A comoonent (Using design 17 the
a prior; values are o(A) = 2 and o(B) = 1 (99 percent
of region <= 1.0))
83
RVfW4
0.996
8.0
1).973
O.9~9
Figure 4.6.9
Ratio of the variance (HINQE(UtI))/variance (ANOVA) for
estimating the A component (Using design 17 the
a priori values are o(A) = 2 and p(8) = 1 (99 percent
of regi on < =1. 0) )
84
!WAH-...
1.OOO~E+OO
1.0002E...OO
S.O
9. 9990E-')l
9.9963E-Ol
Figure 4.6.10 Ratio of the variance (MINQH)/variance (MINQE(U,I)) for
estimating the A comnonent (Using design 17 the
a priori values are p(A) = 2 and 0(8) = 1 (43 percent
of regi on < = 1. 0) )
85
RVSHN
1.12
1.1)7
1. 02
0.97
Figure 4.6.11
Ratio of the variance (MINQH)/variance (ANOVA) for
estimating the B component (Using design 17 the
a priori values are p(A) = 2 and p(B) = 1 (85 percent
of region < = 1.0))
86
RV8~
1. 08
1. 03
8.0
0.99
Figure 4.6.12 Ratio of the variance (r4HfQE(U,I))/variance (ANOVA) for
estimating the 8 component (Using design 17 the
a priori values are p(A) = 2 and p(8) = 1 (44 percent
of regi on < =1. 0) )
87
RV8HM
1. a'f 1
1. a18
8.0
0.995
a.971
i\ho(AJ
Figure 4.6.13 Ratio of the variance (MINQH)/variance (MINQE(U,I)) for
estimating the 8 component (Using design 17 the
a priori values are p(A) = 2 and 0(8) = 1 (84 percent
of region < =1.0))
88.
RVEMN
1.~5
1. 29
6.1)
l.l2
Figure 4.6.14
Ratio of the variance (MINOE(U,I))/variance (ANOVA) for
estimating the E component (Using design 17 the
a priori values are o(A) = 2 and p(B) = 1 (19 percent
of region < = 1.0))
89
!WEHH
0.92
8.1)
1).81
0.59
Figure 4.6.15 Ratiq of the variance (r1INQH)/variance (MINQE(U,I)) for
estimating the E component (Using design 17 the
a priori values are p(A) = 2 and 0(8) = 1 (81 percent
of region < =1. 0))
90
4.7 MINQE{U,I,l)
Rao (1970, 1979 ) suggests the use of priors p(A)
= p(B) = 1
when independent information is not available a priori.
and Searle (1978) suggest a p(A)
= 1 and
Swallow
the use of MINQE{U,I) over
ANOVA for estimating Sa in the one-way classification if Rho(A) is
greater than or equal to 1.
Table 4.7.1 like Tables 4.6.1 and 4.6.2 examines the percentages
·of unknowns where V(M)/V(N)
~
1, V(MH)/V(N) ~ 1, and V{MH)/V(M) ~ 1
for the 15 designs of consideration when priors p(A)
are adopted.
= p(B) = 1.0
One should pay special attention to the fact that the
proportions of unknowns where V(MH)/V(N)
~
1 are invariably larger
than the respective proportion V(M)/V(N)
~
1 for allIS designs.
This undoubtedly can be attributed to the fact that the proportion of
unknowns where VE(M)/VE{N)
~ 1
is very small and VE(MH)
= VE(N).
Also of interest is the large proportions in the column representing
the percentages of V(MH)/V(M)
~
1.
Figures 4.7.1 - 4.7.8 examine the effects of priors
p(A) = p(B) = 1 on the ratio of the variances of the estimates for
design 17. Of special interest is that VA(MH)/VA(M) is less than
or equal to 1 for 71 percent of the unknowns as displayed in
Figure 4.7.3, VB(MH)/VB(r1)
~
Figure 4.7.6 and VE(MH)/VE(M)
1 for 85 percent of the unknowns in
~
1 for 89 percent of those unknowns
considered as shown in Figure 4.7.8.
Figure 4.7.7 indicates that when priors p(A) = p(B) = 1.0
are used and the true unknowns are Rho(A) =Rho(B) = 8.0, then the
ratio
VE(~1)/VE(N) ~
1.88 as compared with 1.45 and .999 for priors
p(A) = 2, p(B) = 1 and p(A) = ~(B)
and 4.6.7.
= 8.0
displayed in Figures 4.6.15
Table 4.7.1 The effects of priors equaling 1 on the ratio of the variances over all unknown
components
--
..
-------- ------"-
-
--------_.
~f~+
V
N -<
\
< 1
VA N --
IJesigll
VAP!!
over 3
grids
-- - - - - - - - - - - - 4\
46
~~HH ~
\
V£
it ~ 1
VW*
~f!ijl
V
N -< 1
~Af!»tl
VA N1
<
-
1
~~fNl
VB
N
over 3
grids
~
1
~lW- ~
1
VmrP
VA
I
~
1
YJ!HNl
VB H ~ 1 V
~~<1
H -
over 3
grids
70
8
71
47
61
67
66
41
92
2
44
53
71
9
74
54
69
10
54
65
91
4
44
92
28
13
88
93
92
70
43
80
87
5
41
99
10
15
94
99
82
68
33
86
85
6
43
48
n
8
78
65
68
69
70
45
92
8
4J
99
15
15
94
99
85
69
31
86
85
10
43
5)
67
8
7J
54
64
69
64
52
92
J]
]2
63
22
11
76
63
65
79
63
89
63
17
43
lI5
32
11
84
86
65
82
71
85
89
18
41
100
11
12
82
100
41
66
16
93
88
21
41
96
15
12
75
96
29
67
18
95
88
tl3
44
67
56
8
77
67
62
74
52
79
92
55
45
74
54
8
78
74
59
16
62
14
92
60
43
79
42
8
81
80
61
76
56
78
92
61
40
81
30
8
83
82
68
76
54
84
92
----------
\0
~
e
e
e
92
RVAM.."
1. 03
0.99
0.95
Figure 4.7.1
~8.1)
Ratio of the variance (f.1INQE{U,I))/variance (ANO'lA) for
estimating the A component (Using design 17 the
a priori values are p(A) = 1 and 0(8) = 1 (85 percent
of region < =1.0))
93
RVAHN
1. 02
1).39
0.95
~8'iJ
iJ.92
Figure 4.7.2
Ratio of the variance (MINQH)/variance (ANOVA) for
estimating the A component (Using design 17 the
a priori values are o(A) = 1 and 0(8) = 1 (86 percent
of region < =1.0))
94
RVAHl1
1.1}00lfE'POO
1. fJ001£+00
8.0
9.9991£-01
9. 9966E-0.1
a.1)
Figure 4.7.3
Ratio of the variance (MINQH)/variance (MINQE(U,I)) for
estimating the A component (Using design 17 the
a priori values are o(A) = 1 and 0(8) = 1 (71 percent
of region < =lor)))
95
RVB~
1. 38
1. 22
8.1)
1.1)7
~ho(A)
Figure 4.7.4
Ratio of the variance (MINQE(U,I))/variance (ANOVA) for
estimating the B component (Using design 17 the
a priori values are p(A) = 1 and p(B) = 1 (82 percent
of region <=1.0))
96
RVBHN
1. 31
1.13
0.95
Figure 4.7.5
Ratio of the variance (MINQH)/variance (ANOVA) for
estimating the B component (Using design 17 the
a priori values are p(A) = 1 and p(B) = 1 (65 percent
of regi on < = 1. 0) )
97
s.t}
Figure 4.7.
~ariance (MlnOH)/~ariance
Ratio of the
(MlnOE(U.l)) for
component
(using
design
17 the
estimating
the
5
6 a priori
are ptA) = 1 and p(5) = 1 (55 percent
~alues
of region
<::
1.0))
98
RVE~
1. 88
1.57
8.1)
1. 26
Figure 4.1.7 Ratio of the variance (HINQE(U,I))/variance (ANOVA) for
estimating the E component (Using design 17 the
a priori values are o(A) = 1 and ~(B) = 1 (11 percent
of reg; on < =1. 0) )
99
RVEH."1
1. Gf!
0.08
8.1)
G.71
~I).I)
0.1)
Figure 4.7.8 Ratio of the variance (MINQH)/variance (MINQE(U,I)) for
estimating the E component (Using design 17 the
a priori values are p(A) = 1 and p(B) = 1 (89 percent
of region < = 1.0))
100
4.8 MINQE(U,I,O)
MINQE(U,I,O) is designed to handle efficiently very large
designs.
It is Rao's MINQE(U,I) with zero priors except for the
prior used for the error component.
In most cases the value of 1 is
adopted for the error component prior value. This method was recently
advocated by Hartley, J. N. K. Rao, and La Motte (1978) because of
the following properties:
unbiasedness, locally best (at zero),
asymptotic consistency, admissibility, best unbiased if the design
is balanced, and computationally efficient.
It would seem that the
latter point far outweighs consideration of the one underlined.
Goodnight (1979) suggests this procedure because of those optimal
properties but seems to make no mention concerning the. potentially
hazardous effect when the true unknown components are not in fact
close to zero.
Table 4.8.1 examines the effects of using zero priors.
proportions V(M)/V(N)
~
The
1 are consistently smaller for all designs
compared with those tables considered previously.
However, for
designs 1 and 6 the variance of the MINQE(U,I,O) of 6 b is uniformly
less than the variance of the ANOVA estimate over the grid. Also,
for 95 percent of the cases considered in design 21 the variance of
the MINQE(U,I,O) of 0 a is less than the variance of the ANOVA
estimate.
TJte combined MINQH estimator minimizes the ratio of the
variances with the ANOVA estimator for a larger proportion of unknowns than
MINQE(U~I,O)
and with the exception of design 18 the
variance of the combined estimators is less than the variance of the
MINQE(U,I,O) for a majority of those scaled unknowns considered.
Table 4.8.1 The effects of zero priors on the ratio of the variances over all unknown components
--, _. -
_._.- --- ,----+- - - - - - - - - - - - - - - - - - - - - - _ . _ - - - - - - - - - - - - _ . _ - - - - - - -
-
_.
-r-
V(HJ < 1
VE(~i. ~ 1
V~f~J
~~fn~ ~ 1 VO
VN N -< 1 VUH
over ]
grids
.- -- . --_._ - ---------- -_._--------------]!l
100
< 1
17
lieS i 1111
VI"!!
- -ij
< 1
V
H -over ]
grids
¥~I'W
VA H
-< 1
----------Y!l~f
VJ7fl
it ~
VB" -< 1 V
1
-----_.... _ - - VE(.'!!
1
Y~,p.
!~!ff
VA H ~ 1 VB
H ~ 1 VEfH) ~
over 3
9rlds
..
54
19
41
80
98
41
99
9
13
91
95
77
99
2
3
9
, 1
, 1
41
4
" 1
' 1
" 1
" 1
34
<
1
2
97
91
99
99
!o
, 1
<
1
' 1
<
1
36
<
1
9
96
89
99
99
6
]6
7
100
<
1
50
7
42
80
99
42
99
II
, I
1
, 1
<
1
l6
" 1
8
99
98
99
99
10
7
20
, 1
<
1
50
21
28
92
98
80
99
IJ
5
II
<
1
<
1
39
14
4
99
98
99
99
17
4
12
<
I
<
1
38
12
2
99
9'J
99
99
W
, 1
1
, i
<
1
35
2
1
67
1
99
99
21
32
95
<
1
<
1
66
95
1
76
27
99
99
!oj
II
2]
- 1
<
1
46
24
Il
99
96
99
99
5!o
9
26
, I
<
1
45
28
7
99
99
99
99
60
14
40
, 1
<
1
50
48
3
99
96
99
99
61
16
49
" 1
<
1
54
59
4
97
92
99
99
u
<
<
__________
~
o
~
e
e
e
102
For priors p(A) = p(B) = 0, the ratios VA(M)/VA(N) and
VA(M'H)/VA(N) are very similar for 0 a as depicted in Figures 4.8.1;
4.8.2 and 4.8.3. For the first time in Figure 4.8.4 we see that for
p(A) = 0 = p(B) and the true Rho (A) = 8 and Rho (B) = 0, the ratio of
the variances denoted VB(M)/VB(N) equals 15.2. On the other hand,
Figure 4.8.5 shows the ratio VB(MH)/VB(N) ~ 3.34.
Figure 4.8.7
examines the case when p(A) = p(B) = 0 and Rho (A) = Rho
then VE(M)/VE(N)
MINQE(U,I,O).
~
(I~)
= 8.0;
118, a truly alanming feature when implementing
103
RVAI'W
1.21
8.l)
1. 00
Figure 4.8.1
Ratio of the variance (MINQE(U,I))/variance (ANOVA) for
estimating the A component (Using design 17 the
a priori values are ~(A) = 0 and p(B) = 0 (12 percent
of regi on < = 1. 0) )
104
RVAHN
1. "2
1.21
8.1)
1. 00
Figure 4.8.2
Ratio of the variance (MINQH)/variance (ANOVA) for
estimating the A component (Using design 17 the
a priori values are p(A) = 0 and p(B) = 0 (12 percent
af reg ion < =1. 0) )
105
RVAHH
1.0000
O.9991t
8.1)
0.9988
0.9981
Figure 4.8.3 Ratio of the variance (MINOH)/variance (MINQE(U,I)) for
estimating the A component (Using design 17 the
a priori values are p(A) = 0 and p(B) = 0 (99 percent
of region < =1.0))
106
RVS"""
15.2
10.ol
s.a
5.S
Figure 4.8.4
Ratio of the variance (MrNQE(U~r))/variance (ANOVA) for
estimating the B component (Using design 17 the
a priori values are c(A) = 0 and p(B) = 0 ~2 percent
of region < =1.0))
107
RV8HN
3.3
2.5
8.0
1.7
Figure 4.8.5 Ratio of the variance (r4INQH)/variance (ANOVA) for
estimating the B component (Using design 17 the
a priori values are peA) = 0 and pCB) = 0 (2 percent
of region < =1.0))
108
a.55
a.22
8.1)
Figure
~.8.6
Ratio of the variance (Ml"QH)/variance (Ml"QE(U,l)) for
estimating the B component (using design 17 the
a priori values are ptA) = 0 and p(B) • 0 (99 percent
of region
<:;
1.0))
109
RVEf'fl
118
79
8.1)
'fO
Figure 4.8.i Ratio of the variance (HINQE(U,I))/variance (ANOVA) for
estimating the E component (Using design 17 the
a priori values are p(A) = 0 and p(B) = 0 ~2 percent.
of reg ion < = 1. 0) )
110
RVEHH
1.4
0.9
O.S
"8,1)
Figure 4.8.8 Ratio of the variance (MINQH)/variance (MINQE(U,I)) for
estimating the E component (Using design 17 the
a priori values are p(A) = 0 and p(B) = 0 (99 percent
of region < = 1.0))
III
rUNIr~Ur.,
5.
NORM QUADRATIC UNBIASED ESTIMATION WITHOUT INVARIANCE
5.1 Introduction
Consider the situation where invariance does not hold and one
must attempt to find unbiased estimators, as in problems suggested
by Focke and Dewess (1972). One special case of this would be the
fo 11 owi ng . Let
Y
= XB
+
(5.1.1)
€
where
x=
1
0
o
1
1
0
o
1
and
E(
E:.
1
D(Y)
E.)
J
=
=0
,
1
0
0
0
o
100
o
o
000
000
0
o
0
l)
0
0
()
0
o
1
o
0
()
1
82
112
To find a quadratic estimator Y'AY which satisfies the conditions
of unbi asedness for f' 6 and invari ance for trans 1ati on of Y by Xe:, then
as previously described Y'AY satisfies (unbiasedness and translation·
invariance) iff tr(A Vi )= f i , i = 1, 2 and
AX
=0
(5.1.2)
•
Let A be a symmetric 4x 4 matrix with typical element a ij . Then
the condition AX = 0 would result in the following system of equations:
(S.1.3)
(S.1.4)
(S.1.S)
a24 + a44
=0
•
(S.1.6)
The condition of unbiasedness gives one the following equations:
But equations (5.1.3) - (5.1.8) result in the following unsolvable
system for f F f 2:
1
(5.1.9)
113
Thus we are not able to find the A matrix meeting the needs of an
unbiased and translation invariant estimator.
One possibility in this situation is tl) consider a least squares
approach described by Seely (l970b). The method suggests solving the
following system of equations:
(5.1.10 )
and the resulting system of equations in this example suggests
solving:
1
(
a ) ( ~1
o 1
62
) =
~
( Y1 (Y 1-Y 3 ) + Y2(Y2- Y4) ) .
(5.1.11)
Y3(Y 3-Y 1) + Y4(Y4-Y2)
5.2 MINQE(U)
If one drops invariance and considers only unbiasedness as
suggested by MINQE(U), it is advisable to use an a priori value So
of S and change Y to Y-XS O and S to (a-sO) denoted y* and 8*,
respectively, and work with the transfonned model in addition to the
transfonnation indicated in (2.2.4).
The class of unbiased estimators
of fie is defined by
C: = {A:
A symmetric, XIAX
= 0,
tr(AV i )
= f.,
1
i = 1, ... ,p}
(5.2.1)
where X and Vi are as in the general model (2.2.1).
Theorem (5.2.1):
Let Va. = U*U~ be non-singular.
If
cJ is non-empty, then the
MINQE(U) of fIe under the Euclidean nonn (2.3.4) is
114
y =I
AiV*A.V*,
A.1
1·
= (Va+XX,)-I(V.-p
V.P')(Va+XX,}-l
1 ala
where A = (AI' ••• , Ap)' is any solution of QA
=f
(5.2.2)
with
Proof:
We have to minimize
But N = I uif i so as before
needs to minimize
tr(NU~U*)
tr(AVaAV a ) + 2 tr(AV aAXX')
=I
aiuif i . So one only
= tr(A Va A {Va
+
2XX,} )
= tr(A TAT
)
a
a
for
T
a
= Va
+ XX'
since
X'AX
=0
•
Similar to the previous proofs by defining
A
*
= [ A'(V
+XX,)-l (V.-P V.P')(V +xX,)-l
1 a
1 ala
a
one can easily observe that X'A*X
= 0 by expressing
and observing the fact that pa X = x.
(5.2.4)
115
= f j and we conclude A* £: c~ .
Given any A£:C~ we can always write A = A + 0 where D£:C~
Now by construction tr(A*V j
C~
}
= {D: 0 is symmetric, X'DX = 0, and tr(DV i } =
°}.
(5.2.5)
Since C~ is a subspace of the set of all n~:n symmetric matrices, we
can directly apply Theorem (2.4.1) part (ii) which concludes that A*
minimizes tr(A. TAT}
a
a iff tr(D Ta A* Ta } ::
tr(A* TOT}
a
a
=I
° for any
0 £:. Cou'
Now
A·
V.P')T-1T 0 Ta }
l tr(T-1(vl·-p
a
alaaa
(5.2.6)
So A* does minimize the above.
Routine calculation of the quadratic form in (5.2.2) gives the
followi ng:
Y*A.Y*
1
= y*' (Va +XX,}-l
(v.-P V.P'}(V +Xx,}-ly*
1
ala
a
(5.2.7)
(5.2.8)
(5.2.9)
where
116
5.3 Explicit MINQE(U) for the One-Way Classification
Let
(5.3.1)
be the general form of the known model or specifically let
=u+
y ..
lJ
a.1 + e lJ
..
We will denote yij
= l, ... ,h;
i
j
= l,. .. ,n i
.
(5.3.2)
= Yij
- bO for some a priori value of the scalar u
and rewrite the above model as
(5.3.3)
Thus we have X
= nIl'
u -and bO scalars U1 (nxh) matrix of O's and
l's, U2 = In where n = L ni , a (h x 1) random vector, e (n x 1) random
vector and a and e are independent.
Hence,
(5.3.4)
Now V1 has typical elements 0ii
o..
=
11
I
[~
l
where
=i '
if
i
if
i ;. i
(5.3.5)
I
and the covariance matrix can be expressed by means of matrices I
and Vt as
where
0*
= e a/0 e
(5.3.6)
117
thus, similarly define
(5.3.7)
where
€
and
Va-1
p
indicate prior values of ee and
= € -1( I
p*,
respectively.
- V1*)
(5.3.8)
where V1* has typical elements pAniP+l)oii' .
The corresponding elements of V-I
are Clbtained as
a
(5.3.9)
Define the following sums:
5t
9
=
n~
r
i=1 (n.p+l)9
h
(5.3.10)
1
1
1
5o =\~. ni =n,
1
o =h
50
Now V-IX has typical element
a
S
-1(
1
n.p+l
)
(5.3.11)
1
and the scalar XlV-Ix can be written as
a
E
-1 51
1
The general element of Ra follows as
(5.3.12)
118
e:
-1[
0ii 10jj 1
_
e
1 ( 1 )( 1,)]
n.p+l 0;; 1 - -$1 n.p+l n. ,p-+l
1
1
1
(5.3.13)
1
and the typi cal el ement of Y*' R
a
e:
-1[ *
e y*
1
Yi'j' - n ,p+1 i ' + - $1(n. ,p+1)
1
1 1
(5.3.14)
(5.3.15)
Now rewriting (5.2.9) in explicit form as
y*1 R V.[V- 1 _ 2V-1X(X'V-lX+1)-lX'V-1 + V-1X(X'V-1X)-IX'V-1]y*
a1
a
a
a
a
a
a
a
(5.3.16)
and the term in square brackets can be expressed as
e:
-1
[0"10
.. ,
11 JJ
Thus
So the scalar
o
nil' +1
0 •. ,
11
119
Y*'R V R Y*-2Y'*'R V V-IX l X'V- 1y*
ala
ala
a
a
(which we will call TI)
n.p
1
n.
_ -2 h\ [y*
1
y*
1
- e:
t..
i+
n
.p+1 i+ -:T (n:.p+1)
S1 II
i=l
1
1
n.
1 (Sl-e:)
[Yi+
* - n P+1
'p
* - Sf
Yi+
(Si+e:)
1
1
ni
ni P+1
(5.3.19)
and the scalar
Y*'R V R Y* -2Y*~R V V-IX l XIV-ly*
a2a·
a2a
a
a
= e:-
2 h
ni
L L
i=l j=l
[y '!t. lJ
e
y*
n1P+1 i+
(which we will call T2)
- -
1
h
{n 1P+1)
L
1=1
Y~+
n1P+1
]
(5.3.20)
To find the elements of Q defined in (5.2.2) we must solve for
scalars of the following form:
(5.3.21)
Now by (5.3.17) the preceding term in square brackets multiplied by
VI has typical elements:
I
nil
n.,p+l ] .
n.p+l
1
1
(5.3.22)
120
From (5.3.13) the general form of RaV i
(5.3.23)
50 RaV Ra VI - 2Ra V V-IX
La XI V-a 1V has typical elements
2
2 a
1
(5.3.24)
thus tr(Ra V2 Ra VI - 2Ra V2 V-IX
La X V-a 1V1)
a
I
(5.3.25)
Now Ra V2 Ra V2 - 2R a V2 Va-1 X La X'V a-1 V2 has typical elements
e:
-2 ~
po; ; •
1
1
1
[0,0,0 ,0JoJo , - n;p+1 - -1
5 ( n op+ 1)( no, p+ 1)
l'
2
poo" o. + p noooo.
,,, + P
n,op+1
,
no.
, _ _--,:-
2
(n op+1) 2:r
S1 (nop+1)(no.p+1)
,
"
121
1 (Si-e )
1
1
(Si- e )
ni
- (
)(
) +..e... - 1 ' - --....,2~-1 (1 ) n p+l n ,p+l
1 (
(+1) (
+1)
SI ~l+e
1
1
SI
Sl+e)
niP
n; ,p
o
1 2 (Si- e )
+ (-1)
Sl
thus
1
t Sl +e).
o
Si
(n p+l)
o
1
1
(5.3.26)
(n ,p+l) ]
o
1
tr{Ra V2 Ra V2 - 2Ra V2 V-IX
La X'V-a 1V2)
a
(5.3.27)
Ra VI Ra VI - 2Ra Yly-1X
La x,v-a 1V has typical elements
a
1
2
-2 [doo.no - doo, !liP
£
n op+l
11
1
11
1
~ 0; i '
pn~1
niP
+1 + 0
n~p2
1
0
"
11
-"";"-'--,ro2
(n P+l)
i
(5.3.28)
122
so tr(R Vi R VI - 2R VI V-ly L XIV-lV )
a
a
a
a
a
a l
(5.3.29)
Now examination of the elements of Ra VI Ra V2 - 2Ra VI V-IX
La XIV-a l V
a
2
reveals that
Q = {tr(R V.R V' - 2R V.V-lX L XIV-lV .)}
a 1 a J
a 1 a
is a symmetric matrix.
Q =
[;
a
a
J
Let the elements of
(5.3.30)
:]
The MINQE(U) procedure estimates flea + f 2ee by means of the linear
combination AITI + A2T2 where we obtain the Ails as a solution of the
system
I;
7\.Q(ij)
1
= f.J
(j
= 1,
2)
(5.3.31)
that is
All + \29
= f1
,
(5.3.32)
A19 + A2 h
= f2
123
For f 1
ea
+
= 0 and
ee: ;
and for f 1 = f 2 = 1 we have
and then we get the estimators of the components of variance
f2
= 1,
we get
a£
separately:
ea
=
~
1
= ~~2
e
e
(5.3.33)
zh-g
(-g T +
1
z T2)
(5.3.34)
Thus we now have the MINQE(U) for the one-way in an explicit solvable
form.
124
6.
MINIMUM NOro1 QUADRATIC ESTIMATION UNDER SOME
SPECIALIZED CONDITIONS
6.1 Introduction
Recall that we are dealing with the general model
Y = Xa + Ue where E(ee')
= el F1 +
+ epv p as described in (2.2.2).
Y = Xs +
natural
U*~*
... + epF p and D(Y)
= e1v!
+ .•.
We transformed the above model as
and considered the difference between Y'AY and the
Then the MINQE of fIe is defined as
estimator£~Ne*.
Y'AY where A is such that
U'AX
~
(6.1.1)
X'AU
*
is a minimum.
In this section the condition of unbiasedness is withdrawn and
~
the following classes of symmetric matrices are considered:
C*
= {A}
Co
= {A: X'AX = O}
CI
= {A:
,
AX
(6.1.2)
,
= O}
(6.1.3)
(6.1.4)
The estimators Y'AY obtained by minimizing (6.1.1) subject to
the restrictions (6.1.2), (6.1.3), and (6.1.4) are denoted MINQE,
MINQE(O), and MINQE(I), respectively.
125
6.2 MINQE, MINQE(O), and MINQE(I)
Theorem (6.2.1):
non-singular, VN = UJIU~ where N is defined in
(2.2.5), and Pa = X(X ' V-a 1X)-X ' V-a 1. Then under the Euclidean norm
(6.1.1) we have the following.
Let Va
= U~~ be
(i) The MINQE of fie is
~
.
= (Va +XX , )-l
YI~Y
where
VN (Va +XX , )-l
(6.2.1)
(ii) The MINQE(O) of fie is Y'A*Y where
A+.
= (Va +XX , )-l(V NaN
-P V P')(V
+XX,)-l
a
a
.
(6.2.2)
(iii) The MINQE(I) of fie is Y'A*Y where
A*
= Ra VNRa
.
(6.2.3)
Proof of (i):
Under the Euclidean norm (2.3.4) the square of (6.1.1) is
(6.2.4)
Now considering A = A*+B the above equals
126
(6.2.5)
+
tr(X'A~X'A*X)
+ tr(X'BXX'BX) + 2 tr(X'A*XX'BX) . (6.2.6)
Without any restriction on A, the minimum of (6.2.4) is attained at
A* if and only if
(6.2.7)
for all symmetric matrices
B. Then A* satisfies the equation
U (U'~ U -N)U' + XX'A U U' + U UtA XX' + XX'A XX'
* -Jr* *
*
** *
***
*
=0
(6.2.8)
or
(V + XX')A (V + XX') = U NU' = V
*
CL
**
CL
N
(6.2.9)
or
A*
= (V
CL
+ XX' )-1 VN(V
j
CL
+ XX,)-l .
(6.2.10)
127
Proof of (ii):
Now if A is subject to the restriction X'AX
= 0, then the
minimum of (6.1.1) is attained at A* if and only if
(6.2.ti)
for all symmetric matrices of the form
B = E - P'a E Pa
Then
A~
satisfies the equation
(6.2. I2)
or A* sat; sfies
Va A*Va - VN - Pa Va A*V a pia + Pa VNP aI + XX'.A*V a
- P XXIA V pI + V A XXI - P V A XXIP'
a
*a a
a *
a a *
a
But since Pa Va = Va P'a = X(Xlv-1Xfx' and X'A*X
=a .
(6.2.13)
= 0, we obtain
(6.2.14)
or
128
·,
(6.2.15)
that is,
(Va + XX')A* (Va + XX') = VN - Pa VNP'a
(6.2.16)
V P'](V + XX,)-I.
* = (Va + XX,)-l[VN - PaNa
a
(6.2.17)
or
A
Now as a check one can easily show that
X'(V
a
+
XX,)-1 P V P'(V
=
X'(V
a N a
a
+
a
+
XX,)-I X
XX,)-1 V (V
N a
+
XX,)-1 X
(6.2.18)
since
and
X1(V + XX,)-l P
a
a
= X'(V a
+
XX,)-l .
Proof of (i i i) :
Now if A is subject to the restriction AX
= 0,
then the minimum
of (6.2.4) is attained at A* if and only if
(6.2.19 )
for all symmetric matrices B of the form
129
1.'
1.
B = PEP
a.
a.
where
and E is symmetric.
Then A* satisfies the equation
{6.2.20}
or
A*
= Ra. VN Ra.
(6.2.21)
6.3 Explicit MINQE without Unbiasedness for the
One-Way Classification
Recall the general form for the known model
y ..
,J
= u + a."
+ e ..
J
i
= 1, ... ,h;
j
= 1, ... ,n i ;
(6.3.1)
or
(6.3.2)
where Xis a vector of l's (n xl).
U is a (n x h) matri x of 0 sand
1
lis, U2 = In where n =i~ln;= S&, a (hx1) random vector, e (nx1)
random vector and a and e are independent.
h
I
130
The dispersion matrix
{6.3.3}
Now we can also rewrite {6.3.2} as
y
= Xu
{6.3.4}
+ U4>
where
4>'
= {a',e'}
,
The fonn of
(6.3.5)
and
a
(6.3.6)
n+h
and these matrices satisfy the expressions:
F.F.
= F.1
1 1
and
131
Let F = Cl l Fl + Cl2F2 where Cl1 and Cl2 are priclr values of 0a and 0e and
F is non-singular. (Note this rules out using Cl1 = 0 or Cl2 = 0.) Now
making the transformation
(6.3.7)
we wish to find scalars (u , u2) satisfying
1
E(,~ N 41*)
= f' e
(6.3.8)
= ~ tr(F}0
+
CIlIa
u2 tr(F )0
2 e
Cl2
= -FIe
or
uh
~
U 51
0
Cll a
+..1....Q.
Ct2
= fl e
(6.3.9)
Now in order to obtain an estimate of 0 a one can select
and
u
2
=0
(6.3.l0)
and similarly choose
u1
=0
and
to estimate 6 2. Thus VN =
U*NU~
woul d be cal cul ated as follows.
132
To estimate Sa one would calculate
2
V = a1 U~F ~U = a1 V
N
h
1
h
(6.3.11)
1
and accordingly choose
(6.3.12)
to estimate Se.
Thus depending on the value of f, VN will take on
the followi ng form:
. (6.3.13)
6.4 Explicit MINQE
Explicit equations for MINQE indicate solving Y'A*Y where
for
(6.4.1)
Now the above term in square brackets has typical elements
(6.4.2)
133
al
where € = a2 and p = a2 • So the elements of
[V- 1 _ V- 1X{X'V- 1X+l)-I X'V-11Y ..
a
a
a
a
1J .
(6.4.3)
=
-1
€
Pni l i n ; ,
h
y 1+
[Yi'+ - n p+l Y;I+ - - 1 (n p+1)
n p+l 1.
; I
(SI+e:) ;' . 1=1 1
r
(6.4.4)
So the scalar Y' [V-l_V-1X~X'V-1X+1)-IX'V-11V [V- 1_V- 1X(X'V- 1X+l)-1
a
a
a
a
1 a
a
a
1
.X'V- 1Y
a
= € -2
r [y. - nl y. - ;=1 1+ n;p+l 1+ (Si+e:)
h
p
•
1
(
n•
1)
ni P+1
r
1=1
h
Y1+
n1p+l
2
1
(6.4.5)
and the scalar y [V-l_V-1X(X'V-1X+l)-IX'V-l]V [V- 1_V- 1X{X'V- 1X+l)-1
a
a
a
a
2 a
a
a
1
·X ' V- ]y
a
l
(6.4.6)
Thus multiplying (6.4.5) by zl and adding it to (6.4.6) multiplied
by z2 as indicated in the calculation of VN described in (6.4.1)
will give us our system of equations to sol VI:!. It should be noticed
that if one is estimating 8 a and 0 e , these estimates will always be
134
positive whereas those estimators described previously may have
negative estimates.
Since MINQE is not an unbiased estimator, one may be interested
in studying the sensitivity of these estimators with respect to
different a priori values and true unknown components.
It is well known that
= tr(AV) + u'X'AXu
E(Y'AY}
(6.4.7)
for Y - (Xu, V) .' Thus in evaluating the expectation of the MINQE
we will need the following explicit forms:
(6.4.8)
(6.4.9)
, tr([V-1_V-lX(X'V-1X+1}-lX'V-1] V [V-1_V-1X(X'V-1X+l}-lx,v-1]V },
a
a
a
2
CL
a
a
a
2
a
(6.4.10)
and the scalar terms
u X'
U
[V-1_V-1X(XIV-1X+1)-lX'V-l]V [V- 1_V- 1X(X ' V- 1X+1)-lX'y-l]Xu
a
a
a
a
1a
a
a
a'
(6.4.11)
XI [V-l_V-lX(X'V-lX+l)-lX'V-l]y [V-l_y-1X(XIV-lX+l)-lX'V-l]Xu .
a
a
a
a
2a
a
a
a
(6.4.12)
135
Expressions for (6.4.8) - (6.4.12) can be obtained by using
expressions (6.4.5) and (6.4.6) and using Hartley's r1ethod of
Synthesis (1978).
The procedure suggests replacing the vector Y
in (6.4.5) - (6.4.6) with the columns of U1 or U2 and then adding the
results to obtain the desired explicit forms. Denote the k column
and i,jth element of U1 as U1kij •
Thus
U1k i +
= nko ki
and we obtain
h
= t.\
k=1
£2
h
\
L
i=1
p
[nko k,'
-
ni
n.p+l
,
(6.4.13)
(6.4.14)
Although a simplified expression for U2 'in terms of the elements
y .. and k cannot be obtained, examination of the patterns in the
1J
elements results in the following expression::
136
(6.4.15)
When adding the sum of squared elements of (6.4.2) we obtain
= e:
-2 1
22
1
2S~ 2pS~
(S1)2
[SO +p S2 - 2pS1 - - 1 + - 1 +
2
]
S1+e: S1+e: (S ll+e:)2·
,
(6.4.16)
Since X is a vector of lIs we can obtain (6.4.11) from (6.4.5)
as
(6.4.17)
and
137
(6.4.18)
Thus we can calculate (6.4.7) now given the unknown values 9a and 6e •
6.5 Explicit MINQE(OL
Explicit equations for MINQE(O) indicate solving YIA*Y where
A*
= (V
+XX)-I(V N-p VNP')(V +XX )-1
I
(l
(l
(l
(l
.
Now
(6.5.1)
for zl and z2 described in (6.3.13). Now X
l
v: 1
has typical
elements
e:
-l(
1
no,p+1)
,
(6.5.2)
So
= € -1
,
no
,
I
no P+1
l
(6.5.3)
138
and the scalar X,y-1Y1y-lX
a
-- e: -2 S22 '
=
e:
-2
1
S2
a
(6.5.4)
(6.5.5)
and
(6.5.6)
So
has elements
(6.5.7)
and
has elements
O•• ,0 •• , 11
JJ
(6.5.8)
139
The elements of
{6.5.9}
So the scalar
=€
-2
~
L
i =1
_
pn;
1";
(Y;+ - n.p+1 Y;+ - ~ (n.~+l)
S, +€
1
1
::r.S1+~
(
Now the elements of
n; )
n.p+1
1
h
L
~= 1
Y~+
2
n p+1] ]
~
1'"
~
L
~= 1
(6.5.10)
140
= € -1 [y i
I
j
I
(6.5.11)
50 the scalar
1
ni
-( n.p+1
) h-')
1
5 +£
1
1,=1
1
y'1.+
nQ. p+l J
J2.
So explicit forms of MINQE(O) of fie indicates solving
(6.5.12)
141
+
Z
2
€ -2
n·
\1 [
i;1 j;1 Yij
h
\
_
e
ni P+1 Yij
_
1
(
1
) h\
Y11+
~]
2
s1~ ni P+1 t~1 nt p+1
(6.5.13)
Note that estimates of 0a and 6e resulting from this system of
equations may be negative.
Now in order to evaluate the expectation of the MINQE{O), we
will need the following explicit forms in addition to equations
{6.4.14} - {6.4.18}.
Now
has typical elements
(6.5.14)
and hence the elements of
142
[V-1_V-1X(X'V-1X+1)-lX'V-1][X(X'V-1X)-lX'V-1V V- 1X(X'V- 1X)-l X']
a
a
a
a
a
a
1a
a
(6.5.15)
and elements
can be obtained as
,
(6.5.16)
Thus we obtain
(6.5.17)
143
and similarly
(6.5.18)
By examining typical elements we obtain
(6.5.19)
and
144
(6.5.20)
Now by construction the scalars
uX'(V +XX,}-1(V -p V P'}(V +XX,}-l Xu
a
a
1 a 1 a
=0
u X' (V +XX') -1(V -p V P· }(V +XX') -l Xu
a
a
2 a 2 a
=0
(6.5.21)
and
(6.5.22)
•
6.6 Expl i ci t mNQE(I}
Explicit equations for MINQE(I} indicate solving V'A*Y where
= RCL VNRCL
(606.I)
0
Now we previously found the scalar V'Ra VIR CL V
= e: - 2
h
I
i=l
[y .
1+
-
n. py. +
1 1
niP+l
-
1
$I
1.
(
n.
h
y n+
1) I
:<.
niP+l 9..=1 nR.,p+1
2
]
(606o2)
145
= e:
-1 h
\
ni
\
i~1 j~1
[y..
1
si
0
lJ -
ni~+l Yi+ -
1
h
Y11+ 2
\
,..
]
"i O+1 g,~1 ng,0+1
()
•
(6.6.3)
So the MINQE(I} estimates of fIe
(6.6.4)
for appropriately chosen zl and z2.
Like the MINQE, MINQE(I} of
Sa and Se will always be non-negative.
Using expressions (6.4.13) - (6.4.17) we obtain the following:
(6.6.5)
and
tr(Ra V2Ra V2 )
-2
= e:
2
1
1
2 2
1
2S
20 S
(Sl)2
[SO + P S2 - 2p $1) - --L + -1 + 2.]
1
1
(S1)2
SI
$1
1
(6.6.7)
and the scalars
uX'R a.VIR a Xu = 0 ,
uX'R V2R Xu
Ct
ct.
=0
(6.6.8)
.
(6.6.9)
146
7.
MINI~lliM
NORM QUADRATIC NON-NEGATIVE DEFINITE
UNBIASED ESTI11ATORS
7.1 Introduction
In estimating variance components in linear models, procedures
which use quadratic unbiased estimators may prescribe estimators
.
.
which take on negative values. Since we are considering the random
effects model where the variance components are no.n-negative it is
desirable to have non-negative estimators for them. The estimators
considered so far with the exception of MINQE and MINQE(I) can assume
negative values although the parametric function being estimated is
non-negative.
In this section we explore the possibility of obtain-
ing unbiased quadratic estimators
y = Y'AY
with A non-negative
defi ni te sYmmetri c denoted A ~ 0 of the parametri c functi ons f.' a
which are non-negative in a for a general model. A MINQE in this
class is denoted by MINQE(U,D), where 0 stands for non-negative
definiteness of the quadratic estimator.
7.2 MINQE(U,D)
The following lemma characterizes the nature of A if fls has
to be unbiased and non-negative.
Lemma (7.2. 1) :
A non-negative and unbiased quadratic estimator Y'AY satisfies
the invariance condition, i ..!.,
AX
=0
•
(7.2.1)
147
Proof:
From unbiasedness we obtain XIAX
= 0 and
since A > 0 we obtain
XI AX = XI HH IX = 0 from which we obtai n HI X :: 0 and hence AX =
o.
Thus in view of Let1ITIa (7.2.1) we need only consider the class of
matrices
f
CUD
= {A: A symmetric,A
~
0, AX = 0, tr(AV i )= f i , i = 1, ... ,p} •
(7.2.2)
We wi],l use the following results by La Motte (l973a) with slight
modifications in the proofs.
LelTl1la (7.2.2):
= 0 and
In order that XIAX
A be non-negative definite, it is
necessary and sufficient that there exist a matrix B such that
A = R() BBIR() .
(7.2.3)
Proof:
Suppose A = R BBIR , then A is n.n.d. and XIAX
()
()
= O.
Now if A
is n.n.d., then there exists a matrix C such that A = CC I . Now
I
I
consider B = V C, R BB'R = p.L CC'p.L = p.L AP,i = A since AX = 0 by
()
a
a
a
a
a
a
Lemma (7.2.1).
Lel1111a (7.2.3):
In order for there to be a non-negative quadratic Y'AY unbiased
for fie it is necessary and sufficient that there exists a matrix B
satisfying
tr(B'R a V.~
, a B)
= fl.
i
= l, ... ,p.
(7.2.4)
148
Proof:
If there exists a matrix B satisfying (7.2.4), let
A = RaBB'R.
B) = f.1 and X'AX = 0 and
a Then tr(AV l.) = tr(B'Ra V.R
1 a
A is n.n.d. Now if Y'AY is non-negative and unbiased for fie, then
A satisfies X'AX
= 0 and
tr(AV i )
= fie
By Lemma (7.1.2) there
exists a matrix B such that A = Ra BB'Ra and the tr(BIR a Vl·R a B)
= fl'.
An important consequence of Lemma (7.2.3) is the following.
Lenma (7.2.4):
= 1,
.•. , p, Vi is positive definite and f i = 0,
then the only vector f for which there is a non-negative quadratic
If for some i
unbiased estimator of fie is f
= o.
Proof:
If V.1 is positive definite, f.1 = 0, then writing BIRa V.R
B = HH '
1 a
we have tr(HH I ) = 0; and writing the columns of H denoted hj , we have
h
L
tr(h.h~)
j=1
J J
h
=o = L
so H = 0 and hence RaB
j=1
=0
hJ~hJ'
since Vi is non-singular.
An immediate consequence of Lemma (7.2.4) is that in ANOVA
models, the only individual component which can be estimated unbiasedly by a non-negative quadratic ;S se' and even
estimable only if all Vi' i
Let us assume that
C00
= 1,
;s so
... , p-l, are singular.
defined in (7.2.2) ;s non-empty for a
given f and estimate fIe by MINQE principle.
have to minimize
~e
For this purpose we
149
II U~U*-N 11
2 or tr(AV AV )
a a
f
when . A£ CUD
(7.2.5)
.
This appears to be a difficult problem in the general case. Of
course, if MINQE(U,I) turns out to be a non-negative estimator in any
situation, it is automatically MINQE(U,O).
Now since C~O is a convex set, we may apply Theorem (2.4.1),
part (i), to solve the problem (7.2.5).
The minimum is attained
at A* iff
for every B
€
f
CUD ;
(7.2.6)
or writing B = A* + 0, the condition (7.2.6) becomes
'r:j
0
H = {O: 0 symnetric, OX
tr(OV i )
= 0,
H
€
= 0, A*
+ 0 ~ 0,
i = 1, ... ,p } .
(7.2.7)
Theorem (7.2.1):
Let us assume our model has an ANOVA structure described
= 1,
the Vi are singular for i = 1,
earlier.
8
Let Vi
~
0, i
, p. Vp is an identity matrix and
, p-l.
Then the MINQE{U,O) of
e is
ee
=
VIA V
*
R
(7.2.8)
where A* = I - G, R = Rank (A*) and G is the projection operator on
to the space generated by the columns of (X, VI' ... , Vp_1) .
150
Proof:
Let us investigate H defined in (7.2.7).
Since GV i
= Vi'
i f p,
we have for 0 e: H
(7.2.9)
i f P •
Multiplying and the
ri~ht
and left of A* + 0
~
0 by G, we have
Now tr{GOGV i ) = 0, i f p, and Vi ~ 0, so
tr{GOGV i ) = tr{K'GOGK) = tr{HH ' ) = 0 so H = 0, and hence
GOG
~
O.
GOGV i
= GOV i = 0
Now since OX
GOG
=0
,
(7.2.10)
i f P •
,
we have
=0
(7.2.11)
by examining the elements of G and the fact that GOV i
Now the tr{DV a A*V a )
=a
Thus
'lIe
si nee Vp
=I
i f p.
(7.2.12)
.
have the mNQE(U, D) for the estimate of
= 0,
6 •
e
151
7.3 Explicit MINQE(U,D} of 6 e for the
One-Way Classification
Remembering the general form of the known model
i
Y··=lJ+a.+e:.·
lJ
1
lJ
= 1, .. .,h;
j
= 1, .. .,n i ;
(7.3.1)
or
(7.3.2)
described previously, the estimator of e defined in (7.2.8) lends
e
one to solve the equations
yl (I-G) y
R
(7.3.3)
where G is the projection operator on to the space generated by the
columns of (X : VI)'
Lemma (7. 3. 1) :
The projection operator G = Pv where Pv is the projection
I
I
operator on to the space generated by the columns of (VI)'
Proof:
VI has typical elements
0 .. ,.
"
Define the vector t = 1. 1 •
J=
= 1 and we can write VI t = X; thus GP = P and
VI
V1
P G = G = G' = GP" giving us the result Pv = G.
Then VI t
VI
II
I
Now a (VI 'V 1 )- has typical elements
(7.3.4)
and the corresponding elements of
152
c..
11
(7.3.5)
t
So the scalar
2
Yi+
(7.3.6)
n·1
Now the
h
Rank(I-P V )
1
= trace(I-P V ) = L n.
1
i=1
(7.3.7)
- h
1
So our MINQE(U D) estimate of
9
h
ee
h
n'1
= (\
Y~'
.~. r
1J
1=1 J=1
-
.L
1=1
y~
2) / (
ni
h
L n.
i=1'
- h) •
(7.3.8)
But this estimator is exactly the ANOVA estimator defined in Searle
(1971a) for unbalanced data and the Henderson Method III estimator
for the one-way analysis of variance.
153
8.
SENSITIVITY OF THE VARIANCE AND MEAN SQUARE ERROR
FOR MINQE( ) USING DIFFERENT A PRIORI
AND UNKNOWN VARIANCE COMPONENTS
8.1 Introduction
.. ,
i.~
Under the assumption of nonnality,
Y - Np(lJ, V) where
V = eaV 1 + eeVl' we will examine the sensitivity of the variance and
mean square error for the minimum nonn quadratic estimators described
. previously.
Using the data considered in a paper by Swallow (1978)
we will see the effects of Q a
classification.
= ea and
Q
e
= 6e
for the one-way
8.2 MINQE(U)
The variance of the
ing.
~HNQE(U)
are obtained by solving the follow-
Let the matrix B have typical elements
(8.2.1)
for i,j = 1,2 and define C = B- 1. For A.1
= (V +XX,)-I(V.-p
V.P')
1
1
Q
Q
Q
·(VQ +XX,)-l, the normality condition allows one to calculate the
variance of the estimate of 6 a as
(8.2.2)
where
V(Y*'A.Y*)
1
= 2 tr(A.VA.V)
+ 4 s'A.VA.s
,
1 1
1 1
Cov(Y*'A.Y*, Y*'A.Y*)
1
J
= 2 tr(A,.VA.V)
+ 4s ' A.VA.s
J
1
J
and the Cij's are elements of the matrix C.
154
Similarly, one obtains the variance of the estimate of 6e by
changing the first index for the elements of the matrix C. One
should note that the variance and covariance terms contain the
vector
~
rather than
~
for 60 , a predetermined
Ideally one would like this term to be
where s =
estimate of the grand mean.
~
- XB
O
o but we will examine some of the effects of this term deviating
from O.
Because of the large number of different matrices needed for
the calculation of the variance and covariance terms, explicit
equations for MINQE(U) were not derived.
The MINQE, MINQE(O), and MINQE(I) are biased so in addition to
calculating variances we will investigate the sensitivity of the
mean square error of our estimates.
8.3 Sensitivity of the Estimators
We will briefly describe the data analyzed in a paper by
Swallow and Searle (1978).
Five groups of several consecutive bottles each were snatched
from a moving production line which was filling the bottles with
vegetable oil.
The oil in each bottle was weighed and the resulting
data appear in Table 8.3.1. A multiple (24) head machine was being
used in the filling and different (unidentified) heads are represented in the five groups of bottles sampled; so variability among
groups reflects, in part, variability among heads.
In an application of this sort, a priori estimates of the
variance components can often be obtained from data collected
previously on the same or a similar production line.
In this case,
155
Table 8.3.1 Net weights (oz) of vegetable oil fills
1
2
3
4
5
15.70
15.69
15.75
15.68
15.65
15.68
15.71
15.82
15.66
15.60
15.64
15.75
15.59
15.60
15.71
15.84
156
the y
expl ai n that other data on the same 1 i ne were avail ab le and
the ANOVA estimates (6 a = 0.0028 and 6e = 0.0025) of the variance
components calculated from these data can be used as the a priori
estimates to obtain the MINQE{ ).
Since our MINQE{U) needs an a priori estimate for the overall
mean we went back to 't h e i r earl i er data and were not abl e to
reproduce his estimates of (Sa
= 0.0028
this original data we calculated ANOVA
6e
= 0.00773381,
and
~
= 0.0025). Using
estimates as (6 a = 0.0010365,
and Se
= 15.67).
The estimates of 8a and Se using the five different types of
estimators discussed are displayed in Table8.3.2. The ANOVA, MINQE(U,I),
and MINQE(U) which are unbiased are very similar to three decimal
places. The three biased estimators considered are also very
similar to four decimal places. Table 8.3.3 examines the variances
and the sensitivity of the mean square error for the estimators
considered when the prior values are in fact the unknown parameters.Of those three unbiased estimators considered, we observe that the
r"1INQE(U,I) and MINQE(U) have uniformly smaller variances than the
ANOVA estimators for both parameters being estimated and a priori
values.
This table also indicates when using those a priori values
derived by the author the MINQE(U) variances are slightly smaller
than the variances of the rlINQE(U,I) for both estimators. The effect
of the constant u being different had no appreciable effect on the
variance of the MINQE(U) estimates unless /ui-XiS O' > 2.5.
Examining the bias terms in Table 8.3.30ne notices a significant
drop in the mean square error for the two parameters.
The biased
157
Table 8.3.2 Estimates of sa and 8 e using a priori values based on
ANOVA calculations of 6a =' .0010365, ae = .00773381, and
U= 15.67
ANOVA
0.00377764
0.0021397
MINQE(U,I)
0.00433971
0.00159406
MINQE(U)
0.00434201
0.00159264
MINQE
0.000364227
0.00293938
r4INQE(0)
0.000352967
0.00288129
MINQE(I )
0.000334888
0.00292345
e
e
-
e
Table 8.3.3 Sensitivity of the variance and mean square error when a priori values are exactly the true
unknO\'In parameters for (Oa ;:: 0.0010365, o~ ;:: 0.0077381) / (Oa = 0.0028, 0e = 0.0025)
159
estimators do, however, show many similarities. The variances of
the MINQE and MINQE(O) estimators are similar
but both estimators
underestimate the true value of the parameter by. differing amounts
and thus have different mean square error terms. The MINQE(O} and
MINQE(I) have the same bias terms but different values for the
variances.
160
9. SUMMARY OF THE RESULTS
The emphasis in this study has been on determining efficient
expressions for calculating Minimum Norm Quadratic Estimators for
two different random effects models.
For the two-stage nested model,
three different procedures for estimating the variance components
were given and then compared using numerical methods.
Similarly for
the one-way random model six different estimators are compared.
Since simple expressions for the variance components and their
variances cannot be obtained, it was not possible to analytically
compare the three estimation procedures to determine under what
circumstances anyone of them can be considered as best,
i.~.,
which procedure gives estimates with the smallest variance.
For the
two-stage nested design a number of unbalanced designs were considered
and for each design the variances of the variance components for
certain values of the variance components and pre-determined prior
values were calculated.
From this part of the study the following conclusions were
obtained:
(1) When a priori values for the variance components are both less
than one, a balance design is suggested as being optimal for all
three estimation procedures.
(2) Prior values can be selected whereupon implementation in the
MINQE(U,I) equations guarantee smaller variances than for
ANOVA estimates for at least 78 percent of the grid areas
considered.
161
(3) Selecting large prior values (i.e., p(A)
= p(B) = 8.0)
gives smaller variances for MINQE(U,I} estimates than for
ANOVA estimates over a majority of the grid for all designs
considered.
(4) Those prior values recommended when using the combined
MINQE(U,I)-Henderson Method III estimator are of the order
p(A}
= 2 and
p(B)
= 1 as
compared with p(A)
= p(B) = 8.0
for the
MINQE(U,I).
(5) If values for priors are scarce and someone recommends using
p(A)
= p(S) = 1,
the combined estimator MINQH appears to have
some optimal properties over MINQE(U,I).
(6) Zero priors should. not be used unless one is using an IteratedMINQE(U,I) and has needs of initial estimates.
Simple explicit equations are given for a variety of MINQE for
the one-way analysis of variance.
Examination of the variance of
these estimates using a classical unbalanced design and specialized
conditions reveals the following results:
(1) Relaxing the unbiased condition appears to reduce the Mean Square
Error of the estimates.
(2) MINQE(U} seems to reduce the variance of the estimates for the
within and between random model.
162
10. LIST OF REFERENCES
Ahrens, H. 1978. MINQUE and ANOVA estimator for one-way classification--a risk comparison. Biometrical Journal 20(6):535-556.
Anderson, R. D. 1979. On the history of variance component estimation. Proceedings, Variance Components and Animal Breeding,
Cornell University, Ithaca, New York.
Anderson, R. D., Quaas, R. L., and Searle, S. R. 1977. Fourth
moments in the general linear model; and the variance of
translation invariant quadratic forms. Paper'BU-630-M in the
Biometrics Unit, Cornell University, Ithaca, New York.
Anderson, R. L. 1960. Uses of variance components analysis in the
interpretation of biological experiments. Bull. Int. Statist.
Inst. 37(3):71-90.
Anderson, R. L. 1966. Non-balanced designs to estimate variance
components. Proceedings of the NSF-Japanese Society Seminar on
Sampling of Bulk Materials, 5.117-5.149.
Anderson, R. L., and Bancroft, T. A. 1952. Statistical Theory in
Research. McGraw-Hill Book Co., Inc., New York.
Anderson, R. L., and Crump, P. P. 1967. Comparison of designs and
estimation procedures for estimating parameters in a two-stage
nested process. Technometrics 9:499-516.
Bainbridge, T. R. 1965. Staggered nested designs for estimating
variance components. Industr. Qual. Contr. 22:12-20.
Brown, K. G. 1976. Asymptotic behavior of Minque-type estimators of
variance components. Ann. Statist. 4:746-754.
Brown, K. G. 1978. Estimation of variance components using
residuals. J. Amer. Statist. Assoc. 73:141-146.
Calvin, L. D., and Miller, J. D. 1961. A sampling design with
incomplete dichotomy. J. Agron. 53:325-328.
Corbeil, R. R., and Searle, S. R. 1976. Restricted maximum likelihood (REML) estimation of variance components in the mixed
model. Technometrics 18:31-38.
Crump, P. P. 1954. Optimal designs to estimate the parameters of a
variance component model. Unpublished Ph.D. thesis, North
Carolina State University, Raleigh, North Carolina.
Focke, J., and Dewess, G. 1972. Uber die Schatzmethode MINQUE von
c. R. Rao und ihre verallgemeinerung. Math. Operationsforsch.
Statist. 3:129-143.
163
Giesbrecht, F. G., and Burrows, P. M. 1978. Estimating variance
components in hierarchical structures using Minque and Restricted
Maximum Likelihood. Commun. Statist. Theor. Method. A7(9):891-904.
Goldsmith, C. H., and Gaylor, D. W. 1970. Three stage nested designs
for estimating variance components. Technometrics 12:487-498.
Goodnight, J. H. 1979. New features in GLM and VARCOMP. Proceedings,
Fourth Annual SAS Users· Group International Conference, SAS
Institute, Raleigh, North Carolina.
Hartley, H. 0., and Rao, J. N. K. 1967. Maximum likelihood estimation for the mixed analysis of variance model. Biometrika 54:
93-108.
Hartley, H. 0., Rao, J. N. K., and La Motte, L. R. 1978. A simple
"synthesis"-based method of variance component estimation.
Biometrics 34:233-242.
Hartley, H. 0., and Vaughn, W. K. 1972. A ,computer program for the
mixed analysis of variance model based on maximum likelihood,
pp. 129-144. In Statistical Papers in Honor of George W.
Snedecor.·. Iowa State University Press, Ames, Iowa.
Harville, D. A. 1969. Quadratic unbiased estimation of variance
components for the one-way classificati()n. Biometrika 56:313326.
Harville, D. A. 1975. Maximum likelihood approaches to variance
component estimation and to related problems. Aerospace
Research Laboratories, ARL-75-0175.
Harville, D. A. 1977. Maximum likelihood approaches to variance
component estimation. Biometrics 34:233-242.
Hemmerle, W. J., and Hartley, H. O. 1973. Computing maximum likelihood estimates for the mixed A.O.V. model using the Wtransformation. Technometrics 15:819-832.
Henderson, C. R.
components.
1953. Estimation of variance and covariance
Biometrics 9:226-252.
Hess, J. L. 1979. Sensitivity of MINQUE with respect to a priori
weights. Biometrics 35:645-649.
Jennrich, R. I., and Sampson, P. F. 1976. Newton-Raphson and related
algorithms for maximum likelihood estimation of variance
components. Technometrics 18:11-18.
Kleffe, J. 1980. C. R. Rao's Minque under four two-way Anova
models. Biometrical Journal 22(2):93-104.
164
La Motte, L. R. 1970. A class of estimators of variance components.
Technical Report No. 10, Dept. of Statistics, University of
Kentucky, Lexington, Kentucky.
La Motte, L. R. 1971. Locally best quadratic estimators of variance
components. Technical Report No. 22, Dept. of Statistics,
.
University of Kentucky, Lexington, Kentucky.
La Motte, L. R. 1973a. On non-negative quadratic unbiased estimation
of variance components. J. Amer. Statist. Assoc. 68:728-730.
La Motte, L. R. 1973b. Quadratic estimation of variance components.
Biometrics 29:311-330.
Liu, L. r1., and Senturia, J. 1977. Computation of MINQUE variance
component estimates. J. Amer. Statist. Assoc. 72:867-868.
Miller, J. J. 1973. Asymptotic properties and computation of
maximum likelihood estimates in the mixed model of the analysis
of variance. Technical Report 12, Dept. of Statistics, Stanford
University, Stanford, California.
Patterson, W. D., and Thompson, R. 1971. Recovery of interblock
information when block sizes are unequal. Biometrika 58:545554.
Prairie, R. R. 1962. Optimal designs to estimate variance components
and to reduce product variability for nested classifications.
Unpublished Ph.D. thesis, Department of Experimental Statistics,
North Carolina State University, Raleigh. Institute of
Statistics, Mimeo Series No. 313.
Pringle, R. M. 1974. Some results on the estimation of variance
components by Minque. J. Amer. Statist. Assoc. 69:987-989.
Pukelsheim, F. 1974. [ZitiertaufS. 5-7, 18]. Schatenvon
Mittelwert und Streuungsmatrix in Gauss-Markoff-Modellen.
Diplomarbeit, Inst. f~r Math. Stochastik der Univ. Freiburg.
Pukelsheim, F. 1976. Estimating variance components in linear
models. J. Multivar. Anal. 6:626-629.
Pukelsheim, F. 1977. On Hsu's model in regression analysis.
Operationsforsch. Statist. 8:323-331.
~1ath.
Rao, C. R. 1970. Estimation of heteroscedastic variances in linear
models. J. Amer. Statist. Assoc. 65:161-172.
Rao, C. R. 1971a. Estimation of variance and covariance components-MINQUE theory. J. Multivar. Anal. 1:257-275.
165
Rao, C. R. 1971b. Minimum variance quadratic unbiased estimation of
variance components. J. Multivar. Anal. 1:445-456.
Rao, C. R. 1972. Estimation of variance and covariance components
in linear models. J. Amer. Statist. Assoc. 67:112-115.
Rao, C. R. 1973. Linear Statistical Inference and Its Applications,
2nd ed. John Wiley, New York, New York.
Rao, C. R. 1979. Minqe theory and its relation to ML and MML
estimation of variance components. Sankhya 41:138-153.
Rao, C. R., and Kleffe, J. 1979. Estimation of variance components.
Technical Report No. 79-1, Dept. of Statistics, University of
Pittsburgh, Pittsburgh, Pennsylvania.
Rao, J. N. K. 1973. On the estimation of heteroscedastic variances.
Biometrics 29:11-24.
Rao, P. S. R. S., and Chaubey, Y. P. 1978. Three modifications of
the principle of the MINQUE. Commun. Statist. Theor. Meth.
7(8):767-778.
Rao, P. S. R. S., Kaplan, J., and Cochran, W. G. 1981. Estimators
for the one-way random effects model with unequal error
variances. J. Amer. Statist. Assoc. 76:89-97.
Searle, S. R. 1961. Variance components in the unbalanced 2-way
nested classification. Ann. ~~th. Statist. 32:1161-1166.
Searle, S. R. 1968. Another look at Henderson's methods of estimating variance components. Biometrics 24:749-778.
Searle, S. R.
1971a. Linear Models.
John
'~iley,
New York, New York.
Searle, S. R. 1971b. Topics in variance component estimation.
Biometrics 27:1-76.
Searle, S. R. 1979. Notes on variance component estimation: A
detailed account of maximum likelihood and kindred methodology.
Paper BU-673-M in the Biometrics Unit, Cornell University,
Ithaca, New York.
Seely, J. 1970a. Linear spaces and unbiased estimation.
Statist. 41:1725-1734.
Ann. Math.
Seely, J. 1970b. Linear spaces and unbiased estimation--application
to the mixed linear model. Ann. Math. Statist. 41:1735-1748.
Seely, J. 1971. Quadratic subspaces and completeness.
Statist. 42:710-721.
Ann. Math.
166
Swallow, W. H., and Searle, S. R. 1978. Minimum variance quadratic
unbiased estimation (MIVQUE) of variance components. Technometrics 20:265-272.
Szatrowski, T. H. 1980. Necessary and sufficient conditions for
explicit solutions in the multivariate normal estimation problem
for patterned means and covariances. Ann. Statist. 8(4}:802-810.
Thitakomal, B. 1977. Extension of previous results on properties of
estimators of variance components. Unpublished Ph.D. dissertation, University of Kentucky, Lexington, Kentucky.
Thompson, W. A. 1962. The problem of negative estimates of variance
components. Ann. Math. Statist. 33:273-289.
Thompson, W.O., and Anderson, R. L. 1975. A comparison of designs
and estimators for the two-stage nested design. Technometrics·
17:37-44.
Thrum, R., and Schmidt, W. H. 1979. Contributions to asymptotic
theory in regression models with linear covariance structure.
Preprint 37/39 Humboldt-Universitat ZW Berlin, DDR.
Townsend, E. C. 1968. Unbiased estimators of variance components
in simple unbalanced designs. Ph.D. dissertation, Cornell
University, Ithaca, New York.
Townsend, E. C., and Searle, S. R. 1971. Best quadratic unbiased
estimation of variance components from unbalanced data in the
one-way classification. Biometrics 29:643-657.
167
11.
APPENDIX
Calculate
N
= L,L.n;j
1
k1
J
= L. n~1· IN
k3
.1
= LL
. ' . n~lJ·/N
·.1
TAB
k12
J
= 1. . (L • n~./n.
)
1J 1 •
1
J
= L.1 L.J Y~J.• /n;J''
and
T
u
= y 2• • •IN .'
then
(11.1)
( 11.2)
( 11.3)
11.2 Variance of Analysis of Variance Estimators
(under Normality)
= 28 e2 /
VE(N)
( 11.4)
(N-m )
.
Calculate
k
4
kg
= I (OJ
)' n~.
lJ
k
= -,
I n.1, (I . n~.)
lJ
1
J
S
= I . (I . n~.
/
1J
k
9
n. )
1,
1
J
k
7
= I . (j-. n~.)2/n~
1J
1•
1
= I n~
i"
J
168
Then
( 11.5)