This research was partially supported by the National Institutes
of Health, Institut.eofGeneralMedicalSciences, Grant No. GM-12868-06.
AN EXTENSION OF KOCH'S METHOD OF ESTIMATING
COMPONENTS OF VARIANCE
y.'
by
Ronald Norman Forthofer
Department of Biostatistics
University of North Carolina
Institute of Statistics Mimeo Series No. 699
July
1970
RONALD NORMAN FORTHOFER.An Extension of Koch's Method of Estimating
Components of Variance.
(Under the direction of GARY G. KOCH.)
Koch's squared difference approach of estimating variance components is modified and extended in this study.
1)
The extensions are:
the proposed technique does not require that the random
variables in the model come from infinite populations,
2)
the estimation technique does not require that the random
variables come from any specific distribution,
3)
the variance-covariance matrix of the modified Koch estimators is presented (assuming a normal distribution of the
random variables), and
4)
the proposed technique can be used even if a mixed model is
being considered.
The modified Koch approach is also compared with the ANOVA estimation method and the maximum likelihood procedure using the variance
of an estimator as the criterion of comparison.
The results show that
the modified Koch procedure is very competitive with the ANOVA procedure in many situations.
ACKNOWLEDGMENTS
The author expresses appreciation to his adviser, Dr. Gary Koch,
and to the members of his advisory committee, Drs. R. Alvarez, D.
Gaylor, R. Helms, and H. Smith, Jr.
Special thanks go to Dr. Gary
Koch for suggesting the problem and for his guidance during the dissertation and also to Dr. Ronald Helms for stressing the importance of a
general formulation for the estimation procedure.
The author also
appreciates the encouragement given to him during his graduate studies
by Dr. James Grizzle.
The financial support of the University of North Carolina Computation Center is gratefully acknowledged.
Finally, the author wishes to thank his wife, lfury, for typing
the dissertation and for her patience during the years of the degree •
•
TABLE OF CONTENTS
Page
LIST OF FIGURES
. . . . . .. . . . . . . . . . . . . . . . . .
~
v
Chapter
L
INTRODUCTION AND REVIEW OF LITERATURE . •
1.1
1.2
1.3
1.4
1.5
1.6
II.
. . . . .
Es t imation .
. • • • •
Variance of Variance Component Estimators • •
Designs . . . . . . . . . . . . . . . . .
Comparison of Procedures and Designs . •
1
1
2
6
9
11
15
2.1
15
16
16
24
31
2.3
2.4
2.5
IV.
. • • .
....
PRESENTATION OF METHOD • .
2.2
III.
Introduction.
Models. . .
1
Introduction . . •
Definition of the Model •
Estimation Procedure • • • .
Selection of Terms.
Variance-covariance Matrix of the Estimates .
EQUIVALENCE OF PROCEDURES • .
39
3.1
3.2
3.3
39
39
Introduction. . • • • • • • .
Equivalence with Koch's Method . .
Equivalence to ANOVA Estimates in the Comp Ie te
Balanced Case
44
VARIANCE FORMULAE. .
55
4.1
4.2
55
55
55
56
65
69
69
4.3
4.4
Introduction •.
'Koch' type Estimators . .
4.2.1 Introduction.
• . . •.
. •••
4.2.2 Nested Designs •
. ........•
4.2.3 Classification Designs
..•.••
ANOVA Estimators • . . . . • •
4.3.1 Introduction • • • . •
4.3.2 Random-Effects Model •
4.3.3 Mixed Model . . . . . •
Asymptotic Variance-covariance ~trix of
Maximum Likelihood Estimates of Variance
Coroponen ts..
. . . . . .
70
80
82
iv
Page
Chapter
V.
5.1
5.2
5.3
5.4
VI.
VI I •
83
COMPARISON OF ESTIMATION PROCEDURES ••
Criterion • • • • • • • •
Procedures to be Compared
.-
.
..
Designs . . . • . . . . .
5.3.1 Random-Effects Model • • • .
5.3.2 1-1ixed Model. •• • • •
Results • • • • • • • • • •
5.4.1 Random-Effects Nodel .
5.4.2 Mi~ed Model ••
83
84
85
86
92
95
96
109
TWO-STAGE NESTED DESIGN. •
112
6.1
6.2
6.3
6.4
112
112
117
118
Introduction • • . •
New Estimators • • • . • • • • '2'
Variances of the Estimates of a .
l
Comparison of Estimators . •
127
SUMMARY.AND FUTURE WORK. .
7.1
7.2
Summary • .
. • • • .
Suggestions for Future Work.
....
127
129
Appendices
A.
DERIVATIVES OF MOMENT GENERATING FUNCTIONS •
131
B.
TABLES OF VARI.ANCES. •
135
LIST OF REFERENCES • • • • • • •
142
LIST OF FIGURES
Figure
2.1.
Page
Terms (and their expected values) from Condition 1. . . .
29
CHAPTER I
INTRODUCTION AND REVIEW OF LITERATURE
1.1.
Introduction
The estimation of the components of variance in the random-effects
and mixed models has received considerable attention in the 1iterature.
However, the fact that a number of articles recently have
appeared presenting new methods of estimation indicates that the
problem of estimating variance components has not yet been solved
satisfactorily.
The purpose of this study is twofold:
1) to modify
and extend Koch's [21] estimation procedure and 2) to compare several
estimation procedures (ANOVA, Koch, and maximum likelihood) by examining the properties of their covariance matrices.
1.2.
Models
Analysis of variance has been used to solve two types of problems,
those with fixed effects and those with random effects.
Churchill
Eisenhart [9] drew attention to these two types of problems and pointed
out that the difference between them is due to the assumptions about
the underlying model.
This difference in the models is demonstrated
very clearly by Scheffe [28].
For example,consider the model
+ xpJ.. f3 p + e.J.
i
= 1,
2, ... , n
(1.2.1)
2
where ~ and the {x.o} are known constants, the {So} are unknown quanJ1
J
tities, and the {eo} are uncorrelated random variables with expected
1
2
value zero and variance 0 • If the {So} are all unknown constants, the
e
J
model is called a "fixed-effects model" or Eisenhart's Model I. If
the {S.} are all random variables, then the model is a "random-effects
J
model" or Eisenhart's Model II.
Eisenhart, however, noted that in
practice most models do not fit precisely into either category, but
rather, are actually a combination of the two.
Consequently, the
term "mixed model" is used to describe a model in which at least one
of the {So} is a constant and at least one of the {So} is a random
J
J
variable.
1.3.
Estimation
S. Lee Crump [7], in 1946, summarized the estimation of variance
components.
He gave an example of the estimation procedure in the
balanced case and noted that work had been done in the unbalanced oneway classification.
He pointed out that the method of fitting con-
stants, or the method of weighted squares of means, both due to
Yates [38], or the method of expected subclass numbers due to
Snedecor [34] could be used in the estimation of variance components
in the unbalanced case.
In 1953, Henderson [17] presented three methods which could be
used to find the estimates of variance components in the non-orthogonal
case.
The three methods are:
Method 1.
Use the sums of squares from the usual analysis of orthogonal data and equate these sums of squares to their expectations which were obtained under the assumption of a random
3
model.
Method 2.
Adjust the data for the fixed effects by obtaining the
least squares estimates for the fixed effects, and then,
using the adjusted data, apply Method 1.
Method 3.
Use some conventional method to compute mean squares for
the non-orthogonal data, and then, equate these mean squares
to their expectations and solve for the estimates.
Henderson pointed out that Method 1 leads to biased estimates if used
with a mixed model or if certain elements of the model are correlated.
Method 2 adjusts for the mixed model, but the estimates are still
biase-d-iftne-correlaEion. of-tne -eIements (in terms of interaction
between the fixed and random effects) still exists.
Method 3 will
give unbiased estimates, but this method may require a great deal of
computation.
Searle [29,30,31] extended Henderson's methods to more complex
designs and introduced the use of matrix methods to Henderson's procedures.
Another contribution by Searle was finding the variance of
the variance component estimates and this is discussed in section 1.4.
In 1968, Searle [32] reviewed the three methods described above and
he added a fourth method which is similar to Henderson's Method 2.
Searle moreover pointed out that Method 2 does not lead to a unique
set of estimates, but that the estimates depend on the choice of a
generalized inverse.
Another method of estimation was presented by Hooke [18] in 1956.
This method is based on Tukey's polykays (generalized or multiple
subscripted 'k-statistics') and bipolykays (an extension of polykays
from sets to matrices developed by Hooke).
Hooke expresses the
4
expected mean squares as linear combinations of bipolykays.
Since
the average value of the bipolykays is a function of the variance
components, Hooke solved for the variance components and found their
estimates by substituting the observed bipolykays for the average
value of the bipolykays.
Note that under this formulation the random
variables in the model are now defined as coming from a population
determined by polykay parameters k , k , etc. instead of coming from
ll
l
a population with mean
~
an d
.
var~ance
2
cr •
In 1967, Hartley and Rao [15] developed maximum likelihood procedures for the estimation of the variance components and the constants
in the mixed model.
In addition to presenting the likelihood equations
and a method of solution, the authors discussed the asymptotic properties of the estimates, i.e. they showed their estimates to be cons istent and asymptotically efficient.
Since the distribution theory of
maximum likelihood estimates is predominantly based on asymptotic
results, the authors also included a section on small sample confidence
regions for the parameters in the model.'
Also in 1967, Koch [19,2lJ developed a general procedure for the
estimation of variance components for random-effects models.
As Bush
and Anderson [5] noted, perhaps the connection between estimating procedures for Model I and Model II parameters should be broken, and Koch's
method does break this link.
The method does not require the analysis
of variance table as most of the other procedures do.
Unlike the max-
imum likelihood procedure which requires the use of a computer, these
estimates" are reasonably easy' to calculate on 'a desk calculator , and
tIiey, are a1so',lUEiased and consis-tent.
5
The method takes advantage of the fact that in a model the
expected value of the squared difference of two observations is a
function of the variance components.
For example, consider the
following model:
+ a.~ + e ~J
..
Y •• = ~o
~J
i
= 1,
2,
j
= 1,
2, ... , n
(1.3.1)
a
i
where' {a.} are independent and normally distributed with mean zero
~
and variance 0
2
(NID(O,02», {e .. } are NID(O;02), {a.} and {e~J'} are
a
a
independent, and
~O
e
~J
is a constant.
~
~
Hence, one has that
One can form the "normalized symmetric sum" of all the unbiased
estimators listed above, i.e. take the average of all possible
2
squared differences whose expectation is 2 0 :
e
a
h
e
= [
L
n. (n. - 1)]~
i=l
a
=2 [
I
~
2
n. - n ]
i=l
1
~
-1
a
L
i=l
a
L
i=l
n.
2
y~ (Y .. - Yin)
j~Q,
~J
N
n. 2
-2
n. [ I~ Y.. - n. Y.
~
j=l
~J
~~.
(1.3.3)
and the average of all the possible squared differences whose expectation is 2
(0
2
a
+ 0 e2 ):
6
h
a
= [
a
Ln.
i=l ~
=2
[ n
2
(n - n.) ]
-1
~=
~
]-
n.
~
·i~k j=l 1(,=1
~
I. a 1n.2
a
}' . I~ L (Y .•
1
[ Ia
i=l
- Yko )
)(
~J
(n - n.)
~
2
In i Y..
j=l ~J
2
] -
(Y
a
-i~l
~ Y7 )
~.
(1.3.4)
Now equate h
value of h
e
to the expected value of h
e
and equate h
a
to the expected
and then solve for the estimates of the variance components:
a
2
2 8 = h
e
e
2 (8 2 + (}2) = h
a
e
a
.e
or
8e2 = h e / 2
(1.3.5)
8a2
(1.3.6)
=
(h
a
- h ) / 2
e
E. Niedokos [25] has extended one of the types of estimators proposed by Koch to two unbalanced mixed models.
He finds estimates of
the components of variance (assuming weighted and unweighted restrictions and correlated and uncorrelated interaction effects) in the twoway crossed classification with interaction and in the two-way crossed
classification with a third factor.
1.4.
Variance of Variance Component Estimators
Crump [8] found the large sample variances of the maximum like1i-
hood estimators in one-way classifications (the model defined by
(1.3.1).
He compared these large sample variances with the vari-
ances of the ANOVA estimators.
His conclusions were the following:
7
2
approaches the
e
2
asymptotic variance of the maximum likelihood estimator of 0 indepene
1) as n
+
00
the variance of the ANOVA estimator of 0
dent of the value of a; 2) as n +
tor of 0
2
is at least (a / (a - 1»
a
00
the variance of the ANOVA estimatimes as large as the variance of
2
the maximum likelihood estimator of 0 •
a
Tukey [35,36] used 'po1ykays' and 'minimal unit populations' to
find the covariance matrix for a set of estimators of variance components.
(Minimal unit populations are finite populations whose values
are all zero except for one value of unity and whose size is as small
as possible for the situation considered.)
In his procedure, Tukey
does not assume that the random variables are normally distributed or
from an infinite population.
However, he does assume the independence
of sets of the random variables.
·e
The procedure is based on two facts:
1) estimates of variance components are linear combinations of polykays
and 2) the averages of squared po1ykays are homogeneous polynomials ·of
degree four.
These two facts enabled Tukey to set up equations in
which the variances of variance component estimators are equal to po1ynomials (in polykays) of degree four with unknown coefficients.
Since,
in arriving at these equations, there are no underlying assumptions of
normality or infinite population, these equations are quite general.
If Tukey then finds the value of the unknown coefficients for any case,
these same values should hold true for all cases.
By considering mini-
mal unit populations, properties of the model, and assuming a normal
distribution, Tukey solves for the coefficients and hence he has found
the covariance matrix for the set of estimators being considered.
8
Searle [29] used matrix theory to find the variance of the estimates.
X }
n
If Y =~' A~, with x' = (xl' x 2 ' ••. , x n ) where {xl' x ' ••• ,
2
is a vector of multivariate normally distributed random variables
with zero means and with variance covariance matrix V, then var (Y) =
2 trace [(VA)2].
This fact enabled Searle to find the variances of
the mean squares.
Searle likewise made use of the fact that several types of matrices had forms which made it easy to calculate their traces.
In his
initial article [29], he showed how the sampling variance of least
squares estimates and the large sample variances of maximum likelihood
estimates in the one-way classification could be obtained.
In his
next articles, Searle [30,31] derived the variances of Henderson's
Method 1 estimators for the two-way factorial and nested classifications.
Mahamunulu [23] extended the work further by deriving the variances of Henderson's Method 1 estimators of the variance components
for the three-way, nested classifications.
Blischke [3,4]" considered
the three-way classification and then the r-way classification, but he
used Henderson's Method 2 to estimate the variance components and then
he found the covariance matrix of the estimators.
Still another approach to finding the moments of mean squares was
developed by Hartley [14] in 1967 and extended by Rao [27].
This pro-
cedure, called 'synthesis' by Hartley, can be used for mixed or random
models to find the numerical values of the coefficients in the formulae
for expected mean squares and for the variances and covariances of mean
squares.
The procedure is based on synthetic data consisting of the
columns of the design matrix.
The method is straightforward, but, if
9
there are many factors, the amount of work and/or time involved may
make it undesirable.
One of the most recent contributions in this area was Searle's
[33] presentation of the large sample variance-covariance matrix for
the maximum likelihood estimators.
These variances are relatively
easy to calculate, requiring only the evaluation of the derivatives
(with respect to the components of variance) of the variance-covariance matrix of the observations and the inverse of the covariance
matrix of the observations.
Once these variances are calculated, one
may compare other methods of estimation with the maximum likelihood
technique by using certain properties of the covariance matrix of
each method of estimation as a measure of "goodness".
Moreover, this
comparison does not require the maximum likelihood estimates themselves, which in practice may be very difficult to compute.
1. 5 •
Des igns
Consider the two-stage nested model given in (1.3.1).
What value
of· 'a' should be assigned to obtain the design with minimum variance
2
for the estimate of 0 , or what value of 'a' should be assigned if one
a
wishes to obtain the best design to estimate a
O
2?
or 0e'
P. P. Crump [6] was among the first to work on finding best
designs for estimating a
nested model.
O
and the variance components of a two-stage
He used the usual ANOVA estimation procedure of equa-
ting mean squares to their expected value and his findings were the
2
following: 1) to estimate 0 , the optimal allocation of the n observaa
tions is to have r classes with p+l observations and a-r classes with
p observations where n=ap+r, r (an integer) < a; 2) to estimate aO' the
10
appropriate design has a
design has a
= 1.
= n;
an d 3) to
.
est~mate
2 te
h
a
e
.
appropr~ate
Crump also presented tables comparing the effi-
ciencies of various designs.
In 1960, Gaylor [10] considered the estimation of variance components for a two-way crossed classfication with the goal of finding
the design which caused the estimates to have minimum variance.
Again one has to decide which parameters are most important, as a
good design for the estimation of one component may be a bad design
for the estimation of another component.
Gaylor found the lower
bound to the variance of estimates or combinations of estimates and
showed designs which attained this lower bound.
He presented tables
of efficiencies of various designs and from these tables he was able
to suggest a procedure for finding a nearly optimal design for estimating the individual variance components.
To find the nearly opti-
mal design it is necessary to know the approximate value of the
ratio of the variance component for rows to the variance component
for random error.
Bainbridge [2] pointed out that the usual balanced nested
designs provide many more degrees of freedom for estimating the
2
e
error mean square (hence a ) than for estimating other mean squares
(and hence, other variance components).
For example, if one wishes
to use a nested design to estimate the components of variance in an
experiment with four factors and eighty observations, one must
choose what type design to use.
Bainbridge presented four designs
to be considered, the first being the usual balanced design, and the
other three designs being of the type suggested by Prairie [26].
The two extreme breakdowns for the degrees of freedom are:
11
Design
Factors
DF
A
9
I
B in A
10
C in B
20
D in C
40
Total
79
IV
A
31
24
C in B
16
D in C
8
Thirty-two A units as 8
groups of 4 A units each;
each group contains an A unit
with 4, 3, 2, and 1 observation
79
If one is most interested in 0
.
d '~n ad'
2 Des~gn
.
most ~ntereste
1.6.
.•
Ten A units of 8 observations
B in A
Total
•
2
one should use Design IV; if one is
a
I should be used.
COMPARISON OF PROCEDURES AND DESIGNS
There are many means of estimating variance components and even
after choosing a method, one must select a design to use.
Bush and
Anderson [5] compared three procedures of estimation (fitting constants, weighted squares of means, and unadjusted sums of squares)
for the two-way classification assuming a random model for various
connected designs.
Based on the variances of the estimates, the
conclusions reached were that one should use a type of balanced connected rectangle design and the method of fitting constants to estimate the components of variance.
An example of a balanced connected
12
rectangle design is one in which the observations are arranged in
the following manner:
1
1
0
0
0
0
1
1
1
0
0
0
0
1
1
1
0
0
0
0
1
1
1
0
0
0
0
1
1
1
0
0
0
0
1
1
Others who compared procedures of estimation and/or designs are
Mostafa, Wang, Goldsmith, and Harville.
Mostafa [24] considered the
estimation problem for two-way crossed classifications with rep1ication:
·e
Y"
"k == l.l + a.l. + b.J + (ab)."
+ e""k
l.J
l.J
l.J
(1.6.1)
He proposed unbalanced designs for the joint estimation of variance
components in the model and compared these unbalanced designs with
balanced designs having the same number of observations.
Mostafa
concluded that the unbalanced designs considered are better for estimating
0;
and O~, but that these designs are worse for estimating
2
°ab'
Wang [37] investigated the one-way balanced model and compared
the merits of the following estimates of
1.
0
2:
a
equating mean squares to their expected values and solving
for their estimates
13
2.
truncated estimates
1'2
cra
3.
=
~
SSE -
a
o
if > 0
otherwise
exponential corrector estimates
Wang gave the exact distributions, the means, variances, mean
square errors, and probability of negative estimates for the various
estimates for certain values of the parameters.
Goldsmith [12] extended the comparison of designs to threestage nested designs assuming a random-effects model.
-e
He systemati-
cally presented and classified sixty-one three-stage designs.
For
various variance component ratios and different sample sizes, Goldsmith compared the designs in five ways using as his criteria the
trace, the determinant, and the adjusted trace of the covariance
matrix of the variance component estimates.
His results are pre-
sented in tabular form, with the choice of the optimal design being
dependent on the relative sizes of the components of variance.
Harville [16] presented a lengthy, detailed review of the literature on the one-way classification.
In this quite extensive study,
Harville discusses "goodness" criteria, Bayesian and non-Bayesian
estimation procedures, and inappropriateness of the model.
He also
comments on the inquadmissibility of estimates, where a quadratic
estimator is said to be inquadmissible if there exists a second quadratic estimator with the same expectation and whose sampling variance
is less than or equal to that of the first estimator for all points
14
in the parameter space with strict inequality holding for at least
one such point.
-e
CHAPTER II
PRESENTATION OF METHOD
2.1.
Introduction
As stated in the first chapter, there have been several articles
in the recent literature dealing with the estimation of the components
of variance.
The scope of many of these articles is limited to a
particular class of designs, if not to a single design.
If the scope
is broader, then it may be very difficult to apply the technique, as
is the case with the Hartley-Rao [15] maximum likelihood procedure.
Harville [16] points out many of the drawbacks of the Hartley-Rao
technique, not the least of which being the large amount of computation necessary for convergence, if indeed the method does converge at
all.
Furthermore, if there is convergence, this convergence may not
be to the true maximum.
Another practical problem with much of what has been done is the
assumption that the variables are sampled from infinite populations.
In many cases, this is a false assumption.
Tukey [35] very early,
and Gaylor and Hartwell [11] more recently have allowed for finite
populations.
In this chapter, Koch's [21] method of estimating variance components·is reformulated and extended.
The resulting procedure has
the following properties: 1) it gives an unique set of estimators of
the components of variance; 2) it is not limited to the infinite
16
population problem; 3) i t does not require the assumption of the normal distribution; 4) it is applicable to the mixed model as well as
to the random-effect model; and 5) it is competitive (using 'efficiency' as the criterion) with theANOVA estimators in many cases.
2.2.
Definition of the Model
The linear model which will be considered is given by
y = X a
I
+
h=l
(2.2.1)
U E.h
h
where X is an n x k design matrix consisting entirely of one's and
zeros with k 2 n;
U is an n x
h
~
matrix of known constants with
~
2
n;
a is a k x 1 vector of unknown constants;
~
~ x
is a
1 vector of unknown random variables
with
E(
.!2n )
=
(2.2.2)
Q
E( ~~' )
(2.2.3)
X is the design matrix for the fixed effects, U is the design
h
th
matrix for the h
random factor and V is an ~ x ~ matrix of
h
known constants.
The random vectors -b , -b , ... , -p
b are assumed to
2
1
be mutually uncorrelated.
and b
hj
(i
~
By the above definition the variables b
hi
j) may be correlated, i.e. the method proposed is not
restricted to infinite populations.
2.3.
Estimation Procedure
In the random model, Koch 121J considered the expectation of the
squared differences of the observations and then he equated the
expectation of the squared difference to the squared difference to
17
obtain estimates of the variance components.
For the mixed model,
the cross product of differences of observations, i.e. (Y
(Y
k
Y~),
-
i
- Y ) x
j
will be considered as well as the squared differences.
To form these quadratic functions of the observation, one uses a
matrix D and premultipliesYY' by D and postmultiplies by D', i.e.
Dyy'D'.
D is chosen so that DX
= 0;
that is, D is orthogonal to the
design matrix of the fixed effects.
Choosing D to be orthogonal to X
enables the components of variance to be estimated without having to
obtain estimates of the fixed effects.
The formation of the D matrix is presented by means of an exampIe.
Consider the model given in (2.2.1) where "the parameters have
the following values: n
= 7,
k
= 3,
p
= 2,
m
1
= 4,
and m2
= 7.
·e
b ' =
-1
b '
-2
X
=
1
1
0
1
1
1
U
=
1
0
0
0
.0
0
1
0
0
1
0
0
0
1
0
1
1
0
0
0 '0
1
1
0
1
1
0
0
0
1
0
1
0
1
0
0
1
0
1
0
0
1
0
1
and U
2
WI and W are immaterial to the definition of D.
2
= 17,
The D matrix for
18
this example is
D
=
1
-1
0
0
1
0
0
0
I·
1
0
-1
0
I
0
0
0
1
0
0
-1
I
0
0
0
0
1
-1
0
0
0
0
0
1
0
-1
0
0
0
0
0
1
-1
0
0
0
1
=
[:1 :J
--1----0
0
0
0 I 1
0
0
0
0
0
0
0
0
[
I
-1
0
1
0
-1
0
1
-1
Choosing this form for the D matrix satisfies the requirement that
DX = 0 and using this D matrix in Dyy'D' gives all the possible cross
products and squares of differences of the observations in the same
level of the fixed effect.
One can see that, in general, D is a
block diagonal matrix, that is
o
D =
o
o
o
D
r
if there are r levels of the fixed factor.
Each of the D. is defined
~
in the same manner as D and D above. Defining the D. in this way
2
~
1
·
.
.
. h·~n t h e ~. th
pro duces t h e tota1 ~ty
0 f d·~ ff erences
0 f 0 b servat~ons
w~t
19
level of the fixed factor.
If there are n. observations in the i
level of the fixed factor, then D has
i
r
hence the D matrix has.I
1=1
th
1
(n.)
2
1
(:i)
rowe and n
i
columns and
r
rows and.I n. columns.
1=1 1
Note that
one is not limited to a model with just one fixed factor in this
method, but that one can consider several fixed factors and call each
possible combination of levels of the fixed factors a 'level' under
the above formulation.
Let H be a vector containing the upper triangle of the symmetric
matrix resulting from taking Dyy'D'.
To estimate the variance com-
ponents, a subset of the terms in the H vector is used.
(The subset
of terms to be used is defined here and the rationale for the selection of these terms is given in section 2.4.)
The method of selecting the terms from H which are to be used in
the estimation procedure is presented by means of an example.
Con-
sider the two-way classification defined by
i = 1, 2, ... , r
Y •• =
1J
j
where a
O is an unknown constant.
C
2,
c
Let r' = (r l , r 2 , ••• , rr)'
'=(cl'c 2' ... , cc)' and~' = (ell' e 12 , ... ,e rc ) with E(!:) = Q,
E (rr')
0
= 1,
2
V •
3 3
=
oi
VI' E (cc ')
= O22
V2' and E (ee ') =
Also assume that .!:.' .£' and e are mutually uncorrelated.
Now
H consists of terms of the form (Y.. - Ykn ) (Y
- Y ) which can be
1J
XI
mn
op
expressed as [ (r
i
- r ) + (c. - c n ) + (e .. - e .n )
k XI
k
J
XI
1J
]
[(r
m
- r )
0
+
20
(cn - cp ) + (emn - e op )].
The terms of H to be used in the estima-
tion procedure are those whose expectations simplify to one of the
..
following forms:
1)
2
E[ (r.~ - r k ) 2 + (c. - c)
+ (e .. - ek 9,)2 ]
9,
~J
J
2)
E[ (r. - r )
3)
E[ (r
4)
E[ (c. - c9,)2 ]
i
2
k
~
- r)
k
2
2
+ (c.J - c9,) ].
]
J
where it is assumed that none of the four expectations are zero.
If
any term of H has zero expectation, then that term is not used in the
estimation method.
Hence the only terms of H to be included are
those terms which satisfy the following condition:
Condition 1:
The terms from
n to
be used in the estimation procedure
have non-zero expectation expressible as the expected
value of the sum of squares of differences of random
variables.
Let G be a vector containing all the terms in
Condition 1.
~
which satisfy
Q is a quadratic function of Y, that is Q
= G(YY').
Therefore the expected value of G is a linear combination of the
variance components Q'
=
2
2
(01' 02'
...
,
Thus the expected value
of G can be expressed as
where the elements of the A matrix are found by taking the expected
21
value of terms -1.
d. 'YY'd.
(d.' is the 1..th row of D and d. is the j th
-J -1.
-J
column of D').
E(d. 'YY'd.) = E(d.' (Xll +. g U1..1h.)
-1. -J
-1.
h&l, !.---:I!
(~'X'+' h&l
g
Q1.. 'Uhf) d.)
:J
--"1
(2.3.2)
Now DX = 0 by the manner in which D was chosen.
E (d. 'YY'.9..)
-:L
:J
= E (d.'
-:L
Since b , l2' ... ,
l
Ep
E(d. 'YY' d .) = d.'
-1. -
-J
·e
-1.
(!
h=l
U~)(!
h=l
11--..1
Therefore,
Q1..' U ') .9..)
h
:J
--"1
are mutually uncorrelated, this yields
g Uh E ('P.1..1h. ') Uh ' d.
h~l
-J
--..1--..1
= -:L
d.'
(2.3.3)
g cr 2 u VhU'd
h~l h h
h-j
(2.3.4)
Therefore,
a .. cr=d.'
-1.J -
-1.
.. ) is the ijth row in A.
where ~j = (aij,l' a ij ,2' .•• , a 1.J
,p
Now, equating E[G(YY')] to Q,(YY') , that is
A 8 = Q,(YY')
enables the components of variance to be estimated.
(2.3.5)
22
Premu1tip1ying both sides of (2.3.5) by Al gives
Al A 8
= Al
G
or
8 = (AI A)-l A'
(2.3.6)
G
This solution for Q is analogous to the least squares solution for
the vector of parameters
E(Y)
=X
~
in the fixed-effects model given by
~
~
where Y is the vector of observations and X is the design matrix.
is given by
A
~
=
(Xl X)-l Xl Y.
Q(YY') corresponds to Y, A corresponds to X, and
In the
fixed~effects
Q corresponds
to
~.
model, a function of the parameters
(Sl' S2' ••• , Sp) is said to be linearly estimable if there exists a
linear combination of the observations (!) which has expected value
equal to the"function of the parameters.
2
2
of the parameters (01' 02' ••. ,
°p2) is
In the same way, a function
said to be linearly estimable
if there exists a linear combination of the observations (G) which
has expected value equal to the function of the parameters.
If XIX has rank r < p, then only r linearly independent parameters can be estimated.
To obtain estimates of the remaining (p - r)
parameters, restrictions must be imposed (for example, S
p
I
Si
= 0).
=0
or
The same is true if AlA is singular; hence, if (A'A)-l
i=r+1
does not exist, a conditional inverse could be used to obtain esti-
23
mates of 0.
Another possible method of obtaining
estimates of
.
.
07
~
for i
=
1, 2, ••• , p when A'A is singular is to take more observations.
Hence, when estimating 0 and A'A is singular, one could include more
terms from H to remOve the singularity from A'A.
The procedure is easier to apply when Model II is being assumed.
In this case, the only terms to be used in G are the squared differences of observations.
The reason for using just the squared differ-
ence terms for Model II is that these terms give the b_est _quadratic
unbiased estimators (BQUE) in the 'complete' balanced case ('complete'
in the sense that there are as many mean squares in the ANOVA table
as there are variance components in the model).
Reformulation of Koch's method in terms of matrices has yielded
the following:
1)
The method may now be applied to the mixed model as well as
to the random model.
2)
The method allows for the introduction of finite popu1ations.
3)
The method gives a unique set of estimators, which Koch's
original formulation did not give except in the complete
(q.v.) case.
4)
The method does· not require that the random variables come
from a normal distribution.
(Koch's original formulation
does not require normality either.)
Furthermore, the estimates obtained are unbiased (by construction),
consistent, and they agree with the BQUE in the complete balanced
24
case.
2.4.
Selection of Terms
In this section, the selection of terms for the mixed model
shall be discussed.
In the estimation procedure for the mixed model
only those terms which had satisfied Condition 1 were selected.
The
question now is just why these terms are used in the procedure,
rather than the complete
li vector or just the diagonal elements of
Dyy'D' as in the random-effects model.
If one just uses the diagonal elements of DYY'D', this very
often leads to a singular A'A matrix.
One practical reason for not
using all the terms of Dyy'D' is that this would constitute using
·e
~L
(nil
i=l
2
2
terms, which may be too large to handle easily.
Using
just the terms which satisfy Condition 1 on the average leads to a
70 percent reduction in the number of terms for the examples considered in chapter 5.
The best reason for using the terms satisfying Condition 1
(rather than some other subset of the terms) is that these terms are
the ones which, when used in the complete balanced case in the mixed
model, lead to the ANOVA estimates.
For example, with a 2
3
factorial
model having one fixed-effects factor and two random-effects factors
where
25
a'
=
ab
=
(aO' aI' a 2 ),
ab
ab
ab
ab
abc
=
=
aC
aC
12
aC
21
aC
22
abc
abc
abc
abc
abc
abc
abc
e
ac
11
x=
111
l12
121
122
211
212
221
abc 222
U
3
and U
6
a
2
=
bc
11
=
£'
(b , b ),
1
2
=
(c , c )
2
1
bC
ll
bC
12
bC
21
bC
22
12
21
22
=
=
1
0
0
0
1
0
1
1
0
0
0
1
0
1
0
1
1
0
1
0
1
0
1
1
0
0
1
1
0
1
0
1
1
0
1
0
1
0
1
0
1
1
0
0
1
1
0
1
1
1
1
1
0
1
1
1
=
U
1
U
2
U = 11
5
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
0
1
0
1
0
1
0
0
0
0
0
0
1
0
1
0
0
1
0
0
1
0
0
0
1
0
1
0
0
0
1
0
0
0
1
1
0
0
0
0
0
1
0
0
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
0
0
1
0
0
0
0
0
0
0
1
0
0
0
1
0
0
0
1
0
= 18
=
b'
U4
(the identity matrix of rank 8).
I
a
O
is the mean; a
1
and
are the effects of the first and second level of the fixed factor;
, {b },' {c.},' {ab .. },' {ac .. },' {bc .. }, and" {abc .. k} are mutually uncori
J
J.J
J.J
J.J
J.J
26
related random variables with zero means and variances
a~, a~, a~b'
2 2 2 . 2 2
a ,a , and a b respect~vely. Note that a b here is really a b
~c
bc
~ c
~ c
~ c
+ the variance component due to random error, as they are inseparable
in this design.
The sum of squares and expected mean squares for
this model are
SS
~
SS B
=
(Y1 + Y2 + Y3 + Y4)
- (Ys + Y6 + Y7 + Ys)
=
(Y l + Y2 + Ys + Y6)
-
E(MS~) =
a
2
~bc
E(MSB)
= a
E(MSC)
= a
2
~bc
2
b
~
E(MS~B) =
c
+ 2
2
a~b
+ 2 abc + 4 a b
+
2
= a~bc
bc
~
L ~2a
+ 2
2
a~bc
2
= a·
+ 4
a~b
2
E(MS~C)
(Y 3 + Y4 + Y + Y ) ]2 / 8
7
s
+ 2
E(MS~C) = a~bc
E(MSBC)
2
2 0'2
~c
+ 2
+
2
a~b
2 0'2
~c
2
+ 2 abc
0'2
~c
2
]2 / S
2
2
2
+ 2 abc
+ 4 ac
27
Equating the mean squares to the expected mean squares, one obtains
the ANOVA estimates of the variance components.
2*
1
4
8c
=
For example,
MSC - MSaBC - (MSaC - MSaBC) - (MSBC - MSaBC) ]
t[
MSC - MSaC - MSBC + MSaBC ]
(2.4.2)
and
To apply the proposed technique, the D matrix is first formed.
this is a mixed model, the D matrix is
1
-1
0
0
1
0
-1
0
1
0
0
-1
0
1
-1
0
0
1
0
-1
0
0
1
-1
\
. l
1
-1
1
- -0
0
0
-1
0
1
0
0
-1
0
1
-1
0
0
1
0
-1
0
0
1
-1
Since
28
Dyy'D' therefore contains 144 terms, but, since this matrix is symmetric, there are really only 78 distinct terms to be considered.
Of these 78 terms, there are 30 terms which satisfy Condition 1.
These terms with their expected values are given in Figure 2.1.
The general form of the equation in Figure 2.1 is
G(YY')
= A cr
and, therefore,
A'A
=4
20
2
12
2
12
8
2
20
2
12
12
8
12
2
12
2
8
8
2
12
2
12
8
8
12
12
8
8
18
12
8
8
8
8
12
12
~e
(A'A)-l
1
8
= -
.~
.5
.25
-.5
-.25
-.5
.25
.5
-.25
-.5
-.5
.5
-.5
-.25
.9
.45
.5
-.9
-.25
-.5
.45
.9
.5
-.9
-.5
-.5
.5
.5
1.
.5
.5
-.9
-.9
-1.
-1.
1.7
For purpose of comparison, $2 and 8 2 will be presented and shown
c
abc
equal to the ANOVA estimates.
The estimates of the remaining vari-
ance components obtained by this method can also be shown to be equal
29
Figure 2.1.
Terms (and their expected values) from Condition 1
(Y -Y )
1
2
(Y -Y )
1
3
2
2
2
0
2
0
2
2
2
2
0
2
0
2
2
(Y -Y )
1 4
2
(Y -Y )
2
2
2
2
2
2
2
2
2
2
2
2
(Y -Y
2
0
2
0
2
2
0
2
0
2
2
2
0
2
0
2
2
2
2
0
2
0
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
0
2
0
2
2
0
2
0
2
2
2
0
2
0
2
0
0
2
0
2
0
0
2 -2
0
~I
2
2
3
4
z
)
2
(Y -Y )
3 4
2
(Y -Y )
S 6
2
(Y - Y7)
S
2
(Ys-Y )
a
2
(Y -Y )
6 7
2
(Y -Y )
6 a
2
(Y -Y )
7 a
(Y -Y ) (Y -Y )
1 2
3 4
(Y -Y ) (Y -Y )
Z 4
1 3
(Y -Y ) (Y -Y )
Z 3
1 4
(Y -Y ) (Y -Y )
S 6
7 a
(Y -Y ) (Y -Y )
S 7
6 a
(Ys-Y ) (Y -Y )
a
6 7
(Y -Y ) (Y -Y )
1 2
S 6
(Y -Y ) (Y - Y S)
=
2 -2
0
2
0
2
0
0
2
0
Z
0
0
0
2 -2
0
0
2 -2
0
2
0
0
2
0
0
2
0
0
0
0
2
0
0
0
2
0
Z
0
0
0
0
0
(Y -Y ) (Ys-Y )
1 4
a
(Y1-Y4) (Y -Y )
2
2
0
0
2
0
Z -2
0
0
0
0
(Y -Y ) (Ys-Y )
Z 3
a
(Y -Y ) (Y -Y )
2 3
6 7
(Y -Y ) (Y -Y )
2 4
S 7
(Y -Y ) (Y -Y )
2 4
6 S
(Y3~Y4)(YS-Y6)
Z -2
0
0
0
0
2
2
0
0
2
0
2
0
0
0
0
0
2
0
0
0
2
0
0
2
0
0
0
0
0
2
0
0
2
0
2
7
(Y -Y 3) (YS-Y )
1
7
(Y -Y ) (Y -Y )
1 3
6 a
1
6
7
(Y -Y ) (Y7-Y a)
3 4
0
0
2
b
2
e
0'2
ab
0
0
2
ae
2
be
0'2
abe
I
·30
to the ANOVA estimates, but this will not be done here.
8~ = ~2 {
4 [ (Y -Y )(Y -Y )
7 a
1 2
+
(Y -Y ) (Y -Y ) ]
s 6
3 4
+
2 [ (Y -Y )(Y -Y ) + (Y -Y ) (YS-Y ) + (Y -Y )(Y -Y ) +
1
2
6
3
S
l
4
z
S
3
6
7
(YZ-Y4)(YS~Y7)
- (Y l -Y3 )(YS-Y 7 ) - (Y l -Y 4 )(Y 6-Y 7) -
(Y -Y )(Y -Y )
2 3
s a
-
1'[
crabe = SO
l
6 [ (Yl-Y Z)
(Y S-Y 6)
2
+
2
]}
(Y -Y )(Y -Y )
2 4
6 a
+ (Y 3-Y 4 )
(Y7~YS)
2
Z
+ (Y l -Y 3 )
+ (Y S-Y 7)
2
Z
+ (Y Z-Y 4 )
+ (Y 6-Y S)
Z
2
+
]-
Z
- (Y l -Y ) (Y -Y ) + (Y -Y )(Y -Y ) + (Y Z-Y )(YS-Y ) 3 S 7
1 3
6 S
7
4
(YZ-Y4)(Y -Y )
6 a
+
(Y -Y ) (Y -Y ) s 6
3 4
(Y -Y )(Y -Y )
3 4
7 a
]}
(2.4.S)
31
Several unbalanced examples were also considered for the mixed
model.
In these examples, the coefficients of the square and cross
product of the variance components in the formulae for the variancecovariance matrix of the estimates of the variance components were
compared.
Upon trying many different combinations of terms in the
estimation process, it was found that using those terms which satisfied Condition 1 gave, in general, smaller coefficients of the
square and cross product of the variance components in the formulae
for the variances of the estimates of a.
2.5.
Variance-covariance Matrix of the Estimates
In this section, the variance-covariance matrix of
Q is
calcu-
lated.
Var a = v = E(aa')
- [ E(a)
] [
E(Q) ]
,
(2.5.1)
E(a) = E [ (A'A)-l A' G
= (A'A)-l A' E(G)
= (A 'A) -1 A'A a
=
a
E(aa') =
(2.5.2)
E [ (A' A) -1 A' GG' A (A'A)-l ]
= (A'A)-l A' E(GG ') A (A' A) -1
(2.5.3)
The terms which make up -G are of the form -1
d. 'YY'd.
which means
--J
that the elements of GG' are of the form (§.i'YY'dj)(~'YY'E..Q,)' This
means that one desires that the expected value of the following terms
32
be found:
1)
(d. 'YY'd.)2
2)
(d 'YY'd.)(d 'yy'd )
q--~
-k--k
3)
(d.'YY'd.)(d. 'yy'd )
4)
(d
'YY'd.)(d
'yy'd )
q--~
-k---9"
5)
(d.
'YY'd.)(d
'yy'd )
-~--J
-k--9"
"-l. -
-J. -
-J.
-~
-~ -
-9"
The other terms which could come from different arrangements of
i, j, k, and 9" reduce to one of the five forms given above.
The ex-
pected value of the five forms above are given next.
1)
E[ (d. 'YY'd.)2
-~ --J.
(X~ +·I
= E {[ £i'
UIA)
x
h=l
(~'X' + h=l
I =-hb 'uh ') -J.d.]2~j
Since DX = 0
E[ (d.'YY'd.)2 ] = E[ (.
-J. -
-J.
~L -~
d.'U b )4 ]
h-=tl
h=l
Since the b. 's are mutually uncorrelated, one has
"-l.
E[ (d. 'YY'd.)2
-:I.. -
-J.
=
I
h=l
6
(£.
E[
J.
'U,--bt.J
p~l
~L E [
L
h < h'
4
] +
!r'.1
(£. 'U,--1:>.1..)
2 (~i 'Uh'~ ,) 2 ]
1r-11
J.
(2.5.4)
2)
E[(~i'YY'di)(E.k'YY'E.k)]
(~'X'
+
I
= E[
di'(X~ +
I
I UIA) x
h=l
b'U ')d.d '(Xct +
U b )(a'X' + ~b'U')d ]
h=l-=tl h -J.-k h=l h.':.h h* h ~
33
Again, since DX = 0
E[ (d.'YY'd.)(d 'yy'd ) ] = E[ (
-l. -
-J..
-k -
-=k
gL. -J..
d~U1A1
b )2 ( g d 'u b )2 ]
L. -k 1A1
h=l
h=l
Since -b , -b , ... , -p
b are mutually uncorrelated, this implies that
I
2
I
E[ (d.'YY'd.)(d 'yy'd )] =
E[ (d.'Ub)2
-l. -J..
-k - -k
h=l
-l.
1A1
(A,.'Uh~1J2]
+
~-u
-L\.
(2.5.5)
3)
E[ .(d.
'yy'd.)(d.
'yy'd
) ]
-l. -l.
-J.. -Q,
= E[
=
( g
( g
L. -~l.' 'U~)3
L.---.L
2. _dn'U~~)
N
Lc-lL
h=l
\
h=l
I E[
h=l
(d.' U b ) 3
-l.
h:=-h
(~n' U~P1.)
N
] +
lc-ll
(2.5.6)
4)
E[ (d.'YY'd.)(d
d' Uh:=-h
b)2 ( g
d' Uh!?h ) x
-l. -l.
-k 'yy'd
- -Q, )] = E[(. g
1. -'
2. -k
h=l l.
h=l
(2.5.7)
=
Since b 1 , b , ... ,
2
~
E[
(I
d.
h=l -1.
I U1.- blJ
x
Lr-1L
are mutually uncorrelated, this implies that
E [ (d. I yy I d . )( d I yy I d ) ]
-1. - -J -k - -JI,
I
= h=l
E [ (.§.1.' I
U1.-~)
X
lr-11
(d. I U~ )
-J
X
lr-11
(2.5.8)
Now if the distribution of the random variables is known, it is possible to evaluate the above expressions.
For purposes of comparison
with other estimation techniques, it is assumed that the random vectors {b· } are distributed N (O,W. ).
h
-h
~From multivariate normal theory, it is known that if !-Nm(Q,R),
then
~'X-N(O,~'Rd).
3 cr 4 '
d'X
It is also known that E{ [.§.'! - E(.§.'!) ]4 }
=
Therefore, (2.5.4) can be rewritten as
.I
4
I
d. )2
E[ (d. ' yy l d.)4 ] = 3
U V U I -1.
cr (d.
-1. --1.
-1.
h h h
h=l h
p-l
+ 6
I .I
h < h'
(2.5.9)
2 2
d.)(d. I Uh,Vh,Uh'
[(d. I UhVhU I -1.
crh crh,
-1.
-1.
h
I
d.) ]
-1.
35
The evaluation of (2.5.5) requires the expected value of terms like
(E.l'X)2 (d 2 'X)2 where X - Nn(Q,R).
Letf -
[~~
J
K·Then
l
i - N2 (Q,[d ] R(E.l'd 2)) ,Le. ! has the bivariate normal distribution •
. d '
-2
The moment generating function of the bivariate normal distribution
shall be used to find E[
(E.l 'X)2
(.S!.2 '1D
2
].
(2.5.10)
The mechanics of taking the derivatives necessary to find
E[ (d 'X)2 (d 'X)2 ] appear in the appendix and only the results are
-1 -
-2 -
presented here.
4
d met!' t 2 )
dt
2
l
dt
(2.5.11)
2
2
0,0
Therefore,
Now (2.5.5) simplifies to
E[ (d.'YY'd.)(d 'YY'd)
-1 -:I.
-k - -k
36
(2.5.12)
To evaluate (2.5.6) one requires the expected value of terms of the
form (£1,!)3 (£2'X).
The moment generating function in (2.5.10) can
be used again.
4
d met!' t 2 )
dti
(2.5.13)
dt 2
0,0
Therefore,
Hence
E[ (d. 'YY'd.)(d. 'yy'd ) ]
-'l. -
-1
-'l. -
-J/,
=
3·
I .I
h=l h
'=1
0
2
(i,
h h
(£1~
UhVhUh ' £1') x
(2.5.14)
The evaluation of (2.5.7) necessitates the calculation of
37
x - Nn(0,
R).
-
To
-
. find E[ . (~l ,!)2 . (d I!) (~3 '~) ]use the moment generating function
2
(2.5.15)
(2.5.16)
This means that (2.5.7) can be simplified to
E[
(~-r' yy ,~-r ) (~k'yy , ~n)
......
XJ
] =
I .~.~
h=l h =1
2 2
0h 0h , [ 2 (E.-1' UhVhUh 'E.k) x
...
(2.5.17)
To reduce (2.5.8) to a form which can be calculated requires the ex-
The moment generating function of
f
is
38
(2.5.18)
4
a m(t ,t ,t ,t )
l 2 3 4
at
l
at
2
at
3
at
4
0,0,0,0
(2.5.19)
Therefore (2.5.8) can be reduced to
E1 (d.'YY'd.)(d 'YY'd)]=
-1. -
-J
4<. -
-)I,
I I
2
2
a ah , [(d.,'uhvhuh,d.) x
h=l h '=1 h
-1.
~J
(2.5.20)
These then are the formulae for the expected value of the terms in
E ( GG' ).
In order to calculate the variance-covariance matrix of
the estimates, [E ( Q )][E ( Q )]' must be subtracted from E ( GG' ).
If the random variables are known to come from some distribution
other than the normal, the equations through (2.5.8) are still valid,
but the remaining equations have to be adjusted to account for the
different distributional form.
CHAPTER III
EQUIVALENCE OF PROCEDURES
3.1.
Introduction
In this chapter, the equivalence of the matrix procedure pre-
sented in Chapter II to Koch's method of estimation will be shown.
The equivalence of the matrix estimators to the BQUE in the complete
balanced case will also be demonstrated for several models.
3.2.
Equivalence with Koch's Method
Consider the two-stage nested design given by equation (1.3.1).
The estimates of
~2
(5
e
=
1
2
(5
a
and
(5
2
e
proposed by Koch [21] are
n.1. (n.1. - 1) ]-1
[I
a
~2
aa = 12
2
1
I
i=l
a
I
i=l
n. (n .., n.) J""l
1.
1.
I
j;l:.Q,
a
I
(Y •• - Yi,Q,)
2
(3.2.1)
1.J
a
I I I.Q,
i l' k j
(Y.. .., Yk.Q,)
2
1.J
~2
(5
e
(3.2.2)
The estimates of the variance components given by the matrix method are
IG: ~.]
(3.2.3)
= (A •A) -1 A' G(YY • )
where A is a
(~) x 2 matrix.
The first column consists of
40
(~) - .I (:i).
two's and the remaining elements of the column are
~=J.!
zeroes.
AlA
The second column contains all two's.
=4
Then (AlA) -1
(~) - I ( :i)
(~) _I ( :i)
(~) - I (~i)
{~l
= -u1
Therefore,
I(:i) -(~)
(~l
I( :i) -(~ 1
(~) - I ( :~
A' G(YY') = ( : )
Then (A'A)-l A' G(YY')
_.
= 1u
(
I
~)
v
+ I
~:i) - (~)
I(:~ - (~) ]
v
+ I
J
w
(~)
-
I (:
i) J
W
41
= -u1
~ - ~ [ (~)
=
[v -
q-(~)
t
w]
+
+ [
r(:!) 1 w]
(3.2.4)
rI ( : i) ] wJ
J
If one adds and subtracts
II
:i) J t, one has
(n - n.)
1.
(n - n.)
1.
=
J-1
v _
J-1
0- 2
e
~2
(5
v
(3.2.5)
a
(n. - 1)
1.
J- 1
t
=
~2
(5
e
(3.2.6)
This equivalence proof covers only one class of designs under the
random model.
However, it is clear that the algebraic procedure may
be extended to include other experimental situations which assume an
underlying random-effects model.
Another method of showing the equivalence of the matrix approach
to Koch's method of estimation is given by the following:
42
Consider the model
Yij = aO
i =
+ r i + cj + eij
j
where a
O
l~
= 1,
(3.2.7)
2, .•. , r
2, ...
~
c
is a constant and {r.}, {c.}, and {e .. } are random variables,
J
~
each with mean zero.
~J
Also assume that the variables are all mutually
2222·22
uncorrelated and that E(r )
i
= aI'
expected value of
-
E[ (Y ..
~J
•
Yk 9,,)
2 ]
=
2
2 ( °2
=
2
2 ( a
2
+ a3 )
i
2
l + ° 3)
k, j
i :f k, j
a12 + a22 + a32 )
2
+ 2 °3
a2
2
2 °1
2
+ 2 °3
2
a 2
ex
2
2 °1
2
2
+ 2 °2 + 2 °3
2
2
=
l
3
=
:f 9"
=
9"
i Of. k, j :f 9"
2
2 °2
a
The
difference is
= 2 (
Let a -
= 03.
E(C ) = a2 , and E(e ij )
j
a 2 2
2
2
°1
Co
2
.2
0
2
°3
Let xl be a vector containing the elements of the set of (Y .. - Yk 9,,)
2
~J
with i
= k,
j < 9,,; let ~2 be a vector containing the elements of the
set of (Y .. - y kn )2 with i < k, j = 9,,; let x be a vector containing
. ~J
N
3
the elements of the set of (Y .. - y kn )2 with i < k, j :f 9".
~J
N
Therefore
one has the equation
(3.2.8)
43
where~'
=
~i.
(1,1, ••• , 1) has as many elements as
Defining the G vector and theA matrix of the matrix procedure
in Chapter II in terms of the above, one has
G=
and
A=
C
11
0
0
~2
0
12
0
~3
0
0
13
~1
Therefore the matrix estimate of cr is given by
A
-cr
= (A'A)-l
=
C'
= C-1
A' G
. ,.
~1 ~1
0
0
. ,.
0
0
~2 ~2
0
0
. ,.
~3 ~3
·~11,..11
0
0
~1
0
. 1,.
0
~2
0
. 1, .
~2
0
.12
.13 .13
or
cr = C-1
/';
-1
C
1
11 '11
1
· ,.
.12 ~2
1
· ,.
1.3 ~3
~3
c
r
L
L j<.Q,
i
r
c
I L
i<k j
r
L
i<k
(Y .. - Yi.Q,)
1J
(Y .. - Ykj )
1J
c
I
j#.Q,
(Y .. - Yk.Q,)
1J
2
2
2
c'
11
0
0
0
12
0
0
0
13
G
44
where h c , h r , and h rc were defined. by Koch I2lJ.
Koch then performed
a non-singular transformation of the hIs and obtained estimates of
the variance components.
The estimate of
provided by the matrix
~
procedure is also a non-singular transformation of the hIs (since C
is non-singular for complete cases) and hence the estimates must be
"'~,
the same for both methods.
Any complete random model can be expressed
as a one-way model equivalent to (3.2.8) and then the succeeding steps
can be applied to show the equivalence of the matrix method to Koch's
method.
3.3.
Equivalence to ANOVA Estimators in the Complete Balanced Case
In order to demonstrate the equivalence of the proposed estima-
tors to the ANOVA estimators in the complete balanced case, three
examples are presented.
From these examples, it is clear ,that the
equivalence does hold in the complete balanced case.
pIe is a randomized block experiment.
The model is given by (3.2.7).
The observations are arranged in the following
c
r
r
r
l
2
:r
l
c
2
c
c
Y
ll
Y
Y
Y
Y
Y
Y
21
rl
12
22
The first exam-
lc
2c
Y
rc
manner~
45
Let
G(YY') =
(Y
(Y
(Y
(Y
(Y
(Y
n
2
"" Yl -2).
1l
_ Y )2
13
n
- Y )2
lc
n -
l1
r·
Y
21
- Y
rc
c-l
)
2
)2
_ Y
rc
)2
The A matrix corresponding to G(YY') is a ( ~) x 3 matrix.
The
first column of A has r c (c"",l) zeroes and the first·column has
2
r c (rc-l) - r c (c-l) two's; the second column consists of r c (r-l)
2
2
zeroes and n (n-l) -2 r c (r-l) two's and none of the zeroes in the
second column are in the same row as the zeroes in the first column.
The third column of A consists of all two's.
The above can be seen
by considering
(3.3.1)
46
and
A'A = 2 rc
'c(r
c (r '" 1)
(r - l)(c - 1)
c(r ""' 1)
A'A
=I
2rc
1)
r(c - 1)
r(c - 1)
r(c - 1)
rc - 1
c(r - l)r{c - 1) (rc - 1) + 2(r - l)(c - l)c(r - l)r x
2
(c - 1) - c (r - 1)2r (c - 1) - (c - 1)2(r - 1)2(rc - 1) 2
c(r - 1)r (c - 1)2 J
1
(A 'A)-l =
2rc(r-1) (c-1)
r
A' Q(YY')
-r
1
c
-c
-r
-c
r+c-1
(Y •. - Y 9-)
k
c
Ii kI j!=9I
(Y •• - Y 9-)
k
(3.3.3)
2
. 1J
r r c c
Ii kI Ij 9-L
2
1J
i!=k j
r r
1
c c
L L 9-L
=
(3.3.2)
r
.
(Y •. - Y 9-)
1J
k
2
In the expression for A'G, instead of considering just the terms
given in G and multiplying by two when appropriate, the subscripts
are allowed to extend over their entire range as 2 (Y .. - Y1• 0 )
1J
(Y •• - Y.
o)
1)(,
1J
2·
r
t
= Y
+ (Y. 0
1!V
-
2
Y .• ) •
II
i7kj 9-
(Y .. - Y .o )
kN
.1J
!V
For ease of computation, let
1J
c c
2
2
,
u
=
r r
c
II Y
i k j79-
2
(Y .. - Yko ) , and
1J
N
=
47
r r c c
v
IIII
=
i k j 51,
(Y .. "" Yko )
~J
!'v
2
rt + u - rv
1
= -::----:---::;.,.....,,..--.-:-:2rc(r-1) (c-1)
(3.3.4
+ cu - cv
t
-rt - cu + (r+c-1)v
Now the ANOVA estimators (BQDE:forthis model) are the following:
,.,2*
°3
=
1
rc(r-l) (c-l) { rc
r
c
I
I
j=l
i"'l
c
c
a12*
=
1. { ~
c
1
r-1 [ -
c
2
Y..
~J
r
I I
j=l i=l
(
Y.. )
(I. c1 Y.. ) 2
I
i=l
r
J=
2
~J
c
r" .
2
,J
(I
Y. j )
j=l i=l ~
r
-
-
~J
r
+
(
c Y.. ) 2 )
I
I
i=l j=l ~J
1(Ir
I
c Y • .)
i=l j=l ~J
rc
2
] -
(3.3.~
2*}
6 '
3
(3.3. I
/-'2*
°2
1 '{ - I [ -1
:',r,
"c-1
r
=~-
To show
r
c
I ( I Y.. )
i=l j=l
th~,equa1ity
~J
2
1
rc
r
(
c
I I
i=l j=l
Y.. )
2
~J
between the two sets of estimators, one
compares the coefficients of the terms appearing in the estimators.
632 can be simplified to
In v
ea~h Y~j
term appears 2(rc-1) times and each Y Y 5l, term (i=k,
ij k
48
jf£, or
i~k,
j=£, or
i~k, j~£)
has a coefficient of -4.
2
1J
each Y.. term occurs
2(c~1)
In
times and each
Y"Y'n (jf£) has a coefficient of -4.
1J 1";v
2
Y.. term occurs 2(r-1) times and each Y..Y , (ifk) has a coefficient
~
~ kJ
of -4.
e;
Dividing these coefficients by the two in the denominator for
results in
e;
and 8;* having the same denominator.
The coeffi-
cients of the terms appearing in 8
2*
are quite easy to see just by
3
considering the definition of 8;*.
One can use the same idea as
2
2 ~2*
~2*
above to obtain the coefficients of terms in 81 , 8 2 , vI ' and v 2 •
A table of these coefficients follows:
-f
r c (r-l) (c-l)
coefficient of
in
2
"X
2
Y..
1J
Y..
Y.£
1J 1
Y, .Y ,
1J k J
YijY k £
83
rc-r-c+l
-2(r-l)
-2(c-l)
2
832*
rc-r-c+l
-2(r-l)
-2(c-l)
2
2
81
0
2(r-1)
0
-2
8 2*
0
2(r-l)
0
-2
2
82
0
0
2(c-l)
-2
8 2*
2
0
0
2(c-l)
-2
1
Hence from the table one can see that for a randomized block design
(assuming a random-effects model) these matrix estimators are BQUE.
The next example shows that the matrix method yields the same
49
set of estimators as the ANOVA estimators for the balanced randomized
block design, but now a mixed model is being assumed.
where a is an unknown vector of constants; X is the design matrix for
the fixed effects; E(b.) = 0; E(b~)
J.
J.
= °12 ;
2
J.J
E (e .. ) = 0; E (e .. )
J.J
E(bJ..bk) = 0; and E(e .. b.) = O.
J.J ].
Y.. =
J.J
]l
(3.3.8)
i = 1, 2; ... , r
+ a. + b. + e ..
J].
J.J
1, 2, •.. , c
j
.
r(r-l)c(c+l)
rc(r-l)
Q(!) wJ.ll have
4·
terms of which
2
are squares and the
others are cross product terms.
The A matrix will have two columns;
the first column contains all two's and the second column contains
rc(r-l)
2
two's and the remaining terms are zero.
A'A
rc(r-l) (C+l)
4
=4
rc(r-l)
2
rc(r-l)
rc(r-l)
2
2
= 2 rc(r-l)
=
1
--:---::-~-::-o-'
rc(r-l) (c-l)
1
-lJ
[
-1
(3.3.9)
c+l
2
G is now composed of square terms and cross product terms, i.e.
(Y .. - Y. n )
J.J
J.N
2
and (Y .. - Y'n)(Y
- Y ) respectively.
km
k0
J.J]'N
Let t= sum of
Corrections
Page 49
Last line on page 49 should be:
Let t = sum of
Page 50
The lines following equation
812 = rc(r-1)2 (c-1)
=
2
rc(r-1) (c-1)
=
c-1
rc(r-1) (c-1)
, 2*
Now 81
2*
and &3
.
.
should be:
(3.3~10)
t
L L
i<k j<m
r
c
i<k
j
(Y J.J
.. -Ykj)(Y.J.m-Yk m)
L L (y .. -Ykj )
2
J.J
2
- &1
of the previous example are the ANOVA
2
2
estimators for 01 and 02
respectively in this example •
'
The table of coefficients is
..
2
Y
ij
r c (r-1)(c-1) x
coefficient of
in
Y.jY.
J. J.e
Y.. Yk .
J.J J
Y Y
ij k1
&2
2
(r-1)(c-l) -2(r-1)
-2(c-1)
2
&2*
3
812
8f.*
(r-1) (c-1) -2(r-1)
-2(c-1)
2
-
0
2(r-1)
0
-2
0
2Cr-1)
0
-2
From this table one is able to see that the matrix procedure yieJ.ds
e.-
50
= sum
cross product terms in Q; let u
v
= sum
"
of all the terms in G.
A' G(YY')
and
Then
=
[
(A'A)-l A' G =
1
rc(r-l) (c-1)
= -r-c
2
61 = rc(r-1) (c-1)
e;
2 t
-2 v + (c+1)
[
1
r-_':;'l""'r'7'(c--""""l""')
-i-(
2
of square terms in G; and let
J
2 t
(c-1) u - 2
J
(3.3.10)
t
r
=
1
rc(r-1) (c-1)
=
c-1
2rc(r-1) (c-1)
c
c
i~k j~R, k~O
(Y, , - YiR,)(Y
- Y )
ko
km
l]
c
r
Ii jo/R,
I
(Y., , - YiR,)
l]
2 _ 62
1
Now 8 2* and 62* are the same as in the previous example.
1
2
The table of
coefficients is
r c
(r~,l)
( c-l)
coefficient of
in
2
x
2
Y..
l]
Y,l],Y.l R,
Y, ,Y ,
lJ k J
Yij YkR,
62
(r-l) (c-1) -2(r-1)
-2(c-1)
2
8 22 *
(r-1) (c-1) -2(r-1)
-2(c-1)
2
812
0
0
0
-2
8 2 '1'
1
0
0
0
-2
From this table one is able to see that the matrix procedure yields
51
the same estimators as the ANOVA method for the randomized block design (assuming
a mixed
model).
The last example is a three-stage nested design with model equation
.. + .e.1J'k
Y.1J'k = ex o + a.1 + b 1J
i = 1, 2,
· .. ,
a
1, 2,
·.. ,
·.. ,
b
j
k = 1, 2,
n
= abc.
22222
; E (b . .) =
; E (e .. k) =
1
2
1
1J
1J
mutually uncorrelated.
°
°
o;.
E(a ) = E(b . .) = E(e. 'k) = 0;
i
1J
1J
The random variables are all
The A matrix for the matrix procedure is an (~ )
first column consists of a
bc
2
elements are all two's.
c
2
AlA
2 abc
Let r
=
=
matrix.
The
zeroes and the other
The last column of A is composed entirely of
two's.
AlA = 2 abc
x 3
zeroes and the other elements are all
The second column is composed of ab
two's.
c
{b .. }, and {e. 'k} are random vari1J
1J
abIes with the following properties:
E (a .) =
(3.3.11)
(a-l)bc
(a-l)bc
(a-l)bc
(a-l)bc
(ab-l) c
(ab-l) c
(a-l)bc
(ab... l)c
abc-l
2
(a-l)b(b-l)c (c-l)
2 a 1)2 c3 (a-I) (b-l) Cc~l)
52
=!
(A'A)-1
e(ab-l)(e~l)
-(a-l)bc(c-l)
o
- (a-l)b e(e:"l)
Ca-l)bc(be-l)
... (a-1)b (b-1) c
2
- (a-l)bc Cb-1)
(a-l.) (b-1)bc
r
o
2
2
(3
a
= . 'i'
i~i'
A' G(YY '.)
a a
bb
cc
2
CY
33' t t,
'i' 'i'
'i' 'i'
k-Yo, 'k' )
1J
1 J
0
0
0
b bee
o , o'k')
ILL
L kL Ik' (Yo.ok-Y
o
1J
1 J .
11
JJ
a a
b bee
0'
0
.12)
...
L LO.kL k'
L (Yo1Jok-Yo1Jok')
2
l.J
0'
L
L L L I I (Yo ok-Yo, o'k')
i i' j j' k k' 1J. 1 J
abc c
2
0
2
(3.3.13)
a
Le t t
=
b bee
I I I LI
i:fi' j j ' k k'
a a
u =
(Y 0 Ok - Yo, 0' k ' ) 2
1J
1 J
2
bee
I L I LI
j :f j ' k k'
(Y
i i'
k ... Yo, 'k ' )
1J .
1 J
0
0
0
a abc c
v =
L I0'. ILL
0 k k'
1o
1
aa
W
Then
2
(Y
J
bb
k ... Y., ok')
1J
1 J
0.0
cc
= L L ILL L
i i ' j j ' k k'
2
(Y Ok ... Yo, 'k ' )
1J
1 J
0
= (A' A) -1 A' G(YY , )
0
53
8 12
Then
1
= r
c(ab-l) (c-l)t
(a-l)bc(c-l) (w-v)
8 22
-(a-l)bc
8 32
2
- (a-l)bc Cb-l) (w-v)
1
= r
I
(c-l)t -
(bc-l) (w-v)
+
+
(b-l)cw]
2
(a-I) (b-l)bc w
(b-l) (c-l)ct .... (a-l)bc(c-l)u
(3.3.14)
2
(a-l)bc(c-l)u - (a-l)b(b-l)c v
.
2
(a-l)b (b-l) c v
The" ANOVK estimators" (BQUE- for. this.model) are
8 12*.
1 { bc 1..
a (Y.
2
-Y
)
CO. 1
~. .
. ..
~=
c
= -b.
L bL1
a
. 1
(Y.. _Yo ) 2
0
J=
~=
a-I
~J.
~
••
}
a (b-l)
(3.3.15)
b
o
I1
(Y
J=
0
0
~J
•
_Y.
~
••
)2
I
c (Yiok-Y .. ) 2 }
k=1
J
~J •
ab (c-l) .
a(b-l)
abc
L L L
(3.3.16)
_
2
(3.3.17)
(Y 'k-Y " )
i=l j=l k=l
~J
~J'
0
ab (c-l)
The table of coefficients for each set of estimates is
Coefficient of
in
.
2
Y"
lJ k
Y.lJ'kYolJ'k'
(;2
1
abc
abc(c-l)
(12*
1
abc
abc( c-l)
3
3
(12
1
8 2*
1
-2
YijkYij 'k'
0
0
0
0
2
-2
2 2
-2
0
0
0
0
YijkYi'j'k'
2
abc (b-l)
2
2
abc (b-l)
ab c (a-I)
-2
2 2
ab c (a-I)
54
Coefficient
of
2
Y,1J'k
' 'k'
Y,1J'kY1J
" 'k'
Y,1.J'kY1J
-
in
62
2
0
82*-
0
2
2
abc(c-l)
2
abc(c-l)
-2
2
abc (b-l)
Y,1J'kY1' , J. 'k'
0
-2
2
abc (b-l)
0
-
Hence one can observe that the matrix procedure leads to the BQUE for
a balanced three-stage nested design. (under the assumption of a random
model) •
These three examples show the equivalence of the matrix estimation procedure to the ANOVA method in the balanced cases of a nested
design and a cross classification (assuming a random-effects model)
and for a cross classification design (assuming a mixed model).
From
these examples one can see the equivalence of the matrix method to the
ANOVA technique in the balanced case (to the BQUE in the complete
balanced random situation).
CRAPTER IV
VARIANCE FORMULAE
4.1.
Introduction
This chapter contains the formulae for the elements of the
variance-covariance matrix for the ANOVA estimators, for the maximum
likelihood estimators, and another derivation for the 'Koch' type
estimators.
Although the variance-covariance matrix of Koch's esti-
mators has already been derived in Chapter Two, another derivation is
presented here.
4.2.
'Koch' type estimators
4.2.1.
Introduction.
In this section another form is given
for the variance of Koch's estimators under the assumption of a random-effects model.
The following derivation is presented because it
considers specifically the design being used, and hence the formulae
given here for the elements of the covariance matrix are easier to
use than the formulae given in Chapter Two.
Consider the symmetric quadratic form 8
= Y'A! where!
-
N(~,
Lancaster [22J has shown that the variance of 8 may be given by
var 8
If 81
= Y'A1!
2 tr[ (VA)2 J + 4~' AVA~
=
and 8
2
= !'A2!,
(4.2.1.1)
then
cov(Sl,8 2 ) = 2 tr[ VAl VA 2 J + 4V' A1VA2~
(4.2.1.2)
V).
56
In the following the matrices A. are all symmetric, and moreover each
.
J.
row and column sum to zero.
Also
~ ~ ~1.
These conditions imply
that (4.2.1.1) and (4.2.1.2) may be simplified to
var S
2 tr[ (VA)2 ]
~
(4.2.1.3)
and
(4.2.1.4)
~
The A. matrices have n rows and n columns where n is the number
J.
of observations.
What follows is a presentation of the matrix A. which is deJ.
fined so that the quadratic form S.
J.
2
cr J..
.
~
is Koch's estimate of
-Y'A.Y
J.-
To find the variance-covariance matrix for the estimates of the
variance components, apply (4.2.1.3) and (4.2.1.4).
4.2.2.
Nested designs.
The nested designs which are considered
are the following:
a).
a).
one-stage
b) •
two-stage
c).
three-stage
d).
r-stage
One-stage nested design
The model equation for the one-stage nested design is
Y.
1.
~
i~1,2,
••• ,n
(4.2.2.1)
where c/'O -is a constant and a , a ,··· ,an are independent and identi2
1
2
2
cally distributed as N~O, cr ), i.e. , NID (0, cr ). Koch's estimate of
1
1
/
57
1
2n(n-1)
Ii4:jI
This estimate can also be formed by using
X'~!
with the matrix A
1
being defined by
= -1n
=
b).
i f i=j
-1
n(n-1)
Two-stage nested design
The model equation is
i=1,2, •.• ,a
j=1,2, ••• ,n.
1.
where a
O is a constant; the {ail are NID(O,Oi) and the {b ij } are
2
NID(O'02)'
The {ail and the {b
ij
} are also independent of one
another.
2
2
Koch's estimates of 01 and 02 are
cri ~ J,
-
{[1
ni (n - ni>f
a
I
[ i=1
ni(ni _1>]-1
n.(n. _ 1)]-1
1.
1.
So
a
\
\
L..
n1.'
n.K
~L. '
L..
j=l $/,=1
\
L..
i rF k
(
YiJ' - Yk $/, )2
i~l
I
i=l
Ii Ii
j F
k
(Y .• - Y )
ik
1.J
2
58
These estimates can also be obtained by defining A
1
!' AiY
=
A2
0" 1
A2
and Y' A Y =
0" 2'
2
n - n
=
n
n.]. - 1
i
L
i=l
2
n.].
I
a
L
i=l
if i=j
2
n.]. - n
if the jth element in the Y vector
2
. t h e same c 1ass as t h
is ].n
e ']. th
n.]. - n
i=l
so that
1
=
a
2
A is def ined by
1
a
2
and A
elem~nt
in the Y vector,
i~j
- 1
=
otherwise.
a
2
n
L
-
i=l
The definition of A
a
2
i,j
2
2
n.].
is
n.].
=
a
L
i=l
-1
2
n.]. - n
- 1
=
a
I
i=l
i f i=j
].'f t h e - J. th e 1ement i n t h e
2
n.]. - n
Y vector is in the same class
. t he Y
as te
h ].. th- e 1 ement ].n
vec tor,
= 0
i~j
otherwise.
•
59
c).
Three-stage nested design
The model equation is
=
Y"
].J k
Cl.
O
+ a.,] .+]b"
. J+
] .COOk
J
(4.2.2.3)
i=1,2, ••. ,a
j=1,2, ••• ,m,
].
k=1,2, ... ,n ..
].J
where
Cl.
2
the {b } are
O is a constant, the {a]..} are NIDtO, 0),
1 mi
ij
a
2
NID(O, 0;) and the {c].'J'k} are NID(O, °3 ).
n].. =
Koch's estimates are
~z ~
~
=
{[
f
n. (n - n.)
'1].
].=
].
J,-l
a
a
m.
m
1:
I
1.
L L
0z
.
=
~[L:
I
{
i=l j =1
n .. (n i - n.,)
1.J
].J
1
n1J
.. n.1.U
m.
1
I:
t
[
~
1J
1.J
I
v
u
~
#
(Y"
1J k
~i ~ij ~ij
L
i=l j=l
L
L
k # v
.
A2
2
2
)
2
'(Y
- Yiuv )
iik
-
(Y
- y .• )2
1.JV
1f
)
ijk - Y••
1JV
2
These estimates can also be formed by defining AI' A ' and A3 so
Z
th8.t .!'A1.! :::; 01 and .!'AZY = ~2' and .!'A~ =
uv
1U
n,. n ..
1J 1J
mt
L
(Y .• k - Yt
n., n.
1
I L L I
i=l
k
j
2
(Y"1J k- Y.1UV )2}
k=l v=l
m.
a
m.].
1 a
J- i=l
I j=l
Ln..
(n., - 1)
I L
].J ].J
i==l j=l
a
I
ILL
L:
j f:. u
J-
I
j=l u=l k=l v=l
i f= t
a ]m,
.
n].'J' and n = In].. •
i=l
n .. n t
1J
U
t
m.1 m.1
A2
I
j=l
83 ,
60
Al is formed by taking
n
n - n.
a1
. j
J.,
J.
=
a
2
n
-
a
2
L
i=l
2
L
i=l
ui
u.
J.
-1
=
n
2
-
i
n ..
J.J
if i = j
m.
a
J.
2
- L L
n ..
J.J
i=l j=l
i f the jth element of the Y
a
2
n.
- 1:
vector is in a different class
J.
i=l
.th
at the first stage than the
i :f: j.
element of Y,
1
=
.th
if the
a
2
1:
n.
i=l
J.
-
J.
J
element of the Y
nl.
a
J.
I I
i=l j=l
2
n ..
vector is in the same first
J.J
stage class but in a different second stage class as
.th
the
=
A
2
J.
element of
!,
i
:f: j.
otherwise.
0
is defined by
-
n.
a
2
i,j
J.
=
a
I
i=l
2
n.J.
n
-
n ..
ii
a
Ii
l]
m.
Ij
J.
a
2
n ..
1
if i = j
m.
J.
L L
J.J
-
i=l j=l
2
n ..
J.J
-n
-1
J.·f
=
\'
2
L n.J.
- I
t h e J.th e 1ement
.
0 f _Y J.8
2
L n J.J..
in the same first stage class
but a different second stage
class as the i
i
f.:
j
th
element of Y
61
1
=
if the jth element of Y is in
I n~o
~J
- n
the same first and second stage
classes as the i
th
element of
Y,iroj.
=
The definition of A
a3
i,j
otherwise.
0
3
is
no
=
0
~J
I I
i
j
-1
2
no
0
~J
if i ::: j
-
n
Of
~t
:::
h e J th e 1 ement
0
0f Y
0
~s
0
~n
the same first and second stage
classes as the i
th
element of
Y,if=j.
=
d).
0
otherwise.
R-stage nested design
The model equation is
(4.2.2.4)
::
i
i
1
2
i
r
::: 1,2, ... ,m
::: 1,2, ... ,m
o
~1
::: 1,2, ... ,m.
0
~1~2'"
i
r-1
62
m.
m
n
- L
i =1
1
n..
1.1~Z·
where
CX
n.
~1
,
~1
= I
i =1
2
n.
~1
.
=
• "~r-1
n..
~1~2
,
n ..
~1~2
m.
1. i
1 2
= L n...
i =1 ~1~2~3
3
,
...
mi i """t
'i' 1 2
r~l
L n . ',. • ••.
i =1
~i12
1. r
r
2
a constant, and {bi }- are NID(0,cr 1 ), {b . . }- are
~1~2
o is
2
'
1
2
NID (0 ,cr ) , •• ", and {b. i
.}- are NID (O,cr ).
Z
~l' Z···~r
r
Koch did not pre-
sent his estimators for this case as the extension is obvious.
I\Z
The Ai matrices, i = 1,2, ••• ,r that make 8 = Y'A ! = cri
i
i
= 1,2, ••• ,r
(Koch's estimate) i
- n.
n
a
l
i,j
n - n ..
~l~Z
~1
=
n
m
2
I
Z
n.
-1
n
"
L.
=1
1
m
i
1
I=1
2
n ..
L I
i
1
i
~l~Z
2
·f t h e J. th e 1ement
- I n.Z~1
1
2
n.
1.
1
if i=j
~
m
i
I
~1
i =1
1
=
are defined as
.J.
0 f _Y·~s
i
th
element of -Y
,
if the jth element of
!
is
in the same first stage class
but a different second stage
-Y ,
0
a
different first stage class than the
class that the i
=
.
~n
otherwise.
th
element of
63
n. . - n . . .
___~-=1~~-=2_ _~~1;::.~~2=-~....::3::.-.
=
L n~
1
L
-
-1
=
i f i=j
if the jth element of Y is in
-I L
the same first stage class but
a different second stage class
than the i
element of Y,
~f'
~
t h e J.th,e 1emen t
1
=
th
- LI I
0 f _Y
.
~s
in the same first stage and
second stage classes but a
different third stage class
t h an t h e ~· th e 1 ement
•
=
0
otherwise.
a _ . . ==
r 1 ~,J
i f i=j
- n
f _Y,
0
64
-1
=
if the jth element of Y is
in a different r_1
th
stage
class but in the same r-2
stage classes before as the
i
1
}: }: ... }:
th
element of
!, i~j
if the jth element of Y has
- n
the first r-1 stage classes
. common W1t
. h t h e 1.th
1n
element of
!'
in different r
but they are
th
stage
classes, i#j
=
0
otherwise.
i f i=j
=
-1
if the jth element of Y
- n
has the first r-1 stage
classes in common with the
i
th
element of Y, but they
are in different r
th
stage
65
classes,
=
0
4.2.3.
i~j
otherwise.
Classification Designs.
The classifications designs
which are considered by Koch are the two-way classification and the
two-way classification design with sub-sampling.
What will be con-
sidered here is the two-way classification design with sub-sampling
as that encompasses the two-way classification design.
The formulae
for the elements of the 'A' type matrices are easily generalized to
the r-way classification but the formulae are just too long and
complicated to be given here.
a).
Two-way classification design design with sub-sampling.
The model equation is
= <x + r. + c. + (rc) .. + e
Y..
o J.
ijk
J.J k
J.J
J
i=1,2, ••• ,r
(4.2.3.1)
j=l,2, ••• ,c
c
r
.. , n .j = L n ..
n i • = . L1n J.J
i=l J.J
J=
where
<X
o is
NID(O,cr~),
r
n
::
c
k=1,2, •.. ,n ..
J.J
L L n ..
i=l j=l J.J
a constant and the {r } are NID(O,cri), the {c } are
j
i
the {(rc)ij} are NID(O,cr;) and the {e ijk } are
NID(O,cr~).
The {r.}, the {c.}, the {(rc) i . }, and the {e ..k} are also mutually
J.
J
J
J.J
independent of one another. The estimators of the variance components given by Koch are
66
o=
2
1
2
i=l
~ {[2
n - r n.2
2
r
~"
r
2
c
- 1: n" j +
L
-1
2
c
LL2
J
J Ir 2 2 LL
i=l j~u k v
2 -1
- [ ·2 n."· - 2
2 n..
i=l ~
i=:l j=l ~J
2
;3 =
~{[L n~"
~
+ [2
+
n~.J
2 Ln~. J. -1 LLLk2 I
-
~J.
~=
I n~.J
~J
- 2
-1
(Y
ijk
- y
ij
Jru
v
Ii~tI Ij 11
k v
2 In~J2.J-1 Li~tI 2 L I L
j
1'~
u k v
.
2 L 2 (Y .. k- Yt )
i~t j u k v
~JUV
Ln. .
i=l j=l ~J
j=l
c
r
2}
(Y .. k - Y. )
~J
~uv·
(Y ' _ Y. ) 2
i Jk
~uv
(Y .. k - Yt . )2 ~J
JV
[n2-Ln~.-2n~.J
~
1
(Y"k-Y
~J
t
uv
)2 -
[I I n:.JIII
~J ~ Ii j k~
)2 }
LLLI
i j k~v
(Y"
~J k
- Y.. ) 2
~JV
These estimates can also be obtained by defining AI' A , A3
2
"'2
',2
"'2
"'2
and A4 so that Y'A1Y = aI' !'AZY = 02' Y'A3! = a 3 , and Y'A 4! = (54"
Al is defined by
2
67
n
i,j =
~
2
n .,.
=
n.
~
n . +
1..J
\' 2
n..
.n 1.....
n
• . . . . ij
1J
,2
\' 2 . \\' 2 .
In.
. 1.
L ni. - ~n'.j+LL n ij
-1
2 . \' 2 . \' 2 . \\ d2
n .+ 'LL u , .
n .,. Ln
.... L..J
. 1.
.. 1J
1'f
t he
J.th e 1 ement
#-
0
th
1,
element of
same row but a different column
th
element of
1, i #- j
otherwise
The definition of A
2
i,j
.
1n
a
j
than the i
n
0 f y '1S
if the jth element of Y is in the
1
In:
. 1. - lIn:.
. 1J
=
j
different row and a different
i
=
=
.,.Hn..
., 1J
column than the i
=
if i
2 .
is
n. ~ n . + n ..
1 . .J
1J
2
2
11 - In:
- In ..+
lIn~1J.
. 1.
J.
~
n . - n ..
.J
1J
1. f t h e J. th· e 1ement
if i
0
= j
f ,_1S
Y' 1n
. a
different row and a different
column than the i th element of
i
=
1
1,
#- j
if the jth element of
!
is in the
same column but a different row
than the i th element of
=0
A is defined by
3
otherwise
1, i #-
j
68
i,j
=
n.
- n ..
J.J
1. •
I n~.
+
---'''"---~'----
2
n ..
-
n - n.J. • - n • j +n J.J
..
-II
J.J
n .. - 1
J.J
if i=j
I I n~.
J.J
- n
1
=
if the jth element of Y is in the same
2
row and same column as the i
n .. - n
J.J
of Y,
=
-1
-------..,..
th
element
i~j
if the jth element of
!
is in the same
row but a different column than the i
element of Y,
-1
=
i~j
if the jth element of Y is in the same
2
2
L n 'J. - I L n J.J..
co 1 umn b ut a d J.· ff erent
element of
=
th
1
---.---------
!'
ro~
h
tan
th
e 'J. th
i~j
J.• f' t h e J. th e 1 ement
0 f Y"J.S
J.n a d'J. ff er-
n2_\n~
_\n2.+\\n~. ent row and column than the i th element
L J.' L • J L.L J.J
of !' i~j.
A
4
is defined by
a
4 i,j
=
-
1
n ..
J.J
-
n ..
J.J
II
2
i f i=j
n
69
-1
=
if the jth element of Y is in the same
2 -n
n,
.iJ
row and the same column as the i
element of Y,
=
o
th
i~j.
otherwise.
If there is only one element in each cell, that is, no sub-sampling,
' not de f'~ne d an d A2
, actua 11y an
t h en A4 ~s
G ~s
3
'
est~mate
0 f 032
+ 04'
2
4.3 ANOVA estimators
4.3.1.
Introduction.
The formulae presented here are limited
to the models which are used in the comparison of the various estimationprocedures.
The two and three-stage nested models and the two-
way classification model with sub-sampling under the assumptions of
Model II are considered here.
The two-way classification design with
sub-sampling is also considered under the assumption of a mixed model.
The particular ANOVA technique considered for each of the above designs is as follows:
Random-Effects Model
a).
Two-stage nested design - Henderson's Method 1
b).
Three-stage nested design - Henderson's Method 1
c).
Two-way classification with sub-sampling - Henderson's Method 1
Henderson's Method 3
Mixed Model
a).
Two-way classification with sub-sampling - Henderson's Method 2
70
4.3.2.
a).
Random-Effects Model
Two-stage nested design
The model used here is the same one that is defined by (4.2.2.2.).
Crump [8] has presented the formulae for the variance of Henderson's
Method 1 estimates of a
2
l
The formulae are
4
= 2 a2
_2
var (12
n-a
_2
var a
l
0; {
2
=
')
....
nO
=
(a-I)
3J +
_1
i=l
n.
w~
,
~
1 + pn.
1
r
1
n.
a
2
I
- -n
where w.
1
and
2
l
1
n-a
a
2c
(l
n2 i=l
=
w.
)2
1
2
n;
1
+ I -2-·
i=l w.1
a
}
2
p
2
n.1
al
a22
,
and nO
= _l_(n-'In~)
a-I
n
The covariance of a-2
l
,.2
cov ( 01_'
b).
~22)
v
=
Three-stage nested design
The model used here is defined by (4.2.2.3.).
Searle [31]
gave the formulae for the elements of the variance-covariance matrix
of the estimates defined by Henderson's Method 1.
To aid in the
presentation of the elements of the variance-covariance matrix,
Searle defined several expressions involving the n .. 's.
1J
--
71
m,
Let
a
k
1
=
2
l
i=l
n,
~.
In
m,
k3
=
2
a,~
LIn"
i=l j=l ~J
m.
kS
=
k 12
~
I.
=
In
k4
=
l
n.,
k6
~J
J -::;.....-i=l.....
~.
m.
1
L L
3
n, ,
~J
i=l j=l
3
l.
L
'=1
n"
~J
n,
1=1
a
2
~
l
j=l
a
Z Z
(I. n,,)
, ~J
J
i=l
n.
= L
no;' •
\'
a
~.
~
\'
a
k
7
2
n,,)
(I.
j
= L
Z
n. (\'n~,)
~. ~. ~J
J
~J
i=l
2
ni ·
a
k
t
2
=
9
k
b
n,
~.
i=l
3
a
3
= L
[n(k
t
t
-e
4
s
= [(n
- k
+
(k
=
(n - k
=
(n - k
12
12
1Z
)
l.
+ k 3 (n - k12 )2 J + (n - k 3 ) 2 k 7
- k ) k + (n - k ) k In]
[(k
12
3
6
12
S
In
)Z (n - 1) (a - 1) - (n - k )2 (a - 1) (b - a)
3
- k )
3
12
m,
Z
+ 2(n - k12 ) (k - k 3) k 4
1Z
3
L
i=l
- k )
3
1Z
- Z(n - k )
3
t
=
2
Z
(n - 1) (b - a)J
[k
3
I(n -
(n + k ) - ZkS/nJ
1
) 2 (n - k )
1
Fsing the above expressions one has
b)
72
-2
var 03
=
2
°43
n - b
cov
- k ) (b - a)/(n - k ) - (a - 1)J •
[(k
12
3
12
cov
var
a;
/(n - k )
1
4442222
2(~1 1 + ~2 °2 + ~3 °3 + 2~4 °1 °2 + 2~s °1 °3
°
var
0~ = [2(k 7 + nk 3 - 2kS)0; + 4(n - k12 )0; a;
+ 2(b - a) (n - a)o~/(n - b)J /(n - k12 )2
2 2 4
cov (01'02) = {2 [kS - k 7 + (k 6 - k 4)/ nJ 02
+ 2 (a - 1) (b - a)o~/(n - b) - (n - k12 ) (k 12 - k 3)
. var -022} /[(n - k ) (n - k12 )]
1
c).
Two-way classification with sub-sampling
The model equation for the two-way classification with subsampling is given by (4.2.3.1.).
Searle DOJ gave the formulae for
the elements of the variance-covariance matrix of the estimates
defined by Henderson's
Method 1.
The method presented by Searle is the following:
let S
=
_2
°1
_2
°2
-2
°3
H
=
1
0
0
-1
0
1
0
-1
-1
-1
1
1
73
-
1
C -
1
TZ.
k
k
12 - 3
1 k,.L 2-k"...
k -k
n-k
k -k
21 1
2
21 3
n'-r-c+1
T3
k1-k
r
=
m
t
T
1
=
n-k
P '"
41
k 2-k:t2
n-k12-k21-k3
T
4
r
T
1
I
=
uncorrected sum of squares for rows
n.
~.
i=l
c
T =
Z
I
n
j=l
~2
uncorrected sum of squares for columns
Y. j ' ,
'j
uncorrected sum of squares for interaction
T4
~2
=
n Y ••• ,
sum of squares for the mean
where
n'
=
k
=
the number of non-empty cells
r
l
k12 =
2
n.
I
i=l
Ii
~.
Ij
c
k
In
n
2
=
I
2
n .In
j=l
2
and
ij
ni •
k 2l =
k
'J
Ij Ii
3
=
2
n ..
~J
n
'j
Henderson's Method 1 estimators are defined by
P S
'e
2
I L n . .In
i j
~J
74
~2
~2
~2
~2
Since 04 is independent of 0'1' 0'2' 0'3
~2
(4.3.2.2.)
2 0'4
n - n'
=
var 0'4
Therefore to find var
are defined next.
Let a
= 2 {~L [~L.....
n~J'
i
(I
j
~J
i
(L
II
var T
4
=2
~
L
i
= 0'4'2
and e
2
b
~J
n
l.J
n
L ni .
i j
! whose elements
a+ ni.b
+ 11,1.J
•• d + e)/n. ] 2
J
1. •
(n .. a+ n
11, ••
j#j'
var T3
v~r
2 }
n i · n.~ ,
n: n ..
j
+
~•
n .. n i , j)
J.J
i:/=i'
var T = 2 {L~
2
222
= 0'1'
b = 0'2' d = 0'3
(11,.
j
II
+
it is necessary to find
~
J
n
'j
[n'
~j
ij
,)
.,
•j
b
+ n l.J
.. d +
e) In. .1
2
J
2
')
a<"}
'J
(a + b + d + e/n
ij
)
2
+
(n . - n .) a
i
iJ
2
75
2
n ..
= 2 L: L:
~J
. . n. n .
~ J
~ •• J
(n
i
•
Ii jL n ~J~.
.. (n. a
a + ti .b
.J
2
LL
n:.
i j
~J
e)2
+ n. jb + n .. d + e) 2
+
~J
1
n
b
+ n.. d +
i.
(n . - n .. )
~J
~J
•J
n.
~.
=2
L
n ..
I
~ (n. a + n .b + n .. d + e) (n. a +
. . n
~.
.J
~J
~.
~
J
Ln
L n~.
.n ..
..
.J ~J
b ....
J~~_
+
n.
d j
~.
n.
J-J
+
e)
~.
LIn .. (n.ja + n .b+ n .. d + e)2
. J.
2
1J
1
•J
n .
•J
~
..1J
+
.
a4 \' \' 2
L L n .. (n.
- n .. )
i j
~J
~.
1J
n
.j
.
I.
n..
= 2 L L~
i
(n. a + n .b + n .. d + e) (a
n
j
1.
•J
2
L: n ..
d
•
l.
l.J
n
.
+
e)
•J
n
'e
1J
1
n.~. n i J·
n. j
+ bn .
.J
+
76
Bush and Anderson IS] also considered this model and they found
the variance-covariance matrix for the estimates defined by Henderson's
Method 3.
.
Note that Bush and Anderson's method is limited to con-
nected designs.
They used the same technique as Searle did for the
two-way classification.
S* =
"'2*
,
0'1
Let
H*
/\2*
"2*
a
1 -1
a a a
0'3
=
1 -1
-1
0'2
p*
a -1
a
n-k
a
n-k
n-k
a
a
n-k
21
T1,T2~T3,k12,kZl'
n' and
lZ
lZ
r-l
T1c =
c-l
1
21
m=
n
T
I
T2
T
3
T~
rc
-k*
I
-k)'c
1
k*
1
~
are defined here exactly the same as
Searle's definition for them.
and
n
rc
n'-r-c+l."
Since T , T ' T are the same as above, then the formulae for
l
Z 3
their variances and covariances are also the same.
formulae will not
appear here.
In order to find
Therefore, those
the variance of TZ
and the covariances which involve TZ, Bush and Anderson defined several
expressions involving the n" 's and these expressions are
lJ
k*1 = tr I
c'rc
Crc Brc
J
k* = tr {rC' C B J2}
Z
rc rc rc
77
k*r
=
tr I
c'rc Cru Bru c'ru Crc Brc J
k*c
=
tr I
c'rc Ccu Bcu c'cu Crc Brc J
k*s
=
tr I
c'rc A'A arc Brc J
freq~encie$
where A'A is a diagonal matrix whose elements are the cell
and B~ is defined as I
ai
(A'A)-l c~ ]-1.
Y is defined to be the
estimated population cell means.
The C' matrices all have n' columns with each column corresponding to an occupied cell.
c'
ru i,j
a'cu
C' is a r x n' matrix defined by
ru
= n ik
if the cell (i,k) is the jth occupied cell
=0
otherwise.
is a c x n' matrix defined by
c'
cu i,j
.th occupied
the cell (k,i) is the J
cell
= ~i
if
=0
otherwise.
c'r and c'.
c
To define C it is necessary to first define
rc
C' is a
r
(r-l) by n' matrix in which each row is a contrast between two of the
r rows in the r x c classification.
If any cell is empty, the column
associated with the empty cell is omitted.
C' is a (c-l) by n' matrix
c
in which each row is a contrast between two of the c columns in the
r x c classification.
C' is also a matrix of contrasts and these
rc
contrasts are determined by multiplying corresponding elements in the
rows of
a'r
and
ct.
c
By taking all the (r-l)(c-l) combinations of rows
of C' and C', the rows of C' are formed.
r
c
rc
When some cells are empty,
some of the combinations of the rows of C' and C' do not form conr
c
78
trasts and hence for these combinations there is ~o row in C;c'
To
understand better the formation of these C' matrices, the example·
which Bush and Anderson used is presented.
Consider a 3 x 3 classi-
fication:
~
2
1
3
Total
13
nl.
23
n2•
33
n3 •
Rows
1
nIl
n
2
n 21
n
3
n
n
Totals
Let nIl
31
n. 1
= 0,
n
12
n
12
n
22
n
32
n. 2
= n 13 = n 22
:::: n
n. 3
23
= n 31 = n 32
n
:::: 1, n
2l
= 11 33 =
2.
Therefore, the C' matrices all have 8 columns as there are 8 nonempty cells.
C'
ru
C'
cu
C'
r
=
=
=
1
1
0
0
0
0
0
0
0
0
2
1
1
0
0
0
0
0
0
0
0
1
1
2
0
0
2
0
0
1
0
0
1
0
0
1
0
0
1
0
0
1
0
0
1
0
0
2
~
1
0
0
0
0 -1
0
1
1
1 -1 -1 -1
-1J
Note that here the first row of C' is a contrast between row 1 and row
r
3 with the column corresponding to cell (3,1) having a zero coefficient
79
= O.
since nIl
2 and row 3.
The second row of C' refers to a contrast between row
r
C' is formed in the same manner except now one is inc
terested in contrasts between columns.
C'
c
=
0
1
0-1
1
[ 1 -1
o
1-1
o
0
o
-lJ
1 -1
The first row of C' refers to a contrast between column 1 and column
c
3 and the second row of C' refers to a contrast between column 2 and
c
column 3.
C'
rc
=
1 -1
0
0
0
0 -1
o
0
1
0 -1 -1
o
0
0
1 -1
1
0
1
0 -1
1
Row 1 of C' is formed by the combination of row 1 of C' and row 2 of
rc
r
ct.
c
Row 2 is formed by the combination of row 2 of C' and row 1 of C'
r
c
and row 3 is formed by the combination of row 2 of C'r and row 2 of C'c'
The combination of row 1 of C' and row 1 of C' does not cause a row in
r
c
C' to be formed as this combination does not result in a contrast.
rc
Having formed all the above expressions, the formulae for the
variance of TZ and the covariances involving TZ are
*
cov(T , T4)
l
4
2 k* 0'3
r
coV~T2 ,Tt)
4
= 2 k*c 0'3
cov(T 3 ,TZ)
= 2 k~ O'~ + 2 n rc 0'44 + 4
k~~
2
2
1 0'3 0'4
80
Having the above formulae enables one to use the method of Searle
given in (4.3.2.1) and (4.3.2.2).
4.3.3.· Mixed Model.
Blischke 13] found the variance-covariance
matrix for the two-way classification model with sub-sampling.
worked with estimates defined by Henderson's Method 2.
He
The model equa-
tion is
(4.3.3.1)
i=1,2, ... , r
1, 2, ... , c
j =
k = 1, Z,
n ..
1.J
· } are NID(O,cr 2 ),
are constants, {r } are NID(O,cr 2
), {e
Z
l
i
ijk
Yj
where
...,
and the {r.} and {e. 'k} are mutually independent.
1.
= ~
notation for subscripts shall be used, e.g. n.
1..
n
=~
~
1. J
The usual 'dot'
1.J
n...
J
n .. , n . =
1.J
.J
~
1.
n .. ,
1.J
Blischke applied the method developed by Searle, i.e.
1.J
-p
var S = P
-1
-1
"2
m var cr
"z ]
z
[H var t H' + m m' var cr
1'2
where S = aI' P = n - k , H
3
=
(1,-1), m
=r +
(4.3.3.2)
z
1
(4.3.3.3)
(P- )'
K
r
- K - 1
f
'
t
=
4
k
3
= !n
I•
Z
n.
1.
1..
and
var
Z cr
z
cr"z = n-r-c+l
z
-~-7"
•
To find T and T , Blischke defined the following:
r
f
a C matrix with
(c-l) rows and (c-l) columns with the matrix being defined by
c .. ,
JJ
= n .J.
r
I
2
n ..
21.
i=l nt.
ifj=j'
81
r
n .. n .. ,
L
= _
1.] 1.] .
'.1
1.=
where j
z,]
= 1,2,
= X
... , (c-l) and j'
_I
•j .
, ,
Tf
= Y~
-
f
n1.']'Y]'
-1
c-l
r
I I I
j=l j'=l i=l
= 1.. I
n, ,n" ,c
1.J 1.J
n,
1..
c'tl
L
n j~l j'=l
Then T =
r
jj'
jj'
n ,n " c
.J.J
Then H var t H' = var T + var T
-
•
Let
c-l
K
I
is the j,j'-th element in C
c-l
r
••• , (c-l).
.A
,
/ n.
K =
= 1,2,
z.] ,
I
j '=1
where c]]
j'
1.] 1. ••
n,
1..
i
= X,1. ••
j"1
n, ,X,
c-l
y,1..
if
n,1..
r
f
- 2 cov(T ,T )
r f
Ii
(Y~
/ n. ) and
1..
82
c-l c-l
jj'
,
L: noon.,.,e
L:
1J
1
J
j=l j'=l
where z .. , =
11
Wii
' = n.1. n.1
t
,
and w.
1.
z. -.
1.
I
i'
z11
.. , ., z
.*
;::;
L z.11'" ,
i
Applying (4.3.3.2) and (4.3.3.3)
enables one to find the variance-covariance matrix of the estimates
defined by Henderson's Method 2.
4.4.
Asymptotic Variance-covariance Matrix of Maximum Likelihood
Estimates of Variance Components
Searle 133] worked with the model
y=xS+Zu
where Y is a vector of n observations;
~
is a vector of fixed effects;
u is a vector of random effects; and X and Z are known matrices, often
design matrices.
The random effects are normally distributed with
'
1 ves aI'
2
·
.
. A wh'1Ch 1nvo
zero mean an d var1ance-covar1ance
matr1x
a22 ,
••• ,
a 2 • Therefore
q
var Y = var Zu = Z A Z' = V.
Searle shows that the asymptotic variance-covariance matrix of the
maximum likelihood estimates of the fixed effects and the q variance
components is
yc:@.,£ )
...
x'
y-l X
o
where T is defined by t .. = tr· {V- l V. V-I V.}, and V. and V. are the
1J
1
J
1
J
2
2
partial differentials of V with respect to a. and a ..
1
J
CHAPTER V
COMPARISON OF ESTIMATION PROCEDURES
5.1.
Criterion
One of the purposes of this study is to compare several pro-
cedures (Koch, ANOVA, and maximum likelihood) for estimating the
components of variance.
The first decision that must be made in doing
the comparison is what will be the criterion for saying one procedure
is better than another procedure.
Bush and Anderson [5] compared pro-
cedures and designs for the two-way classification and they used the
variance of the variance component estimate.
Goldsmith [12] compared
three-stage nested designs and he used the trace, the adjusted trace,
and the determinant of the variance-covariance matrix as his criteria
for comparing the designs.
The adjusted trace is the same as the
trace except that each element involved in the sum has been scaled by
the size of
the variance components involved.
Goldsmith came to the
conclusion that the trace performed best as a criterion for determining
which design was better.
Another possible criterion results from the
subtraction of the variance-covariance matrix of one set of estimators
from the variance-covariance matrix of another set of estimators.
If
this difference is positive (negative) semi-definite, one is able to
discuss the relative merits of the sets of estimators.
This pro-
cedure was tried for several designs and the resulting difference
matrix was never positive (negative) semi-definite.
84
Based on the above, it was decided that considering the variance
of each estimator was the most relevant criterion for this study.
5.2.
Procedures to be Compared
The estimation procedures to be compared are the following:
1) modified Koch
2) ANaVA
a) Henderson's Method 1:
use
the sums of squares from the
usual analysis of orthogonal data and equate these sums
of squares to their expectations which were obtained
under the assumption of a random model.
b) Henderson's Method 2:
adjust the data for the fixed
effects by obtaining the least squares estimates for the
fixed effects, and then, using the adjusted data, apply
Method 1c) Henderson's Method 3:
use some conventional method to
compute mean squares for the non-orthogonal data, and
then equate these mean squares to their expectations and
solve for the estimates.
3) maximum likelihood.
The particular type of ANaVA estimator to be compared is a function of what design is being considered.
This results from the fact
that there is no general theory or method which has been applied to
the ANaVA technique of estimating variance components.
To find the
variances of the
k~aVA
estimators for any design requires a formidable
amount of work.
Therefore in this comparison one is limited to de-
signs for which the variance formulae have already been derived in the
85
·literature.
The following table shows which estimators are compared.
The table also shows for what models the estimators will be compared •
. .RANDOM 'MODEL
Model
Two-way
Classification
Estimation
Procedure
ANOVA 1
I
ANOVA 2
*
ANOVA 3
Modified Koch
Maximum Likelihood
Two-way
with Subsampling
Two-stage
Nested
Three-stage
Nested
I
I
I
I
I
*
I
I
I
I
I
I
. MIXED MODEL
Model
Two-way
Classification
Estimation
Procedure
ANOVA 2
Modified Koch·
Maximum Likelihood
I
I
I
Two-way with
Sub-sampling
Two-way with
Nesting
x
I
I
I
I
The I represents the models for which the estimation procedures are to
be compared.
The
* indicates
that Method 2 in these instances is the
same as Method I as Blischke assumed the mean to be zero.
The X indi-
cates that Method 2 could be compared except for the fact that
Blischke did not allow for the interaction term to be random.
5 .3. .Designs
Twenty-one different designs were considered for the random model
86
and five different designs for the mixed model.
For each of the
designs, different combinations of values for the variance components
were considered.
Ninety-four different combinations of designs and
values for the variance components for the random model were investigated and 18 different combinations were considered for the mixed
model.
It is hoped that by covering a wide range of designs these comparisons would give some idea of the relative merits of the various
estimation procedures over the whole spectrum of possible design configurations and not be limited just to the designs that have been
investigated here.
The designs which were studied are presented in 5.3.1 and 5.3.2.
The model equations will be presented again, but not the underlying
assumptions as these are given in Chapter IV.
5 •.3.1 'Random:-Effec::ts.Model
TWO-WAY CLASSIFICATION
The model equation is
y ..
~J
=
e ..
~J
i
= 1, 2,
j
= 1,
. .. , r
2, ... , c
and the assumptions about this model are given following (4.2.3.1).
For this model, four combinations of values for the variance components and eight different designs were considered.
The different
2
combinations of values are as presented in the following table (aI'
a 22 , an d a32 are
.):
g~ven
87
•
a1
2
a22
a32
1)
49
49
7
2)
49
1
7
3)
1
49
7
4)
7
7
7
The eight designs considered and the parameters are the following:
r
1)
3)
= 7, c = 8, n
S-type Design
= 26
2)
L-type Design
1 110 0 000
110 0 0 0 0 0
1 1 1 100 0 0
110 0 0 0 0 0
011 1 1 0 0 0
110 0 0 0 0 0
o0
1 1 1 1 0 0
11000000
00011 110
110 0 0 0 0 0
000 0 1 111
11111 111
0000011 1
1111111 1
Random Design
4)
L-type Design
110 0 000 1
1 0 0 0 0 0 0 0
011 1 1 100
100 0 0 0 0 0
111 0 110 1
10000 0 0 0
10011010
1 0 0 0 0 000
01110101
1 1 1 1 1 110
o0
o0
0 0 0 1 1 0
1 111 1 110
000 0 1 0
11111 111
88
r
5)
7)
==
7, c
==
8, n
==
36
S-type Design
6)
L-type Design
11110 000
1 110 0 0 0 0
111 110 0 0
1 1 1 0 0 0 0 0
1 1 1 1 1 1 0 0
1 1 1 0 0 0 0 0
o1
o0
o0
o0
1 1 1 1 1 0
1 1 1 0 0 0 0 0
1 1 1 1 1 1
1 1 1 1 1 1 1 1
0 1 1 1 1 1
111 1 1 1 1 1
0 0 1 1 1 1
1 1 1 1 1 1 1 1
Random Design
8)
L-type Design
1 0 0 0 110 1
110 0 0 0 0 0
10101110
110 0 0 0 0 0
11101011
110 0 0 0 0 0
010 0 1 1 1 1
1 1 1 1 1 100
11101010
1 1 1 111 1 1
1 0 1 1 1 1 1 0
1 1 1 1 1 1 1 1
1 1 1 100 1 0
1 1 1 1 1 1 1 1
TWO-WAY CLASSIFICATION WITH SUB-SAMPLING
The model equation is
Y"J..J"k ==
0.
+ e J.."jk
0 + r"J.. + c.J + (rc)"j
J..
i == 1, 2, .• " , r
j
== 1, 2,
k == 1, 2,
and the assumptions are given following (4.2.3.1)"
, c
, n."
J..J
Four combinations
of values for the variance components and three different designs
were considered.
The different combinations of values are
89
2
0'1
2
0'2
2
0'3
2
0'4
1)
36
36
6
6
2)
36
1
6
6
3)
1
36
6
2
4)
6
6
6
6
The three designs and parameters considered are as follows:
r
1)
= 7,
c
= 8,
n
= 43
S-type Design
2)
3)
L-type Design
a
Random Design
21110 000
1 1 1 0
000
1 0 0 0 1 201
1 2 1 1 1 0 0 0
1 1 1 0 0 0 0 0
2 0 1 0 1 110
11211 100
2 110 000 0
21102011
o 112
1 1 1 0
1 2 1 0 0 0 0 0
010 0 1 2 1 1
a 0 112 111
1 2 1 1 1 111
11101010
00011 2 1 1
1 2 2 2 1 111
2 0 1 1 1 110
o 000
111 1 2 1 1 1
1 1 1 200 1 0
1 121
TWO-STAGE NESTED
The model equation is
y ..
1J
= aO +
a.
1
+
b ..
1J
i
= 1,
2,
j
= 1,
2,
and the assumptions are given after (4.2.2.2).
...
...
,
a
,
n.
1
Seven different
designs and three different combinations of the values of the variance
90
components were investigated.
The different values of the variance
components are
2
2
C\
°2
1)
7
1
2)
1
1
3)
1
7
The designs considered are as follows:
1) n
= 25, a =
3, n
2) n = 25, a = 3, n
3) n
= 25, a
=
4) n
c:
25, a
c:
5) n =
2, n
l
l
= 3, n 2 = 5, n 3
=
8, n 2
=
12, n
8, n
=
2
50, a = 6, n
n
l
l
l
=
2
3
=
17
= 9
= 13
= n3
25, a = 20, n = n =
l
2
6, n
6) n = 50, a = 6, n
7) n
l
=
n
n
4 = n 5 = 4, 6 = 5
= n
15
= 1, n 16 = n 17 =
= n 20 =2
= 6, n 4 = 8, n 5 = 10, n 6 = 20
n = n = n = 8, n = n
9
2
4
6
3
5
2, n
2
4, n
3
:;:
THREE-STAGE NESTED
The model equation is
i = 1, 2,
j = 1, 2,
k
=
1, 2,
a
...
...
,
m.
,
n ..
~
~J
and the assumptions about the model are given following (4.2.2.5).
Eight designs and four combinations of the values of the components
of variance were studied. The components of variance and their values
are presented in the following table:
91
0
2
0
1
2
2
2
3
°
1) 49
28
7
2) 49
1
7
3) 21
49
7
7
7
4)
7
To define the designs, consider the following structures given by
-Goldsmith 112], along with the m. and the n .. for each structure.
1
1J
Structure Number
Stage
1
2
1
I
I
2
2
2,2
2,1
2
3
m.
1
n ..
1J
~~ ~
3
4
5
n ~ t
2
1
1
1,1
2
1
The first structure is used when two second-stage samples are taken
from each first-stage sample and in turn, two third-stage samples are
taken from each second-stage sample.
Similarly the second structure
is used when two second-stage samples are taken from each first-stage
sample and in one of the second-stage samples, two third-stage sampIes are selected, while in the other second-stage sample, only one
third-stage sample is selected.
Likewise, the fifth structure is
used in a design when only one second-stage sample is selected from
each first.,..stage sample, and in turn only one third-stage sample is
selected from each second-stage sample.
--------------
The designs considered are
92
presented in the following table where the elements in the table are
the number of each structure involved in the design:
~
Number of
observations
1
2
3
4
5
1
3
0
3
0
18
36
2
3
0
9
0
6
36
3
6
0
6
0
0
36
4
0
3
0
3
21
36
5
0
6
0
3
12
36
6
0
9
0
3
3
36
7
5
0
5
0
30
60
8
5
0
15
0
10
60
number
Design
5.3.2
Mixed Model
TWO-WAY CLASSIFICATION
The model equation is
Y ••
~J
= r.~ +
y. + e ijk
J
i
1, 2,
j
1, 2,
k
=
1, 2,
·..
·..
·..
,
r
,
c
,
n ..
~J
and the underlying assumptions are given following (4.3.3.1).
Blischke assumed that the mean equals zero while the maximum like1ihood procedure and the modified Koch technique do not require this
assumption.
Three sets of values for the variance components and two
designs were studied.
following table:
2
2
The values for 01 and 02 are presented in the
93
2
0"1
2
0"2
1)
49
7
2)
7
7
3)
7
49
The designs and parameters are as follows:
r
1)
=
7, c
=
7, n
= 21
BIB Design
2)
Random Design
a0
a
1101000
11 0
all a
011 1 1 a a
a0
a0
1
100
0
1 1 0 1 0
III 0 1 1
0 1 1 0 1
1
a0
o1
0 1 1 0
010 0
a all a
all
a
1
010 1 0
o a a 001 1
a a a0
1 a 100 0 1
001
TWO-WAY CLASSIFICATION WITH SUB-SAMPLING
The model equation is
Y.1.J'k = a 0 + r.1. + y.J + (ry) ij + e ijk
and the assumptions are that
aa
i = 1, 2,
j
1, 2,
k
= 1,.2,
... ,
... ,
r
c
•.. , n ..
1.J
is a constant,· {Yj} are unknown param-
· 2 2
eters, {r i } are NID(O'O"l) , {(rY)ij} are NID(O'0"2), and {e ijk } are
2
NID(O,03).
It is also assumed that the {r.}, {(ry) .. }, and {e .. } are
1.
1.J
1.J k
mutually independent.
2
of 0"1'
·2
Two designs and four combinations of the values
2
0"2' and 0"3 were considered.
presented in the fo11mviilg table:
The combination of values are
94
f
.
2
0'1
2
0'2
2
0'3
1) 49
49
7
2) 14
49
7
3) 49
1
7
4)
7
7
7
The designs and parameters are
=
n
1)
28
BIB Design
r = 7, c = 7
2)
Random Design
2101000
2 2 1 4 1 1·
0210100
3 120 2 1
o0
210 1 0
o 321
o0
0 2 1 0 1
r = 3, c = 6
1 1
1 000 2 1 0
o1
000 2 1
1 0 100 0 2
TWO-WAY CLASSIFICATION WITH SUB-SAMPLING AND NESTING
The model equation is
i = 1, 2, ... , r
1, 2,
j
no
1.
k = 1, 2,
,
,Q, = 1, 2,
,n
c
ijk
} are NID(0,O' 2 ) ,
is a constant, {Yk} are unknown parameters, '{r
1
i
O
2 2 ·
}
. 2
are NID(O,cr 4 )·
. {s ~ } are NID(0,O'
. 2) , {(ry) i k} are NID(0,O'
'.
~
~
3 ), and {eookn
It is also assumed that the random variables {r o}, {s OO}, {(ry) ok}'
where a
1.
and {eijk,Q,} are mutually independent.
1.J
1.
One design and four combinations
95
of the values of the components of variance were investigated.
The
combinations are
..
2
0'1
2
0'2
2
0'3
0'2
4
1)
36
12
24
6
2)
36
12
12
6
3)
36
36
6
12
4)
6
6
6
6
The design and its parameters are
n
1)
31, r
~
3, c
~
5, n
1
=
1, n
2
1
2
3
4
5
1
2
1
1
2
1
1
1
1
0
1
2
2
1
0
1
0
1
c
r
n.
1
~
2, n
3
~
3
~
2
;
..
:
3
_---_. __.---_...-._----------
I 1
1
2
1
1
0
2
0
1
3
2
1
I 3
11
0
1
1
1
I
5.4.
~
I
J
Results
In this section the results of the study are presented in terms
of a type of asymptotic· efficiency.
The asymptotic efficiency of
estimate A is obtained by dividing the asymptotic variance of the
maximum likelihood estimate by the variance of estimate A.
The actual
values of the variances of the estimators are presented in the appendix .. The actual values of the variances thus also allow the comparison
of designs in addition to the comparison of estimation procedures.
96
The tables in
this section contain the 'asymptotic' efficiencies
of the various methods of estimation for the combinations of designs
and variance component values given in section 5.3.
For the models
considered, the designs are identified by the numbers assigned them in
•
the preceding section •
97
Design
Number 2
(L Design)
Method 1
Method 3
.455
.519
.649
= 49
.448
.518
.717
=
7
.010
.014
.582
2
o1 = 49
.583
.634
.582
2
o2 = 1
.018
.037
.411
o32 = 7
.030
.061
.540
Variance Component
and its value
o12 = 49
2
2
2
o3
0
2
1
1 =
2.
O = 49
2
2
7
0
3
.018
.037
.406
.616
.676
.648
.030
.063
.538
2
o1 = 7
.567
.669
.682
0
2
°2 =
7
.557
.671
.712
2
3
7
.245
.362
.563
2
= 49
1
2
0
2 = 49
2
°3 = 7
.626
.640
Since design 3
.673
.678
is not a con-
.018
.020
nected design,
.752
.752
.069
.078
.112
.127
.040
.057
.866
.866
.070
.098
0
Number 3
(Random)
Koch
=
0
2
49
1 =
2
1
0
2 =
2
0
= 7
3
0
2
1
1 =
2
0
2 = 49
2
0
= 7
0
3
Method 3 is
not applicable.
98
Design
Variance Component
and its value
Number 4
(LDesign)
.760
2
o2 = 7
.752
.765
o3 = 7
.363
.405
o12 = 49
.477
.497
.546
.599
.636
.835
.017
.020
.083
o1 = 49
.525
.551
.488
2
o2 = 1
.040
.114
.046
2
2 = 49
2
o3 = 7
2
2
3
=
7
.069
.185
.080
2
1
2
o2
2
o3
=
1
.026
.026
.039
=
49
.726
.729
.753
=
7
.045
.046
.071
2
= 7
1
.682
.702
.257
.660
.754
.220
.342
.413
.078
49
.797
.795
.896
= 49
.769
.765
.886
=
7
.023
.022
.273
o12. = 49
.845
.844
.803
0
0
0
2
2 = 7
2
0 = 7
3
0
~e
Method 3
.729
O
Number 5
(8 Design)
Method 1
2
o1 = 7
2
"
Koch
2
1
2
0
2
2
cr
3
0
2
2
2
0
3
0
=
=
1
.069
.061
.162
=
7
.122
.110
.269
99
Design
Variance Component
and its value
(J2 ;::: 1
1
cr2 ;::: 49
2
2
cr ;::: 7
3
.160
.842
.841
.799
.149
.145
.270
7
.829
.836
.624
7
.808
.822
.614
7
.429
.465
.271
.678
.717
.779
.691
.734
.830
.019
.026
.636
cr2 ;::: 49
1
.712
.733
.700
2
(J2 ;:::. 1
.037
.080
.472
(J 2 ;:::
3
.068
.140
.623
.035
.082
.472
.749
.773
.751
.064
.142
.622
7
.731
.798
.782
7
.739
.813
.808
7
.361
.514
.632
a 2 ;: : 49
.810
.811
Since design 7
.759
.763
is not a con-
.026
.027
nected design,
(J 2 ;::: 49
1
2
(J2 ;::: 49
(J 2 ;::: 7
3
(J 2 ;:::
1
(J 2 ;:::
2
o2 ;:::
3
7
1
2 =
49
2
2
a3 = 7
0.
~e
Method 3
.081
(J 2 ;::: 1
1
(J 2 ;::: 49
2
(J2 ;::: 7
3
Number 7
(Random)·
Method 1
.085
(J2 ;:::
1
(J2 ;:::
2
2
(J ;:::
3
Number 6
(L Design)
Koch
100
Design
Variance Component
and its value
2
cr 1
==
Koch
Method 1
49
.851
.851
Method 3 is
not applicable.
2
cr 2
2
cr 3
=
1
.087
.111
=
7
.152
.190
2
cr1
=
1
.133
.144
.817
.818
.223
.244
2
cr 2 == 49
2
°3 = 7
Number 8
(LDesign)
~e
Method 3
2
cr 1
=
7
.839
.845
2
cr 2
2
cr 3
=
7
.804
.828
=
7
.452
.502
2
cr 1
= 49
.666
.692
.738
2
cr 2
= 49
.761
.789
.896
2
cr 3
=
7
.024
.031
.636
2
cr 1
= 49
.679
.694
.663
2
cr"2
2
cr 3
=
1
.070
.151
.473
=
7
.124
.249
.629
2
cr 1
=
1
.042
.075
.464
2
cr 2
2
cr 3
= 49
.796
.811
.811
=
7
.075
.131
.613
2
cr 1
=
7
.755
.799
.780
cr 2
2
=
7
.788
.846
.833
2
=
7
.418
.548
.630
cr
3
101
For the S designs and for the experiments where the cells containing the observations have been selected at random, there is almost
no difference in efficiency between the modified Koch procedure and
Henderson's Method 1.
.
Method 3 is very susceptible to the value of
the variance components being estimated, and Method 3 is very poor
when the components of variance are equal.
For the L designs, Method
3 seems to be better than Method 1 or the Koch technique.
Method 1 is
somewhat better than the modified Koch procedure for the L designs
although the difference in efficiency is never greater than 0.117.
TWO-WAY CLASSIFICATION WITH SUB-SAMPLING
EFFICIENCIES OF ESTIMATION PROCEDURES
Design
Number 1
(S Design)
Variance Component
and its value
Koch
Method 1
Method 3
(52
1
(52
2
(52
3
(52
4
= 36
.762
.763
.764
= 36
.728
.729
.746
6
.109
.111
.116
6
.976
.976
.976
(52
1
(52
2
(52
3
(52
4
= 36
.848
.847
.682
= 1
.1l8
.122
.044
= 6
.349
.362
.1l4
= 6
.963
.963
.963
=
1
.101
.103
.106
= 36
.838
.837
.768
= 6
.211
.217
.187
(52
1
(52
2
(52
3
102
Design
ci4 --
Number 2
(L Design)
Koch
Method 1
Method 3
2
.993
.993
.993
Variance Component
and its value
2
(51
2
(5 2
(52
3
(52
4
=
=
=
6
.794
.806
.187
6
.767
.796
.181
6
.682
.725
.114
=
6
.967
.967
.967
(52
1
(52
2
(52
3
(5 2
4
= 36
.666
.709
.778
= 36
.662
.712
.809
=
6
.105
.152
.486
=
6
.972
.972
.972
2
1
2
(5
2
(5 2
3
(52
4
= 36
.712
.733
.699
=
1
.086
.207
.313
=
6
.283
.513
.472
=
6
.959
.959
.959
.024
.150
.414
.724
.747
.731
.134
.295
.562
.992
.992
.992
a12 = 6
.708
.805
.649
2
a2 = 6
.699
.813
.651
a"32 = 6
.647
.794
.476
a24 = 6
.963
.963
.963
(5
2
°1 = 1
2
a2 = 36
a32 = 6
(52 = 2
4
~e
103
Design
Number 3
(Random)
Method 3
Koch
Method 1
a21 = 36
.773
.778
Since design 3
a 22 = 36
.711
.730
is not a con-
a32
6
.006
.132
nected design,
a42
6
.973
.973
Method 3 is
•842
.842
.106
.171
.331
.456
.961
.961
a12 = 1
.134
.157
a 2 = 36
2
.802
.803
a2 = 6
3
.261
.309
a2
=
2
.992
.992
a2
=
6
.798
.818
a2
=
6
.733
.804
6
.682
.757
a2 = 6
4
.964
.964
Variance Component
and its value
not applicable •
4
1
2
0'2
3
In the two-way classification with sub-sampling for S designs
there is almost no difference in efficiency between the modified Koch
technique and Method 1.
Both of these methods also are somewhat better
for S designs than Method 3 which does very poorly if the variance components are equal.
For the L designs, Method 3 seems to be better than
either Method 1 or the modified Koch procedure, although Method 3 again
drops in efficiency when the variance components are equal.
Method1
104
again is a little better than the Koch estimators for L designs with
the greatest difference in efficiency being 0.23.
For random designs
Nethod 1 again is somewhat better than the modified Koch method,
although here the greatest difference in efficiency is only 0.126.
TWO-STAGE NESTED
EFFICIENCIES OF ESTIl1ATION PROCEDURES
Variance Component
and its value
Koch
Method 1
.666
.666
1. 000
.997
1. 000
.589
.613
.666
.666
.851
.999
.997
1. 000
.334
.542
.666
.672
.838
.984
.997
1. 000
.500
.500
.829
.829
.999
1. 000
.992
1. 000
.500
.500
.831
.833
.999
1. 000
.992
1. 000
a1 = 1
.500
.502
.830
.847
2
0"2
.998
1. 000
.990
.999
2
a1
2
a2
Design
Koch
Method 1
1
.582
.582
.851
=1
= 1
2
a1 = 7
2
a2
=1
2
=7
3
Design
2
4
105
Variance uComponent
and its value
2
Design Koch Method 1
Design
Koch
Method 1
6
.633
.634
.895
.895
=1
.997
.997
.805
1.000
0"1 = 1
.949
.949
.686
.706
.935
.935
.805
1.000
=1
.722
.722
.503
.780
0"2 = 7
.772
.772
.793
.986
.831
.831
.998
1.000
1
.832
.832
0"2 = 1
.998
1. 000
.831
.837
.997
1. 000
0"1 = 7
2
0"2
5
2
2
0"2 = 1
2
0"1
2
2
(}1 = 7
2
0"2 = 1
2
OJ.
=
2
2
OJ. = 1
2
0"2 = 7
7
For the two-stage nes ted designs, Method 1 does much better than
Koch's estimators in designs which are very unbalanced.
If one uses
the designs which Prairie [26J has shown to be optimal, then there is
almost no difference in efficiency between the two estimation methods.
(See Chapter VI for a further discussion of two-stage nested designs.)
106
THREE~TAGE
NESTED
EFFICIENCIES OF ESTIMATION PROCEDURES
Variance Component
and its value
Koch
Method 1
.812
.869
.918
.736
.942
.997
.997
.995
.995
.762
.763
.813
.814
.993
.895
.963
.834
.931
.931
.871
.871
.722
.855
.694
.875
.668
.801
.668
.886
.998
.998
.998
.998
7
.840
.921
.820
.921
7
.876
.936
.918
.977
7
.964
.964
.957
.957
.824
.848
.948
.949
.810
.954
.948
.949
.997
.997
.998
.998
.845
.846
.851
.854
.993
.895
.957
.985
.932
.932
.993
.993
2 :;:
49
1
2
0- 2 :;: 28
2
0- :;:
7
3
0-
Design
Koch
Method 1
1
.837
.901
.779
0- 2
:;: 49
1
2
0- :;: 1
2
2
0- :;: 7
3
2 :;: 21
1
2
0- :;: 49
2
2
0- :;: 7
3
0-
2 :;:
1
2
0- :;:
2
2
0- :;:
3
0-
2 :;:
49
1
2
0- :;: 28
2
2
0- = 7
3
0-
~e
.0- 2 :;:
49
1
2
0- :;: 1
2
2
0- :;: ]
3
3
Design
2
4
107
Variance Component
and its value
(1 2 = 21
1
2
(12 =49
(1 2 = 7
3
(12
1
2
(1
2
(1 2
3
3
Design
Koch Method 1
.820
.821
.934
.806
.807
.999
.999
.998
.998
.767
.866
.780
4
=
7
.845
.899
.856
.864
.
=
7
.936
.999
.855
.863
=
7
.976
.976
.970
.970
.910
.910
.892
.892
.961
.962
.979
.980
.998
.998
.999
.999
= 49
.822
.822
.870
.870
.
=
1
.966
.984
.971
.983
=
7
.992
.992
.993
.993
(12 = 21
1
2
(12 = 49
.849
.850
.844
.844
.864
.864
.935
.936
(12 =
3
7
.998
.998
.999
.999
(12 =
1
(1 2 =
2
(12 =
3
7
.881
.887
.879
.883
7
.891
.896
.926
.930
7
.976
.976
.982
.982
.847
.911
.779
.918
.997
.997
(12 = 49
1
(12 = 28
2
(12 = 7
3
(1 2
1
2
(1
2
(12
3
(1
~e
Design Koch "Method 1
2
= 49
1
(1 2 = 28
2 .
(1 2 ;:: 7
3
5
7
6
108
Variance Component
and its value
c1-
= 49
a;
= 28
Design
.827
.885
.736
.942
7
.995
.995
49
.827
.828
1
.963
.834
02
7
3 =
.871
.871
02
.706
.889
= 49
.668
.886
7
3 =
.998
.998
7
.836
.939
7
.918
.977
7
3 =
.957
.957
1
a; =
if-1 =
c12 =
8
Koch Method 1
21
1 =
0
2
2
02
02
1
0
0
2
=
2
2
For the three-stage nested designs, both techniques of estimation
use the same estimate of
2
estimates of 0 1 and
°22 •
°32 ,
hence the comparison must be based on the
For designs 1, 2, 3, 7, and 8, that is, de-
signs involving structures 1, 3, and 5, Method 1 is usually better
than the modified Koch method, although the Koch estimator of
better when
°12 »° 22 •
0; is
For designs 4, 5, and 6, that is, designs in-
volving structures 2, 4, and 5, there is almost no difference in the
procedures although Method 1 always has an efficiency greater than or
equal to the efficiency of Koch's.technique.
109
5.4.2.
Mixed Model
TWO-WAY CLASSIFICATION
EFFICIENCIES
O~
ESTIMATION PROCEDURES
Variance Component Design Koch
and its value
2
0"1 = 49
2
o2 = 7
1
(BIB)
2
0"1 = 7
2
0"2 = 7
2
0"1 = 7
2
0"2 = 49
Method 2
Design
Koch
Method 2
.030
.834
2
(Random)
.386
.735
.000
.571
.013
.571
.016
.721
.416
.674
.004
.571
.221
.565
.008
.521
.369
.399
.015
.571
.453
.545
TWO-WAY CLASSIFICATION WITH SUB-SAMPLING
Variance Component
. and its value
2
0"1 = 49
2
0"2 = 49
2
0"3 = 7
2
14
1 =
2
(}2 = 49
2
0"3 = 7
q
.-
2
0"1 = 49
2
(}Z = 1
Design Koch
1
.704
Design
2
Koch
.607
.313
.205
.017.
.021
.419
.544
.679
.273
.033
.040
.809
.641
.024
.075
110
Variance Component
and its value
Design
Koch
1
.042
Design
Koch
2
. 0'3
=
7
2
0'1
=
7
.477
.600
2
0' 2
=
7
.443
.298
2
3
=
7
.243
.405
0'
2
.177
TWO....WAY CLASSIFICATION WITH SUB-SAMPLING AND NESTING
EFFICIENCIES OF ESTIl1ATION PROCEDURES
Variance Component
and its value
e
"-
2
a1
2
0' 2
= 36
= 12
Koch
1
.5094
.2305
2
3
0'2
4
= 24
.2532
=
6
.0111
2
0'1
= 36
= 12
.5081
.1991
= 12
.1579
=
.0135
0'
0'2
2
0'2
3
2
°4
6
2
= 36
1
2
0' = 36
2
0'2 = 6
3
2 :=
12
4
0'
'e
Design
°
0'
2
1
=
6
.5603
.4461
.0850
.0298
.5768
111
Variance Component
and its value
2
2
2
0"3
2
0
0
= 6
Design
1
Koch·
.6357
= 6
.3565
4 = 6
.1351
The modified Koch technique does best in the mixed model when
there are several observations in each level of the fixed effect.
This
is because the estimation procedure does not form any differ-
ence of observations unless the observations are in the same level of
the fixed effect.
The results for the mixed model show this.
For
design 1 in the two-way classification, three is the maximum number
of observations in any fixed-effect level, and the modified Koch
technique does very poorly.
For design 2 in the two-way classifica-
tion, there are four observations in one fixed-effect level and there
is a considerable increase in the efficiency of the modified Koch
method.
If there are very few observations in the fixed-effect
levels, another method of estimation of variance components (instead
of the modified Koch method) should be used.
However, if there are
several observations in several of the fixed-effect levels, then the
modified Koch technique should be quite competitive (with respect to
efficiency) with the ANOVA estimates.
CHAPTER VI
TWO-STAGE NESTED DESIGN
6.1.
Introduction
In his second article on variance components, Koch [21] stated
that there exist numerous possible ways for forming estimators based
on the concept of squared differences.
The results in Chapter V show
that the ANOVA estimators have an efficiency greater than or equal to
the efficiency of the Koch estimators for the two-stage nested design.
Motivated by this finding and also by Harville [16] who has shown
Koch's estimators to be inquadmissible (i.e. there exist other estimators whose variances are less than or equal to the variances of
Koch's estimators with the inequality holding for at least one point)
in the two-stage nested design, two other ways of combining the squared
differences to form estimators shall be considered.
The variances of
these new estimators shall also be studied to determine whether these
new estimators are quadmissible.
6.2.
New Estimators
The model being considered is defined by (4.2.2.2) and the model
equation is
y ..
~~
..
= a O + a. + b ~J
i
~
=
1, 2,
j
2
EI(y .. - Yk~) J
1.J
:::;
2 a
2(a
2
i
2
2
l
+
cr~)
...
. ..
1, 2,
=
k, j
i :f:. k
i
~
,
a
,
n.
~
113
The estimates presented by Koch TZlJ are
Bi =
~
(hI
hI = I
hZ = I
h ) / Z and
Z
f n.(n-n.)
i=l
~
a
I
i=l
~
n.Cn.-l)
~
~
e~
= h Z I Z where
fi 4.Ik j=lIi f(y.. - Yk.Q,)Z
J-1
J-
a
1
n.
L
i=l
and
~J
.Q,=l
n.
Z
I~ L~ CY.. - Y.;o) •
j 4 .Q,
~J
... x,
A different set of estimators can be defined by considering a different
combination of the squared differences.
h' .
Z ,~
1
= n. (n.-l)
~
and then define
~
a
h
z
Define
= [
I
i=l
a
(n.-l) ]-1
~
L (n.-l)hz .
i=l
~
,~
By this definition the fact that there are 'a' classes in this design
has been taken into consideration.
The new estimators are then
"z
8 Z = hZ / Z
(6. Z.l)
(6.Z.2)
"2
6 2 is the same as the ANOVA estimator of
0z2
as can be seen by the fol-
lowing:
Ii (Y.. _
n.-l
.
1.
~J
~
J=
= _._2_
y. ) 2
~.
Then
-e
h2 =
1
a
I (n.-l)
i=l ~
J-
1 a
I I
i=l
2
ni
(n.-l)--l
(Y .. ~
n i - j=l ~J
I
Y.)
~.
2
]
114
a
2
=
n.
1
I
I
(y
i=1 j=1 ij
n-a
-
f. )
2
1.
1'2
1
a
n.
= h
\'
\'1 (Y .• / 2 = _
2
2
n-a. L .L 1 1J
1-1 ]=
6
Y. )
2
1.
Another set of Koch type estimators can be formed by taking a different combination of the squared differences (Y
1
n.1 n k
2
h 1 ,ik = - (Y .. - Yk~)
nink j=l ~=1 1J
I
I
i
ij
-
Yk~)
2
, i f k.
Define
f k
and next define
This new set of estimators is then defined as
~2
·e
°2
~2
°1
= h2 /
=
(h
1
1'2
2 :;: 6
2
- h )
2
(6.2.3)
/ 2
(6.2.4)
These two sets of estimators will be compared with two sets of
ANOVA estimators.
The first set of ANOVA estimators is the usual
ANOVA estimators
= MSE = 6~
(6.2.5)
= n(a-1) (l£A - MSE)
(6.2.6)
n2-Ini2
where
1
MSE = -
n-a
a
n.1
I I (Y..
i=l j=1
1J
-
f.)
1.
2
and
115
MSA ==
y2
y2
i==l n.
n
a
1
L
\'
~
a-I
__
i. - __
..
~
J.
.
2*
2
The second set of ANOVA estimators has 6
as the estimate of 02' but
2
instead of using MSA, one works with the unweighted sum of squares
between classes, that is
1
~
a
I
MSA == a~l I
... Y
(Y.
~.
i==l
1
a
I (Y.
i=l ~.
E(MSA) = a-I E[
where
Y...
a
Y.
L
=
~.
i=l
a
- Y
2
2
=
°1 +
°2 a
1
- a. 1 n.
~=
~
I
Therefore
a
~MSE
I
.. 1
(6.2.7)
an.l
~=
2
Since all four sets of estimators have the same estimator of 02' the
...
2
estimates of 01 are the only estimators which must be compared.
The
2
estimators of 01 that will be compared are
1)
"2
6 1 = (hI - h 2) / 2 = h 1 / 2 - MSE
hI / 2 =
a
1
n 2- Ln.2
I
~
=
(n-n.) Ii
i=l
~ j==l
a
1
2
n 2... n.2
I n
a
~e
I
i=l
y2 +
~J
2
2~ y .. ... y2
2
~J
n 2... Ln.2
y~
~.
i==l
a
(2
i=l
I n SST
.-
L
i==l
In.
~
]
Iiy~.
. 1
J=
~J
... y~ ])]
~.
n. SSE.~ ]
~
n.
I~ y 2.. ... y2
j=l
a
I
a
1
l
SST ==
Y~.
n.
i=l j==l
l
=
I
where
lJ
..
n.
/ n
and SSE. =
~
L~ (Y .. ... y.
j=l
~J
~
..
2
/ ni )
116
Therefore
A2
a
61 = I
=
L n.
.. 1 1
n SST _.
1
1=
1
2
In
2
n -In.
J
SSE.
(SSA
..
1
2
a
+
a
hI
= a(a-l) ~ #
a
=
SSE) -
a(a-l)
i::::l
..
I
[
t
1
a
SSE.
~
_._1
Lan
i=l· i
1
a
I
+
SSE i
i=1
Therefore
=
1
+
a
~
(Y.
1.
L
i=1
_ Y )2
a-I
SSE
• •
n-a
a(n-a-an.)(n.-l)MSE.
=
3)
~2*
u
l
I
1
i=1
n -In.
(MSA - MSE)
1
4)
6
2**
1
1
(n - a) a n i
= n(a-I)
2
2
-
=MSA-
n-a
I
# k
1
-2
SSE
SSE. ]
h 1 ,ik
i=1 an.
0"
n.
a-
a
=2
I
a
Ii
1
2 - MSE
n-Ln.1
1
1
~
l. 1 -MSE
an.
a
1=
1
1
+
MSA
117
6.3.
.
Variances of the EstimatesofO
2
l
The expressions for the four estimators are all defined in terms
of MSA, ]sA, MSE., and MSE.
Therefore to find the variance of each
~
estimator, one needs the variances and covariances of these four
.
tities.
20
The variance of
~E
is
quan~
4
2
since MSE is the estimator of
n-a
The covariance
The variance of MSE. is therefore given by
~
of MSE and MSA is zero and the same is therefore true of MSE i and MSA.
The variance of lffiA is given by
var(]SA) =
2 2 I (a-l)oi + 2
(a-I)
The variance of MSA is given by
2
4
4
var(MSA) = a-I °1 + a(a-l)
1 .
L2" +
n.
~
(a-I)
2
a
L
i=l
1
4
2] °2
n.
~
The covariance of MSA and MSE (and MSE.) is zero since MSA is also a
~
function of the means as is MSA.
Given the above variances and
covariances, one then has the following formulae for the variances of
the estimators:
118
A2
var(&1) =
I n 2 (a-I) 2 var(MSA) +
1
2
(n -In7)2
~
aI [<Iu~
.
~
- n n.~ - a n +
i=l
a u. )(u. ~
l.
l)j
2
-2
var(o 1) = var(MSA) +
Ia [<u
i=l
var(MSE. ) ]
~
n - a
- a - a n.)(n.
1
~
a n. (n - a)
-l)f
~
var(MSE )
i
= { n (a - 1) ] 2 [ var(MSA) + var(MSE) ]
n 2 - "In 2
i
1
2
2** ) = var (MSA) + ( Ia ) var(MSE)
var(&l
. 1 an,~
~=
6.4.
Comparison of Estimators
A2
2*
Since &1 and &1 both involve MSA they will be compared with each
other and
-2
0
and 6
1
2-J~*
1
will be compared with each other as they are
both functions of MSA.
A2
2*
In the comparison of 6 and 6 ' the variance of MSA plays no
1
1
A
2
role as it has the same coefficient in both var(G ) and var(6 2* ).
l
l
Therefore the only quantities to be compared are
Ia
[(In7
- n. n. - an + a n.)(n.
-_1)
_.....:l=--_ _
__
_
i=l
n - a
...:~==___
- ~)~
[nn /a- In.
J
.:::~
~=__
1
2
versus
J
2
.
~
1'>2
Assume that varC&l)
proved to be true.
n-a
2*
~var(el
) and see if this assumption can be
119
a
1
Assume
I
I' (In~
l·
... n n i - a n + ani)
J.
.
i=l
n- a
l
J
2
2 0
4
2
(n. -1)
J.
>
n-a
a
I
i=l
or
(\n~
- 1)
L J. - n n.J. - a n + a n.)2(n.
J.
J.
>
[ n (a - 1) ]
2
n - a
or
>
n
2
or
or
I
In~(n.
J.
J.
I(n. - 1) ] [
J.
Let a. = n. In.-l
J.
J.
and b. = In.-1
J.
J.
. \' 2 . \' 2
(Lb. )(L a .)
J.
J.
>
>
- 1) ]
[In.(n. _1)]2
J.
J.
Then one has
J.
(Ia.b.)2
J. J.
which is just the Cauchy-Schwarz inequality.
Hence this implies that
the original assumption is true.
~2
In the comparison of 01 with
2**
~1
the variance of MBA plays no
role as this term has a coefficient of one in both
2''o~
var(~l
a
I
i=l
).
var(~i)
and
Therefore the basis of the comparison is
4
(n - a - a ni)(n i ... 1)] 2 2 0 2
[
a n. (n - a)
n.-l
J.
~2
Assume that var(ol)
shown to be correct.
J.
21~*
> var(6'1
versus
a
(Ii=l
1
an
4
i
2 2°2
) n-a
) and see if this assumption can be
120
- a - a n.)(n.
1
1
- l)J 2
n -1
a n. (n - a)
or
1
(a - II
-) 2
n.
<
1
or
(
In~~l
n-a
1
(n - a) (Il:- - I\)
n.
n.
1
1
)2
n.-l
<
I I(n.1 - 1) J I I-T ]
n.
1
1
Let a. = ./n.-l
1
1
1
2
2
2
- I2" (n -2an+a ) + a (a-n)
n.
\1 2
\1
(n-a) (L-)
- (n-a) (n+a) L - <
n.
n.
1
or
a
1
2
(I·-·
)
i;i:l ani
i
1
.
>
_1_
1
. 2
( la.b.
)
1 1
and b. = ./n.-l / n.
1
<
1
1
This results in
( Ia7)( Ib7)
1
1
which is the Cauchy-Schwarz inequality.
Hence this implies that the
initial assumption again is correct.
Therefore the two Koch type estimators can be dismissed as they
are inquadmissible.
There are other possible combinations of the
square differences and some of these other combinations may be better
than the ANOVA estimator, or at least quadmissible with respect to
the ANOVA estimator.
The two ANOYA estimators have also been compared for the same
designs that Koch 120J considered in his comparison of estimates of
the mean in the random-effects model.
In the table which follows, the
design parameters are a = S, and then n, n , n , n , n , and n are
l
2
3
4
S
specified.
(52 . 2
1 (52' and
4
Following these parameters are the coefficients of (51'
(5~
for both ANOYA estimates.
These are
121
Al
coefficient of
=
2 2 + n 21 n.2 2[ (I n.)
1.
. 1.
2
(n _ In~)2
1.
= coefficient
B1
=2
/
°14
of
A12
= coefficient
of
B12
"
= cae ff 1.C1.ent
0
4
)
0; in var(8i*)
.
°12 °22 1.n
f
( p:_)2 _
°24
in var(8
°4·2
2
IL
---=1.::..---2c:::----=1.~+
a
1
)
2
n
3
( IL )2
1
L2" +
(a-I)
n
2*
1
in var (1)2**
VI )
n.n2
=2"
var (1)2**)
vI
2 (n-1) (a-I)
n - a
B2 = coefficient of
n
O~
Ln:1.
n
n2-In~
1.
n
in var(8 2** )
1
\'1
- 1)
A2 = coefficient of
= 2(
2nln~
. 1. J
(a .,.. 1)
= a(a
--
0i in var(8 12* )
=
n.
1.
n
4
n
5
n.
1.
n - a
Al
B1
A12
B12
A2
B2
10
2
2
2
2
2
.500
.500
.500
.500
.225
.225
11
2
2
2
2
3
.510
.500
.458
.467
.175
.183
12
2
2
2
3
3
.515
.500
.421
.433
.139
.150
13
2
2
-3
3
3
.514
.500
.388
.400
.113
.123
123
n
n
n
1
n
2
3
n
n
4
5
Al
B1
A12
B12
A2
B2
25
3
3
3
8
8
.594
.500
.213
.250
.027
.041
25
5
5
5
5
5
.500
.500
.200
.200
.024
.024
26
2
3
5
8
8
.601
.500
.204
.257
.025
.047
26
3
5
5
5
8
.534
.500
.197
.212
.023
.028
27
3
3
5
8
8
.571
.500
.194
.223
.022
.033
28
2
2
8
8
8
.644
.500
.192
.275
.022
.057
28
2
5
5
8
8
.574
.500
.186
.230
.020
.038
28
5
5
5
5
8
.514
.500
.181
.185
.019
.020
29
2
3
8
8
8
.610
.500
.182
.242
.019
.043
29
3
5
5
8
8
.546
.500
.177
.197
.018
.025
30
3
3
8
8
8
.578
.500
.174
.208
.018
.029
31
2
5
8
8
8
.576
.500
.168
.215
.016
.035
31
5
5
5
8
8
.521
.500
.164
.170
.015
.017
32
3
5
8
8
8
.548
.500
.160
.182
.015
.021
34
2
8
8
8
8
.569
.500
.152
.200
.013
.031
34
5
5
8
8
8
.519
.500
.149
.155
.013
.014
35
3
8
8
8
8
.542
.500
.146
.167
.012
.018
37
5
8
8
8
8
.512
.500
.136
.140
.010
.011
40
8
8
8
8
8
.500
.500
.125
.125
.009
.009
.2.
AI; A2
In the table the following inequalities hold: B1
and A12
.2.
B12 with equality holding in balanced designs.
.2.
B2;
The author
has attempted to prove the inequalities, but his attempt has not met
with complete success.
4
I
n 2- n.2
<
4
a(a-1)
.2.
B12, that is
the following has been used.
First note
For example, to prove that A12
I~
l
,
l
that showing the above is equivalent to showing tha t
I
*
-_
a (a - 1)
-<
(n 2 -
In.)2 II-n.
l
l
.
(6.4.1)
If one is able to show that the minimum of the right hand side (ID-IS)
124
of (6.4.1) is a(a-l), then it is true that A12
~
B12.
To do this,
one must find the set' {n.}
the RHS of (6.4.1) subject
J. which minimizes
.
to the constraints
l
n
= nand
i
(n 2
=
~
n
I
> 0 for i
i
n~)
J.
L!n.
= l~
2, .•• , a.
+ A(n
Let
(6.4.2)
1.
Then, upon differentiation of f, one has
a
f
--- = -
ani
2
2 . __1__
(n
Ln )
k k
-
n 2.
1
- 2 nJ.'
1.
L-- k ~
A
i = 1, 2, ... , a
Setting these partial derivatives equal to zero and then solving for
the {n.} and A one then has the critical points of f.
Doing this im-
1.
plies that
A
=-
(n
2
\' 2
- L ~)
1
2
n.
L n1
- 2 n.
1.
J.
i
= 1, 2,
... , a
k
One choice for the {n.} such that the solution for A is consistent for
J.
i
= 1,
.
2, ... , a is n
means that n.
J.
n
= -
a
l
for i
= n2
=
= na
= •••
and this along with
1, 2, •.• , a.
~~ =
0
This means that A = -a(a+l).
To determine whether this critical point is a relative minimum, a relative maximum, neither, or indeterminate, one examines the second partial derivatives evaluated at the critical point that has been found.
2
.'
=
3
n.
1.
and
(n
2
\' 2
- L n. )
k
4
+ -n.
1.
2
~
n
2
(a -2a+2)
125
a2f
n.
.n.
~
= 2 ....l. + 2
2
2""
aniB~j n.= E. A
n;; .
n.
a'
~
~
J
f
=
a
4 -.
n
If one forms the matrix of these second partial derivatives and then
finds the eigenvalues of this matrix, it is possible to determine
whether the critical point that has been found is a relative maximum,
a relative minimum, neither, or indeterminate.
that is, there are just two classes.
For example, let a
= 2,
This is a degenerate case as the
function f is a constant, that is
or
2 n > a(a-l) = 2
Let a = 3.
6
n
Then the matrix of second partials is
5
2
2
2
5
2
2
2
5
This matrix is positive definite which implies that all the eigenvalues
are positive reals.
has been found is
This in turn implies that the critical point which
a relative
minimum.
thing for larger values of a, e.g. if a
tials is
•
16
n
5
1
1
1
1
5
1
1
1
1
5
1
1
1
1
5
One can continue doing the same
= 4, the matrix of second par-
126
This matrix is also positive definite which implies that taking n
i
~
l~
2,.
taking n.
l.
...
~
-na
,
a is a relative minimum.
i
~
1, 2,. •••
~
~
i
n
a
It is des.ired to show that
a gives an absolute minimum, not a
relative minimum, and to do this requires the finding of the other
critical points of (6.4.2) and this is where the attempt becomes unsuccessfu1.
One can use the same procedure in an attempt to prove that
A1 > H1 and A2 < H2, but in these cases again one
can only show rela-
tive, not absolute minima.
For the situations considered in the table, if one knows that
2
2
01 » °2' one should use
.
°
2
2**
to estimate 01.
1
2
should be use d to estl.mate 01.
2
2
If 02 » 01' then
~2*
v
1
CHAP TER VII
SUMMARY AND FUTURE WORK
7.1.
Summary
In this study t;he estimation procedure proposed by Koch [21] has
been
extended
and modified.
The ANOVA technique, the maximum like-
lihood method and the modified Koch procedure have also been compared
using the variance of the estimate as the criterion for the comparison.
The modification of the Koch method is necessary since the procedure does not lead to a unique set of estimators unless there are
as many mean squares in the ANOVA table as there are variance components in the model.
Otherwise the technique results in the solving
of r equations in s unknowns (r> s).
When Koch's procedure results
in a unique set of estimators, the matrix formulation of Chapter II
gives the same set of estimators.
For those situations when Koch's
method does not yield a unique solution the matrix formulation does
give a unique set of estimators.
The extensions presented in this dissertation are as follows:
1) the technique does not require that the random variables in
the model come from infinite populations;
2)" the estimation technique does not require that the random
-e
variables come from any specific distribution (Koch did not
require this either in his formulation);
128
3) the variance-covariance matrix of the modified Koch estimators
is presented assuming that the random variables come from a
normal distribution.
The groundwork is laid for finding the
variance-covariance matrix for distributions other than the
normal also.
4) the method obtains estimates of the variance components under
the assumption 6f a mixed model.
It is not necessary to esti-
mate the fixed effects to obtain estimates of the components
of variance.
The comparison of the various procedures is based on a type of
'asymptotic' efficiency.
The asymptotic efficiency of u.;:,2. is obtained
1
by dividing the asymptotic variance of the maximum likelihood esti-2
.;:,2
1
1
mator, 0., by the variance of U., that is
-2
asymptotic variance of 0.
asymptotic efficiency of 6~ :;:: ~
-=-l
1
variance of 6~
1
The conclusions are presented for each particular model discussed
under the random-effects model and for the mixed model as a whole.
The conclusions are as follows:
1) two-way classification:
the modified Koch procedure and
Method 1 are almost the same for S designs and 'random'
designs.
Both of these procedures seem to do better than
Method 3 which is affected greatly by the values of the
variance components.
,
When the variance components are equal,
Method 3 does very poorly.
For the L designs Method 3 seems
to be better than Method 1 or the Koch technique.
Method 1
does a little better than the Koch approach, although the
129
greatest difference in efficiency is only 0.117.
2) two-way classification with sub-sampling:
the conclusions are
the same as for the two-way classification •
•
3) two-stage nested:
when one uses the designs which PrairieI26]
has shown to be optimal, there is almost no difference in efficiency between Koch's procedure and Method 1.
When there is
a large unbalance in the design, then Method 1 is much better
than Koch's procedure.
4) three-stage nested:
both Method 1 and the Koch method do a
good job here, although Method 1 usually has a somewhat higher
efficiency than the Koch approach.
For the mixed model, if there are only a small number of terms
in each level of the fixed effect, it is best to use Method 2 or some
other technique to estimate the components of variance.
,
•
The modified
Koch technique gains in efficiency as the number of observations in
the levels of the fixed effects increases.
In summary, the efficiencies of the Koch method are-generally
slightly lower than the ANOVA estimators.
The fact that the Koch
technique is very general, that it allows variances to be calculated
without constantly deriving new formulae limited to specific classes
of designs, and that the method is easy to apply are advantages which
may outweigh the generally lower efficiencies.
7.2.
Suggestions for future work
We feel that considering the squared differences or cross-
products of differences is a good approach to the estimation of variance components.
What needs to be done now is the determination of
130
whether there exist better ways of combining these cross-products of
differences to improve efficiency.
Specific problems that require
investigation are the following:
1)
Does there exist a combination of squared differences (and
cross-products of differences) which will make the Koch type
estimators quadmissib1e in the two-stage nested design?
2)
In the mixed model, use the squared difference and crossproduct of differences approach along with the estimation of
the fixed effects.
3)
Investigate what happens to the variance formulae assuming
some distribution other than the normal.
4)
Consider different designs and determine whether the conclusions given above are still valid.
5)
(
Consider a different criterion, such as the determinant and
see if the above conclusions still hold.
6)
Adjust the A matrix of Chapter II by a weight matrix to
account for the different sizes of the variance components
in order to improve efficiency.
-e
'e
APPENDIX A
DERIVATIVES OF MOMENT GENERATING FUNCTIONS
i
2
If m ,f (t ,t 2) = expI
(t 1 r 11 + 2
f1 2 1
dm(t ,t )
1 2
then ~~~-= m(t1,t Z) (t 1 r 11 + t 2 r 1Z )·
dt
1
(t
(t
r
1
11
+ t
2
r
12
)[ r
12
t1
+ (t
1
t 2 r 12 + t 2
r
11
+ t
2
2
r 22 ) ],
r 12 ) x
l}
1 r 12 + t 2 r 22 )
• m(t ,t 2) {r11 r 22 + ri2 + (r12 t 1 + r 22 t 2) x
1
(2 t
(t
-e
t
2
1
r
1
r
r
12
11
r
+ t
12
2
+ t
r
22
2
)(t
r
11
1
r
r
11
22
+ t
+ t
) (t r
+ t z r 22 ) ] }
12
1 12
2
2
2
r
r 12 ) + r 12 1 r 12 +
12
)I r 12 + (t 1 r 11 +
13Z
0,0
This is the result presented in (2.5.11).
The result presented in
(Z.5.13) is derived by considering the same moment generating function as above and finding the partial derivatives and evaluating at
the point (0,0).
4
d m(t , t )
1 z
dt
3
1
dt
{(t 1 r
11
+ t
z r 1Z ) (
Z t 1 r 11 r +
1Z
tz r
11
Z
r
(t
t
1
r
11
2
) + 2 r
r
+ (t 1 r 11 + t z r 1Z ) Z x
12
11 12
+ t 2 r 1Z )(Z t 1 r 11 r 12 + t z r 11 r Z2 +
z riz)}
To obtain the result presented in (2.5.16), one uses the moment generating defined by
-e
x
133
-e
134
To obtain (2.5.19) consider the moment generating function
t
a4m(t 1 ,t 2 ,t3 ,t 4)
Clt 1 Clt 2 .Clt3 Clt 4
0,0,0,0
APPENDIX B
TABLES OF VARIANCES
The tables presented here are in the same order as those in
Chapter V.
Only the values of the asymptotic variance of the maximum
likelihood estimator are presented as one can obtain the values of the
other variances by using the asymptotic variance of the maximum 1ike1ihood estimator and the table of efficiencies presented in Chapter V.
PJillDOM~EFFECTS
MODEL
ASYMPTOTIC VARIANCES OF MAXIMUM LIKELIHOOD ESTIMATORS
TWO-WAY CLASSIFICATION
J
Design 1
Design 2
Design 3
Design 4
(52 = 49
1
(52 = 49
2
(52 = 7
3
854.06
863.34
875.77
919.95
757.99
754.43
743.59
749.62
8.13
8.15
8.14
8.13
(52 = 49
1
(52 = 1
2
(52 = 7
3
748.37
774.12
769.07
821. 91
4.36
4.41
4.51
4.37
7.69
7.56
7.90
7.80
= 1
4.29
4.31
4.14
3.88
= 49
666.67
681. 41
662.52
676.53
= 7
7.81
7.53
7.55
6.97
Variance Component
and its value
t
-e
(52
1
(52
2
(J2
3
-
136
Variance Component
and its value
2
1
2
(J
2
2
(J
(J
3
2
1
2
(J
2
2
(J
3
(J
2
1
2
(J
2
2
(J
(J
3
2
1
2
(J
2
2
(J
3
(J
2
(J1
Design 1
Design 2
Design 3
Design 4
:::
7
27.99
30.61
29.98
35.47
:::
7
26.69
27.75
25.70
26.29
:::
7
7.83
7.88
7.82
7.63
Design 5
Design 6
Design 7
Design 8
:::
49
815.56
823.00
813.25
834.06
:::
49
718.05
719.83
720.05
716.12
:::
7
4.45
4.45
4.45
4.45
:::
49
731.11
740.43
730.51
749.86
:::
1
2.30
2.33
2.30
2.31
:::
7
4.39
4.36
4.37
4.41
:::
1
2.28
2.32
2.28
2.30
:::
49
647.04
651.58
649.85
648.32
:::
7
4.42
4.36
4.44
4.29
:::
7
23.18
24.24
22.99
25.34
7
21.48
21.85
21.74
21.36
7
4.43
4.42
4.43
4.41
2 :::
2 .
2 :::
(J
(J
3
TWO-WAY CLASSIFICATION WITH SUB-SAMPLING
Design 1
Design 2
Design 3
36
473.45
482.23
471.00
2 . 36
422.12
424.68
423.89
6
19.27
19.03
19.21
6
10.04
10.00
10.01
Variance Component
and its value
,
.-
(J
2
1
:::
(J2 :::
(J2 :::
~e
3
2
°4 :::
137
Design 1
Design 2
Design 3
422.29
4"33.45
421.88
4.47
4.39
4.44
18.82
18.47
18.69
9.91
9.86
9.89
1
2.60
2.62
2.60
= 36
362.16
366.04
364.23
Variance Component
and its value
2
O"i
2
cr 2
2
cr 3
2
cr 4
2
cr1
2
cr 2
2
cr3
2
cr 4
2
cr1
2
cr 2
2
cr 3
2
cr 4
= 36
= 1
= 6
= 6
=
=
6
6.35
6.24
6.38
=
2
1.13
1.13
1.13
=
6
22.64
23.98
22.47
=
6
21. 76
22.33
22.03
=
6
18.95
18.65
18.88
=
6
9.94
9.90
9.92
TWO-STAGE NESTED
Variance Component
and its value
Design 2
Design 3.
Design 4
34.51
33.80
50.12
17.49
.09
.09
.09
.11
1
.93
.84
1.17
~52
=1
.09
.09
.09
.11
=1
2.68
2.32
2.46
2.69
=7
4.39
4.45
4.26
5.15
2
cr ::; 7
1
2
cr 2 = 1
2
cr1
2
cr 2
,
~e
2
cr1
2
cr 2
=
Design 1
139
Design 5
Design 6
Design 7
Design 8
2 ;::
49
791.36
772.66
447.22
376.77
2 ;::
28
360.46
245.27
216.03
121.12
7
10.87
8.16
9.77
9.75
2 ;::
49
296.34
380.62
195.64
Variance Component
and its value
0
0
0
0
1
2
2
;::
3
1
2
0 2 --
1
18.83
130.50
8.87
2 ;::
7
10.81
8.11
8.54
o 2 ;:: 21
1
862.05
629.70
294.38
o 2 ;:: 49
2
852.81
616.17
295.32
0
0
0
0
3
2 ;::
7
10.87
8.16
9.78
2 ;::
7
53.23
40.87
20.52
7
50.33
35.30
22.21
7
10.62
8.02
9.38
3
1
2
2
=
0 2 ;::
3
NIXED MODEL
ASYMPTOTIC VARIANCES OF NAXIMUM LIKELIHOOD ESTIMATORS
TWO-WAY CLASSIFICATION
Variance Component
and its value
Design 1
Design 2
49
753.67
772.17
o2 ;::
2
7
7.00
7.00
o2 ;::
7
25.67
27.58
-
7
].00
6.92
2
°1
1
2
02
-e
;::
140
2 .
7
1 =
2
cr = 49
2
cr
193.67
182.91
343.00
326.88
TWO-WAY CLASSIFICATION WITH
Variance Component
and its value
SUB~SAMPLING
Design 1
Design 2
= 49
1340.33
2349.84
= 49
438.33
450.86
7
13.98
8.16
2
1
2
cr
2
2
cr
3
= 14
345.09
404.24
= 49
438.15
450.66
7
13.97
8.15
2
1
2
cr
2
2
cr
3
= 49
747.40
1666.23
2
1
2
cr
2
2
cr
3
cr
cr
cr
cr
2
1
2
cr
2
2
cr
3
=
=
=
1
13.97
6.19
=
7
13.15
7.33
=
7
38.47
57.11
=
7
31. 20
23.63
=
7
13.56
7.93
TWO-WAY CLASSIFICATION WITH SUB-SAMPLING AND NESTING
Variance Component
and its value
Design 1
1690.02
275.79
154.00
4.77
141
Variance Component
and its value
2
1
2
o2
2
o
3
2
0
4
o
0
2
1
2
0
2
2
0
3
2
0
4
-e
!
==
36
1501.71
==
12
210.95
==
12
52.90
==
6
4.74
==
36
2577.71
==
36
1115.32
==
6
43.61
==
12
18.39
6
102.44
6
61.53
6
20.88
6
4.65
2
1 ==
2
0 ==
2
2
0 ==
3
2
0 ==
4
0
Design 1
~-i
-e
LIST OF REFERENCES
1.
Anderson, R. L. 1962. Designs for Estimating Variance Components.
Proceedings of Seventh Conference on the Design of Experiments
in Army Research Development and Testing. Fort Monmouth, New
Jersey. October, 1961, lJSARO Institute of Statistics Mimeo
Series No. 310.
2.
Bainbridge, T. R. 1963. Staggered, Nested Designs for Estimating
Variance Components. ASQC Annual Conference Transactions 93 103.
3.
Blischke, W. R. 1966. Variances of Estimates of Variance Components in a Three-Way Classification. Biometrics 22: 553 - 65.
4.
Blischke, W. R. 1968. Variance of Moment Estimators of Variance
Components in the Unbalanced R~Way Classification. Biometrics
24: 527 - 39.
5.
Bush, N. and Anderson, R. L. 1963. A Comparison of Three Different Procedures for Estimating Variance Components. Technometrics
5: 421 - 40.
6.
Crump, P. P.1954. Optimal Designs to Estimate the Parameter of a
Variance Component Model. Unpublished Ph.D. thesis. North Carolina State University.
7.
Crump, S. L. 1946. The Estimation of Variance Components in Analysis of Variance. Biometrics 2: 7 - 11.
8.
Crump, S. L. 1951. The Present Status of Variance Component Analysis. Biometrics 7: 1 - 16.
9.
Eisenhart, C. 1947. The Assumptions Underlying the Analysis of
Variance. Biometrics 3: 1 - 21.
10. Gaylor, D. W. 1960. The Construction and Evaluation of Some Designs for the Estimation of Parameters in Random Models. Unpublished Ph.D. thesis. North Carolina State University. Instituteof Statistics Mimeo Series No. 256.
-e
11. Gaylor, D. W. and Hartwell, T•. D. 1969. Expected Mean Squares for
Nested Classifications. Biometrics 25: 427 - 30.
143
12. Goldsmith, C. H. 1969. Three~Stage Nested Designs for Estimating
Variance Components.. Unpublished Ph.D.·thesis • North Carolina
State University •. Inst:i.;tute ·oUStatisticsl'[imeo ·Seties N6.624.
13. Graybill, F. 1961.Anln~roduction to Linear Statistical Models.
McGraw-Hill.
14. Hartley, H. O. 1967. Expectations, Variances, and Covariances of
ANOVA Mean Squares by 'Synthesis ll • Biometrics 23: 105 - 14.
15. Hartley, H. O. and Rao, J. N. K. 1967. Maximum-Likelihood Estimation for the Mixed Analysis of Variance Model. Biometrics 54:
93 - 108.
16. Harville, D. A. 1969. Variance Component Estimation for the Unbalanced One-Way Random Classification - A Critique. Aerospace
Research Laboratories 69-0180.
17. Henderson, C. R. 1953. Estimation of Variance and Covariance Components. Biometrics 9: 226 - 52.
18. Hooke, R. 1956. Some Applications of Bipolykays to the Estimation
of Variance Components and their Moments. Annals of Mathematical
Statistics 27: 80 - 98.
19. Koch, G. G. 1967. A General Approach to the Estimation of Variance Components. Technometrics 9: 93 - 118.
20. Koch, G. G. 1967. A Procedure to Estimate the Population Mean in
Random Effects Models. Technometrics 9: 577 - 85.
21. Koch, G. G. 1968. Some Further Remarks Concerning 'A General Approach to the Estimation of Variance Components'. Technometrics
10: 551 - 58.
22. Lancaster, H. O. 1954. Traces and Cumulants of Quadratic Forms in
Normal Variables. J. Roy. Stat. Soc., Suppl. 16: 247 - 54.
23. Mahamunulu, D. M. 1963. Sampling Variances of the Estimates of
Variance Components in the Unbalanced 3-Way Nested Classification.
Annals of Mathematical Statistics 34: 521 - 27.
24. Mostafa, M. G. 1967. Designs for the Simultaneous Estimation of
Functions of Variance Components from Two-Way Crossed Classifications. Biometrika 54: 127 - 32.
25. Niedokos, E. 1969. Estimation of Variance Components in Two Nonorthogonal Mixed Models. Biometrische Zeitschrift 11: 289 - 96.
144
26. Prairie, R. R. 1962. Optimal Designs to Estimate Variance Components and to Reduce Product Variability for Nested C1assifi~
cations • Unpublished Ph~:D. ·thesis. q.NoxthqGarolina State University. Institute of Statisticsllimeo Series No~ 313.
27. Rao, J. N. K. 1968. On Expectations, Variances:; and Covariances
of ANOVA Mean Squares by 'Synthesis'. Biometrics 24: 963 - 78.
28. Scheffe, H. 1959.' The Analysis of Variance.
Wiley, New York.
29. Searle, S. R. 1956. Matrix Methods in Components of Variance and
Covariance Analysis. Annals of Mathematical Statistics 27: 737 48.
30. Searle, S. R. 1958. Sampling Variances of Estimates of Components
of Variance. Annals of Mathematical Statistics 29: 167 - 78.
31.
Sea~le,
S. R. 1961. Variance Components in the Unbalanced 2-Way
Nested Classification. Annals of Mathematical Statistics 32:
1161 - 66.
32. Searle, S. R. 1968. Another Look at Henderson's Methods of Estimating Variance Components (with discussion). Biometrics 24:
749 - 87.
33. Searle, S. R. 1969. Large Sample Variances of Maximum Likelihood
Estimators of Variance Components. To appear in Biometrics.
34. Snedecor, G. 1934. The Method of Expected Numbers for Tables of
Multiple Classification with Disproportionate Subclass Numbers.
J. Amer. Stat. Assoc. 29: 389 - 93.
35. Tukey, J. W. 1956. Variance of Variance Components: I. Balanced
Designs. Annals of Mathematical Statistics 27: 722 - 36.
36. Tukey, J. W. 1957. Variance of Variance Components: II. Unbalanced Single Classifications. Annals of Mathematical Statistics
28: 43 - 56.
37. Wang, Y. Y. 1967. A Comparison of Several Variance Component
Estimators. Biometrika 54: 301 - 05.
38. Yates, :F. 1934. The Analysis of 11ultip1e Classifications with
Unequal Numbers in the Different Classes. J. Amer. Stat. Assoc.
29: 51 - 66.
'.
. / ,• .
?j'