I
I.
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I·
I
A GENERAL APPROACH TO THE ESTIMATION OF VARIANCE COMPONENTS
by
Gary G. Koch
University of North Carolina
Institute of Statistics Mimeo Series No. 447
September 1965
This research was supported by National Institutes of Health
Institute of General Medical Sciences Grant No. GM-12868-01.
DEPARTMENT OF STATISTICS
UNIVERSITY OF NORTH CAROLINA
Chapel Hill, N. C.
I
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
I·
I
A GENERAL APPROACH
ro
THE ESTIMATION OF VARIANCE COMPONENTS*
Gary G. Koch
University of North Carolina
A general method of estimation of variance components in randomeffects models of the nested and/or classification type is considered. If
a given parameter is estimable with respect to some particular experimental
design (i.e., an unbiased estimate of the parameter may be obtained from
the experiment), then the suggested estimator may be readily computed with
only the aid of a desk calculator. The estimates are always unbiased and
consistent (with respect to the structure of the experimental design); in
the case of balanced experiments, they coincide with those obtained from
the analysis of variance.
Secondly, the problem of designing experiments to estimate
variance components is briefly discussed from the point-of-view of the
suggested estimation procedure. As a result, certain non-balanced designs
are seen to yield more efficient estimators of particular parameters in
specified situations than the corresponding balanced design using the same
number of observations.
Finally, the method of estimation is shown to be applicable to
models more general than the variance component one.
computed' and is unbiased and consistent.
*This
res~arch
Again it is readily
was supported by National Institutes of Health Institute
of General Medical Sciences Grant No. GM-12868-0l.
I
I.
I
I
I
I
I
I
I
INTRODUCTION
In this paper, we want to consider experiments in which the
observations may be assumed to be linear functions of independent, unmeasurable
random variables with zero means and unknown variances which are called
variance components.
The purpose of the experiment is to study the
contribution of several sources of variation to the total variation among the
observations by estimating the variance components.
Several methods of
estimating variance components have been developed.
The three best known are:
(i)
setting the mean squares appearing in some particular type of
I_
analysis of variance
I
I
I
I
I
I
solving the resulting equations for the variance components
I
.I
unadjusted)
(weighted or unweighted, adjusted or
equal to their respective expectations and
(see Anderson [2J; and Bush and Anderson [3J ) ;
(ii)
listing certain sums of squares and their corresponding
expectations (there being more sums of squares than
variance components) and performing weighted least squares
(using prior estimates of the variance components or
other information to determine the weights) (see Anderson
[2J) ;
(iii)
assuming that the random-effects obey some probability distribution
function which is completely known except for a constant
2
representing the associated variance component (for example,
.
assuming the random-effects are normally distributed with zero
means) and obtaining maximum likelihood estimates or modified
maximum likelihood estimates (see Anderson and Bancroft [1]).
In balanced experiments, we may say (loosely speaking)
(ii), and (iii) all lead to the same estimators.
th~t
methods (i),
In addition, the estimators
obtained by (i) are the best quadratic unbiased estimates (BQUE) of the
corresponding parameters (see Graybill [7]; Graybill and Hultquist [8];
Hsu [10]).
However, in non-balanced experiments (particularly non-balanced
cross-classification-type designs), methods (i), (ii), and (iii) can become
very complicated computationally; their small sample properties are, for the most
part, unknO'Wl'l and difficult to study.
Hence these methods are of limited
use in studying the efficiency of different experimental designs.
With the comments made above in mind, one sees the need for
another type of estimator which, in non-balanced experiments, is computationally
simpler and theoretically more tractable than those estimators obtained by
the methods (i), (ii), and (iii) while, in balanced experiments, coincides
with them (i.e., it is BQUE).
The method of estimation presented here will
be seen to have the above properties.
The structure of the suggested estimate follows from basic
assumptions made about the form of the covariance matrix in the assumed
model.
Any estimable parameter (see Appendix 1) can be estimated by forming
an appropriate linear combination of certain sYmmetric sums of (linearly
independent) random variables having the same expectation such that the
resulting quadratic form is an unbiased estimate of it.
In many cases, there
I
I
e
I
I
I
I
I
I
I
-,
I
I
I
I
I
I
I
-.
I
I
:3
I.
I
I
I
I
I
I
I
is some choice as to which linear combination to use.
prior information concerning the magnitude (relative or actual) of the variance
components is available, then the suggested estimator would be modified to be
that linear combination which was best with respect to this information.
Because of its generality and structure, this estimation procedure
may be used to evaluate the efficiency of non-balanced experimental designs.
Even with it, however, results have been too difficult to obtain in a clear-cut
form for the general case; but some light is shed on the problem of the proper
choice of design if a few simplifying assumptions are made.
(2) the random effects are assumed to be normally distributed.
Finally, the suggested approach is valid and useful in obtaining
estimators of the variances and covariances of observations from any experiment
in which they are estimable with respect to the assumed model (see Appendix 1).
I
The estimates will be of particular interest whenev"er the variance-covariance
matrix of the observations possesses some structure; i.e., it is a function of
I
,e
I
Namely,
(1) the mean of the observations is assumed to be zero,
I_
I
I
I
I
I
As a consequence, if
a small number (with respect to (N 2 + N)/2 where N is the number of observations)
of parameters which appear in it in a particular pattern.
All of the above
comments will be clarified by the specific applications and examples which follow.
NOTATION:
If a Ildot ll replaces a subscript, then that subscript has been
summed over; if a IIbar" appears over a quantity with IIdots" in
its subscript. then those subscripts have been averaged over.
2.
NESTED EXPERIMENTS
2.1
The Degenerate or One-Stage Nested Design.
I
4
Model:
Y
i
=
~
+ e
i
for
i
=1,
.-I
2, ••• , n.
Assume e , e , ••• , en are independent and identioally distributed as
2
l
N(O,
0
2
e
).
Hence, we have that
where i, j
=1,
=
if i = j
=
if i:4= j
2, ••• , n.
Let us now estimate (0
2
e
+ ~2) by forming the normalized symmetric sum of
all of its unbiased estimators listed above; i.e., we have
~
2.
Ii
= (02e + ~2) =
1. ~
n
i=l
_I
y2
i
Similarly, let us estimate ~2 by the normalized symmetric sum of its
unbiased estimators.
~
=
(~l) lLi=l
J( ~ y.)2
J.
n n
-
i-~-l YJ.~}
We obtain the estimator of o~ by subtraoting gm from gt; that is,
"2
a
e
=
I
Henoe,
(1 )
Y Y.
n n-1 itj=l i J
I
I
I
I
I
I
•
I
I
I
I
I
I
-,
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
,.
I
5
It is "'ell know that
=
and that
model.
~:
is the minimum variance unbiased estimator of
0
2
e
for this
As a result, ",e see that in this case the suggested estimation
procedure leads to the usual estimator of
2.2.
(n-l)
0
2
e
•
The 1\.ro-Stage Nested Design •
Model:
Yij
=
~
+ a + eij
i
for
=1,
i
2, ••• , a and j = 1, 2, ••• , n i •
Assume
(ii)
(iii)
~,a2' ••• , aa are identically distributed as N(O,O: );
2
ell' ••• , e1n , ••• , e al ,···, e a,n are identically distributed as N(O,oa);
1
a
Hence, ",e have that
E
{Yij" Ykl}
2
2
2
= 0a + 0 e + ~
if
i = k
and j
=I.
= 02a + ~2
if
i =k
and j
-4= /.
if
i :f k
=
~
2
",here i, k = 1, 2, ••• , a
Let us no'" estimate
0
and
j
222
a +
0
e +
~
= 1,
,
0
2, ••• , n.:1. and /. = 1, 2,
2 + ~2,
~2 by forming the normalized
a
symmetric sums of their respective unbiased estimators listed above; that is,
n.
n.1. .
a
a
(L
'-1
1.-
n.r1
:1.
L
L
1=1 j=l
2
Y..
:1.J
1
= -n
a
1.
LL
i=l j=l
2
Y' j
1.
I
6
g
-a
&n
={
={
~
i=l
~
n.{n i -1)}-1{
1
i=l
a
n.1
ni
i=1
j=l
j=l
2
~ { (~Yij) -
;i
Y Y./.} =
j1/'=1 ij 1
a
-1
a
1 k
1
a
nn
{( ~
n. (n-n )
~
~
~ Y
Y } =
i=l 1
i } {ifk=l j=l 1.=1 ij kl
(n2-k ) 1=1
1
a
~
i=l
a
ni
and
k
1
~
=
i=l
2
ni
2
2
We obtain the estimators of 0 a2 , °e'
by setting
~
~,
~'iin
respective expectations and solving the resulting equations.
"'2
0
a
"'2
0
e
"'2
~
=
&1
=
~ -~
=
~
can be obtained by using the theory of Appendix 2;
expressions
equal to their
Thus, we obtain
-~
The above estimates are unbiased by construction.
Their variances and covariances
however, the resulting
On the other hand, if we assume that
~
is a known constant
Y.. = Y.. - ~), then the estimates of the variance components are
1J
1J
I
I
I
I
I
_I
and in particular is zero (this can be achieved by performing a transformation
I
.-I
I
become unwie1d1y and are difficult to interpret; hence, they are
not considered here.
,
1J
2 a
1
2
Y ) - ~ ( n~ Y•• ) }
ij
'-1 J'-1 1J
1-
~
where n =
~
2 }
Y..
I
I
I
I
I
I
I
.,
I
I
I.
I
I
I
I
I
I
I
7
In t:us case, the variances and cov'ariances of the estimators are not so
difficult to obtain and provide some insights in the design problem.
If we let
n.
~
••
I
~J
j=l
g ai =
then from the theory of Appendix 2, we obtain
22
4o a 0 e
+
= 204
n.
a
~
I_
I
I
I
I
I
I
I
2
Y••
l.:
Ct·
a ,~
=
Var (g .)
+
= 20 4
a
=
Cov (gt"~ g aJ..)
= 20 4 +
a
~
2
40
40;
0
a
4
20
+ --L
n.
~
20 4e
2
e
n.~
+
ni(ni-l)
o~
,
,
n.~
20 4
V
e,i
= Var {gti-gai}
=
e
(n.-l)
~
Since (gti' gail is independent of (gtk' gak) for i
have that
f
k = 1, 2, ••• , a, we
I
8
2~
=
n
=Var
(g)
a
=
(k - n )2
l
+
4
4 0a
2
a
0
(kl-n)
2
e
+
n
n(k1-n)
e
+
(k -n)2
l
40
I
I
I
I
20 4
e
2 2
4k] 0a 0e
4
O
2 a
2k
V
n
n
2k
va
,
+
2
.-I
* = Var
I
I
At this time, we should note that the estimation procedure can give us a more
well-mown estimator of
0
2 than gt-g •
e
a
g i
e,
Thus, we observe that
1
= gt·-gai
=
--=r
~
n
i
n.~
2}
)
~
(y.j-Y
{ j=l
i•
~
is an unbiased estimator of 0 2 with variance V • =
e~
e
alternative estimator of
0: would
be
204
e
(n.-l)
~
With this in mind, an
_I
I
I
I
I
I
I
e I"
I
I
I
I.
I
I
I
I
I
9
Its variance is, of course,
ve
(g e )
=
•
(n-a)
Finally, one should observe that if prior estimates of 0 a2 ,
other linear combinations of the gai and/or the gti
of the parametric functions
0
2
a'
0
2 + 02
a
e'
0
2
e·
mar
0
2 are available, then
e
lead to better estimators
For example, one might consider
-1
a
1
L
)( L V )
i=l Vai
i=l ai
=(
I
I
=Var
gai
a
as an alternative estimator of
0
2
•
a
The questions concerning what is the best
I_
way in which to use the information of the covariance matrix (if such a best way
I
I
I
I
I
I
I
Let us now see what happens if n1 = n2 = ••• = na = c.
n = ac, ~ = ac 2 , k = ac 2 (c_1)2, k) = ac{c-1)2, k = ac 2 (c-1).
2
4
••
I
exists) will require further research and probably numerical study.
=
V
t
=
V
a
Cat
V
e
=V
=
*=
e
204a
a
40'2 0'2
20'4
a e
e
+
+
ac
ac
-
40'2 0'2
a e
20'4
a
+
a
ac
40'2 0'2
a e
20'4
a
+
a
ac
20'4
e
a (c-1)
•
20 4
e
+ ac (c-1)
In this case,
Hence
I
10
The above suggest that unless
0
2 is very small with respect to 0 2 , one should
e
a
have that c be of moderate size for the design to be efficient in estimating
the parametric functions,
a:, a: + 0:,
a:;
and particularly
should be larger or smaller than moderate according as
17
in addition "a"
2
is larger or smaller.
a
Finally, by computations similar to those used in the next section,
one can verify that if nl
= n2 = ••• = na = c,
~: =
ga - gm
=~
A2
g - g -ae
-_ t
a
g
I
I
I
than
I
I
{MSA - MSE }
e
.-I
MSE
--
_I
where MSA and MSE are the mean squares due to A-classes and error respectively
in the one-way analysis of variance associated with the model.
balanced experiments
~:
and
~~
Thus, for
coincide with the estimators obtained by the
analysis of variance method of estimation and hence are BQUE (see Graybill
[7]; and Graybill and Hultquist [8]).
2.3. Tbe Three-Stage Nested Design
for
k = 1, 2, ••• , n .
i J•
i
=1,
2, ••• , a;
I
I
I
I
I
I
I
••
I
I
II
I
I
I
I
I
I
11
Assume
... , bama ;
. (i)
independent;
are identically distributed as N{O,02)
a ;
(ii)
the
{ai }
(iii)
the
{bij }
(iv)
the
{eijk } are identically distributed as N{O,
2
are identically distributed as N{O, °b)
;
0:) .
Hence, we have that
=
E {Y"J.J k Yt uv }
2 + 0 2 + 02 + IL 2 if i = t, j = u, k = v
0a
be r
= t,
= u,
if i
I
I
I
I
I
I
I
if i = t, j :/:
I
are
a
I_
.-
elll' ••• , e a,m ,n
a am
= ~2
where i, t
= 1,
Let us estimate
2, ... , a
j
k ~
v
u
if i i- t
j
= 1, 2,
u
= 1"
0: + o~ + 0; + ~2
2,
... , ~
;
k
= 1, 2,
... , ffi.t;
v
= 1, 2,
..., nij .
... , ntu
0: + o~ + ~2, o~ + ~2, ~2 by forming the
normalized symmetric sums of their respective unbiased estimators listed above;
that is,
a
(~
m.
~
J.
i=l j=l
n
1
ij
)-
a
~ n ij
~ ~ ~ y2
n i=l j=l k=l ijk
1
I
12
a
mol•
g b =( Z Z nij (n -1»a
i=l j=l
ij
t
1
a
Z
m.:l
Z
Z Y.
Y
}
i=l j=l kfv :ljk ijv
=
i'" Za
1
m.:l
2
n.:l2
j
mi
{ a}
y2
1
_
•••
1:
i=l
y2
i ••
m
i 2
a 2
where n. = 1: n , n= 1: n , k = 1: 1: n .. , k = 1: n.•
1 i=l j=l :lJ
2 i=l :l
:l
j=l ij
i=l i
We obtain estimators of
a
a
0:, o~, 0;, ~2
by setting·g t , gab' ga' ~ equal to
their respective expectations and solving the resulting equations.
Thus, we
obtain
"2
°a = ga - gm
°b = gab - ga
"'2
,
"'2
° e = gt - gab
,
"'2
~
= gm
.
,
1
a (y 2 _miZ y 2 ) }
Z
{ i=l i •• j=l ij.
=
}
Z (Y' j - Z Y. 'k)
(k -n) i=l j=l :l.
k=l :lJ
The same remarks that were made in the previous section apply to the variances
,
--I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
••
I
13
and covariances of these estimators.
In addition, the approach used there can
be applied to this model and hence can provide some insight to efficient designs.
Finally, the procedure can be modified at various places to provide alternative
estimators of the parameters.
ge
=
L~l
1
m.
a
J.
(n -1)
1j
i=l j=l
:E
:E
m.
J.
For example, in
t~e
case of a 2 , we consider
e
n ..
J.J
(Y ijk - Yij / } = -L
n-m
j=l k=l
:E
:E
{a m
:E
i
:E
D
1
ij
:E (Y1jk- Y
1j.)
1=1 j=l k=l
a
where m =
:E
i=l
If nil
m. •
J.
= ni2 =... = n.J.m = c i
for i
=1,
2, ••• , a, then an alternative estimator
i
of o~ is obtained by considering
m.J.
}
y 2 -:E y 2
{ i.. j=l ij.
1
(c.-l)
J.
- Y.
J...
2
) /(m. - 1)] J.
m.J.
c.J.
[:E
:E
}
2
(Yijk - Y1 . ) /m.(c.-1)]
j=l k=l
J.
J.
J.
and forming
""2
a
a
°b = gb = :E c. (gab i - ga i)/ :E c i
1=1 J.
,
'i=l
•
II
14
If
*
0 = O2 = ••• = c a = c, we might use
1
1
2
g ={ Z m.
Z (Y1j - Ii ) } /
b
1=1 j=l
...
a
f }{
m- a -
i
2
0
Z mZ Z
~ijk-Yij) } /
i=l j=l k=l
•
a
trmc
(0 - 1)
}•
Again, it should be noted that it is possible to derive any number of estimators;
we have only considered some of the simpler ones and some of the more traditional
ones.
Suppose now that
~
= ••• = nal = ••• = nab = c ;
gt
= m2 = ••• = rna = b and nIl = n12 = ... = n1b
i.e. , the experiment is balanced.
In this case,
a b 0
2
Z
Z Z y ijk
= -L
abo i=l j=l
k=l
1
gab = abo (0 - 1)
.-Ii
II
II
II
II
I
I
_I
a b
c 2
2
Z Z (Yij.
- Z Yijk )
i=l j=l
k=l
ga
=
1
2
ab0 (b - 1)
a
2
b 2
Z (Y
Z
Y )
j=l ij •
i=l i ••
gm
=
1
2
2
ab c (a - 1)
_ Z y2 )
(y2
•••
i=l i ••
a
;
I
I
I
I
I
I
I
.e· I
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
••
I
15
~ y~
11
"2
1
{ b(a-1) { Z
a Y2
a
bZ Y2} - (b-1{ y 2 _
a = g -g =
- Z
2
2
a
a m ab c (b-1)(a-1)
1=1 i..
1=1 j=1 1j
... 1=1 ~.jJ
=
~ y~~.. _y2• ..1_
(a-1){b ~
~ y2. _ ~ y~ }}
S
1=1 j=1 1J· i=1 ~ ••
1
{ (b-1) {a
ab 2 c 2 (b-1)(a-1)
1=1
1~(ZYi
a 2
= bc
= l-b
o
i=1··
t
[bc
H~
~
b 2 J - (ZY.
a 2 L )/a(b-1)]
/bc - Y2 / b )/(a-1) - (Za
ZY'
••• a c
i=1 j=1 ~ j ./c
1=1 ~··/Ib 0
a
Z (Y.
-I
i=1~·····
)2/(a_1)] - [c
a
Z
b
Z (Y
1=1 j=1
ij •
-Ii
)2/a (b-1)]
}
••
=~c {MSA - .mB}
~b2 = ga b-g a = abc 2 (b-l)(c-l)
1
{C(b-1){ ~
~
1=1 j=1
=
abo
2
={
~ ~ Y~j~ •- 1=1~ y~~..}-(b-1) lk=1
~ ~ Y~'k~ ~ y~.}}
~J i=1 j=1 ~J.
i=1 j=1
a
Z
b
Z
c
Z
2
J•
2
}
c (Yijk-I1j ) /ab(c-1))
) /a(b-1)] - [ Z
~. •
k=1·
- I.
(Y
1jk
i=1 j=l k=l
= MSE
~ (y~~ •• _ j=1~ Y~j~ ·'~111
1
{(C-1) {b
(b-l) (c-1)
1=1 j=1
1{ [c Za Zb(Ii'2
= o
~
2
2
(y1 . y ijk )} - (C-l){
J. k=1
1=1
Yij
2}
) /ab(c-1)
•
16
where MSA, MSB, MSE are the mean squares due to A-01asses, B-01asses, and error
respeotively in the analysis of varianoe assooiated with the model.
balanced
.
exper~ments
ft2 ft2 ft2
0a' 0b' 0e
Thus, for
ooincide with the estimators obtained by the
analysis of varianoe method of estimation and henoe are BQUE.
2.4. An Example of the Estimation Procedure for the Three-Stage Nested Design •
The following example consists of data generated from random normal deviates
2
2
with 0 a2 16, 0b 1, 0 e 4, ~
o.
=
1st stage
1
2
3
4
5
6
=
2nd stage
=
=
Observations
2nd stage
Sums
1
3.230, -1. 588
1.642
2
4.982, 1.144
6.126
1
4.701, 3.639
8.340
2
3.182, 3.680
6.862
1
.705, -5.215
-4.510
2
.229, -1. 717
-1.488 .
1
-6.529, -8.797
2
-7.730,
1
8.855,
2
3.373 ,
1
.757, -1.421
2
1.520
1
-2.095, 0.053
Total
Sums
7.768
-23.056
'"
I:
17.269
Ii
.856
I~
-0.664
1.520
-2.042
-2.042
7
1
8.764, 5.882
14.646
1
-3.299, -4.951
-8.250
8
9
II
II
eli
II
I
-5.998
13.896
3.373
II
I:
15.202
-15.326
-7.730
5~041
1st stage
I~
II
-8.250
16.395
I
17
II
I
I
I
I
I
I
I
I
I
I
I
I
I
I
Z
i, j, k
2:
579.8505
n
= 27
=
968.0249
k1
= 51
gab
= 16.17
= 1444.686445
k
= 87
g
=13.22
2
Y ..
~J.
i,j
2:
.
~
y~~
..
2
a
= 268.796025
g
"'2
"2
a
aa
= 5.31
e
m
= -1.83
=15.05
_2.5. The Multi-Stage Nested Design
.I
Y~jk =
(r)
+ ••• + e. 1i
Model:
~
. = 1 , 2 , ••• , m(1) ;
where ~l
Assume
{ e~2!
~J.
1 2
i
2
=1,2, ••• ,
}, ••• ,
(2)
m.
{e~r). 2
;
~l
••• ~.
2
r
(r)
; i r = 1,2, ••• , mi i ···i 1
1 2
r-
. } are independent,'
~l~···~
r
(ii)
The
{e~~l}
(iii)
The
{e~2~ 1 are
2
identically distributed as N(0,a 2 )
(iv)
The
{e~ j~
2
are identically distributed as N(0,a j ) where j = 3, 4, ... , r.
are identically distributed .s N(O,.il ;
~1~2J
. }
~1J.2 ... ,
18
Hence, we have that
E {Y..
i Yj 1 j 2··· jr}
~1~2··· r
2
2
2
= °1 + °22 + ••• + 0r + J.L if i 1 = J1' i 2 = j2'···' ir=jr
...
• • •
= °12 + 0 22 + ILr 2 ~. f i 1 = J l' i 2 = J 2' i3' "~ J 3
...,
= 1,
j2
... ,.
r
=1,
2,
......I Jr = 1, 2,
2, ••• ,
2
222
2
2
We estimate 01 + ••• + or + J.L , °1 + ••• +or_l + J.L ,
i
••• ,
... ,
... ,
222
° 1 + J.L ,J.L
by
forming the normalized symmetric sums of their respective unbiased estimators
listed above; that is,
Ii
II
II
II
II
ell
II
II
I,
II
II
Ii
II
\\
•
•
-.
I
I.
I
I
I
I
I
I
I
19
,
where n
k
and where n
2
. = ~. i
i·
r-J
J. l 2'" j
By setting gr' gr-l' "', gl' gm equal to their respective
expectations and solving the resulting equations, we obtain the estimators
I_
I
I
I
I
I
I
I
I·
I
i
. = L· ••
i 12···J. j
~
"'2
=
gm
"'2
°1
=
gl - gm
"2
°2
=
g2 - gl
°r_l
=
gr- 1 - gr-G~
"2
The same remarks concerning the variance-covariance matrix of the estimators, the
possibilities for alternative estimators, and methods of studying the efficiency
of different experimental designs apply to this general model as well as to the
I
20
special cases previously discussed.
Finally, we can show that if the experiment is balanced (i.e., all the
(2)
(r)
(2)
are equal to m(r», then the
, . • ., all the mi i
i
~l
1 2••• r-l
above estimators coincide with those obtained by the analysis of variance method
m.
are equal to m
of estimation and hence are BQUE.
The method of proof is identical with that
used in the case of the three-stage nested design and consequently exploits the
one can observe that if the experiment is balanced from some stage and onward
(i.e., all the m. (j) i
~l··· j-l
are equal to m(j) for j
~
t), then the above estimators for
the variance components from the previous stage and onward (i.e.,o~_l,a~, ••• ,a;)
are similar to those which could be obtained by an analysis of variance method of
estimation.
Appendix I.
They are not BQUE,however, for reasons which Will be considered in
.-I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
.I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
••
I
21
3.
CLASSIFICATION EXPERIMENTS
3.1 The Two-Way Classification Designs
Model:
Yij =
~
+ r i + c j + e ij
if an observation appears in cell (i,j) where
i = 1, 2, ••• , r and j = 1, 2, ••• , c •
Assume
(i)
(ii)
(iii)
(iv)
The { ri} , {Cj }, {eij} are independent;
2
The {ri } are identically distributed as N(O,o r );
are identically distributed as N(O,02);
C
The {c j }
The {eij } are identically distributed as N(O,02).
e
Hence,we have that
=
(02
+ ~2)
if i=k, J1=1.
=
(02
+ ~2)
if
r
c
= ~2
where i, k
=1,
2, ••• , r and j,
itk,
j=f.
if ifk, j*1.
I.
= 1,
2, ••• , c provided YijYkl. is defined.
2 02 + IL 2 02 + IL 2
We estimate 0r2 + 0c2 + 02e + r
IL
'
r r' C r,
rIL
2 by forming the normalized
I
22
symmetric sums of their respective unbiased estimators listed above; that is,
g
t
=1.
r
~ ~ y2
n i=l j ij
=1. ~ ~ y2
n
ij
,
1 if an observation appears in cell (i,j)
where nij = { 0 otherwise
r
c
n.J..
r
c
~
n.
= j=l
1: noH' n.j = ~ n ij , n = 1:
...,
i=l
1=1 j=l J. j
By setting
~,
r
2
k2
- ~ n
and k l-~i'
i=l·
c 2
~ n •
~
. j=l .J
=
gr' gc' gm equal to their respective expectations and solving
the resulting equations, we obtain the estimators
"'2
ar
=
,
"'2
=
,
oc
.-I
II
II
II
II
II
II
ell
II
II
II
II
II
II
II
II
I
I.
I
I
I
I
I
I
I
II
I
I
I
I
I
23
Let us now simplify the model by asswning that n 1. = n2 • = •••
n. l
= n. 2 = • • • = n .c
= q.
Hence, n
= pr = cq,
k
=nr. =p,
2
cq = rpq •
l
Thus, we may write gt' gr, e
g m
, g as
g
t
=
-L
rp
~ ~ y2
ij
,
. 1 {2
g c = cq ( q- 1)
again,
"2
~
"2
r
"'2
c
"2
e
, a , a , a
~ Y • -
.J
2}
,
~ ~ y ..
J.J
are constructed from these as above.
Let us now briefly consider the variances and covariances of the
estimators of the variance components.
assume that
~
For reasons previously mentioned, we
is a k;nown constant and, in particular, is zero.
we use the estimators
"2
ar
=gr
,
"2
a = gc
c
,
"2
a
e
= gt
- gr - gc
;
In this case,
I
these may be viewed as linear combinations of the following more basic estimators
g
_ 1
2
tr, i - P ~j Y.:l j
=p
gr,i
t!>"ll {
-1:.
gtc,j
q
=q
gc,j
,
~
i
yt - f Y~j
}
2
Yij
I
I
I
I
I
I
,
,
1
{y2.j
(q-l)
-
~
i
y~j }
•
From the theory of Appendix 2, we obtain
Vtr,i
V
r,i
C
tr,i
= Var
= Var
= COY
(gtr,i)
(gr, i)
2
2
40; (0 c + 0e )
4
+
20 +
p
r
=
2 2
2
40 r (0 c + 0 e )
p
=204r +
(gtr,i' gr,i)
+
2(0 2 + 02 )2
c
e
p
2(0 2 + 02 )2
c
e
p (p-l)
40 r2 (0 c2 + 02e )
204r +
p
_I
,
,
,
=
+
vc,j = Var
(gc,j ) = 204
c +.
Ctc,j = C0 V
= 204 +
(g tc,j g
cJ)
c
+
40 2
c
2 + 02)
r
e
q
q (q-l)
.-I
,
(0
;
I
I
I
I
I
I
I
• I
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
••
I
25
Cov (gtr,i' go,j) = nij
1[ 20~
Cov (gto,j' gr, i) = nij {20 ;
+
2
2
2
+0 )
_~C_qr~--:.:.e_
40
(0
}
/ p
,
}/q
+
,
1. e. ,
"'I ik
is the number of units common to rows i and k while
the number of units common to columns j and
= cq
(q-l) and
~
jr/.
OJ/.
=
rp (p-l).
I..
Note that
~
ifk
°j/.
"'I ik
=
is
~j {~
ni · ~ . }
i*k
J J
I
26
vr
.-II
= Var
}
222
=
=
V
Var (g )
~ c04 +
coo { c
2. 2}
222
2co (o + 0 )
c{o + 0 )
ere +
r
e
q
q (q-l)
+ 2{ I: Tljl.)(04/ q (q-l) )
j=1.
r
II
II
II
II
II
II
ell
II
II
II
II
II
II
e-
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
••
I
27
{q.4
=.:1.n e+ e
2(.2)(.2r + .2)}
a
COy
(gr' go)
ve = Var
2 2
40 r 0 c n
= pqro
(g _ g _ g )
t
r
c
2
402r 0c
= n
=a
n
=
.:1.
n
;
{2.2 .2}
r
0
;
(0 2 + 02 )2
(0 2 + 02 )2
c
e
+
+ 0 4a
+ r q-l e
c
p-l
{2.2 .2
r
Let us now look at Vr , Vc' Va for two special cases:
a.) ~ik = ~ = OJ/. for all i, k and j, I..
V =~
r
n
t
2 2 2~(r-l)oc
4 }
2 2
2
(oc2 + °e)
po + 20 (0 + 0 ) + ( 1) +
2
r
rca
pn(p-l)
4
+
2~C(C-l)0;}
n(q-l)
2'
I
28
Usually, we also have r
a symmetric BIB design.
= c,
= q,
p
Hence,
~(r-l)
~r(r-l)
n(p-l)2
=
= p(q-l)
in this case; i.e., we have
~c(c-l)
n(q_l)2
1
1
=p-l = q-l
in the above.
Note that in this case, gt' gr' gc are the simple averages of equally correlated
random variables.
b.) Balanced Disjoint Rectangle Designs (see Anderson [2], Bush and Anderson
Suppose r
and
= vq,
c
= v P and
'Y
_ {o if columns j,
0jl q if columns j,
ik
l
l
Jo if rows
i,
rows i,
=1: if
disjoint rectangles of dimensions (q x p).
20 2c 02e + 04e
vpq(p-l)
2
20 02 + 04
c e
e
qp(p-l)
[3]).
k not in same rectangle
k are in same rectangle
not in same rectangle
are in same rectangle·
.-I
The design consists of v
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
I_
29
vc =,g
v
V =
e
f
-Zvpq
..£+
p
f·
2
20 202 + 04
20 02 + 20 2c 02e
r c
r e
e
+ pq(q-l)
pq
2 2
r• c +
=1 is
}
2
2
(q_l)04
(p-l)04
04r + 20 2r 0 e + 0 4
04c + 20 2c 0e + 04e
e
r
c +
+
+
p-l
p-l
q-l
(q-l)
04
04
{ 2.2.2
c
r c
r
=~
v
pq + p(p-l) + q(q-l)
The case v
04
r
+ q(q-ll
2
(pq-l)04
20 202
20 02
e
+
c e +
r e +
p(p-l)q pq(q-l)
pq(p-l) (q-l)
the randomized blocks design.
+a;}
}.
In the above series of designs,
we reduce the correlations between the estimates from the respective rows and
the respective columns by taking observations in disjoint (and hence uncorrelated)
blocks.
As a result, some variance terms will be decreased while others will be
increased; the overall effect will determine the efficiency of the design for the
I
I
I
I
I
I
I.
I
I
situation with which an experimenter is concerned.
If we remove the assumption n • = ••• = n r. = p and n •1 = ••• = n •c = q,
l
there is a third general type of design which is of interest for experiments of the
classification type.
Here, we will call such designs "intersecting rectangles II
although the term "intersecting cylinders" would be more appropriate for higher-way
experiments.
This class of designs includes ilL-designs" (see Bush and Anderson [3])
which have the form
**
**
**
**
* *** **
******
I
30
U+
.-I
designs" which have the form
*
*
***
***
*
*
*
*
***
***
*
*
(and hence the same structure as IlL-designs"), and nS-designs" (see Bush and
Anderson [3]) which have the
fo~
****
* *- * *
**
**
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
***
***
With these designs, one constructs estimators of
(i)
(1i)
0
2
r'
0
2
c'
0
2
e
as follows:
Partition the design into non-overlapping sets of the following type
a.
isolated rectangles
b.
series of balanced d1sj oint rectangles
c.
rectangles which are the intersection of two rectangles
For each set, obtain the quadratic forms gt' gr' gc; obtain
~
for the whole experiment as indicated in the beginning of the section
or in some other appropriate fashion.
(iii)
Combine the estimators from the non-overlapping sets in an appropriate
fashion (weighting them according to some natural criteria or according
to their estimated variance-covariance matrix) to obtain gt' gr' gc for
the whole experiment;
"2
"2
"2
then construct or' 0c' 0e from these as before.
I
I
I
I
I
I
_I
I
I
II
II
II
I
.-I
I
I
I.
I
I
I
I
I
I
I
31
Such a procedure as the above is a "reasonable" though arbitrary method of
estimation.
It is not difficult to apply, but it does require a good deal of effort
to study in the general case.
However, special cases of interest to an
experimenter could be readily evaluated with the aid of a computer by using
the theory of Appendix 2.
Finally, if the design is balanced (in the sense that n .. = I for
~J
all i,j and hence p
= c, r = q),
then by computations similar to those used
in the next section, one can verify that
1_
-
~2 =
r
g
r
_ g
m
= 1c {MSR _MSE}
o"2c
= gc
- gm =-1r { MSC - MSE }
"2
= gt
- gr - gc + gm
oe
=MSB
where MSR, MSC, MSE are the mean squares due to rows, columns, error respectively
in the randomized-blocks (two-way) analysis of variance associated with the
model.
ThUS, for this balanced experiment, the recommended estimation
procedure coincides with the analysis of variance procedure and hence leads
to BQUE of
0
2
r,
0
2
c,
a
2
(Graybill and Hultquist [8]).
e
I
32
3.2
The Two-Way Classification Design with Sub-Sampling in Cells
Model:
Y
=
ijk
~
+ r
i
+ c + (rc)ij + e
ijk
j
for i=1,2, ••• ,r;j=1,2, ••• ,c;k=1,2, ••• ,nij·
I
I
I
I
II
II
Asswne
(i)
Theh} ,h} ,{ (rcl iJ}
, {
eiJk}
are independent,
(ii)
The {r i }
are identically distributed as N(O, o~)
(iii)
The {c j}
. 2
are identically distributed as N(O, 0c)
(iv)
The {(rc)i j }
(v)
The {eijk}
are identically distributed as N(O, 02 )
ro '
are identically distributed as N(O,
o:)~
e·'.1
Hence, we have that
if i = u, j = v, k = w
if i = u, j = v, k
if i = u, j
if i
= ",,2
where i, u = 1, 2,
.-I
f
j, v = 1, 2, ••• , c;
*
v
u, j = v
if i =t u, j
... , r;
+w
and
f
v
k = 1, 2, ••• , n ij •
w = 1, 2, ••• , n uv
I
I.
I
I
I
I
I
I
I
33
22
2+2
2 + 0 2 + 0 2 + It 2
It
,0
o2
r
r
c
rc r ,
r + 0 c + 0 rc + 0 e
2
r +
2
2 + 2 2
~,~
~Je
estimate
by
constructing the normalized symmetric sums of their respectiv"e unbiased
0
~, 0 Q
estimators listed above:
n
1 r
c
ij 2
gt = - ~
~
~ Y·
n i=l j=l k=l 1 jk
r
~
{
Y
~
i=l jfV ij.
yiv.} -
I_
I
I
I
I
I
I
I
••
I
1
~
~
Y.. Y }
{ i~u jfv 1J. uv.
=
2 1
{y 2 _
•••
n - k1-k2+ k12
~
Yi2
. _~y2. +~~.}
• • • J.
1J.
c
r
r
c
r
c
2
r
2
where n.1. = ~ n. j , n. j = ~ niJ·, n = ~ ~ n ·, k12 = ~
~ n ·,k1 = -l~=1 n.1.
iJ
iJ
j=l 1
i=l
i=l j=l
i=l j=l
•
c 2
k2
=~
n j
j=l·
By setting gt' grc , gr , gc , gm equal to their respective expectations and
solving the resulting equations, we obtain the estimators
I
34
"2
IJo
"2
o
= gr
"2
=g c -
r
oc
"2
orc
.-I
,
= gm
,
- gm
=grc -
I
I
,
gm
g r - gc + gm
,
I:
we could also have obtained the following alternative estimator of 0 2
e
g
e
--L
n-m
~
r
L
c ni"J
L
L (Yijk i=l j=l k=l
Yij .)
2}
where m is the number of cells occupied
(by more than one unit) by considerations previously discussed.
One can study the
variance-covariance matrix of the estimators and the efficiency of different
experimental designs by using the methods illustrated in the preceeding section.
However, as the models become more complex, this becomes more and more
difficult to do for general designs.
As mentioned before, particular designs
of interest to an experimenter could be evaluated in any given instance with
the aid of a computer.
Suppose now that the design is balanced with p observations in each
of the rc cells.
In this case,
II
II
II
e II
I)
II
I;
I
I
I
I
• I
I
I
I.
I
I
I
I
I
I
I
35
gt = -l- ~ ~ ~ y2
rep
ijk
;
,
II
36
.11
II
II
Ii
II
II
-(e-l)(p-l)(IY~j.-~~j.)
+(P-l)(Y~ •• -zr;•• -ZY~j.+EEI;j·t
=
2
1
rep (r-l)(o-l)(p-l)
-r(p-l)~
_C(P-l)zr2 j +re(p-l)~i2j
...
~..
·.
·
J(P-l)y2
L
y2
-p(r-l)(e-l)~~jk+p(r-l)(e~l)~
A2
e
a
=gt -
gre
= roptP-l)
{
(P-l)(u.IY~jk) - (~~j.- ~~jk)}
= re(~l) {~(Yijk
=MSE
- Yij • )2}
}
I;
I
I.
I
I
I
I
I
I
I
•
~
~
37
where MSR, MSC, MS(RC), MSE are the mean squares due to rows, columns, interaction,
and error respectively in the analysis of variance associated with the model.
A2 A2 A2
A2
Thus, for the balanced experiment, or' 0c' arc' 0e
coincide with the estimators
obtained by the analysis of variance method of estimation and hence are BQUE.
I
38
3.3
The
.-1\
Classification Experiment
Mu1ti-W~
Model:
for
i l = 1, 2, ... ,
~;
i
2
= 1, 2, ... ,
~;
••• ; i r = 1, 2, ... , m ;
r
i
and u = 1, 2, ••• , n i i
1 2·" r
Assume
II
II
are independent;
(ii)
The
{e~~1
are identically distributed as
(iii)
The
{e~:>}
are identically distributed as N{O,.;>
(iv) The
(v)
The
~~~~~are
identically distributed as
)J
(jlj2··· j q
ei i
i
jl j2··· j
t
for all q
=1,
such that jl
II
II
II
II
~(o,.~)
,.
N(O'.~2)
2
are identically distributed as N(O,a j j
j)
1 2··· q
2, ••• , r and (jl,j2' ••• , jq) selected from (1, 2, ••• , r)
< J 2 < ••• <
jq.
eII
I
I.
I
I
I
I
I
I
I
39
Hence, we have that
if i =j1, ••• ,i r =j r ,utw
1
.
2 222
= I.L +01-+02+°12
2 2
= I.L +02
2 2
= I.L +01
I_
I
I
I
I
I
I
I
I·
I
= I.L
where i 1 , j 1= 1, 2, ••• ,
~j
2
i 2' j 2 = 1, 2, ••• ,
1,1
~j
••• j
ir, j r
=1,
2, ••• , mr j
= 1, 2,
the normalized symmetric sums of their respective unbiased estimators listed
above:
I
40
••I
1
•
•
{
=
gm
~i2
1···
-~i2 i
1 2···
J
+•••+(-1) r-1 ~i2
i·'
1··· r
1
{y2 _~2
_••• _~2
+~2
+
+
n2_k _ •••-k +k- •••+(_l)rk
•• •
i 1 •••
••• i r
i 1 i 2 ••••••
1
r -~2
12 ••• r
I
I
I
I
I
I
+(-1)rIyi2
1··· ir·} '
_ eI
where n •.• i ••• i ••• ...;I n i
i where C is the set of all indices other than
jq
j1
C i 1 2··· r
j1,j2' ••• ' j
2
q
j = I n l'
and k j
•
•
1··· q
••• j1··· 1j q•••
By setting gt'
g12 ••• r , ••• , gm equal to their respective expectations and solving the resulting
equations, we obtain the estimators
"2
J.I.
,
= gm
"2
(71 = g1 -.
~
,
"2
(72 = g2 - gm
,
•
;2
12••• r
= g12 ••• r -g12 • •• ( r-1)_ •••+(_1)r-1gr + •••+(-1)r-1 g1+(_1)rgm
•
,
I
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
The same remarks concerning the variance-covariance matrix of the estimators, the
possibility of alternative estimators, and methods of studying the efficiency
of different experimental designs apply to this general classification model
in principally the same ways as with the special cases considered earlier in
the section.
In addition, we have that if the experiment is balanced (i.e. there
are no observations in every cell), then the above estimators coincide with
those obtained by the analysis of variance method of estimation and hence are
BQUE.
The method of proof is identical with that used in the case of the two-
way classification design with sub-sampling and consequently exploits the
special form of k , k2 , ••• , k12 ••• r here.
l
Moreover, if the experiment is
balanced with respect to some factors at all possible combinations of levels
of other factors, then estimators of the variance components associated with
these factors can be obtained by appropriately combining the estimators obtained
from the analysis of variance for each possible combination of levels of the other
factors.
The point of this is that if the experiment has special features, then
the experimenter can (if he desires) use them to construct symmetric sums which
are more appropriate to
his needs and which take greater advantage of his prior
information and the experiment's special features.
As mentioned before, the
suggested estimation procedure offers a special form applicable to general problems;
the principles behind it are very flexible and general and
more appropriate estimators for specific problems.
m~
be used to construct
I
_.
42
4.
AN EXPERIMENTS OF THE MIXED TYPE
where i = 1, 2, ••• , r; j = 1, 2, ••• , 0; k = 1, 2, ••• , ni .;
l
= 1, 2, ••• , n.j.
Assume
(i)
The
b}' b}'
trO)i j} • {Uik}.
b,{}·{
(UO)ijk} • {(rY)ij,{} •
tUV)ij!c.l} ore independent
(ii)
The
{r {04 ·
i} •
{(rv)ij,{} •
trO)ij}' {Uik }.
t
j ,{} • {(UO)ijk} •
UV) ijk,(} ore identioally distributed os N(O••;) •
N(O,o2 ) respeotively •
uv
Henoe, we have that
r
I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
I
I
43
2
2 + 0 2 + 02 + 2 + 2
=r
+ 02r + e
02 + 02 + 0O
rc
u
v
uc
rv 0 UY
if i-it
- , j--j', k--k', /.-_1.'
= lJo2 + 02 + 02 + 02 + 02 + 02
r
c
rc
u
uc
if i=i', j=j', k=k',
/.tl.'
= lJo2 + 02 + 02 + 02 + 02 + 02
r
c
rc
v
rv
if i=i', j=j',
klk',
/.=/.'
2
2
= l.L + 0r + 02 + 02
c
rc
if i=i', j=j',
ktk',
/.+1.'
= lJo2 + 0r2 + 0u2
if
i=i', jfj',
k=k',
= lJo2 + 0 2 + 0 2
c
v
if
iti',
j=j',
, 1.=1.'
if
ifi',
j=j',
, I.fl.'
IL
I
I
I
I
I
I
= lJo2 + 02
r
I_
= lJo2 + 02
c
I
I
I
I
I
I
I
.I
= lJo2
From the above, it follows that all the parameters
~re
estimable (See Appendix 1).
One obtains the estimates of the variance components by constructing the
appropriate normalized symmetric sums to estimate the above parametric functions,
then setting them equal to their respective expectations, and solving the resulting
equations for the estimators.
I
44
5.
.-I
SOME PROBLEMS IN MULTIVARIATE ANALYSIS
5.1. A Simple Two-Way' Model With Sub-Sampling in the Cells.
Model:
Suppose Yijk where i
n
ij
=1,
2, ••• , rj j
=1,
2, ••• , Cj and k
=1,
2, ••• ,
are observations from a multivariate-normal population with the
following structure:
E (Y
ijk )
Var (Yijk )
=
j.&.
=0
2
Cov (Yijk,YuVW)
for all i, j, k
;
for all i, j, k
;
=
Prc 0
2
if i
= u, j =v,
k , w
_I
if i = u, j f v
if i ~ u, j
=0
parameters as follows:
=v
if i =t u, j =\: v
wherever nijk nuvw ±
T 0 where nijk
=~ if Yijk is
o otherwise
I
I
I
I
I
I
defined.
Then we estimate the
I
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
45
"'2
=
gm
"2
a
=
gt - gm
"
Pro
=
(gro - gm)/(gt - gm)
'"
Pr
=
(gr - gm)/(gt - gm)
'"
Po
=
(go - gm)/(gt - gm)
~
,
where gt' gro' gr' ge' gm are the same as in Section 3.2.
Similar remarks
apply to estimation in these models as were made in the oase of varianoe
components models.
One should note here that for cases where negative estimates of
variance oomponents are obtained from the model of Seotion 3.2, the model
considered here may be more appropriate.
5.2. The One-Stage Model for Observation Vectors.
Model:
Y..
~J
Y2j
=
Ij
Ytj
where j
= 1,
2,
..., n •
=
:[ +
~j
=
+
I
46
Assume
{!!~
are independently distributed as N(Q, V).
Hence, we have that
E
{IjI:C} = ~ ~I
=
We estimate
~ ~I
+ V
~ ~I
+ V,
~ ~I
if
j = k = 1,
2, ••• , n
if
j :f k = 1,
2, ••• , n
by forming the normalized symmetric sums of
their respective unbiased estimators listed above; that is
G
t
=l
n
n
~
j=l
I
yl
j -j ,
••I
I
I
I
I
I
I
_I
By setting G , G m equal to their respective expectations and solving .
t
the resulting equations, we obtain the estimators
~
'I, 'I,I
..
V
= Gm
)
n
~ (y - t)(y - Y) I •
= Gt - Gm = ...l..
n-l j=l -j
- -j
-
2
2
If, as in the case of the mixed model, we have V = 0c J,t + 0e It' then
we obtain the estimators
I
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
••
I
47
"2
°e
=1t
rltr (V)"} -
,
"2
00
" is sum of diagonal elements of V.
"
where ~ is a (1 x t) veotor of ones and tr(V)
5.3· A Two-Stage Nested Model for Observation Veotors
Model:
X Jk
txl
=~
tx1
+~j
txI
+ ~ Jk
txl
for j
=1,
2, ••• , 0; k
Assume
(i)
{~j}' {§.jk }
are independent ,
(ii)
{!!oj}
are identically distributed as N(.Q, V ) I
c
(iii)
{§.jk}
~
identically distributed as N(Q, Vel •
Henoe, we have that
=-
~~.
= 'X S'
+ Vo
if j = u, k
if J :f: u
:1=
v
=1,
2, ••• , n
J
I
48
We estimate
~~'
+ Vc + Ve '
~ ~I
~ ~I
+ Vc'
by forming the normalized symmetric
sums of their respeotive unbiased estimators listed above; that is
_1
c
n
j
c
,
nj
II
Gt - n 1: 1: 1jk 1jk = 1 1: 1: I
n
j=l
k=l
jk
-jk
j=l k=l
=..lk1-n
I
I' - 1: 1:
j=l -j. -j.
j=l k=l
{
I'• •
c
",here n = 1: n j and
j=l
~
nj
CO
1:
-
c
1:
X
I
I'
}
,
jk -jk
II }
j=l -j. -j •
,
c
2
= 1: n j • By setting Gt , Gc , Gm equal to their respective
j=l
expectations and solving the resulting equations, ",e obtain the estimators
~
'X. ~I
= G
m
,
,
= Gc - Gm
ve= t
G -G
c
•
1£ Vc = 0 c2 j t' Ve = 0 2rc jt + 0 2e I t' then "'e obtain the estimators
,.
"2
L
j,' V J,; 0
=
2 t o ' ro
°c
t
"'2
"2
°e =
t{tr<v
e) }
1
= t(t-1)
-;;.
{.!i
Ve i
t-
A
tr (V )
e
}.
••I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
I
'I
I.
I
I
I
I
I
I
I
II
I
I
I
I
I
I
I·
I
49
5.4. A Two-Stage Nested Bivariate Model •
Model:
for i
=1,
2, ••• , r
and
j
=1,2,
••• , n •
i
Assume
~
are identically distributed as N
(0),(
are identically distributed as N
(0)(
0
2
,po 0
ye,
e ye ~e
po 0
0
e ye xe' xe
00
(iii>
(
Hence, ,.,e have that
E{Yij Yk,(}
2 + 02 if i = k,
= J.L2y + 0yr
j =
ye
I.
= J.Ly2 + 02yr
=t=
I.
= J.Ly
if i
2 +02
= J.Lx2 + 0xr
xe
if i = k, j =
I.
2
= J.Lx2 + °xr
if i = k, j 4=1.
2
E{Xij ~}
2
= J.Lx
if i = k, j
=t=
,.
k
if i ~ k
o~r
poo
2
y:,
: yr
) )
Pr yr xr' xr
0
;
))
,.
•
I
50
IL
IL
"'x "'y
••II
+ P r ayr axr + p e aye axe if i = k, j = l .
if i = k, j
= ~x ~y
-1=
l
if i f k
We estimate the above parametric functions by forming the normalized symmetric
sums of their respective unbiased estimators
r
ni
g =1 L
L y2
ty n i=1 j=1 i j '
r
r
L ~ - L
{ i=1 i. i=1
1
g
r
=- L
txy n i=1
ni
L X
r
j=1 ij
Y
ij'
g
ni 2 }
{ r
r
L y
Ig
= -L L X t - L
j=1 ij
rxy
k1-n i=1 i. i. i=1
r
{
1{
L
r
x2
i=1 i.'
_
~ -
{2 2}
1
y
2 k
n - 1
••
-
tx
=1n i=1
L
r Y
L
i=l i.
,2.
~y
= 2
n -k
1
X Y
••••
-
-
L
ni
L
x2
j=1 ij
;
I:
ni
}
L X Y
J=1 ij 1j
n}
i
L
x2
i=1 j=1 ij
,.
1
rL X Y
i=1 i. i.
,
r}
{
_ L X2
••
i=1 i.
x2
r
r
2
where n = L ni and k1 = L n.
i=1
i=1 1
Equating the above to their respective expectations and solving the resulting
equations, we obtain the estimators
II
II
II
II
I'
-I
I
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
51
"2
gmy
J.L
ayr
=
gry - gmy
"2
=
gty - gry
axe
y
"2
a
ye
A
"2
=
J.L
=
grnx
a
xr
=
grx - gmx
,.
"2
=
gtx - grx
;
x
"2
j
P
r
= (grxy - grrrxy)/ {(gry - gmy) (grx - gmx)} t
,
1)
= (gtxy - grxy)/ {(gty - gry)(gtx - grx)} t
•
e
One can observe that models of the type studied here (see Tukey [
])
can be generalized both in the direction of the number of components per
observation vector and in the direction of the number of effects.
Procedures
similar to those demonstrated here would be used to find the estimates of the
parameters of interest in more complex situations.
bf the estimators
Appendix 2.
m~
As always, the variances
be obtained by appropriate application of the theory of
Such considerations may lead an experimenter to appropriate
designs and refined estimators for the special situations of interest to him.
I
APPENDIX
A.l.
Basic Theory of Suggested Estimation Procedure.
We assume that the observations from an experiment can be represented by a
vector
X.
nxl
This
may be a vector of univariate or multivariate observations.
Let the mean vector and covariance matrix of I be denoted as follows:
E(I)
=~,
Var(X) = V
Our purpose is to estimate the elements of V by using the information provided
by
X.
Definition:
A parameter will be said to be estimable in the quadratic sense
with respect to a particular experiment (or model representing it) if there
exists a quadratic form
I'e I
which is an unbiased estimator of it.
From the above, we have that
E{I
X'}
= Ll Ll' +
V
= S.
It follows that if a function of the parameters can be written as a linear
function of the elements of S, then it is estimable.
In the general case, not
very many functions of experimental interest would be estimable in this sense.
However, with the models of interest to us, Ll and V, have a particular structure.
Let us now look more closely at a general random effects model.
the observations may be characterized by the following model .
We assume that
••I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
I_
53
....here ~(i), the random variable oorresponding to the k th souroe of variation,
has the following properties:
(i)
=0
E{ek(i)}
Var
for k
Cov {ek(i), ej(h) }
=0
(iii)
Cov {"k(i), "k(h)}
= .~
for k
=1,
2, ••• , s;
I
and i
= 1,2,
... , n;
for j
i, h
*
k
=1,
2, ... , r; i, h
if
ek{i)
=
"k{h)
if
ek(i)
f
ek(h)
=1,
= 1,
2, ... , n;
2, ••• , n.
The ek(i) are normally distributed.
Consequently, we have that
=
I
I
I
I
I
I
I
I·
... , r
{ek(i~ =.~
(ii)
(iv)
= 1,2,
2
2
L ok + ~
if i = h = 1, 2, ••• , n
r
k=l
=
2
2
ok + ~ if i
L
keC ih
where Cih
= {" k
Iek(i) = ek(h)}.
Let Gih
*h = 1, 2,
••• , n
= L o~ + ~2.
k£C ih
0; is estimable in the
quadratic sense if and only if it oan be written as a linear funotion of the {G }
ih
One now sees that a funotion of the parameters
~2, o~, ••• ,
The {G }are estimated by oonstruoting the normalized symmetrio sums of their
ih
unbiased estimators
•
I
54
.-I
where Tih
= {(j,.o I E~(j)
kih = {number of (j,
!)
yto} "
Qjl
= Qih }
combinations in Tih }
and
If
Cf~
may be written as a linear combination of the {gih};
G~
=I
dih
is estimable, then it
that is,
gih
where it is understood that the summation is with respect to distinct values
The suggested estimator of
2
k is then given by
G
This estimator will be a good one if the
{gih}.
If the distribution of
{Qih}
are good estimators of the
X is invariant under groups of transformations
involving certain permutations of observations or groups of observations (which
is the case with balanced experiments of the nested and/or classification
type), then the
{Qih}
often may be viewed as conditional expectations of
unbiased estimators of the {gih } given a sufficien.t statistic.
the
{Qih }
are efficient quadratic estimators.
(See Wilks (14).
As a result,
For example,
in the case of a balanced two-way classification experiment, the distribution of
X is
invariant under permutations of rows and • permutations of columns.
As a
result of these considerations, estimators which are linear combinations of the
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
••
I
I
55
I. {Q
I
I
I
I
I II
I
I
I
ih}Will be effioient quadretio estimators. This result has been proven by
Graybill and Hultquist (8); it, in part, follows from the U-like structure
they possess (See Hoeffding
(9)).
On the other hand, if I is not invariant under any particular group
of transformations, then the suggested estimators are reasonable in the sense
of being unbiased, consistent, and readily obtainable.
Moreover, if they are
of interest, efficient quadratic estimators of the Qih or estimable functions
of them can be derived oy performing weighted least squares on the elements of
where it is assumed that the weight matrix is the estimated variance-
covariance matrix of the elements of I II;
this procedure necessarily
presupposes the existence of prior estimates of the v·ariance components themselves.
I_
The concepts and procedures considered here for the variance-components
model naturally extend to the multivariate model introduced in the beginning
of the section.
Hence, if a parameter is estimable, the suggested estimator
of it is an appropriate linear function of symmetric sums of elements from the
I
I
I
I
I
I
I
I·
I
matrix I II.
A.2.
Variances of Symmetric Quadratic Forms in Normally Distributed Random
Variables.
The results in this section may be found elsewhere in the literature; in
particular, Lancaster
Lemma (2.1).
Let
y
nil
(11), Anderson (2), Anderson and Bush (3).
be distributed according to the non-singular multivariate
normal distribution l~( Q,
nx1
V).
Let S = II Q I be a symmetric quadratic form.
nxn
Then,
=
tr {vQ}
= 2 tr { [VQ)2}
•
I
56
Proof:
Let nxn
F be a non-singular matrix such that F V Ft = In; than
is N(Q,I n ) distributed.
Let W= (F-l)t Q (F- l ) and let
Consider the transformation
= FI
P be an orthogonal matrix such that P' WP = D
nxn
where D is a diagonal matrix with diagonal elements (d , d2 , ••• , d ).
n
l
y be defined by ~ = P y.
nxl
~
y = P'
~
= P' F
I.
Then
y is
Let
distributed according
to N(Q, P' F V F' p); i.e., y is N(Q, In).
yip' (F- l )' Q (F- l ) P y}
{s} = {I'QI} =
E
E
=
=
=
E{
E{
E{
E
{s}
{s} =
= var{
n
I:
n
d
i=l i
I:
n
I: d
i=l
Var
Q'DQ}
~ d ~}
i=l i
u~ }
i
n
=
di = tr(D) = tr(P'wp)
i=1
2
I: d
2
i=l i
.'I
I
I
I
I
I
I
_I
,.
=
= tr {(F-l ), Q (F- l ) }
=
~
= tr
{Q[ (F-l)(F-l ) '] }
{VQ }
since tr(AB)
= tr(BA) if A,B are squares matrices of same dimensions •
.,.,
I
I
I
I
I
I
I
.,
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
57
n
2
2
Similarly, Var {5} = 2 I d = 2 tr(D )
i=l i
=2
tr (Plwpplwp)
=2
tr (w2)
1
= 2 tr { F-11 QF-1F-1 QF-1 }
=2
Corollary (2.2).
52 =
I'Q~
Proof:
tr {(VQ)2 }
Let X be the same as in Lemma (2.1).
If 51 = X'~I and
are symmetric quadratic forms, then Cov {51' 52} = 2 tr { VQ1VQ2}.
Var {Sl + 52} =
Var {Sl} + 2 Cov {51' 52} + Var {52}
1
Var {51 + 52} = 2 tr
{[V(~ + ~)]2J
= 2 {tr[(VQ1)2] +
k
since t~ (
k
I Ai) =
i=l
Lennna (2.2).
2tr [VQ1VQ2] + tr [(VQ2)2].
I tr Ai.
i=l
Let X
be distributed according to the non-singular multivariate
nx1
normal distribution N( 1!:. , v ). Let 5 = II Q Xbe a symmetric quadratic
nx1 nxn
form. Then,
E{ 5 } = tr {( V+1&, 1&,') Q} = tr [VQ} + i!' Q 1\.
var{ 5} = 2 tr {(V+2&&, ill) QVQ}= 2 tr {(VQ)2} + 4 ill QVQ
1&; •
I
58
In addition, if 51 = II Q I
l
and 52 = II Q2 I are any two symmetric
quadratic forms, then
Proof:
Perform the same double-diagonalizing
Y= pi
proof of Lemma (2.1); i.e.,
to
N(pIF~,
F
X.
transformation as in the
In this case,
Yis
distributed according
n
2
as before, S = II Q X =yl D U = L d Ui .
i=l i
In).
2
2
.2
Now U , U , ••• ,
2
1
u~
are independently distributed and have non-central
chi-square distributions with one degree of freedom.
The respective non-
centrality parameters are the squares of the elements in pi F
= (piF 1&,),
non-centrality parameters.
{~}=
then f> l' f> 2'·'" f> n are the respective
As a result,
for
i = 1, 2, ••• , n;
for
i = 1, 2, ••• , n.
Consequently, we have that
E(5)
=
E {
~
d
i=l i
~}= ~ di
i=l
= tr
=
i.e., if we
I
let ~ I = (51' 8 2 ,,,. 8 n)
E
~;
t~
(1
+f>~)
(D) + ~ I D~.
(D) +
~I
FIP D PIF
= tr (VQ) + ~I Q il
.
J
~
••I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
••
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
Ie
I
59
and
Var (5)
=Var
{
J
2
~
d i U~
=
~
d
i=l
i=l i
(2
+
48
~)
= 2 tr {(VQ)2} + 4
~I
FI P n2 pi
= 2 tr {(VQ)2} + 4
U:,'
F' ;. F ~
= 2 tr {(VQ)2} + 4
U:,'
[F' WF F-1 FI-1 FI WF]
= 2 tr {(VQ)2
j
+ 4 11'
QVQ~
F~
•
If 51 = II ~ I and 52 = I' Q2 I where Q1' Q2 are symmetrio, then
~
60
I
.1
ACKNOWLEOODfENTS
The author would like to thank Dr. R. L. Anderson of the Department
of Experimental Statistics at North Carolina State University for aoquainting
him with the problems and results in the area of varianoe-oomponent models and
Dr. James E. Grizzle of the Department of Biostatistios at the University of
North Carolina for suggesting the applioation of the estimation prooedure to
multivariate models.
The author is deeply indebted to both of these men for
many helpful disoussions whioh have led to the preparation of the paper.
I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
·1
I
I
61
REFERENCES
I.
I
I
I
I
I
I
I
[1]
New York:
[2]
I
McGraw-Hill Book Company, Inc., 1952.
Anderson, R. L., "Designs for estimating variance components," Proc.
Seventh Conference.
Des. Expts. Army Res., Dev., Test., ARDDR 62-2
(1961),781-823. lnst. of Stat. Mimeo Series No. 310.
[3]
Bush, N., and Anderson, R. L., "A comparison of three different procedures
for estimating variance components," Technometrics, 5 (1963), 421-440.
[4]
Crump, P. P.,
'~ptimal
designs to estimate the parameters of a variance
component model," Unpublished Ph.D. Thesis, North Carolina State
College, Raleigh, 1954.
I_
I
I
I
I
I
I
I
I-
Anderson, R. L., and Bancroft, T. A., Statistical Theory in Research.
[5] Eisenhart, C., liThe assumptions underlying the analysis of variance,"
Biometrics, 3 (1947), 1-21.
[6]
Gaylor, D. W., "The construction and evaluation of some designs for the
estimation of parameters in random models," Unpublished Ph.D. Thesis,
North Carolina State College, Raleigh, 1960.
lnst. of Stat.
Mime 0
Series No. 256.
[7]
Graybill, F. A., ''On quadratic estimates of variance components," The
Annals of Mathematical Statistics, 25 (1954), 367-372.
II
62
[8]
Graybill, F. A., and Hultquist, R. A., "Theorems concerning Eisenhart's
Model II,"
[9]
Annals of Mathematical Statistics, 32 (1961), 261-269.
Hoeffding, W., "A class of statistics with asymptotically
distributions."
[10]
normal
Annals of Mathematical Statistics, 19 (1948), 293-325.
Hsu, P. L., "On the best unbiased quadratic estimate of the variance,"
London Univ. Stat. Research Memoirs, Vol. 2 (1938), 91-104.
[11]
Lancaster, H. 0., "Traces and cwnulants of quadratic forms in normal
variables," J.ournal of the Royal Statistical SOCiety, Series B, 16(1954),
247-254.
[12]
Prairie, R. R., l~ptimal designs to estimate variance components and to
reduce product variability for nested classifications,"
Unpublished
Ph.D. Thesis, North Carolina State College, Raleigh, 1962.
lnst. of Stat.
Mimeo Series No. 313.
[13]
Roy, S.N., Some Aspects of Multivariate Analysis.
New York:
John Wiley
and Sons, Inc., 1957.
(14]
Wilks, S. S., Mathematical Statistics.
1963.
New York:
John Wiley and Sons, Inc.,
.1
II
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-I
I