QUADRATIC UNBIASED ESTIMATION OF VARIANCE COMEONENTS
IN LINEAR IDDELS HITH AN EMPHASIS ON
TEE ONE-HAY CIASSIFICATION
by
James Howard Goodnight
Institute of Statistics
Mimeo Series No. 850
November 1972
iv
TABLE OF CONTENTS
Page
1.
INTRODUCTION.......
1
2.
STATEMENT OF 'rtfE PROBLEM • •
4
2.1
2.2
2.3
2.4
2·5
2.6
3.
COMPUTING LOCALLY BEST QUADRATIC UNBIASED ESTIMATORS •
3.1
3.2
3·3
3.4
4.
The Mathematical Model
Unbiasedness . • . • .
Invariance Concepts • • •
• • • •
Locally Best Quadratic Unbiased Estimators
Locally Best Quadratic Unbiased Estimators With
Addi tional In-v-ariance Restrictions . . . . .
A Suggestion For Obtaining Invariant Quadratic
Unbiased Estimators • • • • • • • • • • • • •
Forming The Matrices Needed •
Computing The A Matrix
Computing The BA Matrix
Computing Other Invariant Estimators
QUADRATIC UNBIASED ESTIMATORS FOR THE ONE-WAY CLASSIFICATION
4.1 P '1.'''''Tor'd • • • • • • • • • • • • • • •
4.2 The Mathematical Model • • • • • • • •
4.3 Invariant Quadratic Unbiased Estimators For The
Variance Components • • • . . . . . .
4.4 Re.lative Efficiencies • • • • • • • . •
4.5 Choosing Reliable Estimates With Limited Prior
Knowledge • • • • • • • • • • • • • • • • • •
4.6 Comparison of Invariant Quadratic Unbiased
Estimators To The Standard Analysis of Variance
Estimators • • • • • • • . • . . • • . • • •
5.
4
4
5
8
10
15
15
19
21
21
23
23
24
25
29
36
40
SUMMARY AND SUGGESTIONS FOR FUTURE RESEARCH
5.1 Summ.ary..
• • • • • • • • •
5.2 Suggestions For Future Research • • • • .
6.
LIST OF REFERENCES •
7.
APPENDIX • • • • •
.........
7.1 An Algorithm For Factoring A Symmetric Positive
Definite Matrix in Place • • . • . . • •
7.2 An Algorithm For The Inversion Of An Upper
Triangular Matrix in Place • . • . • ••
•• •
7.3 An In Place Generalized Inverse Sweep Algorithm •
1+5
46
50
51
52
53
nrrRODUCTION
1.
The classical approach to the unbiased estimation of -'rariance
components for
-~balanced
data is one of choosing several different
quadratic functions of the data, equating them to their expected
values, and
sol~~ng
the resultant system of equations.
There are, of course, infinitely many quadratic functions
available for equating observed to expected values, and solving to
provide unbiased estimation.
However, much of the previous work in
this area centers around quadratic forms which bear analogy to those
used with balanced data; in particular, much work has been done using
the methods outlined by Henderson [1953J.
Rao [1972J states that the
classical methods lack a clear theoretical basis and that the
classical procedures are:
intui tion ll •
"ad hoc and much seems to depend on
Numerous authors
1
have, in fact, compared two or more of
the classical estimators to determine which, if any, has the smaller
variance for a particular design.
However, the dedsion as to which
estimator among a given set is "best", always seems to depend upon the
unknown values of the components being estim.ated.
Hence, the idea of
achieving a uniformly "best" unbiased variance component estimator in
the "generally" unbalanced situation
2
appears to be improbable.
III
1 Review articles by Searle [1971J ~nd Harville [1969bJ describe much
of the earlier work in variance component estimatioIl and give some
of the important references.
2 Situations s~ch as those that arise from the loss of data, or lack
of data, etc. are implied here. Conceivably, specific designs
wi th planned imbalanced may be found which yield uniformly "best"
estimators. Rao [1971bJ provides some necessary cond.i tiOD.B for
this to occur.
-e
2
fact, Read [1961J has proven that there exists no
ql~adratic
E;stimator
of' the "between!' component in the unbalanced one-way classification
for which the variance (assuming normality) is uniformly smaller than
that of every other quadratic estimator.
Recognizing, perhaps, the fundamental difficulties which arise
from the classical approaches to variance component estimation, authors
of recent papers have focused their attention on choosing quadratic
forms with some
sort~;
of o..p timal properties.
Ha~viJ.le
[1')69aJ, f,:;,1'
example, has used thc:~ results of HUltquist aLld Graybill [1965J on
minimal suffident statistics in conjunction w'i th Koch IS [1967J J.emma
on the variance of quadratic forms to establish the basic form that the
matrix of a quadratic form should have when it is to be used for
estimating the components in a one-way classification.
Townsend [1968J
gives locally best quadratic unbiased estimators for the variance
components associated with the one-way classification with zero mean.
Recently published papers by Rao [1970, 1971a, 1971b, 1972J outline
new techniques referred to as MINQUE (Minimum Norm Quadratic Unbia.sed
Estimation) and MIVQUE (Minimum Variance Quadratic Unbiased Estimation).
Although Rao develops MINQUE and MIVQUE with
fairJ~
relaxed assumptions
on the distributional properties of the random effects involved, he
does consider the special case where the random effects are normally
distributed.
In this case
ML~QUE
aQd MIVQllli estimators are the same
and provide locally Ilbest" quadratic unbiased estimators.
r,rhe present paper is restricted to the quadratic unbiased
estimation of the variance components in linear models for which the
random effects are taken to be norma.lly distributed.
The basic:
-e
3
objectives of this paper are to extend MIVQUE theory tc provide
estimators whose variance is functionally dependent on as few of the
unknown parameters as possible, t.o provide computational techniques
for MIVQUE and its extensions, and to apply MIVQUE techniques to the
unbalanced one-way classification in order to develop an estimator for
which no
~
priori knowledge about the variance components is necessary,
yet the efficiency of which is in some sense optimal.
4
2.
STATEMENT OF THE PROBLEM
2.1
Let the
NX 1
The Mathematical Model
vector of random variables
Y have the linear
structure
(2.1)
(i ::: 0, •.. , m)
where
X.
(with
Xm:::~)'
J.
is an
13 0 is an nO
parameters, and each 13.
N X n.
J.
X 1
vector of unknown non-stochastic
(i::: 1, ..• , m)
J.
matrix of given values
is an
n. X 1
J.
vector of
uncorrelated random variables assumed to be normally distributed with
2
a-.1
mean zero and variance
(i
!
j)
l
n.
Furthermore, each
J.
are assumed to be uncorrelated.
13 i
and
From the above it follows
that
Y
fOV
Normal (Xa!3o' V) ,
where
2.2
Unbiasedness
The problem of quadratic unbiased estimation of the variance
components
... ,
such that the
•
any quadratic form in
a-
2
m
is one of choosing quadratic forms
2
::: a-.
l
(i ::: 1, ... , m) .
I
Y Q.Y
l
The expectation of
Y, as defined by (2.1), is
5
2
Thus a quadratic unbiased estimatoi' of
a matrix
such that
Q.1
1
1, ... , m)
and
if i = j }
if i F j
2.3
(i
cr.
for
1, ... , m •
j
Invariance Concepts
%
By imposing additional restrictions on
I
other desirable properties of the estimator
The concept of I!invariance
(i
~Y
Y
may be obtained.
On the translation of the
t3
t3
0
for any choice of values for the
vector
t3 *
0
condi tion for invariance with respect to
t3
X Q
type of
=
parametel,11
I
is said to be lIinvariant on the translation of the
t3*0
0
A quadratic form Y QY
is considered by Rao, Harville, and others.
condition is also necessaryl unless
... , m)
= 1,
is
0
parameter ll if
A sufficient
I
o =0
This
The appeal for this
0 .
t30-invariance can be seen in that recoding of the data by
subtracting
*
Xd30
does not alter the estimate for a particular effect.
In addition, quadratic forms in
contain any elements of
Po B '
~lf
O
Y,
(2.1),
as defined by
do not
in their variance if they are
t3 o-
invariant.
The variance of a symmetric quadratic form
l
Var (y QY)
•
•
Y/QY
2 . ~ tr QVQV + 4f3if.;QVQX.Jo
lJ
is
(2.4)
1 Expansion of the right hand side of (2.3) with the definition of Y
from
(2.1)
involves the term X~QX.m or
for (2.3) to hold for any non-trivial
X/QI
0
*
t3 0 .
which must be zero
6
Applying the definition of
V given in (2.2), the above variance may
be expressed as
Furthermore, by defining, for any real matrix
ssq(A)
A, the matrix operator
to be equal. to the sum of the squared elements of
at once that
tr (AA / ) = ssq(A).
A ,we have
Thus (2.4) may be expressed more
succinctly as
Var(Y , QY) = 2
'
)22
,/)2
E ssq (X.QX.
~.~. + 4 E ssq (
~OXoQX.
~ .•
..
~
J ~J.
~
~
~J
Clearly, if
Y/QY
is ~o-invariant, which implies
contains no elements of
quadratic form in
~
0, (2.5)
X;Q =
,
f3Jo . Thus the variance of any
~O-invariant
Y is
Var(y/QY)
=2
E
. .
~J
ssq(X~QX.)~~~~ •
~
J
~
(2.6)
J
Harville also considers a more general type of invariance in
regard to the quadratic forms arising from variance component
estimation in the one-way classification.
What he terms "Q'-invariance"
(which implies that the variance of the estimator for the within
component contains no terms involving the between component) is
expanded here to include invariance with respect to any random or nonrandom effect when estimating the components associated with (2.1).
DefiDi tion 2.1:
is said to be
For any
Y defined by (2.1), the quadratic form
~.-invariant
~
m) provided the variance of
i
(for a fixed value of
Y'QY
i
between
does not contain arw
= 0 and does not contain any terms involving
2
~.
l
a.nd
~o
i
0
y' QY
and
terms if
1= 0 •
-e
7
Theorem 2.1:
A necessary and sufficient condition for the quadratic
,
form
to be
Y QY
~.-invariant
l
in any non-trivial si tuation is that
X~Q ;::: 0 •
l
Froof:
Sufficiency is immediately seen upon examination of (2.5).
Recalling that
Xm
= IN establishes necessity also on inspection.
The following lemmas although obvious are presented to show to
what extent additional invariance restrictions may be placed on
quadratic forms and still maintain unbiasedness when estimating
2
cr.
l
(i ;::: 1, ... , m)
~.-invariant
if
J
Lemma 2.2:
cr.2
l
(i
=
J
n. X n
for some
l
l
~uadratic
No
l
for any
Denni tion 2.2:
l
matrix
j
then it is
L.
~.-invariant
estimator exists for
~i-invariant
estimator exists for crj
l
unbiased
n. X n.
l
J
The quadratic form
provided that it is
~.-invariant
J
2
L.
matrix
,
Y QY
invariant quadratic unbiased estimator of
the set
~.-invariant,
1, ... , m) •
X. = X.L
J
X. = X.L
is
Y QY
No quadratic unbiased
Lemma 2.3:
if
I
If' a quadratic form
Lemma 2.1:
is said to be a maximally2
cr.
for as many
l
(i = 1, ... , m)
j's
as possible among
j;::: 0, 1, .•• , m and still maintains its unbiasedness
property.
By using a maximally-invariant
quadratic unbiased estimator, one
insures that the Itgoodnes s It of the estimator depends on the vB.lue of
the parameter being estimated and on as few of the other parameter
·e
8
values as possible.
This property of maximally-invariant estimators
is particularly applicable when little or no
~
priori k..l1.owledge is
available concerning the relative magni tudes of the variance components
being estimated.
2.4 Locall¥ Best Quadratic Unbiased Estimators
When the normality of
Y is assumed) Rao's [1971a] MINQUE and
Rao's [1971bJ MIVQUE estimators coincide and provide locally best
invariant
1
~lE's
m
~
2
K.~.
) where the K. 's are k..l1.own constants.
i=l l l
l
In the context of (2.1) Rao [1972J proves that for any positive
definite matrix
ditions that
for
~o-
W) the minimum of
,
XOQ = 0
and
tr QWQW
,
(i = I) ... ) m) is obtained
tr X. QX. = K.
l
l
subject to the con-
l
when
m
Q=
~ 0l' RX.X~R )
i=l
l l
where
(2.8)
R
0' =
and
(° 1 )
°2)
••• ) Om)
is determined from the equation
. . )th
( l)J
-
where
element of
8o = K
8
is
ssq(X~RX.) •
l
J
A quadratic form)
y' QY) where
m
E(Y'QY) = !:
i=l
1
Q is defined by (2.7 L satisfies
2
K.~.
l l
)
QUE is henceforth used to denote quadratic unbiased estimator.
-e
9
. (
) = 2
Var
Y 'QY
and
tr QWQW
'
)22
Z ssq (X.QX.
rr.rr.
1
J 1 J
lJ
. .
2 tr QVQV ,
is a minimum subject to the unbiasedness and invariance
restrictions.
When
V the minimum variance t30-invariant
W
is obtained.
When reliable
§:
("2
priori estimates
rrl'
2
QUE
of
ZK. rr.
... ,
;i)
are
available for the components being estimated, then the
1
m
1
W matrix of
(2.8) may be computed as
m
W
=
Z
i=l
X.X.,'"rr.2
1
1
1
Furthermore, since the restrictive minimization of
equivalent to the restrictive minimization of
for any
is
Z p.p. ssq(X~QX.) where
..
lJ
(i = 1, ..• , m)
tr QVQV
1
J
1
J
k = 1, 2, •.. , m , the following
W matrix may be used in (2.8) to obtain a minimum variance 13 0 invariant QUE of
2
ZK. rr.
1
1
m
W=
Thus the minimum variance
Z
i=l
X.X~ p.
1
1
1
t30-invariant QUE of
2
ZK. rr.
1
1
may be realized
if the ratios of all variance components to a common variance component
are known.
Rao [1971bJ presents necessary conditions for obtaining a minimum
variance t3 -invariant QUE irrespective of the values of the variance
0
components being estimated.
required of the
X.1
However, due to the restrictive conditions
(i = 1, ... , m)
matrices, it would seem that few
of the generally unbalanced designs met in practice could qualify.
Further inspection of these conditions may lead to specially constructed unbalanced designs (perhaps nested) for which minimum
variance t30-invariant. QUE's are possible.
10
-e
Rao [l972]
SUgb2:"~S
tLLLt when no
~
priori knO"l'lledge is available
for the components being es timo.'cecl Ghi.;:.t the
W matrix of (2.8) be
computed as
m
W ==
Using this
L:
i==l
X.X~
l
l
W matrix corresponds to assigning equal
~
priori weights
to the unknown variance components, and results in the minimization of
L: s s q (X ~ QX .) •
ij
l
J
Due to the lack of additional invariance constraints, the variance of
any t30-invariant
QUE
in the unbalanced case will, in general, have
all terms in the variance expression (2.6) greater than zero.
Hence,
the "goodness" of any estimator will depend on the actual values of all
of the components being estimated.
In the event that some components
are large relative to others, use of (2.9) as Rao suggests could lead
to estimators with possibly undesirable variances.
si tuations where
QUE's
~
priori knowledge is not
would be less riskY.
Rao's
extended to include t3.-invariant
l
maximally.,.invariant
2.5
QUE' oS
availabl~ maximally-invariant
t30~invariant
QUE's
Perhaps in
QUE's
are easily
(i == l, .•• , m)
and thus
•
Locally Best Quadratic Unbiased Estimators With
Additional Invariance Restrictions
Locally best t30-invariant
invariant QUE's for one or more
13·l
which are in addition 13.l
(i == l, ... , m)
13Q' denote that collection of 13.l 's
here.
Let
which
t3.-invariance is sought.
l
QUE's
are considered
for
(i == 0, ... , m)
In the context of this section
t3
Q'
-e
1.1
13
is assumed to contain at least
f3Q'
can not contain
(i
m)
1, ..• ,
Obviously, based on Lemma 2.3,
.
0
Also denote by 13
13m •
not contained in
I
that collection of
The term f3
13Ol
Ol
13.l 's
-invaria..rlt is
used to denote f3.-invariance
with respect to all f3. 's
l
contained in
l
130'
Furthermore, let
130'
13
and
and
Oln
represent the number of elements in
In
respectively, (Q'n + In = m+l).
1
by
X
11
in
,
13 •
I
X
Let
,,2
cr
,
... ,
, ... ,
Y
components
cr
2
l
of
13
Y
Ol
,,2
cr
be
In
cr
2
Yn
§:
Q'n
,and denote
priori estimates of the variance
respectively, associated with the elements
•
The problem, then, is one of finding 13 -invariant
QUE's
0'
2
K. cr. ,
l
(where
X
the incidence matrices associated with the elements
In
11
0'1
13
the incidence matrices associated with the e.1ements in
... ,
x , ... ,
Denote by
KY.' ••. , KY
1
for
(2.10)
l
are a set of predetermined values) for which
n
ssq(X I QX ) "cr2 ,,2
cr
y. y. y. y.
J
l
l
(2.11)
J
is a minimum.
In light of Lemma 2.3, (2.10) is not estimable if f3 -invariance
0'
implies invariance with respect to any of the elements of
which the corresponding
f3 y
for
K value in (2.10) is non-zero.
By constructing the matrix
In
W
I
.
1::
l=y
1
,,,2
X.X.cr. ,
l
l
l
(2.12)
-e
12
(2.11) may be rewritten as
tr QW QW
)'
)'
and by letting
the solution to t3 -invariant
QUE's
ex
is apparent by applying Rao's
basic MINQUE theorem from the previous section.
t3 -invariant
QUE
minimized is
Y'QY
ex
To be specific, the
for (?lO),if it exists,for which (2.11) is
where
Q is determined as follows:
)'n
Q ==
.
l
L:
5.RX.X~R,
l
l
l
==)' 1
where
and
0 I == (5
,
K
where
, ... ,
)'1
==
is determined from the equation
S5
==
. . )th
( l,J
-
S
is
(KY '
and the
element of
K
1
ssq(X' RX
)'
.
l
)'.
).
J
The above results imply that if t3. -invariant
l
more values of
QUE's
for one or
i == 1, ... , m are sought in addition to t30-invariant
QUE's, then one has only to include those
t3.l 's
(for which
t3.l
invariance is sought) with the set ~O of fixed effects and proceed
with Rao's MINQUE estimation procedure as if all the random effects
inclUded with
t3 0 were fixed.
result of the above.
The following theorem is an interesting
13
The unique minimum variance
Theorem 2. 2:
~aximally-invariant
QUE for
~2m in model (2.1) if rank (X) < N is
2
0-m
where
=
y'(I - X(X'xrX')Y/[N - rank(X)],
X
Proof:
The variance of a maximally-invariant
y' QY , for
QUE,
~
2
m
is
SSq(Q)~4m =
2
which is minimized whenever
value of
4
tr QIQI
QIQI~4
,
m
is a minimum, regardless of the
Thus, to find a maximally-invariant
~
m
is necessary to find
tr Q
From (2.13),
2 tr
1,
for
QUE
~
2
m
, it
Q such that:
,
X Q
0,
and
Q = 0lRR , where
tr QIQI
is a minimum
R = I - x(X'X)-X'
and
01
is
determined from the equation
ssq(R)ol = 1 •
Since the idempotent matrix
rank(X)
R is unique, and
ssq(R) = tr (R) = N -
the theorem is proven.
2.6
A Suggestion For Obtaining Invariant
Quadratic Unbiased Estimators
~ a-invariant
value of
i
QUE's
between
1
which are also
and
~.-invariant
1
for some fixed
m have the property that
ssg (X~QX,) = 0
J
l
since
~.-invariance
l
for all
implies
j
X~Q
0, 1, •.. , m ,
If (2.7) is computed with a
O.
l
(2.14)
matrix
W=
m
'
,E
,,,,2
X.X. cr.
1=1
where
",2
is an
cr.
l
~
l
l
l
,E
(2.15)
2
priori estimate of
ij
,
for
cr.
l
i
1
)'" 2t- 2
ssg ( X.QX.
cr,cr.
l
J
(2.16 )
J
l
is minimized subject to the necessary constraints.
"practical"
.
~.
correspon d lng
l
1, ..• , m , then
It would seem that
-invariance could be achieved by replacing the
,,2
cr.
l
in (2.15) by a number whose magnitude is
sufficiently larger than that of any of the
is minimized, the coefficients,
large values of
~~IS.
J
Hence when (2.16)
ssg(X!Q,X.), associated with the
l
J
will be forced to be relatively small.
15
·e
3.
COMPUTING LOCALLY BEST QUADRATIC
UNBIASED ESTIMATORS
3.1
Forming The Matrices Needed
(2.7)
Although equation
~O-invariant
provides the theoretical basis for
(2.1),
QUE's in models described by
a more computationally
oriented procedure is given here.
Suppose that
denoted by
~
priori estimates of
r , .•. , r
are available.
m
l
Q
wi~l
2
v
'"'"m
(i :=: 1, .•. , m) ,
Then upon defining
r.X.X.
~
i'=l
(2.7)
and by applying equation
v
'"'"l2./
,
m
w
matrix
Pi --
l
l
(3.1)
l
W
to this
be generated such
for some set of
K.'s, a
l
that~
E(y'QY) :=: ~ K.O"~ ,
ill
Var(y'QY) :=: 2
'
) 0".0".
2 2
ssq (X.Q;X.
~
. .
J
l
lJ
l
J
,
and
~
ssq(X~QX.)r.r.
ij
l
J
l
J
will be minimized sUbject to the conditions
tr X~ Q;X. :=: K.
l
l
X'Q
o
o
and
(i:=: 1, ... , m) •
l
Thus, values of
r. "close" to
l
will yield an estimator
whose variance is "close" to that of the minimum variance
QUE
of
2
~ K. 0".
ill
~O-invariant
16
-e
Using a suitably chosen positive definite matrix
W as is given
in (3.1), define the following matrices:
x~w-:sco
A
X/W-:SC
0
,
=
X/W-:SC
'--
(3.2)
X/W-:SC
0
where
o
B =
(3.3)
I
and the matrix product:
-
r--
,
BA =
X/RX
where
R = W-
l
X/RY
- W-~o(X~W-~orxbw-l •
Since the submatrix
I
X/RX =
X/RX
o
X/RX of
I
X RX
1 1
X RX
1 2
X;RX l
X
..
X/RX
~
m 1
/
P2
(3.4)
can be partitioned as:
·..
· ..
··
I
·
X RX
m 2
· ..
1
-
X1RXm
X/RX
2 m
··
·
X/RX
m m
-
-(3.4)
-e
17
the quanti ties
(2.7),
equation
8. .
lJ
= s s q (X l~ RX J.) ,
(i, j = 1, ... , m) , needed in
can be computed directly from
(3.4).
Q matrix defined by (2.7) is:
Also note that the
hence
m
=
y' QY
o.Y'RX.X~RY
1::
l
i=l
m
ssq (X~RY)
sUbmatrix
X'RY
l
8i ssq(X~RY) •
1::
i=l
Thus the element p
l
(3.5)
are also available from (3.4) since the
l
may be partitioned as:
X'RY
2
X'RY =
X'RY
m
The
Q. 's
l
that
of (3.5) are computed from the equation
-1
8
is non-singular
0 = 8
.•• , m)
the elements of
K will be:
80
=
K and when estimating
K •
2
0".
l
Provided
(i
:=
1,
if i := j
otherwise
Thus, since
8
and
8- 1
(if it eXists) are s~etric, the
vector associated with the estimation of
8-
1
•
Hence by defining
~,
:=
O"~l will be the
[&i, &~, ... ,
&;J
and
jth
0
row of
18
T
I
=
[SSq(X~RY), SSq(X;RY), •• " ssq(X~RY)J ,
(3.6)
However, (3.6) may also be derived by equating the
T values to their
expected values since:
E(T. )
1.
m
l:
j=l
2
ssq(X~RX. )O"j •
J
1
Hence
E(T)
S l:
where
Since the solution to the system of equations
T
A
=
S l:
is invariant under linear row operations, provided
any set of linear combinations of the
equated to its expected value and
of full rank.
1.
l
%Y,
i
is of full rank,
LIT, can be
"E be solved for, provided
Thus by forming the quadratic forms:
y
where
T. 's , say
S
= 1, 2, ••. , m ,
LIS
is
-e
19
Q1.'
:= RX,X~R
1. 1.
then equating each quadratic form to its expected value and solving,
t3 0-invariant
for each
QUE's
variance t30-invariant
3.2
QUE's
2
0-,
1.
are obtained which are minimum
provided
Computing The
Although the matrix
r
i
:= Pi
(i:= 1, •.. , m) .
A Matrix
A of (3.2) can be computed directly, an
alternate method will be presented which in general will require less
co~puter
storage.
Since the matrix
W is assumed to be positive definite, there
exists a non-singular matrix
i
would be
p:= CA
vectors of
where the
P such that
One such
W:= pp'
P
C matrix represents the characteristic
W stored columnwise, and
Ai
represents a diagonal matrix,
whose diagonal elements are the square roots of the characteristic
W.
roots of
Unfortunately computer routines require both the upper
triangular portion of the
W matrix and the
C matrix which is
N X N to be resident in core.
An
alt~rnate
method for deriving
triangular portion of
P which requires only the upper
W to be in core can be used.
Since the Forward
Doolittle method as described by Rohde
[1964J factors
into two triangular matrices, the
matrix and
Ad
Ed
~
matrix, say
matrix, such
that
where
Ad
is' an upper triangular matrix and
is a diagonal matrix with diagonal elements
:= (DAd)' , where
:=
1
~
1.1.
Hence
D
W,
·e
20
w
and by letting
P :;;: A/Di ,
d
the
W matrix may be expressed as
W
pp '
(A computational algorithm for forming
p'
is given in Appendix
7.1).
Hence
Since the
p'
described above is an upper triangular matrix, it
may be inverted in place using a slightly modified version of the "sweep
procedure'outlined by Schatzoff et al. [1968J as described in Appendix
7·2.
Thus the
A matrix may be computed
~fficiently
via the following:
STEP 1:
Form the upper triangle portion of the
W matrix.
STEP 2:
Use row operations to convert it in place to the
Ad
matrix
of the Forward Doolittle.
e
STEP 3:
Divide each row of the resultant matrix by the square root of
the dia,gonal element of' that row.
-e
21
STEP
4: Invert the resultant matrix in place using the modified sweep
procedure outlined in Appendix 7.2 and name the inverse matrix
l'
(hence
W- l = 1'1 ).
Zo = LX O '
STEP 5:
Compute
STEP 6:
Compute the
2;
= LX , and G = 1Y
A matrix of (3.2) using the matrices of STEP 5
above as follows:
ZIG
o
,
A
Z' Z0
___
Z/Z
(3.8)
Z'G
-
however, only those elements of
A for which
i
~
need be
j
stored.
3.3 Computing The BA Matrix
Once the
A matrix has been computed using (3.2) or (3.8), by
applying Rohde's modification of the Doolittle procedure (to produce a
generalized inversion routine) to the sweep routine outlined by
Schatzoff (see Appendix
nO
7.3),
columns to produce the
the
A matrix may be swept on the first
BA matrix of
(3.4)
in place •.
Once this is accomplished, all the elements needed to estimate all
variance components are at hand and equation (3.6) may be employed.
3.4
Comvuting Other Invariant Estimators
i
From the results of section 2.5, locally best
which are in addition locally best
values of
~.~invariant
~
~O-invariant
QUE's
for one or more
i = 1, ... , m, may be achieved by augmenting the
matrix with the
X.
~
matrices associated with those
QUE's
~.'s
~
X
o
for which
22
invariance is sought, and by redefining the estimated variancecovariance matrix as in (2.12).
Hence, the preceding techniques may
be used to compute estimators which are locally best
QUE's
in addition to being locally best
~O-invariapt
Once locally best ~O-invariant QUE's
are computed using
(3.6),
the
associated with
(30'
achieved.
QUE's
~.
1
(3.3)
2
rri
(i
QUE's.
= 1, ••. , m)
could be swept on
's as if they were columns
The remaining submatrices in
could then be used to compute new
invariant
for
BA matrix of
the columns associated with one of the
~.-invariant
1
X/RX
s and T matrices, and
I
X RY
oDd
~.
1
-
for the remaining variance components could be
23
-e
4.
QUADRATIC UNBIASED ESTIMATORS FOR
THE ONE-WAY CLASSIFICATION
4.1
Foreword
The preceding sections outline the theory and the computational
procedures for obtaining locally best
~O-invariant
variance components associated with model (2.1).
the concept of
values of
~O-invariance
tq include
Section 2.5 extends
~i-invariance
for one or more
i = 1, ••• , m , which in turn leads to the concept of
maximally-invariant estimators.
heavily restrict the form of
general (except for
~
QUE's for the
2
).
m
0-
Although maximally-invariant QUE's
Q, uniqueness
~an
not be claimed in
Thus, in the "generally" unbalanced case
priori estimates of the variance components (for which invariance is
not sought) must be supplied to completely specify the estimator for a
particular component.
When
~
priori estimates are not available, are
there other relatively "safe" prior estimates, based on the design
characteristics, which could be used to p,revent the estimator from
having a variance much larger than that of the "best" estimator were
the true variance components known?
In seeking a partial answer to
this question, we take the approach suggested by Harville and work
with
a simplified model, that of the unbalanced one-way classi-
fication.
As stated by Harville, "virtually every problem that
arises in constructing a complete bOdy of theory for the estimation
of the variance components associated with a comPlex possiblyunbalanced random or mixed model is also encountered, though in simpler
form, in carrying out that process for the possibly-unbalanced one-way
24
random lUodel".
By extending our knowledge of this model, "we
effectively place an 'upper-bound' on what we can hope to achieve in
the way of theoretical results for more complicated models".
4.2
~he
Mathematical Model
The one-way classification model is represented here as:
(4.1)
where
•.• , Yal' •.• , Yan )
a
X is an
o
~
NX 1
vector of
l's,
is a fixed but unknown constant,
Xl is an
NX a
matrix whose
for l's in elements
R
jth
column is zero except
thru 'R
l
2
where
+ ••• + n .. and
J
a'
= (aI' ... ,
aa)
where
a
is assumed to be distributed
2
Normal (0, laO'Q') ,
E'
=
) where E is
, ..• , Eal , ••. , E
an
lnr
a
2
assumed to be distributed Normal (0, IN~e) , a.l'ld a.1
(Ell' ..• ,
and
E..
J.J
E
for all
correlated.
i
and
j
are assumed to be un-
25
·e
From the above,
y ,.., Normal (XolJ., V) ,
where
4.3 Invariant Quadratic Unbiased Estimators
For The Variance Components
Although Harville provides locally best IJ.-invariant
cr
2
e
The variance of any J,L-invariant
and
l
for
through use of Lagrangian functions, a simplier
derivation is given here by applying equation
(K
QUE's
QUE, y' QY ,for
l<lcr;
+
~cr;
are predetermined constants) is:
K
2
Var(y'QY)
(2.7).
=2
4
2 2
4
ssq(X 'QX1)cr + 4 ssq(X 'Q)cr cr + 2 ssq(Q)cr ,
l
a
a e
e
l
where
Hence, infinitely many IJ.-invariant QUE's for
generated by choosing
the matrix
tr Qr =
~
Q
r
~
~cr:
priori estimates, r's, of
such that:
X'Q
or
+
p
K2cr~ can be
and determining
"'·0,
, and minimizing
(4.2)
-e
26
When
11.0-:
r = p , the minimum variance
+
~C1;
we have from (2.7) that the
y'Q Y
minimized (4.2) is
01
QUE
for
is obtained.
Since (4.2) is equivalent to
and
~-invariant
and
02
r
'
tr Qr Wr Qr Wr
~-invariant
QUE
where
of
where:
are determined from the equation:
ssq(X~RX1)
ssq(X~R)
Kl
61
(4.4)
ssq(X~R)
Since
-1
W
ssq(R)
6
K
2
2
can be expressed in closed form, (4.3 ) simplifies to:
C11J 11
C12J 12
C1aJ la
C12J 21
C J
22 22
C J
2a 2a
+ 10 2 J
Qr =
C J
1a ai
where each
e
C..
~J
J ..
~J
C2aJ
2a
represents an
(4.5)
C J
aa aa
ni X n.
J
matrix of l'sJ and each
.. + h.. +l(i=j)] + 20 t .. + °2mij
° 1[(ni + n.J )t ~J
2 lJ
lJ
.
-e
27
The quap.ti ty
~=J
=j
and zero otherwise.
and
m ..
are computed as follows:
gi =
r/(l
value of one if
t .. ,
~J
h.. ,
~J
represents the indicator function, which has a
l-(. .)
i
iJ
The remaining quantities
Let
+ rn )
i
for
(4.6)
i = 1, ..• , a ,
and
G
=
-lIeN - ~. n~g.)
~
~
~
Then
and
The elements of (4.4) can be expressed as:
ssg(R)
=N+
2
~ n.n.t .. + 2 ~ nkt kk '
ij ~ J ~J
k
122
2
Z n.n.t' j + 2 Z nkt kk ,
ij ~ J ~
k
ssg(X1R) = N +
= Z n k2
k
+
~
ij
222
n.n.t .. + 2
~
J
~J
3
and
~ nktk~
k
~
28
·e
Although the above equations could possibly be simplified, they
do serve the purpose of reducing the amount of computer storage required
to study I-L-invariant QUE I s of
lS.cr~ + K2 cr: which minimize (4.2).
Examination of (4.5) leads to the fact that the estimator
l
is a function of the sufficient statistics
2
Z y...
. . lJ
l,J
the class sums) and
••• , Y
Y'Q Y
r
a'
,i.e.,
--
Furthermore (4.5) meets the necessary
condi tions specified by Harville [1969a] for "quadmissibili tylf since
if
For any
n.
l
Q
r
= n~
l
and
n.
J
= n
,
j
•
matrix computed using (4.5),
and
=
4
,2 2
2 ssq(Xl'Qr Xl)cra + 4 ssq(X1Qr)cr
cr
ae
4
+ 2 ssq(Q)cr ,
r e
where
= 4 Z n. (n.C .. + 02) + 2
i l l II
Z
i,j>i
2
n.n.(n. + n.)C .. ,
l
J
l
J
lJ
and
2 ssq(Qr )
=
22
2 Z [no (C .. + 02) 2
+(n. - n. )C .. J + 4
i
l
II
l
1 See Hultquist and Graybill [1965J.
l
II
2
..
Z n.n.C
l J lJ
i,j>i
29
-e
The only restriction that need be placed on the value of
-lin.
that it not be equal to
for any
~
i
=
... ,
1,
r
is
a ; in this case
(4.6) becomes infinite for some value of i .
4.4 Relative Efftciencies
Having stated the model and provided a technique for
infini tely many }.1-invariant QUE I S for
gener~ting
2
2
Kl (J"a + !<2(J"e ' we are now in a
position to study the behavior of the variances of estimators generated
by using
(4.5) with a variety of values for
~ ~
0 , and to compare
them with the variance of the. minimum variance }.1-invariant QUE.
Throughout this section
Q
r
will denote the matrix generated by
(4.5) for a specified value of r
of
Kl
K2'
and
(4.5) with
The matrix
~
Q
0
denotes the matrix generated by
p
=p
r
KJ.. and
For any values of
K
2
minimum variance }.1-invariant QUE for
for any
for any
r
~
and for some specified values
0
r~lative
to
(not both zero),
KJ..(J";
r F 0 •
+ K2
0": .
Y'Q Y is the
P
The
The efficiency of
y'Q Y
r
Y'QpY denoted by
Since little can be shown theoretically about
Eff(Qr IQ p )
due to the
algebraic complexity of the problem, the current author employed
equation
(4.5) in a computer program to examine Eff(Qr IQp ) over a
range of values of
r
and
p
classification designs. Tables
for a number of different one-way
4.1, 4.2, and 4.3 are a sample of the
tables produces in studying the efficienty of the }.1-invariant QUE for
2
(J"a
(K
l
= 1,
K2
= 0).
Tables
4.4, 4.5, and 4.6 are a sample of the
Table 4.1
r=O
r=.25
r=l
r=5
r=10
r=100
r=1000
r=10000
1
ANOVA
e
e
e
Eff (Qr IQp ) when estimating cr; in a one-way classification model with cell frequencies:
3, 5, 59, 20, 50, 21, and 89
p=O
p=.25
p=l
p=5
p=10
p=100
p=1000
p=10000
1.000000
0.150487
0.053123
0.034322
0.032262
0.030465
0.030288
0.030271
0.587925
0.545024
1.000000
0.898088
0.788522
0·770279
0·752912
0.751126
0.750947
0.807093
0.441949
0·933258
1.000000
0·981cxJ7
0.974931
0·968404
0·967695
0.967624
0.671921
0.399638
0.874939
0.983854
1.000000
0.999613
0·998537
0·998388
0·998373
0.611748
0.393582
0.865512
0·979049
0·999621
1. 000000
0·999662
0·999589
0·999581
0.602986
0.387957
0.856543
0·974cxJ9
0·998596
0·999668
1. 000000
0.387385
0.855619
0·973535
0·998456
0.999598
0·999997
1. 000000
1.00000-
0.387328
0.855527
0·973481
0·998441
0·999590
0·999996
1.000001.000000
0·593903
0·999997
0·999996
0.594819
0.593987
1 The efficiency of the standard ANOVA estimator (described in section 4.6) is included here for
comparative purposes
\>I
o
e
e
Table 4.2
Eff(Qr IQ)
when estimating rra2 in a one-way classification model with cell frequencies:
p
22, 52, 33, 88, 68, 48, and 25
p=O
r=O
r=.25
r=l
r=5
r=10
r=100
r=1000
r=10000
;;NaVAl
e
1. 000000
0.559071
0·513653
0·500592
0.498931
0.497430
0.497279
0.497264
0.844936
p=.25
p=l
p=5
p=lO
p=lOO
p=1000
p=10000
0.655821
1.000000
0.996715
0.994470
0.994140
0·993833
0.993802
0.620161
0·996790
1.000000
0·999713
0.999635
0·999555
0·999547
0.999546
0.859781
0.609706
0.608368
0·994316
0·999637
0·999995
1.000000
0.999996
0·999995
0·999995
0.848586
0.607159
0.607037
0·993992
0·999551
0·999981
0·999995
1.00000 1.000000
1. 000000.847303
0.607025
0·993989
0·999550
0·999981
0·999995
1.000001.000001.000000
0.841291
0·993799
0.891645
0.994632
0·999715
1. 000000
0·999995
0.999983
0.999981
0.999981
0.849871
0~994022
0·999559
0.999983
0·999996
1. 000000
1.000001. 000000.847420
1 The effici€ncy of the standard ANeVA estimator (described in section 4.6) is included here for
comparative purposes
\.>J
i-'
e
e
Table 4.3
r=O
r=.25
r=l
r=5
r=10
r=100
r=1000
r=10000
PJifOVA1
e
when estimating ~2a in a one-way classification model with cell frequencies:
Eff(Qr IQ)
p
1, 33, 94, 78, 1, 64, 91, 69, 72, 1, 24, and 42
p=O
p=.25
p=l
p=5
p=10
p=100
p=1000
p=10000
1.000000
0.184756
0.01ll97
0.002280
0.001775
0.001401
0.001367
0.001364
0.646481
0·743460
1.000000
0·766674
0.338290
0.280231
0.231693
0.227077
0.226618
0.906109
0.662315
0·934801
1.000000
0.863808
0.805017
0·739401
0·732190
0·731462
o~ 816273
0.571334
0.815404
0.550680
0·786971
0.898059
0·995820
1.000000
0.528928
0.756779
0.867824
0·983743
0·995623
1.000000
0.526564
0·753486
0.864453
0·982032
0.994662
0·526326
0·753154
0.864113
0·981856
0·994560
0·999937
0·999999
1.000000
0.650962
0·925139
1. 000000
0·995427
0·980770
0.978587
0.978361
0·706138
0·995258
0·994171
0·994055
0.680848
0·999947
0·999936
0.654158
0·999948
1.000000
0·999999
0.651255
1 The efficiency of the standard ANOVA estimator (described in section 4.6) is included here for
comparative purposes
'vi
I\)
e
e
T-able 4.4
e
2
Eff(Q.-r IQ)
when estimating rre in a one-way classification mode~ with cell frequencies:
p
3, 5, 59, 20, 50, 21, and 89
p=O
p=.25
p=l
p=5
p=10
p=100
p=1000
------ .p=l-DOOO
r=O
1.000000
0.140769
0.000017
0.000000
0.000000
0.992288
0.006756
0.838361
0.001706
r=.25
0·700228
1.000000
0.569698
0.000001
r=l
0·990311
0·999020
0.000136
0.013387
0.434284 . 0.007642
0.000077
r=5
r=10
0·989969
0·996338
0·730918
0.026453
0·999755
1.000000
0.286479
0·999814
0·999995
0·999990
0.975695
r=100
0.989955
0.989949
0·998695
0·998681
0.998676
r=1000
0.989949
0·998676
0.999814
0·999990
0·999997
1.00000-
0·999732
1.00000-
r=10000
1
ANOVA
0.989949
0.998676
0·9998J..4
0·999990
0·999997
1.00000-
0·999997
1. 000000
1. 00000-
0·989949
0.998676
0·999814
0·999990
0·999997
1.00000-
1.00000-
........-.
_.
'-
0·992585
1.000000
0·999833
0.999819
"#.
._.--...
-
0·996873
1. 000000
0·987058
0·999979
1. 000000
0·999997
1
1.000000
1.00000-
1 The efficiency of the standard ANOVA estimator (described in section 4.6) is included here for
comparative purposes
VJ
VJ
Table 4.5
--
e
e
Eff(Q IQ) when estimating ~2
r p
e
22, 52, 33, 88, 68, 48, and 25
p=O
in a one-way classification model with cell frequencies:
p=.25
p=l
p=5
p=lO
p==lOO
p=lOOO
p=lOOOO
r=O
l. 000000
0·707l0l
0.l42656
0.006806
0.00l7l7
0.0000l7
0.000000
0.000000
r=.25
0.9972';f8
l.OOOOOO
0·99ll0l
0·965369
0.2l8622
0.002792
0.000028
r=l
0·997227
0.999976
0·999675
l. 000000
0·999B09
0·980934
0.339785
0.005l2l
r=5
0·997223
0·999972
0·999998
0·999955
l. 000000
0·999966
0·996596
0.745375
r=lO
0·997223
0·999972
0·997223
0·999972
l.OOOOO -
l. 00000
0·999998
l. 000000
0·999784
l.OOOOO -
0·978849
r=lOO
0·999998
0.999998
r=lOOO
0·997223
0·999972
0.999998
r=lOOOO
l
ANOVA
0·997223
0·999972
0·999998
l.OOOOO -
0·997223
0·999972
0.999998
l.OOOOO l. 00000 -
l. 00000 -
l.OOOOO -
l.OOOOO l.OOOOOO
l.OOOOO
l. 00000
-
l. 00000
l.OOOOOO
-
l. 00000-
l. 00000
l. 00000
l. 00000 -
-
0·999998
l.OOOOO l. 000000
l. 00000-
l The efficiency of the standard ANOVA estimator (described in section 4.6) is included here for
comparative purposes
\.>J
+"
e
e
e
Table 4.6
Eff(Qr~IQ)
p
when estimating
~2e in a one-way classification model with cell frequencies:
1, 33, 94, 78, 1, 64, 91, 69, 72, 1, 24, and 42
p=O
p=.25
p=l
p=5
p=lO
p=lOO
p=lOOO
p=lOOOO
0.201057
0.-010270
0.002597
0.000026
0.000000
0.000000
0.997504
0·785052
l. 000000
0.995835
0·908862
0·728468
0.028563
0.000297
0.000003
r=l
0.995061
0·998645
l. 000000
0.962757
0.216443
0.002787
0.000028
r=5
r=lO
0.993360
0.997080
0.247723
0.003286
0.996942
0·999949
0·999822
l. 000000
0·970352
0·993222
0·999123
0.998990
0·990550
1.000000
-0.802567
0.039101
r=lOO
0.993166
0.996887
0·998935
0·999902
0·999972
r=lOOO
0·993166
0.993166
0.996886
0·998935
0·999901
0·999972
0·997570
l. 000000
1.00000-
0·999969
1.000000
0·996867
1.00000-
0·996886
0·998935
0·999901
0·999972
1.00000-
1.00000-
1.000000
0·993166
0·996886
0·998935
0·999901
0·999972
l. 00000-
1.00000-
1.00000-
r=O
1.000000
r=.25
r=lOOOO
1
ANOVA
I
1 The efficiency of the standard ANOVA estimator (described in section 4.6) is included here for
com~arative ~ur~oses
\.).J
\Jl
36
tables produced in studying the efficiency of the
~-invariant
QUE for
2
ere (Kl :::: 0, ~ :::: 1) •
4.1 thru 4.6 seems to indicate the
Close examination of Tables
follow:tng:
when estimating
approaches
P
or
2
er
e
using equation
(fixed) from above or below,
increases to a value of one.
approaches
2
er
a
r
Eff(Q IQ)
Furthermore, it appears that as
Eff(Q IQ)
r
P
p
(fixed).
In tables not shown, the
= n2
to one any time
4.5
:;::: ••• == n
P
Eff(~IQp)
ere'
seems to approach an asymptotic value rather quickly as
beyond
r
monotonically
2
Also, when estimating
as
monotonically
P
r
(fixed) from above or below,
increases to a value of one.
(4.5),
r
Eff(Qr IQp )
increases
was equal
a
Choosing Reliable Estimates
With
i
Limited Prior Knowledge
If a reliable estimate is avaLI.able for
estima~e
2
er
a
anq.
in
er
(4.5)
2
e
p, then use of that
should yield fairly efficient estimators for both
due to the seemingly monotonic
prope~ties
In many situations it might be possible to bracket
upper and lower bounds.
true value of
p
Eff(Qr IQp ) •
with feasible
In other words, if we are confident that the
lies somewhere between, say,
choosing a particular value of
(4.5)
p
of
r
between
Po
and
and
Pl ' then
to use in
would seem the most logical choice.
By observing Tables
4.1 thru 4.6, one can see that tbe line
associated with any choice of
r
has its smallest efficiency either
in the first or last position of the line.
This smallest efficiency
-e
37
represents the "worst" one can do if he uses that particular
the true value of
P
bounds of the table.
between
and
P
r
and
in fact lies somewhere within the upper and lower
If we are certain that the true value of
l '
the~for
any value of
r
P
+les
we use in (4.5), the
worst efficiency we could possibly have would be the smaller of either
Eff(~IQ
) or Eff(Q IQ ). Conceptually, we could build ~ list
Po
r Pl
containing all possible r values along with the worst possible
efficiencies associated with each of them assuming
and
of
r , we could pick the
r
lies between
To minimize the risk we are taking in choosing a value
r
value aspociated with the largest
efficiency in this list (of worst efficiencies).
of
P
By using this value
in (4.5) we could provide not only the variance component
estimates based on this
r
value, but also a "guaranteed" efficiency
level for the estimators, which would merely be the worst possible
efficiency associated with that value of
4.1 thru 4.6, one can see that the
On closer examination of Tables
search area for the "best"
r
and
Pl.
r*
then
r*
If we denote by
may be restricted to values between
the value chosen
is equal to the value of
F(r) = min[Eff(Q IQ
r
is a maximum over the range
r.
Po
$
r
Po
),
Po
by this technique,
for which
Eff(Q IQ )J
r
P
r S P .
l
(4.7)
38
-e
The following graph (Figure 4.1) illustrates the above points.
Efficiency
1
r
Figure 4.1
r
*
Typical efficiencies when estimating
(J
2
or
a
(J
2
e
Although the above curves are merely representations of the
efficiencies, the following point can be made:
has a value of 1 1I11.o:;;J.
approaches
P
l
;)
o
from below and since
F(r)
Eff(Q IQ
r
and monotonica.1J.,y decreases as
Eff(Q IQ
and monotonically decreases as
the function
Since
r
r
P
l
Po
)
r
) has a value of 1 when
approaches
Po
ftom above,
of (4.7) is represented by the portion of the two
curves closest to the
r
axis.
As can be seen
F(r)
achieves a
maximum at the point of intersection of the two curves, and it is at
this intersection that
r*
is realized.
Since the theoretical
formulas for these curves are intractable,
can be used for determining
the following algorithm
r*
Algorithm 4.1:
STEP 0:
Assign the desired values for
Kl'
K,
2
and set
EO
=
desired accuracy of F(r*) (~.~., the algorithm terminates
-e
39
when a value of
IEff(Q IQ
r
Set
STEP 2:
Set
STEP 3:
Compute
4:
Po
is found sUGh that
) ~ Eff(Q IQ )\ ~ EO) .
r Pl
and
STEP 1:
STEP
r
r
r
=
L
+ (r
H
- r )/2 •
L
Qr using equation
Set
F
STEP 5 :
If
IF\
STEP 6:
If
F < 0 , go to STEP
=
(4.5).
Eff(Q IQ ) - Eff(Q IQ )
r
Po
r
Pl
~ EO' set r * = r and terminate, otherwise:
7; otherwise set r L
r , and go to
=
STEP 2 .
STEP
7:
Set
r
H
=r
and go to STEP 2.
r*
Since the search area for
is halved at each iteration in the
above algorithm, convergence is quite rapid.
presented in sectio~
Several examples are
4.6 which compare the Qr * estimators (r* being
generated by algorithm 4.1) with the standard analysis of variance
~2
estimators for
a
and
~2
e
When no knowledge about
P
is available other than assuming
P ~ 0 , then algorithm 4.1 may be applied by setting
equal to a pseUdo value for infinity.
instances suffice.
Of course the smaller
the better, especially when estimating
2
~e
A choice of
P
l
~
2
a
' one need not be concerned about making
since it appears that for any fixed value of
p.
o=0
and
Pi = 1000 would in m2-st
can prudently be made
However, when estimating
P
l
as small as possible,
P, the
approaches an asymptotic value quite rapidly as
the value of
P
r
Eff(Q IQ )
r
P
increases beyond
In other words, if it were known that
p were less
40
than say 100, then any choice of
PI
~
100 , whether it be 100 or
10,000 would not make any appreciable change in the choice of
Based on the above discussion, we
r*
uses the
Definition
r*
*
define an estimator which
(4.1).
of algorithm
4.1: The Relatively Safe Quadratic Unbiased Estimator
2
+
Kl~a
( RESQUE ) for
where
for~ally
r
2
K2~e
,
Y Q *Y ,
in the one-way classification is
is determined by algorithm
r
4.1 and Q * is determined by
r
(4.5)
equation
4.6
with
r
r
*
Comparison of Invariant Quadratic Unbiased Estimators
To The
Sta~dard
Analysis of Variance
Es~imators
The standard analy$is of variance estimators for
~
2
a
are:
0-2 = Y'EY ,
e
and
0-a2
y'AY,
where
a-I
N-a I] ,
and where
A is an
elements are
etc.
Both
1/n
Y'EY
l
N X N diagonal matrix whose first
, whose next
and
y'AY
n
2
n
diagonal elements are
l
diagonal
1/n
2
are ~-invariant estimators, and the
,
-e
41
matrices
of
E
(4.5).
and
A have the same structl,lre as does the
In terms of
(4.5),
the
1
matrix
Qr
E matrix has:
1
-(~)(-)
if
N-a n.l
i
== j
if
i
c..
lJ
o
otherwise
and
6
In terms of
(4.5),
2
1
~
==
N-a
A matrix has:
the
K
~
K
a
(N-l)
N-a
n.
l
N
j
K
a
otherwise
N
and
(a-i),
K
62
a
N-a
where
K
N
a
Since both the
A and
Qr ' is there a value of
the
A
and the
r
E matrices?
E matrices have the same structure as
which could be used in
known, setting
r:::; co.
to generate
By Theorem (2.2), the estimator
is a maximally-invariant estimator.
generated by setting
(4.6)
Therefore the
E matrix may be
Hence, if an upper bound on
p
is
+ equal to that upper bound will produce a more
efficient estimator than the standard analysis of variance estimator
42
-e
The corresponding value of
not as easily determined.
l'
associated wi th the matrix
However, by making numerous computer runs,
the current writer found that a value of
harmonic mean of the
I'
= Iln h
n.l 's ) produced a matrix
approximately equaled
Y'AY
Iln
h
p
(where
n
such that
for all cases tested.
knowledge is available which indicates that
from
A is
h
is the
Y'Q Y
l'
Thus, if prior
has a value diffe:!:'ing
, then use of that knowledge in algorithm 4.1 should pro-
duce a more efficient estimator.
To compare further the standard analysis of variance estimators
wi th
~-invariant
QUE's, the following tables give the worst efficiency
achievable provided
p
l i es between
of variance estimators, the
based on setting
r =1
RESQUE
in (4.5).
and
Pl ' for the analysis
estimators, and the estimators
-e
43
Table 4.7
Component
to be
Estimated
cr2
a
cr
2
e
Table 4.8
Smallest obtainable efficiencies when 0 ~ p ~ 10000 of
RESQUE, ANOVA and Qro1 estimators for a one-way classification model with cell frequencies: 3, 5, 59, 20, 50,
21, and 89
Smallest
RESQUE
Efficienoy
Smallest
ANOVA
Efficiency
Smallest
Qr=l
Efficiency
.0427
.62780
.58792
.05312
40.2cb9
·98994
·98994
.00007
RESQUE
r * value
Smallest obtainable efficiencies when 1 ~ P ~ 10 of
RESQUE, ANOVA and Qr=l estimators for a one-way classification model with cell ftequencies:
21~ and 89
Compo.qent
to be
Estimated
2
cr
a
cr
2
e
Table 4.9
Component
to be
Estimated
cr
2
a
cr2
e
3, 5, 59, 20, 50,
Smallest
RESQUE
Efficiency
.Smallest
ANOVA
Efficiency
Smallest
Qr=l
Efficiency
1·9097
·99419
.60298
·97904
3.2500
·99985
.99981
·98705
RESQUE
r * value
Smallest obta~nable efficiencies when O~· P ~ 10000 of
RESQUE, ANOVA and Qr=l estimators for a one-way classification model with cell frequencies: 22, 52, 33, 88, 68,
48, and 25
Smallest
RESQUE
Efficiency
Smallest
ANOVA
Efficiency
Smallest
Qr=l
Efficiency
.0222
.84952
.84493
·51365
16·7847
·99722
·99722
.00512
RESQUE
r * value
-e
44
Table 4.10
Smallest obtainable efficiencies when 1 ~ P ~ 10 of
RESQUE, ANOVA and Qr=l estimators for a one-way classification model with cell frequencies:
22, 52, 33, 88,
68, 48, and 25
Component
to be
Estimated
(J"
(J"
2
a
2
e
Table 4.11
Component
to be
Estimated
(J"
(J"
2
a
2
e
Table 4.12
Smallest
RESQUE
Efficiency
Smallest
ANOVA
Efficiency
Smallest
Qr=l
Efficiency
1.8438
·99990
.84858
.99963
5·5000
,99999
·99999
·99980
RESQUE
r * value
Smallest obtainable efficiencies when 0 ~ p s 10000 of
RESQUE, ANOVA and ~=l estimators far a one-way classification model with cell frequencies: 1 1 33, 94, 78, 1,
?4" 91, 69" 7~,· 1, 24, and 42
Smallest
RESQUE
Efficiency
Smallest
ANOVA
Efficiency
Smallest
Qr=l
Efficiency
.057 4
·70109
.64648
.01119
82,0923
.99316
·99316
.00002
RESQUE
r * value
Smallest obtainable efficiencies when 1 ~ p ~ 10 of
RESQUE, ANOVA and ~=l estimators for a one-\\'ay clasi3ification model with cell frequencies:
1, 33, 94, 78, 1,
64, 91, 69, 72, 1, 24, and 42
•
Component
to be
Estimated
2
(J"
a
2
(J"
e
Smallest
Qr=l
Efficiency
RESQUE
r * value
Effi~iency
Smallest
ANOVA
Efficiency
2.0624
.96166
.68084
.89805
3.6016
·99925
·99893
.96275
Smallest
RESQUE
5.
SUMMARY.AND SUGGESTIONS FOR FUTURE RESEARCH
5.1
Summary
Rao's recent works on MINQUE and MIVQUE provide estimators of the
variance components in linear models.
If normality is assumed, then
MINQUE and MIVQUE coincide, and provide locally best quadratic unbiased estimators which are
effects in the model.
invari~t
to translation of the fixed
As pointed out in section 2,
estimators, which are based on
~
~o-invariant
priori estimates of the components
being estimated, have variances which are, in general, functionally
dependent on all of the true values of the parameters in the model.
The basic weakness of MIVQUE is then, that when
~
priori estimates are
not "close" enough to their true values, or when no
§:
priori estimates
are available, the variance of MIVQUE estimators could in many instances
be unacceptably large.
To help reduce this risk, the concept of in-
variance with respect to one or more of the random components was introduced.
These additional invariance concepts, remove certain terms from
the variance of an estimator in order to protect against situations for
which these terms might be large.
The concept of maximally-invariant
estimators was introduced to denote those estimators for which the
variances are as free as possible of terms involving other components
in the model.
In section
3, techniques are presented for computing MIVQUE
estimators, and also for locally best quadratic unbiased estimators
which are invariant to other random effects in the model.
In that
section, Qne can see that MIVQUE is actually equivalent to selecting a
set of sums of squares from a weighted regression analysis, to equate
46
them to their expected values, and to solve to get variance component
estimates.
To provide some insight into the choice of the prior estimates to
use
when either limited or no
i~ MIV~UE,
able, section
~
priori knowledge is avail-
4 examines what can be done in this respect when working
with the one-way classification.
Relatively Safe
~uadratic
A new estimator, referred to as the
Unbiased Estimator
(RES~UE),
is developed
which in some sense minimizes the risk one takes in choosing the prior
estimates of components to use in
5.2
MIV~UE.
Suggestions For Future Research
One of the problems which arises in
weighting matrix which must be inverted.
MIV~UE
is the size of the
Although in section 3
techniques which can be used are presented, these still require in
general the inversion of the
N X N weighting matrix (where
presents the number of observations).
N re-
Unless techniques can be
developed which circumvent this inversion process,
rendered an interesting but somewhat useless
MIN~UE
technique~
will be
Along this
same line, using an identity matrix as the weighting matrix might
prove quite efficient for those designs which are not very unbalanced,
such as those arising from experiments which were initially balanced,
but for which some observations have been lost.
One of the other computational
prob~e~s,
so far as the size is
Gonoerned, is the amount of storage required to compute the
portion of (3.4).
.
1S
I rr2 ,where
e
X/RX
Assuming the last effect in the model being analyzed
I
is
N X N and
2
rr
e
is the error variance, then
47
the quantities
ssq(X~R), ssq(X;R), ••• , ssq(X~_lR),
and
ssq(R)
are
needed in the expectation of each sums of squares being computed.
Since
the
" X R, ••• , Xm_1R
, , and
X1R,
2
R represent the major portion of
X'RX matrix, computing them in some other fashion would reduce
considerab~
the amount of computer storage required.
In addition to the computational problems described above, there
remains the problem of what can be done when no
available.
c~n
~
priori estimates are
Hopefully the work presented in section 4 describing what
be done in this situation for the one-way classification can be
extended to cover the more general linear models discussed in section
Although Rao [1971bJ presents the basic theory for obtaining
minimum mean square quadratic estimators when the true values of the
variance components are known, Rao [1972J suggests that iterative
estimation using MINQUE's may provide estimators with interesting
properties.
These "interesting properties" might include sma.ll mean
square errors, obtained through the use of equation (4.5) and the more
general techniques of section 3.
As aid to future research in this area, and to make available
MIVQUE's for experiments of under 250 observations, the present writer
has developed a computer procedure which is available in the
statistical Analysis System of North Carolina State University at
Raleigh.
A description of the system and the MIVQUE procedure is
given by Service [1972J.
48
-e
6.
LIST OF REFERENCES
Harville, D. A. [1969aJ. Quadratic unbiased estimation of variance
components for the one-way classificatiop. Biometrika 56:313-326.
Harville, D. A. [1969b J. Variance com.ponent estimation for unbalanced one-way random classification - a critique. ARL Report
69-1080.
Henderson, C. R.
components.
[1953J. Estimation of variance and covariance
Biometrics 9:226-252.
Hultquist, R. A. and F. A. Graybill [1965J. Minimal sufficient
statistics for the two-way classification mixed model design.
JASA 60:182-192.
Koch, G. G. [1967J. A general approach to the estimation of
components. Technometrics 9:93-118.
vari~lce
Rao, C. R. [1970J. Estimation of heteroscedastic variances in linear
models. JASA 65 :161-172.
Rao, C. R. [1971aJ. Estimation of varianc~ and covariance components
MINQUE theory. Journal of Multivariate Analysis 1:257-275.
Rao, C. R. [1971bJ. Minimum variance quadratic unbiased estimation
of variance components. Journal of Multivariate Analysis
1:445-456.
Rao, C. R. [1972J. Estimation of variance and covariance components
in linear models. JASA 67 :112-115.
Read, R. R. [1961J. On quadratic estimates of the interclass
variance for unbalanced designs. J.R.S,S. B 23:493-497.
Rohde, C. A. [1964J. Contributions to the theory, computation, aDd
application of generalized inverses. Ph.D. Thesis, Department
of Experimental Statistics, North Carolina State University at
Raleigh, Raleigh, North Carolina. Untverisity Microfilms,
Ann Arbor, Michigan.
Schat~off,
M., R. Tsao, and S. Fienberg [1968J. Efficient calculation
of all possible regressions. Technometrics 10:769-779.
Searle, S. R. [1971J. Topics in variance com.ponents estimation.
Biometrics 27:1-76.
Service, J. W. [1972J. A User's Guide To The Statistical Analysis
System. North Carolina State University Students Supply Stores,
Raleigh, North Carolina.
-e
•
Townsend, E.C. [1968J. Unbiased estimators of variance components in
simple unbalanced designs. Ph.D. Thesis, Biometrics Unit, Cornell
University, Ithaca, New York. University Microfilms, Ann Arbor,
Michigan.
50
•
7.
APPENDIX
51
7.1 An Algorithm For Factoring A Symmetric Positive
•
Definite Matrix in Place
Denoting by
W the symmetric positive definite matrix which is
to be factored, only the upper triangular portion need be computed and
stored in core.
The algorithm presented here computes the
P
I
out-
lined in (3.7) and stores it in the same core positions previously
occupied by the upper triangular portion of
W without using any other
storage locations for intermediate results.
STEP 0:
Compute the
N(N+l)/2
elements of
Wand denote them by
Wi j
(i::;: 1, ••. , N), (j ::;: i, ..., N)
STEP 1:
Set
k
STEP 2:
Set
STEP 3:
Set
k + 1
and set
K + 1
and if
D
and set
W
•
kk
:=
l > N , then go to STEP
Wij ::;: Wij - WkiWkj/D
k::;: 0 •
for each
5.
(i::;: l, ••. , N)
and
(j ::;: i, ... , N) •
STEP
4:
If
k <
N,
then go to STEP 1.
STEP 5:
Set
i::;: 0
STEP
6:
~et
i ::;: i + 1
STEP
7: pet
STEP
8:
If
and
Wij ::;: W1' J,/D
D=,r;!.
'\!wi i
for each
i < N , then go to STEP
labeled
W
ij
are now
j
::;: i, ..• , n •
6, otherwise stop; the elements
P~ .•
1J
The above algorithm stores only the upper triangular portion of
·
le '1S a 11 zeros.
P I sl'nee the 1 ower tr1ang
In the event that the matrix
W is not of full rank, a singularity check may be inserted folLowing
STEP 1 as follows.
52
STEP la:
D<
If
~ero),
7.2
E
l
(where
E
then stop,
l
> 0 represents a pseudo value for
W is singular.
An Algorithm For The Inversion Of An Upper Triangular
Matrix in Place
Assuming that the elements above and inclUding the diagonal of an
upper triangular matrix denoted by W..
lJ
••• , N)
~
(i
1, ••. , N),
(j
= i,
have been stored, the following algorithm replaces the
elements of
-1
W
W with the elements of
without additional working
storage needed, and is a modification to the sweep technique presented
by Schatzoff,
et~.
STEP 0:
Set
k
::;:
0
STEP 1:
Set
k
::;:
k + 1
STEP 2:
Set
i
::;:
i + 1
set
j = k - 1
Set
j
set
W..
lJ
STEP 3:
STEP
4: If
STEP 5:
If
STEP 6:
Set
STEP
j <
::;:
,
j + 1
::;:
D ::;: W
kk
and if i ::;: k
set
and if
j
,
an,d set
,
then go to STEP 5; otherwise
k , then go to STEP
W.. - wikwkj/D
lJ
N , then go to STEP 3
Set
STEP 9:
If
4; otherwise
.
i < k , then go to STEP 2 •
W ::;: -Wil!D
ik
for each
i
= 1,
7: Set Wkj ::;: Wk/D for each j :: k,
STEP 8:
i = 0 •
W
kk
... , k •
... , N •
= liD.
k < N , then go to STEP 1; otherwise stop; the upper
triangular portion of
W has been replaced by the upper
triangular portion of
W
-1
•
53
The above algorithm takes advantage of the fact that the
i~v8rse
of an upper triangular matrix is an upper triangular matrix and hence
neither
W nor
be stored.
-1
W
needs the lower triangular portion of'
In the event that
check as was given in Appendix
7.3
to
z:-~ros
W might be singular, a singularity
7.1
may be inserted following STEP 1.
An In Place Generalized Inverse Sweep Algorithm
As pointed out by Rohde the abbreviated Doolittle method may be
modified to produce a generalized inverse by setting any row of the
Ad
matrix of the forward Doolittle to zero if its diagonal element
goes to zero during the forward Doolittle procedure, and by setting
the corresponding column of the
B
d
diagonal element which is set of 1.
to the
Ad
and
B
d
generalized inverses.
matrix to zero except for the
ApplYing the backward Doolittle
matrices thus defined, then produces a
This same technique applied to the sweep routine
outlined by Schatzoff et al. would imply that if any pivat element
goes to zero then the matrix will be considered swept on that pivot
ele~ent
once the row and column containing that pivot element have
been set to zero.
Using the modified sweep routine on pivot elements 1, 2, ••• , nO
of the matrix
A of (3.2) would thus be equivalent to mUltiplying
by the matrix
B
of (3.3) yielding the resultant matrix
A
BA
produ~t
of (3.4).
Assuming only the elements
stored (!.~., (i
n
c
A on and above the diagonal have been
= 1, ••. , n r ) and
(j = i, ••. , n c )) ,where
represent the number of rows and columns respectively of
nand
.r
A, the
follo¥ing algorithm performs a sweep on each of the pivot elements
I'
1, •.• , nO ' with the result that the elements
the corresponding elements of the
STE:f 0:
Set
k = 0 •
STEP 1:
Set
k
STEP 2:
If
=
k + 1
D > €l
and
D
(where
BA matrix of
l
STEP 3:
4:
Akj
:=
for
0
i
=i +
set
j
=i
STEP 5:
Set
j
=
j + 1
STEP 6:
Set
B =
gik
C
=
-1
i
=
for
0
... , nr
= k then go to STEP 8·, otherwise
and if
if
j = k , then go to STEP 7.
i < k
,
and
,
and
otherwise
A
{~
-A
if
k < j
otherwise
-
BC/D •
A.. = A..
lJ
lJ
j <n , then go to STEP 5·
c
If
STEP 8:
i < n , then go to STEP 4.
r
Set Aik = -AiJD for each i = 1,
If
Set
Set
~
STEP 10:
and.
1, ••• , k
; then go to STEP 10.
STEP 7:
STEP 9:
i
.
kj
set
ik
= k,
and if
1
ki
set
A
.
Set i = 0
Set
STEP
j
(3.4).
is a pseudo value for zero), then go
to STEP 3; otherwise set
set
lJ
Akk •
=
E
A..
If
Akj = ~/D
for each
j = k,
... , k
... , n c
Akk = lID •
k < nO ' then go to STEP 1·, otherwise stop; the
matrix of
(3.4) has replaced the A matrix of
(z.
.. ./ •
0) •
c:..
BA