UNIVERSITY OF NORTH CAROLINA
Department of Statistics
Chapel Hill, N. C.
A NOTE ON THE BEST LINEAR UNBIASED ESTIMATES
FOR MULTIVARIATE POPULATIONS
by
J. N. Srivastava
November 1962
Contract No. AF 49(638)-213
In this note, we present a lemma on the
best linear unbiased estimates for multivariate populations.
This research was supported by the Air Force Office
Research.
Institute of Statistics
Mimeo Series No. 339
o~ Scienti~ic
A NOTE ON THE BEST LINEAR UNBIASED ESTIMATES FOR MULTIVARIATE
POPULATIONS
1
by
J. N. Srivastava
University of North Carolina
= ==
;::::
:=
::::;:
= = - -- - - - = == = = = = = = = = = ; ;
== = ==== == = = =
Consider the usual multivariate linear model
(1)
Exp(Y)
nxp
where
,
=
A~
man nJ.:h.'1l
n > m and where
Y ) =
Y = (Xl' !..2' ••• , --p
(2)
f~(l)i,
'•
=
,
I
f:ll'
Y1 2'
Yn1 ' Yn2'
L
·.. , YIp - \
·.. .
• •• J
Ynp
)
J
say
lIen) J
is a matrix of np
(3 )
r."
'::"
=
opservationsj
f:ll
~;lPl
S12
LSml sm2
..
\
(4)
= 1,
2, ••• p)
is a known matrix and
=
(~l' §.2'
where the dispersion matrix Z
say
We further as surne that the vectors II( )~)
are all uncorrelated and that for
Var (X (r--) )
... , 1p ) ,
r'''mp~\
is a rna trix of unknown parameters.
(r
A
r
= 1,
2, ••• , n:
, say
is also unkno\vn.
The model (1) for the j-th
variable reduces to
IThis research was supported by the Air Force Office of Scientific Research.
(5)
Exp (y. )
AF
==
.'J
Var (Y )
jr
.2.j
==
j
(J ••
JJ
== 1, 2,
. .. p,
r
==
1, 2,
... n .
If we consider just the j-th variable and ignore the rest, we can obtain from
(5),
the best linear unbiased estimate
~j ~j
mxl vector such that
(6)
Q ==
u
Lemma:
c.
"'J
is any
Let
u
mp unknown parameters, such that for each
j,
Let
"
5.j
is the best linear unbiased estimate of
Q.
Let
z
== -b l' -Yl + ••• + -p-p
bl Y
be any other linear unbiased estimate of
mpxl
c! ~., where
-J -J
P
L:
p
c!
== L:
j==l -J
Then we show that
of
c' f:!.
j==l ._j-j
is estirnable.
(7)
-J
-J
is estimable.
be a linear function of all the
c I r_oj 2.j
A
S.
c!
Q.
Then, provided that the space of the
vector
• • • •., S"mp )
contains at least mp
linearly independent points, we must have
Var (z)
> Var (u),
wha.tever the population dispersion matrix L:
of normality is involved.)
Proof:
Suppose
..... +
(8)
d' y
-p -p
Since
Exp (z) == E(u)
==
Q,
may be. (Notice that no assumption
we have
Exp (z - u)
=
°,
or
=
£1' S2'
-
for all
••• , -p
S•
(b! - d jt
-J
where
OlIn
servations
(10)
is a
)
A
°,
This however implies
= 0lm'
= 1,
j
2, ••• , P ,
Also since
1 x m matrix.
b.
-J
and
d.
are free of the ob-
-J
Y, we have
Var (u)
P
=
Z
j=l
= Var
dt Y )
(~l."l + •••• + -p
-p
(~j ~j)
+
CI' ••
JJ
(-J
d! I -J
d .) CI'j J' r
,
and similarly
Var (z)
=
Let
P
+
I:
(b j' b. )O"j .
J
j=l - -J
A be of rank
r
Let
A.
j
b.)
t
-J
CI' •• t
JJ
W be the vector space of rank n-r, which
~l' .~2J1
Let
••• , ~n-r
... +
be the vector space of rank
columns of
A.
be an orthogonal
~jl' ~'2' ••• , ~.
J
J,n-r
such that
b. = -J
d. + ~'l
Q + IJ. ·2 _Q +
2
--J
J -l
J
vi
-
'W. Then fram (9), there exist constants
(j = I, 2, ••• p)
(11)
t
and let
is orthogonal to the columns of
basis of
(b t
I:
j1j
Then since
c!
S.
-J'-J
IJ..
Q,
J,n-r -n-r
j
r (orthogonal to 'W) generated by the
is estimable as a univariate problem for the
j-th variable, it follows that
A
Rank (A) = Rank (
t)'
~j
= 1, 2, ••• P •
j = 1, 2, ... , P ,
4
i
and hence that
€
j
W, for all
j •
Hence we have from (11),
b! b. =
-J -J
b!, -J
b.
2
• •• + IJ.j,n_r
d ' d.
- j -J
= '-J
d!, -J
d, +
-J
lJ.'l
lJ.'l
+ ••• + IJ..J, n - r fl..,
J
J
J ,n- r
Therefore we get
Ver (z) - Var (u)
+
p
n-r
= z
s=l
n
=
~
s=l
But since
f
~
j=l
2
IJ.js
JJ
(
~
IJ..
j~j'
+
0" ••
~
j~j'
n-r
~
s=l
JS
"t-"J' S "t-"J" s )
IJ.j's O"jj'
-
r
IJ.' Z -sIJ.
7, where -s
IJ.' = (1J.1S ' 1J.2s'
-s
Z
is positive definite,
Z -8
IJ.
> 0,
IJ.'
-S
Since however
z
unless
is different from
0" • ,
JJ
r
_7
··., IJ.ps )
IJ. = 01 (zero vector) •
-s
p
u, we must have H ~
s
0lp' for some
s.
Hence
> Var (u)
Var (z)
,
which proves the leImllB.•
The above lennna opens the door for many new lines of work in multivariate
linear estimation.
It has many interesting corrolaries, of which a very obvious
one is the follo1nng:
Let
xl' x ' ••• , x
2
n
be independent random variables such that
Exp
and
Var
,
j
= 1,
2,
... ,
n ,
5
2
where a j
are unknown and are not necessarily equal.
Then if aI' a 2 ,
n
are any real numbers, the best linear unbiased estimate of ~ a. Q. is
j=l
J
a
lJ • • ,
n
J
n
~
. 1
a
J=
j
YJ.
It is outside the scope of the present note to go into the details of applications of the above leImlla.
However, by way of illustration, two examples may be
illuminating.
A detailed paper will follow later.
Example 1. Consider a 2n factorial experiment with r'repetitions of each
treatment combination.
gn)'
g.
J
=. 0 .
or 1,
Also, assume no blocks to be present.
j
= 1,
Let
(gl' g2' ••• ,
2, ••• , n, represent a treatment combination,
Q(gl' g2' ••• , gp. ) its 'true' effect and
y.~ (gl' g2' ••• , gn ) the corresponding
.
n
i-th (i = 1, 2, ••• , r) observed value. Further suppose that all the 2 r
observations are independent, and that
i = 1, 2,
depends on gl' g2' ••• , gn.
~
•• , r
Notice that this assumption is contrary to the
usual one, 1-There the variances are assumed to stay constant.
However, the appli-
cation of the leImlla (in fact merely the corrollary) shows that if
are any k
(0
~
k ~ n)
say Ai Ai ••• Ai.
i , i , ••• , i
l
2
k
factors, then the estimate of this k-factor interaction,
is the s~e contrast of y (gl' g2' ••• , gn)' asA. A.••• Ai,
~l ~2
12K
is of
Q
(gl' g2' ••• ,
Here
r
~
. 1
~=
y. (gl' g2' ••• , g ) •
~
n
In other words the best linear unbiased estimate of any interaction is unaltered
by the assumption of unequal variances.
Example 2.
As another area of application, consider an experiment, with
say v treatments, repeated at different points of time.
time
At an;}' fixed point of
t, the observations are supposed to be independent, and have a variance
l\.
6
(dependent on t).
Such a situation is obtained when for
e~~ple,
the treat-
ments are the different rations for pigs, and we are studying their growth.
Suppose
T
T
and
T'
are any two treatments, and the design is connected, and
are their true effects.
t
T
t,
Then if we want to estimate something like
where R is the total set of time points, the above lemma says that we could
1\
proceed by first obtaining the best linear unbiased estimate (T
t
€
R, and then the best linear unbiased of
Q is
t
1\
t)
- T
for each
given by
I am than.'kful to Professor S. N. Roy for going through this note and for
his comments.
REFERENCES
f'!:7
Bose, R. C., "Notes on Linear Estimation", Unpublished Class Notes,
UniverGity of North
Carolina~
Chapel IIill, N. C. (1958).
f~7
Henry Scheffe:
L"J..7
Roy, S. N., "Some Aspects of Multivariate Analysis," John Wiley and
Sons, 1957.
The Analysis of Variance, John Wiley and Sons, 1960.