FURTHER CONTRIBUTIONS TO MULTIVARIATE CONFIDENCE BOUNDS
by
S. N. Roy and R. Gnanadeshikan
This research was jo:tntlYBp<)QSored by the United Sta.tes Force
through the Office of Scientific Research of the Air Research
and Development Command, and the Research Techniques Unit,
London School of Economics and Political SCience.
Institute of Statistics
Mtmeograph Seriee No. 155
August, 1956
•
FURTHER CONTRIBUTIONS TO MULTIVARIATE CONFIDENCE BOUNDS*
By S.
Summary.
~J.
In this paper
Roy and R. Gnanadeshikan
the implications of certain results obtained
4 7 on
confidence bounds on parametric functions
connected with multivariate normal populations are fully worked out. This
in earlier papers ;-1, 2, 3,
-
leads to a number of confidence bounds, expected to be useful, but hitherto
unnoticed, (i) on the characteristic roots of one population dispersion matrix
and on roots connected with (ii) two population dispersion matrices, (iii)
the regression matrix of a p-set on a q-set and (iv) multivariate linear
hyoothesis on means, including, in ?articular,the problem of discriminant
analysis.
1. Confidence bounds on roots connected with Z of
Il.
start from the statement 0.1,2) of
~
£2J
N(~,Z).
Let us
and note that the statement is
exactly equivalent to
(1.1)
-
for all nonnull a(p x l)'s, that is, to
,
A
1
,
,
=-=
-....>
'
~
a a
a a
a Sa
a Sa
a Za
--
A. 2
=-=
'
a a
,
- lp,( n) en d ne -1 ( p,n),
where ""1 and A2 stand respectively for ne la
2a
* This research was jointly sponsored by the United States Air Force
through the Office of Scientific Research of the Air Research and Development
Command, and the Research Techniques Unit, London School of Economics and
Political Science.
2
-
Choosing a so as to minimize a
that
-8
,
~a/a 8,
we observe that the seoond part of the
- -- ...
inequality (1.2) implies that
minimize
,
< C
(2:); and choosing a so as to
- mi n
sala
.....a, we notice that the first part of the inequality implies
,
_
,
C • (~) <
A-o in(S),
- -'1 m
nu.n
A.-C
(S)
-~ mi n
,
,
Likewise, choosing a so as to maximize a Sala a,
- - --
we note that the second part
of the inequality implies that >-c (S) -< cmax (~);
"2 max
and choosing
a
EO
as
to
max~.mize a
,
-
-_
,
'Ea/a ...a, we have that (1.2) implies that
and
AIC
max
(S) > C
-
max
- -7 and note that
We use (2.4) of ;-2
( Z) > A C (S) •
- 2 max
(1.3)has a confidence ooefficient > I-a,
-
- -7 .
and also incidentally that (1.3) implies (3.1.3) of ;-2
-
Going back to (1.2) let us take a(p x 1) such that the ith component is
zero.
Then arguing in a similar manner, we observe that (1,2)
(1.4)
>
and
AC
(s(i» > c
(~(i»
- max
1 max
> AC
-
~
max
(S(i»
'
for i . 1, 2, ••• , p, where SCi) and ~(i) stand respectively for the truncated
sample and population dispersion matrices obtained by cutting out the ith
variate.
-
Likewise, if we take an a(p x 1) such that the ith and jth (1
components are zero and then argue in a similar manner, we observe that
r j)
UNCLASSIFIED
Security Information
Bibliographical Control Sheet
1.
University of North Carolina, Chapel Hill, N. C.
2. 'Mathematics Division, Air Force Office of Scientific Research
3. AFOSR-TN-56-380
AD 95816
4. ASTrA
5.
FURTHER CONTRIBUTIONS TO MULTIVARIATE CONFIDENCE BOUNDS
6. s. N. Roy and R. Gnanadeshikan
7.
August, 1956
8. AF 18(600)-e3
9. File 3.3
3
>
(l.?) also
and
(1.5)
for i
r
j •
1, 2, ""
p, where S(i,j) and ~(i,j)
stand respectively for the
truncated sample and population dispersion matrices obtained by cutting out
the ith and jth variates. We can continue this process on to the stage of
cutting out any (p-l) variates, that is, retaining anyone variate.
seen that (1.2)
z=
statements like (1.4),
l
> a pair of statements
t~)
pairs of
(1.3), and also p pairs of
stat~ments like (1.5),
,Le., p statements involving only one variate.
p \
It is
and so on down to
All such statements
p-l)
will thus have a joint confidence coefficient
~
1-a, and will provide us,
from a certain standpoint, with a complete analysis of what the psychologists
call the problem of principal components.
2.
Confidence bounds on roots connected with Zl and Z2 of N(51,Zl)
and N<'~2,Z2)'
Let us start from (J.2 .1) of
£2_7,
put Al .. (nl /n2) ei~(p,~,n2)
(2.1)
... , p).
We next recall (2.3.2)
4
of
L:2_7 ~nd
3
*t
= ASIA
1
also that 0(3 S;1)18 are invariant under a transformation:
1
and S2
~
t
-1
AS *
2A where A is any nonsingu1ar matrix, and put A = ~ ,
and rewrite (2.1), without any loss of generality, for our purpose, in the
~
!'monic a1 form
Al ;: nIl 0(S2 D
ff1
or
or
,
-1
a 3 6 a
>..1 - 1 , 2 -
(2.3)
')
>
~
Alo
mX<:
\'1')
>
-':1
for all nonn1l11 2( p x 1)' s.
c,f (2.03),
i
t
-1
a 8 8 a
21
>.. 2
,
a a
a
Now ch00sing 2 so
1S
to maximize the middle tnrm
nd,e Lh'1t the left P3I't of the inequality (2.3)
(
( 8 3 -1) > 0
1 2 - max 81 D
AI
8- 1
D );
1
and chousing
~
-=====>
so as to rninim:i.ze
~
the middle tdrm of' (2.3), we note that the right part of (2.3)
===»
5
(2.4)
Use is made of (2.3.?) and (2.3.5) of
["2_7
all c(ry )
i
'!'hus it is se",m that (2.3)
whi.ch, therefore, is
~
(;l
to show in
>
["4_7
(
c.
mm SID
Ri
that
81-1 D
).
~
===>
confidence statement
'toli th A
confi.dence coefficient
1 - a, since (2.3) has the confi.dence coeffiQient 1 - a. (2.6) is proved
in a sli.ghtly different way in
["4_7.
We now r,o back to (2.3) and, as in the previous section, take
~(p
x 1)
such that the lth component is zero, argue the s::nne way as from (2.3) to (2.6)
and end up by observing that (2.3) also
=======>
where SCi) SCi) Z(i) and Z(i) have tho same meaning as in the previous section.
1 ' 2 ' 1
2
6
Li.kewis'j,
'lfJ
in the previous
section,~e
note that (2.3) :)lso _ _..;l>
(2.8 )
and so on till we reach the stage where any (p-l) v3riates have been cut out,
Le., allY one var:i.Dk has bel3n retained, whi.ch gives us just the confidence
~ounds
on v,:lriance ratios in the univariate
C,'lS6.
We have thus, with a joint
(2.6),
confjdence c08fficient > 1 - a, confidence statement
stJtHmlJlIts like (('.7),
'fhi:.; agnin, from
(~)
p confidence
confidence statements like (2.8), and so on.
cGrtuin standpoi.nt, provi.des pnrt of the analysis of
Q
(l
prohlem which occurs in the rmJ.ltivclriate generalization of the customary
V':lr;iinCe components
3.
'~ni:llysis
in univariate
dn~llysis
of variancG and
ConfidencE; llounds connected with the regression m;jtrix @(p x q) 01'
a P-Stlt on a q-sot in a (p+q )-v!'lriate norme~ distribution.
of
cov~Jri:mce.
["2_7!'J!ld
Let us stidrt from
obt:-dn, wi th a confidence coefi'i.chmt ~ 1 - a. And for all unit
vectors ~l(P x 1) 3nd 22(q x 1) the confidence statement
(3.1)
_< _d1'B_d2 + Ac 1/2 (8
m:uc
where B(p x q )
q-set) cmd
~(p
on the q-set).
::a
8
1.2
)c 1/2 (S-1)
max
22'
8 -1 ( the sample regression matrix of the p-set on the
12 22
= 1:121:~~
x q)
•
(the population regression matrix of the p-set
Going bC3ck to Lemmas C and E of
£2_7
again we notice that,
7
with retipectto variation over .21 and .22' the maximum values of .2i. B.22 cmd
.2~P.22 are ~espectivGly C;~(BBI) and C;~(~~I). Now, first choosing ~l and
I
~2
80'18
I
i
B.22 and then choosing
.21
and
.22
so as to maxim:tze
and arguing in the same way as in the two previous sections we note
~~d
.;;:L
t.o maximize
-2
that (J.l) _._>
which, therefore, is a confidence statement with
Now going back to (4.4) of
;: l-a.
with
3
1:2_7 which
3
confidence
coeffici~nt
is ~I confidence statement
confidence coefficient l-a, we rewrite it in the equivalent form
,
-
for all nonnull o(p x 1)18.
-
This means that (3.3), with a probability I-a,
implies (3.2), with a probability;: l-a.
~(p x 1)
3S
As in the previous scctions, tako
Buell that tho ith component is zGr", define SCi) •
1.2
B(i) and ~(i)
the truncated matrice::; obtAined by cutting out tho ith variute of the
p-set, 2nd observo that (3.3) also
======>
8
() .4)
(3.3)
Likowise, as in the previous sections, we observe that
cmd so on.
We have thus, with a joint confi.dence coefficient
st~tement (3.2),
so on.
,~so
p statements like
(3.4), (~)
~
statements like
========>
I-a, the
(3.5)
and
This kind of result could be genernlizod by truncating the variates
of the q-set as well, but this will not be discussed here.
4.
Confidence bounds on roots connected wi th multivariate line aIr
hypothesis on means.
L~a.
On,5 of N (,5, Z).
and rewrite
(4.1.4)
I.
Let us start from
- - - - A -------
2.',5
< --- <
(~1~)1/2 -
(~'~)1/2 -
-
r
of L
1_7,
2
')
set)..::tT ~/(n+l),
as
(a'Sa)1/2
a x
(h.1.4)
for all nonnull a(p x l)'s.
a'i
{a'Sa)1/2
-,- 1/2 + A ----:";"-,1::"'/""'2l!""
(g~)
(~ ~)
Ille recall that (1~.1) is a confidence stC1tement
9
with
I
confidence coufficiunt I-a.
We rec,:111 that the maximum values of
(~1~)1/2 Dnd c l / 2(5), reason in the same
of als, are respoctivoly (x'x)1/2,
-
way
- -
<IS
- -
max
in t.he previous sections and deduce that (1.)..1) ===>
1/'2 l/?
_r _x_, __x_ 7-ACmr.
. . .r_·(S) < r~ ~
1/2
I
' ..IJ..
_
...
-
..
-
7<
..
-
r XtxI l7? AC 1/2 (5),
..
-
-
+
max
which is thus a confidence statement with a confid0nce coefficient
~
l-a.
l\rguing as in the prGvious sections and using tho same notation:w bE::ifore
for truncated
E, ~ cmd S obtained by cutting out
the ith variato, the ith
and jth variates (i ~ j), and so on, we have with a joint confidence
coefficient
(
~)
~ I-a,
in addition to
(4.2), p statements like
stcltomonts like
and so on down to the stage of cutting out. 'lIlY (p-l) variates, Le"
retAining
any nne v:]ri 'lte .
4b,
Som8 observations on multi variate line,:.Jr hypothesis on means.
Confidence
bounds connected with univariate and with multivariate linear hypothesis on
means are discussod respectively in chapters 15 and 16 (to which chapter 14
forms the background) of
.
-r3-7.
Here we shall first set up a physically
more general hypothesis and then discuss the associated confidence bounds,
10
I
Let X(n x p) (w:i.th P < n) consist of n row vectors x.(l x p) (with
-~
I
i . 1,2, •.. , n) which are independently distributed, each being N ~E(~i)'
and let E(X) (n x p) • A(n x m)
~(m
x p), where m < n and rank (A) • r
~
m.
Let ~ (n x r) be a basis of A and let us write (as we can, without any loss
of generality)
A(n x m)
2
L:Ar
r
condition
A _7n and let us rewrite the expectation
2
m-r
[IS
E(~)n
• n
L~
~r J
r~1
:-r
p
Here the X is a set of (observable) stochastic variates,
s is
a set of unknown
population parameters, A is a known matrix of constants given by the design of
the experiment and is called the design matrix.
It might consist of numbers
like, say 0, 1, etc. and/or a set of observGd (nonstochastic) quantities, as
in the case of regression problems with concomitant variates.
dispersion matrix Z is also unknown.
The population
This is the model under which we propose
to test the hypothesi.s
(4.6)
r
m-r
where C(q x m), partitioned as above, and M(p x u) are matrices given by the
hyoothesis to be tested and are called the hypothesis matrices.
that rank (M) ~ u ~ p and rank (C)
more that, rowise,
fell
a
S ~
It is assumed
r (~ m < n of course), and further-
012 _7 is a basis of C and, columnwise,
[GIl
On
J
11
is also a basis of C,
-r3-7
We go back to (14.2.13), (14.2.15) and (14.4.6) of
(thi3t are repeatedly used in chapters 15 nnd 16 of
.
..r3..,.7)
and recall
that for H to be testable in the sense of chapter 14 of ~3_7 we should have
O
However, in most realistic problems, the C matrix of the hypothesis is given
in a form such that the last rows are absent and we can, therefore, without
any essential loss of generality, replace (4.6) by
(4.8)
[
l
~ (r x
p)
~(m:r
x p)
M(p x u) .,. 0,
j
and (4.7) by
We now go back to X and observe that x(n x p) M(p x u)
consists of n rows of independently distributed vectors
,
~i(l
£- X*(n x
x p) M(p x u)
*' x u), sayJ such that ~i* is N LE(-~i)'
* M:t
t
if[" .. !i(l
M_7, i.e.iJ ["E(~i)'
say, and that
(1+ .10)
where
u),say_7
.z*_7,
12
(4.10a)
r
[.;
u-r
The H of
O
1]
=
M(p x u)
';2
p
u
(h.B)
can now b8 rewritten so
'" 0
<nd the altornati ve
(4.12)
HI
II
S
to
LCI
r
H can be expressed as
O
c2_7
m-r
[~
.;;
r
= "*(s
x u)
m-r
u
If we now go back to
,
,
(16.6.3) of £3_7 and substitute M (u x p) X (p x n),
I
i.8., X*' (u x n) for X(p
x n), C for C ' und!*(u x 1) for !(p xl), u for p,
1
11
',*.for 11, :md use (15.2.15) of '[3_7, we observe that now the confidence stute-
ment (16.6.3) of ~3_7 is replaced by
(4 .13 )
~
*' (1
x u ) X*' (u x n ) hI (n x r )('
~Al )-1 (r x r) C , (r x s) r---'
U(s x s)h(s x 1)
1
13
for all nonnul1
(I.l..12), (~~,
*
nnd .
all unlt
- (11 x 1)
.'l
-
vectors b(s x 1), where
'r)
*
is given by
t;) by (h.10(J), X* stands for X M, and wher(-)
(4.14)
(4.15)
-l~
I
r
I
I
(n-r) S (u x u) • M (u x p) X (p x n) L I(n)-A (n x r)(A11~)
1
,
x A!(r x n17X(n x
p) M(p
-1
(r x r)
x u)
ltJt.l not.E.J that (I f .1.3) is a set of confidence statemonts, with a joint
confidonce coefficiEnt I-a, on bilinear compounds of 'r)i~, where
'r).* ...
['C 1
C2-7
r f1·1
L
t.h0 null hypothfsis
l.j.c.
M, may bu regarded as measuring the deviation from
'2
Ho'
Further consoqul3nCGS of (4.13).
Starting from (4.13) !md arruing j.n tho
IVlJn8 WlDuor as in sect/ion 3 and setting c (u,s,n..r) = C (say) we nato tb3t
a
(4.13)
a
>
+
- -,
or,. substituting for U U from (1.j..14),
1 / 2 (8*).
raea-7 1/2 cmax
-
14
<
0
where the
and the
1/ 2
rs
max L
3**
-
7 + -rsc a-71 / 2
c l / 2 (3*)
max
'
due to tho hypothesis, i.o., 83** is given by
matr1Jt
due to the error, i.e., (n-r) 3* is given by (4.1,).
matr'1x
Notice that (4.17) is
3
confidence statement with a confidence coefficient
middle term of (4.17) is zero if and only if the hypothesis
->Hl-ais andtrue.thatForthep=l,
M(p x u) will drop out (except for a trivial scalar
O
factor, since u <p) and we shall have the univariate problem when c
(83**)
max
will be replaced by just the sum of squares due to the hypothesis, c
by just the variance due to the error and c~ax L'rJ*'
by just the scalar
max
(3*)
- 1°1_7
' -1 'rJ*_7
f0 1 (AI'J\)
*' Lr 01 ('
'7 -1 ~.*
Al~ )-1 01_
~
Starting from (4.13) and reasoning in exactly the same way as in sections
3 and 4a, we see that (4.13) also implies, in addition to (4.17), p statements
(')*
(1)**
like (4.17) on truncated S 1 , S
and
ith
variat~ (~)statements like (4.17)
and
'r),1,
i
'r)
(i)*
obtaine~ by cutting out any
on truncated S(i,j)*, S(i,j)** and
,. j)*
obtained by cutting out any pair of ith and jth variates (with
I j),
and so on.
These latter confidence
-
confidence coefficient> I-a.
st~tements
will thus have a joint
•
15
It may be noticed that the problem discussed in section 4a is really a
special cnse of the one discussed in 4c; nevertheless, for expository purposes,
it is worthwhile to discuss first a simple problem 1i}s:e the one in 4a and then
take up the most general one in 4c.
5.
Concluding remarks,
Similar confidence bounds that arise in connec-
tion with (i) factor analysis, (ii) classification problems (iii) univariate
variance components analysis and (iv) multivariate variance components analysis,
obtained by generalizing univariate variance components analysis, will be
discussed in a later paper.
REFERENCES
7 Roy,
~1
estimation~,
II
S. N. and Bose, R. C., Simultaneous confidence interval
Ann. Math. Stat., Vol. 24(1953), pp, 513~536.
2 7 Roy, S. N., "Some further results in simultaneous confidence
interva( estimat ion It, .Ann. Math. Stat., Vol. 25(1954), pp. 752-761.
r3
7 Roy, S. N., uA report on some aspects of multivariate analysis",
Institute of Statistics mimeograph series No. 121.
["47 Roy, s. N., ItA not6 on some further results in simultaneous
confidence-interval estimation tl , 1.nn. Math, Stat., Vol. 27(1956), pp •
.