I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
CATEGORICAL DATA ANALOOS OF SOO: MULTIVARIATE TESTS
by
V. P. !tlll'kpar ~
University of North Carolina
and
University of Poena
Institute of Statistics Mimeo Series No. 450
October 1965
(To be published in Roy Memorial Volume)
This research was supported by the National Institutes of
Health Institute of General Medical Sciences Grant No.
GM-12868-01.
DEPARTMENT OF STATISTICS
UNIVERS~Y
OF NORTH CAROLINA
Chapel Hill, N. C.
I
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
I·
I
1.
Problems of association between several continuous variables are
generally handled under the
concerned.
assL~tion
of joint normality of the variables
Similary in the general multifactor multiresponse situation with
observations on several characters of each experimental unit for various factorcombinations, data are generally analysed under the assumption of normality;
the situation is referred to as one involving analysis of variance (ANOVA)
or multivariate analysis of variance (MANOVA) according as whether we have
observations on a single character or several characters, respectively, of
each experimental unit.
These methods are fairly well-developed now and form
a part of the classical statistical theory.
The position is not so satisfactory when the assumption of normality
is discarded.
This is particularly true When the experimentaJ.. data are
categorical in nature, i.e., are given in the form of frequencies in cells
determined by a finitely
multi-~yay
cross-classification with predefined
categories along each way of classification.
data goes back to the pioneering
~.,ork
Even though analysis of categorical
of Karl Pearson (1900) and has been
developed at subsequent stages among others, by Fisher (1922), Cramer (1946)
and, in particular, Neyman (1949), a lot remained to be done.
Barnard (1947) and Pearson (1947) pointed out by considering the
simple 2 x 2
table that the same
2 x 2
table could be the outcome of
different sampling schemes which makes it necessary to assume appropriate
probability models, which may lead to different statistical procedures with
obviously different interpretations appropriate to each experimental situation.
This line of thought was developed extensively by Roy and his students; refer
for example to Mitra (1955), Roy and Mitra (1956), Bhapkar (1959), Roy and
I
Bhapkar (1960).
Formulations of some of these categorical data problems as
analogs of filiOVA, and MANOVA and "Normal association" problems were offered
first by Roy and Mitra (1956) and later on by Roy and Bhapkar (1960);
statistical procedures to handle these were given in Roy and Mitra (1956)
and Bhapkar (1961) respectively.
The present paper is in the same line and
develops further some such methods.
It deals exclusively with suitable
formulations of hypotheses and appropriate tests for multi-response situations.
Section 2 gives notation and preliminaries, section 3 includes some
further results and section 4 gives specific test criteria for some hypotheses;
the test statistic mentioned is usually the Pearron -,: statistic if the
maximum likelihood estimates are easy to obtain, and otherwise is the Neymanstatistic computed from results in section 3.
_-~-
~-...~._
Suppose that
taken from
s
s
population and
n
ij
""'I
independent random samples of experimental units are
populations,
n.
oJ
is the size of the sample from the j-th
is the observed frequency in the l-th category of the
We assume that
j-th sample, i=1,2, ••• ,r; j=1,2, ••• s.
is the probability
that an experimental unit drawn at random from the i-tIl population belongs
to the l-th category and tInt either the sampling is with replacement or, if
without replacement, sampling fractions are negligible, so that the probability
distribution of the observed frequenceies is given by
(2.1)
cI>
=
s
II (
j=l
n OJ.
.1
~
. .'
. 1n ~J.
,
n ij )
. rII Pij
~=l
~=
i-There
I
I
I
I
I
I
_I
Notation
"* ••
.... and Preliminaries
""*" ....
...
-~
--I
I: n' j = noj a given integer and
i ~
2
I: p. j = 1 for each j = 1,2, ••• ,s;
i
~
I
I
I
I
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
,.
I
zero in place of a suffix will indicate sum over that suffix.
Let N
= £ j noJ0'
~j = nij/n oj ' Qj = nOj/N, £' = [Pll,···,P(r-l)l;···; Pls,···,P(r-l)s] and
20'
=
[qll, .. ·,q(r-l)l;· .. ;qlS,···,q(r-l)s] •
According as the marginal frequencies along any dimension or way of
classification are held fixed or left free by the experim:mtal scheme that
dimension will be said to be a "factor" or a "response".
Thus in the above
model (2.1) i refers to response categories while ~ refers to factor
categories;
no
is a random variable while
W
n j
0
is a fixed integer.
i
may be a multiple subscript, say, (i i ••• i k ) with i a = 1,2, ... ,ra '
l 2
a = 1,2, ... ,k so that r = r l r 2 ... r k ; similarly, j also might be a multiple
subscript, say
jlj2••• j t
lath
j~
= 1,2, ••• ,s~,
~
= 1,2, ••• ,t
but with
the distinction that all combinations may not be selected for the experiment.
TIlis will be called a k-response (or k-variate) and t-factor problem where
i
refer to a category of the 3-th response While
a
factor.
j~
to that of the
~h
If a set of real values (scroes) is associated with the categories
along any way of classification (factor or response), that way of classification
~dll
be said to be structured.
These may be, for example, the mid-
points of class-intervals for a response (or factor) or tIle values themselves
if the response (or factor) is discrete or may be any scores assigned on
some other considerations even for a way of classificat ion without any implied
ranking, to start with, for its categories.
Suppose that we I1ave to test the hypothesis
, m = 1,2, •••
,mere F 's are
t
= t (t
~ rs-s)
independent given functions of £.
:3
It is assumed that F's
I
possess continuous partial derivatives up to the second order and that the
ranJ~ of the
tx(rs-s) matrix [OFm(~)/dp ]'J
. .] is
t.
It is assumed that there is
at least one solution such that Pij > 0 for all i, j.
(e.g. refer to [11]) that Ho
~,~
the
It is then well known
can be tested in various ways by using either
or the likelihood-ratio statistic A defined, respectively, by
~ =
~
(2.3 )
=
s
r
~
~
j=l i=l
s
r
~
~
j=l i=l
s
-2 log A= 2 ~
(2.2).
f-
or
I
(nij-nOj~ij)2/nij
I
I
I
r
~ n .. (log n. j -1o g n '~i-'}
].J
].
oJ J
,
are any BAN [11] estimates of pIS obtained subject to constraints
Neyman (1949) has shovm that, in particular, estimates minimizing
xf
or those maximizing
<I>
are BAN estimates.
If the pIS are subject
to constraints (2.2), the equations giving these estimates are, in general,
complicated to solve; the minimum - ~ estimates, though, can be obtained
by solving only linear equations whenever the constraints
pIS.
If the functions
F
ill
(2.2)
are linear in
are not linear, Neyman (1949) has proposed the
technique of 'linearization' to reduce the problem to the linear case whereby
minimum - ~
estimates are obtained subject to constraints
Neyman has proved that each of the statistics in
(2.3),
using any system of
BAN estimates (using linearization if necessary), has a limiting Chi-square
distribution with
t
I
I
(nij-nOj~ij)2/nOj~ij
j=l i=l
where ~'s
--I
degrees of freedom as
with Q's fixed if H holds;
o
he 1ms also proved that these tests are asymptotically equivalent in the sense
N~ClO
that the probability of anJT hro of them contradicting each other tends to 0
4
_I
I
I
I
I
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
as N
~
irrespective of whetherH is true or false. The author (1965) has
o
s:il01m recently that the ~ statistic, whenever it is defined, is identical
00
to Wa1d's statistic (1943) as adapted to the categorical situation and, hence,
possesses the same asymptotic optimality properties as those possessed by
\'1ald's statistic and the
1il~e1ihood-ratio statistic
case of sampling from one population (i.e. for s = 1) and conjectured for the
general case (i.e.
3.
for s
2 2).
-.C'ItI
Some__Further. .:;- Re£lts
~
~
Theorem 3. 1.
Let
H be the hypothesis specified by t
o
I
I
I
••
I
independent
constraints
1_
,I
I
I
I
as sho1-m by Wald for the
m
= 1, ••• ,t,
*
*
s
we assume that these
f mij = fmij-fmrj and f m = f m +j:1 f~j
Let
are independent of the basic constraints
~ p .. = 1.
i=l J.J
where
= 1,2,
m,m'
with
g
lnIll
,=
••• , t
s -1
r
r
r
~ n j (~f 'jf lij~j- ( ~ f mi ·l1.i·)( ~ f ,ija. j ))
j=l 0
i=l mJ. m
i=l' J J i=l m -:t
s -1 r-1 * *
r-1 *
r-1 *
= ~ n .. ( ~ fmi.f ,ija. j -( ~ fmija. j )( ~ f "ja. j )}
j=l oJ i=l
J m
-:t
i=l
-:t
i=l m J. -:t
Then if H holds, the statistic
o
c'G
N
N
-1
c
N
has a limiting chi-square distribution with t
5
d.f.
as N
~oo
with Q's
I
--I
remaining fixed.
It can be shovm easily (e.g. refer to [2]) that (3.2) is the ~-
Proof:
statistic to test
H
o
It may be noted tl1at
~
Q, is the matrix obtained after replacing £ by
in the covariance matrix of
Let a linear
I
I
£ and hence will be referred to as the'sample
covariance matrix' of c; it is nonsingular almost
""
assumed conditions.
T11eorem 3.2
I
and tl1e theorem then follows from Neyman's results.
h~~othesis
everyv~ere
in view of the
be defined by
I
r
(3.3)
i~l a/3iP ij
vn1ere
a's and d's are knovm constants,
with rank ~
=v <
= d jl e/3l
+ d j2 e132 + ••• + djuetm ,
e's
/3
= 1,2, ••• ,k,
I
are unknown parameters, ~ • [d. ]
Jr
s, and the linear functions on the left in (3.3) are
linearly independent and also linearly independent of I:iP.. (== 1, of course).
J.J
sxu
I
_I
Suppose
a/3 j = I:i a/3iqij ,
a' = [~j' ••• , '\j] ,
""j
e' = [elr , ••• , Ek r ]
""r
.(3.4)
and
A. = [\/3' j]
""J
,
13,13'
= 1,2, ••• ,k
kxk
with
T11en the
x~-statistic to test (3.3) is equal to the minimum value of
'I·lith respect to the
e' s
""
and has k ( s - v) degree s of' freedom.
6
I
I
I
I
I
I
I
-,
I
I
I.
I
I
I
I
I
I
I
Proof:
Let!!
~
J
= [b 8j ]
b 8j d
jr
=0
8 = l,2, ••• ,s-v;
,
••
I
= Q"
i.
e.,
r=1,2, ••• ,u.
(3.3), then, implies that
Conversely,
!:
j
b~j
u
!:a... .p .. = 0
p~ ~J
i
j = 1,2, ••• ,s) belongs to the vectorspace orthogonal to that generated by
tile rows of
columns
of~;
!! and, hence, belongs to the vectorspace generated by the
e's
such that !:.aQ.p .. = !: e~ d
which
~ I-"'~ ~J
1 ...1 j 1
Thus (3.3) is equivalent to k(s-v) independent linear
thus there exist
means (3.3) holds.
constraints
I_
I
I
I
I
I
I
I
~
be a (s-v)xs matrix of rank s-v such that
= 1,2, ••• ,k
= 1,2, ••• ,s-v
13
8
nle
•
~-statistic with k(s-v) degrees of freedom to test
then, from theorem 3.1.
(3.6)
can be derived,
Let
=
!:
b~
j
.a
uJ
~.'
~
=
Q •
I-"'J
[~.~,
~.L..
• .,
c' ]
"'S-V
so that
£0
=
~ b8j~j
J
and, hence,
c = B
,...*
,...
(3.7)
,.,here
B
,...* =
[bl1~
~
,
::: bls~ ]
I
bs-v,l~··· b s-v,s~
7
= B
,...
®~
,
'
I
the Kronecker product{sometimes also called the Direct product) of B..., and
Il
""l~
Also
•
Cov (£) = 1?* Cov{~)
1?*'
where
~
~
...,
and ~j
in
~.
""J
of ...,
c is
=
0
...,
~
0
...,
0
...,
~
gives
* ...,A B...,*' ,
A
...,
,
,
0
...,
~
defined by (3.4).
B
...,
*'
0
...,
is the covariance matrix of ~j
Note that replacing £ by
Thus, the
=
·..
0
...,
0
...,
~
0
...,
0
...,
~-statistic, in view of
·..
·..
0
...,
0
...,
~
,
(3.2), is
*' (B*A B*' ) -L*
a'B
l3 a
f"t,I"'"
,...,,...,
IV
,...,
,...,
On the other hand minimum value of S2 with respect to ...,e 's is seen to be
-1
~ ~j ~j ~j j
u
/\
~
r=l
.€r ~
,
-1
Jr""u
,
where
\If
=
...,
and
~'-
~d. i\';~j
j
r2i, ... ,~ ]
1 =
r
is a solution of
u
~
€=l
/\
~
,
e
"'r€""€
a
sample covariance matrix'
where
~
and the
·..
·..
0
...,
*
=BEB
..., ..., ...,
••I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
••
wllere
8
I
I
I.
I
I
I
I
I
I
I
I_
~€
I
I
I
••
I
J
If ue let
=
)i'
... , ~]
[)ii,
,
.... ,
[211' ...
:lU J
and
,
....<I> =
... ,
~
,
~u
then it follows that
)i=£*' A
....
-1
(3.10)
<I>~
........
)i=
hence
v~
1
....<I> = D....*' ....A- D....*
,
ex
....
and
(3.9)
reduces to
-1
A ex
ex'
(3.11)
I
I
I
I
-1
= ~ djlj€~j
,...,
f'IIJ"""
A *' -1
-e'D
A ex
rv
".",
I"IltI
,..."
have to show that (3.8) is equal to (3.11), taking into account (3.10) and
the fact that
~
£ =~
Now D
.... is a
* * = 0....
i.e.,
s xu
B
D
........
•
matrix of rank v so that D*
.... is a
matrix of rank kv; there exist then a
ks x kv matrix
M and
matrix [, both of rank l<;:v, such that
* = ME
.... ....
D
....
From (3.10) then we have
Er W
I"ttI
A l ex = E' M' A- l M E 'e
i"'tJ".",
,...,
".."
N"'"
1"ftI".."
,,·mich gives
,
9
N
,
ks x
a kv x
ku
ku
I
.-I
A
''There
Tl
~
= E,...,,...,e
The second term in (3.11) is then n'M,~-l~ •
The theorem fol10,'l'S if we show that
I
* *')-L*
B*'( BAB
-13 =
,..",
~
noting that
#"ttl
,..",
#"tJ
*M = Q. since
** = Q.. Let
~
~ ~
~, ~,;t
be nonsingular matrices
such that
I
I
I
so that
A- 1
/'fIIJ
= R'R
,...,,.,,,,
*
BA
B*'
t'J1'V"'"
,
= (8'8)- 1
M' ~1 M = (T'T)-l
,."",..",
and
N/'fttl
,.""
#"ttl
IV
•
Then
[~
I
I
=
I,...,
_I
,...,
I
''Thicl1 implies
that is,
*
-L*'
-1
R'
~
S'8B R-1 + R M T'T
,...,
, . . , , . . . , f"'tt/I1"ttI
"""* ,..., M'R
,."., ,..",
IV
vmich leads to (3.12).
,.."
,...,
= I,.."
,
Q.E.D.
Note that a's are the natural unbiased estimates of the quantities
on tIle left in (3.3) while ~j is the 'sample covariance matrix' of ~ •
I
I
I
I
I
I
••
10
I
I
I.
I
I
I
I
I
I
I
sUbject to the constraint
(i)
z. .
p. .
~1'~2 ~1~2
Hypothesis of independence:
•
This hypothesis is expressed by the
(h.A.l)
"i'There, of course,
~-statistic
p.
~l
to test
Z
i ,i2
l
~
= z.~2p..
and
~1~2
POi =
0
H1
(
The well known
2
is
n. .
~1~2
-
~/
n.~lOn Oi2
N
(n.~~
ana.~2~
N
,
d.f.
= (r l -l)(r2 -l).
has been already seen ([15]) to be an obvious analog of the hypothesis
of independence (i.e. no correlation) in the bivariate normal analysis.
(ii)
Hypothesis of equality of two marginal distributions:
Assuming r
l
= r 2,
this hypothesis is expressed by the condition
this may be seen to be an analog of the hypothesis
in the usual notation, in the normal analysis.
and
••
I
=1
~1~2
condition
I_
I
I
I
I
I
I
I
1
a double sUbscript (i i ),
l 2
The probabilities p..
are
Here we have a single population with
11
Let
~
=~
and
~ll
=
~22'
I
N
~ = [Oi1i 2(0.~l0+
i ,i2
1
vn1ere
o. . = 1
if
~1~2
il
= i2
.'I
Clo· ) - a. i-a. . - h. (5])h i (~)J
~2
~l 2 ~2~1
~l
2
and
= 1,2, ••. ,rl -l ,
0 otherwise.
Then the
~l-statistic
is
seen to be
T11e same expression had been obtained by Sathe (1962) for Wald IS criterion
and the t'toTO statisticsmust be identical as observed by the author (1965).
The
statistic (4.A.4) differs from the one proposed by Stuart (1955) in that the
latter deletes the last term in the rectangular brackets for
should be preferred since
matrix of .[N II (~)
(iii)
our statistic
NQ is a consistant est±mator of the covariance
even ,.,hen
is consistent only when
~;
132
is false while the mtrix used by Stuart
H2 holds.
Hypothesis of equality of llmeans ll of two variables:
Let us now suppose that the two responses are
II
structured ll , i. e., have
) associated with
i1
the categories of the first response with a similar system (b. ) for the
an implied ranking, and we have a system of scores
(a
~2
second response.
(4.A.5)
~:
Then the above hypothesis may be expressed in the form
E a p. 0
i 1 ~l
i
1
=
Z b. POi
i
~2
2
2
Let
and
N g
then the
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
.,
=
xf-statistic is seen to be
12
I
I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
I·
I
(4.A.6)
=1
•
may be considered as an analog of the hypothesis of equality of means
~
of
d. f.
tv~
possibly correlated variables in the normal analysis.
in the special case
rl
weal'i.er hypothesis than
= r2
~
Note that
, ~ ==> ~ and thus ~ is a
il
and may be of interest if ~ does not hold.
with
ail = b
Here again we have a single population, but
nOvl
i
is a triple sub-
script (i i i ), i a = 1,2, ••• ,ra ' a = 1,2,3 and r = r r 2 r , the probabilities
l
l 2 3
3
p. . .
are subject to the constraint E. . . p. . . • 1 •
lll2 l 3
lll2 l 3 lll2 l 3
(i)
Hypothesis of complete independence:
(LI-. B.l)
(ii)
I~othesis
of independence of the first response with the last two:
(4.B.2)
(iii)
Hypothesis of independence of the first two responses given the third
response:
(4.B.3)
=
It 11as been pointed out by Roy and Mitra (1956) that
H ,H can be ccnsidered
5 6
analogs of the hypotheses of no multiple correlation and no partial correlation,
respectively, in the normal analysis while H4 is that of the hypothesis of
zero correlations.
The :f-statistics are knownto be
13
I
--I
(n....
.
J. 1 J.2 J.
J. 1 , 2,J.
3
3
1:
i
ni10i,nOi2i,
n
OOi3
for testing H4 ,H and H respectively.
6
5
It may be pointed out here that the hypothesis of pairwise independence
of three responses
I
I
I
I
I
I
_I
that of pairwise independence of' the first two (separately) with tb:l third,
viz. ,
,
and the hypothesis of
"no interaction" between three responses (see, for
example, [4] or [9]) are hypotheses Which have no formal analogs in the normal
analysis where pairwise independence between two sets of variables is equivalent to the complete independence of the two sets of variables.
(iv)
Hypothesis of equality of three marginal distributions is specified
by the conditions
Pi 00 = POi 0 = POOi
'
111
assuming, of course, that
r =r2 =r •
1
3
14
Let
I
I
I
I
I
I
I
-,
I
I
I.
I
I
I
I
I
I
I
II
I
I
I
I
I
I
••
I
~i = [qlOO'~OO' ••• , q(r -1)00] ,
1
~ = [~10'~20'···'~(r -10] ,
1
513 = [~01'CJo02'···'~0(r -1)] ,
q' = [qi,~,q3]
1
,
~l = diagonal (~ 00' i = 1,2, •• ,r l -l) etc.
l
1
~2 = [~li20]
(h.B.6)
N~ = [Saf3 - ~ ]
-1
...A
...
<X,13 = 1,2,3
,
= 1,2,3
,
<X,13
= 11 == [M
]
~
and finally
Note that ...
A is the 'sarqple covariance matrix' of ...
q and, hence, is nonsingular
aJJ:lOst everywhere excludinc, of course, the degenerate case where sone variables
(or rather the associated probabilities) are linear functions of the re.maining
variables (i.e., their corresponding probabilities).
it can be shown that the ~-statistic to test
L:
<x, 13
q' M
q..., -
~~,qJ
-1
m'M
... ;..;0
By applying theorem 3.2
H is given by
9
,
m
...
•
The method can be immediately extended to the case of };:
statistic then has
to the statistic
(k-l)(rl-l) degrees of freedom.
(4.A.4).
only for the special case
the statistic
H
9
(4.B.7)
variables; the
The case
k = 2 leads
Cochran (1950) has considered the k-variate problem
r
l
=r
2
=••• =r
k
= 2; even for this special case
is expected to be more efficient.
is easily seen to be an analog of the hypothesis '\ =~ =~
0"11=0"22=0"33
in the normal analysis.
15
and
I
(v)
HjYJ?othesis of equalit~l of "means" of three variables:
Assume now that the responses are "structured" vrith (a. ), (b. )
~l
and (c. ) as the scores associated with the respective categories.
~3
~2
Then
the above hypothesis is expressed by
(4.B.8)
E c. pOO.
i
~3
~3
3
Let
etc.
N\l
= (.
E
a. b. a •. 0) - 11~2
~1'~2
~
=
[~]
etc.
~l ~2 "'"J.l~2
.
[~(3]
,
-1
:;!:! = ~
,
= 1,2,3.
By applying
Theorem 3.2 it can be shawn that the
xf-
statistic is
(4.B.10)
E
a,(3
mNQ1 1(3 ~
a
(rtF-1m)
, d. f.
0
=2
This can be immediately extended to the case of
statistic would have
k-l
degrees of
I
I
I
I
I
I
el
,
and finally
,vitll a,(3
--I
•
k
variables where the
freedo~
H is seen to be an analog of the hypothesis of equality of means
10
of tl1Tee (possibly correlated) variables in the normal analysis.
I
I
I
I
I
I
I
-.
16
I
I
I.
I
I
I
I
I
I
I
With the basic set up (2.1) we have now
a
(i)
and the probabilities
l 2
constraints
. p. i
!:.
populations indicated by
also! is a double subscript (i i ), i
a
l 2
= 1,2 so that r = r r
~1'~2 ~l
.
2J
p. i .
~l 2 J
= 1,2, ••• ,ra '
are sUbject to the
= 1.
Hypothesis of independence of the two
re~onses:
(4.C.1)
This implies that
H
1
holds for each of the
s
populations and it is knOim
that we have a ~-statistic
(4. C.2)
!:!:
..
.
J ~1'~2
I_
I
I
I
I
I
I
I
I·
I
~;
the sUbscript
s
,
)
(n...~1~2J
which follow·s inunediately from (4.A.2).
~l is seen to be an obvious analog
of the hypothesis of independence of two variables in each of
s
(bivariate)
normal populations.
(ii)
Hypothesis of "no interaction" between the two responses and one factor
essentially means that the nature of association between the two responses
is the same over all factor categories, i.e., for all populations; the
fonuulation depends on what measure of association is chosen for comparison.
is independent of j
(4.C.3)
is independent of j.
17
I
The formulation
112
is due to Goodman (1964) While H is due to Bhapkar
13
These l~otheses are nonlinear and the Wald statistics can
and Koch (1965).
be obtained by the "linearization" technique.
For detai Is the reader is
referred to [9] and [4].
This hypothesis can be seen to be the analog of the hypothesis of
equality of
s
correlations given samples from s
Note that the hypothesis
(iii)
H
ll
is a very special case of the hypotheses
Hypothesis of homogeneity of marginal distributions:
p. O·
~l
(4.c.4)
is independent of j
J
also is independent of j •
POi j
2
~lis
bivariate normal populations.
is seen to be an analog of the hypothesis that
~aj'
aQQj ,
a = 1,2,
are independent of j, using the usual notation, in the normal analysis.
Let
,
~j
~llj = diagonal (~
OJ'
1
.. ]
~12j = [~ 1~2J
(4.C.5)
nOOj Aj
-1
~j
Note that
~j
=
i
and
l = l, ••• ,rl-l
[~j-~j'j]
= M.
""J
,
i =1,2, ••• ,r l -l) etc.
l
o:,~
!:! = Ej!:!j
aDoost everywhere (excluding the degenerate case).
gives the x~-statistic
18
2 = 1, ••• ,r2 -l,
= 1,2,
and t =
is the 'sample covariance matrix' of
i
""
E.M.~
J""J
q. and is nonsingular
""J
Theorem 3.2 immediately
--I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
I
I
II
I
I
I
I
I
I
I_
,
(4.c.6)
to test
H •
14
The extension to the general k-response case is quite
obvious giving a statistic of the same form with {~a(ra-l)}(S-l) degrees of
fl~eedom.
(iv)
H;YI>othesis of equality of 'means':
are 'structured' with {a. } and
I
Coo. sider
~2
~
a. Pi OJ
1
is independent of j
~
b. PO·i j
~2
2
also is independent of j
~1 ~1
i
2
this is an obvious analog of the hypothesis of equality of mean-vectors of
several populations in MANOVA.
r .. = ~ a.
I
I
I
I
I
I
I
I-
(b. } as the score s.
~l
(4.C.7)
Suppose now that the two responses
~J
i
l
1
~
i
l
= (~.
nOOj '\2j
,
~ OJ
~1
=(
nOOjA.Llj
Let
a;;
~l
.
~1'~2
"1 2j
=~
~ OJ) - )' 2ij
i
2
b.
~2
,
CJo.~2J.
7.....j'
=
[7 ij,7 2j] ,
etc. ,
1
,
a. b. ~ . j) - )' il2j
~1 ~2 1~2
-1
Mj = t:j
,
and finally
M=
.....
and
~JL
J .....J
The theorem 3.2 gives
~
L.I
j
1t
'M
"1 - t'M.....J• .....J•
...............
'Y ,.;
fw
u
19
,
d.f. = 2(s-1)
,
I
as the ~-statistic to test
H15.
_.
For the k-response problem we have a
statistic of the same type but with k(s-l) degrees of freedom.
H14
Note that
==>H
15
H15
and the weaker hypothesis
when
H14
(v)
H,ypothesis of linearity of regression:
may be of interest
does not hold.
is also 'structured' with
d.
Assume now that the factor
as the score associated vdth the
J
.ii.t h factor-category (or the .ii.th level of the factor).
If
H15
does not
hold, we may test the hypothesis
(4.C.10)
,
vlhere TI's are in the nature of regression coefficients,1')'s and
urumOlID.
This is an analog of the hn>othesis: J;!;j =
In the notation of
(4.C.8)
!t = ~j !:!j
the
~
+
.11 d j
~'S
are
_I
in MANOVA.
and with
Z,jd j
!i = ~j!1j
,
dj
,
~ = ~j ~
dj ,
~-statistic is seen to be
(4. C. 11)
~ 1'~ Mol 1'01 jl'WU"""tJ"-'u
[t', W']
,...,
,.."
d. f.
MR""',,",
[
= 2 (s -2)
Again the k-response extension is immediate where the statistic would have
k(s-2) degrees of freedom.
It may be pointed out l1ere that the h;ypotheses of the type
and ~6 \lere proposed earlier by Roy and Bhapkar
(1960).
~4'
H
15
The test of the
11J-'Pothesis
.11=2,
\dth
J1
given in 1\6' or in other words, of the hypothesis
20
~5
with
I
I
I
I
I
I
I
~6
•
I
I
I
I
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
I·
I
as the model is provided by the statistic
(4.C.9) - (4.C.ll)
a statistic of this type would have
k
d.f.
=2
degrees of freedom in the general
k-response case.
In the basic set up (2.1) now we have! a double sUbscript (i l i 2 ),
i = 1,2, ••• ,ra , a = 1,2 so that r = r r 2 as before and J also a double
a
l
subscript (j 32 ),
jf3 = 1,2, ••• ,sf3
' 13 = 1,2 with not all combinations
1
selected necessarily for the experiment; let s be the rnunber of (jlj2)
combinations selected so that
by
s:S sls2.
For convenience we shall denote
jl the categories of the 'block-factor' (i.e., the 'blocks') and by j2
the 'treatments' (1. e., the categories of the 'treatment-factor').
(i)
Hypothesis of no 'treatment-effect' on the narginal distributions:
p·O.·
~l J J 2
l
is independent of j2
PO. . j
is also independent of j2 •
(4.D.l)
~2Jl 2
This may be seen to be an analog of the hypothesis that 1-l1"V-l.
,
""U1J 2
cr
j..
, a = 1,2, are independent of j2 in MANOVA. In the notation
aa lJ 2
of (4.C.5) with 1 a double subscript (jlj2) let further
(4.D.2)
~j
,
,
1
the summation being over those j2 only which occur in conjunction with
From theorem 3.2 we have the
~-statistic
21
jl.
I
_.
(4.D.3)
to test H • The method can be innnediately generalized to the case of k
17
response giving a statistic of the same type with (~a(ra-l)}(s-sl) degrees of
freedom.
(ii)
Hypothesis of no 'treatment-effect' on the 'means':
If the two responses are 'structured' with
{a. } and
~l
(b. } as the
~2
associated scores, consider
is independent of j2
(4.D.4)
is also independent of j2 •
This is the analog of the usual hypothesis of no 'treatment-effects' in MANOVA.
In the notation of
For testing H18 we have the ~-statistic
j
~
""
j
1 2
'V'
'.-1
...."'1
j Mj
.
'V
l-l
j
2.... 1 J 2 ...."'1 2
-
~ t' M-lt .-1
j ...." 'l....j 1""'"1
"".-1
,
d •f• -- 2(s - s 1 ) •
1
For the k-response situation the statistic would have k(s-sl) degrees of freedom.
_I
(4.C.8) but with J. a double subscript (jlj2) let further
(4.D.5)
(4.n.6)
I
I
I
I
I
I
I
Here again we note that
would be of interest when
IS.7
IS.7
==> H18 and the weaker hypothesis H18
does not hold.
I
I
I
I
I
I
I
-.
I
I
II
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
II
(iii)
Hypothesis of linearity of regression on treatment level:
He assume now that the second factor is 'structured' and d.
J2
score associated with the category
of the Jeth treatment.
If
~8
j2; in other words, d.
J2
is the
is the 'level'
does not hold, we may consider
~
~ ai~ilOjlj2
= t(l)
+
~jl
1
(1) d
~jl
j2
,
where
~
's and
~'s
are unknown.
This hypothesis is the analog of the hypothesis
that
IJ.. j =
~l 2
in MANOVA.
~"'
~l
+~. d.
NJ l
In the notation of (4.C.8) with
J2
j
= (jlj2) and of (4.D.5) let
further
(4.D.8)
1\9 from theorem 3.2 we have the ~-statistic
Tl1en for testing
(4.D.9)
d
E
d...
l'~
~ j
Ul 2 ul 2
Z:
j
Z· . ~lJ2
E [t,:
j
1
....ul
':!j']
1
]
[~l fijl ] [~l
:!j
R
-j
S
1
....j
1
1
For the k-response problem, the statistic would have k(S-2S ) degrees of
l
freedom.
It may be mentioned here that the hypotheses 1\7' Hl8, H19 were offered
as analogs earlier by Roy and Bhapkar (1960). To test the hypothesis
23
I
"lith
11
's given by (4.D.7), i.e., the hypothesis 11.8 with F1.9 as the model,
we have the statistic
(4.D.6) - (4.D.9)
a statistic of the same type would have
kS
l
degrees of freedom in the k-
response case.
Similarly we can test the hypotheses 11...
NUl
tI1e case may be) and/or
g•
NJ l
=!
=
11
,....
(known or unknown as
under the model 11.9; these details are
omitted.
(iv)
Hypotheses of "no interaction":
This raises a number of problems indeed depending on whether we are
interested in the nature of association between the tHO responses over the
categories of the two factors or in the effects of factors on the marginal
distributions of the responses.
He are then faced ,'lith defining 'interactions'
of different orders and testing hypotheses about these.
Hence this is omitted
in fUrther discussion and the reader is referred to Bhapkar and Koch (1965)
for further details.
Most of these problems can be handled in much the same way as the
s~ler
problems discussed earlier.
Thus, as mentioned earlier, the hypothesis
of equality of marginal distributions of several responses, or of equality
of 'means' of several responses can be tested by methods discussed in 4.B;
the hypothesis of homogeneity of rrarginal distributions or the one of equality
of 'means' etc., for several populations and the general multi-response
problem can be tested by methods in 4. c; the varicus hypotheses of no
24
_.
I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
I
I·
I
'treatment-effect' or of the linearity of the treatment-effect in the general
mul.ti-response two-factor situation (with one factor in the nature of 'blocks')
can be handled by methods in 4. D.
Problems of thi s nature for the general k-
response l-factor situation present no further difficulties and can be handled
in precisely the same way.
It is when we are considering problems of inter-
actions of various types and of various orders that further difficulties arise;
some of these are discussed in
[4] and [5].
Problems of associations similar to those in H , H and H can be
4 5
6
handled in a fa.irly straightforward manner for the general k-response single
population situation as follows:
(i)
~othesis
of COmplete independence of k responses;
(5.1)
It is easy to check that, as in H , we iIl'lIlediately get maximum likelihood
4
estimate ~
ilO••• O
= n ilO•••
"
In
~
_...k 1
~--
~'
n.
I:
i_
i 1·'" K
(ii)
etc. giving the ~-statistic
2
nilO••• o···noo••• :ix
.-
~l··· L
~-l
.K
)
I'~(
co
n. 0
~l···
O••• nO
0. )
•••
HYPothesis of independence of two sets of responses:
,
where we assume, without loss of generality, that the first k l responses form the
first set.
As in
~,
we get immediately the maximum likelihood estimates
25
~k
I
_.
Actually the statistic follows directly from (4.A.2) noting that
can be regarded as one subscript i *
l and (ik +l".~) as
1
HYPothesis of independence of several sets of
(iii)
H_ : p
-"22
i l i 2 ···ik
i 2* •
re~onses:
= p.~l···~kl
. 0
-~O
Oi
.
0
O···PO
Oi
.
••• V"" ••• kl+l···~kl+k2 •••
• •• kl+••• +k+l···~k
t-l
where we assume that there are
t
sets of responses, the first containing
tile first k responses and so on. The statistic for ~2 follows inurediately
l
from (5.2) regarding i * = (il••• i k ) and so on so that r *
l = rl ••• r k etc.,
l
1*
* *
*
1
and replacing k in (5.2) by t with rl ••• rt-(rl+••• +r t ) + (t-l), i.e.,
rlr2... rk-(rlr2 ••• rkl+... +rkl+••• +kt!,"frk)+t - 1
(iv)
(5.6)
degrees of freedom.
Hypothesis of independence of two sets of responses given a third set:
=
here we want to test the conditional independence of the first two sets given
the third set of responses.
The r-statistic is immediately seen to be
I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
26
I
I
I.
I
I
I
I
I
I
I
,
no • • • O'~k
Note that
~O
1 2
is an analog of the hypothesis that the correlation matrix is
the unit matrix, whi1.e H21.' H22 and
~3
are analogs of the hypothesis of no
canonical corre1.ations, WiJLlcs' hypothesis of independence of sets of variates
and the hypothesis of no partial canonical corrleations, respectively, in
the normal muJ.tivariate analysis.
I_
I
I
I
I
I
I
I
••
I
•
+k +1 ••• ~k
27
I
REFERENCES
,,",,,,,,.~.,,~AIWP~..-.J
2 x 2
[1]
Barnard, G. A. (l947), "Significance tests for
BioIOOtrika, ~ pp. 123-138.
[2]
Bhapkar, V. P. (1961), "Some tests for categorical data," Ann. Math. Sta~.,
~ , pp. 72-83.
[3]
----- (1965), "A note on the equivalence of two test criteria for
hypotheses in categorical data," to appear in the March 1966 issue
of Jour. AIOOr. Stat. Assoc.
[4]
----- and Koch, Gary G. (1965), "On the hypothesis of no interaction in
three-diIOOnsional contingency tables," Institute of Statistics Mimeo
Series No. 440, University of North Carolina.
[5]
----- and -----(1965), "The hypotheses of no interaction in four-dimensional contingency tables," Institute of Statistics Mimeo Series
No. 447, University of North Carolina.
[6]
Cochran, W. G. (1950), "The comparison of percentages in matched
samples," Biometril:;:a, ~ pp. 256-266.
[7]
Cramer, H. (1946), Mathematical Methods of Statistics, Princeton
University Press, Princeton.
[8]
FiSher, R. A. (1922), "On the interpretation of chi-'square from cont:ingency tables and the calculation of P," Jou. Roy. Stat. Soc., ~
pp. 87-94.
[9]
tables,"
Goodman, L. A. (1964), "Simple methods for analyzing three-factor inter-
action in contingency tables,"
Jou. Amer. Stat. Assoc., ~ pp. 319-352.
[10]
Mitra, S. K. (1955), "Contributions to the statistical analysis of
categorical data," Institute of Statistics Mimeo Series No. 142,
University of North Carolina.
[11]
Neyman, J. (1949), "Contribution to the theory of the
test, "
Proceedings of the Berkeley S~osium on Mathematical Statistics and
Probability, University of California Press, Berkeley, pp. 239-273.
[12]
Pearson, E. S. (1947), "The choice of statisticaJ. tests illustrated on
the interpretation of data classed in a 2 x 2 table," Biometrika,
~ pp. 139- 167.
[13]
Pearson, K. (1900), "On the criterion that a given system of deviations
from the probable in the case of a correlated system of variables
is such that it can be reasonably supposed to have arisen from random
sampling," Philosophical Magazine, Series 5, ~ pp. 157-172.
-r
28
••I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
[14]
Roy, S. N. and Bhapkar, V. P. (1960), "Some nonparametric analogs of
normal ANOVA, MANOVA, and of studies in normal association,"
Contributions to Probability and Statistics, Stanford University
Press, Stanford, pp. 371-387.
[15]
------ and Mitra, S. K. (1956), "An introduction to some nonparametric
generalizations of' ana.lysis of variance and nmltivariate analysis,"
Biometrika, ~ pp. 361-376.
[16]
Sathe, Y. S. (1962), "Studies of some problems in nonparametric inference," Institute of Statistics Mimeo Series No. 325, University
of North Carolina.
[17] Stuart, A. (1955), "A test for homogeneity of the marginal distributions
in a two-way classification," Biometrika, ~ 412-416.
[18]
Wald, A. (19'+3), "Tests of statistical hypotheses concerning several
parameters when the number of observations is large," Trans.
AJoor. Math. Soci., ~ 426-482.
I_
I
I
I
I
I
I
I
I·
I
29