I
I.
I
I
I
I
I
I
I
I
II
I
I
I
I
I
I
I
•
SIMULTANEOUS TEST PROCEDURES FOR ONE~WAY
ANOVA AND MANOVA BASED ON RANK SCORES*
by
K. R. Gabriel and P. K. Sen
Hebrew University, Jerusalem
and University of North Carolina
Institute of Statistics Mimeo Series No. 546
August 1967
*This research was supported by the National
Institutes of Health (Institute of General
Medical Sciences) Grant No. GM~l2e68-03 .
)
.
..
~<~,
DEPART-MENT OF BIOSTATISTICS
UNIVERSITY OF NORTH CAROLINA
Chapel Hill, N. C.
I
I.
I
I
I
I
I
I
I
I
SIMULTANEOUS TEST PROCEDURES FOR ONE-WAY
ANOVA AND MANOVA BASED ON RANK SCORES*
by
K. R. Gabriel
Hebrew University Jerusalem and
University of North Carolina
and
P. K. Sen
University of North Carolina
SUMMARY.
A MANOVA statistic based on rank scores is proposed and its
distribution derived along the lines of Puri and Sen (1966).
simultaneous inference approach of Gabriel (1967a) this statistic is shown
to provide a procedure which simultaneously tests all subgroups of samples
on all subsets of variables, and to set simultaneous confidence bounds on
all location differences.
.-
I
The formulae for actual ranks, i.e., Wilcoxon
type scores, are worked out explicitly.
Ie
I
I
I
I
I
I
Using the
1.
INTRODUCTION
The resolution of rejections of overall hypotheses into significant
detail, allowing simultaneous inference on subhypotheses, has received a good
deal of attention in the literature.
(For a survey see Miller (1966».
The
original work was principally on univariate normal ANOVA, but this was more
recently extended to non-parametric set-ups (see Nemenyi (1963)
and Sen (1966)
which also cite other references), and to the multivariate normal model (see
Gabriel 1967b), which also cites earlier work).
No multivariate non-parametric
technique seems to have been available hitherto, and the present paper is intended to fill this gap.
Joining the general approach to simultaneous in-
ference being developed by Gabriel (1967a) with the method of derivation of
* This research was supported by the National Institutes of Health (Institute
of General Medical Sciences) Grant No. GM-12868-03.
I
I.
I
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I
.-
I
-2-
statistics used by Puri and Sen (1966), a technique is obtined which has known
asymptotic distributions and allows a permutation approach for small samples.
The present approach relaxes the assumption of multi-normality (as in
Gabriel (1967b».
However, it still requires that all the distributions com-
pared be equal except for shifts in location.
function by
Fk(~)'
the model under which the present approach is valid is
ek )
Fk(X) = F(x +
'" '"
'"
where
~
Denoting the k-th distribution
is any vector of constants.
for all k,
(1.1)
A further extension to provide methods
for the purely distribution-free set-up is not available at present.
Definitions are set out in section 2 of the paper, followed by the
distribution theory in section 3.
Section 4 describes the simultaneous in-
ference procedures and their properties.
The special case of actual ranks,
rather than rank scores, is worked out in detail in section 5.
2.
DEFINITIONS
Consider Co (>2) independent samples of sizes nl, ••• ,n
c
' each of the
o
observations being made on p variables. Denote the a-th obsero
o
vat ion of the k-th sample on the i-th variable by ~), a=l, ••• ,n ,
k
N=nl+ ••• +n
c
k=l, ... ,c, i=l, ..• ,p .
o
0
For any pair (k,q) of samples,
k~q=l, •••
,c , rank the n
= n + n
k
kq
o
q
observations separately on each variable i(=l, ••• ,po)' Write
for the rank of
(i)
~
(i)
among Xkl
(i)
rank scores
n kq
~)
(i)
, ••• ,X
•
qnq
Further define the
(i)
/i)
for each
(i)
""'Xk~,Xql
~~q) ,a=l, ••. ,nk ,
by means of functions
and R, being t in R(l
~
(~(g»
nkq+l
J~i)(n~l)'
i=l, ••• ,p , which depend on n
o
R ~ n) and satisfies the Chernoff-Savage (1958)
I
I.
I
I
I
I
I
I
I
I
-3-
conditions (See also Puri and Sen (1966) for further detail of these regularity
conditions).
Now obtain pairwise mean rank scores differences
n
Iq
/i)
n q 0'.= 1
n kq
_.L
(2.la)
and similarly define
=0
(i)
Ukk
for k=l, ••. ,c
o
(2.lb)
•
and i=l, .•• ,p •
0
Next, rank the observations within each single sample.
f ~(i)
XCi)
~KO'.
among ~(i)
~Kl , ••• , kn •
k
po) matrix V so that its (i,j}th element is
N
an d samp 1 e k d enote b y R(:)
~K~
t he ran k
(po
x
For variable i
0
Now d e f'1ne
Ie
I
I
I
I
I
I
.e
I
(2.2)
nk[
(Note that
I
~i)~, r
nk
I rJ
(i) ( ,+0'.1),
(i) ( 0'.+1)".1 r for all r, which for r=l ex0'.-1 n k n k
O'.=l[nk n k
plains the negative term in v
• Also, this shows, for r=2, the diagonals
Nij
J
J
(2.3)
to be independent of the within sample rankings).
Denote the group of all c
o
samples, or corresponding populations, by
I
I.
I
I
I
I
I
I
I
I
Ie
I
I
I
I
I
-4-
G and any subgroup by G , containing c (1 < c < C ) samples, or distrio
e
e
e- 0
butions.
Similarly, denote the set of all p
by S , containing p (1 < P < P ) variables.
a
a
- a- 0
=
samples in G by N
e
e
I
I
I
variables by S
0
and any subset
Denote the sum of sizes of
n , and let vij(a) be the (i,j)th element of the
k
kEG
inverse of the principal ~inor of V with rows and columns corresponding to
N
a
the variables of S. To test hypothesis H that c populations of subgroup
a
e
e
G are equally located on the p
e
variables of set S , one may use the statistic
a
a
I
kEG
(2.4)
e
O
In particular, to test the overall hypothesis H
o
are equally located on all p
that all c
o
populations
O
o
variables one would use statistic L
o
•
Note that the statistic La is completely unaffected by the within
e
sample variabilities.
The diagonal terms of V are constants (2.3) depending
N
only on sample sizes and not on variability.
However, the off-diagonal
'terms of V (2.2) are affected by within sample correlations -- as expressed
N
through rank scores.
Hence, the statistic La may be affected by the ranke
score correlation within the c
o
samples, and its usefulness as a test sta-
, , f or Ha d epe~d s on t h e equa I'J.ty
tJ.stJ.C
e
distributions Fl(x), ••• ,F
tV
c
0
f th
IatJ.on
'
,
e corre
matrJ.ces
0
f t he
(x), which is, however, implied by the location
o
tV
model (1.1).
a
e
A statistic of the form L was chosen because its distribution is unaffected by the location of samples not belonging to G and variables oute
side S.
a
Thus, given the location model (1.1) the distribution of La is
completely specified under H:.
I.
o
e
To see this note again that the matrix V
N
is essentially a "'Within samples" rank-score variance matrix, and is clearly
I
I.
I
I
I
I
I
I
I
I
-5-
unaffected by any shifts in the location of the samples on any of the variables.
Next, the rank-scores used in obtaining
volve only the variables of S.
U~~) ,U~~)
and
v~j(a),
i,jES , ina
Further, the rank scores used for u1(i) ,uk(j),
a
(q
k,qEG , involve only pairs of samples from G.
e
e
q
Hence none of the elements
which go into the formula for La are affected by changes in location of
e
a
Thus, since H specifies the
samples outside G and/or variables not in S.
e
a
e
relative locations of the relevant samples on the relevant variables, it
completely specifies the distribution of La.
e
This may be summarized (as in
a
Gabriel (1967a)) by saying that the collection of hypotheses H implied by
e
O
H and their corresponding statistics La forms a testing family.
o
e
3.
NULL DISTRIBUTION OF La
e
Under the null hypothesis Ha,the distribution of La may depend on the
e
e
parent distribution F(x) (unless we are essentially dealing with univaraite
til
Ie
or independent multivariate distributions).
I
I
I
I
I
I.
I
I
free.
Thus, unlike the univariate rank
tests, the proposed STP stes will not, in general, be strictly distributionHowever, the Chatterjee-Sen permutation arguments considered in detail
by Puri and Sen (1966) may be used to construct permutationally distributionfree tests.
O
We shall consider the permutation distribution of L in some deo
tail, while the case of general La will follow on the same lines.
c
A set of N* = (N! /
~
i\;ka
1
realizations of L0 can be obtained from a
0
samples by considering all possible N* allocations of the N
(1)
(po)
= (~'
,
.••
,~
), a=l, ••• ,n , k=l, ••• ,c , into c samples of
-1m
-lta
k
o
0
given set of c
vectors
rP n k !)
e
o
sizes nl, ••• ,n c ' respectively.
o
O
Since under H
o
,
the ~ 's are independent and
~ka
identically distributed stochastic vectors, all these N* allocations will be
conditionally equally probable, each having the conditional probability l/N*.
I
I.
I
I
I
I
I
I
I
I
-6-
This compietelyspecified conditional pro;bability law generates the permutation distribution of L~.
I.
I
I
c
are all small, this distribution may
O
be used to find the critical value of L
o
•
If the sample sizes are large, one
can either proceed as in Puri 'and Sen (1966) to find the asymptotic permutaO
tion distribution of L
o
,
or alternatively, one can take a sample of several
hundred allocations from the universe of all possible allocations to give a
consistent estimate of the critical value of LO.
'0
Since, it has been shown
by Puri and Sen (1966) that the asymptotic permutation distribution and the
asymptotic null distribution of rank statistics do coincide, one need only
consider the following large sample approach.
Recall that according to the model (1.1) Fk(x) = F(x+8 ), k=l, ••• ,c ,
'\,
'\, '\,k
0
where 6 , ••• ,6
are all p vectors.
'\,1
'\,co
Denote the ith marginal cdf of F by
o
F [iland the bivariate join,t cdf of the (i, j ) th variates by F [. . 1(x, y), for
~,J
i;'j=l, ••• ,p.
o
Ie
I
I
I
I
I
If nl, ••• ,n
Next, define
/i) (u) = lim J(i) (u): 0 < u < 1,1=1, ••• ,p ,
n~
where n stands for n
kq
n
, for any k;'q=l, ••• ,c '
o
0
Then, let
i=l, ••• ,p;
lli =
(3.1)
o
(3.2)
(3.3)
for i,j=l, ••• ,p •
o
LEMMA 3.1.
If (1.1) holds and the conditions of Theorem 4.2 of Puri and Sen
(1966) hold,
uniformly in '\,
6 , ••• ,6
•
'\,c
1
o
I
I.
I
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I.
I
I
-7-
As the proof follows along the lines of Theorem 4.2 of Puri and Sen
(1966). hence the details are omitted.
Now. define
0.4)
and assume that
v is positive-definite
(3.5)
tV
THEOREM 3.1.
If the conditions of Theorem 5.1 of Puri and Sen (1966) hold.
a
a
then under H and (3.5) (N/N )L has asymptotically a chi-square distribue
e e
tion with p (c -1) degrees of freedom.
a e
Proof.
O
It suffices to consider the case of Lo· since the others follow along
the same lines.
Define
c
I
o
c
~
0
(n n
k q
k=l q=l
/2N)U(i)u(j) •
kq
kq
(3.6)
It follows from (2.4). lemma 3.1 and (3.6) that
0.7)
So. it is enough to prove the theorem for L*o.
o
Now write
0.8)
for a=l ••••• n k • i=l ••••• po' k=l •••• ,c • where
o
c(u)
Then. from the definition of
i t follows that
=
u~~)
f
l
lo
u > 0
0.9)
u < O.
and Theorem 5.1 of Puri and Sen (1966).
I
I.
I
I
I
I
I
I
I
I
Ie
I
I
I
I
I
-8-
0.10)
-(i)
B
+ 0 p (n k-~q ), say.
k
-(i)
B
q
Consequently,
...!...
2N
L Ln
k q
n u(i)U(j)
k q kq
kq
0.11)
It also follows from Theorem 5.1 of Puri and Sen (1966) that
0.12)
for all a,S and i,j=l, ••• ,p.
o
Hence,
E(:B(i)IHo )
k
0
E(i3(i)i3(j) IH o ) =
k
k
~
0
= 0,
(3.13)
-1.. \) .
n
-(1)
k
ij
~-(po)
Consider now the Co vectors n~k = (nkB k , ••• ,nkB
), k=l, ••• ,c • As they
k
o
involve summation over independent and identically distributed random variables
B(~»,
by the vector valued central limit theorem,
(3.14)
Thus, it follows from the well-known results on the distributions associated
with quadratic fOlims in multinormal vairahles that
I.
I
I
0.15)
I
I.
I
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I.
I
I
-9-
has asymptotically a chi-square distribution with p (c -1) d.f ••
o
0
Now, (3.6) and (3.11) imply that L*o has asymptotically a chi-square
o
distribution with p (c -1) d.f ..
o
0
The rest of the proof follows from (3.7).
The corresponding derivation for any La would require the subtotal
e
a
N in place of the total N. Hence the result holds for (N/N )L •
e
e e
4.
SIMULTANEOUS TEST PROCEDURE AND CONFIDENCE BOUNDS
A Simultaneous Test Procedure (STP) of level a for the testing family
a
considered in this paper is to accept or reject each H according as
e
(4.1)
or
where ~
a
O
O
0
0
is the upper a point of the distribution of L under H
•
This STP has the following properties:
(I)
then H: cannot be accepted if
(II)
H~
Its decisions are always coherent in the sense that if H: implies
H~ is rejected;
Its decisions are not always consonant in the sense that if a hypoa
thesis H is rejected it does not necessarily follow that some other
e
H~ implied by H: is also rejected;
(III)
The probability of any type I error in all its decisions is no more
O
than the level a, being exactly a if the overall hypothesis H is
o
true.
a
The probability of a type I error on any particular true H
e
is less than the above.
Property (I) follows readily from the fact that the testing family is
monotone in the sense that if H: implies
Theorem 1).
H~ then L: ~ L~.
(See Gabriel (1967a),
To prove this monotonicity note first that the implication rela-
a
b
tion between H and H is equivalent to the pair of containment relations
e
Sa~ Sb and G ~ G ·
e- f
p
I
I.
I
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I.
I
I
-10-
Next, note that since Sa ::::>.
S
- b
(4.2)
for any k and q.
This is so since the matrix
(See Lemma in Appendix).
nkn
L
q iES
L
'ES
a J
a
V
N
is symmetric positive definite
Finally, note that La sums terms
e
u(i)u(j)vij(a) /2N over
kq kq
G
e
whereas Lb sums smaller terns
f
b
Clearly, then La > L
e -
f'
as was to be proved.
Property (II) is essentially negative and is established by any counterexample.
Property (III) consists of several statements.
of any type I error at all is exactly a under HO •
o
(1) That the probability
This follows readily from
monotonicity which entails that any type I error at all be made if and only
O
if such an error is made on H
o
•
(2) That the probability of a particular
error is no more than that of any error.
This is obvious.
(3) That the
O
probability of any type I error at all is no more than a even if H is not
o
true.
This follows from the fact that the family of hypotheses is closed
under intersection (See Theorem 2 of Gabriel (1967a), where a derivation of
these and other properties is given for STPs in general).
Particular type I error probabilities may be evaluated by means of the
approximate null distributions.
a
H by the STP with critical value
e
a
(a,e)
Thus, the probability of falsely rejecting
sa
is
= PHa (La>
sa )
e
e
(4.3)
I
I.
I
I
I
I
I
I
I
I
-11-
a
which may be obtained from the distribution of La under H .
e
samples,
~
a
N/N
e
will be the upper a(
a,e
e
Thus, for large
) point of the chi-square distribu-
tion with p (c -1) d.f ••
a
e
It is interesting to compare the STP proposed here with the only other
non-parametric multiple comparison method available for unequal sample sizes,
and restricted to univariate ANOVA.
Nemenyi (1963) (See also Miller (1966),
Chapter 3, section 6) essentially tests H
e
(the superscript is superflous in
the univariate case) by means of the sum of squares of rank scores "between"
the samples of subgroup G , though he specifically mentions only G s which
e
e
are pairs of samples (and contrasts).
In particular, he uses the Kruska1-
Wallis rank ANOVA statistic whose asymptotic distribution for any H is chie
square with c -1 d.f., just as is that of L N/N in the present STP.
e
e
e
level critical value
~
a
The a
is therefore the same in both techniques, but the
N
type I error probability on He with Nemenyi's method, a(e)' exceeds that,
Ie
a(e)' with the STP.
I
I
I
I
I
bution.
To see this note that
square with (co-I) d.f. whereas
~aN/Ne
~a is the upper a~e) point of chi-
is the upper a(e) point of that distri-
Since corresponding inequalities must also hold for power comparisons
it is clear that at a given level
Nemenyi's technique offers a higher prob-
ability of obtaining significant detail than the present STP.
This advantage of Nemenyi's technique must be set against the disadvantage that its decision on any sybhypothesis H is influenced not only by
e
the location of the samples in G but also by the location of all other samples,
e
which is irrelevant to H.
e
This is so because Nemenyi computes all his
statistics from the overall ranking of all c
o
samples.
Finaliy , the STP might be adjusted to accepting H: only if (N/Ne)L: ~ sa'
I.
I
I
with the same critical value sa as before.
This would increase the type I
I
I.
I
I
I
I
I
I
I
I
Ie
-12-
error probabilities but destroy the coherence of the decisions.
a
the decision on each H would still depend only on the location of the samples
e
in G on the variables of S.
e
a
I.
I
I
In the univariate case the type I errors would
be asymptotically equal to those of Nemenyi's technique.
Thus this adjusted
a
STP has the advantage that the decision on each H depends only on the truth
e
of Ha itself whereas Nemenyi's technique has the advantage of coherence.
e
The
unadjusted STP has both advantages, but at the expense of lower chances of
rejection.
The STP (4.1) is readily extended to a simultaneous confidence statement about bounds on the location differences
(4.4)
i=l, •.• ,p , k#q=l, ••• ,c , where 8 (i) is the i-th co-ordinate of the vector
k
o
0
~k of (1.1).
x.(i)
-Ka '
Start by defining
a=l, ••• ,n , by
k
I
I
I
I
I
However,
(i)
~
U~q(d)
as
U~~)
computed upon replacing each
(i)
- d, the X s being unaltered.
qB
Next define
N ~
D(i) = sup{d: u(i)(d) >
}
kq
kq
- [l,;avNii ~]
k q
(4.5a)
N k
d(i) = inf{d: U(i) (d) <
kq
kq
- [l,;aVNii nkn q ] 2}.
(4.5b)
P{d(i) < /1 (i) < D~i) , \j i,k,q} > I-a
kq - kq q
(4.6)
and
Then
provides I-a simultaneous confidence bounds for all
To prove (4.6) note first that u(i)(c) is
kq
u: 0 <
U
< 1.
/1k~i)s.
~ in d as J(i)(u)
is t in
n
kq
Hence
d(i) < /1(i) < D(i)
kq - kq - kq
is equivalent to
(4.7)
I
I.
I
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I.
I
I
-13-
(4.8)
that is, to
(4.9
Writing La(~) for the La statistic computed by means of U(i)(~(i»
e
~
e
~
values
in place of U(i)
kq , (4.9) is equivalent to
t a(~) < I; , where G
e
- ex
e
:s
{k,q} and S :: {ilo
Now, by STP property (III), since
(4.10)
a
~
denotes the true location dif-
ferences
(4.11)
Hence
(4.12)
So
that (4.6)
fol1o~s
from the set of equivalences of (4.7), (4.8), (4.9)
and (4.10).
5.
A particularly simple as
WILCOXON SCORES
~ell
as useful special case is that of
Wilcoxon rank scores, i.e., of using the ranks
J(i) (....£L)
n
n+l
. -n+lex
1
~
ex
themsel~es.
~
n•
In this case
(5.1)
Then
U~~) becomes l/(nkq+l) times the difference of the mean ranks of the
k~th
and q·th sample in the joint ranking of all n
these samples on the i-th variable.
kq
One notes that
observations of both
U~~) is proportional to
the two sample Wilcoxon statistic comparing the k-th and q-th samples on the
i-th variable.
I
I.
I
I
I
I
I
I
I
I
-14-
Further, one obtains from (2.2), (2.3) and (5.1) that
(5.2)
and
n
1 c
k (i) nk+l
() nk+l
vN';J" =(R.' ... ----)(R.,j - - 2)/(n+l)2, i;'j=l, ... ,p.
N k=l a=l ka
2
-Ka
k
.
0
L l
A particularly notable simplification is achieved for the simultaneous
confidence bound in (4.7).
Let
(i)
= --ka
~(i) _ xCi)
Zkq,aS
qS f or a= 1 , ••• ,nk ;
Arrange the nkn
q
observations on Zk(i)
Q
q,a~
Q
~=
i=l, ••• ,p.
o
(5 • 4)
in ascending order of magnitude and
Z~~~r)
for r=l, ••• ,nkn q ,
Now define
C(N) _
kq
+1»), ~
,\ng (,,¥+ng
-
(nk+n q )
t' a nknq vNii }~ '
(5.5)
(and note that C(N) for large sample sizes reduces to (nkn s /l2)~(1+o(N-l»).
kq
q a
Then let
m,(N) = ~ n
'Kq
. k q
_ c(N)
kq
and
~(N)
uk,q
= ~ kn q
+ C(N)
kq
Proceeding then as in Miller (1966, pp. 146-149), one observes that (4.7)
is equivalent to
(5.6)
which provides the desired confidence bounds.
I.
I
I
1 , ••• ,n , k~Tq= 1 , .•• ,c •
o
q
denote the rth smallest observation in this set by
Ie
I
I
I
I
I
(5.3)
.L
A graphical procedure due
to Moses is explained in Miller's book and may also be adopted for the
present purposes.
I
I.
I
I
I
I
I
I
I
I
-15-
APPENDIX
LEMMA:
Let x be a (p X1) vector and V a (pxp) symmetric positive definite
'V
x
( 'VI) an d V, correspon d ing 1 y, as
let x be partitioned as
'V
T'nen
Proof:
~
'V- 1~ _>
I.
I
I
Z2
Write
-1
Ie
I
I
I
I
I
'V- 1
~1 11~1
>
'V- 1
- ~1 1l~1
since
is positive definite.
I
-16-
I.
I
I
I
I
I
I
I
I
Ie
I
I
I
I
I
I.
I
I
REFERENCES
CHERNOFF t H. and SAVAGE, I. R. (1958). Asymptotic normality of certain
nonparametric test statistics. Ann. Math. Stat., ~, 972-994.
GABRIEL, K. R. (1967a). Simultaneous test procedures - some theory of
multiple comparisons. University of North Carolina, Institute of
Statistics, Mimeo Series No. 536. (Submitted to Ann. Math. Stat.)
GABRIEL, K. R. (1967b). Simultaneous test procedures in multivariate
analysis of variance. University of North Carolina, Institute of
Statistics, Mimeo Series No. 539. (Submitted to Biometrika).
MILLER, Rupert J. (1966). Simultaneous Statistical Inference.
Hill Book Co., New York.
McGraw-
NEMENYI, Peter (1963).
graphed).
(mimeo-
Distribution-free Multiple Comparisons.
PURl, M. L. and SEN, P. K. (1966). On a class of multivariate multisample rank-order tests. Sankhya,~, 28, 353-376.
SEN, P. K. (1966). On nonparametric simultaneous confidence regions and
tests for the one criterion analysis of variance problem. Ann.
Inst. Stat. Math., 18, 319-336.