il
I.
I
I
I
I
ON ANALYSIS OF VARIANCE FOR THE K-SAMPLE PROBLEM
by
Dana Quade
University of North Carolina
\1
·1
I
Institute of Statistics Mimeo Series No. 453
December 1965
I_
I
I
I
I
I
'I
J
-Ie
I
This research was supported by the National Institutes
of Health Grant No. GM-10397
DEPARTMENT OF BIOSTATISTICS
UNIVERSITY OF NORTH CAROLINA
Chapel Hill, N. C.
I
ON ANALYSIS OF VARIANCE FOR THE K-SAMPLE PROBLEM
'.
I
I
I
I
I
I
I
1
by Dana Quade
University of North Carolina
1.
INTRODUCTION.
Suppose we have k
~
2 random samples, possibly multivariate,
where X.. is the j-th observation in the i-th sample 'for 1 < i < k, 1 < j < n ,
~J
-- i
and there are N = En. observations in all.
~
be G..
~
Let the distribution function of X..
~J
The null hypothesis to be tested is
We shall be particularly concerned with the large sample situation where
N~ ~
I_
I
then our null hypothesis can be reexpressed as
Ho : G.~
1
I
I
I
I
I.
I
I
==
G,
i=l, 2, .•• , k.
The general approach we have in mind is an extended version of one-way
Let fN(xl,x2""'~) be a function of N arguments which
analysis of variance.
is symmetric in the last (N-l) of them.
Next, corresponding to each observation
X define a score
ij
y{J'
L
1
= fN(X ~J
.. ,
[(N-l) X's other than X.. J).
~J
Supported by the National Institutes of Health Grant No. GM-10397
I
'.
I
.. 2-
Then we perform an ordinary analysis of variance based on the scores:
we calculate
I
I
(N-k) I:n.
"
I~
lit
1.
Fa
!I
I
I
that is,
_
1
where y. = -
n.
1.
1
N
k
<y.1. - =)2
Y
(k-l) I:I:(Yij - =)2
Y
,
ni
I:
I: y .. , and test H by referring F to the
I: y.. and Y =
0
j= 1 1. J
i=l j=l 1.J
F-distribution with (k-l, N-k) degrees of freedom.
1.
ni
In the remainder of this paper we present conditions under which such a test
will be asymptotically valid, and we show that several of the tests proposed in
the literature are essentially of this type.
. . 2.
,.,
THE NULL-HYPOTHESIS DISTRIBUTION .
In accordance with Chernoff and Teicher
[2], we shall say that n random vsriables are interchangeable if their joint
distribution function is symmetric.
Then clearly the scores as defined above
I
are interchangeable random variables if the null hypothesis is true.
tl
I
a completely straight:forward extension of Theorem 1 of Chernoff and Teicher,
f
using the multivariate extension of the Wald-Wolfowitz-Noether limit theorem,
we can obtain the following
For every sufficiently large N let (Z~~)} be a set of N interchangeable
k
1.J
random variables, where 1 < j < n~N),
I: n~N) = N, and as N~~ n~N)/N~p. > 0 for
-1.
1.
1.
1.
i=l
1 5 i 5 k. Suppose also that for all N
Theorem 1.
I
I
I
I·
I
Now, by
k
(i)
I:
(N)
n.
~ Z~N)
i=l j= 1
(ii)
1.j
... 0
I
'.
I
-3-
and that as
(iii)
I
I
I
I
I
(iv)
N~
00
1
J:'
k
1
N
t
I z~~)
I
l.J
max
i, j
n
(N)
i
~O
in probability,
[Z~~)]2-+ 1 in probability.
t
l.J
i=1 j=l
Then the k random variables
(N)
n.
1.
1
.IN
~
j-l
z~~)
l.J
i=l,2, ••• ,k
'
have asymptotically a k-variate normal distribution with zero mean vector and
variance matrix
I
I-
v =
Application of the preceding theorem to our problem gives
0.'
Theorem 2. If the hypothesis is true, and if N
A
~
00
and
n./N~
1.
p. > 0 for
1.
lSi S k we have
Assumption A:
max
. j
j
1.,
I y ..
l.J
- =
yl
~ (y .. _ ~)2
l.J
~
0 in probability,
I
I.
I
I
t
I
I
I
I
,e
I
II
I
I
t
I.
I
I
-4-
then the analysis of variance statistic F based on the scores has asymptotically
the F-distribution with (k-l, N-k) degrees of freedom.
-Proof.
for 1
Define
~
i
~
k, 1
~
j $ n •
i
Then the analysis of variance statistic based on the
Z's is identically the same as the statistic based on the y's.
But it can be
verified immediately that the Z's are interchangeable and satisfy all the conditions
(i) - (iv) of Theorem 1.
n
E
1
.IN
Hence the k random variables
i=1,2, ... ,k ,
Z •. ,
j=l
1J
have asymptotically a k-variate normal distribution with zero mean vector and
variance matrix V.
It then follows that the random variable
n.1
k
E
[E
j=l
i=l
has asymptotically a
Z .. J2/ n .
1J
1
x2 -distribution
k
E
1
N-k
with (k-1) degrees of freedom.
n.
1
Z .. J2/ n . ~ 0
[ E
i=l
1J
j=l
1
in probability, and thus
1
N-k
k
(E
k
ni
E
i=l j=l
Z •• 2
1J
ni
E [ E
i=l j=l
Z .. J2/ n .} ~ 1
1J
1
Hence
I
'.
I
II
I
I
I
I
f
-e
•
I
I
I
I
I
I
I
,.
I
-5-
in probability.
Then by Cram~rls convergence theorem ([3J, p. 254) we have that
(k-l)F is asymptotically a
x2 (k-l)
also.
But this is equivalent to saying that
F itself is asymptotically an F(k-l, N-k).
The reader may note that we have passed up several opportunities for obtaining a statistic with asymptotic X2 (k-l) distribution.
the test in terms of F rather than the simpler X2 ?
of personal preference.
He may then ask, why put
This may be partly a matter
However, statement in terms of F may be intuitively
appealing to experimenters accustomed to applying analysis of variance to the
k-sample problem, and it provides a unified approach, since an ordinary analysis
of variance based on untransformed univariate data is clearly a special case of
analysis of variance based on scores.
Alternatively, the test could be regarded
as a conditional one, given the scores; this will be a particularly useful device
if N is small.
Then we have a permutation test based on the analysis of variance
statistic, and the results of Pitman [5J indicate that the F-distribution affords
an excellent approximation to the required conditional distribution.
our approach
3.
t~roughout
However,
is to consider the test as unconditional.
SCORES RELATED TO U-STATISTICS.
Although the general definition used for
scores so far leads to a satisfactory result under the null hypothesis, it is
not evident how this may be extended to the case where the alternative holds
true.
In order to obtain such an extension, we shall suppose in this section
that the scores have the more specialized form
y ..
~J
= E ..
~J
4>(X. "
~J
[m XlsJ),
where the function 4>(xO'x , ... ,x ) is symmetric in its last m arguments,
l
m
1 < m < min(n.), and the summation extends over all possible combinations of m
-
-
1
I
I.
I
I
t
-6-
observations other than X .
ij
We shall also make the following
Assumption Bl:
This is equivalent to assuming that
I
I
I
I
,_
I
O,X1"",Xm are independent and their distributions are all taken from
whenever X
Gl,GZ,.",Gk ,
such that Em
I
I.
I
I
= m,
where X ,X ""X
O 1
m
and define
are independent, X comes from G , and m of the other m XIS
o
i
r
come from G for 1 $ r $ k.
r
I
I
I
I
r
Let {mr } be any ordered set of k nonnegative integers (ml,m2""'~)
Tl
Then we may write
=
k
E
i=l
{m }
p.
1.
E*
R
r
{m }
r
Tli
where
{m }
R r
m1
=
mlP1
m2
P2
m1 ,m2 , '"
~
",Pk
~1
and E* indicates summation over all (m+k-1) possible choices of {m}.
m
r
{m }
max
i, {m }
r
or
r
Tli
,
Hence
I
I.
I
I
I
I
I
I
I
-7-
{m }
r
TJi
min
TJ [ i
Pi
]-m-1
for all i, {m }.
r
We note that Assumption B1 is trivially satisfied if
~
is bounded.
Since
we shall need this stronger condition for the results of Section 4, we state it
here as
Assumption B2:
(The constant c may depend on G.)
Assumption B2 is satisfied in all the examples
to follow.
A few further definitions will be useful.
I_
I·
I
I
I
I
I
I.
I
I
<
Let
where X ,X "",X are independent, Xo comes from G , and m of the other m X's
m
O 1
i
r
corne from G for 1
r
~
r
~
k.Then
8
where
Define also
{m }
r
i
=f
8
(m }
r (x) dGi(x)
I
-8-
I.
(m } (m }
I
e(x) = j •.• j4>(x,xl'x2, ... ,x ) dG(x ) dG(x2 )···dG(xm)= E* R r e
m
l
t
I
e. = je(x) dG. (x),
I
I
I
I
,_
I
I
I
I
I
I
I.
I
I
~
e
r (x),
~
=
je(x) dG(x)
=
k
E
Pie i ,
i=l
and finally
~ = je (x) dG(x)
2
Theorem 3.
1
~
i
~
=
k
E
i=l
Pi ~i
If Assumption Bl holds, then as
N~
and
n./N~
~
p.~ > 0 for
k the k random variables
Nt(N-1)-1
m
Yi
-
e }
i
have asymptotically a joint normal distribution with zero mean vector and finite
variance matrix.
Proof.
We may write
(m }
.. r
Yij = E* E ~J
<p(x .. ,[m XiS])
~J
where the inner summation extends over all possible combinations of m observations other than X.. such that m of them come from the r-th sample for
~J
1
~ r ~
k.
Then
r
I
-9-
'.
I
I
I
I
I
I
I
,_
I
I
I
I
I
I
I.
I
I
{m }
{m }
~r
U. r
~
where
k n
7r ( r)
m
r=1
{m}
[
=
i
U r
and 0.
~r
is Kronecker's delta.
=
so that cI>
u
,
r
k
ni
7r
'-' '-'iJ'
j=l
~
r=l
For 0 5 u
~
{m}
~
r
cI>(X. ,,[m X's]),
~J
m define
u
E cI>(xr,xl+olr(xO-xl),x2+o2r(xO-x2),···,xu+our(xO-xu)'xu+l""'xm),
r=0
is symmetric in its first (u+l) arguments and also in its last (m-u)
arguments.
Then
ni
{m }
{m }
r
E E..
cI>(X .. , [m XIS]) = E. r cI>. ([(m.+l) X,'s], [(m-m.) XIS])
j= 1
~J
~J
~
mi
~
1.
where the summation extends over all possible
~
combinations of (m+l) observations
such that the first (m.+1) of them come from the i-th sample and that m of the
r
1.
others come from the r-th sample for 1
(m )
U. r
~
=
~
r
~
k, r
r i.
Thus
(m )
E.
~
r
cI>m ([(m.+1)
Xi's],[(m-m~) XIS])
~
i
A.
I
'.
I
I
I
I
I
I
-•
-10-
E[U. r
1.
1.
J)
have asymptotically a joint normal distribution with zero mean vector and finite
variance matrix.
Then the same result must hold true also for the k
variables
where
1
Q
i =
~* R
(m.+l) ~
{m} (m}
r U r
i
1.
•
I
(m }
{m}
.!.
N2 (U. r
for 1 =s; i
'.
I .
U-statistic, and hence, using Lemma 3.1
of Bhapkar [lJ, it follows that the k(m+k-l) random variables
m
I·e
I
I
I
I
I
generali~ed
has the form of a k-sample
~
k.
Now
(m }
E[U. r J =
1.
(m }
(m.+l) 8. r
,
(m} (m)
r 8. r
1.
= 81.'
n.-m.
{m}
1.
1.
and hence
E[ Q. J
1.
= E*
R
•
Also,
Yi
so
- Qi = E
* [ ~.
1.
1.
fm \
~ r
_ R
{m } U. r
r
J_1._
m.+l
1.
,
r~ndom
I
-11-
'.
I
{m }
max
{m } var[
r
I
I
I
I
I
.',..
.-
U.
=
~.+l
1.
1
0(1) x 0(1) x O(N) ,
1.. N-l-1
and thus the random variables N2{( m )
square.
r
Yi - Qi} converge to zero in mean
_1.
N-1 -1 -
Then the asymptotic joint distribution of the random variables N2{( m)
1
must be the same as that of the random variables N2{Q. - E[Q.J), and the theorem
1.
1.
is proven.
In the next two theorems we extend the argument used by Sen [7J in establishing the structural convergence of U-statistics.
the random variables e(x .. ) for 1
1.J
_<
i
_<
k, 1
_<
For this we shall need to consider
j
<
-
n..
1.
These variables are mutually
independent, the expected value of e(x .. ) is
1.J
E[e(x .. )J
1.J
= je(x)
dG.(x)
1.
= e.1.
as defined previously, and the variance of e(Xij ) is
•
var[ e(x .. ) J = ~. - e. 2 .
1.J
1.
1.
-.
~
--
-
-..
We have also
Theorem 4.
1
~
i
~
If Assumption B1 holds, then as N~
00
and ni/N~ Pi > 0 for
k we have
I
I.
I
I
yi-e )
i
E[ ( N-l)-l y .. - e (x)..J 2
m
1.J
1.J
uniformly for all i,j.
-Proof.
Starting from y .. as expressed in the proof of Theorem 3, we find that
1.J
I
I.
I
I
I
I
I
I
I
,-
-12-
n,-m, (m}
(m}
N- 1) - 1
r
= ~* .2:...-!. R-_ r V
( m
Yij
~
n
-~
ij
i
fm }
where ~ r
is as defined previously, and
(m }
V
ij
r
=
k
[
7r (
~-l
n -5
r m ir)
r=1
4>(X, " [m Xt s]).
1.J
r
Suppose for the moment that X" is fixed.
1.J
(m }
r
Then, conditioned on X , V
is
ij
ij
a k-sample generalized U-statistic, and we have
{m }
E[V, , r IX, ,] =
1.J
1.J
e
{m }
r (X, ,).
1.J
{m }
Now consider estimating the quantity
(m }
e
r (X . .) not by V,', r but by the mean
1.J
1.J
of independent statistics of the form ~(X, " [m Xts]). The number of such
1.J
independent statistics which can be obtained from the data is the largest integer
n -5
-
•..
•-
-
min
r ( rm ir),,by hypothesis, this number will be at least equal
, r
m1.n
to NPo/2m, where Po = i Pi' for all sufficiently large N. The variance of any
not greater than
one of the independent statistics will be
var[~(X, .,[m Xts])IX,.J
1.J
1.J
< E[4>2(X'j'
1.
[m X's])jx'j]
1.
= 1)
{m }
r
say, where
1)
{m }
r
(x) =
f .. . f~2(x,xll,x12'·· .,x1m1,x21'·· .,x. m.)
k
I.
I
I
mr
Tr
Tr
r= 1 u= 1
k
Hence the variance of their mean will be no greater than 2m
k
1)
(m }
r
dG (x ).
r ru
(X, .)/Np .
1.J
0
But since a U-statistic has minimum variance among all unbiased estimates of
its expected value it follows that
I
I.
I
,I
-13-
(m )
var[V .. rjX .. J
~J
~J
(m)
r _
(m)
e r (X. ,»2IX .. J <
= E[(V ..
~J
~J
~J
{m }
2m" r (X . .)
-
N P
~J
o
Then, integrating over the distribution of X,., we find unconditionally that
~J
I
I
I
I
I
,_
(m )
E[ V,. r
J.J
uniformly for all i,j, (m).
r
I.
I
I
e
(m )
r (X,.) J2
<
J.J-
l
2:'2 = O(-N )
Npo
Note that
e(x)
= E*R
(m )
r
(m )
e
r
(x).
Hence
n,-m. (m )
(m )
(N-l)Ml
e(x.
,)
= E*[ ~ ~ ~r V, . r
Yij
m
~J
n,
J.J
~
~I
I
I
I
I
I
_
= E*
(m )
(m )
n.-m.
1. 1.
r
~
r
[V
..
n,
1.J
1.
R
(m ) (m )
r e r (Xij) ]
(m )
e r
(Xij)J
(m )
n,-m. (m )
(m )
1. 1.
_
R
r
J
e
r (X . .)
~r
+ E*[ n,
1.J
J.
=A+
say, and
But
B,
I
I.
I
I
I
I
I
-14-
E[A2 ]
:5
(m+k-l)2
= 0(1)
•
-e
-
I
I
I
I
I
= 0(1)
(m )
max
[
r
V.
.
E
i,j, ( m )
~J
e
(m )
r ( x..
. )]2
~J
r
~
x 0(1) x O(!)
N
1
x O(N) x 0(1)
uniformly for all i,j.
Hence the theorem follows immediately.
At this point it is convenient to define
s 2
i
= _"_1_
n. -1
~
n.~
E (Y;J' _ y;)2
j=l
...
...
for 1 $ i $ k, and also
k
E
i=l
(n i -l)si
N-k
2
1
= -N-k
k
ni
E
E
i=l j=l
-
2
(Y;J'
... - Yi )
Then we have
Theorem 5.
If Assumption Bl holds, then as
1 $ i $ k we have
I.
I
I
r
(m)
r]2
~
uniformly for all i,j, and also
I
-•
m
n. -m.
[-2:-.2:.
. max
( ) n.
~,m
N~
00
and
n./N~
~
p. > 0 for
~
I
I.
I
I
I
I
I
I
•••
-e
...
-
I
I
I
I
I
I.
I
I
-15-
in probability, and
in probability.
Proof.
Let
n.
Then
n.
-1
1.
1.
E
•
J=
law of large
1
a .. 2 ~ (~._e.2) in probability by Khinchin's form of the
1.J
1.
numbers, and
1.
n.1.
-1
n.1.
E (a .. - b .. )2 ~ 0 in probability
1.J
1.J
j=l
n.
by Theorem 4.
Then by Lemma 1 of Sen [7J
n.1.
-1
1.
probability, and certainly
1
n. -1
1.
in probability also.
But
n.
1.
E [(N-1)-1
m
j=l
., _ e.J2 _
Y1.J
1.
ni
n.-l
1.
[(N-l)-._e.J2
m y1. 1.
and the second of these two sums converges to zero in probability by Theorem 3.
Thus the first part of the present theorem is proven; and the second part
follows immediately from it.
Theorem 6.
If the hypothesis is true, and if furthermore Assumption Bl and
I
I.
I
I
I
I
I
I
I
-16-
Assumption
both hold, then as
.e
I
N~
00
and
n./N~
~
p. > 0 for 1
~
~
i
~
k the analysis of
variance statistic F based on the scores has asymptotically the F-distribution
with (k-l, N-k) degrees of freedom.
Proof.
We present a direct proof instead of reducing the present theorem to
Theorem 2.
Under the hypothesis, the scores are interchangeable.
= (N-l
m) e
that the expected value of any score is E[ Yij ]
We have
for all i,j; let us
write O~ for the variance of any score, and P for the correlation between
N
any two scores.
Now let Z
=
(Zl,Z2"",Zk)' , where the Z's are the random
Then E[~]
variables in the statement of Theorem 3.
=
0 and it is easily verified
that
I_
I
I
I
I
I
I
I
c:
var [ ~ ]
= I: N = N( N-l)-2
m
2
[(
oN
-1
l-P N) DN
+ PN J ]
where D = diag (nl,n2""'~) and J is the matrix in which every element is
N
2 2
unity. Let ~.= ( N-l
m ) 2 (NDN - B ~.) / NON
(l-PN), where B = (n l ,n2 ,···,nk )··
Then ~N = I-D~/N is an idempotent matrix of rank (k-l).
Hence, since by
Theorem 3 the joint distribution of the Z.'s is asymptotically k-variate normal,
~
it follows that the quadratic form
Z'A-2
-
-~~
=
=)2
I:n. ( -y.-y
~
~
2(1_)
ON
PN
is asymptotically distributed as X2 (k-l).
A little algebra will show that
,
Hence by Theorem 5 and Cramer's convergence theorem we have
that
-17-
is asymptotically distributed as X2 (k-l).
And this is equivalent to saying
that F is asymptotically F(k-l, N-k).
We note that making Assumption C is equivalent to saying that the random
variable e(x) should not equal a constant with probability one when X is a
random observation from G.
Theorem 7.
for 1
~
i
If Assumptions Bl and C hold, then as
~
N~
00
and
n./N~
1.
p. > 0
1.
k the test of H based on the analysis of variance statistic F
o
is consistent against any alternative for which
11.=
k
E
i=l
-
Proof.
p.(e. -e)2>0.
1.
1.
From Theorem 3 it follows that under the stated conditions the random
variable
=
in probability.
Hence, using Theorem 5,
in probability, and thus, for sufficiently large N, F is certain to exceed
•-
-
~
-
the critical value F* corresponding to any a .
e
We remark that if for each N the alternative
~
is true, where under
~
I
Il.
I
I)
II
I
Ii
1\
I)
--- e
I
I
-18-
then under reasonable regularity conditions the asymptotic distribution of
F will be noncentral F with (k-l, N-k) degrees of freedom and
no~centrality
parameter
Ep. (0. _0)2
6,,-
-
where 0 = Ep.O ..
1.
4.
1.
1.
1.
~_e2
However, we shall not attempt to present a general theorem.
SCORES RELATED TO MODIFIED U-STATISTICS.
Suppose the function
~
of the
preceding section depends on an unknown parameter .. (which may be vec'torvalued), that is
Then we may consider using an analysis of variance test based on modified
scores
I)
-
I
I
I
I.
I
I
where T has been estimated by a statistic t
of all the N observations.
Assumption D:
If
N
which is a symmetric function
We shall require the following
XO,Xl""'~
are independent observations from G, then
Then we may state
\
I
1
II
-19-
1
•
I)
II
I
Ii
Theorem 8.
If the hypothesis is true, and if Assumptions BZ, C, and D all
hold, then as
N~
00
and
Pi
> 0 for I
~
i
~
k the analysis of variance
statistic F based on the modified scores has asymptotically the F-distribution
with (k··l, N-k) degrees of freedom.
Proof.
Define scores
as in section 3.
Then by Theorem 5 we have that
I~
I
N-k
I)
-e
ni/N~
in probability as
N~
00.
And
so by Assumptiqn D
E[ y ..
~J
J 2 = (N-I)2
- z..
m
~J
O(l)
N '
and hence
k ni
1 E
E [(y .. - y)-(z .. - ~)J2 (N-1)-Z
N- k i = 1 j= 1
~J
~J
m
in probability.
k
E
E (Y;J' - z . .)2(N-1)-Z~ 0
N- k i= 1 j= 1 ' "
~J
m
< _1_
-
ni
But then by Sen's lemma we have
n.
k
~
E
E ( z .. - z-)2 (N-l)-Z ~ ~_e2
~
N- k i=1 j=l
~J
m
1
--;;1'"
I
I;.
Ii
I)
Ii
1\
I'
I
I)
-20-
in probability, where ~_e2 > 0 by Assumption C.
find that
max
i, j
and thus we see immediately that the modified scores satisfy Assumption A.
Finally, since under the hypothesis these scores are interchangeable random
variables, the present theorem follows from Theorem 2.
5.
EXAMPLES
Example 1.
(Kruskal-Wallis Test).
I)
I
I
II
'.
I
I
Let X.. be univariate, and define the
l.J
corresponding score as
Yi'
1\_
I'
I
Now, using Assumption B2 we
J
= R..
l.J
- 1
where R is the rank of X among all the N XiS, average ranks being used in
ij
ij
case of ties. Then
y •.
l.J
= E l.J.•
$(X .. , X) = E •• "ljr(X
l.J
l.J
where
"ljr(z)
=
o
z
1.
2
z = 0
1
z
<
>
0
0
The analysis of variance statistic turns out to be
F
Where
=
(N-k)H
ij
- X)
..21 ..
I
J
n.
~
j= 1
k
H =
~
12
N(N+l)
L.
2
R..
1.J
-
i=l
-
3(N+l)
ni
is the familiar Kruskal-Wallis statistic, usually treated as X2 (k-l).
(It may
be noted that F is one of the alternative expressions considered by Wallace
raJ,
who showed that the approximation to the conditional distribution afforded by
treating it as F(k-l, N-k) is better than the usual one is, at least for certain
very small sample sizes,
approximations.)
although not so good as various other more complicated
In the standard method it is necessary to make special adjustments
for ties; however, in our approach equivalent adjustments are made automatically
as a part of the computation.
Finally, we have
e.1. = p(X.1. >
X}
+ -21P(X.1.
= X},_ e = 21
where X. and X are observations randomly chosen from G. and G respectively.
1.
Example 2.
1.
(Rank Analysis of Covariance).
Let X..
1.J
(0)
(1)
(r)
= (X..
, X. .
, ••• , X. .
)
1.J
1.J
1.J
where X(O) is the response and X(l), ... ,x(p) are concomitant variables.
R.. (u) be the rank of X.. (u) among all the N X.. (u)'s for 0
1.J
1.J
1.J
~ u ~ p.
Let
Then the
rank analysis of covariance procedure proposed by the author [6J consists of
performing an ordinary analysis of variance on scores
= [R
ij
(0) _ N;lJ _ :
c [R.. (u) .. N+l J
2
u=l u 1.J
where the c's are any suitable set of constants.
as y ..
1.J
= I: i. J.
$(X. j' X) where
.
1.
These scores can be obtained
I
I.
I
I)
-22-
4>(X .. ,X) = [\jI(X .. (0) - X(O»
~J
No matter what
CiS
tJ - u=l
i.
-
~J
c ['!r(X . . (u) u
~J
i
u»
-
tJ.
have been selected, we have
I,
1\
I
I
I'
,_
I'
I
1\
I'
I
II
I'
Ie
I
where X.(O) and X(O) are observations randomly chosen from G.(x(O), 00, ••• ,00)
~
~
and G(x(O),oo, •.• ,oo) respectively.
If c = ... =c =0 then the test is equivalent
p
1
to the modified Kruska1-Wa11is test of Example 1.
In [6J it is shown that, from the standpoint of asymptotic relative efficiency,
the optimal choice for £'= (c , .•• ,c ) is~' =
p
1
2I'nJ, n
is the vector of
,
covar~ances
A is the variance matrix of (R ,
iJ
(1)
I'
between R..
where
(0)
~J
, ... ,R..
~J
(p)
).
I'
minimizes
and (R..
(1)
~J
14> I
< 1 + r. Ir u l , Assumption
between R. .
~J
(0)
and (R..
~J
(1)
, ... ,R..
(p)
~J
), and
Suppose we choose c' to minimize
[£'L£ - 2£'!] where! and L are the sample estimates of ~ and A.
using modified scores as in Section 4.
[I'A I -
Then we are
Assumption B2 is satisfied since
C is satisfied unless the mu1 tip1e correlation
, ... ,R. ,
~J
(p).
)
~s
are continuous functions of U-statistics.
unity, and Assumption D since the
CiS
Thus the test with estimated optimal
c's is also asymptotically valid.
Example 3.
(Mood's Squared-Rank Test).
Let X.. be univariate and define the
~J
corresponding score as
y .. = (R., - l)(N - R, .),
~J
LJ
LJ
where R
is again the rank of X among all the N
ij
ij
XiS,
provided that X is
ij
not tied with any other X; let the score attached to each member of a group of tied
observations be the mean of the various scores they would receive if the ties were
I
I.
I
I;
II
I'
I
I
I)
,_
-23-
broken arbitrarily.
Then
y ..
1J
= L: 1.J
..
$ (X .. ,
1
if xl <
x2
or
x2 <
X
1
if x = xl =f x
2
0
or
X
x2 =f xl
1.J
[2 X, s])
where
2"
O' xl' x2 )
$(X
e.1.
o<
o
I:l
1/3 if
X
o=
xl
otherwise.
is the probability that an observation randomly chosen
1
e.1. = e
1:1
xl
= x2
from G. will lie between two observations randomly chosen from G;
sis is true then
o<
1:1
0
If G is continuous then
X
if the hypothe-
1/3.
Now, the squared-rank test proposed by Mood [4] for the case where k
1:1
2 and
there are no ties is based on treating the statistic
I'
_ !!:l)2
(R
2
lj
I
1\
I'
I
II
I)
Ie
I
as normally distributed with mean E[W] = n (Ntl)(N+2)/12 and variance V[W]
l
1:1
n n (N+l) (N+2) (N-2)/180; equivalently,
l 2
may be treated as X2 with 1 degree of freedom.
After some algebra it can be
shown that for this special case the analysis of variance statistic based on the
scores is
I
I.
I
I
I
I'
I
I
Ii
,_
I'
I
I
I
I
I;
1
1
Ie
I
-24-
F(1,N-2) - (N-2)Wi:
- (N-l-W*)
Note, however, that as in Example 1 the extension to k
> 2 is made immediate
through the general approach, and again that no special adjustment for ties is
required.
Example 4.
of
(2-by-k Contingency Tables)
any space
n
and let
nO
a) Let the observations X.. be elements
1J
n.
be any prespecified subspace of
Then the data may
be summarized as a contingency table with k rows, corresponding to the k samples,
and 2
column~
corresponding to observations in or not in
of observations from the i-th sample which fall in
nO'
nO;
let m be the number
i
and let M = Em .
i
The standard test is based on treating the statistic
X2
as X2 (k-l).
y ..
1J
= E 1J
..
Let $(Xij,X)
=
1
M(N-M)
=1
(m.N-n.M)2
k
1
E
1
n.1
~l
or 0 according as Xij E
nO
or ~
nO
or
~
$(X .. ,X) = (N-l) or 0 according as X1. .
1J
J
€
nO'
nO;
then the score
and the analysis
of variance statistic turns out to be
F
The parameter
e.1
n
nO.
Now suppose that the subspace
parameter.
Here
(N-k)X2
(k-l)(N-X2)
is then just the probability that a random observation from the
i-th population will fall in
b)
=
nO
is defined in terms of some unknown
As a definite instance, consider the extension of the median test.
is the real line and
nO
is the portion to the right
of~,
the population
I
••I
I
I
I
I
I
I
I'
I
I
I
I
I
1\
.e
I
-25-
median.
We use modified scores
Z. •
LJ
= ~..
LJ
$ (~; X.. ,
LJ
X)
where $(~; X.. , X) is 1 or 0 according as X.. does or does not exceed~, the
LJ
LJ
sample median.
That this test is asymptotically valid may be shown by the method
of Section 4, although in this case it is easier to apply Theorem 2 directly.
Other instances may be adduced.
I
~I
-26-
,
I
,I.
I
I
I
I
I
I:
REFERENCES
[lJ
BHAPKAR, V. P. (1961). A nonparametric test for the problem of several
samples. Ann. Math, Stat. 32 1108-17.
[2J
CHERNOFF, H., and TEICHER, H. (1958). A central limit theorem for sums
of interchangeable random variables. Ann. Math. Stat. 29 118-30.
[3J
" H. (1951).
CRAMER,
University Press.
[4J
MOOD, A. M. (1954). On the asymptotic efficiency of certain nonparametric
two-sample tests. Ann. Math. Stat. 25 514-22.
[5J
PITMAN, E. J. G. (1938). Significance tests which may be applied to
samples from any populations, III. The analysis of variance test.
Biometrika 29 322-335.
[6J
QUADE, D. (1965).
Statist. Assoc.
I
I_
I
1
I
I
1
,I
I
I·
I
Mathematical Methods of Statistics
Rank Analysis of Covariance.
Princeton
Submitted to J. Amer.
SEN, P. K. (1960). On Some Convergence Properties of U-statistics.
Calcutta Statist. Assoc. Bull. 10 1-18.
[ 8J
.n.
WALLACE,
L. (1959). Simplified beta-approximations to the KruskalWallis H test. J. Amer. Statist. Assoc. 54 225-30.