I
I.
I
I
I
I
I
I
I
I_
I
I
I
I
I
I
I
II
ANALYSIS OF VARIANCE OF DEPENDENT DATA
by
Bruno Baldessari
Department of Statistics
University of North Carolina
an~
Universita di Roma
Institute of Statistics Mimeo Series No. 467
March 1966
CONTENTS:
1.
Introduction and notation.
2.
Conditions c), d) and matrix V.
3.
Conditions d), e) and matrix V.
ACKNOWLEDGMENTS
REFERENCES
This research was supported by the Mathematics
Division of The Air Force Office of Scientific
Research Contract No. AF-AFPSR-760-65
DEPARTMENT OF STATISTICS
UNIVERSITY OF NORTH CAROLINA
Chapel Hill, N. C.
-~
1.
Introduction and notation
The aims of this article are:
a), show that analysis of variance
(ANOVA) can be performed also when the elements of the "sample" are dependent
and b), establish the more general type of dependence which allows us to do
an ANOVA.
To make these statements precise we need some definition
and
notation.
Definition:
By a "sample of n dependent elements" we mean a
random sample of 1 element
drawn from the r. vt. (random vector) X*
in which the n r.v. (random variables) X~, i
~
= 1,
= (X*l'
,
••• ,X* )
n
••• , n, may be dependent [3J.
We will denote this random sample by the r. vt. X
= (Xl'
,
••• , X )
n
and it is clear that the d.f. (distribution function) of X is the same as that
-e
of X* so that the dependence of the n r.v. Xl' ee.,
*
dependence of the r. v. XI, e•• , Xne
normal with mean vector
~
~
is defined by the
If the r. vt. X* is
N(~,V),
(multivariate
and variance-covariance matrix V), where V is p.d.,
(positive definite), then X is N(~,V) and the dependence of Xl' ••• , X is
n
specified by V.
In this article we extend the validity of ANOVA to the case in
which X is N(~, V) so that the elements of the sample may be dependent.
More
precisely, we extend the validity of the methods of the classical ANOVA, (in
which. X is suppose to be N(~,QI), Q > 0, I is the n x n identity matrix) built
null hypotheses Hl , ••• , H~ which specifies ~ distinct linear
relations· on the parameters of the general linear model of the data X, against
up to test
~
the ~ ulternative hypothesis:
~
-e
we
re~uire,only,
at least one
1
= (1,
'.0' "not
H~".
Of the mean vector
that the set of all a priori possible vectors
* of
vector~.
••• , 1) I .
"not Hl ",
the form
~
* = ~ 1,
~
contains
where 1 is the n-vector:
We will supposethat H , H , ••• H'l are such that the total
1
2
1
[
:-L
sum squares X'(I
n-\J)X, (U = 1 1 ' ), can be decomposed into (q+l) positive
,
"
semidefinite quadratic forms: X 80 X, X 81 X, ••• , X 8 X, such that:
q
q
a) •
=I
8j
E
j=O
- n
-1
f
U;
... , q;
b) •
8 2
j
c) •
X· 8 j X is distributed like Q X (n , 2 ~ 8 ~),
j
j
(noncentral chi-square with n degree of freedom and
= 8 j , .j=O,
,
'2
j
-1
J
,
noncentrality parameter 2- 1 ~. 8 ~), where n
j
rank
.'
,
d) •
X 8o X,
e) •
X
8
n
I
F'
j
= 0,
(8 j), j
X
q
no
-y-~-
=
j
••• , q;
... , X . 8
X, are mutually independent; .
(n j , no' ~. 8 j ~,~
0
8
~)'
(.8ne.decor.' s double non-
central F distribution with n j and no degrees of freeI
dom and noncentrality parameters
j
= 0,
Actually, if X is
... , q and for every
N(~, Q
~
,
8j~' ~
~),
80
~.
I) then conditions a) and b) imply
conditions c) and d) which imply condition e).
In the present extension we regard the matrices 80 , •.• , 8 as
q
fixed; typically, if X is
,
N(~, Q
I) the matrices 8 j , j
=0,
••• , q, will be
such that X 8j X, j = 0, ••• , q, are, in some sense, the "best" statistics
to use in order to test H , ••• , H against these alternatives.
l
q
We suppose that the fixed matrices So , ••• , Sq , satisfy conditions
2
I
I
L
r
I
[
er
is distributed like
X-SX
o
j
_I
I
l
l
~
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
a) and b) and our generalization of ANOVA consists in the fact that we establish
the set of ALL p.d. variance-covariance matrices V such that, if X is
then conditions c), d), e) are also still satisfied.
N(~,V)
In fact this set of
matrices is the set of p.d. matrices V of structure:
* = 2 -1("A+A ,)
V
(1)
+ a(I-U),
where the n x n matrix A is:
A
{~;:.::::.~;)
a , ••• , a
n
= 1,
a. > 0, i
,
n
-Ie
with
•
show that if X is N(I-l,V) and if a) and b) are satisfied then, Theorem I):
..
~
••. , n.
The proof of this fact is based on two theorems which,actually
conditions c) and d) are equivalent to (1), and, (Theorem 2):
and e) are equivalent to (1).
is equivalent to saying that
2-
1
conditions d)
We also note that, if X isN(I-l,V), formula (1)
x' (I - n-1 U)X is distributed as a x' 2 (n-l,
J.L'·[I-n-:IuJI-l), as follows immediately from the proof of Proportion 1 of [3J.
In the following we write <A> <=> <8> to mean that the statement
<.A> is equivalent to the statement <-13>, and
< D{x)
= D{Y»
for the statement
"the distribution of the r.v. X is the same as the distribution of the r.v. Y:'
,
To avoid cumbersome formulas we will write < I(X'S.X) > for the
,
J
statement "the r.v. X S. X, j
J
I
I·
I
S for I-n
I-l
-1
'2
U; X
= 0,
'2
S for X (n ., 2
I-l, j
••• , q are mutually independent and, also:
-1 -'.
I-l
J
So I-l).
3
,
S. J.L); F
J
1
S S
I-l, j'
0
for F (n., no ,
J
L
2.
.r
Conditions c), d) and matrix V.
If X is
Theorem 1.
N(~,V)
and matrices So , ••• , Sq satisfy a) and b),
r
then:
'-"'-. V=V
. *> .-". . => <
[
. = D(a X'II 2
D(X~ S .X}
S },.J = 0, ••• , q, and I(X' S .X} >
J
".., j
J
l
Proof of the implication: <=.
Conditions a) and b) imply that:
[
rank (S) = n - 1, so that we have:
,
'2
'
< D(X 'S. X} = D(a X " S }, j = 0, ••• , q, and I(X -S. X}
J
,.., j
J
,
q
< D[X '(J.~- Sj' )X)
-v
= D(a
'2
X
> =>
t
er
' 2
"'S} > <=> < D(X SX}
~,~
l
j
= D[a Xl ~,s ) > <=>
*
[
~V=V>.
The last equivalence follows from the proof of Proportion 1 of [3J.
Proof of the implication:
=>.
First we prove that:
L
[
< V = v*> => <:: D(X',S.X} = D(a x'~ S }, j = 0, ••• , q> •
J
r
,.., j
In fact, from the proof of Proportion 1 of [3J, Cor.ollary 5.1 of
[5], and from condition a) we have:
< a- l S V S = S > <=> < a-l(s + ••• + S )V(S + ••• +S ) =
.
0
4
q
0
q
s0
+. .• +Sj'
E
!
I
••
I
I
I.
I
I
I
I
I
so that, from condition b)
.( V
= V*>
-1
<0'
vie
have:
=> < O'-lS.(S + ••• + S ) V (S + ••• +S ) S.
JO
q
0
q
J
= S.(S
+
JO
••• + S )S.> <=>
qJ
S.VS.
J
J
Now we prove that:
< V = V* > => < I ['
X S.X J >,
J
and this will complete the proof of Theorem 1.
I
I
From what we have just proved we have:
< V
I_
= V*
> <=> <
0'-1 (S
o
+ ••• + S ) V ( S + ••• + S )
q
q
0
= S0 +
••• + S
q
>,
so that, from condition b) we have:
I
I
:: V
= V*>
=> < O'-lS.(S + ••• + S ) V (S + ••• + S ) Sk
Jo
q
0
q
= S.(S
+
JO
••• + S ) Sk' j"k>
q
-
I
i
The last equivalence follows from Craig's condition [4J in Aitken's generalization [lJ from which we have that Sj V Sk
I
•
,
X Sj X, j
= 0,
= 0,
j "
••• , q are pairwise independent.
k implies that the r.v.
(In our particular case,
pairwise independence implies mutual independence as can be seen from the
relevent characteristic functions.)
5
I
3.
_.
Conditions d), e) and matrix v.
Theorem 2.
If X is
N(~,v)
and matrices So , ••• , Sq , satisfy a) and b),
then:
:h
___~o_J
Xl S X
o
= D[F'~,S.,S
},
J 0
j=l, ••• ,q for all ~,and I{XISjX} >
I
The proof of the implication => is obvious from (2).
Proof of the implication <=.
We may suppose
~
= ~*
~1
=
I
I
I
I
because in the second statement of
I
I
_I
< S ~* =
j
= 0,
° > <=> «So+ ••• + Sq ) ~* = ° > => < S.(S
+ .•• +Sq ) ~ * = 0,
J
0
••• , q >
and similarly for
* = 0,
<=> < S.
~
< ,,*1 S j
= 0,
J
~
Now we show that, with
°
J' -
~
= 0,
j
••• , q'> ,
, ••• , q
=
~
*,
>•
and for every p.d. V, we have:
(4 )
6
I
I
I
I
I
I
I
-.
I
I
I.
I
I
I
I
I
I
I
1_
~1ere ~jk'
j
= 0,
••• , q, k
= 1,
... , n J., are positive constants and where
~1
(1) is a (central) chi-square r.v. with 1 degree of freedom and, also,
J{
the chi-squares are mutually independent.
In fact, let ~(to , ••• , t q ) be the characteristic function of
. ,1
X S)
X, so that we have:
the r. vt. ( X1 S X, ••.
o
q .
1 (x- J.i)
* I V-1 (x- J.i * ) }, dx ,
Sj x - '2
wbere
J denotes n-fold integration and dx
From tbe relations:
J.i
*
Sj
follows that with the transformation y
cp(t , ••• , t ) ex
o
q
If
I
that:
IT' 1=0,
n
= Sj J.i * = 0,
=x
j
I
-1
exp (i .~ t. Y SJ' Y - -2 Y V
j
J=v
= 0,
••• , q, it
- J.i * we have;
1
q
J'
= dX, ••• dX •
J
}
Y d y.
V is positive definite then there exists a matrix T such
V
= TT
,
, and the transformation y
,
= Tx
1
gives:
'
T' Sj T x - '2 x x) dX •
,
The matrices T Sj T, j = 0, ••• , q are symmetric and
so that:
I
I·
I
,
,
(T' Sk T) (T
Sj T), j ~ k, and this implies that a matrix P exists
7
I
,
such that:
P P
= I,
,
P
S. P
J
= !I. j '
with !I.. diagonal, j
J
= 0,
••• , q.
Then
the transformation x = P y shows that:
q
t
1 t
ep (t , ••• , t ) oe. J.exp {i .~ t. y' !I.. y - -2 Y y} d y.
o
q
J=v J
J
Now !I. j is positive semidefinite so its diagonal elements are
non-negative and there are exactly n j positive elements in the diagonal of
!I.
j
because:
rank (!I. j )
= rank
(Sj)
= nj ,
j
= 0,
••• , q.
j 1= k, do not have any positive elements in the same row:
0.
Also !I. j and !I.k ,
in fact, !I. !I. =
j
k
From this it follws that:
0) = 1, we see that cp(to , ••• , t q ) is exactly
equal to the right hand side of
(5), so that formula (3) is proved.
Now we prove the implication: <=.
In fact, from (3) and (5)
we have:
t
< D{
X S.X
n
A
no
j
ok
t
-,---} = D{F
J
X "So X
~ S S}, j
' j'
= 1,
••• , q, for all ~, and
0
,
X2
ok
I
I
I
I
I
I
I
_I
where Ajk is the k-tb. positive element in the diagonal of !l. •
j
Since cp(O, ••• ,
_.
I
I
I
I
I
I
I
-.
8
I
I
I.
~
where F(n., n) is a (central) Snedecor's F r.v. with n
J
freedom.
I
1_
< D[
n
J
\
0
k~l
< /\jl =
/\
ok
X2
= /\ j
S
degrees of
S}, j = 1, ••• , q, for allll, and I{X'SjX}> =>
ok
nj
(1)
=/\
01
2
{x
(n j ) },
=D
! (no)
J = 1, ••• , 'l., and
=
a > 0, j = 1, ••• , q, and I{X'SjX}> <=>.
= /\
on
=
2
<: D[X'S.X} = D{al (n.)], j = 0, ••• , q, and I(X''S.X} >
J
J
J
,
r[x' SJ~ <=>
o
=>
q
2
'-2
j~ Sj X} = D(a 7C (n-l)} > <=> < D(X S X} = D(a J. (n-l)}> <=>.
< V
= V* >,
in which the first equivalence symbol is justified by the proportion of [2J.
This completes the proof of theorem 2.
]
-,
I
,.
t
I
0
Il, j' 0
j
J
I
= D[F'
2
k~l /\jk ~jk (1)
D
/
}
XS X
o
J
n
0
1
n.
<: D(X
]
and n
From that 'He have:
X'S.X
]
J
1
j
,
~
J
0
9
I
_.
ACKNOWLEDGMENTS
The author is grateful to Professor N. L. Johnson
for helpful discussions and for revising this article.
I
I
I
I
I
I
I
_I
I
I
I
I
I
I
I
-.
10
I
I
I.
I
I
I
I
I
I
I
REFERENCES
[lJ
Aitken, A. C.
Statistical independence of quadratic forms in normal
varieties.
[2J Baldessari, B.
Biom., 37, (1950), 93.
Observation sur Ie rapport de combinaisons lineaires
de chi-deux.
Pub. de l'Institut de Statis. de l'Univ.
de Paris, (1965).
,
[3J
Baldessari, B.
Observations sur les l'echantillons gaussiens dependents,
Pub. de l'Institut de Statis. de l'Univ. de Paris,(1965).
I_
[4J
I
I
I
Craig, A. T.
Not on the independence of certain quadratic forms, Ann.
Math. Stat., 14, (1943), 195.
[5J
Traybill, A. F. and Marsaglia, G.
Independent matrices and quadratic
forms in the general linear h;ypothesis, Ann. Math. Stat.,
28, (1957), 678•
•
11