Zo.
J
MAXIMUM LIKELIHOOD ESTIMATION FOR THE GROWTH CURVE
MODEL WITH UNEQUAL DISPERSION MATRICES
By
S. R. Chakravorti
Department of Biostatistics
University of North Carolina
and
University of Calcutta, India
Institute of Statistics Mimeo Series No. 893
OCTOBER 1973
Maximum-likelihood Estimation for the Growth Curve
Model with Unequal Dispersion Matrices*
S. R. CHAKRAVORTI+
Department of Biostatistics, University of North
Carolina, Chapel Hill, N. C. 27514
Abstract
The paper considers the maximum-likelihood estimation problem for the
growth curve model -- treated as MANOCOVA model --with unequal dispersion
matrices.
Optimality properties of the estimates have been studied and pro-
cedure for testing equality of several growth curves has been indicated •
.e
*Work sponsored by the Aerospace Research Laboratories, U. S. Air Force Systems
Command, Contract F 33615-71-C-1927.
Reproduction in whole or in part permitted
for any purpose of the U. S. Government.
+On leave of absence from the University of Calcutta, India.
2
1.
INTRODUCTION
Consider the growth curve model as multivariate analysis of covariance
(MANOCOVA) with stochastic predictors (Rao [9]).
The problem of estimation of
the parameters of this model with unequal dispersion matrices does not follow
the usual procedure of either maximum likelihood method or least squares method
when dispersion matrices are equal.
In the univariate situation of linear model under heteroscedastic assumption the methods of estimation have been proposed by C. R. Rao [10] and Hartley
and Jaytillake [6].
The later method follows the procedure of Hartley and
J. N. K. Rao [5] and is free from the defects of the MINQUE method proposed by
C. R. Rao.
.e
Here the method of Hartley and Jaytillake has been generalized for
the MANOCOVA model stated earlier.
The proposed method of estimation yields
estimates which are the solutions of maximum likelihood equations by the steepest
descent method.
The solutions are, in fact, the asymptotic limit to the solu-
tion of a system of first order differential equations.
The asymptotic optimality
properties of these estimates have been studied. 'Also the procedure of testing
equality of several growth curves has been discussed.
2.
THE MODEL
The usual growth curve model (viz., Potthoff and Roy [8])
wri~ten
as
MANOCOVA model, under Behrens-Fisher situation, is given by
(2.1)
where y(t) (lxp) is the a-th observation vector in t-th sample (a=1,2, ••• ,n ;
t
~a
t=l, ••• ,m),
~(t)(lXp) is the vector of unknown constants in t-th group, which
is a p-th degree polynomial in time in growth curve model,
~~t)(lXS) is the a-th
observation vector of concomitant variables in t-th sample (where s = q-p, q
~
p),
3
~(SXp),
the common matrix of regression coefficients of
~~t)
on
~~t), ~~t),
the
error component in t-th sample, distributed as Np(Q, ~t)' where ~t(pXp) is the
conditional dispersion matrix in t-th group.
~(t), ~t'
The problem is to estimate the parameters
S, for t=1,2, ••• ,m,
by the maximum likelihood method.
3. MAXIMUM LIKELIHOOD ESTIMATES
OF THE PARAMETERS OF THE MODEL
The log-likelihood of the model (2.1) is given by
log L = const + ~
.e
m
1
L ntl~~ I
(3.1)
t=l
where, from (2.1), 8(t) = y(t) - n(t) - x(t)S. Then following the method of
-a
-a
-amatrix derivatives (Dwyer and Mephail [2] and Dwyer [3]) we have
(3.2)
n
aalog
I:
-t
L =
t
~2 nt~-tl
+ k2~-tl(
t
~(t)'~(t»~-l
t-l
m
~
~
L ~N
~N
~t'
- , .•• , ,
a=l
~
(3.3)
~
(3.4)
For given
~,
we have the estimates of n(t) and
~t'
by equating (3.2) and (3.3)
to zero,
(3.5)
(3.6)
When
S
is unknown, to estimate ~ we have by substituting (3.5) and (3.6) in (3.1)
4
m
F(S) = const -
where
~t(~)
is some function of
m
~t~lnt log!2 t l = ~tIl~t(~)
S.
(3.7)
Then the estimate of ~S=«S ron » is obtained
by solving
(3.8)
For this we generate an asymptotically convergent sequence from the steepest
descent differential equations
as ron
--ae =
.e
aF(S)
aD
~
j.)mn
t for m=lt"'t s ; n=lt"'tpt
(3.9)
where 8 is the parameter in the parametric representation of the path of
descent S (8) •
ron
Then following Hartley and Jaytillake [6] t we observe that
~
for
8~t
the path coordinates S (8) will tend to a limit S
such that
mn
ron
(3.10)
By following Runge-Kutta procedure (Henrici [7]) a local minimum of F(S) is
attained in this case which depends on the initial values
the system of simultaneous equations (3.9).
Smn (8)
The estimates of
selected for
~t ~t'S and n(t),s
are obtainable by solving simultaneously
(3.11)
and (3.5) and (3.6) by feed back principle.
denoted by 5(t) t
kt
and
The m.l.e.'s thus obtained are
~ respectively for net) t ~t and S.
5
4.
ASYMPTOTIC PROPERTIES OF THE
MAXIMUM LIKELIHOOD ESTIMATORS
We have altogether 2m+1 (matrix) parameters denoted by
_
~-(~
(1)
, •.. ,~
(m)
efficiency of
~.
,
~l""'~m' ~).
To prove the consistency and asymptotic
we assume,
m
L n are large and
t=l t
= nt/n (for t=l, ••• ,m) are bounded away from 0 and 1.
m = the number of groups is fixed, n
(i)
r
t
t
and n =
The stochastic concomitant vector variable x(t) is distributed as
(ii)
~a.
Ns (0,
E*), so that
~
~t
E ~(t)
.e
4.1.
(4.1)
Consistency
A
lim
Prob[~
= ~O] = 1
(4.2)
n~
(1)
(m) 0
0
where ~O=(~O ,···,~O ,k1"",km'~0) is the true value of
To prove this, let us first prove that for
V
e
~O
Let
6~(t)
~(t), ~
=
and
~(t)-~cit), 6~
~t
=
respectively.
[.!.log
L(Yle)]
n
~
~-~O'
where
~
@~@O
= 0(1)
n
~cit), ~O
e.
and
(4.3)
~~
are the true values of
Then from (3.1)
(4.4)
6
where 6y(t) = 6n(t) + x(t)6B. Let us choose anon-singular matrix ~t(pxp) such
-a
-~0-1~~ = !' so that ~t = ~~~t· Also let z(t)=(y(t)_n(t)_x(t)S )A- 1 .
that ~t~t
~a
~a
~o
~a
~o ~t
°
Then z(t) is distributed as Np (0, I ).
~a
~
Under this set of transformations we
~p
have from (4.4)
-1
(t)
where A
E A' = D
= diag(A 1
~t~t ~t
~t
roots of
~-t1
·(t)
.••p
A
"
(t)
) and A.1.
and hence finite, so that V(L(l»
's are the characteristic
= 2ln
t
IA~t)2.
t i
Hence from
1.
assumption (i)
V(.!. L
) = 2 l l "1': (A~t»2 = 0(1)
n (1)
n t i t 1.
n
.tt
Since for fixed
~~t),
(4.5)
E(L(2»=0, we have
= E~ ~Z(t)A E-1 6y(t)'6y(t)E-1A'Z(t)'
L L~a
t
a
~t-t
-a
-a
-t
~t~a
= tr ~ ~A E- 16y(t)'6y(t)E-1A'oI
L L~t~t ~a
-a ~t ~t
t a
-l
t
~6y(t)E-1EOE-16y(t)'
L -a ~t -t-t ~a
a
= ~n 6n(t)E-1EOE-1on(t)'+2~n 6n(t)E- 1EOE-16B'x(t),
~ t ~
~t ~t~t
~ t ~
~t ~t~t ~ ~
+ ~ ~ X(t)6BE- 1EOE-1oS'X(t),
L L ~a
~~t ~t~t --a
t a
This is conditional variance and since
6~(t) and o~ are fixed and ~t or ~~
are positive definite symmetric matrices, we have on taking expectation over
x(t) and assumptions (i) and (ii)
-a
(4.6)
Since (4.4) is conditional likelihood function, L(3) can be treated as a constant and hence its conditional variance is zero and since E x(t)=O
its
~a
~,
7
variance is zero also unconditionally.
Again the three covariance terms are
zero due to the fact that Z(t),s are distributed as Np (0, I).
~a
~
~
Hence the result
(4.3) follows from (4.5) and (4.6).
Now from (4.3) and Chebychev's inequality we have
lim Prob [~ log L (X I~)
(4.7)
n~
where EO is the expectation when true parameter ~O holds.
Also for any ~~~O' we
have from Lemma 1 of Wa1d [11]
(4.8)
_e
If the maximum likelihood estimate
A
e provides
global maximum of the likelihood,
then with probability one,
(4.9)
which satisfies the conditions of theorems 2 of Wa1d to hold.
Hence using (4.7)
and (4.8) we have the result (4.2) from the theorem 2 of Wa1d~~,This establishes
A
the consistency of the estimate
4.2.
~.
Asymptotic Efficiency
To establish the asymptotic efficiency of the estimates
A_ A(l)
§-(~
A(m) A
A A
, ... ,~
'~l""'~m'~) we are to prove the
Theorem 1.
.
fo110w~ng.
The derivative of the log-likelihood, a log
L/a~
is asymptotically
normally distributed with a null-matrix as mean and variance-covariance matrix
as the information matrix ~(§)
A
This theorem will then imply that
with mean
e
~
and dispersion matrix J-1.
~
e is
asymptotically normally distributed
--8
Proof.
The elements of the information matriX! are obtained by considering
second derivatives of log-likelihood with respect to parameters.
Let us first
of all show that the off-diagonal submatrices of the information matrix are zero
and diagonal submatrices are the inverse of the dispersion matrices corresponding
to the estimates
A(t)
~
,
A
~t
and
A
~, t=1~2~
To show this let us denote
respectively.
••• ,m.
(3.2)~ (3.3) and (3.4) by gi t ) ~ g~t) and Q3
Then applying matrix derivatives method we have
au(t)
_~~1~,:"
(t) ,
an
2
= a log L = an (t) 'an (t)
(4.10)
au (t) ,
.e
~1
-~-=
as
where
~rt
-
(4.11)
is a (sxp) matrix with (r,t)th element unity and rest are zero,
r=l, ••• ,s; t=l, ••• ~p and each element of (4.11) is a vector of order (pxl).
(4.12)
where J rt is a (pxp) matrix with (r,t)th and (t,r)th elements unity for r+t and
only (r,t)th element unity for r=t for r,t=l, ••• ,p
~
rest are zero.
9
= ~n
(4.13)
t
(4.14)
for r=l, ••• ,s; t=l, ••• ,p.
(4.15)
Now from the assumptions on the model (2.1) and assumption (ii) of Section 4, it
follows that on taking expectations, expressions (4.11), (4.12) and (4.14) are
zero, which proves that off-diagonal submatrices of the information matrix are
zero.
Again on taking expectations over (4.10), (4.13) and (4.15) and after
some simplifications, the inverse of the dispersion matrices of the estimates
n(t)
-
~ and~,
which are the diagonal submatrices of information matrix are
.-
' -t
obtained as follows.
t=1,2, ••• ,m,
(4.16)
(4.17)
(4.18)
10
where
A~B
is the product notation for «aijb ij » when A and B are of some order
(Rao [10]) and p0 Q is the Kronecker's product notation.
Now to prove the theorem we are only to prove that, unconditionally, any
t
linear function of (gi ),
distributed.
where
g~t) and g3' t=1,2, ••• ,m) is asymptotically normally
Let us, therefore, consider the linear function
~~t)(lXp), ~~t)(pXp), ~3(sxP) are matrices of real elements. Then from
(3.2), (3.3) and (3.4) we can rewrite T as follows.
(4.20)
where F(t) = E-~(t)' F(t) = E-~(t)'E-1 and F(t) = E-~'X(t)'.
-1
-t -1
' -2
-t -2
-t
-3a
-t -3-a
Writing (4.20) as T = E E T(t) where T(t) = T(t) + T(t) + T(t) we note
t a a '
a
1a
2a
3a '
that the undonditiona1 second and fourth central moments of T(t), are
a
(4.21)
(4.22)
(which follow from the assumptions in (2.1) and assumption (ii) of Section 4).
Now E~t) being distributed as N (0, E ), T (t) = E(t)F (t), a linear function of
1a
-~
p -t
- a -1
the elements of E(t) is distributed as N(O F(t)'E F(t». Hence
-a '
, -1
-t-1
11
(4.23)
From (4.20)
,
= tr(E(t)'E(t)F(t)
T(t)
2a
-a
-a
characteristic function of
9'(8)
= Ee
=
.-
e
Since E(t)'E(t) ~ W (1, L ), the
-a
-a
p
-t
- E F(t».
-t-2
-2
Ti~) is obtained as (Anderson [1])
i8T (t)
-i8tr(~t~~t»
2a = e
-i8tr (~t~2(t»
• II-2i8L F(t) I·~
-t-2
(4.24)
On taking logarithm of both sides of (4.24) and expanding r.h.s., we have by
collecting coefficients of (i8)2/ 2! and (i8)4/4!,
(4.25)
From (4.20) it is clear that for fixed x(t) , T(t) is a linear function of the
3a
-a
elements of
f~t)
and hence distributed as N(O,
!~t)~t!~t)'),
where
~t = 23~~~3.
Therefore, the c.f. of T(t) for fixed x(t) is
3a
-a
1Jx (8) =
~(i8)2X(t)R x(t)'
-a -t-a
e
(4.26)
Then from assumption (ii), since x(t) ~ N(O, ~*t) we have the unconditional c.f.
""'a,
""'--
of T3(~) by integrating (4.26) over x(t)
-a
u.
'
)P(8) = IE*-1_(i8)2R
-t
=
II
s
I~/IL*I~
-t-t
- (i8)2E*R
-t-t
I~
(4.27)
12
Hence on taking logarithm of both sides of (4.27) and expanding r.h.s. we have
tr(~~!?3~~~3)
]J (T(t»
2 3a
=
]J (T(t»
4 3a
= 3[4 tr(~t~3~~1~3)2 +
(tr(~~!l3~~~3})2 I
(4.28)
]
Thus substituting results from (4.23), (4.25) and (4.28) in (4.21) and (4.22)
and remembering that
Ti~), T~~)
obtain the unconditional moments
and
Tj~)
]J2(T~t»
are independently distributed we
and
]J4(T~t» which are finite and
L~t v 3 (T(t»
be the third absolute central moment of T(t) •
a
a
same for all a.
Then defining
.e
we have
(4.29)
Since V (T(t»
4 a
B
n
t
Ie n
-+0 as n
t
=
t
]J
(T(t»
4 a
is finite and constant for all a, it follows that
-+00.
Thus for the sequence of independent random variables {T(t)}, all the
a
conditions for Liapounoff's central limit theorem [4] are satisfied and hence
T(t) =
a~:T~t)
is asymptotically normally distributed.
Since T(t) for t=l, ••• ,m
are independently distributed it follows that T defined by (4.19) is asymptotically
normally distributed.
~
This T being linear function of (gi t ) ,g~t) ,g3' t=l, ••• ,m)
it follows that d Log L/dQ asymptotically follows multinormal law with a nullmatrix as mean and dispersion matrix~, whose diagonal elements are given by
13
(4.16), (4.17) and (4.18).
4.3.
Hence the theorem.
Unbiasedness
The small sample property of unbiasedness of the estimates follow in the
same line as proved by Hartley and Jaytillake [6] since from the assumptions in
model (2.1) the condition P(~(t»
-ex
5.
= P(_~(t»
-(1,
is satisfied.
LIKELIHOOD RATIO TEST FOR THE HYPOTHESIS
OF EQUALITY OF SEVERAL GROWTH CURVES
From the model (2.1) it is clear that the desired hypothesis is
- H [11(1) =
o -
.e
(5.1)
We have seen that the parameter matrix of the model (2.1) is
~
_
-
[~
(1)
, ••• ,~
(m)
~~l' ••• '~m'~].
Let the unconditional maximum of the likeli-
hood function, obtained in Section 3, be denoted by L(!I~).
parameter matrix contains m+2 parameters ~
o
= (~'~l'
Now under Ho , the
••• '~m'~) and the model
(2.1) reduces to the MANOCOVA model of the same kind.
So that by the same
procedure as in Section 3 we obtain the maximum of the likelihood function,
Ao
given by L(! l~ ). Hence the likelihood ratio test is given by
(5.2)
Since all our maximum likelihood estimates are asymptotically normally distributed and efficient, -2 log A is asymptotically distributed, under H , as a
e
0
central chi-square with p(m-l) d.f.
ACKNOWLEDGMENT
The author is grateful to Professor P. K. Sen for his help and guidance
throughout this investigation.
14
REFERENCES
.e
II
•
[1]
Anderson, T. W., Introduction to Multivariate Statistical Analysis, (1958)
John Wiley & Sons Inc., New York.
[2]
Dwyer, P. S.. and Macphail, M. S., Symbolic matrix derivatives, Annals of
Mathematical Statistics, 19, (1948), 517-34.
[3]
Dwyer, P. S., Some applications of matrix derivatives in multivariate
analysis, Journal of American Statistical Association, 62, (June 1967),
607-25.
[4]
Gnedenko, B. W. and Kolmogorov, A. N., Limit Distributions for Sums of
Independent Random Variables, (1954), Addison-Wesley, Cambridge.
[5]
Hartley, H. o. and Rao, J. N. K., Maximum-likelihood estimation for mixed
analysis of variance model, Biometrika, 54, (June 1967), 93-108.
[6]
Hartley, H. o. and Jaytillake, K. S. E., Estimation of linear models with
unequal variances, Journal of American Statistical Association, 68,
(March 1973), 189-92 •
[7]
Henrici, P., Discrete Variable Methods in Ordinary Differential Equations,
(1962), John Wiley & Sons Inc., New York.
[8]
Potthoff, R. F. and Roy, S. N., A generalized multivariate analysis of
variance model useful specially for growth curve problems, Biometrika,
51, (June 1964), 313-26.
[9]
Rao, C. R., The theory of least squares when parameters are stochastic and
its applications to the analysis of growth curve, Biometrika, 52,
(December 1965), 447-58.
[10]
Rao, C. R., Estimation of heteroscedastic variances in linear models,
Journal of American Statistical Association, 65 (March 1970), 161-72.
[11]
Wald, A., Note on the consistency of the maximum likelihood estimate,
Annals of Mathematical Statistics, 20 (1949), 595-601 •