BIOMATHEMATlCS TRAlNING PROGRAM
~XIMUM
LIKELIHOOD ESTIMATION IN NORMAL MODELS
WHOSE ERROR VARIANCES ARE SOME FUNCTIONS
OF THEIR MEANS
Mohd-Nawi Bin Abd-Rahman
Institute of Statistics
Mimeograph Series No. 1165
Raleigh, N.C.
1978
iv
TABLE OF CONTENTS
Page
LIST OF TABLES •
vi
LIST OF FIGURES
vii
1
INTRODUCTION AND LITERATURE REVIEW
2
SPECIFICATION AND THEORETICAL CONSIDERATIONS OF THE
MODELS
• • • • • • • • • • • • • • • • •
2.1
2.2
Models ~pecifications and Basic Definitions
Verification of Regularity Conditions for
2.3
Asymptotic Properties of the Proposed Estimators
for Model I
• • • • • • • • • • • •
Extension of Results to Models II and III • • • •
Modell. . . . . . .
2.4
3
1
.... . . . . .
7
7
11
14
18
THE CONSTANT COEFFICIENT OF VARIATION MODEL •
19
3.1
3.2
3.3
19
3.4
3.5
3.6
The Likelihood EQuations • • • • • • •
Solutions for a Special Case of k = 2
Solutions for the General k Case ••
Computing Algoritym • . • • . . •
Asymptotic Properties of Estimates.
Numerical Examples and Application •
22
27
32
32
39
4 THE VARIANCE PROPORTIONAL TO UNKNOWN POWER OF
• • • • . • • • •
47
4.1 The Likelihood EQuations
47
MEAlI MODEL
4.2
Information, Hessian, and Variance-covariance
Matrices . . . . . .
. . . . .
51
4.3 Numerical Solutions to the Likelihood EQuations:
4.4
Method of Scoring • • • •
Applications • • • • •
• • • • • • • • •
59
70
5 LINEAR MODELS WITH ERROR VARIANCE PROPORTIONAL TO AN
UNKNOWN POWER OF THE RESPONSE • . • . • • • • • . • •
5.1
Information Matrix, Efficient Scores, and their
Sc aled Forms . . . . . . . . . . . . . .
5.2 Primary Development to Increase Stability
5.3
5.4
75
of the Solution Algorithm • • . •
Extension to General Data
Application • • • • • • • •
75
80
91
98
v
Page
6
SUMMARY AND CONCLUSIONS
7
LIST OF REFERENCES. .
8 APPENDICES
8.1
8.2
8.3
. ..
........
108
....
.........
112
.
Sample program for solving likelihood equations
in normal populations having constant C.v.
by bisection method • • • • • • • • • • • • •
Sample program for solving likelihood equations
in normal populations whose group error
variance is proportional to some unknown
power of mean • • • • • • • • • • • • •
Sample program for solving likelihood equations
in linear model with error variance proportional to some unknown power of response • . •
115
....
116
119
125
vi
LIST OF TABLES
Page
3.1 Summary of results for simulations (const. c.v.) . . . . .
3.2 Absorbance values of three substances in a
chemical assay for leucine amino peptidase
........
3.3 Detailed output for Azen and Reed's data
(const. c.v.)
4.1 Summary of results for simulations (unknown power) ••
41
43
44
68
4.2 Detailed output for Azen and Reed's data
(unknown power)
•••• • . • • • . • •
71
5.1
Summary for initial simulations (linear model) •
86
5.2
Estimates for initial simulations (linear model) ••
87
5.3 Detailed output for initial simulation.
88
5.4 Summary for the general data
5.5
Summary of estimates for the above •
5.6 Data used in Carr's example
95
99
5.7 Print out obtained for Carr's data.
102
5.8
107
Comparison table of the estimates
vii
LIST OF FIGURES
Page
3.1
A simple flow chart for constant coefficient of
variation model • • • •
4.1 Flow chart in unknown power case
.........
33
64
1. INTRODUCTION AND LITERATURE REVIEW
In many statistical analyses researchers may not be willing to
assume that their normal models have constant or homogeneous variances,
particularly when the observations are taken in groups at various
points of time under different conditions.
Therefore, it is necessary
to find alternatives to the classical assumption of homoscedasticity.
As an example consider a situation in which the response consists of
weights of human males.
a group of preschool
bo~s
The variance for the response values within
is, in fact, smaller than the variance for
the response in a group of adult males.
Similar situations can be
seen in laboratory data when measurements are taken under different
conditions, in economic data when measurements are taken over an
extended span of time, and in agricultural data when measurements are
taken under various biological conditions.
By introducing more appropriate models for the examples cited
above, it may be possible to obtain estimators which are superior
to those which are derived under the assumption of constant variance.
In some instances a more realistic assumption than constant
variance is that the variance of the response is proportional to some
power, known or unknown, of its mean.
This means that the variance
increases or decreases with increases in the mean.
With this observation the following three models are proposed
for investigation.
throughout.
We assume normal models having positive means
2
Model I:
We begin with a normal model whose error variance is propor-
tional to the square of its mean.
If
independent normal populations, the
2
cr.
J
= c 2~.2
J
, j=l, ••• ,k, are k means of k
.
var~ances
cr.2 are
J
.
g~ven
by
j=l, ••• ,k.
J
The constant of proportionality c
~.
2
of variation c, assumed unknown.
is the square of the coefficient.
This model is known as the constant
coefficient of variation model.
Specifically, we write the model as
y .. -NID(~.,c2~/) , i=l, ••• ,n. , j=l, ••. ,k.
J~
J
J
J
The parameters to be estimated are
Model II:
~l""'~k'
and c.
Here we consider a less restrictive model in which the
variance is proportional to some unknown power, A, of its mean.
That
is,
cr
2
j
= c
2 A
~j
, j=l, •• .,k
We write the model as
Y.. ,.....,NID(J.l.,c2~~) , i=l, ... ,n. , j=l, ... ,k.
J~
J
J
J
The parameters to estimated are
~l""'~k'
c, and A.
A can assume
both positive and negative values.
Model III:
problem.
Finally, we extend our investigation to the linear model
We assume that the variance of the response is proportional
to some unknown power of its expectation.
the form
This leads to a model of
3
I-J
Y. _ NID( X~8. , C2(X·~.Q)A)
J
!~
-:J-
-J-
, j =1, ••• , n,
= (Sl' ••• 'Sp)' where ~ , j=l, ••• ,n , are known, and Sl' ••• 'Sp are
such that E(Y.)
J
>
0, all j.
The assumption of constant coefficient of variation as in the
proposed model I is a valid assumption in many types of agricultural,
biological, and psychological experimentations because many times
the treatment that yields a larger mean also has a larger standard
deviation (Khan [21]).
In his theorem 2.1 Lohrding [22] obtains a
set of maximum likelihood estimators for the parameters PI' P2 and c
in model I, for a special case of k=2, and equal sample sizes, by
solving the three likelihood equations in three unknowns simultaneously.
Under this special case the expressions for the estimators are already
complicated.
Zeigler [32] examines certain types of estimators for
the coefficient of variation by means of simulation studies.
Sen and
Gerig [27] observe that for biological and plant populations, incEeases
in numbers are often proportional to the numbers already present,
consequently giving rise to a variance approximately given by
(J
2 = c 2].I 2
.
In their article they assume that c is accurately known
from a past experience.
In this case an estimator consisting of a
linear combination of the sample mean snd standard deviation is shown
to be more efficient than the sample mean alone.
Khan [21] considers
a similar form of estimator using two combined estimators of
].I.
The
asymptotic efficiency of his estimator relative to the maximum likelihood estimator is one.
Under the assumption of known c Azen and Reed [3] consider an
interesting case of a bivariate normal distribution, illustrating
4
the model by a clinical chemistry experiment.
They derive the
estimator of the correlation coefficient by the method of maximum
likelihood.
Asymptotic relative efficiencies of the ordinary estimators
relative to their estimators are also shown.
Literature on attempts to extend and examine the relationship
of the variance to the mean in the manner of mod el II are not know
to us.
Box and Hill [5] hypothesize a form of variance similar to
model II, and use it to generate a method of weighting for generalized
least squares estimation.
A form of relationship as in model III is probably first considered
by Williams [30], where he looks at a simple linear regression whose
response variance is proportional to the square of some function of
the expectation of the response.
The case where this function is the
expectation itself is studied by Amemiya [2].
other
thing~,
In this article, among
he examines the asymptotic relative efficiency of
weighted estimators with respect to the maximum likelihood estimators
of the regression coefficients.
The model in Box and Hill is similar to model III.
To assign
2 2-2ep , a nd n = -J~
x~.Q,
weight w. to the response y. were
h
Var (Y)
j
= cr n
j
j
J
J
2ep-2
they argue that w. == n .
• In order to obtain an estimate of W.,
J
J
J
they approximate nj by Yj' the predicted value of Yj based on the
ordinary least squares method, and derive an estimator for ep using
Bayesian approach.
The method of estimation employed in this report is the method
of maximum likelihood.
Obtaining these estimators involves solutions
of complicated nonlinear equations, and, in general, no closed-form
5
expressions for the estimates can be obtained.
designed for obtaining them.
So algorithms must be
In designing these algorithms the fact
that one model is less complicated than another can be taken into
consideration.
A rapidly convergent method is developed for solving the equations
in model 1.
In models II and III the (Fisher's) method of scoring'
is used for solving the equations.
For the first model we find some bounds for the estimate of c.
These serve as starting values for the solution algorithm which is
the method of bisection.
Because of scaling problems, in many
situations it may not be feasible to apply the method of scoring
directly in solving maximum likelihood equations obtained from models
II and III.
Some rescaling is necessary to increase computational
stability of the algorithms.
The form of the asymptotic variance-covariance matrix for the
estimators in the first model is easily obtained by algebraically
inverting the information matrix.
matrix is more complicated.
For the second model the information
It is possible to invert this matrix
algebraically but the result is very cumbersome and discourages us from
inverting the information matrix for model III.
Only a submatrix of
the information matrix is inverted to obtain a rough form of correlation
between the estimates of c and A in the last model.
This provides us
with enough information to develop a form of scale transformation
on the response, substantially reducing the correlation between estimates
of the two parameters.
possible.
"Independent" estimates of c and A become
6
We
obtain the estimate of the asymptotic variance-covariance
matrix of the estimators in model I by substituting the estimates of
the parameters in the asymptotic variance-covariance matrix.
for models II and III
However,
we obtain each of them by numerical inversion
of the information matrix evaluated at the estimates.
We have also developed an asymptotic chi-square test of hypotheses
of the forms H :
O
III.
c~~ =
y in models I and II and HO:
c~~
=!
in model
In model I we further examine the asymptotic relative efficiency
of the maximum likelihood estimators relative to the sample mean
estimators.
7
2.
SPECIFICATION AND THEORETICAL CONSIDERATIONS'
OF THE MODELS
2.1
Models Specifications and Basic Definitions
We recall the three models which we propose to investigate.
In
model I we assume that we have independent and identically distributed
(i.i.d.) normal random variables Y.. , i=l, ••• ,n., with mean ~. and
J~
J
J
variance c2~~, j=l, ••• ,k. Assume the probability density function
J
(p.d.f.) of Y.. is
J~
(2.1)
f. = f.(y;6) =
J
J
-
n(Y;~J.'
i=l, ••• ,n., j=l, ..• ,k, where 6' =
J
-
c
(~',c)
-
2 2
~J.)'
~'
and
n( ) denotes the normal probability model.
-
=
(~l'
..•
'~k)
and
We assume that 6' lies in
j=l, ••. ,k;
the parameter space for model I.
variables is R = {_oo
y
<
<
+
The sample space for the random
In model II we assume that the p.d.f.
oo}.
of Y •• is
J~
(2.2)
f. = f. (y;6) =
J
J
-
i=l, ••• ,n., j=l, •.• ,k, where 6' =
J
® II
= {(~' ,C,A)
10
< ~jl
.2 ~j
n(y;~.,
(~',
J
c, A).
< ~j2 <
c
2 A
~J.)'
We assume that 6' lies in
+ 00, j=l, ••. ,k;
8
the parameter space for model II.
variables is also R.
The sample space for the random
In model III we assume we have n independent
normal random variables YI, •.• ,Y , having expectations
n
2 'A
2, A
X"~~, ••• ,x:.ft., and variances c (x f3) , ••• , c (.!n~) , respectively, where
1
~' =
are known vectors.
(f3 I ,···,f3 p ) and -xI' ••• 'x
-n
We assume the p.d.f.
of Y. is
J
f.
(2.3)
J
=
f. (y; 8)
J
-
j=l, •.• ,n, where 8' =
(~',
@III = {(~',c,A)I~
< A
-
2
<
+
c, A).
We further assume that 8' lies in
B; 0< c l -:: c < c < +
2
E:
00,
-
00
<A
l
-:: A
oo}
< x~ B < n <
l--J2
where B = {BIO < n
+
00
'
j=l, ••. ,n}.
aD rII
is the
parameter space for model III.
rt is known that under proper regularity conditions the maximum
likelihood estimates possess certain desirable properties such as
consistency and asymptotic normality for a variety of probability
models.
We refer to the work of Bradley and Gart [6] regarding these
conditions in connection with the models which we have presented above.
The regularity conditions.
To apply the results of Bradley and
Cart [6] to a probability model it must satisfy the following three
regularity conditions:
(i)
For almost all y
E:
¥ and for all ~E:aD, assuming 8 is q-
dimensional,
a log f.
a 8r
J
a 2 log f.
J
a 8 ra 8 s
, and
exist for r, s, t=l, ••. ,q and for all j.
a3
.
log f.]
a 8 ra esa 8 t
9
(ii)
For almost all ye:R and for all !e:@, there exis't non-negative
integrable functions
a log f
h~(Y), h~s(Y)' h~~(Y),
a
j
log f
j
aea e
a er
r
a log f.
a log f
]
ae
2
< h
s
j
rs
and
and that Ee{h
j
rs
t(Y)}
<
such that
(y),
a3
log f j
aeaeae
r s t
j
a es
r
h~st(Y)
j
< h rst (Y),'
M., r, s, t=l, .•• ,q and for all j, where M. 's
]
]
are finite positive constants.
(iii)
For all !e:@, the matrix J = «J
[
k
a log f
j
J
(e) = j:l 0jE!
rs ae
r
a log
ae
(e»), where
rs -
f.1
] ,OJ
> 0
is positive definite with finite determinant.
for all j ,
In our case
o
j = nj/n (or lin).
To conform to the set up of [6] we must have k sets of random
variables of size n. which are mutually independent and identically
]
distributed, j=l, ..• ,k.
This is the case for models I and II.
By
using the method of "constant in repeated samples" approach of [28],
model III can be seen to satisfy this condition.
In fact, we assume
that the n observations are composed of r sets of k observations each
(n = rk).
The (kxp) X matrix is identical for each set.
"X" matrix for all n = rk observations is
X
X
first set of k observations
second set of k observations
X
rth set of k observations.
That is, the
10
If we represent the jth observation of the ith batch by Y
then
ji
Thus, with r = n., j=l, ••. ,k, we find that, in this set up, model III
J
is equivalent to model II.
We observe that Y.l, ••• ,Y.
J
Jr
are i.i.d.
random variables, j=l, ••• ,k.
The following terms are frequently used in this report, and it
may be useful to define them formally.
LetQD denote an arbitrary q-
dimensional parameter space of a model whose log-likelihood function
is
Q, (~) •
Definition 1.
The equations a Q, (~) /a.Q. = Q. are called the likelihood
equations.
Under the regularity conditions a maximum likelihood estimate
(m.l.e.), . denoted by.Q., or .Q.(y), can be obtained as a solution to the
likelihood equations.
Definition 2.
If 8'
aQ, (8)/a 8
-
q
is called the efficient score vector.
Definition 3.
j
with typical element
a Q,
<..~)
a 8s
'
11
r, s=l, ••• ,q, is called the information matrix.
Definition 4.
The symmetric matrix
Hrs (!) = (:
H<,~)
with typical element
::a(::) e
r, s=l, ••• ,q, is called the Hessian matrix.
Throughout this report! or !(Z) will be used to denote either
the value of the estimator or the estimator itself depending on the
contexts.
2.2
Verification of Regularity Conditions for Model I
For notational convenience in verifications of conditions (i) and
(ii) we denote f. by f and
J
f = n(y;
in (2.1).
~,
c
2 2
~
~.
by~.
J
Hence we have for model I,
) where the parameter space conforms to that specified
We show detailed verifications regarding the first and
second partial derivatives only.
For the third order we demonstrate
by a typical example.
The p.d.f. for this model is
The partial 8-derivatives of log f up to third order exist, are finite,
and are continuous in~.
(2.4)
a
log f
a~
(2.5)
a
(2.6)
a2
These derivatives are
1
1
= - - + - - (y 2 2
~
c ~
~)
1
+2"3 (y -
112
log f
- (y - ~)
3 2
a c = - -c + c ~
log f
1
= -2
2
a~
~
-21-2c
~
c
~)
2
,
~
,
4
- - (y 2 3
c ~
~)
3
- 24"
c
~
2
(y - ~) ,
lZ
aZ
(Z.7)
log f
Z
=
ac
aZ
(Z.8)
log f
1
-Z
-
~
3
c
c
Z
a ca ~
= - 64
c
(y -~)
Z
,and
~
(y - ~) -
~
Z
c
Z
(y - ~)
33
•
~
~.
All partial derivatives taken with respect to
1
for
~~ j
are
zero.
We observe that all first and second partial a-derivatives, and
m
·
1
·
h
·
f
L aiy i ,
cross pro ducts are po 1ynom1a s 1n y, aV1ng a orm, say, i=O
with finite m.
But
m
L
Ii=O
(Z.9)
Since 0 <
Ea(IYI~)
a.yil <
1
< <Xl,
i E~r '
m
for finite
m
i
I a.11
L
) = i=l
E a (i~O la.IIYl
1
is
*.
finite~
it follows that
~,
E
a
(I YI i)
m
i.e.,
i~O
I ail I yli is integrable.
h~(Y), h~s(Y)'
Thus,
m
and
1.
hr~(Y) may be chosen to be of the form i~O I aill y1 •
We now look at the third order partial derivatives of log f.
iEer.
These derivatives exist and are finite for all
For example,
from equation (Z.8)
a 3 log
a cZ a ~
f
lZ
=~
c
(y - ~)
+
6
~
c
~
(
y -~)
Z
•
~
This derivative is dominated by an integrable function having an
expectation which is bounded by a positive number independent of the
parameters.
For, let
.
hizz(Y) =
lZ
74 I Y
c
~
-
6
I
~I + ~ Y - ~ I
c
~
Z
13
then
6
+4"3
c ].I
~
12
---:r4
+
12
---:r4
f
c ].I
C].l
(y _ ].I) 2dF (y)
6
(y - ].I) 2dF(y) + 4"3
c ].I
=
2
; 4 ( 2 + 2c ].12 + c S].I3)
c ].I
=
M.,
J
J
.
(y - ].I) 2dF(y)
for all !e: ~'r' where Mj is independent of !.
The forms of other third partial derivatives are similar to the
one in the above example and we can find, corresponding to each of
them, constant M. in a simiiar manner.
J
We have just verified that model I satisfies conditions (i) and
(ii) of the regularity conditions.
(iii).
Next we shall verify condition
We need to show that the matrix J = «J
positive definite with finite determinant.
•
T; =
d
rs
(!)))'
log f.
dS
r
J , j=l, ••• ,k, r=l, ••. ,k+l,
Then J
(S) can be
rs -
written as
k
( S) =
rs -
is
Let
where Sr = ].Ir' when r=l, ••• ,k, and Sk+l = c.
J
!e:~,
E
j=l
<5
j
E [ Tj . Tj ]
S
r
s
<5. > O.
J
14
Under conditions (i) and (ii) we have (see for e.g. [29J)
a
(2.10)
Ee
Then
J
(
k
log f.
ae J)
= 0,
r=l, ••• ,k+l, for all !E~.
r
= J=
. l: 1 c.J
where ~ = (Ti, ••• ,T~+l)"
and Var( ) is understood to denote the
variance-covariance matrix.
Thus J is non-negative definite.
J is
positive definite if there does not exist a non-zero vector z =
But, since
j
c.
> 0 and Var(I ) is
J
non-negative definite,
k
z'J z = .l:l
J=
c.J
j
_z'Var(T ) z = 0
if, and only if, ~'Var(Tj) ~ = 0 for all j.
For a given j we observe
j
that the only non-zero elements of the (symmetric) matrix Var(T ) are
E [T3 • T3] , E![T3 •
i
T~+l]
, and
Ee[T~+l
•
T~+l]'
that is, only the
(j,j), (j,k+l, (k+l,j), and (k+l,k+l) elements are non-zero.
The 2x2
submatrix consisting of these elements are positive definite, so that
j
for z'Var(T ) z to be zero z must equal(zl'···'z. 1,0,z.+1, •.• ,zk'0).
-
-
-
-
This is true for all j=l, ••• ,k.
is that z=O.
J-
Hence the only
~
J
which satisfies these
Therefore, J is positive definite in model 1.
We conclude that model I satisfies the regularity conditions.
2.3
Asymptotic Properties of the Proposed Estimators
for Model I
The following three theorems of Bradley and Gart [6] establish
some important properties of the m.l. estimators in model I.
follow from the regularity conditions given in part 2.2.
They
15
Theorem 1.
Let 8 represent a solution to the likelihood "equation and
0
represent the true value of 8. Then as n.
J
k
0
n = I: n , 8 is a consistent estimator of 8
j=l j
!
Theorem 2.
o.
-+- co.
J
= n.ln constant,
J
•
Of all possible solutions to the likelihood equation one
and only one is consistent.
Theorem 3.
0
~
If 8 is the vector of m.l. estimators and 8
of true values, then
rn (8 - !o)
has asymptotically as n
is the vector
-+- co,
n
j
= 0jn,
the multivariate normal distribution with zero means and variancecovariance matrix J-l, where J
o
is the J matrix of condition (iii)
0
o
evaluated at 8 •
We see at once from theorem 3 that the asymptotic variancecovariance matrix of 8 can be obtained as follows.
~
1
~-l
0
Var~ (_8 - 8 )) = J- implies that Var(8) = (nJ) .
-
0
-
= I(!),
show immediately below that nJ
But we shall
0
and hence nJ
o
information matrix evaluated at !o, and Yare!) = (I(!o))-l.
Another useful result when the regularity conditions hold is
that, (see [29]),
a
E
(2.11)
8
(
J
a 8r
r, s=l, ••• ,k+l, for all !E~I'
likelihood,
.Q,
log f.
•
a 8s
J)
+
E
8
a 2 log
( a8 a8
(!) for model I is
.Q,(!) =
r
s
Hence, recalling that the log-
k
we have
a
log f.
I:
n.
J
I:
j=l i=l
log f. (yj i; !) ,
J
16
E
(2.12)
e
2
(~ i
d
(&.»
e ae
r
log f. (y .. ;
J 1.
J
s
log f. (y .. ;
J
J1.
~»
e»
-
= -
a
log f. (y .. ; e)
J
ae
Since Ee (
]1.
-
a
log f. (y . 'i' ; e)
~ eJ
- ) = 0 if jfj' or ifi',
s
r
on interchanging rand Ee, we have, on adding terms with ifi',
E
e
(~
'a
2
i
(_»e
a
ae s
L
. log f.(y .. ; e)
= -
er a es
1.
J
J1.
-
L
•
1.
log f. (y .. ;
J
J 1.
!»
by the linearity of partial derivatives under summation,
= - Ee
aLL
(as
J' i
r
by adding terms with j
+j',
a
log f. (y .. ; !) ae
J
J1.
S
L L
. . log f. (y .. ;
J
J1.
J 1.
!»
m
= -
That is, the information matrix, under the above conditions, is also
given by
I
As
(e)
rs -
2
er a es ),
e (- a i(e)!d
-
= E
we denote the Hess ian matrix (Cd 2i
(2.13)
r,s=l, .•. ,k+l.
(e)
/a er a es »
-
by H(_e), then we have
17
From (2.12), since for model I Y.l, •.• ,Y.
J
are i.i;d. random
Jn j
variables, we may put, conforming to notation for condition (iii),
k
a
=nj~lojE!(
(2.14)
log f.
a8 J .
a
log f.
a8 J).
s
r
Except for n the right hand side is J
(8) of condition (iii).
Thus
rs -
which is exactly what we have promised to show.
This also means that
the information matrix, I(!), is positive definite for all !EQUI.
Denote the asymptotic variance-covariance matrix of !, sometimes
abbreviated as Asy var-cov(!), by V.V has typical element I
rs
(!) which
is estimated by either the (r,s)th element of the inverse of I(!) or
the (r,s)th element of the inverse of
I(~,
then evaluated at 8 = 8.
In either case the estimator of V is consistent.
Definition 5.
The asymptotic relative efficiency (A.R.E.) of an
estimator 8 relative to another estimator 8* is given by
ARE(~, !*) = IAsy Var-cov(!*) I/!Asy Var-cov(!)
where
I· I
denotes the determinant of a matrix.
The correlation coefficient between 8
Prs
=
r
(see [2]).
and 8
s
r,s=l, ••• ,q.
is given by
I
18
2.4
Extension of Results to Models II and III
The first, second, and third partial e-derivatives of the log of
the density functions in models II and III can be shown to be of the
polynomial form given in (2.9).
they were in part 2.2.
They can be dominated in the same way
Hence all the results for verifications of the
regularity conditions obtained above hold also for models II and III.
The asymptotic results for model II follow with no change in principle.
However, for model III we have to use the principle of "constant in
repeated samples" as we have specified in part 2.1.
Then all the
asymptotic results follow.
We have shown that our models satisfy the regularity conditions
under which we establish (i)
I(~)
= Ee(-H
(~»,
simplifying the method
of finding the information matrix a great deal, (ii)
I(~)
is positive
definite for ~EQDI,auII orGUIII , a property which plays an important
role in curvature determination during iterations (see and cf:
Marquart's Method [4]), and (iii) consistency and asymptotic normality
of the m.l. estimators.
Detailed discussions for models I, II, and III are dealt
separately in sections 3, 4, and 5.
In section 3 the iterative method
used for solving the likelihood equations is based on the method of
besection (see for e.g. [19]).
In sections 4 and 5 the iteration
method is the method of scoring (see for e.g. [25], [16], and [18]).
19
3.
THE CONSTANT COEFFICIENT OF VARIATION MODEL'
An alternative to the classical assumption of constant variance
model which has been proposed in some articles is the assumption of
constant and known coefficient of variation model.
As proposed earlier
we consider a more general alternative model without the assumption that
the constant coefficient is known.
The solutions to the likelihood equations are fairly complicated
and therefore it is essential that an efficient numerical method be
developed.
Basically, the method involves some simplications of the
k+l equations into a single equation of the form F(x)=x.
Numerical
solution to this equation is obtained using the method of bisection.
Some numerical examples are presented to illustrate the method.
3.1
The Likelihood Equations
We consider k independent normal populations with
2
NID ( II j' cr j) ,
Yj i -
i=l, •.. ,n.,j=l, .•. ,k, where it is assumed that the coefficients of
J
variation remain constant, i.e.,
c
222
=cr.lll.
J
J
,j=l, ..• k.
Thus, the model is, in fact, of the form
Y.. J 1.
i=l, •.• ,nj,j=l, ... ,k.
2 2
NID (ll., c ll.),
J
J
The parameter space'I is given by (2.1), namely,
20
For k random samples Y.l, .•• ,Y.
of size n , j=l, •.. ,k, from the
j
J
Jn j
above model, the likelihood function L(e), ~ E'I' is given by
.k..
L(~) =
n.
.
-~
-1
2 2 -1
2'
j=l i=l (2rr) (c~j)
exp {- ~ (c ~j) (Y ji - ~j) }
IT ITJ
k
= (2rr)-~ j~lnj
Writing
~
~
k
TT
j= 1
()-n
c~.
J
j
k
~
·exp {
- l~ J=
'[1 ~=
'[1 ( c 2~J'2)-1
for log L(e), the log-likelihood is given by
k
= const- '[1 {no log
J=
J
~J'
+ n. log c +
J
(2c2~:)-1
J
Equating the first partial derivatives w.r.t.
~l'
.••
n.
.[J (y .; _
l
J ...
~=
'~k'
and c to zero
gives the following likelihood equations:
o
j=l, ••. ,k; or
(3.1)
n.
"2" 2 -1 " J
where
~.)
~.
"
C.[l y .. - n.~.)} = 0, j=l, ..• ,k.
J
J ~=
J~
J J
"2"2
Dividing the above equation by n. and multiplying by 2c ~. we get
J
J
"2
"2"
"2
(3.2)
(c + l)~j - ~jYj - 'j = 0 , j=l, .•• ,k,
+2(2c
~.)2}.
J
21
n.
2
and
'P.
J
-1
= n.J
J
~
2
'~l (y .. - ll.) •
1.=
J 1.
J
Similarly, corresponding to c we have
n.
J
~
2
A2 A2 -2 A 2
+ {- (2c ll.) (4Cll.) i~l (Y - \.lj) }] = 0-,
ji
J
J
k
A
[(n./c)
·~l
J=
J
or
(3.3)
~2
c
=n
-1
2 A2
k
.~l n.'P./~.,
J=
J J
J
k
where
.~l
J=
n.
J
= n.
From equation (3.2),
and equation (3.3) reduces to
k
jgl (n/ ].Ij)Yj
=
~2
-1 k
c + 1 - n
.~l (n./ll.)Y ..
J=
J
J
J
Hence
~
k
.~l (n./].I.)Y.
J=
J J J
(3.4)
=n
Equation (3.2) can be reduced to
A_
2 -2
c ].I. + 1l.Y. - (s. + y.) = 0
~2A2
(3.5)
J
J J
n.
where
J
J
J
- 2
2
-1
s. = n. 1.=
(y j i - Yj) , as follows:
·~l
J
J
22
n.
J
" 2
-1
= n. igl CY
- llj)
ji
J
J
n.
n.
J 2
-1 "
-1
= n. igl Y
2n.
llj 1.=
ji
J
J
'¥~
.i\ Yji + llj"2
n.
= n
-1
j
J 2
"2
igl Yji - 211 j Yj + llj
n.
J 2
-2
-1
= {n
igl
Yji - Yj } + y.
j
J
- 211 j Yj
"2
+ ll.
J
Then, substituting '¥~ in equation C3.l) leads to the above equation.
The equations to be solved are
(3.5)*
0, j=l, ••• k; and
k
(3.4)*
jgl
3.2
Cnj/~j)
Yj = n.
Solutions for a Special Case of k=2
We first look at the special case when there are only two groups
involved, i.e., k=2.
Equations (3.4) then become
(3.6)
0,
(3.7)
and equation (3.4) becomes
(3.8)
a .
23
The last equation gives
(3.9)
So equation (3.7) becomes
(3.10)
2 -2
- (s2 + Y2)
From equation (3.6) we have
and so equation (3.10) now becomes
(3.11)
Simplifying this we have,
or
(3.12)
o.
Z4
This is a quadratic equation in
~1.
This gives two solutions for
namely, by
[-b + l(b
Z
- 4ac) ]/Za,
with
4ac
Za
= 2n
-Z
Z Z
(n - n Z) yz + 2n s2
Then
0.13)
Observe here that real solutions to equation (3.1Z) may not exist,
unless
>
a , i.e.,
~1'
25
(3.14)
To see this, suppose that n = 6, n
1
(3.14), and Yl
= 2,n 2 = 4,
a case violating relation
= 5, Y2 = sl = s2 = 1, then
b
2
- 4ac
=
(4)(6)(2)(-36)
so that no real solutions exist in this case.
<
O.
Similarly, for
~2
to
have real solutions we must have
(3.15)
As a consequence of restrictions (3.14) and (3.15) we have the following.
Let r i
r
l
= ni/n,
+ rl - 1
.382
~
r
2
~
~
i
= 1,2.
Then these imply ri - 3r
0, that is .382
.618.
~
r
l
= nl +
.618.
+ 1 ~ 0 and
By symmetry,
This means that the restrictions are satisfied if
n l and n Z are chosen so that .382n
where n
~
l
~
n
l
~
.6l8n and .382n
~
n
2
~
.6l8n
n .
2
However, if the model is correct then the term in (3.13) containing
2 -Z
2 -Z
(sl/Yl - sZ/Y ) will be small and, hence, the likelihood of (3.13) being
Z
negative will be small.
In any case, if (3.14) and (3.15) hold then
(3.13) is always positive.
The solutions for
~l
and
(Znn
(3.16)
l
~2
are
2
-z
- n 2 - nln ) Y +
Z
Z
26
2 -2 2
2
2-2-2
2-2 2
2 2 2 2 ~
n 2) y l s2 + (n 2 + n l n 2 - 2nn l ) YlY2 + 4nn l n 2y 2s l + 4n n 2s l s 2} ~
-2
2 2
2nn l Yl + 2n s2
and, by symmetry,
-2
2
2
(2nn 2 - n l - n n 2) Y + 2nn 2s )
l
l
l
0.17)
-2
2 2
2nn 2Yl + 2n sl
2
2 -2 2
2
2-2-2
2 2-2 2
2 2 2 2 ~
y l {4nn 2 (nn 2 - n l ) y 2s + (n l + n l n 2 - 2nn 2) Yl Y2 + 4nln2Yls2 + 4n n l s l s 2 } .
l
+--=----==----=~--=----==-=;,..-.--=----=:.-::_---==-----=--=---=--=-=-=------=--=-=-2 2
2-2
2nn Y + 2n sl
2 l
Also
2 -2
c = [(sl + Y1) -
0.18)
2
-2
= [(s2+ Y2)
Suppose n
i.e., n
l
l
=
n
2
=
~lYl]/~l ,
h
-
f.l
or
h
2Y2 ]Jf.l 2
J, then n = 2J, and
~ n~/n and n 2 ~ ni/n in both cases, and thus solutions f.l l ,
f.l
2
,
and c always exist and are given by, cf: [22],
0.19)
0.20)
(3.21)
f.l
2
= ~2
222
2 ~
+ ~1{(Y2 + 2s2)/(Yl + 2s l )} ,and
2 -2
[(sl + Yl ) - [~l
c =
-
~l
-
-2
2
2 -2
2 ~ ~
+ 2s l )/(Y 2 + 2s2)} ]]
-2
2
+ ~2 {(Y l + 2s l )/(Y 2 + 2s2)}
Consider the solutions for
then
-2
± ~2{(Yl
f.l
l
, and take c - sl2
=
~
/ -2
Y
l
,
say.
27
~l
== (y
=
~l.
2c}~
(y/2) + (y/2) ,
~l == 0 or y.
i . e.,
of
/2
l /2) + (Y2 ) (y l /Y2) {(I + 2c)/(1 +
But ~l = 0
i
8r ,
that is, not an admissible value
Hence it is reasonable to choose solutions for
the plus sign in front of the radicals.
~l
and
Expressions (3.19),
~2
with
(3.20)~
and (3.21) above change accordingly.
3.3
Solutions for the General k Case
For k > 2 the likelihood equations (3.4)* and (3.5)* must be
solved numerically.
2
= Sj
-2
-2
I Yj' then on dividing equations (3.5) by Yj , we have
(3.22)
This is a quadratic equation in
assumption
~.
J
(~./Y.),
J
J
for each j, and under the
> 0, and assuming y. > 0, we consider only the positive
J
solutions to the equations, namely
(3.23)
2
since 4c(t. + 1) > O.
A
J
Substituting this quantity in equation (3.4),
we have
(3.24)
~2
= (n/2)/
.~l
J=
n./[-l + {I +
J
We now have one equation in one unknown c.
the form
c
A2
= F(c
),
4~2(t~J + l)}~].
This equation is of
28
A2
whose solution is a zero of fCc )
is differentiable to any order.
A2
= F(c
A2
) - c , where F, 'and hence f,
We note specifically that the value
A2
2
~
{l + 4c (t + l)} is taken to be positive.
j
To determine if the method of bisection can be applied to solve
equation (3.24) it is sufficient to show that the function f above
A2
satisfies ft(x) < 0 at its zero. Denote c by x, and let
(3.25)
= -1
aj(x)
+ {l + 4x
(t~ + l)}~
(We use prime notation to denote differentiation only for a.(x) and
J
F(x) that follows).
Then
a~(x) =
(3.26)
d(a.(x»/dx
J
J
~ {l + 4x(t~ + l)}-~ .4(t~ + 1)
=
J
= 2(t~ +
J
J
1)/ {l + 4x(t
2
+ l)}
and the function F(x) reduces to, writing a. for a.(x),
J
J
(3.27)
k
F(x) = ~{.Ll p.a. -1 }-1 , where Pj = n./n.
J=
J J
J
(3.28)
F' (x) =
2
t.
J
+ 1
-~{
k
-1 -2 k
-2
.L p.a. } {.L p. (-l)a.
a. '}
J= l J J
J= l J
J
J
J
J
(3.29)
Since
{(a. + 1)2 -1}/4x, by equation (3,25),
= a.
{l + 4x
Thus
(a. + 2)/4x, and
J
(t~ + l)}~ = a. + 1, then equation (3.26) gives
J
a.'
J
J
=
2a. (a. + 2)/4x (a. + 1).
J
J
J
29
Hence.
k
-1 -2
k
-2
F'(x) = {j~lPjaj}
{j~lPjaj
aj(a j + 2)/4x(aj + 1)} .
(3.30)
k
When F(x)
= x,
i.e.
1 {
~
'~1
J=
-1
p.a.
J J
(x)
}-1 _
- x, then
k
(3.31)
= x.{'~lPj(a.
J=
J
F'(x)
k
+ 2)/a.(a. + 1)}
J
-1 -1
J
k
-1
. [jE 1 p.a. (a. + 2)/(a. + 1)]
=
J J
J
J
= ~{.E1Pja.}
J=
J
k
=
-1
where q. = Pja
j
J
of
~(a.
J
jE1qj .~(aj + 2)/(a. + 1),
J
k
-1
/ j~lPjaj .
+ 2)/(a. + 1).
J
~(a.
J
That is, F'(x) is the weighted average
But, since a. > 0,
J
+ 2)/(a. + 1)
J
Thus, at the point x where F(x)
(3.32)
< (a.
J
=x
+ 2)/(a. + 2)
J
= 1.
is satisfied, F'(x) < 1, or
fl(x) <
o.
Therefore the bisection method can be applied for solving equation
(3.24) .
To apply the method of bisection we must construct some bounds
for c.
We first show that
(3.33)
c
2
u
=
k
- 2
(l/n) .E n.(s./y.)
J= 1 J J J
If we visualize the expression
k
and .E P.a., p.
J= 1 J J
J
= n./n,
J
30
as mean of a , respectively denoted by HM(x) and E(x), being functions
j
of x, then by the relationship HM(x) 2 E(x), we have
(3.34)
~2
k
= ~(j~l
=
c
-1 -1
p.a. )
J J
<
k
- ~ '~1 p.a.
J=
J J
k
= ~ '~1
J=
p. [-1
J
+ {1 + 4~2 (t: + 1)}~]
J
4c~2
+ I)} ~ ]
=~
k p. {1 +
[-1 + '~1
=~
k
~ ~
A2
2
~
[-1 + '~1 p. p. {1 + 4c (t. + 1)} ]
_< ~
k p. '~1
k p. {l + 4c~2 (t 2. +
[-1 + ['~1
J=
J
J=
J
J=
(t 2
j
J
J
J J=
·
l'~ty j~ x Y 2 /( j~ x 2
bY t h e ~nequa
j
j j
J
J
j~ Y2)
j
(C auch Y- S chwarz ) .
Then
(3.35)
~2
~2
k
(2~2
+ 1)2
_<
1 +
1
(t. + 1)}]~
J
= ~ [-1 + {1 +
(3.36)
2
c _< ~ [-1 + ['~1 p. {I + 4c
J=
J
A2
k
2
1
4c ('~1 p.t. + 1) }~] ,
J=
J J
4~2 ('~1
J=
P.t: + 1).
J J
But
Hence we have
~4
4c
(3.37)
A2
+ 4c + 1
c
2
~2
< c
-
1 + 4c
<
2
u
1)}]~],
c
2
u
~2
+ 4c , or
31
A2
Thus c
A2
is an upper bound for c.
u
We make an observation here that if
a given data fits the model and if n., j=l, ••• ,k, are sufficiently
2 -2
A2
large, then s./y. is close to c •
J
J
J
This implies that c 2 is close to
u
Next is to establish a lower bound.
we assume that t
1
2
t2
2 ... 2
Without loss of generality,
t , where tIs are as before.
k
By the
inequality
min (x.) < ~ w.x. < max (x. )
J
j
- J
J J
J
j
where w. is some weight, we have
J
(3.38)
-1 -1
p.a. )
J J
(.L
J= 1
~ ~ a.
~
since a
j
= -1 + {1 +
4~2
(t; +
1)}~
is also such that a
1
2 a 2 2 ...
<
Then
(3.39)
hence t
(3.40)
2
1
A2
is a lower bound for c.
c
2
.t
=
In practice, however we find that
k
k
2
In.t./.L /n.)
J= 1
J J J= 1 J
(.L
provides a more efficient lower starting point for the search however
A2
we cannot prove that it is a lower bound for c.
suggests that it is always true that
Our numerical work
a .
k
32
(3.41)
c
2
Since t l
~
2
cJ..' then
2
A2
< c •
1..2
c~
A2
2
3.4
2
than t l , i.e., min (t .) .
J
j
is closer to c
Computing Algorithm
We construct an algorithm for computing the estimates based on
the method of bisection, with the two bounds given by (3.33) and
(3.40).
The convergence in this case is very rapid.
for convergence is:
F(x)/x = 1.0 + 10
chart for obtaining c, and then
-16
.
~l""'~k'
The criterion
We present a simple flow
using equations (3.23),
the asymptotic variance-covariance matrix for the estimates, and also
an asymptotic chi-squared test for testing H : C' ~ = Y.
o
-
These asymptotic
results are discussed in the next part.
In the chart below 'NITLIM' is the number of iterations allowed.
It is used for stopping the computations when the convergence criterion
is not satisfied in some specified number of iterations.
'NIT' is the
current iteration number.
Some numerical results are shown at the end of this section.
The
properties of the estimates are considered next.
3.5
Asymptotic Properties of Estimates
To enable us to examine some properties of the estimates we first
obtain the variance-covariance matrix for these.
form of the matrix can be calculated.
Only the asymptotic
This is obtained from the
information matrix whose elements are the expectations of the second
partial derivatives of the
log-likelihood,~.
The second partial
Q
______ L
_
READ NITLIM,k,n
y·i,j=l, ••• k;
J .or-I
- ,... n ,•
______'v..
_
COMPUTE y. , s?,
and t?=s?/y?J J
NIT=oJ J J
NIT = NIT + 1
c. = ~(c2+c2) i=NIT
~
u'
R,
COMPUTE F(c~), and
~
criterion p, where
2
2
p = F(c.)/c.
~
~
yes
no
--- - --
[-- c--=c.- - u
~
1
Figure 3.1.
A simple flow chart for constant coefficient of variation
model
33
34
Figure 3.1 (Cont.)
----------------------- ----]
r--- _:j- :~-~;l~:'::~~j- ~~~/~~n_
A
"2
_
2
"2
+-
=-
----------------------- ----J
[
-- - - - - - - - - - - - - - ]
---------r
COMPUTE
......
. Asy var-cov(H), Asy var-cov (.H."
,..
ma..1= "t.i~
--
- - - - - - - - - -
-
-
[---------1---------READ DIM OF TEST MATRIX C',
READ C', and 'i in H: C'll = Y
2
.....
c)
COMPUTE Asy X and
prob of rejection of H
35
derivatives of i w.r.t. the parameters are given below in (3.42),
(3.44), (3.46), and (3.48).
Their expectations are given in (3.43),
(3.45), (3.47), and (3.49).
(3.42)
Taking expectation we have
(3.43)
E
2
e{'} = -(2c +
1) n./c
J
2 2
~.
J
where {.} denotes the expression immediately above.
(3.44)
a2 i/a~.a~
=
J m
i f m#j, and
0
if m#j,
(3.45)
m, j=l, •.• ,k.
(3.46)
a2 i/a~.ac
(3.47)
Ee{·}
J
_
= -2n./c~
..
J
J
2
k
L
2
= -.
[ -n . I c
J=l
J
(3.48)
a'll ac
2
(3.49)
Ee{·}
= -2n/c 2 .
By the results of section 2 the asymptotic variance-covariance
matrix of
(3.50)
~l""'~k
[
and c is given by
Asy var-cov (~l""'~k'
C~)]-l
36
(2c
2
2 2
+ 1)n /c lJ
1
1
2n/c~1
0
0
=
(2c
2
2 2
lJ
k
+l)~/c
2n/c 2
2~/c~
2n /c lJ
1
1
2~/c~
Denote this matrix by a block matrix
k1\
u
v'
a
2
2 2
+ l)n./c lJ., j=l, •.• ,k).
where
A = diag ( (2c
and
y'= u ' = (2n1/clJ1 , ..• ,2~/ClJk)'
2
a = 2n/c • Using the result of ex. 1.58 of [24],
]
J
we have
-1
A
u
B
P
..9.,'
b
=
v'
where
B
b
and
= a
a
-1 + bA- 1 u v' -1
A
(a
-
v' A-1
-1
.E.= -b A u ,
-1
..9.,'= -b v' A
~)
-1
.
In our case
-1
2 2
2
2 2
2
A = diag (c lJ1/n1 (2c + l), ••• ,c lJk/~(2c + 1) ),
v' A- 1 u = 4n/(2c 2 + 1), and
37
i f j=m,
if j,&m,
j, m=l, ••• ,k.
Also
Thus, Asy var-cov
(~l'.'.'~k'
c)
n(2c
2
+ 1)
n(2c
2
+ 1)
n
- c
n(2c
2
2
~2
n
+ 1)
.........
2~2~kc
n(2c
- c
3
2
~
+ 1)
1
n(2c
- c
3
2
2
2
c ~k(l + 2~c In)
2
~(2c + 1)
4
+ 1)
- c
~2
n
n
3
~
k
c
3
~2
n
2
2
c (2c + 1)
n
2n
e =-e0
-
This is estimated by the matrix whose elements are correspondingly
replaced by the estimates
element is of D(n- 1 ).
~l'
..
"~k'
and c.
We observe that each
38
Asymptotic relative efficiency (A.R.E.).
the A.R.E. of
ARE (l!., y) =
(3.52)
I. I
where
(~l""'~k)~
As defined in Section 2
(Yl""'Yk)~
relative to
is given by
Asy var-cov(y) I Asy var-cov (l!.)
denotes the determinant.
Since
Asy var-cov (y) = Var-cov (y)
then
IVar-cov
(y)
I = c 2k
TT
~ ~ In ..
j=l J J
Asy var-cov (l!.) is obtained from
matrix (3.21) deleting its last row and its last column.
Its
determinant is then given by
k
(3.53)
IAsy var-cov
(~) I
= c
2k
/(2c
2
+ l)k
k
1T
~:1T (2c 2 + l)/n.
j=l
j=l
J
J
k
= {c
2k
/(2c
2
+ 1)k-1}]lr ~7/n ..
j=l J
J
Hence
k
(c 2k
(3.54)
k
2
IT ~/nj) I (c
2k
j=l
2
I (2c + 1)
k-1
~
II
2
~. In.
j=l J
J
= (2c 2 + 1)k-1
Since (2c
2
+ l)k-l
> 1, the estimates
efficient than Yl""'Yk'
The efficiency increases as c increases.
The A.R.E. of a single estimator
mean y., is
J
~l""'~k is asymptotically more
~.
J
relative to the group sample
39
(3.55)
2
2 2
2
2
= (c 2 ~./n.)/(c
~. (1 + 2n.c In)/n.(2c + 1»
J
J
J
J
J
2
= (2c + 1)/{2 (n./n)c 2 + I}.
J
(o~
This is also dependent on the relative sample size of each group
stratum).
The larger the relative size the more efficient
~j
A.R.E. is always greater than unity since in general (n./n)
J
Test of hypothesis.
<
will be.
1.
Suppose we denote Asy var-cov (.H.) by ~ ,_ then
~
by the result of Section 2, for testing a hypothesis of the form
(3.56 )
H
o
C'.H.
= 1..
a test statistic for H can be obtained from the consideration that
o
c' .H.
AN
c
(C'.H.,
c't
~
C), where rank (C) is c, and
t~ = Asy var-cov (.H.) is consistent.
Then
(3.57)
L
n
n Asy
3.6
2
X wi th cd. f. .
Numerical Examples and Application
Appendix 8.1 is a Fortran program based on the previous flow
chart and is used to obtain numerical results for real data.
simulated data slight modification is necessary.
For
In this study we use
the random normal deviate generator GGNOR of IMSL to obtain the
'responses'.
Table 3.1 is a summary of results for some simulations
40
studied for this model.
real data.
Table 3.3 is the complete print out for the
This data is taken from [3], p. 462, which is on absorbance
values of 3 substances analyzed in 19 runs of a laboratory test for
serum concentration level of enzyme, leucine amino peptidase.
reproduce the data in Table 3.2.
We
Abbreviations used in the print out
are explained below:
2
J
XBAR:
s. = (lIn.)
S2/XBAR2:
2
c , CVSQ:
ML SOLN FOR CV:
J
l.l
2
c ,
maximum likelihood solution for the coefficient
of variation,
k
CHECKING SUM:
checking equation (3.4),
~
n.y./l.l.
j=l J J
LN:
asymptotic chi-square value, DF:
n.
J
degrees of freedom.
41
Table 3.1.
Parameters
Summary of results for simulations (constant ·c.v.)
True
Y + (std. dev.)
j
8 + (asy std. dev.)
j
.20
•19 70 + .0101
.2019 + .0083
.30
.3051 + .0127
.2980 + .0123
.35
.3478 + .0160
.3469 + .0143
.40
.3642 + .0178
.3689 + .0152
.60
.6161 + .0275
.6102 + .0251
.80
.7889 + .0348
. 7787 + .0321
.85
.8297 + .0405
.8397+ .0346
.4001 + .0142
.40
sample sizes:
n. = 75, j=1, ..• ,7; no. of iter:
J
9
12.00
11. 922 + .108
11. 934 + .102
15.00
15.048 + .128
15.043 + .129
17.00
16 • 860 + . 144
16.856 + .144
18.00
17.914 + .157
17.918 + .153
22.00
22.384 + .195
22.382 + .192
21.00
20.770 + .176
20.760 + .178
25.00
24.829 + .215
24.828 + .213
.099 + .002
.10
sample sizes:
n. = 130, j=1, .•. ,7; no. of iter:
J
3
1122.00
1107.47 + 20.28
1111. 80 + 18.65
1155.00
1162.36 + 19.73
1160.35 + 19.47
42
Table 3.1.
Continued.
Parameters
True
e. + (asy std. dev.)
Y + (std. dev.)
j
J
113
1270.00
1249.12 + 21. 58
1248.51 + 20.95
114
1890.00
1871. 96 + 33.04
1873.97 + 31.44
115
2222.00
2299.54 + 39.08
2295.77 + 38.52
116
2190.00
2142.05 + 36.61
2139.41 + 35.90
117
2590.00
2554.66 + 44.51
2554.99 + 42.87
.1975 + .0048
.20
C
sample sizes:
n. = 130, j=l, ... , 7; no. of iter:
J
4
Other parameters generated for (partial)
III
112
113
114
115
116
117
.20
.30
.35
.40
.60
.80
.85
1.20
1.50
1. 70
1.80
2.20
2.10
2.50
12.00
15.00
17.00
18.00
22.00
21.00
25.00
2000.00
3000.00
3500.00
4000.00
6000.00
8000.00
8500.00
c
n.
-.l.
tliter.
.20
75
5
.50
130
6
.30
130
6
.40
75
7
43
Table 3.2.
Absorbance Values of Three Substances For Leucine Amino
Peptidase
Run
Std.
Cant. (1)
Cant. (2)
Run
Std.
Cant. (1)
Cant. (2)
1
126
73
72
11
122
72
72
2
124
68
66
12
124
70
72
3
116
73
68
13
124
71
71
4
118
69
69
14
124
71
69
5
115
63
66
15
118
73
71
6
116
71
71
16
115
66
63
7
118
73
71
17
121
70
71
8
111
71
71
18
127
75
72
9
121
70
70
19
122
70
69
10
122
71
71
44
Table 3.3.
Detailed Output for Azen and Reed's Data (canst. c.v.)
~AXIMUM ~IKeLIHODO ~LUTICNS
~QOEL:
Y IS NORMAL WITH
THIS IMPLIeS CONSTANT
~eAN
~U
CCE~ICIENT
AND VARIANCE
OF VARIATlON
~eLI:4tNARIES
GROuP
xaAA
085
19
19
19
1
2
3
17.145152
6.349030
5.772853
6~.736842
0.001231
C1.=
ITERATION
S2/XSAR2
S2
120.210S26
70.421053
cu.
CVSQ
O.001:Z3162
FINA~
SCLUTIO/loS ARe:-
.'4L SOLN FOR CV-
ML SOloN FOR CVSQ-
0.035094
0.001232
ML SOloN FOR MEANS
t"eAN
12C.210091
70.424470
3
<;19.733741
CHeCKING SUM-
2299.688506
OF=
Z
!:I-VALue..
0.0
0.001228
0.00t280
0.001187
:
45
Table 3.3 (Cont.)
4SYMPTOTIC VARIANCE COV4RIANCE MATRIX
0.93S176
0.000.49
0.320966
0.000445
Q.000261
0 • .314700
-0.000091
-0.00005::3
-0.0000S3
0.000011
ASYMPTOTIC CORReLATION MATRIX
1.000000
0.000820
1.000000
0.000820
0.000820
1.000!J00
-0.028643
-0.028643
-0.028643
1.000000
46
The results shown in Table 3.1 are typical of many more simulations that have been obtained for this model.
We see, in general, that
the estimates of the asymptotic variances for the maximum likelihood
estimators of the means are smaller than the ordinary estimates of the
variances.
The estimate for c in each simulation is very accurate.
The excellent result shown in Table 3.3 should be interpreted with
care.
That the convergence is attained in a single iteration and the
maximum likelihood estimates being very close to the ordinary estimates
of the means merely demonstrate one of many possibilities.
remember that the data is specially selected as an example.
We
The chi-
square value obtained is for testing the equality of the three means
and, as expected, the hypothesis is rejected.
An immediate generalization of the constant coefficients of
variation model is discussed in the next section.
47
4.
THE VARIANCE PROPORTIONAL TO UNKNOWN
POWER OF MEAN MODEL
The assumption of constant coeffient of variation is equivalent
to the assumption that the error variance is proportional to the square
of the mean, the constant of proportionality being the square of the
coefficient of variation.
If one wants to generalize such a model
the first tendency is for him
to look at a departure of the error
variance from being proportional to the square of the mean.
One
~y
is
to assume that the error variance is proportional to some unknown power
of the mean.
Special cases are:
the power is zero which is equivalent
to the constant variance model, and the power is two which is the previous model.
Under this model one more parameter is being added.
The (k+2)
likelihood equations cannot be reduced to a single equation in the
manner of model I.
Numerical solution to the likelihood equations by
a derivative-free algorithm is found to be unsuccessful.
The method
of scoring is employed and excellent results are obtained.
Sufficiently large ranges of values for the parameters are simulated.
For an illustration of application we use the same data as for
the previous model.
4.1.
The Likelihood Equations
A more general model than that has been examined in the preceding
section is of the form
48
i=l, ••• ,nj,j=l, ••• ,k.
The parameter space for this model is @)II which
is given by (2.2) of section 2.
A
The previous model corresponds to
= 2.
Let L(8),
!
E
~II' denote the likelihood function given y'
Then the function is given by
= (21T)-n/2
n.
J
2 A -1
2
~
~ (c ].1.)
(Y .. -].1 .) } ,
j;"l i=l
J
JJ.. J
k
k
1T (c 2 ].1.) -.n j /2 • exp {J
j=l
~
k
where n =
En.. As before we wri te
j=l J
likelihood is given by
~
for log L(~), and the log-
n.
k
~
= const -
~
j=l
2 A -1 J
A
2 -1
= const - - ~ n.log ].1. - n log c - (2c)
2 j J
J
Taking the first partial derivatives of
(4.1)
~ (y ..
{An./2)log ].1J. + n. log C + (2c ].1.)
J
J
J
~
i=l
JJ..
A
~].1~ E (Y ·
j
w.r.t.
J i
Ji
2
the parameters
~O/~].1.
/2
_ {(2c 2 )-1(_'/].1A.+l)
)2
(
a J = =An
J
~ Yji - ].1j
j ].1j
ox-
- ].1J.)
1\
+ (2c 2)-1].1=.A ~ ( y .. -].1. )(2)( - l)} ,
].J..
J
J i
L.
A+l
2
2 -1
- An./].1. - (2c) (>../].1. ) ~ (Y .. - ].1J.)
J J
J .J..
JJ..
2 -1
A
2
+ (2c) (2/].1.) ~(Y .. -].1.) , j=l, ••• ,k .
J i J J.. J
49
Equating this to zero and adjusting, dropping ",," to ease writing, we
have
2 -1 ~j! (Y
- (A / 2)~J'2 + (2c2)-1, L: ( )~ji - ~j) 2 /n j + 2(2c)
ji - ~j)/nj = 0
1\
i
i
or
, 2 A
(4.2)
I\C
~,
J
2
J
~,)
where 'P, = L: (y.. -
2
and Y ' = (l/n,) L: y.. , j=l, ... ,k J
J i
J~
If A = 2 equation (4.2) becomes
J~
i
(C
/ n,
J
J
2 + 1) ,,2 _ ".y. _ \112, = 0
,...
,...
"I
J J
j
which is the same as equation (3.2).
(4.3)
'd9.,/'dc = -n/c
J'=l ,
,
J
••• ,
k
Further,
3
+ (l/c )
2
A
L: n,'P,/~.
j
J J J
To obtain an equivalent form of equation (3.5) we equate equation
(4.3) to zero to get
(4.4)
c
2
=
= (l/An) L:
j
since
'1J':
= Ac
2
2Yj~j +
~.
J
c
2
= c
2
-
2~
2
,
J
A
(n./~,)(AC
J
J
2
~,
J
-
-
2y.~,
J J
+
2
2~.)
J
So
A-1
(2/nA) L: n.y. /~,
+ (2/nA) L: n j /~jA-2
j J J J
j
i.e.,
50
(4.5)
L:
j
11.-1
n.y .Ill.
J J
J
With A = 2 equation (4.5) reduces to equation (3.4).
Again
(4.6)
Equating this last equation to zero we get
2
-A
n. log ll. = (2c )-1 L: llj (n j log ll.)
J
J
J
j
j
~ L:
(4.7)
'¥J:
2
-A
2 -2
= (2c2) -1 L: llj (n. log 1l.)(S. + y. - 2y.ll. + ll.)
J
J
J
J
J
J
J
j
since
(4.8)
(4.9)
'¥~ = s.2 + -2
y.
J
J
2
2Y.ll. + ll.
J J
J
J
1
n. log ll. = 2
J
c
j J
L:
c
2
= (L: n . log ll.)
j J
J
L:
j
-1
or
also,
1..
{ ,
-A
llj (n j log
L:
j
2Y j llj + llj2)
-A
2 -2
2
llj (n. log 1l.)(S. + y. - 2y.ll. + ll.)
J
J
J
J
JJ
J
Equation (4.2) can be simplified by
'II
2 -2
(s. + y .
J
J
J
II . )
s~pstituting
2
2 -2
2
= s. + y. - 2Y.ll. + ll.
j
J
J
J J~
J
in it to obtain
(4.10)
2
2
2-2
AC ll. + 2 (A - 1) YJ.llJ' - (A - 2) ll. + A(s. + Y.) = 0 , j=l, ••• ,k.
J
J
J
J
51
The k+2 equations must be solved are
(4.10)*
2 A
Ac
. 1..1.J + 2 (A - 1) Yj l..l j
E n.y./l..I.
J J J
j
(4.5)*
(4.9)*
(E n. log l..I )
j
j J
-1
-
2
2
-2
(A - 2)1..1. + A(s. + y.) = 0, j=l, ••• ,k;
1..-2
= E n /l..I
j
j
j
J
J
J
and
2 -2
2
2
-1 E 1..1.- (n j log 1..1.) (s. + Y - 2y. - 2y.1..I. + l..I ) = c
j
J
J
J
J J
j J.
We first attempt to solve this system of nonlinear equations using
a derivative-free method [7], through an IMSL routine called ZSYSTM.
The results are very unsatisfactory, particularly in the sense that it
is very sensitive to the starting values.
We simply conclude that if
we are fortUnate enough to obtain the initial values that are very
close to the solution the method converges well.
Otherwise it is of
little value.
In the method of scoring we solve iteratively the likelihood
equations obtained by equating to zero expressions (4.1), (4.3), and
(4.6).
The information matrix plays an important role in this method.
The inverse of this matrix is the asymptotic variance-covariance matrix
for the estimates.
4.2
Information, Hessian, and Variance-covariance Matrices
To get the information matrix, I(!), the second partial derivatives of t are required.
Differentiating expressions (4.1), (4.3),
and (4.6) w.r.t. l..I , ••• ,l..I , c, and A once again we obtain
k
l
52
(4.11)
Taking expectation of this expression we have
(4.12)
2 2
A c
= -n (--2-
j
+
2 2
1) /e ~j , j=l, ... , k.
Similarly,
if m :f: j, and
(4.13)
=0
Ee {.•}
(4.14)
if m :f: j ,
m,j=l, ••• k
(4.15)
(4.16)
a2£/ ae 2 =
Ee
n/e
2 - (3/e 4 ) ~ n'~j/~'
2 A
{.} = n/c 2
j
4
- (3/e )
J
~ (nj/~~)
j
= -2n/e 2
J
• e
2
~.
J
53
(4.17)
a2J!./aea~. = (n /e 3) -OJ~~+l),¥: -
(4.18)
E!{'}
(4.19)
a 2n~ /aea'A = - (lie 3 )
(4.20)
E {.}
J
e
= -(nj/e 3 )(A/~jA+1 )
=
~
-(lie)
j
e{'} =
E
-n./2~j
J
J
J
~.
log
~.
log
J
2
~.
J
'lI.)/n.,
Yji-"J
~.)
a2 i/aA 2 = (1/2e 2 ) ~ log ~. ~
(y •. -
(4.24)
E {.}
n.e
J i
e
JI.~
J
I
(!>,
2
is
-A-1 2 A
)e~.,
J
J
J
;
~.)2 ~-J.A log ~. ,
J
A
J
~./~.
J
J
~.)(~.
, j=l, •.• ,k
(4.23)
Thus, the infrorna tion rna trix
(
;
J
+ (n./2e )(1-A1og
J
j
i
,
J
2
+ (1/2e )n. (1-A1og
J
= (-An./2~.)
(4.
J J
J
J
~j
2 A
e ~j ,
n. log
J
~
(21 A)
J
~ n.~.-A '¥.2
j
= -(n./2~.)
(4.21)
(4.22)
J
j
J
,
J
54
~ (l+A 2 c 2/2)
2 2
c ).l1
(4.25)
o
•
o
2 2
n (l+A c /2)
k
2 2
c ].Ik
---------------------------------------------------~------------
2n/c
1
2
-
C
1:
•
J
n .1og].l.
J
J
1 '~-' n. 1og].l.· -2·1:.
1
n. . (1 og).l J. )2
•
J
J
C
J.
J
a (k+2)x(k+2) matrix.
J
The elements of this matrix are obtained from
equations (4.12), (4.14), (4.16), (4.18), (4.20), (4.22), and (4.24).
The Hessian matrix, H(8), that is, the matrix of second partial
derivatives of t w.r.t. the parameters, is obtained from equations
(4.11), (4.13), (4.15), (4.17), (4.19), (4.21), and (4.23).
This
matrix is given by
(4.26)
A
2~IA+1)-1
- 2 ( C...
J
=0
j,m=l, ••• ,k;
n . (y j _ ].I.) _ n .
/C 2.
].1A
J
J
J
J
i f j=m ,
otherwise,
55
3 A+l -1 2
2
)
~. - 2n (y . - ~.)/c ~.
j
J
J
J
J
J
= - An.(c~.
(4.27)
J
(4.28)
2 A
- n. (log~. )(y. - llJ.) I c ~J.
J
J
J
n/c
1\+1, k+l <.~) =
(4.29)
2
4
- (3/c )
~
2
J J
A
J
n.~ ./~.
j
,
(4.30)
2
2
ul: A
Hk+ 2 , k+2 (~) = - (1/2c ) ~ (10 g~ J. ) nJ. J. Ill.J
j
(4.31)
3
1\+2, k+2 <..~) = - (l/c )
This matrix is symmetric.
~
j
-A
logll.
J J J
J
r..~ . ll.
,
.
A
When evaluated at 6 , the m.l.e., this
matrix is used for checking maximality of
e , by
showing that it is
negative definite, as shown by the sign of the eigenvalues.
We shall proceed to obtain an expression for the asymptotic
variance-covariance matrix of the estimates
i ,
c and~,
leaving the
actual process of obtaining the solutions until the end of this section
As in the previous section the asymptotic variance-covariance matrix
of
8'=
(~', ~, ~) can be obtained by inverting 1(6 ), the information
-
-0
matrix evaluated
(4.25).
at~o'
I(~)
where
in this case is given by matrix
Again we omit "0" to ease writing.
Now let
[Asy
var-cov(~ ~ ~)] [D bJ~l-l
,
_
-.!'
a
[:,
j
56
where w is the last column vector whose elements are the 'first (k+l)st
of the last column of [Asy
A
var-cov(~
A
~
, c , X)]
-1
, b
= ~ ~ no(log~o)
J
0-
= ~',
x'
and D is the matrix obtained from [Asy var-cov(
J
A
A
~,
c)]
-1
2
J
;
and where
r = -dD
-1
w
s .. -dx'D- l
and
where
a = 2n/c
u'
h
-1 -1
= (a-v'A~)
h
.E.
s.'
,.E.
-1
(a-~'A
~)
=
-hA
-1
= - h v 'A-1 •
and B
= A-I
+ hA- l uv'A- l
For our matrix
-1
= -(c 2 /2n)(1 + A2c 2 /2)
= -
u,..9"
2
= -
3
(Ac /2n)~
,1
57
This gives
2 2
(1 + n.A e /2n)
(B)jj
J
,
and
So
if j;'m.
j,m=l, ••• ,k;
j=l, ... ,k;
i'
where (D) J
-1
is the (i,j)th element of D •
w' = [(An l /2Il l )10g
w'D-1 w
Ill' ••• '
Also,
(A'fi k /2Il k ) log Ilk ' (l/e) L: n j log
j
Il
j
]
=
+ L:
jFm
2 4
2 2
{A e /2n(1 + A e /2)}1l.1l (An./21l.)(An /21l )logll.logll
J m
J
J
m m
J
m
+ {(l - A2 e2 /2)/2n
(L:
j
n.logll.)
J
J
2
•
58
(4.32)
n. (log
j
(4.3~)
r
J
l.l .)
2
J
= -dD -1w
d
Hence
(4.34 )
Asy
'" , c ,
var-cov(~
where G is given by matrix (4.34),
~
by vector (4.33) and h by equation
(4.32).
For a test of hypothesis of the form H :
O
statistic given by equation (3.57).
C'~
=
X
we use the
59
4.3
Numerical Solutions to the Likelihood
Equations: Method of Scoring
To apply the method of scoring we do not consider the likelihood
equations (4.10)*, (4.5)*, and
(4~9)*
but we go back to solve the
equations
aQ./alJj = O,j=l, ... , k
aQ./a '1 = 0, and
aQ. /a A = a •
That is, we look at the expressions (4.1), (4.3), and (4.6) and equate
each of them to zero, and then solve simultaneously for the estimates.
By the method of scoring we are supposed to solve iteratively
the linear equations
(4.35)
1(8)08
= ~(~)
where I(!) is the information matrix,
and
08'
~(!)the
efficient score vector,
= (OlJl, ••• ,OlJk,Oc,oA) a small increments in 8.
In fact, if
8. is the value of 8 at the i th iteration the value of 8 at the (i+l)st
-
-~
iteration is set so that
~-i+l
=
....
8. + -~.
08 .
-].
However, we find that if we use 1(8.) and s(8.) as they are to
-~
--~
obtain the increments 08., where, for e.g.,
---'l.
- =
matrix evaluated at 8
I(~)
is the information
8., scaling problems arise.
~
2
3
c, c , or c of some elements of I(J2. and
~(~)
Divisions by
result in very large
numbers, since C is small compared to other values in the components
of
I~)
and
E(~).
Since equation (4.35) is an ordinary system of
60
linear eauations in k+2 unknowns, multiplying any row of the system
by a constant does not change the final solutions.
~(~)
We scale I(!) and
as follows:
Multiplying equation (j) by 4c2~j
by 2c
3
2
,and '(k+2) by Bc •
Thus 1(6.) becomes, say, T(6.), where
I 4Ac~1A-In1 AC 2~lA-I nllog~l2
I
. 2 2 A-2
I 4AC~kA-I ~ AC 2VA-I ~log~k2
2(c A ~k +2)~
k
- - - - - - - - -1- - - - - - - - --- -I 4nc
2AC 2n k / ~k
c 2 ~n .log~.2
I
j J
J
o
o
T (6.)
=
-~
-
- - - - -
- -
-~
4.
2 2 A-2
2(c A ~l +2)n l •
(4.36)
, j=l, .•• ,k; equation (k+l)
- -
-
2AC 2n/~l
(2Ac2nk/~k)log~~ 14c ~n.log~7
I
where
each element is evaluated at 6 = 6. =
-~
write log~.
J
matrix.
= ~logv7.
jJ
(~.,C.,A.),
~
~
~
J
c2~n.(log~7)2
jJ
and where we
We refer to T(!) as the scaled information
J
We note that this matrix is not symmetric.
The scaled right hand side, or the scaled efficient score vector,
denoted by
(4.37)
t(6.)~,is
--~
2 A
J
t. = 4c ~ . . sj
J
= -ZAc
(4.38)
=
2
A-I
+
J J
n.~.
~+2 = Be
=
2An.$./~.
J J
J
+ 4n.(y.-u.), j=l, ... ,k;
J J J
2c 3 • s k+l
= -2nc 2
(4.39)
the (k+2)-vector given by
2
-2c
2
J
+ 2~(nj/~')~'
J
j
.sk+2
2
2
n.log~.
j J
J
+
-A 2
2rn.~.
j J J
2
~.logu.
J
J
J
61
In the solution of the likelihood equations if at some iteration any
of the
. 's becomes negative the process is terminated.
The itera-
J
tions may be restarted at different starting values in hopes of circumventing the problem.
If negative
. 's continue to arise then we
J
might conclude that the model does not fit the data and that some
other model should be employed.
Our method of scoring is based on the equations
(8.)08.
(4.40)
The starting values.
are the means y .•
2
J
. log
c
2
t(8.) .
-
1.
J
(0) = y., j=l, ••• ,k.
]1j
J
2
we consider the following.
log s.
=
Good starting values for ]1., j =1, •• " k,
Thus we take
J
(4.41)
_1.
""'""J.
Since s.
J
2
== a.]1,
J
For c and
(0)
and y.=]1.
J
J
by
-A , or
+ log y.
J
log s. = log c + A(~log y.)
J
J
This is of the form
v j =a +b).lj' j=l, ... ,k.
As a rough approximation we take
LU.V. - (LU.)(LV.)/k
(4.42)
b
=j
JJ
_
J
LU 2 _ (LU.) 2 /k
j j
(4.43)
. J. J
J
j
J
a=v - bu, where v=(l/k) LV. , ~=(l/k)Lu . •
j
J
j
J
We set A(0) = b, and c(O) = ea as starting values for A and c,
respec tively.
62
In practice, however, we find quite often that b tends to be too
low, while e
a
tends to be too high as the starting values.
·
S 1nce
s.2.= c 2 ~.A , t h en
J
J
ing procedure seems to be more appropriate.
.
·2
2
A = (log s. - log c
J
)/log~
The follow-
. , or
J
.
2
2
A =. (log s. - log c )/log y .
J
J
(4.44)
Now take
(4.45)
c.Q, = O:IU:"" s./y.)/I.in:
j J J J j J
(4.46)
kA
A*
(4.47)
*
2
= (L: log s. - k log
J
j
,
and compute
2
Co,9) /~log
J
i. e.,
Yj
2
2
= (L: log s. - k log c.Q,)/k L: log Y.
J
J
j
j
Then c omput e
(4.48 )
(4.49)
-A*
;c * = (L:I (n. s. /y. )) /L:yu. , and recompute
j
J J J
j
J
s~ - k log c*2)/kL: log y.
A** = (L: log
j
.
Th e s t ar t 1ng
v al ues are then
A
j
J
11
J
(.0) = Yj , J. -1 , ••• , k ,c (0) -- c * ,
~J
and
(0) = A** •
A Monte Carlo Study.
For each of the groups we generate, using
the IMSL routine GGNOR, n., j=l, ••• ,k, N(O,l) random variables
J
(4.50)
z. = (y.. -1Illl.) / (c
1
J1
J
2
~.)
J
~
i=l, ••• ,n.
The response to be studied are obtained from
J
63
i=l, ••• ,n., j=l, ••• ,k;'
(4.51)
J
where y .. comes from NID (ll.,
J
J~
The
chart.
C
2 A
ll.).
J
computing algorithm is summarized by the following flow
The subprogram shown is for the real data.
an obvious modification is necessary.
For simulations
In this flow chart 'INIT2'
is used to enter the data and then to compute the starting values.
'TOL' is a small number which determines the convergence of iteration.
When 'VMAX', the maximum value of the absolute ratios of the increments in the current 6 , i. e., 06 , r=l, ••• ,k+2, to
r
TOL, the iteration stops.
r
er ,
is less than
'NIT' is the current iteration number.
A Fortran program for computation of results is shown in Appendix 8.2.
64
MAIN PROGRAM
o
CALL INIT2
NIT = 0
TOL=lO**(-8)
--=~r-)--J.
l ~~_~~_~ _
----l---[
l
@
NIT=NIT+l
----~~~~~~-~~~~~~------------
Form scaled info matrix
,(8.)
""""l-
and scaled eff. se. vee. t(8.)
Compute
08.
~
=, -1 (8.).t(8.)
-~
v. = I 08 . /8 . I
J
J
J
~
,
j = 1, ...
, k+2
VMAX = max{v. , j=1, ••• ,k+2
J
------------- ----------------------
Yes
Fig. 4.1
No
Flow chart in unknoWn power case.
65
I
No
Yes
[-----~-------]
6.-6'+1
~
-~
[=~~;~~~l
Fig. 4.1
(Continued)
The above logic is necessary for treating negative values of c .•
~
When c + is negative the progr?m sets c i + = c i which is positive
l
i l
and proceeds to the next iteration.
66
e.
-e = -~
IF ITER-i
----------==]-=~------------1. (e)
Jm -
"
"2"A
= T.Jm (e)
/4c f.l.
J
,
.
J =1, •.• , k;
_
.
m-1, ••• ,k+2
"
"3
I k + 1 ,m(6) = Tk+1,m(~)/2c , m=1,
,k+2 ,
I k + 2 , m(S) = T k+ 2 , m(e)/8~, m=1,
,k+2.
-~-------------------][---------------------
l~--_~~~;;(~~:c~:~]
Fig. 4.1
(Continued)
67
Fig. 4.1 (cont.)
SUBPROGRAM (INIT2)
Q
1
I
I
READ k, n. ,
J
READ Yji,j-l, ••• ,k;i=-l, ... ,nj •
_____J
_
COMPUTE
I
I
I
I
I
I
I
I
I
I
Yj
,
I
I
=
I
I
2
Is
j
I
I
I
t
I
I
..
I
I
I
I
I
I
I
I
I
-----1-------
I .
,------------------------------------------,
I
k
_
k
*
k
2
2
k
_,
I cg,== ( E, rr;:: sj/yj)1 E Tn: , A = (l/k)( E log s.-k log c ,}(E log y.I
t______________ _
,______________ J
j=l
I
I
j=l
J
J
j!=L
.
J:
~
,
k
k
**
k
2
*2
k
_ I
c* =- ( E.;,
2 A*) I E ~, A
= (11k) ( E log s. - log c ) I E log y.
j=l njsj 1Yj • j-l J i
j=l
J
j=l
J:'
L
I
j=l
J
1
-------
I
I:
l';'
eJ<.0)
.. Y J. -1
k' c ( 0 ) = c*,' ( 0 )
j ' - ., ... , ,
1\
~
i
1-..1
:
RETURN
I
= A**
68
The following table shows some results for the simulations
performed.
sample sizes: n.=60,j=1, ••• ,5; no. of iter: 10
J
25.1425 +
111
25.00
25.0717 +
112
32.00
32.1672 + 1. 0542
32.1565 + 1. 0085
J.l 3
40.00
40.3560 + 1.2938
41.1989 + 1.2735
114
44.00
44.2961 + 1.3631
44.0725 + 1.3617
115
52.00
52.1378 + 1. 7741
52.4741 + 1. 6300
.8554
.8165
.25
.3000 +
.1830
2.00
1. 9008 +
.3350
c
sa~p1e
sizes: n j =60mj =1, ••• ,5; no. of iter: 8
69
Table 4.1
Continued.
True
Parameters
c
e. +
Yj + (std.dev.)
J
(asy.std.dev.)
1132.00
1135.25
+ 38.73
1141.04
+ 35.89
1225.00
1231.40
+ 40.36
1231.9~
-+ 38.25
1240.00
1282.04
+ 40.11
1276.30
+ 39.49
1444.00
1453.72
+ 44.73
1444.93
± 44.78
1652.00
1656.38
+ 56.36
1664.75
+ 52.98
.
.25
.2625 +
.6166
2.00
1. 9870 +
.6523
sample sizes: n.=60,j=1, ••• ,5; no. of iter: 10
J
Other parameters examined (partial)
II iter
]J1
]J2
]J3
]J4
]J5
c
.10
.20
.30
.40
.50
.08
3.00
n.
.-L
60
.25
2.00
60
5
.15
1.50
60/170
8
.80
1.50
60
10
.08
3.00
60
12
25.00
32.00
40.00
44.00
52.00
A
7
From the above and other tables which are not shown here we
observe, in general, that the estimates of the asymptotic variances
for the maximum likelihood estimators of the means are smaller than the
ordinary estimates of the variances.
are obtained in all the simulations.
Accurate estimates for c and A
70
4.4
Application
As an application of the method developed we consider the data in
Table 3.2 of Section 3.
The results are shown in the preceding table.
The meanings of some of the abbreviations used are:
n
XBAR:
yj =
S2XBAR:
CS:
c
2
2
s.
J
j
2
E y .. /n. , S2: Sj'
i=l J ~ J
2
=
= njs./(n.-l)/n.
J
J
-
2
L:(y .. -y.) In.-I),
i
J~ J
J
,DL: A , ML SOLN FOR CV:
ML SOLN FOR LAMBDA:
LN:
J
=
m.l. e. for c
m.l.e. for A ,
Asymptotic chi-square value"
DF:
degree of freedom,
MATRIX OF SEC PART DER EVAL AT ML ESTIMATES:
,.
matrix evaluated at
e.
PERFORMANCE INDEX:
The Hessian
a value which provides
a measure as to how well the IMSL routine performs for a particular
matrix when the eigenvalues are being computed.
expression (5.55) of the next section.
This is given in
71
Table 4.2
Detailed output for Azen and Reed's data (unknown power).
roIAXlMUM L.lI<ELlHOOD SOLuTIONS
~ODEL:
Y IS NORMAL wITH MEANS
~u
AND VARIANCE
PRELUUNAAIES
ass
1
2
:3
XSAR
19
19
19
69.73684211
INiTIAL VALS: CSa
FIRST 0ARTIAL
52
120.21052632
70.42105263
~.772S5319
0.00123177
~ERtvATIVeS
0.00000000
17.745152.35
6.34903047
52XSAR
0.98584180
0.35272392
0 • .32071407
1.99989201
EVAL AT L.AST ITER
-0.000100000
0.00000000
0.00000000
0.00000000
72
Table 4.2
~INAL
Continued.
SCLNS BY SCORING MeTHOD:
GROUP
I4l!AN
120.21042483
10.42434915
69.7:3.363679
1
2
3
141.. SOLN FOR
ev-
~L
SOLN FOR eVSQ:a
~I..
SOLN FOR LAM8DA-
NO OF
t TeR=-
0.00126831
1 • 993:38940
4
ASYIoIPTOTIC VARI.ANCE CCVARIANCE"'ATRIX
0.93459923
0.0000123;3
0.321:30035
-0.00001199
0.00038942
O• .3152S942
0.00217273
-0.00070158
-0.00073086
0.00339191
-1).02871234
o. 00820900
0.00858698
-0.04286162
1).54340369
0.00001
0.3212':1
0.:31494
EIGENVAL~ES
~ERFuRMANCE
CF
VA~-CCV
INoex-
I4ATRIX
0 • ..38
73
Table 4.2
Continued.
4SYMPTOTt C CCRAELATICh
~ATRIX
1.00000
0.00002
1.00000
-0.00002
0.00122
1.00000
0.03859
-0.02125
-0.022:35
1.00000
-0.04029
0.01964
0.>J207S
-0.998:35
1.000'00
0.0
-8.84670
-0.75452.
0.0
-lS.10082
-1.13891
-3.17609
-15.25040
-1.15780
"'ATRIX CF SEC PART OER
-1.07259
:).0
eVA'"
AoT "'L eSTIMATES
0.0
-3.11401
0.0
0.0
-d.84670
-15.10082
-1:5.25040
-d9879.41863
-7089. 345.a.6
-0.75452
-1.13891
·-1.15780
-7089.34546
-561.02335
-1.33201
- 3.1.1359
-3.17503
EIGZNllAol..UES OF THE
-90438.01553
"ERFOFH4ANCE
INoex-
."'AT~IX
-1.06753
0 •.32
74
As the power is assumed to deviate from two, slightly different
results are obtained for the same data under the assumptions of the
previous model.
In this example it is not surprising that the estimates
for the means and c are close to those of the constant coefficients of
variation model, since the estimate for
interesti~g
is close to two.
It is also
to observe that the convergence is attained in only four
iterations and that thellikelihood equations are accurately satisfied,
as can be seen from the values of the first partial derivatives
evaluated at the stationary points and the negative definiteness of
the Hessian matrix evaluated there.
The idea of the dependency of the variances on the means or
expectations can be extended to a linear model.
to such a model.
Section 5 is devoted
75
5•
LINEAR MODELS WITH ERROR VARIANCE PROPORTIONAL
TO AN UNKNOWN POWER OF THE RESPONSE
In many linear models problems some alternatives to the assumption
of constant variance may be necessary, for example, by formulation of
a form of weighting in the generalized least squares estimation.
In
the present section we consider one in which the error variance is .
related to the response in some manner.
In fact, we assume that the
error variance is proportional to an unknown power of the expectation.
The method of scoring will be employed to solve the likelihood
equations.
By the nature of the relationship between two of the parameters
in the error variance component the correlation between their estimators
is high.
Estimation of two parameters whose estimators are nearly
uncorrelated is achieved by a scale transformation on the response.
Estimates 'of the original parameters can be obtained by a transformation.
This procedure leads to greater stability of the solution
algorithm.
Numerical examples are given.
5.1
Information Matrix, Efficient Scores and
Their Scaled Forms
The model under consideration is a linear model of the form
x' S
-1-
E (y)
= X',g,
x' S
-n -
76
.. ,x.),
8' = (8 , .•. ,8 p )' and
where xi = (x.l,·
~
~pl
o
2
Var (Y) = c
*,
=
o
say.
(x' 8) A
~-
More specifically,
with the parameter space~III given in (2.3) of section 2.
If A = 0
then this model becomes the constant variance model.
The likelihood of
L(_6) =
n:n
~=l
x~
If we denote
-~
(21T)-
-B by
i given y is
~
2
~
2
Al
'2
[c (x~ 8) ]- exp {- ~ [c (x~ 8) ]- (Y; - x; _8) }
~
-
n. and the log-likelihood by
~
-~
-
~,
then
The first partial derivatives w.r.t. the parameters are
(5.1)
a~/aB. =
J
-
~
-2 l: -A
l: -1
- n.) +
A.n. xi. + c
i ni xij(Y i
~
~ ~
J
A(2c 2)-1 l: -A-l xij(Y - n ) 2 ,
ini
i
i
(5.2)
a~/ac = -
nc
-1
+ c
-3 l: -A
ini (Y i
n )
i
2
,
and
.L.L
77
(5.3)
The second partial derivatives and their expectations are
(5.4)
2
2
~ -2 2
-2 ~ -A
a ~/aB. = ~A .n. xi. - c
{in. x ..
J
1 1
J
1 1J
+
-A-l
Ani
x · · (y. - n .)}
1J
1
1
j=l, ••• ,p;
(5.5)
= -
(5.7)
c
-2
~ -A 2
.n. x .. 1 1 1J
1
~
A2
2
~ -2
1.n1. x .. , j=l, ••. ,p.
1J
E {.} = -
(5.8)
(5.9)
(5.10 )
E {.} = -2nc-2 •
2
-3 ~
-A
a ~/acaB. = -c
1. {2n1· X . .(Y1. - n.)
1J
J
1
+
-A-l
2
An.
x .. (y. - n.) },
1
1J 1
1
j=l, ••• ,p;
(5.11)
E {.}
= -A
(5.12)
(5.13)
c
= -
E {.}
= -
c
~ -1
. n. x ..
1 1 1J
-1
-1
c
-3
~
i
~
-A
j=l, ••• ,p.
2
.n. (log n.)(y. - n .) ;
1 1
1
1
1
log n i
78
=-
(5.14)
(5.15)
(5.16)
j=l, ••• ,p;
(5.17)
E {.}
= -
1
L -1
~A. n. X ••
1. 1.
1.J
j=l, .•• ,p.
log n.
1.
Equations (5.5), (5.7), (5.9), (5.11), (5.13), (5.15), and (5.17)
give the elements of the information matrix I(!),
(5.18)
1
0
J
k
(6)
-
= c
-2 L -A 2 + ~A2 L -2 2
..
..
i n i x 1.J
ini x. 1.J
= c
-2 L -A
.. x. k +
ini x 1J
1.
~A
if j=k,
2 L -2
i n i xijx ik
i f j;'k,
j,k=l, •.• ,p;
(5.19)
=
AC
-1 L -1
k=l, •.. , p;
.11. x ..
1
1
1.J
(5.20)
X. °
1J
(5.21)
= 2n/c 2 ;
(5.22)
=
-1 L
C
i
log T'li
and
This matrix is symmetric.
As before, we would like to obtain the numerical solutions for
the likelihood equations by solving the linear equations
79
(5.24)
I(6 ) ·06
-t
-
= -s(6
)
-t
similar to part 4.3, where -6t is some value of -6, according to the
scoring procedure.
To facilitate computations we also scale the
2
Multiply equation (j) by 2c , j=l, •.• ,p;
equations as follows:
2
3
equation (p+l) by 2c ; and equation (p+2) by 8c •
We also refer to
the new matrix as the scaled information matrix and denote it by
T(!).
This matrix, which is no longer symmetric, is obtained from
equations (5.18) through (5.23) and is given by
(5.25)
T· (6) =
J
k
-
=
L
i
L
i
-A
(2n.1 + A2C2n-2) x ..
i
1J
if j=k,
-A
2 2 -2
(2n. +A c n. ) xijx
1
1
ik
i f j'fk,
j, k=l, ... ,p;
k=l, ••. , p;
(5.26)
(5.27)
Tp+l,k(~) = Tk,p+l (!) ·c,
(5.28)
Tk , p+2(!) =
(5.29)
Tp+2 ,k (!) = Tk , p+2 (!) ·4
(5.30)
Tp+l,p+l (!) = 4nc ;
(5.31)
L
2
Tp+ 2 ,p+l (!) = 4c i log n i
(5.32)
T p+l, p+2 (!)
(5.33)
Tp+2, p+2 (~) = c
~AC
c
k=l, ... ,p;
2
2 L -1
x ..
i n i (log n.)
1
1J
2 L
2
log n ; and
i
i
2 2
2 L
(log n.)
i
1
The scaled information matrix is not symmetric.
80
From equations (5.1), (5.2), and (5.3), the scaled right hand
side, here denoted by E(!), is given by
(5.34)
j=l, ••• ,p;
(5.35)
and
(5.36)
That is, the correction 08. is obtained from
--J..
T(8 ) 08
(5.37)
-t
-t
= r(8 )
--t
instead.
We observe that for r(8
) to be defined for any A, n1..
-t
x~
all i.
5.2
Primary Development to Increase Stability
of the Solution Algorithm
Starting values
for~,
c, A.
(5.38)
2 A
2
cr. = c n
i
1.
(5.39)
2
2
log cr. = log c + A log n.
1.
1.
Suppose
i=l, .•. ,n, then
2
2
We approximate cr. by si where
1.
(5.40)
s2 • (y _ n.)2 = (y. _ x~ S)2
i =
i
1.
1.
1. -
For! we take the ordinary least squares estimates given by
_Q
I-'
*'
(0) = ( xx
, ) xl.
S
1.-
>
0 for
81
as starting values.
We then write (5.40) as
(5.41)
= log c
Z
(0)
(0)
=
' where n
i
i
+ Alog n
Xl
.§. (0)
This equation is of the form
and hence, roughly,
E
.u.w. -
1. 1. 1.
E
Cu.)
2
1. 1.
and
b
o
=
w - b
1
In
u , in the usual notations.
The starting values for c and A are taken as
c
= e bo • Since it could happen that y.1.
(0)
it is safer to define
if y.
1.
where 0 is a small number.
Z
In practice, S. can also be approximated
1.
2
by y. for this problem.
1.
Correlation between c and A.
Consider the submatrix of
corresponding to c and A, i.e.,
Zn/c
Z
c
-1 E
i log ni
(5.43)
and suppose now that
t.
1.
= log n.1.
and t
= n-1
E
. log
1.
n..
1.
I(~)
82
Inverting the submatrix we obtain
(5.44)
This submatrix provides a rough estimate of the variance-covariance'
matrix corresponding to c and A.
Thus, the correlation coefficient
between c and A is roughly given by
(5.45)
if
t
> 0, or
PC,A = + ( c 2t + 1 )~ , according as t
(5.46)
where c
2
t
>
<
0,
is the square of the coefficient of variation of t , i.e.,
i
(5.47)
c
2
t
=
L.:
-
2
-2
(t. - t) Int
1
i
2 -2
We usually approximate this by s / t , where
s
2
t
=
L.:
i
-
2
(t. - t) I(n - 1).
1
To effectively estimate c and A simultaneously IPA AI should be
C,A
small.
We make an observation here that when IPA AI is close to unity
2
this would mean that c
t
C,A
is close to zero, and therefore xLI remains
almost constant for i=l, •.. ,n.
In this case the ordinary least squares
estimation is applicable, because
all i.
07
1
2
= c (xtl)A is almost constant for
83
A simulation study.
Let
zi = (y. -
(5.48)
1.
x~ B)/c(x~
~-
B)A/2,
~-
Using the IMSL routine GGNOR we may generate
then z." N (0, 1).
1.
zi' i=l, .•• ,n.
We can then obtain y. by
(5.49)
y. = x' B +
1.
~ -
1.
c(x~
B)A/2 z
and hence y. comes
~ i'
1.
from NID (x'. S, c2(x~
B)A), i=l, ••• ,n.
-1.~-
By the characteristic of the
log function it is wise to generate data in which the y-values are
less than unity, but greater than zero.
desirable situation, namely, that c
2
t
This assures us at least one
is large, and hence
~ is. small.
c,A
The first column
For simplicity we choose an (nxp) X matrix with p=2.
of X is the unit vector.
p~
If we fix So = .05, Sl = 2.00, c = .20, and
A = 2.00 and the elements of the second column of X between 0.0 and
0.60, the values of
x~
B, i=l, .•. ,n, and also y-values, would be
1.-
between zero and unity.
Before proceeding further we first observe that the values of y
need not be restricted to be less than unity.
They can be extended to
values beyond unity so long as they are not too large.
This is because
the values of the log function still vary appreciably in this range.
For such generated y-values above we also compute estimates for
c ' the coefficient of variation of t. or log
1.
t
n.,
1.
~. arid p~ ~ the
c,A
c,A
correlation coefficients between c and A based on the submatrix and
the full matrix
I(~),
respectively.
As before,
p~
~(~)
and
H(~)
are computed
when convergence is attained to verify that the former is sufficiently
close to zero, and the latter for its negative definiteness by computing
its eigenvalues.
84
Tables 5.1 and 5.2 are summaries of some of the results obtained
in initial simulations.
With the hope of improving the results we consider a method proposed
by Marquardt [23] and [4J.
However, when we print out the eigenvalues
and the corrections a!, we find that the information matrix evaluated
at each iterating value is always positive definite, and the corrections
approach zero fast enough that the value of log-likelihood is always
increasing, approaching zero as the log-likelihood tends to the
maximum.
This happens for all the cases considered.
A typical case is
shown in Table 4.3.
The ordinary least squares solutions for f are obtained using the
IMSL routine AGLMOD.
From the output VARB of the routine we obtain
the variance-covariance matrix of
~LS
by multiplying each element of
VARB (a p x p matrix) by the error mean square for the estimates, given
by
~'y
where SS
= y'X
(X' X)*X'y is provided by the routine also, and '*'
denotes a g-inverse.
routine.
- SS)/(n - p)
IER
=0
indicates a correct execution of the
'NO OF ITER' gives the number of iterations required to
converge for a particular problem.
'MATRIX OF SEC PART DER EVAL AT ML
ESTIMATES' is the Hessian matrix evaluated at!, the final solution.
Its eigenvalues follow immediately.
The eigenvalues are all negative.
This means that the Hessian matrix is negative definite.
'PERFORMANCE
INDEX' provides some measure as to how well the IMSL routine performs
for a particular matrix when its eigenvalues are being computed.
expression for this index is given by
The
85
(5.50)
p
=
max
l~i.n
II AIIIII
zj "IIO(N) (EPS)
where w. is the eigenvalue and zj the associated eigenvectors.
J
appears in the note accompanying the routine.
This
When p is less than
unity, as in all of our cases, this indicates that the routine performs
well.
86
Summary for initial simulations (linear 'model)
Table 5.1
Range of y.
~
Range of
x~
-~
c
""
t
...
p""
c, ~
P""e,A""
.05 -
.36
.05 -
.29
.261
.971
.967
.05 -
.72
.07
-
.57
.488
.911
.898
.08 - •• 91
.07 -
.74
.597
.876
.858
.05 - 1.05
.06 -
.85
.693
.844
.822
.06 - 1.18
.07 -
.91
.783
.812
.787
.06 - 1.26
.07 - 1.05
.882
.778
.750
.08 - 1.55
.07 - 1. 25
10112
.700
.669
.15 - 1.84
.19 - 1.47
1.301
.613
.609
.09 - 1.85
.07 - 1.49
1~469
.592
.565
.07 - 2.30
.08 - 1.85
2.319
.4]0
.396
.07 - 2.67
.09 - 2.15
3.701
.250
.260
.13 - 3.04
.09 - 2.45
7.240
.009
.156
87
Table 5.2
Parameters
Estimates for above simulations. Generated for: a = 2.00
a1 = 2.00, c = .20, A = 2.00. Generated data based Oon
d~fferent X matrice~.
MLE
<.±
2 asy std dev)
.0149(-.q290, .0588
2.0848(1.9003,2.2694)
OL8
(±
2 std dev)
.0056(-.0799, .0912)
2.1158(1.8968,2.3347)
.1754( .1339, .2169)
1.6191( .9864,2.2699)
.0388( .0237, .0538)
1. 9421 (1 •7678 , 2 .1164 ) .
.0706(-.0644, .0768)
2.1214(1.8807,2.3621)
•2215( .1448, .2982)
2.1292(1.5411,2.7173)
.0413( .0257, .0569)
.0175 ( .0401, .0751)
2.0044(1.8603,2.1484)
2.1250(1.9126,2.3374)
.1848( .1293, .2402)
1.8781(1.3324,2~4237)
.0562( .0372, .0753)
1.8196(1.6417,1.9976)
.0317 (-.0414, .1049)
1.9325(1.6831,2~1820)
.2140( .1366, .2914)
2.0330(1.3948,2.6711)
.0407( .0238, .0577)
• 0119( -.0564, .0801)
2.0002(1.8610,2.1394)
2.1227(1.9129,2.3325)
.1873( .1394, .2384)
1.8887(1.3598,2.4176)
88
Table 5.3
Detailed output for an initial simulation. Generated
for: So = .05, Sl = 2.00, C = .20, A = 2.00
ORDINARY LEAST SQUARES SOLUTIONS FOR BETA:
""
So
OLS
""
S10LS= 2.1179
... -.0134
VARIANCE-COVARIANCE MATRIX (OLS) :
.003378
-.005145
c
t
.010419
= 3.701
INITIAL VALUES:
( (1))
So
= -.0134,
A (0)
=
0
S(O) = 2.1179,
1
c
(0)
= .1036
.8454
EIGENVALUES OF SCALED INFORMATION MATRIX DURING ITERATIONS
Iter 1
266.39
15.71
2.00
24.71
p = .42
2
1509.88
62.26
8.10
18.91
.18
3
1146.37
18.69
49.84
5.50
.29
4
964.80
18.31
46.70
4.93
.25
5
929.21
18.23
46.37
4.89
.23
6
921. 78
18.21
46.32
4.89
.23
7
920.01
18.21
46.30
4.89
.31
8
919.60
18.21
46.30
4.89
.28
9
919.50
18.21
46.30
4.89
.20
10
919.48
18.21
46.30
4.89
.19
89
Table 5.3
(Continued)
CORRECTIONS AND VALUES OF LOG-LIKELIHOOD (WITHOUT CONST)
Iter 1
.056040
-.108501
.152919
1.362888
LLHD = 31. 7240
2
-.003002
-.028794
-.050278
-.168057
76.0709
3
-.000704
.005339
-.011773
-.094417
80.0860
4
-.000506
.003436
-.001033
-.023790
80.3233
5
-.000133
.000887
-.000155
-.005282
80.3303
6
-.000030
.000197
-.000032
-.001252
80.3307
7
-.000007
.000047
-.000008
..... 000292
80.3307
8
-.000002
.000011
-.000002
-.000069
80.3307
9
-.000000
.000003
-.000000
-.000016
80.3307
10
-.000000
.000000
-.000000
-.000004
80.3307
FIHAL
SOLl~S
BY SCORING METHOD:
IND VAR NO:
ML
BETA
1
.03825557
2
1.99051585
SOLN FOR CV =
= 0.19320118
ML SOLN FOR LAMBDA = 1. 91512064
NO OF ITER = 11
ASYMPTOTIC VARIANCE COVARIANCE MATRIX
0.00012711
-0.00040174
0.00410623
0.00000554
-0.00026828
0.00035539
0.00066344
-0.00236502
0.00116280
0.05997454
90
Table 5.3
(Continued)
EIGENVALUES OF VAR-COV MATRIX
0.06010522
0.00405449
0.00008049
0.00032297
PERFORMANCE INDEX = 0.35
ASYMPTOTIC CORRELATION MATRIX
1.0
-0.55606907
1.0
0.02605990
-0.22211239
1.0
0.24028202
-0.15070604
0.25190279
1.0
MATRIX OF SEC PART DER EVAL AT ML ESTIMATES
-12520.10
-1196.21
-1069.11
229.34
- 1196.21
- 376.74
- 278.25
25. 6
- 1069.11
- 278.25
-3214.86
63.18
229.34
25.26
63.18
- 16.71
- 249.59
-11.89
EIGENVALUES OF THE MATRIX
-12765.92
PERFORUANCE INDEX
-3101. 01
= 0.21
91
5.3
Extension to General Data
Our model may be written as
1
Yl·= x1.~
(5.51)
+ C(X~13)1.U.,
~
1. i=l, ••• ,n;
where u.1. N(O,l).
Multiplying equation (5.51) by a constant factor k, we get
(5.32)
6) ~ u.
-1. 1.
ky.=
x~
-1. (k
!) +
=
x~
-1. (k
~)
+
=
x~
-1. (k
~)
+ ckl-~A (x~k S)~ u .•
1. 1.
1.
kc(x~
kc(x~k6)
~
~A -~A
-
k
u.
1.
This is of the same form as equation (5.51), i.e.,
*
*
* ~A
w. = x~ S + c (x~ S) u.
1. -1. -1. 1.
(5.53)
~~
where .§. = k .§.
*
= ck l-~A
and
c
w.'"
1.
NID(x~
,
or
S* , c * (x., S*) A).
-1. -1. -
We proceed to obtain the solutions for
~,
c* and A. as in the
previous part using the starting values
*(0.) = k~(o)
.§..
, and
c
*(0)
=c
(0)
.k
l_~A(o)
When the final solutions are obtained we compute
-1
S=k
A* and
• .§.
c = k
~~-l
92
The y-values are also transformed back into their original values.
,..
~,..
Using!, c, and A and the original y-values the scaled information
matrix
(8), and then the information matrix 1(6) are recomputed.
Inverting 1(8), the asymptotic variance-covariance matrix of
S,
c, and
,..
A is obtained.
We now determine the transformation factor k.
The transformation factor (choice of k).
that t i
= log(xi
When Yi
+
i),
i=l, ••• ,n; and we assume that x~! ~ Yi' all i.
KYi' then, say
t.*
(5.54)
l.
~
log(ky.)
= log
l.
k + log y.
= log k +
Hence,
We have from before
-*
t =
l.
t.
l.
log k + t , where -*
t = n-1 I t * and -t = n
i
i
-1
~
.
l.
t .•
l.
s for expression (5.47), if we denote the coefficient of variation for
*
t*
i by c t ' we have
(5.55)
where
t ) 2 /(n-l), and St2 as before.
(t * - -*
i
i
=I
93
In part 5.2 we develop an algorithm which solves problem concerning
Yi' i=l, ••• ,n, that are not much greater than unity.
Now we wish to
extend the algorithm so that it can also be used for solving problems
involving Y ,
i
i=l,.l~;n,
having magnitudes beyond those values.
yo, i=l, ••• n, are (much) greater than unity then
~
t
> O.
If
Following the
argument of the last part Ip~.~1
is small when c *t above is large.
~ ~ <
Logically we choose log k
1
-(-)
n·
-t •
e
e
0, or k
L log
= e
e
-t
<
1.
Now,
y.
~
i
I
~
1
n
log(y, )
~
i
1
=
= I
Yi
i
(5.56)
k
n
e 10g(Yi)
=
(L
=•
(L
1
n
1
n
i Yi)
i
,
or
\ )-lin
Yi
That is, k must be chosen so that it is the reciprocal of the geometric
'mean of yo, i=l, ••• ,n.
~
(i)
For convenience, k is computed as follows:
obtain t
= n-1
log y, , then
i
(ii)
set k
= e -t
~
94
Another advantage of the transformation is that we can handle data
whose y-va1ues are large, equivalently, whose x'
~
's are large, because
when the latter are large (~' !) , for certain >.., and (~' !)2 may be
larger than the values the computer can handle.
Tables 5.4 and 5.5 show summaries for some of the results simulated
for the general data.
Table 5.4
Summary for the general data.
Range of y.
~
Range of
"
xi ~LS
C
t
PA A
C,>..
P" A
c,>..
k
12.39-1060.78
12.50-995.00
.141
-.985
-.981
.003
4.78- 124.11
5.60-100.00
.224
-.982
-.952
.026
1. 24- '106. 08
1.25- 99.50
.231
-.961
-.949
.030
.48-
12.41
.56- 10.00
.607
-.888
-.731
.258
.12-
10.61
.13-
9.95
.647
-.773
-.705
.295
.12
3.10
.14-
2.50
19.902
• ellO
.003
1. 031
95
Table 5.5
Summary of edtimates for the above.
Parameters
MLE (+2 asy std dev)
Generated for:
So = 5.00, 13
1
=
150.00,
OLS (+ std dev)
c = .20,
x = 2.00
Generated data based on different X matrices.
4.76(2.13,7.38)
142.70(133.07,152.33)
c
25.59(-36.02,87.19)
131.96(115.88,148.03)
.074(-.023, .170)
2.347(1.999,2.785)
Genera ted for:
60
= .50,
6 = 15.00,
1
c = .20,
•476 (. 213, •738 )
14.170(13.307,15.233)
c'"
A = 2.00
2.559(-3.602,8.719)
13.196(11.588,14.803)
.111 ( .021, .200)
2.347(1.909,2.785)
Generated for:
c
60
= .05,
6
1
= 1.50,
c = .20,
A = 2.00
.048(.021,.074)
2.56(-.361, .872)
1.427(1.331,1.523)
1.320(1.159,1.480)
166(.107, .224)
2.347(1.909,2.785)
Generated for:
6
1
= .10,
6
1
= 4.00,
c
= .20,
A
= 2.00
• 081 ( •048, • 115 )
• 024( -113, .160)
4.000(3.722,4.279)
4.245(3.826,4.665)
c'"
.195 (.158, .231)
'"
A
1.887(1.360,2.418)
96
Table 5.5
Continued.
Parameters
Generated for:
MLE (+ asy std dev)
o=
.10,
1 = 4.00,
OLS (+ std dev)
c = .20,
A
= .50
"
.027(-.055,.109)
.019(-.083, .432)
Sl
"
4.189(3.950,4.587)
4.216(4.904,4.528)
c
.191(.156,.226)
"
A
.380(-.082,.842)
So
Generated for:
c
o=
.10,
1 = 4.00,
c = .20,
A = 3.00
.095(.079,.112)
.015(-.169, .199)
3.910(3.638,4.183)
4.314(3.747,4.881)
.198( .158,.237)
2.943(2.405,3.481)
97
Table 5.5
Continued.
Other values generated for (partial)
So
Sl
c
.10
4.00
.20
.20
.20
.50
.• 20
1.00
.20
2.00
.20
3.00
.20
4.00
.10
2.00
.50
2.00
98
5.4
Application
Using Carr's example concerning the catalytic isomerization of
n-pentane to isopentane, Box and Hill [5], assuming the same model as
ours with
i=l, .•• ,24, where e.
~
~
2 A
NID(n.,c n.), n. = E(y.), and
~
~
~
~
;\=
2(I-~)
and assuming the prior for cr (or c) which is proportional to cr
optimizing e
/\
to obtain an estimate ep, for
(~/y)
~,
-1
,
via a Bayesian
approach, obtain an estimate for!' = (8 , .•• ,8 ) as
4
1
e=
(.n.2ep.
d ~ag
whel1e
W
where
~LS
~
=
(x
i1
~
2
,
(X'WX)-lX'Wy
. 1 , ••• , n )
~=
A
.
They estimate ni by x~ ~LS
is the ordinary least squares estimate of !, and
, ••• ,x
14
).
That is, the elements on the diagonal of Ware
estima ted by
,
(~~LS)
2~-2
, i-I, ... , n.
The data below is reproduced from Table 1 of [5].
We note here
also that in applications we use routine LLSQAR to obtain the ordinary
least squares estimates of the linear parameters, as the corresponding
parts of the initial values.
and develop the method.
Routine AGLMOD is used only to evaluate
In applications many of the statistics pro-
vided by AGLMOD are unnecessary.
99
Table 5.6.
Run
Data used in Carr's example.
~i1
~i2
~i3
r.~
1
. 205.8
90.9
37.1
3.541
2
404.8
92.9
36.3
2.397·
3
209.7
174.9
49.4
6.694
4
401.6
187.2
44.9
4.722
5
224.9
92.7
116.3
.593
6
402.6
102.2
128.9
.268
7
212.7
189.9
134.4
2.797
8
406.2
192 .6
134.9
2.451
9
133.3
140.8
87.6
3.196
10
470.9
144.2
86.9
2.021
11
300.0
68.3
81. 7
.891
12
301.6
214.6
101.7
5.084
13
297.3
142.2
10.5
5.686
14
314.0
146.7
157.1
1.193
15
305.7
142.0
86.0
2.648
16
300.1
143.7
90.2
3.303
17
305.4
141.1
87.4
3.054
18
305.2
141.5
87.0
3.302
19
300.1
83.0
66.4
1.271
20
106.6
209.6
33.0
11. 648
21
417.2
83.9
32.9
2.002
22
251.0
294.4
41.5
9.604
23
250.3
148.0
14.7
7.754
24
145.1
291.0
50.2
11.590
100
Transforming as in [5], i.e.,
y. = l/r.
and
~
~
i=l, .•• ,n;
we obtain the X matrix and y as in Table 5.7.
Using these transformed X and L we proceed to analyze according
to our method and obtain results as in Table 5.7.
The format is slightly
different from that of Table 5.3 since we feel that it is now not
necessary to show details as in there.
'OBJECT
Fm~CTION'
is the value
of log-likelihood evaluated at each iteration, neglecting the constant
term. 'VMAX' is the criterion of convergence value, i.e.,
88.
VMAX = max
{I e~ I,
i
~e
i=l, ... , p+2}.
~
also show only the values of efficient scores at the end of the
iterations, as indicated by 'FIRST PARTIAL DERIVATIVES EVAL AT LAST
ITER' •
Other abbreviations are the same as explained for Table 4.3 of
the last section.
The
'"
f3
,.
(or
~)
values 8
1
'"
= 1. 025, 8
2
=
.053,8
3
=
"
= .130, are back transformed according to the relationships
and e
4
.025,
1m
We then show a comparison table for the ordinary least squares, Box and
Hill, and the m.l. estimates.
It is interesting to note the predicted
values for the ordinary least squares and for m.l.'s, and then compare
them with the observed values.
102
Table 5.7
Printout obtained for Carr's data.
14AJUMUM L.Il(eL.IHOCD sa..UTIONS
Tt1l! X-MATRIX ANO '(-veCTOR
0.015
3.019
. 1 • .:s:s3
0.54"
0.282
0.014-
5.729
1.315
0.514
0.417
0.007
1.450
1.209
0.342
0.149
0.006
2.515
1.t72
0.281
0.212
0.047
10.490
~.324
5.425
1.686
0.043
17.340
/1..402
5.552
3.731
1.78 "
t.752
1.286
0.358
1.227
0.408
0.313
0.010
2. 034
0.009
3.695
o. all
1.330
1.616
1.005
0.011
5.177
1.585
0.955
0.495
0.055
Ib.4S0
3.745
4.479
1.116
0.007
1.981
1.409
0.668
0.197
0.007
2.190
1.047
0.077
0.176
0.020
6.225
2.909
3.115
0.838
0.011
3.423
1.590
0.963
0.378
0.011
3.394
1.625
1.020
0.303
o. all
3.488
1.612
0.998
0.327
O. all
3.401
1.604
0.986
0.303
0.024
7. 092
1.962
1.569
0.787"
0.007
1.801
1.ObS
0.106
0.129
a.Olb
Q.~45
1.316
0.516
0.500
0.004
0.933
1. 095
0.154-
0.104
0.00~
0.503
1.107
0.174
0.086
O.IJ04
o.sss
1.113
0.193
0.01:56
TR~NSFORM
FACTOR.
2.906
"'
103
(Continued)
Table 5.7
INIIiETA(O1.5I
-185.47479089
0.53196022
0.10047575
3
1.26....,6601
U.I TVA&.$:
c"
0.42140643
ITER
OBJECT FN
1
i!
-0.905001710
-0.31:19093880
-U.ll0414800
0.61"074200
0.215041600
0.345714000
0.379952130
0.3d5585930
0 • .3135749410
0.385752100
0.38:57!2440
0.385752520
0.385752540
0.385752540
0.31:15752540
0.335752:140
0.385752540
0.385752540
0.3857525.0
0.3<157525.0
a .3'J5752540
0.3<:1575"540
3
4
(S
<!>
1
3
9
10
I I
12
13
14
15
10
17
la
19
i!"
21
22
Fl~ST ~ARTIA~
'I"'A)(
01
02
02
01
02
'02
02
02
02
02
C2
02
02
02
02
02
02
02
02
02
02
"2
0.139591750 01
0.267029980 08
0.203312470 07
0.725275690 00
0.55068.770 00
0.1&4603600 01
'.979816180-01
0.585017190-01
0.1:50902520-01
o. 11397 46S0-01
0.485730720-02
0.247521250-02
0.119438650-02
0.585!502480-03
0.255627840-03
0.139!564950-03
0.6<113.39040-04
0.332898400-04
0.1~25a7n2O-04
0.794111670-05
0.387952950-05
0.159433810-05
OERIVATIVES !VAL AT
-0.000001.l12
0.00007402
~AST
ITER
0.00001545
-0.00000000
-".0000011 .,
104
Table 5.7
~IN~
(Continued)
SCLNS SY SCURING
U"OVAANQ:
~THOD:
SETA
1.19677597
2
0.OS21H346
3
0.Q2471951
4
'>.12945620
'4\" SOLN ~OA
1111.
SOLN
:-IQ
01" ITER-
~OA
ASYMPTOTIC
cva
1."/lISO"-
0 • .1.331 3233
3.30338835
22
vAAl"NC~
COVARIANC~
.... fRIX
5.1291.3077
-'>.00839772
0.J0002510
-0 • .11452934
0.,)OUOI323
-0.00829200
-0.00000403
-0.00001380
0.00029405
-.:I.00U02700
0.1.l0009347
-0.00037239
-0.00735616
-O.0I.lC02302
(1.00026725
-o.uo 102949
0.02414769
0.1277"739
5.12921070
,).13236235
0.00026392
0.002S0!06
0.00000146
).0000I~62
~ERF3R"ANC= INC~X.
0.17
105
Table 5.7
(Continued)
4SY'4PTOTlC CCRA~L....TtCN I4ArRIX
1.00000000
-0.74005853
1.00000000
-0.86657324·
U.49149046
-0.21351503
-0.0~fo95Qla
-0.010.J794.J
-0.J0908911
,~ATR 1 X
CF
~C
1.00000000
-0.10869051
1.00000000
-0.06373594
0.14929601
-0.25680346
1.00000000
-0.01285429
0.10101767
-0.16799805
O.7<J906988
1.00000000
~"'"T ,)ER eV41. 4r .111. eSTII4ArES
-d.CS
-1517.59
-1716.04
-337.093
-9.7'5
2.60
-1517.59
-34:;790.31
-304205.7?
-01870.15
-2664.39
1566.51
-171 ... lJ4
-304205.79
-390797.20
-71346.39
-1510.28
499.07
-337.093
-01870.15
-71346.39
-18256.19
-65.... 8!!
122.35
-9.75
-2664.39
-1510.28
-654.88
-432.39
73.60
2.60
ob6.51
499.J7
122.35
7!h60
-23.16
-03438.28
-4774.76
-0.20
-400.52
-3.16
=:I.::iC:'lV4I.UES
OF THe ,II4TRIX
-l5dt:c75.~'
.'ei'lF'Ji'l~~ANCe
INoexa
0.29
106
Table 5.7
( Continued)
~ReDICT!O
cas
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24-
ASY
VA... ues
YHATlOLSI
YHAT( .'4I.EJ
-0.094
0.452
0.034
0.264
1.521
3.089
0.393
0.733
0.105
0.749
1.657
0.284
0.045
1.379
0.431
0.453
0.460
0.449
0.558
-0.012
0.474
0.043
-0.079
-0.007
CHI-SORO VA ... •
0.280
0.417
0.159
0.205
1.417
1.791
0.330
0.407
0.264
0.448
1.604
0.234
0.159
0.827
0.357
0.364
0.366
0.363
0.6!!3
0.143
0.463
0.101
0.085
0.087
3827.973
OF..
4-
~IIA
...ue..
0.0
107
5.8
Comparison table of the estimates.
OLS
Parameters
Weighted [ 6]
.335
c
'" ,.
~(</»
3.6(-.8)
,.
k
O
'"
k
k'"
MLE
l
2
3 .8l6( -.908)
16.331
40.00
39.531
-.003
•7 5
.051
-.001
.35
.025
-.007
1.85
.127
....
k
3
For the ordinary least squares estimates we obtain the following
....
~LS:
,.
-64. 811 , 8'"
= .185, 8'"
= .061, and
2
3
OLS
OLS
84
= .432.
kl , k2 , and k3 are computed as before and are
OLS
shown in the above table.
values for
,.
8
l0LS
,.
Then k O'
=
We note that the transformation factor for this data is greater
that unity, as expected from the values of the response Z, in Table
5.7.
We would have obtained the same results if we did not transform
the data.
108
6.
SUMMARY AND CONCLUSIONS
Maximum likelihood solutions are obtained for each of the
following models:
(i) Model I:
i=l, ••• ,n.,j=l, ••• ,k.
J
This is the constant coefficient of variation
model.
(ii) Model II:
i=l, ••. ,n.,j+l, ... ,k.
J
Y •. J1
2 A) ( ,
NI D ( ~ ., c ~. , ~ , c,
J
J-
This the case when the group with mean
has
~.
J
error variance proportional to an unknown power of the mean.
(iii) Model III:
j=l, ..• ,n.
This is a linear model whose error variance is proportional
to an unknown power of the expectation.
To employ the theorems of section 2 to obtain the consistency and
asymptotic normality of the estimators we establish that the models in
question satisfy certain regularity conditions.
These conditions
involve dominating the first, second, and third order partial 8derivatives of the log-likelihood.
This is done in context of
bounded parameter spaces.
For Model I, we first look at the special case of k=2, for
unequal sample sizes.
In the k sample case, by manipulation of the
likelihood equations, we are able to express the estimators of
function of c alone and obtain a nonlinear equation in c.
~.
J
as a
This equation
109
is solved by the method of bisection.
Upper and lower bounds are
obtained for c which serve as starting values.
is discovered through simulation studies.
on this method is constructed.
real data are presented.
A better lower bound
A computing algorithm based
Examples for both of simulated and
In both cases the algorithm gives rapid
convergence.
From the expected values of the second partial derivatives of the
log-likelihood we form the information matrix, which is, in this case,
of a simple form.
This can easily be inverted to obtain the asymptotic
variance-covariance matrix for
~l"
asymptotic relative efficiency of
..
'~k'
and c.
~l'."'~k
From this the
relative to the ordinary
means Yl'."'Yk can be calculated.
For Models II and III reduction of the likelihood equations similar
to Model I is not feasible.
In Model II, we first attempt to solve the
equations using a derivative-free method.
unsatisfactory.
standard method:
However, the result is
Ultimately, we settle on a modified version of a
the method of scoring.
The elements of the information matrix and the efficient scores
involve divisions by some powers of c whose value is usually less than
unity.
These result in unstable values during iterations.
the k+2 equations as follows:
4c
We scale
Multiply equation (j), j=l, •.• ,k, by
2 A
3
~., equation (k+l) by 2c , and equation (k+2) by
J
2
Bc.
With these
scalings the linear equations I (~)~ = i.(~) used in the method of
scoring are stabilized.
Th
initial values for
~l'.'.'~k
are taken to
110
be the sample means Yl""'Yk'
values are employed.
For c and A more complicated starting
The algorithm is tested on simulated data from a
wide variety of models.
The inverse of the information matrix in Model II is cumbersome.
We obtain the estimate for the asymptotic variance-covariance matrix
of ~l""'~k' c, and A by numerically inverting the information matrix
evaluated at the estimates.
We also check for the negative definite-
ness of the Hessian matrix at the end of the iteration.
that the solution so obtained is maximal.
This verifies
This is accomplished by
examining the signs of the eigenvalues.
The information matrix for the last model is much more complicated
than for previous models in the sense that none of the principal minors
are diagonal matrices.
To apply the method of scoring we also scale
the information matrix and the score vector as in Model II.
For this
model it is necessary to examine an approximation to the correlation
coefficient between the estimators of c and A in order to obtain a
transformation of the data which orthogonalizes their estimates.
We begin our empirical study for this model by choosing y-values
to be less, or not much more, than unity.
By this choice the
correlation coefficient between c and A is not close to unity.
Based
on this result we extend the algorithm to more general data by transforming the observations.
This is a scale transformation in which the
scale factor depends on the observations themselves.
This factor is
the reciprocal of the geometric mean of y's.
For each of the first two models and the third we construct a
test about their means and betas, respectively, using asymptotic chisquare tests.
III
In the course of this investigation some topics for 'further
research have arisen.
(i)
Among these are:
Simulation studies to access validity of asymptotic
variances for use in small samples.
(ii)
Relative efficiency of proposed estimators w.r.t.
others in the literature.
(iii)
Multivariate generalization for the proposed models.
112
7
LIST OF REFERENCES
1.
Abramowitz, M. and Stegun, I. A., Handbook of Mathematical
Functions, New York: Dover Publication, Inc., 1965.
2.
Amemiya, T., "Regression analysis when the variance of the
dependent variable is proportional to the square of its
expectation," Journal of American Statistical Association,
68(1973), 928-914.
3.
Azen, S. P. and Reed, A. H., "Maximum likelihood estimation of
correlation between variates having equal coefficients
of variation," Technometrics, 15, No. 3(1973), 457-462.
4.
nard, Y., Nonlinear Parameter Estimation, New York:
Press, 1974.
5.
Box, G. E. P. and Hill, W. J., "Correcting inhomogeneity of
variance with power transformation weighting,"
Technometrics, 16, No. 3(1974), 385-389.
6.
Bradley, R. A. and Gart, J. J., "The asymptotic properties of
HL estimators when sampling from associated populations,"
Biometrika, 49(1962), 205-214.
7.
Brown, K. M., "Computer oriented algorithms for solving systems
of simultaneous nonlinear algebraic equations." in Numerical
,Solutions of Systems of Nonlinear Algebraic Equations, by
Byrne, G. D. and Hall, C. A., New York: Academic Press,
1972.
8.
Chanda, K. C., "A note on consistency and maxima of the roots of
likelihood equations", Biometrika, 41(1954).
9.
Chow, G. C., "Two methods of computing full-information maximum
likelihood estimates in simultaneous stochastic equations,"
"International Economic Review, 9, No. 1(1968), 100-112.
Academic
10.
Churchill, R. V., Brown, J. W. and Yehrey, R. F., Complex Variables
and Applications, New York: McGraw-Hill Book Co., 1974.
11.
Cramer, H., Mathematical Methods of Statistics, Princeton
University Press, 1946.
12.
Dent, T. W. and Hildreth, C., "Maximum likelihood estimation in
random coefficient models," Journal of American Statistical
Association, 72(1977), 69-72.
13.
Hildebrand, F. B., Introduction to Numerical Analysis, New York:
McGraw-Hill Book Co., 1956.
14.
Hinnnelblau, D. M., Applied Nonlinear Programming, New York:
McGraw-Hill Book Co., 1972.
113
15.
Huzurbazar, V. S., "On a property of distributions admitting
sufficient statistics," Biometrika, 36(1949), 71-74.
16.
Jenrich, R. I. and Sampson, P. F.," Newton-Raphson and related
algorithms for maximum likelihood variance component
estimation," Technometrics, 18, No. 1(1976), 11-17.
17.
Kale, B. K., "On solution of likelihood equation by iteration
processes, "Biometrika, 48 (1961), 452-456.
18.
Kale, B. K., "On the solution of likelihood equations by
itera tion processes," Biometrika, 49(1962), 479-486.
19.
Kelly, L. G., Handbook of Numerical Methods and Applications,
California: Addison-Wesley Publishing Co., 1967.
20.
Kendall, M. G. and Stuart, A., The Advanced Theory of Statistics,
Vol. 2, New York: Hafner Publ. Co., 1967.
21.
Khan, R. A., "A note on estimating the mean of a normal distribution with known coefficient of variation, Journal of
American Statistical Association, 63(1968), 1039-1041.
22.
Lohrding, R. K., "A test of equality of two normal populations
means assuming homogeneous coefficient of variations,"
Annals of Mathematical Statistics, 40, No. 4(1969),
1374-1'385.
23.
Marquart, D. W., "An algorithm for least squares estimation
of nonlinear parameters,~ Journal of Soc. Indus. Appl.
Maths., 11, No. 2(1963), 431-441.
24.
Noble, B., Applied Linear Algebra, New Jersey:
Inc., 1969.
25.
Rao, C. R., Linear Statistical Inference and Its Application,
New York: John Wiley and Sons, Inc., 1973.
26.
Rao, C. R., "Simultaneous estimation of parameters in different
linear models and applications to Biometric problems,"
Biometrics, 31(1975), 545-554.
27.
Sen, A. R. and Gerig, T. M., "Estimation of population means
having equal coefficient of variation on successive
occasions," Proc. of the 40th Session of the Intern.
Statist. Instrt., Warsaw (1975), 774-782.
28.
Theil, H., Principles of Econometrics, New York:
and Sons, Inc., 1971.
John Wiley
29.
Wilks, S. S., Mathematical Statistics, New York:
and Sons, Inc., 1963.
John Wiley
Prentice-Hall,
114
30.
Williams, E. J., Regression Analysis, New York:
Sons, Inc., 1959.
John Wiley and
31.
Zacks, S., The Theory of Statistical Inference, New York:
John Wiley and Sons, Inc., 1971.
32.
Zeigler, R. K., "Estimators of coefficients of variation using
k-samples," Technometrics, 15, No. 2(1973), 409-413.
115
APPENDICES
116
Appendix 8.1
Sample program for solving likelihood equations in normal
populations having constant c.v. by bisection method.
~AXlMU~ ~IKELIHoao
SOLUTIONS FOR K GROUPS O~ NORMAL POPULATIONS
HAVING CONSTANT COEFFICIENT OF VARIATION 3Y REDUCING
THE K.l EQUATIONS INTO A SINGLE EQUATION AND SOLVING IT
BY ~ISEeTION MeTHOD
ASYMPTOTIC CHI-SQUAReD TEST ABOUT THE ~EANS IS PROVIoeD
IMPLICIT REAL*S(A-H.o-Z)
REAL*a S( 15) • T (15) .u (15) • Z (250) • VC 250 ) • \If l ( 250) .IIIfZt 250) • AL( 1) •
1w3(250).HC15.1S)
FU!AL.*4 XS.R ANI< • P. G
DIMeNSION N(2S)
~
C
C
0001
0002
C
0003
01);)4
OOOS
o
OOOS
0009
0010
OOll
001Z
0013
1J0140015
0016
C
C
C
C
C
NITLIM-MAX NO O~ [TERATIONS
KaNa OF GROUPS
N(J)-NO OF 085 IN .1 TH GROUP
wFlITE[3.1Z)
FORMAT C/ / . '
;:IREl..I M[MARIES' 1')
wRITEl3.131)
FORMAT(//.'
GROUP
oas
131
1
S2/XBARZ'/)
00 20 .1" •• 1(
NT-NT.N(.1 )
12
0017
Q013
0019
0020
0021
0022
0023
1J024
0025
0026
S(
NNaN(~)
00 30
110
C
C
C
30
"030
20
00340037
1~O
0038
0039
0040
S(.1 l-S(.J) +X
TI.1).. T(J,+X.*Z
T[.1)a(T[.1'-(S(J)··2)/N(J»/N(.J)
S( J)"S(.1) /N( J)
UIJ)aT(J)/(SIJ) •• 2)
C=O.uO
A=O .00
CO 25 .1=1,1<
"'iH TE ( 3.130) .1 • N (
OOJ3
0035
I-l.NN
REAOClol10) X
FORMAT(FIO.S)
.....0.00
OO:U
OO~2
0036
.1).0.00
T(.1).O.OO
0027
0028
0029
WRI~C3.6)
FORMATC//.'
MAXIMUM ~IKELIHOOD SOLUTIONS·/}
WRlTE(3.7)
7
FORMAT(//.·
~ODEL: Y IS NORMAL WITH MeAN
~U AND VARIANCE
C-5QR
1*OIU.*.2·/)
... RITE(3.S)
FORMAT(//.·
THIS IMPLIES CONSTANT COEFFICIENT O~ VARIATION'//)
a
NT"O
REAO(1.100) NITLIM.I<.(N(J).~••• J()
100 FORMAT(2I5.14-15)
0006
0007
25
J) ,S (
.1) ,T ( J
) ,U ( J)
FaRMATI1H.I5.2X,IS.2X.~Fla.ol
C=C~OSORT(UC.1)·N(J)
X~N(.J )
... =W+OSORT I X)
~,,"A+U(.1)*N(J)
XBAR
52
117
Appendix 8.1
Continued.
0041
0042
0043
0044
0045
0046
0047
0048
0049
0050
0051
210
13
28
0052
0053
00S4
40
0055
0056
0057
0058
0059
120
0060
0061
0062
0063
27
00':>5
50
0067
0068
0069
0070
14
0072
0073
0074
0075
0076
220
0064
0066
170
0071
0077
0078
180
190
C
O=A/NT
c=c/w
8=C**2
NIT=O
WRITEl3.210) 8.0
FORMAT(/'.·
CL.·,FlS.6. o CU=',F15.6/)
wRITEe3.13)
CVSO"/)
FORMAT(//,"
ITERATION
C$.=:(8+0)/2.0
OIV1'lO.OO
00 40 J=l,1<
0IV=0IV+N(J)/(-1.0+0SQRTCl.00+4.00*CS*(UCJ)+1.00)))
CSO=NT/(2.00*OIV)
NIT=NIT+l
~RITEC3.120) NIT.CS
FORMATlIH.I5"2X.F16.8)
IF(NIT.EQ.NITLIM)GO TO 50
P=CSo/cs
IFCP.EO.l.00IGO TO 50
IFCCS.LT.CSO)GO TO 27
o.=CS
GO TO 28
B=CS
GO TO 28
SA"","O .00
WRITE(3.14-)
FORMAT(//.·
FINAL SOLUTIONS ARE:-")
CE=OSQRT( CS)
'otIRITE(3.170) eE
ML SOLN FOR CV=",F15.6)
FORMATe/,.'
WRITE C3.220) CS
FORMAT(/,. ° ML SOLN FOR CVSO=".F15.6)
WRITE(3.180)
FORMATe//.'
~L SOLN FOR MEANS')
WRITEC3.190)
MEAN" l
FORMAT!//.·
GROUP
SOLUTIONS FOR THE MEANS
00 29 J=l.K
AM=CSeJ)/2.0)/CS
8~=4.00*cs*eUCJ)+1.00)+1.OO
0079
0080
0081
0082
150
0083
0084
29
0085
0086
0087
160
C
C
0088
0039
300
C
C
C
C
BM=050RTC 8M )-1.0
A14=AM*814
WRITEC3.1501 J.AM
FORMAT(///,I5.3X.F15.6)
UCJ)=AM
SAM=SAM+(NCJ)/AM)*SlJ)
WRrTE(3,loO) SAM
FORMATe//.3X,'CHECKING SUM=·.F15.6l
TESTING ABOUT THE MEANS
VARCOV ~ATRIX FOR MEANS
REAOC1,3001 NRC?NCC?
FORMAT(2I5l
NRC?=NO OF ROWS AND
NCCP=NO OF eeLS OF e-TR~NSPOSED=K
WHERE HO:C-TRANSP*8ETA~GAMMA.BETA
IS
~U-VECT HE~E
118
Appendix 8.1
0090
0091
Continued.
C
0092
0093
4000
310
C
0094
1J095
Y IS C-TRANS MATRIX
C
C
B=CS/12.00*CS+l.00)
00 60 I=I.NCCP
00 70 J=l.NCCP
0096
0097
0098
0099
0100
0101
0102
0103
0104
0105
0106
0107
0108
70
61
00
0109
0110
0111
0112
0116
0117
410
011 a
449
0120
0121
0122
450
0119
0123
0124
0125
012b
0127
0126
0129
0130
420
430
440
v131
01.32
0133
0134
510
013~
0136
v137
0138
0139
500
0140
0141
0142
0143
0144
0145
0146
11147
0148
11149
0150
IF(I.EQ.J)GO TO 61
IFII.LT.J)GO TO 74
ZIIJ-l)*NCCP+I)=(12.00*U(II*UIJ»/NT)*CS*S
ZIII-l)*NCCP+J)=Z(IJ-ll*NCCP+I)
D=II.DO+(2.DO*NIII*CS)/NT).(U(I)*UII»
Z I I I-I ) *NCCP+ 1 ) =a*O/N( 1 )
CALL DGMPRO(Y,Z.wl.NRCP.NCCP.NCCPI
CALL ~GMA6PIWl.y.WZ.NRCP.NCCP,NRCP)
CALL DMP[NVlwZ.NRCP.NRCP.Wl,IRANK.W3)
CALL DGMPRD(Y.U.w2.NRCP.NCCP.l1
CALL DGMSUSlwZ.T.wZ.NRCP,l)
CALL DGMPRD(WZ.wl.T.l.NRCP.NRCP)
CALL DG~PROIT ••Z.AL.l.NRCP.l)
XS=ALI 1)
RANK= ll~ANI<
CALL COTRlxS.RANI<.P.~.IER)
pp=p
o 11:l
0114
0115
00 3111 1= 1 • NRCP
REAOll.400) IYIIJ-l)*NRCP+I).J~l.NCCP).T(I)
FORMATCIOF5.2.F5.2)
CONTINUE
520
:i30
540
Q= l.DO-PP
WRITEI3.410) ALll).IRANI<.Q
FORMATI///.·
LN=' .F2Z.6.·
OF=·.[3.·
P-VALUE=·,FI:>.2)
WRITEI3.449)
FORMATllHI )
wRlTEI3.450)
FORMATI//.·
ASYMPTOTIC vARIANCE COVARIANCE ~ATRlx'/)
L=NCCP+l
YIL*L)=CS*ICS*2+1.DO)/12*NT)
DO 420 1= 1, NCCP
YII*Ll:-CS*CE*U(I)/NT
CONTINUE
00 430 [=1,1<
",RITEI3.440) IZ(IJ-l)*K+[),J=l,l)
CONTl:'>1UE
'WRlTEI3.4401IYII*L),[=I,L)
FORMATI//,2x.8F15.6)
00 500 I=I,K
00 51') J=l,K
HI 1.JI=ZI IJ-ll*K+I)
CONTINUE
HI 1 ,L)=Y( I*L)
HI L oI ) =H( I ,L)
CONTI:'-4UE
HIL,L)-=YIL*L)
00 520 I=l.L
00 520 J=l,l
DIv=HI 1,1 ).Hl J.J)
OIV=O:iQRTIOIV)
Z I I J- 1 I *L + I ) =H I I ,..J) /D I V
CONTI'lUE
wRITE13,530)
FQRMATI//,'
ASYMPTOTIC CORRELATION MATRlx'/)
00 540 I=l,L
wRITEI3.440) (Zl I J-l ).1.+1) .J=l, I)
CONTINUE
END
119
Sample program for solving likelihood equations in normal
populations whose group error variance is proportional
to some unknown power of mean.
Appendix 8.2
C
C
C
C
C
0001
,)002
0003
0004
0005
0.:.106
0007
0008
J009
0010
Jail
OOlZ
0.:.113
001'"
001S
001()
0017
0018
0019
0020
J021
0022
0023
002 ...
0025
,)\)26
00Z7
00,9
lJOZ9
J030
0031
JO,JZ
U033
00.3'"
-.10.35
IJO]o
;,)037
JOJ8
0039
ao'loo
0041
J042
00.. 3
• vo ...
;;045
v046
0047
C
1
:4AXIMUM ~IKELIHCOO SOLUTIONS FOR GROUPS OF NORMAL ~OPULATIONS
~HOSE GROUP ERROR VA~lANCE
IS ~ROPORTIONAL T~ SOME UNKNOWN
PQweR OF THE GROUP ~EAN
•
SOLUTION IS BY FISHER'S ~ETHOO OF SC~RING
WITH ~OOlPICATI0NS
ASYMPTOTIC Oll-SQUAREO TEST ABOUT THE Ml!ANS [5 PROVIOE:;)
IMPLICIT REAL-S(A-H.O-Zl
RE.AI.-e S( 15). T(15) .Z(225) ••111(225) .W.H 225). SLI 15) •
xe(15).PS(15).U(15).w~(225).W4(2251
REAL.'" XS.RANK.P.H
OI!04l!N:iION N(IS)
WRITE(3.1100)
1100 FORMATI//.·
~AxIMUM LIKELIHOOD SOLUTI0NS'/)
wRITEI3.1110)
1110 FORMATI//.'
~ooeL: Y
[S NORMAL wITH ~EANS MU AND VAR1ANC~
1*(MU)**LAM8DA'//)
CALI.. I N1T2<1< .N.S,T ,OL,C,NT)
wRITEt3.1220)
1220 FORMAT(//.'
PRELIMINAAIES'/)
NIT.:oO
",AlTE13.1320)
1320 FORMATI//,'
~ROUP
OBS
XBAA
S2
1
S2XSAR' /)
1'4=1(+1
l. =1'4+1
00 40 .J=-l,K
A=T(.JJ /(N(.J )-1)
",AITEI3,13JO) .J,N(.J),S(.J),T(.JI,A
XBl.J'=S(.J)
40
CONTINUE
1330 FORMAT(lH,lS,2X,IS,2X.3F1S,d)
CS=C**2
.R[TE(J.1390) CS.OL
1390 FCRMATI//,'
INIT[AL VALS: CS=',F18.8,'
OL=',F13.8//J
>'IRtTElJ.S0!55)
50S5 FORMAT(//,'
FIRST P~RTIAI.. JE~lvATrvES EVA~ ~T l.AST IT~~'/)
100
CS=C.-2
OLSaDL**2
ASaO.JO
eSaO.;JO
00 130 Ial,K
A=S([)~-(JL-2.00)
S=A*CS"'OLS
(1-1
.;JO-N(
a+2.aO)
Z(
).L~( ):02
I) *(
~=S([)**(~l.-1.00J
ZIl.~K~I)=4.00.OL.C.N(1).0
~=S([).·2
FaOL.:lCi(E)
) ='1l.*CS*O*N ( [ ) *F
:;a:<8( [)*-2
pst 1 )-:T( I )+G-( 2,oo*xa( I) -S( [» ~~
5L ( 1) =-2. OO-Ol.*CS*N ( t ) *0 +( 2.00 .ot.. *N ( [ ) .PS(
S L ( I ) '" 51.. ( I J + 4 , 00*114 ( 1 ) ~ ( x B I I ) - '5 ( I ) )
Z (L*'''+ {
AS.:oN(t)*PS(t)/IS(I)*~OLI+AS
dS.:oNlI).PS([)*F/(S(I) ••OL)+8S
JO 120 J.:ol,K
1)
/S( [ ) )
C-SOR
120
Appendix 8.2
Continued.
0048
0049
120
130
0050
0051
0052
0053
0034
0055
0056
0057
0058
0059
0060
0061
0062
0063
0004
0005
0006
0067
0068
0009
140
C
C
C
0077
0078
0079
00~5
00d6
0087
0089
00:39
0090
0091
0092
009.3
0094
FS~O.;)O
GS=O.;)O
DO 140 J"'I.K
Z{IJ-l)*L+M)=Z.DO*OL*CS.NIJ)/S(J)
E=S{J).*Z
F"'OLOG (E)
ZI(J-l)*L+L)=ZI(J-l)*L+MI*F
FS=NIJ)*F+FS
GS=N{ J)*F*F+GS
CONT [:-lUE
Z(L*K+MI=4.DO*NT*C
Z(L*K+L)=4.00*C*FS
Z (L*L-l )=CS *FS
Z{L*L)=CS*GS
Z HERE IS THE L BY L SCALED INF MATRIX
SL(LI=-2.DO*CS*FS+(Z.OO*BSI
IOGT=8
CALL LINV2F(Z.L.L.~I.IOGT.W3.IERI
CALL VMULFFlwl.SL.L.L.l.L.L.W2.L.IER)
1112 IS THE INCREMENT VECTOR
C
170
C
C
C
oo . ~o
OO,U
00d2
008.3
00<i4
Z{(J-l)*L+{)=O.DO
SL(~I=-2.;)0*NT*CS+(2.00*AS)
C
C
0070
0071
0072
0073
0074
0075
0076
[FII.NE.J)
CONT INUE
CONTINUE
330
310
99
NIT=NIT+l
DO 17::1 I=I.K
u( I )=WZI I l/S( I)
CONTI NUE
U(MI=lf2(MI/C
UILI=W2IL)/1JL
CALL VA8MXFIU.L.l.J.VMAXI
VMAX IS THE MAX OF THe ABSOLUTe RATIOS OF THE
INCREMENTS TO THEIR CORRESPOND[NG PARAMETERS
TH[S [S THE CONVERGENCE CRITERION
TOL=I.0-8
IFIVMAX.LT.TQLJGO TO 310
00 330 {=I.K
S I [ l=;i I I) "w 2 I I I
CONTI NUE
C=C+lf2( )4)
DL=DL+WZILI
IFIC.LE.TDLI C=C-W21~1
GO TO 100
CONTINUE
00 99 [=1 ,K
UII)=4.00*CS*ISC[I**OLI
5 L I tl = SL I I I / UC[)
CONTINUE
VCMI=~.:JO.CS*C
SLIM)=SLC~)/UCM)
VC LI",·~.DO.CS
SLCLI=SLCLI/UCL)
121
Appendix 8,2
0095
0096
0097
0098
0099
0100
IlI01
0102
0103
01')4
0105
Continued,
150
350
351
4020
4040
4050
~106
0107
0108
0109
0110
0111
0112
011,J
0114
0115
0116
0117
0118
0119
0120
0121
0122
0123
0124
0125
0126
0127
0128
0129
0130
0131
0132
0133
0134
J135
0136
0137
0138
0139
0140
01'1-1
0142
01'+3
0144
l IS THE FIRST PARTIA~ DERIVATIVE OF ~OG ~~HD
EVALUATED AT EACH ITERATION
THIS IS 'ZERO' AT THE LAST ITE~ATION
IlfRlTe(3tl50) (S~(J),J=l,L)
FORMAT(//,2X,dF15.8)
wRITEC3,350 l
FORMA T( 1H 1 )
ilRlTEC3,351l
FORMAT(//.·
FINAL SOLNS BY SCORING METHOO:'//)
wRITE(3,4020)
FORMAT(//.'
GROUP
~EAN'/)
00 4050 l=l,K
WRITE(3,4040) I.S(I)
FORMAT(lH.15,3X.F18.8)
CONTINUE
wRITE( 3,4060) C
FORMAT(//.·
"IL SO~N FOR CV='.FI8.8)
CVSQ=C*C
MRITEC3,40(1) cvsa
FORMAT(//.'
~~ SCLN FOR CVSQ=',F18,8l
ilRITE(3,4070) DL
FORMAT(//.'
ML SO~N FOR ~AMBOA=',FI8,8)
wRITEC3,5000) NIT
FORMATC///,'
NO OF ITER:' .15)
WRITEC3,4000l
FORMAT(//.'
ASY~PTOTIC VARIANCE COVARIANCE MATRIX'/)
00 88 J=l,L
00 88 I=l.~
,
ZCCI-ll*L"J)=Z(( I-ll*~+JI/UCJl
CONTINUE
S~(
C
C
C
4060
4061
4070
;)000
4000
38
CAL~
~INV2FCZ,~,~.WI.IOGT.W3.IER)
00 4080 I=I.L
wR 1 TE ( ,J.l SO)
C WI ( ( J-l I
4080
C
C
C
.~+ 1
l ,J= 1 ,I )
CONTI,~UE
W4 IS THE ASYMPTOTIC VAR-COV
DO 7000 1=I,K
7000 J:l,K
1lf4( (1-1 l*K+J)::WI(
~~TRIX
OF THE MEANS
:)0
7000
CONTI~UE
(I-l)*~+J)
"RITE(J,5041)
EIGENVA~UES OF VAR-COV MATRlx'/)
5041 FORMAT(//,'
CA~~ EIGRFCwl .L,~.2.U,w2,L,wJ,[ER)
IlfRITE(Jd60) (U(2*I-l) ,1=1 .L)
'.RlTE(30179) \113( 1)
FORMAT(//.'
PERFORMANCE 1NDEX='.F12,Z)
179
DO 5020 [':1.~
DO 5020 J<=l,1
VARI="'l « [-1 I *L+ [)
VAR2="'1(J-1l*L+JI
O[V=VARl*VAR2
DIV=OSQRT(:)[ VI
Z ( ( J- 1 ) *~ + [ ) ='01 1 ( ( J- 1 ) *L+ I I /0 I V
5020 CONT INUE
<fRITE(J.5010l
122
Appendix 8.2
0145
0146
01 .. 7
0148
0149
0150
0151
015Z
0153
0154
01~5
0156
0157
0158
0159
0160
01",1
0102
0103
0164
0105
0100
0167
Olod
0169
0170
0171
0172
0173
\l174
0175
0li6
\l1'7
0178
0179
0180
01dl
\lld2
0133
0134
0ld5
\lIdo
01'17
01138
0li9
Continued.
SOlO FORMAHIHI)
IlIRITE(3.5030)
5030 FORMAT(//.·
ASYMPTOTIC CORRELATION MATRIX'/)
00 5040 1=1.1WRITE( 3.160) (Z( (J-ll*L+1) .J=l tIl
160 FORMAT(//.2X.BF15.S)
5040 CONTINUE
AS.aO.OO
8S=0.:)0
FS=O.OO
DO 6010 I:I.K
A=S(! )*S( I)
B=S( I )**01o""s*s ( ! )
G=S*4
DO 6020 J=I. I
IF(I.GT.J) GO TO 6030
E=OL*N(I)/4/2-(OLS+OL)*N(I)*PS(I)/CS/G/2
E=E-Z.OO*OL*N( Il*(XB(I)-S(I)l/CS/O
Zl(J-1)*L+I)=E-Nl!)/CS/8
GO TO 6022
6030 ZIIJ-I)*!.+I)=O.OO
60Z0 CONTINUE
6022 CONTINUE
AS=AS+N( I ) *PSI I) /'3
6S=8S+NlI}.PS(Il*OLOGlS(I)}/8
FS=FS+N(I}*PS(I)*OLCGISII)}*OLOGIS(I»/9
E=-N(I)*OL*PSlI)/CS/C/Q-Z.00*Nll)*IX8(1'-SII})/CS/C/B
Z ( ( 1- 1 ) $I. +M l =E
E=-N(I)/SII)/Z+N(ll*II.OO-OL*OLOG(SlI»)*PS(I)/CS/O/2
E=E-NlI)*OLOGlSlI»*(XBlIl-SlI»/CS/B
Zl I I-I )$I.+L)=E
60 I 0 CONTI NUE
Z(L*K+NI.:zNT/CS-3.00*AS/CS/CS
Z( L*L)=-FS/CS/Z
Z(I-.~+L)=-as/cs/c
6050
6060
o I'JO
6071
0191
0192
\l1-,1,J
01 J4
01'15
0061
C
0196
0197
C
C
DO 0050 J=Z.IJJ=J-I
DO 6050 I=I,.JJ
Zl I J-I )-*L+I I=Z( I I-I )*L+J)
CaNT INUE
.. RITEl3.6060)
FORMAT(//,'
~ATRIX OF SEC P4RT DER EVAL AT ML ESTIMATES'/l
,;)0 6071 1=1,1..
wRITE13.1601 lZIIJ-l)*I.+Il,J=I,I.)
CONTINUE
wR I TEl 3 .606 I )
FORMAT(//,'
EIGENVALUES OF THE MATRIX'/l
CALL EIGRF(Z.L.I..Z.T,WI.L.IlIZ.IER}
.. RITEl3tl601 (TIZ.I-I) .1=1,1..)
·.. RITE(3tl791 '.. Zll)
TESTING HYPOTHESIS ABOUT THE ~EANS
C-ORIME IS THE ~-COL INPUT TEST MATR!X
REAOll,701\l} N~CP
7010 FORI-lAHI5)
OF,SAY,NRC? ROWS
123
Appendix 8.2
0198
0199
0200
0201
0202
0203
0204
0205
0206
0207
0208
02~q
0210
0211
0212
0213
0214
0215
0216
Continued.
~O 7020 I~1 ,NRCP
READ I 1 t 7030' ( Z I ( .1-1 l *NRCP+ I l t,J: 1 t Kl t UI I l
7020 CONTINUE
C
Z HERE IS THE C-pqIME MATRIX
C
U IS THE GAMMA VECTOR OF THE HYPOTHESIS
7030 FORMATI13F5.2.F5.2)
CALL ~GMPRD(ZtW4 •• 1.NRCP.K.K)
CALL OGMABPlwl.Z.w2.NRC?K.NRCP)
CALL ~MPINV(W2.NRCP.NRCP.WltiRANK.w3l
CALL OGMPRD(Z.S.W2.NRCP.K.1)
CALL OGMSUB(W2.U.w3.NRCP,ll
CALL OGMPRO(w3.W1.U.1.NRCP.NRCPl
CALL OGMPRO(U.W3.Z.1,NRCP.ll
XS"'Z(ll
RANK=IRANK
CALL CDTRIXS.RANK.P.H.IERl
pp=p
Q=1.00-PP
wRITE(3.7100) Zl 1 l .I~ANK.Q
LN:t ,F18.6.·
7100 FORMATI///.·
END
OF"", I3.'
P-VALUE=t ,F6.2l
124
Appendix 8.2
Continued.
0001
0002
0003
0004
0005
0006
0007
0008
0009
0010
0011
0012
0013
0014
0015
0016
0017
0018
0019
0020
0021
0022
0023
0024
0025
0026
0027
0028
0029
0030
0031
0032
0033
;)034
0035
0036
0037
0038
0039
0040
U041
0042
0043
0044
0045
00"6
0047
0048
0049
0050
1000
1100
1300
1010
SUBROUTINE INIT2(K.N.S.T.OL.C.NT)
IMPLICIT ~EAL*8(A-H.O-Z)
REAL*a SllS),T(1S),U(lS),QllS)
o U4ENSION N(15)
REAOl 1.1000 I
1(. (NC)
,.J=l,K)
NT=O
C=O.OO
Il/=O.OO
ZL=O.OO
ZU=O.OO
DO 1 0 1 0 .I = 1 • K
NT=NT+Nl.J I
5(.1)=0.00
T( .1)=0.00
NN=N(JI
00 1300 I=l.NN
REAO( 1.1100 I X
FORMAT(II0,1415)
FORMAT(F18.81
S(JI=SIJ)+X
T(.J)=TI.J)+X**Z
T( J) =( T ( J ) - l 5 l J) **2 ) /N ( .I) I /N l .I )
SIJ)=Sl.JI/N(J)
UlJI=T(J)/ISlJ)**2)
C=C+OSQRT(U(J)*NlJII
V=N(JI
'AI=W+OSQRT(V)
ZL=ZL+OLOGlTIJII
ZU=ZU+OLOGIS(JI)
C=C/'II
CS=C**2
DL=ZL-K*OLOGICS)
OL=OL/ZU
C=O.OO
111=0.00
ZL=O.DO
lU=O.OO
DO 1400 J=l.K
U(.J)=TIJI/IS(J).*OLI
C=C+OSQRTIU(JI*NlJ»
v=NIJI
\lI=W+OSQRT( VI
ZL=ZL+DLOGITI.J)1
1400 lU=ZU+OLOGlS(J)}
C=C/w
CS=C**2
OL=ZL-K*OLOGICS}
DL=OL/ZU
RETURN
Er-Cl
125
Sample program for solving likelihood equations in linear
model with error variance proportional to some unknown
power of response.
Appendix 8.3
MAXIMUM LIKELIHOOD SOLUTIONS IN LINEAR ~OOEL wITH
ERROR VARIANCE ~ROPORTIONAL TO SOME UNKNOWN ~OwER
~
OF THE RESPONCe •
C
C
SOLUTION IS BY ~ISHI!R'S ~ETHOD O~ SCORING
wiTH ~DIFICATIONS
ASYMPTOTIC CHI-SQUARED TEST ABOUT THE 8ETAS IS ~OVIOE~
IMPLICIT REAL*e(A-H,O-Z)
REAL*4 XS,~ANK.P,V
REAL*e S(151,Tl901,Zl2251,Wl(22SI,WZ(2251.SL(lSI,X3(90J,-
C
0001
OOOZ
1J003
0004
0005
0006
0007
JOOS
11009
0010
0011
11012
0013
0014
0015
0016
0017
ll<Jld
0019
0020
J021
0022
00Z3
o.oz"
00~5
0026
0027
0028
0029
JO,JO
'"'0.31
J032
,)033
J034
0035
0036
iJ037
iJ033
0039
OJ40
00401
J042
JO .. 3
004"
0045
0046
0047
ilJ4a
C
lU(151,Wa(2251,X(90.151.Y(90J,~(15,151.W4(Z25).Q(2251,O(15).VH(~OI
,.. RITE(3.11001
1100 FORMAT(//,'
~AXl~M ~lKELlHOOD SOLUTIONS'/)
'IlAITEl3.11101
1110 FORMAT(//,'
MODEL: v IS NORMAL wITH MEANS X~RIHe*eETA
ICE C-SQR$(X-PRIMe*eETAI**LAMBDA'//1
CALL INIT3(N.KP.S.C.OL,Y.XI
IfAITE{3.1120 I
llZ0 FORMATl//,'
THE )(-fl4ATRIX AND V-vECTOR'/1
G~O.OO
00 1Z00 lal,N
X8( 1 )-.0.00
00 1209 ,J.al.I<P
X8(II.aXB(II+X(I,~I*S(,J1
1209 CONTINue
~AITE(3,11301 lXlI,~).,J.al,KPI,Y(II
1130 FORMATl/,2X.13FI0.31
G S=GS+OLOG( v ( I I I
YH( I )aX8( [)
1200 CONTINUE
GS2-GS/N
TRF=OEXP(GS)
wAITEl3,12601 TRF
1~60 FORMAT(//.·
TRANSFORM FACTOR=·,FI2.3/1
00 lZ10 [:IIl.N
Y(II=V(II*TRF
1210 CONTINUE
I CHK-=O
NI T=O
',AITElJol3Z01
13Z0 FORMAT(//,'
INOVARNO:
INIRETA 10LSI' .II
)ol=K;:a+l
L"I'4+1
00 40 ,J=l.KP
S(JJ"'S(J)*T~F
~AITEl3.13301
J.SlJI
1330 FORMATl/,I7,12X.F1S.31
40
CONTI:-lue
c=c/(rRF**l~L/2-1.00»
1390
503
100
.R[TElJ.1390J C.OL
INITVALS: C"'·.F13.8.'
wRITe(3.S0S1
FORMATl//.·
OB,JECT FN
ITER
CS=C**2
FOR~AT(//.·
OLS-OL.·Z
00 50 I=l,N
X8(1)"0.00
OL='
.Fl,~. S//)
"~AX'/)
AND VARIAN
126
Appendix 8.3
Continued.
0049
0050
0051
0052
0053
50
1)0~4
0055
0056
0057
0058
0059
0060
0061
0002
0063
0064
00b5
0006
0067
0066
0069
0070
0071
0072
0073
0074
0075
0076
0077
0078
0079
0080
0081
00~2
140
130
120
122
121
0033
00d4
0085
0086
00'37
0088
0089
150
0090
0091
0092
0093
0094
00~5
0096
.:J097
OO<J8
0099
0100
01)1
0102
0103
0104
165
00 50 ,Jal.KP
x8(I)=Xl .,J)*51,J)+XSlI)
CONTINUE
00 120 J=l.KP
00 130 K=t.KP
1145=0.00
00 140 l=l.N
Tl I )='1'1 I )-XSI I)
A=(XS(I)*XBll»**IQL/2)
S=xSll )**2
A5=AS+12.00/A+OL5*CS/B)*X(I.,J)*XlI.K~
CONTINUE
.
WlJ.KlaA5
CONTI NUE
CONTINUE
00 121 J=l.KP
B5=0 • .:>0
FS=O.OO
GS:O.':>O
G=O.OO
DO 122 1= 1. N
T( I )::'1'1 I )-x8( I)
A=(XB(I)*X8(Il)**IOL/21
S=XS( [) **2
SS=B5+xlI.,J)/XBlIl
F5=FS+XlI.,Jl*OLOG(SI/XS( I)
GS=GS+Tll)*xll.,J)/A
E=X( I .,J)*n I I*Tl I I/A/XBl I)
G=G+E
CONTINUE
w(,J.M)::2.00*OL*C*SS
wl,J.L )=OL*C S*FS/2
SLl,Jl=-0L*CS*S5+2.00*G5+0L*G
CONTINUE
00 150 K=l.KP
WlM.Kl=WlK.M'*C
W(L.K)=w(K.Ll*4.DO
CONTINUE
wI M.Ml=4.DO*N*C
A5=0.00
B5=0.00
GS=O.OO
F 5=0.,)0
00 165 1:1.N
A=IXBIIl*XS(Ill**lOL/Zl
S=XBl I 1**2
AS=A5+0LOG(B)*OLCG(BI
F 5=F 5+0LOG lOll
T( 11='1'1 I )-XA( [I
GS:GS+T([I*T(Il/A
B5=SS+T([I*T([)*OLOGlSl/A
CONT[NUE
VLl=-~L*FS/4-N*OLOGlCSl/2-GS/l2.CS)
W(L.L)=CS*45
w(L.~)=4.DO*C*FS
WlM.LJ=w(L.Ml*C/4.00
127
Appendix 8.3
Continued.
0105
0106
0107
alaS
0109
QllO
0111
161
C
C
Z HERE IS THE L BY L
C
0123
0124
0125
0126
0127
0128
0129
0130
0131
C
C
C
INF MATRIX
w2 IS THE INCREMENT VECTOR
170
C
C
C
330
01.32
013.3
;)134
v135
0136
0137
0138
0139
0140
0141
0142
01+3
0144
014;;
0146
::>147
;)148
0149
·)150
0151
SCAL~O
IF( ICHK.Ea.1) GO TO 1220
K=KP
IOGT=8
CALL LINV2FIL.L.L.W1.IDGT.W3.IER)
CALL VMULF.FIW1.SL.L.L.l.L.L.w2.L.IER)
0112
0113
0114
0115
0116
0117
0118
0119
0120
0121
0122
SL(M)=-Z.00*N*CS+2.00*GS
SLIL)=-2.00*CS*FS+2.00*BS
00 161 J=1.L
Z{(J-l)*L+J)=W(J.J)
00 161 l=t.L
IF{ I .NE.J) ZIIJ-l) *L+I )=wl I.,)
CONTINUE
506
310
1230
1240
DC 170 I:1.K
lJ( 1)=W21 I )/S( I)
CONTINUE
U(M)=W2IM)/C
uIL)=IIIZIL)/DL
CALL VABMXF(u.L. 1.J.VMAX)
VMAX IS THE MAX CF THE ABSOLUTE RATIOS OF TH~
INCREMENTS TO THEIR CORRESPONDING PARAMETERS
THIS is THE CONVE~GENCE CRITERION
TCL=I.D-6
IFINIT.Ea.40)GO TO 310
IFIVMAX.LT.TCLIGO TO 310
DO ::l30 I=l.K
S<I)=S(I)+W2II)
IFISII).LE.0.1l00) SII):TOL
CONTINUE
C=C+tI.2(4)
OL=OL+WZIL)
IFIC.LE.O.OOOJ C=TOL
NIT=NIT+l
MR1TEI3.506) NIT.VLl.VMAX
FCRMATI1X.I5.4X.c15.8.3X.EI5.8)
GO TO 100
CONTINUE
DO 1230 I=l.N
YII)=YII)/Tf.!F
CONTI~UE
00 1240 J=1.i<
SI J ):51 J) /TRF
CCNTI:~UE
TRF=T~F**(OL/2-1.00)
C=C*TQF
ICHK=l
GC TO 100
1220 CONTINUE
DO 99 I=l.K
UI I ) =2 • DO *c S
SL( 1 )=SLI I J/U( I)
128
Appendix 8.3
0152
015.3
01540155
0156
0157
0158
0159
0100
0161
0162
;)163
01040105
0166
0107
0168
0109
0170
0171
0172
01'3
01740175
0176
0177
0178
0179
0180
0131
0132
Continued.
99
5055
C
C
C
350
351
4-020
4050
4060
4070
5000
4000
88
018.3
0134
0185
0136
0187
0138
J139
0190
J191
0192
0193
0194-
0195
0190
01~7
0198
J199
J200
0201
C
C
C
4-080
160
CONTINUE
U (MJ==2. OO"CS*C
SI..( 114)==SI..( 104) /Uo-O
U(I..)-=8.00*CS
SI..(I..)==SI..(I..I/U(L.I
IliRITE(.3.S05S)
FORMAT(//.·
FIRST PARTIAL DERIVATIVES EVAI.. AT L.AST ITER'/)
WRITE(3.1601 (SL.(~).~=l,L.)
SL.( ) IS THE FIRST PARTIAL ~ERIVATIVE OF L.OG LLHO
EVAL.UATEO AT EACH ITERATION
THIS IS 'ZERO' AT THE LAST ITERATION
wRITEC3.350)
FURMAT ( IH 1)
WRITEC3.3511
FORMATC//,'
FINAL SOL.NS BY SCORING METHOO:'//)WIHTE{3 ,4020 I
FORMAT(//.'
INDVARNO:
8ETA'/)
DO 4-050 I:l,K
wRITEC3.13301 1,5(1)
CONTINUE
WRITEC3,4060) C
FORMAT(//,'
ML. SOLN FOR CV=' ,F18.81
WRITE(3.4-070) DL
FORMAT(//.'
ML SGI..N FOR L.AII4BOA=',FI8.8)
WRITE(3,50001 NIT
FORMAT(///,'
NO OF ITER:' ,15)
"IRITEC3.4000)
FORMAT(//,'
ASY!4PTOTIC VARIANCE COVARIANCE 14ATRIX'/)
DO 88 J-=I,1..
00 88 1=1,1..
Z( C1-1 )*L+~I=ZC (1-1 )*L.+~)/U(~I
CONTINUE
CALL L.INV2FCZ,I..,I..,~I,IOGT.W3,IER)
00 4-080 1:1,1..
wRITE(3,160)
CW1«J-l)*L+II.~=I,II
CONTI NuE
FORMAT(//,2X,14Fl0.81
\114 IS THE
~SY"'PTOTIC
VAR-CQV
~ATRIX
OF THE SETAHATS
DO 7000 I=1.K
00 7000 J=I,K
1/4«!-I)*K+J)=Wl(CI-ll*L.+JI
7000 CeNT! ,'WE
"IR!TE(3,50411
e:IGENVALUES OF VAR-CQV MATRIX'/l
5041 FORMATC//,'
CAL.L EIGRF{'''1 ,L,L,2,u,w2,L,\II3,IER)
lIRITE(3ol6;)
CU(Z*I-l),I=I,L)
~~ITE(3,17q) \1/3(1)
179 FORMAT(//,'
PERFORM~NCE INOEX=',FIZ.2)
DO 50Z0 I=I,L
DO 5020 J=I, I
VAR 1= ~ 1 ( ( 1- 1 I *L + I )
V AR Z= '" 1 ( ( J- 1 ) *L +J I
:J I V=VAR1*VARZ
DIv=D'iQRT(OIvl
129
Appendix 8.3
0202
0203
0204
U205
0206
0207
0208
1J209
0210
0211
0212
0213
0214
U215
1J216
11217
0218
0219
0220
0221
0222
11223
0224
0225
0220
0227
0228
0229
0230
0231
0232
0233
0234
023~
0236
U237
0238
02-39
0240
024l
0242
021>3
02<1-40245
024b
0247
0248
0249
0250
0251
0252
0253
0254
0255
0250
0257
Continued.
Z ( ( J-1) *1.. +[ ) -=W 1 ( ( J- 1 1*I..·H ) /0 I V
5020 ceNT INUE
1IIIRITE(3.50101
SOlO FORMAT( 1H 1 I
WRITE(3.50301
5030 FORMAT(//.'
ASYMPTOTIC CORREL.ATION MATRIX'/)
00 5040 1=1.1..
WRITE(3.160) (Z(J-l)*l..+11.J=1.I)
5040 CONTI NUE
00 6000 J=1.KP
00 6000 K=1.J
A5=0.;)0
8S=0.':>0
O~O.iJO
E5-=0.00
00 0020 I=1.N
X88=XB ( I) **01..
.0.= x ( I ••J ) . )( ( I • K I / )(8 ( I I / XB ( I I
AS=A+AS
B-=X(I.jl.X(I.KI/XB8
BS=B+'iS
O=B*T( I l/XB( I I
05=0+-)S
E=O*T ( I I / XB ( I I
ES=E+ES
6020 CCNTI:--lUE
w(J.KI=01..*AS/2-BS/CS-Z.DO*OL*OS/CS-OL*(OI..+1.001*~S/CS/ Z
w(K.JI=W(J.KJ
cOO 0 CONTI NUE
00 6010 J=l.KP
GS=O.oo
HS=O.OO
RS=O.':>O
GA5=0.00
HA5=0.DO
DO 6030 [=l.N
X8B-=XB ( I I *'OIOL
G=)( [,J)*T( I I.IXE8
GS=G+GS
H=G*T (I l/XB( I I
HS=H+H5
R=X(I,..J)/X8(I)
RS=R+R5
GA-=G* f)1..0G (XB ( I ) I
GA~GA+GAS
HA=H*DI..CG(XB(II)
HA S-=HA+ HA S
6030 CCNTINUE
~(M,..J)=-2.o0*GS/CS/C-OL*H5/CS/C
)=w( _",.J)
.(I...J)=R5/Z-GAS/CS+HS/CS/Z-DI..*HAS/CS/2
.,(.).I..)=IOI(I..,J)
6010 CONTI;\IUE
FS=O.<JO
P5=0.:>0
05=0.00
-.,( J .....
130
Continued.
Appendix 8.3
02ti8
0259
0260
0261
0202
020.3
0264
026S
0266
0267
1)208
0.209
1)270
0.271
0272
1)273
0274
0275
0276
0.2.77
0278
0279
J280
02'31
1l.2.82
0.2.33
0284
0235
1)296
1)287
0238
0239
0.2.'} 0
02'11
0292
0293
00 ..040 I=l.N
XBS""X8( II**OL
F=T( I I*n I I/xes
FS=F+FS
A=F*OLOG( XB ( I I I
PS=A+PS
8=A*OLOG( XS( I) I
CS=B+QS
6040 CONTINUE
~(M.M)=N/CS-3.00*FS/CS/CS
III (1~.1..1 =-PS/CS/C
WI L.1-4I=w( ~.L)
.~ (I.. .l..I=-QS/CS/2
OC 60,,0 J=l.l..
I=l.l..
Z«J-11*L+II=\lI(I.JI
6050 CONTI NUE
'!flU TE( 3.60601
oObO FCR~~Tl//.· ~ATRIX OF SEC PART ~ER EVAL AT ~L ESTI~ATES'/I
DO 6071 I=l.l..
WRITE ( J. 1 Q 9 I (Z I ( J - 1 I * I.. + 1 I • J = 1 • l.. )
FORMAT(//.2X.14F16.21
169
6C71 CONTI NUE
iIIRITEI3.60&11
EIGENVALUES OF THE ,.-IATR IX'/)
0001 FCR"IAT 1//.'
CAl..L ~I~RF(Z.L.L.2.T.wl.L.W2.1ERI
iIIRITE13.1691 (TI2*1-11.I=1.LI
wRITElJ.1791 "'2(11
WIHTE(J.60801
6080 FORMATI//.·
~REDICTEO VALUES'I
',HI I TE I J .6081 I
60S1 FORMAT( / / . '
YHAT( OLSI
OBS
YHATPILEI '/1
00 6082 I=l.N
',ljRITElJ.60831 I .YH( I I.XSI I I
6082 CONTI.\lUE
6083 FORMAT(lX.I5.~X.F12.3• .3X.F12.3)
00 6050
C
TESTING HYPOTHESI5 AAOUT ~~TAS
C-PRIME IS THE K-COL INPUT TEST MATRIX OF.SAY.NWCP ROWS
REAO( 1.70101 NRCP
7010 FORMAT(IS)
00 7020 l=l.NRCP
QEAO(1.703011Q(IJ-l'*NRCP+l).J=1.KI.0(II
7020 CONTI NUE
a HERE IS THE C-P4IME MATRIX
C
o 15 THE GAMMA VECTOR OF THE HYPOTHESIS
C
7030 FORMATI13FS.2.FS.2)
CALL )G~PRDla.~4.~1.~RCP.K.KI
CALL 0GMAdPlwl.a.w2.NRCP.K.NRC?1
CALL ~MPINV(\II2.NRCP.,\lRCP.\IIl. IRANK.W31
CALL )GMP~D(C.5.M2.NRCP.K.l)
CALL 0GMSUBl~2.0.~J.~RC~.11
CALL )GM~RD(~3.Wl.0.1.NRCP.NRC?1
CALL oJG:-IPRu(Q.';J.Z,l.~RCPol)
XS=Z( 1 )
C
C
0294\.1295
0296
\.12)7
\.1298
0299
0300
0301
1).302
0.303
0304
0305
O.3Jti
0307
0.303
0309
0.310
0311
0312
J313
0.314
R~NK=IRANK
CALL COTRlXS.RANK.P.V.IERl
pp-=p
A= 1 .00-?P
wRITE13.71001 Zlll.IRANK.A
7100 FORMATI///.·
ASY CHI-SORO VAL=' .F12.3.·
1.F6.21
E/I,O
·JF=·. 1 <\0 ••
P-VALIJE=
I
131
Appendix 8.3
0001
0002
0003
0004
0005
0006
0007
oooa
0009
yOLO
;)011
0012
,J013
001'+
0015
0016
0017
0018
0019
J020
v021
0022
0023
0024
002'5
0026
0027
0026
0029
0030
0031
0032
,)033
0034
00.J5
0036
0037
0038
0039
;;<J40
0041
0042
<)043
0044
00 ...5
Continued.
SUBROUTINE INIT3(N.KP,S,C,OL.Y,XI
IM~lCIT ~EAL.a(A-H,a-Zl
REAL*8 Y(901,X(90,IS),S(90),R(13501,U(90).IIY(90),X6(90)
US"O.OO
115"0.00
uvS"O.OO
1155-0.00
ReAO(l,1500) N,KP
1500 FOANAT(215)
C
XMATRIX 15 N X KP
00 1(01) I=I,N
REAC( 1,1(50) (X( I .,J) .,J=1 ,/(Pl .y( I I
S( 1)"Y (1)
1 bOO CONTI NUE
1650 FORMAT(14F5.2)
00 16ol1 ,J=I,KP
00 1601 1=1 ,N
R«(,J-l)*N+l)·XII,,J)
1601 CONTINue
IOGT:aa
CALL U.SQAR (R ,S .N. KP. 1. N.N .10GT •.U. IER)
00 1600 1 =1 .N
ZSzo.OO
00 1900 ,J=l,KP
ZS=ZS+XII,,JI*SI,J1
1900 CONTINUE
X61 1 )aZS
YY=Y( 1 l-x61 11
YY.YV.VY
IF(YY.EQ.O.DO) YY=I.D-4
u<l )=OLOGIYv>
Us=uSfoUll1
ZS=ZS-ZS
RIII=.S*OLOG(ZS)
VS=IIS+R( 1 >
UIiSaUIIS+U(I)*R(Il
VSSaIlSS+R(I >.R(I)
1800 CONTINUE
OL=UVS-<JS*VS/"l
ZS=VSS-IIS*IIS/N
OL=OL/ZS
C-=US/N-OI..*VS/N
C=C/2
c=oeXp(C)
~eTURN
END