t
This research ~JaS supported by the NationRl Ipstitutes of
Health, Institute of GenerA} Hedical Sciences, Grant No.
GH-12868-03 and Institute of Arthritis 1'l.nd Metabolic
Diseases, Grant No. AH-07540-04 •
•
MULTIVARIATE ANALYSIS OF NONLINEAR MODELS
David Mitchell Allen
University of North Carolina
Institute of Statistics Himeo Series No. 558
December 1967
ABSTRACT
ALLEN, DAVID MITCHELL.
Multivariate Analysis of Nonlinear Models.
(Under the direction of JAMES E. GRIZZLE).
Growth curves are usually expressed as polynomial functions of
time.
However, in some instances a more realistic model may require
that the response over time be a nonlinear function of the parameters
and that observations made on the same animal at different points in
time be assumed to be correlated.
One may want to consider several
groups of animals where the parameters may be different for the different groups.
A general model for such data is formulated and two methods
for the estimation of its parameters are developed.
Under the as-
sumptions of normality, one of these methods is shown to yield
maximum likelihood estimates which are known to have desirable large
sample properties, and the other method, which is computationally
simpler, is shown to have the same desirable large sampie properties.
Hypotheses concerning the parameters may be tested using
the likelihood ratio statistic.
An empirical study of a three group asymptotic regression model
shows that the estimates are without serious bias and that the distribution of the test criterion is well approximated by X2 even for
small sample size&.
ii
BIOGRAPHY
The author was born July 15, 1938, in Webster County, Kentucky.
He was reared on a dairy farm near Sebree, Kentucky and was graduated
from Sebree High School in 1956.
He received the Bachelor of Science degree with a major in
dairy science from the University of Kentucky.
active duty in the United States Army.
He served 15 months
He then received the Master
of Science degree in dairy science from the University of Kentucky.
The statistical interests he developed while writing this thesis led
him to continue his studies in experimental statistics at North
Carolina State University where he received the Master of Experimental
Statistics degree in 1966 and completed the requirements for the Ph.D.
in Experimental Statistics in 1967.
During the last two years of his
study at North Carolina State he was a research associate in the
Department of Biostatistics of the University of North Carolina.
then accepted a position as Assistant Professor of Statistics with
the University of Kentucky.
The author is married to the former Hazel Mary Evans of
Henderson, Kentucky.
He
iii
ACKNOWLEDGMENTS
The author expresses his appreciation to his advisor, Dr. James
E. Grizzle , who suggested the topic of this dissertation, and who was
always available for guidance and counsel.
Appreciation is also ex-
pressed to the other members of the advisory committee, Professors
B. B. Bhartacharyya, I. M. Chakravarti, Harry Smith and Oscar Wesler.
All of these men gave assistance and made valuable suggestions in the
preparation of this dissertation.
Mr. Jerry Gentry wrote the original version of program 1 in the
appendix and also served as the author's consultant in computer usage
throughout the study.
Mrs. Gay Goss typed the manuscript.
The con-
tributions of each of these people are greatly appreciated.
Finally, the author expresses thanks to his parents, Mr. and
Mrs. Lilburn Allen, who always emphasized the importance of education and to his wife, Mary, who patiently proofread the manuscript
throughout its development.
iv
TABLE OF CONTENTS
Page
LIST OF TABLES .
vi
LIST OF FIGURES.
. . vii
1.
INTRODUCTION AND REVIEW OF LITERATURE.
1.1
1.2
1.3
1.4
2.
THE MODEL AND THE ESTIMATION OF ITS PARAMETERS
2.1
2.2
2.3
2.4
3.
1
2
4
7
8
8
8
13
17
31
Introduction. . . •
The Gradient Vector.
Preliminary Estimate of ~ .
The Method of Steepest Descent.
The Weighted Linearization Technique.
31
31
33
36
TESTING OF HYPOTHESES AND CONFIDENCE SETS . .
40
4.1
4.2
4.3
4.4
4.5
4.6
5.
Introduction.
. ....
The Model and its Applications.
Es timation of the Parameters . . .
Some Properties of the Estimators
MINIMIZATION TECHNIQUES.
3.1
3.2
3.3
3.4
3.5
4.
Types of Models . .
Estimation. . . . .
Review of Previous Work on Multivariate Methods
for Nonlinear Models.
Objectives of the Present Study . . . .
1
Introduction. • . • • . .
The Problem . . . • . . .
The Likelihood Ratio Test
Distribution of the Likelihood Ratio Test Cri terion .
The Wald Test .
. . . .
Confidence Sets
.
MONTE CARLO STUDY OF THE PROPERTIES OF THE ESTIMATORS AND
TEST STATISTICS. .
5.1
5.2
5.3
5.4
Introduction.
Construction of Data.
Methods of Evaluation
Results of Study . . .
37
40
40
41
43
47
49
50
50
51
52
53
v
TABLE OF CONTENTS (continued)
6.
USE OF THE COMPUTER PROGRAMS AND A NUMERICAL EXAMPLE .
6.1
6.2
6.3
6.4
6.5
6.6
7.
SUMMARY AND SUGGESTIONS FOR FURTHER RESEARCH . .
7.1
7.2
8.
Summary..
.
Suggestions for Further Research.
APPENDICES. . . .
8.1
8.2
9.
Introduction. . . . . • . . . . . • .
General Description of the Programs.
Some Special Features of the Programs
The Subroutine DEFUF.
. •..
The Data Format . . .
A Numerical Example.
Selected Theorems
Computer Programs • .
LIST OF REFERENCES • . .
Page
59
59
59
60
61
62
63
73
73
76
77
77
81
89
vi
LIST OF TABLES
Page
4.4.1
Correspondence between Schatzoff's Notation and the
Notation of this Baper . . • . . • .
47
5.3.1
Hypotheses Considered in the Study.
54
5.4.1
Tests for Unbiasedness and Equality of Means of
Method 1 and M(~thod 2 . . .
. . . . . .
54
Comparison of the Mean Values of the Estimates by
Method 1 and Method 2 with Actual Values of Parameters
55
Number of Likelihood Ratio Statistics Larger than
the Critical Value (a = .05) • . . .
56
5.4.2
5.4.3
5.4.4
Means and Variances of the Likelihood Ratio Test
Criteria. . . . . . . . . . . . . .
5.4.5
56
Chi-Square Tests for Goodness of Fit of the Likelihood
Ratio Test Criteria . . . . . . . . .
. . . •
56
5.4.6
Means and Variances of the Wa1d Statistics.
57
5.4.7
Means and Variances of the Modified Wa1d Statistic. •
57
5.4.8
Comparison of Sample Variance with Average of the
Asymptotic Variance Matrices. . . •
• • .•
58
The Elements of Y' and A' Arranged in the Format
for Program 1 .
.••.
64
6.6.2
The Z' Matrix
66
6.6.3
Data Format for Program 2
69
6.6.4
The Values of
6.6.5
Data Format for Program 2 (Restricted Model).
71
6.6.6
The Values of D(~) and ~ for the Restricted Model
at Each Iteration • . • . • . . . . . . .
72
6.6.1
D(~)
and
~
at Each Iteration.
70
vii
LIST OF FIGURES
Page
3.3.1
Plot of the Z' Matrix of a Hypothetical Experiment . .
35
6.6.1
Plot of the Matrix Z' . . • • . . . . . . • • . • • .
67
CHAPTER L
1.1
~s
INTRODUCTION AND REVIEW OF LITERATURE
of Models
A model, in the statistical sense, is a mathematical expression
which describes the structure of an observation on an experimental
unit.
Examples of models are
8 a
+ 8Za Zi +
l li
y,
1
Y,
1
and
... ,
+ 8P a pi + E: i ,
= exp(8 l a li + 8 Za Zi +
y.
1
a ,
l
8 - 8.8 1 +
1
Z 3
(1.1.1)
(1.1.Z)
E: , ) ,
1
E: i
(1.1.3)
'
where in each case Yi represents the i-th of n observations, ali' aZi'
..• , a
pi
are known independent variables, 8 , 8 ' ... , 8 are unknown
1
Z
p
parameters, and
E:,
1
represents a random variable with mean O.
The model
(1.1.1) is called a linear model since 8 , 8 ' ... , 8 occur in terms of
Z
1
p
the first degree only.
The model (l.l.Z) is nonlinear, but
is linear, and the usual least squares analysis can be performed on the
logarithms of the observations.
The model (1.1.3) is nonlinear; more-
over, no transformation of the observation will make it linear.
Thus,
special techniques are required for the analysis of models of this type.
An alternate method of stating models is in matrix notation.
2
The relation
y
A'
(nxl)
8
+
(nxm) (mxl)
defines a linear model where
~
is a vector of observations, A' is a
known design matrix, l is a vector of parameters, and
random variables with mean O.
(1. L 4)
E
(nxl)
E
is a vector of
Similarly
y
E
(nxl)
(nxl)
(1.1.5)
defines a nonlinear model if the elements of Y(l) are nonlinear functions
1. 2
of.~
and if 1, l, and £ are as in (1.1.4).
Estimation
Estimation procedures for linear models are well known for both
the univariate and multivariate cases.
For example, if the elements of
Y of model (1.1.4) are independent and have equal variances, then the
best linear estimate ofl is produced by the method of
leas~ sgu~~.
The value of 8 which minimizes the sum of squares
is
(1.2.1)
If the elements
Y
are not independent and have variance matrix L, then
the best estimate of l is given by the method of weighted
l~
sguares.
According to this method the value of l which minimizes
(1. 2. 2)
3
is
(1. 2.3)
When E is not known, the practice is to replace it by its unbiased
estimate.
The analysis of nonlinear univariate models has received considerable attention.
Using the representation (1.1.5), almost all of
the methods of estimation require the minimization of
(1.2.4)
with respect to
e.
Draper and Smith [6] describe the basic techniques
to minimize (1.2.4) which are in current use.
They give an extensive
bibliography of work on the general univariate nonlinear model.
An -important general technique for the estimation of the parameters of nonlinear models is an iterative method known as the
linearization
or Gauss-Newton-Raphson method.
At each iteration
K(~)
is replaced by its first order Taylor series expansion about the estimate of
e
obtained from the previous iteration.
model in the form of (1.1.4), and
by the equation (1.2.1).
~
This yields a working
is re-estimated at each iteration
This process is repeated until successive
estimates are sufficiently close.
If the elements of Yare not inde-
pendent, the weighted linearization method can be used.
This method
is the same as the linearization method except that the estimates at
each iteration are obtained in a manner analogous to (1.2.3) rather
than (1.2.1).
A preliminary estimate of
either the weighted or unweighted method.
~
must be available to apply
The weighted linearization
procedure will be discussed in more detail in Chapter 3.
4
Several univariate nonlinear models have received special attention.
Shah and Khatri [19], and Shah and Patel [21] discuss the model
=
y
+
ct
Hx
+ Sp x +
E.
Several authors [5], [11], [12], [23], and [24] have considered
various forms of the asymptotic regression model given by
y
k
t
L: B.p,
ct -
i=l
1
+
(1.2.5)
E.
1
Because of their association with quantal response and with growth, the
logistic model,
Y
=
k/(l + exp( - kbt)) +
has been extensively investigated.
E,
Variations of this model have been
considered by [3], [4], [7], [8], [15], [16], and [20].
Relatively little work has been done in the development of statistical methods applicable to the analysis of nonlinear models when
the observations are dependent.
This dissertation is concerned with
the relatively unexplored area of developing multivariate techniques
for nonlinear models.
1.3
Review of Previous Work on Multivariate Methods for Nonlinear
Models
Beauchamp and Cornell [2] considered the problem of estimating
the parameters of the model
5
Y..
1J
f.(8,t.) +
1 - J
E •• ,
1J
i
1,2, ... ,p,
j
= 1,2, ... ,n,
where the Y.. represent the observations, the f. (8,t.) are nonlinear
1J
1 - J
functions of the elements of
~, ~
is an (mxl) vector of unknown para-
meters, and the t. are (kxl) fixed vectors.
They also assume
J
0,
(1.3.1)
j'!Q"
v iQ,' i, Q,
=
1,2, ... , p
1,2, ... ,n,
j
and that
v = (v .. ) is positive definite.
1J
That is, the observations made for the same t. input vector are correJ
lated, but observations made for different t. vectors are uncorre-J
lated.
If k
=
1 and t is time, observations at the same point in time
are correlated, but observations at different points in time are uncorrelated.
Their technique of estimation proceeds as follows:
i, the Y. ., j
1J
=
1,2, ... ,n are fitted to the f.(8,t.), j
1 -
J
for each
= 1,2, ... ,n
using a modified linearization technique to obtain an estimate of 8
denoted by e(i).
Residuals are calculated by the formula
e ..
1J
A(i)
Y •. - f.(8
1J
1 -
,t.),
J
i = 1,2, ...
,p,
j = 1,2, ...
,n.
6
The estimate of V, denoted V, is given by
and
The quantities Y , f (~,~), and
i
ij
E •• ,
1J
are arranged in vectors
as follows
f
Y
(pn-;l)
=
YIn;
K(§)
(pnxl)
(8,t )
1 - -l
f (~,!y);
l
E
=
E
(pn-;l)
Y
2l
f 2 (§..'!.l)
E
Y
22
f 2 (~'!.2)
E
ln
2l
22
f p (~'!.l)
f p (~'!.2)
Y
pn
f
(8,t )
P - 11
E
pn
7
The model is given by
=
y
where E c..~ £' )
and
~
L:
v
(pnxpn)
(pxp)
!(~
+
~
I
E
(nxn)
denotes the Kronecker or direct product.
The parameters are
estimated using a weighted linearization procedure assuming that
L:
=
V
~
sis tent.
I.
Estimates obtained by this method are shown to be conHypothesis testing is not discussed by Beauchamp and
Cornell.
Khatri [10] considered the model
=
y
(pxn)
B
8
+
A
(pxn) (rxq) (qxn)
'-r
(1.3.2)
E
(pxn)
where Y is a matrix of observations, B and A are known matrices,
e
is
a matrix of unknown parameters, and E is a matrix whose columns are
independently distributed as N (O,L:).
This model represents a signi-
p -
ficant extension of multivariate linear models.
Khatri also derives
the maximum likelihood estimator of 8 as well as likelihood ratio and
largest root tests of hypotheses of the form
C
8
L
(cxr) (rxq) (qxQ,)
1.4
=
o
(cxQ,)
Objectives of the Present Study
Although Beauchamp and Cornell's model is useful in many analy-
ses, the assumptions made about the error structure in (1.3.1) often
are not appropriate.
For example, in the study of growth curves, one
observes several animals each at several points in time.
In this
8
situation it is more reasonable to assume that observations on the same
animal are correlated, but that observations on different animals at the
same point in time are uncorrelated.
This may also be the case when
using compartmental models or experiments using radioactive tracers.
Also the experimenter would like to make inferences about the parameters
in the model.
Khatri's development is applicable to the analysis of growth
curves provided one is willing to assume a polynomial model within each
subject.
The subjects may be blocked according to any design, however.
Unfortunately, many biological systems are known to behave nonlinearly,
and, in this case, it is much more meaningful to use models based on
the underlying physiology rather than strictly empirical polynomial
models.
The purpose of this dissertation is to develop methods for the
analysis of multivariate nonlinear models which are applicable to,
but certainly not restricted to, the study of growth curves that are
nonlinear in the parameters.
Included in these methods will be pro-
cedures of estimation and hypothesis testing.
perform the calculations will be given.
Computer programs to
CHAPTER 2.
2.1
THE MODEL AND THE ESTIMATION OF ITS P APJ\METERS
Introduction
In this chapter the model is defined and examples are given to
illustrate its applications.
Two methods of estimation, of the para-
meters of the model are developed, and some properties of the estimates are examined.
2.2
The Model and Its Applications
Assume that an adequate model for describing the results of an
experiment is
y
(pxn)
F
(8)
(pxq) -
A
+
(qxn)
where Y is a matrix of observations;
F(~)
(2.2.1)
E
(pxn)
is a matrix whose (i,j)
element is
f
j
(8,t.),
-~
i
1,2, ... ,p,
j = 1,2, ... ,q;
A is a known design matrix of rank q; i is a mx1 vector of unknown
parameters whose elements are functionally independent; t. is the i-th
~
fixed input vector; and E is a matrix of random variables whose columns
are independently distributed according to some p-variate distributions
10
with mean
Q and
unknown positive definite dispersion matrix
V
(pxp)
For mathematical reasons we assume that the third order partial
derivatives of f.(8,t ) with respect to the elements of _8 are conJ--:j
tinuous.
We also require that
p+~n
which enables us to derive a non-
singular estimate of V.
To introduce the model, consider some examples.
Example 2.2.1
Suppose that we have
10 animals each of whose response, say
weight, is observed daily for seven days.
weight on the i-th day is given by a - Sp
The expected value of the
i-I
•
This model is called
the asymptotic regression model and is the simplest form of model
(1.2.5).
A polynomial model could be used to describe the response
over the same region.
However, if the asymptotic regression model
fits, we prefer to use it, for unlike polynomial models, the parameters of the asymptotic regression model have definite physical
meanings.
The parameter a represents the upper bound of the weight
of an animal, S represents the potential weight increase during the
course of the experiment, and p characterizes the rate of growth.
The general model (2.2.1) is easily specialized to the asymptotic
regression model.
Let
where Y. represents the seven observations on the j-th animal;
J
11
f 1 (~'-4)
a
-
Sp
i
i-I
1,2, ... ,7,
(aSp) ,
8'
t. = i, i = 1,2, .•. ,7,
""1-
and
A
( I l l I I I I I I 1).
Example 2.2.2
As an extension to example 2.2.1 suppose we have three groups
of six animals such that the expected value of the response is the
same as in example 2.2.1 except that the values of the parameters may
differ among the different groups.
That is, the expected value of the
response of an animal in the j-th group on the i-th day is a
j
i-I
- SjPj .
To apply the general model to this special case let
y
(2.2.2)
where the Y., i = 1,2, ... ,6 are the observations on the animals in
""1group 1, Y. , i
""1-
Y. , i
""1-
=
=
7,8, ... ,12 are the observations on group 2, and
13,14, ..• ,18 are the observations on group 3,
i
j
t.
""1-
i, i
1,2, ... ,7,
= 1,2,3,
1,2, ... ,7,
(2.2.3)
12
and
1 1 1 1 1 1 000 0 0 0 0 0 0 000
A
000 000 1 I I I 1 1 0 0 0 0 0 0
o
(2.2.4)
0 0 0 0 0 0 0 0 0 001 1 1 1 1 1
Note that it would not have been possible to apply the model of
example 2.2.1 to each of the three groups separately because there are
not enough observations within any group to obtain a non-singular
estimate of V.
However this model permits estimation of V as will be
shown in the next section.
Examp Ie 2. 2. 3
Assume the same experimental situation as in example 2.2.2.
Suppose that from knowledge of the biological system we are able to
deduce that the rate of growth and the upper bound of growth are the
same for all groups, and that the initial weights are different for
each group.
The initial weight is represented by the parametric
function a - S, and, since we are holding a constant for the three
groups, differences in initial weight are characterized by differences
in S.
Let the common upper bound be denoted by a and the common rate
of growth be denoted by p.
Then the model is defined by letting
f,(s,t.)
J --.I.
and S'
i
1,2, ... ,7,
j
1,2,3
13
The quantities Y, t., and A are as defined in (2.2.2), (2.2.3), and
-:L
(2.2.4) respectively.
Example 2.2.3 is somewhat analogous to analysis of covariance in
which adjustment is made for different mean initial values.
Example
2.2.3 also illustrates a reparameterization of the model of example
2.2.2, a technique which is used when calculating the likelihood ratio
(LR) test statistics.
Our general model (2.2.1) includes as a special case the model
(1.3.2) given by Khatri.
In that case
F(§) =
B
8
(pxr) (rxq)
and
Y
2.3
B8A + E.
Estimation of the Parameters
First consider the estimation of V.
Let
s
Y(I - A'(AA,)-l A)y',
(2.3.1)
(pxp)
and notice that __
1 _ S is the usual estimator of the dispersion matrix
n-q
of the multivariate analysis of variance model Y
= SA + E.
Theorem 2.3.1
The statistic __
1 _ S is an unbiased estimator of V independently
n-q
of
F(~).
14
Proof:
Let
u'
(lxn)
of
F(~),
~
denote the i-th row of Y,
-.i.
represent the i-th row
(lxq)
s .. be the (i,j) element of S, and v .. be the (i,j) element
1J
1J
of V.
s ..
1J
u~(I - A'(AA,)-lA)u. and
J..
J
E (s .. )
1J
tr«I - A'(AA,)-lA)E(u.u~))
JJ..
tr«I - A' (AA,)-lA)v .. )
1J
(n - q)v ..•
1J
Thus
E(-.L S)
n-q
=
V.
Q.E.D.
Two methods of estimating i are proposed.
(method 1) is to choose as the estimator of
The first method
e that value of
~
which
minimizes
A
Dl (~)
=
log
I (Y
A
- F(])A)(Y - F(])A) ,
I·
(2.3.2)
15
If we have an estimator of
~,
say
~,
then the estimated value of
Y is F(~)A, and thus Y - F(~)A is a matrix of residuals.
1
V =-(Y -
If
e
is close
A
to the actual value of
~,
reasonable estimate of V.
n
F(~)A)
(Y -
F(~)A)
I
should be a
Wilks [26] defines the generalized variance
of p-variates as the determinant of their dispersion matrix.
method 1 yields the value of
generalized sample variance.
e which
minimizes the logarithm of the
Under the assumption of normality
method 1 yields the maximum likelihood (ML) estimate of
proven later.
Thus,
e
as will be
The logarithm of the determinant is used rather than
the determinant because the numbers are of a more convenient size,
minimization procedures are easier to perform, and it is the form
which appears naturally when deriving the LR test statistics.
The second method (method 2) is to choose as the estimator of
that value of
e which
e
minimizes
(2.3.3)
Notice that (2.3.3) is a sum of quadratic forms each of which is
similar to a X2 statistic.
Method 2 may then be considered analogous
to the method of modified minimum X2 •
Note that
tr(S
-1
(Y - F(~)A)(Y - F(~)A)'),
and thus both D (.§) and D (.§) are functions of the matrix.
l
2
16
(Y -
F(~)A)
(Y - F(S?) A) ,
z
YA'(AA,)-l,
Let
(pxq)
and
X(8)
(pxq)
Z-F(5!);
we can then simplify (2.3.4) as follows:
F(~)A)
(Y -
F(~)A)'
(Y -
(Y - YA'(AA,)-lA + (Z - F(~»A)(Y' - A'(AA,)-lAY' + A'(Z' - F'(~»)
s + (Z -
F(~»AA'(Z
s +
X(~)AA'
-
F(~»'
(2.3.5)
X' (]).
This simplification is important since Z and S are the estimates of the
parameters in the linear multivariate analysis of variance using the
model Y
= SA +
E.
By going through the preliminary step of computing
Z and S, the number of input variables for the nonlinear estimation is
considerably reduced.
Now we will examine D
l
(~)
and
D2(~)
using the representation
(2.3.5)
log
and
Is
+ X(~)AA' X' (~)
I
17
tr(S-l(S + X(~)M'X' (~))
=
p + tr(M'X'(~)S-lX(§».
Since the function tr(M'X'(~)S-lX(~» is a monotone increasing
function of D (]) and is of simpler form, D (]) is redefined as
Z
Z
follows:
Z.4
Some Properties of the Estimators
Assume that the columns of Yare independently distributed as
p-variate normal random variables.
The joint density of the elements
of Y under this assumption is
n
'V-liZ
1
= ~ exp{- - tr((Y - F(Q)A) 'v-ley - F(§)A»}
!r2.
Z
(Z1T)Z
n
IV-liZ
=~
!r2.
1-1
exp{- -tr(V (Y - F (Q) A) (Y - F(~)A')}
Z
(Z1T) Z
n
- 1 (S + (z = ~ exp{- -1trV
!r2.
Z
F(~»M'(Z
-
F(~»')}
(Z1T)Z
(Z.4.l)
18
n
~
!!Eo
(2rr)2
exp{- -1
(2.4.2)
2
Applying the Fisher-Neyman Criterion to the form (2.4.1) we see that
Z and S are sufficient statistics for
e and V.
Thus there is no loss
of information by restricting our search for estimators of
e and V to
functions of Z and S.
Theorem 2.4.1
If the columns of Yare independently distributed as p-variate
normal distributions, and E(Y)
i, say
~l' such that Dl (~l)
with respect to
(1)
~l
(2) V
=
= F(&)A, and if there exists a value of
log
Is
A
A
+ X(~l)(AA')X' (~l)
I
is minimum
i, then
is the maximum likelihood (ML) estimator of
i;
= ~(S + x(il)AA'X'(il » is a ML estimator of V.
Proof:
The method of proof is to show that
log ~(!l'V) - log ~ (i,V) ~
a for any value of
~
log ~ (§.l'V) =
- ~ log 2rr - !!. log
2
and
2
11.(s
+ X(8-1 )AA'X' (8--1 ) I
n
_!!Eo ,
2
and V.
Now
19
log l/J (§.., V)
- jE- log
2rr -
~ log Ivi -
t
tdv-l(s +
X<,,~)M'X' <.~»},
(2.4.3)
and thus
1
+ 2" tdV
-1
(S
+ X(.2)M' X(§..» }.
(2.4.4)
Notice that
-
~
log Iv-
1
~(S +
I-~
X(§..)AA'X' (§..»
+ ~ log I~(S + X(~)AA'X' <..~»
I ::
log Ivl
(2.4.5)
O.
Adding expressions (2.4.4) and (2.4.5) we obtain
A
log l/J (§..1' V) - log l/J (8, V)
~ (log
[-%p - t
log
Is
+
X(~AA' X' (~I
1
~(s + X(~AA'X' (~) I + t
Iv-
-
log
Is
+ X(i1 )AA' X(i1 )
tr(V-
1
I~ +
~s + X(~AA'X' (~)J
.
(2.4.6)
The first bracketed term on the right hand side of (2.4.6) is
non-negative by hypothesis.
term is non-negative.
It remains only to show that the second
In order to do this we parallel a technique
originated by Watson as described by Rao [17, p.449].
denote the characteristic roots of V-
1
~(S
+
n
Let A ,A , .•. ,A
l 2
p
X(8)AA'X'(8».
-
20
By theorem 8.1.2 of the appendix we see these roots are all nonnegative.
By theorem 8.1.1 of the appendix the second bracketed term
of (2.4.6) is
.!l(
2
-
-
p
p
L log A.
p
n
2
L (A.
i=l
1
i=l
1
-
P
L
+
i=l
log A.
1
-
A. ) =
1
1),
(2.4.7)
and by theorem 8.1.4 of the appendix every term of the sum (2.4.7) is
non-negative.
Q.E.D.
Theorem 2.4.2
(1) I f q
1 then method 1 and method 2 are equivalent.
(2) If q > 1 method 1 and method 2 are not necessarily equivalent.
Proof:
Let Al <.~) ,A 2 <..~) , ••• ' \ <..~) represent the characteristic roots
of AA'X'
<,~)S-lx<.~).
Note that
q
L
i=l
A. (8) •
1
-
(2.4.8)
Note also that each of the following functions is a monotone increasing function of the preceding function, and thus the minimization of one implies the minimization of the others as follows:
21
D (.§) = log
l
Is
+
Is
+ X(5l) AA I Xl
X(§)AA I Xl
(iL)
<..~)
I
I
(by theorem 8.1. 5)
q
IT (1+ A.(8)).
i=l
1
(2.4.9)
-
I f q = 1, D2(~) is simply Al (.§), and 1
increasing function of D (.§).
l
+ Al (~ is a monotone
Obviously, then, D (i) is a monotone
l
increasing function of D (.§) , and both are minimized by the same value
2
of 8.
a monotone increasing function of D (iL).
l
of i, say
Suppose there are two values
i l and §.2' such that Al (il ) = 5, A2 (il ) = 5, Al (i2 ) = 1,
and A (i ) = 10.
2 2
Note that
A
36 = (1 + Al (iLl)) (1 + A2 (il )) > (1 + Al (i2 )) (1 + A2 (i2 )) = 22
but that
Thus D (iL) is not a monotone function of D (~), and the minimization
2
1
of D (.52.) does not imply the minimization of D (iL).
2
l
Q.E.D.
22
Let
1 be a vector of dimension m + pep + 1)/2 whose first m
~
elements are those of the vector
and whose next pcp + 1)/2 elements
-1
are the independent elements of V
It is well known that lower
bounds for the variances of an unbiased estimator of
~
are given by
*-1
the diagonal elements of the leading mxm minor of V
(~,V) where
B* (~,V) is a matrix whose (i,j) element is
_E(a 2alog ~ (P»)
<P 1J <P
i
1, 2 , ... ,m + p (p + 1) /2 .
i ,j
The matrix
j
B*(~,V) is called the information matrix and B*-l(~,V) the
asymptotic variance matrix of the estimates of the parameters.
Lemma 2.4.1
The matrix B* (~,V) is a block diagonal matrix with two
blocks, the first of dimension mxm and the other of dimension
pcp + 1)/2 x pcp + 1)/2.
Proof:
The method of proof is to show that the expected value of the
cross partial of log
e
~(~,V)
with respect to an arbitrary element of
-1
and an arbitrary element of V
log
- ?- log
Let
6.(F(~)
~
2IT -
~
log
~
is zero.
(~, V)
Ivi - %tr[V-\S
+ X(~M'X' (~].
represent a matrix whose elements are the partial deriva-
tives with respect to
e.
~
of the corresponding elements of
F(~).
23
By theorem 8.1.3,
a 10g1jJ (.§., v)
a8.
1
-1
- 2."
t r [V (\ (S + x(.§.) AA ' X' (.§.) ) ]
1
= tr[V- 1 8.(F'(8))AA'X(8)].
1
2
Thus a
log 1jJ
avjka8.
(2.4.10)
is the (j,k) element of 8.(F' (8))AA'X(8) where v jk
1
-
-
1
-1
is the (j,k) element of V .
It follows that
since
E (X(.§.))
= ..Q.
Q.E.D.
As a result of Lemma 2.4.1 we can obtain the leading mxm minor
of B*-1 (.§.,V) by inverting the leading mxm minor of B*(.§.,V) which we
denote as
B(~,V).
To determine
B(~,V)
we have from (2.4.10) that
a log W = tr[V-18.(F'(8))AA'X(8)].
a8.
J
J
24
Thus
a
2
log l)J
ae. ae.
J
J
1
1
The (i,j) element of
_ E(a
1
tr [V- 8. (8. (F' (e»AA' x(e»]
_
B(~,V)
2
-
is
1
tr[v- 8.(F'(e»AA'8.(F(e»]
log 1/J)
ae.ae.
1
-
J
J
1
1
tr[AA' 8. (F(e»V- 8. (F' (e»].
1
J
-
(2.4.11)
-
Let
K(jz)
=
(2.4.12)
(pq xl)
F (e)
-q-
where F. (jz) represents the i-th column of
-'1
F(~),
and let
u<.~)
(2.4.13)
(pqxm)
be a matrix whose (i,j) element is the partial derivative of the i-th
element of
K(~)
with respect to e ..
J
Notice also that the j-th column
of U(e\
can be represented by 8.J(F(e».
::!../
-Lemma 2.4.2
Let
by (2.4.13), then
B(~,V)
be as defined by (2.4.11) and
U(~)
as defined
25
where 9 denotes the Kronecker product.
Proof:
Let
ck~
denote the
(k,~)
element of AA'.
The (i,j) element
of U'(8)(AA'~V-l)U(8) is
tr[AA' O. (F' (8) )V-lo. (F(8»]
1J-
which is the same as (2.4.11).
Q.E.D.
We have shown that method 1 yields maximum likelihood estimators
which are known to be consistent, asymptotically efficient, and asymptotically N (8,B
m-
-1
(8,V».
-
We now define some notation and present a
lemma in order to prove that method 2 yields estimates with the same
p rope rties •
Suppose that we have k independent replications of an experiment
with model (2.2.1).
Let
Y. represent the matrix of observations and
1-
(pxn)
E.
1-
represent the matrix of random errors of the i-th experiment,
(pxn)
i=1,2, ... ,k.
26
Let
Y(k)
(pxkn)
[AA ... A],
A(k)
(qxkn)
and
E (k)
=
[E l E ... E ] .
2
k
(pxkn)
The observations from all k experiments can be represented together
in one model as follows:
or alternatively as
Let
and
Now let
1
Y(k)
=-
k
E Y. ;
k i=l ~
it is then easily verified that
-
= Y(k)A'(AA ' )
-1
,
27
and that
By Ko1mogarov's Strong Law of Large Numbers and by the Mann-Wa1d
Theo rem [13]
Z
Prob. 1) F ( e)
(k)
-
(2.4.14)
,
and
1
Prob. 1
k"S(k)
as k
700.
(2.4.15)
) nV,
Our minimization criterion
D2(~)'
based on the k experi-
ments, is
~
1
-1
tr(AA'(z(k) - F(~»(ii'(k»
(Z(k) - F(~».
Let
&(~)
be a vector whose i-th element is
g. (e)
1.-
(2.4.16)
i
= 1,2,oo.,m.
28
Let
~l(k)
~(k)
(2.4.17)
~2(k)
=
~(k)
where
~(k)
is the i-th column of Z(k) and let
(2.4.18)
where L(e) is given by (2.4.12).
Lemma 2.4.3
Proof:
nU'
Let
(~) (M ~
I
Let
ck~
~(lO
be as defined in (2.4.16), then
denote the
(b (k)) -l!(k)
(k,~)
element of AA ' .
The i-th element of
(e) is
which is the same as (2.4.16).
Q.E.D.
29
Let
W(~)
be a matrix whose (i,j) element is
2
n
2
W.. (6)
J.J -
a D2k(~
a6.a6.
J
J.
(2.4.19)
- n
i,j
1,2, ...
,m.
Let
C·.
(6)
n
1J'" -
3
a
D2k (.§.)
2 a6 n a6.a6.
n
=-
'"
J
1
(2.4.20)
A
Let
R(~,~)
be a matrix whose (i,j) element is
1
r .. (6,6)
1J - -
= -2
m
L
A
C.. n (t.6
£=1 1J'" J.-
+
(1 - t)_6)(6 n
where
o < t. < 1,
1
i,j = 1,2, ... ,m.
...
'"
-
6 n ),
'"
30
By (2.4.14) and (2.4.15)
l)
)-1) Prob. 1) U' (8)(AA,avn U ' (8)
_ (AA,a(L
~ k~(k)
_
~
,
(2.4.21)
W(~ Prob. 1) _ B(~,V)
(2.4.22)
C. -n Prob. 1) (some finite constant)
1J'"
(2.4.23)
and
i,j
as k -+
1,2, ...
,m
00.
Theorem 2.4.3
Method 2 yields estimates which are consistent,
totically efficient, and asymptotically N (8,B
m-
-1
asymp~
(8,V».
-
~
Proof:
The value 8 which minimizes
D2k(~)
is a solution to the
equations
o.
(2.4.24)
By (2.4.14) and (2.4.15), if k is sufficiently large then equations
(2.4.24) become
which have the solution
~ =~.
This proves consistency of 8.
We now replace the equations (2.4.24) by their Taylor series expans ions about 8 which gives
~
.& (~)
+
W(~ (~ - ~
~
+
R ( 8 , 8)(~ - ~) = 0,
31
or
~
+
[W(~)
A
R(8,8)](~
-
~)
- ..8.(JD.
(2.4.25)
By consis tency and (2.4.23)
R(~,i) Prob. 1)
0
(2.4.26)
(mxm)
as k
~
00.
The solution of (2.4.25) is
We know that
Ik -X(k)(8)
-
is distributed as N
(O,(AAI)-l~v.
pq -
Thus by the Mann-Wald Theorem and (2.4.21),
liZ nU' (~.)(AA '9
(b (k)) -l)X(~)
=
liZ ..8.(JD is asymptotically distri-
buted as
By the Mann-WaldTheorem, (2.4.22), and (2.4.26) we have that
liZ
(i - JD
=
/k [W(~) + R(~,
e) ]-1..8.(8)
is distributed asymptotically as
This proves both asymptotic normality and asymptotic efficiency.
Q.E.D.
CHAPTER 3.
3.1
MINIMI ZATION TECHNIQUES
Introduction
This chapter is concerned with the mechanics of the minimiza-
tion of D
l
(~)
and D (~).
2
Two methods are discussed; one is the
method of steepest descent which can be used here without modification, and the other is a weighted linearization procedure.
considering
Dl(~
and
D2(~
When
the subscript will be omitted when the
comment applies to both functions.
3.2
The Gradient Vector
The gradient vector of D(l) is defined as an mxl vector whose
~-th
element is
dD(l)/d8~
where
8~
is the
gradient vector of D(l) is denoted by
respect to l
then &(l)
~,
say ll' such that
~(l).
element of 8.
The
If D(l) is minimum with
= Q, and thus we can restrict our search for
estimators to values of l such that
of
~-th
~(~l)
=Q
~(~)
= O.
Having found a value
we can examine D(l) at the points
m
of a 3 factorial design within a small neighborhood of ~l and with
m
If we find that D(~l) is the minimum of these 3
center point ~l.
values of
minimum at
D(~),
~l.
8 such that
Let
~(~
then we can be confident that
D(~)
attains a local
Both of the methods we discuss are designed to find
= Q but they do not guarantee that
8~(F(~)
D(~)
is minimum.
represent a matrix whose elements are the partial
derivatives with respect to
8~
of the corresponding elements of
F(~.
33
Applying theorem 8.1.3 <See appendix) the
t r{ <S
+
X (~) AA' X'
~-th
element of the gradient
<i) )-1 <5 ~ <X (~) AA ' X' <i) )}
=
(3.2.1)
The
~-th
element of the gradient vector of D (~) is
2
(3.2.2)
3.3
Preliminary Estimate of 8
The methods of steepest descent and linearization are iterative
and both require a preliminary estimate
of~.
Before discussing
these methods it should be pointed out that the success of either
method depends to some degree upon the selection of this preliminary
estimate of 8 and that if the starting value is close to a local minimum, the procedure may lead to convergence to the local rather than
the global minimum.
This section is devoted to finding a value of
denoted 8 , which is likely to be an adequate starting value.
-0
~,
34
First, compute Z and assume that it is equal to its expected
value F(iU.
It may be possible to find functions of the elements of
Z which equal the parameters.
Suppose
and
E(Z' )
we see that 8
is estimated by zl and that 8
1
2
is estimated by
log z. - log z.
]
~
i>j
t. - t.
~
J
0,1,2, ... ,5.
We would expect that zl would be adequate for a preliminary estimate
of 8
1
and an average of several or all of the estimates of 8
be adequate for a preliminary estimate of 8 .
2
2
to
However, there are
many cases where this method will fail but a knowledge of the physical interpretations of the parameters helps in the search for a
starting value.
Consider another example
and a plot of the response is given in figure 3.3.1.
FIGURE 3.3.1 PLOT OF THE Z' MATRIX
OF A HYPOTHETICAL EXPERIMENT
eZ3
.Z4
eZ2
LLJ
~
Z
eZ
5
.Z6
c>O
(l.
eZ
~
LLJ
7
.Za
a::
tl
t2
t3
t4
t5 t 6
TIME
f7
fa
.z9
t9
·z 10
tlO
36
It is known that 8
1
represents the amplitude of the function or the
greatest distance of the curve from the time axis.
Both max(z.) and
•
1.
1.
- min(z.) estimate 8 and their average (max(z.) - min(z.))/2 would be
1
i
1.
i
1.
i
1.
a logical choice for a preliminary estimate of 8 .
1
this is (z4 - z9)/2.
We know that sin(O)
=
largest value of x such that sin (x)
function is 0 at approximately (t
0 is IT.
and that the next
In our example the
+ t )/2 and the argument of the
7
6
t
sin function at that point is 8
o
For our example
6
2
+
t
7
2
Thus we see that
2IT
t
6
+ t
7
provides an estimate of 8 •
2
3.4
The Method of Steepest Descent
The classical method of steepest descent can be used to minimize
D(~)
modification.
on which
D(~)
The essence of the method is to follow the line
descends most rapidly starting from the estimate given
by the previous iteration.
i-th iteration.
Let 8, be the estimate obtained from the
""1.
The line through 8. on which
""1.
D(~)
descends most
rapidly is defined by
where A > O.
The next estimate,
~+l'
is given by
8.
~+l = ""1.
where A is chosen such that D(8.
m
to A.
""1.
A g(8,)) is minimum with respect
m
-:L
The process stops when we find 8 such that
K(~)
O.
37
3.5
The Weighted Linearization Technique
Another method available is that of weighted linearization.
!.
Let
be as defined by (2.4.12) and
(~
(pq xl)
defined by (2.4.12).
U
(pq xm)
(~)
be as
Let Z be as defined by (2.4.17) and
~(~)
be as
defined by (2.4.18) except that here k=l; so we omit the subscript.
We now have the model
Z
!.(~)
+
(3.5.1)
E:
where
a
-'
and
At each iteration,
K(~)
of (3.5.1) is replaced by its first
order Taylor series expansion about the estimate of
the previous iteration.
e
obtained from
At the i-th iteration, this gives the model
or
(3.5.2)
where
~-l
and S. 1 =
-:L-
is the estimate of
e - e.
-
~-
1.
~
provided by the (i-l)-th iteration
The model (3.5.2) is of the form (1.1.4), and
we estimate S. 1 by the relation (1.2.3), replacing E by an estimate
~-
E.
Let
38
X(~_I)AA'X'(~_l)
where V is proportional to S +
if we want estimates
by method 1, or proportional to S if we want estimates by method 2.
Thus
(3.5.3)
8.1.- 1
and
~
= ~-l + ~-l·
The iterative process is continued until we obtain a value of
~
which is essentially null.
We see that according to (3.5.3) 8
i
=
0
implies that
(3.5.4)
In a manner similar to the proof of lemma 2.4.3 the t-th element of
(3.5.4) is seen to be
(3.5.5)
~
If V is proportional to S + X(8.)AA'X'(8.),
then (3.5.5) is propor--:L
.-'1.
tional to the t-th element of the gradient vector of D
l
given in (3.2.1).
(~)
as
If V is proportional to S, then (3.5.5) is pro-
portional to the t-th element of the gradient vector of
D2(~)
as
given in (3.2.2).
Thus if our iterative procedure converges, the
gradient vector of
D(~)
is null.
The estimate of V, V must be cal-
culated at each iteration when using method 1, but when using method
2, V remains constant throughout the iterative procedure.
39
In univariate nonlinear analysis, it is known that the method of
steepest descent is fairly good initially when the starting point is
far from the nearest minimum, but as the provisional value nears the
minimum other search procedures are superior.
Linearization, on the
other hand, is very good when the provisional value of
the nearest minimum.
plications due to
U(~)
e
is close to
However, the method may break down in some apbecoming almost singular.
One would expect to
encounter the same problems in multivariate nonlinear analysis.
One
of the main advantages of the linearization method is that the computations at each iteration are similar to regression, and useful
statistics are produced as a byproduct.
Smith and Shanno [22] and
Marquardt [14] have proposed algorithms which combine these two
methods in an effort to realize the better features of both.
Their
methods can be adapted to the multivariate case, but it is not our
purpose to pursue the computational aspects that far.
We shall use
the linearization technique in the numerical work presented in
Chapter 5.
CHAPTER 4.
4.1
TESTING OF HYPOTHESES AND CONFIDENCE SETS
Introduction
The estimation of
~
has been discussed in Chapter 2.
In this
chapter we shall examine methods of testing hypotheses about 8 and
establishing confidence sets for 8.
Throughout this chapter we assume
that the columns of Yare normally distributed.
The Likelihood Ratio
(LR) procedure is developed because it has several desirable properties.
However, LR statistics are difficult to compute in many in-
stances, and for this reason a computationally simpler procedure
based on a theorem proved by Wa1d is discussed.
Exact confidence
sets are given for the case where q = 1, and asymptotic confidence
sets are given for q
4.2
>
1.
The Problem
Suppose that we have k functions of l, Gi(l), i
=
1,2, ••. ,k,
and we want to test the null hypothesis
G.(8)
1
-
=0
for all i
1,2, ... ,k
against the alternative hypotheses
G.(8) I 0 for at least one i
1
Let
-
1,2, •.. ,k.
G (8) be a vector whose i-th element is G.(8).
(k;-l) 1 -
Restating the
problem in vector notation, we want to test the null hypothesis
41
against the alternative hypothesis
Q(~
Assume that the elements of
are functionally independent and
that all the second order partial derivatives of G.(8) with respect
~
to the elements of i
4.3
are continuous for i
=
-
1,2, ... ,k.
The Likelihood Ratio Test
Let ¢ be the joint parameter space of 8 and V and let
The joint dentisy function
~
(i,V) is as defined in (2.4.2).
The LR
statistic is given by
max ~(i, V)
iZ.,VE:<!>l
max ~ (i, V)
i, VE:<!>
By theorem 2.4.1,
e- !!:Eo
2
1
(ZIT)
T- (~~o.I~(s
+
X(Q)AA' X'
(~_)) I) ~
(4.3.1)
42
Wald [25] has shown that the LR test has desirable large sample properties.
A disadvantage of this method is that the calculation of both
the numerator and denominator of (4.3.1) often requires a separate
iterative process.
A test of particular importance is that of "fit of model",
i.e. the test of the null hypothesis
H :
O
=
E (Y)
F(E))A
against the alternative hypothesis
s
(pxq)
where S is a matrix whose elements are not restricted.
The likelihood ratio in this case is
_(
A
~~
l.§J
-,Is + x(ilM'x'(ill)
where 8 is the ML estimator of 8.
The usual test statistic based on the likelihood ratio is of
the form
-
m
l
1 og
2
,n
(4.3.2)
/I.
Selection of m and the small sample distribution of (4.3.2) are disl
cussed for certain special cases in the next section.
A useful com-
puting relation is
2
A
log An
=
Dl(i) - Dl(i ),
l
(4.3.3)
43
where
e
is the ML estimator of
the restricted parameter space
4.4
£, and
~1
e
is the ML estimator of
in
~1'
Distribution of the Likelihood Ratio Test Criterion
In linear multivariate analysis of variance it is well known
2
(see Anderson [1, p. 208]) that -m log An is approximately distri1
buted as X2 where
(4.4.1)
and where n
1
is the error degrees of freedom, p is the number of
variates, and q1 is the hypothesis degrees of freedom.
The purpose
of this section is to establish a correspondence between our model and
the linear multivariate model for the purpose of obtaining multipliers
analogous to (4.4.1).
The development given here is not a proof,
but we shall attempt to verify our result empirically in Chapter 5.
For this discussion we restrict ourselves to the case where
m
=
rxq, and where the elements of
denoted
e
~
can be arranged in a (rxq) array,
such that II (~ depends only on the first column of
(rxq)
e,I2 (£)
depends only on the second column of
e,
and so forth.
We
assume that all elements in the same row have the same functional
relationship to their respective columns of
F(~).
Such an arrangement
for example 2.2.2 is
(4.4.2)
44
Further, we restrict ourselves to the hypothesis that the model
fits and to hypotheses of the form
H
8
L
(hxr) (rxq) (qx£)
where h
~
r, £
~
(4.4.3)
y
(hxt)
q; Y is a known matrix; and Hand L are known
matrices of full rank.
We can linearize each column of F(i0 about the same r-dimensional vector and obtain a model of the form
y
*
(pxn)
B
8
A
(pxr) (rxq) (qxn)
+
E
(pxn)
which is seen to be the model of Khatri given in (1.3.2).
We now relate Khatri's model to the linear multivariate model.
Let
and
B
be matrices of full rank such that
B
2
l
(pxr)
(pxp-r)
B'B
1
=
I
(rxr)
and
B'B
2
o
=
(p-rxr)
Let
=
B'y *
1
and
Z2
(p-rxn)
B'y *
2
(4.4.4)
45
which gives the model
A
[:]
+ E
(4.4.5)
where
s
=
O.
(p-rxq)
We can test the adequacy of model (4.4.4) by testing the hypothesis
H:
S
(p-rxq)
0
in the linear multivariate model (4.4.5).
We see that the error
degrees of freedom is n-q, the effective number of variates is p-r,
and the hypothesis degrees of freedom is q.
The appropriate statis-
tic to test fit of model is then
Z
-(n-q-
~(p-r-q+l» log An
which is distributed approximately as X2 with (p-r) q
(4.4.6)
pq-m degrees
of freedom.
Having accepted model (4.4.4), we factor the joint density of
Zl and Zz into the product of the marginal density of Zz and conditional density of Zl given ZZ'
n(p-g)
Z
(ZIT)
This result is given by
46
n
Iv~ll
2
(4.4.7)
!!'<l.
(2IT)2
where
v2
(p-q xp-q)
and
n
(qxp-q)
It is evident from (4.4.7) that the ML estimator of
hood ratio statistics for hypothesis concerning
~
e and the like li-
can be obtained
using the conditional density of Zl given Z2' since the marginal
density of Z2 assumes the role of a constant.
Notice also that the
conditional distribution of Zl given Z2 is a linear multivariate
model
=
[e n J
+
E
(4.4.8)
(qxn)
(qxq+p-r)
From (4.4.8) we see that the error degrees of freedom is n-q-p+r.
From (4.4.2) we see that the effective number of variates is h, and
the hypothesis degrees of freedom
is~.
The appropriate test statis-
tic for testing the hypothesis (4.4.3) is then given by
2
-(n-q-p+r)
t(h-~+l» log An
which is approximately distributed as X2(hx~).
(4.4.9)
47
Schatzoff [18] provides tables from which the exact percentage
2
points of the distribution of -m log An can be computed when the model
is linear.
It is suggested that his correction is appropriate for the
statistics (4.4.6) and (4.4.9).
Table 4.4.1 gives the correspondence
between his notation and the notation used here.
Table 4.4.1.
Correspondence between Schatzoff's Notation and the
Notation of this Paper
Description
Hypothesis
H
e
L
Schatzoff
Fit of Model
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _...->(..:.;;h"-?<.=.r__
) (rxg) (9 ~~)
k
Likelihood ratio
!I. 2
Ertor d.f.
n
Hypothesis d.L
q
Number of variates
p
*
n-q-p+r
n-q
p-r
h
q
*k is the number of observations for which Schatzoff does not have a
notation.
4.5
The Wald Test
To calculate Wald test statistics [25] we need to evaluate the
-1
asymptotic variance matrix B (~,v)
by (2.4.11) at the ML estimates of
Let
H
(~be
=
~
-1-1
(U'(~)(AA'@V
and V.
a matrix whose (i,j) element is
(kxm)
aGo (e)
i
1,2,
,k
ae.
j
1,2 ,
, m. '
~-
J
)U(~»
defined
48
The Wald Statistic for the hypothesis of section 4.2 is given by
(4.5.1)
W
This statistic is approximately distributed as X2 (k), and this test
is asymptotically "power-equivalent" to the likelihood-ratio test.
The remarkable feature of this test is that B-1Ci,v) has already been
calculated during the process of estimation and that any number of
reasonable hypotheses can be tested with relatively little calculation.
The most important special case takes a particularly simple
form; suppose
L
81;;
Ckxm)
Ckxl)
where L is a matrix of known coefficients and
constants.
~
is a vector of known
We have
H(~ =
L,
and
A similar test which has intuitive appeal is to use V =
and the estimate of
ML estimates.
~
1
n-q
S
obtained by method 2 in (4.5.1) instead of the
We call this procedure the modified Wald procedure.
The small sample properties of both the Wald and modified Wald procedures are studied in the next chapter.
A test for "fit of model" is not available using the Wald
method, but this is of little consequence since the likelihood ratio
49
provides a simple test in this case.
4.6
Confidence Sets
The quantities Z and S are independently distributed.
If q=l
then the vector Z is distributed as p-variate normal with parameters
1
K(~)
and; V and the matrix S as p-dimensional Wishard with
parameters n-l and V.
Thus
n(n-l)~' (~)S-l~<.~) is distributed as central Rotelling's
(n-p)n
p
X'(~)S-lX(~ is distributed as F(p,n-p) as is
shown by Anderson [1,p.107].
A 95% confidence set for
~
is
This will not, in general, give an ellipsoidal set due to the nonlinearily of
If q
>
X(~)
1 then exact confidence sets are not known.
Approximate
sets can be computed if we assume the asymptotic distribution of the
Wald statistic.
An approximate 95% confidence set for G(~) is given by
The accuracy of this approximation depends upon the small sample properties of Wald statistics.
next chapter.
These properties will be examined in the
CHAPTER 5.
5.1
MONTE CARLO STUDY OF THE PROPERTIES OF THE ESTIMATORS AND
TEST STATISTICS
Introduction
Two methods of estimation of the parameters were developed in
Chapter 2_.
We have shown that estimates obtained from large samples
when using either method have desirable asymptotic properties, but
the smallest sample size at which it can be assumed that these desirable properties hold for either of the two methods is not known.
In Chapter 4,
cussed.
three methods of testing hypotheses were dis-
The large sample distribution of the statistics obtained by
each technique is known to be a central chi-square under the null
hypothesis, but the smallest sample size for which the distribution
of these statistics is adequately approximated by a chi-square distribution is not known.
The purpose of this chapter is to investigate the small sample
properties of the estimates and the test statistics.
Data which
simulated 100 repetitions of an experiment were constructed using
known values of
e
and V.
Estimates of
~
were calculated by both
methods in each experiment, and the appropriate test statistics were
calculated by all the methods for some hypotheses known to be true.
These estimators and test statistics were examined for the desired
properties when the sample sizes were 12, 24, and 36.
51
5.2
Construction of Data
The model which was used for this study is that of example 2.2.2
except that we used different sample sizes.
The asymptotic regression
model was chosen because it is frequently encountered in biology, and
the values of the parameters chosen are approximately those which were
encountered in an actual experiment.
Notice that 12 is the smallest
sample size which provides a nonsingular estimate of V when there are
equal numbers in each group for this model.
The matrix A is of the form
1 1 ... In/3 0 0 ... 0
A
o
0 ... 0
o
0 ... 0
1 1 ... 1 nl 3 0 0 ... 0
o
0 ... 0
o
0 ... 0
1 1 ... In/3
The vector 8 is
(38.,19., .5, 38.,25., .5, 38.,31., .5).
The matrix of random errors is
where the
and
~
are independently distributed as N CQ,V), i
7
1,2, .•. ,n
52
V
2
11
3
2
3
11
3
2
11
3
2
3
11
3
2
11
3
2
3
11
3
2
11
3
2
3
11
3
2
11
3
2
3
2
3
11.
3
2
11.
3
2
3
11
3
2
a
a
a
a
a
a
a
a
a
a
2
3
a
a
a
a
a
a
a
a
2
3
a
2
3
a
Let £ represent an arbitrary column of E and let
the i-th element of
~.
To generate the E., i
1
=
E.
1
represent
1,2, •.• ,7 we first
construct 72 independent random variables with uniform distribution
over the interval (-.5, .5) by the method described in I.B.M. Corporation Form C20-80ll [9].
random variables;
E
random variables;
E
2
3
Then
E
l
is the sum of the first 24 of these
is the sum of the ninth through the 32nd of these
is the sum of the 17th through the 40th of these
random variables, and so forth.
That
E
is N (Q,V) follows by the
7
Central Limit Theorem.
5.3
Methods of Evaluation
The estimates of 8 were calculated by both
m~thods
for each ex-
periment and, in addition, the difference of the two estimators was
computed.
The mean vector and the sample variance matrix of these
vectors were calculated over the 100 experiments.
The two methods
2
were tested for bias and equality of means by the use of Rotelling's T .
The test for equality of means is analogous to the univariate paired
53
t test.
The methods are evaluated on variance by comparing the corres-
ponding elements of the sample variance
matrices~
giving particular at-
tention to the diagonal elements.
Table 5.3.1 contains a list of hypotheses considered in this
study.
The appropriate LR statistic for the hypotheses
were calculated for each
experiment~
2~
3 and 4
and the appropriate Wald and
modified Wald statistics for the hypotheses
lated for each experiment.
l~
2~
5~
6 and 7 were calcu-
These statistics can be compared to their
hypothesized distributions in three ways.
The first method of evalu-
ation is to count the number of statistics which are larger than the
critical value for a
=
.05 of the hypothesized distribution.
number should be larger than zero and less than 10.
This
The second method
is the comparison of the first two moments with those of the hypo thesized distribution.
The third method is the usual chi-square test for
goodness of fit.
5.4
Results of Study
2
On the basis of the T statistics given in Table 5.4.1 and the
mean values of the estimates given in Table 5.4.2 we conclude that~
for practical
purposes~
both methods have the same mean and are with-
out serious bias~ even for n
=
12.
The covariance matrices of the
two types of estimators are essentially the same for each of the
sample sizes considered.
54
Tab Ie 5. 3 • 1.
Reference
Number
----
Hypotheses Considered in the Study
Degrees of
Freedom
___ J!y££thesis
-------------
1
12
2
3
P
.5 P
1 =
2
.5 P
3
.5
3
6
P
.5 P
1 =
2
.5 P
3
.5
4
Fit of Model
9
5
2
6
2
7
4
19 13
2
25 13
3
31
PI
.5 P
2
.5 P
3
.5
13 1
19 13
2
25 13
3
31
Ct
38
38
38
1
Ct
1
Ct
2
Ct
2
Ct
Ct
P2
P3
Ct
Ct
Ct
1
2
P2
3
3
PI
PI
--------Table 5.4.1.
1\
3
P3
---
Tests for Unbiasedness and Equality of Means of Method 1
and Method 2
2
Hotelling's T (critical value for
n_=--=.1=.2___
_ _ _ _ _ _ _ _ _ _ _ _ ---e
n = 24
Ct
= .05 is 19.39)
n = 36
Method 1
5.41
12.76
10.45
Method 2
5.65
12.27
9.96
.-.:..7.;...•.. :. 43
5.18
Di f fe renc::.ce=---__ _ _ _--=6:..;.:..;:1:..;0:.....-
Table 5.4.2.
n
Method
24
36
a
1
81
P1
a
2
82
P2
a
3
8
3
P3
38.00
19.00
.5000
38.00
25.00
.5000
38.00
31.00
.5000
1
38.12
19.24
.5015
38.02
25.18
.4985
38.07
31.10
.5013
2
38.11
19.23
.5013
38.02
25.18
.4984
38.08
31.11
.5017
1
38.06
19.01
.4985
37.97
24.95
.5018
37.98
31.01
.5010
2
38.07
19.01
.4985
37.97
24.95
.5018
37.98
31.01
.5010
1
37.92
18.90
.4953
38.03
25.09
.5020
38.02
31.06
.5010
2
37.92
18.90
.4953
38.03
25.09
.5020
38.02
31.06
.5010
Actual Values
12
Comparison of the Mean Values of the Estimates by Method 1 and Method 2 with Actual Values
of Parameters
--
V1
V1
56
An examination of Tables 5.4.3, 5.4.4 and 5.4.5 shows that the
distribution of the LR statistic as given in section 4.4 is well approximated by the chi-square distribution even for n
Table 5.4.3.
= 12.
Number of Likelihood Ratio Statistics Larger than the
Critical Value (a = .05)
Hypothesis
12
n
n = 24
n, = 36
1
4
5
6
2
3
3
5
3
6
6
9
4
5
2
6
Table 5.4.4.
Means and Variances of the Likelihood Ratio Test Criteria
Hypothesis
Standard
Error
n = 12
- -Var.
Mean
n = 24
Mean
Var.
n = 36
Mean
Var.
12.90 19.94
11.99 20.53
12.98 19.66
1
12
.49
2
3
.24
2.93
3
6
.35
6.52 13.91
5.48 11. 34
6.29 14.88
4
9
.42
9.82 20.85
8.22 14.03
8.96 22.55
Table 5.4.5.
4.72
2.70
4.16
3.31
7.68
Chi-Square Tests for Goodness of Fit of the Likelihood
Ratio Test Criteria
Critical Value
a = .05
Hypothesis
n
=
12
Test Statistics
n = 36
n: = 24
1
13
22.36
21.52
6.33
18.68
2
6
12.59
8.47
3.07
5.51
3
9
16.92
9.44
8.75
13.14
4
11
19.67
13.53
16.47
7.83
57
As can be seen in Tables 5.4.6 and 5.4.7, neither the distribu-
tion of the Wald nor the modified Wald statistics was adequately approximated by a chi-square distribution for the sample sizes considered
here.
Tab le 5.4. 6.
Means and Variances of the Wald Statistics
n = 12
Mean
Var.
24
Var.
2403.2
7.2
35.8
4.9
25. 7
15.0
457.8
3.8
15.4
2.8
11.8
.20
16.0
1388.8
4.8
25.0
3.2
14.4
.28
42.6
3378.4
8.9
44.2
5.8
23.7
--------
d. f.
Standard
Error
2
3
.24
25.2
5
2
.20
6
2
7
4
Table 5.4.7.
n = 36
Mean
Var.
n
Mean
BYpothesis
Means and Variances of the Modified Wald Statistics
n = 12
Mean
Var.
n = 24
Var.
Mean
n = 36
Mean
Var.
d. f.
Standard
Error
2
3
.24
19.0
1395.3
6.3
27.1
4.5
21. 6
5
2
.20
11. 3
268.1
3.3
11.8
2.5
9.9
6
2
.20
12.1
810.0
4.1
18.8
2.9
12.1
7
4
.28
32.0
1925. 7
7.7
33.4
5.4
20.0
~thes~__
This means the approximate confidence regions given in section 4.6 are
not valid unless the sample size is very large.
The computational ad-
vantages of the Wald method over the likelihood ratio method make further study of the small sample distribution of the Wald statistics
seem worthwhile.
-1
The matrix B
~
~
C8,V) is not a good estimate of the variance of
~
regardless of the method of estimation used, although if evaluated at
CHAPTER 6. USE OF THE COMPUTER PROGRAMS
AND A NUMERICAL EXAMPLE
6.1
Introduction
Appendix 8.2 contains two programs which can be used together to
calculate estimates of
Chapter 2
ter 4.
~
by either method of estimation developed in
and also LR test statistics which were discussed in ChapThis chapter describes some of the features of the programs
and provides instructions for their use.
A numerical example is pre-
sented to illustrate the application of the statistical theory and
the use of the programs.
6.2
General Description of the Programs
The programs are written in Fortran IV (E) for use on the IBM
360 system computers.
With minor modifications the programs can be
used on any computer of adequate size which has magnetic tape and
Fortran capabilities.
No claims are made concerning the efficiency of
these programs relative to some other algorithms.
However, they have
been found to perform satisfactorily on a variety of examples.
The first program, denoted "program 1", computes and prints the
matrices AA'
,
(AA,)-l
V
Z
= _1_ S (labeled as V), and
"n-q
(labeled as V-INVERSE).
These are all clearly labeled.
V-I
(_1_ S)-l
n-q
The matrix Z
is useful for determining the model and for obtaining a preliminary
estimate of
~
with which to begin the iterative procedure.
For this
reason the program is written in two segments rather than as one.
60
-1
Program 1 also writes on magnetic tape the matrices AA', Z, S, and S
The second program, denoted "program 2" reads from tape the matrices
written by program 1, computes e by either method 1 or 2 or both, the
LR statistic to test the fit of model, and D
l
lation of other LR statistics.
(~)
for use in the calcu-
Again, all of these quantities are
clearly labeled.
6.3
Som~cial
Features of the Programs
Program 1 must be used with the subroutines PROUT, INVRTR, MULT,
and ABTR.
Program 2 uses the subroutines ATRB and DEFUF in addition
to those used by program 1.
All of these except DEFUF are as listed
in Appendix 8.2 following the listing of program 2.
The subroutine
DEFUF is specific to the particular model used and will be discussed
later.
In calculating an estimate of Q by method I the program first
calculates an estimate of Q by method 2 and then uses this estimate
as a preliminary value for
calculating~.
Experience has shown this
to be more efficient than using method 1 directly.
The user must specify the maximum number of iterations that
may be done in attempting to obtain convergence of D(e.).
---"1
If the
specified maximum number of iterations are required, a message to
that effect is printed, and then the algorithm proceeds as if the
process had converged.
Due to rounding errors it is possible for
D(e.) to oscillate the fifth or sixth significant digit and thus
---"1
not converge.
In this case, one can, for all practical purposes
assume that the process converges.
If at some stage of the iterative process, D(e.) attains a
-:t.
.
61
smaller value than it does at the point where it converges, a warning
to that effect is printed.
The DIMENSION statement (statement 1) * defines the maximum anticipated dimensions of the arrays used in the program.
These arrays
are listed here with their minimum dimensions in parentheses:
S(pxp),
SI(pxp), C(qxq), Z(pxq), U(pxqxm), ST01(pxqxm), ST02(mxm), ST03(pxq),
F(pxq), and THE(m).
allow for p
~
12, q
The dimensions as they are given in the program
~
6, and m < 20.
The program with these dimensions
will run on the IBM 360/30 or a larger computer.
Any of the values of
p, q, and m can be increased for a specific problem without exceeding
the core storage providing the other two quantities can be reduced
accordingly.
If it is not required to examine Z before proceeding with the
analysis, the two programs can be combined into one by deleting statements 65 through 74 from program 1 and statements 1 through 9 from
program 2.
6.4
The Subroutine DEFUF
To use program 2 one must write a subroutine to define
given by (2.4.12) and
U(~
I(~)
as
as given by (2.4.13) except that for the
program,
U(~)
manner.
The vector 8 is denoted as THE in the program and is already
defined.
is packed as a one-dimensional array in a column-wise
It can be used in defining
E(~)
declared as integers are the quantities P
QQ
=
qxq, PQ
*The
=
pxq, MM
=
mxm, MQ
=
mxq, MP
and
=
U(~).
p, Q
=
=
Also defined and
q, PP
mxp, MPQ
=
=
pxp,
mxpxq, and IN
term statement number, as used in this chapter, refers to
the sequence number on the right margin of the program listing.
62
which is the number of iterations completed.
The process of writing
such a subroutine can be made clear by relating the model of the example in this chapter to the subroutine DEFUF in the Appendix 8.2.
6.5
The Data Format
To use the programs without modification the data should be
punched on cards according to the format described here.
If it is
more convenient for the user to use another format, he may replace
statements numbered 12 and 39 of program 1 and statements numbered
17 and 24 of program 2 with his own equivalent statements.
It is
expedient to define some terminology for describing the data format.
We will consider the standard 80 column punch card as being comprised
of eight fields of 10 columns each.
A vector is said to be punched
if the first element of the vector is punched in the first field on
a card and consecutive elements are punched in consecutive fields,
using additional cards if necessary to contain the vector.
The first data card for program 1 carries the value of n, p,
and q in the first, second, and third fields, respectively.
These
values must be punched without a decimal and right justified in their
respective fields.
and A, respectively.
~l' ~l' ~2' ~2'
Let Y. and a. represent the i-th columns of Y
-.L
-.L
The vectors of Y and A are punched in the order
... , Xu'~·
The elements of Y and A must be punched
with decimals unless they are integer valued, in which case they may
be right justified in their respective fields.
The first data card for program 2 is as follows:
the first
field contains a 1 if estimation is to be by method 1, a 2 if estimation is to be by method 2; the second field contains the value of m,
63
i.e. the dimension of
~;
and the third field contains the maximum num-
ber of iterations allowed.
All of these values must be punched without
a decimal and right justified in their respective fields.
nary estimate of
~
The prelimi-
is punched beginning on the second card.
If the two programs are combined as described in section 6.3 the
data for program 2 simply follows the data for program 1.
6.6
A Numerical Example
In a hypothetical experiment, we have three groups of six mice
each.
All of the mice are injected with tumor producing cells.
The
first group is a control group while group 2 and group 3 are treated
with different drugs.
The weight of each mouse is recorded in grams,
daily, for seven days.
These observations form the matrix y' and are
given on the even numbered rows of Table 6.6.1.
Assume that the response of the animals in each group is of the
form
y
where t is time in days.
in example 2.2.2.
=
a _ spt-l
+
€
Such an experimental situation fits the model
We want to estimate the parameters for each group
and to test the hypothesis that the rate of growth
p
is the same for
each group.
The elements of A' are given on the odd numbered rows of Table
6.1.1 beginning on the third row.
We now prepare the data for program 1.
The first punch card con-
tains the number of mice, 18, in columns 9 and 10.
Column 20 contains
the number of observations on each mouse which is seven, and Column 30
contains the number of groups which is three.
proper format in Table 6.6.1.
The data appear in their
64
Table 6.6.1.
l::l
,..
,..
(J)
(J)
,..
"@
,0
(J)
l::l
9l::l §
Po
(J)
;j
;j
,..
0
0
The Elements of y' and A' Arranged in the Format for
Program ]
t~
;j
0
.j.J
tll
()
l::l
'H
4-1
'H
,..
(J)
'"d
Day (pertains to the
0
'H
1
2
3
l::l
'"d
H
0-10
1
*
18
2
y'
-1
23.7
28.6
32.3
3
1.0
.0
.0
4
a'
-1
y'
-2
24.5
28.3
30.6
5
a'
-2
1.0
.0
.0
6
y'
-3
23.6
28.7
31.5
7
a'
-3
y'
-4
1.0
.0
.0
23.3
26.3
30.9
1.0
.0
.0
25.5
30.1
31.0
a'
-5
1.0
.0
.0
y'
26.6
29.7
31.3
a'
1.0
.0
.0
14
y'
-7
22.2
27.9
33.2
15
a'
-7
y'
-8
.0
1.0
.0
21.2
28.3
33.1
.0
1.0
.0
u
1
2
3
1
8
4
9
10
5
11
12
6
13
7
16
8
17
*Dimensions
a'
-4
y'
-5
-=-f>
~
a'
-8
only)
5
6
7
31-40
41-50
51-60
61-70
32.5
33.0
31.1
33.1
32.2
34.3
34.8
35.0
32.6
33.0
34.0
33.9
31.1
34.0
33.6
32.0
30.6
32.4
33.6
34.2
31.5
32.9
32.5
33.0
34.4
34.4
33.3
33.8
35.0
35.7
34.3
33.7
Columns
.j.J
tll
;E:
4
y~
~
11-20
21-30
7
3
65
Table 6.6.1.
Continued
I:l
l-l
Q)
,.c
§
I:l
P.
;:l
0
l-l
0
l-l
Q)
1
I:l
Q)
CIl
;:l
0
~
l-l
Q)
,.c
§
c:
"BCil
C,.)
18
9
19
.j.J
til
1
(J
.r-i
4-l
.r-i
2
3
c:Q)
"Cl
0-10
H
51-60
61-70
31.4
31.9
31. 7
33.2
33.5
34.7
35.5
36.3
35.2
33.4
32.4
32.3
31.0
31. 6
32.3
34.3
36.8
38.7
38.7
39.3
38.6
37.4
37.2
36.4
38.2
38.3
36.2
35.9
36.2
36.5
36.9
37.7
36.7
38.0
39.0
40.3
37.7
39.7
40.0
40.1
-:.g
y'
25.7
29.7
31.4
a'
.0
1.0
.0
~
21
~10
.0
1.0
.0
22
XII
23.4
30.0
33.7
23
~11
.0
1.0
.0
24
X12
23.1
29.0
31.2
25
~12
.0
1.0
.0
26
X13
22.4
29.1
34.4
27
~13
.0
.0
1.0
28
X14
24.0
32.6
35.5
29
~14
.0
.0
1.0
22.3
29.6
35.7
14
30
15
,
,
,
,
,
Xi5
,
31
~15
.0
.0
1.0
32
Y16
23.6
31.9
35.7
33
a
.0
.0
1.0
34
X17
24.3
31. 3
34.6
35
~17
.0
.0
1.0
36
X18
24.4
31.8
35.0
37
~18
.0
.0
1.0
16
17
18
7
41-50
30.5
13
6
31-40
27.7
12
5
21-30
21.9
11
only)
11-20
X10
2
4
y~
-.I.
Columns
.j.J
20
10
3
Day (pertains to the
0
'r-i
,
16
,
,
66
Program 1 gives us the matrix Z which is preseqted in Table
6.6.2.
Table 6.6.2.
The Z' Matrix
Day
Group
1
2
3
4
5
6
1
24.5
28.6
31.3
31. 7
33.3
33.3
33.5
2
22.9
28.8
32.2
33.4
33.6
33.2
33.9
3
23.5
31.0
35.1
37.4
38.1
38.0
38.3
7
A plot of the matrix Z is given in Figure 6.6.1.
The vector
~
is defined by the relation
where the subscripts pertain to the groups.
We now find a preliminary estimate of
~
for the use of program 2.
From an examination of Figure 6.6.1, we choose 34. as our preliminary
estimate of 0.1'34.5 as our estimate of 0. ' and 39. as our estimate of
2
0. •
3
The technique for finding preliminary estimates of the (3. and Pi
~
is illustrated for group 1.
Setting
~l
gives
24.5
a - (3
AA
A
A
(3p
a
a
2
(3p
a
a
28.6
(3p
a
-
3
"A'4
(3p
AA
5
(3p
A
A
a - (3p
6
31.3
31. 7
= 33.3
= 33.3
= 33.5
equal to its expected value
FIGURE 6.6.1 PLOT OF THE MATRIX Z·
40
39
38
37
36
35
34
33
(/) 32
~
« 31
a:
(!)
30
~ 29
~ 28
w
3= 27
26
25
24
23 ..
Group I
Group 2 ......
Group '3 ----1. . . . 1. . . .
22
21
20 I
2
3
4
5
TIME (DAYS)
6
68
We see that a
= 34.
implies
=
S
9.5
5.4
f3p
AA
Sp2
S;3
2.7
=
2.3
S~4 = .7
S~5 = .7
AA
Sp6
thus the estimate of S is 9.5.
.5;
Dividing each number into the number
following it yields
p
5.4/9.5
.57
p
2.7/5.4
.50
p
2.3/2.7
p
=
p
p
=
=
.7/2.3
.85
.30
.7/.7
1.00
.5/.7
.71
The average of these values is .66 and is taken for our preliminary
estimate of p.
Repeating this process for groups 2 and 3 yields
= (34.,9.5, .66,34.5,11.6, .68,39.,15.5, .63).
The subroutine DEFUF for this problem is given in the Appendix.
We now prepare the date for program 2.
The first card contains a 1 in
column 10 so that we will obtain estimates by both methods.
The
69
number of parameters, nine, is in column 30, and the maximum number of
iterations allowed, 20, is in columns 29 and 30.
cards contain
Table 6.6.3.
e.
-0
The second and third
The data for program 2 is given in Table 6.6.3.
Data Format for Program 2
Columns
Card Number
0-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80
1
1
2
34.
3
.63
9
20
9.5
.66
34.5
11.6
.68
39.
15.5
The results of the estimation procedures are given in Table
6.6.4.
The X2 (12) statistic to test the fit of model is 14.770 which
is considerably less than the critical value of 21.026.
We now want to test the hypothesis that the rate of growth is the
same for each group.
We rewrite the subroutine DEFUF with the res-
triction that p is the same in each group.
is also given in the Appendix.
Such a revision of DEFUF
This version uses
Notice that there are now only seven parameters.
~
defined by
The preliminary
estimates of parameters are the estimates obtained using the unrestricted model except for p which is estimated by the average of
PI' P2 and P
3
of the unrestricted model.
restricted model is given in Table 6.6.5.
The data format for the
A
Table 6.6.4.
D(~)
The Values of
A
and.§. at Each Iteration
A
A
A
PI
a2
S2
.66
34.5
7.8
.44
33.4
8.7
1. 2782
33.4
4
1. 2782
1
1
1
2
A
A
A
A
A
Pz
a
3
S3
11.6
.68
39.0
15.5
.63
33.3
10.4
.47
38.0
14.5
.49
.46
34.7
11.6
.51
38.8
15.1
.51
8.7
.46
34.7
11.6
.51
38.8
15.1
.51
33.4
8.7
.46
34.7
11.6
.51
38.8
15.1
.51
16.6992
33.4
8.7
.46
34.7
11.6
.51
38.8
15.1
.51
16.6992
33.4
8.7
.46
34.7
11.6
.51
38.8
15.1
.51
Method.
Iteration
D(.§.)
2
0
2
al
Sl
20.3281
34.0
9.5
1
5.6172
32.2
2
2
1. 2843
2
3
2
P3
'-I
o
71
Table 6.6.5.
Data Format for Program 2 (Restricted Model)
Columns
0-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80
Card Number
1
1
2
33.4
20
7
8.7
34.7
11.7
38.8
15.1
.49
The results of the estimation procedures for the restricted model
are given in Table 6.6.6.
Our hypothesis in the format of (4.4.3) is
[0 0 1]
1
0
o
1
[0 0]
-1 -1
from which we see h
1 and t
= 2.
The approximate multiplier m of
1
2
(4.4.9) for log An is
-(18 - 3 - 7 + 3 - t(l - 2 + 1))
-11.
By the relation (4.3.3),
2
n
log A
= 16.6992 - 16.7358 = -.0366
where 16.6992 is the value of
D1(~)
from the last line of Table 6.6.4,
and 16.7358 is the value of D (~) from the last line of Table 6.6.6.
1
The X2 statistic with two degrees of freedom is .403 which is
considerably less than the critical value of 5.991.
the rate of growth is the same for each group.
We conclude that
Table 6.6.6.
The Values of D(8) and
~
for the Restricted Model at Each Iteration
~
Method
Iteration
D(8)
(Xl
Sl
(X2
S2
(X3
S3
p
2
0
1. 5511
34.4
8.7
34.7
11.6
38.8
15.1
.49
2
1
1. 3391
33.7
9.2
34.6
11.5
38.6
14.9
.50
2
2
1.3390
33.7
9.2
34.6
11.5
38.6
14.9
.50
2
3
1.3390
33.7
9.2
34.6
11.5
38.6
14.9
.50
1
1
16.7359
33.7
9.2
34.6
11.5
38.6
14.9
.50
1
2
16.7359
33.7
9.2
34.6
11.5
38.6
14.9
.50
1
3
16.7358
33.7
9.2
34.6
11.5
38.6
14.9
.50
1
4
16.7358
33.7
9.2
34.6
11.5
38.6
14.9
.50
-...J
N
CHAPTER
7.1
7.
SUMMARY AND SUGGESTIONS FOR FURTHER RESEARCH
Sunnnary
In the analysis of growth curves, the analyst may encounter a
situation where the response over time is a nonlinear function of
the parameters and where the observations on the same animal at different points in time are likely to be correlated.
One may also want
to consider several groups of animals where the parameters may be
different for the different groups.
A general model, which includes as a special case a model for
the experimental situation above, is formulated.
Y
(pxn)
F(e)
+
A
(pxq) (qxn)
where Y is a matrix of observations;
This model is
E
(pxn)
F(~)
is a matrix whose (i,j)
element is
f
j
(e , t . ) ,
-~
i=1,2, ... ,p,
j
1,2, ... ,q;
A is a known design matrix of rank q;
~
is an mx1 vector of unknown
parameters whose elements are functionally independent; t. is the i-th
~
fixed input vector; and E is a matrix of random variables whose
columns are independently distribution according to some p-variate
74
distribution with mean
Q and
unknown dispersion matrix
V
(pxp)
For mathematical reasons we also require that the third order partial
derivatives of f.(8,t.) with respect to elements of _8 be continuous.
J --J
We find an unbiased estimate of V to be
__1 _ S where
n-q
,
S = Y(I - Al (AA )-lA)y ' •
This is the same estimate which is used in linear multivariate ana1ysis.
In Chapter
2, two methods for the estimation of
Method 1 is to find
~
log
~
are presented.
which minimizes
I (Y
A
A
- F(i1)A) (Y - F(8)A)
\.
(7.1.1)
tr«Y - F(8)A) I S-l(y - F(~)A)).
(7.1.2)
I
Method 2 is to find 8 which minimizes
Method 1 and method 2 are equivalent if q
= 1 but not necessarily equi-
valent i f q > 1.
Under the assumption of normality we find that Sand
Z
= YA
I
(AA I
)
-1,
the sufficient statistics for the parameters of the linear multivariate
model, are also sufficient statistics for the estimation of 8 and V of
the present model.
The minimization criterions (7.1.1) and (7.1.2) are
given respectively in terms of the sufficient statistics by
75
A
log Is + (z - F(~))AA'(Z - F(~))'I
and
Method 1 yields the maximum likelihood estimators of
e
which are known
to be consistent, asymptotically efficient, and asymptotically
ly distributed.
normal~
Estimates obtained by method 2 are also consistent,
asymptotically efficient, and asymptotically normally distributed.
In Chapter
3
(7.1.1) and (7.1.2).
we discuss two methods for the minimization of
One method is the classical method of steepest
descent, and the other is a weighted linearization procedure.
The likelihood ratio and Wald test procedures are developed in
Chapter 4.
Exact
confidence sets are available when q
=1
and
asymptotic confidence sets can be determined for q > 1.
Chapter 5 presents an empirical study of the properties of the
estimators and test statistics for a three group, asymptotic regression model.
The results of this study indicate that both methods are
without serious bias and are, for practical purposes, equivalent even
for small sample sizes.
The distribution of the likelihood ratio test
statistics is well approximated by the chi-square distribution even
for small sample sizes.
The distribution of the Wald statistics is
not adequately approximated by a chi-square distribution for the
sample sizes considered.
Computer programs are given to do the calculations.
A numerical
example is presented to illustrate the application of the statistical
theory and the use of the programs.
76
7.2
Suggestions for Further Research
The Wald test criterion is much easier to compute than the like-
lihood ratio and also provides a means for establishing confidence
sets
for~.
Abetter knowledge of the small sample distribution of
Wald statistics would be very valuable.
CHAPTER 8.
8.1
APPENDICES
Selected Theorem
In an effort to improve continuity of the text, several theorems
which are used in this development are stated here.
reference to a proof is given for each theorem.
A proof or a
The first two
theorems establish some relations involving the characteristic roots
of matrices.
Theorem 8.1.1
A
If
is real symmetric with characteristic roots
(pxp)
then
IAI =
P
A. ,
II
i=l
1
and
P
tr(A)
L
i=l
Proof:
Let
A ••
1
C
be a nonsingu1ar orthogonal matrix containing the
(pxp)
characteristic vectors of A.
=
Ici IAI Ic'l
ICAG'I
=
=
p
II
i=l
A.
1
and
78
tr(A)
= tr(AC'C)
=
P
L:
tr(CAC') =
•A
P
i=l
A••
1.
Q.E.D.
Theorem 8.1.2
If
A
is non-negative definite and
(pxp)
definite then the characteristic roots of AB
Proof:
B
is positive
{pxp)
-1
are all non-negative.
We can find C such that
B-
1
The characteristic roots of AB-
= CC'.
1 are the solutions for A in the fol-
lowing equation:
IAB-
Ic'l
IAB-
l
- All
l - AIl Ic,-ll
IC'AC - All
Thus the characteristic roots of AB
-1
=
= o.
are the same as those of C'AC.
Since C'AC is non-negative definite its characteristic roots are nonnegative.
Q.E.D.
The next theorem gives the derivatives of two matrix functions.
79
Theorem 8.1.3
Denoting O(Y) as a matrix whose elements are the differentials
of the corresponding elements of the matrix Y, we have
and
if
IY I
> O.
This theorem can be verified directly from the definition.
Theorem 8.1.4
For all real x we have e
x
> 1
+ x.
Proof:
Let f(x)
x
= e - 1 - x.
x
=e
f' (x)
it follows that f(x)
~
f(O)
Since f(O) and
0 for x > 0
- 1 [: 0 for x < 0
= o for all x.
Q.E.D.
Equivalently, e
x-I
~
> x for all x.
Hence, for all x > 0 we have
log x.
Theorem 8.1.5
If
x-I
(Anderson [1, p. 103])
B and
A and
Dare nonsingular matrices, and if
(pxq)
(pxp)
(qxq)
I
I.
I
I
I
I
I
I
I
.-
I
I
I
I
I
I
',-.
I
80
C
(qxp)
are arbitrary matrices of the dimensions indicated, then
8.2 COMPUTER PROGAAKS
PROGRAM 1
lJlMENSICN Stl441,Slll441,CI301,li121,UI144CI,
1STU l ( 144 C1 ,S lOLl4<J<J) ,S I 03 I 711 ,F (72) , Th I:: 120 1
INTEGER p,W,PP,JQ,~W
WRITEI3,1261
126 FOR~All'lS~FFICIANT ST~TISIICS FeR MuLl IVARIATE NON-LINEAR',
I' ANALYSIS',I,
j'OALGCRITHM BY OAVIU M ALLEN ',I,
4'OPI<.UGRAM dY JI':I<.RY G GUdlO'tI,
~'OCEPARTMENr
Of
eiOSTAll~lICS',I,
NOAT~ lAKCLINA'1
o'OUNIVERSITY Of
KI::AtJIl, 114)i\l,P,.l
114 FCRMATOllOl
CF=I\-(,;;
pp=p*1'
lJu 102 1=1,1'1'
102 5111=0.0
WQ=.l*i.i
OC llJ3 1= 1,.Ii,;
103 (,111=(.0
I'\,,= t' *w
DC 104 l=l,P~
104 5101(11=0.
DO 105 1=1,;'4
REACll,U.l61Ifll<l,K=1,PI
kEAtJIl, lCt) I ZI LI ,L=l , ... 1
DO 115 J= 1,.,
DC 115 t<=l,F
L=I J-li *P+K
115 S(LI=SILI+FIJI*F(KI
DC 116 1<=1,,,"
DO 116 J= l,p
L=I K-l)*P"J
116 STLIILI=SICIILI+fIJI*ZIK)
tJO 117 J= 1, Q
DC 117 K=l,i,)
L=I J-li *C+K
117 CILl=CIll+ZIJ)*ZIKI
105 CONT If\UE
10f: I-CRMAH8FlO.01
Wi<ITEI3,lC71
107 FCRMATI'O AA* MATRIX')
CALL PROLII C,Q,QI
Ct.LL INVRTRI C,./,I,), U,Q,\,j,DETl
\>oRIHU,1081
loa fORMATl'l AA* INVeRSE')
CALL PRUUT! U,I,),QI
CALL M~LTlSTLl,P,~,U,Q,C,ll
WR I It:! 3, I 0 9 I
109 FeRMATl'l Z-MATRIX'I
CALL P~CUT( l,P,~1
CALL ABTR(Z,P,y,SlGl,I',Q,SI)
DC 118 l=l,PP
001
002
003
U04
005
OOb
CCl
U08
009
Cle
Ull
OIL
013
014
Ob
016
U17
018
Gl'i
020
021
022
02"3
024
02S
02b
027
028
029
03C
031
032
C33
034
035
036
037
038
039
040
041
04.2
043
044
04S
04b
C47
04d
049
050
051
0~2
118 S(I)=S([)-SI(I1
CALL INV~TR( S,P,P,SI,P,P,DET)
DETl=ALCG(oET)
DO 20e 1=1,1'1'
U([)= S(ll/CF
200 STOl([)=SI([I*CF
WI<.1TE(3,1l0)
110 FUl<MAT ('1 V-j"1ATK IX' I
CAll F~(LTI L,I',PI
WRITE(3,1111
111 H.J<MH('1 V-INVEl<SE')
CALL I'RO~l(SlOl,P,PI
wRITEISIP,W,PQ,PP,WW,N
WRITe(51( C( 1),I=l,Q~1
VIR 1 TE (511 l ( I ) ,I =1 , PC)
WIUlEIS)( 5111,1=1,1'1')
VlRITEI51IS1(II,[=I,PPI
WRITEIS)OLTl
WRITEI3,ll31
113 FUR~Alt'O iNC CF CLJTPU1')
STOP
eND
053
054
055
056
OH
058
059
060
061
062
063
064
065
066
067
068
C6S
070
071
e12
073
074
PKOGRM 2
o [Ml:NSION
51 1441,::,[ I 1.441 ,L (36) , l ( 121 ,U( h4CI ,
I~TOlll4401,ST0214uOI,ST03(72),f(72),TH[(~OI
[NTEGER
P,C,FP,C~,P~
REAC('»)P,(J,f'Y,PP,IolIol,!~
RlAC(:>llCIIl,I=l,QQ)
KeAtH 5) ( l I [ ) ,I =1 ,P~I
KEAO(5)(S( 11,1=1,1'1'1
001
002
003
004
005
006
007
READISIISIIII,I=I,~p)
OCS
READI 51Dl: Il
WRITEI3,211
009
010
27 FCR~AI('I~ULTIVA~lATE ~O~-LINcAR ANALYSIS',/,
I'GALGURITHM AND PRCGRAM BY DAIiID M ALLEI\",1,
2'ODEPARTMENT OF riUSTATISTICS",I,
3"OLJI~1 \lERSI TV Cf f\(;iHh (ARLlINA" I
IN=O
RE:At:(I,lI~ET,M,MIN
1 fORMAll41101
DET= O.
~M=M*M
M(;;=~*C
MP=M*P
MPI,,=MF*C
RE AD ( 1 ,2) IT HE I II , 1=1 , MI
2 FORMAT( 8f 10. C)
60 DtTM=999999~9.
61 01.:1 l=OE T
CAll CEfUf(F,U,T~E,P,P"MPQ,IN)
DC 43 1=1 ,PI,;
43 F(I)=lCll-F(I)
If( MET-2 )8, 8,3
8 CALL f'LLT(S[ .P,P.F,P,C,SfCl)
CAll ATRBIF,P,Q,STUl,P,Q,Sl021
Cll
012
013
014
015
016
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
2d
3
44
't
48
41
46
OcT=O.
DO 28 1=I,QQ
DtT=DET+CIlltST;J2111
L=2
GU TO 4
CALL A8TRIC,<J,I,j,F,P, .. ,STOll
CALL ~UlTIF,F,~,STCI,W,p,SrL2)
OU 44 1= I ,P P
STCZIll=Sll)+SlLJ211)
CALL IIWlllf<ISrU,p,p,SI,F,P,CETl
OtT=ALUGIOET)
L=1
(,AlL /lLU lSI ,P,P,u,P,MI",STCl)
IfIOtT-OETMI4t,41,47
OETM=CET
DU II 1=1,/1
DU 46 J=I,P<..l
K=( l-l1*PI,,+J
ST02IJl= STLlllKI
CALL MULTISTu2,P,J,L,<..l,~,SI031
DC 11 J=I,?Q
033
034
035
036
u31
03d
C3S
040
041
042
043
044
C4~
040
U47
C4~
049
O~U
u51
052
053
054
u55
G56
K=II-l)*P~+J
11 STLlllK)=$IU3IJI
CALL ATR~IU,p~,~,SrUl,f~,/I,SI(21
LALL IN~RTKIST02,/I,M,SlCI,/i,/I,OTT)
CALL MULHSI,P,P,F,P,<J,ST031
CALL /lUlllsr(3,p,~,C,~,~,srC2)
CALL ATRoIU,?Q,M,ST02,FW,I,SlG31
CALL /lULT(STC1'i..","1,STO~,M,I,ST02)
v.KITEI3,301
30 FUKMATIIHCI
WRIT~13,51IN,l,Dtl
5 FURMAlt'OITEKATICI. NUMl:1ER ',[4,'.
WRITEI3,21)
21 FCR/lAlt'OTt-ElII VECTOR')
\o.ld TE13,1911 THEI II ,I =1,1"1
WRITEI3,L21
22 FUKI"AlI'0-1/2 GRHlt(',T Vi:CTOtl')
WRITd3,l<JltSTG3111,l=I,MI
19 feRMAT 11h), 10E13.l:d
1(',=1(',+1
!l-1JEl-DtTlI',S,1
7 DC 4~ l=I,M
45 THElll=THtlI)+STCZII)
IF1MIN-INI50,tl,61
~
0~7
0~8
C5~
060
061
C62
063
064
O'0I1,'=',EI4.6)
IFICET-[lTMI49,~9"O
5C iIlkITE13,511
51 FURMATI'CwA~NING-uIU NLT LUN~~MGE LR DIG
49 IN=1
Wid TEl 3,t3)
63 fUKMATI'OLACK OF FIT UR X MATRIX'I
CALL FRCuTlf,P,Ql
IFIMET-2IIO,35,6
10 MET=3
GO TO 60
6 C I=P-M/Q-Q+ 1
Cf=(',-Q
CF=CF-.5*Cl
CF=CF*IDU-OETLI
~(I
ClhVERGE TO MINIMUM')
Ob?
066
C67
068
069
07C
071
072
073
074
075
C76
077
078
019
080
081
082
083
C84
085
086
C~l
088
089
090
091
es.::
NOf=P*\'-~
0'.l3
094
wK ITt( ;, ; ; I
33 FUkMATI'OTEST FUR FIT GF MUDcL'~
32
>;RITEU,321CF,1I01-
FUR~ATI'CCHI-SJUAI<t:=',FII..3,'
('is
U'I6
097
Cc,e
OU.J,{t:tS (I' r'j<ltC(~=',151
35 wKlfE13,621
6l
fLk~ATI'C~IIU
LF LLTPLT'I
09'1
SlOP
!G0
tl\[)
SUdK~uTINt:
C[FUF FOK Tre THKEE GKUUP
SUd~UUTINE
iJ It' E115
A~Y~PICIIC
~EGRESSIUN
MCUtL.
001
DtFUFII',U,TI'E,P,PJ,~P~,11I1
002
I ell u 11 I, f 11 I, THI: I 1 1
II\Tt:Gl:.I<
003
p,P~,t'P":
004
005
006
007
It-liN 11C3, lC4, 10
104 DC lu2 I=l,~P(,i
102 Ulll=C.
103 CU 101 I=l,P
cca
I2=P+I
009
010
1 3=P + 1214= Pi.;+ I
I':.>=I'i,;+14
ell
012
It>=3·PI,J1oP+I
013
014
17=Pi,;+ 16
Tb=Pi,;+17
*
*
015
016
017
19= 6 PI.< + 2 P + I
110=Pc;,+19
Ill=P~+IlC
018
r= 1-1
019
020
021
t-( I I=TH:HI-IH,U I*TrE13IHI
UII I =1.
Fl UI=1I-<tl41-THClSI*lHt:lel**1
I' 113I=TfE l71-Tl-t 1:3 1*1 HI:( 91 HI
UI 141 =- THE I :II ** 1
U 1 15 1=- T I/- EI 21 *l" Hll31
IT - 1. I
UI161=1.
L( I7I=-THEI61**r
U1 I d 1=- T* r Hl: ( ':.> I 1 Ht: ( 6 1** 1 1- 1 • I
UII91=1.
*
on
023
024
**
C25
026
*
L;l
lie I =- 1 t1 to I
<j
I
027
028
Ol9
** T
030
lUI UIllll=-I*Tft{tll*TI-<EI9IHIT-I.I
kETUkl\
031
032
tNlJ
SUUkUUlll\t: Cl:rLf fLR IHL THKEt: uRCUF ASY~FTCTIC R~GkESSICN MCDl:l WITH
kArt U- GRCWrh IN ~ll GkOUPS.
THE
SAMi:
SUBROUTINE r.[FUflf,L,Tt-l:,P,PQ,~P~,11I1
OIMt:NSIUN UllI,F(1),TH[1 1)
INTEGEk P
1 I' I IN-II104, 104,10.3
104 00 102 1=1, ~F(';
10" LlII=C.
103 00 101 1=1,f'
001
002
003
004
005
006
007
vOl:!
(JUY
12=P+I
13= 2 • ~f' + I
I4=j .*HI
010
Oll
I~=7.*P+I
LlU
I6=10.~P+I
eu
I7=14.*P+I
Ie=17.*p+I
I9=18.*P+I
110=19.*P+I
Ill=2C.*P+1
014
Ob
016
T=I-l
fIII=THtlll-THcIZI*THtl/l**T
UII)=I.
FI121=T~EI31-Tht(4)*fHEIJI**T
FI131=THEIS)-THElol*THE1/)**T
U I 14 ) =- THI 7 )
** T
OLL
OZ~
ull':»=!.
024
UlI71=l.
UI I d 1=- T~ E 17 1* *T
\..l 191=-1* I HE 12 1* Trit I (P'* IT-I. 1
u ( 110 1=- T *T He I 't 1 *f HE I 7 1** I T- 1• I
020
02~
UI I 6 I =- THt I 7 I ** T
101
017
018
Gle,
OlO
021
LlIl11=-T*THElbl*l~t(71**IT-l.1
KETUK.~
END
SUbRUUTINE ATK0 IX,~A,~X,y,Nr,~y,ll
OIHi\SIC,., Xlll.YHI,zlll
N= 1
i"l=1
LlL 02
L 1= 1
J=l,I"Y
DC id
1'\=,.1
1= I, MX
Gn
OLd
029
C3C
031
032
OLll
Ge2
003
00"
CC5
006
QUi
GCe
009
L=L 1
)(.x=o.
CIu
DC 64 l<=l,"'Y
X X= XX+ X I L 1* Y (;\1 I
L=L+l
OU
Cll
013
014
64 M=M+I
l
U~
OI~
1= XX
[\=,.,+1
010
63 L I=L 1+I\X
017
OIU
CIS
020
62 Ml=,"1I+NY
KE:TUK'"
E:ND
SLdRCLTINE AtTR
IX,NX,~x,y,Ny,~y,ll
UII'lENSIuN XIll,Ylll.llll
1\=1
IF I '''1X-M't1 h,14,f'j
14 .H= 1
DC 11 J=l,to.Y
72
M=Ml
L=Ll
003
004
005
Cl,;6
007
L 1= 1
DG
OCI
002
I=I,NX
OOl:!
CC'>
010
011
XX=o.
DC 73 j<=l,/'Y
XX= XX+XI U*VI MI
L=l+NX
73
/'=1'HI\V
ZINI=XX
N=N+l
72 1l=1l+1
71 M 1= M 1+ 1
RE:TU RN
75 CAll ElllT
020
on
022
SUbKULTINE MlLT Ill,Nll,~X,¥,l\y,~V,LI
ION X I 11 ,Y I 1) , Z I })
002
MI=l
co
52 J= 1, MY
"'=Ml
DO r; 3 I = 1 ,N X
L= I
ZZ=O.
0054 K=l,MX
ll=U+XILj*VIMI
L=L+f\X
54 M=M+l
M=M 1
ZINI=ll
53 N=N+I
"'1=~l+f\Y
"to lUKN
tND
SUbRCLTINE Pl<CUl U,Nl<,I\C,}
U IMENSlUN AllI ,lI 1101
17 FORMAl I U" I
10 fURMATIlHIII
13 FCJRMATllH ,ICE D.6}
foIA=NC
lFIMAI26,43,33
26 MA=-MA
wRIT E13,10 j
33 NM=[ABSINR*NCj
NT= 1
f\A=NR
[F(NAI32,43,Jl
31 NS=10*NA*INT-l)+1
DO 4 1=1,10
4 BII 1=0.0
NL=NS+NA-l
IFINS-NMI30,30,43
3C DO 72
N2=N
l=l
~4
014
015
016
017
(,18
019
EN C
o ( ,. Ef\5
1'<=1
5~
012
013
N=N~,Nl
BIll=AIN21
L=l+l
IF(l-10123,23,2~
001
003
004
005
006
007
008
009
010
011
012
013
014
G15
016
017
C18
019
OCI
002
003
004
005
006
OCl
OOB
009
010
011
012
013
014
015
016
017
018
019
020
021
022'
023
024
014
K 1= I
KRS =I
KkF =I\A
DU 4 KK= I, NA
C=tJdLEl elK I) )
01:>
016
017
Old
Cl"
LC =CC*U
1FILJlt,5,t
020
6 C=I.LJO/C
on
DC 7 K=Kf<S,KHIF I K-K II c, 7, t
8 dIKJ=SNGLlbIKI*CJ
023
024
CU
C25
7 CLNTII\L£
KCS=KK
KF=I\A
\)C g 1=1,1\1',1\/1
1F ( I-K k SIll, I ~ , I I
I i [=-C13LEI~(K(SIJ
020
027
C2l3
029
030
031
032
Kk=KKS
Du I i K= I, KF
033
IF(I\-K(SI14,12,14
14 u(K)=5NGL(ct(KKI*t+d(K) 1
12 Kk=Kktl
I ' Kf=I\F+I\A
034
035
C3!:
037
9 KCS=KCS+NA
C3il
C3'1
~(K11=SI\GL(OI
0=-0
DU 16 11= KK ,,\ I" ,1\1\
IF(I\-KlI17, It, 17
17 B(KI=SI\Gl(O*E(KII
16 CLNTINlJE
KR:>=Kf<S +I\A
Kkf-=KRFtl\1l
4 Kl=I\kS+KK
C=S NGL (CC I
24 KETUKI\
5
040
041
C42
043
044
C45
040
047
C4E
049
WklTE(~,2CIKK,NA
STOP
0:>0
051
END
0'J2
20 FCkMAT I' II\VIHk PIVOT '. I'J,' elF
',151
Z3 NZ:NZ+NA
IFlNZ-NMI24,24,2'
25 l:l-l
DO 56 1= 1, l
IFlBlII-9661100.121,56,21
56 CONrINUE
WRITEl3,771
GL TO 72
21 wRIrEl3, 13111H 11 ,I=I,Ll
72 CONTINUE
Nl=NT+l
WRITEl3,151
15 FORMAT (/ II)
GO TO 31
32 NA=-NA
MR:MA
41 NS:l0*/lJ-S
00 49 1=1,10
49 13 l 1 ) =0.
If (MRI 43,43,42
42 IF(MR-I0145,45,44
45 M=MR
GO TO 46
44 M: 10
46 MR=/ooIR-I0
DU 73 N=NS,NII ,MA
N2=N
L:O
47 L:L+l
BIL):A(N21
N2:N2+1
If (L-/oI147,48,48
48 00 60 1=I,l
If (t3 l 11-9661100.150,60,50
60 CONTINUE
wRI TE( 3,77)
GO TC 73
50 I'lRI IE U ,131 ( e I II ,1 =1, l)
73 CON T INUE
NT=NT+ 1
I'lRITEI3,15)
GU TO 41
43 RETURI\
END
2
18
3
10
SUBRCUTINE INVRTR(A,NA,HA,B,NB,Mti,C)
01 HENSION A(l),Bll)
DUU I3L E PRECISION CC,D,E
KK=O
If l NA- 11 5,2,3
C=A l 1)
IFlC1l8,5,18
B( l):I./C
GO TO 24
CC=1.00
NM=NA*NA
DO 10 K:l,NM
B( K) =A (10
OZ5
026
027
028
02<;
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
050
051
052
053
054
055
056
057
058
059
060
061
C62
063
064
065
066
067
068
001
002
003
004
005
006
OC7
008
009
010
011
012
013
89
LIST OF REFERENCES
1. Anderson, T. W. 1958. Introduction to Multivariate Statistical
Analysis. John Wiley and Sons, New York.
2. Beauchamp, John J. and Richard G. Cornell. 1966.
nonlinear estimation. Technometrics 8:319-326.
Simultaneous
3. Berkson, J. and J. L. Hodges, Jr. 1961. A minimax estimator for
the logistic function. Proc. 4th Berkeley Symp. Statist. Probab.
4:77-86.
4. Berkson, J. 1960. Nomograms for fitting the logistic function
by maximum likelihood. Biometrika 47:121-141.
5. Cornell, R. G. 1962. A method for fitting linear combinations
of exponentials. Biometrics 18:104-113.
6. Draper, N. R. and H. Smith. 1967.
John Wiley and Sons, New York.
Applied Regression Analysis.
7. Grizzle, J. E. 1961. A new method of testing hypotheses and
estimating parameters for the logistic model. Biometrics 17:
372-385.
8. Hitchcock, Shirley E. 1962. A note on the estimation of the
parameters of the logistic function, using the minimum F2
method. Biometrika 49:250-252.
9. I.B.M. Corporation. 1959. Random Number Generation and Testing.
Data Processing Techniques. Form C20-80ll.
10. Khatri, C. G. 1966. A note on a manova model applied to problems
in growth curve. Annals of the Institute of Statistical Mathematics 18(1):75-86.
11. Lipton, S. and C. A. McGilchrist. 1963. Maximum likelihood estimators of parameters in double exponential regression. Biometrics 19:144-151.
12. Lipton, S. and C. A. McGilchrist. 1964. The derivation and
methods for fitting exponential regression curves. Biometrika
51:504-507.
13. Mann, H. B. and A. Waldo 1943. On stochastic limit and order
relationships. Ann. Math. Statist. 14:217-226.
90
14. Marquardt, D. W. 1963. An algorithm for least squares estimation
of nonlinear parameters. J. Soc. Ind. Appl. Math. 2:431-441.
15. NeIder, J. A. 1961. The fitting of a generalization of the
logistic curve. Biometrics 17:89-110.
16. Oliver, F. R. 1964. Methods of estimating the logistic growth
function. Applied Statistics 13:57-66.
17. Rao, C. Radhakrishna. 1965. Linear Statistical Inference and Its
Applications. John Wiley and Sons, New York.
18. Schatzoff, Martin. 1966. Exact distributions of Wilk's likelihood ratio criterion. Biometrika 53 (3 and 4):347-358.
19. Shah, B. K.·and Khatri, C. G. 1965. A method of fitting the regression curve E(Y) = a + ox + Bpx. Technometrics 7:59-65.
20. Shah, B. K. 1960. Fitting of logistic curve by least square
method. J. Univ. Baroda 9:11-14.
21. Shah, B. K. and I. R. Patel. 1960. The least-square estimates of
the parameters for the Makeham's second modification of
Gompertz's law. J. Univ. Baroda 8:1-10.
22. Smith, F. Beckley, Jr. and David F. Shanno. 1966. An algorithm
for the estimation of nonlinear regression parameters. Annual
Meeting of the A.S.A., August 15-19, 1966.
23. Stevens, W. L.
267.
1951.
Asymptotic regression.
Biometrics 7:247-
24. Turner, M. E., Monroe, R. J. and H. L. Lucas, Jr. 1961. Generalized asymptotic regression and nonlinear path analysis.
Biometrics 17:120-143.
25. Wald, A. 1943. Tests of statistical hypotheses concerning several parameters when the number of observations is large. Trans.
Amer. Math. Soc. 54:426-82.
26. Wilks, S. S. 1932. Certain generalizations in the analysis of
variance. Biometrika 24:471-494.