.
,
THE SINGIE PROCESS LAW:
A STUDY IN NONLINEAR REGRESSION
by
Maloolm E. Turner, Jr., Robert J. Monroe, H. L. Luoas, Jr.
"
. ' ~e
'I(
c,
...w:
'.
..
This researoh was supported in part by the Revolving Researoh
Fund of the Institute of Statistios, of the Consolidated University
of North carolina through a grant i'rom the General Eduoation Board
for' researoh in basiostatistioal methodology.
Institute of Statistios
Mimeo Series No. 235
July, 1959
...
ABSmACT
The Single Process Law: A Study' in Non-
TORNIiR, .MALCOLM ELIJAH, JR..
.
linear Est:iJnation.
.
(Under the direction of ROBERT JAMES MONROE and
HENRY LAWRENOE IDOAS, JR..).
Some
~ortant
d1.t'ferences between linear and nonlinear regression
models are described using a geometric device, due to R. A. Fisher, of
representing all possible samples as points in an n-d1mensional observation space. For illustrative purposes the two-d:lmensional case is
considered in some detail. Differences between the classes of models
noted are (1) orthogonality of sums of sql1ares (represented as square
distances), (2) d:istribution of sums of squares, (3) existance of sufficient statistics. EssentiaJ.l.y nonlinear models (1•.2,., those not capable
.e
of being linearized) are characterized by non-orthogonality, unusual
distribution of sums of squares, and unavailability of
tics.
It is pointed out that
esse~t1aJ.ly nonlinear
~ficient statis-
models are most often
encountered in science and should therefore be given due cons:1deration.
Attempts to study' the properties of these nonlinear models are hampered
by ·the fact that although there is only one kind of linear model there
is an 1n:f:inite variety of nonlinear ones and hence there is a need for
the definition of classes of nonlinear models.
The most interest:lng ones
will have simply interpretable d1:fferent1al equations since the power
of description of these latter is greater.
An interesting fam1J.y is
given by the eJq)ression l' = .( + ~ (x - yO) 6 + e which arises from the
differential equation d
11 /d
r=
6( ?J'""--4/ (~ ;""'(6). Since by taking 6
to be vat'ious values we obtain special processes such as the Itexponential
process", ttlinear process l ', ltparabolic process u , "inverse-square process tt,
etc. 'rhe name s:InW process
'.
...
l!1i: is suggested for the generic model.
A:fter a discussion of ana:Q'tic properties of the special cases certa:In
statistical problems concerned with discrim:i.nating the special cases
4,'
.~
are taken up.
Computational schemata. for obta1n:Ing max:tmum. likelihood
estimates and asymptotic confidence l::Lm1ts are set out.
In order to
:tJJ.ustrate the model-build1ng potential of the s:Ingle process law, as
well as to illustrate the est:imation procedure, an original theory of
middle and long-distance track records is sketched and numericalJ.y
fitted to the world records for 1957. It is concluded that lithe excess
of average speed over the asymptotic value is inversely proportional to
the 3/7 th power of the time of the race. II
It is possible that the ~­
totio confidence limits will not be satisfactory for small samples.
.e
Three exact tests of significance and their associated confidence limits
are described.
It is thought that one of these, the parabolic test, has
not been recorded previously and that this test may prove to be useful.
Application to the track record e:xanrple has been made. Review' of some
important results :In l:Inear path regt"ession is made :In
o~der
to conven-
iently extend the results to the nonlinear situation :In which the single
process law describes each connection in the causal network
0
Computa-
tional methods are described which are s:imple generalizations of the
uniw.riate techniques.
The surprising fact is brought out that problems
of ident1f'ication are the same :In the l:Lnear case and :Ln the particular
•
nonlinear case stu.died and for this reason certa:Ln estimation algorithms
are also valid for the nonlinear system. A sampling investigation has
been carried out :Ln the univariate situation in order to
~lore
properties of the maximum likelihood estimates of the :Lndicial or
the
e:q;>onential parameter 8. In particular, comparison of the finite
..
variances and the asymptotic ones have been made. so as to 3udge the
adequacy of the asyinptotic methods. It appears that, with a 2% or
smaller coefficient of variation at the center of the curve and with
at least ten
equidista.n~ spaced
observations, we may wst the large
sample techniques. Larger rates of error will not permit resolution of
the special forms of the single process law and hence are of little
practical interest to the model-buUder. Brief presentations of a
cubic approximation valid for large "f6 and a s:1mplified estimator based
.~
upon a finite differenoe appro:x::lmation for the derivative are given in
the appendices- •
.
"
o
v
1.0 TABLE OF CONTENTS
Page
:
LIST OF TABLES • •
•
3.0 LIST OF FIGURES. • •
4.0
0
•
••
•
0
•••••
o
•
•
0
•
•
INm<DUCTION • • • • •
4.1
4.2
• • • • • •
••••
0
•
•
0
•
0...
• • •
•
0
•
•
•
•
•
0
• •
0
•
0
•
• • •
o
•
•
0
•
viii
.0.
1
o
••
0
o
o
0
• •
0
0
•
•
•
0
•
0
0
•
0
••
•
0
•
0
0
•
•
•
•
O.
0
0
•
0
0
•
9
9
11
13
•
•
0
O.
0
•
••
•
0
•
•
0
•
0
0
•
•
O.
LJXELIHOQD 0
•
•
•
•
0
o
•
• • • •
18
IntroductQryRemar.ks • • • • 0 0 • • • • 0 • • • • • • •
Maximizing the Likelihood • • • • • • • 0 0 0 • • • • • •
The Method of First Derivatives
An Iterative Scheme for the Single Process Law • 0 • • •
Restrictions on the Iteration • • • • •
.A Numerical Example
18
18
0
0
•
•
•
•
•
0
0
0
0
•
0
0
0
•
.0
•
0
0
•
0
0
0
0
•
•
•
•
0
0
0
0
0
0
0
0
0
0
0
0
0
•
0
0
•
0
SUMMARY OF ASIMPTOTIC RESULTS
16
19
21
25
26
28
7.1 .Asymptotic Variances
28
7.2 Asymptotic Confidence Idmits
B
9
0
6.0 ESTIMATION BY MAXIMIZDlG B
8.0
2
000
0
0
1
••••••
0
7.0
0
• •
•
5.1 Need for a General Approach • • •
5.2 The Single Process Law
5.3 Discussion of the Curves • • • •
5.4 A Geometric Criterion of Ollrva'b1re
5.5 A Tabular Summar,y
,:e
•
Objectives of the S~ar • • • • • • • • • • • • • • • ••
Some Remarks on the Differences between Linear
and Nonlinear Models • • • • • • • • • • • • • • • • •
THE SINGLE PROCESS LAW •
•
• • • vii
•
30
'l"HEORY OF MIDDLE AND LONG DISTANCE TRACK RECORDS:
.AN EXAMPLE OF THE SINGLE PROOl!SS LAW' 0 0 0 0 0 0 0 •
8.1 Uses of the Single Process LEw .'
8.2 A Theory of Track Records • • • • • ~ •••
8.3 Fitting the Theory to World Record Data 0 •
0
0
•
0
0
31
31
31
•••••
0
•
0
•
••
0
•
•
•
•
••
•
•
•
•
•
••
34
•
0
37
O.
37
,
9.0 SOO EXACT TESTS
~
9.1 Introduction
0
0
CONFIDENCE LD1ITS
.•
•
•
0
0
~
•
•
•
o
•
• • ••• • •
•
0
•
•
•
0
0
0
•
0
•
9.2 A Test of Assigned Nonlinear Parameters in the Case of
Replication
9.3 A Test Based upon the TEl¥lor t s Series Expansion. • • ••
9.4 The Parabolic Test for Nonlineari'for • • . 0 • 0 • 0 • • •
9.5 Confidence Limits for the E:xponential Parameter in the
Track Record Example Based upon the Parabolic Test. ••
0
0
•
0
0'.
.'.
0
•
••
•
•
•
0
•
•
0
•
••
37
39
41
42
vi
1.0 TABLE OF CONTENTS (cont:1nu.ed)
.
Page
~
10.0 LINEAR PATH REGRESSION • • • • • • • • • • • • • • • • ••••
46
10.1 The Orig:lns of Path Regression .Analysis • • • • • • • •
E~ations • • • • • • • • • • • • • • • •
10.3 The Reduoed Struotural E~ations • • • • • • • • • • • •
10.4 The Equations of Identification • • • • • • • • • • • •
10.5 Estimation in the Case of Under- and Just-identified
Schemes • • • • • • • • • • • • • • • • ~ • • • • • •
10.6 Classification of Path Schemata • • • • • • • • • . • • •
10.7 Some Examples of Particular Path Schemata • • • • • • •
46
46
48
49
NONLINEAR PAT'H REGRESSION
57
10.2 The Struotural
ll.O
• • • • • . • • • • • • • • • • -. •- •
50
52
54
The Structural Equations for Pa~ Following the
Single Process Law ••••••••• _••••••••• 57
ll.2 Identification. • • • • • • • • • • • • • • • • • • • • 59
ll.3 Some Examples . . . . . . . . . . ; . . . . . . . . . . 61
ll.4 The Gaussian Iterant for Max:1mwn Likelihood Estimators • 63
ll.l
12.0 A SAMPLING mvESTIGATION • • •
..• • • • • • • • • • • • • •
-
66
12.1 Objectives • • • • • • • • • • • • • • • • • • • • • • • 66
12.2 The Saxnp1es • • • • • • • • • • • • • • • • • • • • • • 66
12.3 Estimation when ~ is Known • • • • • • • • • • • • • • • 67
12.4 Estimation when 11 is Not-Known • • • • • • • • • • • • • 69
12.5 Discussion and Summar,y • • • • • • • • • • • • • • • • • 71
13.0 SUMMARY AND CONCLUSIONS
14.0
• • • • • • • • • • • • • • • • • • • 73
LIST OF REFERENCES • • • • • • • • • •
• • • • • • • • • • • • 75
15.0 APPENDIX A. A POLYNOOAL APPROXIMATION • • • • • • • • • • • 77
16.0 APPENDIX B.
A SIMPLJFIED ES'rIMATOR.
• • • • • • • • • • • • • 79
2.0 LIST OF T.A.BIBS
Page
Some special £orms o£ the single process law • • • • • • • • •
.
•
Constants £or special extinction curves • •
o
0
o.
0
•
o
•
Some world track. records £or 19 51 • • •
•
0
•
•
0
0
• • • • 34
43
Values of·t for va:rious hypotheses.
10.1.
11
0
o
•
••••••
Classification of path schemata ••••
0
0
•
0
0
0
0
•
•
••••••
•
0
•
0
1202. Estimated biases and va:riances of the ma.:ximu.m likelihood
estimate for delta • • • • • • o·
0
•
•
•
•
0
•
'"
• • • • • • •
••••
12.1. Th:irteencutves chosen for investigation showing
oomputed'values of eta • • • •
0
• • 17
•
•
•
•
•
0
•
•
•
68
•
•
•
•
70
. v:t:l.1
3.0 LIST OF FIG11R!S
"Page
4.1. Diagram showing anaJ.ysis of distances for a line&'l:'
mo'deJ. • •
4.2 •
0
•
e-
•
0
0
•
0
•
0
0
•
•
•
•
0
0
0
o·
•
•
0
•
0
•
•
0·.
o·
•
•
0
•
•
•
•
•
0
•
•
•
0
•
•
•
0
•
•
..
0
•
0
•
• • 12
•
0
0'
0
9 .1. Determination of con:f'1dence 11m:Lts for the track record
e:xample using the parabolic test method • • • • • • • • • •
0
5.2. Relationship between the
~ar8llleter
6
0
•
•
•
•
•
7
0
•
0
5
0
ft-
SpeciaJ. exliinction curves • • • • • • • • •
'~.".'
0
Diagram showing anaJ.ysis of distances for a nanline&'l:'
mOdel •
,
0
A. criterion and the exponential
•
•
0
•
•
0
•
•
0
0
•
•
•
.'
•
•
15
44
h.l
~jeot1ves
of the Study
This monogr:oaph reports the reSlllts of a study dealimg with a class
of nonlinear regr:ession models oalled herem
~
lQs-expoment:ial model.
~
single process
l!! or
There haw been several spec:i.tio objectives
of the s'b1c%v'. First of all, we wished to e2plore the elementar1'
JJrtic geometry of the curves. How cio they bebaw? HcJw
CaD
th87
aube
ollaraoterized?Section 5.0 is devoted to considerati_ of this sort.
A s.cOD! objectiTe was to desoribe and record. methods and formulae tor
ob1'AW1:1Jlg the ma:x:bmzm likel1.1lGod estimates aDd their 8B)"1Iptotic oonfidence llmits. Especial regard was
w begiwn
to thee:xponential param-
eter lriIdch. serws as an :f.ndex of tbe particular class member, that- is,
'.e
to the parameter wldoh might be used for moclel-l:IllUd1ng. Suoh methods
aDd fG1"JllUlae are foud :1B lectiODB 6.0 ad
7.0. A 't1Ltrd objectiTe was'
to :t.Dvestigate by'Monte Carlo procedures the properties of the ma"dllQll1
likellhood est_tea for a small sample. In particular we wished to
find out haw bad the asymptotic variance might be for purposes of describing confidence
1mt.s about the EaPOneDtial pa:tameter. Can we use
as,mptoticresults for the purpose of empirical model selection"l Th1s
part of the stuq is described in Section 12.0. A fourth and f:lRal
orig1nal objective was to extend the method of path regression palIsis
to COTer the case :in which each casual pathwa\r behaves according to the
single process Jaw 0
After a brief review of the l:lDear renl ts in Sec-
tioD 10.0 su.ch am exteBsion is g:Lven in Section 11.0.
'e
2
In addition to these four origiul objectives several other points
concem1ng the s:Lmgle process law were investigated. .An orig1nal tlJeory
of :m:i.chne and long distaDce taoack records has been developed in order to
:UJ.u.strate the model building potentialitr of the siDgle process law.
This theory is outlined h Sect.ion B.G. Seme e:xact tests of sip:L:f'i-
cance and coDf:Ldellce regioBS for the single process law are presented 18
Sect:ton 9.00 A cub:Lc approx:blLat:ton to the s:f.:ngl.e prooess law and a
sblpl:t:.t::Led est:hu:tor based upon f1D:L te differences are
'br1~
described
in Appendioes .A. and B» r8SPeotiveq.
4.2 Same Remarks on the Dif'fercmces between Linear and Nonl:lnear Models
RegreSSioR models ma:y be conveniently grouped into three classes:
In the
(1) linear, (2) pseudo-nonlinear, u.d (3) essentitY;( DoB1ineF.
·.e
first case the unlmcnm par_eters each ooeur to tke first power and the
errors are simpJar additive. ID tJae secoad oue these co1'&d1t:tou
JJIq
Dot
hold lMt l1JleariV can be achieved either b;r reparameter1Hticm or 'b;y
tn:ns:fonnat1on of the variables. These changes must be such. as
1;0
either
. leave aa addit:Lve error still additive or to produce additiv:tty wheD the
e~r
was not orig:b'tal1y so. !here rema.has a large class of models 'Wtd.ch
are neither linear nor can be made linear br
tIJq
cOJllVEmiemt device 0
Tkese wnsti-mte tbe tb:1rd class and these models mq 'be teZ'1lled eHEm-
t:talJ.:t llo!llDea:r
noD.1.illearl:ty 0
1n order to CGrmote the ftmdamental Datura of th.e
Unbapp:i1yi most plv'sical ad biological processes are best
descriBed 'b,y tAese essentially nnlinear models.
fut is 11afortuDate ma:y be 'seen by resort to
R• .4.. Fisher (1939).
811
The reascms wlv" this
explana:tory dev:tce due to
two o'beervat:1ou 71 aDd 72 obe7:1Dg
Let us oons:fderi for example i
the simple liDear medel
(4•.1)
am
where t1te ':1 are ~ ad :bdepndentq d:1str:1bl1ted
2
'2.
.
.
with apeotaUon zero ad constat'VU'1a1lce 0":1 • C1. T1Ie two-cl:ImeD-
:1 • 1, 2
11cmal space made up of aU p08S:1'b1.e· pa:lrs of ••ervat:1cms mq 'be te:naed
n. remts .t aa;r parttoula1."
the obsemUG}! space.
tn-valued ample
pNd1lces a pair (,-p 72) whioA mq be repNseaiied as a point, the cHerDUO! popt 1m the ob!er!!':1a !!!!Ice.
)Tow:1B
tJit.e liIIear eDJI,pl• •mg
couWeNd .. also laave t1Ie e2pected 'n.J.-aes
E(TI) ..
IJ;,
E(,-2) • ,~
(4.2)
Tmls, the expec-ted locus of all ..i .enat.i.en pcdnw,\l !!. matter
true. Talu.
!t !!!!It !!11
--
estUlate cd' 13, sq
l\
~,
:is g1vea 'tv the s\'raigAt liDe (_.2).- Fer 8JI1'
we haft estimated ol?!enati.-
"
72,"
A
nat ie!.
f:l;
1\
A
ad (71 , 72) falls on the l:fM (4.2). The marlmmn l1kel1hood .atimate
of ~ :Is eqa1Ta1.ent to the least ••ares est_te aDd :is foud. 8,r 1d.a1n
mim13g tlte sqaare d:1stanee
A
SSE • (1'1"71
)21+.
'2.
+ (72"72) • .
This m:ba:bmm sa of sqaares :J.s cle&'rl1' obta1ned by taldDg t!Je perp_
dicuJ.ato to (4.2) pass:1mg through (7 , 7~. See Figure Q.~1. 'l'1Ie
1
A A
.
estDated point (71,72) :f.s thea g:l:,n '" the :f.Dte1'seetiGa of (4.2) and
1\
1\
A
the perpendicular. Then p - 'Tl/x,. - 7 2/~.
Let
'"-S
'U
now
CODeicier S01I18 ~
Jpp'th'l:ta" for .:ample, P •
Q.
We
get the hyp'thetical observationf
A
".
01'20 -0
The s'lQaI'e .d:J.stance Mtwe. t1ae est:1mated point and the Jvpothes:J.sed
po1nt :l.s o'brleuslT
and the sqaare distance between the obsenatioa poat and the}vpoth....
s:J.sed point is
4
4 20)2
SST • (71 10)2 + ("2
The tkree distances
'tt":iagle with
VasE, VSSR,
VSS'l' as
aDd
VSfJr
CQ1lStitute tle sidees of a ri.ght
b1PoteJ.nlse. !he P7tbagorean theorem theD holdEJ
g:t.T.l.Dg
SST • SSR. + SSE
We see that ••. SUlof esClJ.'lares for the distance betwe. observation ad
~otltes:ts
is decomposed iBto two _thogga1 compo_ta, ODe correspond-
:ing to iDe d:istaDce DetweeD esttma:te 81IGl Jvpo'thes:l.sand the other between
eest1mate ad obl!lervation. It is alIIo we tbatboth SSR. ad SSE are·
8\1IlIS
of esqu&red aormal deviates when the DUll lvPoth.es1s is trae ad Ia_ce the
USlla.1. d:l.stri_t:tcl)D
.-17
:l.s applicable.
The l:1mearity of (4.2) tlIr ___
4
3
2
1
o
Fig.
4.1
o
1
2
3
Diagram. showing anaJ.ysis of distances for a linear model
aceOUllts for the fact tlIat our estimates are nff1c1at s1aoe 1IDCler
::
these c1rcwllJJtaRces tlae 1at'ormation cecerD1Dg , depends·OJ'I1T upe SSE.
In the ease of aD esseat1a.1.:Qr ncmJ.1Dear model the s1tuat1oa is
..~ d1f'fernt.
TlIe mod.el
x
:11 . , .
1 • 1, 2 and
8
1
1
'
+~,
as 'before, is illustrated :lD Figure 4.2. We haw
1(7 ) • p;'
1
1(7 ) ..
2
·.e
It
p"2
Xt/;. -.2 the upected 10. . of o'bsenat1a p01llts 18 a paraala.
JbIr
118 eo_1der
the JIIIl1 lVPot1lesiB p -0. 'fhen
A
;y, .Q
20
¥e. Dote (1) saT ;
sa + SSE,
(2) SIR. and SSE are
!!i nms
of s~ed
normal dev:tates UDder . . .n bT,pothes:1s ad __e tlB 11Sl1al d:ls1ir1-
1lJt1t1on theory ww1d ut appq
8'9'8D
11" the lack of orthogonaliV were
allCMtd for I ad (3) a ntf:1c1ent est:lmate for p dees at ex:l8t s1ace
t.he precision of
8IlpOB
-
mrr estimate _ t depend
the nrvaiQre of (k .3).
DOt oD1T l1pem SSE 'bttt also
KDGJH'ledge _f tJd.s eunature wnld have
to .. 1UrJl:1slled b.Y appropriate aae:Ul.arz statistics in the absence of
direct knowledge of p.
7
:
..
3
2
1
o
Fig.
4. 2
Diagram showing analysis of distances for a nonlinear model
S
The rElll81"ks 1ft this seetin app11'
pae~
to aUl1aear 81d . . .
l:Inear models aDd to samples of a:DT size. We see that noD1.:I.nea1' models
d:1f'fer 1D a .1te :f\mdameDtal
wa:r
from liDear .Gdels.
It is 1Dd.eed
unf'or1aDate that the majority of models 8DCftIltered :111 ....n;.developecl
areas of researca belO1lI to the Glass ....t we tea 88sent1."T ncml.1aea:r.
:
$.1 Need far a CkmeralApproach
J.a·.caested 111 Seot1an 4.0 there 1s a oaplete
-.
aad ..t1~1
th80r7 of .t18t1081 inter.ce !II the ·oaae of 11Jaear models am!, lit,.
traaltorDl&tlon" 1n the ca. of pseu.do-RoDJ.1llear models.
DO noh complete tbeory
c~ ~
met.
Qr:a 'the 0~a.:t7"
for esleDt:LaJ.l,. Donl1nea:r model..
.'*7 .wiDear DlOcle1
is a ua1...e problem ad mut be
studied .!!! !!!! when W ..ellc.s are to be drawn r~gc'd1ng a
algebraic fcom.
!e a
part1f:aJ..ar
It 1. apparent" tberefare" that there 1s ccma:l.derable
ad'V8zrtage to be gained 1nstlldy.1ng models of lOlIe g&l1eraJ.1t7 although
the degree of laaraJ.1t7 18 liDd.ted b7 the great amaurt of labor :1Imtl'V'-
ed :1D 1Irvest1gat:trag
able.
~e:x: S)'IItems
with the pr:11l:l.t1ft methods . . a't'aU-
Systems of models sat1s:r,:lDg rimple dif'fererat:LaJ. equatiol1s are
p1"8:tarable to other systeu as objects of stud,. moe their power of
~caJ.
descr1pt1on 11 greater"
$ .. 2 The S1Dgle Proc.ss Law
The above c0l1s1derat1oJlS mo't:1:"ated the present :1nvest1gation" Thus"
a system of models val sought which
(1) was somewhat more .genaral tbaa that unall7 app11ed 1Il pari1ft-
:tar s1tll&'\1es;
(2) . . not toe complex ...
cma.iR too _7 pa.ttametera f«r
practi-
cal :1Jrftstigat1oll;
(3) ad vas gehar&ted b;y & rdaple meaa1nghl dUferent:Lal etpJatioll •
.After lDspect10n of a numb.. of ca»d:i.dates the follow1Dg model
was
10
selected as being promising:
-.
where y and
%
~,
are observables} IJ.,
T' &11d 8 cre constut paraa8t..s and
e :1s - a random element
- normally
distributed. with expectation zero ad..
.. .
.,.
.-
.
2
constat 'V'8.'r1ance a. Thus, this 1, a clals:1caJ. regression model of the
esaemt:t.a:L17 nonlinear var1et7' Thill model resultl frOll 1rltepo.":LDs the
d1:t:terent:taJ. equation
.. .,(
~. ~t
~ 1'8
d;
aDd r~acing ~ 'by 7-£ and ~ b7
-
%'0
.~
eel by the proportional1ty constant 8 0
-.e
(5.2)
Datee of the procen ia detC'.lll:la-
:a,.. bepect1Gn
it ma7 b,
seeD
tut
the process is linear i f 5-1" 18 f&1I8drat:1c :1t p2, :1s hy,perbo11c :1t ....1,
In addition the FHe..
and follows the merse squre law :1t ..-2.
""II
I.
to'R'rd the e%pOiteDt1al 11m:tt
,
7 • .,(+
~e
-ziT +s
as I tfmtis toward iDf1Dit7. .Tb18 series of model. determ1Red by a-.gle
1
pazoaEter is best depicted :in tEll"U of a :pa:rameter 1- which til the
iJrvene· of 1
0
See Table
5.1.
u
Table 5.1.. Some iJpee1al :forma of the tingle process law
:
'-1
Algebraic Form
Model
.L:
-.
y. -< +
.. ..5
y--<+
'T. -< +
1 ..0
S
@
<:J!I'2y)2
+1
-
pe-xlT + s
Quadratic parabola
'T. -< + p(x-2T) 2- +
straight line
y. -< +
The more gemeraJ. calle 6
,.e
JoT +
>0
,Cx-y)
S
+ S
i8 a gemeraJ.izat:l.on of the regresa10n ...:Lope
of the well known "allometric equation II .. For conftnience the paezial
lIMel (5.1) will be referred to henceforth as tbe single :process
l!!.
-
!biB choice of terms 1s not meant to :tmpl,. that all male ph,..ieaJ.
.
processes cu be_bsaed
.
80
dmply b7 equation (5.1); however, it i.
telt that . . g8rae:raJ.:l.ty should 'be ftggestecl :1n the
chGBeD
:name.
:b
altEII"M.t:i:n name cc:niLd 'be the log-e;ponent1al model mee (5.1) . . 'be
written in the form
5.. 3 ]I)1sRs81_
Gf the Curves
T'her'e i8 a smooth tranl1:t:l.oD .t'J:t0Jll
pt-Gce811
ODe
spee:laJ. ease of the s:I.iaIle
law to .other. 1'1da:ract isngpltecl b,.1Pipre
S .1. • ......
t'hat the e:xponential law wh1cn has- a single asymptote .- 18 the trau1ti9
.
e
~
-.
-.
I'
II
!
I
I
!
I
i
"b
j.
~
,
~
II
II
Q)
g.
to
(J)
fO
,
!
6
j
~g
~e
~
r-I
•
'l.t\
•
~
btl
-------------------
w:t.th
:
lICUl"Y&ture" JII'OIN88J:nt17 increases . . ODe pt-oeeeda
De a..,J:Ilptne 0
.:f'rom the straight line through the pa:rabolaa" the expcment1aJ." aa4 lIpward
tbr_gh the varins h,perbelas. The
Jl1(;)RotOD1cal17 iDcreaaing or
pDabolas"
.r
~b&las
momotoDicaJ.17
and
the e:xponeDt:t.aJ. azoe
decreasing
fu.nct:lo.s.Ds
en:rse pass tbrngh amaxhnm or m1lWm.mJ how..,.., p-eateet;
:tutereBt :1. centered
:I.D e:lther the moJlO'toD1cal17 iDereas:l.ng sep.l8lllt .f tbe
"
"
"cu:ne as :tn"growth" processes or :b.l the _DGtn:I.oa11y decreaaiRg 88pleJlIt
as :t.n -deeaT' or "ctiaGttea- precesses
ed to tlJoseC'tll"'l'eS ffl11! which d?j lei;
cases can 'be generated
0
The present 41lftlsid. ::I.. Um:1.t,
< e .ad
2
d ~ /d'
2
~ () abee the otlter
.f'rom thosest'tldS.ed lq linear transformations of
tile a:es .fhat is, oDl,. tile ezIi:bot:ten ,. decay form of the tam:l17 1s
eons:Ldered explicitly, DO 1011 of generality be:tng emgemderecl t1aereb7.
".e
For en,nl.oce we
wm
take
x-T6 to be alwa,..
areat.. thaD .. equl
to
zero.
504 A Geometricariter:LoJ1
We ba'veseen
of OarTature
that the parametED" S measures the tendency to n:ne. It
would 'be co:avemerrt to 'be able to a p ~ 7 pick out the properS (azact"
henee !he proper member of the fam:L1.y~ by some form of graph:lcaJ. ~.
l'm"or'tunately, the parameter S :t.s singalarly 1msu1ted tor this parpose.
HoweTer"
a s1mple
tr~!OD
exl.sts wtd.ch prO'9':ldea a eoravedeJrt taeleE:
of C'D1'V&ture amd wbich bas a silIQUe grapldcaJ. :lnterpretation
:ru.nct1ouJ.
equation
0
We have the
14
e017ellpOJldiJag to (S.l). When ; . 0 the "in:!.t:1al 1 ftJ.ue of
'1 18 at..... b7
The roue Gf ~ correspoJlCl1Jlg to (~(/) -.1.)/2 _,. b. termed the bal.f-lUe ad
18 g:tTeD by
8''''&'1''17.. the "f'8llle of ~ corresponding tG
th:ree-pa:rt.....1U..ct is
Let us
1IOW
(11. -.1.)/'4 -7 be ca11ec! t1Je
giTeD \)7
clle.f'iDe a new e:riterion 1. f . the
~..._ ~...ll:f'. by
cnD"'geS
found. b7 tiv.14illS the
thehalf;"l:1:re. 'lIms
~1S/~ rJ"
• 1_;-'l/e .
1. •
1_2-1/"
'1'1ae relatleaabips
Flgare
S.2.
(rJ
0
4) ad (S .S)azoe cba:rted :fer 'U1e
'Y8ri.01lS
cu:rves b
e "
,
e
-1
...
I
I
'"~
parabola
exponential
inverse squ,a,re
...1
~
mroerbola
~
I
I
I I I I II
~
~
,
~
inverse cube-root
2-
1.5
1.7071. ..
I
I
3
2.4142 ...
I I I
.
-2 inverse square-root
-3
e
"
-'-"~
6
o
.
I
4
3.8284 •••
I
5
I
I
6
~
I 1 • l
7
8
~
9 10
A
6.6568 ...
Fig•. 5.2. Relationshijf between' the A criterion and the exponential parameter 6
~
\.Jl.
16
We now can determine the particular
;
course
0
cnD"'9"8
(6) .frem a plot ot tot8 c, '. '
J. more general criterion is read1ly fOUDd to be
'llkere;pl is the 'V3J:ae of ;cGft"8.,.nd:t.lll to any f'ractl1e-l1f'e ad ~
cOft"e.,.Dd8to '8D'T ether .fractUe-l1t8J aDd P:1. +
':1. •
P2
P2 + CI2 • lo .
505 A TUalar S'a1rImI.t7
We will term1mate this prnatat10n by giv.trtg a table (Table 5.2) :bl
Which a mraiber of important cha:rarieri.8tios COe S\111IPJ8:rised tor epeo:La1
.".diea n:rves. It is seen that the parameter· ,T',.PpeeSt'.q the Itpobt
of e:z:biaetia 8ll for the straight linel theB8tln'ee-qurtC"s life lt f . 1;1\.
pa:ra1M1a, and the'fIIbalf-life ue ffW tile reetuplar lqper'bola. '!'he
t!iDtegratiea eonstaat lll
,
d_les
a8
thel8bitial
~loeit7.
ffR the
stra.1gbt l:ble ·"as the "imt:1al level llll f . theexpomential.
,
e
Table 5.2.
6-1
1.0
-5
Name
of Curve
6
P
.".
straight Line
1
<0
>0
2
>0
>0
Q)
>0
>0
-
Inverse Square -2
Law
>0
>0
- 2y
-1
>0
Parabola
Exponential
0
--5
-1.0
Rectangular
Hyperbola
-1.5
-2/3
-2.0
Asymptote
>0
Inverse Square -1/2 >0
Root.' Law
>0
- y
>0
-¥
>0
e
e
"
a
Constants for special ext.inction curves
b
~50
c
J..d
~100
"noint. of ext.inction n = ~7l'f50
Init.ial
Velocit.y
Init.ial
level
~25
"bal.f-life"
~75
- (3y
y/4
y/2
1.,./),
.,
1.5
(2-(3)y
(2-{2)y
y
2y
1.7071
- 4,By
(log 2}y
(log 4),.
--
2.0
-l!
-'-
2.h1lJ2
2
4r>Y
4~2
(3
.(log
(1
....lL
p
3{4-,'19
Y
~)y
{3-2)y
y/3
y
,g(8D -3)y
3
~
-'2'
.
3
7y/18
Vy/2
(2 {2-2)y
y
-¥2{2-1)y
3,./2
2y
~y
---
15y/ 2
--
3,.
B
y
_....lL
4y3
(d
3·0
-2
y
3·8284
-
p
,. ?j4l/9
5.0
_-1-
,..;::J2
a The curves all have t.he following characteristics in connnon:
(1)
(2)
(3)
(4)
Finit.e. posit.ive init.ial vaJ.ue of 'f(r= 0)
df /df~O for all vaJ.u~s of
d
/d
~ 0 for all vaJ.ues of 'Y
IlIin
-< except. for the straight line vaJ.ues for l~hich we will ignore be~nd the point.
1=
b
~ = y6
c
IflOO = 6y>
p
(~/6)
d
J.. ..
(21 / 6)
(1 _
2
2
J-I6).
6
p + q
=1
6= -
log 2 ;
10g{J..-l)
(If=
y>
7 =-<).
>0 •
_ 1
_ -}-/6
•
for _
Q)
<
1:
<+
(J).
>.. > 1 •
6
t-'
~
6.1 IDtroductory Remarks
In practice, ome 11.81' be confronted with a situ.tion like this:
are a'ft11a\)le
whic~ are bel1ned to foUR' thes:mde process
.
.
however, the parametric 'V8lue of' the apo_t
<!•.!.,
.
na~e
of the process is unknown.
l!!
data
(S .1) J
S) govermng the
.
ThwJ, S as well as the otb.er param-
eters _at be est:hDated from the data. Alternativeq, 6 U7 be kDon so
that es'MmatioD is c,onfiRed to -', fB, ad y. .A. .further common special .
-
.
case 18 the one in which the lhlit -' 1s kDonJ in particular, it mq o;f'tea De zero 0
In other cases the product y8 may \)e known. This chapter
:is devoted to presentation ·of a procedure for finding the ma:x::bmIm l1kel1Jaeod est:bnatee of the 'nr10l1S _1t1RatiODS of parameters whioh mq be
·e
.
UDkDcnna 18 practice. .l e1Dgle computatiODal metllad 18 fOUld fer aU
cases.
6.2 MarJm:1sUl' tke Likelihood
For a seri_ oft ebsen'at1eD8 t1ae l:1kel1hood faDetin 18
L
e •
. 1
(2110'2.)
)1/2 exp
f-
1.2.
2a""
~ [,. -
..( -
fB (Je-yS) SJ 2
I1Ta •
J
(6.1)
and thuS the "log 11ke~odlt is
where summatioll 18 over the N G'bservations 1m the sample.
pointed O11t the likelihood Us a
max;imum
As is
comm~
at the same point :1D parametric
19
space as floes the log likel:l.hoed. so 'that it is sufficient to maximize
the log likel:i.hoGdo
In the usual
W1Q"
we locate the maxi1'l1lml by dUt81"-
ntiatiag ], with respect to each unknown parameter 1D tl1rD and setting
the re8111ting apressions equal· to Bero
0
The equations o'bta1Dedby
eClUting the part:1al derivatives to zero are _Dl1Dear ad
us~
camnot De uplicit:Qr sol'ftd for the -xim::lB:1ng val:.es of the parameteN,
the IIwxiwm likel1hGOCi estimates. 11 Several daTices have been described
(for example, see Will, 1936, Whitaker aRd JtolDiDson, 194!1.; Dem1Ilg, 1943)
for f:bld:1ng the maxim:' zing ftlUII e ID ae-ral these deT1ces either
enta1l some form of interpolation :1n the param.et.'ric space or else .Glle
form o:f'successi'V$ approach to the max:lmum point using derivatives or
Qifferences o 'fhe or:l.gi:u of the former approach date 'but to anti..iV
whereas the latter approach was first emplOJed 'by Newtono Gauss appears
to bave been the first to EllIploy the second approach fer the parpose of
max:Jm'JziDg the likeli1loodo lathe case of regression R. Ao l:l.8ber •
'
(1925) :important "metllod of scor:Lngli is eqllivaJ.ent to Gauss' procedure.
Fisherlls Jlethocl is rioJa with _Dg in terms of the generaltl:Jeo17 of
estmation lNt usua.lll' :J.nvolves the tediou algebra of' dete:rm1laiag
e:xpectatiou of'ceria1D variates
0
hus.' method, geDeraJ.ly' speald.ng,
is simpler to appq bat is 170 'IDe regarded mereq as a eomputatinal
device.
at' course, once the eqll1valence of the two methods i8 reeopized
1m a special case Gauss' process :mq be interpreted :Ln terms of' the
Fisheriu WormatiGli concept.
603 T11e Met.'hod of' First Der:l:ntives
1'here are
problem.
many.~
to describe Gauss t approaeJa to the max:Im1zatio
'!'he follOwing has some meritsg
20
Let
'lIS
suppose tbat we have the repssion model
y ..
where ! is a vector
ot
unknown
supposed to be normaJ.ly and
t(@.,x) + s
parame'bers aDd
:bldependen~
It
is the randOll component
dis'tri'buted with e:x;pectatiG'D
zero and unknCMl· var1ance ,;.. I t Jl'1&X1mlIm likelihood .st:tmates
R. are
substituted fer the parameters! we have
'1' .. f<J!.,x) +
e
(6.4)
where e :ls an "est:imateU ot s.· Ther1 the est1mate of the observation y .
~
B.v the rule tor the
·e
• f(R"x) •
total derivative we get
(6.6)
where "Yi ..
e "I
Y. 9 b •
i
Then to a first approJdm.ation
(6.7)
where
The zero subscript indicates that trial values tor tl19 undeterm:1ned
estimates have been interjected.
Thus, we get
(6.8)
and
./It.
"
1\
Y , Yo + A~"'lO + AbiT20 + ... + e
•
Now (6.9) is an ordinary linear regression equation and estimates mq
be found in the usual way.
From the linear regression estimates mq 'be
21
found :Improved values for the original nonlinear regression est:lmates.
Tbu.s, the process is iterated find:Lng successive improvements.
It may
be seen from (6.9) that trial values for est1mates which are li.r:Maar in
(6.4) are not needed to pr:Lme the iterative process. This po1nt is
1llustzated in the next section.
6.4 An Iterative Scheme for the Single Process Law
The single process law is of the fom. (6.3) and mnce we may write
down from inspection the linearized form analogous to (6.9).
Before
doing this we note that the linearization is not unique. There is
nothing in the derivation of (6.9) which specifies the definition of the
parameters.
Tbu.s, we could reparameterize (5.1) as followsl
Let
1.1. •
then
(6.10)
1'6,
22
for this statement is offered bllt this form wUl be used in the present
presentation and has been used in the sampling study desoribed in Seotion 12.0. The form of the single prooess law embodying "( and p is
(6.12)
Then, we have b,y differentiating with reSPect to ao' b0'
0 ,
0
and r 0
-e
A
y.l.... c b
Lj.V
0
{ o.
0
l/r -1
-2 (:xl- -i..)
0
r3
ro
o
Now, let
z.. •
~
(x-
o . l/r
..e.
)
r
0 •
ro
c
l/r
...s.
)
r
o
d
0
0
o
o l/r-l
Z2. • (x- -Sa )
0
Z3 • (x-
(:x:-m )
1
0 . l/r
0
(x- ..e. )
0 In(x- ..9. )
r2
ro
ro
0
__
0
d-l
..
(X'1l )
0
0
c
d
h(x- ..2. ) • (x-m ) 0 h(X'1l) •
ro
0
0
J
•
23
So, substituting (6.15) into (6.14) and then substituting the result :lnto
(6.9) we get
(6.16)
where
b
bc
B • - ~ (c -c ) + ...JL2 (r -r )
2
r2 1 0
~
1 0
o
0
b
B3 • The entire procedure
(1)
;f (rl-ro)
•
o
::ts ;now as follows t
Tr:laJ. values mo •
coIro
and do • llro are somehow obta1Ded,
perhaps by a graphical method,
(2)
The transforms (6.15) are obtdned for each data pointJ
(3)
Then y is regressed on tle "new" variables ~, Z2' and Z3 to
obtain estimates of Bo' B , B , and B , and from these latter
2
l
3
improved estimates ofm and d are obtained from the relations
(4)
Steps (2) and (3) are repeated using the :improved Utr:ial"
values
~
and dl to obtmn st:Lll better values m2 and d ,
2
24
(5)
These latter are used to obtain m and d and so the process
3
3
is iterated until corrvergenceJ
(6)
After the final iteration we have
a • Bo
n. cd
r • lId
•
In order to obta:tn Bo " Bl " B2", and B we have to solve the normal
3
equations
ztZj,. zty
-
for B where
Z•
~2 ~3
1
Zu
1
Z2l z22 z23
, :sa
-
•
•
•
1
Znl ~2
•
•
•
•
Zn3
!..
(Z'Z) -1 Z'% •
(6.20)
2,
The matt'1:K: prQduct ZIZ is found to be
N
...
~Zl
~~
~Z2
~zi
~~Z2 ~~Z3
~Z3
~Z2
~Z2~ ~~
~Z3
~Z3~ ~Z3Z2 ~Z~
~Z2Z3
•
It is apparent that :l.f 8 were known the last row and column of ZIZ
should be deleted. Similarly:l.f T and 5 were known the last two rows
and columns should be deleted y1eld:ing the ordinary' straight line
regression case. If T were known the next to last rOW' and. column should.
·e
be deleted.
If -< • 0 is known then the first row and column should be
deleted bu.t if -< is known and non-zero then 1" • Y--< is substituted in
-
(6.20) and the first row and column of ZIZ deleted as before. Therefore,
one computational method suffices for fitting the general s:ingle process
law and for £itting the various special cases as well.
60S Restrictions on the Iteration
In some cases curvature :in the parametric space may cause gross1y
eDggerated corrections to the trial est:1mates to be :indicated. Such
large corrections i f applied ma:y often lead the iterative process outside of the region of convergence. Several courses may be taken to prevent over1y large oscillations. A s:1:mple remedy is to place bounds upom
the solution so that whenever a V$lue is indicated 'beyond the bounds the
'boundary'value itself is used for the next iteration. If the process
26
converges to one or the other of the bound.a.ry values then the- bounds
must be widened. Another course has been su.ggested by Wegstein (1958).
st
Inswad of using Pi in the i + 1 iteration we use the "dampedlt estimate
* • uP i _l + 'Vpi
Pi
(6.21)
w.re Pi is the esUmate of some parameter after the i th iteration and
u + v • 1. EJcper1ence has shawn the need for some such restriction on
the solution.
other types than those mentioned are of course possible.
The device (6.21) has been used success~ in the carrying out of the
sampl:tngs'bldy described :tn Section 12.0.
In sane cases it mq be convenient to Itfix"
o~.
j..L
and converge for 6
By taking several values of I.L the best est:bnates mq be found by
interpolation.
6.6 A Numerical Example
The following data mq be used to :illustrate the iterative prGcesst
x
1
2
3
4
5
6
7
8
9
10
y
162.47
108.10
81.12
66.n
55.06
49.37
39.72
33.81
30.90
.30.73
27
'1'1-1al values m0
• -1.00
and d • -1.00 were tried and the follow:1ng
-- 0
results for successive iterations were found
Iteration
m
d
0
-1.00
-1.00
.,6
.69
.67
.67
1
2
3
4
---
.23
.38
.43
- .44
---
Then for the final iteration we have the estimated relationship
(6~22)
Rarely in our uperience has it been found necessary to compute more
than six iterations.
Of
course, over-conservative use of the device
described in the last section mq produce a very slow rate of convergence.
701 .A.8'1JBP'tot1c Va:r:lances
In thi. chapter i. presented the la:rge sample remt. relating to
the variances
aDd distributions
of t'he 'll!8."x:f m m likelihood estimates.
. .. -.
-
101.tioB of the mormaJ. equations ...
-
a1'Y8D
:bI Seet1n 6.0 to 'be
t )-1 t
I - ( ZZ
Zyo
(7.1)
-
The as;ymptot1call1' 1mb1aseci estimate of
J
The
is as usual given by
or
·e
.2.
iT; { ]/ - 1
0
]1 ~ l!l:J.]Z:!.1- 1 2 ]Z2" - 13 ]Zy}. (7.3)
the element in the i th row and j th column of
1d
I has been well mown since Gauss t~t the 'Variance a~ of any
Let u.doote by c
(Z~ Z)-l
0
function of the Btas (1' say) 1s as,mptot1cally given by .
whare 1'i • .·:f'/~1 and t j •
et>/eB j
0
We
m&)"
use
&8
tbeast1mated 'V'a'riaz1ce
of the faDct10D
(7 oS)
29
:By a.ppl~g (p.9) to the form (6.10) of the single process law we
obta.1n the l:Lm1ting relations
a-:eo
(7.6)
'!'hus" from (7.$) we obtain the as,mptotio estimates
·e
S:Lm:tla:rly" for c • mId we find
aad for r • lid
2
... 2A.2
sr • c.33.
'I
,-,
I',
. : .:,":',,' ..
':
'
"
.... ,
...
.- . . ., .
'
'.'.
til
4
d
•
(7.9)
30
FlnaJ.ly, we find the asymptotic estimate of the var:l.ance of a predicted y
for givan x:
2A
I
s (y x)" s
2 .J.,
J..
~ ~ cij ZiZj
i-o j=O
(7.10)
where Zi and Zj a:t'e evaluated for the g:tven x.
7.2
Ma'x:f,mwn.
A~otic
Confidence L:l.m:1ts
likel1hood estimates are, under suitable conditions, asymp-
totically normally distributed with expectation equal to the parametric
'V8J.ues and "O'a'riances given by (7.4). This fact -7 be utUized to obta:J.n
~t:l.c
confidence 11m:l.ts use.tul under certain cirC'WllS'tances. We haTe
a - ttP2 ~ -< oS. a + tnSa
b -
ttflb .s p .s b
+
t~
•
(7.11)
•
with
l~w
confidence where t w is studEmtt S t for the .. level of l!iign1:f'icance with H-4 degrees of .freedom. Sir RonaJ.d Fisher (1939) has said
11 •••
that the ordinary tests will still benfficiently e:xact when the
radii of ou:rvature of the spaces concerned are aJ.l large compared with
the distances." Thus, it is quite possible that these "aQJllPtot:1c"
:results will suffice even for "small" aamples in some cases. Sectio&'1 12.0
1s devoted to asamplUig exploration of the adequacy of the as)"Jllptot:1c
results for certain oases.
8 00 THE THEORY OF MIDDIE .AND LONG DISTANCE TRACK RECORDS.
AN EXAMPlE OF THE SINGIE HtQCESS LAW
8..1 Uses of tb.eSingle Process Law
It is anticipated that the single process law wUl be quite useful in
e1Tlp!r.ically describ.ing monotonic "growthSI or "decay" curves.
.
Ind~
the
-
special cases are in wide use for this purpose. HOW81W, t1l1e fact that
the d1fferential equation of the slmgle process law is of an eas:1l7 interpt"etable form suggests that it will lend itself read1l7 to the theoretical model-bullding process. As an :tllustration of this point we
proceed to der1w a theory to describe the form of the world record curve
for middle and long distance track events. The general line of reasoning
is atter A. V. H1l1 (1927).
802 A Theory- of Track Records
We def:'JJ1e the following symbols:
7 The total energy (measured as volum.e of oxnen) '• •eill,OYer
resting required to run a distance in a certain time at a
g:l.'Vell constant speed.
x The distance run.
t
The time to run x.
x
!he constant speed.
Xo The greatest speed-for which the energy (oxygen) required
is not greater than the energy' ava.i.lable for an :l.ndefinite17
long period of t:ime. Thus, at speed Xo the energy required
is equal to the maxIJmml amount of o~n whioh can be
'brought 1r.rto the 'body .from the atmosphere.
r
If)
The maxLmwn rate at which oxygen can be brought into the
bod7 .from the atmosphere ..
The ma.x1mwn tolerable ~ debt be1Gnd which complete
is m.anifest ..
e~st:l.on
.32
To a .nrst appro:x:1mation we may say that
(e.l)
..
where b ud k are arbitrary constants of scale. Integrating we get
(8.2)
The constant of integration is obviously zero since no e:=ess energy is
reqUired to run a race of zero duration. Equation (8.2) cu be used to
describe the experimental data shown in Figure 11 of Hill (1927).
Hill
measured d.1rectlythe e:=ess oxygen reqUired by u athlete to run for
g:!;ven periods of time at constut speeds. This experiment, of course,
does not correspond to actual contest conditions, but does serve to
:JJ.luatrate the relation (8.2) which was not given by Hill. The meaning
o£ the relation is clear.
-
If a runner runs at his own asymptotic speed
Xo then the excess energy required is obv.L~sly proportional to the time
:run where the proportionality constant is rJ but i f he runs faster than
Xo then the energy required wUl be increased monotonically with the
~vetopnce
of his speed from the a.,mptotic speed.
Now, the speed cannot be increased indefinitely since there is an
upper bound on energy available. This upper bound is given 'by
Yo
max
-rt+]) ..
(8.,3)
For all but the shorter races (say, la'ss than 4G& meters)
the time lost
I
in atta1n1ng opt:Lmwnspeed is negligible ud itc&r1 bea,een tkat the best
t1:me £or a race can be obtained by mainta1Dl.ng a nearly constant speed
throughout the race.
It may 'be supposed that records are broken when a
33
rwmer with a great amount o£ available energy is able to select that
constam speed Which equates the
nia:x:Lmwn a'VaUable energy to the reqa:lred
energy. In this case
k
:rt + D • :rt + b(t - :i:)
o t
which upon cancelling :rt from both sides and re&'t'%'anging g:1vel
(e.,)
o
•
(0 t)-l!k
x-x
o + -D .
relating speed· to time :run. The constant.
*0'
D, b, and k are constaDt.
pertaining to the individual :runner. D is the max:lmum oxygen debt the
runner can tolerate and b and k relate to the mechanical ef.ficiency o£
-
the runner. body, i.eo, to the amount o£ work requi:red to overcome the
·e
v.L.cosity o;f the ,muscles
0
~
i. a function of the rate at which .oxygen
can be ,otten to the tissue.. Relation
pro..... o£
q
(a.,)
could be used to studT the
ind:tv.lduaJ. runner :I.n hi. pe:r:tod of training} however, our
interest at present is in characterizing the world record curve ..Although
xO'
D, b, and k are personal parameters it may be supposed that
£or world record holders
~ and
D approach the human maxLtrmm. for all and
that b and k are similarly close to the
.~
m:1nimam for all and that
therefore it is appropriate to treat %0' D, b, and k as constant parameters ;for all record holders as a :f.'1rst approximation.
O'bv:LouIly
(e.,)
is a special oale of the single process law
(8.6)
34
where
~=t
-<
=:
:i:
o
~ • (D/b)l/k
..,.0
Thus (8.6) is equivalent to
(8.7)
8.3 Fitting the Theory to World Record Data
Data for world track records were taken fr<m1 Hansen (1958).' These
data are presented in the first 3 columns of Table 8.1. The average speed
·e
for each record has 'been -computed and is presented in oolumn 4 of this
table.
Table 8.1. Some world track records for 1957&
Distance
Enslish Units
Meters
220 yd.
~01·.17
400.'
Boo ..
1 mi ..
a Source:
1000.
1609.35
2000 ..
3000.
5000.
10000.
20052.
\ .1ime
H
' seconds
20.0
. 45.2
105.7
139.0
237.8
302.2
472.8
816.8
1710.4
3600.0
Average Speed
Meters/second
Predicted
Average Speeds
Meters/Second
10.06
8.85
7.57
7.19
6.77
6.62
6·35
6.12
5.85
5.57
10.15
8.65
7.55
7.28
6.82
6.63
6.37
6.09
5.80
5.60
The World Almanac and Book of Facts for 1958 edited by
H. Hansen.
35
The events which had clearly 1nferior average speeds as well as the
shorter races
in which "starting" plaY'S a major role have been omitted.
.
.--
-
We have aSl'UlI1ed a regression model, derived from (8';), of the form
x• Xo + ( ~ t) -11k
and f.1tted this model by the
+ e
met~ds of ~ction
(8.8)
6.0. In order to employ
the iterative method a trial value of 6 • - l/k is required.
This is
qu:Lck1y provided by the use of the A. criterion. From a plot of the above
data we est:lmate the "3/4 life" to be appro:z:tmately 540 seconds and the
lfJ./2 life" to be about 80 seconds.
This gives a value of 6.75 for A. and
referring to the nomogram relating A. to 8 (F:tgure 5.2) we estimate 6 to
be about - 0.4. This value we then take as our trial estimate. :By
·e
iteration we find the maxLmum likeJ.:Lhood estimate of 6 to be - 0.426.
The ma.:xiJmlm
li~l:Lhood
est:Lmate of .( is 5.04 and that for
p :Ls 18.34.
Thus, we have the fitted curve
Predicted values for the average speed are given in Column 5 of Table 8.1
al~ng
with the observed average speeds. It will be noted that the fit is
rather good.
For esthetic reasons one may prefer to replace the decimal estimate
by a simple rational fraction.
The fraction -3/7 is quite close numeri-
cally to the estimate (317 • 0.4285) and may be used instead. In this
case the law of track records (8.9)
CQl
be stated thus:
"l'he excess of
36
average speed over the a.s,mptotio value is inversely proportional to the
3/7th power of the t1me. tt In Short, we
may
say that WEJ have demonstrated
that middle and long distance 'tt'ack records obey the uinverse 3/7SiDJi law. 1I
~othesesother
class of tenable
than that 5 • .. 3/7 are, of course, possible.
~otheses may'
about the unknown parameter.
The
be found by construct:ing cont'idenoe limits .
Three msthods of constructing e:xaot confi-
denoe l:t.mits are disoussed in the next section and munerical results are
obtained for the present example
5 • .. 1/2 is not acceptable.
·e
0
We will see that the b3Pothesis
9.0 SOMEEUOT
_'l'S
AND CONJrJDENCE LIMI'l'S
901 Introch1ction
The e:x;pression exact test is used to denote a test which rejects
-.
the null h;vpothesis at precisely' that relative frequency corresponding
to the type I error for the test.
These tests are to be d:lBtinga.ished
from asrmptotic tests which generaJ.ly' reject the null b1PQthesis at a
somewhat higher relative freqllency than the assisned tn>e I error. It
JIII1St not be concluded that in a particular case an exact test is necessarily to be preferred over an as,mptotic one since the available exact
tests mq suffer from deficiencies such as 1napprQpriateness for a particular purpose Qr inadequate power. In some cases it IDa\Y' be that a non-
pro'bab:Uistic rejection rule, such as the one based upon relative likelihood, is to be preferred over available probab:Uity tests.
In the present chapter we consider three exact tests of s1gn1:t'icance
and the corresponding confidence statements 0
9.2 A Test of. Assigned Nonlinear Parameters in the Case of Replication
Let us take 11 • "(6 as in previous chapters 0 Then the single process model is of the foll~g form for tm· jth replication of the level
of x:
(9.1)
i • 1,2, ooo,k; j • 1,2,ooo,ro B\1t now suppose that we substitllte some
other values of the parameters (say 110 and 60 > for 11 and 6 then (9.1)
would have to be replaced by
(9.2)
38
Now, if we wish to test the null l\VPothesis
we can· test for udepartures from l:inearitylt :in the
Zi
iii
(Xi -
11
0
)
°0 •
Then the
SUll'l
usual way.
Let
of squares of residuals a.bout the fitted
linear model (9.1) is given by
SSE' • Syy- Srr;y2/Szz
where
S:Lm1larJ.;y, the residual sum of squares a.bout the fitted model (9.2) is
g;l.ven by
(9.6)
Then the null. hypothesis (9.3) is tested by the variance ratio
F
iii
SSE' - SSE ken - 1)
.SSE
.k - 2.
where F follows the F-d.:1stribl1t1on with (k-2) and k(n-l) degrees of
freedom when H is true.
o
39
The test (9.7) is well lmawn :In the linear case but not so well
known in the nonlinear case.
The nonlinear application has been con-
sidered previously by Hurst1 . (195~) •
A
joint confidence region for the nonlinear parameters !-L and 5
(and hence also for "( and 5) mq be found by inverting (9.7) to give
SSE'.Tt • SSE + F11' (SSE)
where F is the tabuJ.ar value of F for the (1-11) th confidence coeff'i11'
ciento Now SSE' is calculated f'or various combinations of!J.o and 50
untU the confidence region
SSE' ~ SSE'
11
is del:1mited.
9.3 A Test Based upon the Taylor's Series Expansion
In :many cases only a single observation has been made at each level
of' I., Eo J., WUliams (1959) has suggested an e:xact test of the null
l\vpothesis (9.,3) obtained £rom the expansion of the model (901) about the
point (!-Lo' 50) 0 ThuS, we replace the model (901) by the approximate linearized form of Section 600,
(9.10)
See (6.16) for derivation.
The coefficients
B2 and B are proportional to
3
o and !-L and between 50 and 5. Henee a test of
the di.t'£erences between !-L
the null lvPothesis
1 Personal communication
40
is approx1mately equivalent to a test of the rmll hypothesis (9.3).
1\ " 1\
Linear regression theory supplies the test for (9.11).
If B , B and
1
2
A
B are the minimum varianoe unbiased est:hn.a.tes then the residual sum of
3
squares for (9.10) is given by
(9.12)
where
h • 1,2,3. When H is t.a:ue, (9.10) reduces to (9.1) and the residual sum
o
of squares is given by
·e
2
SSE' • 8 .. 8 /8
yy
ly
11
where
Then the varianoe ratio for testing (9011) is
(9016)
and F has the F-distribution with 2 and (k...4) degrees of freedom when H
is we
0
o
We IIl8\V ola:lJn to have an EmJ.ot test of an "approximatel1' equiva-
lent lvPothesis
0
A oonfidenoe region
may
be found by oaloulating F for various values
of 110 and 6 and determining the region such that
0
41
F
<
-
F
11
(9.17)
where F11 is the tabular value of F for a confidence coefficient (1-11) ..
It maybe supposed that the test (9.16) is more powerful than the
test (7,,7) for small departures from!-L o and 50 • However, this point
has not been investigated.
In the case of replication at eaoh point we can replaoe the denominator of test (9.16) by the swn of ,,·quares (9.6).
9.4 The Parabolio Test for Nonlinearity
.An approaoh s:bnilar to that of the last seotion but differing in
details is given below.
If IJ. • !-Lo and 5 • 00 then (9.1) ma;y be written
·e
(9.18)
for the case of single replication. However, in the case that the rmll
~othesis is
not true, then instead of (9.18) we would have
Expanding (9.19) in a McLaurin series and preserving terms through the
qaadratic, we get
(9.20)
Now to a first appro:Jdmation the null
Ho
~othesis
: B2 • 0
(9.21)
is equivalent to (903) 0 .\ test of (9.21) is well known and is eas~
performed. If Bl and B2 are the m.in:1mum variance unbiased est1mates then
42
the residuaJ. sum of squares about the predicted
curve corresponding to
(9.20) is given by
A
1\
SSE == Syy - B1Sly - B2S2y
-.
where Syy and Sly are given by (9.13) and S2y is given by
.
S2y •
k
~
2
Zio 3"i -
~
(~
2 k
Zio ) (~ 'Yi)/k •
The test statistic is
F
SSEI-ssE k-3
•
SSE
1
where F has the F-distribution with 1 and (k-3) degrees of freedom and
SSEI is obtained £rom (9.14).
This test too may be described as an e:xact
test of an approximately equivalent bn>othesis.
·e
Again, the confidence region is given by
F
oS
Fft •
In some cases, it m.ay be desirable to retain terms higher than the
second in the series expansion. In arry case the numerical work involved
.in performing the test is relatively light and that involved in determin-
ing the confidence region quite heavy.
9.5 Oonfidence Limits for the Exponential Parameter in
the Track Record EJcample Based upon the Parabolic Test
Values of F given by the formula (9.24) have been computed for various values of the exponential parameter 6 for the example of world track
records.
Table 9.1.
The somewhat more convenient values of t • ± ff are reported in
43
Table 9.1. Values of t
.for various hypotheses
t
5.43
,·.6
..
-.5
-.46
-.426
-.4
2.89
1.81
.67
- .20
-1.52
-.36
-.3
-3.50
Under the null hypothesis (9.21) t bas student's 'b-distribu.tion
with 7 degrees of freedom.
The tabular value of t with 7 degrees of
freedom at the 5% significanoe level is found to be 2.365. By graphioaJ. interpolation we find the 99$ oonfidenoe statement
(9.26)
- 0 .48 ~ 5 ~ - 0.33 •
·e
The detaUs of the interpolation are
shown
in Figare (9.1).
apparent from these data that the s:lmple hypothesis 5 • -
It is
0.5 is not to
be aocepted at the 5% level of signifioanoe. Likewise, the hypothesis
5 • -.33 is to be rejected.
Of all simple rational fractions only -3/7
comes at aJ.l olose to the maximum likelihood estimate although several
others (notably -2/5) would not be rejected at the assigned level.
In closing we note that the parabolic oonfidence l:imits converge,
not to the max:imum likelihood estimate (5 •
ltzero ourva'blre tt estimate (5 • -0.415).
-0.426), bu.t rather
to the
This phenomenon often occurs
when oonfidence limits are not fiduci.aJ. l:imits, as has been pointed out
by Fisher (1956).
Both est:bnates are consistent so that it can be seen
that our parabolic limits have the asymptotio Itenolosure Q or convergence
.
e
e
'
.
•
"
'
Student's
t w.tth7 d.f.
4
3
Ix
:II
,calculated value of t
.1
2
1
o
-1
-2
-3
-4
-.6
-.5
-.4
-.:3
5
Fig. 9.1 Determination of confidence limits for the track record example using the parabolic test method
45
properw while for small samples, although not possessing this property,
they do produce emct probabiliw inequalities of confidence.
Similar
remarks apply to the limits based upon the test of Williams (9.16).
,e
.
46
10.0 LINEAR PATH REGRESSION
10.1 The Origins of Path Regression Analysis
Sewall Wright (1921) oonceived of the notion and term of path
regression analysis.
The notion has been mch extended and developed
in the econometric literature. See especially Hood and Koopmans (19,3)
N
and Wold (1956). Wright himself bas come to favor the use of standardized variates" the analysis of which he refers to as ltpath analysis tl •
Summaries of Wright's views mq be found :in his review papers" Wright
.
(1934" 1954). May (1954) has argued against the use of standardized
variates even when these have a rational basis"
!.!.."
when the u:Lndependent" variables bavenorm8J. 'd1Striblltdlons.When the independent
variables are "fixed"" i..!.." have no probability distribu.tion" then
·e
standardization as Wright advocates is meaningless. For elementary'
presentations of the regression method" see Kempthorne (19$7) and 'lUrner
and Stevens (19,9).
In this section will be presented a recapitulation of the iDain
resul ts of linear path regression ana.J.;ysis. An extension to the nonlinear case using the single process law will be described in the next
section
0
10.2 The StX'llctural EqI1ations
Let us suppose that we have p "secondary" variables which are
casuaJ.ly related to one anotherand to a set of q 'tpr:iJnary" or "controllable" variables oWe :imagine that the q primary variables (denoted
~ a" ~b'
HO"
~ q) have values which can be chosen at will by the
47
e:x;per:tmenter.
The p secondary variables (denoted by
r; l' '? 2'
..., '7 p)
are then supposed to be completeJ.y detemined. We further assume that
all rates of change are constant.
'.
tives
1'7 ij
is H where
PIa
~2a
•••
~pa
-<:La
~a
•••
.(
~lb
?;12b
•••
"l pb
-<:Lb
~b
• ••
-),b
•
•
•
•
•
•
-<:Lq
~q
•
•
•
•
•
~lq
H' -
·e
Thus, the matrix of partial deriva-
'72q
•••
~pq
-------------
1ll
721
•••
~pl
712
~22
•••
"lp2
•
•
•
lp
•
•
•
•
•
•
r2p
•
•
•
•••
,?w
•
pa
•
•
•
•
• ••
~q
-----------1
-<:L2
•
~l
•••
~l
1
• ••
~2
•
•
•
•
•
•
~
~
(10.1)
•
•
•••
1
and where the "path regt"ession coefficients", .,(ij' are constant.
Solu-
tion of (10.1) yields the p Itst'rllctural equations lt
"ll--<:L+-<:La ~ a+").b-;b+ • • .+"1.q ~ q
+").2 2+-1.3 '13+ •• .+-1.p p
r;
f
1J 2~+.,(2a -;a+~b ?fb+ • • .+~q ,. q+.,(21 ~1
+~3~ 3+·
0
o+.,(2p? p
(10.2)
•
•
48
where -<:t., -<2' ••• , 1> are constants of :IJ'ltegration. These equ,at1ons
..
mq be written :IJ'l matrix notation as
-<:t.
-'1a -'1b
~ ~,. ~b
•••
"1.q
••• ~q
•
•
•
.-<
p
-<
''Pa -),b ••• -),q
I
I
0
I ~1
-<:t.2
-'13
• ••
0
~3
••• -<2p
I •
I ••
I
I
-),1 1>2
1
"1.p
~a
;b
~3 •••
•"
•
0
e
•"
!tjl
• ~2
•
•
•
~p
"q
--111
?J2
•
•
·e
•
'?p
or
•
brefore, the ttstru.cture lt of the causaJ. system is completely described
10.3 The Reduced Stru.cturaJ. Equations
From (10.3) we get by multiplication
(10.4)
49
and
~-~1 .~~
1. • A:L!-
•
(I - A2)
If (I - A ) is non-singular we get the Ilreduced structural equationsl!
2
(10.5)
T.tm.s, the secondary' variables
2f. are e:x;pressed as linear functions of the
primary variables.f-. and it is seen that the necessary and sufficient
condition for this to be possible is that (I - ~) be non-singular.
10.4
The Equations of Ident11'ication
Let B be a pX(q+l) :matrix of regression coefficients as follOWs:
,·e
B.
NOw we
wm
~l
~la
~lb
•••
~lq
~2
~2a
~2b
•••
~2q
•
•
•
•
•
•
•
•
. ~p
~pq
~pb
•
•
•
•
•••
•
~pq
state the ltident11'ication equationa lt as follows:
·1
B • (I - ~)- ~
•
(10.6)
The reduced structural equations (10.5) ~ then be written
the structural equations for an ord.::1.nary'Mllltiple regression system.
We note that it may or may not be possible to solve (10.6) for
(A:L,
A ) uniquely or at all. If it is not possible to find a solution
2
at all, the system is said to be nunder-identified!'. If a unique
solution exists the system is said to be It just-identified", and if
more than a single solution exists, the system is said to be !loveridentified" •
If
we expand the right hand side of (10.6) we w:U1 find, in general,
that some of the
ions on the
.<IS
~ IS
are zero as a consequence of the casual restrict-
{in particular, some of the casual pathways will be
absent and the corresponding
all those -;
IS
.(IS
will be zero).
Ist us agree to omit
from the rows of {10. 7~ having zero ~ IS as coefficients.
Then, we wOuld replace (10. 7) by
~l·!i '1
·e
o/2·~~2
•••
(10.8)
where each 1.i and ~ i is of order less than or equal to q-:-+ 1.
10.5 Estimation in the Case of Under- and Just-identified System
The under- and just-identified cases are of especial interest because
in these
~ases
possibilities:
•
regression type estimators are avaUab1e. We consider two
(1)
All~' s
are non-m111 and
Zj-,?j+.lj
~ - ~j
j -
1, 2, ••• , n.
Then by substitution (10.7) becOJlles
or
(10.10)
For n observations we have
·e
z{
~
z?.-
~
••
.
~
'
•
••
B' +
•
••
•
~
or
y. IE' + E •
(10.11)
Now, i f the n x p elements of the !terror" matrix are uncorrelated and have constant variance;'
the~ the minimum variance
unbiased estimator is given by the well known expression
~, • (X'X)-l X'Y •
(10.12)
Ii' in addition E is distributed as a nm1tivariate normal disA
tribltion, then, of course, B' is the sufficient estimator for
B1 •
-
In the case of just-ident::1.£ication B' may be replaced by
52
1\
B' in (10.6) and est:i:mates of
(A:t"
A2) solved for. In the
case of under-identification" it will not be possible to solve
~
~
for all elements of (~" ~) explicitly unless a sufficient
number of .! priori oont.:t"aints can be assumed.
(2)
More often than not restri~t10ns on
some
XIS
~
A:t and A2 will
force
's to be zero and then we must omit the corresponding
from the estimation equations.
In this case a separate
regression analysis mu.st be performed for each y variable.
Henoe" we get the :individual est:bnators
(10.13)
-e
h
remarks following (10.12) conoerning distrib\1tion and iden-
tification apply s:tmila:rly to (10.13).
For trea1:ment of the over-identified oase see Hood and Koppmans
(1953). Refer also to Tttrner and Stevens (1959).
10.6 Classification of Path Schemata
There are several types of oasual relationships which are conveniently distingaished. First of all there is the situation in whioh a
single secondary variable (or response variable) is determ:1ned bT one or
I/lore
pr~
variables.
This is the case Gf ttordinarytt regression" the
single stntctural eqt1ation corresponding to that of the usual multiple
regression" model. A second case is the situation in which two or more
seoondary variables are joint:l;v determined by some or all of a oommon
set of pr:t.:m.ary variables. We tenn this oase the ease of ltjointtt regression. In a third situation there is a cha1n of cause and e££eot leading from one or more pr:l.m.a.ry variables to a secondary variable and then
on to stU! another Itseoond.ary11 variable.
This oase we term the oase
of "0ha1n" regression. A £inal type involves cycles of causation or
Qfeedbaokll from one response variable to another and back again after
possibly passing through one or more other seoondary variables.
is the oase o£ "oyclic 1t regression. All of these oases
This
indiv1du~
or in oombination have been synopt1oalJ.;y' treated in the previous sections
of this chapter. Here we wish only to note that the several oases are
d1~tingu1shed by
having different kinds of (I - .1.2) matrioes. For this
reason we term (I - A ) the ltclass1fioationu matrix.
.
2
-
The type of regres-
/\.
sion is identified as fn'. 'fable'9 .1.
Table 9.1. ClassifioatioE of path sohemata
, Ordinary
It is
1
Joint
I
Chain
triangular
Cyclio (feedbaok)
Eon-triangular
~vident
that eny soheme not involving feedback will possess a
triangular classifioation matrix.
In th:ts oase the identifioation
equ~­
tions (10 .. 6) are of a partioularly s:lmple variety and algorithms have
been dev:l.sed by 'Wright and his followers for writing down these
equ~tions
10.7 Some Examples of Partioular Path Schemata
We will consider several verr s1mple cases of linear casual networks, nonlinear analogu.es of which we will consider in the next chapter.
(1)
two pr:1.Inary variables and one secondary variable.
The case of
We represent theflo'W' of cause and effect by a Itpath diagram"
as follows
The stru.ctural eqllation is
(10.14)
or
0
(-<:J. -<:La -<:Lb
1
• (1J
~a
~b
---
'1
and
(10.15)
from (1006)
0
ThIls, we get the obvious result that if' one obtains
the usual multiple regression est:tma.tes
ing 7 , on x and
a
1
Jlb
I;.
,..
"i
~l' ~la' ~ib by
regress-
then we would estimate the path coefficients
by equating aooording to (10.15) t
A
A
1\
/\
1\
,...
"'). -"1
"ia-. ~la
-'lb- ~lb •
(2)
.\ oase in whioh both joint and ohain regression ooours.
Consider the path diagram
~Q
rX!«
~ ~l
~~~2J
~ b oC.,.I." 712-
Prooeeding as before we obtain
.e
I-~.(.~
:)
and aooording to (10.6) we get identification equ.ations
(::
~:la
f3~~
1b ).
P
~2b
.
C~ :J (~
- (~ )(~
- "'2+~1~
0
1
~
0
~b
"ia ~b)
0
~
(
~b)
-<21"').a
~b
)
~b "1. • (10.16)
b ·21 b
~ +-<
Now (10.16) is solved for the path ooeffioients and the
~ fS
are
replaced by regression est:ima.tes to give est:lmates of the path
ooeffioients.
(3)
An eDllple of feedback.
The path diagt'Gl of a simple case
of cyolic regt'ession is given 'Pelow.
~
O(I~:>
tI
P(tl~
~1> olYq~
1)
I \o1~1
?}'J..i
We have aooording to (10.6)
Regt'ession estimates of all path ooeffioients are then
found.
eas~
1100 NONLINEAR PATH :REGRESSION
11.1 The Structural Equations
£orPathways Following the Single Process Law
"--.
The ideas of path analysis presented in the last chapter can be extended to the case of nonlinear causal relationships between var:Lables.
In this chapter we will study the situation in which the relationships
can each be
r~resented by
the single process law. The general approach
is valid for other relationships as well.
As in the linear case we describe the structure of a hypothetioal
network of oausal pathways by a system ofpartiaJ. d1.f'ferential equations.
In the linear case each of the partial derivatives are constant -
the
constants being termed Itpath coeffioients". In the non-linear case the
analogues of the path ooeffioients (i .e., the pa.n:ial derivatives) are
.e
not oonstant.
Let the matrix of partial derivatives be given, as before, by
~la
"l2a
'lb
92b
0
•
0
t
H
,?:Lq
•
• ••
·...
0
'lJpa
'lJpb
0
0
•
•
0
r2q
~pq
- .'.. - - - .-- -. - .0.
~
fll
'21
?12
~22
•0
•
71
1:>
• • 0
.0.
'1P2
•
•
0
0
•
P2p
'l)p1
•
.' ...
o/pp
t
H
•
;
,
HI
Let us now consider the mu.ltivariate analogue of the single process law
given by
I
==
H •
(11.1)
The analogy in form to the univariate ca.se is evident.
Compare with
equation (S.2).
A solution of (11.1) is
7l·-<:L+~l ("a-~a)6:l.a(~b-~1)~b •• •("(r~q) 6:l.q(
I/J
:
1
)(?2-~) 6:1.2. 0o(~p_p) 6:l.p
2a("b-~)52b ••• ('fq~q)52 q(Pl--<:L) 821 ( 1
5
'/ 2=="'2+~2(1a-~a)
).
u<?p-i» 82P
•
(11.2)
••
'!'hus:' equations (1102) are the "structural equations" for the causal
.
network arising from allowing each pathway to individually follow the
single process law.
11.2 Ident:1:f':Lcation
If one el::Lm::Lnates the
l' s from the right hand side of (11.2) in the
spirit of the preceding chapter to form "reduced equations", the problem
of ident::Lf::Lcat::Lon arises once more.
Generally, one does not place
restrictions on the scale faotors -< or
~
when describing the causal
network. When a particular pathway is non!"'exLstent, this fact is represented by setting the appropriate exponent 8 equal to zero. Hence, we
will not be surprised to discover that the problem of ide:nt::L:rica.t::Lon
primarily concerns these exponents, although the constants of
(~, s),
integrati~n
signi:f'ying initial conditions of the system, also may be involved.
Let us trans.t'orm(11.2) by taldng logarithms. We set
.
-
?J 1• = log(~l
,
-
-'J.)
'72 • 10g('2 - ~)
•
0
0
,
~ p • 10g(~p - ..(p )
t
~ a = log(1a - lJ.a )
~ = log ~l
;~
~ • log ~2
•
log(~
-
~)
•
0
0
•0
0
~~ • 10g{~a
- lJ.a )
Ap • log~p
and ":i.j • 8ij 0 Then (1102) becomes
.e
•o•
Now the transformed structural equations (1l03) are identical nth the
corresponding linear structural equations (10.2) of the last chapter 0
Since this is true, we can apply all of the results of the earlier
chapter regarding identi£:Lcation and reduction directly to the present
problem.
61
11.3 Some EDmples
In t111s section we Will consider the same path diagrams as cons:ldered
in the linear case.
(l) The "multiple regression l ' analogue with diagram as follows
The structural equation is then
(11.4)
(2) The second example with diagram
re;r _....:?J;.;.;;Ja~~ 1)J
t9
.e
1/
'Jp -~"""~?lt
andetructural.. e~t1tm'S:
(11.5)
yields by substitution or b,. the methods of the last chapter
the reduced structural equations
(1l.6)
62
(3) The .feedback example
..
produces structural eql1&t1oJ'ls
'? 1= -i + ~l ('a -
\-La)
?J 2 • -<a + Pa(~b - ~)
~a (fa _ ~)~a
6ah
<'1 -
-'l)
621
and reduced structural eqaat10ns
Comparison o.f the results .for these tbree exa:mPles with the corre'Bponding linear results will serve to illustrate the analogies. The
important point to note is that, under suitable conditions, reduced
structural equations can be .faund, even in this nonlinear case, SIlch
that each secondary variable can be exprrssed as an explicit function
o.f only the primary variables.
63
11.4 The Gausm.an Iterant for Ma.xLmum Likelihood Est1mators
In considering estimation of parameters for the linear case, we
..
restricted our attention to systems in which reduced equations could be
found and in which a state of under- or just-identifications e:xisted •
We s:JJnilarly confine our consideration to these m.tuations for the nonlinear case.
After reduction we have p independent nonlinear regression eqaations,
each of the form
As before, we linearize by expanding in a Taylor t 21 series and neglect
terms beyond those containing the f1:rst derivatives. First, we
.e
trial estimates Mao'
1\,0' ••
I
Mqo ' Dao , 1\0'
~o • ~c1(Xa - Mao)
X30 ..
~oI(;, ·1\0)
•
••
for all observational vectors.
I
I
I
I
I
"
Dqo
I
IlIl1~
obtain
We then compute
64
In addition, we def'ine
(11.11)
••
•
•
••
Now we have by use of' Taylor! s series
".A
A
'1 -: Yo + YBoo(B
A
"
o - Boo) + YB:J.o{B:J. - ~o) + ~ao(Ma - Mao)
"
+ y~o(~ - !\o)
.e
•
••
(11.12)
·••
and by substitution we obtain the linear equation
65
as an appro:x:Lmate substitute tor (1.1.9). By the usual methods ot linear
regression we now obtain the est1mates Bo" Bl , •••, B2qtl ~d :f'ro~ (11.11)
we get the :Lmproved estimates Ma, ..., Dq • These oan now be used as our
"trial n estimates and the entire process repeated until convergence.
If' any subset
ot parameters is known ! priori then, as in the s1mp1.e
regression case, the appropriate XI s coe simplY' omitted and the analysis
oarried out without them. For e:xample, i f it is known that all processes
have vertical asymptotes zero then we omit X20 through XqI'l,o •
When there is more than a single response variable the application ot
(11.13) separately to eaoh provides multiple est1mates tor the vertioal
1\, ..., Mq • In order to prov1de unique, effioient
a,
estimates ot these values we fit, instead of (ll.13), the family of
a~otes M
.e
hyperplanes
"Yi • BOi + ~i~Oi + B2~Oi ... + Bq+1Xq+l,Oi '+ Bq+2,iXq+2,Oi + ...
.
(11.14)
where i • 1, 2, ..., p and B2, B3, • H' Bqtol are common tor all i.
Linear regression theory provides the m:Ln1ml1m variance unbiased estimators
for :t1tting (11.14).
Asymptotic tests and oonfidence 11m:lts are tound as before by ap·
plying linear results to the final iteration. Exact tests of the type
di8cussed in Section 9.0 are also applicable to the general oase of this
chapter, but, of course, the numerical d1.f:tioulties are II'lI1oh increased.
12.0 A SAMPLING INVESTIGATION
12.1 Objectives
In the case of models linear in the unknown parameters and with
normaJ.ly distributed errors, max:lmum likelihood est:iJnators of the
coefficients possess a number of des1rable properties. .Among these are
. p?,~~~10n unbiasedness and m1n:1mum W1anoe. properties possessed by
est:tma.tes taken from samples of any size whatsoever. Maximum likelihood est:iJna.tors for models nonlinear in the unknown parameters acquire
these two properties only
asymptotic~.
Not Imlch is known about the
behavior of the est:imators:for small samples in the nonlinear case. For
large samples we
e~ect
the asymptotic results to approx:bnate:b' hold. Is
it safe to rely upon the asymptotic results for small numbers of observations J say ten or twenty?' The objectives of the stlldy reported in this
chapter .re to
e~lore
the behavior of the max:lmum likelihood
est~tor
for the e:x;ponential parsmeter 6 oceuring in the single process law (5.1)
and to compare the results with those produced by use of the asymptotic
theory of Section 7.0.
12.2 '!'he Samples
Thirteen stru.ctural equations of the .form
(12.1)
were selected for study. These thirteen equations correspond to thirteen
distinct points in the two-d:1.ma!onal nonlinear parameter space (y, 6)
and were chosen so as to more or less completely cover the useful. parameter region. For convenience the constant ..( is taken in every' case to
67
be zero and the scale factor ~ is chosen such that
Values of
Ifj -
3:> when ~ • 5.5.
1have been compu~d for the ten values ~1:.1)' r2-2,~ .:i,rJD=10.
Thus the thirteen curves intersect midway along their computed course.
..
The selected sets of parameters and computed points are shown in Table
12.1 for each of the thirteen curves.
S:1mulated data were prepared by generating 98 sets of ten no1'mal
deviates for each of the thirteen curves.
The 12,740 normal dev:Lates were
obtained by use of the IB-I 63:> Progam developed in the N. C. State College
.
2
oomputation laboratory (1958) • These normal deviates were added to the
.
computed values of Table 12.1 so as to give 98 sets of points for eaoh of
thirteen curves. That is, 1274 s:bnulated experiments were obtained, 98
replications for each point in the parameter space.
.e
choice of the scale factor
~
The convention in
:implies that the coefficient of variation is
2% for all curves at the midpoint
r-
5.50
12 0 3 Est:ima.t1on when IJ. is Known
Maximum likelihood estimates were obtained for all3 curves.. :til ,the'
sample under _the assumption of
mown
IJ.
where as before IJ. • "y"6. .Analysis
of the estimates of the exponential parameter a has been made
0
In partic-
ular, evaluation of the estimited bias
bias (d) •
a-a
(12.2)
2 Library Progr~ .. ~ND~ Random ~omal.dE:t~te~'l in range -4 to,,4•.
1958. Department' of ~ei:'imental Statistics:, North Carolina'
State College. Raleigh, N. C.-
j Due'to"a ~isha.P in ~aI'd ~dling'S~ral c~~~s had .~ ~ discarded.
Five cases were omitted .from each of curves No. 4 and No. 7 and
one case amitted .from each of curves No. 10 and No. 11. Thus 1262
simulated experiments were an~ed.
..
e "
e
Table
CURVE
Y
{)
12.1- Thirteen curves chosen f'or investigation showing computed values of' eta
I
1
2
3
4
5
6
7
8
9
10
11
12
13
I
0
1
1
2
2
2
3
3
3
4
5
6
8
-1/2
-1
-4
-2
-8
+8
-1
-4
+4
-2
+4
+2
+2
1
I 117.26
162.9J
651.60
180.9J. 327.26
867.33
106.25
164.19
410.10
112.9J
147.41
143.20
102.04
2
I
82.92
108.33
314.24
125.35
207.16
499.44
85.00
122.07
280.10
91.12
118.74
118.34
88.89
I
I
I
I
67.70
81.25
169.62
92.09
134.42
276.06
70.83
92.63
183.78
75.31
94.47
95.86
76.64
58.63
65.00
99.43
70.51
89.17
l45.51
60.71
71.56
114.73
63.28
74.13
75.74
64.31
52.44
~.17
62.07
55.71
60.36
72.~
53.12
56.15
67.25
53.92
57.26
57.99
~.88
47.87
46.43
.40.72
45.12
41.60
33.84
47.22
44.67
36.30
46.49
43.45
42.60
45.35
44.)2
40.62
24.47
37.29
29.15
14.57
42.9J
35.98
17.51
40.9:>
32.30
29.59
36.74
41.46
36.11
19.64
31.34
20.74
5.68
38.64
29.31
7.17
35.60
23.45
18.94
29.02
39.09
32.9J
14.26
26.70
14.96
2.95
35.42
24.11
2.27
31.53
16.56
10.65
22.22
37.08
29.~
10.60
23.02
10.93
SI
32.69
20.02
.45
28.12
11.31
4.73
16.33
3
4
~
e
.'
.'.
5
6
7
8
9
10
~
69
and of the estimated variance
has
been made. The results are shown :In Table 12.2. Confidence limits
have been obtained for the bias by use of Student's t-distriblltion.
The
-
upper and lower 95% points are given in the table. We note that in three
of the cases· biases are signii'icantq' different from the asymptotic expectation of zero. However, even in these cases the bias is quite small,
be:lng in every case less than 0.3%. It appears that under the conditions
assumed here, bias is of little consequence in the region of the
paran1~
eter space studied.
Confidence limits for the variance of estimate d have been computed
2
using the x • distriblltion and upper and lower 95% points are reported
also in Table {12 •.2J. No case bas a variance signii'icantq greater than
tbB asymptotic value.
For ease of discussion the "finite efficiencies",
found by dividing the asymptotic variances by the est:Lmated variances,
are shown. Judging by the lower 95% l:lm1t the finite efficiency'mq :in
some cases be as law as 66%; however, this is unlikely to be so and we
have little reason to reject 100% as a sufficient approx:bnation in all
cases s'b1d1ed..
Thus, the asymptotic variance would seem to prov:l.de a use-
ful basis for construction of confidence 1m!ts under the conditions
examined •
12.4 Est1mation when Po is Not Known
The constructed samples for curves 2: ~and 3 have been
an~zed
under
the assumption that Po was unknown and hence must be est:bnated s:1multan-
eously with 6 and the linear parameters -< and
~.
'!'he results for the
.,
e
'
.
.
..
•
e
,I
Table 12.2. Estimated biases and variances of' the maximum likelihood estimate for delta
Curve
IL
Parameters
lJPy6a
5
y
Bias . of Es+.i1llAte d
lower
known
Finite Efficiencv ('t)
lower point upper
est.
9~
95%
limit
limit
point b
estjmate
upper
-.0003
+.0031
+.0065
.00090
.00116
.00158
.00158
100
136
176
9~
limit
o·
Variance of Estimate d
lower
point
upper as;vmptotic
estimate
value
9~
9%
limit
1:imit
9%
limit
1
0
-1/2
2
1
-1
-1
-.0026
+.0009
+.oo1lO
.00090
.00117
.00159
.001.42
89
121
158
3
1
-4
-4
+.0053
+.008.5*
+.0117
.00020
.00026
.00036
.00028
78
108
140
4
2
-2
-4
-.0118
-.0026
·+.0066
.00146
.00200
.00270
.00270
90
121
166
5
2
-8
-16
-.0025
+.00.5J.
+.0]2.7
.00442
.00513
.00781
.00.5J.6
66
90
117
6
2
+8
+16(-5) +.0014
+.0035*
+.00)6
.00034
.00044
.00060
.00047
78
1D7
138
7
3
-1
-3
-.0001
+.0084
+.0169
.00489
.00667
.00862
.00670
78
100
137
8
3
-4
. -12
-.0180
-.0082
+.0016
.00729
.00944
.01287
.01123
87
119
1~
9
3
+4
+.0024*
+.0044
.00032
.ooo4l
.00056
.00052
93
127
162
1D
·4
-2
-.0119
+.0016
+.01.5J.
.01347
.01763
.02378
.01628
68
92
121
11
.5
+4
+20(-9) -.0260
-.0070.
+.0120
.00670
.00876
.01182
.00964
82
liO
])..4
12
6
+2
+12(-1) -.0066
-.0029
+.0008
.00104
.00135
.00184
.00171
93
127
164
13
8
+2
+16(-5) -.0101
-.0008
+.0085 .•006~
.008.5J.
.01160
.01004
87
li8
153
2
1
-1
-1
-.0728
-.01>2
+.0424
.06273
.08J2.3
.11072
.07275
66
90
li6
3
1
-4
-4
-.0933·
-.0487*
-.oo!tl.
.03769
.04881
.066.53
.05867
88
120
156
unknown
+12(-1) +.0004
-8
a There is an equivalence between certain negative values of IL and certain other positive values due to the convention of
taking (x - ILJ positive. Negative equivalents are given in parentheses for all positive IL.
b An asterisk signifies significant departure from asymptotic e:xpectationzero at the
5% level.
-.;a
o
71
criterion parameter 8 are shown at the bottom of Ta.ble 12.2. A smaJ.l
b.1t significant bias was discovered for curve 3. It is interesting
to note that the bias is negative when 11 is
when 11 is known.
the latter.
unknown bu.t
is positive
The bias is much greater in the former case than in
For unknown 11 the bias is estimated to be 1.2% of 8
whereas for known 11 the percentage :is estimated to be 0.2%.
Again the f:Lnite efficiencies do not depart significantly from
the asymptotic value of 100%
0
As expected the asymptotic variances
are considerably increased in the case that 11 is not known !. priori.
This :increase is about
~
.e
50 fold for curve 2 and about 200 fold for curve
limited experience has been obtained for the case of IJ. unknown
b.1t so far as these l:imited results go it a.ppears that the asymptotic
variance m.q provide a reasonable guide to the precision of the estimate
of 8, even when the vaJ.ue of "" is not specified by the model.
1205 Discussion and Su.rnmary
Constructed curves covering the ItusefUll • range of the parameters
and consist:Lng of a series of observations of intermed:tate length (n-10).
aildwith a reasonable rate of error (2% at the midpoint) have been subjected to ma:x:!mum likelihood
~:is.
In order to check on the form of
the CIlrve a sb:>rter series than one of length 7 or 8 seems inadvisable.
A length of 10 :is still better. Longer series yet supply little advantage
not gained by replicating the original series. .An advantage of replication is that the first exact test of Chapter VI can be employed. In
many cases the error rate is greater than that studied here. Replication
72
provides a_ ~ethod of reduoing the int.'rinsio error rate to any desired
level. A rate of about 2$ with 10 equidistant points is required in
order that 8 be determined to the nearest integer when
-.,.
~
is not
mown
11 priori. Equidistant points have been used in the study sinoe this
arrangement
plausib~
has best average effioienoy for disorimina1iU1g
against unspeoified alternative models
0
If the model is known to be
of the single prooess form then more effioient designs oan be found.
See Box and Incas (1959)
0
-
The estimated sampling varianoes, resulting from estimating the
criterion parameter 8 for each of the constructed curves, did not
differ significan'l"Jq from the asymptotic value. We concluded therefore
tha t under the studied aonditions, which as seen above are quite realis-
tic ones, the asymptotio value seems a safe guide as to the precision of
the criterion parameter and hence the proper curve can be selected .from
e:c;perm.ental resultsit' these have sufficient precision and about 10
equidistant points covering the main course of the curve are ezg.ployed.
The biases were of an un:1mportant magnitude for purposes of modelbu.ilding.
73
13.0 SUMMARY AND CONCWSIONS
A class of nonlinear regression models, possessing very interesting
and useful analytic properties, has been described in some detail.
A
single e:x;ponential parameter serves to distinguish the various members
of the olass. Since suoh £requ.ently occuringprocesses as the linear
process, parabolic prooess, e:x;ponent1al prooess, and 1nverse-squ.are process are members of the class it is suggested that the general model be
oalled tbesingle process
1!"!o
The appropriateness of the name is :further
suggested by the simple differential equ.ation used to generate the model.
The single prooess law model is one that we refer to as an essentiallY
nonlinear model, which is to sq'that olassical linear regression teohniqtles are not avaUable. After a brief reSUl1l(~ of some important cU.stinc"·
-e
tions between linear and nonlinear models iterative methods of obtaining
ma.x:iJmml likelihood estimates and asymptotic confidence lindts for the
parameters in the single prooess law are developed.
Next we consider the great potentialities of the single process law
for the prooess of model-bu.Uding.
This potentiality' is illustrated by
the development of an original theory of world track reoords andtbe
theory is applied to the offioial records.. It is suggested as a oonolusion that the average excess velocity· (over the asymptotio value) of a
middle or long distanoe nnner is inversely proportional to the 3/7 the
power of the time of the race.
Some tests of significanoe and derived confidenoe limits for small
samples are discussed •
One suoh test based upon a Parabolic expansion
appears not to have been previously reported. Confidenoe limits obtained
from this test are applied to the track reoord example.
74
A swmnary of some results in linear path regression analysis and a
nonlinear extension based upon the single process law are given.
Comptl-
tational procedures are indicated for the nonlinear case which are.
simple generalizations of those for the univariate situation.
A sampling study for the purpose of
e~loring
the adequacy of asymp-
totic results for small samples has been carried out. Within the limitations of the study it appears· that for the purpose of model-bailding,
that is for selecting the appropriate process, the asymptotic results
May well be quite adequate.
We generally conclude that the technique of the single process law
is a useful and manageable tool for anaJ.yzing problems of the monotonic
growth or extinction variety, whether the problem is of the s:1mple
-e
We-dimensional type or whether complex multi-dimensional causal networks are involved.
75
14.0 LIST OF REFliRENCES
Box, G. E. P., and Incas, H. L. 1959. Design of experiments in nonlinear situations. Biometrika. 1&:77-90.
Deming, W.E. 1943. The Statistical Adjustment of Data.
and Sons, Inc. New York.
John Wiley
Fisher, R. A. 1925. Theory of statistical estimation. Proc.
Cambridge Phil. ,Soo. ·~:700-725.
'Fisher, R. A. 1939. The sampl~g distribution of some statistics .
obtained from nonlinear equations. Ann. Eugenics. 2,:238-249.
Fisher, R. A. 1956. Statistical Methods and Scientific Inference.
Hafner PUblishing Co. New York.
Hansen, H. (Ed.). 1958. The World Almanac and Book of Facts for 1958.
New York World-Telegram. and The Sun. New York.
Hill, A. V. 1927. Muscular Movement in Man.
New York.
McGraw-Hill Book Co., Inc.
Hood, W. C., and Koopmans, T. C. (Ede.). 1953. Studies in Econometric
Method, Cowles Commission Monograph No. 14. John Wiley and
Sons, Inc. New York.
Kempthorne, o. 1957. An Introduotion to Genetics Statistics. John Wiley
and Sons, Inc. New York.
Takey, J. W. 1954. Causation, regression, and path ana~is. Statistics
and Mathematics in Biology (0. Kempthorne !!. .!l., Eds.).
The Iowa State College Press. Ames.
Turner, M.
E., and Stevens, C. Do 1959. The regression analysis of
causal paths. Biometrics. ~: 236-258.
Wegstein, J. H. 1958. Accelerating convergence of iterative processes.
Commu.nications of the Assoc. for Oomputing Machinery•
.,(No. 6):9-13.
Whitaker, E., and Robinson, G. 1944. The Calcu~us of Observations,
4~ edit. Blackie and Son, Ltd. london.
Will, H. S. 1936. On a general solution for the parameters of any funotion with applioatiOn to the theory of organic growth.
Ann. Math. Stat. 1.16$-190.
W:U1iams, Eo J.. 1959 (in press). Regression Analysis.
Sons, Inc. New York.
John Wiley and
76
..
Wold" H. 1956.· Causal inference from observational data: a review of
ends and means. J. Roy. Stat. Soc." Sere A (General).
119, Pt. I: 28-61 •.
Wright" S. 1921. Correlation and causation. J. Agri. Res. g,Q,:557-585.
".,
Wright" S. 1934. The method of path coefficients. .Ann. Math. Stat.
~:161..215.
wright" S. 1954. The interpretation of mltivariate systems. Statistics and Mathematics in Biology (0. Kempthorne 11. -'b., Eds.).
The Iowa State College Press. Ames •
.e
77
APPENDIX A. .A POLDrCMIAL APPROXIMATION
..
When Ix
I <' p.)
for all values of x required the model
y • ~
+ ~(~p.)5·+
8
can be e:xpanded 1n a converging series
y •
~ + ~1(~)5+5(-p.)0-1x +(1/2)5(0-1)(-p.)5-2x2+(1/6)5(0-1)(0-2)(-p.)5-3~
+ ..
~J +
t
or
2
y • Ao + ":J.x + A2x +
~x3 + ... +
t
where
A • ~ + ~ ( _p.) 5
o
~ • ~5(-p.) 5-1
.e
A • (1/2) {36(e-l) (-p.) 5-2
2
3 • (1/6)135(0-1) (5-2) (-p.) 0-3
A
o
•
•
I I «< 'p.I we may·terminate the series after the cubic term.
It' x
this case we have
-< • Ao -
P ..
A:t .A2/2~
2
~ (~) A/I~
5· ~/~
p. •
-A:L Ai2~
2
In
78
where
..
~
2
... 2A2 -
3A:I.A3
2
~ • A2 - 3A:J.A3 •
.An est:1mate of 5 is tlms given by
"
5·
~2 _ ~~
2
.....'1. '3
~2-~f3
where "'"
A:J., A
A2, and "A are the usual least "squares estimates obtained from
3
the cubic approximation.
The asymptotic variance for the 'tcubic lt estimate of 5 is found to
.e
79
APPENDIX B.
A SIMPLIFIED ESTDUTOR
The differential equation for the single prooess law was stated
.
to be
~
•
If' values of
tf are
ohosen to be equidistant (l\ • 1 without loss of
generality) then D may be estimated. by passing a parabola exaotly
through three suooessive values of y and evaluating the derivative at
the first.
By following this prooedure we arrive at the approx:lmate
relationship
.e
for i • 1,,2, ..." n-2 and where Di • -3Yi + 4"i+l""i+2. The parameter
5 oould be estimated by regressing Y on Y and D • However, suocessive
i
i
i
values of Y are oorrelated a.rd a proper prooedure would involve these
i
oorrelations. In fact the matr:Lx of varianoes and covariances is found
to be
~2+B 2+C 2 ~Bl+B2Cl
1 1
0
0
•••
AC
42
0
•••
AC
3 l
A B +B C
3 2 3 2
A B +B C
3 2 3 2
A2
+B 2
"+C2
3 3 3
0
A4C2
A B +B C
4 3 4 3
~Bl+B2Cl
~.
A,}+B ~+C 2
,22
A Cl
3
0
•
•
•
0
•
•
•
Arf3
•
•
•
A4B +\C
3
3
Af.3 • •• (j2
~2+B42+C42 . A5~4+~f:4···
A5B4+Bf4 Atjf+Btj?+c
•
•
•
•
•
0
2
5 •••
80
where
Ai • -3~ -25 +3~
Bi •
4xi -4~
A possible estimation procedure would be to approx:1mate ~ by
substitution of trial values of ~ and 6, and then to estimate the
regression coefficients by generalized 'least squares. Hence
! -(x' ~-lx)-lx~ ~-lZ
1\,
-A"
A 1\
where! • (-z&<, 25, IJ.).
.'
This procedure is even more tedious than
direct application of ma:x::l.mwn likelihood methods as described in Ohap-
-
.
ter III•
A s:1mplified procedure might have some merits.
For example we
0>
could take all off-diagonal elements of ~ to be zero as an approx:1mation. In this case tba above procedure reduces to ordinary weighted
regression
~sis.
Weights then are s:i.mpq
A still siIrij;>ler approximate method is found by' taking the weights
be constant.
gated.
e·
to
The properties of these procedure$ have not been investi-