THE THEORY OF NONLINEAR REGRESSION AS IT RELATES TO
SEGMENTED POLYNOMIAL REGRESSIONS WITH ESTIMATED JOIN POINTS
by
A. Ronald Gallant
Institute of Statistics
Mimeograph Series No. 925
Raleigh - May 197)+
ABSTRACT
This report is a summary of the topics in nonlinear regression which are
particularly relevant in applications of segmented polynomial regressions with
unknown join points.
These are nonlinear regression models whose distinguishing
characteristic is that the partial derivatives of the response function with
respect to the parameters to be estimated may not exist.
The regularity conditions usually assumed in nonlinear regression theory
to obtain convergence of the modified Gauss-Newton method of computing least
squares parameter estimators, the asymptotic normality of these
estimators~
and
the asymptotic distribution of Likelihood Ratio test statistics for hypotheses
of parameter location are reviewed.
The extent to which the derivative conditions
can be weakened to accommodate segmented polynomial regressions constrained to be
once continuously differentiable with respect to the input
var~able
is
discussed~
Hartley's method of testing hypotheses of the location of subsets of the
parameters which enter
~
response function nonlinearly is reviewed.
This method
does not require the imposition of regularity conditions involving derivatives
and its use in segmented polynomial regressions which violate the derivative
conditions is discussed.
The topics presented are illustrated with examples taken from applications.
THE THEORY OF NONLINEAR REGRESSION AS IT RELATES TO
SEGMENTED POLYNOMIAL REGRESSIONS WITH ESTIMATED JOIN POINTS
by
A. Ronald Gallant *
*
Assistant Professor of Statistics and Economics, Institute of Statistics,
North Carolina State University, Raleigh, North Carolina 27607
1.
INTRODUCTION
This report summarizes the theory of nonlinear regression which is particularly
relevant in statistical applications of segmented polynomial approximating functions
where the join points are uqknown and must be estimated from the data.
statement of the
theoretic~
A formal
results cited is included as an Appendix.
The nonlinear parameters appearing in such models are, of course, the unknown
abscissae where one polynomial submodel stops and the next polynomial submodel
starts; the remaining coefficients of the model enter in a linear fashion,
many data analysis situations the join points are known
particular interest in the analysis (Fuller,
1969).
~
In
priori or are of no
In the former case
the
model is linear subject known restrictions on the parameters (referred to as
B ' B , ...
O l
in the later sections)
and in the latter case the same is true
provided one is content to treat his visual estimates of the join points as
~
priori knowledge.
In these situations, Section 2 may be of interest as a
means to put the mod.el in proper form for standard multiple regression computing
programs.
On the other hand, one can avoid reparameterization in these situa-
tions by employing the numerical methods given in Gerig and Gallant
(1974).
This algorithm is available as a SAS procedure upon request from the allthor.
2
2.
REPARAMETERIZATION OF SEGMENTED POLYNOMIAL REGRESSIONS
It will be convenient in the later sections to be able to express or represent
a segmented polynomial regression according to the specification commonly employed
in nonlinear regression theory:
to inputs
A set of responses
generated according to the regression equations
(t
where the errors
e
t
= 1,
2, •.. , n)
are independently and normally distributed with mean zero
and unknown variance
The response function
f(X, e)
'"
form depending on k-dimensional input vectors
a p-dimensional Parameter
a
~
*
~
Segmented polynomial regressions are usually written as
where
g(x)
is the sequence of grafted polynomial submodels
g(x)
=
Ci
corresponding to the partitioning of the interval
r- l<X:S;Ci r
[Ci
O' CirJ
~
and
The true but unknown
f"V
value of the parameter is denoted by
function~l
contained in a known set
contained in a known set 0
~
has a known
f"V
into
3
The endpoints
aO
and
ar
are known but the intermediate points of join
are unknown and must be estimated from the data.
(j == 1, ... , r)
The submodels
g.(x, ~.)
J
NJ
consist of the polynomials
==
As pointed out by Fuller (1969), it is often desirable to impose a continuity
restriction on the model
j == 1, 2, ..• , r-l
and, in addition, require that the response functions
g(x)
be once continuously
differentiable by imposing
j == 1, 2,
... , ;r-1 •
As mentioned earlier, it is helpful to be able to express such a regression
model in the single equation form
in order to apply the ideas of nonlinear regression theorY.
along the lines (Gallant and Fuller, 1973) summarized here.
This may be done
4
The basic idea is to rewrite the grafted polynomial model in terms of the
basis functions
1, x 1
X
2
,
••• ,
x
q
i = 1, 2, .,., r-l
z
k
z
~
0
=
o
Note that
TO(z)
is discontinuous,
T (Z)
once continuously differentiable,
and so on.
Tl(z)
3
z < 0
is a continuous function,
TO(a
i
- x)
Tl(a
i
- x)
B
O
be deleted from the basis, imposition
of the once continuously differentiable restriction
functions
is
is twice continuously differentiable,
Consequently, the imposition of the continuity restriction
requires that the fUnctions
T (z)
2
B
l
requires that the
be deleted from the basis, and so on.
As an example, the quadratic-quadratic model
SUbject to the continuity restriction
is written in terms of the basis functions as
5
where the correspondences between parameters in the two representations are
86 =
~l·
The data given in Table 1 can be represented by this model as seen
from the least squares fit in Figure 1.
If the quadratic-quadratic model is restricted to be continuously
differenti~
able by imposing
one obtains
where the parameter correspondences are
The data given in Table 2 can be represented by this model as seen in Figure 2.
FIGURE 1.
,.-•
Quadratic-Quadratic !-lodel Subject to
Fitted to the Data of Table 1
.
.
B
a.r..d
O
B~
...L.
~.-----~~--9---------.
o'"
'".,III
·
C"l
I •
'"
'"
0
•
-·
C"l
M
f
I
,•
I
I
c
c
c
I
••
•••
•
,•
•
j
•
•
!\I
·
'"M
e
co
,
••
+
t
••
'"
,'"
",
·
I
I
I
I
:•
•
'"'"
'"
0
·
lZ)
III
..
:
,..•
0
•
•
'"'"o
Cl'
~--~---.-----~---.---------.-----c
..,.'"'"
•
-
0
..'"'"
0
0-
0
•
"00
0
N
.0
••
0
...
o
o
0
0
o
••
C>
.0
N
'"o
-•
•
·
'"
I II
,....
F'IGURE c:..
Quadratic-Quadratic Model Subject to
Fitted to the Data of Table 2
o
o
j- -------.---------.--------_._--------.---------.,
I
•
I
I
I
I
•••
.,
C'>
·
I'll
PI
I
I
I
,
I
••
I
I
I
,,
,
•
,•
I
I
I
I
I
•
•
I
.
I
e
C>
co
'"e
·
l"'I
e
<:>
oil
PI
:
•
:
:
'"·
0-
I
I
I
I
II
I
••
.
·
c
C>
I
I
•
<>
••
01)
til
I
:,
f
I,
••
•
••
I
I
,
I
••
.,•
I
I
""
Q
,...'"
·
ot'J
N
o
.
o
C>
~--------.---
......
•...
1ft
•
"
:
......
co
-.
00)
N
.----.---------.---------.--------c
...
......
...
...
...
......
....,
....
.
.
'"•
<:>
<:>
N
<:>
Cl
C>
<:>
CD
...
<:>
<:>
C
...
·
~
I II
TABLE 1.
Specific retention volwne of' cycloheptene in polyethylene terephtha1ate
Reciprocal ~ 10 3 of
temperature in degrees Kelvin
2.69323
2.72182
2.76395
2.80269
2.82885
2.85388
2.87686
2.87852
2·9019~
2.92568
2.93772
2.95)+20
2.95945
2.97885
2·99132
3.01386
3.05997
3.10173
3·139'71
3.18471
3.18471
3·23310
Source:
Hsiung (19'74)
Natural logarithrn of
specific volume in ce. per gm.
0.35680
0.27624
0.061.1-185
0.00116999
-0.038708
-0.011101
0.068929
-0.0135 1).7
O.o6861U
0.20731
0.29910
0.38649
0.43128
0.44319
0.5651()
0.80240
0.9539h
1.11987
1.27006
1.41183
1.40225
1.66507
TABLE 2.
Specifio retention volume of methylene chloride in polyethylene
terephthtiLate
.~
Reciprocal X 10 3 of
temperature in degrees Kelvin
Natural logarithm of
specific volume in ee. per gm.
2.54323
2.60960
2.67952
2.75330
2.79173
2.82965
2.87026
2.91120
2.94637
1.16323
1.10458
0.98832
0.87471
0.62060
0.51175
0.35371
0.6695 1+
0.85555
1.07086
1.22272
1.29113
3~00030
3.04228
~,09214
3~13971
3.19081
Source:
Hsiung (1974)
1.381+80
1.46728
6
3.
NONLINEAR REGRESSION THEORY
The topics in nonlinea.r theory which are of primary importance in appl:Lcations are:
i)
the modified GaussrNewton method for computing least squares estimators
(Hartley, 1961);
ii)
the consistency and as;ymptotic normality of the least squares estimator
(Jennrich, 1969; Malinvaud, 1970; Mal:Lnvaud, 1966); and,
the asymptotic null and non-rl1ull di"tribution of Likelihood Ratio tests
]11)
for the location of parameter estimates (Gallant, 1973b; Ga,llant, 1974),
The resu+ts in these references are obtained subject to the requirement that the
seoond
ord~r
partial derivatives of
exist and be continuous.
f~)~)
with respect to the parameters
As we have seen from the previous discussion of grafted
polynomial models, this requires thE; deletion of the functions
'1'l(Q'i - x)
and
'1' (O'i - x)
2
T (Q'1 - x),
O
from the set of basis functions for gr~fted polynoB '
O
mial models or, equivalently, the imposition of the conditions
B
2
~
on the re~ponse funct+on
B ,
l
and
g(x) •
In Gallant (1973a)) a set of assumptions are listed which are sufficient to
obtain these results and allo\v the restorlttion of the functions
'1'2 (Q'i -:J1:)
to
the set of basis funotions or, equivalently, the elimination of the need to
impose the restriction
B
2
on the response function
g(x) •
Subject to choosing
one's inputs as "constant in repeated ·sample s" (Theil, J971, p. 364), i t is
verified in Gallant (1971) that the quadratic-quadratic model sUbjept to
and
B ,
l
BO
7
satisfies this set of assumptions and the additional assumptions
obtain the
conv~rgenoe
of the modified Gauss-Newton method.
proof employed are not lmique to
th~
re~uired
to
The methods of
quadratic-quadratic model mld, working by
analogy, one should be ab+e to verify that these assumptions are satisfied by
any segmented polynom;i.al model subject to
B
o
and
B
l
when the inputs are
" cons tant in repeated samples."
A listing of Gs,llant I s (1973a) assumptions mld a formal statement of the
theoretica:t- results are given in the Appendix to this report.
A summary of these
tOI?ics as applied to segmented polynomial regression mQdi='ls with unknO"',,'Il join
B and
O
points and subject to the conditions
Assuming that the
m~thods
B
1
is presented below,
of Section 1 have been employed to write the model
in terms of the basis functions as
(t::::l,2, ... ,n)
we define:
x
V
'"
=
(Yl' Y2' ... , iYn )'
f(~)
:;:
(f()Sl'
f(X, a)
=
the
'" '"
:;:
the
SSE(~)
;;::
ftn
"2
a
(y)
"oJ
p X 1
f(02'
,!l),
... ,
(n ~ 1),
f(x ,8))'
"'il""
.th
vector whose J
element is
O~.
f(/S,
J
X(!!)
,@.
!),
(n X: 1),
:;:
::::
nxp
t:;:l (Yt
the
p X 1
SSE(e)/n
1"'-'
matrix whose t
r(x
"'t'
e)) 2
ry
tp
row is
== (X ,. ,£(~)
vector minimizing
l' fl(0t' ~),
) '(¥.; -
SSE(~)
,!:<,.~)),
over
n
rv
~),
8
The starting value
plotting the
d~ta,
~
for the modified Gauss-Newton. method is obtained by
choosing join points by visual inspection, and estimating the
remaining parameters by ordinary multiple regression methods,
As mentioned
earlier, a grafted polynomial model is linear in those parameters which are not
join points and they may be estimated by ordinary least squares.
The combined
visual E::stimates of the join points and corresponding least squares estimates
the remaining parameters provide the starting value
~
The modified Gauss.Nevnop algorithm (Hartley, 1961) prooeeds as follows:
0)
From the starting estimate
Find a
1)
Set
AO between
-e.l = ~
Find a
Al
0
1
compute
such that
ComputE1
+ Arh
betw~en
and
~
0
and
1 such that
of
9
These iterations are continued until
terminate~
according to some stopping rule
such as
(i ;:: 1, 2, ... , p)
and simultaneously
where
•
is some tolerance.
(The use of the modified Gauss-Newton method to fit
grafted polynomial models is explained in more detail in Gallant and Fuller
(1973).)
Let
i
be the value of' the
paramet~r
to which the iterations have
~onverged
and let
£ =
Assuming that
!
consistent for
is, in fact, the least squares
*
estimato~',
it is strongly
converges in distribution to a p-dimensional
~
multivariate normal with mean zero and a variance-covariance matrix estimated
cons!ste~tly
by
£.
Consequently,
on~
may expect
to be a reasonably accurl;l.te 95% confidence interval for
Consider the hypothesis of location
H;
where ,Q.
against
has been partitioned according to
A:
8i
in applications,
10
~
with
being
an
r
~
1
vector of nuisance parameters.
v~ctor minimizing t~=l (yt • f(.?St:, (s:<.,~))
r X 1
value of Q
treating
r-J
~
to be the
(Q,,:to)' Q.
over
The
is obtained using the modified Gauss.Newton method as above by
f\<st' (R" rro))
as a response functi.on with parameter R,'
The Likelihood Ratio Test for
is larger than
T(~)
Define
c
where
H against
A
:is to reject
F(T(X) ~ clHJ = a.
Let
Hwhen
The Likelihood Ratio test statistic
may be characterized as
T(y)
where
c
n
'i"
X +
converges almost surely to zero.
approximatiqg random variable
X
C
n
The point
c*
such that the
satisfies
P[X > c * IH]
=a
is
where
p~r
F
ex
denotes the upper
ex • 100
numerator degrees freedom and
percentage point of an F-distribution with
n-p
denominator degrees freedom.
(See the
Appendix for the non..l}ull behavior of X.)
In applications, one would test the hypothesis
A:
,.t -:t
~
by computing
and rejecting when
T(~)
'~2 (-X)
exceedS
and
c*
H:
against
~(;:(), fonming the ratio
Also, one may set a
T (~)
"'2
"2
= " (-X) /0
confidel'J.c~
interv.al
(-l,.)
11
~
~bout
a coordinate
hypothesis
H:
T
of
i
by putting all
in
T
I
T :;;:: T
o is accepted; that is, for which
for which the
r~)
< c* •
As an example, we may apply these methods to the data given in Table 1 using
the quadratic-quadratic model subject to
B
and
O
B
l
The results from a modified Gauss.,Newton fit are displayed graphioally in Figure 1
and the munerical results are given :i,n Table 3.
The modified Gauss-Newton method
did not perfonn well for these dat(:l. llecessitating a choice of a start value very
near the correct answer as determined from the entries in 'rable
for this poor performance seems to be a very poorly conditioned
matrix; see Table 3.
is much
b~tter
Examples where
the performance of the
'+.
The cause
F/(~)F(!)
Gauss~Newton
are given in Gallant and Fuller (1973).
-
e,
The asymptotic normality of
the Likelihood Ratio test, and Hartley's
test (discussed in the next section) may each be used to set a
interval on the point of join
e •
5
95%
confidence
Using the numerical results in Tables 3 and
4 the confidence intervals thus obtained are:
1)
method
Asymptotic nOlmality of the least squares estimator
- [2. l'r8, 3.705 J
2)
Inversion of the LikelihouG Ratio test
3)
Inversion of Hartley's test
The lengths of these intervals are
1.53,
.06,
and
.09
respectively.
TABLE 3-
MOdified Gauss-Newton Results
••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
•
••
•
MODIFIED GAUSS-N~WTON ITERATIONS
•
•
•
•
•
•
••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
STEP
0
ERROR SU~ OF
SQUARES
--------- .........
PARAMETER
NUHAER
..............
0.341431340-01
1
3
4
5
1
0.341411500-01
1
2
3
4
5
INTERMEDIATE
---_ ....._-----...
ESU ... ATE
-0.'555089450
0.321643220
-O.448383b2U
0.296730800
0.293998240
02
02
01
02
01
"0.555120690
0.321666030
-0.448420480
0.296129150
0.294010190
02
UNABLE TO IMPROVE ON THIS ESTIMATE
02
01
02
01
••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
•
•
•
•
•
•
CHECK THESE ITERATIONS TO SEE IF
BEEN ACHIEVED BEFORE USING
T~E
~ONVERGENCE
HAS
FOLLOWING RESULTS
•
•
•
•
•••••••••••••••••••*••••••••••••••••••••••••••••••••••••••••••••••••••••
TABLE 3.
(continued)
.00 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •
•
•
•
CORRELATtON MATRIX GF THE ~ARA~[ttA tSTIMATES
•
•
••••••••••••
~
••••• u.o~ •••• ~ •••• *.oo ••••••••••••••••••• ~
0
•
•
••••••••••••••••
I
I
I
I
1.0000ll0
-0.999935
O.99~73'
-0.426821
.. 0.794536
-0.999935
1.000000
..0.999934
0.430923
0.789949
I
I
0.999739
"'O.9999~4
l.dOOOOG
·0.434.,92
-0"85379
I
I
-0.426821
O.4309~3
-0.4:14192
a,OGOOGO
-0.1681R3
I
I
-0.794536
0.789949
-0.785319
~GU6st83
.1.000900
I
•
I
I
f
i
f
I
I
I
I
I
I
.------------._-~-~.
__ ._~----- ..... -.~---_._ ...~~.._-~--~-~----_._---.
MODIFIED GAUSS-NEWTON PARAMETER tsrlMAT£S
I
.-_.---------------------------.._-~~ .•.._-_._..~.-._-----._--~---.--.
PARAMETER
NUMBER
ESTIMA tE
--------- --------------.
,,
,
f
.....
$U~IOA~O
_-~.-
..-._ERROR
1
2
3
-55.S1~1
14.5208
32.1666
4
29.6730
2.94010
9.45202
1.5)661
2.03211)
O.390!JOI
5
-4.484~O
ESTIMATED VARIANCE
.--~--._--------------.--------_._
2-StATISTIC
--------.--3.8229
3.4031
-2.918~
H.6021
.7.5271
0.1551810-02
RESIDUAL SU~ OF SQUARES
NUMBER OF 085ERVATIONS
NUMBER OF PARAMETERS
0.341411500-01
22
S
....-.---.-._-.-.-.-..
I
I
_._-~--------.
f
I
TABLE 4.
Tests for the point of join using a quadratic-quadratic model sUbject to
the data of Table 1
Corresponding Least Squares Estimates
Of the Linear Parameters
Hypothesized
Join Point
e;
61
62
83
e4
2.85999
2.86999
2.87999
2.88999
2.89999
2.90998
2.91998
2.92998
2.93998
2.94998
2.95998
2.96998
2.97998
2.98998
2.99998
3.00998
3.01998
-0.12112
-5.56382
-11.3023
-17.4076
-23.9942
-31.0819
-38.6981
-46.8439
-55.5085
-64.6956
-74.4353
-84.7119
-95.6625
-107.171
-1190070
-131.308
-143.985
-3.87760
-0.31805
3.43134
7.41454
11.7057
16. 317l
21.2595
26.5508
32.1643
38.1059
44.3938
51.0141
58.0540
65.4347
73.0443
80.8461
88·9031
1.37240
0_79124
0.17964
-0.46917
-1.16719
-1.91637
-2.71821
-3.57560
-4.48384
-5.44356
-6.45749
-7.52281
-8.65336
-9.83591
-11. 0518
-12.2945
-13.5741
37-.6367
35.7040
34.0864
32.7759
31. 7538
30·9664
30.3796
29.9583
29·6731
29·5143
29·4709
29.5398
29.7347
30.0233
30·3753
30.7831
31.2578
The .05 critical point for Hartley's test is 3.63.
The .05 critical point for the Likelihood Ratio test is 1.262.
SSE
0.097515
0.085059
0.073391
0.063047
0.053955
0.046236
0.040295
0.036153
0.034143
0.034271
0.036548
0.040612
0.046188
0.053705
0.063389
0.075120
0.08867)+
B
and B
O
1
for
Hartley's
Test
Likelihood
Ratio Test
F(y)
T(y)
7.50994
3.95173
2.05554
1.41018
1.35767
1.55706
1.83090
2.12069
2.35441
2.68254
3.26427
4.27518
5·90773
7.76941
9.18396
9.76091
9.66892
2.85608
2.49126
2.14952
1.84656
1. 58027
1.35419
1.18018
1.05887
1.00000
1.00375
1.07044
1.18947
1.35278
1.57294
1.85658
2.20016
2.59714
4.
HARTLEY'S METHOD
The theory presented in the previous gection required the deletion of the
functions
TO(ai - x)
and
Tl(a
- x)
i
from the set of basis functions or,
equivalently, the imposition of the constraints
~ction
g(x).
B
and
O
B
l
on the response
In applications where it is not appropriate to impose these
conditions (see Figure 1) it is still possible to test some hypotheses and set
exact confidence intervals on some parameters using a method due to Hartley
(1964) •
In those applications where the imposition of
and
Hartley's method is not recommended for setting confidence intervals.
is appropriate,
Usually
such intervals will be wider than those constructed using the metbods of Section 2
and sometimes the confidence interval will not contain the least squares estimate
of the parameter.
Consider the hypothesis
H;
A:
ag~inst
,. = r.:.{)
T
rv
for the regression model
where
e
rv
has been
partitione~
according to
~
If the regression model with
~
,
(I
"" R.,..r,
,)
specified to be
is a linear model in the remaining parameters Q
~,
then Hartley's method is
13
~pplic~ble.
For example, the quadratic-quadratic model subject to
86
join point
specified as
~~ is
which is linear in the parameters
H:
~
(8 ,8 , 8 , 8 , 8 ),
1
2
3 4
5
is true and one fits the
where
denotes
arb~trarily
line~r
If the hypothesis
model
chosen regressors then the least squares estimate
is normally distributed with the zero vector as its expectation.
the standard F-test tor the hypothesis
is test for
H: r!
=.:!o
The regvessors
the alternative
A;
~t
~
B with the
O
HI:
0
rv
=0
Consequently,
in the linear model
in the nonlinear model
must be chosen to obtain good power for the test under
f Xo'
A method of choosing the extra regressors which
works well for grafted polynomial models is the following.
Plot the residuals
from fitting
against
x
t
for various choices ot
plots, try to choose
reg~essors
~.
From a visual inspection of these
which are functions of the
a "good" least squares fit to the plotted residuals.
x
t
and would give
For example, the choice
14
yields adequate results for the hypothesis
quadr~tic
model sUbject to
H:
8
6
= 86o
of the quadratic-
B •
O
A confidence interval may be set on the point of join for grafted polynomial
models with a single join polnt using this method.
Letting
join point, one includes in the confidence interval
which the hypothesis
HI:
i
~
2
corresponding to
I
H:
~
correspopd to this
those points
~
=
for
is acoepted.
In
models with multiple joins
this same process can be used to construct joint confidence regions.
One might note that Hartley's method depends only on the assumption of
normal errors
e
t
so that probability statements based on the method are exact
and not approximate as with the methqds of Section 2.
However, in this author's
opinion, the deficiencies of the method noted earlier outweigh the advantages
of exact probability statements in situations where either approach is applicable.
As an example, we may apply these methods to the data given in Table 2 using
the quadratic-quadratic model sub.ject to
B
O
The parameter estimate
~
= (-56,
Table 5.
test is
86
SSE~)
-8.8,12, -.93,2.86)'
(8 1 , 8 , 8 , 84 , 8 ) for
2
5
3
by ordinary least squares and then varying 86 as seen in
is obtained by minlinizing
specified
45~
with respect to
A 95% confidence region for
86 obtained by inverting Hartley IS
TABLE 5.
Tests for the point of join using quadratic-quadratic subject to
Table 2.
EO
for the data of
15
The disconnectedness of this confidence region can, of course, be eliminated
by including the omitted interval
(2.875, 2.885).
truth of the statement
"I( I
contains
This would not alter the
16
REFERENCES
Bill~ngsly,
P. and Topsoe, F. (1967) "Uniformity in weak convergence," Z.
Wahrscheinlichkeitstheorie und Verw. Geb., 7, pp. 1-16.
)
;
Fuller, W. A. (1969) "Gr""fted polynomials as approximating functions," ...--The
1
Australian Journal of Agricultural Economics, 13, PP' 35- -1-6.
inference for nonlinear re ression models.
University.
Gallant, A. R. (1973a) "Inference for nonlinear models," Institute of Statistics
Mimeograph Series, No. 8rr5. Raleigh: North Carolina State University.
Gallant, A. R. (1973b) "The power of the likelihood ratio test of location in
nonlinear :,regression models," Institute of Statistics Mimeograph Series,
No. 889. Raleigh: North Carolina State University.
Gallant, A. R. (1974) "Testi,ng for the location of a subset of the parameters
of a nonlinear regression model," Unpublished.
Gallant, A. R. and Fuller, W. A. (1973) ,,It,itting segmented polynomial regression
models whose join points have to be estimated," Journal of the American
Statisti~al Association, 68, pp. 144-147.
i
Gerig, T. M. and Gallant, A. R1 (197)-1-) "Computing methods for linear models
subject to linear parametric constraints," Journal of Statistical Computation
and Simulation. In press.
I
Graybill, F. A. (1961) An introduction to linear statistical m?dels, Vol. I.
New York: McGraw-llill.
Hartley, H. O. (1961) "The modifi.ed Gauss-Newton method for tll'l fitting of
non-linear regression functions by least squares," Technometrics, 3,
pp. 269,,280.
Hartley, H. o. (;1.964) "Exact confidence regions for t le parameters in non.
linear regress+on laws," Biometrika,
51, pp. 347-353.
,
Hsiung, P. o. (197}-I-) Stud;y- of the interaction of organic liquids with pol;ymers
b~l'.y~m...e..;,;an~s..,.....;.o.;;:.:r~€i:.;;.;a ...s,.....-.,.;.c;.;:h..;.r_o...
rh;.;;.a..;.t,..,o~e:!.:;;.r..;.a:.l?..;, ;h;j4't. • Ph •D. di sser tat i on, North Carolina Stat e
Univel'::;ity.
Jennrich, R. 1. (1969) "Asymptotic properties of non-linear least squares
estimators," Annals of Mathematical Statistics, '+0, pp. 633-6 1+5.
i
Malinvaud, E. (1966) Statistical methods of Econometrics.
McNally and ComltanY.
Chicagol
Rand
Malinvaud, E. (1970) "'rhe consistency of nonlinear regression," Annals of
Mathematical Statistics, 41, pp. 956-969.
.
'
(
Theil, H.
Inc.
(1971)
PrInciples
of econometrics.
,
New York:
John Wiley and Sons,
18
Notation.
For the regres,-;j
C,(i
:nodel
(t == 1, 2, .. "
where
f(x, e)
rv
is a known fune: Vi em defined on
'"
but unknown value of the po.ramc t(~l'
Y
==
( y l' y 2' ... ,
\ /
Yn )
~
1:. X Q
n)
denotes the true
define:
(n X 1) ~
(n X 1) ,
==
==
*
~
and
the
p X 1
()
38:
vcctUI" whose jth element is
f~, ~)
J
==
th e
~ p
p"
f'
ma~rlX
!(~)
=
the
n X p
matri.x vlhose tth row is
..@.(t)
==
an;>, Borel mEasurahle fun~tion of
Wh ose l',J,th element
~/f(~t'~) ,
X with SSE(~)
with respect to the hypothceis
against
where
e
,.."
has been pl;lrtitioned acc'jrding to
A:
~
1.'S
f .!o
== infQ.SSE~)
,
19
define:
(n )( n)
(n ~ n) ,
<;]
"P
'!o (R,)
:;:::
f(X, e)
:;:::
rv
rv
~OV<.)
;::
r! ((~) .=!;o»
the
(n X 1)
rXl
n
the
>;
r
vector
,
who~e
.th
J
element is
ep0
j
f(x, (g.. ~»
"1
ttl
matrix whose trow is ;;f~t' ~, ~»
J
,
(n )( n) ,
= l - .~./R)
~o(g)
:;: :
,!~ * )
(n X n) ,
- rf.o(g)
(n X 1)
Densities
and Distributions.
t
i
squared density function with
1961, p. 74) and let
Let
0
2
n(t;~,
and let
2
0)
v
G(t; v, A)
Let
g(t; v, A)
denote the
non-centr~
degrees freedom and non-c~ntra.lity
A
(Graybill,
denote the corresponding distribution function.
denote the no~al density function with mean
~
and varianc~
W(t;~, ~2) denote the corresponding distribution function.
H(x; vl ' v~, Al , ~2)
chi-
to be the distr~bution function given by
Define
20
0,
:x < 1, "2 > 0 ,
= 1,
x
X~
> 0 ,
x > 1 •
Remark.
The distribution function
H(x; vI' v ' ~l' A2 )
2
is partially
tabulated in Gallant (1973b) and can be approximated by the non-central
distribution with argument
v (x-l)/v ,
2
l
VI
numerator degrees freedom,
denominator degrees freedom, and non-centrality
Al
when the parameter
F"
v
2
~2
is
small.
Definition.
let
(Malinvaud, 1970)
Let
~
be the Borel subsets of
[~};=l be a sequence of inputs chosen from
indicator function of a subset
!.
Let
I
A of .-v
1. • The measure IJo
n
A
on
05)
!
and
be the
~,Q)
is
defined by
for each
A e ~ •
Definition.
,
on
(1, CL)
'"
j
(Bi11ing~ly and Topsoe, 1967)
is said to converge weakly to a measure
n'"
~
(1,
a)
.-v
....
on
.-v
real valued, bounded, continuous function
as
A sequence of measures
co
g
with domain
1
f'Y
(IJo )
n
if for every
21
For each assumption below it is impltcitly assumed that any lower numbered
assumption necessary for existence of terms is satisfied.
ment of AssU1Jlption '7
for the
G
I".....~
requir~s
For example, the
Assumption 6 for definition of IJo
me~surability of
state~
tmd Assumption 2
(x:
f"'I¥
~.
Q, is a closed subset of HP •
~.
f(x,
~.
For given
e)
is continuous on 1)( n
~fttJ
,-.,..;,.....
n
•
X
and almost every
there is a
!
in
~
minimizing SSE(e) •
""
~.
The errors
(etl
are independent and identically distributed
2
0 > 0 •
with mean zero and finite variance
is a compact subset of
~.
weakly to a measure
The sequence of measures
IJo
on
(!,
~.
If ~f,t*
~.
Given M > 0
(IJo n l
.
~)
and
lcQ
k
R.
then
there is an
iJ'Le:
N and a
then
~.
closure
ff
There is a bounded open sphere
(~)
detennined by
£f
f(~,
j) f
f(~~ ~* )) ~ 0 •
K such that for all
Ittll <
K •
containing
is a subset of .Q •
~.
!f(~,~)
converges
exists and is continuous o,n ~ x Q, •
*
~
whose
22
~.
The matrix
is non... singular.
~.
There is a function
tp(2S,~)
which is uniformly bounded for
such that
~.
The response function
f,
inputs
[~}
and errors
(e )
t
are such that given a sequence of random variables
n -
as
it follows that
~
p
as
n -
m
for
i
~
....
0
1, 2, ••• ~ P •
The verification of Assumption 8 is unnecessary when those which
pr~ceed
it as satisfied and
~.
.G
~s
bounded.
Also, the verification of Assumptions 12, 13 is unnecessary when those
which preceed them and
~.
continuous over
1
The partial
XQ
exist and are
derivat~ves
.
Additional assumptions which are required to derive the asymptotic
of the Likelihood Ratio test of
been partHioned according to
!'
agains t
H:
=
(R.' ~ :L')
are:
A;
,!
'I ~
when
p~operties
!
has
23
~.
J [f(,3, ~* )
- f(;S' (I<.,~)) J2 dJJo (~)
~.
closure .....
~
R.o
There is a unique point
over ~
which minimizes
= f.R,:
There is a bounded open sphere
containing
Ro
whose
is a subset of ....R.
~.
The matrix
is non-singular.
If a regression model satisfies Assumptions 1 through 8 then
Theorem.
converges almost surely to ~
Proof.
ana. variance-covariance matrix
Theorem.
.
Moreover,
n .1,(;, (t)!(,i)
0
converges alm913t
Gallant (1973 a) .
We are given a regressiqp model
and the data pairs
Condition~.
S
1.1
_
,t .
Proof.
to
2
converges in distribution to a multivariate normal with mean
~
surely to
~
If a regression model satisfies Assumptions 1 through 13 then
Theorem.
f'J
a~2 converges almost surely to
and
Gallant (1973a).
i
*
In(e'" - e)
*
!
such that:
(Yt'.<st.)
(t
= 1, 2, .•• ,
n) •
There is a convex, bounded subset
S
of
RP and a ~ interior
24
1) Zf(~J ,,~)
exists and is continuous over
2)
~. Simplies the rank of
3)
SSE(~O) <: inf(SSE(,.g):
4)
There do not exist
~
,f,
!(~)
Find
for
.@o'l
in
2.
S
hO which minimizes
S) •
such that
. and
Construct the sequence
t;:: 1, 2, •. n n •
p.
a boundary point of
%.. SSE (@.') ==! SSE (~ ") =
Construction.
is
S
SSE(e') ;:: SSE(e/')
•
t"oI
I"'J
~kJ~=l as follows:
SSE(~ +~)
over
Ao
;:: (A: 0
~
XS
1.
~O+%IS) .
1)
Set !l ;:: ~ + h~ •
Compute £1;::
Find
Al
[X (~l ),t(,@,l) ) ..lX
I
which minimizes
I
(,@,l)(.;z - .!(!l) J
S$E(~l + ~l)
over
•
A1 ;:: tA~O ~ ~ ~ 1 ,
~l + 1£1 e S}
Conclusions.
The for the s~quence
is an interior point of S
~
C~/k)k=l
for
k == 1, 2,
converges to a limit
3)
!
SSE(~* )
=0
I"'J
•
i t follows thi3-t:
*
~
...
which is interior to S
t
25
Proof:
Gallant (1971).
......--.,.-
£0, k.o' £0,
In the following theorem,
~ =
functions evaluated at the point
Ro
and
.1.
denote the corresponding
P
NO
given by Assumption 16,
For a regression model with normally distributed errors which
Theorem.
satisfies Assumptions 1 through 13 and Assumptions 16 through 18 the Likelihood
Ratio test statistic for
H:
T
rv
= ,..;.0
'T
against
A:
T
'"
# T°
may be characterized
P<J
by
where
c
n
converges in probability to zero and
If the condition
~
= £0
is satisfied, the random variable
X has the distri-
H(X; p-r, n-p, Al , A2 ) where Al = ~(£ ~ £o)~/(2~2)
A2 = ~.1.~/(2a2) .
bution function
When the null hypothesis
satisfied and
Proof.
X
~s
H:
~;:: ~
distributed as
Gallant (1974),
is true the condition
H(x; p-r, n-p, 0, 0).
and
£0£;:: ~
is