•
INFERENCE FOR NONLINEAR MODEIS
by
A. RONALD GALLANT
Institute of Statistics
Mimeograph Series No. 875
Raleigh - June 1973
REVISED - June 1975
•
ABSTRACT
Inference for Nonlinear Models
The study considers estimation and hypothesis testing problems for a
regression model whose response function is nonlinear in the unknmm
parameters.
The results of Malinvaud are extended to obtain asymptotic normality for
the least squares estimates of the nonlinear parameters.
The conditions set
forth do not require the existence of second order partial derivatives of the
response function in the parameter.s and do not require that the parameter
space be bounded.
Hypothesis testing problems in the situation where the errors are
normally and independently distributed are considered.
For the hypothesis of
location with variance. known, a necessary and sufficient condition :for the
existence of a uniformly most powerful test is obtained and the asymptotic
.-
null and non-null distribution of the likelihood ratio test is derived.
A
special case investigated is a test of location when some parameters enter
the model in a linear fashion.
D1FERENCE FOR NONLINEAR MODElS
Y
A. R. GALLANT
1.
Introduction.
~
The unknown parameter
The regression model considered has the structure
80
is contained in the parameteter space 0
a subset
are contained in X a subset of Rk ;
t
the rule of formation of the sequence [xt }t=l is assumed known. The
sequence of random variables [etJ;=l are assumed independent each with
of RP •
The input variables
x
ClI)
distributi.on function F(e) • This determines the (marginal) probability
n
measures
denotes the n dimensional Borel sets
on (R J 3) where
n
and also determines the measure
over
A s~le
... ,
is generated according to the model in order to estimate
test hypotheses as to the location of
e
8.
In Section 3 we extend the results of Malinvaud (1970) to obtain
measurability, strong consistency, and aymptotic normality for the least
squares estimator.
In addition technical results on characterization
and rates of convergence are obtained for use in later sections.
assumptions employed allow 0
The
to be an unbounded set and do not require
that the second order partial derivatives of f
in
8 exist.
These
relaxations of the usual assumptions were
11
This research ~as supported in part by the Bureau of the Census Contract
No. 1-35044 and by a N.D.E.A. Title IV Fellowship.
-2-
motivated by a desire to accommodate Problem B below and to develop an estifuation
theory sufficiently general to include the grafted polynomial response functions
which have seen use in applications-
A grafted quadratic-quadratic model
is used as the example in Section 9-
It should be noted that our assumptions
do not rule out the case when the response function is linear iri the parametersThe remainder of the paper considers these
tl~
hypothesis testing
problems :
Problem A-
The form of the response function
~
well as the sequence
H:
to test
eo
1"\
€
Problem B-
For given
a.
€
(0,1)
is known as·
and given n we wish
against K:
and
The form of the response function is known as well as the
~
sequence
HH
tXt} -
f(x,e)
tXt} -
Some of the parameters enter the model in a linear
fashion so that the parameter space has the form
and the response function
For given
K:
eO
€
a, €
has the form
(0,1)· and given n we 1'1ish to test
OK when
H:
eO
€
0H
against
-3The theoretical setting we adopt to approach Problem£. A andB follows
Briefly, the probability pYa (B)
Lehmann (1959) •
falling in the set
is given by P~(Be) where
from an
B
... , en
• • • ,- e ):
n
A test of
cp(:r)
'P
H:
eO
€
0H
against K:
mapping the sample
is given by Ee'P
=
eO
(Yl' ••• , Yn)
J cp(y)~
of obtaining a sample
(y)
uniformly most powerful .(UMP) ,
€
o.K
is a
+ f(x
(B )
into [0,1].
n
n
,S»
Bl •
€
measurable function
The power function of
The lisage of the terms level
and most powerful
(MP)
a.,
are standard
(Lehmann, 1959, p. 60 ff).
2.
Notation,
definitions, assumptions.
Definition 2.1.
~~~
and given n
For the regression model described in Section 1
define
W'
= the transpose of a matrix W.
w+
= the
" II
= the Euclidean norm.
Moore-Penrose inverse of a matrix W (Rao, 1965, p. 25).
... , e
e
n
)' •
y
Vf(x,S)
=
the
tlf(x, a) = the
p X 1
vee t or wh ose J.th e1 ement ·1S
p X P matrix whose i,jth element is
....La f(x,e).
o J.
2
oe~oe.
1.
J
.
f(x,e).
-4F (9)
n
= t ·h e
n Xp
~(e)
= Ily -
f
n
' whose tth row ~s
.
matr~x
(6)1\2 •
Pn (e)
=I
- Pn (6) •
IA(x)
= 1 if . x
a
= the
€
A;
= 0
if x
4A •
Borel subsets of X.
~
8n (Y)
= a (Bn ) measurable function such
tha~
~
~(6n)
= infQ~(e) •
When considering Problem B, the following additional notation is
convenient:
Definition 2.2.
Given the conditions of Problem B, define:
~ - ~
a(8(2»
= the n X.l vector whose tth element is ao(xt ,S(2»'
A(e(2»
= the n X Pl matrix whose t,jth element is a j (xt ,S{2»'
a
= a(OS (2» •
H
~
= A(08(2»
,. (8)
= ([~(fn(e) - aH)] , 0 8(2» •
•
I
-5-
=I
- PH •
,.,..
.en (y)
Definition
2.3.
...
~
for each A
define
Definition 2.4.
as
JJ
n
to be the measure on
(x,a)
['lin!
on
(x,a)
(Billingsly and Topsoe, 1967)
A sequence of
is said to converge weakly to a measure
if for every real valued, bounded, continuous function
n
-+ <:0
which satisfies
a.
€
~~~
(x,a)
H
(Malinvaud, 1970)
_~~-
from X and an n
measures
intO ~(e)
= a (3n ) measurable function such that
on
V
g over X
•
The following definition is an extension to a sequence of vector
valued· random variables of the definition given by Pratt (1958).
~~.
variables, and
if for every
€
[Zn1 be a sequence of vector valued random
a sequence of (strictly) positive real numbers.
[a}
n
We say. that
Let
Zn
is order in probability an
and write
n
0
p
(a)
n
For. each
~.
=
0 (a )
p
n
> 0 there an M and an N such that
P[a-lliz II;?;M]<€ for n>N.
n n
We say
that Zn is of smaller order in probability than
.
Z =
Z
·n
if· a-lz ~ 0
n
n
assumpti~n
as
n
-+ <:0
an
and write
•
below it is implicitly assumed that any lower
numbered assumption necessary for existence of terms is satisfied.
For
-6-
7
example,the statement of Assumption
a
of lJ.and Assumption 2 for the
requires Assumption 6·for definition
measurability of· [x:
~~.
0
~.,
f(x, e ) i s continuous on
~~_~.
For given
in 0
minimizing
is a closed subset of
lly -
f
.
n
n
(e)1/
2
f(x,e) /:. f(x,eO)} •
RP •
X X0 •
and almost every
y (pieo)
there is a
e
*
•
[ e 1 are independent and identically
t
distributed with mean zero and finite variance cr2 > 0 •
~.
The errors
~.
X
~~~~~.
The sequence of measures· [lJ. n }
is a compact subset of
converges weakly to a measure
~0.Ei'
all
n> N. and all
~.
closure
f1'
e
€
M> 0
0
if
l
determined 'by
N and a
~=lf2(Xt,e)<
K such that for
M then
There is a bounded open sphere 0°
~~~~.
\7f(x, e)
Ass~tion
The matrix
11.
is non-singular.
tXt}
(x,a).
there is an
n-
•
\Iell <
K.
containing
is a subset of 0
~
e.
Given
on
IJ.
k
R
exists and is continuous on . X X 0 •
eO
whose
-7~ ..
(x, e)
for
€
x x IT'
There is a function
cp(x,8)
which is uniformly bounded
such that
The response function
inputs
tXt}
and errors
are such that given a sequence of random variables
a. s.
0
n ~
and
en----~) 8
fa}
n
with
~.
f,
ret}
f'J
as
n
~ ~
for
i
~
it follows that
= 1, 2, ••• , P •
As shown later the verification of Assumption 8 is unnecessary when
those which preceed it are satisfied and
~.
is bounded.
(2
Also, the verification of Assumptions 12, 13 is unnecessary when those
which preceed them and
~.
continuous over X X (2
The partial derivatives
2
b
b8.be. f(x,8)
J. J
exist and are
•
.<N"-"'J<::1VVQ'~~~l;:::eN'::e""sNto'Vim~at~J.::;;:·o~.
As stated earlier, our objective is to
obtain a large sample theory sufficiently general to accommodate the linear
model, the partially linear model of Problem B, and the class of nonlinear
models typified by the example.
e.
If a regression model satisfies Assumptions 1, 2, 3 for
then there is a Borel measurable function "8 mapping Rn into
n
~.
given n
-8n
"" = intO ~ (e)
~ (an)
such that
for all
y
in a set with
("pr eO )
probability one.
Proof.
_
_'
Construction.
Q(e,y) =
Q
'il.
(8)
0 : /lell < i}
obtaining the measurable function
~~i) = into. (8) •
J.
satisfies
€
Apply Lemma 20fJennrich (1969) with
by Assumption 1.
and
Let O.J. = (8
Since
infn~(8)
which is compact
n
Y =R
J.
n
R - O. which
J.
® =0.,
'?f.J. :
is measurable by Assumption 2
the sets
•
are J.n
Let
10
Ui'·
n
Y~ = Y*."'" Ui. - 1l y*.
'.'
...
J.
J=
J
Le+v
co
Y = U. 1 Y.
J.=
J.
t the
no t e -loh
u a
. t•
are d'J.S J. oJ.n
YJ.'
and set
en(y) = eO I,...,(Y) +
Y
Verification of properties.
li~ _
co
~=l '?fi (y)
I y . (y)
.J..
By Assumption
3 pitf (U~=lYi) = 1
""en
is the limit of measurable functions hence measurable.
Fbr given
Y
Ui=l Yi
*
" "
for some i hence~(8n)
= ~\Ai) =
€
iniQ
co
Y is inYi
~(8).
Lemma 3.1.
for all
eO
n > N.
C
Yi
~
n
Let a regression model satisfy Assumption 1 through 8
Then there is a bounded open subset
such that for almost every realization
estimators is in
and
S
for
n
(pe)
co
..•.. --,
J.
P
of
RP
containing
a sequence of least squares
sufficiently large.
+ n
S
lIyll
-9-
n- l Ilf (8° )11 2 :s; sUPxf2(x,eo) and n- l llel\2 a.s·
1J2 by the strong
n
Law of Large numbers, there is an M for almost every realization of (et }
Since
I
~
such that
A
n -~
A
IIfn (8 n ) II < M. By Assumption 8 1\8 n II <
Ef
B> K large enough that
lZ.~ ~:.,g.
valued function
Let @l
g(x,8)
~
S
€
P
= (8:
be continuous on X X @l.
J g(x,e)
Proof.
~
e
over @l
as
n
dVn (x)
-0
~
-0
Theorem 2.
n
> N.
Let the real
If a sequence of measures
then
J g(x,8)
dV(x)
•
0
Let a regression model satisfy Assumptions 1 through 8 for
A
all n > N and let An be a least squares estimator.
A2 a.s.
2
and On
IJ
as n ~
I
Proof.
~w'
-0
Let
Then
A
n
•
S be as in Lemma 3.1.
If we take @l
= (2 n S as the
parameter space of the regression model it will satisfy Assumption (a)
of Jennrich (1969).
Jennrich when
e
Choose
n
(X,G)
on
v
Malinvaud (1970, p. 967)·
~
for
and X c pk be compact sets.
R
(V } converges weakly to a measure
n
uniformly in
11811 < Bf.
K
We next show the model satisfies Assumption (b) of
is the parameter space.
-10uniformly on ® by Assumptions 1, 2, 5, 6 and Lemma 3,2;
e 1= eo
space
be a least squares estimator for the model '\ITith parameter
0
® and set
11·n = ""en
0
en
and by
*....
""
+ (S n - 8n )(1 - Is(S n »
,."
6 of JennrichS
n
Th~orem
a.s,
•
E
($)
Q
Is(en ) = 1,
""
0
0
= en ,
~
Lem.-na 3.3.
~~-"'-''''''''''''''''''-I
[~}
such that
[a'}
I-'n'
On.'
(3n _a_,_s_,__•
Lemma 3.2
..••
00
n
eo •
X
n
F'n\On
~ ) Fn\P
(';;(3 )
n
as
=
f .. (C!4(3)
n1J
f . . (a,~) - f .. (C!4(3) =
nJ.J
pe(E)
co
for
n
11:f)
a·s·
= O,we
have
~\On
2
•
By
SUfficiently large.
o
•
(J
0
uniformly on
0
IT'
1J
X· ff
;
J >.be
v
a.s.
-__a
• f ..( eo,
o
1J
•
F'FO •
n -IF ' (a) F «(3)
n
f(x,a) >.at
u
1
J >.et i
v
n
f(x,a) >. be
U
eO) •
0
ff
o.
X
fi:' •
where
f(x,~) du n (x).
•
J
j
f(X,~)
d/,L(x)
By
as
0
further, the uniform limit
continuous f'unctions is continuous on
a, s'
n
Then
We may write the elements of
~
-1
be sequences of random variables with range in
!n
€
= (J""2n
n
®
Let a regressio~ model satisfy Assumptions 5, 6, 9, 10, 11,
and
Proof.
n -1 Q (e' )
--n.n
and
""
n
and
with
o
(a, (3)
.
is a least squares estimator for the model with parameter space
Lemma 3.1,except for realizations in
n -
® and.
7.
en*
Let
Let
e€
then
by Assumption
Now
if
f.. ( C!4 ~ )
J.J
As a consequence
of
-ll~--2~'
Let
[e t } ,
X,
(xt } satisf'y Assumptions 4, 5, 6
respectively and let @c RP be compact•. If g(x,8)
is continuous on
X X e then for almost all realizations of ret}
!n ~nt=l
uniformly for all
~.
e.
over
By
e€
t'
e) e t - 0
@•
Lemma ).2,
n-
l
~=l g2(xt ,e) -
Apply Theorem 4 of Jennrich (1969).
Y~.2~·
respectively.
Let
[et } ,
g(x)
where each
V= [
X,
J g2(x,e)
[xt } satisf'y Assumptions 4, 5, 6
g.
~
= (gl (x), ••• ,
is continuous over X.
J gi(X)
gj(x) ~(x)]p X p
~ (x) ) I
Then
and Np[~'V} denotes the p
Proof.
Apply Corollary 1 of Jennrich (1969).
requisite tail products follows from Assumption 6.
8 ,
n
~
and dispersion
V.
~
,.
uniformly
n
dimensional multivariate normal distribution with mean
matrix
ctL(x)
Let
•
where
g(x
The existence of the
D
Conclusions (a), (c) of the theorem which follows characterize
S2
n '
as linear and quadratic fUnctions of the errors
similar to
those occurring in linear regression models plus remainders of specified
,.
probability order.
to
eO •
Conclusion (b) specifies the rate at which
en converges
-12'!heorem 3.
Let a regression model satisfy Assumptions 1 through 13 for
~
all
n
>
,..
Nand l e t e
such that for all
n
be a least squares estimator.
n> M det[F'(eo)F <eo)] > 0
n
'!hen there is an
M
and:
n
(a)
('6 -
(b)
eO), =0 (.,.1)
PlJn
n
If, in addition, Assumption 15 is satisfied then:
!rl~~.
as
n
n
-'00
Assumptions 6, 10, 11 imply
hence the existence of
det[n-1F~(eO)Fn(eO)J.-.
det(F'F) > 0
M.· All statements below are for
> max[M,N] •
Conc1usion·(a).
Let
cpn(e)
=
given by Assumption 12 and let en
Ci'n (en)
,..en e
0
,..,
fn(e )
n
° then ~n
~n
V Q (8 )
n n
8n
e
n
= ~n en
and by Assumption
'
= ~nen + 0 p (1)
,..,
= 2Fnt (e n )(y
eO(1 - I O(9
o
and
- f (e)}
n n
~n V Q
n
10
<e'n )
n
» •
-~,..,
~n V Q
.
=
0
p
(1)
n
(8 n ) = 0
If""
°
Ft(e)e - G vn(e· - e )' ,
n n
n
n
is
Note that
8n
•
If
hence by
Using the fact
and substituting the expression for
given by Assumption 12 we have
0p(1) = n
where
= enIOo(9n) +
cp(x,e)
is measurable by Assumptions 2, 10 and the measurability of
Theorem 2
that
(cp(xl'e), ••• , cp(xn,e»' where
,
-13-
By Assumption 13
Now G a.s •• F'F as we will show below and
n
that \[n (an - eO)1 ~ N •
P
n-%F'(eo)e ~ N
n
p
by Lemma 3.5 so
Thus
(I - [1 F'(eo)Fn (eO)l-l G ) \[n ~ - eO)
n n
- n
n
t. 0
hence
,.., - e °)' + 0 (1) = \[n [F' (e ° )F (e °) ] -1 F' (e ° )e + 0 (1) •
'{n (e,.. n - e ° )' = \[n (e
n
p
n
n
n
p
Gn ~ F'F
and the elements of
To see that
•
note that
n- 1F'n <en)Fn (e 0) ~
F'F by Lemma 3.3
_
are uniformly bounded by Assumption 12
hence
Conclusion (b).
By Chebishev's inequality the elements of
'{n [F'(eo)F (eo)]-lF'(eO)e are 0 (1) •
n
n
n
p
Conclusion (c). Omitting the subscript
n we have
"" - 8 °)' - lie"" - 8 °II 2cp(e)
. . . 112 +
.. lIy - f(e ° ) - F(e ° ) (8
using conclusion (a) and letting
probability order
0 (1)
p
we have
j
represent a random
0
P
(1)
p x 1 vector of
-14-
.. IIpL (ao)e
.. IIpi- (eo )e11
(n- 1z)F(eo)j -
1I~_eoll:2 cp(e)1\2 +
-
0
2
+ lIop<n-~)F(eO)j +
p
lie -
p
(1)
2
eO 11 cpfe)11 2
- 2e,pI.(eo)[F(eo)jo (n- 1z)
p
0
+
lie -
2
eOIl cp(e')J +
0
p
(1)
Now the second term of this expression is bounded by the square of the sum
which converges in probability to zero since
Lemma 3.3 and
(l/n)cp'(8)cp(B)
(1/n)F'(eo)F(eO) -. F'F
is uniformly bounded for all
ri.
by
The
absolute value of the term
is bounded by
20p (1) II n -1ze 'pi (e 0) 1111 n-1zcp(e') 11
~ 20 p (1)[e'e/nJ\[cp'(s)cp(e')/nJ1z
and the latter term is 0 p (1)
uniform bound on cp(x,e)
to-n2
= 1lY"(eo)eIl
'
by the Strong Law of Large Numbers and the
given by Assumption 12.
2
+ 0 p (1)
. + 0 p (1) +
which establishes conclusion (c).
0
p (1)
Thus, we have shown that
.. ~a ..
Conclusion (d).
the term
What is required is to use Assumption 15 to show that
21\7) .. e01l2e'r(eO)cp(a')
obtained above is
I
: By Taylor s theorem, there is a function
II e
.. e
°112cp(x,e)
-
e:
.
0
p
(1) rather than 0 (1) •
·0-0
X x O-t n
p
such that
2..
= ~(e - e ° )V f(x,e)(e - e ° )'
then
-1.
° )cp\e) = . \e
~
n
.."'"
21l'e .. e ° \I 2e',LTu(e
.. e ° )~t=l
u t " 2 f(xl,eHe
.. e ° )1 • .
sh~
If we
d(x,e)
that
(lin)
..
u td(xt,e)
denotes a typical element of
from conclusion (b).
Now
·n
~t=l
E(Zl) = 0
and
-to
The sum
converges in probability to zero where
2
V f(x,e)
the desired result will follow
as
compact
a
Thus, by Chebishev's inequality
n ... oo •
6
d(x,e)
is uniformly continuous over
lie -
such that
< (r1 + e) -Ie •
Zl ...
8°11< 0 and
(x,e) e X x
X
0
p
(1) •
x n
·0
Since X x
. Given e >
0°
0
is
there is
f."o imply Id(x,9) - d(x,eo) I"
Then for almost every realization of
(e }
t
there is an
N.
such that
so that
Thus
Z2 a.s.
~ 0
Theorem 4.
all
an
~
n
be a least squares estimator.
Then
The first conclusion follows from (a) of Theorem 3, Lemma 3.3
=~
n
= eO , and Lemma 3.5.
,...,
,..
01
n = Sn = en
-,...".
3 .3 wi th
n
Let a regression model satisfy Assumptions 1 through 13 for
n > N and let
Proof.
with
1
whic h impies
Theorem 5.
The second conclusion follows from Lemma
• 0
If a regression model s.atisfies Assumptions 1, 2, and 14 it
,.,. , . . . , . . ~ ~
satisfies Assumptions 3 and 8 for all
n.
If a regression model satisfies Assumptions 1 through 11 and 15 for all
n> N it satisfies Assumptions 12 and 13.
-15~.
Assumptions 1, 2, 14 imply Assumption 3.
First statement.
K = su~lIell + 1 "and Assumption 8 follows.
Set
By Assumption 15 and Tay1or'~ theorem there is a
Second statement.
8: X X rf -
function
cp(x,Fl)
rf
such that Assuro.ption 12 hoHs with
=
e = eo
o
~n}
Let
satisfying ~n
e
Pn [%(8n) = inf %(6)]
are measurable functions
D
is the
n
rf ,
-
n
where
E:
1
~n a.s·
be given.
e.J.t n (e)
•
eO,
and
By Jennrich
rf
with range in
p X p' matrix with row index
i
(1969) Iemma. 3 there
such that
and column index
j
defined by
Assumptions 1 through 11, 15 imply that Assumptions (a)' through (d) of
Jennrich are satisfied for the regression model with parameter space replaced
by
ff •
t7",' I?f
III
\0,
"
n
By Theorems 6, 7 of Jennrich ~
n
_ 6,0)' _s.,_I
surely to zero·
satisfied.
0
N •
p
Thus
By
~mma
a. s.,
I
8°
and
3. 4 the elements of D
Dn,,;n (111 - 6°)'
n
p. 0
converge almost,
and Assumption 13 is
-16-
will consider Problem A under the assumption that the errors
independently and normally distributed with known variance
Letting m represent Lebesgue meas~e,
with respect to m.
2
o known.
4It
o <02 <co •
f
~
= c(8)~
(9 ) - f n (9)
no.
and a function
•
Let a regression model have normally distributed errors,
For the hypothesisH:
fn
(9 ) f. f (8 )
on
where
2 ,
are
~ will have the density function
There is ann X 1 vector
such that
Lemma 4.1.
0
t
The results of this section are stated in terms of
Condition 4.1.
~
e
1
eo
eo
=
and 0 < a. < 1
vs.
IS.:
the unique
eO
a. e.
=
8
1
at level
m most pow'erful
test is
1
a'y S c
o
ay>c
'cp(y) =
I
.
where
c
and
za.
satisfies
qi
(z )
a;
=
lIall
0 Z
= a. •
a.
(qi
+
a f n (8 0 )
I
is the distribution function of a
standard normal random variable.)
Proof. . The proof follows from the Neyman-Pearson fundamental lemma
~
(Lehman, 1959,p·
65).
D
ct
-17Theorem 6.
~~
a)
Let
a regression model have normally distributed errors~
If Condition 4.1 is satisfied there is a uniformly most powerful
test for Problem A given by
cp{y)
1
~'y
S
o
~'y
> c
c
=
where
c
(~
=
1I~1I cr
is given by Condition 4.1;
b)
Z
a,
+ ~'fn (e 0 ) •
if ~ = 0
set cp{y) :=
a,.
)
If Condition 4.1 is not satisfied there does not exist a uniformly
most powerful test for Problem A.
Proof.
Conclusion (a).
~
vs.
K:
eO --
e1
at level
Lete
l
If ~
€
OK
and consider testing H:
eO
€
0H
c{e ) = 0 then f n
(e o
) =·f
n (el)
l
and for any test W. which is level a, for H we have E W = E W < a,
e0
e1
= Ee cP = Ee cP so cP is MP for (H, Ki). If c{e l ) > 0 and ~ ~ 0 the
o
1
MP test CPl of (H, Ki) at level a, is given by Lemma 4.1 with
= c{el)~
a,.
or
c = c{e l ) [ II~II cr za, + ~/fn{eo)]· Hence cp{y) =CPl(y) •
Thus for every 8 € OK cp is MP for H: 8° € 0H vs. K: eO = 8 •
a
and
Given choices of 8 , 8 € OK let a i = [fn{e o ) - fn{e )]
1
2
i
(i :;: 1, 2). There is a choice of 8
such that a ~ 0 or else Condition
l
1
4.1 will hold with ~ = O. There is a choice of e
such that a ~ cal for
2
2
any c ~ 0 or else Condition 4.1 will hold with ~ = a • Consider
l
Conclusion (b).
-18H:
eo
r-.
€ u
eO = e.
vs.
H
at level
J.
ex. with associated unique
a. e. .
m MP tests
=
CPi(y)
given by Lemma 4.1.
for
(H, Ki )
(i
If there exists a
= 1,
2)
We will show that
cannot exist.
(i
= 1,
cp
for Problem A it is
I{ y:
test
UMP
and we haveCP1(Y)
CPl;' CP2
a.e.
2)
= cp(y) = ~2(Y)
m so that a
a.e.
MP
m.
test for Problem A
UMP
Set
Now A is the intersection of two open half spaces of Rn so that A = ¢
implies
al ,
a2
a
€
l
= ca 2
OK'
c > O.
where
Then A
This is false by the choice of
is anon-en:q;>ty open set and
meA) >
o.
n.
consider Problem B under the assumption of normally distributed errors,
(J2
known.
The results of this section are stated in terms of
~..
[O,~)
such that
~.
buted errors,
vs.
IS.:
eO
(l
= Al
There is an
n X 1
vector
i\
and a function
c:
OK ....
P~ [aH - fn(a)] = c(e)i\ •
Let the regression model of Problem B have normally distriknown, and let. A
l
at level
ex. where
€
OK
For the hypothesis
•
fn(r (8
1
» ;. f n (1\)
m most powerful test is
cp(y)
1
a'y S c
o
a'y> c
=
H:
the unique
aO
€
a. e.
OR
-19where
~.
By Lemma"4.1
vs·
IS.:
level
(H ,
r
(H, K ) . Let
l
for
eO = 8 •
1
for
a,
is the unique
cp
a.e.
Since any test which is level
IS.)
8
EeCP
we will have . fP
€
0H
MP
for
MP
I).:
test of
a,
for
(H, Kl )
(H,
IS.)
if
cp
eO = reel)
must be
is level
then
= ~
[y:
a/yS; c}
I
Ie
=
because
P~~ = 0
~.
0
2
implies
qi
(z )
"
a,
a/~ = 0
I
Let a regression model have normally distributed errors,
known.
a)
If Condition 5.1 is satisfied there is a uniformly most powerful
test for Problem B given by
cp{y)
1
Tl/YS; c
o
Tl'y> c
=
a.
-20where
is given by Condition 5.1;
(11
b)
if'TJ = 0
= a. .J
set q:>(y)
If' Condition 5.1 is not satisf'ied there does not exist a unif'ormly
most powerful test for Problem B.
Proof.
,...,..,..,..,.
The proof of Theorem 6 may be used word for word with the
substitution of ?' (8.)
J.
for
80
(i
= 1, 2),
Lemma
5.1 for Lemma 4.1,
Condition 5.1 for Condition 4.1, and Problem B for Problem A.
~~~.
For Problem B if' a(8 (2»
ana there is a 8(2)! 08(2)
such that
= 0
D
for all 8 (2)
€
°(2)
then Condition 5.1
Y"If(8(2»! 0
is not satisfied.
~.
Suppose, to the contrary; that
F-Itn(8) = c(e)11
Then Pt!A(8(2» 8(1) =c(8)11 for all 8
Pl
~
ranges over Rand c(e);;: 0 we have PjiA(8'2»
8
€
OK.
8(2)
i
0
8 (2) •
OK
€
e
= 0
.
for all
Since 8(1)
for all
I
~.
As a consequence of Proposition 5.1 there do not exist UMP
tests in such common situations as H:
S2X
0
f{x,S) = 81e
or H: 8 = 083 vs.
3
S~
K:
= 082
vs.
K:
S~ !08 2 when
S x
S~! 083 when f(x,8) = 81 + S2e 3
(under reasonable choices of {xt } ) •
6.
Likelihood r tio test for Problem A.
distributed errors,
Under the assumption of normally
cr2 known, the likelihood is
-21Theorem 8.
,..."....----~
(J2 known,
Let a regression model with normally distributed errors,'
satisfy Assumptions 1, 2, 3.
The Likelihood Ratio test for
Problem Ais
where
and
cl
is chosen so that ES ~= a
(provided such a
o
cl
exists).
A
~.
suItHL(S)
= L(So) and by Theorem 1 suItL(S) = L(Sn)
The asymptotic distribution of tl(y)
~.
Under the hypothesis
with p
is given by
Let a regression model with normally distributed errors,
a2 known, satisfy Assumptions
2
(x (p)
H:
l through 11 and 15.
SO
= ao
denotes the distribution fUnction of a cro!'squared random variable
degrees freedom.)
Then there is an M such that for all
n>
M
where:
o
-22a)
X and Y are independent.
n
n
X is distributed as a non-central x2 with p degrees freedom
n
and non-centrality A = a- 2 6' P (8°)0 (Graybill, 1961, p. 83).
b)
n
Y is distributed as a normal with mean 0'-2 6' p~(eO)6
n
variance 40- 2 6' PJ.(AO)6 o.
c)
and
n
Proof.
~'
The first conclusion follows from the second by putting
= o.
6
Apply Theorem 3 and
= Xn
Set ~
n
+ Yn + 0 p (1) •
= ~n = eO
in Lemma
By Assumption 11· det (F'F)
rank
(F (eO»
3.3 then det(n-1F'(eO)F
(6°»
n
n
> 0
-+
det (F'F) •
and there is an M such that for all
= p. It is easy to verify that X
n
n.
have the required distributional properties.
0
n
= rank (p (eO»
n > M
and· Y
n
When Theorem 9 applies we denote the large sample approximation of the
0
critical point
X""
x2 (p)
c
in Theorem 8 by
l
c!;
that is,P [X ~ cll
•
,.,
9n (Y)
=
«y - ~)'~[Ahl+,
0
8 (2»
=
Gwhen
-23is
(IBn)
measurable and ~atisfies
,.,,.,
~ (8 ) == infQH ~ (8)
n
for every y
€
n
R •
As a consequence we have
Theorem 10.
~ _ . ~
Let the regression model of Problem B have normally
0'2 known, and satisfy Assumptions 1, 2, 3.
distributed errors,
The
Likelihood Ratio test is
=
where
t
2
(y)
is chosen so that
su~ Ee~2 == a
,., H
(provided such a
A
Proof.
sun....
L(8) = L(8)
~UH
n
Theorem 11.
buted errors,
exists).
2
A
and by Theorem 1
The asymptotic null distribution of t (y)
2
~
c
sun....L(8)
= L(e n ) •
-u
is given by
Let the regression model of Problem B have normally distri-
0'2 known, and satisfY Assumptions 1 through 11 and 15.
Under the hypothesis
H:
Proof.
~
As in the proof of Theorem 9 there is an
•
n
M such that
n > M implies
-24statements in the proof are for
the sUbscript
n
n
> max [M ,N}.
By Theorem
3 and omitting
we have
A
t (Y)
2
=
0-
2
lie
+ f(e
O
)
-
f(S) 11 2 -
0-
llFel1 2
2
-
0
2
0p (1)
,..
=
Since
0-
(P(6°) - PH)
2
/Ie +
~
6°6.) -
A.H 8(1)11 2
- (J-2 I1p.L e Il2 + opel)
is a symmetric, idempotent matrix with rank PI)
the
<;.
H
result follows.
When Theorem II applies we denote the large sample approximation of' the
critical point
c
2
in Theorem 10 by
c~ ;
that is,
P[X ~ c~]
= a,
where
= 1,
2
2
X "" X (:P ) •
2
approach is to replace
A
2
on
.·;\2
(provided .. 0n
0
2
in the test statistics
exists and is non-zero).
ti(y)
i
The test statistics thus
obtained are:
•
by
-25The corresponding tests are:
o
where
di
is chosen such that
s.(y)< d.·
J.
J.
= a (provided such a di eXists).
SUIhHEeVi
The same conditions which allowed us to approximate the critical points
of ~.(y)
by cJ.'
* allow us to approximate the critical points d.J. of
J.
V.J. (y) by c..
*J. To see this note that if Assumptions 1 through 8 are
2
2
satisfied for all n > N then ;~ a. s.
0- • Then if t (y) ..!.- X (f )
i
i
when eO satisfies Hit follows that
cJ.'
I
=
9.
.~~~.
function is to take
f(x,S)
At times, a desirable choice of a response
to be joined polynomial submodels constrained
to be continuous and once differentiable in x;
nomial splines.
sometimes called
poly-
Instances of their use when the join points have been fitted
by visual inspection of the data are found in Fuller (1966) and Eppright .
(1972).
We will consider a case when all parameters, including the abscissae
of join points, are estimated by least squares; fitting methods are discussed
in Gallant and Fuller (1973).
k
letting T (r) = r
if r ~ 0 and 0
k
quadratic model will join point abscissa
e
if r < 0,
5
a quadratic-
constrained to be once
continuously differentiable in x may be put in the form
-26Note that the model is of the form described in Problem B with
, a(8 ) = 0 , and A(e ) the
5
5
5
2
x t' T (85 - x t
• Since T2 (r) is
2
T~(r) = 2Tl (r) the second partial
(A ,
8 ,
eel) = l 8 2, 3 8 4 ), ·A (2) =
nX4 matrix with rows (1, x t '
only once differentiable with
derivatives of f(x, e)
~t
n
4
=R
X
8
»
do not exist at all points in
X X
0
us further specify the model by taking X = [a, b]
[c, d]
- = < a < c < d < b <=.
where
and
=
The inputs
{xtJt=l
from X will be chosen as "near" replicates according to the following
construction.
Choose
q ( ~ 5)
points
(Zi}{=l from
five satisfy Zl < Z2 < c < d < Z3- < Z4 < Z5.
}=. 1
(Z"..
l.J J=
for
from X converging to the
i = 1, 2, ••• , q and such that
Choose
X
so that the first
q sequences
= [
at the rateL: . 1 Z. . - Z. ]2 <
Z.
J=
1.
l.J
1.
=
Zll < Z21 < c < d < Z3l < Z4l < Z5l •
Assign the inputs according to xl =Zll'
x 2 = Z2l' ••• , x q = Zql '
lastly, take the errors ret} to satisfy Assumption 3,
= Z , ••••
q+l
12
take 8° interior to
x
n,
and assume
8 ~ 0 •
4
The quadratic-quadratic model thus specif'ied can be shown to satisfy
Assumptions 1 through 13 for all
found in Gallant (1971).
n
~
5; a detailed verification may be
In the next few paragraphs we will sketch the
verification of Assumptions 8, 12, 13 for this response function.
Assumption 8.
The sequence of measures
(Un}
determined by
(Xt }
converge weakly to the measure defined by U(A) = q-l L:{=l I A(Zi). The
matrix n- 1A'(8 )A(8 ) is positive definite forn ~ 5 and using Lemma 3·2
5
5
its elements can be shown to converge uniformly to the elements of
-27provided cS 8 S d.
5
it follows that
trace
n""l
Since
n""l
~~=lf2(Xt'
~~=lf2(Xt'
8) = n",,18(1)A'(8 )A(8 )8'(1),
5
5
e) S M implies
~i=l 8~S
(n-1A'(e5)A(85»""~.
G(8 ) is positive definite for c S 8 S d
5
5
1
and tr~ce (n- A'(8 )A(8 »-1 converges uniformly to trace G",,1(8 ) as
5
5
5
n .... 00 provided c S 8 S d. Then given € > 0 there is an N such that
5
l
for n> N, n- ~=l f 2 (xt , e) S M implies
Assumption 12.
Given the point
(ro' sO)
the fUnction
s.T (r)
2
be put in the form
The last term above can be put in the form required by Assumption 12.
e·
can
-23Assumption 13.
eo
ff
If,
is a closed and bounded sphere containing
there is a finite bound
f~ (x, e)
where we have written
Given ~
n
K
ff
with range in
such that
b~
for
and
~
a· s.
n
(k
f(x,S)
k
I
= 1,
2, ••• , p) •
eO
,.
~
= n-:e
E. E {f'(z .. , 1)' ) - f'(Z., ~ )} e ..
1. j
1.J
n
1.
n
J.J
+ E. tnin)i {f'(ZJ."
1.
~n)
- f'(Z1."
eO)l. (m.lon )-~ r.e
..
J ~J
=x +y +Z
n
n
n
Now
Ixn 1S
i
n-
E.r!: 1 If'(Z .. ,
1. J=
1.J
a.s.
co
I
(J
~n )
- f'(Z., ~)I
1.ll
K (E. E. M (Z .. - Z.)
J. J=
1.J
1.
r"
21..
le .. 1
J.J.
-29The last term on the right can be made arbitrarily small by varying M
(m. )-i' t.e..
hence
X
n
(m~n)i
[f'(Zi' en) - f '(Z.J.' -eO)1.
Finally,
12.
a·s •• 0.
Z -L. 0
n
Since
J.n
J
a·s·
I
J.J
q -i •
s,.
o
N
l
=0
and
we have
Y
n
a. s. .....
0
by Chebishevrs·inequality.
~.
I wish to express my appreciation to Professor .
Wayne A. Fuller for his encouragement and guidance during the preparation of
my dissertation of which this work is part.
-30REFERENCES
•
[1]
Billingsly, P., and Topsoe, F. (1967) Uniformity in weak convergence.
Z. Wahrscheinlichkeitstheorie und Verw. Geb. 7, 1-16.
[2]
Eppright, E. S., et. ale (1972) Nutrition of infants and preschool
children in the north central region of the United states of
America. World Review of Nutrition and Dietetics 14, 270-332.
[3]
Fuller, W. A. (1969) Grafted polynomials as approximating functions.
Austral. J. Ag. Econ. 13, 35-46.
[ 4]
Gallant, A. R. (1971) statistical inference for nonlinear regression
models. Ph. D. Dissertation, Iowa state University.
[5]
Gallant, A. R. and FUller, W. A. (1973) Fitting segmented polynomial
models whose join points have to be estimated. J. Amer. Statist.
Assoc. 68, lh4-l)-f.7.
.
[6]
Graybill, F. A. (1961) An introduction to linear statistical models,
Vol. 1. Wiley, New York.
Jennrich, R· I. (1969) Asymptotic properties of non-linear least
squares estimators. Ann. Math. Statist. 40, 633-643.
•
•
[8]
Lehmann, E. L.
[9]
Malinvaud, E. (1970) The consistency of nonlinear regressions.
Math. Stat. 41, 956-969.
[10]
Pratt, J. W. (1959) On a general concept of "In Probability."
Math. Statist. 30, 549-558.
[11]
Rao, C. R.· (1965) Linear statistical inference and its applications.
Wiley, New York•
(1959)
Testing statistical hypotheses.
Wiley, New York.
Ann.
Ann.