•
..
RECURSIVE M-TESTS FOR 1HE CONSTANCY OF
MULTIVARIATE REGRESSION RELATIONSHIPS OVER TIME
by
Pranab Kwnar Sen
Department of Biostatistics
University of North Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 1458
April 1984
RECURSIVE M-TESTS FOR THE CONSTANCY OF MULTIVARIATE REGRESSION
RELATIONSHIPS OVER TIME
Pranab Kumar Sen
Department of Biostatistics,
University of North CarOlina,
Chapel Hill, N.C, 27514, USA
Key Wo~ds and Phrases : Asymptotic theory; change-points; CUSUMstatistics; invariance principles; ~ecursive M-statistics; ~eg~e­
ssion p~ametel>s; sequential detection p~oblem; stopping ~les.
ABSTRACT
Based on a general class of recursive M-estimators of regression parameters in a general multivariate linear model and allied
•
M-statistics, some testing procedures for a possible change in the
regression relationships occurring at an unknown time point are
considered. The (asymptotic) theory of the proposed tests rests on
some invariance principles for recursive M-estimators and related
residual M-statistics, and these are studied too.
1. INTRODUCTION
Let XI, ••• ,X be independent random p( _> I)-vectors, taken at
-n
ordered time points tl, ••• ,t , respectively. At time point t. , we
n
1
also have a q( > I)-vector c. of known regression constants, and
-
-1
the following linear model is assumed to hold
•
X.
_1
=
Q(i)
~
-
c.
_1
+
e. , i=l, ••• ,n
_1
(1.1)
(i)
_
(i)
where the 13
( - ((13 .. ,)) '-I
.. '-1
) are lmknown
JJ
J- .···.P.] - •...• q
(pxq) matrices of regression parameters and the e. are independent
-1
and identically distributed (i.i.d.) random vectors (r.v.) with a
distribution function (d. f.) F defined on the p-dimensional Euclidean space EP. A special case of this model is the usual shift
= 1 for every i ~ 1; in this case. the f3(i)
i
relate to location paramters (vectors). In the context of continumodel where q=l and c
ous inspection plans [viz •• Page (1954) J. one may want to test for
the identity of these location vectors against a composite alternative that at an unknown time point ( t
for some m :l<~n). a
m
change occurs. In a somewhat more general situation. one may like
to test for the null hypothesis
••• = 6 (n) =
~
( unknown ) •
(1.2)
against the composite alternative that for some m
eel) = ••• = f3(m)
t
S(m+l) =
= f3 (n)
(1.3)
where m (1 ~ m < n) is unknown. Thus. if H stands for the alternam
tive hypothesis in (1.3), we may formulate the composite alternative hypothesis as
X
.
n-l
=
U
m=l
H
(1.4)
m
For the univariate case (i. e., p=l) and normal F, this testing
problem has been treated in detail in Brown, Durbin and Evans(1975)
and detailed references to earlier works were also cited there. A
very similar problem may also arise in the context of a (generalized) sequential detection problem [ viz., Shirayayev (1963, 1978)
for the specific location model J. where a series of independent
random vectors is observed sequentially, such that at each time
point the model in (1.1) is assumed to hold, and for some unknown
m (possibly +00 ) (1.3) holds: The problem is to raise an.alarm if
m <
+
00
•
In this sequential scheme, one needs to choose a positive
•
integer valued r.v. (i.e., a stopping rule) N , such that if m <
+00
,
E(N-m)
+
should be as small as possible. while, if m = +00
,
+
= max(N-m,O)]
then E(NI m = 00)
[or some other characteristic of (N-m)
.
should be as large as possible { or the probability of a false
alarm i.e., p{ N <00 1m
•
= +oo}
should be as small as possible]. It
may be noted that in this setup of a sequential detection problem,
one has a genuine sequential procedure based on a well defined stopping rule N, while, in the setup of (1.1)-(1.4) {which is usually
referred to as the change-point model ],the testing procedure may
or may not be (quasi-) sequential in nature. If one adapts a quasisequential procedure, i.e., (1.2) tested against (1.3) recursively,
though n is specified in advance, one may be able to have an early
stopping, while, in a non-recursive testing scheme, the test for
the change point is made only when all the n observations have been
obtained. Methodologically, the testing procedures in the sequential
detection problem and the quasi-sequential ones in the change-point
problem are very similar, and, we intend to pursue these here.
For the quasi-sequential procedure in the change-point problem,
recursive residuals playa very important role. Along with a general
formulation of such recursive residuals, some invariance principles
for them were considered in Sen (1982a). These enable one to test
for (1.2) against (1.3) recursively, when F need not be multi-normal
(when the sample size is not small). Recursive rank tests for the
change-point problem were considered by Bhattacharya and Frierson
(1981) for the particular model where under the null hypothesis ,
the random variables are i.i.d. ; these tests are adaptable for the
particular case of the shift model, but, in general, for the regression model the null hypothesis fails to ensure the identity of the
d.f.'s of the X. , and hence, their recursive ranking scheme may
~
run into difficiculties for the general model in (1.1). However, in
such a case, one may use aligned (signed) rank statistics based on
recursive residuals, and the general asymptotic theory of such recur-
·e
sive residual rank tests was developed in Sen
(1982~
(l983~.
Earlier, Sen
considered general recursive V-statistics and studied the
asymptotic theory of recursive tests for the change-point model based
•
on such recursive V-statistics (and their jackknifed estimators of
variances). For the univariate shift model, recursive M-tests for
the change-point problem were studied in Sen(1983b). We intend to
extend the methodology of this paper to general multivariate linear
models.
Along with the preliminary notions. some non-recursive M-tests
..
for the constancy of regression relationships over time are considered briefly in Section 2. Recursive M-tests are then introduced
in Section 3. Along with the basic regularity conditions. some invariance principles for such recursive M-statistics are presented
in Section 4. These results are then incorporated in Section 5 in
the general formulation of the asymptotic theory of recursive Mtests. The concluding section deals with some general remarks.
2. NON-RECURSIVE M-TESTS
Our proposed tests (both recursive and non-recursive ones) are
based on some M-estimators of regression parameters and some related
(aligned) M-statistics. First. we introduce these estimators and
,
statistics. Let 1jJ = { 1jJ(t) = (1jJI(t) ..... 1jJ (t) ) • t e: R =(_co.co)}
-
-
p
be a suitable vector of score-functions • and. for every
«(b).n))
e: RPq , we write _B' = ( bl
•••• ,b ) and let
N
_
B
=
..
(2.1)
~P
n
,
1-1 'n(b.) = L. IC' n 1jJ.(X .. - b.c.) , I~J~JJ; l~~~q,
nJ~ -)
1=
1~
J 1)
-)-1
where _1
c.
= (c.l'
•••• c.1q )
1
,
••• ,X. )
= (X.I,
1
Ip
and X.
_1
,
• i=l, ••• ,n. We
assume that for each j (=1 ••••• p), 1jJ. is nondecreasing, continuous
and skew-symmetric (i.e.,
J
1jJ.(u)
+
)
1jJ.(-u) = O. ~ u e: R ). 1jJ. has a
J
J
bounded derivative inside (-k.,k.) (where 0 < k. < co) and 1jJ.(x) =
J
1jJ.(k.)signx for
)
J
J
J
Ixl > k .• In passing, we may remark that this
J
-
)
boundedness condition on the
1jJ. is not unnatural for the M-proceJ
dures. In fact, for the particular Huber-estimator. one has 1jJ.(x) =
)
x for
Ixl <k. and
k.signx for Ixl >k .• Such a boundedness condi-
- J)
)
tion induces robustness against outliers, and, can mostly be justified on the ground of ( local ) minimax properties • Also, we assume
that corresponding to the d. f. F in (1.1), the jth marginal d. f. ,
denoted by Ffj J
j(=l, ••• ,p) , f
,
is syrronetric about the origin
,
so that for every
= O. Concerning the regressors c.
_1
R 1jJj(x)dF[jJ(X)
•
e
•
)
•
we assume that there exists a positive definite (p.d.) and finite
matrix
~o
n
•
' such that for every (fixed) t
-1 !nt]
,
c . c.
L. 1
-1~1
1=
= II -1 C- I nt ]-+
£
(0,1), as n
tC
+
00,
(2.2)
-0
We also assume that
max
< . < (n -In
L: . 1
I ~_q
1=
Ic.. Ir
l)
(2. :;)
) = 0 (1 ) , for r=3,4.
Note that (2.2) and (2.3) ensure that
lmax
'
<1<n
max
.
1~J~q
Ic. . I
n -~
1)
0 , as n -+
-+
A,
(2.4 )
00.
A
Based on the terminal M-estirnator §n = (§nl' ••• '§np) of §', we
consider the residuals
A
X .
_nl
=
X. - 8 c.
-l
i= I, ... ,n.
-n-1
(2.5)
We define the residual CUSUM M-scores hy
( 2.6)
Mk
_n
Also, we define a pxp matrix S
...n
S .. ,
=n
Finally, let
O
M
~nk
n))
-1 n
L:. lIP.
1=
)
A
= ((
S .. ,)) by letting
n))
A
eXnl)
.. )\j!.,ex
.. ,) ,
)
nl)
j,j'=l, ••• ,p.
(2.7)
be the pq-vector obtained from rolling out M k '
-n
O
for k=l, ••• ,n. Conventionally, we let M k = o for k=D. Note that for
'" O
_n
k=n, M = 0 , by the definition in (2.1). Consider then the set of
-nn
A
-
partial scores statistics
(2.8)
The test for the change-point model is then based on the partial
sequence in (2.8). lf,for some k : l~k~n,
value
J: nk
exceeds a critical
, we reject the null hypothesis of constancy of the regrna
ession relationships over time. Otherwise, we accept the null hypoth£
esis. TILe problem is therefore to determine the critical level £ ,
na
so that the level of significance of this test is equal to the pre-
.e
assigned value a :0 < a <1 •
°=
Let W.
1
{W.0 (t), t
1
£
(0,1) }
i=l, ••• ,m be m independent copies of
a standard Brownian bridge, and let BO = {Bo(t), t £ (O,l)} be defined
by letting (Bo(t))2 = L~ 1(~(t))2, t : (O,l~. Thus, BO is an m-dimenm
1=
1
(
m
sional tied-down Bessel Process. Let then A m)be defined by
a
SUP
p { O<t<l
0
A(m) }
>
Bro(t)
et
(2.9)
ex.
=
Then, we shall show that under the assumed regularity conditions,
t
~
net
(2.10)
"
A~m), we may refer to Kiefer(1959) for m <
.'
A(m)
as n
et'
For the critical levels
~
00.
5. For higher values of m, a recent programme worked out in DeLong
(1981) may be used with advantage.
To establish (2.10), we need to prove the stochastic convergence
of 5
to P, where P
and
,
E1J;,(X .. -S,c.)1J;.,(X .. , ,-n
JJ
JJ
J 1J ~J_1 J
1J
B.lc.) , for j,j';:l, ••• ,p, and also an invariance principle for the
-J -1
O
aligned t>1-statistics { M ; k < n }. First. we note that by virtue
-n k
of Theorem 3.1 of JureCkovaand Sen (1984) , as extended to the mul;:((a~.,))
a~,,;:
tivariate case in a coordinatewise manner,
we have under the assumed
regularity conditions,
1
n~(
where
r
"
1
B - 6 ) ;: n-~
-n
;: Diag. (Y l , ••• ,Yp ) with
00
y. ;: J
J
_00
for j;:l, ••• ,p.
I
, as n
+k.
1J;.(x)dF ,](x);: J kJ
r
J
J
-.
J
~
00
'
1J;.(x)dF .](x),
IJ
J
Let us define SO ;: ((So,
_n
'I
nJJ
))
,
(2.11 )
(2.12)
by letting
S° .. ,;: n-lL~ 11J;.(X .. -S~c.)1J;.,(x", -S~,c.), j,j';:1, ••• ,p.(2.l3)
nJJ
1;:
J
1J -J-1
J
1J
Also, we make use of the fact that the
-J-1
1J;. are all bounded and they
J
satisfy the Lipschitz condition that for every x, y £ E and j;:l, ••• ,p,
I~·(x) - 1J;·(y)1 < Klx-y! , where K« 00) does not depend on x,y,j.Then
J
J
by (2.5), (2.7), (2.11) and (2.13), we conclude that
o
_k
as n ~ 00
(2.14 )
-n
-p
On the other hand, under H ' the -1
X.- SIC.
are i.i.d.r.v., so that by
__1
O
the Kintchine strong law of large numbers, we have
5
(n 2) ,
E* ,
a.s., as n
~
00
(2.15)
•
This establishes the stochastic convergence of S to E*. Next, we
-n
we consider the following asymptotic linearity result on M-statistics
and this extends Lemma 3.1 of Jureckova and Sen (1984) to the multivariate case in the setup of an invariance principle.
•
Let K* = 10,lJ
.
[0,1]'
!
l
KJq (C Eq + ), and for every n( .:. 1), 5
x {-K,
I-K.K]q and j (=1 •••• ,p), define (a q-vector)
E:
N .(s,t)
-nJ
-
=
Ll'~{ns]
c. [1jJ.J (Y IJ
.. + n- \- 'c.)
- 1jJ.J (Y IJ
.. )
-1
_1
k
- n- 2 t ,C.y. ]
..
-
-1
(2.16)
J
S~c. and the y. are deined by (2.12). Then, for
IJ
-J-1
J
each j, {N . (s, t), (s.t) E: K*} is a q-variate process defined on the
-nJ
space Oq[K*].
where Y..
1J
= X.. -
Lemma 2.1. Under the assumed regularity conditions, as n
-+-
00
sup
liN. (s,t) II
= a (1)
(s,!) EK*
-nJ
p
[Note that here we assume that H holds.]
max
(2.17)
1~2P
O
The proof follows precisely on the same lines as in the proof of
Lemma 3.1 of Jureckovi and Sen(1984); the only extra step needed here
.2 s2 and~,!
is the reconstruction ( for 51
N .(sl,t) - N .(s2,t) - N .(sl'u)
-nJ
-nJ
-nJ
-
E
[-K,K]q )
N .(s2'u)
nJ
-
+
1
= L[
nS 2
]<' [
1~
nS l
1
] c.[1jJ.(Y .. +n-"2t ,c.) -1jJ.(Y .. +n-~'c.)
J
-1
IJ
1
-1
J IJ
-"2
'
- n (t-u) c.y. ] ;
-".,.
_1 J
-
-1
(2.18)
the fourth central moment of the right hand side of (2.18) is bounded
by
K(Sl-S2)211!-~114 , for some K«
(0), uniformly in 51'5 ,
2
!
and u ,
so that the rest of the proof follows as in Juretkova and Sen (1984).
Now, by virtue of (2.11) and Lemma 2.1, we may conclude,following
some routine steps, that under H and the assumed regularity conditions,
O
o;~~n
IIBnk - [
~(~)
-
~n(~)~~l~k
]
II
=
Opel)
(2.19)
On the other hand, the ~(~) involve independent matrix-valued su~~ands,
so that the classical invariance principle holds under H and the given
*
regularity conditions. As such, if we define W
-n
*
~n (t)
*
_~ AO
= (E 00 ~n)
~n[nt]
, t
E:
O
=
{W_n* (t), t
[0,1],
E:
[O,l]} by
(2.20)
.
*
0
0'
and If W = ( WI , •••• W ) be a vector of pq independent copies of a
pq
standard Brownian bridge on [0,1], then, we conclude from the above
that under H and the assumed regularity conditions, as n
O
-+-
00,
*
w
-n
W*
7St
....
(2.21)
in the Jl-topology on oPqrO,l].
E
Since ~n converges ~n probability to * ,by virtue of (2.8) and
(2.21), we conclude that under H and the assumed regularity condiO
tions, as n + 00
~
max ('
O<k<n .l...nk
Bo2(t)
pq
sup
O<t<1
oU
(2.22)
..
and this ensures that (2.10) holds.
The weak convergence in (2.21) may also be incorporated along
with 'contiguity of probability measures' in the study of the asymptotic power properties of the test for local (contiguous) alternative hypotheses. Towards this, we assume that in (1.3), we have a
sequence {men)} of positive integers, such that as n
n
-1
men)
+
(2.23)
n : 0 < n <1 ,
and there exists a non-null pxq matrix
6
-
S
...m(n) We denote by
+ ~ ,
-m(n)+l
, such that in (1.3)
0
...l<.
-
(2.24 )
n 20 •
{K} , the sequence of alternative hypotheses for
n
which (2.23) and (2.24) hold.
Also, we assume that for each j( =1,
••• ,p), the d.f. F[j] admits an absolutely continuous probability
density function (p.d.f.) f[j] with a finite Fisher information
I(f[j])' This along with the assumed regularity conditions insure
the contiguity of the probability measure WldedK } to that under
n
the null hypothesis. As such the stochastic convergence of S to
-n
L* under HO extends to that under {K} as well. Further, (2.21)
n
also extends to that under {K } , where we need to adjust w* by a
n
-
suitable drift function. We denote by 00 the rolled out pq-vector
form of 8 , and define T=
- lL*f
-- and let
-
-
~o(t)
W
O
-t (1- n)
{
= -1T(l-t)
cT
-
= {wo(t), 0 <
~~ ~
t
O
=
-~
0
0 , O<t< n ,
<&>~o)-~OO,1T;;: 1;
-0
2
-
(2. 2S)
--
~ I} . Then, we have under {K } ,
n
W* + W
Thus, if we define
w~(t)
rr ~ c )
~
in the Jl-topology on OPq[O,l].
=
{w*(t), O<t<l}
[~o(t)]' [~o(t)]
O;~
(2.26)
by letting
< 1
(2.27)
I
then under {K } and the assumed regularity conditions, we have
max
n
r
(2.28)
O<k<n "'-nk
•
Thus, the asymptotic power function of the test is given by
B0 2(t) + w*(t) > A(pq) , for some t E [0,1] }.
(2.29)
pq
0
a
We shall make some further comments on this test in Section 3 •
p{
3.
RECURSIVE M-TESTS
Note that the use of the terminal estimator
"'-
B
-n
to define the
residuals in (2.5) has rendered a non-recursive character of the
test based on the !:nk in (2.8), though the !:nk are computed at
the successive observations. To construct some recursive tests, one
needs to use some recursive estimates of the regression parameters
and to incorporate them ill the definition of so called recursive
residuals. Also, one needs to estimate
~*
recursively. With these
in mind, we proceed as follows.
Based on ~l' ••• '~k ' the M-statistics ~jt(~j)' j=l, ••• ,p,£ =
l, ••• ,q are defined as in (2.1), for k=l, ••• ,n. With the same set
'"
'"
'"
of regularity conditions as in Section 2, let ~~ = (Bkl, ••• ,Skp) be
the M-estimator of B , based on the ~j£ ' for k=l, ••• ,n. At the
kth stage, we nmy consider then the residuals
'"
x....
=
~kl
Bk - IC'
_1
X. -
_1
-
(3.1)
, i=l, ••• ,k (k=l, ••• ,n).
A
For k _< q, conventionally, we may let _X 1. = _0, i=l, ••• ,k. We let
k
t
x... . = (X , l' ••• 'X , ) and define the recursive M-statistics by
~K1
kl,p
k1,
'"
~*
A
A*
A*'
M._ = (M_1, ••• ,M. )
:"K
-1<.
'Kp
Conventionally, we let
A*
~
A*
k
A
.
, M.,
= L 1~'(X"
,), I <;r:y.
-KJ
1= J IJ,J
-
=
~
(3.2)
, for k=O, ••• ,q. Also, analogous to
(2.7), we define _Sk ' replacing n and the -nl
X . by k and ~Kl
x... , , respectively, for k=l, ••• ,n. Consider then the partial scores statistics
= n
-1
A* t--l A*
{(~) ~k (~k)} , k
= q+l, •••• ,n,
(3.3)
£, (for some £> 0) and £ I
, otherwise;
where ~k = ~k i f ch p (Sk»
- -p
ch (A) stands for the smallest characteristic root of A. Unlike in
p -
-
(2.8), here the estimate
singular for k
~
~k
is used at the kth stage, and
~k
may be
q. While, we may use a generalized inverse in (3.3)
to eliminate this problem, there is little loss of generality in
letting, conventionally, ~:k
we have used for the ~o
~Kn
=
matrices of order pxq , whereas in (3.3),
we have used p-vectors for the
A*
~\
• This is mainly in the spirit of
using robust recursive residuals, and the use of these lower dimensional vectors may enhance the power of the procedure. We may also
note that since the observations X. are taken sequentially at the
-1
*
ordered time points {t i } and the recursive statistics1:
are based
nk
only on tIle set ~l' ... '!k ' k ~ 1, so that a test for the null hypothesis H can be based in a quasi-sequential manner. Operationally,
O
the test procedure consists in computing the ~:k at the successive
K),
S::K
time-points: If, for some k( =
for the first time,
exceeds
some critical level t * , one stops at that time point t along
na
K
with the rejection of HOi if no such K(~ n) exists, one proceeds on
to the last stage and accepts H • The basic problem is to determine
o
the critical level,such that
p{
~:k
>
•
0 , for k ~ q. Further, in (2.8),
t~a for some k: k ~ n I
H }
O
=a
,
(3.4)
where a is the desired level of significance of this quasi-sequential
procedure. We shall provide suitable asymptotic approximations for
* under the same regularity conditions as in Section 2. In
the t na
this quasi-sequential procedure, one allows an early stopping based
on the cumulative evidence at the successive stages. The stopping
number K is a positive integer valued random variable and
K< n
with probability one.
To study the asymptotic properties of the proposed M-test (and
to determine t *
na in (3.4)), we need to study first some invariance
principles relating to the recursive estimators ~* and ~k' for
k = l, ••• ,n. We present these basic results in the next section.
These results are then incorporated in Section 5 for the study of the
general properties of the proposed tests.
•
4. INVARIANCE PRINCIPLES FOR RECURSIVE M-STATISTICS
To fix notations, we define for every
•
IjJ(X.-Bc.)
-
-1
-~1
,
=
=
,
(~l'
,
•••
,
EPq
'~p) £
(1lJ (X.]-b c.), .... 1lJ lX. -b c.))
l 1 ~ l _1
P Ip _p_l
for i=l, ••• ,n. Note that by definition, the
•
~
1lJ(X.-Bc.)
_ _1 __ 1
,
(4.1)
are bounded
random vectors for every i and B. Hence, if we choose any sequence
{k } of positive integers, such that
n
-+ Q;)
-+ 0 as n-+ co ,
k
but n -~k
n
n
then, we have
max
(4.3)
) II -+ 0 , in probabi Ii ty,
_
II n -~ Li<k 1lJ_ex..
k<k
-n
where the X.. are defined by (3.1). We also define the Sk'" as in
_11
JJ
(2.7) , but for a sample of size k, for k=l, ••• ,n. Note that
A
~1
~
I I~kl I
(4.4)
is bounded with probability 1 •
Consequently, by (3.2), (3.3),(4.2), (4.3) and (4.4), we have
max!*
-+ 0 , in probability, as n -+ 00 ;
(4.5)
k<k
nk
-n
in the sequel, we choose k = [nn J where n = 625/i296 « ~).
n
Next, we note that under H ' { ~(§); k ~ O} forms a ( pxq
O
matrix valued) martingale sequence (actually with independent zero
mean increments ), where the scores are all bounded (ensuring the
existence of moments of all finite orders ). Hence, we may use the
law of iterated logarithm and conclude that under H and the assumO
ed regularity conditions, as n -+ 0 0 ,
k~~<n
{(k loglog
n--
k)-~
o
P
(1) •
(4.6 )
Further, we define n = n
and let
o
n 1 = [n 5/6 J , n -- In 25/36J , n -- 1n 125/216 J a~ d n -- kn. (4.7)
2
3
4
Defining the N .(s,t) as in (2.16) and letting N .(s,t) be equal to
-1/12
-nJ_~ 1/12
~
-nJn
(loglog n) 2 N . (5, n
(1oglog n) ~), for s,t £ K*, we obtain
~nJ
~-
as in Lemma 2.1 that under H and the assumed regularity conditions,
O
n-l:~s<l
II
N*.(s,t)11
= a (1),
-nJ
p
fot every j( =1, ••• ,p). In a similar manner, we have
1--
tSUPK*
£
(4.8)
max
1<'<
max
sup
t EK*
_J.:J> n r+l In r-<s<l
_
II
II
N. (s,t)
-nrJ
-
=
(4.9)
aP (1),
1/12
for every r (=0,1,2,3). Combining these and noting that n
<
r
1 10
k /
for every k : n
< k < n , r=O,1,2,3, we obtain that unr+1 - r
der H and the assumed regularity conditions, as n ~ 00,
O
max
max
sup
.
k -1/10 liN (1 t) II
l~j:p k~k~! £(10g10g k)~I-K,K]q
-kj '-
=
a
((10g10g n)
1.<
2 ).
(4.10)
P
By virtue of (2.16), (3.6) and (4.10), we obtain that under H and
o
the assumed regularity conditions, as n
max
k <k<n
n- -
1
kg/lOll C6-k -8)
- r- M (8)C- l
_ -k __ k
~
00
= aP ((1og10g n)~). (4.11)
II
Let Ek denote the conditional expectation given ~i' i~ k, for
k >1, Recall that the ~. are all bounded and satisfy the Lipschitz
-
J
A
condition. Then, on letting _1
Z. =
we note that the
::: ElF1- lcll:·11
-I
2.
)]
Ilz.1I
-1
+
martingale propert)'
A
~(X
.. )-~(X.
_ -11
_ -1 -Be.
--1 )-E.1- l~(X")
_ -11 ,i>
-
are bounded r.v"
[. 1Z, ::: 0 and
1-
_1
q,
2
Fllz·11
_1
incrcasl's. Consequently, using the
0 as i
of the partial sums L:'<k
Z.,
1
_1
k?. 1 , it f011-
ows by standard steps that under H and the-assumed conditions,
O
max {n -~
k<n
II
Li<k ~i
II
}
0, in probability, as n~. (4,12)
~
Also, note that
A
J ~(~ -(~i-1
E.1- IV;_ (X
.. ) =
-11
- ~):i)dF(~)
A
= JI~(x- (B.
- B)c.) - ~(x)] dF(x)
- - -1-1 - -1
"
- -ruS.
1 - 8)c: + o(
(~i-l -~):ill ),
_1- -1.
II
(4,13)
so that
max
k <k<n
n--
II
n
-~
~
""'i<k {Ei-1~(~ii)
J.
2
n-- I.
A
A
~
-
k
< n {o (
+
II (~i -1
I(~i-l
-
-
~):i}
II
~)~ i II)}
(4,14)
(1) ,
::: 0
P
where the last step follows from (4.11) along with the fact that by
virtue of the martingale invariance principle under the d -norm [c.f.
q
Theorem 2.4.8 of Sen (1981 )], for every n >0,
•
= 0 (1) •
(4 • 15)
p
•
,
Therefore, from (4.12) and (4.14), we obtain that under H and the
O
assumed regularity conditions, as n + 00 ,
max
k <k<n
I,
A
Iln-~2 Li<k {~(~ii)- ~(~i-~~j)
n--
A
+
~(~i-l-~)~i}11
~ O. (4.16)
Finally, recall that by (4.11), simultaneously for all k:k <k<n
-~
n
n- -
~
Li<k{ ~(~i-~~i) - £(§i-l -§)~i }
1-
-~~
{
-1}
-2/5
'"'i<k ~(~i-§~i) - ~i-l (§)~i~l~i
+Op(n
loglogn)
-~ i - I ' -1
-2/5
= n L1'<k I 1J!(X 1·-Sc.) -E·_l1J!(Xo-Sco)coc. lCo ] +0 (n
log1ogn)
_
- - --1
J- - -J --J ... J-l- _1
P
(
-2/5
= -~
(4.17)
n Li<k
Lj<i a ij ~(~j- ~~j) ) +Op(n
loglogn),
=
n
-
,-1-
where a IJ
.. = -coCo
.. = 1 and, conventionally, we let
-J_1- Ie.
_1 for j<i, a 11
a.o
o for j >i Note that
1J
L.Jo~l· a
2
1J = 1
0
+
•
' - 1 ' -1
L.J <1- 1 -1-1co C. 1 -J-J
co c ...COl
c.
l- -1
0
--1
= 1 + c!C.
-1-1-
0
,
-1
l( L . . 1 c.c. )C. lC'
J<I-
'-1
::: 1 + c.C. lC'
::: 1
+
= 1 +
-J-J
-1--1
-
...1-1- -1
-1
'
Tr.eC.
1e.c.)
_1- -1 ...1
-1
Tr. (C.
1C - I )
... 1- -1
-q
0
= 1 + 0 (i -1), by (2.2) ;
(4.18)
= 0 , for every i 'f i' = 1, ••• ,n.
t
J IJ 1 J
Thus, if we define W ={ W (t), t £ rO,l]} by letting
...n
...n
L. a. ao
0
W (t)
-n
= n
0
-~ *-~
f
A
L:i~[nt]
IjJ (X
......11
0
•
)
,
t
£
[0,1]'
(4.19)
(4 .20)
then, by an appeal to (4.16), (4.17), (4.18), (4.19) and Theorem 1
of Sen (1982a), it follows that under H and the assumed regularity
O
conditions, as n + 00 ,
W
...n
where W =
~
{
W
~(t), t
in the Skorokhod Jl-topology on op [0,1], (4.21)
£ [O,l]}
consists of p independent copies of a
standard Brownian motion on (0,1]. Also, using (2.14), (2.15) along
with (4.11), it follows that under H and the assumed regularity
O
conditions, as n
~
II
k max
<k<n
CXl
,
S
-k-
n--
II
L_*
~
0, in probability.
(4.22)
•
The invariance principles in (4.21) and (4.22) provide the basic key
to the study of the main results pertaining to the recursive M-statistics. Defining
~
as in after (4.21), we consider a p-parameter
Bessel process
B = {B (t), t E {O,l] }, where
p
p
2
B (t) = {W(t)]' {Wet)] , t E {O,l] •
P
-
(4.23)
-
Then, from (3.3), (4.5), (4.20), (4.21) and (4.22), we conclude that
under H and the assumed regularity conditions, as n
O
max
*
--4sup B2 (t)
k < n ~nk
~
O<t<l p
~
CXl
,
r-
(4.24 )
We shall find this result very useful in the next section.
As in Section 2, we may consider some local (contiguous) alternatives {viz., (2.23)-(2.24)] and extend these invariance principles
to such cases too. In this context, we assume that as k increases,
-c
- k.
~= k -l~k
t... 1 c.
c
1=-1
II
where II~
<
(4.25 )
CXl
First, we note that by virtue of the contiguity of probability measures under {Knl
to that under HO ' the stochastic equivalences in
(4.16), (4.17) and (4.22), holding under H ' remain in tact under
O
{K 1 as well. Secondly,the tightness of {W } (under H ),ensured by
O
n
n
(4.21), also holds under {K } as well. Hence, to study the weak conn
vergence of {W } under {K }, all we need to study is the convergence
n
n
of the finite dimensional distributions of W to those of some pn
dimensional Wiener process with an apprpriate drift function. In
view of (4.17) being validated under {K } , the asymptotic multin
normality result will follow readily through the use of some standard central limit theorems on the
~(X.-Bc.)
-
-1 --:1
, and hence, we need
to study the nature of the drift function only. Towards this, recall
S (n ) = _Sand _m
B (n ) + 1 = _S
_m
EPq ), for every i -< men), ~(X.-Sc.)
_ _ 1 __ 1
that 1 by virtue of (2.23) (where we take
+
n-~ 0- , for some (fixed)
e-
E
has expectation 0 , while for i > m(n),
1
n-~r0ci
1
+
-
~ (X.
-
_1
-Bc.) has expectation
_ .... 1
o(n-~). As such, for every k ~ men), under
K ,
n
,
n -~ L'<kL ,<. a. ,~(X.- Sc.) has
1._ )_1. 1.)- ... ) - ...)
k > men) = m ,
•
e~pectation
equal to 0 , while, for
E{n -~ L,
<k L '<' a,. ~ (X. - Sc.)
K }
1._ J_1 1.)- -J
--J
n
IL
L
+ 0(1)
= n- m<'<k
1. . rn<'<;'
J 1. a.,fec.
1.J---J
- -1
, -1
}
= fe { (k/n)c... k n Lm<'<kI
1 m<J,<·I-c.C.
1 -J-1- lC']c.
_1 -J
+
0(1)
= fe
+
0(1)
{(k/n)~k n-lLm<i~k [~i-l
--I
{ (k/n)~k
- n Lm<i<k
! -
-
~rn]~~~l~i
-I}
~m~i-l]:i
- - I
.
-1
= ~~
= £~ { (m/n):m + (m/n)Lm<i<k (rn
}
+ ~(l)
i -
~m)((1-l)~i_l)Ii_l~i
= fe { (rn/n)Lm<i<k [m
-1
+ 0 (1)
-1
-l-}'"
~m(i-l)~i_l](i-l)
:i-l + ~(l)
= fe {(m/n)Lm<i<k I
!
+
-
- ~i-l]
}
~ck)JI~ + ~(l)]/(i-l)} +~(l)
= £~ {(rn/n)~ log(kj'rn)} + 0(1), by (2.2) and (4.25).
(4.26)
Hence, if we define w={w(t), O<t<l} by letting
0 ,
wet)
- -
--
O<t<1T ,
~(E*)~~~~ log(t/1T)
= {
then, we conclude that under
tions, as n
~n
£"
{K }
n
, 1T <t
~ I,
(4.27)
and the assumed regularity condi-
+00
~ + ~, in the Jl-topology on DPrO,I] •
(4.28)
Since (4.22) holds under {K } as well, the weak convergence result
n
in (4.28) also provides an extension of (4.24) to that under {K }.
Towards this, we define
w*(t)
=
-
w*
,
= {w*(t),
-
n
O~t~l}
by letting
Iw(t)] {wet)] , 0 < t < 1.
--
Then, under {K } and the assumed regularity conditions, as n
n
max r*
k<n clnk
sup {B 2 (t) + w* (t)}
O<t<lp
•
(4.29)
+
00
,
(4.30 )
The asymptotic properties of the proposed tests may now be studied
with the aid of (4.24) and (4.30).
5.
ASYMPTOTIC PROPERTIES OF RECURSIVE M-TESTS
For the Bessel process B , defined by (4.23), we define the
p
critical level
pf
~up
A
p,a.
B
by
>
(t)
•
A
}
O<t<l
P
p,a.
For specific values of p and a.
(5.1 )
a. ( 0 < a. <1) •
=
these critical levels have been
computed by DeLong(l980). Let us now consider the critical value
£*
in (3.4). By virtue of (4.24) and (5.1), we conclude that under
na.
the assumed regularity conditions (of Section 2),
£*
as n
na.
-+-
(5.2)
00
The rapidity of the convergence in (5.2), of course, depends on F,p,
q, the ~i as well as the scores ~l"""~P • Since, we are dealing
here with boundedly continuous scores, the rate of convergence is
expected to be faster than that of the case of maximum likelihood
estimator based procedures (where the score functions are essentially
unbounded in the majority of the cases). Moreover, the M-statistics
considered here are robust against outliers and gross errors, so that
the dependece of (5.2) on the underlying d.f. F is likely to be less
stringent than in the case of the other procedure.
Let us next study the consistency of the proposed tests for any
fixed alternative. Note that under (1.3) , for some m: min
TI
< 1 and S
6
_m+ 1 - _m
m, the
~k
=
-+- TI
:
0 <
6 ( f 0 ), (4.4) remains valid, and for k <
still converge stochastically to
stochastic limit of S_k is given by _L*
+
r*, while for k
I**,
where
_k
> m, the
I** is
_k
. a posi-
tive semidefinite matrix whose elements are all bounded. On the other
hand, for k > m, the ~(~k) would have centering vectors progressing
away from 0 (as k moves away from m), and hence, by (3.3), for k
-
greater than m, the
JC .. k will
n
be stochastically larger. Simple compu-
tations show that for every k: min
1
J: *
kin < 1, nk converges
* n
stochatically to a positive constant, so that
k exceeds the null
*
2
n
critical level £
(converging to a fixed ~
) with probability
na.
p,a.
converging to 1, as n -+- 00. This establishes the consistency of the
proposed test for any fixed alternative.
<
J:
We have noticed in SCL:tion 3 that
the recursive M-tcsts allow
the possibility of an earlier stopping. Define the stopping variable
•
K as in before (3.4). Note that by definition, we have for every k:
l~k~n-l,
K > k}
= p{ ~nr
t*
< £*
na
n } = O. Hence, by
, for every r < k},
(5.3)
while, p{ K >
(4.24) and (5.2), under H and the
O
assumed regularity conditions, for every t E (0,1), as n ~ 00 ,
p{
•
p{ K >
[ntJ
I
~
H}
o
sup B2 (t) <)..,2
}
O<u<t P
- p.a
p{ sup B2 (u) <t -1;x.2
}.
O<u<l p
p,a
p{
=
(5.4)
As a result, we conclude that
Jl P{
-+
a
sup B2 (u)<t- l \2
} dt •
O<u<l p p,a
(5.5)
We may again refer to DeLong(1980) for some numerical studies relating to the right hand side of (5.5) (made in a different context).
In a similar manner, it follows that for any fixed change in the
regression parameter (i.e., for a given
for every
p{
n>
0, as n
K >In(
TI +
-+
TI
:0 <
TI
< 1
and
~
~
a ),
00
n )JI
~
# 0 ,0 <
TI
<
I}
-+
(5.6)
1 ,
so that with a probability converging to 1, the procedure terminates
soon after a change-point occurs. Notice that there is always a probability of stopping earlier even if no change in the
B. occur; of
_1
course, such a probability is usually quite small.
Let us next consider the case of local (contiguous) alternative
hypotheses, treated in Sections 2 and 4. Here, by virtue of (4.30),
we conclude that under
{K}
n
and the assumed regularity conditions,
the asymptotic power of the recursive M-tests is given by
p{ B2 (t) + w*(t) >;..,2
, for some t £ [O,lJ } ,
P
·e
where
pp.
(5.7)
w* is defined by (4.29). Here also, the results of DeLong
(1980) may be used to provide some numerical results. There is ,
however, a difference in the nature of the two drift functions in
the two cases, and bence, the results may not be strictly comparable.
Parallel to (5.4) and (5.5), we have here'
I
p{ K> int]
E( n-lK
I
K }
n
K)
n .
-+-
-+-
2
P{,,~uP<t
(B (u)+w*(u))<;.,2
},
v--U
p
- p,a.
Jlo P{OSUP-(B 2 (U)+W*(U)) < \2
} dt.
<u<t p
- p,a.
(5.8)
(5.9)
Now, looking at (4.27) and (4.29), we gather that for TI < t
w*(t)
and
w*(t)
2
=0
depend on the
-I
TI (c 8'f(E*)
=
-1
-
f8c) ( log(t/TI))
2
,
~
1,
•
(5.10)
< TI. Whereas TI , C , 8 and log(t/TI) do not
- particular set of score functions (i.e.,
), the matrix
for t
~
~ (~*)-l~
depends on the choice of ~ and on the under;ying d.f. F.
The same matrix appears in the non-centrality parameter of the usual
M-test statistic for the classical linear model , and hence, the
usual asymptotically optimal choice of
~
for such models
remain in
tact in the current context too.
6. SOME GENERAL REMARKS
We may observe that the right hand side of (5.4) is bounded from
below by 1 - a. , uniformly in t
E
(0,1], and as t becomes small, this
converges to 1. Thus, the right hand side of (5.5) is bounded from
below by 1 - a. , and, in reality, it is quite close to 1. This shows
that in the null case, there is not much chance of an early termination. On the other hand, in (5.8) and (5.9), for t > TI, the drift
function w*(t) is positive and it reduces the corresponding probabilities. Hence, the expected stopping time becomes smaller. The study
of the forms of the right hand sides of (5.8) and (5.9) demands the
distribution theory of the first passage time for a Bessel process
with a segmented square log-function, as in (5.10). Such results are
not that precisely known. However, numerical studies (as in DeLong
(1980,1981)) or simulation studies can be made in specific cases to
gather more ideas. Whereas the recursive M-tests enjoy this scope
for an early termination, the non-recursive ones in Section 2 do not.
One may, however, attempt to compare (2.29) and (5.7) for the study
of the asymptotic relative efficiency (A.R.E.) of the recursive tests
with respect to the other one (keeping aside the issue of early stopping). There are some technical problems in this context. First, in
,
(2.29). we have a pq-parameter Bessel process. while in (5.7), we
have a p-parameter Bessel process, where q
~
1
even, for q= 1,
these two processes are not the same. In (2.29), we have a
•
tied-do"~
process vanisfung at both the ends, while in (5.7), the Bessel process vanishes at the lower extremity only. Hence, comparing solely
the two drift functions w* and w* may not provide any real picture
o
of their A.R.E. Secondly. w (t) in (2.25) is segmented linear in t.
o
while
wet) in (4.27) is segmented logarithmic in t. Even if we
ex~ress
(2.29) in terms of a (drifted) Bessel process (on the entire line
10,00) ). the range of that process will be different from that in
(5.07) and the two drift functions will also be of different forms.
As such. we are not in a position to study the Pitman A.R.E. of the
non-recursive tests with respect to the recursive ones (based on the
same set of scores). Intuitively, however, for q > 1, the recursive
procedures may perform better than the non-recursive ones for a broad
class of situations. Numerical studies in specific cases are suggested to cast
~re
light on the relative performances of these proce-
dures. We conclude this section with the remark that the recursive
procedures in Section 3 may easily be adapted in the context of the
sequential detection problem. Actually, (4.11), (4.15), (4.16) and
(4.17) all extend naturally to a.s. results (as n
~
00), and hence,
in a sequential setup, one may use (4.17) and a.s. invariance principles for the triangular schemer E. k E. . a .. ~(X.-Sc.) , k > k }
1<
J<l 1J- -J --J
- n
and construct sequential procedures oased-on the recursive M-statistics. These invariance principles provide asymptotic expressions for
the probability of a false alarm as well as for the quickest detection when there is a genuine change-point.
ACKNOWLEDGEMENTS
This work was partially supported by the National Heart, Lung
-e
and Blood Institute. Contract NIH-NHLBI-7l-2243-L from the National
Institutes of Health.
BIBLIOGRAPHY
Bhattacharya, P.K. and Frierson, D. (1981). A nonparametric control
chart for detecting small disorders. Ann. Statist. 9# 544-554.
R.L., Durbin, J. and Evans, J.M. (1975). Techniques for testing constancy of regression relationships over time (with disscussions). J. Roy. Statist. Soc. Sere B 37# 149-192.
Bro~l,
DeLong, D. (1980). Some asymptotic properties of a progressively censored nonparametric test for mUltiple regression. J. Multivar.
Anal. lO# 363-370.
DeLong, D. (1981). Crossing probabi Ii ties for a square root boundary
by a Bessel process. Commun. Statist.# Theor. Meth. A lO# 2197
- 2213.
Jureckova, J. and Sen, P.K. (1984). On adaptive scale-equivariant
M-estimators in linear ~de1s. Statist. Dec, 2# in press.
Kiefer, J. (1959). K-samp1e analogue of the ~olmogorov-Smirnov and
Cramer-von Mises tests. Ann. Math. Statist. 30# 420-447.
Page, E.S. (1954). Continuous inspection schemes. Biometrika 4l#
100-1l5.
Sen, P.K. (1981). Sequential Nonparametrics: Invariance Principles
and Statistical Inference. New York: John Wiley.
Sen, P.K. (1982a). Invariance principles for recursive residuals.
Ann. Statist. lO# 307-312.
Sen, P.K. (1983a). Some recursive residual rank tests for changepoints. In Recent Advances in Statistics: Papers in Honor of
HeT'man CheY'noff's Sixtieth Birthday (eds: M.H. Rizvi, J.S.
Rustagi and D.Siegmund). New York: Academic Press, pp.371-391.
Sen, P.K. (1983b). Recursive M-tests for the change-points. Bull.
InteY'nat. Statist. Inst. (Madrid Confer.), pp. 206-209.
Shirayayev, A.N. (1963). On optimum methods in quickest detection
problems. Theor. Probability Appl. 8# 22-46.
Shirayayev, A.N. (1978). Optimum Stopping Rules. New York: SpringerVerlag.
•