•
INVARIANCE PRINCIPLES FOR RECURSIVE RESIDUALS
by
Pranab Kumar Sen
Department of Biostatistics
University of North Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 1304
September 1980
INVARIANCE PRINCIPLES FOR RECURSIVE RESIDUALS*
•
by
PRANAB KUMAR SEN
University of North Carolina, Chapel Hill
ABSTRACT
A general class of recursive residuals is defined by means of
lower-triangular (orthonormal) transformations and, for these residuals,
some (weak) invariance principles are established under appropriate
regularity conditions.
The theory is then incorporated in the study
of the robustness of some CUSlTh1 procedures.
k~S
1979 Classification No:
Key Words & Phrases:
60F17, 62FOS.
Constancy of regression relationships over time;
CUSlTh1 tests; invariance principles; orthonormal transformations;
recursive residuals; tightness; Wiener process.
•
*Work partially supported by the National Heart, Lung and Blood
Institute, Contract No. NIH-NHLBI-7l-2243 from the National Institutes
of Health.
-2-
1.
INfRODUCTION
In the context of testing for constancy of regression relationships
over time, Brown, Durbin and Evans (1975) have considered some CUSUM
f
tests based on suitably defined recursive residuals.
When the error
components are assumed to be independent and identically distributed
(i.i.d.) according to a normal distribution, these recursive residuals
are mutually independent and distributed according to a common normal
distribution, so that the Brownian motion approximation can be readily
incorporated for some CUSUM test procedures based on these residuals.
The situation differs considerably when the errors are not normally
distributed:
the residuals remain uncorrelated but not necessarily
independent or normally distributed.
Several discussants of the Brown
et al. (1975) paper, including P. R. Fisk, G. Phillips and R. E. Quandt,
have raised the issues of incorporating more general forms of orthonormal transformations for the definition of the recursive residuals
and establishing weak invariance principles for related
CUSill~
test
statistics without imposing normality on the distribution of the errors.
Naturally, these would cast light on the robustness of the CUSUM procedures suggested by Brown et al. (1975).
The object of the present investigation is to study some weak
invariance principles for recursive residuals generated by a class of
orthonormal transformations (when the errors are not necessarily normally
distributed).
Along with the preliminary notions, the orthonormal trans-
formations are introduced in Section 2.
theorems and their proofs.
Section 3 deals with the main
Section 4 is devoted to the applications of
the main theorems to some CUSUM procedures based on recursive residuals
•
and relates to some studies of their asymptotic properties .
-3-
2.
PRELIMINARY NOTIONS
Consider the regression model
(2.1)
§~~t+et,t=l,
Yt
where, at time
(for some k
~
... ,n,
t, Yt is the dependent variate, ~t = (xtl, ... ,x tk ) I
1) is the vector of regressors, §t = (Stl"",Stk) I
is the vector of regression coefficients and the
components.
For testing the null hypothesis
e
are the error
t
(HO)
that §t = §
(unknown), V t along with the i.i.d. character of the
e , i.e.,
t
the constancy of the regression relationships over time, Brown, Durbin
and Evans (1975) considered the regression residuals
where, for every r (= k, ... ,n),
(2.3)
XI = (xl"'" x ), Y I = (Y , ... , Yr )
~r
~
~r
l
~r
Their CUSUM test is then based on the statistics
(2.4 )
w}
D+ = (n - k)-1/2 s -l{ max
n
n k+l~r~n r
and D = (n - k)-1/2 s -l{ max jW
n
n k+l~n~n r
I},
where
(2.5)
s2n
When the
e
= (n -
t
k)-l(Y
n
- Xb
~n~n
)/(Y - X b ).
~n
~n~n
are i.i.d.r.v. (random variables) with a normal
distribution, the w.1
are also i.i.d.r.v. with the same normal dis-
tribution, so that the Brownian motion approximation for the W can
r
readily be incorporated in the study of the distributon of D~ or Dn
•
-4-
However, when the
et
are normally distributed, the wi
are not
necessarily independent, nor normally distributed, and hence, invariance principles for the recursive residuals remain to be explored.
t
Note that by (2.1) and (2.2), for each r(> k), w
r
as a linear combination of e., i
~
1
can be expressed
r (plus a nonstochastic component
which vanishes under HO)' Keeping this in mind, we conceive of a
sequence {D., i ~ I} of i.i.d.r.v. IS, assume that
1
(2.7)
2
EU?1 <
00
v = (Vm+l""'Vn )' = AU, n
~
EU.1
(2.6)
:=
0, 0 <
C5
~n
where m
IS
~n~n
a nonnegative integer,
with row vectors
=
,
m + 1,
6n is an (n - m)
a~nJ. = (a.,O
.), m + 1
~J
~n-J
~
J'
~
x
n and the
n matrix
a.
~nJ
satisfy
the following (orthonormality) condition:
t
A A' = -n-m
I
(2.8)
~n~n
Note that by (2.8), a.a!
~J-J
=
1,
~
generated by the class of A
~n
~
j
The class of residuals V
~n'
(of lower triangular or trapizoidal
as special cases, for which m = k.
forms) in (2.8), contains the wr
In view of the normality of the
m + 1.
e
in (2.1), Brown et al. (1975) did
t
not require any further condition on the ~t (or the A). However,
~n
in our case, in the absence of this normality assumption, for the study
of invariance principles pertaining to the
Yn ,
we may need some addi-
tional regularity conditions, which are posed below.
We assume that
(2.9)
-5-
Generally, for recursive residuals, the
are close to 1, akj ,
kk
k are all close to O.
0 and akj , j <
In this respect, we may need the following conditions (where
j
>
k are all equal to
a
°rs
stands for the Kronecker delta):
I)} ~ cl
<
n
sup{ max iII k=( .) Z(a..· - ak " 1) I}:::;
n>m l:::;i<n-l
mVl + Kl
1+
C
(Z.lO)
sup{ max max (k1akJ" - 0kJ"
n>m m<k:5;n l:::;j:::;k
(Z .11)
•
00,
z < 00,
where c l and C z are finite positive numbers. It may be remarked
that (Z.lO) implies (Z.9), but, not conversely. The main results are
then presented in the next section.
3•
THE MAIN THEOREMS
For every n(> m), we introduce a stochastic process
by letting V. = 0, i : :; m and
1
W
n (t)
(3.1)
= (I"1:::; k
n
(t)v")/(0~),k
1
n (t)
Wn belongs to the D[O,l]
Then,
Jl-topology.
process on
of {W}
n
to W.
[en - m)t], 0 : :; t : :; 1.
space endowed with the
Also, let W = {W(t) , 0 : :;
[0,1].
=m +
t :::;
I}
Skorol~~od
be a standard Wiener
We are primarily interested in the weak convergence
Towards this, we consider the following two theorems
under different sets of regularity conditions.
Theorem 1.
Under (Z.6), (Z.7), (Z.8), (2.9) and
(3.Z)
Wn + W, in the Jl-topoZogy on D[O,l].
V
'
Under (2.7), (2.8), (Z.lO) and (Z.ll), (Z.6)
Theorem 2.
~nsures
(3.Z).
r
-6-
[Note that in Theorem 1, under the more stringent condition that
v < 00, we are able to eliminate (2.10) and (2.11), while, in Theorem 2,
4
(2.10) and (2.11) eliminate the need for v < 00 and the finiteness
4
2
of a in (2.6) suffices.]
To establish (3.2), we need to show that (i) the
Proof of Theorem 1.
finite-dimensional distributions (f.d.d.) of {W}
converge to those
n
of W and (ii) {W}
n
o~
tl
< ••• <
k (to) = k., 1
n
J
J
~ 1
tr
~
is tight.
j
~
and
~
1),
= (Al, ... ,Ar)'(f Q), on letting
r, we have
I~ lAoW (t.) = (av"i1--=rri)
J= J n J
(3.3)
r(~
For (i), for arbitrary
-l,r
,
L' lA L k V
J= J l~ j 1
0
0
0
r lAo I .a }Uo
~
= (avn
- m) -lIn 1
1= J= J S=l Sl 1
{I
0
kj
0
0
= I~1= lCnl.(U./a),
say,
1
(where, conventionally,
I~ask = 0, for
o
(2.8), (3.3) has mean 0 and varlance
Now, by (2.6) and
k > q).
,n
2
Li=lcni
+
,r
,r
Lj=lL~=lAjA~(tjAt~)
,r
2
= E(Lj=lAjW(t j )) (> 0).
Hence, it suffices to establish the asymptotic
normality of (3.3). For this, we appeal to a special central limit
..
v
theorem in Hajek and Sid&k (1967, p. 153), and, we require to show
only that
2
lim{ max c .} =
n+oo l~i~n nl
(3.4)
Since
~'~
<
00,
o.
we obtain from (3.3) that for every
i: 1
~
i
~
n,
-7-
(3.5)
~ r(A'A)(,n
.Ia . 1)2/(n - m)
~ ~
L S =1 SI
so that (2.9) insures (3.5).
Hence (i) holds.
To establish the tightness of {W},
note that for every
n
(m
~)kl <
(3.6)
k2
< k3(~
n), by virtue of (2.7)-(2.8), we may write
kj+l
kj+l
.
'.
k
IV.
=
'.
k
la.U
=
b
.U
,
say,
J
= 1, 2,
Ll = .+ 1
Ll =.+ -nl-n
-nJ-n
J
J
where [by (2.8)]
(3.7)
b~nJ-nJ
.b'. = (k.J+ 1 - k.),
J. = 1,2 and b-nlb'2
J
-n =
o.
Note that E(U. U. U. U. ) = 0 whenever at least one of the indices
11 1
2 13 14
iI' i 2 , i , i
occurs with multiplicity 1.
3 4
Hence, (3.8) is equal to
,
8-
(3.9)
\n bl·
Z b Z·
Z
(n - m) -Z (v 4/a 4)L'-l
11
1
4
:0; (v 4/a )(n -
+
\
Z'
3(n - m) -Z{ 1 ':1:'
L
(b1Z1,b Z
J
:0;1 1 J:o;n
+
blibZibljbZj)}
m)~(I~=lbii) (Lj=lb~j)
3(n - m)-Z{ (L~=lbii)
=
{(v 4 /a 4 )
+
+
(Lj=lb~j)}
~nl~~Z
(as
0)
3}(k3 - kZ)(k Z - kl)/(n - m)-Z.
Thus, (3.1), (3.8), (3.9) and Theorem 15.6 of Billingsley (1968) insure the
tightness of {Wn } (where note that Wn (0) = 0, with probability 1). Q.E.D.
Proof of Theorem 2.
Since (Z.10)
vergence of f.d.d. 's of {W}
~(Z.9),
the proof of the con-
to those of W, as has been sketched
n
in (3.3)-(3.5), holds for the hypothesis of Theorem Z.
Thus, it
suffices to show that under (Z.6), (Z.7), (Z.8), (Z.lO) and (Z.ll),
{W}
n
(3.10)
is tight.
Wn(t)
(av'i1-=-m)
Since the a 11
.. D.1
n-1L~=la?
1
11
+
For this, note that by (3.1),
-1\
.. D.1 + (av'i1=Ii1)
1:0; kn (t) a 11
L'
are independent
-1\
\
.. D.J
1:0; kn (t) L'J<1. aJJ
L'
r.v. 's and (Z.lO) insures that
1 and n- l { max a?}
1<'<
-1-n 11
+
0, as
n
+ 00,
under (Z.6) and
(Z.lO), {Wnl } converges weakly to W; see Problem 1 (p. 67) of
Billingsley (1968) in this respect. This, in turn, insures that
IS
tight.
{Wnl }
Thus, it remains to establish the tightness of {WnZ }'
Let Sk = Dl + ... + Dk , k ~ 0 and So = O. Then, by the Hajek-
R~nyi inequality, for every
c
>
0, 0 :0; a
<
l/Z
and q ~ 1,
-9-
r{ ls;k;q
max (k/n) -an -1/ 2 !Sk l
(3.11)
<
c-Z{n-lL~=l (k/n)-za}
>
ccr}
< c-zf:/nt-Zadt
= (1 - 2a ) -1 c -2( q/ n )1-2a .
Also, U.1 = S.1 - S.1- l' i
~
1, so that for every m + 1
<
q s; n,
,i-1
)
(crrn-:TIi) -l(,q
L'
1 . -la .. U.
1=m+ L J- 1J J
(3.12)
= {en -
+
m)/n}-1/2{I~J= l(am+ l'J
I~-l
l(a. l'
J=m+
J+ J
+
,~
.
L 1 =J+
where by (2.10), lia .. I < c '
1
1J
~
+
I~1=m+ 2(a1J
..
- a .. l))(S./crlll)
1J+
J
2(a .. - a .. 1)) (S./crInJ},
1J
1J+
J
~
1 s; j s; i - 1, i
m + 1, while by
(2.11), jjI~1= (.Jvm) +2(a 1J
.. - a 1J+
.. 1) I < c 2 ' ~ i ~ 1, q ~ m + 1.
by (3.11) and (3.12), we have for
q = m + [en - m)8]
+
Hence,
1, 0 < 8 < 1,
,i-1 .. U. I
max (cr/n - m) -l l ,k
L'
1L·-la
1=m+ J- J1 1
(3.13)
m<k S;q
S;
c(C
s; c(c
= c(c
1
1
1
+
c ){(n - m)/n}-1/ 2I f=l j -1(j/n)a
2
+
q/n 1 a
c ){(n - m)/n}-1/2(!
t- + dt)
2
o
+
c ){(n - m)/n}-1/2 a -1(q/n)a,
2
with probability greater than
(3.14)
If we let a = 1/3, then for every
E
> 0
and
n
> 0, there exist a
.
-100: 0 < 0 < 1 and a sample size nO (= nO(s,n)), such that for
q = m + [en - m)o] + 1, the right hand side of (3.13) is bounded
from above by s, while (3.14) is bounded from below by 1 - n, for
every n
~
nO' i.e.,
p{ sup Iw 2(t)
o<t:s;o n
(3.15)
I
>
s} < n, V n
~
nO.
On the other hand, defining k (s) = k and k (t) = q, for
n
n
o :s; s < t :s; 1, we have by (2.10) and (3.10),
(3.16)
:s; (n - m)
-1,q-1
. ,q
2
Li=l (q - kVl)Lj=kvi+1 a ji
.-z}
:s; c12(n - m)-l{k( q - k),q
Lj=k+1 J.-2 + ,q
Li=k+1 (q - 1.),q
Lj=i+1 J
2
2
Z
:s; c1{(q - k) j(n - m) }{(n - m)(q + 2k + 2)j3q(k + I)}
Z -1
2
:s; c10 {(q - k)j(n - m)}
Hence, using Theorem 12.3 of Billingsley (1968, p. 95) along with (3.16),
we obtain that for every s > 0, n > 0
there exist a
(and 0 > 0, defined by (3.15)),
p: 0 < p < 1 and a sample size nO' such that for n
~
nO'
-11-
p{ o~s<t~s+p~l
sup
Iw 2(t)
n
(3.17)
- W2(s) 1 >
n
€} < n·
Then, (3.15) and (3.17) insure the tightness of {Wn2 }.
4.
Q.E.D.
SOME APPLICATIONS TO CUSUM TESTS
We shall discuss the role of Theorems 1 and 2 in the context of
some CUSUM tests considered in Brown et a1. (1975).
the simple location model, where, in (2.1), k
=
Consider first
~t =
1 and
1, V t.
In this case, br = Yr = r-1I~1= 1Y.'
r ~ 1 and
1
r
~
2.
TI1erefore, we have here m = 1 and for every j
(4.2)
~
2:
a ..
JJ
Now, (4.2) insures (2.10), while
a· 1
J
= ... = aJ..1- l' V j
~
2 insure
(2.11) with c 2 = o. Thus, by Theorem 2, (3.2) holds here under (2.6).
Also, s + o,in probability under (2.6) [see Sen and Puri (1970)],
n
so that for the CUSill4 test for a shift in location, under H and (2.6),
O
(4.3)
sup Wet)
O~t~l
and D
n
+
V
sup IW(t)
O~t~l
I.
This explains the robustness of the CUSUM test for nonnorma1ity; finiteness
of the second moment of the e
suffices.
t
Consider next the general regression model in (2.1).
assume that for some A > 1/2,
(4.4)
O(n -A ),
Vn
~
m + 1.
First, we
-12Then, by (2.2) and (2.3), we have for every j
(4.5)
a .. = (x!
(X!
IX. 1)
~J
~J- ~JJl
a..
JJ = {1
(4.6)
-1
x.)/(l
~1
+
+
~
m + 1,
1 1/2
x! (X! IX. 1)- x.)
,1:::; i :::; j - 1,
~J
x!
(X! IX. 1)
~J ~J- ~J-
~J-
~J-
~J
-I }-1/2
x~J.
,
so that by (4.4), (4.5) and (4.6), for every 1:::; i : :; j - 1, j
a 2.. : :; (x!(X!
IX.
1)
~J
~J ~J Jl
(4.7)
:::; (x!
(X!
1)
~J
-J- IX.
~J-
-1
-1
x~J.) (x!
(X! IX.J- 1)
~1 -J-
x.)
~
m + 1,
2
~1
-1
x.)
~1
=
0 (J' -
2A
).
Further, by (2.9) and (4.7),
(4.8)
{ -I (n -
: :; max n
m<k:::;n
n : :; max {
m<k:::;n n
•
max {I
m<k:::;n
=
+
2}
,n
k)L·=ka'k
J
J
k[a2.. + ,nL'=k+1a'k]
2}
JJ
J
0(k- 2A +1)}
J
=
0(1),
as A > 1/2
Hence, (2.9) holds, so that Theorem 1 holds whenever v 4
(4.4) holds.
Next, we proceed to relax the condition that v4
for every j ~ m + 1 and 1:::; i : :; j - 2,
(4.9)
l
a J1
.. - a.·
J1+ 11
Thus, if for every j
=
~
l{x!(X!
IX. l)-l(x.
- x.
~J ~J- ~J~1
-1+ 1)}/{1
m + 1 and 1:::; i : :; j - 2,
<
00
and
Note that
<
00.
+
x!(X!
1)-lx~J.}1/2 1
-J -J- IX.
~J-
-13(4.10)
then (2.11) holds, while (2.10) holds when (4.4) holds with A = 1.
As a result, Theorem 2 holds when (4.4) holds with A = 1 and (4.10)
holds.
Under these extra conditions, we do not need that v 4 < 00.
As a simple example, consider the classical polynomial regression
model where
~~ = (l,t, ... ,tP), for some
(4.11)
p ~ 1, 1 ~ t ~ n.
In this case, (4.4) holds with A = 1 and (4.10) holds.
related CUSUM test, (4.3) holds whenever
0 < 0 <
Thus, for the
00
So far, we have considered the case where H
O holds i.e., the
= ~. + e., where the e.
are
U have all expectation O. If U.1
11
1
t
i.i.d.r.v. with mean 0 and if we define ]1. = L- .~.a .. , i ~ m + 1,
1
then for the v.1 - ]1.,
1
Hence, whenever
function
y
1 ~
~
t
J
1J
m + 1, the same invariance principles hold.
{(0/~n--~m)-1]1~(t)' 0 ~ t ~ I}
= {Yet), 0
J~ 1
~
•
converges to a smooth
I}, then the asymptotic power of the CUSUM
test can be expressed in terms of the boundary crossing probability of
the drifted Wiener process W + y.
In general, y
is not so simple as
to allow an algebraic expression for this probability.
However, the
recent results obtained by De Long (1980) (in a different context)
presents excellent scope for adequate simulation studies.
•
-14-
REFERENCES
[1]
BILLINGSLEY, P. (1968). Convergence of Probability Measures.
John Wiley: New York.
[2]
BROIiI!N, R. 1., DURBIN, J. and EVANS, J. M. (1975). Techniques
for testing constancy of regression relationships over time
(with discussions).
Jour. Roy. Statist. Soc. Ser. B~ ~Z,
149-192.
[3]
DE LONG, D. (1980). Some asymptotic properties of a progressively
censored nonparametric test for multiple regression. Jour.
Multivar. Anal., !Q, in press.
[4]
[5]
•
.
HAJEK, J. and ~IDAK, Z.
Press: New York.
(1967).
Theory of Rank Tests.
Academic
SEN, P. K. and PURl, M. L. (1970). Asymptotic theory of likelihood ratio and rank order tests in some multivariate linear
models. Ann. Math. Statist.~ 41, 87-100 .