•
.
"
ON ADAPTIVE SCALE-EQUIVARIANT M-ESTIMATORS IN LINEAR MODELS
by
v
Jana Jureckova
Department of Statistics and Probability
Charles University, Prague, Czechoslovakia
and
Pranab Kumar Sen
Department of Biostatistics
University of North Carolina, Chapel Hill, NC
27514
Institute of Statistics Mimeo Series No. 1422
•
December 1982
•
On Adaptive Scale-Equivariant M-Estimators in Linear Models
v
/
*
JANA JURECKOVA and PRANAB KUMAR SEN
Based on tile concept of regression quantiles (due to Koenker and
Bassett ), an adaptive, scale-equivariant version of M-estimators
in linear models is considered. Various properties of the proposed .
estimator are studied.
KEY WORDS: Adaptive estimator; linear models; M-estimators; regression
quantiles; scale-equivariance.
1. INTRODUCTI ON
Let XI"",Xn be n independent random variables (r.v.) with distribution
functions (d.f.) FI, ••• ,Fn ,respectively, all defined on the real line R.
•
Consider the simple linear model where
F. (x) = F( x - S'c.) ,
1
-
_1
X E
R , i=l, ... ,n,
13' = (131" .. ,Sp) is an unknown vector of p(
~
(1.1)
1) parameters, the
~i
are
known vectors of regression constants, and the d.f. F is assumed to be
absolutely continuous and symmetric about O. Let
~n
= (Mnl"" ,Mnp ) , be the
11uber M-estimator of 13 , i.e., it is a solution of
. L~=l ~i W( Xi - !'~i ) =
where the score-function
X
W(x)
= {
kS~nx
W is
Ixl
, Ixl
2
(1.2)
defined by
<
k
>
k
(1.3)
for some specified value of k( > 0). We write
x-n =
,
(XI,oo.,Xn )
and ~n = ~n(~n) •
(1,4)
M is known to be a robust estimator of ~ having other desirable properties
-n
•
* Jana Jure~kova is a Professor, Department of Statistics and Probability,
Charles University, Prague, Czechoslovakia, and Pranab Kumar Sen is Cary
C. Boshamer Professor, Department of Biostatistics, University of North
Carolina, Chapel Hill. The second author's research was supported by tile
National Institutes of Health, Contract NIH-NHLBI-71-2243-L •
-2-
to.? However, it has one discouraging property, namely, unlike the R- or Lestimators,
to
~n
d'~n(~n)'
d~(x)
is not scale-equivariant
i.e.,
~n(~n)
is generally not equal
for all d > O. This is particilarly due to the fact that
~(dx)
•
t
for all x, whenever k<oo. On the other hand, the choice of k in (1.3),
linked to the minimax property of M
_n for the error-contamination model, may
generally depend on the scale parameter of F or on the amount of contamination
through the percentile points of F, which may not be precisely known. For this
reason, Huber (1973,1981) suggested the use of a studentized version, viz.,
n
E. 1 c. ~((X. - t'c.)/s ) ,
1=
-1
1.
- -1
n
(1. 5)
where sn is some suitable estimator. of the scale parameter. We consider here
an adaptive (scale-equivariant ) estimator which does not require the locationscale model for the d.f. F and is somewhat related to the studentized verSlOn.
The proposed estimator is based on the concept of re2!ession quantiles due to
Koenker and Bassett (1978) and the same is introduced in Section 2. The main
•
results on the properties of the proposed estimator are presented in Section
3
0
The concluding section deals with a general class of M-estimators and their
corresponding adaptive versions.
2. AN ADAPTIVE M-ESTIMATOR
Following Koenker and Bassett (1978), we define for every a:O < a < 1,
~
$ (x)
a
=
Pa(x)
= X$a(x) , x
a
- I (x
0) ,
£
The a-th regression quantile
n
E·1= 1 Pa (X.1 - _
t'C.)
_1
X £
(2.1)
R,
(2.2)
R.
~(a)
is then defined as any solution (t) for which
is a minimum.
(2.3)
Consider the residuals
A
AI
X.1 ( a) = X.1 -
_13(a )c.
_1 , i=l, ... , n
and , for every a : 0
< a
<!z , define ~n(a) = Diag(anl(a)"" ,ann(a))' by letting
A
aniCa)
A
1, if X. (1
=
1
{
(2.4)
-a
) < 0 < Xl· (~)
0, otherwise,
~
for i=l, .•• ,n.
(2.5)
•
-3-
•
Also, let ~n = (~l""'~n)
I
,and let
_'
~n(a) = (~n~n(a)~n) (~n~n(a)~n)
, (2.6)
Thus, L ( ) is the ordinary least squares estimator of 8 based on the n- 2 [an]
-n a
= 1,
observations for,which ani(a)
squares estimator of
of
~,we
~
and is therefore termed the trimmed least
• Having obtained this scale-equivariant estimator
consider the residuals
A
,
= X. - L ( )c. , i=l, ... ,n ,
e,1
(2.7)
-n a -1
and the corresponding empirical d.£.
1
A
Fn (x) = n
-1 n
1:. 1 I ( e. < x ) ,
A
1=
Further, let
=
•
~n(a)
(2.8)
R.
X £
1-
f sup{ x
F (x) < P },
\..inf{ x
F (x) > p},
A
n
O<p<~,
-
A
n
-
~
<p< 1 ,
(2.9)
with an arbitrary convention for p = ~. Let then,
A_I
l ( a ) )/2, a £ (O,~)
k = ( F (1 ~ a) -
F-n
n
n
(2.10)
Finally, keeping (1. 3) in mind, we define
A
{
IjIn(x)
=
x, Ixl
, knsgnx,
<
-
Ixl
k
n
(2.11)
k
n
where kn is defined by (2.10). Then, the adaptive estimator
defined as the solution of
>
E~ 1 c. ljJ ( X. - t'C. ) = 0 •
1=
-1
n
1
of 8 is
(2.12)
--1
In the back of our mind, in (1.3), we let k
symmetry of F,' F-l(a)
*
~n
=
F-l(l-a), so that by the assumed
-k. The relationship between k and k provides the
n
basic key to our subsequent analysis.
3~
=
*
ASYMPTOTIC PROPERTIES OF M
_n
The estimator L
_n (a ) has been studied in further detail by Ruppert and
Carroll (1980), and its relation with M by Jureekoval (1983a,b). We intend to
•
-n
'
employ these results along with the Bahadur-representation type results for
weighted empirical d. £. (due to Ghosh, and
Sen (1972))
to study the asymptotic
*
properties of M
_n and its relationship with M
_n • Note that by the results of
-4-
Ruppert and Carroll (1980), under appropriate regularity conditions, including
limn~ n-l~~~n = 9 (positive definite)
(3.1)
and
o,
•
(3.2)
for every a : 0 < a <J."
~
n ( ~n(a)
-
~
-1
2
"'hI" ( 0
•.np
_ , Q
_ • a (a,F) ) ,
)
(3.3)
where a 2 (a,F) is the asymptotic variance of the a-trimmed mean, and is given by
S
2
2
(1-2afl{ !_~ x dF(x) + 20. So. } , where F(sa) = 1- a.
(3.4)
a
Further,
1
n'i
II
~n (a) - ~n (a)
II
pO,
as n
+
00
(3. 5)
,
l
where M
_n ( a ) is defined as in (1,1) - (1, 4) with k = ka = F- (I-a). Actually, for
p=2, -1
c. = (l,e.)',
i=l, ••• ,n, (3.5) has been improved by Jure~kova(1983b)
to
1
.
°
nJ., II M
L ()
(3.6)
_n (a ) - -n
a II = p (n -Ja) , as n + 00 •
It may be noted that (3.6) holds in the general case of p .::. 2, and, further,
under appropriate regularity conditions, we shall show that
n~11 M"'()
_n a
-
M
_n ()
a
II
=0p (n-"'), asn
+00
•
(3.7)
,
where M* ( ) is defined by (2.10)-(2.12). This would establish the equivalence
-n a
of all the three estimates M
L (a ) and M
(of _~ ) upto the order
_n (a )' -n
_n'" ()
a
0p(n- 3/ 4), so that they all share the connnon properties of the Huber estimator
~Il(a)
• In addition,
~n(a)
is not scale-equivariant but
'"
~n(a)
is so. Hence,
the proposed adaptive estimator combines the desirable properties of the
l~ber
estimator and,at the sametime, is scale-equivariant.
We define e.1 = X.1 -
~'c ..
.,.-1
, i=l, ••• ,n, so that these are i.i.d.r.v. with the
d.£. F. Let then K'" = [-k, kJP , and, for every ~
E:
_ -~ n
-k
Z ( a ) (t)
2:.c. ljJ (e.1 - n 2 _t 'c.
) ,
-n
- - n
1- 1 -1 a
-1
where ljJa is defined by (1.3) Witll.k = ka • Also, let
*
-!z E._
n
-k
S
_n (x; t)
_ = n
1- l c.
_1 I( e.<
1 _ x. + n 2t'C.),
__ 1
Let then
K"', let
(3.8)
X E:
R, t
E:
K"'.
(3.9)
•
-5-
•
II
*
*
Wn(x,!) = II~n(x;!)
- .~n(x;~)
- f(x)9n!
(3.10)
-1 '
= nee
(->- Q, by (3.1)) and f=F'. Also, we assume that
-n-n
= n-1~1= 1c-1. ->- c as n ->- '" , where II~
II < "'.
C
-n
-'111en, we have the following
where
0
~l
(3.11)
Lemma 1. Under (3.1). (3.2). (3.11) and the assumed absoZute aontinuity of the
densi ty funation f. over [-k. k'] • as n
BUp{
W (x.t) :
n
The proof
-
t £
-
K* • -k
x < k }
<
•
->- '"
!<
= 0 (n-' )
(3.12)
p
parallel to Theorem 3.1 of Ghosh and Sen(1972), and hence,
TllnS
is omitted. Actually, Ghosh and Sen considered an almost sure order, and
hence, needed more stringent regularity conditions. In view of Theorem 3.1
of Springer and Sen (1983), the Bahadur-type representation holds even without
the concordance-discordance·condition of Juretkova' (1971), so that (3.12) also
•
holds without such extra conditions •
Note that by (3.8) and (3.9),
~n(et)
(!)
=
'"
1_",
*
1jJet(x)~n(x;!)
'!
£
(3.13)
K* ,
while, by the classical central limit theorem (for independent summands),
~n(et) (E)
rv
X ( ~, g.J~", 1jJ~(x)dF(X) ) ,
(3.14)
and hence,
=0
II ~n(li) (E) II
P
(3.15)
(1) •
Further, by (3.10), (3.12) and (3.13),
sup{ II~n(et) (!) - ~n(et) (E)
+
= sup{ I I~n(et)(!) - ~n(et)(~)
9n!(1-2et) II :
+
!
£
K*
}
9n!'{~'" f(x)d1P et (x)}I I: t
£
K* }
£
K* }
k
*
*
= sup{ I I I_ etk {~n(x;~)
- ~n(x;!)
+ f(x)9n!} d1P et (x) I I: t
et
~ sup{ wn (x;t)
< x -< ket } = op(n-~ ).
- : ~_ £ K*, -ket-
(3.16)
I
Therefore, on letting -t = n~(M_n(et) -_B ), we obtain from (1.2), (3.15) and (3.16),
•
n\
~n(et)
-
~
)
g
(1-2et)-lg~1~n(et) (E)
-1 n
If we define Fn (x) = n E'_lI(e.
< x) and let
11-
+
op(n-~
).
(3.17)
-6kOn ~ ( F-l(l-a)
- F-ICa)
)/2 ,
n
n
(3.18)
is defined as in (2.9), then, noting that k
~ F-l(l-a), we have
a
by the classical results on sample quantiles that whenever f(k ) > 0
a
'
nJ, I kO
~
(1) •
nka p
(3.19)
where
F~l
•
°
Also, i f we let
SO(x;t)
~ n-l~_lI(e.
< x + n-J,t'c.)
, x £ R,
n
11 _ -1
!-_
£ K*
(3.20).
then, S~(x;£) ~ Fn(x), and parallel to (3.12), we have
I:
sup{ nJ, ISo(x;t)
n
_ - SO(x;O)
n
_ - f(x)c't
_n_
~ 0p(n-l.i). (3.21)
x £[-k,k] ,t £ K*}
Using (2.7) - (2.10), (3.3) and (3.21), we conclude that
~
0 (1)
P
(3.22)
Thus, by (3.19) and (3.22),
1
n~
I kn
-
k
a
t
~ 0 (1).
. p
(3.23)
We may virtually repeat the steps in (3.16) -and verify that for every £ E(O,J,),
sup
£.::.a<J,
t
sup
£
K*
liZ
(t) - Z
(0) +Ot(1-2a) II
_n(a) _
_n(a) _
~-
Further, if we let for
~
0 (n -1<4
p
(3.24)
) - •
•
u (: [-k,k] ,
1
~n(u) ~ n~{ ~n(a)(£) ~ E~1~ 1 c.{
_1
~n(a+n-~'u)(£)}
~ a (e.)
1
~a+n -\-U (e.)}
,
1
(3.25)
then, by routine computations,_ we have, for every u,
EW
_n (u)
~ _0 alldE(W
(u» 'O-l(W
-n:::n
_n (u»
~ u 2 {f(ka
n- 2 +O(n-J,lu3
1 )
(3.26)
'
and a similar result holds for W (u) - W (u') • Thus, visualizing W as a
-n
-n
-n
-
(multivariate) process on C[-k,k] and using the Kolrnogorov existence theorem,
we conlude that for every k( 0 < k < =),
sup{
By (3.24)
Ilw-n (u) II
: lui < k
-
}, ~
° (1)
(3.27)
p
(3.27) and (1.2), we have the following
for every k
0 < k <=,
1
nn (a,k) ~ sup{ Iln~(M
_n (a+n -\-u) - _fl ) (1-2 (a+n -~»
-l~l~n(Cl+n"~) (£) II :' lui <k} ~ 0p (n -!.o) • (3.28)
As such, we have
0 -1Z
sup{lln\M
_n (+
Cl n -1.;)
l.i - 6) - (1-2a)-1 :::n
_n ()(O)
Cl _
II:
lui <k}
•
-7-
•
< nn (a,k) + 2n-\1-2a) -1(1-2a-2n-~kfl.sup{IIQ-lz
-n _n (a+' n-4)
lJ (0)
_ II: lui <k }
+ n-~(1-2a)-1.sup{1 l~l~n(u)1
I :
lui <k}
= 0 (n-~) + 0 (n -l:i) + 0 (n-l:i) = 0 (n -~).
P
P
p
. p
111erefore, by (2.11), (3.23) and (3.29), we conclude that
. (3.29)
n~( M*(
) - ~ ) = (1-2a)-lo-lZ ( ) (0) + 0 (n-~) ,
-n a _
;:n -n a p
(3.30)
so that (3.7) follows from (3.17) and (3.30).
4. RFMARKS ON GENERALsaJRE FUNCfIONS
In addition to the.Huber M-estimator, we may also consider other M-estimators
and obtain the corresponding adaptive versions based on the trimmed least
squares estimators. For example, if we consider the score function
•
(4.1)
then, for the location model, the corresponding M-estimator reduces to the
..
-l{
n- rna]
}
[na]Xn : rna] +[na] Xn:n-[an] + Li=[na]+l Xn : i '
Wlllsonzed mean n
where the Xn.1
.' are the. ordered r. v. corresponding to Xl, ••• ,Xn • Estimate
of the regression vector ~ based on the score function in (4.1) can also be
considered; its refined relation with the trimmed least squares estimate has
also been studied by Jureckova (1983llJ. We may easily obtain the adaptive
version of this M-estimator by replacing in (4.1), F-l(a)' = _F-l(l_a) by
-k " defined by (2.10). In such a case. (3.7) holds under no extra regularity
n
.
conditions. TIle adaptive estimator will be scale-equivariant, while, the
original M-estimator (based on some assumed value of F-l(a) ) will not be so.
In general, for an·arbitrary score function having a constant value outside
a compact interval, the proposed adaptive version will workout well, and
•
a relation like (3.7) will follow under. quite general conditions.
•
•
•
REFERENCES
Ghosh, M. and Sen, P.K. (1972). On bciunded length confidence interval for the
regression coefficient based on a class of rank statistics. Sankhya, Ser.A
34, 33-52.
lluber, ,P.J. (1973). Robust regression: Asynq>totics, conjectures and Monte Carlo.
Ann. Statist. !, 799-821.
Huber, P.J. (1981). Robust Statistics., New York; John Wiley.
Juretkova', J. (1971). Asynq>totic relations of M-estimates and R-estmates Ln
linear regression model. Ann.Statist. ~ 464-472.
Jureckoval, J. (1983a). Winsorized least ,squares estimator and its M-estimator
counterpart. ,In Contributions to Statistics : Essays in honour of Norman
L. Johnson (ed. P.K. Sen), North -Holland, Amsterdam,pp.
Jureckova; J. (1983b).Robust estimators of location and regression parameters
and their second order asynq>totic relations. To appear in the ~. 9th
Prague Conference on Statist. Dec. Theory 'and 'Infonn.
Koenker, R. and Bassett, G. (1978). ,Regression quantiles. Econometrica
46, 33-50.
Ruppert, D. and Carroll, R.J. (1980).Trimmed least squares estimation in the
linear model. Jour. Amer. Statist. Asso. ~, 828-838. "
Springer, J.M. and Sen, P.K.(1983). On M-methods in multivariate linear models.
(to appear).