.
M-ESTIMATORS AND L-ESTIMATORS OF LOCATION:
UNIFORM INTEGRABILITY AND ASYMPTOTIC
RISK-EFFICIENT SEQUENTIAL VERSIONS
by
JANA JURECKOVA
Department of Probability & Statistics
Charles University
Prague 8, Czechoslovakia
and
PRANAB KUMAR SEN
Department of Biostatistics
University of North Carolina
Chapel Hill, NC 27514, U.S.A.
Institute of Statistics Mimco Series No. 1280
APRIL 1980
M-ESTIMATORS AND L-ESTIMATORS OF LOCATION; UNIFORM
INTEGRABILITY AND ASYMPTOTIC RISK-EFFICIENT SEQUENTIAL VERSIONS
by
JANA JURECKOVA
Department of Probability &Statistics
Charles University
Prague 8, Czechoslovakia
and
PRANAB KUMAR SENl
Department of Biostatistics
University of North Carolina
Chapel Hill, NC 27514, U.S.A.
ABSTRACT
Sequential M- and L-estimators of location minimizing the risk
asymptotically as the cost of one observation tends to
structed.
0 are con-
Their asymptotic risk efficiencies are shown to coincide
with the asymptotic efficiencies of the respective non-sequential
estimators; this enables to construct the asymptotically minimax
sequential M- and L-estimators in the model of contaminacy.
The
asymptotic distributions of the stopping times are derived for both
types of estimators.
The theorems on uniform integrability and moment
convergence of (non-sequential) M- and L-estimators, developed as the
main tools for the proofs, have an interest of their own.
AMS 1970 Subject Classifications:
Key Words & Phrases:
Primary 62L12; Secondary 60F25
M-estimator, L-estimator, asymptotic risk-
efficiency, sequential point estimator, moment convergence.
lResearch of this author was supported by the National Heart, Lung
and Blood Institute, Contract NIH-NHLBI-7l-2243 from the National
Institutes of Health.
-21.
INTRODUCTION
Nonparametric sequential point estimation of location has
received considerable attention during the recent past.
Ghosh and
Mukhopadhyay (1979) and Chow and Yu (1980) have considered asymptotically
risk-efficient sequential point estimation of the mean of a population
based on the sequence of sample means and variances, while Sen and
Ghosh (1980) have extended the theory to general U-statistics.
Sen
(1980) has considered the problem of estimating the location of
symmetric (but unknown) distribution based on a general class of rank-
order (or so called R-) estimators and established the asymptotic riskefficiency of the proposed sequential procedure.
In the classical
non-sequential case, the R-estimators form one of the three main
groups of robust competitors of classical estimation procedures; the
other two major groups are formed by M-estimators and L-estimators
[viz., Huber (1973, 1977)].
The theory of asymptotically risk-
efficient (sequential) point estimation based on a broad class of Mand L-estimators is developed in the current paper.
Unifor,m integra-
biZity and moment-convergence properties of these M- and L-estimators
playa fundamental role in this context.
Along with the preliminary notions, the proposed sequential point
estimation procedures are outlined in Section 2.
Section 3 is devoted
to the study of uniform integrability and moment convergence of the
M-estimators.
Section 4.
Parallel results for the L-estimators are considered in
These results are then applied in the proofs of main
theorems of Section 5 concerning the properties of the proposed
sequential procedures.
In particular, the Section 5 deals with the
asymptotic risk-efficiency and with the asymptotic normality of the
-3-
allied stopping times.
Similarly as in the case of R-estimators
[Sen (1980)], it is shown that the asymptotic risk efficiencies of
"
•
sequential estimators coincide with the asymptotic efficiencies of
their non-sequential versions.
This among others enables to extend
the asymptotic minimax properties of M- and L-estimators in the
model of contaminacy to the sequential case.
2.
Let
THE PROPOSED SEQUENTIAL PROCEDURES
~ I}
{X., i
1
be a sequence of independent and identically
distributed random variables (i.i.d.r.v.) with distribution function
(d.f.)
about
T
n
~e
F (x) ==F(x -8), xsR == (_00,00),
8
where
F
(unknown) is symmetric
0 and 8 is the unknown location parameter to be estimated.
be a sui table estimator of
'J
2
2
n == nE (T n - 8)
and assume that
Xl' ... , Xn
exists for all
n
~
(2.1 )
nO'
and
nO (~l)
for some
based on
8
Let
'J
2
n
-+ 'J
2
as
We conceive the loss (in estimating
~
by Tn)
8
(a, c) == a(Tn - 8)
2
(2.2)
0 < 'J < 00.
n +00,
(2.3)
+ cn,
Then the risk is
-1 2
An(a, c) == E~(a, c) == n a'J n +cn.
(2.4)
We like to minimize (2.4) by a proper choice of n. The optimal choice
where
a and c
of
generally depends on the unknown
n
are positive constants.
as asymptotically as
choice of
n
is
no(c),
no(c) ~b'J,
where
q(c)
~r(c)
n'
for any fixed
c
as well
In this asymptotic case, the optimal
where
b == (a/c)
denotes that
following procedure:
and let
c 1- O.
F,
Let
{Vn }
!<
2
(2.5)
A ()(a,c)~2'J&
nO c
lim q(c)/r(c) ==1. This suggests the
dO
be a sequence of estimates of 'J
and
be an initial sample size
(~2)
and
h(>O)
be an
-4-
arbitrary constant.
Define a stopping number
N =min{n
c
"
~n': n~b(v
n
N(= N )
c
by
+n -h ) } , c>O
and consider the sequential point estimator
(2.6)
•
TN
based on
Xl, ... ,X
c
The risk of estimating
by
is then
c
2
A*(a, c) = aE(T
-8) +cEN
c
Nc
We are primarily interested in showing that
8
Nc
.
TN
(2.7)
A*(a,c)/A
()(a,c)-d as c+o,
nO c
which means that the sequential procedure is asymptotically (as
c
equally risk-efficient as the optimal non-sequential one, if
were
v
(2.8)
+0)
known.
The convergence (2.8) has been studied by more authors [referred
to in Section 1] in the case that
{T }
n
is either the sample mean,
U-statistic or some case of R-estimator.
In the current paper, we
shall show that (2.8) holds for a broad classes of M-estimators
and L-estimators (i.e., the estimators of maximum-likelihood type and
of linear combination of order statistics type, respectively).
An M-estimator
M
of
·n
is a solution of the equation
8
(2.9)
with respect to
t,
(so that
is \
S (t)
n
where
in
~
is some nondecreasing score function
t) .
is defined by
More precisely,
M = (M* +M**)/2,
n
n
n
(2.10)
where
M* = sup{t: S (t) >O}
n
n
and
M** =inf{t: S (t)<O}.
n
n
Under suitable regularity conditions on
~
and on
F
(2.11 )
[viz., Huber
(1964)], to be specified later on,
1
2
L{n~ (Mn - 8) } ~ N(0, v (M)) as
where
n
-T 00
(2.12)
-5-
V~M) = cr~M/Y (M)' cr~M)
(2.13)
= flO1jJ2 (x)dF (x) ,
_00
00
f {-f'(x)/f(x)}1jJ(x)dF(x)
Y(M) =y(1jJ, F) =
d
d
(2.14)
(>0)
2
f' (x) =dxf(x) = ~ (x) is assumed to exist almost everywhere.
dx
In Section 3, we shall show that (2.1) and (2.2) hold for M-estimators
and
generated by a class of bounded 1jJ-functions.
2
estimate
V(M)
as follows.
2
s n (M) = n
let
<P
Let
-1 n
L·1= l1jJ 2 (X.1 - Mn ),
n ~ 1,
(2.15)
be the standard normal d.f. and let
Mn = sup{t: n
In this case, we shall
-~
O<E<l.
<P(-T E) =E,
Snet) >T / S n (M)}
o. 2
(2.16)
M+ = inf{t: n -~S
n (t) < -To. / 2S n (M)}
n
+
dn(M) =Mn
where
0.(0 <a < 1)
-Mn(~O),
n
p
-+
v
" and Sen (1980 a, b).
by Jureckova
(M)
c
"
{V" n } = {Vn(M)}
Xl' ... ,Xn
~
... ~ X
n,n
and
5
Nc(M) '
is denoted by
so that
e
(2.19)
)
that (2.8) holds for
Ln of location
Ln
Xn,l
have been studied
,,-h }
and we shall show in Section
where
vn(M)
The stopping number defined by
=min{n ~n': n ~b(Vn(M) +n
The L-estimator
(2.18)
v(M) =cr(M/Y(M);
in fact, stronger convergence properties of
(2.6), corresponding to
Then, it follows from
increases,
vn(M) =v'n dn (M/ 2T o./2
N
(2.17)
is some preassigned number.
Jure~kova (1977) that as
Put
{l\1 (M)}.
N
is typically
c of the form
= L~1= lC'X
n1 n,l.
are the order statistics corresponding to
(2.20)
-6-
c nl. = c n,n-l+
. 1 ~ 0, V 1 ~ i ~ n,
and
ll=
\~ 1c nl. = 1.
(2.21 )
. = 1 , ... ,n
(2.22)
Denote
J (t) = n.c nl. for i n- 1 < t ~!.
n '
n
1
and suppose that
J n (t) -+ J (t) a. S ., t
E:
(0, 1), J (1 - t) = J (t)
~ 0,
(J (t) dt = 1.
o
(2.23)
Then, under some regularity conditions [viz., Huber (1969)],
(2.24)
where
V~L)
0000
=
Jf
[F (x i\ y) -F (x) F (y) ]J (F (x) ) J (F (y)) dxdy
(2.25)
We proceed to estimate V~L) by
n-l n-l
2
(L) = L
L c .c .{n(i i\j) -ij}(Xn,l+
. 1 -X .)(X . 1 -X .),
n
i=l j=l nl nJ
n,l
n,J+
n,J
v
(2.26)
where under suitable regularity conditions [viz., Sen (1978)]
/\
as
asymptotic normality results pertaining to the
Gardiner and Sen (1979).
vn(L)
are due to
The stopping number, defined by (2.6), for
is denoted by
N?) =min{n
(2.27)
n-+ oo
so that
~n': n ~b(Vn(L) +n- h )}.
We shall show in Section 5 that (2.8) holds for
(2.28)
{L (L)}
corresponding
N
to a class of J-functions which vanish outside of a c compact
subinterval of (0, 1) .
In the remaining of this section, we state the basic regularity
conditions on
F,
~
and J,
pertaining to our study.
Sections 3 and
4 are devoted to the study of the uniform integrability and the moment
convergence of
{Mn } and {L },
n
respectively; these results are used
-7 -
in Section 5 in the proofs of (2.8) for the corresponding sequential
procedures.
Assumptions 'on F:
density
f
We assume that
such that
f(x) =f(-x), V xe:R
f(x)
Moreover,
F
F has an absolutely continuous
is \
in
x
for
and
x ;:::0.
(2.29)
is supposed to have finite Fisher information, i.e.,
00
reF) = {f'(x)/f(x)}2dF (x) <00
(2.30)
f
and we assume that there exists a positive number
l
(not necessarily
an integer or ;::: 1), such that
(2.31)
_00
.e
Assumptions
on~:
We assume that
~
is nondecreasing and skew-
symmetric, i.e.,
~(x)
= -~(-x)
is
!
in
x e:R+ = [0, 00),
and that there exists a positive number
~(x) =~(k) ·sign x
Moreover, suppose that
~
for
k
(2.32)
such that
(2.33)
Ixl ;:::k.
could be decomposed in the absolutely
continuous and step components, i.e.,
~
where
~l
(x)
(2.34)
(x) = ~l (x) + ~2 (x) 'I x e: R
is absolutely continuous [inside (-k, k)]
pure step-function having a finite number of jumps inside
we denote the jump-points by
J-
J
constant
Y(M)
~2
J
a =-k
o
and
am+ 1
=k.
is
(-k, k);
for
and let
a. ,
where
a. 1 <x<a.,
and
Then the
defined in (2.14) is equal to
00
YeM)
=f ~i(x)dF(x)
_00
+I;=l((3j -Sj_l)f(a j ) >0.
(2.35)
-8-
Put
where
£.
c* < 00
£.
£.
c£.(x) = Ixl F(x)[l -F(x)], x e:R; ct=sup C,e(x)
xe:R
is given by (2.31). Then
lim c£. (x)
'
(2.36 )
and
=0
x-+±oo
We assume that
and that there exist an
c.=c
'1;?:0,
nl
n,n-l+
(0 <0.
1
0
0.
0
such that
<2)
c . =c
. =0
nl
n,n-l+l
i sk
for
n
where
as
n -+00.
(2.38)
Moreover, denoting
i -1
J (t) =n'c . for -<t S i
n
nl
n
n'
we assume that
lim J ( t)
n-+oo n
where the function
J(t)
J(t)=J(l-t);?:O,
= J (t )
i=l, ... ,n
a . s ., t e: [0, 1]
has bounded variation on
(2.39)
(2.40)
[0, 1] and
te:[O,l], fJ(t)dt=l.
(2.41)
o
In the context of L-estimators, the assumption of finite Fisher
information may be replaced by a weaker assumption
sup
F
-1
(aO)SXSF
3.
If' (x)
-1
I<
00
•
(2.42)
(1-0. )
0
MOMENT CONVERGENCE OF M-ESTlMATORS
Uniform integrability and some moment-convergence properties of
1.<
{n 2 (M -e)}
n
are studied here.
The following lemmas are needed in the
sequel but they have an interest of their own.
-9~
Under the regularity conditions on
LEMMA 3.1.
and F of Section
2~
for every
(3.1 )
where
(3.2)
PROOF.
Note that for every
Po{ /Ill M
n
I > t}
= Po
{m Mn
t > 0,
> t} + Po{vn M < - t} = 2 Po
n
{m Mn
> t},
(3 .3)
where, by (2.9) - (2.11),
~po{n-1Sn(t//Il) ~O}
PO{m M >t}
n
= Po {n -1 S
n
(tim)
-)l
n
(t)
~
-)l
n
(t)},
(3 .4)
and
1
- )In(t) = -E n- S (tlrn) = -EO~(X1 - (tim))
o
=
EO[~(X1) -~(X1
n
- (tim))]
= j""[F(X + (tim)) -
=r)()~(X)d[F(X)
-F(x + (tim))]
F(X)]d~(X)oo
_00
k
=
J
.
[F(x + (thin)) -F(x)]d~(x)
-k
= r[F(X + (tim)) -F(x) +F(-x + (tim))
o
[as
= r[F(X + (tim)) -F(x -
-F(-x)]d~(x)
~(x) +~(-x)
(t/rn))]d~(x)
[as
=0, 'V x]
F(x) +F(-x) =1, V x]
o
k
=
(2tlm)J
f(x+(et/rn))d~(x)
[where
lei
<1]
o
?
(2t/m)f(k + (t/rn))[~(k) -~(O)]
?
( 2t
I /Il) f (k + c 1 ) ~ (k), 'V t
!,;
= (2c ) ~ (k)
2
E:
(0, c
[as
/n]
f(x) is \
[as
(tl In) .
Therefore, by (3.3) - (3.5), for every
in
~(O) =
x, x
~ 0]
0]
(3.5)
O<t <c/n,
-10In
!.:
PO{lnM
I nI >t}$2Po{-L·
n 1= lZ,
n1 ~(2c2)~l/J(k)(t/ln)}
(3.6)
where
Z . =l/J(X. - .-!..) - E l/J(X. _.-!..) i =l, ... ,n
n1
1
Olin'
rn
are independent r.v. with mean
1.
0,
bounded by
2l/J(k)
(3.7)
with probability
Hence, using Theorem 2 of Hoeffding (1963) on r.v.· (3.7), the
desired result follows from (3.6).
Under the regularity conditions on l/J and F of Section
LEMMA 3.2.
for every
Q.E.D.
t
2~
> 0,
(3.8)
where
m=
PROOF.
I-n ; IJ
° =1
and
n
for
n = 2m,
We consider only the case
analogous.
Note that for every
° =0
n
n = 2m;
for
n = 2m + 1, m ~ 1. (3.9)
the proof for
n = 2m + 1
is
t > 0,
PO{/i1"!Mn I >t}=2P {v'nMn >t}$2P {Xn,m+ l~-k+(t//i1")}
(3.10)
O
O
where X
$ ~ "$ X
are the order statistics corresponding to
n,l
n,n
Xl'" .,X ·
n
Since the right hand side of (3.10) equals to that of (3.8)
the proof of (3.8) is complete.
For any
a
E:
Q.E.D.
[0, 1], put
p (a) = 4a (1 - a) ,
LEMMA 3.3.
n-l
2n m _ 0n
(
where
PROOF.
m, on
n(~l)
For every
and
Jfa
pea)
and
so that
a
>~
0 $ P (a) $ 1.
(3.11)
,
l. m
n-m-o n
n
u (1 -u)
du $ 2(p(a))
(3.12)
are defined by (3.9) and (3.11).
We shall again prove (3.12) for
n = 2m
only; the proof for
-11-
n = 2m + 1
is analogous.
Note that by repeated partial integration,
the left-hand side of (3.12) reduces to
2L~:~[~)ai(1 _a)n-i ~ 2P{B(n,
a)
~ ~}
~2P{~(n, a) - a~
where
B(n, a)
1
a >2'
(3.13)
}-a} ,
is a binomial r.V. with parameters
(n, a).
Since
by using Theorem 1 of Hoeffding (1963), we may bound the right
hand side of (3.13) by
2 [pea)]
n
Q.E.D.
.
We are now in a position to prove the main theorems of this
Section.
For every
THEOREM 3.1.
there exists an
r > 0,
n
r
«
such that,
(0)
under the regularity conditions (2.29) - (2.34),
Eo{nr/2IMnlr}
PROOF.
Let
c
l
>k >0, where
<00
"
uniformly in
n;::n .
is defined by (2.33).
k
(3.14)
r
Note that
Eo{nr/2IMnlr} = fOOrtr-lpo{rnIMnl >t}dt
o
fOO )rtr-1PO{rn!Mnl >t}dt = I
nl
+I
n2
, say. (3.15)
clm
Then, by Lemma 3.1,
I
nl
~
c l In -c t 2
2 r e2t r-l dt
f
a
uniformly in
n
r
[f]
00
~
2
t r-l dt < 00
(3.16)
0
n=1,2, . . . .
=
_ct
2r J e 2
+
1,
where
On the other hand, if we let
l
is defined by (2.31)
use (2.37) and Lemmas 3.2 and 3.3, we get
(3.17)
-121
n2
tr-l[p(F(~k
:5:zrj'X>
+ (tm)))]ndt
clm
•
00
l,
n du
:5:2rn r / 2f u r -[p(H-k+u))]
c
1
-b} foo
n-n
:5:2r(CV(r-l)/i{ n r / 2 [p(F(-k+c )}]
r
l
[4F(y)(1-F(y))]b dy
C -k
1
where
cl
b > 1/ t.
P(F(C
1
Since
>k,
C1
11 ~
n
n
r/2
b
(F (c
(4F(y)(l-F(y)))bdy
as
0
n
~
n
o
and it converges to
r
n
Finally,
+00.
Hence,
n
as
so that
is uniforml y
by (2.36) and (2.37).
<00
c -k
1
uniformly in
n-n -b
r
- k))
2
and converges to
r
is any number satisfying
1
F(cl-k»F(O)=Z'
it ho 1d s
and hence,
-k)) <1
bounded for
foo
is given by (2.37) and
<00
(3.18)
+00.
I
n2
< 00
This completes
the proof of the theorem.
LEMt~
~
Under the reguZarity conditions on
3.4.
m(M
-8) 11
where
1
rn
Y(M) n
L'n1= l~(X'1
and F,
~
(3.19)
-8) =0 (n-'I)
P
is defined by (2.14).
PROOF.
The Lemma was proved in Jure~kova (1980) [Theorem 3.3].
Under the reguZarity conditions on
LEMMA 3.5.
t he sequence
I
n -~\n
Li=l ~ (Xi - 8 1
2r
~
and F of Section 2,
. +"orm",y
"1'
'1.
.-S un'1.-J
'1.-ntegra b"1",e J+"or
L
n = 1,2, ...
and
E8In-~L~_1t)J(X. _e)12r~ a2(~)(2r);
1-
as
n ~ 00-,
PROOF.
where
Since
all orders of
r! 2
1
a (M)
1
-8)
r=l, 2, ...
(3.20)
is defined by (2.13).
E8~ (Xl - 8) = 0
t)J(X
,
and
exist.
I~ (y) I :5: ~ (k)
< 00, V Y £ R, moments of
Hence, the result follows directly
from the moment convergence result of von Bahr (1965).
-13-
Under the reguZarity conditions (2.29) - (2.34),
THEOREM 3.2.
lim E ( rn 1M _el)2r
n
8
= (0(M/'Y(M))2r(2r)I/(2rr!)
(3.21)
n-+<x>
ho Zds for
PROOF.
r = 1 ,2, . ..
.
It follows directly from the uniform integrability in Theorem
3.1, from (3.19) and from Lemma 3.5.
ourselves to even integer
In
-~
n
Ii:lljJ(X
-8)1
i
r
2r.
In fact, we need not confine
Since, by the Jensen inequality,
is uniformly integrable for any
we may prove on parallel lines that as
n
r' E:[2r -2, 2r],
-+00,
(3.22)
_00
for any fixed real
For any
LEMMA 3.6.
p{
Is~ -0~1
0 > 0,
there exist positive constants
~cn-l-o,
>d
'rf n
~nO'
(3.23)
Let us define
s
Since
and
E: > 0
such that
c and nO
PROOF.
r > O.
ljJ2(X. -8),
1
02
n
=n
-lIn
2
. lljJ (X. - 8), n
i =l, ... ,n,
1=
1
~
(3.24 )
1.
are i.i.d. bounded valued r.v., by
Theorem 1 of Hoeffding (1963), for every
E: > 0
there exists an
n >0
such that
02
p{ I sn
2
-2nn
-00 1 >E:/2}$2e
, V n~1.
(3.25 )
Again, by virtue of Theorem 3.1,
1
I n I > r;}
1 ~ d $ cn -1-0 , 'rf n ~ nO
= p{ rn IM I > T'n
n
2
By (2.32) - (2.34), ljJ
is of bounded variation on R,
(3.26)
p{ M
ljJ2 (y) =
where
Y E: R.
ljJi, ljJ2
ljJ~ (y)
so that
(3.27)
+ ljJ; (y),
are nonnegative and
ljJi
is
~
while
ljJZ
is
!
in
Henc e ,
1
1 1 + ljJz(Y +ZE:)
1
sup IljJ 2 (y +t) -ljJ 2 (y) 1$ ljJi(y -ZE:)
-ljJi(y·+ZE:)
-ljJZ(Y
Itl$~
I
I
-21 2 )
I.
(3.28)
-14-
Since, by (2.33),
~i
~2
and
are bounded, by using (3.28) and the
Markov inequality, we obtain that for every
P{
1 \,n
2
1\' 2
sup 1 -L'-l~ (X. +t) --L~ (X.)
1
n 11
n
1
jt\=:;2 E
II}
>2£'
£' >0,
=:; c'n
-1-0
(3.29)
.
(3.23) then follows from (2.13), (2.15), (3.24), (3.25), (3.26) and
(3.29).
Q.E.D.
Under the regularity conditions of (2.29) - (2.34), for
THEOREM 3.3.
every
and
>0
E
there exist a C,
0 > 0,
and an
0 < C < 00
nO
«
(0)
such that
(3.30)
PROOF.
~
Since
v
(1969), we obtain that for every fixed
K«oo) and
exist an
( - K =:;) t 1 < ... < t (=:; K)
.m
m(= m )
KE
~
is nondecreasing, exploiting technique of Jureckova
and a set of points
£
>0,
there
such
that
!-<
!-<
sup In- 2{Sn (n- It) - Sn (O)} + ty (M)
J
ltl~K
+ max \ In 1.1 (t.) + t. Y(M)
l~j ~m
where
1.1.(t)
J
n
J
J
I + £/2,
(3.31)
is defined by (3.5) and
Irn 1.1 n (t)
sup
+
ty (M)
I -r 0
as
n
(3.32)
+00,
It\::::k
The random variables
i = 1, ... ,n
EZ
q)
nl
=0
'
are independent for any fixed
E (Z
q)) 2 = 0 (n -\) ,
n1
j (= 1,2, ... ,m)
and
-15Proceeding as in Theorem 3.1 of Jure~kova and Sen (1980b), we claim that
•
under (2.32) - (2.34),
(3.33)
Thus, by Markov inequality,
1-<
1-<
p{ max In- 2s (n- 2t.) -S (0)
l::;;j::;;m
n
J
n
:;
where,
I~
J=
given
I P{ln- 12 s
0,
n
(n-~t.)
J
-rnjl
-S (0)
n
we may choose
q
n
(t.)1 >E'}
J
-rnlJ n (t.)1
J
q 2
>E'} =0(n- / ) (3.34)
q/2~1+o.
so large that
(3.31), (3.32) and (3.34), we obtain that for every
p{
sup In-~[S (n-\) -S (0)] +t (U)
It I::;;K
n
n
y .'1
I >d
1 o
::;;cn- - ,
E
>0,
V n
From
0>0,
~nO
(3.35)
and (3.31) follows readily from (3.23), (2.15) - (2.18) and (3.35).
Q.E.D.
4.
MOMENT CONVERGENCE OF L-ESTIMATORS
First, we consider the following theorem on a. s. representation
of
L .
n
Let
THEOREM 4.1.
L
n
be an L-estimator of the form (2.20) with
satisfying (2.21) and (2.38).
c ., 1::;; i ::;; n,
n1
Let d.f.
F
have
the absolutely continuous symmetric density satisfying (2.29) and
Then>
(2.41).
POI'
J'
any
n
~ nO' there exists a sequence
n1 }~+1
1=1
{Y.
of i.i.d. random variables with standard normal distribution such that
L
In'2(L
n
- 8) - (n + 1)
_~
n+l
I
1
I
a .Y .
= 0 (n -~log n)
j=l nJ nJ a.s.
as
n -roo ,
(4.1)
where
a .
nJ
n
,n+l
1
= . I . b.
- L' 1 --1 b .,
nl
1= n + n1
b. = c . / f (F
n1
n1
1=J
PROOF.
We may put
8 =0
-1
i
(--1) ), 1::;; i ::;; n .
n +
without loss of generality.
(2.29) and (2.41), for every
(4.2)
Note that by
-16- .
2
{F(x)[1 -F(x)]o\f 1 (x)/f (x)l}
sup
F
-1
(B) <x<F
-1
(I-B)
~
(4.3)
[4/f 2 (F -1 (B))]
Iflex) I ~YB<oo
sup
F-1(S)~x~F-l(I-B)
where
B.
may depend on
Y (> 0)
S
As such, the condition (3.2) in
Theorem 6 of CxBrgB and R6vesz (1978) holds for the case of
F
-1
(S)
~
~
x
F
-1
(1 - S), and hence, we may virtually repeat the proof of
their lbeorem 3 [using our (4.3) instead of their more stringent (3.2)]
and claim that for every
{B (t): O~t~l}
-1
F
(B)
~x~F
nO)'
there exists a Brownian Bridge
such that
n
sup
n(~
jf(x)q (x) -B (F(x))1
-1
n
n
1;
= O(n- 2 10gn)
a.s.
(4.4)
(I-B)
where
J
q (x) = n 2 [X
n
. -F
-1
n,1
By (4.4), (4.5), we have for
max
< -k
k + 1<'
-l-n
n
i-I
(F(x))] f o r - - < F(x)
n
1
~
i
~
n.
(4.5)
n-+ oo
jrn[X. -F-1(~1)] -B(i/(n+l))f(F- 1 (i/(n+1))!
n, 1
n
n
+
.
1
= O(n
a.s.
-~
10gn),
(4.6)
so that by (2.20), (2.21) and (2.38),
IrnLn -L~lb.B(i/(n+l))1
= O(n- 1z10gn)
1= n1 n
a.s.
with
b.
n1
(4.7)
given by (4.2).
{W (t) = (t + 1 ) B
n
n
thus there exist
t E:R+} is a standard Wiener process on R+,
(-t~I):
+
n+1
n1 1=
{y .}. 1
of i.i.d. random variables with the
standard normal distribution such that
m
L
W (m) =
Y ., m = 12, ...
n
i=l n1
Therefore,
Ii1+T
Bnn+
(~1) = rn+1 [Wnn+
(.-2-1 ) - n +i lWn (1)]
i
. n+1
= '\ Y . __1_ '\ Y
L. nJ
n + 1 . L. nJ"
j =1
]=1
i=1,2, ... ,n,
(4.8 )
•
-17 so that
•
1 n+1
\,n B (_1_' ) = 1
\,n b [\,i Y
_1_' \,n+1 y ]
=
rn+I
a .• y .,
L.i=l n n + 1
rn+T L.i=l ni L.j =1 nj + n + 1 L.j =1 nj
n + 1 j =1 nJ nJ
I
(4.9)
Q.E.D.
(4.1) then follows from (4.7) and (4.9).
LEMr~\
Under (2.21), (2.38) - (2.41) and (2.42), it hoZds for any
4.1.
positive integer
r,
· E[
1 L.\,n+1 a · y ]2r = v 2r
1 1m
"
~ . ~ j=l nj
nj
(L)
n
v~L)
where
PROOF.
• (2r)!
--
(4.10)
r.12r
is defined by (2.25).
-~\,n+1
Note that
have a normal distribution
(n + 1) . L.' 1 a .• Y .
J=
nJ
-l n+1 2
"laO
J = n1
V
nJ
with mean zero and variance
(n+1)
I
*2
Ln ' say.
(4.11)
To prove (4.10), it thus suffices to show that
.
*2
2
(4.12)
llmv Ln =v(L)'
n~
but it readily follows from (4.2), (2.21), (2.38) - (2.40).
LE~~
Under the assumptions of Therorem 4.1, for any positive
4.2.
integer
there exists a
r,
and an integer
C >0
r
EO(1n L )2r:o;C
n
r
<00
'V n
Regarding (2.20) and the fact that
e = 0,
get by Jensen inequality that under
(In IL n
e I)2r
=
(rn IL I) 2r
n
:0; (
I
c ".
i=l n1
.I
1=1
(4.13.)
l'
cniF- (n ~1) =0,
. _ F-1 (_i_)
n,1
n +1
.
(rn
I
X
_ F -1 (_1_)
c
L.i=k +l ni n ni
n +1
$
such that
rn IX
n-k
\'
r
~n r .
n
PROOF.
n
n
I)2r
I)2 r
n
so that
n-k
$L.1= k n +1c n1.E(lnlx. n1.
Eo(rniL 1)2r
n
holds for
n
n
~
n,
r
.
-F-1(~1)
n +
we
1)2r <C* <00
r
as it follows from Theorem 2 of Sen (1959).
(4.14 )
-18-
Let
THEOREM 4.2.
L
n
be an L-estimator of the form (2.20) with the
coefficients satisfying (2.21), (2.38) - (2.41).
Let
d.f.
have
F
the absolutely continuous symmetric density satisfying (2.29) and
Then~
(2.42).
for every positive integer
lim E [/il (L
n+oo e
PROOF.
\i
2
(L)
A2
v n(L)
(4.15)
be the asymptotic variance (2.25) of
be its estimator (2.26) .
m(L
n
there exist
C >0
and
such
nO
- 8)
and
Then we shall need the following
Under the assumptions of Theorem 4.2, to any
THEOREM 4.3.
o > 0,
- e)] 2r = v~~) (;r) !
2 r!
It follows directly from Theorem 4.1, Lemma 4.1 and Lemma 4.2.
Let
let
n
r,
that~
for
E
>0
and
n:2: no'
-1-0
Iv~ (L) - v~L) I > d ~ Cn
p{
PROOF.
Let
F
n
(4.16)
be the empirical d.f. of
Xl' ... ,X .
n
Then by (2.25),
(2.26), (2.39) and (2.40),
-F (x)F (y)]J (F (x))J (F (y))
n
n
n n
n n
- [F(x "y) -F(X)F(Y)]J(F(X)J(f(y))}dXd Y.
(4.17)
n > 0,
Now, for every
I
xER
P{supF(x)-F(x)
p{X
where
k I <F
n, n+
-1
-2nn2
-n } =P { Xn,n-k >F -1 (1
(a )
O
0 < p (n) < 1.
Lebesgue measure
n
I >n}~2e
n
(4.18)
,Vn:2:l,
-a O) +n
}
~ [pen)]
n
, Vn2':l,
(4.19)
Al so, excepting at countably many points (with
0),
J(t)
has a derivative (with respect to
t)
=0 V i~k, with probability
n,n-i+I'
n
2
we may 'replace the domain R
in (4.17) by
(4.19) and the fact that
c
ni
-c
while on this compact region, (4.18),
-19(2.39) - (2.40) and the boundedness of IJ (t)
•
result when
J(t)
suppose that
is continuous inside
J(t)
I
lead us to the desired
[aO' 1 -a o]'
Next, let us
has only a finite number of saltus on
[aO' 1 -a ]'
O
Excluding small neighborhoods of these saltus points, repeating the
above proof and finally using the boundedness of
neighborhoods, the proof follows.
Finally,
bounded variation, and hence, for any
J(t)
n >0,
J(t)
IJ(t)
I
for these
is a function of
can have only a
finite number of points of discontinuities at which its saltus is
n.
greater than
Therefore, the proof for the case of a finite number
of points of discontinuity extends to that of a countable number. Q.E.D.
PROPERTIES OF SEQUENTIAL M- AND
5.
Let
.e
Xl' X2 ,· ..
to the d.f.
F(x -8)
L-ESTI~~TORS
be i.i.d. random variables distributed according
such that
F
is symmetric and satisfies
regularity conditions (2.29) - (2.31).
Unless otherwise stated,
will denote either M-estimator generated by the
~-function
T
n
satisfying
(2.32) - (2.35) or the L-estimator with the coefficients satisfying
(2.21), (2.38) - (2.41).
m(T
N
c
n
- 8)
and
v
2
will denote the asymptotic variance of
its estimator (2.15) or (2.26), respectively.
be the stopping variable defined in (2.6) and let
TN
Let
be the
c
estimator based on
THEOREM 5. 1 •
Xl" ",X N .
c
Under regu Zari ty condi tions of Seeton 2, for any
h >0
(in (2.6)),
p
N/nO(c) -->-1
where
nO(c)
E(N/no(c))--+l as c i- 0
and
(5.1)
is defined by (2.5);
InO(c)(T N -8)/v 11+ N(O, 1)
(5.2)
c
and
lim{A*(a, c)/"A
ci-O
nO
() (a, c)} ;::1
C
(5.3)
-20-
where
A* (a" c) and
PROOF.
Put
b =
are given by (2.7) and (2.4), respectively.
f~) ~,
E: 0 < E < 1,
and
nc*=[bl/(l+h)],
where we choose
c~O.
Then , by (26)
.,
with probability 1.
c
c >0
as
f
N 2 b ll (l+h)
For every
b ~OO
so that
(5.4 )
put
n lc = [nO( )
c ( l-)E]
c
so small that
(5.5)
n' Sn* <n
<n (c) <n .
Ie
0
2c
c
(2.6), Theorem 3.3 and Theorem 4.3 (on noting that
V n Sn
lc
),
as
Then, by
nib sV(l -E),
c to,
P{N $n } =p{v Sn/b, for some n: n* $n sn }
c
.l c
n e e
l
sP{v sV(l -E), for some n: n* Snsn }
n e e
l
(5.6)
n
$Ln:~*p{lvn -vi >EV} =o[(n~)-o] =0(co / (2(1+h))) ~o.
c
Similarly, noting that
n/b 2v(1 + E), V n 2n2c'
P(N? n) =P { m<b(VA
c
S p{v
n
$ p{ Iv
m
(where
n $n
lc
'
then
c to.
as
(5.7)
by Theorems 3.3 and 3.4.
N/n (c)
O
n/nO(c) <1 -E
E{Nc·I(N c >n 2c )}/n (c) -+- O.
O
as
n >0)
l
-vi >n} =O(n- - o )
Then (5.6) and (5.7) imply that
if
n 2n2c'
+m -h ), V n'Sm$n }
-v <n}
n
we have for
p
--+
1
as
c to.
Moreover,
and, by (5.7),
c to,
so that
(ENc)/nO(c)-l
Thj s proves (5.1).
Lemma 3.4 and Theorem 4.1 ensure that the distribution of
L
{n 2 (T
n
-8)}
is asymptotically normal,
2
N(O, v)
as well as the
"uniform continuity in probability" of this sequence in the sense of
Anscombe (1952).
Hence, (5.2) follows from Anscombe (1952) theorem.
Finally, to prove (5.3), we may follow the ideas of the proof of
Theorem 3.2 of Sen (1980), where the Lemmas 3.1 -3.6,4.1,4.2 and
Theorem 3.1 -3.3, 4.1 -4.3 of Sections 3 and 4 provide the analogous
-21-
'e
•
tools to apply the same technique in the current context.
intended brevity, the details are omitted.
Hence, for
Q.E.D,
v
It has been proved by Jureckov§ and Sen (1980 a, b) that
d A
L{ n (Vn(M) -V(M))/S} ~ N(O, 1) as
n+ oo
(5.8)
and
sup
{ndlvm(M)-Vn(M)\}~O as
m:lm-nj<on
where
ljJ = ljJl
v(M) and vn(M)
+ ljJ2
with
ljJl
(5.9)
are given by (2.13) and (2.18), respectively,
being the absolutely continuous component and
ljJ2
the step-function component and where
if
ljJ2
$ 0,
0+0
d =}
if
ljJ2:: 0
and
d =}
and
222
2
2
2 _ [( 0 /40 (M)) + v 0 1 - (r;/ Y1) ] / 0 0
S - m
2
{
1(f3· -S· 1) f(a.)
I·J=
J
J-
J
if
1
d=2
if
1
d=4
(5.10)
where
0;
=f
ooljJ4
-0~M)'
(X)dF(X)
oo
oi
= f (ljJl(X))2 dF (X)
_00
-Y~~~
(5.11)
_00
r; = fooljJ2 (x) ljJ I (x) dF (x) -
0~fvl) Y(M)
(5.12)
_00
(in the case
ljJ:: ljJl '
(2)
where
0
(M)
and (2.14), respectively), and
with jumps
THEOREM 5.2.
and
Y(M)
are given by (2.13)
a , ... ,a
are the jump-points of
1
m
ljJ2
(S. - S. 1),
J
J-
If the constant
h
in (2.6) satisfies
under the regularity conditions on
F
and ljJ
h > d,
then,
of Section 2, as
c + 0,
(5.13)
PROOF.
Note that
. by (2.6), whenever
A
bV
N
$
c
so that if we put
n
N
c
Oc
A
$
b(V
N
-1 + (N
c
= [bv] + 1,
Nc >n',
-h
c
-1)
)
we get from (2.5) and (5.14)
(5.14 )
-22-
-h
A
b(v
-V) :::;N -n
:::;b(V -1 -v) +b(N -1)
,
N
C
Oc
c
N
c
c
p
whenever N > n'. Now, by Theorem 5.1, Nc/n
--+ 1,
c
Oc
d
-h
d<h, n (N -1)
-+0 as ci-O, and
oc c
-1 d
-l+d
b nac~vnOc
as ci-a.
Thus, regarding that
(5.15) and (5.16).
REMARK.
na(c)
~nac'
(5.15)
•
while for
(5.16)
(5.13) follows from (5.8),
(5.9),
Q.E.D.
Theorem 5.2 shows that, if
1/J
2
:: a,
the rate of convergence to
the asymptotic normal distribution in (5.13) is faster than in the case
1/J
2
::
a.
Let us now consider the asymptotic distribution of the stopping
variable corresponding to the L-estimator.
Gardiner and Sen (1979)
proved
(5.17)
and
as
o i- a
(5.18)
are given by (2.25) and (2.26), respectively, and
where
K
2 =Jf1fl (Si\t-st)La(s)Lo(t)dF -1 (s)dF -1 (t)
(5.19)
oa
with
o : :; t
L (t) = L (t).J (1) (t) - L (1 - t)J (I) (1 - t),
l
1
O
r-
and
J(l)(t) =t·J(t),
(5.20)
:::; 1
L1 (t) = 2
t uJ (u) d F-1 (u) ,
(5.21)
o
THEOREM 5.3.
If the constant
h
under the Y'egularity conditions on
then~
in (2.6) satisfies
F
and
J
of Section
2~
as
c i- 0,
(5.22)
e.
-23PROOF.
.
(5.22) follows from (5,17) and (5.18) similarly as in the proof
of Theorem 5.2 .
6.
Let
ASYMPTOTIC MINIMAX PROPERTY OF SEQUENTIAL
M- AND L-ESTIMATORS
Xl' X2 , ...
be a sequence of i.i,d. random variables distri-
buted according to d.f.
unknown;
by
{T}
n
where
F
is symmetric but generally
F is only supposed to belong to an appropriate neighborhood
F of a given d.f.
8
F(x -8)
T
n
Suppose that the loss incurred in estimating
G.
is given by (2.3).
Let
T denotes the set of sequences
of translation equivariant estimators, asymptotically normally
distributed and such that the minimum asymptotic risk
>.
in
() (a, c)
nO c
(2.5) exists and satisfies
.e
limp2 ( )/(4ac)] =,}(Tn , F)
nO c
ctO
where
2
v (T , F)
n
(6.1 )
'r/ FE: F
is the asymptotic variance of
if
F (x - 8)
is the underlying d.f., and for which there exists sequential point
estimation procedure
TN
with the risk satisfying
c
(6.2)
lim[>.*(c)/>. ()(a, c)] =1.
c+O
nO c
Then we may consider the limit
e(Tn , F)
=
rc
(6.3)
lim >.*(c)
dO
as a measure of efficiency of the sequential point estimator
TN
if
c
F
is the underlying distribution.
Similarly as in the non-sequential estimation procedures, for an
appropriate family
F of distributions, there may exist an M- or
L-estimator providing the saddle-point of the function (6.3) over
T
xF.
We shall formulate such result for the case that
F represents
-24-
the contaminated distribution
G; it is an extension of the Huber's
(1964) result to the sequential case.
F*1 be the set of distribution functions
Let
F~ ={F = (1 -s)G +sH:
where
•
S'C
[0,1)
(6.4)
H sH}
is a fixed number,
G is a symmetric d.f. which has
twice continuously differentiable density
I(G) <00
and
] Ix\idG(x) <00
g,
g
i > 0,
for some
while
all absolutely continuous symmetric d.f.'s with
some
i >0.
FsFi.
mean provided we assume that
2
H; =(I-I sH: ]X dH(X) <oo}
in (6.4).
H is the set of
f\x1idH(X) <00
for
Then, for the M- and L-estimators considered in this paper,
(6.1) -(6.2) hold for
and
is strongly unimodal,
(6.1) -(6.2) also hold for the sample
F sF;,
while
where
F; = {F =
(1 - s)G + sH: H s HZ}
G satisfies the same condition as
Further, for translation-equivariant U-statistics, (6.1) -
(6.2) hold [c.f., Sen and Ghosh (1980)] for
H*
where
3
is the class of all d.f.'s
for which the kernel (generating the U-statistics) has finite i-th
absolute moment for some
.e. > 2).
Finally, (6.1) - (6.2) hold for a
general class of R-estimators [c.f. Sen (1980)] with absolutely
continuous (but possibly unbounded) score function, provided
where
F*
for some
is a sub-class of .F~
4
s >1/6.
4
and let
Tl
and (6.2) for
sF;
for which sup f (x) (F (x) [1 - F (x)])
Let
It is easy to verify that the intersection of
F*
1
F~
be the set of distributions defined in (6.4)
be the set of sequential estimators satisfying (6.1)
F s Fi .
Then there exist a sequential M-estimator
Tel) and a sequential L-estimator
N
c
-s
x
is a non-null set.
F*
THEOREM 6.1.
F
(2)
TN
c
an d
FOE
F*1 such that
< 00,
-25e(T
,
ho Zds faY'
i
Nc
~e(T~i),
' F )
O
=1 ,2,
F ) =
O
c
V TN
£
[I(Fo)]~~e(T~i),
F)
(6.5)
c
T1 and
F £
Fi . .
c
PROOF.
follows from J-Juber (1964) that the asymptotic variances of
It
M-estimators with the
~-functions
satisfying (2.32) - (2.34) have a
saddle-point which corresponds to the density
fo(x)
=
(1 _E)g(k)eq(x+k),
x <-k
(l - E)g(X) ,
-k
(l - E)g(k)e-q(X+k),
k~x
~
x
~
k
(6.6)
and the corresponding minimax M-estimator is the maximum-likelihood
estimator corresponding to
fO'
i.e.,
xER,
g I (x)
-k < x < k
g(x)
(6.8)
k<x
q
and
q =-
(6.7)
x < -k
-q
where
i.e.,
k
is related to
E according
(6.9)
It follows from Theorem 5.1 that the sequential M-estimator
corresponding to
~O
T(l)
N(M)
c
and to the stopping rule (2.19) is the solution
of (6.5).
Moreover, it follows from Jaeckel (1971) that the L-estimator
T(2) corresponding to the weight function.
n
(6.10) .
-26is asymptotically equivalent to
(2) .
T (L)
with the stopping rule
N
c
alternative solution of (6.5).
T(l)
n
N (L)
c
•
The sequential L-estimator
given by (2.28) is then an
•
REFERENCES
•
ANSCOMBE, F.J. (1952). Large sample theory of sequential estimation.
Froc. Camb. PhiZ. Soc. ~~, 600~607 .
CHOW, Y.S. and YU, K.F. (1980). The performance of a sequential
procedure for the estimation of the mean. (Unpublished),
CSORGO, M. and REVESZ, P. (1978). Strong approximations of the
quantile process. Ann. Statist. 2, 882-894.
GARDINER, J.C. and SEN, P.K. (1979). Asymptotic normality of a
variance estimator of a linear combination of a function of
order statistics. Zeit. Wahrsch. Verw. Geb. ~Q, 205-221.
GHOSH, M. and MUKHOPADHYAY, N. (1979). Sequential point estimation
of the mean when the distribution is unspecified. Comm.
Statist. A2' 637-652.
HOEFFDING, W. (1963) .
random variables.
Probability inequalities for sums of bounded
Jour. Amer. Statist. Assoc. ~~, 13-29.
HUBER, P.J. (1964). Robust estimation of a location parameter.
Ann. Math. Statist. ~2' 73-101.
HUBER, P. J . (1969). Theorie de l' inference statistique robuste.
Seminaire de mathematiques superieures. Universite de Montreal.
•
HUBER, P.J. (1973) . Robust regression: Asymptotics, conjectures
and Monte Carlo . Ann. Statist. 1, 799-821.
IIUBER, P.J. (1977). Robust methods of estimations of regression
coefficients. Opert. Statist. Ser. Stat. ~, 41-53.
JAECKEL, L.A. (1971). Robust estimates of location: Symmetry and
asymmetric contamination. Ann. Math. Statist. ~~, 1020-1034.
JURECKOVA, J. (1969). Asymptotic linearity of a rank statistic in
regression parameter. Ann. Math. Statist. ~Q, 1889-1900.
JURECKOVA, J. (1977). Asymptotic relations of M-estimates and
R-estimates in linear regression model. Ann. Statist. ~, 464-672.
v
..
JURECKOVA, J. (1980). Asymptotic representation of M-estimators of
location. Opert. Statist. Ser. Stat. 11, (in press) .
JURECKOVA, J. and SEN, P.K. (1980a). Invariance principles for
some stochastic processes relating to M-estimators and their
role in sequential statistical inference. Sankhya~ Series A
(under consideration for publication).
-28JURECKOVA, J. and SEN, P.K. (1980b). Sequential procedures based on
M-estimators with discontinuous score functions, Institute of
Statistics Mimeo Series No. l279. The University of North
Carolina, Chapel Hill, NC.
SEN, P.K. (1959). On the moments of sample quantiles.
Statist. Assoc. Bull. 2, 1-19.
Calcutta
SEN, P.K. (1978). An invariance principle for linear combinations
of order statistics. Zeit. Wahrsch. Verw. Geb. ~£, 327-340.
SEN, P.K. (1980). On nonparametric sequeritial point estimation of
location based on general rank order statistics. Sankhya~ SeY'. A
i£ (to appear).
SEN, P.K. and GHOSH, M. (1980). Sequential point estimation of
estimable parameters based on U-statistics (under consideration
for publication).
von BAlm, B. (1965). On the convergence of moments in the central
limit theorem. Ann. Math. Statist. ~£, 808-818.
•