,
ON STABLE LAWS FOR ESTIMATING FUNCTIONS
AND DERIVED ESTIMATORS
by
Pranab Kumar Sen
•
Department of Biostatistics
University of North Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 1477
December 1984
CN STABtE JAWS FUR ESl'JMATING roNcrrooS AND DERIVED ESTlMATORS
BY
*
PRANAB KUMAR SEN
University of North
CaroZina~
Chapel HiU
Stable laws for M-estimators, maximum li.keli.hcx::ld and other estimators are obtained through parallel results for the estimating functions
and relative carpactness of
t.
SCIre
related estimating functional processes.
~~~~~~. Let {p e} ee:8 be a family of probability rreasures on ~,A.),
irxiexed by the
par~ter
e
€: 8, where the par~ter space 8 is a subset of the
p-dimensional Euclidean space Ff, for
SCIre
p > 1. Let {X., i > l}be a sequence
-
~-
of independent randan vectors (r. v. ) [not necessarily identically distributed
(i.d.) J, such that under P e
-
'
Xi has a probability density function (p.d. f. )
f. (x,6) , for i > 1. Let n. (x,e) , i> 1
~
be Ff-valued functions on)E;x 8. Then,
-
~
an estimating funation (p-vector) may be defined [ viz., Huber (1967), Hi9:'jek
•
(1970), Inagaki(1970,1973) , and others] as
S (t)
n
(1.1)
and if T
n
~=
= T (Xl' ••• ,Xn )
n -ex
(1.2)
r.~ 1
=
S (T
n
n
n.~ (X.,
t) ,
~
be a
)
-+
rf-valued
t
Ff ,
r. v., such that for scme given ex >
0, in probability, as n
then, Tn is tenmed a derived estimator of
(MLE)~
e:
Huber's M-estimator and
SCIre
e;
-+
0,
00
the maximum likelihood estimator
others all belong to this class of estimators
derived fran suitable estimating functions. In the literature, the specific
case of ex
•
=~
has been treated in detail [see the references cited above
J,
_1
where the asynptotic nonnality of n ~ (e) plays a vital role. It is quite
n
AMS Subjeat Classifiaation Numbers: 60F05~ 62E20~ 62Fl2
Key Words and Phrases: Asymptotia distribution~ estimating funations~ M-estimators~
maximum likelihood estimators~ relative aompaatnes8~ stable laws.
* Vbrk supported by the Office of Naval Research, Contract No. N00014-83-K-0387
-2-
conceivable that in a general setup, for sore a >
-
~
e
, n -as (e) may have
n
(asymptotically) a Unultivariate) stahle distribution
(note that the
characteristic: e3:ponent of the stahle law in our notation corres{X)nds to a-I,
•
arrl the particular cases of nonnal and cauchy distributions correspond to a =
~
and 1, respectively) • In the nonnal case, the asymptotic Unulti-)nonnality
t
of the estimating flmction n -'1; (e) and of the derived estimator (Le.,
n
n~(Tn - e) )
are equivalent. The question may therefore arise whether a similar
equivalence result holds when, in general, n-aSn (e) has asyrrptotically a stable
law, for sc:ne a
~ ~
, and the present note provides a (distributional-) invariance
result in this direction. It is shown that under suitable regularity corrlitions,
an asyrrptotic stable law for n
stable law for
n
I-a
-a
S (e) ,for
n
~
a
d~,lJ,
ensures an equivalent
( Tn - e ).
Along with the preliminaJ:y notions, the main theorem is presented in
section 2, and its proof is outline::1 in section 3. The last section deals with
e
•
M-estimators of location in the general rmlltivariate case.
2. The main theorem.
In the regular case of a
=~
(Le., asymptotically nor:ma1.
law), it has been assUIred I c.f. Inagaki (1973)J that En.(x.,e), i> I all
~
~
-
exist and are continuously differentiable (with respect to e); the differential
coefficient matrices play the daninant role in the asymptotic equivalence
results. In the general case, though Elln. (x.,e) I Jk may_exist for evei:y-k < a-I,
~
l.
we may not be in a {X)sition to asStmE that Eni (Xi' e) is finite ( particularly,
when a
= 1)
or the variance of
n. (X. ,e) - n. (X. ,e') is finite (when a>
~
~
~
~
~).
Hence, to justify (1.2), in the general case, we asStmE that
(2.1)
n -as (e) has asymptotically a stable law G with centering parameter 0,
n
where we confine ourselves to
(2.2)
<
a
<
I.
we may note that for a >I, nI-a converges
to 0 as n -+
00,
•
so that even if
-31
n -o. ( Tn - e) has asynptotically a non-degenerate distribution, Tn nay fail to
be a consistent estimator of e , and hence, we nay not have much interest in
the asynptotic properties of
{T}
n
For every d e: Ff , we let
n.~~
{.X., e+
U. (x., e,d) =
(2.3)
~~
d} -
n.~~
(.X.,e
-
) , i > 1-
For an arbitrary block B = (u,vJ (where u < v and roth belong to Ff ), we
define the increment functions as
'"
Ui (B) =
(2.4)
1:{j~O,l;l<k<P} (-I)
p-1:k jk
Ui (xi,e, u + J(v-u», i .:::. 1,
where J = Diag(jl'''. ,jp}' and let A (8) be the Lebesgue IOOaSure of B. Then,
every <XIllpaCt set ec c e ,
we
assume
d
,D and an r ( > 1), such that for every i > 1,
r
E~ll Ui(.Xi,e,dlll ~Dlldllr, \ld: Ildll <do'
o
that for
there exists IX>sitive numbers
(2.5)
Eell Ui
,.
(}3}ll r ~
D{ACB)J
r
,'rju,v :llull<do ' Ilvll
~do·
Let us also denote by
where the e. are p-vectors and e. has 1 in the jth IX>sition and 0 elsewhere,
J
J
for j= 1, ... ,p. Let then
-
rn = n-1 E.n
(2.7)
we
1
~=
r. , n
~
> 1.
-
assume that there exists a IX>sitive integer n
rn
(2.8)
is nonsingular ens) for every n > n
-
o
0
,such that
•
Note that i f in (2.2), a. = 1, n 1-0. is also equal to 1. In this case, we nay
need to assume that (2.5) holds for every d
..
o
> 0, and this will be referra:l to
as (2.5 I ) . Also, in this case, we may nea:l to strengthen (2. 6) - (2.8) to
limn-.oo Iln-lE~
lEeU.~ (.X.,e,d)
~=
~
(2.9)
where
fn
+ f nd II
=
0, VCfixa:l) d e: RP ,
does not depend on dand it satisfies (2.8). Then, we have the following.
-4-
Theorem 1 • If in
n1 - a
(2.10)
(2.2)~a €
rn (Tn
I~,l)
, then under
(2.1)~
(2.5)~
(2.6) and (2.8)
has asymptoticaUy the stable lC1JJ) G ~
- e)
where G is defined in (2.1). For a = 1
~
under
(2.1)~
(2.5') and
(2.9)~
(2.1V
The proof of the theorem is provided in the next section. Note that for
a =
~
,
"
, in (2.5), we may take r = 2, although the second m:::m:mt may not exist
for a >
~
; for our purp:>se, r > 1 suffices. Also, our (2.5) is m::>re easily
verifiable than the s\lP-nonn m:::m:mt corxlition in Inagaki(1973) or others.
3. Proof of the main theorem.
~
First, we proceed to construct a sequence of
-~~-----------------------
estimating functional processes ani establish its tightness ( or relative
compactness ); these are then incorporated in the proof of the main theorem.
For sane arbitrary positive
of
II? •
K«
oo}, let C
For every n ( ~ 1) and (fixed) e
stochastic process
W
n
= { Wn (u)
€
= I-K,KJ P
be a CCIlp3.Ct subset
0, we consider a (vector-valued)
; U € C } (belonging to the SPaCe oP[C]
,
l.
endoweD with the (extended) Skorokhod Jl-topology ) , where
(3.1)
Wn (u)
= n-a
...n
l..·11
1=
a-I
u.(X.,e,n
u)
1
1
-
a-1
EeU.(X.,e,n
u)],
1
1
U € C.
Then, we have the following.
LamIa 3.1.
(3.2)
For every a
sup
u€C
€
[~,l),
-+
under (2.5) , or under (2.5') ~ a
o,
in probability,
~
n
-+
00
•
Proof. Without any loss of generality, in (2.5) [or (2.5')J, we let r
Note that by (2.3) and 0.1), for every (fixed) u
€
C,
= 1,
€
(1,2J.
W (u) involves ( n)
n
independent sumnands. Thus, using a version of the LP-convergence theorem [ viz.,
Chatterjee (1969)J, we have, for every r
Now, by (2.5)
€
(1,2J,
(for a < I), for every do > 0, there exists an no ' such that
-5-
na-lilull < do ' for every u e: C and n
saIOO.
Hence, for n > n
-
0
~ no ; for a = 1, (2.5') ensures
the
, the right hand side Crhs} of (3.3) is bounded fran
above by
(3.4)
where c *
r
«
00) is a positive number independent of u e: C. Since the rhs of (3.4)
converges to 0 as n
-+
00
,
by using the Markov inequality, we obtain that for
finitely many ( say, rn) (arbitrary) points u ' ••• , urn ( all belonging to C ),
l
(3.5)
[Wn(ul), ••• ,Wn(u )]
m
-+
(0, ••• ,0) , in probability, as n
-+
00
•
Thus, to establish. (3.2), it suffices to verily the tightness of {H }
n
'I'OWal::ds this, we define a block B = (u,v] (for u,v e: C) as in before (2.4),
so that as in (2.4), the incran=nt of W over the block B is given by
n
E .
(3.6)
W (B) - E
n
-
= n
{jk=O,l;l~k~p}
-a '("n
ui=l
(-l)P-
k Jk
'("
W (u -hJ (v-u»
n
(-l)P -Ek jk •
U{jk=O,l;l~k~}
I u.~ (X.~ , e,na-I (u-hJ (v-u» - EeU.~ (X.~ , e,na-I (U-hJ (v-u) )]
--a
=
n
-a I:.n II U . (B)
~=
m
-
EeU . (B) ] , say,
n~
where the U . (B) are independent r.v., so that proceeding as in (3.3) and (3.4),
m
we obtain that under (2.5)
(3.7)
EeIIWn(B)llr
(for n
~
no) (when a <1) or (2.5') (when a = 1),
~ c;n-(r-l) [A(B)]r, forsarer> 1,andeveryBCC.
This in accordance with the nn..tltipararreter version of the classical Billingsley
(1968) inequality ensure the tightness of W • Q.E.D.
n
Next, we note that by (2.5), (2.6) and (2.7), for every a e: L!z ,1),
(3.8) II n -aE.n 1 EeU.
~=
~
ex. , e,
~
n a-Iu)
+ -rn u II
-+
0, as n
-+
00
,
unifonnly in u e: C, while, (2.9) is just the sarre result for a = 1. Therefore,
fran (3.2) and (3.8) Cor (2.9) for a = 1), we obtain that as n
-+
Q)
,
-6-
a l
sup lin-o.t!lI n.
,6+n - u) - n. (.X. ,e)
ue:C
~
1.1.
1.1.
ex.
(3.9)
+ rn ull
-+
0, in .probability.
~
..
Now', by (2.1), n -o.~=l n (.Xi,e) = n-o.Sn (8) has asynptotically a stable law G
i
(.with centering parameter 0) , and without any loss of generality, we asSl.lrre
that G is nondegenerate ( otherwise, the results will be trivial). For norrlegenerate G , n -as (e) is 0 (1) and is nondegenerate too. On the other hand, using
n
p
(2.8) and (2.9), we claim that for every e: >0 , there exists a canpact set Ce:
(in
#)
such that on writing T = 8 + n-(1-a)u (with T definerl as in (1.2»,
n
n
n
C I 8 } > 1 - e: , for every n > n •
e:
- 0
It may be mted that by virtue of (2.8), though u may not be unique, all such
l3.10)
p{
u
n
e:
n
solutions are convergent-equivalent , in probability, and hence, for the asyrnptotic distribution of T
n
, anyone of these would be usable. Consequently, from
(3.9) and (3.10), we obtain that as n
(3.11)
n
-a
S (T ) - n
n n
-a
S (8)
n
-+
00,
+ n I-a -rn (Tn
- 8
0, in probability,
-+
t
so that using (1.2) and (3.11), we arrive at (2.10) and (2.11). This canpletes
the proof of the theorem.
We may note that the asynptotic linearity result in (3.9) has been userl as
1
the main tool in the proof of the asymptotic nonnality of n ~ (T - e) for the
n
regular case of a =
~;
it plays the
sanE
role in the general case of a e:
[~,l].
One of the advantages of using the estimating function in (1.1) (instead of the
usual likelihood function) is that it may be userl to study the asyrrptotic
behaviour of the MLE when the nDdel may be incorrect. For exarrple, if h. <.x, e) ,
1.
i > 1 be the assUIt'Ed p.d.f. 's for the r.v. 's
-
X., i > 1, while the true p.d.f. 's
1.
-
are the f. <.x,8), i > 1, the n. (.,8) will be definerl in tenus of the h. (.,e)
1.
-
1.
1.
whereas (2.5) through (2.9) can be verifierl with respect to the ture p.d.f.'s
f. (. , e). This will reveal the robustness of the MLE for departures from the
1.
assUIt'Ed m::xle1. A classical case relates to nonnal h. C. , e) against cauchy f. C., e) ,
1
i .:: 1, where , we would have a = 1 and a stable law of the cauchy type.
1.
~. ~!~~~_~~~_~~_~~~~~~~..~~",~SX2~~~~~. Let {xi = (Xil'''· ,Xip ) , , i.::
l}
be a sequence of independent i.d.r.v. having a p(> I)-variate continuous
distribution function (d.f.) F, such that the jth marginal d.f. is symnetric
about a location
8
j
, for j =l, ••• ,p, 8 = (8 , ••• ,8p ) is the location vector.
1
'1'0 esti.nate 8 , we use a vector Iji
;
of score functions
Iji. (u), u E: R , j=l, ••• ,p,
J
where we assuma that for each j, Iji. is nondecreasing and skew-symnetric • Then,
J
we have the set of estimating functions
, ... ,
(4.1)
Iji (X.
P J.P
- t ) ]
P
, t
E:
rf .
l'bte that marginally, each r.~ lljJ. (X.. - 8.) has a distribution symretric about
~=
J
~J
J
0, arxi S (t) is oonincreasing in each of its p a.rgurrents. Hence, for (1.2), we
n
may locate a closed rectangle for which S (t) = 0, and the centre of gravilty
n
of this closed. rectangle may be taken as the M-estimator of 8 • The sample rrean,
median and MLE (vectors) are all particular cases of these M-estimators.
•
'...l
Whenever the 1jJ, are continoous and satisfy a Lipschitz condition, it is
J
easy to verify that (2.5) holds; we do not need the Iji. to be bounded in this
J
context. l-breover, if the 1jJ. have continuous first order derivatives a1.Irost
J
everywhere (a.e.) (i.e., the set of points for the discontinuities of the
derivatives is of neasure 0) or if the
1jJ. are continuous while the marginal
J
densities have all finite Fisher infonnation, then <2.6) holds; we do not need
L:l. 7)-(:l.~) {as we are dealing with i.i.d.r.v. 's here]. (2.9) is of course nore
restrictive and demands
SCIre
sort of linearity of the expected values of
Iji . (X .. -8 . -d .) - 1jJ. (X.. - 8. ).
J
~J
J
J
J
~J
J
Thus, (2.10) holds under quite general regularity
conditions, while for (2.11), we need a nore precise linearity result on the
expected score-diffe.....-ences. In this context, it may not be necessary to assume
that the 1/1. are nonotone. But, then one needs to assume that S (e) has location
J
n
o (in sane neaningful way), and one needs to verify (1.2) also ( as there may
be multiple (non-equivalent) roots). Finally, for a. <3/4, one may also allow
jl1llp discontinuities for the
ljij (finitely often) .For sane related linearity
-8-
v
,
results, we may refer to Jureckova and Sen (1981 a,b).
BIILINGSLEY, p. (1968). Convergence of Probability Measures. John Wiley, New York.
CHATI'ERJEE, S.D. (1969). An
HAJEK,
rP- convergence theore.
Arm. Math. statist.iQ., 1068-1070.
J. (19701. A characterizatiDn of limiting distributions of regular
estimates. Zeitschrift Wahrsch.
veJ:W.
Geb. 14, 323-330.
HUBER, P.J. (1967). The behaviour of maximum likelihood estimators under
oonstandard conditions. Proc. Fifth Berkeley Syr!J>. Math. Statist. Prob•
.!.,
221-233.
INAGM<I, N.
(1970). On the limiting distribution of a sequence of estimators
with unifonnity property. Ann. Inst. Statist. Nath.
INAGAKI, N. (1973).
~totic
33.,
1-13.
relations between the likelihood estimating
function and the maximum likelihood estimator. Ann. Inst. Statist. Math.
~
, 1-26.
~ , J. AND SEN, P.K. (1981a). Invari.ance principles for sane stochastic
processes related to M-estimators and their role in sequential statistical
inference.
.rtJREO<ovA',
Sankh.ya ,
Ser.A 43 , 191-210.
J. AND SEN, P.K. (198lb). Sequential procedures based on M-estimators
with discontinuous score functions. Jour. Statist. Plan. Infer.
~,
253-266.
e
,