•
•
•
•
ARE BAN ESTIMATORS THE PITMAN-CLOSEST ONES TOO?
by
Pranab Kumar Sen
Department of Biostatistics
University of North Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 1468
October 1984
ARE BAN ESTIMA'IURS THE PI'!MAN--cr.a;EST
By
Univepsity of
cm:s
'lID ?
*
PRANAB KUMAR SEN
Nopth·Carolina~
Chapel Hill
For estimators admitting an asyrrptotic representation, it is shown
that asynptotic efficiency in the usual sense leads to the 'closest'
character in the pibnan-sense. The results are based on the asynptotic
distribution theory of regular estimators along with
sate
elerrentary
inequalities on non-central chi squarErl distributions.
1. ""1""_"...,_'\1_
Introduction.
Let ( X, A, ll) be an arbitrary rreasure space with II sigma.....
"""
"'V
finite, and let { X.; i > I}
~
-
be a sequence of independent and identically
distributErl (LLd.) randan vectors (r.v.) with X. taking values in X with
~
a probability distribution Pe(dx) = fe(x)dll(X) , x
•
C
X,
e =(el'U.,ep )
£
e
EP , the p-climensional Euclidean space, for sate p ~ 1. In the sequel , X
t
will be taken as E for
•
£
sate
t
?.
1 and
A as the Borel field in X • It may
not be actually necessary to asS'l.1rre that these r. v. 's are Ld., and the
necessary m:xlifications can be done quite routinely. However, for the sake
of s.i.nplicity of presentation, we shall stick to the Lied. setup. we
define the log-likelihood function by
(1.1)
We ass'l.1rre that the usual regularity conditions
on the density f (and its
partial de:d vatives) needed in the context of the asynptotic distribution
theory of maximum likelihood estimatops (MLE) [viz., Hajek (l970) , Inagaki
(1970,1973) and others] hold. Also, we define
•
* Work partially supported by the National Heart, LlIDg and Blood Institute,
Contract: NIH-NHlBI-7l-2243-L from the National Institutes of Health.
AMS Subjeat C"Lassifiaation N'WTIbeps: 62 G 20, 62 F 12.
Key Wores and Phpases. Asynptotic efficiency, BAN estimators, closest estimators' correlated quadratic fonns, infonnation matrix, Mahalanobis nonn,
M-estimators, nl1e, regular estimators, R-estimators.
-21
.
(a/aellog
" Ln (el Ie=
- y ' y e: e ,
2
I(S) = E{ -(a /aeae'11og fS(~) } , e E e.
~ n (y)
(1.2)
(1.3)
= n -~
~
'!ben, there exists a sequence {e } of MLE of S , such that
n
"en)
~n(
(1.4)
t
0, in P s
+
'
as n +
00
,
and the celebrated Hc(jek-Inagaki theorem also asserts that as n.+
1
1
A
n~ ( Sn - e)
(1.5)
- 1- (e)~n(S)
Note that by virtue of (1.5),
(1.6)
n
k
2
,
•
s
we also have
"
Sn - e )
(
+ 0 , in P
00
0 , l-l (S) ), as
N(
n +
00
•
In the sequel, any sequence of estimators satisfying (1.5) will be tented
(in the usual sense) BAN estimators of e •
Let p(a,b) be a zretric defined on
sequence
(1.7)
of estimators of
P { p(T ,e) <
S
n
-
the estimator T
n
rJ?X#,
arrl let {T } and {T*} be t\o.o
n
n
e • If then for every T* ,
n
* e) }
p(T,
n
~
1/2 ,
is tenred the a ZOBest eBtimatO:l'
of e , in the sense of
Pitman (1937) .{Tn}is tenred an asymptotiaaZZy aZosest estimato:l' of e if
(1.8)
lim inf
n-+«>
P {
e
< p (T * , e)}
p (T , S)
n
-
n
> 1/2, for every other {T* }.
n
The object of the present study is to show that for
general class, the BAN estimators
In this
p=l) ,
T*
n
belonging to a
in (1.5) are asynptotically closest •
context, we may remark that for the unipara:meter case (i. e., for
p (a,b) is isatDrphic to
a-b may be chosen for p(a,b).
Ia-b I.
we
However, for p > 1, various nonns of
have chosen particularly the MahaZanobiB
no:rm involving the infoY'Tflation mat:l'i3; 1 (e) , i.e.,
(1.9)
P (a,b)
=
I
(a-b) (1 (e» (a-b) •
'Ihis nonn enables us to inco:rporate
~
probability inequality (on the
ordering of two quadratic fonns in correlated nonnal variables ) in the
derivation of the main result. As such, this inequality will be considered
first in section 2. The main "result is then presented in section 3. sate
general remarks are also listed there.
•
-3-
this section is the following.
I
"
Theorem 2.1. let U = (U1'U
)
2
•
be a (2p-) vector having a nultinonnal distribu-
tion with rrean vector 0 and dispersion matrix
(2.1)
E = (Ell
L2l
12
L )
L
with
22
L = L
12
II
positive definite (p.d.),
L
positive semi-definite (p.s.d.).
- L
ll
22
Then,
>
(2.2)
where the equality sign holds only when
1/2 ,
E
= L
•
22
ll
Proof. There exists a non-singular B such that
(2.3)
-1
B'ElIB
= Ip
and
B'L
-1
B = L
22
= Diag(
i , ••• , i ) ,
p
l
l<;<p~
~-
where i. is the jth characteristic root of EllE-2l2'
J
0 <i
<... <i
-p-
< 1.
- l -
letting U. = BV., j=1,2 and noting that V = (V~,V;)' has a nonnal distribution
J
J
with null rrean vector and dispersion matrix ( IP
L~l ' we have
•
I I)
(2.4)
I'::' '
= p{ VlV
VV
2 2
l
P*
EI p{ V V
=
•
I
1 1
= v,
Now, given V
2
P
}
< vlv
-
IV
2
= v}
J.
VI is normal with rrean Lv and dispersion matrix I - L ,
p
so that V~ (I - L) - ~1 has the non-central chi squared distribution function
p
(d.f.) , H (x;
p
(2.5)
~
v
fj, )
v
,
with p degrees of freedan (DF) and noncentrality parcnreter
= v'L(Ip -L} -1Lv
< il(l-i }-1v'Lv =
l
-
where the equality sign in (2.5) holds only when
*
~
v
i
l
, say,
= ... :::
i
p
• Also, oote
that
(2.6)
•
V~Vl
=
V~ (Ip-L) (Ip-L}-~l ~ (l-il}V~(Ip-L}-~l
'
where the equality sign holds when the characteristic roots are all equal.
Therefore, we obtain from (2.4) and (2.6) that
(2.7)
P*
~ EI
=f
p{
H (
p
V~ (Ip -L) -~l .::.
(l-il}-lv'v
I V2
= v } J
(l-il}-lv'v~ fj,v ) dP(v) ,
with the equality sign holding when the
i. are all equal.
J
-4O
d.f. H (x;O) (= H (x), say), and
p
P
(2.8)
tll-~;V2 =
(1-
(1- tIl -~;IL-~2
(with the equality sign holding for
every
(2.9)
x'~x'::'Oand
Hp(x'; 0)
~
0' >0
t
~ t~l (1-
= tp
= •••
l
tIl -~;LV2
l, on noting that for
•
,p~l,
Hp(X; 0)':: Hp(X;O') ,
we conclude that
(2.10)
where we have made use of the identity that
(2.11)
Hp(Yi a)
= exp(-a/2l
Lr>O (a/2)
r
(r!l
-1
a
Hpt2r(Yl,
\I
y .::. 0 and a
2:
0,
and , where in (2.10), the equality sign holds when all the t. are equal.
J
Writing Y = x/(l-R. ) and following sane standard steps, it follows that the
l
right hand side of (2.10) is equal to
(2.12)
(1-t )P/2 Lr>O
l
t~ {r! {P/2' 2r +p / 2 }-1 f~ e -Y/2yP/2+r-~~2r (y/tl)dH~ (y)
r,
'
p/2
1 ,~
00 0
0
_
_
- (1 t l )
Lr>O t l ( , p/2+r /r. 1P/2 ) f 0 Hpt2r (y/t l )dHpt2r (y) •
~ 1, so that ~2r (y/t l ) .:: ~2r (y),
Now, by the hypothesis of the theorem, t l
for every Y
(0,00), where the equality sign holds when t
E:
00 0
a
f 0 H2r+p (y)dH2r+p (y)
= 1/2,
l
=
1, while,
for every p .:: 1 and r ~ O. Thus, the right hand
side of (2.12) is bounded fran below by
(2.13)
(1/2) (1-t )P/2 Lr>O
l
t~
(\ p/2+r
where the equality sign holds when
t
l
1
/
=
r! , p/2 t)
= 1; in the ultimate step, we have made
use of the well known identity that (I-a) -p/2
= Lr>O
a
Since the equality sign in (2.10) holds when all the t
lower bound in (2.13) holds when
t
l
1/2,
r
j
, p/2+r' / (r! rp/2' ) •
are equal, while the
= 1, the proof of the theorem is canpleted
by noting that p* .:: 1/2, where the equality sign holds when t
i.e.,
L
ll
=
L
22
•
l
=
... = t P = 1,
.
-5-
passing,
In
-1
L
..
, where
\ve
may remark that in (2.2), we may replace
L is any convex ccmbination of L
ll
arrl L
22
-1
-1
L
by L 22 or
ll
' and the inequality
p* .:: 1/2 (along with the condition for the attainroont of the lower bound )
re:nain in tact.
3. Pitlnan-closest characterization. we consider here BAN, est.irrators of
~ ----~---------------------------"for which (1.5) - (1.6) hold, though 8 need not be the role. Also, let T be
n
the class of all est.irrators {T } , for which
n
n 1,
(3.1)
(~:(~) e)
~
N2p
l
0
,G
p
~1e»)); EI le)-I p = p. s.d.
Tnen, we have the following.
'lheorem 3.1. For
p (a,b) defined by (1.9) and for every {T}
n
l~ inf
(3.2)
n .+
P { P(8 ,8)
8
n
00
where {8 } is any BAN est.irrator of 8
•
, and the equality sign in (3.2) holds
n
when LI(8)
=
I
T,
~ P(Tn ,8) } ~ 1/2 ,
A
•
£
•
P
Proof. Note that by virtue of U.5) and (3.1), for any BAN est.irrator 8
A
n
T belonging to the class
n
(3.3)
n!z(Sn -
8) _
Tn - 8
T,
N ( 0
2
P
and
/1- (8)
1
\Cl(8)
'Iherefore, the proof follows directly by appealing to Theorem 2.1 [ along with
the Sverdrup (1952) theorem, which justifies the asyrrptotic treat::Irent], and
hence, the details are anitted.
As has been noted in Section 2, in the definition of p (a,b) in (1. 9), one
may replace
1 (8) by
L- l o r any convex canbination of
1 (8) and
L: -1. However,
the choice of 1 (8) appears to be the Il'Ost natural one ( in view of the basic
asstmption that for T
n
£
(Le., p=l), one may take
T, L1 (8) - I
p (a,b)
p
is p.s.d.). In the unipararreter case
= Ia-b I ,
as was originally considered by
pitnan (1937).
'1'0 appreciate fully the scope of this characterization, it may be
quite
-6-
appropriate to, examine the regularity conditions concerning the class
of estinators. Note that if T
is an unbiased estimator of a , then
n
1 (T - a)L
(3.4)
n
n
-r
(a)d~
= 0 ,
n
so that if differentiation (with respect to the elem:mts of a ) is permissible under integration ( with respect to the prcxiuct rreasure
(3.5)
I
p
{-I
L (a)d~ } +
n
n
In~(Tn -a)~n (e)Ln (a)d~n
!.:
'lherefore, the actual covariance of n 2(T -a) and
n
In fact, the unbiasedness of
If ET
n
= b n (8) + e ,
resPeCt to
(3.6)
e)
~
~),
n
we have
\I
= o.
(a) is the identity matrix I •
n
p
is not very crucial in the asyrrptotic setup.
T
n
and the bias (vector) b (8) is differentiable (with
n
and the pxp matrix
(a/aa)b (e)
n
converges to a null matrix, as
n
-+
00
,
then the right hand side of (3.5) , instead of being 0, will also converge to
1
a null matrix, and hence, the covariance of n~(T -a) and ~ (e) converges to
n
I , as n
p
-+
00.
Also, for any unbiased est.i.Iretor
n
T
of a , it follows fran
n
a general result in Rao (1973,p.265) that nE[ (T -a) (T -a)'] n
n
C l (a) is
p.s.d., and under (3.6), the same result extends to possibly biased estimators
in an asynptotic setup. Hence, the two conditions an the covariance matrix of
the asynptotic nultinonnal law in (3.2) hold under the usual regularity conditions Pertaining to the validity of (1.5). However, to justify these two
regularity conditions, it is not necessary to i.rcp:>se the existence of the first
and secorrl order I'lDYel1ts of {Tn} • It may be enough to assurte that there exists
a sequence {T* } of randan vectors and a sequence {u }of stochastic vectors,
n
such that
(3.7)
n
Un converges in probability to 0 as n
n!.:2( T - a)
n
= Tn* +
U
n
-+
00
,
and
where ET* exists and ET*T*' satisfy (3.1).
n
nn
Indeed, a representation of {T*} in teI.Ins of independent sum:narrls leads to an
n
easy avenue towards the verification of the joint nonnality in (3.2). Hence,
we proceed on to make further carments on such a representation.
•
-7-
Rajek(1970), Inagaki(1970,1973) and others oansLdered a general class
*
~n (8)
of estimators which may be presented in tenns an estimating function
n -~l:i~l n (Xi' 8)
•
•
respect to
If
T
n
where
=
A (8) = En (Xi' 8) has continuously differentiable (with
8) elarents and the n(X. ,8) have a finite, p.d. covariance matrix.
1
be a system of solution of this system of equations, i.e.,
~
(3.8)
* (T)
n
n
0,
+
in Pe
'
then, by appealing to the Hcljek-Inagaki theorem, we conclude that under quite
general regularity conditions,
(3.9)
where
n~
(T
n
- 8 )
A-l(e) ~*(el +
= -
n
0
p
(1) , as
n
+
00,
(in P8l
A(8) = ( a/a 81 A(8) is assurred to be of full rank (p). In Particular, this
representation applies to the estimators obtained by the method of least squares
as well as Irethod of m:m:mts. r-bre exmronly, the M-estimator's [c.f. Huber (1981)]
also satisfy (3.9) under quite general conditions ; we may refer to
01.8) and
JureCk.ova
sen
(1984) for sate specific ncdels. For ranked based (i.e. R-)
estimators, though the estimating function is of a fo:rm different frcm
an
asyrrptotic representation
(3.10)
(1981,
!-::
n 2 (Tn - e ) =
where Ee g 8 (Xi) = 0 and
n
-~
~
* (e),
n
similar to (3.9) holds, i.e., we have
n
Li=l
ge (Xi)
+ 0p (1) ,
ge (Xi) has a p.d. covariance matrix for which (3.1) holds.
We may again refer to Sen (1981, 01.8) for sate specific rrodels. For estimators
based on linear functions of canbinations of order statistics, an asynptotic
representation similar to (3.10) holds under very general regularity conditions
[ see C1apter 7 of
sen (1981) J.
Note that the first tenn on the right hand side
of (3.9) can also be expressed as in the first te:rm on the right hand side of
<3.10). Hence, for all such estimates, we may interpret (3.10) as an asymptotic
•
representation in tenns of average over iOOependent sumnands. we denote by
T
o
the entire class of estimators for which (3.10) holds , where the dispersion
matrix of the ge (Xi) satisfies the condition in (3.1). Note that for any estimator
,
Tn
belonging to To ' ( ge(Xil, (a/ae)log f8(Xil ) , i=l, ••• ,n, •• are i.i.d.r.v.,
-8-
so that by virtue of the classical central limit theorem (in the multivariate
case) and (l.l), (1.2) and (3.9), we conclude that the asynptotic (joint)
nonnality in (3.1) holds. Thus, we have
To C
T
, and this leads to the
•
following.
'Ibeorem 3.2. ri'he asyrti>totic PitInan-closest characterization in 'Iheorem 3.1
holds for the class of estimators ( T ) for which an asyrrptotic representation
o
as in (3.10) holds.
In a finite sanple setup, ITOstly, for location and scale Parameters,
Pitman (1937) characterized the closest property in tenus of sufficient
statistics (assuming the existence of the latter), while, the current results
provide srn'e extension of his characterization in an asynptotic setup where
the existence of sufficient statistics is not required , nor
a location/scale Pararreter
e
be necessarily
; the asyrrptotic sufficiency of mle or BAN estimators
provides the link in this context.
we conclwe this section with a remark that Kaufman (1966) , Hajek(1970)
and Inagaki(1970,1973) considered the 'closeness' of an estimator in the
following sense: Let S be any syrmetric (al:x:>ut the origin) and convex subset
in
If
(3.11)
and let C be a class of estimators of 8 , such that
lim
~
~Pe{n
n~
for every {T }
n
£
A
(e
- 8 ) £ S}
_>
k
2
lim
n-XlOP e {n (Tn - e ) £
S}
n
C • 'l"'hen, {(~ } has the asyrrptotic closest character
n
to tne class C ). The ~jek-Inagaki theorem characterizes the class
which the mle enjoys the property in (3.11). '!heir
the same, though for both of them,
r1 (8) - I
p
,
relative
C for
C and our T are not strictly
is p.s.d., and this, in view
of the mUltivariate eran£r-Rao inequality, is not a very restrictive condition.
While the proof of (3.11) exploits the basic Anderson (1955) inequality, we
are not in a position to use the sane'directly in the proof of our theorens.
HOINever, our 'l"'heorem 2.1 provides a visible and directly verifiable proof
of
the Pitrnan-closest characterization in the asynptotic case.
.,
[1]
Proc. Amer. Math. Soc. 61 170-176 •
•
..
Anderson, T.W. (1955). The integral of a syrmetric \IDinoda1 f\IDction.
[2]
Hajek, J. (1970). A characterization of limiting distributions of regular
estimates. Zeit. Wahrsch. verwo Geb. Z4 1 323-330 •
[3]
Huber, p.J. (1981). Robust Statistics. New York: John Wiley.
[4]
Inagaki, N. (1970). On the limiting distribution of a sequence of estimators
with \IDifonn property. Ann. Inst. Statist. Math. 22, 1-13.
[5]
Inagaki, N. (1973). Asyrrptotic relations between the likelihood estimating
f\IDction and the maximum likelihood estimator. Ann. Inst.Statist.Math. 25,1-26.
[6]
Jureckova, J. (1984). Robust estimators of location and their second order
asymptotic relations. Internat. Statist. Rev. {CentenniaZ VOZ.)I in press
{7]
Kau:f:m3n, S. (1966). Asyrrptotic efficiency of the maximum likelihood estimator. Ann. Inst. Statist. Math
[8]
0
Z8 1 155-178.
Pitman, E.J .G. (1937). The "closest" estimates of statistical pararreters.
Proc. Cambridge PhiZ. Soc. 33, 212-217.
[9]
Rao, C.R. (1973). Linear StatisticaZ Inferenae and its AppZiaations. New
York: John Wiley.
[10]
Sen, P.K. (1981). SequentiaZ Nonparametrics: Invarianee PrincipZes and
StatisticaZ Inference.
[11]
New York : John Wiley.
Sverdrup, E. (1952). The limit distribution of a continuous function of
random variables. Skand. Aktuar. 35, 1-10.