ASYMPTOTICALLY EFFICIENT ESTIMATORS OF LOCATION AND SCALE
PARAMETERS IN THE TWO-SAMPLE SEMIPARAMETRIC
LOCATION-SCALE MODEL
by
Byeong U. Park
University of North Carolina at Chapel Hill
November 1987
ABSTRACT
We observe X1"",X m and Y1, ... ,Y n where Xi's are Li.d. with density g and Yi'S are
independent of Xi's and Li.d. with density
asmptotically efficient estimator of (b..,p).
those of Park (1987) in two points.
g«
.-b..)jp).
This paper presents an
The results of this paper are different from
At first we consider unpaired two samples with
different sample sizes, whereas Park (1987) dealt with paired sample cases. Secondly, we
do not assume location-scale structure in Xi'S, while on the other hand he considered the
case where Xi'S have that structure. It turns out that having location-scale structure in
Xi's or not, does not make any significant difference in terms of efficient score function but
one may prefer our consideration when one is not quite sure about the structure of the first
sample.
AMS 1980 Subject Classification: Primary 62G05, Secondary 62G20.
Key words and phrases: Semiparametric location-scale model, asymptotic lower bound,
asymptotic linearity, efficient score function, efficient influence function.
~
.....
ru
'0
-1-
1. Introduction
We observe X1, ...,X n ; Yl'''.'Y n where Xi's are LLd. random variables having a
common density g with respect to Lebesgue measure and Vi's are independent of Xi's and
i.i.d. with a common density g [. p-~]. Our interest is in estimating the parameter
with minimal conditions on the density g.
efficient estimator of
(~,p)
(~,p)
In this paper we present an asymptotically
with minimal conditions on the density g.
A slightly different problem is as follows: Xi's and Vi's are the same as the above
except that now g( .) is of the form go [ .a-It], in other words, Xi's have another common
location-scale structure.
This problem has a long history since Stein (1956) first
investigated that one can estimate
(~,p)
as well with go unknown as with go known at the
presence of unknown nuissance parameter (Jt,a), so called, adaptation is possible.
Some
earlier works of this sort include Weiss and Wolfowitz (1970) and Wolfowitz (1974). But
they put too many conditions on the density go.
A completely definitive result was
obtained by Park (1987) for the case m=n where he used only the minimal conditions on
the density go' which is finite Fisher information, to get asymptotic efficiency. Problems
with location shift only were considered by Bhattacharya (1967), van Eden (1970), Beran
(1974, 1978) and Stone (1975).
Of course with the model described in the first paragraph in this section, adaptation
is not possible because the score function for
(~,p)
is not orthogonal to the nuissance
parameter scores (scores for g). But the efficient score function for
(~,p)
(for definition see
Begun et al. (1983) or Bickel et al. (1986)), which determines the difficulty of the problem,
is equivalent to that with the model described in the second paragraph where Xi; Li.d.
go [. a-It], Vi; Li.d. go [. a;~ -
i] up to constant term which depends on Jt and a as we will
see it in Section 2. Hence in terms of difficulty of estimating
(~,p),
the two models are
closely related. But in this paper we deal with the first because we may encounter the
situation where we do not know even the structure of Xi'S.
-2-
Briefly speaking, our estimator of (~,p) will be constructed in section 3 as follows:
First we find an initial est.imatof'~~N'~NfWhichmakes us be able to locate (~,p) with rate
of convergence Nt, then we add an estimate of the efficient influence function to that. In
fact, our estimator turns out to be asymptotically linear with the efficient influence
function.
One final note is thatt
w~e a.void considera.ble technical details in proving our main
.
:.1,,,
theorem because they can be.referred to Park (1,987), instead we will sketch the proof in
section 4.
2. Preliminaries
First we start with stating the conditions we need.
(Cl)
m
m+n
-.,"
~
i
:
p
as 'tn, n
~
e"
".- \'
00 .-' ,
Note that our model can be molded t6,one' s~p:le'modelby introducing a random indicator
variable Z such that P[Z=l]=p: ,Hence, we obse~ve LLd. (X1,Zl)' ... , (XN,ZN) (N=m+n)
~
D~::"PIJ8Z-j '3;r1~~-;-:UJ.~t:;·
"
~j "'\
with density function f(x,z,~,p,g) with respect to the product f.J- of Lebesgue measure v and
; 1,\:1' J)!lmYio;':'/
. ,7~, ; ' ,
counting measure on R 1 x {O,l}, where
. ' "1 j'~. " bql':l tt .. .• "r; ..·.
J..,
'-'
~U
r
_ ....... :J.L
f(x,z,~,p,g) = {~g[X;~J
•
I -,
.
\.J.JI.,...~_~J\S,.JJ:'
{g(X)}l-Z p(z) ,
p(O) = p and p(l) = I-p = q.
-3-
2.1 ..jN - consistent estimate
which satisfy
'l).'[i
j[j
'
•
0N)"= !(~N,PN)
ili~!;~,')
J
i
.1 - '
(2.1)
-
IN -eonsist~it'7stimate
At first we need to construct
= (~,p)
Jl i 'jr[:
J
'11
.
of ()
(1)
N2((}N - (}) = Op
(},g
where P (},g is the probability measure having density I( '~g) 'with respect to the product
measure p. Let ?jJ(x)
= 2x/(1-x2)
and '~(x)
= x?jJ(x)-i""
arid define the M-€stimate ON =
(~N,PN) corresponding to 'I/J and X to be any solution of
(2.2.a)
(2.2.b)
(2.3.a)
EHx;t>] IZ=lL
(2.3.b)
E
~[~~I,~;-~];
\
hUR
[x [x;~ JI z=11;:Z:1£{xt Xi) Iz#:0lJ fJ~;r'(!"r: ",
•
has a unique solution ()
=
(~,p)
)
"
-
\~..
.,
~~-'
(;
!y'.'
I:';.) fJ.~~
i1 .~i ~..::.:
:.-
:J
<.
l-.'~
and the estimate defined through the equation (2.2)
~(.,; 1-."~
-}
,"'·':,1 dJf·'.~.-
(:!::>.\.
"
converges to this solution with rate N2. By Maronna (1976) these two requirements are
"rlL\rl'If {r"·
met with our 'I/J and X defined above (actually we need a simple two-sample generalization
_l~.\.
"..
of the results of Maronna (1976)).
,-':
~;:'
"
yy
~".
. L-y)
,
I
__ . j
r
.
-4-
2.2 Efficient score function
Let
'# be
the collectibn
of~U probability densities on R l with respect to Lebesgue
measure v and set"~('':T,{Ib,g,1 g. €:,1lLthe collection of all probability measures P (J,g'
which has density f.C·I,~g) :~ith: respect:~9. At; with g varying over
defined in Begun et al. (1983), an~,Bi,ck~1
estimating
(J
ert,¥.
x=
is the orthogonal component of· the score function for
(x-~)/p.
tangent space of
~,
As
P98~), the efficient score function for
nuisance parameter g::~ Now the score filli~c,ti~n ·for 8, :denoted by
where
'# and (J = (~,p) fixed.
l
(J
= (ll'
to the scores for
l2) T, is
If we let ~ (following the notation in Bickel et al. (1986)) be the
the space of all scores for g (for definition of tangent space see
Pfanzagl (1982) or Bickel et al. (1986)), it can be easily verified that
~ = {z h(X)
+ (1-z)h(x) I J h(x)g(x)
dx = O}.
Now it simply involves a straightforward calculation to get
(2.4) IT UI ~) =
[~{ z ~ (X) + (1-z) ~ (x) }, 7! {z.(X-~ (X) + 1) + (1-z) (x ~ (x) + I)}]
T
where IT(· I~) is the projec~io~ ?f,erat8~ on~. Hence the efficient score function for
*.
'..1
(J,
(2.5)
e\x,z,(J,g) =
'11.',
~:l i"J,J~.rTIs<!J)q f.[11,~)~<'
l-tI(ilJ~)iJnL
;,
=
J.),J.!Y
denoted by l , is
estimating
f; ,!f,t'1
J,!..
..
10'1 ( ,., ~:': '
y[;
(Sd
,G OJ.!
~[(P-l) z ~ (X) + P(I~)i'~ (~),..(P-l)Z{X-~jX) + I} + p(I-z) {x ~ (x) + I}]
T
Remark 2.1. Suppose we observe Li.d. (Xl' Zl)"'" (X N, ZN) with density function
f(x,z,J.t,a,~,p,g) with respect to J.t where
-5-
f(x,z,tt,a,~,p,g) - . ap,g [x-~
ap
- {L
=
p(O) = p, p(l) = 1-p
q and g E
_I!:.J
a}Z
, {l
&? [!:::E:J
~H: }l~Z
,) [jJ>,(Z}'i
,I,
,~;' the coilection bf..~tIdensitiesg with respect to
Lebesgue measure v which make (tt,a) identifiible.Tllen'· after':'going through a simple
calculation we arrive at the following effident'score:fUnction.
.'
i
1
*
(2.6)
f (x,z,O,g)
x=
where
(x-tt)/a
=P
1
a
I!:.
a
and
x
=
(x-~)/ap
- tt/a.
Comparing these two efficient score
functions in (2.5) and (2.6), we can see that they are essentially the same except the
constant matrix which depends on (tt,a). Hence if terms of the difficulty of estimating
(~,p),
the two models are closely related as one can expect it from their structures.
t-- {"I" .;.
"
J
2.3 Asymptotic lower bound
Let
'n
* *T
, ."... ,', i:. ( ,:c!
f f (X,Z,~,g).·
\-
(2.7)
1* = E p
•
I
'
..
Ii .;..
O,g
Then by Theorem 3.1 of
Begun:'~~t ~'. (198;31)',!"':;hic~;)i~)'~
extended version of the
representation theorem of Hajek (1970) for semiparametr'ic' cases and a special case of
proposition 9 and 10 of Le Cam (1972), the limi,~:q:a~ )0£ every regular estimator (for
definition see Begun et al. (1983), or Bickel et al. (1986)) may be represented as the
._,
i
.
.
:g
convolution of N(0, 1; 1) with,' :some 'distiibutidn~wlliichhdepends
on (O,g) only.
Also
Theorem 3.2 of Begun et al. (1983), which is a special case of Hajek-Le Cam-Millar
asymptotic minimax theorem (see Hajek (1972), Le Cam (1972), (1979) and Millar (1979,
1984)), states that E f( Z*) is the asymptotic minimax bound for all "generalized
-6-
procedures" (see Millar (1979))'lV:here l;,R2~ R+ is a bowl shaped loss funciton (Le. f(x) =
f(-x) , {xlf(x) ~ c}
Z* - N(O, 1;1).
js c<my~~·Jor ..~~~r~,~: ~,O ¥1d l is continuous a.e. Lebesgue) and
In·£,p~~i~u1Btr:,
if l.'is· the squar~ error loss function, tr(l;l) is the
asymptotic minimax·bw.1P.d. l,',
j \'
\r::'
3. Main Result
Our main goal in this paper is to c<"?Ilstruct an estimate of
asymptotically normal with mean
°
o and variance 1;1 defined in section 2.3.
will be obtained by adding an estimate of the efficient influence function
f
*
= (/:)..,p) which is
-
(Xj,Zj,O,g) to the initial estimate ON'
This estimate
k1;1 ~j =1
First of all we need to estimate the unknown
density g. Let
(3.1 )
where Xi = (Xc/:)..)/p, K(y) = e-Y/(1+e-Y)2, logistic density function and {b , N ~ 1} is a
N
sequence of positive cdn~~S£;d):n~E!rgmgi to! ,zere}"at a rate to be chosen later. Here the
choice of the kernel function K is not critical but it should have some properties described
in section 2, Chapter 1 of Park (1987). Now let
f:::.
'"
(3.2)
{)
hN(x,O)
:" -
7
(¥:
-
,I
= [ax gix,O)]
A.,'
.~
, " d -. /.--, (;
... 1..... ..'1,:
'J.,lLx.
1
\,.'
/ gN(x,O) .
l~:]. ",
.f>'
r: (~:. '·y:l,', ,
As an estimate of g,jg, hN'may have good'properties but as we can see it in the expression
of
*
e
.
.
-
in (2.5), h N also should serve as an estimate of x(g/g)(x) in the form of x hN(x,ON)
which might be (actually turned out to be) uncontrollable even if h N has good properties as
an estimate of g/g. Hence we rather use a smoothly truncated version of h N defined by
-7-
--l 00 at a rate to be chosen later, b( . ) is a'symmetfic 'function such that b(x) = 1 if
N
o ~ x ~ 1, c(x) if 1 ~ x ~ 2, 0 if x ~ 2 andc(x) is alsfuoofh ftiiio~n such that c(l) = 1, c(2)
where c
= 0,
c'(I)
0, C'(x) ~r'O, Ic~(xYl;~;Hi"'and Icw(x) I ~ H2 for
= c'(2) = c-(l) = c-(2) '==
some positive numbers H l and H 2 whenever 1 ~ x ~ 2. This sniddth truncating function
b( .) has been used in Park (1987).
.*
Let l (x,z,O) be an estimate of the efficient score
*
.
-
function l (x,z,O,g), defined by replacing (gig) with ~(.,ON) and p with miN in the
expression of l * in (2.5). Namely,
(3.4)
.*
l (x,z,O)
I
x= (x-~)I p.
'~
.
(C
'.'
',','
itl"
r
n . -'.
--
= P [-N z hN(x,0N.).+ NP-Z[~.hN~~~rON~' -N z{x hN(x,ON)
+
where
n ..:... -" m "~ . , ,'
1
N(l-z){x h~(x',ON) + i}]T
With 1* defined by
(3.5)
Here is our main theorem.
(3.7)
l-
in PO-probability for any sequence (ON,gN) such that, IN 2 (ON-O)-t I --l 0 and
' . , I .n
•
N,gN
,'0.YI
\... 1'(".r
, \
c"t,
and
II .II v is the usual L2( v)-norm.
I)
Note that
(3.8) .
1
1* = - p(l-p)
p2
I
I
l,g
2,g
-1
] = p(l-p)I**
T
+ I}
-8-
= J x a jg dx, a=a;1,2. 'Uenceincorporating with the condition mj(m+n)
a,g
an equivalent statementito(-3;7) can be writteil:. as follows; Ji:
g2
where I
.'1
'l~"
)
/
p,
f /
~i
j1r (ON - ON)
(3.9)
-1
N(O,I;1) .
-+
\
."
• -I,d
The statement in (3.9) is more familiar to us than the statement in (3.7) in two-sample
problems.
Remark 3.1. It is an immediate consequence of Theorem 3.1 that under the condition
of Theorem 3.1, we have
for 0 < a < 1 where
x2(1-a)
is the 100(1-a)th percentile of chi-square distribution with
degree of freedom 2. As discussed in Park (1987), the confidence region based on (3.10) is
..
regions for
°
based on
. I-
.
re~ular es~imates
.
,
lias tlitsmallestafea asy,r;nptotically among all the confidence
optimal in the sense thltt'it
!,\j
It:'
(1-a).
with the correct asymptotic coverage probability
J - ' .• I t ; : '
'~ ,../;_
(
Our proof of Theorem 3.1 is based on Lemma 3.1.
; ..:
Before stating the lemma, we
i .--~
introduce an important property of estimates, namely asymptotic linearity.
.
:?'--:q.:::"';':':::
;~~
.
Definition. An estimate ON is called "a4ymptotically linear at (O,g)" if
n
11\1
°-
tAl
N (ON -
-1
N
"'-ii ~'<·r'
*
N 1* b j =1 l (Xj,Zj,O,g)) = op (1)
O,g
where i * and 1* are defined in section 2.
Lemma 3.1. If ON is asymptotically linear at (O,g), then the conclusion of Theorem
3.1 is true.
-9-
Proof. The lemma follows itmhediatelyfrom Le Cam's third lemma (see Hajek and
Sidak (1967) Chapter 6).
A quick alternati've"proo£ can be;:givenras follows:
J.
A
From the
A
asymptotic linearity of ON it is easy to see that (N2 (ON-(J), AN) converges weakly to Z with
. ',)'
,,/'
o
J.t= [-t
for some
(3.11)
02 >
02]'
t]
[1;1
E=
2
T
t
(J
J.
A
O. The characteristic function of N2(ON-ON) underJ~O
is
, N,gN
T
exp[i u Nt(ON--:0N)] = E p
Ep
exp[i u
0N,gN
T
N~(ON-O) -i u T t] + 0(1)
0N,gN
,
it.;
C',<
•
=ERO;f~p'l;l~Ti,l;j~t?~?,(J)i+A':N T i)~~t]+ 0(1) .
"
By considering almost surely
~:,'\
I
,ff
~~
2:
.r""::."
1
,(:'::"'::.:'1'.)"
't ','
-J!.-:-.:'
.~~.
convergentv~r;i~ns "~t6:~t'(e~':"ot A~)
and Z as in Beran
(1977), the characteristic function in (3.11) converges to
;
(3.12)
d,.l,~"'}
T
E exp[(i u ,l) . Z] exp(-i uTt)
n('~
'/ 1.:n.. ~~
If'''-{-',:
,;.
(:,~.~ ·"-.)~TI· ,:f~'')
T
by Vitali's theorem. Since E exp[(i u ,l).Z] = exp[-t
,
"'(U'J'
<;.
11
, ;.
1-: ., "'lS(.: ,Y.;
.v
1. :.:::
"\.,
.
'.
+ i uT t],
T
,
)
'
4. Proof of Theorem 3.1
:-f.f9'~~.-,;·-1
U I;1U
!)7)lLEJ (,: ,,'\J .?
is equal to exp[-t uT 1;1 u]. This concludes the proof.
'~-I \~~
: _*.1
}
the term in (3.12)
.
..
We avoid technical details here because the corresponding results can be easily
derived from Park (1987) by putting J.t =
Jtn =
0 and a =
an =
1 there.
-10 -
,
Sketch of Proof. By Lemma 3.1, it is sufficient to show that ON is asymptotically
linear at (O,g). Put
!",
,
Note that I( ON) = 1* . andL::
,;:;
,11 N
= { I+I- (ON) N jE
+
f)
1
'*
7J7) l
(X j ,Zj,O)I8=u
}
N
[i-1(ON) - i- (O)] N-t EJ=1 l* (Xj,Zj'O)
1
, 1
-.l. N
'*
*
+ 1- (O)N2~j -:-1 {l, (Xj,Z,j'O) -l (Xj,Zj,O,g)}
,
-.l.
*
+ (I-1(O) - 1;1) N 2 E =1 l (Xj,Zj,O,g) .
..
Now it is easy to see
J
r · · , ·
th~ all
'-,
I
(
the. ~etIDs al>Qve;go to 2~O if we show
.. - .....
~.__
\'-
\,
"'
(4.1)
(4.2)
and
,
(4.3)
sup
I0'-01 ~M
I1( 0') -
,
1( 0) I ~ 0
in PO-probability.
We can easily derive
,g
(4.4)
,',
."
.1.-
N2(ON- O)
-11-
and
(4.5)
-..I.
N
2
'*
*
E
{f (Xj,O,D) - f (Xj,O,D,g)}
{Z.=O}
J
'
"
-I
0,
' ...... , ;.1 _
i\-'"
\
.
using the arguments in Lemma 3.1 - 3.5 in Park (1987). (4':1) fOllowS' from (4.4) and (4.5).
Now using the same arguments involved in the proofs of Lemma 4.1 '- 04.11 and Theorem
4.1, 4.2 in Park (1987), we can get
f'
(4.6)
•
!
_
' -.
{)
{Zj=l}
p~.
~
and
(4.7)
'.
2
"
- NEon f'* (Xj ,I,D) = -1'~
(p-l) P 1** + op
1
~g
f
(1)
I;
1
-N
E_
{Zj-O}
{) '*
1
on
f (Xj,O,D) = 2
-,
2
P (1~p)I**:- op (1)
p,:.
D,g
where 1** is given through the equation (3.8). Putting (4.6) and (4.7) together leads to
.
,
(4.1). Finally using the arguments in the proof of 'Theorem 4.6 in Park (1987), we can
"
easily obtain
~)
J :::::-
!.
(4.8)
and
(4.9)
Now (4.3) follows from (4.8) and (4.9) immediately. _
-12 -
( , i
, ,
'\0
•
•... '.'
I"
.. .I.
l'j
References
.-.
"
Begun, J.M., Hall, W.J., Huang, W. and Wellner, J.A. (1983).
Information and
asymptotic efficiencyd'tf'OpaDametric-nonparametric models.
Ann. Statist. 11,
432-452.
Beran, R. (1974). ,i.Asymptotically efficient adapVive tank .estimates in location models.
\
Ann. Statist; 2,' 6J";74.~,
Beran, R. (1977).;,; Robust location estimateS'j'Ann. Statist. 5,431-444.
, ',:" j.'~ I
'
~('.L,
it.. '-,"; ":
Beran, R. (1978). An efficient and robust adaptive estimator of location. Ann. Statist. 6,
292-313.
Bhattacharya, P.K. (1967). Efficient estimation of a shift parameter from grouped data.
Ann. Math. Statist. 38, 1770-1787.
Bickel, P.J., Klaassen, C.J., Ritov, Y. and Wellner, J.A. (1986). Efficient and adaptive
inference in semiparametric models. Preprint.
Hajek, J. (1970). A characterization of limiting distributions of regular estimates.
Wahrsch. verw. Gebiete 14, 323-330.
Z.
Hajek, J. (1972). Local asymptotic minimax and admissibility in estimation. Proc. Sixth
Berkeley Symp. Math. Statist. Probab. 1, 175-194. Univ. of California Press,
Berkeley.
Hajek, J. and Sidak, Z. (1967).
Academia, Prague.
Theory of Rank Tests, Academic, New York and
Le Cam, L. (1972). Limits of experiments. Proc. Sixth Berkeley Symp. Math. Statist.
Probab. 1, 245-261. Univ. of California Press, Berkeley.
Le Cam, L. (1979). On a theorem of J. Hajek. In Contribution to Statistics; Jaroslav
Hajek Memorial Volume, Ed. J. Jureckova, 119-135. Reidel, Dordrecht.
Maronna, R. (1976). Robust M-estimators of multivariate location and scatter.
Statist. 4, 51-{)7.
Ann.
Millar, P.W. (1979). Asymptotic minimax theorems for the sample distribution function.
Z. Wahrsch. verw. Gebiete 48, 233-252.
Millar, P.W. (1984). A general approach to the optimality of minimum distance
estimators. Trans. of Amer. Math. Soc. 286, 377-418.
Park, B.U. (1987). Efficient estimation in the two-sample semiparametric location-scale
model. Part of Ph.D. Thesis, University of California, Berkeley.
Pfanzagl, J. (1982). Contribution to A General Asymptotic Statistical Theory. Lecture
Notes in Statistics, Springer, New York.
-13 -
Stein, C. (1956). Efficient nonparametric testing and estimation. Proc. Third Berkeley
Symp. Math. Statist. Probab. 1, 187-196. Dniv. of California Press, Berkeley.
Stone, C. (1975). Adaptive maximum likelihood estimators of a location parameter. Ann.
Statist. 3, 267-284.
1~.
r
~
~
<
van Eden, C. (1970). Efficiency-robust estimationi'of looation.·· Ann. Math. Statist. 41,
172-181.
Weiss, 1. and Wolfowitz, J. (1~70). Asymptotically efficient nonparametric estimators of
location and scale parameters. Z. Wahrsch. verw. Gebiete l-6~:~134-J50.
Wolfowitz, J. (1974). AsymptoticaJly efficient nonparametric es.timMors of location and
scale parameters, II. Z. Wahrsch. verw. Gebiete 30, 117-128.
\
t
-;
"
",.:,,-:
T ..
"":"'J,.fl
_:';/{~~:(1
':~·',~·i.·~f>.··:
EJrr .t;
~:t
~
~~}
'\_~
u)·)
~
~)
tt) :,·ti;,~J.~~t
"iIi{
D.~"~!fF:~
!<... 1.
,b~_~f
,L)i
~
,\
[i
nI;
'-'.