of Convergence for the Distance
Distribution Function Estimators
Dennis D. Boos
Inst. of Statistics Mimeo Series # 1390
ABSTRACT
Rates of Convergence for the Distance between
Distribution Function Estimators
The normed difference between "kernel" distribution function
estimators
'"
F
n
investigated.
F
is
n
'"
Conditions on the kernel and bandwidth of Fare
n
II Fn-Fn II ~> 0
suPxlg(x) I and Ll norm
given so that
I Igi I~·
and the empirical distribution function
a
n
as
U-+clO
,for"both the sup-norm
Applications include equivalence in asymptotic distribution of
'"
T(F)
n
T(·) •
and
T(P)
n
"(to order
a )
n
for certain robust functionals
1.
Introduction
The empirical distribution function
= n -ItlI(Xi~x)
Fn(x)
is the
most widely used nonparametric distribution function estimator.
However,
integrals of kernel density estimators form a large class of smooth
competitors.
These estimators may be expressed as
Fn(X) = ~
00
1 {::iJ
J K[~n4
=
dF (y) ,
(1.1)
n
_00
where
K is a distribution function on
"bandwidth."
(-00,00)
and
b
n
> 0
is the
Winter (1973,1979), Yamato (1973), and Azzalini (1980)
have studied the convergence of
of the observations.
'"
F
n
to
F, the distribution function
The purpose of this present note is to give
sufficient conditions on
F, K, and
b
n
so that
a II F -F II ~ 0
n
n n
II.II . Of particular interest is the sup-norm
I IFn-Fn II00 = sup~
-...,...x<00 IFn (x)-Fn (x)1 and a n = n~. Then
n~1 IFn-Fnl loo~> 0 implies weak convergence of n~[Fn(F-l(t»-t] ,
for several norms
the Chung-Smirnov property (law of the
I I Fn-FI 100 )
logarithm for
of Winter (1979), and the asymptotic equivalence in
distribution of
n~[T(Fn )-T(F)]
robust functionals
i.e.,
it~rated
T(·)
IT(G)-T(H)I <
given in Section 3.
and
n~[T(Fn )-T(F)]
for the many
which are Lipschitz with respect to
ci IG-HI
00 •
1
More details and applications are
The main results proved in Section 2 are simple
and rely on the usual integration by parts representation of
'"
F
and
n
Serfling's (1981) generalization of a theorem of Sen and Ghosh (1971)
on the uniform convergence of
t
-+
0 .
F (x+t) - F(x+t) - [F (x)-F(x)]
n
n
as
2
2.
For functions
II G-HII~
G and
Main Results
H on
• sup-co<x<col G(x)-H(x) I
(-~,~)
and
define
II G-Hll l
•
Co IG(x)-H(x) Idx
The sampling situation is
(5)
X , •.. ,X are independent real-valued random variables having
1
n
the distribution function F.
The first two theorems require
(C)
K
has support in a compact interval
[c,d], -co<c<d<co •
Lemma 2.1 of Winter (1979) justifies the use of integration by parts
·e
to reexpress
A
F
n
of (1..1) as
F
THEOREM 1.
Suppose that
the Lipschitz condition
a n bn .!:el> 0
and
n
(x"b y)dK(y) .
n
(5)
(C)
and
IF(x)-F(y)I < Llx-yl
hold and
on
nb n flog n ~>. co, then
an II Fn - Fn 1·1 co
!E.!.;:;
0
as
~.
F satisfies
(-co,co).
If
3
PROOF.
d
a n I IFn-Fn IIClO -< a n sup
x
f IFn (x-bny)-Fn·(x)/dK(y)
c
d
< a n sup
x
f
IFn (x-b n y)-F(x-b n y)-[Fn (x)-F(x)]/dK(y)
c
(2.2)
d
+. a sup
n x
f
IF(x-bny)-F(x)ldK(y)
c
< a sup
-
n x
+ a nbn L
sup
IF (x+t)-F(x+t)-CF (x)-F(x)]1
Itl«d-c)b
n
.n
n
(2.3)
1
IYldK(y) •
c
Lemma 2.2 of Serfling (1981)
a Q .
ReWTite the first term of (2.3) as
n n
yields
·e
wpl
for uniform random variables.
as
n-.co
Following the remark on page 194 of
Sen and Ghosh (1971), this result also holds for all
Lipschitz.
Then
a Q ~> 0
n n
since
F which are
a [b log n/n]~ ,. a b [log n/nb
n n
n n
n
~> 0 •
REMARKS.
The proof shows that the weaker result,
wpl, holds if only
assumed.
The bandwidth
be estimated in most
b
n
a b
n n
,. 0(1)
The usual choice of
(see Section 3) so that Theorem 1 allows
b
n
- n
The next theorem strengthens conditions of
Theorem 2 allows
b
n
- n
b
n
-a.
is
is allowed to be random since
~pplications.
to reduce conditions on
wpl
In particular, if
,a. >
~
•
-a. ,
n
is
must
n~
a. > ~.
F
a
a
b
n
and
n
,. n~
K in order
then
i"
4
Suppose that
THE-GREM 2.
Let
F have derivatives
IlfilOo < 00
with
and
f
and
(S)
and
f'
Ilf'll oo < 00.
(C)
hold and
which exi,s.t
If
a nb n
2
fd xdK(x)
eye~ywhere
on
0 .
=
c
(-00,00)
~> 0 and
nb flog n wpl> 00 , then (2.1) holds.
n
PROOF.
The proof is the same as for Theorem 1 except that the
second term of (2.2) is expanded by Taylor's theorem to yield
a sup
n x
-e
d
d
c
c
I [-f(x)(b ny)~f'(tn*)(b ny) 2 ]dK(y) I
IJ [F(x-b n y)-F(y) ]dK(y) I = a n sup
x J
EXAMPLE.
on [0,1].
Suppose that
F
is the uniform distribution function
Then Theorem 1 applies but not Theorem 2.
Theorems~l
and 2 are similar in spirit to Theorems 3.2 and 3.3 of
Winter (1979) which conclude that
lim sup{2n/loglog n} ~II F -F II < 1
n
00 n-+oo
Here, condition
(2.4)
wpl.
C and a slightly stronger condition on
b
n
than
Winter required yield (2.1) which in turn yields (2.4).
The last theorem in this section applies to
THEOREM 3.
a b
n n
Suppose that
(S)
holds and
II· I 11
•
flxldK(x) < 00.
If
wpl> 0 , then
a
n
II Fn- Fn II 1
~> 0 as n-+oo
(2.5)
5
PROOF.
00
00
a
n
J IFn (x)
- F (x)ldx < a
n
- n
J f IFn (x-bny)
_00
-00
00
_00
00
= a
n
- F (x)ldK(y)dx
n
00
f
f
-00
iFn (x-b ny) - Fn (x)ldxdK(y)
-00
00
= a Ib
n n
I
f
lyldK(y) •
00 -
The interchange of integrals is justified by Fubini's theorem, ahd
the last step follows since
tion function
J:oo[G(x+a)-G(x)]dx
G and constant
a > 0
=a
for any distribu-
(see Chung (1974), p. 49,
probe 16).
·e
3.
A)
Applications
Weak convergence of
bridge.
Choose
a
n
n~ •
=
may be carried out in
Billingsley (1968).
to
If
CeO,l]
If
K
o
W , the Brownian
is continuous, then the convergence
using Theorems 4.1 and 13.1 of
K is not continuous, then the space
nrO,l]
is appropriate and Theorems 4.1 and 16.4 of Billingsley yield the
result.
metric
(In verifying the latter it helps to note that the uniform
p(x,y)
=
tlx-yl
1
00
dominates either of the Skorohod metrics
given by Billingsley.)
B)
Theorem 4.2 of Sen and Ghosh (1971) mentioned in the proof of
Theorem 1 yields for
sup
-«x<oo
sup
Itl~dn
F Lipschitz
n~IF (x+t) - F(x+t) - [F (x) - F(x)]1 wpl> 0 ,
n
n
(3.1)
6
n~dn increases at a rate not slower than that of log n but
where
not faster than that of
with
a
= n~
n
n
Statistical functions
and
C
0 (1),
a
a. <
~
The results of Theorems 1 and 2
•
allow (3.1) to hold with
c)
n
a.
p
then
T(F).
If
F
n
a [T(F )-T(F )] ~> 0
n
n
n
Xi + Y
n
Xi
F
n
IT(F)- T(F ) I < C IIF -F Hoo
n
n
-n
nn
and
have the same asymptotic distribution up to order
extension is to replace
.....
replaced by
T(F)
n
a
and
T(F)
n
A trivial
n
by the perturbed random variable
(e.g., Pitman location alternatives).
Some specific
T(·)
are given below.
1.
Quantile estimation,
Suppose that
and
F'(F-l(p»
I IG-HI 100
>
o.
T(F)
= F-l(p) = inf{x:F(x).~
For distribution functions
p} •
G and
H
sufficiently small, one can verify that
·e
(3.1)
.....
G= F
Letting
Fn-l(p)
n
and
H
=F
in (3.1) and
a
n
= 1,
we get
~> F-l(p) • Under the conditions of Theorem A of
~ilverman
(1978) which include uniform continuity of
F'
and
K' ,
we haveF '(F -l(p» = F'(F -l(p» + [F '(F -l(p» _ F'(F -l(p»]
n
n
n
n
n
n
~> F.-r-(F-l(p». Thus applying (3.1) with G" F and H .. 'n and setting
n
a
n
= n~
yields
n~[T(Fn ) - T(Fn )] -E-> o. Azzalini (1980) has
apparently shown this result directly and also obtained the optimal
rate for
2.
b
n
of the order
n
_~
.
L-estimators with smooth score function,
Boos (1979a), Theorem 1, showed that
with respect to
T(·)
1
< C n~ •
l
T(F) = !F-l(t)J(t)dt •
Theorem 2 requires
b
n
has a Frechet differential
I 1.11 00 under certain conditions on J
and
F.
7
Similarly, it can be shown that if
[0,1]
J
is bounded and integrable on
and equal to zero in neighborhoods of
Alternatively, if
IT
° and 1, then
is continuous on [0,1] (and thus bounded), then
J
(Fn ) ...
T (F )
n
I
Each of these bounds follows directly from the representation
T(G) ... T(H) -:r:~S(H(X»
3.
Other
... S(G(x»]dx where
example~
S(t) .. 1~ J(u)du •
inelude classes of R-estimators, minimum
distance estimators, and one-step M-estimators.
·e
"
T(F)
n
"
because of the smoothness of F
4.
For certain cases
may be easier to handle than: T(F )
n
In Boos (1979b)
n
the minimum value
of
00
[l-F (e+x) - F (e-x)]2dF (6+x)
n
n
n
was proposed as a test statistic for detecting asymmetry in F.
limiting distribution of the Zocation of the minimum
as that of the minimum
T(F)
n
= dF
n
(8(F»
n
e(F )
The
as well
n
is not hard to find.
However, details of proof are made difficult by the fact that
d
F
(8)
n
is a step function.
Consider the following approach.
If
H
is a
continuous distribution function, then simple calculations show that
8
and thus
(3.2 )
First let
S(P ) ~> S(F)
n
proof).
H. ~
G· F and
Then (3.2) and (2.1) implies that
n
(see Lemma 2 of Beran (1978)
Note that consistency of
not shown
16(G)-6(H)
does not follow since we have
a(F )
n
I' ~ cl IG-HI 1
00
for the technique of
However, we can proceed directly
•
A
and find the limiting distribution of
Section 2, for analogous details).
and
H
nT(F)
n
A
= Fn
and choosing
a
n
A
n
= n in (2.1), we get that nT(F)
n
have the same limiting distribution.
only one
A
F
n
(see Boos (1979b),
Then using (3.2) again with
necessary to get useful results since
·e
nT(F)
-+
00.
distribution of
a(p)
n
and
Note also that
meeting the above requirements must be found since
is the statistic of interest.
= Fn
Here, Theorem 2 is
nb flog n
n
G
T(F )
n
Koul (1980) has obtained the asymptotic
using linear rank methods.
9
REFERENCES
Azza1ini, A. (19811)' "A Note on the Estimation of a Distribution
I
Function and Quanti1es by a Kernel Method," Biometrika, 68,
326-328.
Beran, R. (1978), "An Efficient and Robust Adaptive Estimator of
Location:'Annals of Statistics, 6, 292-313.
Billingsley, P. (1968), Convergence of Probability Measures, New York:
John Wiley.
Boos, D. D. (1979a), "A Differential for L-Statist1cs," AnnaZ,s of
Statistics, 7, 955-959.
Boos, D. D. (1979b), "A Test of Symmetry Associated with the HodgesLehmann Estimator," Institute of Statistics Mimeo Series
·e
n 1248,
North Carolina'State University, Raleigh.
Chung, K. L. (1974), A Course in Probability Theory, New York:
Academic Press.
Kou1, H. (1980), "Some Weighted Empirical Inferential Procedures for a
Simple Regression Model," preprint.
Sen, P. K., and Ghosh, M. (1971), "On Bounded Length Sequential
Confidence Intervals Based on One-Sample Rank Order Statistics,
Annals of Mathematical Statistics, 42, 189-203.
Serf1ing, R. J. (1981), "Properties and Applications of Metrics on
Nonparametric Density Estimators," Department of Mathematical
Sciences Technical Report No. 324, The Johns Hopkins University,
Baltimore.
10
Silverman, B. W. (1978), "Weak and Strong Uniform Consistency of the
Kernel Estimate of a Density and its Derivatives,"
Annal.s of
Statistias, 6, 177-184.
Winter, B. B. (1973), "Strong Uniform Consistency of Integrals of
Density Estimators," Canadian Journal. of Statistias, 1, 247-253.
Winter, B. B. (1979), "Convergence Rate of Perturbed Empirical
Distribution Functions," Journal. of Applied Probability, 16,
163-173.
Yamato, H. (1973), "Uniform Convergence of an Estimator of a
Distribution Function, Bul.l.etincof Mathematiaal. Statistias, 15,
69-78.
·e