A LAW OF
•
TI~E
ITERATED LOCARITHM FOR
NONPARAMETRIC REGRESSION FUNCTION ESTIMATORS*
Wolfgang HardIe
Universitat Heidelberg
Sonderforschungsbereich 123
1m Neuenheimer Feld 293
D-6900 Heidelberg 1
Summary:
University of North Carolina
Department of Statistics
321 Phillips Hall 039A
Chapel Hill, NC 27514
We prove a law of the iterated logarithm for nonparametric regression
function estimators using strong approximations to the two dimensional empirical
process.
We consider the case of Nadaraya-Watson kernel estimators and of esti-
mators based on orthogonal polynomials when the marginal density of the design
variable X is unknown or known.
Keywords and Phrases:
Nonparametric regression function estimation, law of the
iterated logarithm, kernel estimation.
AMS Subject Classifications:
60FlO, 60G15, 62G05
to
~
*Research partially supported by the Air Force of Scientific Research Contract
AFOSR-F49620 82 C 0009.
*Research partially supported by the "Deutsche Forschungsgemeinschaft" SFBl23,
"Stochastische Mathematische Hodelle."
1.
Introduction and Background
A series of papers recently appeared on consistency of nonparametric regression
function estimators and rates of consistency.
phic review).
(See Collomb, 1981 for a bibliogra-
In the present work we obtain pointwise rates of consistency by de-
monstrating a law of the iterated logarithm for a large class of regression function estimators.
The estimators we shall look at are of the following type:
(1.1)
m (x)
n
where' {K:
r
=
n-
1 n
I
i=l
K ( )(x; X.)Y.
r n
1
1
rEI} denotes a sequence of delta functions (or kernel sequence) and
{(X.,Y.)} i=1,2, ... ,n are independent observations of a distribution with unknown
1
1
positive density f(x,y).
Most nonparametric estimators of m(x) = E(Ylx=x) are of this form,
for in-
stance, the Nadaraya-Watson kernel estimator (more generally delta function estimators) or orthogonal polynomial estimators.
A major result in the theory of consistency of kernel tyPe estimators has
been obtained by Collomb who gave necessary and sufficient conditions for consistency of the Nadaraya-Watson kernel estimate.
For generalizations and related
work see the bibliographic review of Col 10mb (1981) where parallel work on orthogonal polynomials is also presented.
Stone (1977) considered the estimator de-
fined in (1.1) and gave general conditions on the .weights K (x; X.) for m (x) to
r
1
n
r
be consistent in L . i.e. for
EI~n (x)-m(x)
•
whenever EIYl
r
<
00
Ir
~ 0
Stone, however, points out that it is not clear from his
results when an estimator of the Nadaraya-Watson type, to be discussed in section
~
r
4, is consistent in L .
In the field of density estimation Wegman and Davies
(1979), Hall (1981), csorgH and Hall (1982) have given a law of the iterated
logarithm for different kinds of density estimators.
-2-
We begin by showing a law of the iterated logarithm for the shifted estimate
(1.2)
m (x) - Em (x)
n
n
.
That is, we center m (x) around its expectation.
n
We could also center it around
m(x), the regression curve, but since the bias is purely analytically handled, it
suffices to look at (1.2).
The handling with these bias terms using different
smoothness assumptions of m(o) and Kr (0) is delayed to the sections where we apply the general result of section 2.
In section 4 we show a law of the iterated
logarithm for the Nadaraya-Watson kernel estimate with known and unknown marginal
density fx(x) of X and in section 5 we show a similar result for estimators based
on orthogonal polynomials.
~
2.
A law of the iterated logarithm for a special triangular array.
Let tX.,Y.)} be a sequence of independent and identically distributed rv's
1
1
2
with pdf f(x,y) and cdf F(x,y) and Ey <oo.
As in (1.1) let {K : rEI} be a sequence
r
of real valued functions each of bounded variation and define
n
S (r) = I {K (X.)Y.-E[K (X.)Y.]}
n
i=l r 1 1
r 1 1
which is actually a multiple of (1.2) where we omitted the design point x for
convenience.
Define also
a(r,s) = cov{K (X)Y,K (X)Y}
r
s
and
2
a (r) = a(r,r) .
We will establish conditions similar to Hall (1981) and CsorgH and Hall (1982)
under which S (r), r=r(n)d follows the law of the, iterated logarithm.
n
monstrate that
limsup ± [¢(n)]
n-+=
-1
Sn (r(n)) = 1
a.s.
We de-
-3-
where
~(n)
2
= (2na (r)loglogn)
1/2
The set' {S (r), n 2: I} is, in fact, a tri-
.
n
angular sequence, and in this section it is shown that under certain assumptions
S may be approximated by a Gaussian sequence with the same covariance structure.
n
A law of the iterated logarithm can then easily be deduced using techniques similar to Hall (1981).
We shall also make use of the Rosenblatt transformation (Rosenblatt, 1952)
transforming the original data points {(Xi'Yi)}~=l into a sequence of mutually
{(Xi'Yi)}~=l.
independent uniformly distributed over [0,1]2 random variables
This transformation was also employed by Johnston (1982) and Mack and Silverman
(1982) to obtain strong uniform consistency of Nadaraya-Watson kernel type re-
gression function estimates.
Define
vn(un) = Ixl!u
IdKr(n) (x)
I
+
IKr(n) (-un)
I,
n2:l
n
where {u } is a sequence of constants O<u ~oo.
n
n
Theorem 1.
Suppose that the sequence of kernels Kr(n) and {un} satisfy
(2.1)
where {a } is a sequence of positive constants tending to infinity.
n
00
L
n=3
00
(2.2)
I
n=3
a(r) -2 (loglogn) -1 [E'{K2 (X) I ( IXI>u ) }]
r
n
0
<
00
a(r) -2 (loglogn) -1 [E'{ K2 (X)oI( IXI~u )oY 2 oI( IYI>a ) }] <
r
n
n
00
•
Then on a rich enough probability space there exists a Gaussian sequence
{Tn }
.
with zero means and the same covariance structure as {S (r)},
n
S (r) - T = o(n l / 2a(r) (loglogn) 1/2)
n
n
a.s.
and such that
-4-
The main idea of the proof is as in Hall (1981) (for density estimators) and in
HardIe (1983)
(for regression estimators) the strong approximation of F (z)-F(z)
n
(density case) and of F (x,y) -F(x,y) (t'egression case) respectively.
Hall employs
n
~1ajor,
for the case of density estimation the results of Kom16s,
Tusnady, (1975).
We will make use of a similar result (for the two dimensional case) by Tusnady
/\
The fundamental connection between the regression estimator m Co) and
(1977) .
n
its strong approximation by a Gaussian process is established by the following
lemma.
Lemma 1.
On a rich enough probability space there is a version of a Brownian
Bridge B(x',y'), (x',y')E[O,l]
2
P{suplen(x,y)
x,y
such that
I
>
(Cllogn+u)logn}
<
C2 e
0
-C 3u
,
where C ,C 'C are absolute constants and
l 2 3
e Cx,y)
=
n
Proof.
n[(F (x,y)-F(x,y))-B(T(x,y))]
n
l 2
This is clear from Tusnady (1977) and the fact that n / [F (T-l(x',y') n
F(T-l(x' ,y'))], (x',y')E[O,1]2 is the empirical process of {(X. ,Y.)}~ 1 (Rosen1
1
1=
blatt, 1952).
The following theorem establishes now under regularity conditions on the covariance matrix a(r,s) that a law of the iterated logarithm (LIL) holds for
~n (x) the regression function estimator as defined in (1.1).
Theorem 2.
(2.3)
when r
Suppose that (2.1) and (2.2) hold and that
2
lim limsup sup la(r(m),r(n))/a (r(n)) - 11
E:-+O
Jl"+OO mEf
n,E
n,E
= {m:
Im-nl~En}.
limsup
n~
±
Then
[¢(n)]
-1
S (r)
n
=1
a.s.
= 0,
-5-
Condition (2.3) is the same as in Hall (1981) but with
II
0(r ,r ) =
y2 K (x)K (x)f(x,y)dxdy instead of his 0
r
r
r ,r
l 2
2
l 2
l
in the case of estimating the uniform density.
3.
=
I Kr
(x)K
l
r
(x)dx
2
Proofs.
To establish Theorem 1 we set
00
T =
n
nIl
_00
K (x)ydB(T(x,y)) ,
r
B(x',y') being the Brownian Bridge of Lemma 1 and show that the difference
1
R = n- (S (r)-T ) =
n
n
n
satisfies
n-lil
K (x)yde (x,y)
r
n
R = 0(n- l / 20(r)(10glogn)1/2)
(3.1)
a.s.
n
Note first that T has the covariance structure ascribed to it in Theorem 1.
n
This follows from the fact that the Jacobian J(x,y) of T(x,y) is J(x,y) = f(x,y),
the joint density of (X,Y) (see Rosenblatt, 1952) and the following lemma, stated
without proof.
Lemma 2.
Let Gr(x,y) = Kr(x)y.
Then
11
(Zl,Z2) =
(II
00
11
G (T-l(X',y'))dB(x',y')
rl
, II
00
G (T-l(x' ,y'))dB(x',y'))
r
2
has a bivariate normal distribution with zero means and covariances
=
II
K (x)K (x)y2 f (x,y)dxdy
r
r2
l
- [II
Kr (x)yf(x,y)dxdy]
l
To
~emonstrate
[II
Kr (x)yf(x,y)dxdy]
2
(3.1) we split up the integration regions and obtain
IR
n
I
7
~
I
R.
j=l J,n
-6-
where
R
= In-
I
I,n
f
\xl~u
n
J
Iyl~a
n
K (x)yde (x,y) I
r
n
~ v (u ) o2 oa on-Iosup Ie (x,y) I ,
n n
n
n
x, y
.
1 n
(2)
R
= InR.
2,n
i=I l,n
I
I,
R(2) = [K (x.)oI(lx.l>u )oY.oI(IY.I~a)]
I
n
I
I
n
i ,nr I
- E[Kr (X)I(lx\>un )YoI(IYI~an)]
R~3) =
1,n
[K
r
(X.)oI(\x.l~u )oY.oI(IY.I>a)]
lIn
lIn
- E[Kr (X)oI(lx\~u n)oYoI(IYI>an )]
R~4)
1,n
= [K (X.) oI( Ix.l>u ) oY. oI( IY.I>a )]
r lIn
I
I
n
- E[Kr (X)oI(lx\>un )oYoI(IYI>a)]
,
n
R
5'n
R
-
n
-11 f
.\x I>u
f
Iyl~a
n
- n-II
6,n -
R
- n
7'n -
-I,
f
Ixl~u
f
Ix I>u
K (x)ydB(T(x,y))
r
n
f
n
Iyl>a
I,
K (x)ydB(T(x,y))
n
f
Iyl>a
n
n
r
K (x)ydB(T(x,y))
I,
I .
r
From Lemma 1 we deduce that n-Isuple (x,y)! = O(n- I (logn)2)
n
x,y
a.s.,
-7-
and so by condition (2.1) we conclude that
(3.2)
R
= 0(n- l / 20(r) (loglogn) 1/2)
l,n
a.s.
Next observe that {R~2)} l~i~n are independent and identically distributed random
1, n
variables. We then have by Markov's inequality that for any £>0
n
p(n-l,
I
R(2) 1>£'o(r)n-l/2'(10glogn) 1/2)
i=l i,n
So with the assumption Ey2<oo and condition (2.2) it follows with the Borel-Cantelli
Lemma that
(3.3)
R
= 0(n- l / 20(r) (loglogn) 1/2)
2 ,n
a.s.
The terms R ' R
may be estimated in the same way using Markov's inequality
3 ,n
4 ,n
and condition (2.2) and we therefore have
R
= 0(n- l / 20(r) (loglogn) 1/2)
a.s.
R
= 0(n- l / 2o(r) (loglogn) 1/2)
a.s.
3,n
(3.4)
4,n
The remaining terms, R
' Rand R
are all Gaussian with mean zero and
S
.,n
6,n
7,n
standard deviations
respectively.
Therefore, R ,n ' for instance, can be computed by
S
P(R
S ,n
>£n
-1/2
o(r) (loglogn)
1/2
)
-8-
where
~
denotes the cdf of the standard normal distribution.
A similar equality
holds for R
and R ; therefore; we conclude in view of condition (2.2) and
7 ,n
6 ,n
the usual approximations to the tails of the normal distribution that
R
=
o(n -1/2a(r) (loglogn) 1/2 )
R
=
o(n
R
=
o(n
5,n
(3.5)
6,n
7,n
-1/2
-1/2
a(r) (loglogn)
a(r) (loglogn)
1/2
1/2
a.s.
)
a.s.
)
a.s.
Finally the desired result of Theorem 1 follows by putting together statements
(3.2)-(3.5) respectively.
The proof of Theorem 2 follows in much the same way as Theorem 1 in Hall
(1981, p. 49).
We only have to note that lemma 1 in Hall (1981, p. 49) has to
be replaced by (2.3).
Setting Y= 1 in all our derivations shows that Hall's
result follows from ours.
4.
Kernel estimators.
Two types of kernel estimates of the regression function m(x) will be con-
sidered here.
The first is due to Nadaraya (1964) and Watson (1964) and is mo-
tivated by the formula
m(x)
=
{fyf(x,y)dy}/fx(x) .
We define the Nadaraya-Watson estimate as follows:
n
1 n
*( ) _ ( h)-l L K((x-X.)/h)Y./[(nh)- L K((x-X.)/h]
m
.1
1
1
·1
1
n x-n
1=
1=
Consistency and asymptotic normality of m*(x) were considered by Schuster (1972),
n
Johnston (1979), Mack and Silverman (1982) among others.
~
If the marginal density
fx(x) is known, it is appropriate to replace the density estimate in the denominator of
m~(x)
by the true density fX(x).
This leads to the following estimate:
-9-
mn (x)
= (nh)-
1 n
L K((X-X.)/h)Y./fX(x)
1
1
. 1
1=
considered by Johnston (1979,1982).
2
2
2
Let us define S2(x) = E(y Ix=x), V (x) = S2(x) - m (x), and assume that
fx(x), m(x) are twice differentiable and S2(x) is continuous.
We assume further
that the kernel K(o) is continuous, has compact support (-1.1) say and that
f:
uK(u)du = o.
l
large enough u .
This implies that v (u ) as used in (2.1) is constant for
n n
We will make use of the following assumptions:
n
5
nh /loglogn
(4.1 )
00
L
(4.2)
n=3
~
0
as n
~
00
2
h(lOglOgn)-loE[y I(]Y\>a )]
n
<
00
where {a } is as in (2.1), (2.2) such that
n
(4.3)
lim limsup
£~o
•
n~
sup
mEr
Ih(m)/h(n) - 11 = 0 .
n,£
We then have the following theorem for
Theorem 3.
mn (x) .
Under the assumptions above
limsup
n~
±
1/2
[m (x)-m(x)](nh/21oglogn)
n
The Nadaraya-Watson estimate follows also a LIL as the following theorem shows.
00
Theorem 4.
Under the assumptions above and
2
n- h- l <
I
n=l
limsup ± [m*(x)-m(x)](nh/2loglogn)
n~
n
1/2
00
-10-
Note that the only difference between Theorem 3 and Theorem 4 is the different
2
Since in general S
scaling factor.
(x)~V
2
(x) we may expect closer asymptotic
confidence bands for m*(x).
This observation has already been made by Schuster
(1972) and Johnston (1982).
This papers together with HardIe (1983) thus solves
n
the question raised by Johnston (1982) whether one should be able to compute
asymptotic confidence intervals for m*(x).
n
Johnston derived (uniform) confidence
intervals for m (x) only.
n
Proof of Theorem 3.
We first show that we cotid center m (x) around Em (x) .
n
n
This follows from
I
f/h)m(u)
·
= f x (x) -1 h - K(
(x-u)
fx(u) du = m(x)
Em
(x)
n
+
O(h 2)
using the smoothness of m(o) and fX(o) and the assumptions on the kernel K(o)
(Parzen, 1962; Rosenblatt, 1971).
From assumption (4.1) it thus follows that the bias term (Em (x)-m(x))
n
vanishes of higher order.
•
limsup
(4.4)
±
A
So it remains to show that
A
[m (x)-Em (x)]/(nh2loglogn)
n
n~
=
[S 2 (x)ofX(x)
n
A
where m (x) =
n
1/2
n
f
K2 (u)du] 1/2
a.s.
n
L
i=l
K((x-X.)/h)Y.
1
.
1 =
i~l
Kh(\)Y i
From the assumptions on the kernel K(o) we conclude that
° (u)
= h -1 K(u/h)
n
is a delta function sequence (DFS) (Watson and Leadbetter, 1964).
We make now
use of this general approach in terms of DFS' s and obtain the following:
h
0
a 2 (h) = h
~
4It
J on2
(x-u)S 2 (u)fx(u)du-h[ f 0n(x-u)m(u)fX(u)du] 2
S 2 (x)ofX(x) f K2 (u)du
as n
~
<Xl
•
This follows from Watson and Leadbetter (1964) by noting that S2(O)f (o) is
x
-11-
2-1 00 2 (u) } is itself a DFS.
continuous and .{h( f K)
.
n
The use of this DFS-technique
would also considerably simplify Hall's proof (1981) for Rosenblatt-Parzen kernel
density estimates.
To establish (4.4) with the use of theorem 2 we have to show that (2.3) holds.
We
mu~t
thus demonstrate that if h,k+O such that h/k+l (in view of assumption
(4.3)), then
h -1 cov { K((x-X)/h)Y
(4.5)
But EK((x-X)/h)Y
=
Kcex-Y)/k)Y} + 1
h f 0n(x-u)m(u)ofX(u)du
=
o(h
+1/2
), and so by the computations
for a 2 (h) above it remains to demonstrate that
h -If [K((x-u)/h)-K((x-u)/k)] 2S2(u)fX(u)du+ 0 .
2
From the boundedness of S (0) and fX(o) it is clear that the integral above is
dominated by
Mf
[K(U)-K(uh/k)]2du .
The kernel K is continuous and so K(uh/k) + K(u) a.e. and i t follows that (4.5)
holds.
Assumption (2.1) follows from (4.2) since K(o) has compact support and thus
v (u )
n
n
=
const. for n large enough.
2
In view of the asymptotic formula for a (h)
above we have by assumption (4.2)
a
n
2
= o((na 2 (h)loglogn) 1/2 /(logn))
which is assumption (2.1).
Finally, assumption (2.2) follows immediately from
2
(4.2) since K has compact support and as above a (h)
~
h
-1
.
Theorem 3 thus f01-
lows from theorem 2.
Proof of theorem 4.
To prove theorem 4 we decompose
m*(x)
- m(x)
n
= [(nh) -1/\mn (x)-m(x)fn (x)]/fX(x)
-12-
-1 n
where f (x) = (nh)
L K((x-X.)/h) is a density estimate of fx(x).
i=l
1
n
Hall (1981), Theorem 2 it follows that
(4.6)
limsup ± [f (x)-f (x)](nh/21oglogn)
n
X
n~
Now from
1/2
2
if we use assumption (4.1) which ensures that the bias (Efn(x)-fX(x)) = O(h )
vanishes.
Note that Hall's assumption (11) is not necessary here since we assume
that K(o) has compact support.
makes
m*(x)-m(x)=o(l) a.s ..
n
\' -2 -1
From Noda (1976) we conclude that In
h
<
00
This and (4.6) thus yIeld that the second term on
the RHS of the decomposition above is of order o[nh/21og1ogn) 1/2) a.s.
The first summand of the decomposition above can be written as
(nh)
~
-1 A
A
..
(m-Em)/fX + ((nh)
-1 A
Em-mfX)/fX - m(fn-Efn)/fX + m(fX-Efn)/f x
As in the proof of theorem 3 it follows by assumption (4.1) that the bias terms
((nh)
.
(4.7)
-1 A
Em-mf ) and (Efn-f ) vanish.
X
x
(nh)
-1 A
It remains to show
A
(m-Em) - m(fn -Efn )
follows the LIL, i.e.
limsup ± [(nh)-l(~_E~) - m(f -Ef )](nh/2loglogn)1/2
n
n~
= [V 2 (x) ofX(x)
0
n
J K2 (u)du] 1/2
a.s.
This can be deduced from theorem 2, if we rewrite (4.7) as
-13-
2
hOG 2 (h) = ho f 0u(x-u)
[5 2 (u)-m 2 (x)]fX(u)du
.
+
2
V (x)ofX(x)
f
2
.
K (u)du
as
n~
.
As above in the proof of theorem 3 we conclude that (2.3) holds.
Theorem 4 thus
follows from theorem 2.
S.
Orthogonal polynomial estimators.
Estimators of the regression function m(x) based on orthogonal polynomials
fit also in the general framework developed in the first section.
•
We define the
estimate based on a system of orthonormal polynomials on [-1,1] as follows:
l
~ (x) = nn
..
n
L
K (x;X.)Y./n
m
1 1
i=l
-1 n
L K (x;X.)
1
.1= 1 m
where m = men) tends with n to infinity and
m
K (x; X.) =
m
1
t:
'0
J=
e.(x)e.(X.)
J
J
1
and {e.(o)} is the orthonormal system of polynomials.
J
In the case of a known marginal density fX(x) we consider
n
l
m'(x) = n- L K (x; X.)Y./fX(x)
n
i=l m
1 1
2
2
As in section 4 let S (x) be the second conditional moment of Y and V (x) the
conditional variance respectively.
We further assume that
fX(x) has compact support in (-1,1)
2
(1_x )-1 / 4f (x) is integrable on (-1,1).
X
-14-
We consider only the case of e.(o)
= p.(o) = orthonormal
here and assume that the following
holds~
J
.
lim limsup
(5.1)
E:-+O
00
\
L
(5.2)
sup
\m(p)/m(n)-l!
pE r
n-¥X>
J
Legendre polynomials
=0
n,E:
m-1 o(loglogn) -1 E(Y 2 oI( IYI>a )) <
n
n=3
00
•
when· {a } is as in (2.2), (4.2) a sequence of constants tending to infinity such
n
that
a
n
= 0 (n 1/2m( log1ogn) 1/2 / (logn) 2)
5
n/(m log1ogn)
(5.3)
~
0
as
n
~
00
•
We have then the following theorem for m'(x) and ~ (x).
n
n
•
Theorem 5.
Under the assumptions above
1imsup
±
[m'(x)-m(x)](n/2mlog1ogn)1/2
n
n-¥X>
a.s.
and
~
limsup ± [m (x)-m(x)](n/2m1oglogn)
n-¥X>
Proof.
We first show that the LIL for m'(x).
n
follow as theorem 4 from theorem 3.
(Em'(x)~m(x))
n
1/2
n
As in theorem 3 we show first that the bias
is negligible.
=
mCx) + OCm
-2
The second assertion will then
)
-15-
y a slight modification of the argument proving theorem 1 in Walter and Blum
(1979).
..
By the same arguments as in Hall's (1981) proof of his theorem 3 (p. 60)
we conclude that
Assumption (2.1) follows now from (5.2) and
IldKm(x;
u)
I
2
= O(m )
.
Assumption (2.2) follows also from (5.2) so we finally derive the desired result
from theorem 2, since (2.3) may be proved as in theorem 3 using (5.1).
Acknowledgement
The author wishes to thank R. Carroll, S. Cambanis for helpful suggestions
•
and remarks .
References
Revue
1.
Co11omb, G. (1981). Estimation non-parametrique de la Regression:
Bibliographique. International Statistical Review 49, 75-93.
2.
CsorgH, S. and Hall, P. (1982). Upper and lower classes for triangular
arrays, Z. f. Wahrsch. verw. Geb. 61, 207-222.
3.
Hall, P. (1981). Laws of the iterated logarithm for nonparametric density
estimators. Z. f. Wahrsch. verw. Geb. 56, 47-61.
4.
HardIe, W. (1983). Asymptotic maximal deviation of M-smoothers.
for publication.
5.
Johnston, G. (1979). Smooth nonparametric regression analysis.
Stat. Mimeo Series #1253, IJniversity of North Carolina.
6.
Johnston, G. (1982). Probabilities of maximal deviation of nonparametric
regression function estimation. J. Mult. Anal. 12, 402-414.
7.
Komlos, J., Major, P., Tusnady, G. (1975). An approximation of partial sums
of independent rv's and the sample df I. Z. f. Wahrsch. verw. Geb. 32,
111-131.
8.
Mack, Y.P. and Silverman, B.W. (1982). Weak and strong uniform consistency
of kernel regression estimates. Z. F. Wahrsch. verw. Geb. 61, 405-415.
Submitted
Inst. of
-16-
8a. Nadaraya, E.A.
9.
.-
(1964). On estimating regression. Theor. Proh. Appl. 9, 141-142.
Noda, K. (1976). Estimation of a regression function by the Parzen kernel
type density estimators. Inst. Stat. Math.
, 221-234 .
...
..
10.
Parzen, E. (1962). On estimation of a probability density function.
Math. Stat. 33, 1065-1076 .
11.
Rosenblatt, M. (1952).
Stat. 23, 470-472.
Remarks on a multivariate transformation, Ann. Math.
12.
Rosenblatt, M.
Curve estimates.
13.
Schuster, E.F. (1972). Joint asymptotic distribution of the estimated regression function at a finite number of distinct points. Ann. Math.
Stat. 43, 84-88.
14.
Stone, C.
645.
15.
Tusnady, G. (1977). A remark on the approximation of the sample df in the
multidimensional case. Period. Math. Hung. 8, 53-55.
16.
Walter, G. and Blum, J. (1979). Probability density estimation using delta
sequences. Ann. Stat. 7, 328-340.
(1971).
(1977).
Ann. Math. Stat. 42, 1815-1842.
Consistent nonparametric regression.
16a. Watson, G.S. (1964).
359-372 .
Ann.
Ann. Stat. 5, 595-
Smooth regression analysis, Sankhya,Series A, Vol. 26,
17.
Watson, G.S: and Leadbetter, M.R.
Series A, 26, 101-116.
(1964).
Hazard analysis II.
Sankhya,
18.
Wegman, E.J. and Davies, H.I. (1979). Remarks on some recursive estimators
of a probability density. Ann. Stat. 7,316-327.