One Nonparametric Estimation of the Bernoulli
Regression
Elizbar Nadaraya1, Petre Babilua2, Grigol Sokhadze3
1,2,3
Faculty of Exact and Natural Sciences, I. Javakhishvili Tbilisi State University
2, University St., Tbilisi 0186, Georgia
1
[email protected], [email protected], [email protected]
Abstract— The nonparametric estimation of the Bernoulli
regression function is studied. The uniform consistency
conditions are established and the limit theorems are proved for
continuous functionals on C[a,1 a] , 0 < a < 1/2 .
numbers converging to zero and nbn   .
II.
Theorem 1. Assume that
(a) K  x  is a function with bounded variation.
Keywords— Bernoulli regression, kernel estimation, consistency,
uniform convergence, Wiener process.
I.
(b) p  x  and
variable x  0,1 , i.e. p = px  = PY = 1 | x ([1]-[3]). Let
x i , i = 1, n , be the division points of the interval 0,1 which
are chosen from the relation
xi
h  x  dx =
0
Then the estimate (1) is asymptotically unbiased and
consistent at all points x  0,1 where p  x  is a continuous
function. Moreover, it has an asymptotically normal
distribution, namely:
d
nbn  pˆ n  x   E pˆ n  x    1  x  
 N  0,1 ,
 2  x =
2i  1
, i = 1, n,
2n
where hx  is the known positive bounded distribution
density on 0,1 . Let further Yi , i = 1, n , be independent
Bernoulli random variables with
PYi = 1 | xi  = pxi  ,
 t  =
h
i =1
1
 x  xi  2 
 Yi ,  = 1, 2,
 bn 
 xi  K 
where K  x   0 is some distribution density (kernel) and also
K  x  = K   x  , x   ,   , bn  is a sequence of positive
 u  du.

e
itx
K  x  dx .
(a1) Let nbn2   , then
P
Dn = sup pˆ n  x   p  x  
 0,
xn
 n = bn ,1  bn  , 0 <  < 1.




1
nbn
2

n
p n  x  =
K
function on 0,1 . Let further  t   L1  ,  ,
Y1 , Y2 , , Yn . Such problems arise in particular in biology ([1],
[3]), in corrosion studies [4] and so on.
As an estimate for p  x  we consider a statistic ([5], [6])
of the form
n
h  x
conditions of Theorem 1, and also p  x  is continuous
estimating the function p  x  , x  0,1 , by the sample

p  x  1  p  x  
Theorem 2. Let K  x  , p  x  and h  x  satisfy the
P Yi = 0 | xi  = 1  p  xi  , i = 1, n . The problem consists in
 x  xi 
h 1  xi  K 
 Yi
bn 

1
i
=1
pˆ n  x  = p1n  x   p2 n  x  = n
,
 x  xi 
1
h  xi  K 
(1)

 bn 
i =1
h  x  are also functions with bounded
variation on 0,1 , and h  x    > 0 , x  0,1 .
INTRODUCTION
Let a random value Y takes two values 1 and 0 with
probabilities p (“success”) and 1  p (“failure”). Assume that
the probability of “success” p is a function of an independent
STATEMENT OF THE MAIN RESULTS
(b1)
If
n
 s/2  s
bn
  for some s  2 , then Dn  0
n =1
a.s.
Corollary. Under the conditions of Theorem 2,
sup pˆ n  x   p  x   0
x a ,b
almost surely for any fixed interval a, b   0,1 .
Assume that bn  n ,  > 0 . The conditions of


P  max Tn  t  >   
a

t

1

a


Theorem 2 are fulfilled:
n bn   if
1/2
1
0<  ,
2
 G   
and


n s/2bn s
n =1
s2
<  if 0 <  
, s > 2.
2s
T n t  = n
Tn  t  = n
  pˆ u   E pˆ
n
n
a
t
H1 : p  x   p1  x  ,
  pˆ u   p u  u  du,
n
K  x = K  x ,


K  x = 0
 x   p1  x   p0  x   dx > 0,

1 2
.
Further note that the functionals
f1 x  = sup x  t  ,
.

for
K  x  dx = 1 . Let further p  x  and h  x 
satisfy the condition (b) of Theorem
0 < inf p  x   sup p  x  < 1 , x  0,1 .
1
and
(c) If p  x  is continuous on 0,1 and nbn2   as
where W  t  a  , a  t  1  a , is the Wiener process.
(d) If nbn2   , nbn4  0 and p  x  has bounded
derivatives up to second order, then the distribution
of f Tn    converges to the distribution of
f W    .
Corollary. By virtue of Theorem 3 and Theorem 1 from
[7, p. 371] we can write
1 a
x
2
 t  dt
a
are continuous on C  a,1  a . Therefore Theorem 3 also
implies
d
f1 Tn   
 f1 W  
and
on C  a,1  a , 0  a  1/2 , the distribution of

a t 1 a
f 2  x    =
n   , then for all the continuous functionals f 
f T n   converges to the distribution of f W    ,

and
x 1 ,


0

Theorem 3. Let K  x   0 satisfy the condition (a) of
besides,

 0  x   h  x  p0  x  1  p0  x  
12
and,
1 a
a
a
1
,
H0 : p  x  = p0  x  , a  x  1  a
when the alternative hypothesis is
where
Theorem

This result enables us to construct the test of the level
 u    u  du,


h u 

 u  = 
 p  u  1  p  u   


x2
1


exp 
dx  , 0 < a < .
2
2 1  2a  
 2(1  2a) 
0 <  < 1 , for checking Hypothesis H 0 , by which
Let us introduce the following random processes:
t

2
d
f 2 Tn   
 f 2 W   .
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
S. Efromovich, Nonparametric curve estimation. Methods, theory, and
applications. Springer Series in Statistics. New York : Springer-Verlag,
1999.
J. B. Copas, Plotting p against x . Appl. Statist., 1983, Vol. 32, No. 1,
pp. 25-31.
H. Okumura and K. Naito, Weighted kernel estimators in nonparametric
binomial regression. J. Nonparametr. Stat., 2004, Vol. 16, No. 1-2, pp.
39-62.
K. B. Manjgaladze, On some estimate of the distribution function and its
moments. Soobshch. Akad. Nauk Gruz. SSR, 1986, Vol. 124, pp. 261263 (in Russian).
E. A. Nadaraya, On estimating regression. Teor. Veroyatn. Primen.,
1964, Vo. 9, pp. 157-159 (in Russian).
G. S. Watson, Smooth regression analysis. Sankhyā Ser. A, 1964, Vol.
26, pp. 359-372.
I. I. Gikhman and A. B. Skorokhod, Introduction to the theory of random
processes. Moscow: Izdat. Nauka, 1965 (in Russian).