•
A NOTE ON ESTIMATIID A UNIMODAL DENSIrrTby
Edward J. Wegman
Department of Statistics
University of North Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 599
November 1968
•
1
Tl1is research arises in part from the author's doctoral dissertation
submitted to the Department of Statistics, University of Iowa. It was supported in part by PHS Research Grant No. GM-14448-0l from tile Division of
General 1,1edical Sciences and in p:l.rt by U.S. Air Force Grant AFOSR-68-l4l5.
A Note on Estimating a Unimodal Densitj·l
Edward J. Wegman
1.
T~lis
Introduction.
paper is concerned with the problem of estimating
a unimodal density with unknown mode.
Robertson [5J has SilOwn in the case
that tIle mode is known a solution can be represented as a conditional expectation gi. ven a cr-lattice.
Brunk [2) discusses such conditional expectations
as well as other problems.
A cr-lattice,
of subsets of
0
s.,
•
and
O.
If
0
of intervals containing a fixed point,
S. , if the set,
A function,
[f
> aJ,
shall say a function
to S.(M).
J..L)
is a collection
closed under countable unions and countable intersections
and containing both
denote as <L (m).
a,
of sUbsets of a measure space (0,
f
is the real line, then the collection
m,
is a cr-lattice wilich we shall
f , is measurable with respect to a cr-lattice,
is in S.
for each real
is unimodal at
M 'When
f
a.
In this paper, we
is measurable with respect
This definition is equivalent to a more usual definition as seen in
the following easily verified remark.
Remark:
A function
decreasing at
Let
0
x
for
f
is unimodal at
x < M and
f
be the real line and let
M if and only if
is nonincreasing at
x
f
for
J..LI=" be Lebesgue measure.
the set of square integrable functions and
which are also measurable witIl respect to
~(s.)
s..
is nonx
> M.
Let
be those members of
~
be
~
We shall adopt the following
definition of conditional expectation with respect to a cr-lattice.
1
TIlis research arises in part from the autllor' s doct:)ral dissertation
submitted to the Department :::>f Statistics, University of IJWa. It was supported in part by PHS Research Grant No. GM 14448-01. from tile Division of
General Medical Sciences and in part by U.S. Air Force Grwlt AFOSR-68-1415.
2
L ' then g € ~ (ot) is equal to E(f It),
2
conditional expectation of f give ot if and only if
Definition 1.1.
If f
€
J
(1.1)
for every
J
=
f • a(g)d7\
tile
g • a(g)d7\
a, a real-valued function such that
a(g)
€
L and
2
a(o)
= 0,
and
l
(1.2)
for each A
€
S.
(f-g)d7\
with 0 < 7\(A) <
00
0).
In
0
•
(Brunk [lJ shows such a function,
of Lebesgue measure
=s
g, exists and is unique up to a set
this paper, the estination of the density will
be based upon a strongly consistent estimate of the mode such as those of
Nada.raya [.3J or of Venter [6J.
(The author is aware of a third estimate, as
of now unpublished, given by Robertson and Cryer).
2.
mode
An estimate of the density.
f, to be estimated has
Y2 =s ••• =s Yn is the ordered
n chosen according to the density f. Let {M} be a
M which is unknown.
sample of size
The density,
Let us assume Yl
=s
n
sequence, (we shall not yet require
(M)
n
to be a consistent estinate of M.
In fact, our only restriction now is Yl ~ Mn ~ Yn for
be
the largest observation less tllan or equal to Mn •
~
= [Y2,
Y.3), ••• , Aq(n)
Mn ),
Aq(~)+l
2: 2.) let Yq(n)
Let Al
= [Mn,
= [Yl , Y2 ),
Yq(n)+lJ, ••• ,
Notice, if Mn is an observation, Aq(n) = t and will be
deleted from this collection of intervals. With this convention in mind, let
An
= (Yn- l ,
= [Yq(n)'
n
us define
(2.1)
ynJ.
3
where
n
I . is the indicator
A
J.
be tIle lattice of intervals containing M. Then
Finally let S.
of A..
and
n .
J.
in
is the number of observations in Ai
i
= E(g
n
1.£),
n
n
the conditional expectation of
intervals containing
g
n
given tl1e a-lattice of
M , is an estimate of the density.
n
Robertson [5J was
A
able to show that
M mlen M
n
n
f
is an unimodal maxinn.un likelihood estinBte with mode
n
is not an observation.
M
If
is an observation, additional
n
restrictions are necessary to have a maxinn.un likelihood estimate.
extension was done (see Wegman
One such
[7]), but since this is neither difficult nor
central to the result of this paper, we shall not pursue this extension.
3. Conditional expectations of the
~
density.
In this section we
shall represent the conditional expectation of the true density with respect
to the lattice of intervals containing a point in
port of
be in L
f.
2
,
Since the definition of conditional expectation requires that
we shall make that assumption
Theorem 3.1.
If
f
point in the support of
i.
ii.
i1.
If
If
If
Proof:
(y: fey) > 0) , the supf
he~e.
is a density function unimodal at
M
and
m is a
f , then
m > M , there is an interval
[a, m]
E(f!s.(m))
=
and
E(t\.t(m»
=
E(tl.£(m))
=
E(fl.£(m))
=
f
on
[a, m]c
such that
(m-a)-1·1
tdA on [a, mJ •
[a,m]
m < M , there is an interval [m, b] such that
m
= M,
t
on
Em, b]c
and
(b-m)-1·1
on Em, b] •
[m,b]
E(f 1.£ (m)) = f
everywhere.
Case iii is easily verified and case i i can be shown in a manner
similar to case 1.
We shall show case 1.
Let
a
= sup(y
4
·1
.:s M : (m_y)-l
fd"
2:
fey)} • To observe that the
[y,m)
set is not en:q>ty, notice for any y.:s M, for which
element of the set.
and
Let us define
1
f*(Y) = (m-a) -1 .
sh01'1 f*
fd" for
y
for
y
belonging to
belonging to [a, m).
is an
[a, m)c
We want to
[a,m)
= E(flt(m».
difficult to verify
= f(y)
f*(y)
f(y).:s rem) ,y
Clear~
f*
f*(z) < f*(m)
is unimodal at
m.
for each
Hence
f*
z, so it is not
€ ~(.t
(m».
We need
only sllow (1.1) and (1.2) hold.
Let
a'
= sup(y < m
: (m_a)-l . j f d " . : s fey)} • For y such that
[a,m)
a .:s y < a', (m-a) -1.
fd" = f*(y) .:s f(y) and for y such that
J [a,m]
r
a' < y .:s m, f*(y)
o < ,,(A) < 00.
2:
fey).
Now let A
[b , b ].
2
l
l
(f-f*)d"
::s m,
since we have
1
(3.2)
[bl,m]
a.:s b
(3.3)
But
l
< a' , since f - f* 2:
r
J [bI,a')
j [a,m) (f-f*)d" =
0
=
2
> m, we may write,
1
(f-f*)d"
[b , m]
l
f* - f > 0
(f-f*)d"
.
on
(a', m]
::s 0
on (a, a') ,
(f-f*)d"
<1
-
(f-f*)d"
[a,a']
0 , so that
1
(f-f*)d"
[a,a']
This together with (3.3) gives us
(3.4)
If b
1
[b , b )
l 2
If
with
Since we are dealing with Lebesgue measure, we may assume A
is a closed interval, say
Now, if a'.:s b
be any element of ,t(m)
= -~
(a',m]
(f-f*)d"
5
Finally, if b
<a
l
f
(3.5)
=0
(f-f*)d'A
[bl,m]
Hence, combining (3.1) with either (3.2), (3.4), or (3.5), we have for
A in S. (m)
0 < 'A(A) <
such that
co ,
This shows condition (1.2) 110lds.
Borel function with
1
9(f*)
(f-f*)· 9(f*(m) )d'A
€
L
2
and
To show (1.1) holds, let 9 be any
9(0) = O.
= 9(f*(m»·1
[a,m]
(f-f*)d'A.
[a,m]
~(f-f*)
Thus
Then jcf-f*) • 9(f*)d'A =
f*
4.
= E(f~(m»
Let
= o.
and the proof is complete.
Consistency.
estimates.
• 9(f*)d'A
r
But
(f-f*)dA.--o, so
{a,m]
we will show consistency of our
In this section,
n t = rlim sup IF (y) - F(y)
Ln-. co
n
y
this set has probability one.
n
to a random variable
Y with density
•
It is well known that
We shall assume henceforth that all observations
arise from points in
t
I = 'jJ.
If F
is the distribution function corresponding
distributed uniformly over the set
f , it is known that
[0, k].
Z
Moreover, if Y
l
=k
~
Y
2
. F(Y)
~
•••
is
~
Y
n
are the order statistics corresponding to a sample from F , then
Zl
=k
• F(Yl ), ... , Zn
=k
• F(Yn )
are distributed like the order statistics
from a uniform distribution with range
distribution by
If.
If Yl
0
to
k.
We shall denote this uniform
~ Y2 ~ ••• ~ yn is an ordered sample and
zl = k • F(Yl)' ••• , Zn = k • F(Yn )
are the transformed points, we denote
the empirical distribution based on
zl' ••• , Zn
sup
z
I~(z)
n
- ffk(z)
I = supy IF n (y)
- F(Y)
I'
n'
by ffk.
n
= [lim
n-. co
sup
z
Since
I~(z)
n
- ffk(z)
I = 0_1~
6
> O. TIle reader may satisfy himself that for points in
for each
k
follol'r.i.n~
statements are true.
Of, the
TIle largest observation less than and the smallest observation
1.
greater than a number in the support of
2.
Corresponding to every pair,
of
r
l
f
converge to that number.
< r 2 ' of numbers in the support
f , there is eventually a pair of observations
y
r
l
()
n
and y
r
2
()
n
r l < Yrl(n) < Yr (n) < r 2 •
2
Vie shall implicitly use these statements in the proof of the consistency
satisfying
theorem.
Our estimate is given by
point in the support of
f
f n = E(gn 1.£).
n
and let
t
Suppose
= f n (Yo).
Yo
Let
Pt
is an arbitrary
= [fn > t J
and
2: t J. If J.il(Tt ) = (L €£n: A(Tt-L) >0) and J!2(Pt ) =
(L €.£n
A(L-P ) > 0) , the result of Robertson [4J gives us.
t
Tt
=
[in
(4.1)
and
(4.2)
f n (Yo)
= sUPI£U (p ) [A(L-pt
2 t
)]-1·1
gn dA
L-P
t
(Notice that Robertson's theorem is stated for finite measure spaces.
Once
the sample in question has been chosen we may restrict our attention to
[Yl' y)
n
for which
A[Yl' Yn ) <
00
.)
The proof of the next theorem uses
metl10ds similar to those of Robertson [5J.
Theorem 4.1.
If
f
a sequence converging to
is an unimodal density with mode
m in the support of
f , then let
M and
f
rn
(M)
n
= E(f !.t(rn))
A
and let
f
n
be the estimate described in section 2.
is
For every y
0
< rn
7
f (y -) < lim inf
mo
and for every y
o
sup
f n(yo) -<m
f (y +)
o
f n(yo) -> lim
inf
f n(yo) ->m
f (y +)
o
>m
f (y -) > lim sup
mo
-
(4.4)
i n:>(y ) < lim
Inequalities (4.3) and (4.4) hold for points in
0', hence with probability
one.
Proof:
Inequalities (4.3) and (4.4) are proven in a similar manner
a1thougi.1 they will require different forms of the representation theorem given
in [4].
vie slla11 restrict::>ur attention to (4.3).
and let
a
and a'
be defined as in section 3.
the ordered sample based on a point from
in the interior of the support of
f.
less than
~
x.
There is a number
0'.
Let
Let us suppose
Let
< x in the support of f
the smallest observation greater than
Ysn
( )
> Y.un
( ).
Let
= f n (x)
t
Let
~.
and Pt
be
a number
be the largest observation
interior is an open interval (including possibly (-
that
.:s y2 .:s .•• .:s yn
x < m and x
Let
Ys(n)
Y1
m> M
00 ,
00
».
since the
Let Yu(n)
be
n be sufficiently large so
= [fn
~
> t]
•
Since
f
n
is
constant on intervals between observations (a fact demonstrated by Robertson
in [5]),
Yi(n)]
€
Pt = [Yj(n)' Yi(n)J.
~2(Pt)
Notice for sufficiently large
so that by (4.2),
fn(X)
~
for sufficiently large
(Yj(n) - Yu (n»-l
·1
[Yu(n)' Yj (n»
n.
But
so t:i1at for sufficiently large
n
gn d "
m, [Yu(n)'
8
= f(~)-l
If Zi
on
(-
01)
,
• F(Yi) , i
= 1,
f
is nondecreasing
m)
so that for sufficiently large
~
fn(X) ~n
If k
2, ••• , n , then since
= f(~)-l
-1
n
• (j(n) - u(n»
1
• (Zj(n) - Zu(n»- •
, then
n- l • (j(n) - u(n»
Finally, for sufficiently large
~~OI) Zu(n)
=
If(z
) - !f(Z
)
n j(n)
n u(n)
n
l
• F(Xl ) , lim inf [Zj(n) - Zu(n)J > 0 .
Thus because of the uniform convergence of If to If,
Since
=
[f(xl)J-
n
lim
n-+
01)
[~(Z
n j ( n) ) - ~(z
n u(n) )J. (Z j (n) - Zu(n) )-1
=
-l~~
= f(x ) ,
l
so tha.t
lim inf
f n (x) -> f(Xl )
•
In the discussion of section 3, we saw for x < a' , f (x) < f(x) .
m
Hence we have
f (x-) < lim inf f (x)
m
n
for
x < a' •
-
Using the representation given by
(4.1),
9
we can show in a similar manner
that
f(x+) > lim sup
-
f n (x)
x <a
Again by the discussion of section 3, for
at < X < m , fm(X)
~
Clearly then for
f(x) •
x <m•
for
fm(x)
= f(x) and for
y <a
o
f (y -) < lim inf f (y ) < 11m sup f (y ) < f (y +).
mo nono-mo
Moreover, for
at < x < m ,
A
(4.6)
f (x+) > lim sup f (x)
m
n
Now let Yo be chosen in
(a, m).
XJ. < Yo < x2
and a <
XJ. < at
Then there is a pair
and at < x2 < m.
f (x -) < lim inf f (y ) < lim sup
m l
noSince
f m is constant on
f n(yo) -<m
f (x +).
2
[a, mJ , we have
(4.3)
This provides the desired conclusion,
the support of f.
(xl' x ) such that
2
Clearly by (4.5) and (4.6)
if Yo
If fey -) = 0 , the result is clear.
o
is in the interior of
The remaining cases
may be proven in a similar manner.
A
If
f
is continuous, then lim f (y ) = f(y)
n-.co
n
0
0
for
y
0
f
m.
Using
this and applying methods similar to those of the Glivenko-Cantelli Theorem,
we have the following corollary.
Corollary 4.1.
hold, then f
n
If f
converges to
is continuous and the conditions of Theorem 4. 1
f
m
uniformly except on an interval of arbi-
10
trari1y small measure containing m.
This holds for all points in
0'
,
hence vdth probability one.
If the limit of the sequence (M) is
n
M, which is the case if M
n
is a
strongly consistent estimate of the mode, then we may state the following
corollary.
4.2. If (M)
converges to M, the mode of f, with probn
Corollary
ability one, then with probability one,
for
x <M
and
A
A
f(X-) > lim sup f (x) > lim inf f (x) > f(x+)
-
n
-
x > M•
for
n-
A
In addition, if
f
is continuous, with probability one
f
n
converges to f
uniformly except on an interval of arbitrarily small measure containing M.
Of course,
fn
also may have the maximum likelihood property mentioned
A
in section 2.
The maximum likelihood property causes
so that if f
is continuous we may be willing to sacrifice the maximum
.
f
n
to be a step function,
likelihood property in order to obtain a continuous estimate.
our continuous estimate is sufficiently close to f
estimate will also be consistent.
f
n
continuous.
Theorem
, then our continuous
Clearly, there is no unique way to make
In the following, let
f
be a continuous density.
4.2. If (fn*) is a sequence of continuous density functions
such tI1at,
1.
f n * is unimodal with mode
2.
f *(M ) =
n
n
If we assure
n
f n (Mn )
,
Mn ,
11
If f
is constant on B , ••• , B ,
k
n
l
once on each B , j = 1, ••• , k ,
then f n*
= fn
at least
j
then ,'lith probability one
f n* converges to
f
uniformly except on an in-
terval of arbitrarily small m:!asure containing M.
Proof:
Let A be the interval of artibrarily small measure.
Corollary 4.2,
choose
sup
c If (x) - f{x) I
xeA
n
n sufficiently large so that
sup
xe A c
converges to zero.
Let
By
£>0
and
Ifn (x) - f{x) I < £/3 •
It is easy to see that conditions 1, 2, and 3 are sufficient to imply for
any x in the support
Ifn*(x) - f n (x)
I -< supy.
~
f n (y.-) I ,
Ifn (y.+)
~
~
m1ere y. , i = 1, ••• , n are the set of observations.
~
tracting f{Yi)
But adding and sub-
and using the triangle inequality,
sup
c If (y.+) - f (y.-) I < 2sup
n ~
n ~
y.eA
xe AC
Ifn (x)
~
Hence
sup
xeA
or for sufficiently large
c Ifn*(x) - f n (x)
I =s
n
sup
c Ifn*(X) - f(x)
xeA
This completes the proof.
(2/3)e ,
I =s
£.
- f{x) I •
12
5. Acknowledgements.
I wish to thank Professor Tin Robertson for
suggesting the topic from which this work arose and for his guidance as my
thesis advisor at the University of Iowa.
13
REFERENCES
[lJ
BRUInc, H. D. (1963). "On an extension of the concept of conditional
expectation," Proc. Amer. Math. Soc. 14, 298-304.
[2J
BRUNK, H. D. (1965). "Conditional expectation given a cr-lattice and
applications," Ann. ~. Statist. 36, 1339-1350.
[3J
NADARAYA, E. A. (1966). "On nonparametric estimates of density
functions and regression curves," Theory Prob. ~. 10, 186-190.
[4J
ROBERTSON, Tim (1966). "A representation for conditional expectations
given
[5 J
cr-lattices,"~.
Math. Statist. 37, 1279-1283.
ROBERTSON, Tim (1967). "On estimating a density which is measurable
with respect to a cr-lattice, "!E!!. Math. Statist. 38, 482-493.
[6]
VENTER, J. H. (1967). "On estimation of the mode~ ~. Math. Statist.
38, 1446-1455.
[7J
WIDMAN, E. J. (1968).
.Q!l Estimating! Unimodal Density, an unpub-
lished dissertation at the University of Iowa.