The research for this work was partially sponsored by the National Science
Foundation under Grant No. GU-2059 and the United States Air Force under
Contract No. AFOSR-68-1415.
MAXIMUM LIKELIHOOD ESTIMATION OF A UNIMODAL DENSITY, II
by
Edward J. Wegman
Department of Statistics
University of North Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 647
NOVEMBER 1969
Maximum Likelihood Estimation of a Unimodal Density, II
by
Edward J. Wegman
1.
Introduction.
Several authors, Grenander [3], Robertson [6], and
Rao [4], have described the MLE for a unimodal density when the mode was
estimate~
known as well as some of the
properties.
The MLE for a unimodal
density when the mode is unknown was described in [7].
was also established in [7].
Strong consistency
We wish to describe some additional properties
in this paper.
2.
timate with unknown mode and
Yl
YZ
<
~
In defining
known mode.
< •••
Y
n
<
and let
Al(n)+l
=
[Ln,Rn ], Al (n)+2
R -L
n n
E·
the sequences
smaller than
Yl(n)
L
n
Al
=
f
tively; and
,
n
*
E >
n
be the maximum likelihood es-
the maximum likelihood estimate with
0
Let
was a predetermined number.
be the ordered observations sampled according to the
density
'
n
f
~
Let
Asymptotic Distribution.
and
[y l ,y 2 ),
=
A2
=
[y 2 ,y 3 ), .•• ,
(Rn'Yr(n)]' •.. , ~
{L}
n
Yr(n)
and
{R}
n
=
Al(n)
=
(Yn-l,Y n ].
converge to
[Yl(n),L n ),
Here
Land
R respec-
are respectively the largest observation
and the smallest observation larger than
Rn .
L
n
and
R
are determined by the maximum likelihood procedure and at least one of
L
n
or
n
R
n
is an observation for each
n.
If
L([Ln ,R])
n
is the a-lattice
The research for this work was partially sponsored by the National Science
Foundation under Grant No. GU-2059 and the United States Air Force under
Contract No. AFOSR-68-l4l5.
2
of intervals containing
n
~
the maximum likelihood estimate,
[L ,R ],
n
A
E(g IL([L ,R ]»,
n
n n
is given by the conditional expectation,
n
,
where
k
L
=
n
•
i
-1
[nA (Ai) ]
• I
A
i
i=l
Here
n
and
I
is the number of observations in
i
is the indicator of
Ai
Ai'
A is Lebesgue measure
Ai.
In a similar manner, let
Ai * = [Yl'Y Z), .•• , Aq(n) = [Yq(n),M),
Aq(n)+l = [M'Yq(n)+L]' Aq (n)+2 = (Y q (n)+1'Yq (n)+2]' •.. , An = (Yn-l,Y n ]·
Here
M is the known mode and
than
M.
Yq(n)
is the largest observation smaller
Notice with probability one,
the a-lattice of intervals containing
f *, is given by
n
E(g */L(M»
M~ Y
j
M,
for each
If
j.
L(M)
is
the maximum likelihood estimate,
where
n
n
gn
*
L
n
i
-1
•I *
* . [nA(A.*)]
A•
1.
1.
i=l
Of course,
n.*
shown that
M E (L,R),
1.
[y O( )'Y ()].
~n
rn
is the number of observations in
and
hence
g *
n
1.
agree except possibly on
A similar situation was the case in Lemma 5.4 in [7].
If we require only that some neighborhood of
points of increase of
N ,
R
In [71, it is
A.*.
f
L,
say
N ,
L
is a set of
and similarly some neighborhood of
is a set of points of decrease of
f,
R,
say
we may use the arguments of
Lemma 5.4 in [7] to obtain
Lemma 2.1.
R+ n
Let
are elements of
n > 0
N
L
one, for sufficiently large
be an arbitrary number such that
and
n,
N respectively.
R
~n
and
f *
n
L-n
and
Then with probability
agree on
c
(L-n,R+n) •
3
Hence, for any
x
4
[L,R],
for sufficiently large
~ n (x) = f n *(x).
n,
An immediate theorem follows
Theorem 2.1:
tribution as
For
x
4
~ (x)
[L,R],
n
has the same asymptotic dis-
f *(x).
n
Rao [4] through some very clever, but rather tedious arguments develops the asymptotic distribution of
~ (x),
could be applied to
n
f *(x).
n
Arguments similar to these
but are avoided by use of Lemma 2.1.
assumes a non-zero derivative of the density,
f,
at each point
Rao
x
where
the asymptotic distribution is to be found.
3.
A Characterization of
~.
Reid (see [1] and [2]) gave a geo-
n
metrical interpretation of a conditional expectation with respect to a
a-lattice,
L, when L consists of intervals with the right (or left)
endpoint fixed.
L(M) ,
If the a-lattice is
the conditional expectation
may be characterized by applying Reid's method individually to the right
and to the left of
of some function
H(x)
= J(_oo,x]hdA.
M.
h
E(hIL(M»,
To find
L(M),
with respect to
To the left of
of the greatest convex minorant of
M,
and
R be fixed with
h
R-L
determine
E(h!L(M»
the conditional expectation of
M,
by the
H.
has bounded support,
= E.
is given by the slope
H and to the right of
slope of the least concave majorant of
Let us assume that
the conditional expectation
{x: h(x) f
a}.
Let
L
We want a geometrical interpretation of
h
with respect to
L([L,R]).
Robertson
4
[5] gives a representation of conditional expectations.
This represen-
tation holds on finite measure spaces, hence the requirement that
bounded support.
H
Let
= {L* E L: A(L*-P y )
>
=
y
E(hIL)(x)
a}.
=y
and
P
Y
L
=
sUPL*EH [A(L*-P )]-1 •
L([L,R])
and
y}.
>
has
Let
Then
Y
Letting
= {x: E(hIL)(x)
h
x E [L,R],
jL*-P
hdA.
Y
it is clear that
P
is empty,
y
so that
In fact, this supremum is a maximum and
maximizing interval, for all
A(L*)
-1
fL* hdA.
sUPL*EH (A(L*» -1 •
y
• fL*hdA.
Let
a
=
xEL*,
inf L*
H
= L([L,R]).
E(hIL([L,R]»(x)
and
b
=
the conditional expectation with respect to
sup L*.
L(M) ,
If
L*
=
As in the case of
it is not difficult to
see we may apply Reid's method individually to the left of. a
right of
b.
is the
and to the
Thus we have,
Theorem 3.1:
The conditional expectation of a function,
bounded support, with respect to a a-lattice,
L([L,R]),
h,
with
is given by the
following procedure.
Find the interval
(H(b) - H(a»/(b-a)
tion is given by
[a,b]
containing
is maximized.
On
(H(b) - H(a»/(b-a).
slope of the greatest convex minorant of
[L,R]
[a,b],
h
A
= gn'
Let
A
the conditional expecta-
To the left of
a,
A
= f( -oo,x ]gn dA.
b,
H.
the theorem applies since the support of
Gn (x)
it is the
H and to the right of
is the slope of the least concave majorant of
If
such that
is
it
5
Corollary 3.1.
F,
n
If
e
in Theorem 3.1,
n
may be replaced by
the empirical distribution function.
The proof is straightforward and is left to the reader.
It is inter-
esting to note that Theorem 3.1 implies Theorem 3.1 of [7] if the condition of
port.
f
being continuous is exchanged for
f
having bounded sup-
The author is indebted to the referee of [7] for pointing out this
fact.
References.
[1]
Brunk, H.D. (1956), "On an inequality for convex functions,"
Proc. Amer. Math. Soc., 2, pp. 817-824.
[2]
Brunk, H.D., Ewing, G.M., Reid, W.T., "The mimimum of a certain
definite integral suggested by the maximum likelihood estimate of a distribution function," Bull. Amer. Math.
Society, Abstract 60-6-684.
[3]
Grenander, Ulf. (1956), "On the theory of mortality measurement,"
Skan. Aktuarieltidskr., 39, pp. 125-153.
[4]
Rao, B.L.S. Prakasa (1966), "Asymptotic distributions in some nonregular statistical problems," Technical Report No.9,
Department of Statistics and Probability, Michigan State
University.
[5]
Robertson, Tim (1966), "A representation for conditional expectations
given a-lattices," Ann. Math. Statist., 22, pp 1279-1283.
[6]
Robertson, Tim (1967), "On estimating a density which is measurable
with respect to a a-lattice," Ann. Math. Statist., 38,
pp 482-493.
[7]
Wegman, E.J. (1970), "Maximum likelihood estimation of a unimodal
density function," to appear Ann. Math. Statist., April, 1970.
Maximum Likelihood Estimation of a Unimodal Density, II
by
Edward J. Wegman
Abstract.
This paper is a sequel to the earlier paper, "Maximum
Likelihood Estimation of a Unimodal Density Function."
The MLE of a uni-
modal density with unknown mode is shown to agree, for sufficiently large
n
and on certain regions, with the MLE of a unimodal density with known
mode.
The asymptotic distributions of the MLE's then agree.
Also a geo-
metrical interpretation of the MLE of a unimodal density with unknown
mode is given.
Keywords:
Maximum Likelihood Estimation
Unimodal Density
Asymptotic Distribution
Geometric Characterization
Mode
Subject Classification Numbers:
62.15
62.70
60.05