RANGE-PRESERVING UNBIASED ESTIMATORS
IN THE MULTINOMIAL CASE
by
Wassily Hoeffding
University of North Carolina at Chapel Hill
Summary
£ = (PO,PI,
Consider estimating the value of a real-valued function f(£),
,Pr)' on the basis of an observation of the random vector
X = (XO,X ,
,X r ) whose distribution is multinomial (n,£). It is known
I
that an unbiased estimator exists if and only if f is a polynomial of degree at most n, and in this case the
unbiased estimator of f(£) is unique.
However, in general, this estimator has the serious fault that it is not
range-preserving; that is, its values may fall outside the range of
feE).
In this paper, a condition on f is derived which is necessary for the unbiased estimator to be range-preserving, and is sufficient when n is large
enough.
l
Key words and phrases:
Range-preserving estimator, unbiased estimator,
prior range, posterior range, binomial distribution, multinomial distribution.
1.
Introduction
For clarity, the results will first be stated and proved for the bi-
nomial case.
The extension to the multinomial case, which is straight-
forward except for the more complicated notation, will be dealt with in
the last section of the paper.
Let the random variable X have the binomial (n,p) distribution,
O~p~l,
and consider the unbiased estimation, based on an observation of X,
of a real-valued function f(p).
It is known that an unbiased estimator of
f(p) exists if and only if f is a polynomial of degree at most n, say
f(p)
(1.1)
where
m~n.
Since
p
k
n x(k) n x
n-x
= I -crf (x)p (l-p)
, k=O,I, ... ,n,
x=O n
where n(k) = n(n-l) ... (n-k+l), an unbiased estimator t (X) of f(p) in (1.1)
n
is given by
m
(1.2)
t
n
(x)
=I
x=O,l, ... ,n;
n~m
.
k=O
By the completeness of the binomial family (Lehman (1959))
this is the only
unbiased estimator of f(p).
In general the values of t (x) can fall outside the range of f(p).
n
Thus the unbiased estimator of p(l-p) is x(n-x)/n(n-l), and its maximum
with respect to x exceeds 1/4 = max p(l-p).
The main results for the binomial case, nroved in Sections 3 and 4
respectively, are the following two theorems.
Theorem 1
Let f be a non-constant polynomial.
In order that a range-
page 2
preserving unbiased estimator of f(p), Ospsl, exist, it is necessary that
either
(a)
f(O)<f(p)<f(l) for O<p<l; f'(O»O, f' (1»0,
(b)
f(I)<f(p)<f(O) for O<p<l; f' (0)<0, f' (1)<0.
or
The necessary condition of Theorem 1 is not sufficient for the unbiased
estimator t n (x) of f(p) to be range-preserving whenever n
~
deg(f).
Thus
the polynomial f(p) = (p_c)3, O<c<l, satisfies condition (a), but for c>I/2,
2
t 3 (1) = c (I-c) > (I-c)
Theorem 2
of Theorem 1.
3
= f(l).
However, we have the following result.
Let f be a polynomial which satisfies
Then there exists a number N(f)
~
estimator tn(x) of f(p) is range-preserving for n
condition (a) or (b)
deg(f) such that the unbiased
~
N(f).
The needed definitions are given in Section 2.
2.
Definitions
Let P be a family of probability distributions on a measurable space
(X,A) and consider estimating the value of a function e(p) defined for
PEP which takes values in the k-dimensional euclidean space
mator t(x) of e(p) is a measurable function from X to~k.
It.
An esti-
In accordance
with [1], the set 8 = {e(p): PEP} is called the prior range of e(p). Informally, the posterior range of e(p) given the observation x from X, denoted
8 x ' is the least set in which e(p) is known to lie when the value x has been
observed.
A general definition of the posterior range is given in [1]. In
the present binomial case, P = P is the family of the binomial
n
distributions P , OSpSI, with n fixed.
p
of f(p) is 8 = {f(p): OSp$I}.
8
x
= {f(p): O<p<l}
Let f(p) = e(p).
The posterior
p
rang~
8
if x=I,2, ... ,n-l,
(2.1)
8 = {f(p): OSp<I}, 8 = {f(p): O<psl}.
n
0
x
(n,p)
The prior range
is given by
page 3
An estimator t(x) of e(p) is said to be range-preserving if its values
are confined to the posterior range of e(p): t(x)
3.
€
0 ,
X
X.
X €
Proof of Theorem 1
We need the following result ([1], Proposition 3.2), stated in the setting
described in Section 2.
Proposition A Suppose there are two members Po and PI of P such that
a)
the convex hull of 8 has a supporting hyperplane II at the point e(p )'
o
and
PI is absolutely continuous with respect to PO' and e(P ) t H.
l
b)
Then no unbiased estimator of e(p) is range-preserving.
In our problem the prior range 8 of f(p) is the closed interval [min
{f(p);
O~p$l},
max{f(p): O$p$l}].
[0,1] at a point PO'; 0,1.
such that
so chosen.)
f(Pl)'; f(po)'
Suppose f(p) attains its maximum
in
Let PI be any point in the open interval (0,1)
(Since f is a non-constant polynomial, PI can be
The binomial distributions (n,Po) and (n,Pl) are absolutely con-
tinuous with respect to each other.
Proposi tion A
(with H being the point
f(pO)) implies that the unbiased estimator of f(p) is not range-preserving.
Hence, in order that
. or <f(l) for O<p<l.
it be range-preserving, it is necessary that f(p)<f(O)
Similarly, we must have f(p»f(O) or >f(l) for O<p<l.
This proves the necessity of the first conditions in (a) or (b).
To complete the proof, first assume that f(O)<f(p)<f(l) for O<p<l.
It is sufficient to show that the unbiased estimator t (x) of f(p) is rangen
preserving only if f' (0»0 and f' (1»0.
According to the definitions in Section 2, for t (x) to be range-pren
serving it is necessary that tn(l»f(O) and tn(n-l)<f(l).
By (1.2) and (1.1),
page 4
t (1)
n
=
aU
tn(n-l) =
a l l1
+
m
I
o
ak -
-1
m
I
0
=
f(O)
k~n
-1
+
f' (O)n
-1
,
1
= f(l) - f' (l)n- ,
and the stated conditions follow.
For case (b) the completion of the proof is similar.
Note that the necessity of the first condition in (a) or (b) of
Theorem 1 has been deduced from Proposition A and deals only with the
prior range of f(p).
In contrast, the conditions involving the signs of
the derivatives of f at 0 and 1 have been obtained from the requirement
that the values of the estimator must be confined to the posterior range
of f(p).
4.
Proof of Theorem 2
We shall assume that the polynomial
(4.1)
(m~l)
satisfies condition (a) of Theorem 1:
(4.2)
f(O)<f(p)<f(l) for O<p<l; f' (0»0, f' (1»0.
The case (b) can be reduced to case (a) by a simple change of notation.
We must show that there exists an integer N(f)
~
deg(f) such that for
I
n
~
N(f) the unbiased estimator
m
(4.3)
t (x) = I a x(k)/n(k)
n
k=O k
is range-preserving.
(4.4)
The latter is true if and only if
f(O)<t (x)<f(l), x=l,2, ... ,n-l,
n
page S
f(O)~tn(O)<f(l),
(4.5)
f(O)<tn(n)~f(I).
By (4.3), tn(O)=aO=f(O) and tn(n)=r~ak=f(I), so that conditions (4.5)
are satisfied.
holds for n
It remains to show that there is an N(f) such that (4.4)
N(f).
~
We shall first show that there is a positive E=E(f) such that
(4. b)
flO)<t n (x)<f(l) if O<x/n<£ or l-E<x/n<l.
The proof will be completed by showing that for every £>0 there is a number
N=N (f ,E) such that for n2:N
f(O)<t n (x)<f(l)
(4.7)
Since x
(k)
/n
(k)
~
if £
~
~
I-E.
k
,m
(k) (k)
(x/n) , we have tn(x)-f(O) = Llak x
/n
I~ ~ (x/n)k, where a+ = max(a,O).
L~ a~ £~
x/n
~
Choose £1 = EI(f) > 0 so that
< f(I) - f(O).
Then
t
Also, with a
n
(x) < f(l)
= max(-a,O),
where al=f l (0»0.
Choose £2 = E2 (f) > 0 so that
Then
t
n
(x) > f(O)
if 0 < x/n
S; E
2
•
In a similar way one shows that the positive number £3(f) can be so
chosen that
page 6
f(O)<tn(x)<f(l)
if 0 < (n-x)/n S £3(f).
Setting £(f) = min'=l 2 3£' (f), we have shown that for every polynomial f which
1
"
1
satisfies condition (4.2) there is a positive £=£(f) such that (4.6) holds.
Let £
E (O,~).
(4. 7) holds for
We must show the existence of a number N=N(f,E) such that
n~N.
We have
It can be shown by induction on k that
k
(4.8)
Since x(n-x)/n
2
x(k)
x
x(n-x) ,
k - (k) S (k)
2
2
n
n
n (n-l)
0 S
k=2,3, ... ,n.
S 1/4, we obtain
1E = max{f(p) :ESf(p)SI-E}, -s
f = min{f(p) ~
Note that f >f(O), 1 <f(l). Hence we can choose a positive
-€
E
For a fixed E
ESf(p)SI-E}.
E
(O,~) define
n = n(f,E) such that
f£ - n
IE
> f (0) ,
+
n < f(l).
Now choose N = N(f,E) so that
Then if
E S
x/n
f
1-£ and n?N,
S
£
-
n
S t
n
(x) S
1£
and therefore f(O) < t n (x) < fell.
+
n
This proves the statement around (4.7). II
page 7
5.
The multinomial case
Let the random vector X = (XO,X , ... ,X ) take values in the set X of
r
l
points
~
= (XO,x '" ,x r ) with nonnegative integer-valued coordinates whose
l
sum is n, and let X have the multinomial (n,E) distribution:
x.
n!
r
1
l: ,
TI o Pi' X € X, E.€ -r
r
TI Ox ·I
i
(5.1)
pdx=x} =
(5.2)
l:
-r = {p.. = (PO" .. ,P r ) : Po
2:
0, ... 'P r
2:
0, I~ Pi = U.
It will be assumed that the unknown parameter vector E. varies over the
entire simplex -r
l: , the dimension r as well as the sample size n being fixed.
Consider the unbiased estimation, based on an observation of
value of a real-valued function f(E.), E. €
~r'
!, of the
As in the binomial case, an
unbiased estimator of feE) exists if and only if f is a polynomial of degree at most n, and then the unbiased estimator is unique.
The latter
follows from the completeness of the multinomial family (Lehmann [2], p 132).
We shall be concerned with the local behavior of f at the vertices
01"\,0- , ... ,0- r of ~
l: , where O. = (0'0,0'1, ... ,0. ),0 .. = lor 0 according
-1
1
1
1r
1J
1
i
as i = j or if. j. For i = O,l, ... ,r, let:e.. = (PO,···,p·1- I'P.1+ 1'''''P)
r
---v
be the vector E. with the component p. deleted.
1
i
Denote by f (E.) the polyi
nomial f(E) with p. replaced by I - I·4'P.
and write
1
1
Jr
J
jl
ji
ji+l
jr
Po ... Pi-l Pi+l "'P r ' i=O,l, ... ,r,
(5.3)
where the sum I m is extended over the nonnegative integers jl, ... ,j r whose
i
sum is ~ m = deg(f). Thus (5.3) is the Taylor expansion of fi(E. ) at
£
i
= (0, ... ,0).
vertex
o.
-1
of l:.
It may be thought of as the expansion of f(£) at the
page 8
Note that a
i
i
i
= f. (0, ... ,0) = f(8.), and a ... 0,···,a ...
lO
0 ... 0
1
--:J..
O 01
are the first order partial derivatives of f. at the point £i
1
Due to the identity
r (j i)
j .
ITO
xi
r
1
(5.4)
IT p. = L
0 1
X€x
(t~j i)
n
n!
IT
it follows from (5.3) that forn
of f(£),£
€
Ir
IT
r
x. !
0 1
~
=
(0, ... ,0).
r x.1
p. ,
0 1
deg (f) the unbiased estimator t (x)
n-
can be written in each of the forms
(5.5)
i=O,I, ... ,r,
whereX
i
= (XO,···,X.1- I'X,1+ 1""'X)'
r
The following two theorems are extensions of Theorems I and 2 from
the binomial to the multinomial case.
Theorem 3
bution,£
€
deg (f)
n.
s;
Let the random vector! have a multinomial (n,£) distri-
~, and let f(£): ~
~l,
+
be a non-constant polynomial,
In order that the unbiased estimator t n(x) of fen)
be rangeL
preserving it is necessary that
(i)
both the maximum and the minimum of f(E) in
at one (or more) of the vertices of
(ii)
~
-r
~
-r
are attained only
;
if the minimum of f(£) in -r
~
is attained at the vertex -1
8., the
first partial derivatives of f i (£i) at £i = (0, ... ,0) are all positive;
(iii)
if the maximum of fen)
in -r
~
is attained at the vertex 0.,
the
L
J
j
j
first partial derivatives of f j (E. ) at E. = (0, ... ,0) are all negative.
page 9
Let f be a polynomial which satisfies the conditions of
Theorem 4
Theorem 3.
t
n
Then there exists a number NCf) such that the unbiased estimator
of fCE) is range-preserving for n
Proof of Theorem 3:
~
NCf).
The necessity of condition Ci) follows from
Proposition A of Section 3.
Suppose that the minimum of fCE) in L is attained at the vertex ii'
Then, for t
tnC~)
that
for
n
Q, ~
(20
to be confined to the posterior range of f, it is necessary
> fCt\) if xi
= n-l (hence
~
=
1 for some k ~ i and XQ,
=
0
i,k). By (5.5) this implies condition (ii).
The necessity of condition (iii) follows in a similar way.
Proof of Theorem 4:
that if
o.
~
~n
As in the proof of Theorem 2, it is first shown
is in a sufficiently small e-neighborhood of one of the vertices
at which fCp) attains its maximum or its minimum, but x/n ~ 0., then
1
min f(p) < t n (!J < max fCE). Here conditions (ii) and Ciii) of Theorem 3
. 1S
. sown
h
'"
2 <
\r. ~ m, we h ave
d
Th en 1t
are use.
th
at '1 f ~/ n E ~r'
- LlJ
i
(5.6)
X. j.
nr1 (2..)
1
n
I
_
nr1 xi(j1')hn (rrl j1·)
where Cr,m depends only on rand m.
I~ C
For r
n- 1
r,m
= 1,
'
(5.6) follows from (4.8);
in the general case it can be proved by induction on r.
(5.6) implies
C5.7)
x
I f Cn) -
t n (!J
I
~ C (f) n
-1
,
Since
page 10
where C(f) depends only on f.
Using (5.7), the proof is completed by an argument similar to that used
in the proof of Theorem 2.
References
[1]
Wassi1y Hoeffding (1983) Unbiased range-preserving estimators.
schrift for Erich L Lehmann, Wadsworth, Belmont, pp 249-260.
[2]
E. L. Lehmann (1959)
A Fest-
Testing Statistical Hypotheses, Wiley, New York.