Gertrude NI. Cox
•
UNBIASED ESTIMATION OF FUNCTIONS OF THE BINOMIAL PROPORTION
by
:SuJ1tKumar.Mitra
This research was supported, in
part, by the United States Air
Force through the Office of Scientific Research of the Air Research
and Development Command
Institute of Statistics
Mimeograph Series No. 122
Pebruary 1, 1955
UNBIASED ESTIMATION OF FUNC'IJONS OF THE BINOMIAL PROPORTION*
by
~
·Su·;j it Ktmlal' Mitr'a ;. ..".
Indian Statistical Institute and Uhiversity of North Carolina
1.
Summary.
Explicit characterisations are given for the class of esti-
mable functions (EPF) of the parameter p in the Binomial Population and the
subclass G of EFF's. possessing uniformly minimum variance unbiased estimators
(UMVUE).
For EFF's not possesing UMVUE, the explicit expression for the
unbiased estimator with minimum variance at some p'E.Cl
space) is given.
(the parameter
The subclass G* of G of EFF's possesing unreasonable
UMVUE has been considered when
.C1 = L- 0,1_7.
Such estimators are neces-
sarily inadmissible in the Waldian senae, with any loss function which
increases with the absolute error.
No attempt is however made either to
prove or disprove the admissibility of reasonable UMVUE (in general) in
such a case except for f(p)
=p
with loss function proportional to squared
error, when it is shown that the reasonable UMVUE for p is admissible also.
2.
Introduction.
In a parametric point estimation problem, we are usually given a sample
apace X, a family of probability ,measures ~ lJ. } eE 1-1 , defined over a
e
field ~x of sets in X and a single valued function fee) defined on 1-1
whose value is desired to be estimated on the basis of a random observation
*
This research was supported, in part, by the United States Air Force
through the Office of Scientific Research of the Air Research and Development Command.
2
An estimator t(~) of fee) is a mapping t (!) of X onto a space T (usually
a subset of
.el f ,
where
i-lf
1s theiinage of
specifying for each !€X, the number t"(E)tT,
o~:f(O}
, when
~
1-1 t.ui'de~
whi~h' will
the" transf.9nnB.t ion f),
bOe talten as an estimate
is observed
The estimator t(!) 1s said to be unbiased 1f
(2.1)
t(~)
for all e
€.Cl •
is said to be a locally minimum variance unbiased estimator (LMVUE)o'
of f(O) if t(!) is unbiased and if, for some Q'€
.(1 ,
(2.2)
for all other unbiaBed estimators
t'(~)
of f(O).
If the inequality (2.2) holds for all
01€1-1,
t(!) is said to be
a uni-
formly minimum variance unbiased estimator (UMVUE) of f(O).
An estimator t(x)
0' f(O) may be said to be reasonableL-3_7 if,
a set of probability measure zero (w.r.t. ~Q for all 0
of
€
i-1 _7,
excluding
T is a subset
i-1 f·
Any function f(O) is said to be estimable if an unbiased estimator t(x)
of f(O) exists.
Such a function will be called an estimable parametric
function (EPF).
3.
The Clas s of EPF for the Binomial Case.
Let (Xl' x2 ' ••• , Xn ) be n independent observations on the chance variable x, where
(3.1)
Prob {x
Prob {x
= 1 I p} = p,
= 0 I p} = I-p,
3
and where l' €
r
=
1-1 L- a.
subset
t-o,1_7
of
J.
It is well known that
n
L xi is a sufficient statistics for I' and also that
i=l
,
k
= 0,
1, ••• , n.
Hence, for any EPF- f(p} (the subscript n is introduced to indicate its possible
n
dependence on the sample size n), there should exist a function t(r) of r
satisfying
E { t(r) II'}
= Ln t(k)
n
k
n-k
(k) I' (1-1')
= f(p)
for ~ p€
1-)_ .
K=O
It follows from the completeness of the family of measures (3.2) when
1-1
consists of (0+1) or more distinct points L~Lehmann and8cheffeL-4_7_7 that
the relation
(3.3) should be an identity in
1', that:is
rep) should be uniquely
expressible in the form (3.3) or f(p) should be a ploynomial in p of degree
< n. Conversely, any polynomial in
f(p)
I' of degree
~
n is uniquely expressible as
=
and admits an unbiased estimator t(r)
= C.
r
Incidentally it also follows from
the completeness of the family (3.2) that this estimate is the unique UMVUE.
When
1-1
consists of only m ~ n dixtinct points, (3.3) is equivalent to
solving m equations in (n + 1) unknowns; hence any arbitrary function
or
ptlS
estimable and there are more than one function of the sUffieient statistics r,
each of which is an unbiased estimator
or r(p).
4
4.
The Class G of EPFn admitting UMVUE.
When
1-1 consists
of m ~ (n + 1)
distinct points we have seen in section 3 that any EPF n admits UMVUE.
only real interest is for the case where m < n.
The
we shall prove
Theorem 4.1.
For a
1-1 which
1-1 excludes the
constants, (b) if 1-1
consists of m ~ n distinct points (a) if
two end points 0 and 1, then G consists only of real
includes 1 but excludes 0, then G consists only of functions of the form
Co + clpn, (c) if
1-1
includes 0 but excludes 1, then G consists only of
functions of the form Co + C2 (1 - p) n and (d) if
1-1
includes both 0 and 1,
then G consists only of functions of the form Co + Clpn + C2(1-p)n, where
CO' Cl ' C2 are real constants.
Proof of (a)
1-1 be
Let the distinct points of
It is given that 0 < p.1 < 1, for i
= 0,
denoted by PO' Pl' ••• Pm-l'
Let Vro be the class
1, ••• , m - 1.
of all functions Z(r) of r such that
( 4.1)
E {Z(r)
I Pi} = 0,
for i
= 0,
1""1 m-l,
or
(1 - p.)n-k
1
(4.2)
= 0,
for
i
= 0,
or
n
L Z(k)aki
k;O
where
( 4.4)
=0
for i
= 0,
1, ••• , m - 1,
1, ••• , m-l,
5
e
Any Z(r)
€
VO may be obtained by assigning arbitrary values to Z(m), Z(mtl),
r
••• , Zen) and solving for the equations
m-l
L
k=O
n
Z(k)~i
= -L Z(k) a ki
k=m
= 0,
1, ••• , m - 1.
i 1 OJ' since Pi 1 Pj • Hence we have the
following expression for the determinant of the matrix of equations (4.5):
It is easy to see that:fo~ i
1 j,
i
Q
(4.6)
Hence (4.5) always leads to unique solution, once Z(m), ••• , Zen) are assigned.
As an illustration we may take Zl(k)
Zl(m) = 1.
= 0,
for k = (m +1), ••• , n and
Then Zl(O), Zl(l), ••• Zl(m - 1) vill be given by
m-l
L
i=O
",m
""'i
a
ki
k
= 0,
1, ••• , m - 1,
where
(4.8)
We shall now prove
Lemma 4.1.
Proof of Lemma:
Zl(k)
1 0,
for k
= 0,
1, 2, ••• , m - 1.
Suppose Zl(S) = 0 where S is an integer and 0 ~ 8 ~ m - 1.
Zl(O), Zl(l), ••• , Zl(8 - 1), Zl (8 + 1), ••• , Zl(m - 1) should therefore
satisfy
=-
(n)
m ami for i-.~O,l, ••• , m - 1.
6
«
Hence if A be the matrix
a ki
))
o ~ k ~ m-lik
o < i < m-l
«
augmented matrix
a
ki
o ~ k ~ mik
o < i < m-l
I BI
(4.10)
=
II
kf S
, then rank A should be the same as
S
(~)
o<k<m
0
:f
~ S
But it can be easily verified ;t~2_7 that
rank B.
where
))
of equations (4.9) and B be the
n-
II
i
(gi - OJ)
>j
on-8 '
i,j=O,l, ••• ,m-l
S is the (n-8)th elementary symmetric function
~ Q., Q.
~l
~2
, ••• Oi
(n-8)
of the O's and where the summation extends over all possible combination
i l , ••• i(n_8)
over integral values 0
S i1
< i 2 < ••• < 1P-8)
8ince Qi ' s are positive and distinct it follows that Ok
IB I f
(4.11)
>0
~
m _ 1.
and
O.
••• rank B = m, where as rank A can only be
~
(m-l).
Hence there cannot exist
Zl(k)'s satisfying (4.9) which proves lemma 4.1.
Z(r)
€
(4.12)
or
i-3_7
that if t(r} be the UMVUE of its expectation and if
o
0
V , then t(r) Z (r) €V. Therefore,
It is well known
r
r
i
= 0,
1, ••• , m-l,
7
(4.13)
n m-l m ki
t(k)Zl(k) = -t(m)( ) L ~i a
m i=O
= t(m)Zl(k),
for k
= 0,
1, ••• , m • 1.
Since Zl(k) ~ 0, it follows that
(4.14)
t(k) = t(m),
Let us next take Z2(r)
and Z2(m+l)
=1
€
= 0,
1, ••• , m - 1.
o
r ' where Z2(k)
V
=0
for k
= 0,
(m + 2) ••• n
and when Z2(l), ••• , Z2(m) are the solution of
m
k:l Z2(k} a ki
(4.15)
for k
= -a(m+l)i'
for i
= 0,
In a similar way we can show that Z2(k) ~ 0, for k
using the fact that t(r) Z2(r)
€
1, ••• , m - 1.
= 1,
2, ••• , m,
and,
V~ , we can show that in order for t(r)
to be an UMVUE it is necessary that
t(k)
=t
(m+l),
k
= 1,
2, ••• , m.
Proceeding step by step in an exactly analogous manner we can show that in
order that t(r) may be an UMVUE of its expectation, it is necessary that
teO)
= tel) = ••• = ten) = C
or, in other words, E iter)
I Pi} = c,
i
= 0,
1, ••• , m - 1, which proves
Theorem 4.1 (a).
Proof of (d).
Pi(i
=.:
1-1
consists of 0, 1 and (m - 2) other distinct points'
1, ••• , m - 2).
8
C
Notice that in this case in order that Z(r)€V , it follows from (4.2) that
Z(O)
= Zen) =
(4.16)
r
° and Z(k}, for k = 1,
n-l
~
k=l
Z(k)
••• , n - 1,
(n) k
)n-k
k Pi (l-P i
=
°
i
should satisfy
=.
I
1, ••• , m - 2
Proceeding exactly as in proof of (a) we may show that in order that t(r)
may be a UMVUE of its expectation
( 4.17)-
it is necessary that
t (1) = t(2) = ••• = t(n-l} = CO'
whereas, since Z(O)
= Zen) = 0,
we may assign arbitrary finite values
to
teo)
and ten) and still satisfy the condition t(r) Z (r) € VOl
r
If we put teO)
= Co +
C2 and ten)
= Co
+ C we have
l
which proves (d).
The proof of (b) and (c) is similar to that of (d) and is omitted for
brevity.
5. A Lower Bound to Variance of Unbiased Estimators of EPF n f(p) at p'€ 1-1
and the LMVUE p I of f(p).
When
1-1
easily verified that for any EPF
n
consists of m? (n + 1) points it can be --
r(p), and any unbiased estimator t(r} of t(p),
we have
Var {t(r)
I pI}
J-ie'shall therefore consider the case m < n.
9
Let
Pi
,for i
= 0,
From (3.3) we have, if
1, ••• , m - 1, be the m distinct points of
li for 1
= 0,
1-1 .
1, ••• , m - 1 be m real constants,
n
L t(k)
k=O
or
E {t(r)
Ll
r(l_ )n-r
( II)
r Pi
P
1
n r
1
n rIps}
(r) P ( 1-P ) -
s
m-l
= r.
s
1=0
l.f(p.)
~
~
o ~ s an integer
Hence, using Cauchy Schwartz's inequality, we have
(5.4)
where
and
a.1S ()
r
Therefore,
)n-r/ Pr( I-p )n-r •
= Pr(
1 I-Pi
S
S
~
(m-I).
10
The above method is essentially due to Bhattacharyya L-l_7
ot t(p} and show that the limit given by
we shall now obtain theLMVUE
P
(5.8) is attained.
s
In (5. 4 ) the sign of equality El.olds if 3 t(r) and 1 0 , 11' ••• ,lm_l such that
and E {t(r) I P j
= f(P ), j
}
j
= 0,1,
••• , m - 1,
(5.10)
But
j
E {ai (r)
s
I Pj J'"
E {a. (r) a. (r)
~6.
(5.11)
j
or
1.
~
= L f(Pj)C
j
ij
•
JS
= 0,
that is,
= 0,
I P S j=C ij
1, ••• , m - 1.
• Therefore,
1, ••• , m - 1,
.
11
Hence the LMVUE
of f(p) is given by
p
s
It can be easily verified that the LMVUE
Ps
for f(p) is unique.
6. Unreasonable UMVUE. we shall prove
Theorem 6.1.
If
1-1 = L-O,l-1,
the UMVUE of any EPF
n
the necessary and sufficient condition that
f(p) of the form LC.(~) pi(l_p)n-i be reasonable is
~
~
tba.t either
(6.1)
C
<C. <C
o -
~
-
or
n
for all i .- 0, 1, 2, ••• , n.
Proof:
Suff1c4.ency
Let C0
~ C i ~ Cn'
for all i = 0, 1, ••• , n.
The UMVUE of f(p) is given by
t(r) :: C
r
We have
(6.2)
Moreover, we have Co
= f ( 0)
and Cn :: f (1) •
Note that f(p) is continuous, and
Co = info
f(p), C
l
p€L-O,l}
Hence from (6.2) it is clear that t(r) is reasonable.
be shown in the other case also.
Necessit¥.
Let there be some integer
J,
such that
= sup
P€
rep)
L-o, 1_7
The same thing could
12
(6.4)
and t(r}
We shall show that t(j)
<
= Cr •
f(p) and hence t(r) is unreasonable.
inf
p€1-1
is continuous, inf
Since f(p)
f(p) is attained either at 0 or 1 or
p€i-l
at some p*, 0 < p* < lIn the first two cases, we observe that since f(O)
= Co
and fell
= Cn ,
we have
t(J)
=C
j
In the last case, since 0 < p* < 1 and teO)
r
= 1,
2,. .'•• , D-l..
we have inr
f(p)
> Cj , ten) > Cj and t(r)
= r(p*) = E iter) I p*}
~
Cj for
> Cj = t(j).
p€i-l
Hence t(r) is unreasonable.
When 3
integer j such that
o < j < n, Cj
it can be shown
si~ilarly
> CO'
Cj
> Cnand
that
t(j}
> sup
p€i-l
and hence t(r) is unreasonable.
rep)
Cj
~
Ci
1
= 1,
2, ••• , n - 1,
13
It is evident therefore that in both the cases
t(r) could be uniformly
improved (with any loss function which increases with the absolute error) if
t(r} is replaced by another estimator t*(r), where
t*(r}
= t(r)
t*(j)
= f = info
for r ,
J,
and
f(p) in the former case
l-1
pe
=r
= sup. f(p) in the latter case.
l-1
pe
•••
t(r) is inadmissible in the Wa1dian sense •
n
i
n-i
Example
(i) f(p) = (i) p (l-p)
,
o < i < n.
In this case t(r)
But sup
f(p)
=0
if r , i
t(i)
= (~)(~)i(n~i)n-i
= 1.
< 1.
l-1
pe
(ii)
= pq,
f(p)
Here t(r)
= nr~n-r~
n-1
sup
= "41 •
p q
q
= 1-p.
and
l-1
pe
But tCn;l)
= RD1
n)
n
and tC'2 4TIi-1)
if n is odd,
if n is even.
In both the cases the estimates are
>
i.
As an immediate corollary to the above theorem it follows that
14
Corollary 6.1 Lf, for some &, 0 < &< 1,
1-1
excludes anyone or both
segments (0, &) and (1-&,1), the UMVUE of any EPF
7. Admissibility of the UMVUE for p.
n
the
is unreasonable.
Here we prove that if
1-1 = L-0,1_7
and the loss function be proportional to the square of the error, the UMVUE
= rln,
for p, given by t(r}
is reasonable and also admissible
t(r) is reasonable is apparent from the fact that 0 ~
rln
~
1
L-5_7.
That
for all r.
As is clear from the earlier discussions the class of admissible estimators
is the subclass of all non-randomised estimators, based on the sufficient
statistics r.
Without any loss of generality we may take the loss function to be
= (p-&) 2 •
For any estimator t(r) the corresponding risk function will be givBn by
=
n
I.
r=O
(p-t(r»
2 n
r
n-r
( r )p (l-p)
corresponding to any a-priori distribution function
~(p)
of p the average risk
is
1
(7.3)
RtL-~_7
=
J
Rt(p)d~(p)
0
1
n
=
C.
r·-O
fer)
J
(p-t(r»
2
d~*(plr)
0
where fer) is the marginal frequency function for r and
aposteriori distribution of p for any given
r.
~*(p
I r) is the
15
Clearly the Bayes solution t~(r) (the estimator for which RtL-~_7 is a
minimum) is given by
1
t~(r)
~ pd~*(pl
•
r).
o
~(p)
Now it under
trated at p
=0
the whole mass of the a-priori distribution is concen-
= 1,
and p
fer)
=0
for all r
r 0 or n and hence from the
point of view of minimising RtL-~_7 it is 1mmaterial how we define t(r) for any
r
r0
or n and thus we have no unique Bayes solution.
>0
fer)
But if otherwise, then
for all r and in this case it is clear that there exists an unique
(7.4).
Bayes solution given by
Therefore, for any
~
for which the entire mass of the distribution is not
concentrated at 0 and 1, the Bayes solution
Corresponding to d~~(p) =
u
,1)
p8-lq8-l dp) 8
Bi.B,o
222
(7.4) is admissible.
>0
2
R (p).8 (p+q, )+(n-28 )pq
(n+-28) 2
t8
and
8
(7.7)
= 2n
2
- 0(8 ).
r
The risk function generated by UMVUE to(r) • 'n
-
(7,8)
R
t
o
(p)
=~ ,
is
we have
16
and the corresponding average risk
Now it possible let there exist an estimator t(r) uniformly better than
to(r), that is, let
Since R (p) = 0 at
to
p = 0 and • = 1, it follows that the strict inequality holds at a point
with strict inequality holding for at least one pel-1.
different from 0 or 1.
Again from (7.2) it is clear that both Rt{p) and
Rt (p) are continuous functions of p.
o
o
<a <b <1
Hence there exists
€
> 0 under
such that
mt(p)
< Rt (p) o
€,
whenever a ~ p ~ b.
From (7.10)and (7.11) it follows that
b
RJ-~8.J < Rti-~8.J -
(7.12)
<
J
d~8(p).
a
Now
b
J
b
d~8(p)
=
1282
( l5)
J
a
a
P8-1. ( I-p )8-1 dp
J
b
vs:.2r:l+f"ls:.
'u
= ~8
( rl+8)2
a
s:. 1
s:. 1
pU(l_p}U-
dp
17
J
b
~~
[p(1-pU-
1
2
dp + 0(8 ).
a
From (7.9), (7.12) and (7.13) it follows that
J
2
dp + O( 6 ).
a
for sufficiently small 6, RtL-~ 6
••
_7 < Rt
fact that tt) is the Bayes solution w.r.t.
Hence to(r) =
nr
L-~5-7
6
which contradicts the
~6(P).
is admissible.
References
L-l
-
7
Bhattacharyya, A (1946): "On some analogues of the amount of information
and their uses in statistical estimation~ Sankhya,8, l-l~,
201-218, 315-328.
L-2_7
Browne, E. T.
L-3_7
Lehmann, E. L. "Theory of estimation" (Mimeographed Notes recorded by
Colyn Blyth) University of California Berkeley 1949.
L-4_7
Lehmann, E. L. add Scheffe H. (1950)"Completeness, Similar regions and
Unbiased est1mation,"Sankhya,lO, 305,-340.
L-5_7
Wald, A.
"Introduction to the theory of determinants and matrices/,
Vol. I, Mimeographed Notes, University of Norhh Carolina.
(1950) Statistical Decision Functions, John Wiley and
Sons (1950).
Acknow legement •
My thanks are due to D. Basu for suggesting the problem to me and for his
valuable help in the earlier stages of the preparation of this paper.