Gertrude M. Cox
ON i'iINIMUlvI VrtRIANCE ESTIMATION OF LOCATION liND SCALE PARHMETERS
by
Suj1t
~umar
Mitra.
This research was supported in part
by the United States Air Force, through
the Office of Scientific Research of
The Air Research and Development Command.
Institute of Statistics
Mimeograph Series No. 126
February, 1955
ON MINIMUM VARIaNCE ESTIMATION OF LOOATION AND SCALE PARAMETERS
by
Sumit Kumar Mitrd
Indian Statistical Institute and University of NGrth Carolina
1.
Swwnary.
It is shown that uniformly minimum variance unbiased estimators
(UMVUE) of location and scale parameters, when they eXist, satisfy Pitman's configurational condition
L-4_7.
As an illustration the case of rectangular popula-
tion with range depending on location and scale parameters has been considered in
some detail.
2.
Introduction.
We often come acrQSS point estimation problems where the statis-
tical distribution concerned does not admit UMVUE for the unknown parameters.
In
such cases we require an alternative set of criteria for chosing an estimator
possessing locally desirable properties.
For purposes of estimating the unknown location parameter
parameter
a in any population, Pitman
L-4_7
~
and the scale
considers only such functions of the
sample observations X = (Xl' x 2 ' ••• , x ) which respectively satisfy the configuran
tional conditions.
(2.1)
(2.2)
for all real c and k
> O.
Among all functions which satisfy the above conditions he picks out as a
suitable estimator the one which has the minimum second moment about the unknown
parameter.
1. This research was supported in part by the United States Air Force, through
the Office of Scientific Research of the Air Research and Development Command.
2
I/hen a is known the condition (2.1) is replaced by
Similarly when A is known, (2.2) is replaced by
(2.4)
The above configurational conditions although appearing to be arbitrary at
first are in fact no lesa general than the condition of unbiasedness.
Girshick and Savage
L-2_7
have considered estimators with the translation
property (2.3) and have shown that if A is a translation (or location) parameter,
and is the only unknown
4It
pa~ameter
in the distribution, and the loss function be of
the form L(b - A), then, under certain further restrictions on L, the optimal estimator in this class of pure invariant
est1ma~ors
the best Pitman estimator) is unbiased and
(which incidentally happens to be
min~ax.
Dealing with a positive random variable whose distribution is known except for
some multiplicative constant
0,
(the known location parameter A may be assumed to
be zero) Blackwell and Girshick L-l_7·have shown that, if the loss on estimating a
by s depend only on the relative error
a/s,
there always exists a minimax estimator
for a which is a pure invariant estimator haVing the multiplicative property (2.4).
Nothing, however, is known about the admissibility of such estimators.
3. The UMVUE of Location and Scale Parameters. We shall prove
Theorem 3.1.
The UMVUE for location and scale parameters (when they are the only
unknown parameters) in any statistical distribution, if they eXist, satisfy Pitman's
configurational conditions.
3
Let us consider the case where both
proaability distribution of !
= (xl'
~
and a are unknown and let the joint
x 2 ' ••• , Xn ) be given by
x
... , -!!-)
a
-~
0.1)
We shall first prove
If 8(!) be the UMVUE for any
Lemma 3.1.
the UMVUE for
e(k~+
parame~ric
function
e(~,a),
c, ka) where k and c are real constants and k
8(k! + c€) is
>0
and
€
is the
n-vector (1, 1, •.• , 1)
It is easy to verify that the joint probability distribution of
if..
= (yl'
Y2' ••• , yn) ::: k x +
C€
is given by.
y
-k~-c
nka
).
Hence if,
then
E {8 c.~)
0.4)
}
= E {8 (k! + c € )}
= e (k~ + c, k a) •
Thus it can be easily seen that, there exists a one-one correspondence between
unbiased estimators of
estimator 8*(!) of
e(~,a)
e(k~+
8(~)
e(k~
+ c, ka) implying thereby that any unbiaBed
c, ka) can be uniquely expressed as,
8*(~)
where
and
= 8(k! +
is an unbiased estimator for
CE),
e(~,
a), and vice versa.
4
Now let
o(~)
for e(A,o), o*(!)
be the
m~VUE
=
+ c€) and 0i(!)
o(k~
for e(A,o),
estimators for e(kA+ c, kO).
~l (~)
= 0l(~
be any other unbiased estimator
+ c€) be the corresponding unbiased
Let V(A,O), Vl(A,O), v*(A,cr) and V1(A,o) be the
variances of 0, 0 , o*} o! respectively.
1
It follows from (3.2) that
(3.6)
v*(A,G)
o(~)
Since
= V(kA +
c, kO)
is the UMVTJE for e(A,cr), we have
(3.8)
V(A,O)
< Vl(A,o), for all A and
0.
Therefore,
v*(A,O)
< Vl(A,o), for all A and
0,
which proves lemma (3.1).
In particular, when e(A,o)
= A,
we see that if be!) be a UMVUE for A,
b(k! + c€) is a UMVUE for kA+ c and following an argument exactly similar to that
. case ~--~k~--~--b(kx + C€) - c is also a UMVUE for
in lemma 3.1, it can be shown that in th1S
A.
From the uniqueness property of the UMVUE
L-3_7, L-5j
it follows, therefore,
that
(3.10)
that is, be!) should satisfy condition (2.1).
The necessity of conditions (2.2), (2.3) and (2.4) in appropriate cases can be
similarly established.
We have thus a method of formal derivation of statistics having uniformly
mimimum variance among unbiased estimators of location and scale parameters.
5
4. Illustrations. Rectangular Population (Range depending on parameters).
(a)
Both
~
and cr are unknown.
Let the elementary probability density of the chance variable
(4.1)
for i
where a and
~
>a
= 1,
2, ••• , n.
are known constants, A, cr are unknown parameters and the parameter
space consists of the upper half of the Euclidean plane (to be denoted by R~).
In this case it is well known that
(4.2)
and
are minimal sufficient for A and cr and that, for
density of
e,
TJ
(~,cr) €
R~,
the probability
is given by
(4.4)
: : ° otherwise.
It is thus clear that the UMVUE for
functions of
e and
TJ
alone
~
and a, if they exist, must be explicit
L-3_i.
The only unbiased function of
e and
TJ which satisfy (2.1) and (2.2) are those
given by
b(~,q) =
TJ + b (}-TJ,O) ::: q.+ (e-q) b (1,0)
s(e,TJ)
s(e-TJ,o)
and
(4.6)
=
= (~-TJ)s(l,O),
6
where the constants b(l,O) and s(l,O) are properly chosen so that for all
E
(4.7)
LTl
+ (s-Tl) b(l,O)!A.,a} = A. ,
and
(4.8)
E
t (s-Tl)s(l,O) I A., a }
= a •
In fact it will follow from the following lemma that they are respectively the only
unbiased estimators of A and a based on sand T} and as such are the UMVUE for
A.
and a.
Lemma 4.1.
The family
cr
of distributions (4.4) for (A.,a)e R* is complete
L-3_7.
2
Proof.
t(~,T})
Let
(4.9)
be such that
f t(s,Tl) I A.,a}
E
0,
;
for all (A., a)€ HI',
2
that is,
A.+f3a
(
(4.10)
I
L
j
Meta
or, i f we put
.
,;
Q
2
to
t(~}TJ) g(~,T};A.,a)dsl dT);
°
rj
=
A. + f3a,
I;)
,'2
(4.11)
i
L
J
TJ
Therefore,
(4.l2)
1
==
t(~,T})(~_~)n-2de}
for all
(~,()')€ R~,
7
12(~1'°) =
(4.13)
9 +(~ -a)o
1
t(Sp:!'XS-.Ql)n-2 ds ==
J
9
1
Therefore,
1
(~-a)
(4.14)
= t(Q1
+
(~-a)o,gl) ~(?_a)0_7n-2
=0,
for all (Q1'
Hence i f cr >
(4.15)
t(9
0)
R; •
£
°
1
+ (~-a)cr, ~1) = t(A + ~cr, A +
acr) ;
0, for all real A )
that is,
(4.16)
t(~,~) = 0,
for all ~ > ~.
We may note howevor that
B
(~,~):~=~
t(~,~)
may be assigned arbitrary finite values .on the. set
Co
d-
.whichhas zero probability measure, with respect A ,which proves
~emma
4.1.
(b)
(J
known and oqua1 to 1.
In this case without any loss of generality we may assume a
A function
(4.17)
b(~,~)
~
0 and
~
=c
>
o.
satisfying ·(2.3) must be of tho form
b(~,~) =" ~ + b(~
- ~)
=~
+
b(R) where
R = ~ - ~.
Hence to determine the best unbiased invariant estimator of A we have to
choose b(R) in such a way that
~ +
b(R) is unbiased and has minimum variance
among all other estimators of a similar type.
8
(4.18)
Now lot, E {1')\R}= A + e(R), and,
(4.19)
(4.20) We havo E {1J+b(R)}
(4.21)
=
A + E(e(R)
and
2
+ b (R) .,,: 2o(R)o(R)
RJ= v(R)
E {( 1') + b(R) - A)2/
+ b(R»
If 1') + b(R) is to be unbiased, b(R) has to satisfy tho condition
E{e(R)
(4.22)
+
b(R)}
= o.
Hence from (4.21) it follows that the proper choice of b(r.) will be
R-c
b(R) = -e(R) ;~.
Hence the best unbiased
(4.24)
inv~riant
*
b (~, 1')=
estimator of A is given by
R-c ti +-.2)
1') + 2 = -r-
-
C
~
Let us now consider the following Gstimator for A:
1
n
n-l
n-l
= - --- ~ + --- ~,
=
where A is any point on the real axis.
O
if 1') < AO;
if A < 1') <
0-
-
~ ~ ~ ~
~
< A +
- 0
1')
+ C
C,
and
9
It is c3sily soen th.:lt b(t:;,Tj) is uniformly unl1i3sed and moreC"ver has zoro variance
at A = A ' Thus b~~(';,Tj) cannot be the UMVUE for A; hence by Theorem 3.1, in this
O
case, tho LMVUE for A does not exist.
In fact, Lehmann and Scheffe
L-3_7 have
shmTn that for such a population the
only estimable 9arametric functions possessing U11VUE are the real constants.
(c)
A known and oqual to zero.
In this case a function s(';,Tj) satisfying (2.4) is of tho form
(4.26)
Honce to determine the bost unbiased invariant ost;_mator for a one has to consider
functions of the typo .;h(t) and choose h(t) such that .;h(t) is unbiased and has
uniformly the smallest variance among all other
un~}iased
functions of a similar
type.
(4.27)
If
E(.;lt)
E(.; 2 1t)
= ae(t)
=:
cy 2v( t)
it follows therefore
(4.28)
= 1, and,
E(e(t)h(t»
E t ';h( t)
(4.29)
r2
t
2
= (J"2E h ( t )v( t ) J
2
= a L-Elh(t)N'V(t) -
e(t)
}
2
At'/V(t)
+
=
:f ~ ~ )
where A co E( e
(j
*
E(h(t)o(t»
_!
A2
2
E(o(t»7
vrtJ-
f + i _7,
Ar/;{t)
e(t)
2 L- E { h(t)'V'v(t)-
10
Hence the proper choice of h(t) will bo
(4.30)
and tho best unbiased invariant estimator will bo given by
*
S (s,~)
(4.31)
If a and
~
1
*
S (s,~)
S
s(r,~),
are of opposite signs,
(4.32)
~
e(t)
= r vet)
n+l
=n
5 ,
It is also welJ Imo"m that in this case
•
when simplified, will reduce to
5
where
5'
=:
max
t:
I B'
~
1"\
}.
is minimal sufficient for a and that the
family of Drobahilit·. r distributions of ) for
(J'
> 0 is complete.
n+l
Hence uSis the
unique UFVUE for a.
Now let a and 8 bo of the samo sign and, for simplicity, let us consider the
case a
= 1, A = 2.
In this case
*
8 (r,~)
will roduco to
£2 n+ 2 _t n+ 2_7
(4.33 )
L-2 n +l _t n+l _7
where
(l~. 34)
whero
Z
is. the
Riemann' 8 Zeta function
Hence
(4.35)
,
1\=
1
...
.192
~
n
+
0(1)
'"3
n
.
,
11
•
It may be noted that the variance of the best unbiased invariant estimation is
given by
°2 (t' '.
(4.36)
1)
°
Let us now consider the following estimator for
=
,
__£_+ n
n-l
n-l
T}
=
<
i f T}
00i
T}
:s s
~
211 and
if
where 00 is any point on the positive part of the real axis.
that s(£,11) is unbiased and has zero variance at
the UMVUB for
0.
° = 00.
It is easy to verify
Hence 5*(£,11) cannot be
we shall in fact prove a more general result similar to that of
Lehmann and Scheffe stated in 4(b).
Theorem 4.1.
For a rectangular population with range oa to
the same sign and if the parameter
space for
° consists
~a
if
and
Q
~
are of
of the positive part of
the real axis, the class:of estimable parametric functions of
0,
possessing UMVUE
consists only of real constants.
Proof.
As before, let us, for simplicity, consider the case 0
f(S,11) be a function defined over the triangle 00
(4.38)
E {f(g,'Il)
I 0o} = 0,
~
11
~
£
~
= 1,
~
=2
and let
200 such that
where 00 is any positive number.
we shall show that it is always possible to extend the definition of f(s,TJ) over
all ordered number pairs s,11 in the strip T}
~
£
~
2TJ, such that
for all
° > O.
12
h
possible
(h)
WtiY
of achieving this is as follows.
If (~,~) is an ordered number pair and if
(k being any integer negative or positive) define
2 -kn .
(4.40)
(B)
If (~,~) is an ordered number pair and if
d.efine
(4.41)
It is easy to verify that with this extended definition of f(s,~), (4.39) is
satisfied.
L-5-7
Now let t(~,~) be a UMVUE of its expectation.
It is well known
L-3_7,
that in this case if f(~,~) be such that
(4.42)
E
i f ( £, ~) I ° ~ = 0,
E
tf
a.nd
-'
,
2
.
(£,~)
1°0 5
for all
° > 0,
< co, for some 0 O >
0, then
(4.43)
Now, for any
0
> 0,
O
measure of the set
for any real c and
°> 0,
(4.44)
and L"(c,o,OO) be the lebesgue measure of
let
+
L (c,o,oO) be the lebesgue
l~
"
Unless
t(~,~}
measure on
is identically a real constant except for a set of zero lebesgue
So
{(£,Tl):
::
o
0
o)
a c such that both L+(c,5,a
0~
~
.s ~ :s 20 0 }
and L-(c,5,a
o)
there always exists a 5
are positive.
, for all
(4.46)
> 0 and
Let us define
(~,Tl)€
It is obvious that
(4.47)
and we may extned the definition of
~ ~
s :s 2~
f(~,~)
to all ordered number pairs in the strip
such that
E If(~,Tl)
(4.48)
1 0
3 = 0, for all a> O.
which contradicts (4.43).
Hence t(~,~)
9
c, a.e., on so' for all a > 0, which proves theorem 4.1.
References
Blackwell, D and Givishick, M. A, Theory of Games and Statistical Decisions
John Wiley and Sons, Inc., New York (1954).
Givishick, M.A. and Savage L. J., "Eayes and m1nimaxestimatee for quadratic
loss functions," froc. St:.c. Berko Sym.p. on Math. Stat. a.nd Prob.,
University of California Press, Berkeley and Los Angeles (1950)
53-75.
Lehmann, E. L. and Scheffe, H.,"Completeness, similar regions and unbiased
estimation," Sankhya, Vol. 10, (1950) 305-339.
14
•
L-4_7
Pitmano, E. J. G., "The estimation of the location and scale parameters of d.
continuous population of any form" Biometrica, Vol. 30 (1938) 391-
421.
Rao, C. R.,"Some theorems on minimum variance estimatioo;' Sankhya, Vol.
12
(1952), 27-42.
Rao, C. R.,"M:l.Ili!num variance estimation in distributions admitting ancilliary
statistics,l1 Sankhya, Vol. 12 (1952), 53-56.