Exercise 6.10/ page 301
T (X) = X(1) , X(n) is a minimal sucient statistic for θ in the case of U nif (θ, θ + 1); show that
it is not complete.
We have an intution that T (X) is not complete because we have seen (in Example 6.2.17 and
also for location parameter distributions in Example 6.2.18) that the range R = X(n) − X(1) is
an ancillary statistic for θ. In particular, R is a function of T (X). Let h(x, y) = y − x, then
R = h(T (X)).
To solve the exercise we will use (the inverse of) denition 6.2.21:
It is enough to nd a function g(T ) for which Eθ (g(T )) = 0 for all θ but g(T ) 6= 0, i.e.
Pr (g(T ) = 0) 6= 1.
Since function g should meet the requirement Eθ (g(T )) = 0 for all θ, it would be helpful if g(T )
is of the form
g(T ) = h(T ) − E(h(T ))
where h(T ) is an ancillary statistic, i.e. whose distribution does not depend on θ, so that Eθ (h(T ))
is the same for all θ. This way, Eθ (g(T )) = E [h(T ) − E(h(T ))] = 0 for all θ.
As we already said, R = X(n) − X(1) is an ancillary statistic for θ and can be written as a function
of T .
So, we can dene function g as follows:
g(T ) = h(T ) − E(h(T )) = X(n) − X(1) − E X(n) − X(1) .
Obviously, Eθ (g(T )) = 0 for all θ. However, Pr (g(T ) = 0) 6= 1 as X(1) and X(n) are continuous
random variables.
Thus, denition 6.2.21 does not hold and T (X) = X(1) , X(n) is not a complete statistic for θ.
Comments: The result can be generalised: if a function of a sucient (or minimal sucient)
statistic T is an ancillary statistic, i.e. S = h(T ) ancillary, then the sucient (or minimal
sucient) statistic is not complete. We can dene the function g(T ) = h(T )−E(h(T )) = S −E(S)
for which E(g(T )) = 0 for all θ but Pr (g(T ) = 0) 6= 1.
Exercise 7.37/ p. 362
A good starting point is to nd a sucient statistic for θ. Then, we check whether it is also
complete. If it is, we can use Theorem 7.3.23 to nd the best unbiased estimator of θ.
To nd a sucient statistic for θ we apply the Factorization Theorem.
The joint denisty of a sample of size n is:
n Y
n h
n
n
i
Y
1
1
1
I(−θ < xi < θ) =
I(|xi | < θ) =
I max (|xi |) < θ .
f (x|θ) =
f (xi |θ) =
i
2θ
2θ
2θ
i=1
i=1
i=1
n
Y
Let g (maxi (|xi |) |θ) =
1 n
2θ
I [maxi (|xi |) < θ] and h(x) = 1.
Then, f (x|θ) = g (maxi (|xi |) |θ) h(x) for all x and θ.
1
Thus, according to the Factorization theorem, T (X) = maxi (|Xi |) is a sucient statistic for θ.
(Observe that X(n) = maxi (Xi ) and therefore T (X) = maxi (|Xi |) may not coincide with X(n) .
For example, when X(2) < 0 it may be that |X(2) | > |X(n) |. For instance, for the sample x =
(−2, −1.28, 0.5, 1) we have x(n) = x(4) = 1 and maxi (|xi |) = maxi (2, 1.28, 0.5, 1) = 2.)
The next step is to check whether T (X) = maxi (|Xi |) is complete for θ. For this we have to apply
Denition 6.2.21. To do so we have to determine the density of maxi (|Xi |) and subsequently of
|X| rst.
(
Let Y = |X|, then 0 < y < θ, x =
set
y = g1−1 (y)
, and according to Theorem 2.18 (page 53)
set
−y = g2−1 (y)
d −1 d −1 −1
−1
fY (y) = fX (g1 (y)) g1 (y) + fX (g2 (y)) g2 (y) =
dy
dy
1
1
1
= fX (y) ∗ 1 + fX (−y) ∗ 1 =
+
= , 0<y<θ.
2θ 2θ
θ
Thus, Y = |X| ∼ U nif (0, θ) and therefore the density of W = maxi Yi = maxi (|Xi |) is f (w|θ) =
nwn−1 θ−n , 0 < w < θ (for the latter see Example 6.2.23 or Theorem 5.4.4 on page 229).
Thus, the density of T (X) = maxi (|Xi |) is f (t|θ) = ntn−1 θ−n , 0 < t < θ.
In Example 6.2.23 it is shown that f (t|θ) is complete, hence T (X) = maxi (|Xi |) is complete
statistic for θ.
In Example 7.3.13 it is shown that E(T ) =
estimator of θ.
n
θ
n+1
. Hence, the statistic φ(T ) =
n+1
T
n
is an unbiased
Moreover, φ(T ) is based only on T which is complete for θ. Hence, according to Theorem 7.3.23
T is the unique unbiased estimator of θ.
φ(T ) = n+1
n
Exercise 7.48/ p. 364
a) First we have to nd the MLE of p. The likelihood function is of the form:
L(p|x) = f (x|p) =
n
Y
f (xi |p) =
i=1
n
Y
pxi (1 − p)1−xi = p
P
xi
P
(1 − p)n−
xi
i=1
and the log-likelihood is
log L(p|x) =
X
xi log p + n −
X
xi log(1 − p) = nx̄ log p + (n − nx̄) log(1 − p) .
The rst derivative of log L(p|x) with respect to p is
∂
1
1
n
log L(p|x) = nx̄ − n (1 − x̄)
=
(x̄ − p) .
∂p
p
1−p
p(1 − p)
For log L(p|x) = 0 we get p = x̄. The second derivative of log L(p|x) with respect to p is
∂2
1
1
log L(p|x) = −nx̄ 2 − n (1 − x̄)
<0.
2
∂p
p
(1 − p)2
2
Hence, log L(p|x) attains its maximum at p = x̄ and p̂M LE = X̄ .
In exercise 7.40, page362, we showed that X̄ attains the Cramér-Rao lower bound.
b) Since Xi are independent, E(X1 X2 X3 X4 ) =
i=1 E(Xi ). Since Xi are identically distributed
as Bernoulli(p), E(Xi ) = p. Thus, E(X1 X2 X3 X4 ) = p4 , i.e. the estimator W (X) = X1 X2 X3 X4
is an unbiased estimator of p4 , for n ≥ 4.
Q4
To nd the best unbiased estimator we will use Theorem 7.3.23. To do so we have to nd a
complete statistic for parameter p.
In example 6.2.22 (page 285) it is shown that the statistic T ∼ Bin(n, p), 0 < p < 1, is complete
statistic
for p. In our case,
P
P Xi ∼ Bernoulli(p), i = 1, . . . , n, and we know that then T (X) =
Xi ∼ Bin(n, p). Thus,
Xi is complete statistic for p.
Another way to nd a complete statistic for p is to use Theorem 6.2.25 (Complete statistics in the
exponential family). Bernoulli(p)
is an exponential family distribution and it is written in the
P
form f (x|p) = h(x)c(p) exp ( k wk (p)tk (p)) as follows:
x
1−x
f (x|p) = p (1 − p)
= (1 − p)
p
1−p
x
p
= (1 − p) exp x log
.
1−p
n o
p
The set {w(p) : 0 < p < 1} = log 1−p : 0 < p < 1 contains an open set in R. Hence, according
P
P
to the theorem the statistic T (X) = ni=1 t(Xi ) = ni=1 Xi is complete statistic for p.
p
p
(Observe that for 0 < p < 1, 1−p
>0 and log 1−p
∈ (−∞, ∞) which is an open set.)
Dene the statistic φ(T ) = E(X1 X2 X3 X4 |T ). This is a function only of the complete statistic T .
E(φ(T )) = E [E(X1 X2 X3 X4 |T )] = E(X1 X2 X3 X4 ) = p4 , i.e. φ(T ) is an unbiased estimator of p4 .
Thus, Theorem 7.3.23 applies, which means that φ(T ) is the unique best unbiased estimator of p4 .
It only remains to determine the exact formula of φ(T ).
φ(t) = E(X1 X2 X3 X4 |T = t) =
1 X
1 X
1 X
1
X
x1 x2 x3 x4 P (X1 = x1 , X2 = x2 , X3 = x3 , X4 = x4 |T = t)
x1 =0 x2 =0 x3 =0 x4 =0
Observe that if at least one of X1 , X2 , X3 , X4 takes the value 0 then the whole term
x1 x2 x3 x4 P (X1 = x1 , X2 = x2 , X3 = x3 , X4 = x4 |T = t) is 0. So, only the term where all
X1 , X2 , X3 , X4 take the value 1 remains. Thus,
φ(t)
=
=
since
Xi 's
are independent
=
Note that the statistic
E(X1 X2 X3 X4 |T = t) = P (X1 = 1, X2 = 1, X3 = 1, X4 = 1|T = t)
P
P (X1 = 1, X2 = 1, X3 = 1, X4 = 1, ni=5 Xi = t − 4)
P
=
P ( ni=1 Xi = t)
P
P (X1 = 1)P (X2 = 1)P (X3 = 1)P (X4 = 1)P ( ni=5 Xi = t − 4)
P
P ( ni=1 Xi = t)
Pn
X
can
take
the
values
0,
.
.
.
,
n
−
4
and
also
i
i=5
i=5 Xi ∼ Bin(n − 4, p).
Pn
However, since t = 0, . . . , n, it can happen t − 4 < 0 and then P ( i=5 Xi = t − 4) = 0 and the
estimator is φ(t) = 0 for t < 4.
Pn
3
For t ≥ 4 the estimator is of the form:
n−4
n−4
t−4
n−t
p∗p∗p∗p∗
p (1 − p)
t−4
t−4
t(t − 1)(t − 2)(t − 3)
= =
.
φ(t) =
n(n − 1)(n − 2)(n − 3)
n
n
t
n−t
p (1 − p)
t
t
So, we see that φ(T ) is indeed a function only of T .
Exercise 7.49/ p. 364
We need to compute rst E(Y ), i.e. E(X(1) ), and for this we need to determine the density of
X(1) . Based on Theorem 5.4.4 (page 229)
a)
fX(1) (x) =
n!
fX (x) [1 − FX (x)]n−1 = nfX (x) [1 − FX (x)]n−1
0!(n − 1)!
Since Xi are iid from exp(λ), fX (x|λ) = λ1 e− λ x , x ≥ 0 and FX (x) =
1
´x
1
1 −λ
e t dt
0 λ
= 1 − e− λ x . Thus
1
1 1 1 n−1 n − n x
fX(1) (x) == n e− λ x e− λ x
= e λ , x≥0.
λ
λ
i.e. Y = X(1) ∼ exp( nλ ) and therefore E(Y ) = nλ .
Hence the statistic W (Y ) = nY is an unbiased estimator of λ.
b)
To nd a better estimator of λ we apply Theorem 7.3.17 (Rao-Blackwell Theorem).
For this, we need to nd a sucient statistic for λ. Since Xi are iid from exp(λ) and exp(λ) is
an exponential family distribution we can apply Theorem
Pn 6.2.10 (sucient statistic in exponential
family distributions). Based on the theorem, T (X) = i=1 Xi is a sucient statistic for λ.
Thus, according to Theorem 7.3.17, φ(T ) = E(nX(1) |T ) is uniformly better unbiased estimator of
λ.
We have to determineP
the exact form of φ(T ). To do so we can think as follows:
P φ(T ) is a function
only of T (X), i.e. of ni=1 Xi . We also know that if Xi ∼ exp(λ) then T = Xi ∼ exp(nλ) and
E(T ) = nλ. Since E(φ(T )) = λ this means that φ(T ) = Tn .
P
Note that using
Theorem 6.2.25 we can show that T (X) = ni=1 Xi is complete for λ (the set
− λ1 : λ > 0 ∈ (−∞, 0) i.e. it contains an open set in R). Moreover, φ(T ) = Tn is an unbiased
estimator of λ based only on the complete statistic. Thus, according to Theorem 7.3.23 φ(T ) = Tn
is the unique best unbiased estimator of λ.
Note that both the method of moments and the maximum likelihood method yields the same estimator for λ which is X̄ , i.e. φ(T ). Thus, both methods yield an estimator which is the best unbiased
estimator of λ.
c)
Based on the estimator W (Y ) = nY = nX(1) we get λ̂ = 12 ∗ 50.1 = 601.2.
Based on the estimator φ(T ) =
T
n
=
P
Xi
n
, we get λ̂ =
1545.6
12
= 128.8.
We observe that the two
P estimates are quite dierent. Between the two we might prefer the one
based on φ(T ) = Tn = nXi because this estimator is the best unbiased estimator of λ > 0.
4