Other Estimators of the Mean Vector
Admissibility
Consider estimators m and m∗ of µ:
I The sum-of-squared-errors loss function is
L(µ, m) = (m − µ)0 (m − µ) = ||m − µ||2 .
I
The risk function R(µ, m) is the expected loss:
R(µ, m) = Eµ [L(µ, m)]
I
m∗ is as good as m if
∀µ, R(µ, m∗ ) ≤ R(µ, m)
I
m∗ is better than m if it is as good, and
∃µ, R(µ, m∗ ) < R(µ, m)
I
m is admissible if there is no m∗ that is better than m.
NC STATE UNIVERSITY
1/6
Statistics 784
Multivariate Analysis
Other Estimators of the Mean Vector
Bayes estimation
I
If x1 , x2 , . . . , xN is a random sample from Np (µ, Σ), and µ has the
prior distribution Np (ν, Φ), then the posterior distribution of µ is
multivariate normal with mean
−1
−1
1
1
1
Φ Φ+ Σ
x̄ + Σ Φ + Σ
ν
N
N
N
and covariance matrix
1
Φ−Φ Φ+ Σ
N
I
−1
Φ
The posterior mean of µ is “a kind of weighted average of x̄ and ν.”
NC STATE UNIVERSITY
2/6
Statistics 784
Multivariate Analysis
Other Estimators of the Mean Vector
I
The posterior mean minimizes the expected risk
r (ν, Φ, m) = Eν,Φ [R(µ, m)]
and is called the Bayes estimator.
I
I
I
Any Bayes estimator is admissible.
Any admissible estimator is a Bayes estimator, or the limit of Bayes
estimators.
An estimator m is minimax if
sup R(µ, m) = inf∗ sup R(µ, m∗ )
m
µ
I
µ
x̄ is minimax.
NC STATE UNIVERSITY
3/6
Statistics 784
Multivariate Analysis
Other Estimators of the Mean Vector
Improved Estimation of the Mean Vector
I
As an estimator of µ in a random sample of N from Np (µ, Σ), x̄ is:
I
I
I
I
the maximum likelihood estimate;
minimum variance unbiased;
equivariant: Ax + b = Ax̄ + b.
But it is not admissible with respect to sum-of-squared-errors loss,
when Σ ∝ Ip and p ≥ 3.
NC STATE UNIVERSITY
4/6
Statistics 784
Multivariate Analysis
Other Estimators of the Mean Vector
The James-Stein estimator:
I
Take Σ = NIp , so that X̄ ∼ Np (µ, Ip ).
I
Then E[L(µ, X̄)] = p.
I
Now for any fixed ν, let
m(x̄) =
p−2
1−
||x̄ − ν||2
(x̄ − ν) + ν
I
This estimator shrinks x̄ toward the arbitrary ν.
I
If p ≥ 3, m(x̄) is better than x̄, proving that x̄ is not admissible.
NC STATE UNIVERSITY
5/6
Statistics 784
Multivariate Analysis
Other Estimators of the Mean Vector
I
Note that if ||x̄ − ν||2 = p − 2, m(x̄) = ν; that is, the shrinkage has
gone all the way from x̄ to ν.
I
If ||x̄ − ν||2 < p − 2, the “shrinkage” goes beyond ν.
I
A more sensible estimator is
+
m (x̄) = 1 −
p−2
||x̄ − ν||2
+
(x̄ − ν) + ν
where (x)+ denotes the positive part of x:
(
x x ≥0
+
(x) =
0 x <0
I
I
I
m+ (x̄) is better than m(x̄).
m+ (x̄) is minimax.
But m+ (x̄) is also not admissible.
NC STATE UNIVERSITY
6/6
Statistics 784
Multivariate Analysis