Hw 6
�
7.24 For n observations, Y = i Xi ∼ Poisson(nλ).
�∞
y −nλ
α−1 −λ/β
1
a. The marginal pmf of Y is m(y) = 0 (nλ)y!e Γ(α)β
e
dλ =
αλ
Thus, π(λ | y) =
f (y|λ)π(λ)
m(y)
=
−
ny
y!Γ(α)β α
λ
λ(y+α)−1 e β/(nβ+1)
y+α
β
Γ(y+α)( nβ+1
)
∼ gamma(y + α,
β
).
nβ+1
�∞
0
λ
λ(y+α)−1 e− β/(nβ+1) dλ.
Alternatively, one could use the relation that the Posterior is proportional to the Likelihood
times the Prior (note set 15, page 4). In this case one only needs to carry the terms that
involve
constant. π(λ | x) ∝ L(λ | x)π(λ) =
�� −λ λx �and the rest go into the proportionality
P
α−1 −λ/β
x
+α−1
e λ i
1
−λ(n+1/β)
i
λ e
∝e
λ i
. Note that this is the kernal of a Gamma
i
xi !
Γ(α)β α
with the same parameters listed above.
β
β
β2
1
b. E(λ | y) = (y + α) nβ+1
= nβ+1
y + nβ+1
(αβ) and V ar(λ | y) = (y + α) (nβ+1)
2.
7.25
b. For one observation of X and� θ the joint pdf is h(x, θ | τ ) = f (x | θ)π(θ | τ ) and the
∞
marginal pdf of X is m(x | τ ) = −∞ h(x,
� θ | τ )dθ. Thus, the joint pdf of X = (X1 , . . . , Xn )
and θ = (θ1 , . . . , θn ) is h(x, θ | τ ) = i h(xi , θi | τ ), �
and the marginal pdf� of X is m(x |
�∞
�∞ �
�∞
�∞
�n
τ ) = −∞ · · · −∞ i h(xi , θi | τ )dθ1 . . . dθn = −∞ · · · −∞ h(x1 , θ1 | τ )dθ1
i=2 h(xi , θ i |
τ )dθ2 . . . dθn . The dθ1 integral is just m(xi | τ ), and this is not a function of θ2 , . . . , θn . So,
m(xi | τ ) can be�pulled out of the integral. Doing each integral in turn yields the marginal
pdf, m(x | τ ) = i m(xi | τ ). Because this marginal pdf factors, this shows that marginally
X1 , . . . , Xn are independent, and they each have the same marginal distribution, m(x | τ ).
a. We will use the results and notation from part (b) to do this special case. From
� ∞ part
(b), the Xi s are independent and each Xi has marginal pdf, m(x | µ, σ 2 , τ 2 ) = −∞ f (x |
� ∞ 1 −(x−θ)2 /(2σ2 ) −(θ−µ)2 /(2τ 2 )
θ, σ 2 )π(θ | µ, τ 2 )dθ = −∞ 2πστ
e
e
dθ. Complete the square in θ to
write the sum of the two exponents as −
„
θ−
»
2
xτ 2
+ 2µσ 2
σ 2 +τ 2
σ +τ
2 2
2 σ2 τ 2
σ +τ
–«2
−
(x−µ)2
.
2(σ 2 +τ 2 )
Only the first term
�∞
involves θ (call it −A(θ)). Also e−A(θ) is the kernel of a normal pdf. Thus, −∞ e−A(θ) dθ =
�
�
√
√
(x−µ)2
1
2
2
√ στ
2π √σστ
,
and
the
marginal
pdf
is
m(x
|
µ,
σ
,
τ
)
=
2π
exp
−
=
2
2
2 +τ 2
2πστ
2(σ +τ )
σ 2 +τ 2
�
�
2
(x−µ)
√ √1
exp − 2(σ
, which is a normal pdf with mean µ and variance σ 2 + τ 2 .
2 +τ 2 )
2π σ 2 +τ 2
9.26 Let X1 , . . . , Xn be iid observations from a beta(θ, 1) pdf and assume that θ has a
Γ(θ+1) θ−1
r−1 −θ/λ
1
gamma(r, λ) prior pdf. f (xi | θ) = Γ(θ)Γ(1)
xi = θxθ−1
and f (θ) = Γ(r)λ
e
. Then the
rθ
i
��n
� 1 r−1 −θ/λ
θ−1
n+r−1 �n
θ−1
1
posterior distribution for θ is f (θ | x) ∝
θ e
= Γ(r)λr θ
( i=1 xi )
i=1 θxi
Γ(r)λr
“
”
P
�
−θ/ 1−λ Pλ log x
θ
i
i
e−θ/λ ∝ θn+r−1 ( ni=1 xi ) e−θ/λ = θn+r−1 e−θ(1/λ− i log xi ) = θn+r−1 e
∝ gamma
16
�
�
n + r, 1−λ Pλ log xi . Let Gα be the α quantile of this distribution. Then a 1 − α Bayes
i
interval of θ is (Gα , ∞). Another interval is (Gα/2 , G1−α/2 ).
�
9.27 a. Let Y = i Xi ∼ gamma(n, λ), the sufficient statistic. By multiplying the likelihood
� 1 �a−1
1
n−1 −y/λ 1
and the prior we find that the posterior distribution of λ is π(λ | y) ∝ Γ(n)λ
e
ny
Γ(a)ba λ
� �n+a−1 −1/(λ(y+1/b))
e−1/(bλ) ∝ λ1
e
, which is the kernal of a Inverse Gamma(n + a, (y + 1/b)−1 ),
so clearly Inverse Gamma distribution is conjugate with the gamma likelihood. The pdf of
n+a
(y+1/b)
1
this is π(λ | y) = (y+1/b)
e− λ .
Γ(n+a) λn+a+1
The Bayes HPD region is a Bayes posterior interval that is of the minimum length. It is of
the form {λ : π(λ | y) ≥ k}, which is an interval since π(λ | y) is unimodal. It thus has the
− 1 (y+1/b)
− 1 (y+1/b)
1
1
form {λ : a1 (y) ≤ λ ≤ a2 (y)}, where a1 and a2 satisfy an+a+1
e a1
= an+a+1
e a2
.
1
2
(Theorem 9.3.2 on pages 441-442 in Casella-Berger shows this approach finds the posterior
interval of minimum length.)
b. To solve this one using (a) one need only recall that S 2 ∼ χ2n−1 which is a Gamma((n −
2
1)/2, 2). The posterior distribution is IG( n−1
+a, ( (n−1)s
+1/b)−1 ). So the Bayes HPD region
2
2
is as in part (a) with these parameters replacing n+a and y+1/b,
1
(n−1)/2+a+1
a2
− a1 ((n−1)s2 /2+1/b)
2
e
1
(n−1)/2+a+1
a1
1
− a1 ((n−1)s2 /2)
e
2
1
.
c. As a → 0 and b → ∞, the condition a1 and a2 becomes
(n−1)/2+1
a2
− a1 ((n−1)s2 /2+1/b)
e
1
(n−1)/2+1
a1
− a1 ((n−1)s2 /2)
e
1
=
.
9.33 Let X ∼ n(µ, 1) and consider the confidence interval Ca (x) = {µ : min{0, (x − a)} ≤
µ ≤ max{0, (x + a)}}.
a. For µ > 0, Pµ (µ ∈ Ca (x)) = Pµ (µ ≤ max{0, (x + a)}) = Pµ (µ ≤ 0) = 0 = Pµ (µ ≤ x + a) if
x+a < 0, Pµ (µ ≤ x+a) if x+a ≥ 0. Therefore, Pµ (µ ∈ Ca (x)) = Pµ (µ ≤ x+a) = P (Z ≤ a)
where Z ∼ N (0, 1). So, Pµ (µ ∈ Ca (x)) = 0.95.
For µ < 0, Pµ (µ ∈ Ca (x)) = Pµ (µ ≥ min{0, (x − a)) = Pµ (µ ≥ x − a) = P (Z ≥ −a) = 0.95.
For µ = 0, Pµ (µ ∈ Ca (x)) = 1 because 0 ∈ Ca (x) for any a and x.
� max{0,x+a}
� max{−x,a}
2
2
b. P (µ ∈ Ca (x)) = min{0,x−a} √12π e−(x−µ) /2 dµ = min{−x,−a} √12π e−t /2 dt
= P (Z ∈ min{−x, −a}, max{−x, a}) = P (Z ∈ (−a, a)) = 0.9 if x ∈ (−1.645, 1.645),
P (Z ∈ (− | x |, a)) → 0.95 if | x |→ ∞.
17
=