Homework 5 Solutions
Problem 1
Part a.
In general, I gave people full credit for saying that θA and θB are dependent because both are functions of θ,
and thus observing one tells us something about the other. That being said, we should be more rigorous in
our justification of the claim that θA and θB are dependent.
One straightforward way of rigorously showing that θA and θB are dependent under this prior distribution is
to compute their covariance.
Cov [θA , θB ] = E [(θA − E [θA ]) (θB − E [θB ])]
= E [θA θB ] − E [θA ] E [θB ]
= E θ2 γ − E [θ] E [θγ]
2
= E θ2 E [γ] − E [θ] E [γ]
= V [θ] E [γ] 6= 0,
so θA and θB are dependent.
A prior of this form is justified when we believe the average number of children born to men with a bachelor’s
degree is proportional to the average number of children born to men without a bachelor’s degree.
Part b.
p (θ|y A , y B , γ) ∝ p (y A |θ) p (y B |θ, γ) p (θ) p (γ)
∝ p (y A |θ) p (y B |θ, γ) p (θ)
nB ȳB
∝ θnA ȳA exp {−nA θ} (γθ)
exp {−nB γθ} θaθ −1 exp {−bθ θ}
∝ θnA ȳA +nB ȳB +aθ −1 exp {− (nA + nB γ + bθ ) θ}
Part c.
p (γ|y A , y B , θ) ∝ p (y A |θ) p (y B |θ, γ) p (θ) p (γ)
∝ p (y B |θ, γ) p (γ)
∝ (γθ)
nB ȳB
exp {−nB γθ} γ aγ −1 exp {−bγ γ}
∝ γ nB ȳB +aγ −1 exp {− (nB θ + bγ ) γ}
Part d.
samp.theta <- function(y.A, y.B, gamma, a.theta, b.theta) {
n.A <- length(y.A)
n.B <- length(y.B)
a.n <- n.A*mean(y.A) + n.B*mean(y.B) + a.theta
1
b.n <- n.A + n.B*gamma + b.theta
return(rgamma(1, a.n, b.n))
}
samp.gamma <- function(y.B, theta, a.gamma, b.gamma) {
n.B <- length(y.B)
a.n <- n.B*mean(y.B) + a.gamma
b.n <- n.B*theta + b.gamma
return(rgamma(1, a.n, b.n))
}
y.A <- scan(file=url("http://www.stat.washington.edu/~pdhoff/Book/Data/hwdata/menchild30bach.dat"))
y.B <- scan(file=url("http://www.stat.washington.edu/~pdhoff/Book/Data/hwdata/menchild30nobach.dat"))
a.theta <- 2
b.theta <- 1
a.gammas <- b.gammas <- 2^(seq(3, 7))
diffs <- numeric(length(a.gammas))
S <- 5000
for (i in 1:length(diffs)) {
thetas <- gammas <- numeric(S)
theta <- mean(y.A)
gamma <- mean(y.B)/theta
for (j in 1:S) {
thetas[j] <- theta <- samp.theta(y.A, y.B, gamma, a.theta, b.theta)
gammas[j] <- gamma <- samp.gamma(y.B, theta, a.gammas[i], b.gammas[i])
}
diffs[i] <- mean(thetas*gammas - thetas)
}
plot(a.gammas, diffs,
ylab = expression(paste("E[", theta[B], "-", theta[A], "|", y[A], ", ", y[B], "]", sep = "")),
xlab = expression(paste(a[gamma], "=", b[gamma], sep = "")))
2
0.35
0.30
0.25
0.20
0.15
E[θB−θA|yA, yB]
20
40
60
80
100
120
aγ=bγ
As we increase aγ and bγ together, we increase the strength of the prior belief that γ = 1, i.e. θA = θB .
As we would expect, the plot shows that as we increase the strength of our prior belief that θA = θB , we
conclude that E [θB − θA |y A , y B ] will be closer to 0, i.e. our estimates of θB given the data will be closer to
our estimates of θA given the data.
Problem 2
Part a.
px (x) = Pr (X = x)
=
L
X
Pr (X = x, Y = yl )
l=1
=
L
X
pxy (x, yl )
l=1
py (y) = Pr (Y = y)
=
K
X
Pr (X = xk , Y = y)
k=1
=
K
X
pxy (xk , y)
k=1
px|y (x|y) = Pr (X = x|Y = y)
Pr (X = x, Y = y)
Pr (Y = y)
pxy (x, y)
=
py (y)
=
3
Part b.
Pr (X ∗ = x) =
L
X
Pr (X ∗ = x, Y ∗ = yl )
l=1
=
L
X
Pr (X ∗ = x|Y ∗ = yl ) Pr (Y ∗ = yl )
l=1
=
L
X
px|y (x|yl ) py (yl )
Given by sim. procedure
l=1
=
L
X
pxy (x, yl )
l=1
= px (x)
4