Economics 203A Homework # 6
Anton Cheremukhin
November 13, 2005
Exercise 1 Let Yn be a statistic such that limn→∞ E [Yn ] = θ and limn→∞ σY2n = 0. Prove that Yn
£
¤
is a consistent estimator of θ. Hint: E (Yn − θ)2 = (E [Yn − θ])2 + σY2n . Why?
E [(Yn −θ)2 ]
E [(Yn −E[Yn ]+E[Yn ]−θ)2 ]
=
=
Proof. Pr [|Yn − θ| ≥ ε] ≤
2
ε
ε2
2 +(E[Y −θ])2
2
E [(Yn −E[Yn ])2 ]
σ
n
+ (E[Ynε2]−θ) + 2E[Yn −E[Yεn2]](E[Yn ]−θ) = Yn ε2
ε2
σ2 +(E[Yn −θ])2
ε2
limn→∞ Pr [|Yn − θ| ≥ ε] ≤ limn→∞ Yn
i.e Yn is a consistent estimator of θ.
=0
⇒
p limn→∞ Yn = θ
Exercise 2 Let X1 , . . . , Xn represent a random sample from the pdf f (x) = (1/θ) exp (−x/θ),
0 < x < ∞, zero elsewhere. Find the MLE θb of θ. Also, find the MLE of Pr (X ≤ 2)
´
³ Pn
¡
¢
Q
Q
xi
→ max
Proof. L(xi , θ) = i f (xi , θ) = ni=1 1θ exp − xθi = θ−n exp − i=1
θ
θ
´
´ Pn
³ Pn
³ Pn
−1−n
−n
i=1 xi
i=1 xi
i=1 xi
+ θ exp − θ
FOC: −nθ
exp − θ
=0
θ2
P
n
θ̂ = n1 i=1 xi = x̄
¡ ¢
R 2θ
R2
Pr (X ≤ 2) = 0 exp − xθ d xθ = 0 exp (−x) dx = 1 − e−2θ
MLE of Pr (X ≤ 2) is equal to 1 − e−2θ̂ due to invariance principle.
¸
∙
£
¤
1 1 1 1 1
0
, y 0 = 14 17 8 16 3 ,
Exercise 3 Let X =
2 4 3 5 2
Calculate the following:
(a) X 0 X
(b) det (X 0 X)
−1
0
(c) (X X)
(d) (X 0 X)−1 X 0
(e) (X 0 X)¡−1 X 0 y
(f) X (X 0 X)−1 X 0
¢
−1
(g) trace X (X 0 X) X 0
(h) I − X (X 0 X)−1 X 0
¡
¢
¡
¢
(i) trace I − X (X 0 X)−1 X 0
(j) I − X (X 0 X)−1 X 0 y
0
Proof. X X =
∙
¸⎢
⎢
⎢
⎢
⎣
1
1
1
1
1
2
4
3
5
2
⎤
⎥ ∙
¸
⎥
⎥ = 5 16
⎥
16 58
⎦
¸
5 16
det (X X) = det
= 34
16 58
¸
∙
¸−1 ∙ 29
8
5 16
− 17
−1
0
17
=
(X X) =
8
5
16
58
¸ ∙ 13
¸ ∙− 17 34
¸
∙ 29
8
3
11
5
13
1
1
1
1
1
−
−
−
−1
0
0
17
17
17
17
17
17
17
=
(X X) X =
8
3
1
3
5
2
9
2 4 3 5 2
− 17
− 17
− 34
− 17
34
17
34
0
∙
1 1 1 1 1
2 4 3 5 2
⎡
1
0
−1
(X X)
0
Xy=
∙
13
17
3
− 17
3
− 17
2
17
5
17
1
− 34
− 11
17
9
34
13
17
3
− 17
⎡
¸⎢
⎢
⎢
⎢
⎣
14
17
8
16
3
⎤
⎥ ∙ ¸
⎥
⎥= 2
⎥
3
⎦
⎤
⎡ 7
1
4
1 2
17
17
17
1
5
3
⎢ 1 4 ⎥ ∙ 13
¸ ⎢
3
11
5
13
17
17
17
⎥
⎢
⎢
−
−
4
3
7
17
17
17
17
17
⎥
X (X 0 X)−1 X 0 = ⎢
=⎢
1
3
2
9
34
⎢ 172 17
⎢ 1 3 ⎥ −3
−
−
7
5
17
17
34
34
17
⎣ −
⎣ 1 5 ⎦
17
17
34
7
1
4
1 2
17
17
17
¡
¢
trace X (X 0 X)−1 X 0 = 2⎡
⎤ ⎡ 10
7
2
1
4
1
4
7
− 17
− 17
− 17
17
17
17
17
17
3
5
3
7
1
12
⎢ 1
⎥ ⎢ −1
− 17
17
17
17
17
17 ⎥
17
17
⎢
⎢
−1
4
3
4
3
7
5
4
27
⎥=⎢ −
I − X (X 0 X) X 0 = I − ⎢
34
34
17 ⎥
⎢ 17 − 17 34
⎢ 172 17
7
5
23
2
⎣ −
⎦ ⎣ 2 −7 −5
−
17
17
34
34
17
17
17
34
7
2
7
1
4
1
4
7
−
−
−
−
17
17
17
17
17
17
17
17
¡
¢
−1
trace I − X (X 0 X) X 0 ⎡= −1
⎤
⎤ ⎡
⎤⎡
10
1
4
7
2
6
14
−
−
−
17
17
17
17
17
3
7
1 ⎥⎢
12
⎥
⎢
⎢ −1
17 ⎥
−
−
−
17
17
17
17
17 ⎥ ⎢
⎥ ⎢ 3 ⎥
⎢
¡
¢
−1
4
3
5
4 ⎥⎢
27
0
0
⎢
⎥
⎢
I − X (X X) X y = ⎢ − 17 − 17 34 − 34 − 17 ⎥ ⎢ 8 ⎥ = ⎢ −3 ⎥
⎥
11
2
⎣ 2 −7 −5
⎦ ⎣ 16 ⎦ ⎣ −1 ⎦
17
17
34
34
17
2
10
7
1
4
−5
3
− 17
− 17
− 17
17
17
⎡
2
− 17
7
17
5
34
23
34
2
− 17
2
17
7
− 17
5
− 34
11
34
2
17
7
17
1
17
4
17
2
− 17
7
17
7
− 17
1
− 17
4
− 17
2
17
10
17
⎤
⎥
⎥
⎥
⎥
⎦
⎤
⎥
⎥
⎥
⎥
⎦
Exercise 4 Consider the classical regression model y = X β + ε where ε ∼ N (0, σ 2 I)
n×k
(a) Let P = X (X 0 X)−1 X 0 . Show that P = P 0 and P P = P
(b) Let M = I − P . Show that M 0 = M, MM = M, and trace (M) = n − k
b where βb = (X 0 X)−1 X 0 y. Show that e = Mε
(c) Let e = y − X β,
(d) Show that βb = β + (X 0 X)−1 X 0 ε.
¢
¡
(e) Show that βb ∼ N β, σ 2 (X 0 X)−1
(f) Show that βb is independent of e
³
´. q
2
0
b
s2 (X 0 X)−1 ∼ t (n − k).
(g) Let s = e e/ (n − k). Show that β − β
¶
µ
q
q
−1
−1
(h) Assume that k = 1. Show that the interval βb − 1.96 σ 2 (X 0 X) , βb + 1.96 σ 2 (X 0 X)
¶
µ
q
q
−1 b
−1
2
0
2
0
b
contains β with 95% probability. Show that the interval β − 2.576 σ (X X) , β + 2.576 σ (X X)
contains β with 99% probability.
(i) Assume that n = 30 and k = 1. Also assume that σ 2 is unknown. Finally assume that s2 = 1
and X 0 X = 25. Construct a 95% confidence interval for β.
Proof. P 0 = (X (X 0 X)−1 X 0 )0 = (X 0 )0 ((X 0 X)−1 )0 X 0 = X (X 0 (X 0 )0 )−1 X 0 = P
P P = X (X 0 X)−1 X 0 X (X 0 X)−1 X 0 = X (X 0 X)−1 X 0 = P
M 0 = (I − P )0 = I − P 0 = I − P = M
MM = (I − P )(I − P ) = I − 2P + P P = I − P = M
trace (M) = trace (I − P ) = trace (I) − trace (P ) = n − k
2
MX = (I − P )X = (X − X (X 0 X)−1 X 0 X) = X − X = 0
e = y − X (X 0 X)−1 X 0 y = (I − P )y = My = M(Xβ + ε) = Mε
βb = (X 0 X)−1 X 0 y = (X 0 X)−1 X 0 (Xβ + ε) = β + (X 0 X)−1 X 0 ε
¡
¢
¡
¢
⇒
βb ∼ N β, σ 2 (X 0 X)−1 X 0 X (X 0 X)−1 = N β, σ 2 (X 0 X)−1
ε ∼ N (0, σ 2 I)
e = Mε ∼ N (0, σ 2 MM 0 ) = N (0, σ 2 MM) = N (0, σ 2 M)
Hence, e and βb have joint normal distribution.
E[(X 0 X)−1 X 0 ε · (Mε)0 ] = σ 2 (X 0 X)−1 X 0 M 0 = σ 2 (X 0 X)−1 (MX)0 = 0
Hence, they are independent.
2
b
s2 /σ 2 = e0 e/(σ
(n − k)) ∼ χ2 (n − k) is independent of β.
³
´. q
βb − β
σ 2 (X 0 X)−1 ∼ N (0, I)
³
´. q
b
Hence, β − β
s2 (X 0 X)−1 ∼ t (n − k)
¸
∙
q
q
−1
−1
=
Pr βb − 1.96 σ 2 (X 0 X) < β < βb + 1.96 σ 2 (X 0 X)
¯
¸
∙¯ ³
q
´.
¯
¯
σ 2 (X 0 X)−1 ¯¯ < 1.96 = 0.95
Pr ¯¯ βb − β
∙
¸
q
q
−1
−1
Pr βb − 2.576 σ 2 (X 0 X) < β < βb + 2.576 σ 2 (X 0 X)
=
¯
¸
∙¯ ³
q
´.
¯
¯
Pr ¯¯ βb − β
σ 2 (X 0 X)−1 ¯¯ < 2.576 = 0.99
³
´. p
³
´
βb − β
1/25 = 5 βb − β ∼ t (n − k)
t0.05h(29) = 2.045231
⇒ i
Pr βb − 0.409 < β < βb + 0.409 = 0.95
Exercise 5 Consider the classical regression model y = X β +ε where E [ε] = 0 and E [εε0 ] = σ 2 In .
n×k
Notice that we are not assuming that ε has a normal distribution here.
(a) Show that βb = (X 0 X)−1 X 0 y is an unbiased estimator for β.
(b) Show that s2 = e0 e/ (n − k) is an unbiased estimator for σ 2 .
−1
0
0
b
Proof.
h i β = β + (X£ X) X ε ¤
E βb = E [β] + E (X 0 X)−1 X 0 ε = β + (X 0 X)−1 X 0 E [ε] = β
e = Mε
e0 e = ε0 M 0 Mε = ε0 MMε = ε0 Mε (see previous exercise)
ε0 Mε = trace (ε0 Mε) because it is an 1x1 scalar.
trace (ε0 Mε) = trace (Mεε0 ) by the properties of trace
E [e0 e] = E [ε0 Mε] = E [trace (ε0 Mε)] = E [trace (Mεε0 )] = trace (ME [εε0 ]) =
trace (MIn ) = σ 2 trace (M) = σ2 (n − k)
Hence E[s2 ] = E[e0 e/ (n − k)] = σ 2 ⇒ s2 is an unbiased estimator of σ 2 .
¡
¢ .³ σ ´
√
∼ N (0, 1),
Exercise 6 Suppose that U1 , . . . , Un are i.i.d. N (µ, σ 2 ). Show that U − µ
n
i
h
based on which show that Pr U − 1.96 √σn < µ < U + 1.96 √σn = 95% and
h
i
Pr U − 2.576 √σn < µ < U + 2.576 √σn = 99%
3
Proof. Ui ∼ iid N (µ, σ 2 )
(U − µ)/σh∼ N (0, 1/n)
⇒
⇒
¢ ∼ iid (0, 1)
√ (U¡i − µ)/σ
⇒
n Ui − µ /σ ∼ N (0, 1)
£√ ¡
¢
¤
σ
Hence, Pr U − 1.96 √n < µ < U + 1.96 √σn = Pr | n U − µ /σ| < 1.96 = 95%
i
h
£√ ¡
¢
¤
Pr U − 2.576 √σn < µ < U + 2.576 √σn = Pr | n U − µ /σ| < 2.576 = 99%
Exercise 7 Let the observed value of the sample mean X of a random sample of size 20 from
N (µ, 80) be 81.2. Find a 95% confidence interval for µ
√
Proof. CI(µ) = (81.2 ± 1.96 √80
) = (77. 28, 85. 12)
20
Exercise
mean of a random sample of size n from N (µ, 9). Find n such
¡ 8 Let X be the sample
¢
that Pr X − 1 < µ < X + 1 = .90 approximately.
Proof. 1 = 1.645 √3n
⇒
n = (1.645 ∗ 3)2 ≈ 24
Exercise 9 Let a random sample of size 17 from N (µ, σ 2 ) yield x = 4.7 and s2 = 6.12. Determine
a 95% confidence interval for µ.
p
Proof. CI(µ) = (4.7 ± 2.12 6.12/17) = (3. 428, 5. 972)
Exercise 10 Let two independent random samples, each of size 10, from two normal distributions
N (µ1 , 1) and N (µ2 , 1) yield x1 = 4.8, x2 = 5.6. Find a 95% confidence interval for µ1 − µ2 .
√
Proof. CI(µ1 − µ2 ) = ((4.8 − 5.6) ± 1.96 0.1 + 0.1) = (−1. 676 5, 0.0765)
Exercise 11 Let X n denote the mean of a random sample of size n from a gamma distribution
¢± p
√ ¡
with parameters α = µ > 0 and β = 1. Show that the limiting distribution of n X n − µ
Xn
is N (0, 1)
Proof. E[Xi ] = µ = V ar[Xi ]
√
d
by CLT n(X n − µ) −→ N(0, µ)
p
by LLN X n −→ µ
¢± p
√ ¡
d
bu Slutsky n X n − µ
X n −→ N (0, 1)
Exercise 12 Let X n and s2n represent, respectively, the mean√and¡ variance
¢± of a random sample of
2
size n from N (µ, σ ). Prove that the limiting distribution of n X n − µ sn is N (0, 1).
√
d
Proof. by CLT n(X n − µ) −→ N(0, σ 2 )
p
by LLN s2n −→ σ 2
¢± p
√ ¡
d
bu Slutsky n X n − µ
s2n −→ N (0, 1)
Exercise 13 Let x be the observed mean of a random
sample ¢of size n from a distribution having
¡
σ
2
mean µ and known variance σ . Find n so that x − 4 , x + σ4 is an approximate 95% confidence
interval for µ.
√
d
Proof. by √
CLT n(x − µ) −→ N(0, σ 2 )
Hence, Pr[| n(x − µ)|/σ ≤ 1.96] = 0.95
That means Pr[x − 1.96 √σn ≤ µ ≤ x + 1.96 √σn ] = 0.95
1
√
≈ 1.96
⇒
n ≈ (1.96 ∗ 4)2 ≈ 62
4
n
4