BS1.
(a) Suppose that X ∼ N ormal(µ, 1). Let Y = X 2 .
(i) Compute the pdf, expectation, and variance of Y .
4 MARKS
(ii) Suppose µ = 0. Write down the joint pdf of X and Y .
2 MARKS
(b) Suppose that U and V are independent N ormal(0, 1) random variables. Let
√
R = U2 + V 2
be the random variable recording the distance of the random point (U, V ) from
the origin. Find
P [R > 1]
4 MARKS
(c) Suppose that X1 , X2 , . . . , XK+1 are independent and identically distributed
Gamma(α, 1) random variables. Let
Xk
Yk = PK+1
j=k
k = 1, . . . , K
Xj
Find the joint distribution of (Y1 , Y2 , . . . , YK ). Show all work.
10 MARKS
1
BS2.
~ = (X1 , . . . , Xk )T is a random vector having a N ormal(~µ, Σ) disSuppose that X
tribution. Suppose that
T
~ = X
~ 1, X
~2
X
~ into
represents a partitioning of X
~ 1 , a k1 × 1 vector, and
• X
~ 2 , a k2 × 1 vector
• X
where k = k1 + k2 .
~ 1.
(a) Derive the marginal distribution of X
4 MARKS
~ 2 , given that X
~ 1 = ~x1 .
(b) Derive the conditional distribution of X
4 MARKS
(c) Derive the marginal distribution of Z where
~ 1 + ~aT2 X
~2
Z = ~aT1 X
where ~a1 and ~a2 are k1 × 1 and k2 × 1 real vectors respectively.
2 MARKS
(d) Suppose µ
~ = ~0. Consider the random variable Y defined by
Y =
k−1
X
λi Xi
i=1
for real constants ~λ = (λ1 , . . . , λk−1 )T .
Find ~λ such that
M (~λ) = E[(Xk − Y )2 ]
is minimized, and compute the minimum value of M (~λ).
In the expression for M (~λ), the expectation is taken with respect to the relevant
joint distribution.
10 MARKS
2
BS3.
(a) State and prove Jensen’s Inequality in the one dimensional case.
8 MARKS
(b) Consider two univariate probability densities f0 and f1 each with support R.
(i) Define the Kullback-Leibler divergence between f0 and f1 , KL(f0 , f1 ), and
demonstrate that the divergence is non-negative.
6 MARKS
(ii) Consider the (squared) Hellinger distance between f0 and f1 defined by
Z
p
1 ∞ p
Hell(f0 , f1 ) =
( f0 (x) − f1 (x))2 dx.
2 −∞
Show that
0 ≤ Hell(f0 , f1 ) ≤ KL(f0 , f1 ).
Hint: for all x > 0,
x − 1 ≥ log x.
6 MARKS
3
BS4.
(a) State and prove the Central Limit Theorem for a sequence of independent and
identically distributed random variables, X1 , . . . , Xn , with finite mean and variance.
5 MARKS
(b) Find an approximation, for large n, to the distribution of the following statistics derived from the sample mean random variable
n
1X
Xi .
X̄n =
n i=1
(i) X̄n2 , if Xi ∼ N ormal(µ, 1) for i = 1, . . . , n, with µ 6= 0.
3 MARKS
(ii) − log X̄n , if Xi ∼ Exponential(λ) for i = 1, . . . , n.
3 MARKS
(iii) X̄n2 , if Xi ∼ Student(ν) for i = 1, . . . , n, with ν > 2.
3 MARKS
(c) Suppose Xm ∼ Binomial(m, θ) and Yn ∼ Binomial(n, φ) are elements of
sequences of independent random variables indexed by m and n respectively. Find
an approximation to the distribution of statistic
Xm /m
Tm,n = log
Yn /n
when m and n are large. Find also the probability limit of Tm,n , that is, the quantity ψ such that Tm,n converges in probability to ψ.
6 MARKS
4
BS5.
Suppose that the densities in family Fk = {fj : j = 0, . . . , k} have common
support.
(a) Show that the statistic
(
~ =
T (X)
~
~
f1 (X)
fk (X)
,...,
~
~
f0 (X)
f0 (X)
is minimal sufficient for Fk .
)
(1)
5 MARKS
(b) Now let F = {fθ : θ ∈ Θ}, for some index set Θ, be a family of densities with
common support, and suppose that, for some positive integer k,
(i) fi = fθi ∈ F for i = 0, . . . , k, and
~ as defined in part (a) is θ-sufficient.
(ii) T (X)
~ is θ-minimal sufficient.
Show that T (X)
5 MARKS
(c) Show that one-to-one functions of minimal sufficient statistics are minimal sufficient statistics.
5 MARKS
~ is a
(d) Use the results from part (b) and (c) to show that X is β-sufficient if X
random sample from a Γ(α, β), β > 0, α known, family.
5 MARKS
5
BS6.
~ = [X1 , . . . , Xn ]0 be a random sample from an Exponential(θ) distribution
Let X
with Eθ [Xi ] = 1/θ. Let τk (θ) = Pθ [X1 > k], for θ, k > 0.
(a) Find the maximum likelihood estimator of θ.
4 MARKS
(b) Find the maximum likelihood estimator τ̂k of τk (θ).
p
(c) Find the asymptotic distribution of (n) [τ̂k − τk (θ)].
4 MARKS
4 MARKS
(d) Show that no unbiased estimator of τk (θ) has a variance that achieves the
theoretical Cramér-Rao lower bound.
4 MARKS
~ = [X1 , . . . , Xn ]0 be a random sample from a χ2ν distribution.
Let now X
(e) Find the uniformly minimum variance unbiased estimator of
d
log Γ(ν/2).
dν
6
4 MARKS
BS7.
Let fg be given densities, g = 1, . . . P
, G, and let π = [π1 , . . . , πG ]0 be a probability
vector (so that πg ≥ 0 for all g and g πg = 1). Suppose that Y = [Y1 , . . . , Yn ]0 is
G
X
a random sample from the mixture distribution fY (y) =
πg fg (y).
g=1
(r)
(r)
(G)
[π1 , . . . , πg ]0
(r+1) ~
Given current values ~π =
derive an EM iteration ~π (r+1) , a
function of ~π (r) and Y~ , such that L(~π
|Y ) ≥ L(~π (r) |~π ), where L(~π |Y~ ) is the
likelihood of ~π based on Y~ .
h
i
~ = (Yi , Z
~ i ) , i = 1, . . . , n, where Z
~i =
Hint: Use the augmented data set X
[Zi1 , . . . , ZiG ] ∼ Multinomial(n = 1, π1 , . . . , πG ) to write the complete data likelihood.
20 MARKS
7
BS8.
(a) Let θ be a parameter of a distribution and X = (x1 , x2 , . . . , xn ) a sample
of size n from the distribution. Consider an estimator W (X) for τ (θ). Give the
definition of bias in terms of W (X) and τ (θ).
4 MARKS
(b) Now consider X = (x1 , x2 , . . . , xn ) from Bin(n, p). Characterize functions of
p that are unbiasedly estimable.
6 MARKS
p
is not
(c) Again consider X = (x1 , x2 , . . . , xn ) from Bin(n, p), show that 1−p
unbiasedly estimable.
4 MARKS
p
p
is called the odds and log 1−p
the logit of p. Consider
(d) In Epidemiology 1−p
p̂
again X = (x1 , x2 , . . . , xn ) from Bin(n, p) that yields θ̂ = log 1−p̂
. Derive an
approximation for V ar(θ̂).
6 MARKS
8
BS9.
Please answer the following.
(a) State the definition for almost sure convergence.
2 MARKS
(b) State the strong law of large numbers.
2 MARKS
(c) Consider the data set x1 , x2 , . . . , xn and the population of all possible nn bootstrap resamples x∗i = (x∗1 , x∗2 , . . . , x∗n ) from this data set. Let θ̂i∗ represent a statistic
calculated from the ith bootstrap resample. Give the expression for V ar∗ (θ̂), the
variance of the population of θ̂i∗ ’s. Describe V ar∗ (θ̂) in terms of the empirical distribution function of the original data.
6 MARKS
(d) Next consider taking B bootstrap resamples from the original data set
x1 , x2 , . . . , xn . Again, let θ̂i∗ be a statistic calculated from the ith bootstrap sample
x∗i , and define
B
B
1 X ∗
1 X ∗ 2
∗
θ̂ −
θ .
V arB (θ̂) =
B − 1 i=1 i
B i=1 i
∗
Repeat this process m times to obtain the values V arB,j
(θ̂) for j = 1, . . . , m. Show
P
m
1
∗
∗
10 MARKS
that m i=1 V arB,i (θ̂) converges almost surely to V ar (θ̂).
9
STUDENT NAME:
STUDENT ID#
McGILL UNIVERSITY
Department of Epidemiology, Biostatistics and Occupational Health
Biostatistics Part A Comprehensive Exam (BIOS 700)
Theory Paper
Date: Friday August 21st, 2009
Time: 13:00 - 17:00
INSTRUCTIONS
Answer only six questions out of BS1-BS9.
Questions
BS1
BS2
BS3
BS4
BS5
BS6
BS7
BS8
BS9
Marks
This exam comprises the cover and 9 pages of questions.