Engineering Probability & Statistics
Sharif University of Technology
Hamid R. Rabiee & S. Abbas Hosseini
November 21, 2015
CE 181
Date Due: Azar 14th , 1394
Homework 5
Problems
1. A univariate Gaussian (Normal) distribution with mean µ and variance σ 2 is defined as
N (x|µ, σ 2 ) = √
(x−µ)2
1
e− 2σ2
2πσ
Suppose we have N i.i.d random variables sampled from a Gaussian distribution N (x|µ, σ 2 ).
Derive the distribution over the set of random variables. This distribution also defines an N
dimensional random variable x, for which, dimensions are statistically independent. Describe
why this equivalence holds.
2. Let x ∼ N (0, 1). Show that P (x > x + |x > x) ≈ e−x for large x and small .
3. The Exponential Family of distributions over x, given parameter vector η, is defined to be the
set of distributions of the form
p(x|η) = h(x)g(η) exp {η T u(x)}
Express these list of distributions as members of the exponential family and derive expressions
for η, u(x), h(x), and g(η).
(a) Gamma distribution
Gam(λ|a, b) =
1 a a−1
b λ
exp(−bλ)
Γ(a)
(b) Multi-variate Gaussian distribution
N (x|µ, Σ) =
1
(2π)N/2 |Σ|1/2
exp
n
o
1
− (x − µ)T Σ−1 (x − µ)
2
4. Suppose that X1 , . . . , Xn form a random sample from a normal distribution for which the value
of the mean θ is unknown and the value of the variance σ 2 > 0 is known. Suppose also that the
prior distribution of θ is the normal distribution N (θ|µ0 , v02 ). Find the hyperparameters of the
posterior distribution of θ given given that Xi = xi (i = 1, . . . , n).
5. Suppose that observations are to be taken at random from the normal distribution N (X|θ, 1),
and that θ is unknown. Assume that θ ∼ N (θ|µ, 4). Also, observations are to be taken until the
variance of the posterior distribution of θ has been reduced to the value 0.01 or less. Determine
the number of observations that must be taken before the sampling process is stopped.
1
6. Suppose that X1 , . . . , Xn form a random sample from a distribution for which the p.d.f. f (x|θ)
is as follows:
θxθ−1
for 0 < x < 1
f (x|θ) =
0
otherwise.
Suppose also that the value of the parameter θ is unknown (θ > 0), and the prior distribution of
θ is the gamma distribution with parameters α and β (α > 0 and β > 0). Determine the mean
and the variance of the posterior distribution of θ.
7. we have two independent RV’s X and Y which both are exponential with same parameter λ if
we define Z = X + Y find its Probability Density Function(PDF)
8. Prove the following identity.
E [y] = E [E [x|y]]
Suppose that N is a counting random variable, with values {0, 1, · · · , n}, and that given (N = k),
for k ≥ 1, there are defined random variables X1 , · · · , Xn such that
E (Xj |N = k) = µ
(1 ≤ j ≤ k)
Define a random variable SN by
SN =
X1 + X2 + · · · + Xk
0
Show that E(SN ) = µE(N ).
2
if(N − k), 1 ≤ k ≤ n
if(N = 0)