Math 561: Theory of Probability (Spring 2016)
Lecture: 15
The Characteristic Function
Lecturer: Partha S. Dey
Scribe: Chris Gartland <[email protected]>
The Characteristic Function
Definition 15.1. For a random variable X, define the characteristic function of X, ϕX : R →
C, by ϕX (t) = E(eitX ).
Proposition 15.2. Let X, Y be random variables. Then
(i) ϕX (0) = 1.
(ii) |ϕX | ≤ 1.
(iii) ∀n ≥ 0, if E(|X|n ) < ∞, then ϕX ∈ C n (R).
(iv) If X ⊥ Y , ϕX+Y = ϕX · ϕY .
(v) ∀t ∈ R, ϕX (t) = ϕX (−t).
(vi) ∀a, b, t ∈ R, ϕaX+b (t) = eitb ϕX (at).
(vii) E(ϕX (Y )) = E(ϕY (X)), and if X ⊥ Y , E(ϕXY (1)).
R1
Example 15.3. (i) X ∼ Unif([0, 1]). ϕX (t) = 0 eitx dx =
R∞
λ
.
(ii) X ∼ Exp(λ). ϕX (t) = 0 eitx λe−λx dx = λ−it
eit −1
it .
(iii) X = X1 − X2 , X1 ⊥ X2 , X1 , X2 ∼ Exp(1).
ϕX (t) = ϕX1 −X2 (t) = ϕ(X1 )(t)ϕ(X2 )(t) =
1
1
1
=
1 − it 1 + it
1 + t2
The density of X is given by f (x) = 21 e−|x| , known as the “Laplace density”.
t2
(iv) X ∼ N (0, 1). ϕX (t) = e− 2 .
1st Proof of (iv): For each θ ∈ R,
1
E(eθX ) = √
2π
Z
e
R
2
− x2 +θx
θ2
e2
dx = √
2π
Z
e−
(x−θ)2
2
dx = e
θ2
2
R
z2
Now, it can be checked that the functions from C to C defined z 7→ E(ezX ) and z 7→ e 2 are
analytic, and the previous calculation shows that they agree on R. The identity principle then
implies that the must agree everywhere, so in particular ϕX (t) = E(eitX ) = e
15-1
(it)2
2
t2
= e− 2 .
15-2
Lecture 15: The Characteristic Function
2
1 R ∞ itx − x2
− t2
2 dx = e
by a contour integral argument.
2nd Proof of (iv): φX (t) = √
−∞ e e
2π
3rd Proof of (iv): By Gaussian integration by parts, E(Xg(X)) = E(g 0 (X)) for all differentiable
g with kgk∞ + kg 0 k∞ < ∞. Applying this to g(x) = eitx yields E(XeitX ) = E(iteitX ). Then
ϕ0X (t) = E(iXeitX ) = E(−teitX ) = −tϕX (t). ϕX also has initial data ϕX (0) = 1, and so the
t2
unique solution to this initial value problem is ϕX (t) = e− 2 .
1
4th Proof of (iv): Let (Xi )∞
i=1 be iid with distribution P(X1 = ±1) = 2 . Then E(X1 ) = 0,
X1 + . . . Xn
√
Var(X1 ) = 1, and so
⇒ X by CLT. This implies ϕ X1 +...X
n (t) → ϕX (t) for all t ∈ R.
√
n n
Next, ϕ X1 +...X
n = ϕX1 +...Xn
√
n
√t
n
= ϕX1
√t
n
n
. It’s an easy check that ϕX1 = cos, so for all
t ∈ R,
n
4 n
2
t2
t
t
− t2
√
1
−
cos
=
lim
ϕX (t) = lim ϕ X1 +...X
(t)
=
lim
+
O
=
e
n
√
n→∞
n→∞
n→∞
2n
n4
n
n
Example 15.4. X ∼ N (µ, σ 2 ). ϕX (t) = e
2 2
itµ− σ 2t
.
(Xi )∞
i=1
Proposition 15.5. If
is a sequence of random variables with Xi ∼ N (0, σi ) for some
∞
(σi )i=1 bounded away from 0 and ∞, and Xi ⇒ X for some X, then σi → σ for some σ and
X ∼ N (0, σ 2 ).
2
σn
Proof: Notice that e− 2 = ϕXn (1) → ϕX (1). Since (σi )i is real and bounded away from 0
and ∞, ϕX (1) is real and bounded away from 0 and 1. Thus, there exists σ ∈ (0, ∞) such that
φX (1) = e−
σ2
2
. This implies σi → σ. Let Z ∼ N (0, σ). Then for every t ∈ R,
ϕX (t) = lim ϕXn (t) = lim e−
n→∞
n→∞
2 t2
σn
2
= e−
σ 2 t2
2
= ϕZ (t)
implying X ∼ Z.
Inversion of the Characteristic Function
For any random variable X with absolutely continuous CDF, let ρX denote its density.
Lemma 15.6. Let Z be a random variable with Z ∼ N (0, 1), and let σ > 0. Then for any random
2 2
R
1
−izy e− σ 2z dz.
variable X, ρX+σZ (y) = 2π
R ϕX (z)e
Proof: For any random variable X,
Z
Z
2
√
σ2 z2
Z
σ
− X2
2σ
2πσρX+σZ (0) =
e
dP = E ϕ Z (X) = E ϕX
=
ϕX (z) √ e− 2 dz
σ
σ
2π
R
R
Thus, the desired equality holds for y = 0. The general case follows by applying this result to the
random variable X − y:
Z
Z
2 2
σ2 z2
1
1
− σ 2z
ϕX−y (z)e
ϕX (z)e−izy e− 2 dz
ρX+σZ (y) = ρ(X−y)+σZ (0) =
dz =
2π R
2π R
Lecture 15: The Characteristic Function
15-3
Lemma 15.7.
be a random variable. If
R Let X−ity
1
ρX (y) = 2π R ϕX (t)e
dt.
R
R |ϕX (t)|dt
< ∞, then X has a density ρX and
Proof: Let Z be independent from X with Z ∼ N (0, 1). By the preceding lemma, ρX+σZ (y) =
2 2
R
1
−ity e− σ 2t dt. Then we note that, for any g ∈ C (R),
b
2π R ϕX (t)e
Z
Z
Z
σ 2 t2
1
g(y)ρX+σZ (y)dy =
E(g(X + σZ)) =
ϕX (t)e−ity e− 2 dtdy
g(y)
2π R
R
R
Z
Z
σ 2 t2
1
=
g(y)e− 2
ϕX (t)e−ity dydt
2π R
R
We also note the fact that as σ → 0, X + σZ ⇒ X. Then
Z
Z
σ 2 t2
1
E(g(X)) = lim E(g(X + σZ)) = lim
g(y)e− 2
ϕX (t)e−ity dydt
σ→0
σ→0 2π R
R
Z
Z
Z
Z
1
1
g(y)
ϕX (t)e−ity dydt =
g(y)
ϕX (t)e−ity dtdy
=
2π R
2π R
R
R
which implies ρX (y) =
1
2π
R
R ϕX (t)e
−ity dt.
Theorem 15.8 (Inversion Lemma). For any random variable
R T −ita −e−itb
limT →∞ −T e it
ϕX (t)dt = P(a < X < b) + 21 P(X ∈ {a, b}).
X
and
a
<
b,
Tightness
Lemma 15.9.
1
u
Ru
−u (1 − ϕX (t))dt ≥ P |X| ≥
2
u
.
Proof:
Z u
Z
Z
1 u
1 u
1
eiuX − e−iuX
itX
itX
(1 − ϕX (t))dt =
(1 − E(e ))dt = E
(1 − e )dt = E 2 −
u −u
u −u
u −u
iuX
2 sin(uX)
1
1
2
=E 2−
≥ 2E 1 −
≥ 2 · P(|uX| ≥ 2) = P |X| ≥
uX
uX
2
u
Theorem 15.10 (Lévy Continuity). Let (Xn )n be a sequence of random variables and ϕ : R → C
a function continuous at 0 such that ϕXn (t) → ϕ(t) for all t ∈ R. Then there exists a random
variable X such that ϕX = ϕ and Xn ⇒ X.
Proof: It is necessary
and sufficient to show that {Xn } is tight, R or equivalently, that
u
supn P |Xn | ≥ u2 → 0 as u → 0. By the lemma, it suffices to show supn u1 −u (1 − ϕXn (t))dt → 0.
R
R
u
u
By DCT we have u1 −u (1 − ϕXn (t))dt → u1 −u (1 − ϕ(t))dt. Since ϕ is continuous at 0,
R
u
1
u −u (1 − ϕ(t))dt → 0 as u → 0.