Advanced probability I
Lecture 3
Convergence of random variables: further results
Independence
Kolmogorov’s zero-one law
1 / 15
Convergence of random variables: further results
The following provides a useful sufficient criterion for a sequence to
converge almost surely.
Lemma 1.2.12. Let (Xn ) be a sequence of randomPvariables, and let X
be some other variable. Assume that for all ε > 0, ∞
n=1 P(|Xn − X | ≥ ε)
is finite. Then Xn converges almost surely to X .
Proof. We need to prove
∞
∞
P(∩∞
m=1 ∪n=1 ∩k=n (|Xk − X | ≤
1
m ))
= 1.
By the Borel-Cantelli lemma, P(|Xn − X | ≥ ε i.o.) = 0. Therefore,
P(|Xn − X | ≤ m1 evt.) = 1. From this, the result follows.
2 / 15
Convergence of random variables: further results
Lemma 1.2.13. Let (Xn ) be a sequence of random variables, and let X
be some other variable. If Xn converges in probability to X , there is a
subsequence such that Xnk converges almost surely to X .
Proof. Let (εk )k≥1 be a sequence of positive numbers decreasing to zero.
For any k, n∗ ≥ 1, there is n > n∗ such that P(|Xn − X | ≥ εk ) ≤ 2−k .
Take a subsequence (nk ) such that P(|Xnk − X | ≥ εk ) ≤ 2−k . Lemma
1.2.12 then yields the result.
3 / 15
Convergence of random variables: further results
For a double sequence (xnm )n,m≥1 , we say that xnm converges to x as n
and m tends to infinity if for all ε > 0, there is k ≥ 1 such that
|xnm − x| ≤ ε whenever n, m ≥ k.
Definition 1.2.15. Let (Xn ) be a sequence of random variables.
• (Xn ) is Cauchy in probability if for all ε > 0, P(|Xn − Xm | ≥ ε) tends
to zero as n and m tends to infinity.
• (Xn ) is almost surely Cauchy if P((Xn ) is Cauchy) = 1.
• (Xn ) is Cauchy in Lp for some p ≥ 1 if E |Xn − Xm |p tends to zero as
n and m tends to infinity.
4 / 15
Convergence of random variables: further results
The definition of being almost surely Cauchy is well-formed because of the
following.
Lemma 1.2.14. Let (Xn ) be a sequence of random variables. Then
∞
∞
(Xn is Cauchy) = ∩∞
m=1 ∪n=1 ∩k=n (|Xn − Xk | ≤
1
m)
Proof. Note that a sequence (xn ) is Cauchy if and only if for each m ≥ 1,
there exists n ≥ 1 such that for all k ≥ n, |xn − xk | ≤ m1 .
5 / 15
Convergence of random variables: further results
Theorem 1.2.16. Let (Xn ) be Cauchy in probability, almost surely or in
Lp . Then, there exists a random variable X such that (Xn ) converges to
X in probability, almost surely or in Lp , respectively. In the Lp case, X
has p’th moment.
Proof. For the Cauchy in probability case, note that for each k ≥ 1, there
is n∗ such that for n, m ≥ n∗ , P(|Xn − Xm | ≥ 2−k ) ≤ 2−k . Pick a
subsequence (nk ) such that P(|Xn − Xm | ≥ 2−k ) ≤ 2−k for n, m ≥ nk .
a.s.
Then Xnk −→ X for some X . Finally, from
P(|Xn − X | ≥ ε) ≤ P(|Xn − Xnk | ≥ ε/2) + P(|Xnk − X | ≥ ε/2)
P
the convergence Xn −→ X follows.
6 / 15
Convergence of random variables: further results
What have we learned so far about convergence in probability, almost sure
convergence and convergence in Lp ?
• Limits are almost surely unique.
a.s.
Lp
P
• Xn −→ X or Xn −→ X implies Xn −→ X .
P
a.s.
• Xn −→ X implies Xnk −→ X for a subsequence.
• Being Cauchy implies convergence.
• Almost sure convergence and convergence in probability are preserved
by continuous mappings, addition and multiplication.
P
• “Fast” convergence in probability ( ∞
n=1 P(|Xn − X | ≥ ε) finite)
implies almost sure convergence.
7 / 15
Independence
Let X1 , . . . , Xn be a finite family of random variables. These variables are
independent if it holds for A1 , . . . , An ∈ B that
P(X1 ∈ A1 , . . . , Xn ∈ An ) =
n
Y
P(Xk ∈ Ak ).
k=1
As σ(Xk ) = {(Xk ∈ A) | A ∈ B}, we may reformulate this as that
X1 , . . . , Xn are independent if
P(∩nk=1 Fk )
=
n
Y
P(Fk ),
k=1
whenever F1 ∈ σ(X1 ), . . . , Fn ∈ σ(Xn ). This reformulation leads to a
natural generalization of independence of variables to independence of
σ-algebras.
8 / 15
Independence
Definition 1.3.1. Let I be some set, and let (Fi )i∈I be a family of
σ-algebras. We say that the family is independent if it holds for any finite
sequence of distinct indicies i1 , . . . , in ∈ I and any F1 ∈ Fi1 , . . . , Fn ∈ Fin
that
P(∩nk=1 Fk ) =
n
Y
P(Fk ).
k=1
The definition above is useful because:
• It will allow us to consider independence of variables, events and
σ-algebras in the same framework.
• It encompasses possibly infinite families, and is thus applicable to, for
example, sequences of random variables.
9 / 15
Independence
Lemma 1.3.2. Let I be some set and let (Fi )i∈I be a family of
σ-algebras. Assume that Fi = σ(Hi ), where Hi is stable under
intersections. If it holds for any finite sequence of distinct indicies
i1 , . . . , in ∈ I and any F1 ∈ Hi1 , . . . , Fn ∈ Hin that
P(∩nk=1 Fk )
=
n
Y
P(Fk ),
k=1
then (Fi )i∈I is independent.
Proof. Consider the case |I | = 2. Fix F2 ∈ F2 and define
D = {F1 ∈ F1 | P(F1 ∩ F2 ) = P(F1 )P(F2 )}.
Then D is a Dynkin class. From this, the result follows.
10 / 15
Independence
Definition 1.3.3. Let I be some set and let (Xi )i∈I be a family of random
variables. We say that (Xi )i∈I is independent if (σ(Xi ))i∈I is independent.
If (Fi )i∈I is a family of events, we say that (Fi )i∈I is independent of
(σ(1Fi ))i∈I is independent.
Lemma 1.3.4. Let I be some set and let (Xi )i∈I be a family of variables.
The family is independent if and only if it holds for any finite sequence of
distinct indicies i1 , . . . , in ∈ I and any A1 , . . . , An ∈ B that
P(∩nk=1 (Xik ∈ Ak )) =
n
Y
P(Xik ∈ Ak ).
k=1
11 / 15
Independence
Let (Fi )i∈I and (Gi )i∈I be families of σ-algebras. Let (Xi )i∈I be a family
of random variables. Properties of independence:
• If (Fi )i∈I is independent and Gi ⊆ Fi , (Gi )i∈I is independent.
• If (Xi )i∈I is independent, (ψi (Xi ))i∈I is independent.
• If J, J 0 ⊆ I are disjoint and (Fi )i∈I is independent, σ((Fi )i∈J ) and
σ((Fi )i∈J 0 ) are independent as well.
Also, for a family of events (Fi )i∈I , (Fi )i∈I is independent if and only if
P(∩nk=1 Fik )
=
n
Y
P(Fik )
k=1
for all finite sequences of distinct indicies i1 , . . . , in ∈ I .
12 / 15
Kolmogorov’s zero-one law
We now show some results where independence is involved.
Definition. Let (Xn ) be a sequence of random variables. The tail
σ-algebra of (Xn ) is the σ-algebra ∩∞
n=1 σ(Xn , Xn+1 , . . .).
For an event F to be in the tail σ-algebra, it is necessary and sufficient that
it can be written as something “depending only on (Xn )n≥k ” for all k ≥ 1.
Example 1.3.11. The event (Xn converges) is a member of the tail
σ-algebra of (Xn ).
◦
13 / 15
Kolmogorov’s zero-one law
Theorem 1.3.10 (Kolmogorov’s zero-one law). Let (Xn ) be a sequence
of independent variables. Let J be the tail σ-algebra of (Xn ). For each
F ∈ J , it holds that either P(F ) = 0 or P(F ) = 1.
Proof. Let F ∈ J and let D = {G ∈ F | P(G ∩ F ) = P(G )P(F )}. D is a
Dynkin class containing σ(X1 , . . . , Xn ) for all n ≥ 1. By Dynkin’s lemma,
σ(X1 , X2 , . . .) ⊆ D as well. In particular, J ⊆ D. Thus, P(F ) = P(F )2 , so
P(F ) = 0 or P(F ) = 1.
A consequence of Kolmogorov’s zero-one law is that for a sequence of
independent random variables (Xn ), either Xn is almost surely convergent,
or Xn diverges almost surely.
14 / 15
Kolmogorov’s zero-one law
Lemma 1.3.12 (Second Borel-Cantelli). Let (Fn ) be a sequence of
independent events. Then P(Fn P
i.o.) is either zero or one, and the
probability is zero if and only if ∞
n=1 P(Fn ) is finite.
∞
∞
Proof. Recall that (Fn i.o.) = ∩∞
n=1 ∪k=n Fk . Note that as (∪k=n Fn )n≥1 is
decreasing, we have for any m ≥ 1 that
∞
(Fn i.o.) = ∩∞
n=m ∪k=n Fk ,
so with J the tail σ-algebra for (1Fn ), (Fn i.o.) is in J , and thus by
Kolmogorov’s zero-one law, P(Fn i.o.) is either zero or one.
P∞
We already know that if P
n=1 P(Fn ) is finite, then P(Fn i.o.) = 0. For
the converse, show that if ∞
n=1 P(Fn ) is infinite, P(Fn i.o.) = 1.
15 / 15