Monotone sequences
Borel-Cantelli lemma
Final remarks
Monotone sequences of events
Def.1.1: A sequence (An )n≥1 of events is increasing if An ⊂ An+1
for all n ≥ 1. It is decreasing if An ⊃ An+1 for all n ≥ 1.
Example 1.2: If (An )n≥1 is a sequence of arbitrary events, then:
the sequence (Bn )n≥1 with Bn = ∪nk=1 Ak is increasing,
the sequence (Cn )n≥1 with Cn = ∩nk=1 Ak is decreasing.
Monotone sequences
Borel-Cantelli lemma
Final remarks
Continuity of probability measure
Probability measure is continuous along monotone sequences of
events:
Lemma 1.3: Let (An )n≥1 be a sequence of events.
If (An )n≥1 is increasing with A = limn An = ∪n≥1 An , then
P(A) = P lim An = lim P(An ) .
n→∞
n→∞
If (An )n≥1 is decreasing with A = limn An = ∩n≥1 An , then
P(A) = P lim An = lim P(An ) .
n→∞
n→∞
Remark 1.3.1: If (An )n≥1 is not monotone, the claim of the
lemma is not necessarily true (find a counterexample!).
Z
This property is equivalent to σ-additivity of probability measures!
Monotone sequences
Borel-Cantelli lemma
Final remarks
Example 1.4: A standard six-sided die is tossed repeatedly.
Let N1 denote the total number of ones observed. If individual
outcomes are independent, then P(N1 = ∞) = 1.
Solution.
We show that P(N1 < ∞) = 0.
S
Notice: {N1 < ∞} = n≥1 Bn with Bn = no ‘ones’ after nth toss .
T
Also, Bn = m>0 Cn,m , Cn,m = no ‘one’ on tosses n + 1, . . . n + m .
Since P(Cn,m ) = (5/6)m and Cn,m ⊃ Cn,m+1 for all m ≥ 1,
the continuity of probability gives P(Bn ) = 0 for all n.
P
By subadditivity of probability, P(N1 < ∞) ≤ n P(Bn ) = 0 .
Monotone sequences
Borel-Cantelli lemma
Example 1.5: Let X ≥ 0 be a random variable with
P(X < ∞) = 1. For k ≥ 1, denote Xk = k1 X and put
A(ε) ≡ |Xk | > ε finitely often .
Show that P A(ε) = 1 for every ε > 0.
Solution.
Let Ω0 = X is finite ≡ {ω ∈ Ω : X (ω) < ∞}; then P(Ω0 ) = 1.
Denote Bk = {|Xk | > ε} = {ω : |X (ω)| > kε}; then
Bk ⊃ Bk+1
and
for all k ≥ 1,
\
!! Bk = Bk infinitely often .
Ωc0 = X = ∞ =
k≥1
Consequently, A(ε) = Bk finitely often
≡ Ω0 for all ε > 0.
Final remarks
Monotone sequences
Borel-Cantelli lemma
Final remarks
Remark 1.5.1: The previous argument shows that
\
{ω : Xk (ω) → 0} ≡
A(ε) ≡ Ω0 ;
ε>0
i.e., the sequence of random variables Xk converges (to zero)
with probability one (or almost surely), P(Xk → 0) = 1.
Monotone sequences
Borel-Cantelli lemma
Final remarks
Sequences of events
Let A1 , A2 , . . . be an infinite sequence of events in Ω, F, P . We
shall often be interested in finding out how many of the An occur.
Eg., for every fixed n ≥ 1, the events Bn and Bnc ,
def
Bn =
∞
[
Bnc
Ak ,
k=n
≡
∞
\
Ack ,
k=n
mean
at
least
one
of
the
events
A
,
k
≥
n,
occurs
, and
k
none of Ak , k ≥ n, occurs ≡ only some Ak , k < n, can occur .
S
Therefore, n≥1 Bnc ≡ finitely many of Ak , k ≥ 1, occur and
def
A =
[
n≥1
Bnc
c
≡
\
n≥1
Bn ≡
∞
\ [
Ak
n≥1 k=n
is the event infinitely many of Ak , k ≥ 1, occur .
Monotone sequences
Borel-Cantelli lemma
Final remarks
Borel-Cantelli lemma
The event
An occur, sometimes
that infinitely many
of the written An infinitely often or An i.o. is
∞
\ [
An i.o. = lim sup An =
Ak .
n→∞
n≥1 k=n
Lemma 1.6: Let A = An i.o. . Then:
P
a) If k P(Ak ) < ∞, then P(A) = 0, ie., with probability one
only finitely many of the Ak occur.
P
b) If k P(Ak ) = ∞ and A1 , A2 , . . . are independent events,
then P(A) = 1.
Monotone sequences
Borel-Cantelli lemma
Final remarks
Remark 1.6.1: Observe that the independence condition in the
2nd Borel-Cantelli Lemma is essential. Indeed, consider some
event E ∈ F with 0 < P(E ) < 1 and define
An ≡ E
for all n ≥ 1.
Then A = E and P(A) = P(E ) 6= 1. Of course, the events Ak
defined above are not independent.
Monotone sequences
Borel-Cantelli lemma
Remark 1.6.2: For X ∼ U(0, 1), let An = X < 1/n .
P
We have: P(An ) = 1/n, so that n P(An ) = ∞ ;
however, A = An i.o. = ∅, so that P(A) = P(An i.o.) = 0.
Final remarks
Monotone sequences
Borel-Cantelli lemma
Final remarks
Example 1.7: [Infinite monkey theorem]
A monkey hitting keys at random on a typewriter keyboard for an
infinite amount of time will almost surely (ie., with probability
one) type any particular chosen text, such as the complete works
of William Shakespeare, and in fact, infinitely many copies of the
chosen text.
Monotone sequences
Borel-Cantelli lemma
Final remarks
Remark 1.7.1: If the key distribution does not change with
time, can use both monotone approximation or Borel-Cantelli
Lemma.
If the key distribution changes with time, it is often easier to
apply the Borel-Cantelli Lemma.
Monotone sequences
Borel-Cantelli lemma
Final remarks
Example 1.9: A coin showing ‘heads’ with probability p is tossed
repeatedly. With Xn denoting the result of the nth toss, let
Cn = {Xn = T, Xn−1 = H}. Show that P(Cn i.o. ) = 1.
Solution.
We have C2n i.o. ⊂ Cn i.o. , with mutually independent events C2n
satisfying P(C2n ) ≡ pq.
The result follows from the second Borel-Cantelli Lemma (or via
monotone approximation).
Monotone sequences
Borel-Cantelli lemma
Final remarks
Example 1.10: Let (Xk )k≥1 be i.i.d. with P X1 > x = e −λx for
all x ≥ 0.
Xn
1
Then Xn ∼ λ1 log n in the sense that P lim sup log
=
n
λ = 1.
n→∞
Solution.
For ε > 0, denote
n
o
n
o
1+ε
1−ε
def
def
Aεn = ω : Xn (ω) >
log n , Bnε = ω : Xn (ω) >
log n .
λ
λ
P
Then P Aεn = n−(1+ε) with n P(Aεn ) < ∞, thus giving
P( Aεn infinitely often ) = 0.
Similarly, events Bnε are independent with P Bnε = n−(1−ε) ;
P
therefore, n P(Bnε ) = ∞, implying
P Bnε infinitely often = 1 .
Monotone sequences
Borel-Cantelli lemma
Final remarks
Records
Remark 1.10.1: Let (Xk )k≥1 be i.i.d. exponential r.v. with
def
distribution P(Xk > x) = e −x , and let Mn = max1≤k≤n Xk . Then
P(Mn /(log n) → 1) = 1 .
[Similar results for other distributions, eg., gaussian, geometric etc.]
Monotone sequences
Borel-Cantelli lemma
Final remarks
Example 1.11: Let variables (Xn )n≥1 be i.i.d. with X1 ∼ U[0, 1].
For α > 0, we have
P Xn > 1 − n−α = n−α ,
so that
P Xn > 1 − n−α i.o.
(
1,
=
0,
α ≤ 1,
α > 1.
A similar analysis shows that
1
P Xn > 1 −
i.o.
=
n(log n)β
(
1,
0,
β ≤ 1,
β > 1.
Monotone sequences
Borel-Cantelli lemma
Final remarks
Proof of the Borel-Cantelli lemma-1
Recall:
∞
T S
A = An i.o. ≡
Ak .
n≥1 k=n
a) For every n ≥ 1, denote
def
Bn =
∞
[
Ak ≡
n
at least one of Ak with k ≥ n occurs
k=n
The sequence Bn is monotone (Bn ⊇ Bn+1 ) and satisfies
A ⊆ Bn for all n ≥ 1, so that
P(A) ≤ P(Bn ) ≤
∞
X
P(Ak ) → 0
k=n
as n → ∞, whenever
P
k
P(Ak ) < ∞.
o
.
Monotone sequences
Borel-Cantelli lemma
Final remarks
Proof of the Borel-Cantelli lemma-2
b) Since
Ac =
Bnc =
S
n≥1
where
∞
T
Bnc ≡
k=n
Ack ≡
∞
S T
n≥1 k=n
Ack ,
none of Ak , k ≥ n, occurs ,
it is enough to show that P(Bnc ) = 0 for all n ≥ 1.
By independence and the inequality 1 − x ≤ e −x for x ≥ 0,
m
m
m m
\
Y
n X
o
Y
c
c
1−P Ak ≤ exp −
P(Ak )
P
Ak =
P Ak =
k=n
k=n
k=n
k=n
so that
P
Bnc
= lim P
m
\
m→∞
as the series diverges.
k=n
Ack
∞
n X
o
P(Ak ) = 0 ,
≤ exp −
k=n
Monotone sequences
Borel-Cantelli lemma
Final remarks
Sequences of events and their limits
By the end of this section you should be able to:
• state and apply the continuity property of probability
measures along monotone sequences of events;
• state and prove the Borel-Cantelli lemma; apply it to infinite
sequences of events.