### Lectures 6-10: Axioms of Probability Theory

```Lectures 6-10: Axioms of Probability Theory
Let X be a set.
We denote by 2X the set of all subsets of X .
Definition
A collection of subsets F ⊆ 2X is called an algebra if
I
∅, X ∈ F
I
A ∈ F ⇒ Ac ∈ F
I
A, B ∈ F ⇒ A ∪ B ∈ F
1 / 16
Algebras of sets
Exercises:
1. Check that
A∪B
c
= Ac ∩ B c
and use this fact to show that if F is an algebra then
A, B ∈ F ⇒ A ∩ B ∈ F.
2. For any set X , show that the collections F = {∅, X } and
F = 2X are algebras.
3. Let X = R. Show that the collection F consisting of finite
unions of intervals (closed, open or half-open) is an algebra.
2 / 16
Pre-measure
Definition
A function P from an algebra of sets F ⊆ 2X to the set of
non-negative real numbers R≥0 is called a pre-measure if for any
disjoint sets A, B ∈ F (A ∩ B = ∅) one has
P(A ∪ B) = P(A) + P(B).
Exercises:
1. Let X be any set and A ⊂ X . Take any numbers x, y ≥ 0.
Show that
F = {∅, A, Ac , X }
is an algebra. Check that P : F → R≥0 given by
P(∅) = 0, P(A) = x, P(Ac ) = y , P(X ) = x + y
is a pre-measure.
3 / 16
2. Show that for any pre-measure P on an algebra F one has
I
I
P(∅) = 0
A ⊂ B ⇒ P(A) ≤ P(B)
3. Let F ⊆ 2X be any algebra and x ∈ X . Show that the
function δx : F → R≥0 given by
(
0, x ∈
/A
δx (A) =
1, x ∈ A
is a pre-measure.
4. Show that for any finite collection of pairwise disjoint sets
A1 , . . . An ∈ F, Ai ∩ Aj = ∅ one has
P(
n
[
i=1
Ai ) =
n
X
P(Ai ) .
i=1
4 / 16
σ-algebras
Definition
An algebra of sets F ⊆ 2X is called a σ-algebra if
A1 , A2 , . . . ∈ F ⇒
∞
[
Ai ∈ F.
i=1
Definition
For any collection of subsets X ⊆ 2X , the σ-algebra generated by
X is defined as
\
σ(X ) =
F
X ⊆F ⊆2X
where the intersection is taken over all σ-algebras of subsets of X
containing the collection X .
5 / 16
Exercise: Show that σ(X ) in the above definition is indeed a
σ-algebra.
We see that σ(X ) is the minimal σ-algebra which contains all sets
in X .
Example: Let X = R and X is a collection of all intervals. Then
σ(X ) is called the Borel σ-algebra.
Exercise: Show that Borel σ-algebra contains all open and closed
sets.
6 / 16
Measure
Definition
A pre-measure on a σ-algebra F is called a measure if
A1 , A2 , . . . ∈ F, Ai ∩ Aj = ∅ ⇒ P(
∞
[
Ai ) =
i=1
∞
X
P(Ai ).
i=1
While pre-measure is an additive function of sets, a measure is a
Exercise: Show that for a measure P and any countable collection of
sets A1 , A2 , . . . ∈ F one has
P(
∞
[
i=1
Ai ) = lim P(
n→∞
n
[
Ai ) .
i=1
7 / 16
Main theorem: extension of a pre-measure
A pre-measure P on an algebra of sets F is called a measure if it is
A1 , A2 , . . . ∈ F,
Ai ∩ Aj = ∅ ,
⇒ P(
∞
[
i=1
∞
[
Ai ∈ F
i=1
∞
X
Ai ) =
P(Ai ).
i=1
Theorem
For any measure P on an algebra F there exists a unique extension
of P to a measure on σ(F).
8 / 16
Extension of a measure
Examples:
1. Let F be the algebra of finite unions of subintervals in
X = [a, b] and λ be the length function, i.e.
[
X
λ
[ci , di ] =
(di − ci )
i
i
Then one can extend the length uniquely to all Borel sets: for
every A ∈ σ(F) the length λ(A) is well-defined.
9 / 16
Extension of a measure
2. Let F be the algebra of finite unions of subintervals in
X = [a, b] and p(x) be a non-negative integrable function on
[a, b]. Then we define
[
XZ
P
[ci , di ] =
i
i
di
p(x)dx .
ci
This is a measure, and therefore it has the uniqueRextension to
Borel sets. Now, for any A ∈ σ(F) we can write A p(x)dx
meaning the value P(A) of the extension.
10 / 16
Extension of a measure
3. Let F be the algebra of finite unions of rectangles in
X = [a, b] × [c, d] and α be the area function on F. This is a
measure, and therefore it has the unique extension to Borel
sets. Now, for any A ∈ σ(F) the area α(A) is well defined.
4. (product of measures) Let Pi be a measure on Fi ⊆ 2Xi for
i = 1, 2. Then
[
X
P( Aj × Bj ) =
P1 (Aj )P2 (Bj )
j
j
is a measure and extends to the σ-algebra generated by finite
unions of products of sets from F1 and F2 respectively. This
σ-algebra is denoted F1 × F2 and P is denoted P1 × P2 . For
example, α = λ × λ.
11 / 16
Probability measures
Definition
A measure P : F → R≥0 on a σ-algebra F ⊆ 2X is called a
probability measure if P(X ) = 1.
In the above examples:
R
p(x)dx
P(A) = R A
X p(x)dx
is a probability measure on Borel subsets of X = [a, b].
P(A) =
α(A)
α(A)
=
α(X )
(b − a)(d − c)
is a probability measure on Borel subsets of X = [a, b] × [c, d].
12 / 16
Axioms of probability theory
Recall that a probabilistic experiment is an experiment which
I
can be repeated infinitely many times;
I
has a well-defined set of possible outcomes, which we denote
by Ω.
Subsets A ⊆ Ω are called events, and we want to have a
probability law which assigns to an event A its probability P(A).
13 / 16
Axioms of probability theory
Definition
A probability space is a triple (Ω, F, P) where
I
Ω is a set;
I
F ⊆ 2Ω is a σ-algebra of subsets of Ω;
I
P : F → R≥0 is a probability measure, i.e. we have P(Ω) = 1.
14 / 16
Examples:
1. Consider an interval [a, b] (a = −∞, b = ∞ are possible) and
let p(x) be a non-negative integrable function on [a, b] such
Rb
that a p(x)dx = 1. Then we have a probability space:
I
I
I
Ω = [a, b];
F is the RBorel σ-algebra;
P(A) = A p(x)dx.
Probability measures constructed by means of integration are
often called distributions. p(x) is called the density function
of the distribution.
1
satisfies properties of a
Exercise: Check that p(x) = b−a
density function on a the finite interval [a, b]; this is called the
uniform distribution on [a, b].
Exercise: Let λ > 0. Check that p(x) = λe −λx is a density
function on [0; +∞]; this is the exponential distribution with
parameter λ.
15 / 16
2. (Discrete models) Let Ω = {w1 , w2 , w3 , . . . } be a countable
set. Then for anyPsequence of numbers {p1 , p2 , p3 , . . . } such
that pi ≥ 0 and i pi = 1 we can define a probability
measure on F = 2Ω by
X
P(A) =
pi .
i:wi ∈A
In the case when Ω is a finite set, one can define uniform
1
. In this case we
probability law by taking pi all equal to |Ω|
have
X 1
|A|
=
.
P(A) =
|Ω|
|Ω|
i:wi ∈A
16 / 16
```